Is an AI chatbot an informant or an instrument?

Consider someone reading a barometer. The barometer does not tell her anything; no entitlement to take its word, no assurance, no norm of assertion is in play. She nevertheless comes to know the pressure when conditions are right: the instrument is well calibrated, she is competent with it, she is using it in conditions for which the calibration holds, and she has reason to think all this is so. No knowledge is passed to her; her knowledge is generated by her reliable use of a reliable instrument, and the epistemic load lies with her.

The thesis

When a user comes to a warranted true belief through a language model, her standing is that of a competent reader of an instrument. It depends on the instrument’s reliability in the conditions of use, on her competence in operating and interpreting it, and on the absence of reasons for doubt. No warrant is transmitted from the system to the user. Her warrant, when she has it, is generated by her use.

Others have proposed different labels. Freiman argues that beliefs acquired from conversational AI form a third, technology-based kind; Duede places such systems between instruments and epistemic agents; Alvarado calls AI an epistemic technology. These are live alternatives and I do not claim to refute them. My claim concerns the structure of the warrant, and it can be accepted under any of these names.

Two practical differences from testimony

Testimony lets a hearer defer, because the warrant runs through the speaker. Instrument reading places the load on the user, whose appropriate attitude is calibrated reliance rather than trust in someone’s word.

And when an informant misleads you, you have social remedies: you can challenge her, ask for her grounds, appeal to others who know her. When an instrument misleads you, the remedies are engineering ones. A chatbot’s conversational surface invites the first kind of response, but the structure underneath is the second kind.

An instrument without transparent calibration

The classical epistemology of instruments presupposes transparent calibration. For a thermometer we know what it is calibrated to, by what procedure, and with what failure modes, so principled judgements about its reliability in a given case are available. For language models we have partial accounts of what they are calibrated to and how they fail, and no general method for predicting reliability at the level of a single answer.

Opacity so described, though, is a limit on what a user can know about a connection that obtains, and the deeper trouble is with the connection. A barometer is opaque to someone who does not understand aneroid cells, and the remedy is to learn, or to consult someone who has. Here there is no corresponding remedy, because what is missing is not the user’s access to the calibration but the calibration itself: nothing in the production of an open-domain answer is keyed to the truth of the particular content emitted. That is why I call these systems degenerate instruments. A working barometer meets the Tracking Minimum; they do not. The record a user projects from past success to the present case is all the connection there is, which is why checking that record is not a formality.