Most of what any of us knows, we know because someone told us: the date of a treaty, the shape of a continent, the safe dose of a medicine. Philosophers call this testimony, and the question they have long asked about it is what makes it work, what it is about a competent and sincere speaker that lets her word give me knowledge of something I did not see.
Chatbots answer questions in the grammar of testimony. They state that the treaty was signed in a given year, in the declarative voice a historian would use, and they are often right. It is natural to treat them as witnesses, sources whose word can make us know. I think that is a mistake, and the mistake does not depend on their being often wrong. It would remain if they were never wrong.
What makes a word worth taking
Start with a case that has nothing to do with machines. A thermometer sits in a room held at a constant temperature by a thermostat. It is broken: it would read the same whatever the temperature, and it happens to read the right number. Its readings are correct, and correct for a stable, explicable reason, yet no one thinks that a person who posts them every hour is passing on knowledge of the temperature. What the thermometer lacks is any connection between a reading and the temperature it reports. Vary the temperature and the reading would not move. Its accuracy is real, but it is the thermostat’s.
The leading accounts of testimony disagree about a great deal. Burge grounds our entitlement to accept what we are told in the presumption of a rational source; Williamson, in the norm that one assert only what one knows; Goldberg, in the reliability of the speaker’s own production; Faulkner, in the assurance a speaker gives; Lackey, in her statement’s being a reliable indication of the truth. Read carefully, each requires the same thing of the speaker: that the truth of what she asserts, or the evidence for it, figure in the explanation of why she asserts it, and figure there through her own operation rather than through a proxy. I call this the Tracking Minimum. The broken thermometer fails it, and so does any source whose accuracy, however impressive, is borrowed.
Two kinds of reliability
There are two things one can mean in calling a source reliable. One is a high rate of true outputs under the conditions in which it is used; the other is that its outputs would follow the truth if the truth were different. The first is a fact about a record, the second about what the source responds to. The broken thermometer is reliable in the first sense and not at all in the second, and testimony needs the second.
It does not need belief, as Lackey’s creationist biology teacher shows. Stella does not believe the evolutionary theory she teaches, and her students, on her word, come to know it. But consider what she still has. Plant an error in her textbook of a kind her competence can detect and she will not teach it; change the evidence within the reach of what she can assess and her teaching changes with it. She lacks the belief. She does not lack the connection.
Where the chatbot stands
On the best current description, a large language model’s answer to an open question is selected because it is likely given the patterns the model learned from a very large body of human writing, adjusted by training against human approval. Where what is likely to be written and what is true coincide, which in well-covered domains is often, the answer is right. The truth reaches it through a proxy: true things tend to be what gets written down. Hold the proxy fixed and vary the truth, and the answer does not move. That is the thermometer’s situation, at a far larger scale and with far more practical value, and it is why I hold that current systems, on the best current evidence, do not transmit testimonial knowledge, however often they are right.
The evidence on the other side is real and should be conceded rather than finessed. Researchers have found structure inside these models that correlates with the truth of what they represent, and intervening on it can make their answers more truthful. That shows the structure is a lever. It does not show that the lever governs production when no one is pulling it, which is what the Tracking Minimum asks about. Nor does the argument depend on denying that these systems understand, intend, or have minds; I make no claim about that. The condition is third-personal, and a machine could meet it. Some come close in narrow domains, where training checks their answers against a verifier. Open-ended conversation is not such a domain.
What you are doing instead
The better model is the instrument. Reading a barometer, you are told nothing; you come to know the pressure because the instrument is reliable in conditions like these, you know how to read it, and you have no reason to doubt it. The knowledge is generated by your use, and the responsibility for using it well is yours. Relying on a chatbot has the same structure. When it goes well, you know because you had good reason to think the system reliable on questions like yours and nothing in the answer gave you reason for doubt. The judging was yours, whether or not you noticed doing it.
The comparison is less comfortable for the chatbot than it sounds. A working barometer meets the Tracking Minimum: vary the pressure and the needle moves, in virtue of the instrument’s own mechanism. On the best current evidence the chatbot does not, so the trouble is not merely that you cannot see how it is calibrated. In respect of the particular content it emits, it is not calibrated to anything, and the record you project from past successes to the answer in front of you is all the connection there is.
Why the label matters
It may be said that nothing practical turns on the label, so long as the answers are usually right. Deference, remedies, and chains turn on it.
A witness lets you defer, since the warrant runs through her. An instrument does not, and treating a chatbot’s answer as testimony invites you to stop checking at the point where checking is your job.
When a witness misleads you, you can challenge her, ask for her grounds, compare her with others; when an instrument misleads you, the remedies are engineering ones. The chatbot’s conversational manner invites the first kind of response, but its answers to your follow-up questions come from the same process as the answer you are questioning, and so are not independent evidence about it.
Testimony travels: a historian tells a journalist, and the journalist tells you. A chatbot preserves no one’s grounds, so chains of its answers, reposted and repeated, do not carry knowledge as chains of testimony do. They carry text.
Instruments are among the great sources of human knowledge, and calling the chatbot one is a correct description rather than a demotion. A practice that mistakes such an instrument for a speaker is in trouble in proportion to how heavily the instrument is used, and an accurate description does its own corrective work.