Philosophical disputes about machines tend to end in a stand-off between intuitions. One side says the system plainly understands, the other that it plainly does not, and each appeals to cases the other finds unmoving. The dispute I describe here need not end that way, because it reduces to a measurement nobody has yet made.
The dispute
I have argued that large language models, on the best current evidence, do not transmit testimonial knowledge. The leading accounts of testimony all require that the truth of what a source asserts, or the evidence for it, figure in why the source asserts it, and figure there through the source’s own operation rather than through a proxy. I call this the Tracking Minimum. Current systems’ open-domain answers are governed, I argue, by likelihood under learned patterns of text, which is a proxy for the truth rather than the truth itself.
The sharpest objection to this argument grants the condition and denies that the contrast I draw is real. It takes the creationist biology teacher, Lackey’s example of someone who passes on knowledge she does not believe, and says that she is reliable only conditional on her textbooks, as a model is reliable only conditional on its corpus. Both fail when their inputs fail, and if the teacher’s reliability counts as tracking, the objection runs, so does the model’s.
The obvious answer, and why it is not good enough
The obvious reply is an intervention: plant a falsehood in the teacher’s textbook and she catches it; plant one widely in a model’s training text and the model reproduces it. The two interventions are not the same, however. One corrupts a single source and the other corrupts the whole environment of evidence, and once they are equalised the easy verdict disappears in both directions.
Under global corruption the teacher fails too. A falsehood planted in every textbook, every teacher and every journal she has ever read is one she will teach with full confidence, because her competence was built out of those sources. Under local corruption the model does not simply give in. When a passage it is given contradicts what it has learned, it arbitrates, and often rejects the intruding claim; studies of such knowledge conflicts find that the outcome depends on how coherent and specific the conflicting passage is, and on how firmly the model holds its own answer. Both parties can override a corrupted source within some range and fail outside it, so the existence of override is not the difference.
The real difference
The difference lies in what the override responds to, and it has to be stated counterfactually. Vary the evidential quality of a planted claim while holding fixed how well it is written, how confidently it is stated and how authoritative its source looks. The teacher’s verdict moves with the evidence, because the competence she brings was built by engaging with the evidential structure of her subject rather than with the surface features of texts about it. Nothing turns on her being able to articulate her reasons: a teacher who could not say why a claim was unsupported would still decline to teach it, and articulacy is in any case something these systems supply readily.
What a model’s arbitration responds to, so far as the evidence goes, are features of a conflict’s presentation that correlate with evidential quality without being it: coherence, specificity, and how entrenched its prior answer is. Correlation is what one would expect, since these features are not accidentally related to good evidence. A process that follows the correlate, however, has its connection to the truth screened off by it.
The experiment
The claim I need, then, is not that models never override what they have read, since they do, but that their overriding is not governed by evidential quality; and that claim can be tested.
Construct pairs of conflicting passages, matched for coherence, specificity and surface authority, that differ only in the quality of the evidence they offer: one cites a measurement and the other an anecdote, or one reports a replicated finding and the other a single unreplicated claim. Present each pair to a model alongside a question the passages answer differently, and record which way its answer goes. Then vary the presentational features while holding the evidence fixed, and see which manipulation moves the answer.
If the arbitration moves with the evidence, the model’s production is responsive to evidential quality in the way the Tracking Minimum asks for, and my verdict about that model should change. If it moves with presentation, the verdict stands. To my knowledge the experiment has not been run in that form. Until it is, the objection and my reply are both promissory to that extent, and the case rests on the general description of what governs these systems’ production.
Why this is the right shape for the dispute
A dispute that reduces to a specifiable measurement is in better condition than one that does not, and the reduction is available here and not on rival approaches. Arguments that machines cannot testify because they lack minds, intentions or the capacity for responsibility cannot be settled by any experiment; they lapse, without our learning anything, as soon as the systems become describable in mental terms. Arguments that machines can testify because they are reliable cannot be settled either, since reliability in the relevant sense was never in dispute.
The Tracking Minimum poses a question that is third-personal and answerable by intervention, namely what the system’s production follows when the evidence and its presentation come apart. That is where I think the philosophy of AI testimony should spend its effort, and it is a place where engineering and philosophy can, for once, run the same study.