Explanation

An explanation is an account of why something is the case or why something was done. In the philosophy of science the leading accounts have been covering-law, unificationist and causal; on Woodward’s interventionist version, to explain an outcome is to show how it would have changed had its causes been different.

Explanation does two jobs in my work. The first is the familiar one in the ethics of AI. Explainable AI promises accounts of automated decisions, and much of the debate runs together an explanation that is causally accurate and one that would satisfy the person affected. Miller argued, from the social sciences, that people want explanations that are contrastive, selective and social; a causally complete account may be none of these.

Language models add a further gap. When a model is asked to explain an answer it has already given, what it produces is a further output of the same generative process, not a record of the process that produced the answer, and the displayed rationale can be disconnected from what actually drove the output (Turpin et al. 2023). Reasoning traces generated before the answer are better placed, since they are causally upstream of it, but how faithfully they reflect it is partial and varies (see chain-of-thought faithfulness).

The second job is structural. The Tracking Minimum is an explanatory condition: a source transmits testimonial knowledge only if the truth of what it asserts, or the evidence for it, figures in the explanation of its asserting it. The test is interventionist rather than psychological. Hold the evidence fixed and vary the proxies the process consults; then hold the proxies fixed and vary the evidence. A process that tracks follows the evidence; one that exploits a proxy follows the proxy. To ask for an explanation of a model’s output in this sense is not to ask the model. It is to ask what its outputs would do under intervention.