Right to explanation

A person is refused credit, a visa or a welfare payment by a system that scored them. They ask why. The right to explanation is the idea that they are owed an answer, and that “the computer decided” is not one.

What the law says

The debate began with the European Union’s General Data Protection Regulation, which applied from 2018. Three parts of it matter. Article 22 gives a right not to be subject to a decision based solely on automated processing that has legal or similarly significant effects. Articles 13 to 15 require that people be given “meaningful information about the logic involved” in such processing. And Recital 71, which is not binding, speaks of a right “to obtain an explanation of the decision reached”.

Sandra Wachter, Brent Mittelstadt and Luciano Floridi argued that this does not add up to a right to have a particular decision explained. On their reading the binding articles give a right to be informed, in advance and in general terms, about how a system works, and the one explicit mention of explanation sits in a recital. Andrew Selbst and Julia Powles replied that “meaningful information about the logic involved” must be meaningful to the person and usable for exercising their other rights, which is a right to explanation in all but name.

The EU’s Artificial Intelligence Act of 2024 added a provision headed “right to explanation of individual decision-making” for decisions taken on the basis of high-risk AI systems.

What an explanation would have to be

Law aside, there is a question of what would satisfy the right. A description of the model’s design explains the system and not the decision. A list of the features that weighed most may not be faithful to what the model did. A counterfactual, such as “had your income been higher by this much, the loan would have been granted”, tells the person what to change but not whether the rule is fair.

Where I stand

I think the right is better grounded in what the person is owed than in what the system can produce. Someone whose standing depends on a decision needs to be able to see the criteria applied to them and to contest their application. I call that legibility, and I treat it as an obligation that falls on the institution whether or not its model is interpretable. If a system cannot give reasons that a person could dispute, that is an argument against using it for such decisions, not a limit on what the person may ask.