The Chinese room

What it tests Can a system understand a language merely by following rules for manipulating symbols, however well its answers pass for understanding?

The case

Searle imagines himself locked in a room with boxes of Chinese characters and a book of rules, in English, telling him which characters to pass out when certain characters are passed in. He knows no Chinese. The people outside send in questions in Chinese and receive answers that a native speaker might have given, and they conclude that someone in the room understands Chinese. He understands nothing: he is matching shapes by the rules.

John Searle published this in 1980, in the paper “Minds, Brains, and Programs”, with comments from 27 other researchers, as a reply to claims that AI systems such as Roger Schank’s story-handling software literally understood the stories they processed. A computer, he argued, is in the position of the man in the room, since it handles symbols by their form alone. His target was what he called strong AI, the view that a suitably programmed computer has a mind in the same sense we do. He did not deny that machines can think. Brains are machines, he said, and brains think.

What it is meant to show

The narrow conclusion is that running the right software cannot by itself produce understanding, so passing the Turing test would not show that a machine understands. The broader claim is that syntax is not sufficient for semantics: manipulating symbols by their shape cannot give them meaning. Searle drew the consequence that functionalism and the computational theory of mind, on which a mind is defined by what it does with information, are false. Minds, on his view, result from biological processes, which computers can at best simulate.

Replies

The systems reply. The man is only the processor in a larger system that includes the rule book, the memory and the characters, and the system as a whole understands. Searle answers that he could memorise the rules and do the work in his head, leaving no system beyond himself, and still not know what the Chinese word for hamburger means. Critics rejoin that the memorising man would then host a second agent, a Chinese speaker distinct from him, which is the virtual mind reply. David Chalmers is among those who have endorsed a version of it.

The robot reply. Put the computer in a robot with cameras and limbs, so that its symbols are tied to things by causal connection with the world, in the way causal theories of reference, such as those of Hilary Putnam and others, tie words to things. Searle replies that the sensors supply only more symbols: he could be handed a tape of digits from a camera and still not know what they mean.

The brain-simulator reply. Suppose the software simulates every neuron firing in a Chinese speaker’s brain. Searle replies that the man could operate valves and water pipes arranged like those neurons and still understand nothing, and that a simulation is not the thing simulated. Paul and Patricia Churchland agreed that the room understands nothing but thought the argument trades on our ignorance, comparing it to waving a magnet and finding no visible light.

The other-minds reply. We attribute understanding to other people only on the basis of behaviour, so we must attribute it to a computer that behaves as well. Searle answered that the question is what we attribute, not how we know, and that the computation can exist without the mental state. The author of the Stanford Encyclopedia entry suggests that this reply may be too short.

The intuition reply. Many critics say the argument rests on an intuition that nothing in the room could understand. Daniel Dennett called the case an intuition pump, and Steven Pinker argued that it slows the computation to a pace at which we no longer recognise it as understanding.

Antecedents and variations

The idea is older than Searle. Leibniz imagined walking inside a machine that thinks and finding only parts pushing one another, and Alan Turing described a human following written rules to play chess without knowing the game. Ned Block’s Chinese Nation, from 1978, has the population of China phoning one another to play the part of neurons.

Why it still matters

Searle was writing against hand-built software that could answer a few questions about restaurants. Language models are trained on text and produce paragraphs their designers did not anticipate, and critics ask whether the argument still applies. Defenders answer that it was never about how good the answers are: if a system’s understanding would consist only in processing symbols by form, the room says it has none, however fluent it is. The critics’ worry is that “real” understanding then becomes undetectable, and so idle. The robot and externalist replies suggest a more tractable question, how a system’s words are connected to the world.

In my own work on whether language models can testify I rest nothing on this argument. What governs a system’s outputs, not whether it understands, is what my argument turns on, and I do not deny that such systems may understand what they produce. The case is set out in Can AI testify?