Can you cite ChatGPT as a source?

The question usually arrives as one about formatting. Underneath it is a question about what a citation is, and the answer to that settles most of the practical cases.

What a citation does

A citation does two jobs. It gives credit: these words or this idea came from that person. And it gives warrant: this claim is supported by that work, which you can find and check, and whose author answers for it.

A chatbot’s output can be the object of the first job. If you quote what it wrote, or your work is about what it wrote, cite it as you would cite any text you are discussing. Style guides now have formats for that.

It cannot do the second job. When you write “unemployment fell by two points (ChatGPT)”, the citation promises the reader that someone stands behind the figure and that it can be looked up. Neither is true. The model’s answer is not stored anywhere a reader can open, the same question may give a different answer tomorrow, and nobody has taken responsibility for it.

Why it is not like citing an encyclopaedia

It may be said that we cite reference works without knowing the authors. The difference is in how the content is produced. An encyclopaedia entry was written by someone who consulted sources, was checked by an editor, and is corrected when it is wrong. Its being true is part of why it says what it says.

A language model’s answer to an open question is produced by a process in which the truth of that particular answer plays no part. A false answer comes out the same way as a true one. That is the reason I argue that what a chatbot tells you is not testimony: there is no one whose knowledge is being passed to you. Citing it for a fact treats it as a witness, and it is not one.

What to do instead

Use the chatbot as you would use a well-read friend with an unreliable memory: for leads.

1. Ask it where a claim comes from, and treat the reply as a place to start looking.
2. Find the original paper, book, dataset or statute.
3. Read the relevant part yourself. Fabricated references look exactly like real ones, and real ones are often misdescribed.
4. Cite what you read.

If you cannot find an original, you do not have a source, and the claim should come out.

Where I stand

I think the instinct behind the rule against citing chatbots is right and is often defended for the wrong reason. The trouble is not that the systems are inaccurate, since many are accurate much of the time and getting better; it is structural. A citation hands the reader on to someone answerable. A language model can be relied on as an instrument, by a user who knows its limits and checks it. It cannot be the place where a reader’s checking ends, because nothing in it is answerable to the reader at all. The person who is answerable is you, which is the best reason to have read the source.