Moral status of AI

A stone has no moral status: nothing you do to it wrongs the stone. A dog has some: it can be treated cruelly. The question about artificial systems is whether any of them, now or later, belongs with the dog and not with the stone.

It helps to separate two roles. A moral agent is something that can do right and wrong and be held responsible. A moral patient is something that can be wronged. The two come apart: an infant is a patient and not an agent. Whether a machine could be an artificial moral agent is one question. This entry concerns the other.

What would ground it

The main accounts differ on which property does the work.

• Sentience. If a being can suffer or enjoy, its suffering counts. On this view the question reduces to whether a system is conscious, which no one at present knows how to settle.
• Rational agency. On views descended from Kant, what matters is the capacity to set ends and act for reasons. A system might have this without feeling anything.
• Relations. Mark Coeckelbergh argues that moral consideration is not read off an entity’s inner properties, which we cannot inspect, but arises within the social relations in which it appears to us.

Eric Schwitzgebel and Mara Garza argue from parity. If an artificial being differed from a human in no morally relevant respect, only in being made of other stuff or having been manufactured, it would deserve the same consideration. They add that makers would owe such beings more than strangers do, as parents owe children.

Two ways of going wrong

The risk runs in both directions. We might deny status to beings that have it, and treat as property something that can be harmed. Or we might grant it to systems that only look the part, and divert concern, law and money from people and animals to products built to seem needy.

Where I stand

I do not think any current system has been shown to have moral status, and I do not think the question is closed. My own work argues that we can decide what to believe on a system’s say-so without first deciding whether it has a mind. I doubt the same shortcut exists here. Whether something can be wronged does seem to depend on what it is, and how a system talks about itself is poor evidence, since it was trained on human descriptions of inner life. Under that uncertainty the sensible policy is caution in what we build, before the question of what we owe to it is forced on us.