Abhinav Saxena
Glossary

Artificial moral agent

Also called: AMA, artificial moral agents, machine ethics, moral machine, machine moral agency, AI moral agency

Definition

An artificial system whose behaviour is open to moral assessment; the debate concerns whether any machine can be a moral agent, and what kind of agent that would require it to be.

By · Updated · 2 min read

A system decides which loan applications to refuse, or which patient is seen first. Its outputs help and harm people. Is it a moral agent, something that can do right and wrong, or only a tool in the hands of those who are?

Moor’s four kinds

James Moor’s much-used classification separates four things the phrase might mean. An ethical-impact agent is any machine whose operation has ethical consequences, which is nearly all of them. An implicit ethical agent has safety or fairness built into its design, as a cash machine is built not to short-change. An explicit ethical agent represents ethical considerations and reasons with them. A full ethical agent does that and has what we take a human moral agent to have: consciousness, intentionality and free will. The live disputes concern the third and fourth.

Agency without a mind

Luciano Floridi and J. W. Sanders argued that the fourth category sets the bar in the wrong place. On their view, something is an agent, at a chosen level of description, if it is interactive, autonomous and adaptable, and it is a moral agent if its actions can cause moral good or evil. That allows machines to be moral agents without minds. They pair it with a distinction that does much of the work: such an agent can be accountable, a source of moral action to be corrected or switched off, without being responsible in the sense that deserves praise or blame.

Objections

The standard view replies that moral agency requires understanding reasons and being able to answer for one’s conduct, and that nothing lacking these can do more than behave. A different worry is practical: describing machines as moral agents invites the people who build and deploy them to step back.

Where I stand

I take seriously the idea that agency comes apart from consciousness (agency without consciousness). But I resist the conclusion that responsibility thereby moves to the machine. When an AI system causes harm, responsibility does not vanish into it; it has to be traced among the people and institutions that designed, deployed and relied on it (the responsibility gap).

Related terms

AgencyThe capacity to act: to do something, rather than merely have something happen, for reasons that are in some sense one’s own.ResponsibilityThe relation between an agent and an outcome in virtue of which the agent can rightly be praised, blamed or asked to answer for it.The responsibility gapThe worry, due to Andreas Matthias, that learning machines can cause outcomes for which no one can fittingly be held responsible, since no one could control or foresee them.AutonomySelf-governance: acting on reasons one has reflectively endorsed, rather than merely on whatever desires one happens to have.ConsciousnessThe fact that there is something it is like to be a creature: experience from the inside, as distinct from function described from the outside.Moral status of AIThe question of whether an artificial system could matter morally for its own sake, so that there are things we owe to it and ways of treating it that would wrong it.

Questions that turn on it

QuestionShould AI have rights?

Referred to by

AI alignmentOrthogonality thesis

Further reading

  1. Moor, J. H. (2006). The nature, importance, and difficulty of machine ethics. IEEE Intelligent Systems 21(4): 18–21.doi.org/10.1109/MIS.2006.80
  2. Floridi, L. and Sanders, J. W. (2004). On the morality of artificial agents. Minds and Machines 14(3): 349–379.doi.org/10.1023/B:MIND.0000035461.63578.9d