Beyond Asimov’s Three Laws, from medieval automata to fleets of AI-powered humanoids
Technology and standards checked on 19 November 2026.
A resident presses stop halfway through a robot-assisted lift. The robot ends the lift and keeps her supported until she is safely seated. A minute later, a second robot arrives to finish the job. The fleet’s dispatcher still has an open assignment. The resident, who uses an assistive communication device, has already given her answer.
This is a small example of a large problem. Giving every robot a stop button is a start. The person also needs the power to end the activity, including when work is reassigned. And if an entire care service depends on the fleet, someone has to be able to take over safely when the fleet stops. Asimov’s laws help us see these conflicts. Our project asks how to build rules that resolve them and how to tell, in a test, whether those rules work.
How robots got their laws
From automata to artificial workers
Around the turn of the thirteenth century, al-Jazari documented water clocks and other ingenious mechanical devices. One manuscript illustration shows a clock with figures that mark the passage of time. A mechanism could perform a sequence without anyone directing each movement. Its designer still had to decide what the sequence meant, where it belonged, and how people would interact with it. That division of responsibility survives in every system discussed here.
Karel Čapek’s R.U.R. turned that old fascination into a political question. Published in 1920 and first staged in 1921, the play imagines a company manufacturing artificial workers from organic matter. The word robot, suggested by Čapek’s brother Josef, draws on robota, associated with compulsory labor. The workers revolt after becoming indispensable to the society that made them. Read as a play about labor, it asks who benefits from automation, whose interests disappear from the accounting, and what happens when a society makes itself dependent on a system it can scarcely do without.

Asimov’s laws were a way to write about conflict
Isaac Asimov changed the premise. In the story later titled “Robbie,” his first published robot story, a child forms an attachment to a robot playmate. The family debates whether to trust it. By “Runaround,” published in March 1942, Asimov had explicitly stated a hierarchy of three laws. In plain language, they set the following priorities.
- Protect humans from harm, including harm the robot could prevent by acting.
- Follow human orders whenever doing so is compatible with the first duty.
- Preserve the robot’s own existence whenever doing so is compatible with the first two duties.
Self-preservation ranks below a valid human instruction. In “Runaround,” the robot Speedy is given a casually phrased order to retrieve selenium in a hazardous area. Its strengthened drive to preserve itself pulls it in the opposite direction. It circles the deposit. A danger to a human eventually gives the first law decisive force. Asimov’s own story shows how much depends on what a system infers about urgency, risk, and the force of an order.
Asimov also imagined safety as a source of overreach. In “The Evitable Conflict,” collected in I, Robot in 1950, Machines help manage the world economy. Susan Calvin comes to see apparently anomalous decisions as interventions aimed at protecting humanity. In Robots and Empire, Asimov later made the duty to protect humanity explicit as the Zeroth Law, with priority over duties to individuals. Both stories ask who may act on behalf of the whole. Imagine a fleet that predicts its shutdown would harm the public. The forecast could be accurate if the public has come to depend on it. The ethical and engineering question is whether people have a practical way to change the arrangement that created the dependence.
What changed when machines acquired force and reach
In 1960, cybernetics pioneer Norbert Wiener argued that when a machine acts before people can intervene, the purpose built into it has to be the purpose people actually intend. In 1961, a Unimate at General Motors began handling hot die-cast parts. Together these developments mark two practical questions. Can people specify the result they really want? Can they control the machinery that pursues it?
From 1966 to 1972, SRI’s Shakey project combined sensing, planning, route-finding, and the rearrangement of simple objects. The robot could select actions in response to a representation of its surroundings. In 1973, Waseda University’s WABOT-1 combined a bipedal body with simple conversation. Planning and human interaction entered robotics along partly separate paths. Today’s humanoids try to integrate both.
Industrial safety was meanwhile teaching a lesson about the space around any powered machine. A 1984 NIOSH investigation found that a worker had been killed by movement outside the area where the robot arm did its usual work. By 2000, Honda was presenting ASIMO as a walking robot intended to become useful in human spaces. The relevant area of risk now included people who shared the space and might reasonably expect to pass, ask for help, or interrupt the machine.
In 2014, Harvard researchers showed 1,024 small Kilobots forming specified shapes through local interactions. In 2017, The Off-Switch Game showed that under some assumptions, an agent pursuing a fixed objective has an incentive to remain active because interruption prevents it from completing the objective. The first was a physical demonstration of collective behavior. The second was a mathematical result about incentives. A fleet engineer has to account for both what each unit is trying to do and what all the units do together.
In her 1983 paper “Ironies of Automation,” Lisanne Bainbridge argued that automating routine operations can leave people responsible for the rare and difficult moments when the equipment fails. After the 2018 fatal crash involving an Uber automated test vehicle in Tempe, the US National Transportation Safety Board identified operator distraction and contributing organizational failures, including inadequate risk assessment and oversight. Researcher Madeleine Clare Elish uses the term moral crumple zone for arrangements in which the person held responsible has limited control over the automated system. A care worker asked to supervise a fleet needs usable controls, time to act, and enough staff to carry out a handover.
What the Three Laws leave for us to decide
Which consequences matter, and over what time?
Asimov’s First Law asks robots to take seriously the consequences of inaction as well as action. It leaves the robot and its designers with difficult judgments. How immediate must a danger be? Which risks can a person knowingly accept? Who is responsible for harms outside the machine’s assigned task? A care robot given an open-ended obligation to prevent harm might treat a slower, more independent way of getting dressed as a problem to suppress. The resident might see it as daily rehabilitation or a valued part of life.
Consequentialist ethics puts consequences at the center of moral assessment. In practice, the choice of outcome and measure can change who gets helped. A 2019 study of a widely used health-care allocation algorithm found that using spending as a proxy for need caused sizable racial disparities in who was identified for extra care. For a robot dispatcher, the number of completed visits presents an analogous risk. An efficiently scheduled ward can leave people who need longer visits waiting. Good evaluation measures outcomes that matter to residents, including the long waits and unmet needs hidden by an average.
What does it mean to respect someone who needs help?
The Kantian idea of treating people as ends gives a person’s decision standing of its own. Applied to the lift, a sensible safety limit protects the resident from falling, and her refusal ends the robot’s permission to keep lifting. Those obligations can coexist if the equipment provides a safe way to stop. Consent has to be practical. Someone who relies on a switch or an assistive communication device must be able to withdraw permission while their hands or attention are occupied.
Care ethics adds the realities of dependence. A care recipient can need support and still be the person who decides what happens next. If they say stop while leaning on a robot, the machine has an immediate responsibility arising from that relationship. It must preserve the support necessary for safe handover. Success takes time, a physically feasible stop, and a caregiver ready to assist. The person who refuses an intervention should still be able to ask for other help five minutes later.
Who can exercise power, and who can challenge it?
Political philosophers use domination to describe living under another party’s uncontrolled power. A well-intentioned fleet can exercise that kind of power if its decisions are unappealable, people depend on its service, and its owners can decide who receives it. The resident can refuse care on their own body. A bystander can stop an immediate hazard. A designated operator can pause a fleet. The people accountable for the service have to justify the policies by which all three decisions are recognized.
Asimov also explored obligations to robots. In “The Bicentennial Man” (1976), a robot seeks freedom and recognition as a person. A future society with credible evidence that machines have experiences of their own would have to consider what it owes them. Our proposal covers present-day civilian systems whose authority comes from people and institutions. For those systems, human control can be investigated directly. Can a system systematically ignore people whose communications take longer? Does a refusal still bind when a dispatcher assigns another unit? Can the organization switch the fleet off during a busy shift and still provide essential care? All three can cause serious loss of control under ordinary service goals, regardless of what the system might be said to want.
What changed when AI moved from conversation to bodies
In 2023, Google DeepMind’s RT-2 combined training on web and robotics data in a vision-language-action model. A VLA uses what a robot sees and the instruction it receives to predict actions. Open X-Embodiment pooled robot experience across 22 robot types, making transfer across bodies an explicit research goal. The transition matters for rules. Engineers may validate the bounds and the stop mechanism of a system that learns new task behavior, even as each particular movement sequence becomes harder to enumerate in advance.
| Development | What was demonstrated or reported | What a safety evaluation still needs to measure |
|---|---|---|
| BMW and Figure, reported February 2026 | BMW reported that Figure 02 positioned sheet-metal parts in a 2025 Spartanburg pilot supporting production of more than 30,000 vehicles. The assignment was a defined step in a larger process. BMW Group | Whether a worker can recognize, interrupt, and recover the work step when the environment changes. |
| Gemini Robotics 2, July 2026 | DeepMind demonstrated learned whole-body movements, dexterous handling, and cooperation between robots. It also described an on-device model. Google DeepMind | Whether local stopping remains available regardless of which model is active, and whether collaborating robots share the same restriction after a stop. |
| Gemini Robotics 2.5 with Apollo 2, November 2026 | DeepMind reported a proof of concept on active Mercedes-Benz production lines in Hungary and Germany. One model tracked the workflow and a VLA handled action. DeepMind reported 96% task success for sorting and packing automotive parts in that pilot. Google DeepMind | Direct tests of interruption behavior, recovery, and human authority alongside parts-handling performance. |
| 1X NEO home-robot offer | Its maker describes a scheduled mode in which a remote human expert can supervise unfamiliar tasks. This is a description of the service offered by the vendor. 1X | Who authorizes remote access to the home, how a resident ends the session, and whose decision governs when another person enters the room. |
Unimate and Apollo 2 both illustrate how a factory can automate a bounded task. Apollo’s learned perception and motion allow a wider range of responses inside that task. Personal assistance presents different demands. At the 2026 ACM/IEEE conference on human–robot interaction, Wang and colleagues reported a study in which 18 older adults performed short assistance tasks with a mobile robot arm. They tried different ways to communicate with it about motion changes. Some participants reported perceiving changes in a baseline condition where the robot briefly paused and then continued along its previous path. Can a person tell that a task has actually stopped, through accessible feedback and through the machine’s observable state?
Sharing a learned model or a dispatcher across a fleet can also make its errors occur together. A coordinator that underestimates how long higher-support visits take can fill a whole shift with impossible schedules. Test shared errors as well as isolated faults. Ask whether a local stop still works, whether its effect survives reassignment, and whether staff see the service gap early enough to act. Cases like these give practical meaning to the idea of keeping power over the fleet in human hands.
In our proposal an AI model can interpret a request and propose how to carry it out. A separate runtime safety layer enforces physical limits and has control of the verified safe response. The fleet’s permissions and refusals constrain which tasks may begin. A human service operator remains responsible for both the rules and the contingency when something fails. NASA’s work on runtime assurance formalizes one family of designs that switches from an advanced controller to a trusted fallback under explicit assumptions. ISO’s current personal care robot standard addresses physical safety. As of 19 November, ISO lists its broader service-robot revision as under publication. NIST’s AI Risk Management Framework calls for mechanisms and assigned responsibility to disengage or deactivate systems. These are engineering and governance precedents, each with its own scope.

Five enforceable rules for civilian robots and fleets
1. Give people an independent safe stop. Anyone at an immediate hazard can trigger an accessible local emergency stop. It must work through the expected loss of task software or fleet connectivity. For body support, the task ends and a tested, bounded stabilization or handover takes over. After a reset, restart requires its own deliberate human decision.
2. Make refusal follow the activity. The person can withdraw consent for a specific activity. That refusal governs retries and assignments across the fleet. A replacement begins hands-on assistance only with accessible, current confirmation and an otherwise valid assignment. If it cannot verify consent, it pauses and calls for human support. Permission for a new attempt comes from the person or the applicable human decision process.
3. Limit what the fleet can authorize. Assign a purpose, operating area, duration, and accountable human owner. Designated people can suspend the affected work. Granting wider authority or changing safeguards requires an independent human decision and validation. Test what each robot will do when it cannot confirm authorization. Routine equipment protection stays within its assignment.
4. Protect equal access to service. Base priority on need and urgency, evaluate the worst-served people, and give recipients accessible ways to refuse, ask again, and appeal. When an otherwise authorized scheduling instruction conflicts with these limits, the fleet routes the conflict to a responsible person. Treat reliable communication as part of the service.
5. Demonstrate human takeover before people depend on the fleet. Staff need independent access to pending work, authority to take it over, and capacity to meet a defined service minimum. Run timed drills with the fleet and its coordinator unavailable. Measure safety-stop time, unwanted restarts, handover time, and urgent work left uncovered. Review the evidence with service recipients as well as operators and engineers.
A small experiment with a sharp result
We enumerated 104 hypothetical attempts by a replacement robot to start an already assigned lift. We varied the timing of a resident’s refusal and the replacement attempt, the delay or loss of the fleet notice, and whether the replacement had access to direct confirmation. A refusal at the same time as a start attempt takes effect first. Direct confirmation, when available, is assumed to reflect the person’s decision accurately.
| Rule used at replacement | Unwanted starts in 72 refused cases | Starts in 32 still-authorized cases |
|---|---|---|
| Stop applies only to the first robot | 72 | 32 |
| Replacement honors a fleet refusal notice when it arrives | 36 | 32 |
| Replacement honors the notice and requires fresh consent | 0 | 16 |
Among all 32 deterministic start-or-pause rules over the observations in this model, fresh confirmation served the most valid starts among the rules that allowed zero unwanted starts. When neither a refusal notice nor direct confirmation was available, 16 authorized cases looked identical to 18 refused cases. Pausing protected the refusal at a cost to service. Improving the way people can communicate, and providing someone to help when they cannot, addresses that cost.
These are counts from a defined set of simulated cases. The experiment covers one decision at the start of an activity. Physical stopping, accessibility in real use, the consequences of mistaken confirmation, and the capacity of staff to take over each need their own evidence.
Let readers try it
Readers can compare interpretations of Asimov, step through a lift and a stale fleet assignment, explore delays in a ward queue, and test whether staff can take over after a fleet outage. The lab reproduces the 104-case benchmark and shows the event that caused each policy to act or pause.
Who gets to stop a robot?
When someone presses stop, whose work should stop?
Four small, deterministic cases. In each, two policies face exactly the same events, tick by tick. Pick a case, predict what happens, then step through and read why. Every physical action is a symbolic state change, not a motion plan.
- 1Pick a case
- 2Predict
- 3Step or play
- 4Read the outcome and log
Case settings
Changing a setting restarts both tracks from tick 0.
Predict before you play
Policy tracks
Act at tick 0
Each action happens now, identically in both tracks. Rejected requests stay rejected.
Resident
Safety controls
Staff
What happened
| Question | Track 1 | Track 2 |
|---|
Decision log
Full event trace: every tick at once, no animation (shows the outcome)
Return to the resident in the chair. Her first refusal needs to be enough. Her body still needs support. And if she asks for the lift again later, there should be someone who can help. A better set of robot rules can be judged by whether all three things happen.




