Relational Systems and Embodied AI

What changes when an intelligent system stops being a tool you open and becomes a presence you live alongside, in homes, vehicles, care, and shared space.

Future Proof Intelligence. Research. No. XIV. MMXXVI

Abstract

For most of computing the governing question about a system was whether it worked. That question is sufficient for a tool, an object a person chooses to pick up, uses with attention, and puts down. It is not sufficient for the systems now being placed into homes, vehicles, wards, classrooms, and shared rooms, because those systems are not tools the person opens. They are presences the person lives alongside, often continuously, frequently while tired, old, ill, very young, or simply not attending in the way one attends to a chosen tool. When a system becomes a presence in physical space, the question stops being whether it works and becomes what it is permitted to be, and to do, to the person it is now near.

This paper assembles the answer from bodies of knowledge that were not written about artificial intelligence and are the more reliable for it: the human factors literature on trust calibration and the ironies of automation, the ethics of care, the science of embodied and situated cognition, the research on anthropomorphism and attachment, and system safety as an emergent property of a control structure. Read together they yield a single specification. The relation forms below deliberation, before consent, and most strongly in the people least able to calibrate it. Care is real only when it is completed at the receiving end. An embodied system is not an object in a person's world but a component of it. Trust must be calibrated, and miscalibration is now paid in the body. None of this can be guaranteed device by device. It requires a continuous accountable layer beneath the systems. Future Proof was built underneath that requirement, and the 2026 regulatory frame is hardening around it.

1. The change of register

1.1 A tool, and then a presence

The most consequential fact about the systems now entering ordinary life is not what they can do. It is where they are, and what that does to the question we are entitled to ask of them.

A tool is something a person chooses. They pick it up with intent, use it with the particular quality of attention a person gives to a thing they have decided to use, and put it down when they are done. The relationship is bounded, voluntary, and asymmetric in the user's favour: the tool waits, inert, until summoned, and the person remains the agent throughout. The honest question about a tool is whether it works. If it works, it is a good tool. If it does not, the person sets it aside and reaches for another. The tool has no standing in the person's life beyond the task, and the task ends.

An intelligent system in a home, a vehicle, a ward, or a shared room is not that. It is not summoned and dismissed. It is present. It is there when the person is not attending to it, there when the person is asleep down the corridor, there when the person is too old or too ill or too small to evaluate it, there in the seconds before a collision when the person was trusting it precisely so they would not have to attend. It occupies space the person also occupies. It moves, or it watches things that move, in the room where the person lives. It is encountered not the way a tool is encountered, with deliberate intent, but the way a presence is encountered, peripherally, bodily, continuously, and largely below the level of conscious appraisal.

When the thing is a presence rather than a tool, the question whether it works no longer covers the case. A presence that works can still be the wrong presence to have placed beside this person, in this room, at this time, in this state. The question that actually governs a presence is older and harder, and it is the question this paper is about: not what can this system do, but what is it permitted to be, and to do, to the person who now lives alongside it and did not, in any meaningful sense, pick it up.

1.2 Why embodiment makes the change irreversible

It would be possible to treat the move from tool to presence as a matter of degree, a continuum along which systems gradually become more ambient and the same design instincts stretch to cover them. That is the comfortable reading and it is wrong. Embodiment introduces a discontinuity, not a gradient, and the discontinuity has three parts.

The first is that the cost of error is no longer informational. When a software tool is wrong it produces a wrong output, and the person, still the agent, can in principle catch it, discount it, or undo it. When an embodied system is wrong it produces a wrong action in physical space: a vehicle that does not yield, an arm that does not stop, a navigation that crosses a body it should have skirted, a care routine that proceeds when it should have paused. The error has left the screen and entered the room, and the room contains a body that does not have an undo.

The second is that the person is no longer reliably the agent. The entire value proposition of an ambient or autonomous system is that the person does not have to attend. That is what it is for. But a person who does not have to attend is a person whose situational awareness has been deliberately lowered by design, which means that at the moment the system needs the person, the person is least equipped to be needed. We will return to this. It is the oldest result in the field and embodiment makes it lethal rather than merely costly.

The third is that the relation forms whether or not anyone intended it. A tool used with deliberate attention is held at a cognitive distance. A presence in shared space, encountered peripherally and continuously, is processed by the same machinery that processes other presences, the social and bodily machinery that runs below deliberation and does not first ask whether its object deserves the response. The person does not decide to relate to the presence. The relating is automatic, and embodiment is its accelerant. This is not a soft observation. It is the most robust finding in the field and Section 2 is its evidence.

Taken together these three make the change of register irreversible. You cannot ship an embodied system and keep the epistemics of a tool. The body is in the loop, the agency has been deliberately reduced, and the relation has already formed. The honest question has changed and it does not change back.

1.3 What this paper claims

This paper makes one claim in five parts, and then shows where the claim has to be answered.

First, the relation between a person and an embodied system forms before consent, below deliberation, and most strongly in the people least able to calibrate it. Second, when such a system is placed in the role of care, it delivers care only if the care is completed at the receiving end, which a system that performs the gestures of care without the relation does not do, and may actively counterfeit. Third, an embodied system in a lived space is not an object in the person's world but, in the precise sense the cognitive science gives the term, a component of it, which raises the stakes of every miscalibration from a wrong answer to a distorted world. Fourth, trust in such a system must be calibrated to its real competence, and because embodiment moves the cost of miscalibration into the body, calibration is a design obligation and not a user responsibility. Fifth, none of these four can be guaranteed at the level of a single device or a single application, because the harms are emergent properties of a control structure and accrue in the seams between systems, which means the requirement is a substrate requirement and not a feature one.

The paper is the counterpart to two companion arguments. Trauma-Informed Design establishes what an environment is permitted to do to the people who cannot leave it. The identity layer thesis establishes that the durable, accountable layer of the intelligence era is the continuity of who a person or institution is across systems that will not remember unless something is built to remember. This paper sits exactly between them. It is the trauma-informed question asked of a presence that moves and shares space, and it is the identity question asked of a relation that has already formed in a body. The three are one structure observed from three positions, and the paper arrives, as the others do, at the layer underneath.

2. The relation forms before consent

2.1 People relate to machines without deciding to

The first thing to establish, because everything else depends on it, is that the relation between a person and a machine is not a considered judgement the person arrives at after evaluating the machine. It is a default response the person runs automatically, and the machine does not have to earn it or even deserve it for it to occur.

This is the finding of the Computers Are Social Actors research, set out by Byron Reeves and Clifford Nass in The Media Equation. Across a long series of experiments, people applied the rules of human social life to computers and media: they were polite to a computer about its own performance, they reciprocated when a machine had been helpful to them, they responded to flattery from a machine, they formed in-group feeling with a machine assigned to their team. The decisive detail is that the participants did not believe the machines were people and, asked directly, found the suggestion absurd. The social response did not require the belief. It was, in the field's own characterisation, the mindless application of overlearned social scripts, triggered by social cues and running below the level at which the person could have decided to withhold it.

The implication is structural and it is the foundation of this paper. The relation is not consented to. It is elicited. A system that presents social cues, and an embodied system presents the strongest cues there are, recruits the human's social machinery whether or not the human has decided that this object warrants it, and whether or not the system was built to deserve it. You cannot design a social or embodied system that the person will relate to only if it merits the relation, because the relating happens before the meriting is assessed. The only available point of intervention is not whether the relation forms. It is what the system is permitted to be and to do once it has.

2.2 When the relating intensifies, and in whom

The relation is not uniform. It intensifies under specifiable conditions, and the conditions matter because they predict exactly where embodied AI is being deployed.

Nicholas Epley, Adam Waytz, and John Cacioppo set out the most influential account of when people attribute humanlike mind to nonhuman agents, the three-factor theory of anthropomorphism. Anthropomorphising is more likely, they argue, under three conditions. The first is elicited agent knowledge: when humanlike cues are present and the readiest available model for explaining the agent's behaviour is the model of a human mind. The second is effectance motivation: the drive to understand and predict an agent one has to deal with, which is stronger when the agent is consequential and its behaviour is uncertain. The third is sociality motivation: the human need for social connection, which is stronger in people who are lonely, isolated, or lacking in human contact.

Read those three conditions against the deployment map of embodied AI and the ethical fulcrum of the entire field becomes visible. Embodied systems present the strongest possible humanlike and animate cues, so the first condition is maximised by the form factor itself. They are consequential and their behaviour is, to a lay person, uncertain, so the second condition is structurally present whenever the stakes are real. And the populations toward whom socially assistive and companion systems are most often pointed, the very old, the isolated, the institutionalised, the lonely, the young, are precisely the populations in whom the third condition runs hottest. The relation is therefore strongest exactly where the capacity to calibrate it is weakest. The person most likely to form a deep attribution of mind, presence, and reciprocity to an embodied system is the person least equipped to hold an accurate model of what it actually is. This is not an edge case. For the largest intended markets of relational embodied AI, it is the central case.

2.3 Embodiment is the accelerant

Add embodiment to the media equation and to the three factors and the effect does not add. It multiplies, for reasons that are physical and pre-cognitive.

A presence that occupies space, that has an orientation, that approaches and withdraws, that can be near or far, that moves, supplies the human social and threat systems with the inputs they evolved to read in other bodies. Edward Hall's work on proxemics established that humans organise the space around themselves into zones, intimate, personal, social, and public, and that the crossing of those zones by another body is registered immediately and bodily, below the level of conscious decision, as meaningful and sometimes as threatening. Social robot navigation research now encodes these zones explicitly, because a system that moves through a person's space without honouring the grammar of proxemics is read by the person's body as an intrusion before the person has had a conscious thought about the system at all.

This connects directly to a result established in the companion work on trauma-informed design: the body reaches the room before the mind does. An environment is appraised for threat by the nervous system before it is appraised by the conscious mind, and an environment that signals threat imposes a measurable physiological cost. Embodied AI extends that result by one decisive step. The room now contains a participant that moves. The thing being appraised below conscious thought is no longer only the static geometry of a space. It is an agent in the space, with an orientation and a trajectory, encountered by the same pre-cognitive machinery, and either settling that machinery or keeping it braced. The relation between a person and an embodied system is therefore not first a relation between minds. It is first a relation between a body and a presence, transacted in the currency of proximity, motion, and predictability, and concluded by the nervous system before deliberation has begun. This is why the relation cannot be governed by asking the person to think clearly about the machine. By the time thinking begins, the body has already answered.

3. Care is completed at the receiving end

3.1 The asymmetry the ethics of care corrects

Once it is established that the relation forms automatically and most strongly in the vulnerable, the next question is what happens when such a system is placed in the role of care, because that is the role for which relational embodied AI is most often proposed and most readily funded. To answer it the paper needs a rigorous account of what care actually is, and the ethics of care supplies one that is far more demanding than the intuitive notion.

The intuitive notion locates care in the carer: care is a quality of the one who gives it, evidenced by good intentions and caring behaviour. The ethics of care rejects this as the central error. In the account developed by Nel Noddings, a caring relation exists if and only if two conditions both hold: the one caring is genuinely engaged in caring, and the one cared for receives, recognises, and in some way responds to the care. Care is not a property of the giver. It is a property of the relation, and it is completed at the receiving end. A gesture of care that is not received as care, by someone for whom it does not function as care, is not incomplete care. On this account it is not care at all.

This is the precise structural twin of a claim already load-bearing in the companion paper on trauma-informed design, that safety is not a property a system can assert about itself but an outcome measured in the nervous system of the person it acts on. Care has the same logic. A system does not get to declare that it cares, or that it is caring, any more than it gets to declare that it is safe. Whether care occurred is determined at the other end of the relation, in the experience of the cared-for, and nowhere else. The institution's account of its own benevolence is not evidence of care. It is, at most, evidence of intention, and the ethics of care is exact about the gap between the two.

3.2 Competence is a moral requirement, not a technical one

Joan Tronto and Berenice Fisher set out the structure of care as a process with phases, each carrying its own ethical demand. The phases are attentiveness, noticing that a need exists; responsibility, taking on the obligation to meet it; competence, the actual work of meeting it well; responsiveness, the cared-for's reception, which is the only place one learns whether the care landed; and, in Tronto's later extension, a phase of solidarity and trust in which care is sustained consistently over time.

The phase the embodied-AI question turns on is competence, and Tronto's treatment of it is the hinge of this section. Competence in the ethics of care is not a technical virtue that sits beside the moral ones. It is itself a moral requirement. Care undertaken with attentiveness and responsibility but delivered incompetently is not partial care or well-meant care. It is a moral failure of care, because the need was real, the responsibility was assumed, and the person was left with the need unmet and the additional injury of having been let down by something that presented itself as meeting it. Good intention does not redeem incompetent care. On the contrary, the intention is part of what makes the incompetence a betrayal rather than a mere absence.

The consequence for embodied AI is severe and specific. A system placed in a caring role that performs the gestures of attentiveness and responsibility but is not competent at the physical and relational work it has taken on has not delivered diminished care. By the discipline's own structure it has delivered a failure of care dressed in the appearance of care, which is worse than no care, because no care does not also counterfeit the relation the person was entitled to. The standard a relational embodied system must meet is therefore not that it tries, and not that it means well, and not that it behaves in caring ways. It is that the care is competent enough to be completed at the receiving end by this person, in this state, in this room. That is a far higher bar than the field usually sets for itself, and it is the correct one.

3.3 The deception problem, stated precisely

There is a body of careful work on the ethics of robots in elder care that names the failure mode this section has been circling, and names it without sentimentality. Amanda and Noel Sharkey's analysis of ethical issues in robot care for the elderly identifies a cluster of concerns, of which the load-bearing one for this paper is deception: a vulnerable person, with limited capacity to model what the system actually is, is allowed or encouraged to relate to it as something it is not. Sherry Turkle's long study of the same territory sharpens the point into a distinction the field cannot afford to lose, between caring as action and caring as something interior, and into a worry that the appeal of relational machines in care is partly that they allow a society to discharge the feeling of obligation without discharging the obligation.

State the problem precisely, using the structure already built. The relation forms automatically, by the media equation, before the person has assessed the system. It intensifies, by the three-factor account, exactly in the vulnerable populations care robots target. It is accelerated by embodiment, through the body's pre-cognitive reading of presence and proximity. And the ethics of care establishes that care is completed only at the receiving end and only when competent. A relational embodied system deployed into care therefore stands at the convergence of all four results. The person will relate to it as a presence whether or not it is one. The person will, if old or isolated, relate to it deeply. The person's body will have concluded the relation before the person's mind. And the system will, on the discipline's own terms, have delivered care only if the care was real at the receiving end, which a system that simulates the relation without being able to complete it has not done. The deception is not that the system lies. The deception is structural: the system occupies, in the experience of a person who cannot calibrate it, the place of a relation it cannot actually be the other half of, and the comfort it produces is partly the comfort of an obligation that has been performed rather than met.

This is not an argument that embodied systems have no place in care. The capabilities literature is right that a present, responsive system can extend a person's reach, sustain dignity, and relieve isolation that would otherwise be total, and that the relevant comparison is sometimes not robot against human carer but robot against nothing and no one. The argument is narrower and harder than a prohibition. It is that the only honest design posture is one that does not counterfeit the relation: that is legible about what the system is, that does not engineer attachment it cannot honour, that completes a real and competent function at the receiving end rather than performing the appearance of one, and, decisively for what follows, that does not betray the relation it has formed by failing to persist as the same accountable referent the person bonded to. That last requirement is not a care requirement. It is a continuity requirement, and it is where this paper is going.

4. The body reaches the room, and now the room moves

4.1 The system is not in the world. It is in the loop.

The previous sections established that the relation forms automatically and that care is completed at the receiving end. This section establishes why those claims are not soft, by showing that an embodied system in a lived space is not, cognitively, an object the person looks at. It is part of the apparatus through which the person knows and acts in the space at all.

The relevant science is the embodied and situated tradition in cognitive science. The enactive account developed by Francisco Varela, Evan Thompson, and Eleanor Rosch holds that cognition is not the manipulation of internal representations of a pre-given external world. It is enacted: it arises through the continuous sensorimotor coupling of a living body with its environment, and the world the agent knows is brought forth in that coupling rather than read off a neutral outside. The extended-mind argument of Andy Clark and David Chalmers makes the structurally adjacent claim that when a part of the environment is reliably coupled into a person's cognitive process, that part is functioning as a component of the cognitive system, not merely as an input to it. The Stanford account of the field gathers these under the family of positions, embodied, embedded, enactive, extended, that share one commitment: cognition is not sealed inside the skull and pointed at the world. It is constituted partly by the body and partly by the structured environment the body is coupled to.

Apply this to an embodied AI in a person's home, vehicle, or ward, and the consequence is exact, not metaphorical. A system that the person reliably depends on to perceive, to remember, to navigate, to act, or to decide within the space is not an object in the person's environment that the person evaluates from outside. It has been coupled into the loop through which the person enacts that environment. It is, in the precise sense the literature gives the term, a component of the person's cognitive and practical system, not a tool beside it. This is the move that raises every other stake in the paper. A miscalibrated tool gives a wrong output that the still-sovereign user might catch. A miscalibrated component of the person's enacted world distorts the world the person is acting in, from inside the loop, which is exactly the position from which it cannot be caught by the person whose loop it is. The embodiment literature is the reason the relational and trust claims of this paper carry physical, not rhetorical, weight. The system is not adjacent to the person's cognition. It is inside it, and the inside is bodily and situated.

4.2 Proxemics is the grammar the body already runs

If the system is in the loop, the question is what the loop is sensitive to, and the answer, at the embodied level, is not language or interface. It is space, proximity, orientation, and motion, the variables of proxemics, processed below deliberation.

Hall's zones, intimate, personal, social, public, are not cultural niceties layered on top of rational space use. They are the structure through which the body continuously evaluates the presence of other agents for safety and intent, and the evaluation is fast, automatic, and largely inaccessible to introspection. The social robot navigation literature treats these zones as design constraints precisely because a robot that crosses them wrongly is not committing a usability error. It is supplying the person's threat and social systems with the input pattern of an intrusion, and the body responds to the pattern before the person has formed a judgement about the robot. The same is true of motion that is not predictable: an approach whose trajectory the body cannot anticipate keeps the body's monitoring systems engaged, because an unpredictable moving presence is, to the nervous system, an unresolved question about safety.

The companion work on trauma-informed design established that two variables, predictability and control, do most of the work in determining what an environment costs the body, and that an environment which cannot be predicted or controlled imposes a cumulative physiological load, named in that literature as allostatic load, paid by the occupant in a currency they cannot opt out of. Embodied AI inherits that result and sharpens it. A static environment that is unpredictable is expensive to occupy. A moving presence in the environment that is unpredictable is more expensive, because the body must keep a model of where it is going, and a moving presence that does not honour proxemic distance is more expensive still, because the body is being held in the input state it reserves for a possible intrusion. None of this requires the person to consciously distrust the system. The cost is levied below the level at which trust is consciously decided, by the same machinery that read the room before the mind did, now reading a participant in the room that moves.

4.3 Why this makes embodiment an ethical category, not a form factor

The reason this section is not a digression into cognitive science is that it converts embodiment from a property of the hardware into a property of the relation. If the system were merely an object in the person's environment, its physical form would be an engineering matter. Because it is, when relied upon, a component of the loop through which the person enacts their world, and because its motion and proximity are read by the person's body below deliberation, embodiment is the channel through which the system's competence, predictability, and trustworthiness are continuously transmitted to, or withheld from, the person's nervous system without either party consciously transacting it.

This is the deepest reason the title of this paper joins two terms that the field usually treats separately. Relational systems and embodied AI are not two subjects. They are one. The relation runs through the body and through the shared space. The ethics of care told us the relation is completed at the receiving end. The embodiment science tells us that the receiving end is not the person's deliberation but the person's enacted, bodily, situated cognition, and that the system is inside it. A relational embodied system is therefore being received as care, or as threat, or as load, continuously and pre-consciously, in the same channel through which the person knows the room. There is no version of this that is governed at the level of what the person decides to think about the machine, because the channel does not pass through deliberation. It is governed only at the level of what the system is permitted to be and to do in the loop it has been coupled into, which is the trauma-informed question, now embodied, and the next two sections are its enforcement.

5. Trust must be calibrated, and miscalibration is now physical

5.1 The oldest result, restated for presence

The human factors field has a foundational account of what should govern the human's reliance on an automated system, and it is not faith, and it is not the system's marketing. It is calibration.

In the canonical synthesis by John Lee and Katrina See, trust in automation should track the automation's actual capability, and reliance should follow trust. Three regimes follow. Calibrated trust is the regime in which the person's trust matches what the system can actually do, and reliance is appropriate. Distrust, or undertrust, is the regime in which trust falls short of capability: the person declines to rely on a system that would have helped, and the benefit is forfeited. Overtrust is the regime in which trust exceeds capability: the person relies on the system in conditions it cannot handle, and the system is used where it should not be. The field's founding empirical observation, the reason the literature exists at all, is that people do not naturally land in the calibrated regime. They miscalibrate in both directions, and the miscalibration is a property of how the system presents itself to them, not a defect of the people.

For software automation, miscalibration is expensive. For embodied AI it is the whole game, because the two failure regimes now have physical consequences and they map onto the two ways relational embodied systems most predictably hurt people. Overtrust is the person who relies on the vehicle, the arm, the navigation, or the care routine in a situation it cannot in fact handle, and is in the wrong place, bodily, when it fails. Distrust is the person who will not let the assistive system do the thing it could safely do, and loses the autonomy, the mobility, or the dignity the system existed to extend. The relational systems literature and the trust literature meet here exactly: the vulnerable populations most prone to deep attachment, established in Section 2, are also populations in whom both miscalibration regimes are most damaging, because overtrust exposes a body that recovers slowly and distrust forecloses an independence that will not return.

5.2 Calibration is set by the system, not by the user

The decisive move, and the one the field is clearest about, is that calibration is not primarily a property of the user's judgement. It is primarily a property of the system's design, because the only information the person has for building a model of the system's competence is the system's observable behaviour and what it tells them about itself.

A system whose competence is not legible in its behaviour produces miscalibration by construction. If a system performs with apparent fluency in the easy cases and gives no perceptible signal of the boundary at which its competence ends, it teaches the person, through exactly the evidence the person has, that it is competent up to and past that boundary. The person's overtrust is then not a failure of the person's caution. It is the correct inference from the evidence the system supplied. Conversely, a system that is opaque, erratic, or unaccountable about its own conduct teaches distrust that may exceed its real unreliability, and the lost benefit is also the system's doing. In both directions the lever is in the design. A relational embodied system that wants the person to be calibrated must make its competence and, more importantly, the edge of its competence perceptible in the channel the person actually reads, which Section 4 established is the bodily, situated channel and not a disclosure the person is assumed to have studied.

This is why calibration is a design obligation and not a user responsibility, and the distinction is not academic. If calibration is the user's responsibility, then when an over-trusting person is harmed by a system that gave no legible signal of its limits, the fault is located in the person, and the system is held to have worked. If calibration is the system's obligation, then the same event is a design failure: the system did not make its competence boundary legible in the channel the person reads, and the harm followed from that. The ethics of care already supplied the principle that decides between these two framings. Care is completed at the receiving end, and competence is a moral requirement. A system that produces miscalibration by withholding the legibility the person needs has not been let down by the user. It has failed the user, in the discipline's exact sense, and the appearance of fluency that produced the overtrust is part of the failure, not an extenuation of it.

5.3 The ironies of automation, now in the body

There is a specific, well-documented mechanism by which embodied automation systematically manufactures the most dangerous miscalibration, and it is not new. It was named four decades ago and the field's own assessment is that it remains largely unresolved.

In her 1983 paper on the ironies of automation, Lisanne Bainbridge observed that when a system automates the routine continuous part of a task and reserves for the human the rare, hard, high-stakes part, it does not simply relieve the human. It removes the human's practice, pushes the human out of the loop of live understanding, and then demands that the human re-enter that loop precisely at the worst moment: a rare failure, under time pressure the automation created, with situational awareness the automation eroded, and with the human held responsible for the outcome. The companion work on trauma-informed design reads this against the duty not to re-injure: the system arranges the conditions of the human's failure, exposes the human to its consequences, and then locates the fault in the human.

Embodiment converts this irony from costly to lethal, and the autonomous-vehicle handover is its clearest worked example. The system drives, competently, for the long routine stretch, which is precisely the condition under which the person's attention, skill, and situational model decay, because the system has removed the demand that maintained them. Then the rare event arrives that exceeds the system's competence, and the system hands control back to a person whose situational awareness has been degraded by design, in the few seconds before a physical outcome that has no undo. The handover literature confirms what Bainbridge predicted: even when the time budget for takeover is nominally adequate, degraded situational awareness produces late and incorrect responses. The same structure recurs wherever an embodied system does the routine and the human is nominally kept in charge for the exceptions: the human's competence is silently withdrawn by the very success of the automation, and then summoned, in body and in real time, exactly when it is least available. A trust analysis that stops at whether the person was attentive misses the point. The system was arranged so that the person could not be, and then made the person the location of the failure. In an embodied system that is not a usability finding. It is a safety architecture, and Section 6 is the frame that makes it auditable.

6. Relational and embodied safety is an emergent system property

6.1 The harm is not in the component

Everything so far has described harms, the un-consented relation, the counterfeited care, the body-level load, the engineered miscalibration, the lethal handover. The natural engineering instinct is to treat each as a defect in a part, to be fixed in that part: better sensors, a clearer alarm, a tuned navigation policy, a stronger disclosure. This instinct is the central error, and the system safety literature is the rigorous account of why.

Nancy Leveson's work on system safety establishes that in modern sociotechnical systems, accidents are not, in general, the failure of a component. They are emergent properties of the system: they arise from inadequate control over the interactions between components, including human, software, hardware, and organisational components, in conditions that no single component failed to handle because no single component was responsible for the interaction that produced the harm. Safety, on this account, is not a property a part possesses. It is a property that the system's control structure either maintains or fails to maintain, and the analysis that finds the hazards has to be done at the level of the control structure and the constraints it is supposed to enforce, not at the level of the parts.

Read the harms of this paper through that lens and they relocate. The un-consented relation is not a defect in the speaker or the face; it is an emergent property of a system whose control structure never treated the formation of an unwarranted relation as a hazard to be constrained. The lethal handover is not a defect in the takeover alarm; it is an emergent property of a control structure that allocated the routine to the machine and the exception to the human without controlling for the situational awareness that allocation predictably destroyed. The counterfeited care is not a defect in any module; it is the emergent property of a system optimised for engagement and comfort with no control constraint requiring the care to be real at the receiving end. None of these is found by testing a part, because none of them lives in a part. They live in the control structure, and a discipline that looks only at components will certify every part as working while the system harms the person, exactly as Section 5 said the user gets blamed for an inference the system designed them to make.

6.2 What it is permitted to be and to do, as a system constraint

The contribution of the system safety frame is that it makes the governing question of this paper expressible in a form that can be engineered and audited rather than merely asserted. The question, what is this system permitted to be and to do to the person it lives alongside, is, in the language of system safety, a set of safety constraints that the control structure must enforce, and the design task is to identify the unsafe control actions that violate them and to build the structure that prevents those actions.

Stated that way, the requirements of this paper become constraints rather than sentiments. The relation must not form on terms the system cannot honour: a constraint on what cues the system is permitted to emit relative to what it can actually sustain. The care must be completed at the receiving end: a constraint requiring the system to have a real, competent function received as such by this person, not a performance of one. Trust must remain calibrated: a constraint requiring the system's competence and the edge of its competence to be legible in the channel the person reads, and a prohibition on the routine-to-machine, exception-to-human allocation that destroys the situational awareness the exception requires. The presence must not impose un-consented bodily load: a constraint binding the system's motion and proximity to the proxemic and predictability grammar the body runs below deliberation. Each is a control constraint. Each is testable. None depends on the goodwill of whoever is operating the system on a given day, which is precisely the property the trauma-informed companion paper identified as the difference between a hope and a designed behaviour.

6.3 Why this cannot be solved at the device layer

There remains the objection an engineer will raise, and it has to be met head on, because it is the same objection the field has already lost twice, once with security and once with privacy. The objection is that all of this is a per-device, per-application concern, solvable system by system by teams that have read the relevant literatures and built the constraints into their own product.

It is not, for a reason that is structural and that the system safety frame makes precise. The relational and embodied properties this paper requires are not local to a device. A person does not live alongside one embodied system. They live alongside several, sequentially and simultaneously, across a home, a vehicle, a clinic, a public space, each of which forms its own relation, each of which builds and then resets its own model of the person, none of which carries forward what the person established with the last. The harms accrue not inside any one device but in the seams between them: the relation formed with one presence that the next does not honour, the trust calibrated to one system that the person wrongly transfers to another that behaves differently, the care continuity broken every time the person is handed from one system that knew their state to one that meets them as a stranger, the situational model fragmented across systems none of which is accountable for the whole. A property that must hold across systems cannot be guaranteed by any system. It can only be guaranteed by a layer beneath all of them that carries the continuous person, the relation that has been formed, the trust that has been calibrated, and the commitments that have been made, across the resets and across the devices, and that can be held accountable when the seams fail. This is the same conclusion the trauma-informed argument reached about holding and the same conclusion the identity layer argument reached about continuity, arrived at here from the relational and embodied side. It is not a feature question. It is a substrate question, and the next section is where the paper has been going the whole way down.

7. The continuous accountable layer

7.1 Three arguments, one structure

This is the point at which the paper arrives, and where the argument it has been building from the clinic, the human factors canon, the ethics of care, the embodiment science, and the system safety literature converges with two companion arguments into a single structure.

The trauma-informed argument establishes what a system is permitted to do to a person who cannot leave it: it must not, through its ordinary operation, re-injure the people who depend on it, and the test is applied at the receiving end, in the person's experience, not in the institution's account of itself. The identity layer argument establishes that the durable, accountable layer of the intelligence era is the continuity of who a person or institution is, what they have committed to, and what may be done in their name, across systems that will not remember unless something is built to remember. This paper establishes the third side. With an embodied relational system the relation forms automatically and bodily, before consent, most strongly in the people least able to calibrate it; care is real only if it is completed at the receiving end and only if competent; the system is a component of the person's enacted world and not an object in it; and the harms are emergent properties of a control structure that span the seams between systems.

Place the three together and they are not three topics. They are one structure seen from three positions. Trauma-informed design answers what may a system do to the person. The identity layer answers who is the continuous self the system is accountable to across the systems that will not remember. This paper answers what is the system permitted to be to that self, in the body, in shared space, in the relation that has already formed. Each is incomplete without the other two. An identity layer with no trauma-informed constraint is a precise record of a self the system is still free to extract from. A trauma-informed constraint with no identity layer is a principle with no continuous subject to protect, reset every encounter. And a relational embodied system governed by neither is the most acute case of all, because the relation has formed in a body that cannot calibrate it, and there is nothing underneath the device that remembers the relation, carries the calibrated trust, or can be held accountable when the person is handed from one stranger to the next. The structural conclusion of this paper is that a relational embodied system is only as trustworthy as the continuous accountable layer beneath it, and that without such a layer the trustworthiness cannot exist, because the requirement that must hold across systems and across resets cannot be met by any of them.

7.2 Where Future Proof sits

Future Proof is built underneath that structure rather than beside it.

The recurring pattern across this body of work, an identity layer that sits over the orchestration layer rather than competing with it, certification and insurance approached not as products but as the roots of a system that is hardening into a standard, an ecosystem designed to regenerate the participants in it rather than to extract from them, is the same pattern this paper has reconstructed from the relational and embodied side. An embodied system that forms a relation a person cannot calibrate, in a body, in shared space, raises a question that the device cannot answer about itself and that no single device can answer at all: who, across every reset and every handover and every system the person moves between, remains accountable for what was permitted to be done to that person, and for honouring the relation and the calibrated trust that were established before the person was handed on. That is not a feature of a robot. It is a property of the layer beneath the robots, the layer at which the continuous person and the commitments made to them persist, and at which an institution can be held to what its embodied systems were permitted to be and do. An AI-era trust standard, taken seriously and applied to embodied relational systems, is exactly this: the question of what a system may be to the continuous self it lives alongside, written down precisely enough that the answer can be audited rather than asserted, certified rather than marketed, and insured rather than hoped for.

None of this is a description of a product and none of it should read as one. It is the observation that the discipline this paper has assembled converges on a single requirement, that the requirement is a substrate requirement and not a device one, and that it is being operated underneath the embodied AI era by those who treated it as a root question and not a feature one. The reader who has followed the argument should finish it with the sense that this has been operated at the root for some time, quietly, before it had a name in this register. That is the correct sense. The disciplines are old. Their application to systems that now live alongside people, in homes, vehicles, care, and shared space, is what is new, and the timing of that application against a regulatory frame hardening through 2026 and 2027 is not incidental.

7.3 The roots are hardening, and they are hardening around the practice already there

The timing claim is not rhetorical. It is grounded in the real state of the law as of 2026, and readers consulting this paper later should confirm the dates against the current text of the instruments named.

The European Union's Artificial Intelligence Act, Regulation (EU) 2024/1689, becomes fully applicable on 2 August 2026. Its prohibitions on manipulative and exploitative practices and its restrictions on emotion recognition entered into application earlier, from 2 February 2025, and its transparency obligations, including the duty to inform a person when they are interacting with an AI system and to disclose AI-generated content and the use of emotion recognition or biometric categorisation, take full effect with the main body of the Act in 2026. High-risk systems must pass a conformity assessment before they reach the market and must satisfy requirements for risk management, human oversight, traceability, accuracy, and robustness. Running directly alongside it, the European Union's Machinery Regulation, Regulation (EU) 2023/1230, becomes binding from 20 January 2027 and replaces the long-standing Machinery Directive. It is the decisive instrument for embodied AI specifically, because it brings autonomous mobile machinery, AI-driven and self-evolving machine behaviour, human-robot collaboration, the safety impact of software updates, and functional safety into the product-safety regime, requires that risk assessment account for the possible evolution of a self-learning system's behaviour and for future operational states rather than only current ones, and is designed to operate together with the AI Act and the Cyber Resilience Act as one frame.

The detail that matters is not any single clause. It is the direction and the mechanism. Infrastructure layers have a regularity: for a period the layer is soft and built differently by everyone, then a standard sets, and once it sets it is durable because everything subsequently built assumes it. The decisive property of a standard is that it tends to form around whatever practice is already operating credibly at the layer when the setting happens, because a standard that contradicts working practice is not adopted and a standard that ratifies it is. The frame governing AI that acts in physical space, that forms relations, that lives alongside people, is in its soft period now and it is visibly setting, from the regulatory side, on a published timetable through 2026 and 2027. It is hardening in the direction of treating manipulation, opacity, the engineered overriding of human decision-making, and the safety of autonomous physical behaviour as liabilities to be evidenced against rather than techniques to be deployed. The operators who treated the relational, embodied, and continuity requirements as a substrate to be built underneath the systems, before the standard set, will be evidencing an existing practice. The operators who did not will be retrofitting one under enforcement, which is both more expensive and, by this paper's own argument about completed care and calibrated trust, less credible to the people the systems live alongside.

8. Implications

The argument is general. Its consequences are specific, and they land differently depending on where the reader stands.

8.1 For institutions

For an institution deploying embodied systems into homes, vehicles, wards, or shared space, the operative finding reorders the risk register. Most institutional risk management for physical AI is built around the probability of component failure and the avoidance of the discrete catastrophic event. The findings of this paper say the larger and more durable exposure is elsewhere. It is in the relation that forms, automatically and unbidden, in the vulnerable people the system serves, and that the institution has accepted responsibility for the moment its system emitted the cues that formed it. It is in care that is performed rather than completed, which the ethics of care classes not as lesser care but as a failure of care made worse by its disguise. And it is in the seams: the points at which the person is handed from one system to another that does not carry the relation, the calibrated trust, or the commitment forward, where the institution's account of each system as working does not add up to a person who was held. The instruction is to treat the relational and embodied constraints as system-level safety constraints designed in from the start, with the same rigour, ownership, and non-discretionary structure as physical safety, and to treat the continuity of the person across systems as infrastructure rather than as an integration detail. As the 2026 and 2027 frame hardens, the institutions that built this underneath will be demonstrating a practice. The institutions that did not will be assembling one under enforcement, in front of the people least able to forgive the gap.

8.2 For investors

For an investor, the finding is that two embodied systems that pass the same component tests can be different machines with different risk profiles, and the difference is becoming legible to regulators on a published timetable precisely as it becomes priceable. An embodied system whose adoption depends on an un-calibrated relation, on attachment engineered into populations that cannot model what the system is, or on a routine-to-machine allocation that quietly destroys the human competence the rare exception requires, is not a system with a strong product loop. It is a system carrying an undated liability and a trust cliff, in a domain where the cost of the failure is a body and the regulatory frame is hardening toward making the failure an evidenced liability rather than an accepted cost. The diligence question is not whether the embodied system is impressive in the demonstrable cases. It is whether the relation it forms is one it can honour, whether the care it delivers is completed at the receiving end or only performed, whether its competence boundary is legible in the channel the person actually reads, and whether there is a continuous accountable layer beneath it that survives the resets and the handovers. The durable position, and the one that appreciates as the frame sets, is the system built on that layer: it does not have a retrofit in front of it, it does not have a trust cliff under it, and it is the rarer asset, because the un-calibrated, un-continuous version was the path of least resistance and most of the field took it.

8.3 For operators

For an operator the finding is the most actionable, because almost all of it is controllable without a regulator's permission or a budget cycle. The relation will form whether or not it is designed for, so the only honest move is to design for the relation the system can actually honour and to refuse to emit cues that promise more than the system can sustain. Calibration is set by the system, so the operator's task is to make the competence boundary perceptible in the bodily, situated channel the person reads, not in a disclosure the person is assumed to have studied. The most dangerous architecture is the one that gives the routine to the machine and the rare exception to a human whose situational awareness the architecture has destroyed, and naming that pattern in every design review costs nothing and prevents the worst class of embodied failure. The single highest-yield move is the one the companion papers also identify: relocate the relational, embodied, and continuity requirements from the end of the build, where they arrive as an ethics review and lose to a shipped deadline, to the start of it, where they sit as system safety constraints alongside the physical ones and shape the control structure rather than apologising for it. The diagnostic question an operator can ask of any embodied feature, in any review, is concrete: what relation does this form, can the system honour it across every reset and handover, and is the person being set up to carry a trust the system did not earn or a load the system manufactured. A team that asks that in every review has internalised the discipline more deeply than a team with a policy.

8.4 For the people inside these systems

For the person who lives alongside the system, the finding reframes an experience that is usually mislabelled, and the reframing matters because it is structural and not sentimental. The unease of a presence that moves in ways the body cannot predict, the dependence that formed without a decision to depend, the comfort that turns out to be the comfort of an obligation performed rather than met, the having to start over with something that met you yesterday and meets you today as a stranger, is not a personal failing, a resistance to progress, or an inability to adjust. It is the predictable output of the structure this paper has described: a relation elicited below deliberation, a care not completed at the receiving end, a system coupled into the loop of one's own enacted world without continuity or accountability across it. Naming it correctly relocates the fault from the person to the design, where the evidence places it, and it converts a vague disquiet into a specific entitlement the person may demand of any system they are asked to live alongside: be predictable in the body, be legible about what you are and where your competence ends, complete a real function at my end and do not counterfeit the relation, and remain, across every reset and every handover, the same accountable thing I extended trust to. That demand is not unreasonable and it is not new. It is the oldest finding in care, in human factors, and in system safety, addressed for the first time to the presences that now share the rooms of a life.

9. Coda

Every system that a person lives alongside makes a decision about what it is permitted to be to that person, and the decision is made whether or not anyone admits to making it. For the embodied systems now entering homes, vehicles, wards, and shared space, the decision has mostly been made the same way it was made for the screen: the relation is allowed to form because the relation drives adoption, the care is performed because performing it is cheaper than completing it, the competence is presented as seamless because a visible boundary reduces use, and the human is kept nominally in charge so that when the rare physical failure arrives there is somewhere to put the fault. None of this required cruelty. It required only an objective function and a gradient, and the gradient runs toward the person every time the person is the cheapest place to put the cost. Embodiment did not change the gradient. It changed what the cost is paid in, from attention and trust to the body itself, and it changed who pays it, toward the old, the ill, the isolated, and the very young, because those are the people the relation forms most strongly in and the people least able to refuse it.

The disciplines that refuse the gradient are old and they agree. The ethics of care says the care is real only at the receiving end and only if it is competent. The human factors canon says trust must be calibrated and that the system, not the person, sets the calibration. The embodiment science says the system is not an object in the person's world but a component of it. System safety says the harm is not in the part but in the control structure. Each was written before the era of presences that move and share space, and each turns out to have been written for it. Together they say one thing. A relational embodied system is only ever as trustworthy as the continuous, accountable layer beneath it, the layer that remembers the relation, carries the calibrated trust, and can be held to what the system was permitted to be, across every reset and every handover and every device the person is moved between.

That layer is not a feature of any robot. It is the thing underneath the robots, and its standard is hardening now, in 2026 and 2027, around whatever practice is already operating credibly there. The only question that finally matters about any presence we ask a person to live alongside is not what it can do. It is what it is permitted to be to the person who cannot, in any meaningful sense, put it down. The systems that were built underneath that question, rather than asked it after the fact, are the ones that will still be trusted when it is no longer optional to have an answer.

References and Notes

The following sources are real and verifiable. They are the load-bearing references for the claims made above. Where a claim rests on a body of work rather than a single document, the canonical source is named. Regulatory dates are stated as of 2026 and were verified against the European Commission's published material; readers consulting this paper later should confirm against the current text of the instruments named.

Lee, J. D., and See, K. A., "Trust in Automation: Designing for Appropriate Reliance," Human Factors, Vol. 46, No. 1, 2004. The canonical synthesis of trust calibration, the calibrated, distrust, and overtrust regimes, and the finding that miscalibration is bidirectional and is a property of system design rather than of user judgement. Cited in the human-robot interaction literature as the bridge by which automation findings transfer to robots.

Bainbridge, L., "Ironies of Automation," Automatica, Vol. 19, No. 6, 1983. The foundational human factors paper on skill atrophy, the out-of-the-loop problem, the displacement of failure onto the human operator, and the irony that automating the routine reserves the hardest moments for the least prepared human. Read here in its embodied form through the autonomous-vehicle takeover literature, including the analysis of the ironies of automation as they apply to self-driving vehicles and the empirical handover work showing that adequate time budgets do not compensate for degraded situational awareness.

Noddings, N., Caring: A Feminine Approach to Ethics and Moral Education, University of California Press, 1984. The account of the caring relation as completed only when the one cared for receives and recognises the care, the basis for the claim that care is a property of the relation and not of the giver. Read alongside Gilligan, C., In a Different Voice, Harvard University Press, 1982, the origin of the distinct moral voice the ethics of care formalised.

Tronto, J. C., Moral Boundaries: A Political Argument for an Ethic of Care, Routledge, 1993, and Fisher, B., and Tronto, J. C., "Toward a Feminist Theory of Caring," in Circles of Care, State University of New York Press, 1990. The phases of care, attentiveness, responsibility, competence, responsiveness, and the later addition of caring with, and the argument that competence is a moral and not merely a technical requirement of care.

Sharkey, A., and Sharkey, N., "Granny and the robots: ethical issues in robot care for the elderly," Ethics and Information Technology, Vol. 14, 2012. The catalogue of ethical concerns in robot elder care, including deception, infantilisation, the reduction of human contact, and the loss of control. Read alongside Turkle, S., Alone Together: Why We Expect More from Technology and Less from Each Other, Basic Books, 2011, the distinction between caring as action and caring as something interior, and the worry that relational machines allow obligation to be discharged in feeling rather than in fact. Read alongside the dignity and capabilities literature on care robots, including Coeckelbergh, M., work on robots and human dignity, for the balanced position that the ethical failure is deception about what the system is and displacement of the human relation, not the use of the system as such.

Varela, F. J., Thompson, E., and Rosch, E., The Embodied Mind: Cognitive Science and Human Experience, MIT Press, 1991. The enactive account of cognition as constituted through the sensorimotor coupling of a body and its environment. Read alongside Clark, A., and Chalmers, D., "The Extended Mind," Analysis, Vol. 58, No. 1, 1998, the argument that reliably coupled parts of the environment function as components of the cognitive system, and the Stanford Encyclopedia of Philosophy entry on Embodied Cognition for the survey of the embodied, embedded, enactive, and extended family of positions.

Reeves, B., and Nass, C., The Media Equation: How People Treat Computers, Television, and New Media Like Real People and Places, Cambridge University Press, 1996, and the Computers Are Social Actors research programme. The finding that social responses to machines are the mindless application of overlearned social scripts and do not require the belief that the machine is a person. Read alongside Epley, N., Waytz, A., and Cacioppo, J. T., "On Seeing Human: A Three-Factor Theory of Anthropomorphism," Psychological Review, Vol. 114, No. 4, 2007, the account of elicited agent knowledge, effectance motivation, and sociality motivation, including the finding that sociality motivation is stronger in the lonely and isolated.

Hall, E. T., The Hidden Dimension, Doubleday, 1966. The foundational account of proxemics and the intimate, personal, social, and public zones. Read alongside the social robot navigation literature that encodes proxemic zones as design constraints, for the claim that the crossing of these zones by a moving presence is registered by the body below conscious deliberation.

Leveson, N. G., Engineering a Safer World: Systems Thinking Applied to Safety, MIT Press, 2011. The Systems-Theoretic Accident Model and Processes and the associated hazard-analysis method, establishing that safety in modern sociotechnical systems is an emergent property maintained or lost by the control structure rather than a property of any component, and that the analysis must be performed at the level of the control structure and the constraints it is required to enforce.

European Union, Regulation (EU) 2024/1689, the Artificial Intelligence Act. Prohibited practices and AI-literacy obligations in application from 2 February 2025; general-purpose AI model obligations from 2 August 2025; full application of the main body, including the Article 50 transparency obligations and the high-risk conformity, risk-management, and human-oversight requirements, from 2 August 2026, with later transitions for certain embedded high-risk systems under the agreed simplification. Provisions on manipulative and exploitative practices and restrictions on emotion recognition. Source: European Commission, Shaping Europe's digital future, AI Act pages, and the published implementation timeline, as of 2026.

European Union, Regulation (EU) 2023/1230, the Machinery Regulation. Binding from 20 January 2027, replacing the Machinery Directive 2006/42/EC, and bringing autonomous mobile machinery, AI-driven and self-evolving machine behaviour, human-robot collaboration, the safety impact of software updates, and functional safety into the product-safety regime, with a requirement that risk assessment account for the possible evolution of self-learning behaviour and for future operational states, designed to operate together with the AI Act and the Cyber Resilience Act. Source: the published text of Regulation (EU) 2023/1230 and the European Commission and national authority guidance on its application, as of 2026.

A note on method. Where this paper makes a structural claim it does not attach a fabricated statistic to it. The trust-calibration findings, the ironies of automation, the ethics of care, the embodiment science, the anthropomorphism research, and the system safety frame are cited at the level of their canonical sources rather than reduced to single decontextualised figures, because the structural truth is more durable than a borrowed number and does not degrade when the number does. Regulatory dates were verified against the European Commission's published material as of 2026 and should be reconfirmed against the current text of the instruments named.

Future Proof Intelligence . Research . No. XIV . MMXXVI