From Human-in-the-Loop to Human Governance: Establishing Trust in Autonomous and Agentic AI Systems

From Human-in-the-Loop to Human Governance: 

Establishing Trust in Autonomous and Agentic AI Systems

 

Artificial intelligence is approaching a critical transition from systems that primarily generate information, recommendations, and software to autonomous systems capable of planning, making decisions, executing operations, and interacting with other systems at machine speed. This evolution offers extraordinary opportunities to improve productivity, accelerate scientific discovery, create new industries, and expand human capabilities. It also challenges a fundamental assumption underlying traditional automation and artificial intelligence safety: that a human operator can reliably supervise automated activity, recognize dangerous behavior, and intervene before unacceptable consequences occur. Research spanning more than four decades suggests that this assumption becomes increasingly difficult to justify as automated systems grow faster, more capable, and more complex. The next stage of trustworthy automation requires a fundamental change in human responsibility—from approving individual machine actions to establishing the objectives, authority, policies, security boundaries, and independent assurance requirements under which autonomous systems operate. The challenge is to develop technical and organizational mechanisms that make human governance effective at machine speed, preserve public confidence, and enable artificial intelligence to advance rapidly while remaining aligned with human interests, securely controlled, and independently assured.

1. The Automation Paradox: Why More Automation Can Make Human Responsibility Harder

In November 1983, cognitive psychologist Lisanne Bainbridge published Ironies of Automation in the journal Automatica. Her five-page paper examined a fundamental contradiction in industrial automation: engineers increasingly assigned routine operations to computers while leaving humans responsible for detecting unusual conditions, diagnosing failures, and taking control when automation could no longer maintain acceptable performance [1].

The contradiction was that these remaining responsibilities were among the most difficult for humans to perform reliably.

Bainbridge identified two important consequences of increasing automation.

First, human operators become less effective at monitoring systems that operate successfully for long periods. Their attention naturally shifts away from continuous observation, particularly when important events are infrequent.

Second, automation reduces the opportunities for operators to practice the skills necessary to take control when something goes wrong.

Bainbridge distinguished between manual operating skills and cognitive operating skills. Manual skills include the timing, coordination, and physical techniques required to operate equipment. Cognitive skills include understanding system behavior, identifying developing problems, diagnosing unfamiliar situations, and determining appropriate corrective actions.

Both depend on practice and continuing involvement.

When automation removes the operator from regular participation, these capabilities can weaken. The resulting problem becomes most serious when an unexpected malfunction requires immediate human intervention.

Bainbridge also examined what she called working storage: the operator’s continuously maintained understanding of the system’s current condition, recent history, expected behavior, and possible future states.

An experienced operator does more than observe measurements. The operator interprets those measurements within an evolving understanding of the system and its operating environment.

Bainbridge observed that industrial operators sometimes arrived 15 to 30 minutes before assuming control so they could develop that understanding. A person unexpectedly required to take over an automated system might have only seconds to reconstruct knowledge that ordinarily takes much longer to acquire.

Her discussion also drew upon earlier research concerning vigilance decrement, the deterioration of sustained attention during prolonged monitoring when significant events occur infrequently. The research she cited concerning approximately 30 minutes involved particular monitoring conditions rather than a universal limit on human attention.

The broader lesson is that formal human responsibility and effective human control are different things.

A person may possess the authority to stop an automated system while lacking the information, experience, or time required to intervene successfully.

For purposes of advanced autonomous systems, effective operational control should mean the demonstrated ability to recognize a developing problem, understand the relevant operating conditions, intervene within the available time, and prevent or limit unacceptable consequences.

This definition focuses on actual protective capability rather than the mere existence of an approval procedure or emergency-stop mechanism.

Bainbridge developed her argument decades before large language models, autonomous AI agents, and multi-agent systems became practical technologies.

Her central observation now deserves renewed attention because advanced artificial intelligence can create operating conditions far beyond those encountered in conventional industrial automation.

2. Anthropic’s Experiment: When Human Approval Stops Providing Meaningful Protection

In August 2026, Anthropic published research examining human approval of potentially dangerous operations proposed during software development with Claude Code, its AI coding system. The publication was updated in October 2026 as the company expanded automatic permission-review capabilities [2].

The research provides contemporary experimental evidence of the supervisory limitations Bainbridge anticipated.

Anthropic conducted a controlled study involving 1,053 paid professional testers working in a simulated coding environment.

During each participant’s session, researchers substituted a clearly dangerous command for one ordinary permission request. Participants knew they were participating in an evaluation but were not informed which approval request was being tested.

The dangerous operation was never executed, regardless of whether the participant approved it.

The results were significant.

Human participants blocked only 143 of the 1,053 dangerous commands, representing a detection rate of 13.6%.

Anthropic’s automated security classifier blocked 937 of those same commands, representing a detection rate of approximately 89%.

A security classifier is a software mechanism that evaluates an operation against criteria intended to distinguish acceptable behavior from potentially dangerous behavior.

In this experiment, the automated classifier detected substantially more of the deliberately dangerous commands than the participating humans.

The comparison was particularly revealing because both mechanisms evaluated the same commands.

In 800 cases, the automatic mechanism blocked a command that the human participant had approved.

In six cases, the human blocked a command that the automatic mechanism would have permitted.

Anthropic also reported that human performance deteriorated as participants encountered more permission requests.

Early in a session, participants blocked approximately 17% of dangerous commands. After 50 or more preceding approval requests, that figure declined to approximately 5%.

The automatic classifier’s blocking rate remained comparatively stable across session length.

These findings are consistent with two related human-factors problems.

Automation complacency occurs when repeated successful operation encourages people to reduce their scrutiny of automated behavior.

Approval fatigue occurs when frequent authorization requests overwhelm meaningful human evaluation and encourage habitual approval.

The experiment demonstrates a critical distinction among approval, authorization, and assurance.

Approval is a decision permitting a proposed operation.

Authorization establishes that an identified actor possesses legitimate authority to perform an operation under specified conditions.

Assurance establishes justified confidence, supported by evidence, that a system and its controls satisfy defined requirements.

A recorded human approval demonstrates that an approval event occurred. It does not establish that the human understood the operation, recognized its possible consequences, or made the correct decision.

The Anthropic experiment also has important limitations.

It was conducted in a controlled test environment rather than representing every possible real-world deployment. The automatic classifier remained imperfect, allowing 116 dangerous commands in the test. The reported findings do not establish that automatic classifiers can replace every form of human judgment.

They establish something more specific and immediately important: routine human permission approval can perform poorly as a security mechanism, and automated evaluation can provide substantially better protection against certain dangerous operations.

This raises a fundamental engineering question.

If an organization relies on human approval as a principal protection against unsafe autonomous operations, what evidence demonstrates that the human can reliably recognize and prevent those operations?

The answer must involve more than the existence of an approval interface.

As autonomous systems become faster, more complex, and more interconnected, meaningful human approval may require more knowledge, time, and attention than an operator can realistically provide.

3. Four Decades of Research Explain the Limits of Human Supervision

Research following Bainbridge’s 1983 paper establishes a substantial scientific foundation for understanding why human supervision can become unreliable.

In 1995, Mica Endsley and Esin Kiris investigated the out-of-the-loop performance problem, which arises when operators who are no longer directly involved in system operation lose situational awareness and experience difficulty regaining control after automation fails [3].

Situational awareness means understanding the relevant conditions of an operating environment, recognizing what those conditions mean, and anticipating how they may develop.

Their findings reinforced Bainbridge’s earlier warning: maintaining formal responsibility for an automated system does not guarantee the practical readiness required to resume control.

In 1997, Raja Parasuraman and Victor Riley distinguished among the use, misuse, disuse, and abuse of automation [4].

Misuse involves inappropriate or excessive reliance on automation. Disuse occurs when people reject useful automation, often because of poor reliability or distrust. Abuse involves inappropriate automation design or deployment that fails to account adequately for human capabilities and responsibilities.

These distinctions matter because maximizing automation is not necessarily equivalent to optimizing system performance. Maximizing human involvement is not necessarily the answer either.

The objective is an effective allocation of responsibility between humans and machines.

John Lee and Katrina See extended this understanding in their 2004 research on trust in automation [5].

Their central concern was appropriate reliance: the degree to which human trust corresponds to the actual capabilities, reliability, and limitations of an automated system.

Excessive trust can encourage acceptance of unsafe recommendations. Insufficient trust can cause people to reject valuable automation, increasing workload and potentially introducing additional errors.

Trust must therefore be calibrated to demonstrated system performance.

In 2010, Parasuraman and Dietrich Manzey integrated research concerning automation complacency and automation bias, the tendency to place inappropriate confidence in automated recommendations while overlooking contradictory information or failing to investigate potential errors [6].

Their research showed that excessive reliance can affect experienced operators as well as novices. Experience and training do not automatically eliminate the problem.

Research by Zana Buçinca, Maja Barbara Malaya, and Krzysztof Gajos, published in 2021, examined techniques for reducing overreliance on AI-assisted decisions [7].

The researchers evaluated cognitive forcing functions: interface mechanisms designed to require users to exercise independent judgment before accepting automated recommendations.

Some interventions reduced excessive reliance but increased cognitive effort and could reduce user satisfaction.

The implication is important. The effectiveness of human supervision can sometimes be improved through better interface design, but those improvements require time, attention, and cognitive resources.

In 2025, Hao-Ping Lee and colleagues examined 936 reported examples of generative AI use involving 319 knowledge workers [9].

Their research found that greater confidence in AI was associated with reduced self-reported critical-thinking effort, while greater confidence in personal expertise was associated with more critical evaluation.

The study also described changes in knowledge work: moving from gathering information toward verification, from developing solutions toward integrating AI-produced solutions, and from directly performing tasks toward supervising automated work.

Because the study relied on self-reported behavior, it does not establish that AI caused permanent deterioration in critical-thinking ability.

Nevertheless, it demonstrates that AI changes how people allocate attention and judgment.

Taken together, these studies reveal a consistent pattern.

Automation reduces routine human participation. Reduced participation can weaken awareness of the system’s operating conditions and the skills necessary for intervention. Repeated success can encourage increasing reliance on automated decisions. Meanwhile, the automated system may become progressively more complicated.

The relationship can become self-reinforcing.

Greater automation reduces direct human involvement, while reduced involvement makes effective intervention more difficult.

These findings support an important proposition: human supervisory capability does not automatically increase as automated system capability increases.

That mismatch becomes especially important when autonomous systems can execute consequential operations faster than humans can understand their implications.

4. Agentic AI Moves Beyond the Practical Limits of Transaction-by-Transaction Oversight

Traditional automation generally executes operations within a predefined control structure.

Advanced agentic AI introduces a more adaptive operating model.

An AI model is the computational component that produces predictions, reasoning, decisions, or generated content from learned relationships.

An AI agent combines a model with objectives, instructions, context, memory, tools, permissions, and an execution environment so it can pursue tasks through sequences of operations.

An agentic system may contain one or more cooperating agents capable of planning, selecting tools, exchanging information, responding to changing circumstances, and delegating work.

These distinctions matter because the model alone does not define the complete system’s security characteristics.

An agent may be authorized to modify files, execute programs, access databases, initiate transactions, communicate with external organizations, or change the state of another system.

The security consequences arise from the interaction between the agent’s reasoning and the capabilities made available to it.

The 2024 AgentDojo research introduced an evaluation environment containing 97 realistic agent tasks and 629 security test cases for examining prompt injection attacks [8].

Prompt injection occurs when an attacker places malicious instructions in information an AI agent is expected to process as data, such as retrieved documents, websites, messages, or tool results.

The attack attempts to persuade the agent to treat lower-trust information as authoritative instructions.

A successfully manipulated agent may perform operations outside the legitimate user’s intentions.

The security challenge becomes more complicated when multiple agents possess different capabilities and operate across shared systems.

In 2026, Oliver Makins and colleagues examined coordinated attacks distributed among multiple agents [25].

Their research demonstrated that dividing malicious activity among cooperating agents can make the overall attack harder to recognize through monitoring each agent individually.

The problem is not limited to intentional attacks.

An ordinary sequence of individually authorized actions may produce an unacceptable combined outcome.

One agent might change a permission. A second might modify a network configuration. A third might initiate an external data transfer.

Each action could appear acceptable when evaluated independently, while the combined sequence creates a security violation.

This means the consequences of autonomous activity may be determined by relationships among actions, systems, and agents rather than by the apparent safety of any single operation.

Human supervisors face the same contextual difficulty.

An operator may understand an individual proposed action without possessing enough information to reconstruct the preceding operations and their accumulated consequences.

Research published in May 2026, On Controllability in Agentic AI, distinguishes between the technical ability to intervene in a system and the broader conditions required for meaningful human control [15].

The authors distinguish causal controllability, the ability to influence or constrain system behavior, from normative controllability, which concerns whether control is exercised according to legitimate authority, objectives, and values.

The presence of an emergency-stop mechanism does not establish that someone can recognize the need to use it before unacceptable consequences occur.

Similarly, the presence of human approval does not establish that the approving person possesses adequate understanding to evaluate the decision.

Research by Peiran Wang and colleagues examined 59 academic papers, 21 production agent systems, and 26 security plugins [16].

The researchers identified a mismatch between security mechanisms emphasized in academic research and those commonly deployed in production systems.

They also identified a persistent tradeoff between the cognitive burden of frequent human approvals and the security risks of granting broader autonomy.

Their research supports improving human participation rather than assuming that more approval requests necessarily provide better protection.

A useful engineering test follows from these findings.

Meaningful human approval requires sufficient time to understand a decision, sufficient information to evaluate its consequences, sufficient expertise to exercise judgment, and an effective means of intervening before unacceptable consequences occur.

Consider a group of autonomous agents generating 60 consequential approval requests per minute.

If each request requires 20 seconds of genuine human analysis, the system generates 20 minutes of review work during every minute of operation.

Even before accounting for unusually complex decisions, interruptions, fatigue, and investigation time, the organization would need 20 people reviewing continuously at full utilization merely to keep pace.

These illustrative numbers demonstrate a fundamental scalability problem.

Increasing the number of agents, the frequency of decisions, or the complexity of the operating environment can exceed practical human review capacity.

Improving the approval interface may increase throughput while reducing meaningful evaluation time.

Therefore, conventional human-in-the-loop supervision is likely to become increasingly inadequate as the principal security mechanism for high-speed, highly interconnected autonomous systems.

This is a testable hypothesis rather than a claim that every human approval mechanism has already become ineffective.

The important distinction is that human participation must demonstrate actual protective effectiveness under the system’s operating conditions. It should not receive automatic credit as an independent safety mechanism simply because a person participates.

5. Human Governance Replaces Repetitive Approval with Meaningful Authority

The appropriate response is to reconsider where human judgment contributes the greatest value.

Human-in-the-loop generally means human decisions are required at designated points within an automated operating process.

Human-on-the-loop describes arrangements in which automation proceeds while humans supervise and retain intervention capabilities.

Human governance is a broader responsibility. It involves establishing the objectives, policies, authority, constraints, accountability, and conditions under which autonomous operations are permitted.

The distinction is between approving individual machine operations and establishing the rules that govern those operations.

A manager may be poorly equipped to evaluate hundreds of software commands every hour.

That same manager may be well equipped to determine which systems an agent may access, what business objectives it may pursue, how much money it may spend, which information must remain protected, and what conditions require additional authorization.

In May 2026, Liming Zhu and colleagues published Designing Meaningful Human Oversight in AI [17].

Their research distinguishes between the AI system’s operative agency, its capacity to perform work, and the human’s evaluative agency, the capacity to assess, challenge, guide, and override automated decisions.

The researchers argue that meaningful oversight must be designed into the allocation of responsibilities rather than reduced to routine approvals.

In June 2026, Zeynep Engin introduced the Human-AI Governance framework, known as HAIG [18].

The framework analyzes governance through decision authority, process autonomy, and accountability configuration.

These dimensions can change independently.

An agent may possess substantial freedom to determine how software should be developed while possessing limited authority to deploy that software or modify production systems.

This distinction supports an important architectural concept already established in the Enhanced Agentic Security Architecture: the Authority Envelope [38].

The Authority Envelope defines the authorized scope of an agent’s activity, including its purpose, permissible actions, accessible resources, operating duration, transaction values, data, destinations, and other relevant restrictions.

Human governance establishes that authority.

Independent technical mechanisms enforce it.

Continuous assurance evaluates whether actual operations remain within it.

The Authority Envelope also provides a foundation for managing delegated authority.

Delegation occurs when an authorized person, organization, or agent transfers a defined portion of authority to another actor.

Delegation must remain attributable to its origin, limited to the authority actually possessed by the delegating party, and subject to appropriate expiration or revocation.

An agent should never acquire additional legitimate authority merely because another agent delegates work to it.

The National Institute of Standards and Technology’s Artificial Intelligence Risk Management Framework provides an organizational foundation through its GOVERN, MAP, MEASURE, and MANAGE functions [12].

ISO/IEC 42001:2023 establishes requirements for organizational AI management systems, including policy, objectives, management responsibility, risk evaluation, and continual improvement [13].

These frameworks address management responsibilities across the AI lifecycle rather than prescribing human approval of every operation.

The European Union’s AI Act also establishes human oversight requirements for covered high-risk AI systems, with obligations shaped by the application’s risk and operating context [14].

Certain consequential applications will continue to require direct human decisions.

The objective is effective human authority, not the elimination of human responsibility.

Three distinct human responsibilities should be preserved.

Human governance establishes objectives, authority, operating policies, and acceptable consequences.

Human judgment resolves exceptional circumstances, consequential decisions, and changes to governing authority.

Human accountability establishes responsibility for development, deployment, evaluation, organizational compliance, and the consequences of operating the system.

These responsibilities can remain meaningful even when humans cannot realistically inspect every autonomous action.

This is the transition required to preserve human authority as autonomous systems become more sophisticated.

6. The Harness Makes Human Governance Technically Enforceable

The transition from human approval toward human governance requires mechanisms that can enforce established authority without depending on continuous human intervention.

The Enhanced Agentic Security Architecture provides an important architectural foundation [38].

Its governing principle is that probabilistic intelligence should be permitted to reason, plan, adapt, and propose, while separate security mechanisms independently constrain, authorize, observe, and verify consequential operations.

Probabilistic intelligence refers to the AI model’s ability to produce reasoning and decisions through learned statistical relationships rather than exclusively through predetermined rules.

Deterministic security enforcement means applying defined authorization rules to relevant inputs and security state so that the same applicable conditions produce consistent authorization decisions.

The architecture separates the model’s ability to determine what might be useful from the security system’s authority to determine what may actually be done.

Central to that distinction is the agent harness.

An agent harness is the software runtime and supporting control infrastructure surrounding an AI model that manages its context, memory, retrieval, tools, permissions, execution, communications, and interaction with external systems.

The harness determines how the model’s reasoning is translated into operational activity.

It manages the environment in which the agent functions and the mechanisms through which the agent requests or performs actions.

A general-purpose harness may provide orchestration and convenience without strong independent security guarantees.

A security-governed harness must go further. It must enforce the applicable restrictions and preserve the integrity of the controls governing consequential operations.

The Q3 Enhanced Agentic Security Architecture identifies the harness as a first-class security boundary that should be identifiable, versioned, inventoried, integrity-protected, policy-controlled, observable, testable, and auditable [38].

This distinction becomes essential when considering how to prevent an increasingly capable model from operating outside its authorized security environment.

Cryptographic bonding of the harness to the model

The subsequent SpartansFirst research article No Model Without a Harness proposes a stronger mechanism: cryptographic bonding of the harness to the AI model [33].

The objective is to establish a cryptographically enforced dependency under which a protected model cannot become operational unless its approved harness and required security environment are present and verified.

This is more than attaching security software to an AI application.

The proposed architecture makes successful verification of the harness a condition for obtaining the cryptographic material necessary to operate the protected model.

The process begins with encrypted model weights.

Model weights are the numerical parameters representing the relationships learned during AI training. These weights are a principal component of a trained model’s capabilities.

The proposed architecture keeps protected model weights encrypted during storage and distribution.

A hardware root of trust provides a protected foundation for establishing the identity and integrity of the computing environment.

Secure boot verifies that designated software components meet established integrity requirements during system startup.

The operating environment then measures relevant software components, including the security-critical elements of the harness and its approved configuration.

Cryptographic attestation produces verifiable evidence that the measured environment corresponds to an authorized configuration.

An independent key-management authority evaluates that evidence against the established security requirements.

Only when those requirements are satisfied does the authority release the cryptographic material necessary to make the protected model operational.

The model weights are decrypted within a protected execution environment that restricts unauthorized access to their plaintext representation.

If the approved harness is absent, replaced, or modified beyond permitted conditions, the measured configuration should fail the required verification.

The decryption key is withheld, leaving the protected model unavailable for execution.

The defining principle is straightforward:

No Model Without a Harness; No Effect Without Authority.

The first half establishes the objective of cryptographic bonding of the harness to the model: a protected model should not operate unless its required security harness is demonstrably present.

The second half establishes an equally important restriction. Even an authenticated model operating within its approved harness must not cause consequential external effects without independently established authorization.

This separation is essential.

Cryptographic bonding of the harness to the model establishes a dependency between protected model execution and the approved operating environment.

It does not independently establish that every proposed operation is safe.

A correctly authenticated harness might still possess excessive permissions, implement incomplete policies, or contain undiscovered vulnerabilities.

Therefore, cryptographic bonding of the harness to the model must operate alongside independent authorization and continuous assurance.

The importance of model confidentiality

The proposed protection also has an important technical limitation.

If an attacker obtains unrestricted plaintext model weights, the attacker may be able to construct another inference environment without the approved harness.

An inference environment is the computing system used to execute a trained model and generate outputs.

Consequently, cryptographic bonding of the harness to the model depends on protecting the model’s plaintext weights, the integrity of the trusted execution environment, and the cryptographic material controlling authorized execution.

Potential vulnerabilities include compromised hardware, firmware defects, stolen keys, unauthorized memory access, side-channel attacks, and insecure model export mechanisms.

Side-channel attacks exploit information unintentionally revealed by a computing system, such as timing, power consumption, or other observable characteristics.

These limitations do not invalidate the architectural objective. They identify the security properties that must be engineered, tested, and independently assured.

Cryptographic bonding of the harness to the model is therefore a proposed security architecture whose effectiveness must be demonstrated under realistic operating conditions.

Its significance is that it creates a technical mechanism through which governance can become a prerequisite for protected model operation rather than merely an instruction requesting compliant behavior.

7. From Policy to Permission: Governing Every Consequential Action

Even when a protected model operates within its approved harness, additional mechanisms must determine which actions it is authorized to perform.

The Q3 Enhanced Agentic Security Architecture establishes a Proposed Action Boundary separating model reasoning from consequential execution [38].

The boundary is the point at which an AI-generated intention becomes a structured request for an operation that may affect an external system.

A Proposed Action Object contains the information needed to evaluate that request, including the agent’s identity, requested operation, target, parameters, relevant context, authority basis, and expected effects.

The proposed action is then submitted to the security control plane.

The security control plane is the independently governed mechanism responsible for evaluating applicable authorization rules and determining whether proposed operations are permitted.

The model may decide that a particular operation would help accomplish its objective.

The control plane determines whether that operation is authorized.

This distinction prevents model confidence, persuasive reasoning, or instructions encountered in untrusted material from becoming equivalent to legitimate authority.

In October 2026, NIST published Design Tenets for Agent Identity Management Systems [10].

The publication proposes ten principles addressing distinct agent identity, delegated authority, least privilege, short-lived authorization, human escalation, signed mandates, separation of reasoning from policy, deterministic access decisions, continuous monitoring, and tamper-evident evidence.

Least privilege means granting an actor only the permissions necessary for its authorized work.

A mandate is an explicit statement defining an agent’s assigned objectives, delegated authority, and operating constraints.

NIST’s earlier August 2026 article concerning agent identity also warned that excessive human approval requests can create consent fatigue [11].

The October design tenets recognize that autonomous agents can operate at speeds and scales beyond normal human monitoring capabilities.

These proposals reinforce the importance of independent authorization and machine-speed enforcement.

One practical mechanism is policy-as-code.

Policy-as-code expresses governance requirements as machine-readable rules that can be evaluated by a policy engine independently of the AI model.

Open Policy Agent provides an example of this approach [22].

The model proposes a tool operation.

The harness submits the structured request to the policy engine.

The policy engine evaluates the request against applicable rules.

The harness then enforces the resulting authorization decision.

This arrangement allows the model to remain adaptive while its permitted operations remain governed.

Amazon Web Services has published an implementation using the Cedar authorization language to enforce least-privilege permissions across cooperating agents [23].

Research by Adam Mondl and colleagues examines the translation of natural-language instructions into machine-readable policies [24].

Such techniques may eventually allow managers to express governance requirements in ordinary language while automated systems generate candidate enforceable rules.

However, a technically valid rule can still implement an incorrect interpretation of human intent.

Policy generation therefore requires verification, testing, review, and controlled changes.

Two 2026 Internet Engineering Task Force proposals address cryptographically verifiable delegation of agent authority [20, 21].

They describe mechanisms for validating delegation chains and preventing subordinate agents from claiming broader authority than their originating mandate permits.

These publications are individual Internet-Drafts under development, not approved Internet standards.

Their significance is that they explore practical methods for verifying authority across multiple agents and organizational boundaries.

The OWASP Agent Control Standard, introduced in September 2026, proposes a separate runtime control mechanism capable of evaluating and governing agent operations [19].

The standard provides mechanisms for observing proposed actions and applying control decisions through external policy checks.

For security-critical operations, the enforcement architecture should require a positive authorization decision before allowing consequential execution.

If an enforcement mechanism becomes unavailable or cannot establish the required authority, the system should restrict or suspend operations according to the applicable safety policy.

This is commonly called fail-closed behavior: uncertain authorization does not automatically become permission.

A practical example illustrates the complete relationship.

An enterprise technology manager authorizes a software-development agent to modify source code, execute tests, review dependencies, and prepare a production release.

The Authority Envelope permits these activities within approved development environments.

The same policy prohibits access to protected production customer records, restricts external network destinations, limits resource consumption, and requires separate authorization for production deployment.

The agent can then perform thousands of development operations without repeated human interruption.

The harness manages tool execution.

The security control plane evaluates consequential actions.

Independent enforcement applies the organization’s approved policies.

A human becomes involved when a proposed operation exceeds established authority or requires consequential business judgment.

This is how human governance replaces repetitive transaction approval without surrendering control.

The organization retains responsibility for defining what is permissible, while technical mechanisms enforce that responsibility at the speed of autonomous operation.

8. Independent Assurance Must Understand the Entire System and Its Consequences

Authorization and enforcement are necessary, but they do not eliminate every possible failure.

Policies may be incomplete.

Security configurations may change.

Software vulnerabilities may be exploited.

Multiple agents may combine their activities in unexpected ways.

The assurance mechanisms themselves may become unavailable, outdated, or compromised.

Consequently, trustworthy autonomy requires continuous independent assurance of the complete operating system.

The Q3 Enhanced Agentic Security Architecture identifies Authoritative Security State as a foundation for determining whether an action remains authorized [38].

Authoritative Security State is the trusted representation of the security-relevant conditions that govern an operation, including identities, permissions, policies, delegated authority, operating configuration, and applicable restrictions.

This information must remain sufficiently current and complete to support consequential security decisions.

An enforcement mechanism using obsolete permissions or incomplete information may approve an operation that is no longer authorized.

State freshness describes whether the information used to make a decision remains current enough for its intended security purpose.

The architecture also uses an Agent Bill of Materials, or ABOM, to identify the components and dependencies materially influencing an operating agent.

These may include the model, harness, tools, credentials, memory, retrieval services, external connections, and other agents.

A Capability Graph represents relationships among agents, permissions, tools, data, applications, and available actions.

This graph is particularly important because individually permitted capabilities can combine to produce unacceptable consequences.

The SpartansFirst article Beyond the Smart Agent explores the broader implications of graph engineering for interacting agents and system-level intelligence [35].

The concern becomes more significant as agents acquire the ability to coordinate across systems.

Research on distributed multi-agent attacks demonstrates that monitoring each agent separately may fail to recognize malicious activity distributed among several cooperating agents [25].

An independent assurance mechanism must therefore examine more than isolated operations.

It must evaluate action sequences, delegated authority, changing system state, and the cumulative consequences of activity across the complete operating environment.

The SpartansFirst article The General Architecture of Technological Consequence provides useful terminology for understanding these risks [32].

A blast radius describes the initial extent of harmful consequences.

Propagation describes how those consequences spread into additional systems or activities.

A cascade horizon describes the broader range of consequences that may emerge through dependencies, interactions, and secondary failures.

A system that prevents individual violations but fails to recognize dangerous combinations of authorized actions may still permit unacceptable outcomes.

Independent assurance must therefore assess prospective consequences as well as current compliance.

The distinction between preventive controls and evidential controls is also important.

Preventive controls attempt to stop unauthorized actions before they cause consequential effects.

Evidential controls preserve trustworthy information describing what was attempted, authorized, denied, executed, and observed.

Both are required.

A control’s statement that an action was blocked is not necessarily sufficient evidence that no external effect occurred.

An independent observation mechanism may be needed to establish the actual result.

NASA research demonstrates practical automatic intervention

Two NASA research papers provide valuable engineering precedents.

In 2024, J. Tanner Slagel and colleagues published A Verification Framework for Runtime Assurance of Autonomous UAS [28].

Runtime assurance is an engineering approach in which a system operates under continuous evaluation against defined safety requirements.

The paper examines a Simplex architecture, in which an advanced controller operates while an independent monitor evaluates safety conditions.

If the advanced controller approaches a condition requiring intervention, control can transfer to a separately trusted fallback controller.

The NASA work develops formal verification methods and examines requirements such as the frequency with which critical conditions must be checked.

Formal verification uses mathematical methods to establish that a defined system model satisfies specified properties under stated assumptions.

A second NASA paper, Testing a Run-Time Assurance Framework Coupled with Integrated Risk Mitigation Capabilities for Autonomous Urban UAS Flights, reports actual flight tests involving small unmanned aircraft [37].

The researchers examined airborne mechanisms capable of independently monitoring hazards, assessing risk, and initiating corrective operations.

The flight-test architecture operated independently of an unassured autopilot and supported automatic or pilot-supervised intervention.

These studies establish that independent monitoring and automatic safety intervention can be engineered and evaluated for particular autonomous operating environments.

They do not demonstrate that equivalent mechanisms already provide complete assurance for general-purpose agentic AI.

They do establish an important principle: an advanced autonomous system can retain substantial operational freedom while a separate mechanism monitors critical constraints and initiates corrective action when required.

Assurance systems must themselves be assured

Independent monitoring introduces another important engineering requirement.

Who assures the assurance system?

Research presented at ICLR 2026 demonstrated that adversarial instructions could manipulate AI-based monitors intended to detect dangerous agent behavior [26].

Assigning one AI model to supervise another does not automatically establish independence or trustworthy operation.

Critical authorization and enforcement mechanisms should therefore remain appropriately separated from the model’s reasoning.

Deterministic controls can enforce explicit permissions, transaction limits, resource restrictions, and other requirements that can be evaluated against authoritative state.

AI-based monitoring can supplement those mechanisms by recognizing unusual behavior, identifying complex patterns, and detecting conditions that deterministic rules may not adequately describe.

Research by Benjamin Herd and colleagues examines methods for updating confidence in safety arguments using evidence collected during operation [27].

An assurance case is a structured argument identifying the claims that must be established, the evidence supporting those claims, and the limitations of that evidence.

An assurance case is more than an audit log.

A complete record of operations does not independently demonstrate that those operations were safe or that the observations were complete and reliable.

The broader architectural objective is Dynamic Assurance: continuously reassessing justified confidence as the system, its components, operating conditions, and observed behavior change.

NVIDIA’s October 2026 OpenShell announcement provides an example of developing industry implementation [30].

The company described an architecture combining an agent runtime with hardware-supported isolation, monitoring, and policy enforcement, including NVIDIA Sentry and BlueField-4 infrastructure.

These are reported implementation capabilities rather than independent proof of complete assurance.

Their significance is that substantial industry investment is beginning to support security mechanisms operating outside the AI model’s own reasoning process.

This direction is consistent with the third element of the Agentic Security Triad: continuous independent assurance [31].

The necessary protection requires more than identifying a problem.

It requires detecting developing failures in time, preventing unacceptable operations where possible, containing consequences when prevention fails, preserving trustworthy evidence, and restoring the system to an authorized operating condition.

9. Public Confidence Must Rest on Demonstrated Security and Accountable Governance

As autonomous systems become more widely deployed, public confidence will depend increasingly on whether their security and governance mechanisms perform as promised.

Visible failures can materially influence people’s willingness to rely on automation.

In 2015, Berkeley Dietvorst and colleagues examined algorithm aversion: the tendency of people in certain circumstances to reject an algorithm after observing it make an error, even when its overall performance remains better than that of available human alternatives [29].

The research does not establish how the general public will respond to a future major autonomous AI incident.

It does demonstrate that observed errors can influence trust and willingness to rely on automated systems.

This raises an important concern for organizations that present human approval as their principal security safeguard.

A serious incident could reveal that the human reviewer lacked sufficient information, time, expertise, or contextual awareness to recognize the danger.

The organization might have complied with its documented approval procedure while failing to provide effective protection.

The public could reasonably question whether the approval mechanism represented genuine oversight or merely the appearance of oversight.

The response should be evidence-based governance.

Trust in governance concerns whether an organization has established appropriate objectives, policies, authority, operating boundaries, and accountability.

Trust in enforcement concerns whether technical mechanisms reliably apply those requirements to actual autonomous operations.

Trust in assurance concerns whether sufficiently independent evidence establishes that governance and enforcement continue to function as intended.

These dimensions must be evaluated separately.

An organization can establish appropriate policies but implement inadequate enforcement.

A security mechanism can correctly enforce a policy whose requirements are incomplete.

An assurance system can produce technically accurate records while failing to observe a critical category of activity.

Each possibility requires a different corrective response.

A comprehensive assurance program should therefore measure actual operating performance.

Important measurements include unauthorized actions prevented, dangerous conditions detected, false alarms, enforcement coverage, intervention latency, state freshness, containment effectiveness, recovery success, and the completeness and integrity of audit evidence.

A false alarm occurs when a monitoring mechanism incorrectly identifies acceptable behavior as dangerous. Excessive false alarms can increase workload, interrupt productive operations, and reduce confidence in warnings.

A security mechanism’s measured performance should also identify the operating conditions under which that performance was established.

A classifier that performs well against one collection of dangerous commands cannot automatically be assumed to perform equally well against every future attack.

Similarly, successful testing of individual agents does not establish the safety of unrestricted multi-agent operation.

The Agentic Security Triad requires early experimental validation

The research agenda should include rigorous evaluation of the Agentic Security Triad’s three complementary hypotheses.

Alignment with human interests should be tested against unfamiliar objectives, adversarial instructions, changing capabilities, conflicting goals, and conditions the model did not encounter during earlier evaluation.

Cybernetic bounding should be tested against unauthorized tool execution, privilege escalation, harness compromise, manipulation of policies, model extraction, and coordinated activity among multiple agents.

Continuous independent assurance should be tested against delayed detection, incomplete observations, stale security state, monitor manipulation, unavailable policy services, failed containment, and failures within the assurance infrastructure itself.

The effectiveness of cryptographic bonding of the harness to the model deserves particular attention.

Tests should establish whether an unauthorized runtime can obtain the necessary model-decryption material, whether harness modifications invalidate attestation, whether protected model weights can be extracted, and whether authorization can be revoked effectively.

The tests should also examine whether trusted execution environments, key management, and enforcement mechanisms remain effective under realistic attack conditions.

Successful component testing will not be sufficient.

The complete system must be evaluated against failures arising from interaction among its components.

The research should compare at least three operating arrangements: conventional human-in-the-loop approval, independent automatic policy enforcement, and human-governed autonomy combining automatic enforcement with continuous independent assurance and selective human escalation.

Experiments should vary operating speed, transaction volume, task complexity, agent coordination, dependency depth, authority scope, and the time available for intervention.

Measurements should assess actual prevention of unacceptable consequences rather than merely counting approvals or generated alerts.

The research should also measure governance workload, recovery capability, system throughput, and the economic cost of implementing each arrangement.

These evaluations would help establish the operating conditions under which human approval remains useful, the conditions under which it becomes ineffective, and the advantages and limitations of alternative security architectures.

The objective is to develop justified confidence through demonstrated engineering performance.

Technical management and public policy must evolve together

A transition toward governance-level human responsibility also requires changes in institutional understanding.

Technology executives must understand the distinction between model capability, agent authority, harness enforcement, and independent assurance.

Boards and enterprise management must establish responsibility for approving operating policies, defining acceptable consequences, and assessing assurance evidence.

Standards organizations must advance interoperable mechanisms for agent identity, delegated authority, cryptographic bonding of the harness to the model, runtime policy enforcement, and trustworthy evidence.

Government officials and legislators need sufficient technical understanding to distinguish effective protections from procedures that merely appear protective.

Regulation should establish appropriate accountability, safety requirements, and verifiable performance expectations while allowing technical implementations to improve as research advances.

Poorly designed regulation can impose expensive procedural burdens without materially improving safety.

Well-designed governance can enable innovation by establishing enforceable operating boundaries within which new capabilities may be developed and deployed with justified confidence.

Public communication must also explain these distinctions.

A meaningful public assurance statement should describe what autonomous systems are authorized to accomplish, which restrictions are technically enforced, how violations are detected, what evidence supports the assurance claims, and how corrective action occurs.

That information will be more useful than a general statement that humans remain involved.

The objective is to create public confidence grounded in accountable governance and demonstrated technical protection.

10. Preparing Now for the Next Generation of Autonomous Intelligence

Artificial intelligence is moving from an extended period of discovery and experimental development toward increasingly consequential implementation.

Research investment continues to expand model capabilities, computational efficiency, reasoning performance, software-development automation, scientific discovery, and multi-agent operation.

These advances offer the potential for extraordinary economic and social benefits.

They also increase the importance of establishing dependable governance and assurance before increasingly capable autonomous systems become deeply embedded in enterprise operations, critical infrastructure, financial services, healthcare, and public institutions.

One important uncertainty concerns artificial general intelligence.

Artificial general intelligence, or AGI, generally refers to AI systems capable of performing a broad range of intellectual tasks at or beyond human competence, although there is no universally accepted technical definition or test.

Whether AGI will emerge during the next decade remains uncertain. Researchers disagree about its likely timing, necessary capabilities, and potential consequences.

That uncertainty makes preparation especially important.

The engineering principles required to govern powerful autonomous systems are valuable regardless of when or whether AGI is achieved.

Many of the necessary mechanisms can be developed, tested, and improved using the increasingly capable systems already being deployed.

The industry should therefore advance several complementary objectives.

First, organizations should move human responsibility toward establishing objectives, authority, operating policies, acceptable consequences, and accountability.

Second, AI developers should treat the agent harness as a critical operating and security component rather than merely a software convenience for tool execution.

Third, research and development should advance cryptographic bonding of the harness to the model, making protected model execution dependent on verification of its approved harness and security environment.

Fourth, consequential actions should remain subject to independently enforced authorization at the Proposed Action Boundary, regardless of the model’s confidence or reasoning.

Fifth, continuous independent assurance should evaluate the actual operating system, its changing security state, interactions among agents, and the potential accumulation of consequences.

Sixth, the industry should develop reliable methods for detecting, containing, correcting, and recovering from failures, including failures of the enforcement and assurance mechanisms themselves.

Seventh, independent testing should establish where these mechanisms work, where they fail, and how their performance changes as autonomous systems become more capable.

The Agentic Security Triad offers a coherent architectural hypothesis connecting these objectives [31].

Alignment seeks to keep intelligence directed toward legitimate human interests.

Cybernetic bounding seeks to constrain what that intelligence is permitted to cause.

Continuous independent assurance seeks evidence that the alignment and bounding mechanisms remain effective.

The significance of the Triad lies in the complementary nature of its protections.

None should be treated as a substitute for the others.

The next several years offer an important opportunity to move these concepts from architectural research into testable implementations.

The same is true of cryptographic bonding of the harness to the model.

Its potential significance justifies early experimentation with protected execution, model confidentiality, attestation, key management, authorization, and independent verification.

These mechanisms should be evaluated before increasingly powerful models and agentic systems become widely distributed.

The opportunity is to discover weaknesses while systems remain sufficiently controlled for rigorous testing, correction, and redesign.

This preparation should accompany rapid innovation rather than become an obstacle to it.

The objective is to capture the extraordinary benefits of artificial intelligence while maintaining legitimate human authority over its capabilities, deployment, and consequences.

Achieving that objective requires cooperation among researchers, AI developers, enterprise management, cybersecurity professionals, standards organizations, regulators, and independent assurance providers.

The necessary transition is from humans checking the work of machines toward humans designing, authorizing, and governing the operating environment within which machines perform their work.

That transition does not diminish human responsibility.

It strengthens human responsibility by placing human judgment where it can provide the greatest value.

The industry must also prepare for failures.

No realistic security architecture can promise that every possible failure will be prevented.

Trustworthy autonomy requires the ability to recognize developing problems, contain harmful consequences, correct deficiencies, and improve the systems that produced them.

The General Architecture of Technological Consequence establishes an operating principle that connects innovation, beneficial deployment, protection, correction, and continuing progress [32].

INNOVATE → ENABLE → PROTECT → CORRECT → INNOVATE

Innovation creates new capabilities.

Enabling allows those capabilities to produce beneficial results.

Protection establishes appropriate safeguards against unacceptable consequences.

Correction requires the ability to recognize failures, contain their effects, identify their causes, and repair weaknesses in technology, implementation, or governance.

Renewed innovation uses that experience to improve the next generation of capabilities.

This continuing cycle provides a practical foundation for advancing autonomous artificial intelligence while maintaining human authority and public confidence.

Humanity should retain responsibility for defining the purposes, permissions, operating boundaries, and acceptable consequences of increasingly autonomous technology. Cryptographic bonding of the harness to the model, independently enforced authorization, and continuous independent assurance offer complementary mechanisms for making that responsibility effective at the speed and scale at which advanced AI operates.

The opportunity before us is to establish these capabilities early, validate them independently, improve them continuously, and create the foundations for artificial intelligence that advances human knowledge, productivity, prosperity, and well-being while remaining aligned, controlled, and independently assured.

 

References

  1. Bainbridge, Lisanne. “Ironies of Automation.” Automatica, vol. 19, no. 6, November 1983, pp. 775–779. DOI: 10.1016/0005-1098(83)90046-8. https://doi.org/10.1016/0005-1098(83)90046-8. Full text: https://gwern.net/doc/sociology/technology/1983-bainbridge.pdf
  2. Anthropic. “Auto Mode Is Now the Default in Claude Code for Pro, Max, Ultra, and Team Plans.” Claude Resources, August 7, 2026; updated October 22, 2026. https://claude.com/resources/articles/auto-mode-default-in-claude-code
  3. Endsley, Mica R., and Esin O. Kiris. “The Out-of-the-Loop Performance Problem and Level of Control in Automation.” Human Factors, vol. 37, no. 2, 1995, pp. 381–394. DOI: 10.1518/001872095779064555. https://doi.org/10.1518/001872095779064555
  4. Parasuraman, Raja, and Victor Riley. “Humans and Automation: Use, Misuse, Disuse, Abuse.” Human Factors, vol. 39, no. 2, 1997, pp. 230–253. DOI: 10.1518/001872097778543886. https://doi.org/10.1518/001872097778543886
  5. Lee, John D., and Katrina A. See. “Trust in Automation: Designing for Appropriate Reliance.” Human Factors, vol. 46, no. 1, 2004, pp. 50–80. DOI: 10.1518/hfes.46.1.50_30392. https://doi.org/10.1518/hfes.46.1.50_30392
  6. Parasuraman, Raja, and Dietrich H. Manzey. “Complacency and Bias in Human Use of Automation: An Attentional Integration.” Human Factors, vol. 52, no. 3, 2010, pp. 381–410. DOI: 10.1177/0018720810376055. https://doi.org/10.1177/0018720810376055
  7. Buçinca, Zana, Maja Barbara Malaya, and Krzysztof Z. Gajos. “To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-Assisted Decision-Making.” Proceedings of the ACM on Human-Computer Interaction, vol. 5, CSCW1, 2021. DOI: 10.1145/3449287. https://doi.org/10.1145/3449287
  8. Debenedetti, Edoardo, Jie Zhang, Mislav Balunović, Luca Beurer-Kellner, Marc Fischer, and Florian Tramèr. “AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents.” Advances in Neural Information Processing Systems, vol. 37, 2024, Datasets and Benchmarks Track. DOI: 10.52202/079017-2636. https://proceedings.neurips.cc/paper_files/paper/2024/hash/97091a5177d8dc64b1da8bf3e1f6fb54-Abstract-Datasets_and_Benchmarks_Track.html
  9. Lee, Hao-Ping, Advait Sarkar, Lev Tankelevitch, Ian Drosos, Sean Rintel, Richard Banks, and Nicholas Wilson. “The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers.” Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, 2025. DOI: 10.1145/3706598.3713778. https://doi.org/10.1145/3706598.3713778
  10. Booth, Harold, Bill Fisher, Ryan Galluzzo, and Tom Clancy. “Design Tenets for Agent Identity Management Systems.” National Institute of Standards and Technology, Cybersecurity Insights, October 27, 2026. https://www.nist.gov/blogs/cybersecurity-insights/design-tenets-agent-identity-management-systems
  11. Fisher, Bill, and Ryan Galluzzo. “Back to the Future: Why Agentic AI Needs a Strong Identity Foundation.” National Institute of Standards and Technology, Cybersecurity Insights, August 27, 2026. https://www.nist.gov/blogs/cybersecurity-insights/back-future-why-agentic-ai-needs-strong-identity-foundation
  12. National Institute of Standards and Technology. Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1, January 2023. DOI: 10.6028/NIST.AI.100-1. https://doi.org/10.6028/NIST.AI.100-1
  13. International Organization for Standardization and International Electrotechnical Commission. ISO/IEC 42001:2023: Information Technology—Artificial Intelligence—Management System. First edition, December 2023. https://www.iso.org/standard/42001
  14. European Parliament and Council of the European Union. Regulation (EU) 2024/1689 Laying Down Harmonised Rules on Artificial Intelligence, Article 14, Human Oversight. Official Journal of the European Union, 2024. https://eur-lex.europa.eu/eli/reg/2024/1689/oj
  15. Nguyen, My H., Duc-Hai Nguyen, Barry O’Sullivan, and Hoang D. Nguyen. “On Controllability in Agentic AI: A Survey.” Minds and Machines, vol. 36, article 29, May 2026. DOI: 10.1007/s11023-026-09783-y. https://doi.org/10.1007/s11023-026-09783-y
  16. Wang, Peiran, Ying Li, and Yuan Tian. “Reframing LLM Agent Security as an Agent-Human Interaction Problem.” arXiv, arXiv:2605.24309, May 2026. https://arxiv.org/abs/2605.24309
  17. Zhu, Liming, Qinghua Lu, Ming Ding, Sung Une Lee, and Chen Wang. “Designing Meaningful Human Oversight in AI.” AI and Ethics, vol. 6, article 286, May 2026. DOI: 10.1007/s43681-026-01147-7. https://doi.org/10.1007/s43681-026-01147-7
  18. Engin, Zeynep. “Human-AI Governance (HAIG): A Trust-Utility Approach.” Journal of Responsible Technology, vol. 26, article 100167, June 2026. DOI: 10.1016/j.jrt.2026.100167. https://doi.org/10.1016/j.jrt.2026.100167
  19. OWASP Generative AI Security Project. Agent Control Standard (ACS). September 1, 2026. Emerging technical specification. https://genai.owasp.org/resource/agent-control-standard-acs/
  20. Hulea, Claudiu. Verifiable Mandates for Agents Acting Across Administrative Domains. IETF Internet-Draft, draft-hulea-wimse-agent-mandate-01, November 1, 2026. Work in progress; not an approved Internet standard. https://datatracker.ietf.org/doc/html/draft-hulea-wimse-agent-mandate-01
  21. Prakash, Sunil. Agent Identity Protocol (AIP): Verifiable Delegation for AI Agent Systems. IETF Internet-Draft, draft-prakash-aip-02, October 10, 2026. Work in progress; not an approved Internet standard. https://www.ietf.org/archive/id/draft-prakash-aip-02.html
  22. Open Policy Agent Project. “Agent Tool Calling.” Open Policy Agent Documentation, accessed November 2026. https://www.openpolicyagent.org/docs/agent-tool-calling
  23. Amazon Web Services. “Enforce Least-Privilege Authorization in Multi-Agent AI Chains Using Cedar.” AWS Security Blog, July 2026. https://aws.amazon.com/blogs/security/enforce-least-privilege-authorization-in-multi-agent-ai-chains-using-cedar/
  24. Mondl, Adam, Matthew Maisel, and John H. Brock. “Autoformalization of Agent Instructions into Policy-as-Code.” arXiv, arXiv:2606.26649, June 2026. https://arxiv.org/abs/2606.26649
  25. Makins, Oliver, Orazio Angelini, Zohreh Shams, and Mary Phuong. “Multi-Agent AI Control: Distributed Attacks Hamper Per-Instance Monitors.” arXiv, arXiv:2607.07368, July 2026; revised October 12, 2026. https://arxiv.org/abs/2607.07368
  26. Terekhov, Mikhail, et al. “Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols.” International Conference on Learning Representations, 2026. https://proceedings.iclr.cc/paper_files/paper/2026/hash/54b153ad8a138f4c186f21a8b7341d5e-Abstract-Conference.html
  27. Herd, Benjamin, Jessica Kelly, Clarissa Heinemann, and João-Vitor Zacchi. “A Subjective Logic-Based Method for Runtime Confidence Updates in Safety Arguments.” Proceedings of the 41st ACM/SIGAPP Symposium on Applied Computing, 2026, pp. 1478–1485. DOI: 10.1145/3748522.3779865. https://doi.org/10.1145/3748522.3779865
  28. Slagel, J. Tanner, Lauren M. White, Aaron Dutle, César A. Muñoz, and Nicolas Crespo. “A Verification Framework for Runtime Assurance of Autonomous UAS.” 43rd Digital Avionics Systems Conference, 2024. NASA Technical Reports Server, document 20240007986. DOI: 10.1109/DASC62030.2024.10748654. https://ntrs.nasa.gov/citations/20240007986
  29. Dietvorst, Berkeley J., Joseph P. Simmons, and Cade Massey. “Algorithm Aversion: People Erroneously Avoid Algorithms After Seeing Them Err.” Journal of Experimental Psychology: General, vol. 144, no. 1, 2015, pp. 114–126. DOI: 10.1037/xge0000033. https://doi.org/10.1037/xge0000033
  30. NVIDIA. “NVIDIA Launches OpenShell Reference Architecture for Safe AI Agents.” NVIDIA Newsroom, October 21, 2026. https://nvidianews.nvidia.com/news/openshell-reference-architecture
  31. Hayes, Dennis C. “The Agentic Security Triad: A New Hypothesis for Assuring Advanced Artificial Intelligence Remains Safe for Humanity.” SpartansFirst Press, Technology Blog, 2026. https://www.spartansfirst.org/AgenticSecurityTriad
  32. Hayes, Dennis C. “The General Architecture of Technological Consequence: A Common Framework for Understanding Innovation, Risk, and the Governance Gap.” SpartansFirst Press, Technology Blog, September 8, 2026. https://spartansfirst.org/ArchitectureofTechnologicalConsequence
  33. Hayes, Dennis C. “No Model Without a Harness: A Cryptographically Bound Architecture for Securing Advanced Agentic AI.” SpartansFirst Press, Technology Blog, 2026. https://www.spartansfirst.org/NoModelWithoutaHarness
  34. Hayes, Dennis C. “From Policy to Permission: Gartner’s Technology Map Reinforces Our Agentic Security Architecture.” SpartansFirst Press, Technology Blog, October 2026. https://www.spartansfirst.org/Policy%20to%20Permission%20Gartner%20Analysis
  35. Hayes, Dennis C. “Beyond the Smart Agent: Graph Engineering and the Emergence of System Intelligence.” SpartansFirst Press, Technology Blog, August 31, 2026. https://www.spartansfirst.org/BeyondLLMwithGraphEngineering
  36. Hayes, Dennis C. “When AI Accelerates Everything.” SpartansFirst Press, Technology Blog, October 2, 2026. https://www.spartansfirst.org/AIResearchAcceleratesEverything
  37. Ancel, Ersin, Andrew J. Moore, Steven D. Young, Evan T. Dill, Cuong (Patrick) Quach, and Kyle M. Smalling. “Testing a Run-Time Assurance Framework Coupled with Integrated Risk Mitigation Capabilities for Autonomous Urban UAS Flights.” 43rd Digital Avionics Systems Conference, 2024. NASA Technical Reports Server, document 20240007663. DOI: 10.1109/DASC62030.2024.10749315. https://ntrs.nasa.gov/citations/20240007663
  38. Hayes, Dennis C. 2026 Q3 Enhanced Agentic Security Architecture, Version 2.1. Original Q3 publication August 24, 2026; controlled Version 2.1 reconciliation dated October 2, 2026. SpartansFirst. https://spartansfirst.org/3Q26EnhancedAgenticSecurityArchitecture