For transformation leaders, chief AI officers, risk owners and the managers who sign off on machine recommendations. Human oversight of AI fails in two opposite directions, and most enterprises measure neither. What a real override requires, what to measure, and which judgement is genuinely yours.
The short answer
Most enterprises treat human oversight as a binary. Either a person approves the output or they do not, and the presence of that approval step is taken as evidence that a human is in control. The evidence says otherwise. Oversight fails in two opposite directions, and an organisation that measures only approval volume cannot see either one.
The first failure is over-acceptance. Reviewers trust a system that is usually right, stop verifying, and approve by default. The paperwork still records a human decision. The second failure is over-rejection, where reviewers dismiss outputs wholesale because the system is noisy, irritating or threatening, and lose the value it was providing. Both produce a clean audit trail. Neither is oversight.
The useful question is therefore not "do we have a human in the loop" but "under what conditions is that human's override a real one". That is answerable, and the answer has more to do with capability and incentive design than with the interface.
Why this became a live problem in 2026
Two things converged. The number of decision points where a human is asked to approve machine output rose sharply, and the legal deadline that would have forced enterprises to design those points properly moved.
On scale, Gartner forecast in August 2025 that 40% of enterprise applications would feature task-specific AI agents during 2026, up from less than 5% in 2025. That is a step change in the number of approval prompts arriving in front of ordinary employees, and it happened inside a single year.
On the regulatory side, Article 14 of the EU AI Act requires that high-risk AI systems be designed so that natural persons can effectively oversee them while in use, including the ability to intervene, override, interrupt or stop the system. The capability must be built into the system, not asserted in documentation. The Digital Omnibus on AI, Regulation (EU) 2026/1744, was published in the Official Journal on 24 July 2026 and entered into force on 27 July 2026. It moved the main compliance obligations for stand-alone high-risk systems under Annex III, which include systems used in employment and education, from 2 August 2026 to 2 December 2027, and for high-risk systems embedded in regulated products under Annex I to 2 August 2028. Article 50 transparency duties applied on 2 August 2026 as originally scheduled.
The deferral changed the deadline. It did not change what Article 14 asks for, and it did not slow the deployments. Enterprises now have roughly a year and a half of running agents in production before the obligation to demonstrate effective oversight becomes enforceable, which is time to find out whether their oversight works rather than time to postpone the question.
The two ways human oversight fails
The human factors literature has names for both, and they predate generative AI by decades.
Commission errors occur when a reviewer accepts an incorrect automated recommendation. Omission errors occur when a reviewer misses something because the system did not flag it, or rejects a correct flag. Goddard, Roudsari and Wyatt's systematic review in the Journal of the American Medical Informatics Association defines automation bias as exactly this pair: omission and commission errors resulting from using automated cues as a heuristic replacement for vigilant information seeking and processing. Their central finding is that decision support usually improves overall performance while introducing a new class of error that organisations fail to recognise, because they are measuring the average and not the new failure mode.
That last point is the one enterprises keep rediscovering. An agent that raises average decision quality can simultaneously introduce a category of mistake nobody was making before, and aggregate metrics will hide it.
The mechanism behind over-acceptance is not stupidity. It is a rational response to volume. The OWASP threat taxonomy for agentic applications lists overwhelming the human in the loop as a named threat: flood a reviewer with approval requests and their review degrades into a reflex, at which point an attacker can place a consequential action inside a stream of trivial ones. Approval fatigue is not a soft cultural issue. It is a documented attack surface.
Over-rejection is less discussed and equally expensive. A system that interrupts too often, explains too little, or is perceived as a threat to the reviewer's own judgement gets dismissed on principle. The organisation pays for the system, absorbs the workflow disruption, and captures none of the benefit.
What thirty years of clinical evidence already shows
Enterprises deploying agents in 2026 are re-running an experiment that healthcare completed long ago. Clinical decision support put a machine recommendation in front of a qualified professional at scale, measured what happened, and published the results. The results are sobering.
Poly and colleagues' systematic review in JMIR Medical Informatics found average alert override rates ranging from 46.2% to 96.2%. Clinicians were dismissing between roughly half and nearly all of the alerts the system raised. That alone is not damning: many alerts deserve dismissal.
The important number is the second one. The same review found that between 29.4% and 100% of overrides were judged appropriate, depending on the alert type. Read that as a failure rate and the picture changes.
| Alert type | Overrides judged appropriate | What that implies |
|---|---|---|
| Drug-allergy interaction | 63.4% to 100% | Reviewers were mostly right to dismiss |
| Drug-drug interaction | 0% to 95% | Wildly setting-dependent, so the system is not the variable |
| Dose | 43.9% to 88.8% | A material share of dismissals were wrong |
| Renal | 27% to 87.5% | In some settings most dismissals were wrong |
| Geriatric | 14.3% to 57% | In the worst settings around six in seven dismissals were wrong |
The analogy has limits worth stating. Clinicians are licensed, the alerts are narrow and rule-based, and patient safety creates scrutiny most commercial decisions never get. An enterprise agent recommending a pricing change is not a drug interaction alert. What transfers is not the numbers but the structure of the problem: a qualified person, a machine recommendation, volume, and a decision that gets recorded either way. Three findings survive that translation.
First, override rate on its own says nothing. A 90% override rate can mean the system is badly tuned or that the reviewers have stopped engaging. The number does not distinguish them.
Second, the spread within a single alert type is enormous. Drug-drug interaction overrides were appropriate somewhere between 0% and 95% of the time depending on the setting. The same system, the same class of recommendation, and radically different human performance. What varies is the local conditions: workload, training, whether dismissing carries any consequence.
Third, and most useful, healthcare measured the thing that matters. It did not stop at counting overrides. It went back and asked whether each override was correct. Almost no enterprise deploying agents in 2026 does this, which means almost no enterprise knows whether its human oversight is working.
The override test: four conditions
An override is real when all four of the following hold. This is MASSIVUE practitioner guidance rather than a regulatory standard, and it is designed to be applied to a specific decision point rather than to an organisation as a whole.
The fourth condition is the one that decides the outcome, and it is the one nobody designs. If the only decision that ever gets audited is the override, reviewers learn that approving is free and disagreeing is expensive. They will approve. No amount of explainability tooling changes that arithmetic, because the arithmetic is about accountability, not comprehension.
Applying the test is quick. Take one decision point where a person approves machine output, and ask the four questions of it. In most enterprises the first decision point examined fails on time and on safety, and the fix is a workflow and accountability change rather than a technology change.
What to measure instead of "we have a human in the loop"
Four measures, borrowed from how clinical informatics evaluates the same problem, and applied per decision point rather than per system.
| Measure | What it is | What it catches |
|---|---|---|
| Override rate | Share of recommendations the reviewer does not accept | A rate near zero or near total. Both indicate the human has stopped deciding. |
| Override appropriateness | Retrospective sample of overrides scored as correct or incorrect by a second reviewer | The failure the override rate hides. This is the measure almost nobody collects. |
| Acceptance appropriateness | The same retrospective sample applied to accepted recommendations | Rubber-stamping. Without it you only ever audit disagreement. |
| Time per decision | Actual seconds between the item appearing and the decision being recorded | The temporal condition failing. Falling time per decision with rising volume is the signature of approval fatigue. |
The second and third are the ones that create work, and they are the ones that produce the answer. Sampling is sufficient: a periodic second review of a modest random sample of both overrides and acceptances will surface a systematic problem long before an incident does. The cost is a reviewer's time on a scheduled basis. The alternative is discovering the failure through the outcome.
One design warning. Do not set a target override rate. The moment a rate becomes a target, reviewers will produce the rate, and you will have replaced one uninformative number with a gamed one. Measure appropriateness and let the rate settle where it settles.
A second warning applies to anything you collect by asking. Self-reported data about AI use is biased by what disclosure costs the person reporting, which is the reason employees hide their AI use at work even in organisations that believe they have visibility. Measure the decision, not the account of it.
The judgement that is genuinely yours
The reason human oversight is worth designing well is that a reviewer does contribute something a model cannot: information about the situation and the people in it that never entered the system. That the customer on the flagged account called last week and explained the anomaly. That the team the recommendation affects is three weeks from a deadline. That the supplier the model scored badly is the one that carried you through a shortage.
Using that kind of information deliberately in reasoning and decision-making is the component of emotional intelligence usually called emotional reasoning, and it is the single strongest argument for keeping a person in the decision. It is also, without discipline, the exact mechanism that produces both failure modes. The same faculty that lets a reviewer see what the model missed lets them mistake a reaction for evidence.
The two feel identical from the inside, which is why the question has to be asked explicitly rather than trusted to instinct. Confirmation bias makes a reviewer notice the evidence that supports their initial reaction to an output and skip the rest. Attribution error makes them read a model's mistake as evidence the system is generally unreliable, and a colleague's mistake as evidence about that colleague, when in both cases the situation usually explains more than the character does.
The practical form of this is a habit rather than a training module. Require that an override carries one line stating the information the reviewer held that the system did not. Not a justification and not a category code. One line of fact. Reviewers who cannot write that line will usually realise mid-sentence that they were about to act on a feeling, and the ones who can write it have just generated the evidence you need for the appropriateness review. It is the cheapest control in this article and the closest to free.
Designing oversight that survives scale
The uncomfortable arithmetic is that meaningful review does not scale linearly, and agent deployment does. If every agent action generates a checkpoint, the reviewer headcount required does not exist. Three responses work, and they work together.
Tier the decisions. Not every action needs a human. Reserve review for the actions where an error is expensive or hard to reverse, and let the rest run with sampling and post-hoc audit. This is the same graduated approach that determines whether an agent survives in production or gets rolled back, applied to the review step rather than the deployment.
Batch and space the reviews. Continuous interruption is what converts review into reflex. Scheduled review blocks with genuine time per item outperform real-time approval prompts on everything except latency, and latency is usually a design preference rather than a requirement.
Name the owner. Oversight without an accountable owner degrades quietly, because nobody is responsible for noticing that it has. Deciding who holds that accountability is an operating model question rather than a tooling one, and it belongs with the other decision rights an AI operating model has to allocate. The related question of who owns the output once an agent produces it has to be settled at the same time, because a reviewer who does not own the outcome has no reason to spend attention on it.
None of this is a technology purchase. It is workflow design, accountability design and reviewer capability, which is why it tends to stall in organisations that have treated AI adoption as a tooling programme. Building the capability side of it is what MASSIVUE's AI Workforce Transformation practice covers, through AI skills assessment and gap analysis, AI role design and organisational restructuring, and fluency programmes delivered through MASSIVUE Academy.
For the leaders who have to design these decision points, the relevant starting point is AI for Business Executives (AIBE), which covers what AI can and cannot do and how to lead its use responsibly. Where the systems in question sit in hiring, promotion, pay or performance, which is Annex III territory under the EU AI Act, Applied AI HR Talent Management covers those high-risk obligations directly.
Sources
- Goddard K, Roudsari A, Wyatt JC. Automation bias: a systematic review of frequency, effect mediators, and mitigators. Journal of the American Medical Informatics Association, 2012;19(1):121-127. academic.oup.com
- Poly TN, Islam MM, Yang HC, Li YC. Appropriateness of Overridden Alerts in Computerized Physician Order Entry: Systematic Review. JMIR Medical Informatics, 2020;8(7):e15653. pmc.ncbi.nlm.nih.gov
- Regulation (EU) 2024/1689 (EU AI Act), Article 14, human oversight. artificialintelligenceact.eu
- Regulation (EU) 2026/1744 (Digital Omnibus on AI), published in the Official Journal 24 July 2026, entered into force 27 July 2026. Analysis: Lewis Silkin, The Digital Omnibus on AI enters into force today, 27 July 2026. lewissilkin.com
- Gartner, Gartner Predicts 40% of Enterprise Apps Will Feature Task-Specific AI Agents by 2026, Up from Less Than 5% in 2025, 26 August 2025. gartner.com
- OWASP, agentic AI threats and mitigations guidance, including the overwhelming human-in-the-loop threat. owasp.org
Regulatory position stated as at 19 August 2026. This article is editorial analysis and practitioner guidance, not legal advice.
Frequently asked questions
When should a human override an AI recommendation?
When they hold information the system could not have had, and can say what it is. Context about the situation, the customer or the people affected that never entered the model is a legitimate basis for overriding. A general feeling that the output looks wrong is not, and treating it as one is how over-rejection starts.
Is a human approval step enough to satisfy Article 14 of the EU AI Act?
No. Article 14 requires that high-risk systems be designed so a natural person can effectively oversee them, including the ability to intervene, override, interrupt or stop the system. The capability has to be built into the system and workable in practice. An approval click that a reviewer has no time to think about, and no authority to make stick, is unlikely to meet that standard.
Did the Digital Omnibus remove the human oversight requirement?
No. Regulation (EU) 2026/1744 deferred the main compliance dates for high-risk systems, moving Annex III obligations to 2 December 2027 and Annex I to 2 August 2028. It did not change what Article 14 asks for. Article 50 transparency obligations applied on 2 August 2026 as originally planned.
What is automation bias?
The tendency to use an automated recommendation as a substitute for looking properly at the evidence. It produces two error types: commission errors, where a reviewer accepts an incorrect recommendation, and omission errors, where a reviewer misses something the system did not flag or dismisses a correct flag. The term comes from human factors research and predates generative AI by decades.
What is a healthy override rate?
There is no target worth setting. Clinical studies found average override rates between 46.2% and 96.2% across settings, and the rate alone did not indicate whether reviewers were performing well. Measure whether the overrides and the acceptances were correct, on a sampled basis, and let the rate be whatever it is. Setting a target rate reliably produces a gamed number.
How do we stop reviewers rubber-stamping AI outputs?
Reduce the volume reaching them by tiering which decisions need review, give genuine time per item, audit accepted recommendations rather than only the disputed ones, and remove the career asymmetry that makes disagreeing more expensive than approving. Requiring one line of fact stating what the reviewer knew that the system did not is a cheap and unusually effective control.