Human in the Loop AI Governance: Is Your Approver a Rubber Stamp?

Human in the loop is AI governance only when you can show the approver catches errors, and most approval logs cannot show that. Suppose a reviewer approves 99% of agent actions in four seconds: the log proves a click, not oversight. This guide is for the CTO, operations head or compliance lead who has put a person in front of an AI agent and now has to defend that choice. It shows how to measure the approver rather than the agent, how many test cases make that measurement trustworthy, and which approvals to keep.

Singapore's Infocomm Media Development Authority (IMDA) added override-rate and response-time monitoring to its agentic AI framework in May 2026. On 30 June 2026, Bank of England Deputy Governor Sarah Breeden said that relying on a human in the loop for all agent actions is unlikely to be realistic. Clicking approve on everything no longer counts as an answer on its own.

Why do human approval queues turn into rubber stamps?

Human approval queues decay because the agent is usually right. Automation bias, the tendency to over-trust an automated system, grows as that system proves reliable. IMDA's Model AI Governance Framework for Agentic AI (Version 1.5, May 2026) names the trigger: over-trust is a concern "especially when it has performed reliably in the past."

The research behind that sentence is decades old. Parasuraman and Manzey's 2010 review in Human Factors separates errors of omission (missing a problem the system did not flag) from errors of commission (following the system against better evidence). Their abstract states that automation bias "occurs in both naive and expert participants, cannot be prevented by training or instructions, and can affect decision making in individuals as well as in teams."

Expertise does not rescue you. In a 2023 lab study in Radiology, 27 radiologists rated 50 mammograms with a purported AI assistant that was deliberately wrong on some cases. In the study's results, very experienced readers rated 82.3% of cases correctly when the suggestion was right and 45.5% when it was wrong. For inexperienced readers the study reports 79.7% and 19.8%. Specialists, reading their own kind of image, were still pulled along by a wrong hint.

Agent work is also hard to check. Lyell and Coiera's systematic review of 40 studies (JAMIA, 2017) scored how many steps it takes to confirm that automation is right, which they call verification complexity. In single-task experiments, only tasks rated medium or high produced automation bias, and the authors tie the effect to cognitive load, while noting that few studies tested significance against a control. Our inference: a tool call with ten parameters, or a five-step plan, sits at the hard end of that scale.

People also hand work back to the machine as trust builds, on purpose. Anthropic's February 2026 data on its own Claude Code product shows users with fewer than 50 sessions turn on full auto-approve roughly 20% of the time, rising to over 40% by 750 sessions. These are developers on a coding agent, not staff clearing refunds, but the lesson carries: oversight drifts towards whatever the design makes easy.

What do IMDA and the EU AI Act ask of the approver?

IMDA and the EU AI Act both name automation bias, and neither sets a number that proves a review was real. IMDA's framework is voluntary guidance that suggests indicators such as override rates and response times. The EU AI Act is binding law that requires effective human oversight for high-risk systems only. This is general information, not legal advice.

IMDA's Version 1.5 (published 20 May 2026, updated 5 June 2026) is the more specific of the two. Its "what's new" table says it "added more practices to guard against automation bias, such as monitoring human override rates and response times." Section 2.2.2 suggests tracking the human override rate, "the frequency at which humans reject or modify agent actions. A low rate may signal rubber-stamping behaviours," and response times, where "a shorter time may signal automation bias or review fatigue." For high-risk actions, users "can be required to provide an additional written justification before approving or rejecting." That is a playbook, not a rule, as our reading of the IMDA framework against the EU AI Act sets out.

The EU AI Act, Regulation (EU) 2024/1689, is law, with a narrower reach. Article 14(4)(b) requires that people overseeing a high-risk system are enabled "to remain aware of the possible tendency of automatically relying or over-relying on the output produced by a high-risk AI system (automation bias)." Article 14(4)(d) adds the power to "disregard, override or reverse the output", and Article 26(2) requires deployers to assign oversight to people "who have the necessary competence, training and authority, as well as the necessary support."

The dates moved this summer. The Digital Omnibus on AI, Regulation (EU) 2026/1744, entered into force on 27 July 2026. Under it, Chapter III Sections 1 to 3, which contain Articles 14 and 26, apply from 2 December 2027 for systems listed in Annex III and from 2 August 2028 for those covered by Annex I. Classification turns on the use case. A sales assistant or an invoice matcher usually falls outside both annexes. An agent that scores the creditworthiness of individuals, prices life or health insurance, or filters job applicants is listed in Annex III. Outside the annexes, Article 14 is a design benchmark, not a duty.

How do you test whether an approver is real?

You test an approver by checking whether they catch errors you already know about. Override rate and review time, the two indicators IMDA's May 2026 framework names, describe behaviour but cannot prove attention. Add run length, an agent-blind second review and seeded test cases, and the five together tell you whether the human is a control.

Start with the base rate. If an agent is right 99% of the time, a reviewer who approves everything unread looks 99% accurate and has caught nothing. The arithmetic gives one hard number: if reviewers override less often than the agent is wrong, at least the difference is getting through. The converse fails. A reviewer who overrides 3% of actions may be fixing typos while waving through the one wrong payment.

Each signal, its warning sign and its blind spot (calibrate every threshold on your own data):

The last row makes the others readable. A probe is a test action with a known wrong answer, such as a refund to a closed account. Build each one by copying a real past item and changing one field that matters, so it looks like the work. Block probes at the execution layer, never with a visible test flag: once reviewers can spot the harness, the catch rate measures the harness. Rotate the patterns monthly, with compliance sign-off.

The agent-blind review builds on Goddard and colleagues' 2012 systematic review of 74 studies, which found, on limited evidence, that offering supportive information rather than commands reduced automation bias. Remove the recommendation and you learn whether your first reviewer added judgement or only echoed the agent.

Small probe counts mislead. Ten probes caught eight times reads as 80%, but the 95% confidence interval (the range the true catch rate plausibly sits in) runs from about 49% to 94%. The ranges below use the Wilson score method.

Seed enough to separate outcomes. At 30 probes per tier, a queue catching 27 (range 74% to 97%) is distinguishable from one catching 18 (42% to 75%). At ten probes, nine caught (60% to 98%) and six caught (31% to 83%) overlap heavily.

Illustrative example: a support team's agent proposes 2,000 refunds a week, and a periodic audit puts its error rate at about 1%, so roughly 20 bad refunds a week. Reviewers override 10 actions a week, a 0.5% override rate, with a median review time of 4 seconds. Even if every override hit a real error, at least 10 bad refunds a week are approved. The team seeds 20 probes over a month and reviewers catch 8: 40%, with a range of 22% to 61%. Even the top of that range is poor. The decision: stop asking people to review 2,000 refunds. Let refunds under a fixed amount to verified accounts run inside hard limits with sampled audit, send the remaining 300 or so to forced-choice review, and judge the change by whether the catch rate rises. The numbers are made up. The arithmetic is the point.

Which redesigns keep the human meaningful?

Meaningful human oversight comes from asking people to make fewer decisions, each one harder and better supported. IMDA's May 2026 framework warns that requiring human approvers to continuously oversee agents "can also lead to automation bias and alert fatigue." The Bank of England reached a similar view on 30 June 2026.

Sarah Breeden's speech at the ECB Forum on Central Banking, "Agents of change", was about trading and payment agents and financial stability, not company workflows. The sentence travels anyway: "relying on a human in the loop for all agent actions is unlikely to be realistic. More sophisticated governance and accountability frameworks may be needed." Anthropic draws the same conclusion from its own data, where experienced users interrupt more often (about 9% of turns against 5% for new users): "effective oversight doesn't require approving every action but being in a position to intervene when it matters."

Four redesigns follow from the evidence.

Approve the boundary, not every action. A person signs off a refund ceiling, a list of permitted recipients or a data scope once, with a written reason. The agent then acts inside those limits, enforced in code rather than in the prompt. Our post on tiering agents by reversibility covers how to draw the tiers.

Make the remaining reviews forced-choice. Replace a single approve button with approve, edit, reject or escalate, nothing preselected and no approve-all on high-risk queues. Ask for a reason on high-risk items, as IMDA suggests. Show the evidence before the agent's recommendation, so the reviewer forms a view first.

Cut verification effort. Lyell and Coiera suggest that strategies to minimise automation bias "might focus on cognitive load reduction." In practice: show what changed and how far it sits from normal, not a raw log. IMDA asks for approval requests that are "contextual and digestible, while making the risk clear."

Hold reviewers to accuracy, not throughput. Skitka, Mosier and Burdick found in a 2000 study that making participants accountable for their overall performance or their decision accuracy led to lower rates of automation bias. The study did not test throughput targets, but our inference is plain: a scorecard that counts items cleared per hour pays for the rubber stamp. Score errors caught on probes instead.

Who decides what still needs a human approver?

A human approver earns their place where an action is hard to reverse and a careful reviewer can check it in the time the business will pay for. The CTO, the managing director and compliance each own part of that call. IMDA's May 2026 framework says deploying organisations and their human overseers remain accountable, whether the agent was built in-house or bought.

Route each action type with that rule. Reversible actions run inside hard limits with sampled audit. Irreversible actions a reviewer can check in time get forced-choice review with probes. Irreversible actions nobody can check in time are an evidence problem: fix the approval request before adding people.

For the CTO or technical lead, the work is plumbing. Log per decision the reviewer, risk tier, outcome and reason code, both timestamps, whether the evidence panel was opened and whether the item was a probe. Build the probe harness so test actions can never execute. Deny by default when approval infrastructure fails, for example when no supervisor can be reached, as IMDA recommends.

For the MD or owner, the decision is economic. On the illustrative refund queue, even 30 seconds per item would cost about 17 reviewer hours a week across 2,000 reviews, and catch little. Three hundred reviews at two minutes each cost 10 hours and buy an actual look. Our post on designing the confirm step covers that remaining step.

For compliance and legal, the shift is in what counts as evidence. An approval log shows a click. Per-tier override rates, time against a floor you timed yourself and a probe catch rate are better evidence that the click was a review. They are still your evidence, not a standard IMDA or the EU AI Act has set, and they do not classify the system or discharge any duty under Monetary Authority of Singapore (MAS) guidance or the Personal Data Protection Act. Decide in writing which figures you would produce if a supervisor asked. Our post on where accountability lands in the agent value chain covers what the deployer cannot hand off.

For reviewers, the redesign should feel like relief. Fewer items. A score that rewards catching the wrong refund.

Where Origin Pi stands

We treat the approver as a component with its own failure rate. That rate climbs in exactly the conditions a successful deployment creates: an agent that has been right for a month and a queue that keeps growing. So it gets measured in week twelve, not only at go-live.

So we design agent workflows the other way round, starting with bounded permissions: hard limits in code that let routine, reversible work run without a person, so the confirm step is reserved for actions that can hurt you. We design that step as forced choice with the evidence up front. And we build the audit trail to record the approver as well as the agent: tier, timing, reason, evidence opened, probe results.

Keep the split visible. Override rates, response times and deny-by-default come from IMDA. Probes, agent-blind review and the sample-size rule are our additions, not anyone's standard. None of it makes an agent compliant by itself. It does make your oversight testable.

Your first step this quarter: seed 30 known-wrong test actions into your highest-risk approval queue, with compliance sign-off and execution blocked, and report the count caught with its range. If the range sits low, change one design variable, such as volume or evidence order, and run the same probes again.