Approach

How it works

Agents review, people decide

Human attention is the scarce resource, so we focus it on what matters instead of spreading it thin. Toyota solved the same problem with jidoka, "automation with a human touch": machines detect an abnormality and stop on their own, so they no longer need to be watched over, and people respond only where something has gone wrong (Ohno, 1988). The system has four parts.

1. 01Risk is assessed independently

   AI is a poor judge of its own work. Coding agents rated 99.9% of their own pull requests 8 out of 10 or higher for confidence, with no difference in how often those changes broke things (Ferdous et al., 2026), and model confidence is a poor predictor of whether code is correct (Spiess et al., 2025). So the producer never sets the risk. Deterministic signals from the work itself, such as what it touches and how far a failure would reach, are combined with a separate assessor, independent of whatever produced the work, whose job is to argue for higher risk. Where they disagree, the higher tier stands.

2. 02Agents review adversarially

   Deterministic checks run first. Then independent AI reviewers each take one lens, such as correctness, security, privacy or provenance, and try to find what is wrong. They are chosen to fail differently from each other and from the producer, for example models from different vendors, and they don't see each other's findings, so they can't anchor on them. This is where the routine volume gets handled.

3. 03People decide where it matters

   Work escalates to a person when the reviewers disagree or can't converge within a set number of rounds, when the independently assessed risk is high, or when a critical finding is unresolved. Non-convergence is not an error; it is the signal that judgment is needed. Escalated work arrives with a brief of why it was escalated and what the reviewers found, and each finding needs a human decision, not an acknowledgment.

4. 04Sample what was cleared, record everything

   Not every unit needs full inspection; risk-stratified sampling can make quality measurable instead (Cobb & Mills, 1990). Low-risk work that passed automatically is still spot-checked at random. The rate of problems those samples find measures how well the system is working and recalibrates the tiers over time. Every decision, whether by a rule, an agent or a person, goes into an append-only record.

Adversarial review runs at every tier; risk decides how deep it goes. Deterministic checks and AI review always come first, so when work escalates, the person starts from a brief: why it was rated high-risk, what the checks and reviewers found, and where they disagreed. Attention goes straight to what matters.

Principles

Why it is built this way

1. 01Verification is the bottleneck now

   For most of computing history, producing the work was the slow part and checking it was cheap by comparison. AI has inverted that. Review capacity is fixed by people's time and expertise; AI output grows with compute.

2. 02Neither extreme works

   Reviewing everything doesn't scale: attention thins until review becomes a formality. Trusting the model doesn't either: the risk is still there, only no one sees it until something fails. The work is in the middle.

3. 03Decide explicitly what reaches a human

   Every organization already decides what gets human review, usually implicitly, by habit or by whoever has time. Make the routing rule explicit, risk-based and written down, so it can be checked and improved.

4. 04Risk hides in work that looks routine

   Rubrics that classify work by what it touches miss where failures come from. Coding agents broke fewer things than people overall, yet in maintenance work the picture reversed: their chores and refactors broke things far more often than their new features or fixes, and more often than people's (Ferdous et al., 2026). Those are exactly the changes that get the lightest review.

5. 05Assume reviewers will drift

   Attention fades over a long review and familiarity with a tool lowers scrutiny. Complacency with reliable automation is a well-documented property of how people supervise machines, not a lapse in diligence (Parasuraman & Manzey, 2010). Airline pilots' attention drifts while automation flies (Casner et al., 2014); programmers look less closely at model-generated code than at code written by people (Al Madi, 2023). Design oversight on the assumption that reviewers will become less critical of AI output over time, and check whether attention is still being spent where it matters.

6. 06Agreement between models is not confidence

   Models trained on similar data share blind spots, so several of them agreeing proves less than it seems. In our own systematic review, three models from different vendors agreed far more with each other than with the human reviewer, and none of the records one model wrongly kept was caught by the other two. Choose checkers that fail differently, and keep a person on the decisions that matter.

7. 07Let rules make the call

   Model judgments inform the decision; deterministic rules make it: accept, reject or escalate to a person. Use every deterministic check the domain offers, such as tests, linters, scanners and policy rules, before spending human attention.

8. 08Keep an honest record

   Record what the AI did, what was checked, by whom or what, and why it was accepted. Accountability needs a record that is independent of the system being overseen.

What we are not claiming

The evidence is still thin in places

The field is young. Much of what is written about AI oversight rests on adoption volume rather than measured outcomes. We say what the evidence supports, say where it is thin, and expect to revise.

Our own systematic review (9,502 records screened, 72 studies retained) used AI models as assistants throughout. They were useful assistants and poor gatekeepers: they proposed 91.6% of the tags the human reviewer assigned, but correctly flagged only 57.4% of the studies the human later excluded. That is a reason to design oversight carefully, not to take the human out of it.