If AI Writes and Reviews the Code, What Is the Human Approving?

TL;DR: AI writes the code and separate AI agents validate it through enforced quality gates and inspectable evidence. The human does not blindly trust either agent, but decides whether the remaining risk is acceptable, and owns the result.

Six months ago, an AI agent implemented a change that passed lint and all tests, but the implementation was still wrong. I caught it during code review.

As AI increased my development output, review was becoming the bottleneck. But handing that work to another AI raised a harder question:

Could another AI agent find the same kind of failure without simply inheriting the assumptions of the AI agent that wrote the code?

Finding out turned into a verification system, built in pieces alongside the actual work.

Today, AI writes and reviews most of my code. I still approve every pull request, and I still own the result. But what I review has changed.

I review the evidence the system produced, the uncertainty that remains, and the consequences of being wrong.

This article explains how that system works, what its PASS actually establishes, and what still requires human judgment.

There is a larger version of the validation flow diagram on its own page, big enough to read gate by gate.

1. The verification system between AI-generated code and human approval

The diagram has three lanes and two human boundaries.

At the top, the human owns the requirements, acceptance criteria, architecture rules and agent rules. At the bottom, the human owns the residual risk and the decision to ship.

Between those boundaries:

1. The orchestrator owns the workflow. It tracks state and starts the next work stage.
2. Subagents own the work. Requirements, architecture, implementation and validation happen in separate, narrow contexts.
3. Artifacts own the state. The spec, acceptance criteria, plan, diff and validation reports live outside any agent conversation.

The flow is:

Human definition
  -> requirements and architecture
  -> task-level developer-validator loops
  -> whole-feature validation
  -> repository CI and separate PR review
  -> risk-based human approval

Every failed gate returns the work to implementation and validation. Nothing advances just because an agent says it is ready.

I described the wider quality pipeline in How I Validate Quality When AI Agents Write My Code (opens in a new tab). Here I want to focus on the harder question: what prevents AI validation from becoming an expensive rubber stamp?

2. A real example: how green tests became a blocking FAIL

The task was to lock navigation while a pipeline was running. Before implementation began, it had three acceptance criteria:

After implementation, every visible mechanical signal was green:

The validator mapped the implementation back to the original criteria:

The code contained a PipelineRouteGuard, but it depended on router context that the production application never supplied. Its unit test fabricated that context, so it verified an isolated callback under conditions that did not exist in the real application.

The validator agent then ran a falsification test: temporarily remove the guard and rerun the suite. All 6,104 tests still passed.

That turned a plausible concern into a VERIFIED, HIGH finding and blocked the task. The suite had executed code, but it had never proved that the requested behaviour existed in the production integration.

This example contains the central design principle of the factory:

The behaviour being gated must be defined independently of the implementation.

3. Validation needs a definition of correct that predates the code

AI-generated tests can encode the same wrong assumption as AI-generated code. If test intent is derived from the code, implementation and tests may agree perfectly while the user-facing behaviour is wrong.

In this agentic coding workflow, I work with a requirements agent until the intended outcome is clear and the acceptance criteria are testable. The architect then designs inside human-owned architecture rules and divides the work into bounded tasks.

That creates a traceability chain:

business goal
  -> acceptance criterion
  -> implementation plan
  -> task
  -> code change
  -> validation result

The validator can walk that chain in reverse: why does this change exist, which agreed behaviour should it deliver, and what evidence shows that it does?

Passing tests remain useful evidence. They are not the definition of correctness.

The developer's tests confirm that the code behaves as expected. Independent validation determines whether that expectation matches the task.

4. Independent AI code review shares evidence, not reasoning

A common failure mode in AI code review is asking an agent to “review your work” inside the same long-running context that produced the implementation. The model already has a coherent story for why its solution is correct. A detailed review prompt does not remove those inherited assumptions.

Starting a fresh agent is necessary but not sufficient. If the new context receives the developer's reasoning, summary and persuasive explanation, it inherits the developer's frame.

My validator starts from the original source artifacts instead:

• the task and requirement
• the acceptance criteria
• relevant architecture rules
• the implementation plan
• the changed code and diff
• its own role file and finding rubric

It receives none of the developer's conversation or claims about why the implementation should pass.

The validator reruns the relevant build and tests itself, verifies that the required artifacts exist, and then evaluates what mechanical checks cannot decide: whether the acceptance criteria are met, whether the implementation follows the architecture, whether it introduces unnecessary changes or duplication, and whether an undocumented decision should have been escalated.

This is context isolation, not statistical independence. Agents from the same model family can still share blind spots. A fresh context reduces anchoring and confirmation bias; it does not eliminate correlated errors.

The separation works because three roles have different responsibilities:

• The orchestrator transfers control. It tracks whether a task is waiting for implementation or validation and follows the standard transition when the result is unambiguous.
• Specialist subagents perform narrow work. A developer implements one task; a validator tries to disprove that it is ready.
• Artifacts carry the handoff. Requirements, code, tool output and findings remain durable and inspectable outside the conversations.

There is no conversational handoff between specialists. They communicate through shared artifacts.

On PASS, the orchestrator commits and moves to the next task. On FAIL, it starts a new developer-validator loop; workflow exceptions are handled without taking ownership of the underlying technical decision.

The technical finding stays in the report for the next developer and for later audit. The orchestrator does not need to adjudicate which technical argument sounds more convincing.

This is the difference between a review bot and a gate. A review bot writes comments that may be ignored. A validator controls whether the workflow may continue.

5. A suspicion cannot block the workflow

AI code review becomes useless when one real problem arrives with twenty plausible suspicions. If a human must investigate every hunch, automation has only moved the bottleneck.

Every finding therefore answers two separate questions:

1. How serious would the issue be if it exists?
2. What did the validator do to show that it exists?

Severity uses project-specific anchors:

Evidence records what the validator actually established:

The workflow decision comes from a project-owned matrix:

A severe but unverified issue triggers targeted verification: the validator must name and run the concrete check that would confirm or kill it. Speculative low-value findings are dropped.

These thresholds are not universal. I have adjusted them over months of real use because an uncalibrated reviewer either misses defects or floods the workflow with plausible noise. I tune them against real and historical defects: does the validator find the known problem without flooding the report? Humans define what is dangerous for the project; the agent applies that decision.

6. An agent's claim is not evidence

An agent saying it ran a check is still only an agent claim.

Claim:    "I ran the tests and they passed."
Evidence: A recorded report naming the command, result, test count and failures.

In one implementation, a wrapper runs the test command and writes a structured result into the workspace. A stop hook verifies that the report exists, is current and can be parsed. The workflow uses the recorded result to decide whether the task continues.

The useful pattern combines model judgment with deterministic enforcement:

• models evaluate requirements, architecture and code paths
• scripts run and capture mechanical checks
• hooks or workflow code verify expected outputs
• both leave inspectable artifacts

An artifact is not automatically true because it exists. A model-written summary is weaker than captured tool output. Captured output is weaker than a reproduced failure when the question is whether a bug exists.

The artifact makes the claim durable and checkable. Its provenance determines how much weight it deserves.

7. Passing every task does not prove the feature works

A feature is not merely the sum of its tasks. Local gates can pass while behaviour is broken across task boundaries.

After task-level validation, the factory runs lint, type-checking, relevant test suites and the production build. It then runs a multi-agent review of the whole feature, each agent from one narrow perspective:

The number of agents is not the point. Each reviewer needs a narrow question, relevant source artifacts and a clear reporting threshold.

Each flagged issue gets a separate validation context. The validator must confirm that the problem exists in the changed code before it enters the final report. A finding that cannot be reconfirmed dies there.

I have not measured the results systematically, so these are working estimates rather than benchmark data. In my own use, task-level validation finds something worth correcting in roughly half of the tasks. Final validation sends about one in four completed features back for further work, usually for architecture, code reuse or business-logic issues rather than basic test failures.

The token accounting is measured, though. One feature through every gate above:

Task-level validation is the cheap part. The Phase 1 developer spent 427 calls and 87,864 output tokens. The validator that checked it spent 63 calls and 5,674, and read about 3% as much cached context. The other two rounds have the same shape.

Final validation is where the verification cost actually sits. Six reviewers over the whole diff, plus one that was stopped, produced 107,771 output tokens between them, more than any single developer agent. That is the price of asking the same diff several narrow questions instead of one broad one.

Checking the work costs a fraction of doing it.

The factory is not equally deep for every change. Review scope and gate depth should scale with the uncertainty and consequence of failure; the design principles remain the same even when a low-risk task uses fewer stages.

A blocking finding returns the work through implementation and validation. A pass allows the orchestrator to open a draft pull request.

Repository CI still runs every deterministic check it ran before any of this existed: the build, the full test suite, linting and static analysis. The agents replaced none of them. Nothing in this system is a substitute for the engineering practices that were already load-bearing, and a red pipeline blocks the pull request exactly as it always did.

A separate AI reviewer, GitHub Copilot, then examines the final merge diff. It runs on a different model family than the agents that wrote the code, so it does not carry the same assumptions about what correct code looks like. That shrinks the shared blind spot.

A PASS therefore means only this: every defined quality gate completed without a blocking finding. Its confidence is bounded by the quality of the acceptance criteria, the coverage of the checks and the blind spots shared by the models.

8. AI cannot own the rules that judge AI

Did I build the system, or did AI build the controls that approve AI?

If a coding agent can rewrite the rubric, architecture rules or role files that judge its work, the loop is closed in the wrong direction. The agent can gradually redefine success around its own output.

In my projects, agent rules, architecture rules and other decision boundaries are human-owned and locked. Agents may read them and propose changes, but they cannot push changes to them.

Locked does not mean frozen: repeated findings can produce new rule proposals, but only a human can approve them into the harness.

Inside those documented boundaries, agents can act autonomously. If implementation requires a new architecture choice, product rule or risk decision that the documents do not resolve, the agent must stop and escalate instead of improvising.

AI acts within documented decision boundaries. Undocumented product, risk and architecture decisions are escalated to humans.

The technical enforcement of those locked controls is described in 5 months. 100% Claude Code. Zero architectural drift. The accountability point here is simpler: agents must not own the controls that decide whether their own work is acceptable.

9. The human reviews what the system cannot decide

After all these quality gates, the pull request still reaches a human.

AI can generate and inspect more code in a day than I can realistically reread line by line. If responsibility means pretending otherwise, human-in-the-loop review becomes either a permanent bottleneck or a superficial ritual.

Instead, I inspect the scope, evidence and consequences:

• Which files, services and system boundaries changed?
• Which business behaviours are affected?
• Which parts of the change are critical, and which are routine?
• What evidence did the quality gates produce?
• Which assumptions or decisions were escalated?
• How visible, reversible and expensive would a failure be?

High-consequence changes receive a deep human read. Authentication, permissions, payments, migrations and security controls do not get the same review depth as visible, easily reversible UI code.

Low-risk changes are still reviewed, but the depth of human inspection is proportional to the cost, detectability and reversibility of failure.

AI reviews the volume. Humans review the risk.

More precisely, the human reviews the evidence, the uncertainty that remains and the consequences of being wrong.

10. Approval means accepting the risk that remains

When I click Approve, I know:

• what the change was supposed to do
• which evidence allowed it to pass
• which decisions were escalated
• where the verification system is still uncertain
• what failure would cost

My approval does not mean the system proved the change correct. It means the defined quality gates found no blocking evidence and I accept what remains unproven.

If I ship the change, I still own the result.

Review was becoming the bottleneck. Now it is a decision, not a queue.

After about two years of working with AI, I trust the verification system around the models more than any individual model judgment.

Trust comes from the verification system around the model.

Tools: Claude Code (orchestrator on Opus, developers on Sonnet, task validator on Haiku), Beads and Linear for issue tracking, GitHub Actions for CI, GitHub Copilot for the separate PR review.