← All writing

AI Engineering

Verification Loops: Make Coding Agents Show Proof

Mani G 3 min read

Stacked code cards flowing into a green loop of check marks around a code block, with one orange cross marking a failed check

An agent saying "done" is not proof of anything. That is the whole idea behind verification loops, a term the Claude Code team pushed into the spotlight this week. Instead of trusting a coding agent's claim that it fixed the bug, you give it a way to check its own work and make it show you the result: a passing test, a clean build, a screenshot of the feature working.

Sounds obvious. Most good ideas do, right after someone writes them down.

What verification loops actually are

On November 16, Addy Osmani published a post on the claude.dev blog about how the Claude Code team checks agent-written code. The headline number: about 90% of that team's code is now written and reviewed by Claude. At that volume, nobody can read every diff.

His point is blunt. Without verification loops, you can only run one or two agents at a time, because you have to check everything yourself. With them, you can run dozens or hundreds in parallel. The bottleneck was never typing speed. It was you.

The loop works like this:

  1. The agent makes a change.
  2. It runs checks: tests, build, linter, or the real app end to end.
  3. If something fails, it tries to fix it.
  4. If a check still fails after three rounds, it stops and calls a human.

That last rule is my favourite part. Failures decide where human attention goes, instead of a person skimming everything and catching nothing.

A quick example

Say you ask an agent to add a discount code field to a checkout page. A unit test passes. The agent says done. But the field breaks the "Place order" button on mobile, and no unit test knew to look.

Now picture the same task with a loop in place. The repo has a skill called verify that boots the app and walks through a real checkout: add item, apply code, pay, land on the confirmation page. The agent runs it, sees the button fail, and fixes it before you ever see a pull request. You review evidence, not promises.

The post names three kinds of tests worth the effort:

  • End-to-end tests, because they are hard to fool. The feature works or it does not.
  • Property tests, where a library like Hypothesis or fast-check throws thousands of odd inputs at a rule such as "a balance can never go negative".
  • Differential tests, where the old system acts as the answer key during a migration. Same inputs, compare outputs.

The part people will skip

The sharpest rule in the post: a change must never be able to edit the check that judges it. The team reads its review rules from the base branch, not from the pull request. Otherwise an agent can "pass" by quietly loosening the test. Anyone who has seen a flaky test get a sleep(5) added to it knows this is not paranoia.

Flaky tests get no mercy either. Claude is not allowed to skip them, add sleeps, or retry until green. If it cannot find the root cause, it deletes the test and writes down why.

In my view, this is the most useful agent advice of the year, mostly because it is so unglamorous. No new model, no new framework. Just tests that run, and an agent that has to look at them.

The numbers back it up. In an earlier post, the same team said merged pull requests more than doubled from June to September while the share that had to be reverted fell by more than 40 percent. Boris Cherny, who created Claude Code, has estimated that giving Claude a way to verify its work makes the final result two to three times better.

A surprising related fact

A new benchmark called SWE-CoWork tested coding agents as if they were new hires on a team. Across the evaluated models, 31% of the requirements agents reported as completed failed their tests. Nearly a third of "done" was not done.

So here is my question for you: if your agent had to show proof before saying done, how many of your current checks would actually catch it lying?