Intent-Driven AI Code Review: Automate Verification, Keep Human Judgment

Nov 01, 2026 497 views
Illustrated computer monitor with lines of text, reaching hands, an atom, and an eye, evoking AI code review

AI can check a change consistently; it can’t take responsibility for deciding whether the change should exist. That distinction is the premise behind a different kind of code review: agents verify the work, while people judge whether it matches the team’s intent.

Most AI review workflows still ask a model to inspect a diff and pronounce it good or bad, then ask a person to skim the response and approve. The model can read every line without tiring, but it doesn’t know a team’s unwritten standards. Even documented instructions don’t guarantee it will follow them.

“Judgment needs intent, and accountability needs a person.”

A diff also can’t tell an agent what the team decided earlier, or whether the right answer is to build the feature at all. That’s why an AI review needs more than a second opinion on code. The question is what a review looks like when verification belongs to agents and judgment stays with people. Aviator says it has built that approach into Aviator Verify; its design has seven parts.

The pieces of an intent-driven review

Click to enlarge image.

The framework checks each change against two layers. Acceptance criteria say what the change is meant to do. Invariants capture the engineering rules that should hold regardless of the change.

Either layer alone leaves a gap. Criteria without invariants depend on engineers remembering every team standard—or catching an agent’s familiar mistakes afterward. Invariants without criteria can keep code within the rules while letting the agent solve the wrong problem or build more than needed.

Aviator’s seven steps connect those layers to a human decision:

  1. Invariants, before work begins: standing team rules, drawn from past code reviews.
  2. Intent and acceptance criteria, during the coding session: why the change is being made and how it should behave.
  3. Core decisions, during the coding session: the choices made along the way and who made them.
  4. A test plan, at submission: applicable invariants and acceptance criteria turned into scenarios.
  5. Evidence, at submission: results from a running environment to support each verdict.
  6. A human review: a person considers intent, decisions, and evidence, then approves or asks for changes.
  7. Refreshed invariants, after merge: waivers and recurring review comments help refine the rules.

1. Put team rules where verification can use them

Every review applies some standard, but in many teams those standards live in senior engineers’ memories and surface as repeated comments. Use the Money type for currency. Don’t write directly to the users table. Emit a counter on every error path. When rules like these stay unwritten, coding agents can repeat the same mistakes indefinitely.

An invariant makes a recurring rule explicit so it can be checked on every relevant change. Taken together, these rules form what Aviator calls an AI slop registry: a record of patterns a team keeps correcting.

Files such as CLAUDE.md can guide generation, but the same nondeterministic agent that writes the code is also expected to obey those instructions. That doesn’t show whether it did. Invariants belong in the verification process, where a deterministic script—or a separate agent—can return a verdict for each rule.

Much of the raw material already exists in pull request history. Aviator Verify can read GitHub review comments and draft invariants from recurring patterns. It can also draw on CONTRIBUTING.md and agent instruction files, or draft a rule when someone tags @aviator invariant in a PR comment or Slack thread. Teams can create invariants with an Aviator Verify free account and keep them if they later stop using Verify.

Whether those rules help depends on how they’re managed:

  • Assign ownership. Send each proposed invariant to the team responsible for it, with an explanation of why it was suggested. It becomes an active check only after that team approves it.
  • Describe the required outcome. Leave implementation details open so a rule still works after code is renamed or moved.
  • Limit where it runs. Apply a rule only where it can matter; a frontend check, for example, shouldn’t run against database migrations.

2. Capture intent from the work itself

A specification or detailed Jira ticket can tell a reviewer what a change is supposed to do. But specs take time, are often written before implementation teaches the team anything, and can drift once coding starts.

The coding session may hold a more current account. When an agent asks a question and an engineer answers, the exchange can define both the purpose of the work and how the result should behave. The intent explains why the change exists; acceptance criteria describe what it must do.

Verify captures that information from the session. Before the agent opens a pull request, an agent skill and hook run verify submit. The agent extracts the intent and criteria from the session, and the engineer can correct them before submission.

Click to enlarge image.

3. Record the choices hidden behind the diff

Many changes involve decisions that could have gone another way: where to store state, what to do when a dependency fails, or whether to reuse a helper or write a new one. Those choices can shape an AI-written pull request without appearing clearly in its diff.

“An AI-written PR is full of calls like these, and almost none of them show up in the diff.”

Verify stores decisions alongside intent and marks whether a person or an agent made each one. That gives reviewers something specific to question. The argument for focusing review on those choices is also laid out in “How to review AI code without reading the code.”

4. Turn requirements into a test plan

At submission, Verify uses the intent and acceptance criteria, then examines the diff to select relevant invariants. It combines both sources into scenarios an agent can run to check behavior. The scenarios come from the requirements, rather than treating the code itself as the definition of success.

That division lets the engineer concentrate on the purpose of the change while the invariant registry checks the team’s standing standards.

5. Ask for evidence, not just an opinion

If one model both writes and reviews code, the two passes may share the same blind spots. Verification needs to be separate from generation, and its results should be backed by something a reviewer can inspect.

“When the same model writes and reviews the code, they share the same blind spots.”

Verify can start a preview environment and send verifier agents through the scenarios. They collect screenshots, database state, server logs, and console logs; each verdict on a criterion or invariant links to supporting evidence. For interface changes, the verifier clicks through the UI to check the result.

A preview environment isn’t required at the outset. Code scans can check structural issues directly from the diff, without extra infrastructure, and some invariants can become deterministic scripts. A preview becomes useful when a criterion asks about behavior the diff alone can’t reveal.

6. Let people make the call

The reviewer’s job shifts toward examining evidence. They read the intent, inspect the areas where the core change happens, consider the recorded decisions and the verdicts for each criterion and invariant, and can open the live preview to see the behavior firsthand.

Click to enlarge image.

The pull request remains the team’s home for the change. Verify adds a review link, leaves the diff in place, and reports results through a status check similar to CI.

The reviewer decides whether the evidence matches what the team agreed to build. They can approve, challenge a decision, request another scenario, or waive a failed invariant with a reason. That final judgment has an owner: a person.

7. Use reviews to improve the rules

When a review ends at merge, its lessons can disappear. The same mistake then returns in a later pull request, along with the same comment.

Invariants offer a way to retain those lessons. An author can waive a failed rule with an explanation—for instance, because it doesn’t apply, the rule is wrong, or the fix will come separately. The team that owns the invariant can see each waiver.

Click to enlarge image.

Teams can track which invariants apply most often, how frequently they fail, and why people waive them. Repeated waivers are a signal to rewrite or narrow a rule. A review comment that keeps recurring can become a draft invariant. Over time, the registry can reflect what the team has learned and check future changes against it.

Start with the rules you already repeat

Introducing all seven steps at once may be too much. Start with invariants: they’re the simplest piece to establish, and existing PR history gives teams a source for them. The first month may mean doing conventional code review while the registry takes shape. In the second, the author argues, invariants should catch enough issues to shorten reviews. By the third, reviewers can focus on intent, decisions, and evidence, returning to the diff when something doesn’t add up.

That timeline is a projection, not a guarantee. The underlying idea is more durable: recurring review comments are potential rules, and documenting them gives agents a standard they can be checked against. Aviator Verify turns those rules into invariants, checks changes against them, and leaves decisions to people.

Create your team’s invariants with the Aviator Verify free account; the invariants remain yours if you stop using the product.

The original post, “A step-by-step guide for how to kill the line-by-line code review,” appeared on The New Stack.

Source: Ankit Jain · thenewstack.io

Comments

Sign in to comment.
No comments yet. Be the first to comment.

Related Articles

A step-by-step guide for how to kill the line-by-line cod...