How to merge more PRs without scaling code review

Coding agents now open pull requests faster than any team can review them. You cannot fix that by hiring more reviewers or by asking people to read faster. To merge more PRs without scaling code review, stop treating every PR the same and answer one question for each one: does this PR need a human review? A short list of rules sends the risky areas to a person every time. An AI review judges the rest, and when it is unsure, a human looks. You roll it out slowly, starting in a mode where it only comments. It only works if your engineering is already solid.

I built this as an open-source GitHub Action, Needs Human Review. This post explains the idea behind it, which comes from chapter 8 of my book, and how to use it without putting your main branch at risk.

Try it now

If you want to see what it would decide on your own PRs, add it in shadow mode. It only comments and never merges. The full workflow file is in the README. The important part is this step:

- uses: alfonsograziano/needs-human-review@v1
  env:
    OPENCODE_API_KEY: ${{ secrets.OPENCODE_API_KEY }}
  with:
    mode: shadow
    protected-paths: |
      src/payments/**
      **/migrations/**
      *.tf

Leave it for a couple of weeks and compare its answers with what your reviewers decided. You can see it on real pull requests in the public demo repo.

Why code review became the bottleneck

For years, writing the code was the slow part. A reviewer could keep up because the author could not produce much more than one or two PRs a day. Coding agents changed that. One engineer can now run several agents at once, and each of them opens pull requests all day. I wrote about where the constraint moves in what happens when coding is not the bottleneck anymore. Code review is one of the first places it lands.

I have been that reviewer. Ten PRs every morning: a typo, a new test, a config tweak, another config tweak. By number six, my "review" was scrolling to the bottom and typing LGTM. Somewhere in that pile there were maybe three PRs that really needed me, and they got the same twenty seconds as the typo. The PR that touched payments got the same tired look as the one that fixed a comment.

Reading everything does not make you safe. It makes reviewers tired, and tired reviewers approve things without reading them. What I wanted was simple: show me the three PRs that need my attention, and let a system mark the other seven as safe to merge. Then my review time goes where it changes the outcome.

Not every PR needs a human

This is the idea that makes people uncomfortable. If a change is small, well tested, easy to undo and far from anything critical, a human reading it adds very little. A human reading a payment change adds a lot. Treating both the same way wastes attention on the first and starves the second.

So the goal is not "review faster". The goal is to know which PRs need a human, and to spend your reviewers' time only there.

Two things decide: your rules, then an AI review

First, your rules. A short list of paths and checks where a human always has to look: payments, auth, database migrations, infrastructure as code, CI workflow files, lock files, PRs from forks. If a PR touches one, a human reviews it. This part is plain code. No AI, no judgment, no exceptions. It works like a CODEOWNERS file, and its power is that it is boring: fast, predictable and impossible to argue with.

Then, an AI review. Rules only catch the risks you can name in advance. They never catch the subtle logic error or the clean-looking refactor that breaks a caller. So every PR that passes your rules goes to an AI review that reads the change and answers the question: does a person need to see this? It gives its reasons in plain words.

Yes, this means you are using AI to decide when not to trust AI. That is only safe under one rule: it has to fail safe. When the AI is unsure, a human looks. When it crashes or times out, a human looks. The AI can add a reviewer, but it can never remove one, and it can never override your rules.

What makes the AI review worth trusting

A model that skims a diff is not enough. Needs Human Review is built so the AI part earns its place:

• It reads more than the diff. The agent opens the whole changed file and the untouched code the change can reach. A function that now returns null looks fine in a diff. It is a bug if a caller 30 lines below does not check for it.
• It has to prove it read the code. The action checks the agent's tool log. If a changed file was never opened, the answer does not count, and a human looks.
• It is read-only. The agent can only read and search files. It has no shell, no web access and no way to edit anything.
• PR text cannot talk its way past the rules. The rules never read the PR description. The AI treats PR text as data, and any text that tries to instruct it is a reason to ask for a human.
• You set the bar. The review prompt is a file you can extend or replace, and you choose how much risk may skip review and how sure the AI has to be. LLMs are poorly calibrated when they give confidence as a percentage (Xiong et al., ICLR 2024), so the action asks for one to five stars with fixed meanings instead.

The first release was tested on 7 scenarios as real PRs on the demo repo, including a prompt injection in a PR description and a change that broke two untouched callers, which the review named with their line numbers. It answered all 7 correctly. That is a smoke test, not a benchmark, so treat it as a tool to try, not a promise.

Roll it out like any other risky change

You would not give a new tool the power to merge on day one, so it has three modes:

1. Shadow. It comments on every PR with its answer and what it would have done. It changes nothing. This is where you collect evidence.
2. Advisory. It adds a needs-human-review or safe-to-merge label and a human-review status check. A human still merges. You can make the status required, so a "human review required" answer blocks the merge button.
3. Autonomous. The PRs that need no human review merge on their own.

Move up only when the data says so: how often its answers match your reviewers', and how often merged PRs get reverted. Critical paths never climb this ladder. A human reads them, always.

Watch the overrides

Once it runs, the most useful number is how often a human disagrees with it. If people keep reviewing PRs it said were fine, it is too loose, so tighten the prompt or add a rule. If they keep merging PRs it flagged, it is crying wolf, and soon everyone will ignore it. Both kinds of drift are quiet, and both are easy to see if you look at the overrides every few weeks.

The catch: this only works on a mature codebase

Skipping human review is the last layer you add, not the first. It sits on top of tests, type checks, linting, security and secret scans, an LLM review of the code, safe deploys with feature flags and a fast rollback, and alerts that catch what slips into production. Each of those layers makes a bad merge cheaper. If they are weak, auto-merge does not make you faster. It just ships bugs faster.

So if your CI is flaky or your tests are thin, do not start here. Fix the floor first. My guide on how to set up your repository for AI coding agents is a good place to begin, and the self-improving harness post shows how to keep that setup getting better.

Where to go next

• The tool: Needs Human Review on GitHub. Start in shadow mode, and open an issue when it gets a PR wrong. Wrong answers are the most useful feedback right now.
• The thinking behind it: chapter 8, "Verification and Quality Gates", of AI-Native Software Engineering, which covers the five layers of safety around AI-generated changes and the human in the loop on top of them.