How we made AI code review 3.4× faster, 65% more thorough, 50% cheaper

What moved instead

We cut the system prompt from 38,500 characters to 3,400 — deleting every worked example and trap catalogue. Precision went from ~73% to ~94%. Fewer tokens in, better output out.

The failure this explains

For months our triage and dedup ran on a cheap helper model while the reviewing agents ran on the frontier one. Triage silently capped recall; dedup collapsed and posted five comments describing one root cause on a customer PR. Putting the whole pipeline on one model took that PR to six distinct findings — at no extra cost per token.

Result

Ensemble-grade coverage at a third of the price. This single restructuring is the largest cost reduction in the reviewer that had nothing to do with changing models.

The same principle runs one level down. Before the agent starts, we resolve the callers of every symbol the diff removed with ripgrep and hand it the answers — concrete caller data instead of hoping it greps. Work a lookup can do deterministically should never cost an agent turn, and every turn it saves comes straight off the critical path.

The reading

We sent essentially the same context and got a better review back in under a third of the time. That is not a pricing story. Prefill is parallel; generation is serial. Cutting 54% of output tokens and 40% of round trips is the entire latency result — a model that reaches the same conclusion in fewer words and fewer turns is faster for reasons that have nothing to do with its rate card.

This is the one lever where the model itself moved the number, and it is worth being precise about which property paid: efficiency, not power. The same benchmark that rewarded a terser model had already refused to reward a stronger one.

The counterintuitive part

86% of our input is served from cache, so re-reading a large repository context is nearly free — uncached input is two tenths of one percent of the bill. Establishing that cache is the single biggest line item on it. The lever most teams reach for is trimming context, and that is not where this bill lives.

We assumed the answer was prefix stability — how often you invalidate the cache rather than how much you send. That was wrong, and the next section is what the number turned out to mean.

Which moves the target

Each turn appends a tool result, and that increment is cache-written exactly once. The 59% block is not the cost of standing up context — it is paid per turn, putting 93% of the bill in reach of anything that stops the agent searching. Section 01 measured the same thing from the other end and we misread it: raising the turn ceiling bought 50% more turns at 1.37× the cost. Agents grind to any ceiling.

Naming the edge is what makes it cheap

A ranked file list tells an agent where to look. Naming the edge tells it what to look for — and lets it skip a file instead of opening it to discover it was irrelevant. Depth two is in the graph and deliberately left there: on a 33,000-node repository it reaches 87 files, and returning them costs more than the reading it replaces.

The condition that decides it

Comment volume and acceptance rate must hold. A graph that cuts tokens by making the agent incurious is a regression dressed as a saving, and it would pass a cost dashboard cleanly — so acceptance, not spend, is the number this ships against.

The one we didn't ship

A nudge that pushes agents to re-sweep for a second defect looked like a clean +30% recall win. One run, high variance, +49% latency. It is still off. A loop that ships every apparent win is a slower way of guessing.