Skip to content

The Review Bottleneck: AI Made Writing Code Cheap. It Made Trusting Code Expensive.

Blog  ✺  AI  ✺  Strategy  ✺  Engineering

Your engineering team is merging twice as many pull requests as a year ago. The AI coding tools are visibly working. And your releases are not arriving one day earlier. Both facts are well documented. The industry has settled on a comforting explanation: the work moved. It deserves a harder look.

The explanation is not wrong. Writing code got faster, and the effort shifted to reviewing and verifying what was written. But "the work moved" is doing a lot of quiet work as an excuse, and the data behind it says something less comfortable than the phrase suggests.

TLDR: The research nonprofit METR ran a randomized trial with 16 experienced open-source developers across 246 real tasks: with AI tools they took 19% longer, while predicting AI would make them 24% faster, and still believing afterwards it had made them 20% faster. Telemetry from Faros AI covering 10,000 developers in 1,255 teams shows why the feeling and the number diverge: engineers merge 98% more pull requests with AI, code review time rises 91%, and the standard measures of software delivery, how often you ship and how long a change takes to reach production, remain flat. Only 5% of enterprises see measurable returns. The work did move, from writing code to trusting it. But moving a bottleneck is not the same as removing one, and organizations that keep measuring the writing are congratulating the wrong machine.

The study nobody wanted

Start with the result that made the industry flinch. In mid-2025, METR did what almost nobody in this debate does: a controlled experiment. Sixteen experienced developers, working on large, mature open-source codebases they knew well, were randomly assigned real tasks with and without AI tools. Not a survey. Not a demo. Time, measured.

With AI, they were 19% slower.

The number that matters more is the perception gap around it. Before the study, the developers predicted AI would speed them up by 24%. After the study, having lived through the slowdown, they still believed it had sped them up by 20%. The measured reality was the opposite. Writing code with AI feels fast, because the model produces plausible material instantly. Verifying that material, understanding it, testing it, discovering where it subtly does not fit a mature codebase, is slow, and apparently does not register as "the AI costing me time."

Be careful with the scope, because the authors were. Sixteen developers, expert ones, on codebases they knew intimately. The study does not show AI slows everyone down, and the same body of research shows gains of 20 to 56% on well-defined, greenfield tasks. That split is exactly the jagged frontier described on this blog before: AI accelerates the easy, self-contained work and struggles precisely where your senior people spend their days. But the perception gap generalizes, and it should worry every leader whose AI business case rests on asking people whether they feel faster. The people closest to the tool, with the strongest incentive to know, misjudged the direction of the effect. Not the size. The direction.

Feeling faster, measured slower

Effect of AI tools on completion speed, 16 experienced developers on 246 real tasks (METR, randomized trial, 2025)

What developers believed What the trial measured

Data: METR, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity", July 2025.

View as table
MeasureEffect on speed
Predicted before the trial+24%
Perceived after the trial+20%
Measured by the trial−19%

What the telemetry shows

One study of sixteen people is a data point. What makes it credible is that the pattern repeats at scale, in production data rather than in a lab.

Faros AI analyzed the delivery telemetry of 10,000 developers across 1,255 enterprise teams. The activity numbers look like a triumph: 98% more pull requests merged, 21% more tasks completed, three-quarters of engineers using AI assistants. Then the pipeline: code review time up 91%. And at the end of it, the measures that actually reach the business, deployment frequency, lead time from commit to production, failure rates, largely unchanged. More code is entering the pipeline. Software is not leaving it faster. Faros counts 5% of enterprises seeing measurable returns, an almost eerie echo of MIT's finding that 5% of AI initiatives produce profit-and-loss impact.

Any production engineer will recognize this shape from the factory floor. Speed up one machine at the front of the line, and if the narrowest machine sits downstream, the factory does not produce one unit more. What grows is the pile of half-finished inventory in front of the constraint. In software, that pile is open pull requests, and the constraint is a senior engineer reading code at the same human speed as in 2022, now with twice as much of it to read and a subtler failure mode: the code in front of her looks right, compiles, and was written by no one she can ask.

More code in, nothing more out

Change with AI coding assistants, telemetry from 10,000 developers in 1,255 enterprise teams (Faros AI)

Code produced Review load Delivery to production

Data: Faros AI, "The AI Productivity Paradox". Delivery = deployment frequency, lead time, failure rate, recovery time (largely unchanged).

View as table
MeasureChange
Pull requests merged+98%
Code review time+91%
Tasks completed+21%
Delivery to production~0% (unchanged)

"The work moved" is not absolution

Here is where the emerging consensus deserves pushback. "The work moved" is true as a description and dangerous as a conclusion, because it is being used to explain away flat delivery numbers as a temporary adjustment. Give it a year, the argument goes, and review practices will catch up.

aerial photography of concrete roads
Photo by Denys Nevozhai / Unsplash

Three problems with that.

First, verification was never a side activity that can be compressed at will. Reading code has always dominated writing it; the industry's own folklore says code is read an order of magnitude more often than it is written. AI did not create the review bottleneck. It revealed it, by removing the writing constraint that used to hide it. What is new is the ratio: the cheap half of the job got dramatically cheaper, so the expensive half now dominates, and the expensive half is precisely the part that requires your scarcest people.

Second, the quality of what enters review has changed character. A May 2026 working paper calls this the productivity-reliability paradox and locates the cause not in model capability but in specification discipline: AI generates plausible code from underspecified intent, and every ambiguity the author did not resolve up front becomes a defect the reviewer must catch downstream. The paper's summary line is worth pinning to a wall: specification discipline, not model capability, is the binding constraint. Buying a better model does not fix a vague ticket.

Third, and most practically: while organizations wait for review to "catch up," they are measuring the wrong end of the pipeline. Pull request counts are rising, so dashboards glow green, and the activity is mistaken for impact in exactly the way this blog has documented across the wider AI program: inputs celebrated because outputs are dark. An engineering leader who reports the 98% and not the flat lead time is not lying. He is reading the speedometer of the machine that was never the constraint.

Managing the constraint instead of feeding it

The playbook for this is forty years old, and every operations engineer in Switzerland has read it. Goldratt's theory of constraints says: find the bottleneck, exploit it, subordinate everything else to it, and only then invest in widening it. Applied here, it inverts most current AI-tooling decisions, and it yields concrete moves at each step.

Exploiting the constraint means treating senior review capacity as the scarcest asset in the system and spending it deliberately. Two practices do most of the work. First, risk-tiered review: a configuration change, a test, a documentation fix does not deserve the same depth of human reading as a change to payment logic or access control. Define the tiers explicitly, route the low-risk tier through a lightweight automated path, and reserve senior eyes for the changes that can actually hurt you. Second, small batches: review effort grows faster than linearly with change size, so a two-thousand-line AI-generated pull request is not reviewed, it is skimmed and approved. Cap the size of what may enter review. A model can generate two thousand lines in one prompt; nothing obliges you to submit them as one change.

Subordinating everything else to the constraint is the uncomfortable step, because it means deliberately slowing the fast machine. Set a limit on open pull requests per team, and when the limit is reached, the next contribution is a review, not more code. Make authorship carry weight again: whoever submits an AI-generated change explains it, has run it, and can answer for every line, which converts the first and cheapest review, the author's own, from a formality into a filter. And label AI-generated changes as such, so the reviewer knows to look for the failure mode this code actually has, the plausible-but-wrong, rather than the failure modes human code has.

Widening the constraint comes last, and only after the first two steps, because capacity added to an unmanaged bottleneck is simply consumed. Here the investments are unglamorous. Test coverage as a precondition: AI-assisted changes land only where a failing test would catch a regression, which turns part of the verification burden into something machines carry. Specification before generation, the discipline the working paper identifies as the binding constraint: an hour spent writing down what the code is supposed to do is bought back at the review stage, the one stage you cannot scale by buying licenses. And yes, AI on the review side too, as a pre-filter that clears away the shallow defects before a human reads for design and context. Used that way it widens the bottleneck. Used as the final gate, AI reviewing AI, it removes the bottleneck by removing the verification, which is how flat delivery metrics eventually become interesting incident reports.

What none of this means is measuring success by generated volume. If the delivery measures, ship frequency, lead time, failure rate, do not move, the AI program has produced inventory, not throughput.

green trees near gray wall
Photo by Jas Min / Unsplash

The pile in front of the machine

Goldratt's novel about the factory made one point stick for a generation of engineers: an hour saved at a non-bottleneck is a mirage. The plant in the book drowned in efficiency, every machine locally optimized, mountains of work-in-progress, nothing shipping. It recovered when management stopped celebrating busy machines and started staring at the one that was actually full.

The AI coding wave is the same plant, rebuilt in software. The writing machine just got five times faster, its dashboard is spectacular, and the pile in front of review grows. The work moved; the discipline has to move with it, from generating more to verifying better, from activity metrics to delivery metrics, from feeling faster to being faster. Sixteen developers in a study felt 20% faster while being 19% slower. Organizations do not get to skip that lesson at scale.

A faster machine in front of the bottleneck does not make the factory faster. It makes the pile bigger.

Mehr

Governance Debt: Brussels Moved the Deadline. It Did Not Move the Work.

The Platform Trap: Everyone Is Building an AI Platform. Almost Nobody Is Building One People Want to Use.