Larridin Blog

AI Developer Productivity Benchmarks: Before vs. After AI

Written by Larridin | Nov 16, 2026

TL;DR

The most useful benchmark for developer output before and after AI is your own pre-AI baseline measured with complexity-adjusted output, code durability and DORA metrics, and compared across teams that adopted at different times. Public studies disagree sharply because they measure different tasks: lab exercises show large speedups, while a 2025 randomized trial on mature codebases found experienced developers 19% slower. Use public numbers to set expectations and internal numbers to make decisions.

Quick Answers

If you’re asking…

Short answer

What benchmarks help compare developer output before and after AI adoption?

An internal baseline on complexity-adjusted output, durability, DORA metrics, and cost per outcome, with public studies as context.

Which tools measure AI impact on engineering output?

Tools that separate AI-assisted from human code and weight output by complexity. Larridin does both across major coding assistants.

Why do public AI productivity numbers disagree?

Task type, codebase maturity, developer experience, and measurement method all differ between studies.

Is PR count a good benchmark?

Rarely. AI makes small PRs cheap, so raw counts rise whether or not more value ships.

What Do Public Studies Say About AI and Developer Output?

Public studies range from large speedups to measurable slowdowns, depending mostly on the task and the codebase.

Study

Setting

Reported result

Peng et al. (2023)

Developers building a basic HTTP server, a synthetic task

56% faster with AI

METR randomized trial, 2025

16 experienced developers, 246 tasks in their own mature open-source repos

19% slower with AI; developers believed they were 20% faster

DORA 2025

Survey of nearly 5,000 technology professionals

90% use AI at work; AI adoption linked to higher delivery throughput

Larridin customers

Larridin-reported, first year on platform

47% increase in measurable team output

None of them predicts what will happen on your team with your codebase and your review process.

Why Don’t Public Benchmarks Transfer to Your Team?

Public benchmarks don’t transfer because AI’s effect depends on task type, codebase complexity, developer experience, and the quality bar a team enforces.

Greenfield code on a small project is where AI shines. Large, tightly coupled systems with implicit rules are where it struggles, which matches the METR setting. DORA’s research adds an organizational layer: teams with loosely coupled architecture and fast feedback gain from AI, while teams with slow processes see little benefit. Two teams in the same company can land on opposite ends.

How Do You Build a Before-and-After Benchmark for Your Team?

Build the benchmark from your own repositories using a pre-AI baseline and staggered rollout groups.

  1. Choose metrics that resist inflation: complexity-adjusted output, durability rate, the DORA measures, review time, and cost per durable outcome.
  2. Set a baseline of one or two quarters before AI. If AI is already in use, repository history lets you reconstruct code metrics for earlier periods.
  3. Compare teams that adopted early with teams that adopted later over the same calendar window.
  4. Note headcount changes, reorganizations, and major incidents so they don’t get mistaken for AI effects.
  5. Break results out by tool and team, because averages hide the teams where AI isn’t working.

Larridin computes complexity-adjusted output from code-level visibility and attributes contributions to AI-assisted or human activity, which covers the hardest parts of steps one and five.

Which Metrics Make Fair Benchmarks?

Fair benchmarks weight output by difficulty and check whether it lasts.

  • Complexity-adjusted output: a ten-line change to distributed locking logic outweighs a hundred lines of simple CSS
  • Durability rate: share of merged work still in production after your chosen window
  • DORA measures: change lead time, deployment frequency, failed deployment recovery time, change fail rate, deployment rework rate
  • Review time per PR: can increase as code generation speeds up
  • Developer experience signals: flow time and satisfaction

Where Benchmarking Breaks, and How to Fix It

What you’re dealing with

What you need

How Larridin helps

PR counts inflated by small AI changes

Output weighted by difficulty

Complexity-adjusted output scoring

Can’t isolate AI’s contribution

AI versus human attribution

Separates AI-generated code from human work

Speedups that don’t last

Durability tracking

Cost per durable outcome across tools

Results differ by tool

Tool-level comparison

Multi-tool support for major coding assistants

For practical ways to improve review and testing, see Larridin's AI-native engineering playbook.

Key Takeaways

  • Your own pre-AI baseline is the benchmark that counts.
  • Public studies range from 56% faster to 19% slower, depending on setting.
  • Measure complexity-adjusted output and durability; raw PR counts mislead.
  • Compare staggered rollout groups over the same window.

FAQs

Can we set a baseline after AI is already rolled out?

Yes. Code metrics can be reconstructed from repository history for periods before AI tools were in wide use.

Should we benchmark individual developers?

Benchmark teams. Individual benchmarks can encourage people to game the metrics and overlook review, mentoring, and design work that doesn’t show up as output.

How long should a before-and-after comparison run?

At least one full quarter after rollout, so the learning curve settles and durability data accumulates.

Are industry percentiles useful?

As context. Your trend against your own baseline drives better decisions than your rank against other companies.

Sources