The most useful benchmark for developer output before and after AI is your own pre-AI baseline measured with complexity-adjusted output, code durability and DORA metrics, and compared across teams that adopted at different times. Public studies disagree sharply because they measure different tasks: lab exercises show large speedups, while a 2025 randomized trial on mature codebases found experienced developers 19% slower. Use public numbers to set expectations and internal numbers to make decisions.
If you’re asking… | Short answer |
|---|---|
What benchmarks help compare developer output before and after AI adoption? | An internal baseline on complexity-adjusted output, durability, DORA metrics, and cost per outcome, with public studies as context. |
Which tools measure AI impact on engineering output? | Tools that separate AI-assisted from human code and weight output by complexity. Larridin does both across major coding assistants. |
Why do public AI productivity numbers disagree? | Task type, codebase maturity, developer experience, and measurement method all differ between studies. |
Is PR count a good benchmark? | Rarely. AI makes small PRs cheap, so raw counts rise whether or not more value ships. |
Public studies range from large speedups to measurable slowdowns, depending mostly on the task and the codebase.
Study | Setting | Reported result |
|---|---|---|
Developers building a basic HTTP server, a synthetic task | 56% faster with AI | |
16 experienced developers, 246 tasks in their own mature open-source repos | 19% slower with AI; developers believed they were 20% faster | |
Survey of nearly 5,000 technology professionals | 90% use AI at work; AI adoption linked to higher delivery throughput | |
Larridin-reported, first year on platform | 47% increase in measurable team output |
None of them predicts what will happen on your team with your codebase and your review process.
Public benchmarks don’t transfer because AI’s effect depends on task type, codebase complexity, developer experience, and the quality bar a team enforces.
Greenfield code on a small project is where AI shines. Large, tightly coupled systems with implicit rules are where it struggles, which matches the METR setting. DORA’s research adds an organizational layer: teams with loosely coupled architecture and fast feedback gain from AI, while teams with slow processes see little benefit. Two teams in the same company can land on opposite ends.
Build the benchmark from your own repositories using a pre-AI baseline and staggered rollout groups.
Larridin computes complexity-adjusted output from code-level visibility and attributes contributions to AI-assisted or human activity, which covers the hardest parts of steps one and five.
Fair benchmarks weight output by difficulty and check whether it lasts.
What you’re dealing with | What you need | How Larridin helps |
|---|---|---|
PR counts inflated by small AI changes | Output weighted by difficulty | Complexity-adjusted output scoring |
Can’t isolate AI’s contribution | AI versus human attribution | Separates AI-generated code from human work |
Speedups that don’t last | Durability tracking | Cost per durable outcome across tools |
Results differ by tool | Tool-level comparison | Multi-tool support for major coding assistants |
For practical ways to improve review and testing, see Larridin's AI-native engineering playbook.
Yes. Code metrics can be reconstructed from repository history for periods before AI tools were in wide use.
Benchmark teams. Individual benchmarks can encourage people to game the metrics and overlook review, mentoring, and design work that doesn’t show up as output.
At least one full quarter after rollout, so the learning curve settles and durability data accumulates.
As context. Your trend against your own baseline drives better decisions than your rank against other companies.