AI can produce more pull requests, but that doesn't necessarily mean more valuable work. Spark data shows why complexity-adjusted velocity tells a different story than PR counts.
TL;DR
The Short Answer
PR counts measure volume, not the complexity of work delivered. In one Larridin customer, AI-assisted work accounted for 66% of merged PRs but 56% of complexity-adjusted velocity (CAV), or about one-third less output per PR than human-only work. In a separate Spark, CAV per engineer grew 1% for AI-assisted work versus 16% for human-only work. Measure both volume and complexity-adjusted output to understand AI's contribution.
Key Wins
- See why AI can win on PR volume while each PR carries less weight
- Learn what complexity-adjusted velocity measures and why it matters
- Report engineering output by complexity as well as count
What’s a Spark Insight?
A Spark Insight is a data-backed finding Larridin surfaces automatically from how your teams actually use AI, flagging the patterns worth a leadership conversation.
Who This Matters To
CTO · VP Eng · Eng managers
Quick Answers
| If you’re asking… | The answer | Go deeper |
|---|---|---|
| Do PR counts measure AI coding productivity? | Not well. In one org, AI-assisted work was 66% of merged PRs but only 56% of complexity-adjusted output. | The data |
| What is complexity-adjusted velocity? | A measure of engineering output that weights each change by its complexity, so a large, difficult change counts for more than a trivial one. | Definition |
| Do AI-assisted PRs contain less complexity-adjusted output? | In one org, each AI-assisted PR carried about a third less complexity-adjusted output than a human-only PR. That does not establish lower business value. | The data |
| How should I measure AI coding output? | Track complexity-adjusted velocity per engineer, split by AI-assisted vs. human-only work. | How to measure |
PR counts are easy to report and easy to inflate. AI makes that problem bigger: it can generate many small changes quickly. Leaders need a measure that rewards meaningful work, not just more of it.
What You’re Struggling With, and How to Fix It
| What you’re struggling with | What you actually need | How Larridin helps | Proof from Spark data |
|---|---|---|---|
| “PR counts are up, but we’re not shipping more.” | Output weighted by complexity | Measures complexity-adjusted velocity (CAV) for every change | One team: AI 66% of PRs, 56% of CAV |
| “I need to report AI productivity honestly.” | A complexity-adjusted output metric split by code origin | Breaks CAV down by code origin | AI-assisted PRs carried about 1/3 less complexity-adjusted output per PR |
| “AI should be making engineers more productive.” | Per-engineer trends | Tracks CAV per engineer over time | Separate Spark: CAV/engineer +1% (AI) vs. +16% (human) |
| “Review time is climbing.” | A link between volume and review load | Shows review time alongside PR volume by origin | One org: AI-assisted review time +79% vs. +11% for human PRs |
What This Means for You
If you’re a CTO
The question you’re asked: “How much is AI increasing engineering output?”
What to do: Report complexity-adjusted velocity, not PR counts. Show the split by AI-assisted vs. human-only work.
For more on connecting AI usage to engineering outcomes, see AI Monitoring for Engineering Leaders.
If you’re a VP of Engineering
The question you’re asked: “Are we using AI on the right work?”
What to do: If AI-assisted PRs carry less complexity-adjusted output each, examine the types of work AI is doing before changing task assignments.
If you manage an engineering team
The question you’re asked: “Why does my team look busier but not faster?”
What to do: Check whether PR volume is rising while value per PR falls. Set goals on outcomes, not counts.
What the Sparks Data Shows
- 66% vs. 56%: AI-assisted share of merged PRs vs. share of complexity-adjusted velocity, across 11 engineers over 12 weeks in one customer organization.
- +1% vs. +16%: Growth in CAV per engineer for AI-assisted vs. human-only work, reported in a separate Spark.
- +79% vs. +11%: Increase in PR review time for AI-assisted vs. human-only PRs in the Spark above.
Why Do PR Counts Overstate AI’s Contribution?
Because PR counts treat every merged change equally, regardless of its complexity.
A one-line fix and a major feature both count as one PR. When AI-assisted PRs are smaller, PR counts can overstate AI's share of complexity-adjusted output.
In one team of 11 engineers over 12 weeks, AI-assisted work made up 66% of merged PRs but 56% of complexity-adjusted velocity. Each AI-assisted PR carried about a third less complexity-adjusted output.
Spark callout: In a separate Spark, CAV per engineer grew 1% for AI-assisted work and 16% for human-only work. That's another reason to track output per engineer, not just how many PRs involve AI.
Myth: More PRs means more output.
Reality: In one team, AI-assisted work accounted for 66% of merged PRs but 56% of complexity-adjusted output.
What Is Complexity-Adjusted Velocity?
A measure of engineering output that weights each change by its difficulty.
Complexity-adjusted velocity (CAV) accounts for the size and complexity of each change, so meaningful work counts for more than trivial work. It gives a fairer comparison between AI-assisted and human-only output, and between teams with different kinds of work.
Complexity-adjusted output is not a direct measure of customer value, revenue, or business impact. A small change can be valuable; pair CAV with quality and business outcomes.
What Should Engineering Leaders Do With This?
Compare PR volume with complexity-adjusted output, then investigate the gaps.
If AI-assisted PRs carry less complexity-adjusted output each, examine the work being assigned to AI. If review time is also rising, as it did by 79% in one Spark, check whether review is offsetting some of the speed gains. Measure both in your own organization before changing priorities.
How to Measure This in Your Own Org
- Tag work as AI-assisted or human-only. Use tool telemetry, commit metadata, or PR labels.
- Weight each change by complexity. Move from counts to complexity-adjusted velocity.
- Compare PR share with complexity-adjusted output share. A gap shows where PR volume overstates output.
- Track CAV per engineer over time. Larridin's Engineering Performance view compares AI-assisted and human-only Engineering Output, including trends by engineer.
Frequently Asked Questions
Is complexity-adjusted velocity the same as story points?
No. Story points are estimates made before the work. CAV is measured from the completed changes themselves.
Should I stop tracking PR counts?
Keep them as an activity signal, but don’t use them as a productivity measure on their own, especially when comparing AI-assisted and human work.
Does this mean AI isn’t making engineers more productive?
Not necessarily. PR counts alone cannot answer the question. Pair complexity-adjusted output with quality, delivery outcomes, and business results in your own organization.
Spark a Discussion About What Your Engineering Output Is Really Worth
See complexity-adjusted velocity for AI-assisted and human work in your org, and bring it to your next engineering review.