AI can produce more pull requests, but that doesn't necessarily mean more valuable work. Spark data shows why complexity-adjusted velocity tells a different story than PR counts.
The Short Answer
PR counts measure volume, not the complexity of work delivered. In one Larridin customer, AI-assisted work accounted for 66% of merged PRs but 56% of complexity-adjusted velocity (CAV), or about one-third less output per PR than human-only work. In a separate Spark, CAV per engineer grew 1% for AI-assisted work versus 16% for human-only work. Measure both volume and complexity-adjusted output to understand AI's contribution.
Key Wins
What’s a Spark Insight?
A Spark Insight is a data-backed finding Larridin surfaces automatically from how your teams actually use AI, flagging the patterns worth a leadership conversation.
Who This Matters To
CTO · VP Eng · Eng managers
| If you’re asking… | The answer | Go deeper |
|---|---|---|
| Do PR counts measure AI coding productivity? | Not well. In one org, AI-assisted work was 66% of merged PRs but only 56% of complexity-adjusted output. | The data |
| What is complexity-adjusted velocity? | A measure of engineering output that weights each change by its complexity, so a large, difficult change counts for more than a trivial one. | Definition |
| Do AI-assisted PRs contain less complexity-adjusted output? | In one org, each AI-assisted PR carried about a third less complexity-adjusted output than a human-only PR. That does not establish lower business value. | The data |
| How should I measure AI coding output? | Track complexity-adjusted velocity per engineer, split by AI-assisted vs. human-only work. | How to measure |
PR counts are easy to report and easy to inflate. AI makes that problem bigger: it can generate many small changes quickly. Leaders need a measure that rewards meaningful work, not just more of it.
| What you’re struggling with | What you actually need | How Larridin helps | Proof from Spark data |
|---|---|---|---|
| “PR counts are up, but we’re not shipping more.” | Output weighted by complexity | Measures complexity-adjusted velocity (CAV) for every change | One team: AI 66% of PRs, 56% of CAV |
| “I need to report AI productivity honestly.” | A complexity-adjusted output metric split by code origin | Breaks CAV down by code origin | AI-assisted PRs carried about 1/3 less complexity-adjusted output per PR |
| “AI should be making engineers more productive.” | Per-engineer trends | Tracks CAV per engineer over time | Separate Spark: CAV/engineer +1% (AI) vs. +16% (human) |
| “Review time is climbing.” | A link between volume and review load | Shows review time alongside PR volume by origin | One org: AI-assisted review time +79% vs. +11% for human PRs |
The question you’re asked: “How much is AI increasing engineering output?”
What to do: Report complexity-adjusted velocity, not PR counts. Show the split by AI-assisted vs. human-only work.
For more on connecting AI usage to engineering outcomes, see AI Monitoring for Engineering Leaders.
The question you’re asked: “Are we using AI on the right work?”
What to do: If AI-assisted PRs carry less complexity-adjusted output each, examine the types of work AI is doing before changing task assignments.
The question you’re asked: “Why does my team look busier but not faster?”
What to do: Check whether PR volume is rising while value per PR falls. Set goals on outcomes, not counts.
Because PR counts treat every merged change equally, regardless of its complexity.
A one-line fix and a major feature both count as one PR. When AI-assisted PRs are smaller, PR counts can overstate AI's share of complexity-adjusted output.
In one team of 11 engineers over 12 weeks, AI-assisted work made up 66% of merged PRs but 56% of complexity-adjusted velocity. Each AI-assisted PR carried about a third less complexity-adjusted output.
Spark callout: In a separate Spark, CAV per engineer grew 1% for AI-assisted work and 16% for human-only work. That's another reason to track output per engineer, not just how many PRs involve AI.
Myth: More PRs means more output.
Reality: In one team, AI-assisted work accounted for 66% of merged PRs but 56% of complexity-adjusted output.
A measure of engineering output that weights each change by its difficulty.
Complexity-adjusted velocity (CAV) accounts for the size and complexity of each change, so meaningful work counts for more than trivial work. It gives a fairer comparison between AI-assisted and human-only output, and between teams with different kinds of work.
Complexity-adjusted output is not a direct measure of customer value, revenue, or business impact. A small change can be valuable; pair CAV with quality and business outcomes.
Compare PR volume with complexity-adjusted output, then investigate the gaps.
If AI-assisted PRs carry less complexity-adjusted output each, examine the work being assigned to AI. If review time is also rising, as it did by 79% in one Spark, check whether review is offsetting some of the speed gains. Measure both in your own organization before changing priorities.
No. Story points are estimates made before the work. CAV is measured from the completed changes themselves.
Keep them as an activity signal, but don’t use them as a productivity measure on their own, especially when comparing AI-assisted and human work.
Not necessarily. PR counts alone cannot answer the question. Pair complexity-adjusted output with quality, delivery outcomes, and business results in your own organization.
See complexity-adjusted velocity for AI-assisted and human work in your org, and bring it to your next engineering review.