The Developer AI Impact Framework: Measuring Productivity When AI Writes the Code

TL;DR

• AI coding tools can genuinely accelerate engineering teams -- but without the right measurement, they can also slow you down while making it feel like you're going faster. More code, more PRs, more activity on the dashboard, while rework eats the gains and costs compound.
• The Developer AI Impact Framework measures what actually matters in AI-native engineering across five pillars: AI Adoption, AI Code Share, Complexity-Adjusted Throughput (CAT), Code Quality (turnover rate), and Cost & ROI.
• AI tool costs now range from trivial to substantial -- $20-60/month for inline completion, $200-$2,000+/month for agentic tools. As costs scale, the margin for error shrinks, and rigorous measurement shifts from nice-to-have to financial governance.
• Most organizations measure adoption and stop. The highest-performing engineering teams measure across all five pillars, connecting tool usage to business outcomes -- and they achieve 2-4x the ROI of teams that measure only activity.

The Problem: Traditional Frameworks Weren't Built for This -- and the Stakes Are Higher Than Ever

DORA and SPACE are foundational frameworks. They shaped how an entire generation of engineering leaders think about productivity, and for good reason. When those frameworks were developed, every line of code in your codebase was written by a human. Every commit reflected human effort. Every PR represented a human decision.

That assumption no longer holds.

In 2026, AI coding tools generate 30-70% of committed code in high-adoption organizations. GitHub Copilot, Cursor, and Claude Code don't just autocomplete variable names -- they generate entire functions, test suites, and boilerplate modules. This is a structural shift in how code gets produced, and it requires a structural shift in how productivity gets measured.

What makes this urgent is not just that traditional frameworks produce inaccurate numbers. It is that the outcome variance in AI-assisted engineering is enormous. Two teams can adopt the same tools, spend similar budgets, and get radically different results:

• One team achieves genuine 1.5-2x acceleration on meaningful work, compounds the gains quarter over quarter, and builds a durable competitive advantage.
• Another team sees a surge in activity -- more PRs, more commits, more lines of code -- but code churn doubles, rework eats the velocity gains, and rising token costs erode the ROI. The dashboard says faster. The codebase says otherwise.

Both teams look identical on traditional metrics. PRs are up. Deployment frequency is up. Activity is through the roof. The difference between them is invisible to DORA, SPACE, or any volume-based measurement system.

"Feels faster, isn't" is the most expensive failure mode in AI-assisted engineering. Code churn has nearly doubled since AI coding adoption went mainstream (3.3% to 5.7-7.1%, per GitClear). AI tool costs now range from trivial ($20-60/month for inline completion) to substantial ($200-$2,000+/month per engineer for agentic tools). When the investment is this large and the outcome variance is this wide, measurement is not a reporting exercise -- it is the mechanism that determines which outcome you get.

Where DORA Metrics Break

DORA's four metrics -- Deployment Frequency, Lead Time for Changes, Mean Time to Recovery, and Change Failure Rate -- were designed to measure software delivery performance. They served us well. But three of the four are now distorted by AI-generated code:

• Deployment Frequency increases when AI makes it trivially easy to ship small changes. More deployments do not necessarily mean more value delivered.
• Lead Time for Changes drops when AI generates code in seconds. A shorter lead time used to signal a healthy CI/CD pipeline. Now it may simply mean a developer accepted an AI suggestion without thorough review.
• Change Failure Rate may temporarily appear stable, but rising code churn suggests that AI-generated code is being quietly rewritten or reverted within days of being merged -- a quality problem that Change Failure Rate does not capture.

Only MTTR remains relatively unaffected, because recovery from incidents still depends primarily on human judgment and system architecture.

Where the SPACE Framework Falls Short

The SPACE framework introduced a valuable multi-dimensional approach: Satisfaction, Performance, Activity, Communication, and Efficiency. Its insight -- that productivity is multidimensional and that developer experience matters -- remains correct.

But SPACE's Activity dimension (commits, PRs, lines of code) directly rewards volume. When AI can produce ten PRs in the time a human writes one, Activity metrics become noise. SPACE's limitations in the AI era are not a failure of the framework's principles. They are a consequence of building on assumptions that no longer hold.

The Data Confirms the Problem

GitClear's analysis of over one billion lines of code found that code churn -- the percentage of code that is rewritten or deleted within weeks of being committed -- has risen from a pre-AI baseline of 3.3% in 2021 to between 5.7% and 7.1% by 2024¹. This is not a marginal increase. It represents a near-doubling of wasted engineering effort, and it aligns precisely with the timeline of widespread AI coding tool adoption.

The implication is straightforward: teams are producing more code, but a growing share of that code does not survive. Any productivity framework that counts output without measuring durability is producing misleading results.

The AI Impact Hierarchy

Before measuring productivity, engineering leaders need a model for understanding what they are actually trying to achieve with AI tools. The AI Impact Hierarchy describes five ascending levels of AI impact, each building on the one below:

Most organizations measure Level 1 -- adoption -- and declare success. They know that 60% of their developers have used Copilot this month. What they do not know is whether those developers are using it for trivial autocompletions or complex feature work (Level 2), whether it is genuinely accelerating delivery (Level 3), whether the code it produces survives beyond the first sprint (Level 4), or whether the investment is generating a positive return (Level 5).

The Developer AI Impact Framework is designed to measure across all five levels. Each pillar corresponds to one or more levels of the hierarchy, ensuring that organizations do not confuse adoption theater with actual productivity gains. For a deeper exploration of this maturity model, see What Is the AI Impact Hierarchy?.

The 5 Pillars -- Deep Dive

Pillar 1: AI Adoption

What it measures: Whether developers are actually using AI coding tools, how frequently, and how usage distributes across the team.

Adoption is the prerequisite. If developers are not using AI tools, nothing else in this framework matters. But adoption alone tells you almost nothing about productivity. A team with 70% weekly active users might be seeing transformative gains, or it might be seeing 70% of developers accepting low-quality autocomplete suggestions.

Key Metrics:

• AI Active User Rate (DAU / WAU / MAU): The percentage of developers who actively use AI coding tools on a daily, weekly, or monthly basis. WAU is the primary tracking metric because it smooths daily variability while remaining responsive to trends.
• Tool Adoption by Type: Usage broken down by interaction mode -- inline completions (Copilot tab completions, Cursor autocomplete), chat-based assistance (Copilot Chat, Claude), and agentic workflows (Claude Code, Cursor Composer). Deeper modes indicate deeper integration.
• Adoption Distribution: The split between power users (daily, multi-mode), casual users (weekly, single-mode), and non-users. A healthy distribution skews toward power users over time.

Red Flags:

• WAU below 30% after initial rollout period (4+ weeks)
• Adoption flat or declining for 4+ consecutive weeks
• Usage concentrated in a small group of power users with no growth in the middle

Benchmarks:

Adoption measurement is the entry point, but it is not the destination. Once you know developers are using AI tools, the next question is what percentage of your codebase they are actually producing with those tools. That is Pillar 2.

Pillar 2: AI Code Share

What it measures: The percentage of committed code that was generated or substantially assisted by AI tools, measured at the line, commit, and pull request level.

AI Code Share answers a question that adoption metrics cannot: how much of your actual codebase is AI-produced? A team can have 70% WAU but only 10% AI-assisted code (indicating shallow usage), or 40% WAU but 50% AI-assisted code (indicating a smaller group of deeply effective power users).

Key Metrics:

• AI-Assisted PRs %: Percentage of pull requests containing at least one AI-generated or AI-assisted code segment.
• AI-Assisted Lines %: Percentage of committed lines of code that originated from AI suggestions, completions, or generations.
• AI-Assisted Commits %: Percentage of commits that include AI-generated content, measured via tool telemetry and attribution signals.

Red Flags:

• High AI Code Share (>40%) with no quality metrics in place -- this is a blind spot, not an achievement
• AI Code Share rising rapidly without corresponding improvement in complexity-adjusted throughput
• AI Code Share concentrated in boilerplate and test stubs rather than feature code

Benchmarks:

The distinction between AI-assisted lines and AI-assisted PRs matters. A PR might contain 200 AI-generated boilerplate lines and 20 carefully crafted human lines. The PR is AI-assisted, but the high-value work may still be human. This is why AI Code Share must be interpreted alongside Pillar 3 (velocity) and Pillar 4 (quality), never in isolation.

Pillar 3: Velocity (Complexity-Adjusted Throughput)

What it measures: Engineering output weighted by the complexity of the work delivered, segmented by AI-assisted versus human-only contributions.

Raw velocity metrics -- PRs merged, commits shipped, lines of code written -- are now unreliable proxies for engineering output. When AI can generate a 500-line PR in minutes, counting PRs tells you how prolific your AI tools are, not how productive your engineers are.

Complexity-Adjusted Throughput (CAT) addresses this by weighting each unit of work according to its complexity:

CAT per engineer per week becomes the primary velocity metric. It reflects the value of work delivered, not just the volume.

Key Metrics:

• CAT per Engineer (weekly): The sum of complexity-weighted points delivered per engineer per week, calculated across all merged PRs.
• Delivery Volume (AI vs Human): Total CAT points broken down by AI-assisted and human-only work, revealing whether AI is being applied to complex work or only to easy tasks.
• Cycle Time (commit to deploy): Time from first commit to production deployment, tracked separately for AI-assisted and human-only PRs.

Red Flags:

• CAT flat or declining while raw PR count increases -- AI is inflating volume without delivering real output
• AI-assisted work concentrated in Easy (1pt) complexity -- AI tools are being used for low-value tasks only
• Cycle time for AI-assisted PRs longer than human-only PRs -- review bottlenecks are negating AI speed gains

Benchmarks:

A team averaging 8 CAT points per engineer per week with a rising trend is performing at or above industry average. A team at 14+ points is in the top quartile. But CAT only tells you about output speed. It says nothing about whether that output survives. That is where Pillar 4 comes in.

Pillar 4: Quality

What it measures: The durability of AI-generated code -- whether it persists in the codebase or gets rewritten, reverted, or deleted shortly after being merged.

Quality is the pillar that most organizations miss entirely. They measure how much code AI produces and how fast it ships, but they do not ask the critical follow-up question: does it stick?

Code Turnover Rate is the primary quality metric. It measures the percentage of committed code that is substantially rewritten or deleted within 30 or 90 days of being merged, tracked separately for AI-generated and human-written code.

Key Metrics:

• Code Turnover Rate (30-day, AI vs Human): The percentage of lines committed in a given period that are modified or deleted within 30 days, segmented by origin. This is the early warning metric.
• Code Turnover Rate (90-day, AI vs Human): The longer-window measure that captures deeper quality issues -- code that passes initial review but creates problems during subsequent development.
• Innovation Rate: The percentage of PRs and commits that ship new features versus bug fixes versus infrastructure/configuration work. See Innovation Rate: Features vs Fires for a deeper exploration.

Red Flags:

• AI code turnover more than 2x human code turnover -- AI is generating code that does not survive
• Overall 30-day turnover rate above 18% -- code churn is consuming engineering capacity
• Core logic AI turnover exceeding 25% -- AI-generated business logic is unreliable
• Innovation Rate below 40% -- engineers are spending more time fixing and maintaining than building

Benchmarks:

The code churn data from GitClear confirms a pattern that many engineering leaders intuitively suspect: AI tools are producing more code, but a significant portion of that code is not production-durable. This does not mean AI tools are failing. It means that quality measurement must be built into any productivity framework from day one, not added as an afterthought.

Pillar 5: Cost & ROI

What it measures: The financial return on AI tool investment, accounting for both the value generated and the hidden costs of code rework.

AI tool costs are no longer trivial. Seat-based licenses for inline completion tools (Copilot, Cursor) run $20-60 per engineer per month. But agentic AI tools -- Claude Code, Cursor with high-autonomy agents, custom LLM pipelines -- introduce usage-based token costs that range from $200 to $2,000+ per engineer per month depending on usage intensity. A 50-person team can easily spend $10,000-$50,000 per month on AI tooling, making rigorous ROI justification essential rather than optional.

Engineering leaders are under increasing pressure to justify this spend. Pillar 5 provides the framework for doing so honestly -- accounting not just for time saved but also for the real cost structure of modern AI tooling and the rework costs introduced by low-quality AI-generated code.

Key Metrics:

• Total AI Tool Cost per Engineer (monthly): The fully-loaded cost including seat licenses, token/usage costs, and implementation overhead (training, admin, integration). This is not just the license fee -- it is the actual spend.
• Time Saved Value: Hours saved per engineer per week, multiplied by loaded engineering cost (salary + benefits + overhead).
• Net ROI Multiplier: The comprehensive return calculation that accounts for rework.

ROI Formula:

Net ROI = (Productive Value of Time Saved - Rework Cost from Code Turnover) / Total AI Tool Cost

For a deeper guide on structuring this calculation, see How to Measure AI Coding Tool ROI.

Red Flags:

• ROI below 2x after 90 days of deployment -- tool investment is not generating sufficient return
• Token costs growing faster than output gains -- usage is scaling but value is not
• ROI calculation excludes rework costs or uses only seat license fees as the cost denominator -- the number looks good but is incomplete
• Time saved concentrated in low-value tasks -- engineers are faster at things that do not matter

Benchmarks:

Cost Structure Reality:

AI tool costs now fall into three tiers, and most teams use tools from more than one:

Engineers using agentic tools heavily can generate $500-$2,000/month in token costs alone. This is not a problem -- agentic tools deliver disproportionately more value on Medium and Hard work -- but it means the cost denominator in your ROI calculation must reflect actual spend, not list price.

Example Calculation:

Consider a 50-person engineering team using a mix of inline completion and agentic AI tools:

This is the honest math. At 1.6x, AI tools are generating positive returns -- but not the 10x that vendor marketing claims. And notice how sensitive the calculation is to its inputs: if your team saves 7 hours per week instead of 5, ROI jumps to 2.6x. If your rework rate drops from 15% to 8% through better prompt engineering and review standards, ROI climbs further. If token costs run higher than $400/month average because engineers are using agentic tools for low-value tasks, ROI drops below breakeven.

This sensitivity is exactly why Pillar 5 matters. The difference between a 1.6x and a 4x ROI is not the tool -- it is how well your team uses it, and whether you have the quality gates (Pillar 4) to prevent rework from eating the gains.

The key insight: organizations that skip quality measurement systematically overstate ROI by 20-40% because they undercount costs (ignoring token spend) and do not account for rework. Including both the real cost structure and the rework deduction is what separates credible ROI analysis from advocacy math. See AI Value Realization Score for how to distill this into a single executive metric.

The Qualitative Layer: Developer Experience Surveys

Telemetry shows what is happening. Surveys show why.

A team might show declining CAT scores, and telemetry alone can confirm the trend. But only a developer survey can reveal whether the root cause is tooling friction, poor prompt engineering skills, organizational resistance, or a change in the type of work being assigned. For a comprehensive guide to structuring these surveys, see Developer Experience Surveys for AI-Native Teams.

Five Survey Dimensions:

Survey Cadence:

Survey data pairs directly with telemetry. A developer who reports saving 8 hours per week but whose code turnover rate is 25% is generating volume, not value. A developer who reports saving only 2 hours but whose turnover rate is 3% is using AI precisely and effectively. Neither data source tells the full story alone.

The Composite Metric: AI Value Realization Score

Engineering leaders need a single number they can report to executives that answers the question: "Is our AI investment actually working?"

The AI Value Realization Score combines telemetry and survey data into one composite metric:

AI Value Realization Score = (WAU Rate x 0.2) + (AI-Assisted Code Rate x 0.2)
                           + (Perceived Time Savings Index x 0.3) + (Quality Score x 0.3)

Where:

Quality Score = 100 - (Turnover Rate x 2) - (Post-Acceptance Edit Rate x 1.5)

Component Breakdown:

Interpreting the Score:

For a complete guide to calculating and tracking this metric, see AI Value Realization Score: One Number for AI Engineering ROI.

How to Implement the Framework

The Developer AI Impact Framework is designed to be adopted incrementally. Do not attempt to measure all five pillars on day one.

Phase 1 (Weeks 1-2): Adoption Baseline Start with Pillar 1. Instrument your AI coding tools to capture WAU, DAU, and adoption distribution. This requires only API access to your tool admin dashboards (Cursor, Copilot, Claude Code) and a simple aggregation layer.

Phase 2 (Weeks 3-4): Code Share and Velocity Add Pillars 2 and 3. Connect your Git infrastructure to measure AI-assisted code percentage and begin classifying PRs by complexity for CAT calculation. This phase requires Git metadata analysis and a complexity classification model.

Phase 3 (Weeks 5-8): Quality Add Pillar 4. Begin tracking code turnover rate at 30-day windows, segmented by AI-generated versus human-written code. This requires historical Git data and a code attribution pipeline.

Phase 4 (Weeks 9-12): ROI and Surveys Add Pillar 5 and the qualitative survey layer. With all four preceding pillars instrumented, ROI calculation becomes a derivation. Launch the biweekly pulse survey to begin collecting perceived time savings and task fit data.

Phase 5 (Week 12+): Composite Scoring Once all five pillars and survey data are flowing, compute the AI Value Realization Score as your executive-level tracking metric.

Larridin operationalizes all five pillars using your existing tool stack -- Cursor, Claude Code, GitHub Copilot, and standard Git infrastructure -- without requiring developers to change their workflow or adopt new tools.

Frequently Asked Questions

How do you measure developer productivity when AI writes the code?

You measure the value of what gets delivered, not the volume of what gets produced. The Developer AI Impact Framework replaces raw output metrics (PRs, LOC, commits) with complexity-adjusted throughput (CAT), which weights each unit of work by its difficulty. A developer who ships one architecturally complex feature (8 CAT points) is producing more value than one who ships eight trivial config changes (8 x 1 = 8 points), even though the latter generated more PRs. Combined with code turnover measurement, this separates genuine productivity from AI-inflated volume.

Do DORA metrics still work for AI-native teams?

DORA metrics remain partially useful but are no longer sufficient. Deployment Frequency and Lead Time for Changes are structurally inflated by AI-generated code -- more code ships faster, but that does not necessarily mean more value is delivered. Change Failure Rate misses the code churn problem entirely. MTTR remains relevant. The Developer AI Impact Framework does not replace DORA so much as it extends the measurement surface to account for AI's impact on code volume, code quality, and the relationship between the two. See Why DORA Metrics Break in the AI Era for a detailed analysis.

What is complexity-adjusted throughput?

Complexity-Adjusted Throughput (CAT) is a velocity metric that weights engineering output by the difficulty of the work delivered. Each PR or unit of work is classified as Easy (1 point), Medium (3 points), or Hard (8 points) based on the nature of the change. CAT per engineer per week replaces raw PR counts and LOC as the primary velocity measure. The exponential scaling (1-3-8 rather than 1-2-3) reflects the reality that hard engineering problems require disproportionately more skill, context, and judgment than easy ones. See What Is Complexity-Adjusted Throughput? for the complete methodology.

How do you measure AI code quality?

The primary quality metric for AI-generated code is code turnover rate -- the percentage of AI-generated lines that are rewritten or deleted within 30 or 90 days. Pre-AI code churn baselines were around 3.3%. In organizations with heavy AI coding tool adoption, overall churn has risen to 5.7-7.1%¹. Healthy AI code quality means keeping AI code turnover within 1.5x of human code turnover. When AI code turnover exceeds 2x human turnover, it signals that AI-generated code is creating rework rather than accelerating delivery. See What Is Code Turnover Rate? for measurement methodology.

What is a good AI adoption rate for engineering teams?

A healthy AI coding tool adoption rate is above 50% WAU within 90 days of deployment, with a target of 60-70% WAU at maturity. Industry average WAU currently sits at 30-40%. Leading engineering organizations achieve 60-70% WAU with 25-35% of developers qualifying as power users (daily usage across multiple AI tool modes). Below 30% WAU after the initial rollout period typically indicates adoption barriers -- licensing friction, training gaps, cultural resistance, or poor tool-workflow fit -- that require targeted intervention.

How do you calculate ROI on AI coding tools?

ROI on AI coding tools equals the productive value of time saved minus rework costs, divided by total AI tool cost -- including token and usage-based costs, not just seat licenses. The formula is: Net ROI = (Productive Value of Time Saved - Rework Cost from Code Turnover) / Total AI Tool Cost. Most organizations understate costs (using only seat license fees when agentic tools like Claude Code can cost $200-$2,000/month per engineer in token spend) and overstate returns (ignoring rework from low-quality AI code). Industry average ROI is 2.5-3.5x; top-quartile organizations achieve 4-6x through better prompt engineering, quality gates, and targeted use of agentic tools on high-complexity work. See How to Measure AI Coding Tool ROI for a step-by-step calculation guide.

Is AI-generated code creating technical debt?

Not inherently, but without quality measurement, it can. The GitClear data showing doubled code churn since AI adoption suggests that a significant portion of AI-generated code does not survive in the codebase long-term¹. This is effectively technical debt in the form of rework. However, the solution is not to reduce AI code generation -- it is to measure code turnover rate and use it as a quality feedback loop. Teams that track AI code turnover and feed quality signals back into their development practices achieve AI code durability comparable to human-written code. See Code Churn in the AI Era for the full analysis.

Footnotes

Related Resources

Framework Components

• What Is Complexity-Adjusted Throughput? The Metric That Replaces LOC
• What Is Code Turnover Rate? The AI Code Quality Metric
• AI Code Share: What Percentage of Your Code Is AI-Generated?
• Innovation Rate: Are Your Engineers Shipping Features or Fighting Fires?
• What Is the AI Impact Hierarchy?
• AI Value Realization Score: One Number for AI Engineering ROI

Context and Analysis

• Why DORA Metrics Break in the AI Era
• Why the SPACE Framework Falls Short for AI-Native Teams
• Code Churn in the AI Era: Why It's Doubled and What to Do
• DORA Metrics Explained: The Complete Guide (2026)
• SPACE Framework Explained: What It Measures and Where It Falls Short

Practical Guides

• Developer Productivity Benchmarks 2026
• AI Coding Benchmarks 2026: Adoption, Output, and Quality Data
• How to Measure AI Coding Tool ROI for Engineering Leaders
• Developer Experience Surveys for AI-Native Teams
• AI-Native Engineering Teams: What They Are and How to Measure Them

Hub

• Developer Productivity Intelligence Center

The Developer AI Impact Framework was developed by Larridin.