Costs often spike because each step in an agent’s loop carries more context forward. File reads, tool calls, and retries add to that history, so later steps process far more input than earlier ones. One task can end up costing much more than a pricing page suggests.
Key Takeaways
- A 2026 study of eight frontier models completing SWE-bench Verified tasks found that agentic coding tasks consumed roughly 1,000 times more tokens than code reasoning and code chat, with input tokens driving most of the cost.
- A LeanOps audit of 30 engineering teams attributed 62% of the agent bill to re-sent context. That’s just one vendor’s production data, but the pattern aligns with broader research showing that agentic workflows repeatedly ingest large inputs.
- Context accumulation is manageable. Prompt caching, tighter session boundaries, context pruning, and model routing can reduce waste, but leaders first need enough spend attribution to see which agents, teams, and use cases deserve investigation.
How Context Accumulation Drives Agent Costs
Many chat and completion APIs don’t automatically retain conversations. Therefore, agent frameworks carry forward the information a model needs for the next step: the original request, prior responses, tool definitions, file contents, test results, and other observations.
That history can grow with every round. In the 2026 frontier-model study, Bai and colleagues found that the agent framework carried the full conversation history forward unchanged from one round to the next. The researchers found an average input-to-output token ratio of about 154:1 for agentic coding tasks. Input tokens dominated even when token caching was enabled.
In a basic agent loop, total token use can grow much faster than the number of steps. If every step adds 1,000 new tokens, a 20-step workflow doesn’t use 20,000 input tokens. Because earlier context is sent again with each step, cumulative input is 210,000 tokens. An Augment Code analysis uses this example to show why a 20-step loop can consume more than 10x as many tokens as a simple per-step estimate suggests.
Not every agent works this way. Newer APIs and agent architectures can use server-side state, caching, summaries, or constrained context windows. The expensive pattern is the naive one: append everything, resend everything, and continue the loop without a cost boundary.
What the Research Shows
The strongest available evidence is the 2026 study of eight frontier models working on SWE-bench Verified tasks. Agentic coding averaged 4.17 million tokens per task, compared with 3,390 for code chat and 1,190 for code reasoning. Runs on the same task varied by as much as 30x in total token use, and the higher-cost runs didn’t consistently produce better results.
The study also found that repeated file views and modifications were associated with higher-cost runs. That matters for enterprise budgeting: the expensive behavior isn’t always a harder problem or a better answer. It can be an agent revisiting the same material, retrying failed approaches, or carrying large tool outputs through the rest of the task.
Production data points in the same direction, although they should be treated as vendor evidence rather than an independent benchmark. LeanOps reported a 20-fold spread between its 10th- and 90th-percentile developers on the same tools. Vantage similarly notes that a session with twice as many turns can cost three to four times as much because later turns carry more accumulated context.
Why Aggregate Spend Data Is Not Enough
A monthly total can tell finance that agent costs went up. It can’t explain whether the increase came from productive use, runaway retry loops, oversized file reads, a model choice, or a few unusually long sessions.
To diagnose context accumulation, teams need to examine:
- Input tokens compared with output tokens
- Total tokens and cost per completed task
- Loop length and retry count
- Cost spikes after file reads, test runs, or other large tool outputs
- Spend concentration by agent, team, model, and use case
A high input-to-output ratio is a reason to investigate, not proof that the workflow is inefficient. Some tasks legitimately require large codebase or document context. The better question is whether that input produced a durable result or reflects repeated work that could have been constrained.
4 Ways to Reduce Context Accumulation Cost
1. Cache Stable Context
System instructions, tool definitions, codebase summaries, and other repeated prefixes are strong candidates for prompt or context caching. Anthropic, OpenAI, and Google Cloud all have caching options that reduce the price of reused input, although the implementation and discount vary by provider.
Caching can materially reduce the cost of static content. It doesn’t solve the entire problem, because new file contents, tool results, and reasoning added during the loop are unique and continue to grow.
2. Set Session and Phase Boundaries
Start a fresh session when a task is complete. For longer work, create boundaries between planning, implementation, testing, and closeout rather than carrying the full history through every phase.
This reduces the amount of stale context attached to later calls. It also gives teams a clearer unit for measuring cost per task instead of treating one continuous session as a black box.
3. Prune Tool Results and Available Context
An agent rarely needs every line of every file or the complete output of every test run for the rest of the task. Keep the relevant sections, summarize older results, and clear tool output that no longer affects the next decision.
Tool definitions also consume input tokens. Limiting each step to the tools and context it can actually use reduces repeated overhead without taking useful capabilities away from the broader workflow.
4. Route Work by Task and Step
Reserve expensive frontier models for work that benefits from their reasoning or code-generation quality. File parsing, classification, routine transformations, and other lower-risk steps may be suitable for cheaper models.
The right routing decision depends on the task and required quality. The goal is to stop paying frontier-model rates for context-heavy work that a lower-cost model can handle reliably. The model tier routing framework covers this decision in more detail.
Make Agent Cost Patterns Visible Before the Invoice
These interventions begin with visibility. Leaders need to know which agents are driving variable spend, who owns them, and what business or engineering output they produced.
Larridin’s Token Spend and Insights consolidates AI spend across tools and models, separates human and agent spend, and traces tokens to teams, agents, and use cases. That organization-level view helps leaders identify where costs are concentrated and which workflows need deeper session-level analysis. Projected-overage alerts also surface budget risk before quarter close.
Frequently Asked Questions
Why does an agent cost more than a chatbot for the same task?
A chatbot may answer in one exchange. An agent can plan, read files, call tools, run tests, revise its approach, and repeat those steps. Each round can include substantial context from earlier rounds. EY illustrates the difference with a customer-service example that goes from $0.04 for a simple linear interaction to $1.20 for an orchestrated workflow involving tools, reasoning, and iterative loops.
How do we know whether context accumulation is driving our costs?
Compare input and output tokens, then examine cost per completed task, loop length, retries, and large tool outputs. If input dominates and costs climb sharply in longer sessions without a corresponding improvement in results, context accumulation is a likely contributor. Confirm the pattern at the task or session level before changing limits across the organization.
Does prompt caching solve the problem?
It helps with repeated static content. It doesn’t eliminate the growing context created during the task. Anthropic and Google Cloud document cache-read pricing at 10% of standard input cost for supported models, while OpenAI automatically discounts eligible cached input. The savings can be substantial, but pruning and session design are still necessary.
Is some context accumulation unavoidable?
Yes. Reviewing a large codebase, managing a multi-file refactor, or tracking interdependent steps requires context. The goal is to keep useful context while removing stale, repeated, or irrelevant material and to measure whether the added cost produces a better outcome.
See Where Agent Spend Is Concentrated
Larridin connects AI spend to teams, agents, use cases, and outcomes so leaders can identify where agent-driven costs warrant investigation before they become budget surprises.
Book a discovery call to build a clearer agent cost picture.