How Much Do AI Coding Agents Cost in 2026? Benchmarks & Math

Benchmark claims checked against the Latitude repository and writeup, Artificial Analysis, and AWS on October 21, 2026. API prices checked against official Anthropic, OpenAI, and Google pricing pages on October 21, 2026. Provider behavior, pricing, and model support can change.

Token prices are only the visible part of an AI coding-agent bill. The real cost appears when an agent reads a repository, calls tools, edits files, runs tests, retries a failed patch, and waits for a human to review the result.

This article uses three kinds of evidence and labels each one: third-party published benchmark results, illustrative calculations using published prices, and PromptSphere measurements. PromptSphere has not yet run its own Claude, GPT, or Gemini provider benchmark, so this article contains no PromptSphere measurements.

The metric that actually matters

A coding agent can produce valid code and still fail the task. It may edit the wrong file, break an existing behavior, stop before running tests, or require several rounds of human correction. A useful definition is:

cost per accepted task =
(total API spend across all attempts
+ total review minutes × reviewer hourly rate
+ rework cost from failed attempts)
÷ accepted tasks

• Total API spend across all attempts: every billed token for every attempt, including failed, refused, and retried runs: uncached input, cached input, cache writes, output, and reasoning tokens.
• Total review minutes: human time spent reading, testing, and commenting on agent output, for accepted and rejected attempts alike. Convert minutes to hours (÷ 60) before multiplying by the hourly rate.
• Reviewer hourly rate: the loaded cost of the person doing the review in your organization.
• Rework cost from failed attempts: engineer time spent fixing, reverting, or redoing work after an attempt that did not pass, priced at the same hourly rate.
• Accepted tasks: tasks whose agent-produced patch passed verification tests and human review without being rewritten by a person.

This definition includes the cost of failure. The right model for a team depends on task difficulty, test quality, failure cost, and how much human review is available.

Why token price alone is misleading

A coding agent is not one request. It is a loop of repository inspection, planning, tool calls, edits, test execution, error analysis, and revision. The same system prompt, tool definitions, repository map, and file context may be sent repeatedly. For the context side of this problem, see PromptSphere’s guide to context rot and long-context degradation.

• Repeated context: stable prefixes can benefit from prompt caching, while dynamic timestamps and request IDs can reduce cache hits.
• Reasoning output: a model that spends more tokens thinking may have a higher invoice but a lower failure rate.
• Different tokenizers: the same codebase can produce different token counts across providers. Anthropic’s pricing page states that Claude 4.7 and later models use a tokenizer that produces approximately 30% more tokens for the same text (Anthropic pricing, checked October 21, 2026).
• Tool overhead: tool schemas and tool results are billed as input tokens. Anthropic documents a tool-use system prompt of several hundred tokens per request on current models, before your own tool definitions are added.
• Failed attempts: a patch that does not pass tests still consumes tokens, time, and review capacity.
• Refusals and errors: a model that declines a task or hits an API failure has a different production cost from one that completes it.

What public benchmarks report

Public benchmark results are valuable when their task definitions and harnesses are visible. They are not directly interchangeable. A cost per solved task from one repository cannot be combined with a pass rate from another repository and presented as one controlled comparison.

Latitude: traced cost, latency, and solved tasks

Third-party published benchmark result. Latitude gave five frontier models (Claude Opus 4.8, Claude Sonnet 5, Claude Fable 5, GPT-5.6 Sol, and GPT-5.6 Luna) the same bug-fix jobs through an agent loop with four tools: list files, read file, write file, and run tests, with a 24-step budget and default model settings. The 18 tasks were split into 12 single-file bugs and 6 multi-file bugs, run with tests visible and then with tests hidden, for 410 runs in total. A run counted as solved only when the full suite passed against pristine test files.

With tests available, the author reports that every model solved every run it attempted, and that cost per solved task on the hard tier varied 14x for the same outcome. With tests hidden, one re-entrancy bug separated the field: the writeup reports that both GPT-5.6 models solved it in every trial, while no Claude model got past 5 of 13. The writeup puts the total API spend for the benchmark at about $32. These are Latitude’s results, not PromptSphere’s.

Limitations you should carry with these numbers:

• Task scope: the repository describes each task as a tiny JavaScript project with one planted bug; the writeup puts them at 25 to 250 lines with a bug report that includes a reproduction. That is far friendlier than repository-scale work, ambiguous requirements, or design changes, which the author says were not tested.
• Harness and authorship bias: the author discloses that the harness was built and operated by Claude Fable 5, which was also a contestant, and that the tasks were written by Claude-based agents. A family-level bias cannot be ruled out.
• Settings and conditions: each model ran at default settings, without reasoning-effort overrides, over two days from a single region.
• Not a product comparison: the benchmark used its own minimal loop, not Claude Code, Codex, or another commercial CLI.

Treat the 14x spread as evidence that cost can diverge sharply when outcomes are equal, not as a prediction for your repository. Source: Latitude coding-agent benchmark repository and Latitude writeup (July 14, 2026).

Artificial Analysis: a multi-benchmark index

Third-party published benchmark result. Artificial Analysis combines DeepSWE, Terminal-Bench, and SWE-Atlas-QnA into a coding-agent index and publishes charts for task-normalized pass@1, token usage, cache hit rate, average pay-per-token API cost per task, agent wall time, and safety refusal rate. The page also defines a reward-hacking rate: the share of attempts scored zero for earning a reward without demonstrating the capability. Check the live page for which agents currently have that metric.

The page notes that its cost figure includes cached-input discounts and cache-write charges where available, but excludes subscription plans, infrastructure, engineering, and supervision costs. It also warns that cache hit rates can vary significantly by provider routing. Results reflect specific agent variants, model settings, and harness configurations, so they are useful for forming a prior, not for predicting the bill on your repository.

Source: Artificial Analysis coding-agent benchmarks (checked October 21, 2026).

AWS agent-cost-bench: a reproducible framework

AWS publishes an open framework for comparing coding CLIs and models on a real repository with verification tests. It has two modes: one runs the same model through several coding CLIs, and one compares several models inside one CLI. It reports cost in USD and native units together with quality and duration. The framework supports a useful principle: bring your own repository and define what “done” means in tests.

It does not make provider access free. Each CLI or model still requires its own subscription, license, or API credits. The value is reproducibility, not free inference.

Source: AWS sample agent-cost-bench.

An illustrative cache scenario

Prompt caching can change the economics of repeated repository context. PromptSphere’s separate prompt-caching measurement guide explains cache hits, cache writes, and the telemetry needed to verify savings. The chart below is a modeled example with generic rates, not a PromptSphere experiment and not a claim about a particular provider’s measured performance. It assumes a medium feature with 1.5 million input tokens and 40,000 output tokens.

Illustrative calculation using published prices. The table applies the same 1.5 million input and 40,000 output tokens to standard pay-as-you-go list prices checked on October 21, 2026. “90% cache hit” bills 90% of input at the provider’s cached-input rate and 10% at the normal input rate. For simplicity it excludes cache-write charges, tokenizer differences, batch discounts, and retries, so it shows the shape of the savings, not a measured bill.

Two details matter more than the totals. First, Google lists Gemini 4 Argon’s standard prices doubling on December 1, 2026 (to $4 input, $20 output, and $0.20 caching), which would raise the same scenario to $6.80 without cache and $1.67 at a 90% hit rate. Second, Google lists implicit caching only for that model, so you cannot force a 90% hit rate; you have to measure it. Anthropic lists 5-minute cache writes at 1.25x the base input price and 1-hour writes at 2x, which the simplified table leaves out.

A real harness should record uncached input, cached input, cache writes, output tokens, reasoning tokens, tool calls, wall time, and retries for every request. It should also record the prompt layout. Adding dynamic text at the top can destroy a stable prefix without changing the task itself.

How to design a fair experiment

Start with a pinned repository commit and task cards that can be reset to a clean state. Every model should receive the same system instructions, tool definitions, task text, repository snapshot, maximum turns, timeout, retry rule, and acceptance tests.

Hidden tests should be copied into the workspace only after the agent stops. A run should fail if the agent changes tests, edits forbidden files, leaves the repository in a broken state, or claims success without satisfying the verifier. This connects directly to PromptSphere’s AI agent evaluation release-gate checklist.

Report task-level results, medians, and confidence intervals when the sample is large enough. Include the exact model IDs, harness version, pricing date, repository commit, and limitations. If the study has only a small pilot, call it a pilot.

Which model should a team choose?

If every diff receives careful human review and the repository has strict automated tests, begin with the lowest-cost model that meets the quality bar; the Latitude results are consistent with this when tests are available. If a failed patch costs an engineer an hour, reliability may matter more than the cheapest token price. If agents must work without runnable tests, test diagnosis quality separately, because that is where Latitude saw models diverge. For large repositories, test long-context behavior separately. For asynchronous test generation, compare batch pricing separately from interactive workloads; Anthropic and Google both list a 50% Batch API discount. When a router can send simple work to a cheaper model and escalate difficult work, compare that policy with a single-model baseline; PromptSphere covers this idea in its guide to LLM model routing.

Six Mistakes That Inflate Coding-Agent Bills — and the Fix

1. Mistake: placing dynamic dates, request IDs, or timestamps at the top of the prompt, which breaks prefix caching. Fix: keep system instructions, tool definitions, and repository maps in a stable prefix and append changing data after it.
2. Mistake: treating effort labels such as “high” or “max” as equivalent across providers. Fix: compare measured reasoning tokens, cost, and pass rate per setting instead of label names.
3. Mistake: overlooking context thresholds and scheduled price changes. Fix: record the pricing date with every result and recheck provider pages before publishing or budgeting; Google, for example, lists a December 1, 2026 price increase for Gemini 4 Argon.
4. Mistake: counting only prompt and file tokens. Fix: include tool schemas, tool results, and provider tool-use system prompts in the input-token budget, and trim large tool outputs.
5. Mistake: accepting the model’s own claim that a task is done. Fix: let external tests, run against pristine test files, decide success.
6. Mistake: quoting a public benchmark number without its task definition and harness. Fix: cite the source, date, task scope, and harness with every number, and rerun the comparison on your own repository before drawing conclusions.

What PromptSphere has and has not measured

FAQ

Is Claude Code or Codex cheaper per task?

There is no single answer. Cost per task depends on the model, effort setting, harness, caching, and task mix, and the same product can appear several times on a leaderboard at different settings. Artificial Analysis publishes average API cost per task for individual agent variants, and the AWS agent-cost-bench cli-compare mode is built to run the same model through different CLIs on your own repository. The Latitude benchmark did not test either product; it used its own minimal agent loop.

How much can prompt caching save for coding agents?

It depends on your measured hit rate. On list prices checked October 21, 2026, Anthropic bills cache hits at 0.1x the base input price on most models, and OpenAI lists cached input for GPT-5.6 models at one-tenth of the normal input price. In the illustrative table above, a 90% hit rate cuts the scenario cost by roughly 70% to 75%, but cache writes, cache expiry, and prompt layout can erase much of that in practice.

Do subscriptions beat API pricing?

Sometimes, but the published benchmarks do not answer it. Artificial Analysis states that its cost figures are pay-per-token API costs and that many users access coding agents through subscription plans instead. AWS agent-cost-bench reports native units such as credits or premium requests next to USD, which is the right way to compare a plan with metered API use on your own workload.

Why is cost per accepted task better than cost per million tokens?

Because token price ignores retries, refusals, review time, and rework. In Latitude’s published benchmark, models reached identical outcomes at a 14x difference in cost per solved task, and one model refused 29 of 82 runs. In the illustrative worked example above, human time is 96% of the total. Cost per accepted task captures all of it.

Does a public leaderboard predict production cost?

No. It can provide a useful prior, but prompt layout, tools, repository size, test quality, retry policy, and human review can change the outcome.

Sources and further reading

• Latitude coding-agent benchmark
• Latitude writeup: Coding ability is not what separates frontier models (July 14, 2026)
• Artificial Analysis Coding Agent Benchmarks
• AWS agent-cost-bench
• Anthropic model pricing
• OpenAI API pricing
• Google Gemini API pricing