The Cost of an Agent Is the Shape of Its Work

The first theory anyone forms about the cost of an AI coding agent is that long conversations are expensive, and it is a reasonable one. A thread that has been open for a week, holding hundreds of exchanges and the full text of every file it has ever read, sounds like the thing that drains the account. The setup measured here is the fleet of coding agents that works on this site, and it mostly contradicted that theory. Showing it took ten days of counting, three claims that had to be withdrawn, and one number that turned out to be junk.

Two meters, and only one of them is money

When a bill arrives larger than expected, the obvious first move is to look at how many tokens were used, and the obvious second move is to assume that the busiest conversation is the expensive one. Both moves run into the same problem, which is that a token count is not a bill unless every token was billed at a rate.

Part of the traffic here ran on a provider that charges per token. Part of it ran on a flat monthly plan, where one more call in a month already paid for costs nothing at all. Those are not two versions of the same number. Adding them together produces a figure that looks like spending, moves when nothing about the spending changed, and points at the wrong sessions. The first version of the measurement did exactly that: it ranked sessions by token volume and put the largest one at the top of a list of costs, when almost every call in it had run on the flat plan. The most active conversation in the fleet had a marginal cost near zero.

So the accounting that has to be settled before any optimisation is decided is not “how much did the tokens cost” but “which tokens had a price”. Everything in this account afterwards is measured against metered traffic only.

The multiplier almost nobody counts

A car’s odometer records distance, and distance feels like a fair proxy for cost. Drive further and you spend more. What the odometer cannot tell you is how the car was driven: the same hundred kilometres can be a smooth motorway run or an hour of stop-start traffic, and the fuel differs by a factor that has nothing to do with the number on the dial.

The same shape showed up here. The size of a conversation plays the odometer’s role: it is visible, it grows, and it feels like the thing to watch. What actually set the cost was the number of times the conversation was sent. Every tool call an agent makes is followed by another model call, and that model call carries the entire conversation again. Measured over a two-day window, the ratio was 1.02 model calls per tool call, with no batching visible beneath it.

Where the comparison stops being exact matters, because it is the whole point. An odometer accumulates across a car’s lifetime, so distance and fuel stay roughly proportional. A conversation does not accumulate the same way: the context window fills up and then stops growing, while the re- sending continues at full size. Past that point, cost stops tracking how much the agent knows and starts tracking how many times it was asked. A task that needs nine hundred tool calls sends the whole window nine hundred times; one that needs forty sends it forty times. No amount of extra context in either changes that multiplier, which is why the lever is fewer calls rather than less context.

That is a counter-intuitive place to land, and the natural objection is that a large context must at least be expensive to carry. The answer is that most of it is not.

Why carrying more context costs less than it looks

Nearly all of the input volume was cached. Across ten days, 97.7% of input tokens were cache reads rather than fresh ones, and a cache read is billed at roughly two per cent of the normal input rate. The reason is mundane: most of what is re-sent on every call has been sent before, unchanged, and the provider recognises it and charges less for the second look.

That single fact moves context size to the bottom of the list of things worth attacking. The per- call cache-hit distribution showed most calls near the ceiling: slightly over half at 99% or better, and about six in seven at 95% or better. That leaves little upside left to recover, by construction. If every input token in the window had been a cache read, the input side of the bill would have fallen by a little over half. That is a real saving, but a bounded one: new material, whether a person’s words or a tool’s output, must be sent at full rate the first time and cannot be cached away.

What the bill was actually made of

Pricing the logs splits the metered total into three parts: output at roughly 39%, fresh input at roughly 38%, and cached context at roughly 23%. The numbers are inferred from the logs against published rates, so they are a shape rather than a receipt, but the shape is stable enough to act on. Output is billed at several times the input rate and can never be cached, because it does not exist until it is generated. The cheapest useful call is one that says less and still carries the decision.

Most cost discussions go the other way. Reducing what an agent is told feels like frugality, and it is the smaller term. Capping the length of the reply is uncomfortable, because the reply is the part a human reads, and it is the largest term.

Where the work concentrated

Calls were not spread evenly. Counting sessions across the window, the three largest held 52% of model calls, and one run on its own held 38%. That run took a single long goal and the review of it through 3,849 calls across about ten hours of continuous work.

That session is also the best illustration of the metering problem from the top of this piece. Almost all of its calls ran on the flat plan rather than the metered one, so its share of the bill was negligible even as it dominated the activity. Ranking by size aimed the first round of work at the one conversation that could not have reduced the bill, while the modest sessions that did carry the cost sat further down the list.

Three numbers that did not survive being checked

The most useful part of this record is not the ratios. It is the three claims that were made confidently, measured properly, and withdrawn.

The first was that roughly 30% of the agent’s turns in long sessions carried no content, and that these were wasted turns. They were not wasted turns and they were not empty. Every one of the 1,963 turns in question carried a tool call, and the record names that as the reason. Turns with neither text nor a tool call were 42 out of about 11,400, or 0.4%. The claim had confused a normal tool- call-only turn with a turn that did nothing.

The second was that roughly a quarter of calls produced nothing. That number came from subtracting one session counter from another, and the two counters did not measure the same thing. The comparison was discarded rather than explained.

The third was that a background review step was quietly running while switched off in its own configuration. It was not running. Reading the setting directly returned false, and the last recorded call of that kind had happened two days earlier. The trap is one anyone reading a log will meet: an all-time total still carries every call the job made before it was disabled, so a table of old rows reads as a live drain.

Each of these went the same way: a plausible reading of a large number, published as a finding, then contradicted by a smaller and more careful measurement of the same thing. That sequence is the more useful result, because it says something the ratios cannot: how much confidence this kind of counting deserves.

A measurement that cannot report its own error is not a measurement, and the same bar applies to anything built on top of it. Two successive versions of the automated meter raised alarms that were false, calling ended sessions still running and reporting an idle conversation as a day of work, before the definition of “live” was tightened. An alarm that fires on healthy traffic teaches its reader to ignore alarms, and the next real one arrives pre-dismissed. Fixing the definition of the alarm was the fix; attaching a paragraph-long explanation to a false alarm would have been the appearance of one.

Which counter to trust, and which one is junk

Deciding where to look for the truth was part of the work, and the answer was not the field that sounds most authoritative. One counter, the API call count recorded per session, matched the reconstructed log exactly in the session that was checked in detail: 3,849 calls, 3,849 records. A different field, the tool call count, disagreed with the underlying message records by a factor of thirty in the same session. It was not slightly noisy. It was unusable, and any ratio built on it was meaningless.

The log had a trap of its own. Log files rotate at about five megabytes, so a parse that read only the current file saw 101% of one lane’s calls and 15% of another’s. That produced a discrepancy that looked exactly like a faulty counter and was in fact a partial read. The correction was to parse every rotated file and to state the coverage alongside the number. A total quoted without its coverage cannot be checked.

The levers that survived

Ranked by what the measurements actually supported, the changes that held up were modest and unglamorous. Fewer model calls comes first, which in practice means doing independent work in one call rather than several. Shorter output comes second, because it is the largest single term and the least reusable. Bounding what goes into the context comes third: offsets, limits and counts rather than whole files, though this is the smallest of the three.

Beyond the arithmetic, the shape of the work mattered more than any single setting. A narrow, goal- shaped run with a fresh context cost a few tens of calls, and so did the reviewers that checked its output; the expensive thing was the long thread kept alive across many goals, where every new question paid to re-send everything the previous ones had established. Keeping a session alive is genuinely cheaper than starting a new one, because a fresh start re-reads the whole prompt at the full rate and throws away the cache. Both facts are true at once, and they pull in opposite directions: stay in one thread rather than restarting it, and do not use one thread for every goal.

One change did not survive contact with its own premise. A limit on how long a single run may continue had been declared as 300 and was being enforced at 90, because the setting was bridged process-wide and each secondary process inherited whichever value the first one launched with. The declared number was not the number in force. The fix was to set it explicitly everywhere so that whichever path resolved it gave the same answer, and then to measure what the cap was actually cutting. Truncating a run costs a follow-up, a re-prime and a person’s attention, so a cap set below the ninety-ninth percentile of real turns buys interruptions rather than savings.

What is not proven

The numbers in this piece come from logs and from a local database of per-session usage. They are not the provider’s own accounting, which could not be read: there is no usage endpoint, and the vendor’s interface is behind a login. The rates used to turn tokens into proportions are published rates, not observed charges, so the split near the middle of this note is a reasoned estimate and should be read as one. For the same reason, no figure here is expressed in currency. The proportions carry the lesson; the totals belong to whoever pays them.

Nor has the lesson been tested forward. Nothing here reports a before-and-after measurement of a change applied and shown to reduce spend, because the changes that followed are recent and their effect is not yet separated from the ordinary variation of a working month. The claim this note can honestly make is narrower: these are the ratios that were measured, and these are the ones that were thrown away.

That asymmetry is the point. The instinct on first seeing a large bill is to look for the big thing being carried around and shrink it. What the measurement kept pointing at instead was how often the same thing was sent, how much of the answer could have been shorter, and how many of the audit’s own first conclusions did not survive being re-checked. The counter that looked authoritative was junk, and the one with an unassuming name was exact.