87% of the time, the AI agent is idle. And the biggest optimization removes zero tokens.

For five consecutive weeks, the Acefleet team intercepted every request made by a single developer to Claude Code via a local reverse proxy. The result: 4.2 billion prompt tokens, $3,035 at list price, with 90.5% consisting of content that had already been sent previously. This isn’t a bug, but rather a design choice: cache reads account for ~98% of the prompt volume, and the provider has already realized a 6.1× cost savings from caching. But behind these numbers lies a far less obvious picture—and it overturns nearly all assumptions about what to optimize in AI agents.

The Main Enemy Isn’t the Tokens—It’s You

87.7% of the wall-clock time of sessions is idle. Of that, the agent spent 233.8 hours (204 instances, 69 minutes each) holding a pending tool call and waiting for approval that no one ever gave. No tool in the stack will tell you that an agent has been waiting for you for an hour. ACE Sidecar flags this as a live alarm, and no classifier is needed for this—the fact of idling is deterministic. Developers label this as a “ceiling” rather than a “saving,” and this distinction is crucial: a transcript cannot distinguish between “the person would have returned sooner” and “they were away for a long time.”

Optimizations That Looked Promising but Failed

The most valuable lesson isn’t in the numbers, but in how metrics can stifle intuition. Here are a few levers the team planned to implement or considered promising:

• Deduplication of repeated file reads. It seemed like an obvious goal. It turned out to be worth $0.33 over 36 days, because 97.6% of “repeat reads” request a different range of strings and aren’t duplicates at all. We scrapped the optimization.
• ttl_keepalive — the biggest cost driver (6.4%). But it removes zero tokens. This is an accounting measure that doesn’t change anything the model sees.
• Context exhaustion trigger. The team assumed the problem was rate-limit exhaustion. There were 3 rate-limit errors across 162 sessions. The actual issue is context exhaustion, and auto-compaction is already handling it in-band. The real gap is durability: compaction creates a lossy summary that expires along with the session. A resume brief on disk—that’s what’s missing.

The only lever the team is ready to enable in production is Bash truncation with head+tail preservation. Its priority was initially lowered (high risk: output consists of signals, stack traces, and test failures), but was later raised. Analysis showed that 90% of Bash output consists of sed/cat dumps, grep summaries, and git diffs, which the agent reviewed once and never opened again. Keep head+tail, exempt diagnostics—and almost all value is preserved with near-zero risk.

Why the metric must come before the optimizer

ACE Sidecar is Phase 0: observation only, zero traffic changes. The proxy relays requests as-is, writes to a local SQLite database, doesn’t upload anything externally, doesn’t store credentials, and doesn’t automatically approve tool calls (structurally unable to do so—permission decisions don’t go through the API). The dashboard reads existing transcripts from disk, so you get metrics for sessions already accumulated prior to the first relay.

Day-over-day, the median volume change is −12.2%, with an IQR ranging from −48% to +76%. Volume routinely halves or doubles. This eliminates an entire class of alerts: any budget based on the rate of change will trigger constantly and mean nothing. Only a cumulative cap against a period-based budget survives.

Conclusion

All findings come from a single developer, a single machine, over 36 days. The cost structure should be generalized, but the magnitudes are specific to this fleet. The team is explicitly requesting a second fleet for validation. The main takeaway, however, is broader: a measurement tool must exist before an optimizer. Without it, you’ll be optimizing deduplication at $0.33 while overlooking 233 hours of downtime that no tool in the stack can detect.

Source: Hacker News - Newest: ""AI" "LLM""