2 min read
In short
Most articles on LLM costs don't take caching and batch discounts into account—and are off by a factor of 2–3. CostPerPrompt attempts to calculate the actual price based on workload type: chatbot, agent, RAG, voice.
When you estimate the cost of AI features, you’re almost certainly wrong. Not because the model is expensive, but because you’re using a linear formula—“tokens × price”—and ignoring two factors that change everything: prompt caching and batch API.
Caching repetitive context cuts input token costs by up to 90%. Batch processing offers an additional ~50% discount if you’re willing to wait for the result. Most articles titled “How Much Will Your Chatbot Cost?” don’t account for either of these factors—and their estimates are either 2–3 times too high or too low.
The CostPerPrompt website aims to fix this. It’s not just a price table for 232+ models, but a set of calculators tailored to specific scenarios.
What’s calculated separately:
- API Cost Calculator — arbitrary load, accounting for caching and batching
- Chatbot Cost — simulates a real conversation with a growing history and context resending
- Agent Cost — multi-step cycles, tool calls, retries
- RAG Cost — indexing, retrieval, and generation calculated separately
- Voice AI — STT + LLM + TTS calculated per minute and per call
- GPU Rental — H100, A100, RTX 4090 from 10 providers
The most interesting calculator is Agent Cost. Multi-step agents with tool calls and retries are the scenario where a naive estimate breaks down the most. The website states explicitly: agents cost 10–30× more than you think. This isn’t marketing hyperbole, but a consequence of the architecture: each step is a new request, each request carries the entire context, and retries double the cost.
Model prices on the website are updated automatically. At the time of this review, the table of flagship models included GPT-5.6 Sol, Claude Fable 5, Gemini 3.1 Pro Preview, DeepSeek V4 Pro, and others. The price range is huge: from $0.09 per million input tokens for DeepSeek V4 Flash to $10 for Claude Fable 5. Output tokens are 3–5 times more expensive than input tokens across the board.
A special bonus is the GPU calculator: the same chip can cost up to 5× more from different providers. If you’re considering renting hardware instead of using an API, this is the first screen you should check out.
Practical takeaway: if you’re budgeting for an AI feature and don’t factor in caching and batching, you’ll either overpay or get a bill you weren’t expecting. The tool isn’t perfect—it won’t account for your specific latencies, rate limits, or peak load. But it’s more reliable than simply multiplying the model’s price by the number of requests and hoping for the best.