Estimating the cost of an AI application starts with usage, not a guessed monthly figure. This guide shows how to model input and output tokens, requests per user, caching, batch processing, retries, storage, and operational overhead so you can build a repeatable LLM usage budget and update it when your traffic, prompts, or provider pricing changes.
Overview
An AI API cost estimate is a planning model. It is not a promise that your final invoice will match the forecast. The estimate becomes useful when it connects four things: the number of requests, the tokens processed by each request, the price assigned to those tokens, and the additional services required to operate the application.
A simple model is:
Monthly AI cost = input token cost + output token cost + other usage costs + operational overhead
For a model priced separately for input and output, calculate the two token categories independently:
Input cost = monthly input tokens ÷ billing unit × input price
Output cost = monthly output tokens ÷ billing unit × output price
The billing unit might be a million tokens, a thousand tokens, or another unit defined by the provider. Use the exact unit shown on the provider's current pricing page rather than assuming that every model uses the same one.
This approach works for a chatbot, summarization workflow, classification pipeline, RAG application, content assistant, or internal developer tool. It also makes model comparison more meaningful: you can compare the cost of completing the same task under the same prompt, output, traffic, and quality assumptions.
How to estimate LLM token cost
1. Define the request unit
Start by deciding what one billable interaction means. For a chat application, it may be one user message and one model response. For a document workflow, it may be one document processed from start to finish. For a RAG system, it may include query rewriting, retrieval-related calls, answer generation, and an optional citation check.
Write the request flow down before estimating. A single visible feature can create multiple API calls. If one user action triggers a classifier, a retrieval call, and a final answer, count all three.
2. Estimate monthly requests
Use a traffic model that separates users from actions:
Monthly requests = monthly active users × requests per user
If usage varies, create low, expected, and high scenarios. For example, a publisher might estimate the number of active editors, articles processed per editor, and review passes per article. A creator tool might estimate subscribers, sessions per subscriber, and AI actions per session.
3. Estimate tokens per request
Separate the average input from the average output. Input can include system instructions, conversation history, retrieved passages, user text, metadata, and formatting requirements. Output includes the model's generated answer, structured fields, or classification explanation.
The core calculation is:
Monthly input tokens = monthly requests × average input tokens per request
Monthly output tokens = monthly requests × average output tokens per request
Do not estimate only from the visible user prompt. Long system prompts, repeated history, and retrieved context can dominate usage. If the application sends the complete conversation on every turn, model that growth explicitly.
4. Apply the current model price
Create a pricing row for each model you are considering. Record the model name, input price, output price, billing unit, effective date, and any relevant conditions such as cached-input pricing, batch pricing, minimums, or separate charges. Because provider plans and model terms change, treat these values as editable inputs rather than permanent facts.
A useful spreadsheet layout is:
| Input | Example formula or value |
|---|---|
| Monthly users | Enter forecast |
| Requests per user | Enter average |
| Average input tokens | Measure or estimate |
| Average output tokens | Measure or estimate |
| Input price per billing unit | Copy from current provider pricing |
| Output price per billing unit | Copy from current provider pricing |
| Retry rate | Use an observed or clearly labeled assumption |
| Cache or batch adjustment | Apply only when the workflow qualifies |
5. Add retries and secondary calls
Retries are part of usage, even when the user sees only one successful result. If a request has a retry rate of r, use:
Adjusted requests = base requests × (1 + r)
For a workflow with several calls, estimate each call separately. A concise classification prompt may have a different token profile from a long-answer generation prompt. This also helps you identify which step is responsible for most of the budget.
Inputs and assumptions that change the estimate
Prompt and context design
Prompt engineering affects cost as well as output quality. Repeated instructions, unnecessary examples, excessive retrieved passages, and full conversation history increase input tokens. A shorter prompt is not automatically better, but every instruction should have a purpose. Review system prompts and context assembly as part of cost control.
For structured outputs, include only the fields the application needs. A strict schema can improve reliability, but a large schema and verbose explanations may increase input or output usage. See the guide to structured AI outputs for implementation considerations.
Caching
Separate reusable content from request-specific content. Stable instructions, product documentation, or repeated reference material may be candidates for provider-supported caching, application-level caching, or precomputed results. Do not reduce the estimate automatically: document which tokens are cacheable, whether the provider applies different rates, and how often the cache is refreshed.
Batch processing
Jobs that do not need an immediate response may be processed in batches if the selected provider and workflow support that option. Model batch work separately from interactive traffic. It may have different timing, limits, or pricing conditions, so confirm the current terms before applying a discount or reduction in the spreadsheet.
Storage and surrounding infrastructure
The model invoice is only one part of AI application pricing. Add estimates for file storage, databases, vector indexes, embeddings, serverless execution, queues, observability, bandwidth, authentication, and human review where applicable. For a RAG application, include document ingestion, chunking, embedding generation, retrieval, and re-indexing—not just the final answer call. The RAG evaluation guide can help connect quality testing with the workflow you are budgeting.
Quality and review costs
A low token bill is not a successful budget if poor outputs create rework. Track acceptance rate, correction time, escalation rate, and the cost of human review. If you are building a content workflow, pair the financial estimate with a fact-checking and editorial review process, such as the one described in the AI content fact-checking workflow.
Worked examples
Example 1: a document summarization tool
Assume a planning scenario with 2,000 documents per month. Each document creates one request with an average of 6,000 input tokens and 800 output tokens. The monthly volume is therefore:
- Input tokens: 2,000 × 6,000 = 12,000,000
- Output tokens: 2,000 × 800 = 1,600,000
To turn those volumes into currency, divide each total by the billing unit shown by the selected provider and multiply by the current input or output rate. Then add a retry allowance, storage, and any document ingestion or review costs. If the workflow uses a second call to extract action items, calculate that call as a separate line instead of hiding it inside the summary estimate.
Example 2: a conversational assistant
Assume 500 monthly users, 20 messages per user, 1,000 average input tokens, and 350 average output tokens. The base monthly volume is:
- Requests: 500 × 20 = 10,000
- Input tokens: 10,000 × 1,000 = 10,000,000
- Output tokens: 10,000 × 350 = 3,500,000
If the application retries 3% of requests, estimate 10,300 requests before calculating tokens, or apply the retry allowance separately to the relevant call. Then test a second scenario in which conversation history raises the average input from 1,000 tokens to a higher value. This sensitivity check often reveals why costs rise as sessions become longer.
Example 3: comparing two models
Keep the workload constant and change only the model row. Use the same requests, token assumptions, retry rate, and operational costs for both options. Then add quality results from a representative evaluation set. A cheaper model that requires more retries, longer prompts, or extensive human correction may not be cheaper in practice. For coding workflows, combine cost estimates with task-based testing rather than selecting a model from price alone; the coding model comparison guide provides a useful evaluation mindset.
When to recalculate your AI API budget
Recalculate before launch, after the first meaningful usage sample, and whenever a major input changes. At minimum, revisit the model when:
- A provider changes input, output, cached, batch, or embedding prices.
- You switch models, providers, regions, or API features.
- A prompt, system instruction, response schema, or context window changes.
- Traffic forecasts, user behavior, or requests per user move materially.
- A RAG index, document corpus, or retrieval setting changes.
- Retry rates, latency targets, moderation checks, or fallback models are added.
- Observed token usage differs from the planning assumption.
For an operational review, export actual request counts and token totals, then compare them with the spreadsheet by feature and model. Replace guesses with measured averages, but retain low and high scenarios for planning. Keep a dated change log for prices and assumptions so the estimate remains auditable.
The most practical next step is to create one worksheet for traffic, one for token assumptions, one for provider pricing, and one for totals. Record input and output separately, include every secondary API call, and add a contingency line without disguising it as model usage. Before each launch or budget review, update the pricing inputs, rerun the low, expected, and high cases, and verify that the resulting quality is acceptable for the application.
