The short version
- AI is billed every time it is used, so costs grow with every customer, every page and every agent step.
- Most savings come from using the right model for each job, sending less text and reusing what you can.
- Providers offer big built-in discounts, like half price for batch work, that many teams never switch on.
- Track cost per useful result, not just the monthly total.
Why the bill grows so fast
Training a model is a one-off cost that most businesses never pay, because they use models from providers like OpenAI, Anthropic or Google. What they pay for is inference: each time the model reads your text and writes an answer. It is metered, like a taxi, and the meter runs on every request.
Three things make the meter spin faster than people expect:
- Scale. A pilot with ten users looks cheap. Roll the same feature out to every customer and the cost multiplies.
- Long prompts. Instructions, policies, examples and conversation history are re-sent with every request, and you pay for all of it every time.
- Agents. An AI agent does not answer once. It plans, uses tools, reads the results and asks the model again, sometimes dozens of times for one task.
Teams have noticed. In the FinOps Foundation's State of FinOps 2026 survey of cloud cost practitioners, 98% now manage AI spending, up from 31% two years earlier, and AI cost management is the number one skill teams say they need to develop.
8 ways to cut it
1. Use the smallest model that does the job
Sorting emails, pulling fields out of a form or tagging a support ticket rarely needs the most powerful model. Smaller models are much cheaper and faster. Route simple work to them and save the big model for genuinely hard questions.
2. Send less text
Trim long instructions, remove examples that do not help and stop re-sending the whole conversation when a short summary would do. For document questions, send only the relevant passages instead of the entire file.
3. Turn on prompt caching
If every request starts with the same long instructions or the same reference document, providers can cache that part. On Anthropic's API, cached input is billed at a tenth of the normal input price, and responses start faster too.
4. Batch anything that can wait
Overnight reports, bulk tagging and back-office processing do not need an instant reply. OpenAI's Batch API, for example, costs half the normal price in exchange for results within 24 hours. Other major providers offer similar batch discounts.
5. Keep answers short
Output tokens usually cost more than input. Ask for the format you need, such as three bullet points or a JSON object, and set a sensible maximum length.
6. Put limits on agents
Give agents a maximum number of steps, a time limit and a budget per task, and stop them when they loop. A runaway agent can burn more in an hour than a chatbot does in a week.
7. Reuse answers people already got
Many questions repeat. If ten customers ask how to reset a password, there is no need to generate ten fresh answers. Caching common responses cuts cost and speeds things up.
8. Measure cost per outcome
A monthly total does not tell you whether AI is worth it. Track cost per resolved ticket, per processed invoice or per qualified lead, and compare it with the old way of doing the work.
Related guideYour AWS bill is leaking. Here is where.The ten everyday cloud leaks that also sit underneath most AI products, and how to plug them.Make someone own it
AI costs creep for the same reason cloud costs do: nobody is watching. The fix is simple.
- Give every AI feature an owner and a monthly budget.
- Tag requests by feature or team so you can see who is spending what.
- Set alerts on daily spend, so a runaway agent is caught in hours, not at the end of the month.
- Review the biggest cost drivers monthly and try one of the eight levers on each.
Questions people ask
What is AI inference cost?
It is what you pay every time an AI model is used to produce an answer, usually charged per token (roughly, per chunk of text read and written) or per hour of GPU time if you run models yourself. For most businesses it is a far bigger cost than training.
Why do AI agents cost more than chatbots?
A chatbot usually makes one model call per question. An agent plans, calls tools, reads the results and calls the model again, often many times for a single task, and each call re-reads a growing history. The tokens add up quickly.
What is the fastest way to reduce an AI bill?
Use a smaller, cheaper model for simple tasks and keep the large model for hard ones. Then turn on prompt caching and move anything that does not need an instant answer to a batch API, which OpenAI prices at half the normal rate.
Should we host our own models to save money?
Only if you have steady, high volume and the skills to run GPUs well. Self-hosted GPUs that sit idle are very expensive. Many teams save more by choosing models wisely and caching than by moving infrastructure.
How do we know if an AI feature is worth its cost?
Measure cost per successful outcome, for example per support ticket resolved or per invoice processed correctly, and compare it with what that outcome cost before.
Want your cloud and AI costs watched for you?
We are building a cost optimiser for growing teams that spots waste in cloud and AI spending and shows what each fix will save. Join the waitlist to try it first.
Join the waitlistSources
Prices and discounts change often and differ between providers and models. Check your provider's current pricing before making decisions.
.jpg){kind=link}