The cost of an AI feature is almost never the invoice from the model provider. That invoice is the tip: add infrastructure, quality assurance and the time of the people who resolve disputed cases. Count tokens only and the project economics look excellent — right up to the first quarterly report.

What makes up the price of a single request

  • Input tokens: the instruction, conversation history and retrieved document fragments. In RAG systems this is usually the bulk of the bill.
  • Output tokens: more expensive than input, which makes answer length a parameter you control, not a fact of nature.
  • Repeated calls: query rewriting, tool calls, self-checking. One “user request” easily becomes three to five model calls.
  • Storage and search: the vector database and re-indexing when documents change.
  • A human in the loop: expert time spent reviewing a share of the answers.

A practical habit: log the cost of every call from day one of the pilot, and aggregate it per completed user task rather than per model call. The business counts tasks, not tokens.

Three levers that really lower the bill

The first is caching. If the instruction and knowledge base are stable, you should not pay for them again on every call. The second is routing: simple requests go to a cheaper model, hard ones escalate. The third is context length — careful retrieval that brings five relevant fragments instead of thirty “just in case” often cuts the bill several times over and improves quality along the way.

How to calculate payback

Take one process and measure it in hours before adoption: tasks per month, minutes per task, cost of an employee-hour. That is your baseline. After the pilot, measure the same things: the system will close some tasks fully, speed up others, and some will have to be redone after it — that last part is a cost too.

Economics is measured on the difference between “before” and “after” in the same process, not on a model's promised performance.

Then it is simple arithmetic: hours saved minus the cost of requests, infrastructure and review. If the difference is positive and holds for two or three months, the solution can be scaled. If it only holds for ideal requests, you are still measuring a demo, not a process.

What not to promise

Do not put “freed-up headcount” into the payback model until it is clear what those people will do instead. Count throughput: how many requests, contracts or tickets the same team handles per month. That number is verifiable and does not start arguments.