Analysis
How much does AI implementation cost? What drives the cost and how to calculate it
Most of the cost of implementing AI is not tokens: German companies using AI name infrastructure (51%), data preparation (50%) and system integration (41%) as major cost items, and tokens only 8%. The model itself is getting cheaper fast: the cost of a query at GPT-3.5 level fell from USD 20 to USD 0.07 per million tokens in less than two years. A cost estimate should therefore start with the process and the data, not with the API price list.12
What the cost of an AI implementation consists of
In a 2026 Bitkom survey, companies using AI said which items represent a major cost for them1:
| Cost item | Share of companies naming it a major cost |
|---|---|
| Infrastructure (cloud, computing power) | 51%1 |
| Data preparation | 50%1 |
| Integration with existing systems | 41%1 |
| Licences and subscriptions | 21%1 |
| External consulting | 21%1 |
| Staff training | 19%1 |
| Tokens (model usage fees) | 8%1 |
The conclusion is simple: the API price list is the smallest part of the bill. What matters are the data, the integrations and the people who will work with the system.
The model itself is getting cheaper fast
The cost of a query to a model of GPT-3.5 quality fell from USD 20 per million tokens in November 2022 to USD 0.07 in October 20242. Epoch AI estimates that the price of reaching a fixed level of quality falls by a factor of 9 to 900 per year, with a median of 503. This does not, however, cover reasoning models, which use considerably more tokens per task, and agents often rely on them3.
How to reduce the cost of running models
- Batch processing: tasks that do not need to be completed immediately cost half the price at Anthropic4, OpenAI6 and Google8.
- Prompt caching: fixed instructions and documents used repeatedly are cheaper when read again; at Anthropic a cache read costs 0.1 times the input price5, and at OpenAI the discount reaches 95%7.
- Matching the model to the task: a cheaper model handles simpler tasks, and the more expensive one is kept for tasks that need it.
Costs that are easy to overlook
Machine learning systems are expensive to maintain: a study presented at the NeurIPS conference showed that the model code itself is only a small part of the system, while ongoing maintenance can be very costly12. According to McKinsey, around 20% of respondents say that the cost of running AI, including tokens, limits its use10, and only 27% of organisations using generative AI review all of its outputs before use11. Quality control is a cost that has to be planned from the start.
Why AI projects do not reach production
In a Gartner survey, on average only 48% of AI projects made it into production, and moving from prototype to production took 8 months13. S&P Global found that the share of companies abandoning most of their AI initiatives before production rose from 17% to 42% within a year14. RAND names the main causes as misunderstanding the problem, lacking the necessary data and choosing technology for its own sake rather than for a real problem15.
What an implementation can deliver
In a study published in the Quarterly Journal of Economics, access to an AI assistant increased the number of issues resolved per hour by 15% on average16. In an experiment published in Science, professionals using ChatGPT wrote 40% faster, and the quality of their texts rose by 18%17. There is another side, though: in an experiment with 758 consultants working on a task beyond the model’s capabilities, those using AI were 19% less likely to reach the correct solution18.
How we calculate the cost in practice
We do not quote prices up front, because without the process and the data they would be guesswork. We start with a process map, then build a prototype on real data and measure two things: the quality of the output and the cost of a single task. Only on that basis is the scale of the implementation decided, and we show the ongoing costs separately.
Frequently asked questions
How much does it cost to implement AI in a company?
There is no single figure. The cost depends on the process, the quantity and quality of the data, the number of integrations and the cost of running the models. Research shows that the largest items are infrastructure, data preparation and integration with existing systems. We prepare our estimate from a process map, separately for the prototype and for ongoing operation.1
What is the biggest cost of an AI implementation?
According to a Bitkom survey of 603 German companies: infrastructure (51% of responses), data preparation (50%) and integration with existing systems (41%); 8% named tokens as a major cost.1
Does AI implementation pay off?
It depends on the task. In a study published in the Quarterly Journal of Economics, an AI assistant increased the number of issues resolved per hour by 15% on average, but in an experiment with consultants working on a task beyond the model's capabilities, there were 19% fewer correct answers. The effect has to be measured on a prototype.1618
Sources
- Bitkom: Erstmals nutzt die Mehrheit der Unternehmen KI
- Stanford HAI: The 2025 AI Index Report — Research and Development
- Epoch AI: LLM inference prices have fallen rapidly but unequally across tasks
- Anthropic: Batch processing
- Anthropic: Prompt caching
- OpenAI: Batch API
- OpenAI: Prompt caching
- Google AI for Developers: Batch API
- Eurostat: Use of artificial intelligence in enterprises (isoc_eb_ai)
- McKinsey & Company: The state of AI in 2026: On the road to ROI
- McKinsey & Company: The state of AI: How organizations are rewiring to capture value
- NeurIPS 2015: Hidden Technical Debt in Machine Learning Systems (Sculley et al.)
- Gartner: Gartner Survey Finds Generative AI Is Now the Most Frequently Deployed AI Solution in Organizations
- S&P Global Market Intelligence: Generative AI experiences rapid adoption, but with mixed outcomes
- RAND Corporation: The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed
- The Quarterly Journal of Economics: Generative AI at Work (Brynjolfsson, Li, Raymond)
- Science: Experimental evidence on the productivity effects of generative artificial intelligence (Noy, Zhang)
- Harvard Business School / SSRN: Navigating the Jagged Technological Frontier (Dell’Acqua et al.)