The most important AI chart for startups may not be the one showing larger models, higher benchmark scores or billion-dollar training runs. It may be the one showing something quieter: the price of asking a capable model a question has collapsed.
Stanford’s 2025 AI Index Report, working with data from Epoch AI and Artificial Analysis, estimates that the cost of querying an AI model at roughly GPT-3.5-level performance on MMLU fell from $20 per million tokens in November 2022 to just $0.07 per million tokens by October 2024. That is a reduction of more than 280 times in about 18 months.
The phrase “per million tokens” sounds technical, but the business implication is simple. The raw cost of running useful language-model inference has moved from something a product team had to budget carefully to something that can disappear into the background of ordinary software usage.
This is not a clean comparison between one exact model and its later version. The AI Index is careful: it is comparing the price of reaching a fixed capability threshold, not the sticker price of a single vendor’s model. The threshold was a model scoring the equivalent of GPT-3.5 on MMLU, a widely used benchmark for multitask language understanding. The $0.07 figure came from Gemini-1.5-Flash-8B, a smaller and cheaper model that could reach that older performance level.
That distinction matters. It is also the whole story.
The weird economics of falling intelligence costs
Most software gets cheaper slowly. Cloud storage falls over time. Bandwidth improves. Chips become denser. Moore’s Law trained the technology industry to expect exponential progress, but usually on the scale of years and decades.
AI inference has been moving on a stranger clock. The AI Index reports that, depending on the task, large language model inference prices have been falling anywhere from nine to 900 times per year. A later Epoch-linked analysis of AI inference prices found that the price of achieving a given benchmark level has been falling extremely quickly across knowledge, reasoning, maths and software-engineering tasks.
Part of this is hardware. GPUs, accelerators and data-centre utilisation improve. Part of it is competition. Once multiple labs can offer similar capability, price pressure becomes brutal. But a large part of it is model design: smaller models getting better, distillation, routing, quantisation, caching, better serving systems and the simple fact that the industry is learning how to do the same work with less compute.
That is why the comparison is so powerful. GPT-3.5 felt like a frontier model in late 2022. By late 2024, GPT-3.5-like capability had become something a much smaller, cheaper model could provide at a tiny fraction of the price.
What this means for startups
For startups, falling inference cost changes the shape of the possible product. In 2023, many AI features were constrained by unit economics. A founder could build a demo that looked magical, but every user interaction carried a cost. If the model had to read long context, generate long answers or run repeatedly inside a workflow, the economics could break before the product found real usage.
When inference gets 280 times cheaper at a fixed capability level, some of those constraints loosen. A feature that once had to be rationed can become always-on. A support bot can summarise every ticket rather than only escalated cases. A writing tool can run multiple drafts in the background. A compliance product can scan more documents. An education app can generate feedback more often. An internal knowledge tool can answer small questions that were not worth an expensive call before.
The most immediate effect is experimentation. Cheap inference lets teams test more product ideas without needing a model-cost spreadsheet for every interaction. It also changes UX. Instead of asking users to wait for one expensive response, products can run chains of smaller model calls invisibly: classify, retrieve, rewrite, check, personalise and summarise.
That makes AI feel less like a feature and more like an ambient layer in software. The user may not know a model ran six times in the background. They only notice that the product feels more responsive, more organised or more aware of context.
The catch: cheap tokens do not mean cheap AI
There is a trap in the price-collapse story. Cheaper inference does not automatically mean AI bills fall. When the marginal cost drops, usage often expands. The same thing happened with storage, bandwidth and cloud compute. Lower prices create new use cases, and new use cases create more demand.
There is also a frontier premium. The AI Index notes that state-of-the-art models remained much more expensive than smaller alternatives. In early 2025, models such as OpenAI’s o1 and Claude 3.5 Sonnet carried far higher output-token prices than cheap small models. The cheapest model that can do the job is not always the model that can do the hardest job.
That means the practical question for startups is no longer simply “Can we afford AI?” It is “Which parts of the product need frontier intelligence, and which parts only need good-enough intelligence at near-commodity cost?”
The companies that answer that well can build much better margins. They will use expensive models selectively: for complex reasoning, sensitive decisions, difficult code, high-value enterprise workflows or tasks where failure is costly. They will use cheaper models for classification, extraction, rewriting, routing, formatting and everyday assistance.
Commoditisation arrives fast
The price collapse also cuts the other way. If GPT-3.5-level capability becomes nearly free, then “we have AI” stops being a defensible claim. A startup cannot build a durable company around access to a model that everyone else can call cheaply.
That pushes advantage elsewhere. Distribution matters more. Proprietary data matters more. Workflow knowledge matters more. Trust, compliance, integration and taste matter more. The model is still important, but the model alone is less likely to be the moat.
This is the uncomfortable part of the AI boom for founders. A technical advantage can vanish quickly when the underlying capability curve moves faster than product cycles. What looked expensive and rare in a seed deck can become a commodity before the next fundraise.
But that same compression creates opportunity. If model capability gets cheaper faster than incumbents can reorganise, startups can attack workflows that used to be economically impossible. They can offer AI-rich products at software-like prices. They can automate long-tail tasks that were previously too small to justify human labour and too expensive to justify machine inference.
The real collapse is in permission
The deepest change may be psychological. High inference costs forced teams to ask permission before using AI everywhere. Every feature needed an economic defence. Every background process looked like a line item.
At seven cents per million tokens for GPT-3.5-level capability, the question changes. Teams can ask what should be intelligent by default. Not because every task deserves a frontier model, but because many tasks now deserve some level of model assistance.
This is why the price collapse has few obvious parallels in computing. It is not just that one service got cheaper. It is that a new kind of software input, natural-language intelligence, moved from premium scarcity toward commodity availability in less than two years.
The winners will not be the companies that simply spend the least on tokens. They will be the companies that understand what cheap intelligence makes possible, and what it makes worthless.
In late 2022, GPT-3.5-level AI was something developers reached for carefully. By late 2024, the same class of capability could be priced like infrastructure dust. That is not the end of the AI business model. It is the beginning of a harsher one: when intelligence becomes cheap, the product has to be genuinely useful.
