“Token Prices Are Falling, but AI Costs Keep Rising” Infrastructure Bottlenecks Deepen the Cost Paradox, While Supply Stabilization Could Reshape the Labor Market

AI token prices have plunged over the past few years, yet real-world AI spending continues to climb.
Exploding compute demand is strengthening the pricing power of AI infrastructure providers, including data center operators.
Once infrastructure bottlenecks ease, a sharp improvement in automation efficiency could trigger sweeping changes across the labor market.

As token prices in the artificial intelligence (AI) market continue to decline, companies are paradoxically seeing their overall AI expenditures rise. The primary reason is that the rapid proliferation of AI agents has driven a surge in absolute token consumption, while the supply of AI infrastructure needed to process that demand has failed to keep pace. However, if AI infrastructure capacity expands and supply-demand conditions for computing resources eventually reach equilibrium, lower token prices are likely to translate into materially lower automation costs, potentially triggering a profound transformation across the broader economy.

The Changing Cost Structure of AI

According to a July 8 report by U.S. investment publication Barron's, average token prices for premium AI models currently range from USD 0.50 to USD 30 per one million tokens. This represents a dramatic decline compared with the early years of the generative AI boom in 2022 and 2023. Consulting firm Optimus Partners likewise found that the average enterprise application programming interface (API) cost per one million tokens fell from USD 18.40 in the first quarter of last year to USD 6.07 in the first quarter of this year. The figure does not represent the official pricing of any single model but instead reflects effective real-world pricing derived from approximately 2.4 billion API calls spanning multiple models and enterprise workloads.

Despite the decline in token prices, however, companies' actual AI spending continues to increase. The expansion of AI agent services has significantly increased the number of tokens required to complete individual tasks. By design, large language models (LLMs) do not inherently retain the context of previous interactions between separate API calls. Consequently, AI agents must repeatedly include prior conversations, previously analyzed files, search results, intermediate calculations, and earlier outputs within the input context at every stage of a workflow. As these iterative task loops become longer, input token consumption grows exponentially.

The cost burden becomes even heavier when retrieval-augmented generation (RAG) capabilities are combined with external tool invocation. While AI agents may appear to answer a single question with a single response, they typically execute multiple internal processes—including document retrieval, summarization, content comparison, error verification, and response refinement—before generating a final output. Every stage generates new inputs and outputs, while the accumulated context from earlier steps continuously expands the amount of information that must be processed. Moreover, AI agents frequently repeat identical tasks multiple times whenever outputs fail formatting requirements or validation criteria, leading to explosive growth in token consumption.

AI Infrastructure Emerges as the Critical Bottleneck

As token usage continues to expand, demand for the infrastructure required to operate AI models inevitably rises alongside it. Commenting on the trend, one market expert said, "AI model developers are currently caught between an intensifying performance race and relentless pricing pressure. The rapid proliferation of open-source and low-cost models continues to drive token prices lower, while research, development, and training costs for frontier models remain extraordinarily high."

"By contrast, AI infrastructure providers are strengthening their market position by absorbing soaring token demand amid persistent supply shortages," the expert continued. "Expanding AI data center capacity cannot be accomplished quickly because securing land, connecting to power grids, procuring transformers and cooling systems, and deploying high-density server infrastructure all require considerable time."

Major institutions have reached similar conclusions. In its Global Data Center Outlook released this year, U.S. commercial real estate services and investment management firm JLL stated that landlords continue to maintain a clear negotiating advantage in the data center market and projected that rental rates will continue rising through 2030. Goldman Sachs likewise concluded that the high barriers to entry associated with AI data centers—including power density requirements, advanced cooling systems, and elevated construction and operating costs—have strengthened the pricing power of infrastructure owners. Because meaningful supply expansion remains difficult in the near term, existing data center operators are benefiting simultaneously from high utilization rates and rising lease prices.

The AI infrastructure shortage has also begun reshaping labor markets. Surging investment in data centers has intensified shortages of skilled construction workers. In major U.S. data center hubs such as Northern Virginia and the Dallas-Fort Worth metropolitan area, projects are increasingly being delayed by several weeks due to shortages of qualified electricians, while substantial wage premiums have emerged. The New York Post reported in May that some young electricians working on U.S. data center projects are earning annual compensation of as much as USD 260,000. By comparison, the nationwide median annual wage for U.S. electricians stood at USD 62,350 in 2024, underscoring the extraordinary premium now being paid. Major technology companies including Meta and Google have also launched large-scale workforce training initiatives to secure skilled personnel needed to build and operate their own data centers.

A Turning Point for the Labor Market Approaches

Experts believe that once the current AI infrastructure crunch subsides and computing bottlenecks are resolved, the industry could enter an entirely new phase. As falling token prices begin translating into genuinely lower automation costs, the economics of workforce deployment could fundamentally change. Until now, the prevailing view across much of the industry has been that implementing AI often costs more than employing human workers. In April, Bryan Catanzaro, Vice President of Applied Deep Learning Research at Nvidia, told Axios, "Our team's computing costs far exceed employee compensation." During the same month, Uber Chief Technology Officer Praveen Neppalli told The Information that the company's AI budget had been exhausted within just a few months due to surging usage of AI coding tools such as Claude Code. "The budget exceeded our expectations by such a wide margin that we'll probably have to start planning from scratch," he said.

The inefficiency of current AI spending has also been documented by numerous studies and industry analyses. In January 2024, researchers at the Massachusetts Institute of Technology (MIT) analyzed the economic viability of AI automation in computer vision tasks and found that only about 23% of occupations generated a cost advantage through AI substitution. For the remaining 77%, human labor remained more economical once system deployment and operating expenses were taken into account, despite technical feasibility for automation. MIT reached a similar conclusion in its State of AI in Business 2025 report, noting that despite companies collectively spending between USD 30 billion and USD 40 billion on generative AI tools and systems, 95% of organizations had yet to realize measurable financial returns.

That equation, however, could change once large-scale data center expansion gathers pace and constraints on computing resource supply begin to ease. Automation efficiency is expected to improve rapidly across standardized functions—including repetitive administrative work, customer service, basic research, coding assistance, and document drafting and review—leading companies to evaluate AI deployment costs not merely as innovation investments but as operating expenses directly comparable with labor costs.