According to JPMorgan's July report, Data Center Watch, AI inference call prices and short-term rental rates for high-end GPUs showed synchronized declines at the time.

This trend directly indicates that the most acute phase of AI infrastructure strain is beginning to ease. For average users and enterprise clients, lower input Token pricing makes large model inference more affordable. For AI application companies and cloud providers, the retreat in H100 rental prices suggests compute markets are no longer characterized by universal queueing as during previous shortage periods.

Yet this is not a signal of declining demand. The tracked sample shows LLM Token processing volume has grown several-fold since the start of the year, with top-tier model utilization remaining in the 70%-80% range, and high-performance GPU utilization exceeding 80%. Price moderation appears more like a convergence of supply, open-source models, and inference efficiency catching up with demand—rather than a sudden weakening of AI demand.

By July 2026, input Token prices for mainstream models continued to decline. According to the report’s sample, the price per million input Tokens for certain mainstream models fell roughly within the $0.2–$5 range, representing an average monthly decrease of about 3% compared to June.

This range requires qualification. Output Token prices for high-performance closed-source models such as GPT-4o and Claude 3.5 Sonnet are typically significantly higher than their input counterparts; prices for lightweight models like GPT-4o mini cannot be directly compared with flagship models. Thus, the reported low prices primarily reflect input Tokens, lightweight models, or hosted provider sampling—not a uniform drop in total invocation cost across all mainstream models.

Even so, the direction remains clear. For AI application companies, unit inference costs are falling for equivalent tasks such as Q&A, code generation, search augmentation, or customer service calls. Over the past few years, one of the hardest financial puzzles in large model commercialization was “the more you use, the more you lose.” A downward trend in input Token unit pricing at least brings some high-frequency applications closer to a sustainable cost threshold.

More crucially, this price reduction occurred amid expanding usage volume. From early 2026 through July, LLM Token processing volume in the report’s sample grew several-fold, with certain niche segments increasing over fourfold. Top models including OpenAI, Anthropic, and Meta maintained stable utilization rates between 70%-80%, while the surge in Llama-series open-weight models played a significant role in driving increased demand.

The market is not witnessing “prices dropping because nobody is using” — rather, it’s seeing providers, open-weight ecosystems, and infrastructure supply collectively drive down per-unit pricing under growing scale.

The rapid growth of open-weight models emerged as another key theme in the July Token market.

The report shows that open-weight model Token usage rose 26% month-over-month in July and reached 63 times the level of the same period last year. Weighted-average transaction prices increased 36% month-over-month, and Token expenditure rose 71% month-over-month.

This indicates open-weight models are not merely undercutting closed-source competitors via low pricing. As performance increasingly approaches frontier levels, certain open-weight models are experiencing concurrent growth in invocation prices, usage volume, and spending.

In July, open-weight models’ share of total spending climbed to 20%, up from 12% in June and 10% in May. Although closed-source models account for only 29% of total Token volume, they still contribute 80% of spending—indicating high-performance closed-source models continue to capture most commercial value, yet open-weight models are rapidly closing the gap.

By Token usage volume, the top five models in July were MiMo v2.5, DeepSeek v4 Flash, GLM 5.2, DeepSeek v4 Pro, and MiniMax M3—collectively accounting for 50% of total Token volume.

By Token expenditure, the top five were Claude Opus 4.8, Claude Opus 4.7, Fable 5, Kimi K3, and GPT-5.6 Sol—representing 54% of total spending. Notably, Kimi K3 entering the top five by spending marks the emergence of open-weight models into high-value domains previously dominated by closed-source models.

For enterprise clients, expanded model options help reduce dependency on single closed-source vendors. Yet, as this report shows, the impact of open-weight models goes beyond price compression—it also includes competition for higher-value invocation scenarios and budgets.

The GPU leasing market has also exhibited similar structural divergence.

In July 2026, the average rental price for H100s among non-hyperscale cloud providers stood at $2.68 per GPU-hour, a 1.1% month-over-month decline—the first such drop after seven consecutive months of increases.

This decline is far less than 15%–25% and should not be described as a “20% drop in H100 rental prices.” The data tracks average monthly rental prices calculated per GPU-hour—not weekly spot rental rates for individual cards.

This signals a response from the supply side. More GPUs are entering the cloud leasing market, intensifying competition among cloud providers and compute platforms. Short-term rental prices are no longer dictated solely by extreme scarcity. Compared to the peak shortage phase, the pressure of unavailability, unrentability, and long queues has eased.

Yet high-end resources remain expensive. H100 prices remain significantly above A100 levels, overall GPU utilization stays between 75%–90%, and high-performance GPU utilization exceeds 80%. As long as utilization remains in this range, the rent decline likely reflects the elimination of some shortage premium—not a surplus of compute supply.

For AI companies, this shift brings two layers of impact.

First, inference operational costs become easier to reduce. Inference demands continuous, stable, low-latency compute—so a drop in GPU rental prices improves cost structures for API providers, AI search engines, code assistants, and enterprise AI agents.

Second, training cost pressures do not simultaneously disappear. High-end training depends not just on GPU count, but also on cluster interconnectivity, memory bandwidth, scheduling efficiency, and stable power supply. A decline in H100 rental prices does not equate to proportional reductions in large-scale training costs, nor does it mean all AI startups can access equally high-quality cluster resources at low cost.

The least relieved segment in this price relaxation is memory.

From May to June 2026, the report sample indicated a rapid rise in DRAM spot prices, with categories such as DDR5 16Gb rising over 90%-100% cumulatively—only stabilizing by July. In contrast, HBM prices remain elevated due to persistent AI accelerator demand, with contract prices lagging behind spot prices by 1–2 quarters.

For high-end AI accelerators, the GPU chip itself is not the sole bottleneck—HBM, advanced packaging, and data center delivery also affect real supply. Even if some GPU prices fall in the leasing market, upstream memory constraints and high-end cluster delivery will still limit the pace of overall cost reduction.

This explains why “the tightest moment of compute stress is easing” cannot be directly equated with “compute shortages have ended.” A decline in lease prices indicates relief from some short-term supply pressure. However, high-end training and large-scale inference clusters remain constrained by memory bandwidth, advanced packaging, and the rollout cadence of next-generation GPUs.

Timeframe matters. This report reflects price and utilization changes observed in the July 2026 sample—best interpreted as a snapshot of AI compute tension receding mid-year, not a comprehensive summary of prices across all cloud providers, regions, or long-term contracts.

Declining Token and GPU lease prices are favorable for AI application companies—but not necessarily positive news for model providers.

Lower invocation prices stimulate demand, expand API usage, and enable more enterprises to integrate large models into real business workflows. The issue arises when price declines outpace improvements in inference efficiency, squeezing profit margins for model vendors and cloud platforms. Especially amid rising competition from open-weight models and enhanced client bargaining power, the high-price premium of closed-source APIs faces continued pressure.

For hardware and cloud providers, the retreat in H100 short-term rental prices also signals the normalization of previously hyper-high returns driven by extreme scarcity. Differences in long-term contracts, regional variations, bulk volumes, and spot pricing methodologies result in significant disparities in actual prices across customers. Relying solely on average weekly rental rates cannot directly imply synchronous revenue declines across all cloud providers.

The clearest signal from this price tracking is that mid-2026, AI infrastructure supply has caught up with part of the demand—unit invocation and short-term rental costs are cooling. But boundaries remain clear: Token volume continues to grow rapidly, high-end GPU utilization remains high, output Token prices are more expensive, and HBM remains constrained. Cost reductions first appear in more market-liquid segments, while the hardest-to-relieve bottlenecks remain high-end clusters and upstream memory supply.

Original: LawDong BlockBeats