China's AI Compute Faces Scarcity Amid Low Utilization
- tech360.tv
- 1 hour ago
- 3 min read
China's artificial intelligence compute landscape exhibits a fundamental contradiction, according to a recent JPMorgan report. While major cloud providers face acute scarcity of high-end processing, newly established local AI computing centres report significantly low utilisation rates, sometimes as low as twenty percent.
A report by JPMorgan recently indicated that up to eighty percent of data centres across China could be sitting idle. This striking estimate, however, conflates several distinct types of infrastructure. It combines traditional internet data centre hosting facilities, general cloud computing services, large supercomputing centres, and the AI-specific computing centres constructed over the past two years. Lumping these fundamentally different categories together presents a potentially misleading overall picture of the nation's total AI capacity.
But the figure is not without significance. It highlights a genuine structural issue within China's AI compute environment. On one hand, major cloud providers, including Alibaba Cloud, Tencent Cloud, and Huawei Cloud, currently experience an acute scarcity of advanced compute resources. On the other, dozens of recently built local AI computing centres operate with reported utilisation rates of between twenty and thirty percent.
The disparity between paper compute capacity and its actual effective output continues to widen across the country. For example, a cluster comprising one thousand graphics processing units, if equipped with inadequate interconnect bandwidth and a rudimentary scheduling system, may deliver less true throughput than a well-tuned two hundred GPU cluster. Many local computing centres were primarily developed as infrastructure assets, functioning as investment vehicles connected to land, subsidies, and industrial park branding, rather than as direct responses to genuine market demand from the industry.
And the market's centre of gravity is experiencing a pronounced shift. Its focus now moves increasingly from AI model training to inference. Training activities can be economically centralised in western hubs that benefit from cheaper green energy sources. Inference operations, however, necessitate close proximity to end-users and specific business scenarios to function efficiently. This fundamental change in market dynamics means many computing centres built predominantly for training now confront an awkward operational reality: the demand has relocated, but the supporting physical infrastructure has not adapted accordingly.
The deeper bottleneck for domestically produced AI chips does not primarily concern single-chip performance alone. Instead, it encompasses the entire engineering ecosystem required to support these components effectively. CUDA remains the industry's de facto standard for parallel computing. The substantial disparity between "usable" and "production-ready" domestic alternatives is still measurable in terms of required developer hours, additional migration costs, and persistent operational uncertainty for businesses.
So China's real challenge in the AI compute sector is not merely about constructing additional physical capacity. It involves organising the vast existing capacity into a coherent, efficient, and readily accessible national computing network. This undertaking requires more than simply adding more hardware; it necessitates comprehensive system-level integration, advanced scheduling, and optimisation across the entire landscape. This approach aims to maximise the utility of current resources rather than focusing solely on expansion.
China's AI compute landscape presents a structural contradiction of high-end scarcity and low local utilisation.
A JPMorgan report initially estimated eighty percent of Chinese data centres were idle, but this conflates different infrastructure types.
The market is shifting from training to inference, leaving some training-centric infrastructure poorly suited to current demand.
Domestic AI chip development faces an engineering ecosystem bottleneck, not just single-chip performance.
The primary challenge involves organising existing compute capacity into an efficient national network.
Source: Pandaily
