Why the AI Industry’s Thirst for New Data Centers Can’t Be Satisfied
Supply bottlenecks slow the scramble to build bigger, more powerful facilities
By Tom Dotan
Follow
and Asa Fitch
April 24, 2024 5:30 am ET
The frenzy to build data centers to serve the exploding demand for artificial intelligence is causing a shortage of the parts, property and power that the sprawling warehouses of supercomputers require.
The lead time to get custom cooling systems is five times longer than a few years ago, data center executives say. Delivery times for backup generators have gone from as little as a month to as long as two years.
A dearth of inexpensive real estate with easy access to sufficient power and data connectivity has builders scouring the globe and getting creative. New data centers are planned next to a volcano in El Salvador and inside shipping containers parked in West Texas and Africa.
Explore Audio Center
Earlier this year, data-center operator Hydra Host found itself in a bind, searching for 15 megawatts of power needed to operate a planned facility with 10,000 AI chips.
The company went from Phoenix to Houston to Kansas City, Mo., to New York to North Carolina to find the right space. It is still on the hunt.
The locations that had the power didn’t have the right cooling systems required to keep the servers operational. New cooling systems would take six to eight months to arrive, thanks to a supply crunch. Meanwhile, buildings that had the cooling didn’t have the transformers required to receive the additional power—those would take up to a year to arrive.
“With what we’re seeing, the fervor to build is probably the greatest since the first dot-com wave,” said Hydra Host Chief Executive Aaron Ginn. He said the search for the right parts and space has taken months longer than expected.
The demand for computational power to create AI systems has surged since late 2022, when OpenAI’s ChatGPT started showing the technology’s potential. Demand for computer servers equipped with new generations of AI chips—the most popular of which are graphics processing units, or GPUs, from Nvidia—is overwhelming existing data centers.
“You had this tsunami, and there’s going to be a shortage of data-center inventory,” said Raul Martynek, chief executive of data-center company DataBank.
Creating and deploying complex AI systems requires unprecedented numbers of chips. Analysts estimate that training the version of ChatGPT that came out in 2022 required more than 10,000 of Nvidia’s GPUs, while more recent updates have required significantly more—putting further strain on data centers. Large tech companies have struggled to get their hands on supplies.
The amount of data-center space in the U.S. grew 26% last year, according to real-estate firm CBRE, and a record amount was under construction. The price of available space is rising while vacancy rates are negligible—a sign that supply isn’t keeping up with demand.
Bill Vass, vice president of engineering at Amazon Web Services, said a new data center pops up somewhere in the world every three days.
It generally takes a year and a half or two years to put up a large, new data facility, said Jon Lin, the general manager of data-center services at Equinix, one of the world’s biggest data-center operators. It is difficult for the industry to suddenly scale up when demand skyrockets because of the extensive planning and supply-chain management required, he said.
“These are not easy problems where you can pivot on a dime and triple capacity very quickly,” said Lin.
Cloud giants like Amazon Web Services, Microsoft and Alphabet’s Google are investing billions of dollars in new data centers. Google’s capital expenditures—almost half of which was on its data infrastructure—jumped 45% from a year earlier to $11 billion in the three months through December.
Microsoft has tried to control costs during its data-center build-out by pulling back on spending in other areas. The company spent more than $30 billion on data centers in 2023, according to an estimate from market-research firm Dell’Oro Group.
The rush to build has ratcheted up the time needed to acquire some critical data-center components. Transceivers, which connect different networks of servers, now take months longer to arrive than before. Labor costs have also become an issue as data-center builders have faced a shortage of construction workers trained on these types of sensitive installations. Getting smaller networking components, like cables that connect different racks of servers, can take months.
The AI chips needed to make the technology work are famously in short supply. The single biggest maker, Nvidia, has been overwhelmed by demand from a range of companies and cloud-infrastructure providers. At the outset of the AI boom, the lead time to get graphics processing units for AI computation in data centers rose to several months, though the wait time has fallen in recent months.
The time it takes to get large backup power generators has gone from less than a year to almost two years, data-center executives say.
The specific requirements for the new GPU-driven AI data centers are pushing builders to look for places where they can get lots of reliable, affordable electricity. Amazon recently bought a data center next to a nuclear power plant in Pennsylvania. Meta Platforms is planning $800 million of computing infrastructure in El Paso, Texas.
SHARE YOUR THOUGHTS
What steps should be taken to meet the growing demand for data centers? Join the conversation below.
Digital-infrastructure company Standard Power is planning to supply power to data centers in Ohio and Pennsylvania with modular nuclear reactors similar to those used to power some submarines and aircraft carriers. NuScale Power, based in Portland, Ore., says it has the approval to supply the reactors to Standard.
“The data-center market needs power yesterday,” said Clayton Scott, NuScale’s chief commercial officer.
Armada, a San Francisco startup, builds data centers inside of shipping containers. The company can drop these portable facilities, full of Nvidia chip-powered servers, in locations such as remote areas of Texas or Africa that are near inexpensive sources of power like gas wells.
El Salvador—after trying to build a better business environment with a crackdown on gangs—has slashed taxes on AI. Marc Seal, who runs a company that invests in AI data centers, is considering tapping the country’s volcanoes to power such data centers using green, geothermal power.
Data center development is booming across the U.S. thanks to AI. Some industry analysts estimate global capacity to double by 2030. But it faces a big obstacle: getting enough power. Graphic: Ryan Trefes, JLL
Write to Tom Dotan at tom.dotan@wsj.com and Asa Fitch at asa.fitch@wsj.com
