Rapid swings in AI data centers\u2019 power demands are straining vital equipment, causing batteries, generators and cooling systems to malfunction or wear out far sooner than expected.<\/p>\n
As the AI boom accelerates, these technical problems suggest added costs and unforeseen reliability problems, with even a few minutes of lost uptime hitting data-center developers\u2019 revenue. They come at a time when investors and lenders are already jittery about hyperscalers\u2019 hundreds of billions of dollars of spending, amid growing concerns that these facilities could be depreciating much faster than estimated. <\/p>\n
The problems are also a potential source of wider instability in power grids that are already straining to keep the lights on.<\/p>\n
\u201cAI does create very unusual power demand,\u201d said Amber Villegas-Williamson, principal consultant at the Uptime Institute in the UK, which advises electricity suppliers and data centers on standards and reliability. \u201cIt\u2019s like over-revving your car wears out the engine faster than keeping a constant speed.\u201d<\/p>\n
Data centers have been around for decades, gulping down electricity while they ensure that everything from your favorite streaming show to your online grocery order functions smoothly. But facilities designed for AI computing are different because their demand is so large and swings much more dramatically. <\/p>\n
Power increments equivalent to the consumption of factories, towns or even cities can appear and disappear within seconds, creating repeated shocks that connected equipment struggles to absorb. <\/p>\n
A gigawatt data center is equivalent to a city the size of Boston, half of which can flicker on and off every few seconds, said Shannon Miller, founder and president of Mainspring Energy Inc., which works on micro-grid projects for industrial and data-center customers. Some AI campuses planned in Texas, the Midwest and other states are more than five times bigger, consuming nearly as much power on average as New York City.<\/p>\n
AI data centers put particular strain on their power supply when they are training new models \u2014 a process that mobilizes all of the graphics processing units in unison. Like the digital equivalent of bees swarming or a school of fish changing direction, hundreds of thousands of GPUs can power up and down on a millisecond basis.<\/p>\n
AI at times sees power usage spike as much as 50% above its design capacity, \u201cso a 1 gigawatt facility may use 1.5 gigawatts for a split second,\u201d said Drew Baglino, a former Tesla Inc. executive who started Heron Power Electronics Co. The company is developing equipment to manage power fluctuations for Nvidia Corp.\u2019s even more energy intensive next-generation of servers, due in 2027. <\/p>\n
Most equipment isn\u2019t designed for such big swings in power consumption. Jon Parrella, chief executive officer of energy-storage developer Terraflow Energy, likens it to driving a Ferrari and shifting straight from sixth gear to first. \u201cYou can\u2019t swing that fast,\u201d he said. <\/p>\n
This story is based on interviews with more than three dozen power experts in the US and Europe, including generators and other power suppliers, data-center developers, grid operators, utilities, investors, standards developers, insurers and regulators, almost all of whom said the physical stresses on the facilities were evident. <\/p>\n
Cranks on small natural gas combustion engines used to generate power at data centers have broken off, several of those people said. At xAI\u2019s Colossus computing facility in Memphis, Tennessee, gas-fired turbines had developed cracks, one of the people said. Batteries were installed within the system to help smooth out power swings and reduce the strain on spinning turbines, the person said.<\/p>\n
SpaceX, the parent company of xAI, didn\u2019t respond to requests for comment.<\/p>\n
Turbines have also cracked at much smaller data centers in the UK, said Andrew Cunningham, CEO of GeoPura Ltd., which is providing hydrogen for use in fuel cells that smooth out power flows at some sites.<\/p>\n
Cracks or wear on devices can cause electrical arc flashes \u2014 when a current jumps between conductors \u2014 potentially damaging AI chips, said Jennifer Scanlon, CEO of UL Solutions Inc., which tests and certifies new technology.<\/p>\n
There is a suite of equipment such as batteries, capacitors, transformers and flywheels that can help stabilize power flows. However, in the bid to build AI computing capacity quickly not enough of these technologies are being used at new data centers, several of the people said. Batteries that have been installed for this purpose have sometimes needed to replaced within months or even weeks due to the high strain, according to the Uptime Institute and other people working with operators.<\/p>\n
This problem is seen at data centers around the world, from the Middle East and Africa, to Europe and the US, said Villegas-Williamson of the Uptime Institute. <\/p>\n
Reliability Problems<\/p>\n
These issues are already causing delays or curtailing operations \u2013 and therefore revenue \u2013 at some AI computing facilities. <\/p>\n
To ensure that a planned 2.67-gigawatt AI campus in West Texas can achieve the 99.999% reliability required by Microsoft Corp., extra time was baked into the schedule for engineering, said Chris James, CEO of Joulent Inc., which is developing the facility with energy giant Chevron Corp. This means power delivery will begin in 2028 instead of 2027, he said. <\/p>\n
If essential equipment breaks down prematurely, \u201cthe financial consequence is not primarily replacing a pump or a breaker or some power component \u2014 it\u2019s the the value of that expensive compute capacity not generating revenue because it\u2019s offline,\u201d said Jason Hoffman, chief strategy officer at data-center builder and operator Switch.<\/p>\n
The cost of downtime in terms of lost revenue varies widely, with estimates ranging from thousands to hundreds of thousands of dollars per minute, depending on the type of facility and its workload.<\/p>\n
Data centers are built on the assumption that, once they are online, they will operate around the clock 365 days a year, said a person involved in the financing of such facilities. In reality, some are seeing uptime closer to 80%, and unless resolved this could hit investors in certain projects in the next 12 to 24 months, the person said. <\/p>\n
Any issues with reliability add to wider concerns about the returns generated from hundreds of billions of dollars of planned AI investments. The rate of depreciation of another crucial piece of equipment at data centers, the GPU racks themselves, has raised questions about whether the industry can be as profitable as it promises. <\/p>\n
These problems with reliability also have the potential to destabilize the wider power grid. This sprawling web of high voltage lines, transformers and power plants requires constant calibration, something that has become more challenging with each passing year due to aging equipment, rising demand and extreme weather.<\/p>\n
The expansion of intermittent wind and solar generation, which often result in big swings in supply from one hour to the next, are already a destabilizing force. AI data centers can put this volatility on steroids. <\/p>\n
\u201cThese loads are extremely dynamic or fluctuating, which causes grid instability and can lead to, if not corrected, potential blackouts or power outages,\u201d said Sreemant Roy, a power-quality expert and global offer manager at Schneider Electric in Nashville, Tennessee. Of particular concern is a data center\u2019s ability to cause sub-synchronous oscillations in the power flow, which can damage equipment connected to other parts of the network, he said. <\/p>\n
\u201cThat has made utility companies globally very worried,\u201d Roy said. <\/p>\n
Within the last two years, the North American Electric Reliability Corp. \u2014 the top US regulatory body establishing standards aimed to keep the lights on \u2014 has repeatedly warned and issued alerts that data centers are one of the greatest risks to grid stability.<\/p>\n
NERC evaluated more than 33 gigawatts of operational data centers in the US an found about three quarters of their load models \u201care insufficient to represent data-center dynamic behavior,\u201d according to a September report. Earlier this year, the agency issued a rare level-three alert requiring big data centers to address these immediate risks and submit their responses by Aug. 3.<\/p>\n
From top to bottom, the AI industry is aware of these issues and actively working on solutions. <\/p>\n
Nvidia started working more closely with power experts when it developed Blackwell GPUs, which were first released in 2024 and have become pervasive in data centers, said Dion Harris, senior director of hyperscale infrastructure solutions at the chipmaker. <\/p>\n
\u201cWe\u2019re building the chips and processors\u201d but also using them in the company\u2019s own data centers, Harris said. Nvidia is working to make its deployments a lot smoother \u201cboth on the data-center build out, design and engineering phase, as well as on the power delivery.\u201d <\/p>\n
Data-center users have deployed techniques to smooth out the power fluctuations of AI workloads by running side computations \u2014 essentially dummy math that isn\u2019t part of the training process \u2014 to keep GPUs operating steadily. However, this approach has been criticized for wasting electricity at a time when power demand is surging. <\/p>\n
Last year, the National Laboratory of the Rockies near Denver, Colorado set up a test-bed on behalf of the US Department of Energy to figure out how to integrate AI safely onto the grid, said Martha Symko-Davies, the NLR\u2019s program manager for the DOE\u2019s office of electricity. <\/p>\n
The site has GPUs and power generation on site that developers can use to figure out if their setup can handle the variability of AI. One power supplier said they will use the facility to test batteries, software and other equipment intended to smooth out oscillations between the data center and grid that can cause damage on either side.<\/p>\n
\u201cWe have the opportunity right now to get it right,\u201d said Symko-Davies.<\/p>\n
Malik writes for Bloomberg.<\/p>\n","protected":false},"excerpt":{"rendered":"Rapid swings in AI data centers\u2019 power demands are straining vital equipment, causing batteries, generators and cooling systems…\n","protected":false},"author":2,"featured_media":131701,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[24,20995,4830,25,1067,65730,65732,16963,2010,17218,32239,1509,11237,4554,65731,156,573],"class_list":["post-131700","post","type-post","status-publish","format-standard","has-post-thumbnail","category-ai","tag-ai","tag-ai-boom","tag-ai-datum-center","tag-artificial-intelligence","tag-battery","tag-connected-equipment","tag-electricity-supplier","tag-facility","tag-gpu","tag-grid","tag-hundred","tag-revenue","tag-thousand","tag-time","tag-uptime","tag-us","tag-year"],"_links":{"self":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/131700","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/comments?post=131700"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/131700\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media\/131701"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media?parent=131700"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/categories?post=131700"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/tags?post=131700"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}