AI Factories are Factories: Overcoming the Industrial Challenges of AI Commercialization

This article is part of VentureBeat Magazine’s special issue, “AI at Scale: From Vision to Feasibility.” Read more of the case here.

If you were to travel back 60 years in time to Stevenson, Alabama, you would find the Widows Creek Fossil Plant, a 1.6-gigawatt generating station featuring one of the tallest smokestacks in the world. today, There is a Google data center Where the Widows Creek factory once stood. Instead of being powered by coal, the facility’s aging transmission lines bring renewable energy to power the company’s online services.

This transformation, from a carbon burning facility to a digital factory, symbolizes a global shift to digital infrastructure. We are about to see intelligence production take off at full speed thanks to AI factories.

These data centers are decision-making engines that gobble up computing, networking, and storage resources while turning information into insights. Data centers are densely packed Appear in record time To meet the insatiable demand for artificial intelligence.

The infrastructure to support AI inherits many of the same challenges that have defined industrial plants, from power to scalability and reliability, requiring modern solutions to century-old problems.

The new workforce: the power of calculation

In the age of steam and steel, work meant thousands of workers operating machines around the clock. In today’s AI factories, production is determined by computational power. Training large AI models requires enormous processing resources. According to Aparna Ramani, Vice President of Engineering at… dead,The growth of training these models revolves around a Worker four a year Across industry.

This level of expansion is on track to create some of the same bottlenecks that existed in the industrialized world. There are supply chain constraints to get started. GPUs – the engines of the AI ​​revolution – come from a few manufacturers. It’s incredibly complex. They are in high demand. So it should come as no surprise that they are Subject to cost fluctuations.

In an effort to avoid some of these supply limitations, big names like AWS, Google, IBM, Intel, and Meta are designing their own custom silicon. These chips are optimized for power, performance, and cost, making them specialized with unique features for their workloads.

However, this shift is not limited to hardware only. There are also concerns about how AI technologies will impact the labor market. Research published by Columbia Business School He studied the investment management industry and found that AI adoption leads to a 5% decline in labor’s share of income, mirroring the shifts seen during the Industrial Revolution.

“AI is likely to transform many, perhaps all, sectors of the economy,” says Professor Laura Feldkamp, ​​one of the authors of the research. “I’m very optimistic that we will find meaningful employment for a lot of people. But there will be transitional costs.”

Where will we find the energy to expand?

Regardless of cost and availability, the GPUs that serve as the AI ​​factory’s workforce are notoriously power-hungry. When the xAI team brought the Colossus supercomputer cluster online in September 2024, it reportedly had access to between seven and eight megawatts of power from the Tennessee Valley Authority. But the 100,000 H100 GPUs in the lineup need much more than that. So, xAI brought in VoltaGrid mobile generators to temporarily make up the difference. In early November, Memphis Light, Gas & Water reached a more sustainable agreement with TVA to provide XAI with an additional 150 megawatts of capacity. But critics say the site’s consumption strains the city’s grid and contributes to poor air quality. And Elon Musk She already has plans For another 100,000 H100/H200 GPUs under the same roof.

According to MackenzieData center energy needs are expected to nearly triple current capacity by the end of the decade. At the same time, the rate at which processors multiply their performance efficiency is slowing. This means that performance per watt is still improving, but at a slow pace, and certainly not fast enough to keep up with the demand for computing power.

So, what will it take to keep up with the frenetic adoption of AI technologies? Report from Goldman Sachs He notes that U.S. utilities need to invest about $50 billion in new generation capacity just to support data centers. Analysts also expect data center energy consumption to drive about 3.3 billion cubic feet per day of new natural gas demand by 2030.

Scaling becomes more difficult as AI factories increase in size

Training the models that make AI factories accurate and efficient can take tens of thousands of GPUs, all running in parallel, for months at a time. If the GPU crashes during training, The operation must be stopped, returned to a fresh checkpoint, and resumed. However, as AI factories become more complex, the probability of failure also increases. Ramani addressed this concern during AI Infra@Scale presentation.

“Stopping and restarting is very painful. But what makes it worse is that as the number of GPUs increases, the probability of failure also increases. At some point, the volume of failures can become so massive that we lose a lot of time to mitigate these failures, and you barely finish Training tour.

According to Ramani, Meta is working on near-term ways to detect failures sooner and get back to work more quickly. In the future, research into asynchronous training may improve fault tolerance while simultaneously optimizing GPU utilization and distributing training runs across multiple data centers.

Always-on AI will change the way we do business

Just as factories in the past relied on new technologies and organizational models to scale the production of goods, AI factories feed on computing power, networking and storage infrastructure to produce tokens. The smallest piece of information used by the AI ​​model.

“This AI factory is creating, creating and producing something of great value, a new commodity,” Nvidia CEO Jensen Huang said during his conference. Computex 2024 Keynote. “It’s completely replaceable in almost every industry. That’s why it’s a new industrial revolution.”

McKinsey says generative A.I He has the ability to add Equivalent to $2.6 to $4.4 trillion in annual economic benefits across 63 different use cases. In every application, whether the AI ​​factory is hosted in the cloud, deployed at the edge or self-managed, the same infrastructure challenges must be overcome, as with an industrial factory. According to the same McKinsey report, achieving a quarter of this growth by the end of the decade will require another 50 to 60 gigawatts of data center capacity to get started.

But the result of this growth is poised to change the IT industry indelibly. Huang explained that AI factories will enable the IT industry to generate information worth $100 trillion from industry. “This will be a manufacturing industry. It’s not a computer manufacturing industry, but using computers in manufacturing. This has never been done before. It’s completely unusual.”

Daily insights into business use cases with VB Daily

If you want to impress your boss, VB Daily has you covered. We give you the inside scoop on what companies are doing with generative AI, from organizational transformations to hands-on deployments, so you can share insights to maximize ROI.

Read our privacy policy

Thanks for subscribing. Check more VB newsletters here.

An error occurred.