How new AI demands are fueling the data center industry in the post-cloud era

The increasing use of artificial intelligence (AI) means a rapid increase in data use and a new era of potential data center industry growth over the next two years and beyond.

This shift marks the beginning of the “AI Era,” after a decade of industry growth driven by cloud and mobile platforms, the “Cloud Era.” Over the past decade, the largest public cloud service providers and internet content companies propelled data center capacity growth to unprecedented levels, culminating in a flurry of activity from 2020 to 2022 due to the surge in online service usage and low-interest-rate financing for projects.

However, there have been significant shifts across the industry in the past year, including an increase in financing costs, build costs and build times, combined with acute power constraints in core markets. For example, typical greenfield data center build times have extended to four or more years in many global markets, roughly twice as long as a few years ago when power and land were less constrained.

Meanwhile, the largest internet companies are engaging in an accelerating race to secure data center capacity in strategic geographies. For each of the global technology companies, AI is both an existential opportunity and a threat with unique challenges for data center capacity planning. These dynamics are likely to result in a period of increased volatility and uncertainty for the industry, and the stakes and degree of difficulty of navigating this environment are higher than ever before. 

Flexible data center capacity planning can allow for changing inputs in rapidly changing markets. Looking back, the Cloud Era gave rise to a completely new set of market-propelling customers with different needs than previous generations. Industry players that were able to address these evolving needs won an outsized share during the last industry cycle. 

Key considerations for data center industry executives and their investors should be:

• Scenario planning to capitalize on the evolving needs of the market. 
• Proactive, yet flexible, strategies for market selection, facility design and other future decisions.

New era of buying data center capacity: Programmatic buying is over

During the Cloud Era, public cloud service providers became more sophisticated in forecasting the ramp-up of demand and adopted a more programmatic approach to procuring capacity. For several years, these buyers typically procured relatively standard amounts of third-party capacity structured with an initial commitment, followed by a reservation and a right of first offer for the same quantity. However, as demand eventually outpaced the original forecasts, cloud service providers (CSPs) had to return to the market for more capacity. 

Over the past two years, customer behavior has notably shifted: With the benefit of hindsight, data center customers are now increasingly willing to sign significantly larger deals, particularly in markets where power is currently relatively more available to avoid last-minute scrambles for more capacity and complicated footprints. They have also demonstrated a willingness to lease capacity at higher prices in markets where capacity is constrained. 

Key consideration for executives and investors: 

• Prior models and expectations may need adjustment to reflect this Evolution in customer buying behavior. 

Self-build data center development approaches are evolving

The largest cloud and internet companies, the hyperscale buyers in the data center industry, have historically preferred to build capacity themselves in markets where there is significant expected demand, potential economic advantage and manageable risk.

However, intense competition has led these players to rely more on leased capacity from third parties to get a more efficient route to market. In response, there are signs that the self-build strategy may be shifting.

Hyperscaler organizations acknowledge that it is unrealistic to self-build everything, and leasing will continue to play an important role in capacity procurement. As a result, hyperscalers are relying more heavily on leasing for speed to market advantage, while also considering smaller self-builds to potentially offset future demand. This suggests a potential increase in the total number of self-builds and a more heterogeneous mix of self-builds and leased capacity within cloud regions and even individual availability zones. For third-party suppliers, assessing the threat of potential future migration risk, given this dynamic, will be increasingly important. 

Key consideration for executives and investors: 

• The shifting mix of self-build vs. leasing across the industry and within specific local markets may alter the size of the addressable market, execution decision-making, and potential risks.

Increased power demand for AI workloads, cooling shift to liquid

AI workloads require power-hungry graphics processor units (GPU), resulting in much higher power density requirements within the data center. Currently, the AI market is relatively homogenous at the server infrastructure level, with Nvidia holding about 95% of the GPU market for machine learning (ML).

Therefore, the majority of high-end AI workloads are run on similar hardware: Specifically, chassis consisting of eight of Nvidia’s latest AI-specific GPUs (H100s), with each chassis consuming 5 to 6kW of power. Up to six chassis can fit in a single data center rack, resulting in total rack densities in the 30 to 40kW range, compared to approximately 10kW/rack densities for commodity public cloud workloads. 

As a result, hyperscalers and data center operators must find ways to effectively cool the equipment. Some major hyperscalers have announced plans to shift to liquid cooling solutions or raise the temperatures within their data centers to support these higher densities. 

Key considerations for executives and investors: 

• Current designs should support the future needs of power-dense workloads as densities shift over time.
• Selecting different cooling technology options may need to consider both economic and sustainability concerns.

Environmental, Social and Governance (ESG) demands

The data center industry ESG considerations are primarily focused on sustainability. To achieve their sustainability goals, industry participants have announced ambitions related to renewable energy usage, water usage and reduction of their carbon footprints. Data center operators are employing a variety of strategies, where available, to meet these goals:

• Efficiency improvements
  • Energy-efficient designs using technologies such as free cooling, efficient power distribution and efficient lighting systems
• Renewable energy usage
  • Procuring renewable energy from the grid
  • On-site renewable generation, including solar and wind
  • Power purchase agreements (PPAs) for long-term renewable energy, specifying amount and price
• Water usage
  • Air-cooled systems
  • Closed-loop water systems to reduce water use
  • Rainwater harvesting and water recycling
  • Water-free cooling, such as evaporative cooling or adiabatic cooling
• Carbon neutrality
  • Energy recovery using heat from IT equipment
• Waste reduction 
  • The ability to utilize these strategies will vary widely by market depending on local climate, local energy mix and other factors such as the need for worker safety. 

Key considerations for executives and investors: 

• ESG strategy should be differentiated from competitors. 
• An ESG strategy should strive to address desired, measurable change, or it may run the risk of being labeled as “greenwashing.”

AI plugins: Next wave of ecosystems

OpenAI has recently announced plugins to support third-party services, such as popular online ordering and reservation applications. These plugins are designed to help developers access and integrate external data feeds directly into OpenAI’s language model, allowing for more sophisticated training and prompting capabilities.

This new functionality could potentially reshape existing data center ecosystems around specific industries or data sources. As this dynamic evolves, it will be essential for operators to identify future “magnets” for these communities of interest and offer a relevant set of connectivity products to support the needs of these ecosystem participants. 

Key considerations for executives and investors:

• To support ecosystem development, the right set of products, partners and infrastructure is critical. 
• It is important to identify the highest-value customers in this new market environment and determine how the sales organization is equipped to target them.

Conclusion

The stakes have never been higher for data center industry participants to develop proactive, flexible strategies to navigate this new era and build the right data center capacity in the right markets. AI is driving increased data storage demand, which is positioned to outstrip supply in the near term. Builders, investors and users will benefit from flexible data center infrastructure strategies that can harness the AI revolution and lead to outsized growth.   

Gordon Bell is EY-Parthenon principal for strategy and transactions at Ernst and Young LLP.

Lillie Karch is EY-Parthenon senior manager for strategy and transactions at Ernst and Young LLP

The views reflected in this article are the views of the authors and do not necessarily reflect the views of Ernst & Young LLP or other members of the global EY organization.

Originally appeared on: TheSpuzz