Driving AI performance: Insights and strategies from the DCD Broadcast: AI Week 2024

At DatacenterDynamics (DCD) Broadcast: AI Week 2024 from July 29 to August 2, Vertiv's leadership and partners addressed how the evolution of artificial intelligence demands more from high-performance computing (HPC) infrastructure. The event was a forum for exchanging insights on overcoming complexities in selecting appropriate technologies and guiding decision-makers in building resilient AI systems from chip to grid.

The executive perspective on the data center AI revolution

Vertiv CEO Giordano Albertazzi and DCD CEO Dan Loosemore opened DCD Broadcast: AI Week 2024 to discuss the rapidly evolving AI landscape. Albertazzi shared his insights, noting, "The future is challenging to predict, but one thing is certain-acceleration is happening." This statement set the stage for a conversation focused on the necessity of speed and flexibility in designing systems that can adapt to the continued evolution of artificial intelligence.

Emphasis on design for the evolution of artificial intelligence

Albertazzi emphasized the critical importance of design adaptability, stating, "You must have a design that enables the evolution of artificial intelligence." He acknowledged the inherent challenges in preparing for an unpredictable future, urging the importance of flexibility to accommodate the dynamic nature of technological advancement.

Holistic approach to data center systems

A key point of Albertazzi's message was the necessity for a holistic approach in data center management, especially coordinating power and thermal management systems. He advised, "Think about the entire system, not just the liquid cooling." Albertazzi championed the idea of efficient heat reuse amid increasing high-density racks, underscoring designs to manage thermal challenges in such areas.

The future of power infrastructure

Looking toward the future, Albertazzi discussed the evolving power infrastructure requirements to effectively support AI data center demands. He highlighted the need for a power infrastructure that is "much more agile, intelligent, and integrated," reflecting Vertiv's commitment to these ideals. His vision includes data centers as community assets that support local energy production and help balance grid demands without causing disruptions.

Innovation through collaboration

Albertazzi is optimistic about collaboration between the silicon and infrastructure sectors to foster innovation. He emphasizes the importance of partnerships with companies like Nvidia and Intel, enabling aligned mutual roadmaps and the achievement of each other's goals: "No single player can achieve these goals in isolation." He reaffirmed Vertiv's pride in contributing to industry efficiency and integration.

[Watch "The executive perspective on the data center AI revolution" on demand]

Develop grid resilience by achieving power independence

Kyle Keeper, Vertiv's Senior Vice President for Power Management, stated on the second day of the DCD Broadcast: AI Week 2024 that the data center industry faces ever-growing power challenges from AI workloads and increasing requirements to decarbonize operations.

BYOP innovation

Arturo di Filippi, Offering Director for Large Power at Vertiv, introduced "bring your own power" (BYOP) to address points with power availability. Marinella Massacesi, Vertiv's Business Development Manager in EMEA for Energy Storage, added, "BYOP means that if customers can't access the power needed to run their facility, we can help them."

Applying the BYOP strategy, the Vertiv™ Dynamic Power portfolio of solutions includes battery energy storage systems (BESS), uninterrupted power supply (UPS) systems, and fuel cells to help customers address power and decarbonization challenges. The panel discussed practical examples and case studies where these technologies enhanced energy efficiency and resilience. Furthermore, these solutions can pave the way for customers to be good grid corporate citizens and even generate revenue by supporting the grid.

Solutions beyond the grid

The panel has established that a one-size-fits-all solution to power challenges doesn't exist. For addressing the specific needs of the facility, it is crucial to have a holistic and comprehensive view of the data center design. "We need to speak about the entire ecosystem or the power train because the challenges are not just applicable to the load itself or even one specific piece of equipment. They are applicable and present across the entire power train. We, as Vertiv, can provide all the products going from the grid to the road itself. And for each of them, we must understand the main points we must face and address," emphasized di Filippi.

To support operators and help address their challenges, Massacesi extends an invitation: "What we want is a one-to-one discussion with our customers. We need to understand the challenges and problems they are facing in terms of critical loads protection as well as energy efficiency, access to power, or intelligent power management."

[Watch "Develop grid resilience by achieving power independence" broadcast on demand]

How to increase facility and rack densities

The third day of DCD Broadcast: AI Week 2024 was focused on the critical challenge of densifying data centers for AI workloads.Vertiv Senior Solutions Architect John Menoche, Product Manager for the Infrastructure Solutions Team Aaron Badowsk, and DCD's Senior Editor Dan Swinhoe, discussed the rising power consumption of GPU chips. Menoche noted, "The overall density has increased substantially, in some cases by an order of magnitude in the last few years." Now, at 400 to 600 watts, this demands advanced cooling and power management strategies.

Cooling challenges in high-density racks

Badowski emphasized the exponential growth in computing power alongside infrastructural challenges, referencing Moore's Law and Nvidia's advancements. He pointed out the need to rethink traditional cooling methods, explaining, "There's great exponential growth on the chip side, but then on the infrastructure side, there's this big inflection point." As conventional air cooling reaches its limits with rack densities exceeding 100 kW, direct liquid cooling is becoming the preferred method for enhanced heat rejection.

Evolving for efficiency

Looking to the future, the panelists discussed the implications for data center operators. Menoche stressed the importance of modular construction and strategic placement of power and cooling infrastructure, noting that existing facilities may require extensive retrofits. Badowski added, "Customers have just accepted the inevitability that liquid will end up in the data center," reflecting a growing industry consensus on the need to adapt for a competitive advantage in an AI-driven environment.

[Watch "How to increase facility and rack densities" on demand.]

Cooling AI technologies

Vertiv's Business Development Director for Sales Greg Stover, Nvidia's Senior Mechanical Engineer Bahareh Eslami, and Microsoft's Director for Systems Technology Husam Alissa discussed the evolving landscape of thermal management in AI technology. They highlighted a significant shift from theoretical discussions on advanced cooling solutions to practical implementations to meet the thermal demands of high-density AI chips. Stover emphasized the urgency of scaling liquid cooling solutions globally, noting that the focus has shifted from advocating for these technologies to effectively executing them in data centers.

Addressing heat loads

The increasing heat loads from AI applications present significant technical challenges, necessitating a transition from traditional air cooling to efficient liquid cooling systems for racks operating at 500 kilowatts and beyond. Eslami emphasized the importance of using modeling and simulation tools to design effective cooling strategies that can adapt to rapid advancements in AI technology, ensuring robust and scalable solutions.

Complex thermal challenges

Thermal management poses complex challenges, from chip design to HPC infrastructure. Alissa highlighted the need for collaboration among industry stakeholders to develop innovative AI cooling solutions, such as cold plates and immersion cooling, that can meet reliability and decarbonization requirements.

The discussion underscored the necessity for continuous collaboration and adaptable cooling strategies to address the evolving challenges posed by the evolution of artificial intelligence. The panel emphasized that strategic partnerships are crucial for developing scalable data centers to meet the growing AI data center demand.

[Watch "Cooling the cutting-edge of AI technologies" on demand.]

The future of AI and HPC

The final day of DCD Broadcast: AI Week 2024 featured a distinguished panel discussing "The future of AI and HPC." Panelists Stephen Liang, Chief Technology Officer and EVP Infrastructure & Solutions, and Martin Olsen, Vice President of Global Product Strategy at Vertiv, as well as David McCall, Vice President of Innovation at QTS Data Centers, explored how these technologies are advancing and the implications for the future.

The shift from CPUs to GPUs

Liang highlighted the evolution of artificial intelligence, particularly emphasizing the significant shift in AI capabilities brought about by GPU parallel processing. This advancement has drastically shortened the time required to complete complex tasks, reducing what would have taken years to just a few hours. Consequently, the demand for HPC infrastructure, including high-density racks, AI cooling systems, and efficient power trains, has been on the rise.

Olsen provided a historical perspective on the shift from CPUs to GPUs in computing, noting that while GPUs dominate due to their efficiency in parallel tasks, CPUs remain vital for general computing. The discussion also underscored the necessity of industry standards and benchmarks to implement rapid technological advancements effectively. The panelists stressed that collaboration across the tech stack-from chip manufacturers to data center operators-will be crucial for progress as AI and HPC grow.

Industry standards and collaboration

The panel emphasized the symbiotic relationship between AI and HPC, stressing their importance in fueling future innovations. They pointed out that energy-efficient hardware, market trends, and geopolitical influences will significantly shape advanced computing in the years ahead. We can address challenges and seize opportunities by promoting open dialogue and collaboration among industry players.

[Watch "The future of AI and HPC" on demand.]

Explore the AI transformation

The DCD Broadcast: AI Week 2024 highlighted the current advancements in AI and HPC and outlined a pathway for their future, motivating businesses to engage with these rapidly changing technologies. The question isn't if AI will come to your business, but when-if it isn't already there.

To access resources to transform your data center and unleash AI's potential with expert guidance, visit Vertiv AI Hub: Transforming Modern Data Centers To Support AI and HPC Demands.

Public link

Please use the above public link if you want to share this noodl on another website.