Immersion cooling systems: Advantages and deployment strategies for AI and HPC data centers

Data center infrastructure is rapidly expanding, fueled by the ongoing rise of artificial intelligence (AI) and high-performance computing (HPC) workloads. As rack densities continue to increase, operators evaluate their liquid cooling strategies and plan for future capacities of up to 500 kilowatts (kW) per rack. According to Dell'Oro Group, the worldwide data center liquid cooling market grew 52% in 2023, to $745 M, accounting for 12% of total data center thermal management revenues (see Figure 1).

Figure 1. Dell'Oro Group predicts that the liquid cooling market will grow at a compound annual grow rate (CAGR) of 45% from 2023 to 2028, reaching over $4.8B in annual revenue. This growth is driven by the expansion of data center liquid cooling from niche HPC deployments to mainstream cloud, colocation, and enterprise data center deployments. By 2028, they expect liquid cooling to account for 36% of total data center thermal management revenues.

While rear door heat exchangers (RDHxs) and direct-to-chip cooling manage current heat levels, the increasing densities suggest additional cooling solutions. Data center immersion cooling (or "liquid immersion cooling") is an energy-efficient option that offers superior cooling for high-density workloads.

Data center operators are evaluating liquid cooling technologies to increase energy efficiency as processing-intensive computing applications grow.

Liquid cooling options for data centers

Understanding immersion cooling

Immersion cooling (see Figure 2) is a liquid cooling method in which servers and other rack components are submerged in a thermally conductive dielectric liquid or fluid within a sealed tank. This liquid, known for its excellent thermal transfer properties, absorbs heat from IT equipment for efficient HPC cooling.

Figure 2. Vertiv™ CoolCenter Immersion is an example of an immersion cooling tank. The IT equipment is placed horizontally in a sealed tank resembling a server rack on its back, following a typical 19-inch standard. Servers are inserted from the top and positioned vertically.

The absorbed heat is then pumped to a coolant distribution unit (CDU) containing a brazed plate heat exchanger. The primary cooling loop, connected to a chiller, dry cooler, or waste heat recovery system, is separated from the secondary loop containing the dielectric liquid. This setup effectively transfers heat outside the building, enabling cost-efficient and effective cooling.

Components of immersion cooling systems

Data center immersion cooling involves several essential components crucial for a successful deployment. Key components of the liquid immersion cooling system include:

• Tank: Sealed containers that hold IT equipment and dielectric liquid. These tanks contain various sensors, a power distribution unit (PDU) and other components.
• CDU (see Figure 3): Circulates and pumps coolant in a closed-loop system within the tank from the chilled water supply to cool the servers. These units include the following components:
  • Heat exchanger -
  • Pumps - Circulate the liquid into the tank around the servers, removing heat from the IT components and heating up the fluid. The liquid to the brazed plate heat exchanger to cool down before passing through a filter and returning to the tank to start the cycle again.
  • Flow modulating valve - Adjusts the cooling capacity of the CDU to match the heat load within the tanks, enabling optimal thermal management and efficiency.
  • Filter - Removes contaminants and impurities from fluids to ensure optimal operation and longevity of systems.
  • Power supply - Enables seamless power transition with an integrated AB transfer switch.
• Piping: Connects the tank to the CDU to facilitate the circulation of the dielectric liquid.

Figure 3. The Vertiv™ CoolCenter Immersion CDU manages and distributes chilled liquid to immersion cooling systems, efficiently removing heat from submerged data center equipment.

Efficiency and performance of immersion cooling
for HPC

Immersion cooling can enhance data center performance by efficiently managing high heat outputs. In AI and HPC workloads, immersion cooling systems have delivered cooling capacities of up to 100 kW per tank (42 or 52 rack units (U)). This technology can provide an effective solution for intense computational demands.

Cooling capacity of immersion systems

Immersion cooling absorbs 100% of the heat from IT components since they are fully submerged in the fluid, minimizing the need for air cooling units and reducing heat transfer steps. This system's capacity to handle all heat from components eliminates residual heat in the air. Moreover, the fluid used in immersion cooling operatesat higher temperatures than direct-to-chip cooling, allowing longer free cooling periods. When using a chiller, the smaller temperature difference can improve energy efficiency.

However, room cooling units will remain essential in a data center to maintain air quality. Controlling humidity and using a filtration system can prevent dust build-up, facilitate clean air circulation, and maintain breathable conditions by reducing excess CO².

Environmental impact

Immersion cooling systems often use a mineral-based dielectric liquid from natural gas. Natural gas can be used as a source material to create a dielectric fluid through the "gas-to-liquids" process, where it is converted into a synthetic liquid suitable for immersion cooling applications. Although the chemical industry purifies and converts this oil into a new dielectric liquid, this prevents CO² emissions. The immersion cooling liquid is not consumed, as is the case with natural gas in electrical power plants.

Data center operators aiming for an oil and natural gas-free immersion cooling liquid can use bio-based dielectric liquids as an alternative. These liquids, derived from biological waste, reduce reliance on mineral oil and natural gas, offering a more environmentally conscious option.

Unexpected benefits

Immersion cooling can offer a quieter data center environment. Unlike the noise prevalent in air-cooled centers-due to extensive air recirculation-immersion cooling is predominantly silent, with only the soft hum of pumps, significantly reducing overall noise levels.

Vertiv™ AI Hub:

Vertiv™ 360AI includes a complete portfolio of power, cooling and service solutions that solve the complex challenges arising from the AI revolution.

Not all data centers requiring liquid cooling solutions are greenfield installations. When retrofitting an existing air-cooled data center with immersion cooling, key considerations for operators include space layout and power infrastructure.

Space and layout

Immersion tanks have a different footprint compared to traditional racks, so immersion cooling systems don't fit a standard data center layout with racks. To deploy immersion tanks effectively, facility teams can assess available space and plan accordingly, possibly reconfiguring layouts to accommodate the required space. Before operations begin, they must clear a designated zone for tank and CDU implementation.

Power infrastructure

Immersion cooling systems need a direct power supply, requiring facility teams to reconfigure their power infrastructure. Data center operators can use an A and B feed with a UPS backup and connect CDUs to a power source similar to the IT equipment. This allows CDUs to stay operational with the IT systems and provide cooling even during a power outage.

Customizing immersion cooling systems

Immersion cooling can be tailored to meet the unique requirements of different data centers by addressing specific needs such as network components and PDU. For instance, some setups require more space for network switches beyond the standard 2U compartment, while others need customized PDUs for higher power output, more outlets, or alternative configurations. Liquid flow rate adjustments are also feasible, but the primary customizations often involve network and PDU setups.

Scalability of immersion cooling

Data center operators can efficiently scale their immersion cooling deployments to accommodate future growth. Each CDU provides a cooling capacity of 200 to 240 kW, allowing data centers to scale in 240 kW increments. With this design, operators can add one to four tanks per CDU, with each tank supporting up to 100 kW. Therefore, to achieve the full 240 kW capacity, multiple tanks would be required. This modular approach suits large data centers with high-density server deployments, allowing for precise scaling according to demand.

Best practices for immersion cooling deployments

The monitoring requirements for immersion cooling systems differ slightly from those of traditional air-cooled data centers. Most operators might already know about some parameters, such as temperature and pump seeds, as these are also used in CDUs for direct-to-chip cooling.

However, immersion systems introduce new data points, such as multiple tank levels requiring unique alarm setups. These alarms are categorized as urgent or non-urgent, similar to air-cooled system alerts for server room door status. Existing building management systems (BMS) can easily integrate these additional data points, such as fill levels and tank temperatures, into their frameworks. Although the specific data points might have different names, the fundamental features and integration processes align closely with those of conventional systems.

Create a roadmap for liquid cooling
installation by downloading our guide:

Deploying liquid cooling in the data center

Emerging trends and prospects in immersion cooling

The future of immersion cooling is advancing with increased cooling capacity. We'll likely see a combination of liquid cooling solutions, including various innovative methods integrated into specialized systems.

Another significant trend is the improvement of dielectric liquids, providing alternatives that help reduce carbon emissions. Moreover, immersion cooling allows for increased cooling system temperatures, which enhances waste heat recovery processes. This progress opens opportunities for district heating applications and other uses.

Choose your path to high density

Your journey to high-density can be a smooth ride. Vertiv designs advanced solutions to power and cool the most challenging deployments. Unleash your AI Evolution with Vertiv today.

Public link

Please use the above public link if you want to share this noodl on another website.