AWS Builds Custom Liquid Cooling in 11 Months for AI Chips

Amazon Web Services (AWS) has developed a custom liquid cooling system for its data centres in response to the thermal challenges posed by AI workloads. 

As reported on the company’s website by Alex Davis, the cloud provider completed the project from initial design to production deployment in 11 months. AWS says this is to help address the inability of traditional air cooling to manage heat from dense GPU configurations.

The data centre industry is currently having to confront the AI chip challenge, as the technology is generating heat levels that outpace the capacity of traditional cooling infrastructure.In response, AWS is evaluating existing cooling solutions in the market.

“We’ve crossed a threshold where it becomes more economical to use liquid cooling to extract the heat,” says Dave Klusas, AWS's senior manager of data centre cooling systems.

AWS liquid cooling system uses direct-to-chip approach

The AWS cooling solution employs a direct-to-chip methodology, placing cold plates directly onto processors. Liquid circulates through sealed plates via tubes, absorbing heat and transporting it away from server racks. 

Additionally, the system operates as a closed loop, with heated liquid moving to rejection systems for cooling before recirculation.

This approach differs from air cooling systems that rely on airflow circulation through server racks. Air-based systems pull external air through facilities and circulate it past electronics to remove heat before expelling it outside. However, AI chips performing trillions of calculations per second generate heat levels that require airflow volumes beyond practical or economic limits.

Liquid cooling addresses this limitation through density advantages. AWS highlights in its research that liquid is more than 900 times denser than air and enables greater heat absorption capacity. 

LIkewise, the closed-loop design prevents increases in data centre water consumption, with the engineered fluid operating at temperatures comparable to hot tubs.

“We call it cooling, but our goal isn't a comfortable, 68-degree data hall. Our goal is to move just enough air through our servers to keep them from overheating and use the lowest amount of energy and water to do that,” Dave explains. 

“In the summertime, that actually means our data halls are pretty warm.”

Custom cooling solutions designed to optimise efficiency

The tech giant took just four months to move from design to prototype, before taking 11 months to deliver the first unit in production. This included time to develop designs, build a supply chain, write control software, test everything and manufacture systems.

AWS also emphasises the importance of flexibility, given how quickly technology is developing in contrast to the time it takes to build a data centre. 

Advancements in technology can change the thinking on how to balance liquid with air cooling and AWS says its designs are built to adapt to changing needs. 

“We designed our liquid cooling systems to make it easy to add them to data centres where they’re needed, but avoid the expense of adding them where they’re not,” says Dave.

Another key element of the cooling system is the custom coolant distribution unit (CDU) AWS has developed, which the company says is more powerful and efficient than its competitors. 

“We invented that specifically for our needs,” Dave says. “By focusing specifically on our problem, we were able to optimise for lower cost, greater efficiency, and higher capacity.”

Deployment strategy targets existing air-cooled facilities

The cooling system installation requires minimal modifications to existing air-cooled data centres. 

This capability enables rapid global scaling to meet AI infrastructure demands without constructing new facilities or implementing comprehensive retrofits.

AWS initially developed the system at its research and development centre, where the company tests all data centre components before deployment. Following laboratory validation, the team deployed test units in production environments. 

The system has completed testing phases and achieved readiness for scale deployment, the company says.

Current deployment plans involve summer ramp-up activities to increase cooling workload capacity, with expansion into additional data centres following performance validation. The specific locations and deployment schedules will depend on customer demand patterns and operational requirements, according to AWS.

AI workloads benefit from high-density chip configurations that minimise physical spacing between processors. This approach reduces communication latency, improving performance while reducing cost and energy consumption. 

However, AWS notes that the resulting heat concentration exceeds the thermal management capabilities of traditional air cooling systems.

As AI adoption accelerates across the AWS customer base, the company notes that transitioning towards more liquid cooling-based solutions will support current data centre infrastructure limitations. 

Its cooling system development reflects the company’s committed approach to improving customisable AI and data centre solutions. 

Dave adds: “[our solutions] can be deployed precisely where liquid cooling is needed to meet our customers' demands.”

Explore the latest edition of Data Centre Magazine  and be part of the conversation at our global conference series, Tech & AI LIVE and Data Centre LIVE. 

Discover all our upcoming events and secure your tickets today.

Data Centre Magazine is a BizClik brand