Increased computing power and performance demands, combined with technological advancements such as artificial intelligence (AI), Internet of Things (IoT), and machine learning (ML), have resulted in rising temperatures within data center infrastructure and IT equipment. More compute means more power, more power means more heat, more heat needs more cooling which consumes yet more power, and the cycle continues.
Given the inefficiency of traditional air cooling systems — and the combination of sustainability-minded design goals, governmental regulation, rising power costs, and the technological demands to cool cutting-edge CPUs and GPUs — the need for more efficient and less power-hungry cooling systems is clear.
Is immersion cooling the future of high-performance computing? If one looks at the impressive forecasts, it certainly appears so. The immersion cooling market is projected to grow from $243 million this year to $700 million by 2026. Driving these phenomenal growth projections are an ever-increasing number of data centers and the rising need for cost-effective and eco-friendly cooling solutions.
Immersion cooling is the hot new thing in computer cooling with several key advantages that are winning over more and more converts. It can handle far greater power densities, it operates cleanly and quietly, and it costs much less than traditional air cooling. All of these advantages have additional value for high-performance computing, which pushes processors and budgets to their limits.
Research shows that the high-performance computing segment makes up the second-largest share in the immersion cooling market. The large share is explained by the fact that high-performance computing (HPC) is gaining traction across many different industries with enterprises looking to scale up complex applications. Researchers, however, point to increased retrofitting costs of existing infrastructure, coupled with the requirement for new specialized infrastructure, as possible restraints for the growth of the immersion cooling market.
What Is Immersion Cooling?
Immersion cooling is, as the name suggests, dropping temperatures through the submersion of components in liquid. Instead of blowing cool air over your electronic parts, you submerge the entire system into a dielectric liquid.
Much like a plastic layer around a cable does not conduct electricity, dielectric fluid also does not conduct electricity. When a current reaches the liquid, it does not travel through the liquid. This allows the computer equipment to function even when submerged.
Some dielectric liquids are sourced naturally, such as mineral or biological oils, while others are engineered. These liquids work tremendously better at conveying heat away from the heat source when compared with air —1,200x or more!
Dielectric liquids are typically divided into two types: single-phase liquids which are designed to circulate around the equipment and then through a heat exchanger, and two-phase liquids that boil at low temperatures to dissipate the heat even more rapidly. We’ll get into the pros and cons of these two approaches later.
Immersion cooling can employ several different configurations. For instance, the electronics can be fully contained within closed systems for each computer, or within a single large vessel that holds multiple machines, or in some combination. Each arrangement requires different components to distribute the heat.
One can describe the immersion cooling process as a revolution in evolution. It is an evolutionary advance in applying immersion cooling — which has been around a long time and used in many industries and applications — to using new fluids and systems appropriate for use with modern servers and IT. This revolutionary technology marks a rapid shift to sustainable, cost-efficient data center cooling solutions.
Given the demand for green IT to address rising costs and climate change-driven policy, liquid immersion cooling is poised to become a staple of data centers, rendering air cooling systems redundant. Immersion cooling offers many unique advantages that include yielding an extremely efficient operation that slashes electrical costs while enabling “green” computing and a quieter work environment.
In addition, immersion cooling eliminates many of the moving parts associated with air cooling. The liquid approach also means a smaller real estate footprint, offers a much lower total cost of ownership (TCO) and is substantially easier to maintain.
Tech giants, including Intel, Facebook, and Google, have put their weight behind liquid immersion cooling, while industry and government are working to make this technology standard. If all of these advantages have you thinking: this sounds like the future of data centers, then you are on the right track.
Single-Phase Immersion Cooling vs. Two-Phase Immersion Cooling
During single-phase, immersion cooling, the equipment is immersed in a thermally conductive dielectric liquid — typically a synthetic hydrocarbon. The facility’s available coolant can be used to dissipate heat from the server setup. The coolant always remains in a liquid state. The technique uses ‘open baths’ since the coolant is non-evaporative.
With two-phase immersion, as in the single-phase approach, servers and other equipment are also immersed in a thermally conductive dielectric liquid. The liquid circulates to make direct contact with hot components. The liquid evaporates off the electronics into a gas (the second phase) to remove heat, then re-condenses back into a liquid. In two-phase immersion, the dielectric liquids in use are fluorocarbons with low boiling points. The electronics cook off the coolant into a vapor, which then condenses on a heat exchanging coil to return at a lower temperature to the basin.
Those same low boiling points also make two-phase liquids highly evaporative. As such, these systems must be sealed with the liquid inside to prevent evaporative loss of the fluid. This makes two-phase systems more complex and less practical than single-phase immersion cooling.
Both immersion approaches involve drawing the heat out of the components through liquid instead of air. The difference is essentially in whether that cooling liquid undergoes an intermediate vapor stage. The phase change from liquid to gas removes heat in two-phase cooling, while one-phase cooling instead dumps the heat into another outlet through liquid only.
What Are Some Use Cases for Immersion Cooling?
Immersion cooling has already and impressively racked up several use cases where it has ably demonstrated its value. Many of the applications involve high-density computing, settings in which it has proven challenging to cool equipment sufficiently with traditional air cooling.
Supercomputing
Supercomputing for scientific or engineering activities places extreme demands on computers within a small volume, requiring huge cooling capacity. This has made supercomputing one of the main drivers of immersion cooling adoption.
High-performance computing (HPC) requires intensive power for the processors themselves as well as for cooling. Immersion solutions like GRC's ICEraQ One can drastically decrease energy consumption – removing 10-20% of server use and 95% of cooling use. This enables innovative applications as well as makes the physical requirements more manageable.
Supercomputing centers are often loud enough to require hearing protection, but immerison cooling operates silently. It also cuts around half of the infrastructure costs, saving more money for the servers themselves.
Immersion cooling increases the number of operations per second, per watt, and per dollar, yielding more powerful supercomputers. Not only are they faster, these supercomputers are also more sustainable thanks to their decreased resource use.
Enterprise HPC
Businesses make use of high-performance computers, as do scientific and other institutions. For enterprise HPC, immersion cooling offers greater efficiency and cost savings. As with other applications of immersion cooling, this results in smaller footprints and lower latency, plus more reliability.
It’s tough for businesses to anticipate their computing needs. Immersion cooling makes this easier, given that the modular units allow for constructing what is needed when needed. Gone are the days of having to commit to far larger buildouts than what’s required.
GRC’s ICEraQ Quad liquid immersion cooling solutions cost half as much as other alternatives. For private or public clouds and other use cases, you can eliminate nearly all cooling costs, as well as maintenance and operating expenses.
AI and Machine Learning
Artificial intelligence, including the statistical approach called machine learning, requires relatively intensive computing resources. These systems often use high-density CPUs and GPUs, as well as custom chips like Google’s TPUs. Cooling all of these processors has proven challenging for air cooling systems. Liquid immersion, however, handles the heat, literally!
AI does numerous parallel computations with powerful processors, pulling several times more electricity than average. These draw heavily on the electrical equipment and generate heat. Resultantly, AI data centers have become larger to accommodate these needs.
The real-time processing of AI pushes processors to the limit. Immersion cooling raises these limits, thus enabling faster, smarter computing. Immersion in a dielectric fluid provides practically unlimited cooling. This lets artificial intelligence expand to run on as much hardware as necessary, including new GPGPUs.
Data mining, another computing-intensive application of statistics and pattern detection, also necessitates the need for these data centers to use liquid immersion cooling. Along with analytics, fraud detection, and other low-latency functions, these operations make considerable use of power-hungry hardware accelerators.
Blockchain and Cryptocurrency
Cryptocurrency has taken off over the last few years, turning into one of the core drivers of immersion cooling. Crypto applications often employ dedicated hardware like GPUs and ASICs that require considerable cooling.
Mining cryptocurrency requires massive computational resources. As such, this endeavor has worked alongside supercomputing as a key motivation for using liquid immersion cooling. Efficient heat management technology allows these intense processes to run smoothly. Crypto users and admins want to extract as much performance as they can out of their hardware. Immersion cooling allows for safe overclocking. The greater efficiency yields higher profits.
Blockchain is rapidly expanding into areas other than cryptocurrency. Expansive networks to verify data require heavy computation. The complex mathematical calculations produce heat that dissipates more efficiently with immersion cooling. To generate a profit, blockchain applications need to minimize power use wherever they operate. Speed matters, given the rapid ups and downs of cryptocurrencies. GRC’s immersion cooling addresses these needs. You can use them anywhere you want, however remote.
GRC even offers a custom blockchain rig in a container, the HashTank. Complete with networking, remote monitoring, electrical infrastructure, and fire detection, these extremely efficient mining centers can handle 648 kilowatts of processing. You can also install the HashRaQ in an existing data facility.
High-Frequency Trading
As more capable computers have become available, the finance industry has put them to use for low-latency transactions. Huge amounts of money ride on mere milliseconds. Thus the cooling capacity and processing enhancements of immersion technology are a good fit.
The speed pressures of high-frequency trading require data centers near where they trade. This puts a premium on equipment density, which along with the exacting computational criteria makes cooling efficiency downright necessary. Also, the data centers must operate within limited power and space envelopes.
Fortunately, immersion cooling can deliver more than enough capacity right in busy metro areas. Even without revamping a site, you can install a full-fledged data center where you need it. The powerful processors in use for HFT can then glide through operations with minimal cooling overheads.
Why Should You Go For Immersion Cooling?
There are several motivations for using immersion cooling, which apply to different organizations. These reasons vary and could include handling greater power densities than previous cooling technologies support; decreasing costs for a data center; increasing energy efficiency.
You Need To Expand Beyond Your Power Envelope
In the data economy, IT resources often push beyond the available power. For cases like these, you can use liquid immersion cooling to cut down on your power consumption and increase your computing capacity.
As you gradually replace your older air-cooled systems, you’ll see electricity use (and the accompanying costs) drop. The immersion cooling units use less power, and they also make the servers themselves run more efficiently. You can fit in double the computers, which may even remove the need to build another data center.
It’s the extreme energy efficiency of immersion cooling that allows more processors within the same power envelope. Data centers that have maxed out their power yet need more computing should consider immersion cooling. It makes more sense than bringing in additional power.
An oil and gas business, for instance, had an obsolete data center that could no longer support its functions effectively, as it burned through more power on cooling than on computing. With ICEraQ immersion cooling, however, they seamlessly decreased their use of the power envelope, revitalizing the facility with double the servers within weeks.
You Face High Capital Expenses for a New Data Center
The upfront costs to build a traditional air-cooled data center are high. From building design through the installation of air handlers and chillers, the costs quickly add up. Raised floors and other requirements can make a new data center unaffordable. By eliminating the need for all the air handling infrastructure of the traditional air-cooled data center, immersion cooling can cut your capital expenses in half.
With immersion cooling, you have a smaller footprint so you can also decrease capital expenditures on building the data center itself. A smaller building will suffice, and you don’t face the same initial costs on generators and other electrical equipment. Instead, you can build a smaller data center with only the cooling capacity that you need now.
Traditional cooling necessitates computational fluid dynamics (CFD) analysis, a difficult task to prevent the data center from overheating. It also imposes other preventive measures, including construction and engineering work. The mounting costs can undermine the very profitability of the endeavor. Liquid immersion cooling minimizes the amount of capital necessary for preparing the structure. Instead, the investment can go into IT resources that directly contribute to the organization.
Air cooling is constrained to around 15 kilowatts per rack, an order of magnitude less than immersion cooling. This forces the development of larger, costlier data centers, which can become prohibitively expensive. Liquid immersion solutions, on the other hand, with their lower absolute cost and their modularity enabling gradual deployments, sidestep the tricky situation of high CAPEX.
If You Are Looking for Extreme Energy Efficiency
Energy efficiency, the ultimate determinant of data center functionality, is much higher with immersion cooling than with air. Many of the server applications now gaining steam are GPU-intensive, pressing the limits of available power. Data centers are on track to use a fifth of the world’s electricity production in the next few years.
Liquid immersion can cut energy use in half. This drops the cooling cost from 70% to 3% of the power requirement for processing! Organizations in areas with large energy costs are already relying on GRC’s efficient cooling solutions. Examples include the greenest supercomputer on the planet in Japan, along with geoscience businesses in Australia and the US. The growing trend towards green data centers and sustainable computing will only make this motivation to use liquid immersion cooling more compelling.
Today’s uses of data exceed what air cooling was designed to support, and tomorrow’s uses will need even more energy. Liquid immersion drastically ups the efficiency of data centers, opening new avenues of possibility.
The more efficient use of power saves money, increases server density, and protects the environment’s scarce resources. It also gives you flexibility in where and when you add capacity. As business and IT increasingly progress together, having complete control of your energy use can propel your profits.
This need for more efficient power use applies not only at the level of the data center but also at the level of the rack. By packing potentially ten times as many servers into the same volume, each rack becomes a super-dense computer trove.
Liquid Immersion: Myths vs Facts
Public perception of liquid immersion cooling has not yet caught up with the development of the technology itself. Numerous myths surround the technology, clouding decisions as to which cooling solution to deploy. The right information can assist you in finding a safe and cost-effective system.
It's a New and Unproven Technology
Liquid immersion cooling has extensively been rolled out across a range of contexts including a number of the most intensive computing facilities on the planet. Some of the main telecoms, government agencies, and supercomputing and search facilities use immersion cooling. The last decade, for example, has seen installations by the United States Air Force (USAF), the National Security Agency (NSA), and several multinational oil and gas enterprises.
The cooling technology has worked consistently for years, totaling millions of hours of server activity. Users have found liquid immersion cooling extremely reliable, considerably exceeding air cooling setups. After numerous iterations of product development, this is now a well-established technology.
Liquid immersion technology has expanded to encompass multiple products for a range of high-performance needs. For instance, GRC's ICEraQ is ideal for use with artificial intelligence, enterprise HPC, and cloud services.
Research and testing on immersion cooling go back decades even before modern commercial applications of the technology. It now supports servers from Dell, HP, IBM, and practically any other manufacturer. This technology has undergone exhaustive testing and it performs far better than air cooling in even the most demanding use cases.
Liquid Immersion Is More Expensive Than Traditional Cooling
The opposite is true. Liquid immersion is actually much less expensive than traditional cooling. Organizations have found that they spend around half as much on data centers and energy with liquid solutions, saving literally millions of dollars on deployments. Still not convinced? Try this TCO calculator.
With liquid immersion, even the servers themselves cost less to run. Bathed in cooling oil, they don’t require inefficient fans. This cuts down the cost of running the servers by about a fifth, in addition to the financial savings from the infrastructure.
The savings start at installation with the lower initial requirements, then continue to accrue throughout the operation of the cooling system. Both CAPEX and OPEX are noticeably less with liquid. The cooling costs are 95% lower than with air. Myth busted!
Liquid Immersion Cooling Is a Complex Solution
While liquid immersion differs from traditional air systems, it’s mechanically far simpler – not more complex. For example, GRC’s immersion cooling systems have almost no moving parts, which simplifies maintenance immensely in comparison to high-wear air cooling. An annual filter change for the liquid replaces the extensive service requirements of air coolers.
Moreover, the elegant design of liquid immersion systems gives technicians easy access to servers for hot swapping and the like. It also makes the entire data center simpler, without the need for hot and cold aisles, raised floors, chiller plants, humidity controls, air handlers, and much of the electrical equipment necessary for air cooling. Plus you can install it practically anywhere.
The technology of liquid immersion is not complex, it basically involves pumping a liquid around. The parts are straightforward. Use of the system is also easy, as it connects directly with existing material including servers, power, and networking. This solution displaces the hugely complex components and supporting infrastructure of traditional air cooling.
Liquid immersion is such a simple solution that you can deploy a complete server setup in mere weeks, with installation measured in days. It’s reliable and remote monitoring is a snap. Cooling with liquid results in efficiency as well as simplicity.
The Coolant Used is Highly Expensive
In single-phase liquid immersion cooling, the kind of immersion that has become commercially successful, you use an inexpensive fluid, like GRC ElectroSafe. This substance remains within the cooling system throughout the years of its operation.
While ElectroSafe has 1,200 times the cooling ability of air, this does not result in bigger costs. On the contrary, the entire system costs far less than air cooling. There are several variants of the coolant that you can select from, all of which are safe, affordable, and widely available.
While there are two-phase coolants that cost more, you don’t need those for immersion cooling. Instead, the inexpensive single-phase coolant fits easily into the much lower total cost of ownership: half as much as air cooling. Also, unlike two-phase fluids, most single-phase fluids are non-evaporative and designed to be used continuously for 15 or more years. In summary, the coolant for liquid systems is not highly expensive, particularly when the cooling solution uses a single-phase liquid.
It’s Hard to Integrate With Legacy Data Centers
Immersion cooling works surprisingly well with legacy data centers. You can upgrade an older data center with its existing infrastructure. You can even incorporate an area of high-performance liquid-cooled servers within an operational air-cooled facility.
Installation of immersion cooling systems is quite easy, so you can fit it into your legacy data centers as necessary. Regardless of whether you have raised floors, concrete floors, or even unused rooms or loading docks, adding on a immersion cooling system is seamless.
The adaptable technology allows you to make more efficient use of older data centers’ space and power. Whether you partially or entirely retrofit the data center, you can then support new workloads including high-performance computing. Modular coolers like the ICEraQ can accommodate legacy facilities easily.
Air Cooling vs Liquid Immersion Cooling
While both air cooling and liquid immersion cooling use a medium (air or liquid) to ship heat away from components, they differ immensely in how they function mechanically, how cost-effective they are, and how much computational capacity they can support. They also differ in terms of where they can operate and how susceptible they are to environmental hazards.
Mechanism
Air cooling pushes air from outside through a series of equipment including air handlers and computer room air conditioning (CRAC). The cooled air goes through a raised floor into a cold aisle, then through the servers where it siphons off heat, then into a hot aisle. From there the air returns to the air handlers to restart its journey.
Liquid immersion cooling, in contrast, entails directly dunking the servers in a fluid that transfers heat over a thousand times more effectively than does air. This liquid circulates through a cooling distribution unit (CDU) that dissipates the heat through a heat exchanger, returning the liquid to the start.
Air cooling pushes heat through complex mechanisms that cost a lot in money and space. This contrasts against immersion in a non-evaporative liquid. As the heat in a immersion cooling system is handled by the facility’s coolant, the waste heat can even be reused to warm the building’s water and air.
The liquid immersion approach has a simpler mechanism, which requires only three moving parts. This drastically aids all aspects of running a data center. It’s easier to build a data center for immersion cooling, costs less, equipment breaks down less often, and supports greater computational density.
Cost
When we compare the costs of air versus immersion cooling, we must consider the initial costs (CAPEX) as well as the ongoing costs (OPEX), the two different components of total costs (TCO). Also, for a fair comparison, we should include not only the cooling devices themselves but also any necessary accessories.
The far greater efficiency of liquid makes it capable of cooling more servers in a smaller space at a lower cost. It also decreases the complexity of the machinery, making maintenance easier. Air cooling costs around two times as much upfront due to all of the external cooling equipment, space, battery backups, and raised floors.
Even after construction, air costs more than liquid. This again derives from liquid’s more efficient heat transfer properties. Fans draw power continuously, while refrigeration also drains power. The bottom line: air adds a heavy toll to data center operational costs. Immersion drops 90% of the cooling costs, a good part of the total data center energy budget.
Performance
When it comes to performance, air cooling tops out at around 30 kilowatts per rack, with a massive loss of efficiency at around half that. By contrast, immersion cooling can handle over 100 kilowatts per rack, in some cases twice that.
Modern HPC loads extend beyond the capacities of air cooling, and this trend will only intensify. Liquid solutions have ample capacity and will become necessary for data centers in the coming years.
Data centers increasingly are running energy-intensive loads like artificial intelligence and crypto. These often employ dedicated processors that are multiplying in power consumption. As such, we need the order-of-magnitude gains of immersion cooling.
Location
Air cooling restricts data centers to certain locations where air filtration and other infrastructure will fit, while immersion cooling has fewer requirements and can be installed anywhere. As edge computing becomes the norm, this advantage will make immersion cooling more compelling. Reliable operation is already making immersion cooling popular for out-of-the-way installations.
Humidity, temperature, and other factors can affect the performance and reliability of cooling systems. Air cooling uses outside air, so it loses much of its efficacy in warm locations, where many data centers are situated. On the other hand, immersion cooling does not have this dependency. It works with a coolant that remains potent regardless of outside weather conditions.
Air cooling also imposes location constraints from the additional size necessary for spacing out servers to dissipate heat. With immersion cooling, you can pack servers much more closely together, without the need for hot and cold aisles or anxieties over hot spots.
Immersion cooling offers exemplary simplicity, allowing it to function in hot or cold climates, wet or dry locations, in cities, or ‘far from civilization’ locations. This technology frees you to install data centers where you want.
Protection from Environmental Hazards
Server equipment normally undergoes environmental wear and tear which gradually degrades reliability. Data centers exposed to the elements, like edge computing sites, face additional stresses. The air contains moisture and small particles, which interfere with the integrity of the systems. Liquid immersion coolant coats the electronics in safe armor against these threats. Using air as a coolant foregoes this protection.
The oil in immersion coolers acts as a protective casing against dust, oxidation, corrosion, vibration, and numerous other risks. The computer equipment remains safe, with the coolant acting continuously to maintain a low operating temperature.
By contrast, air cooling brings in air laced with contaminants that can neutralize IT equipment. Lacking the protective liquid, traditional systems’ mean time between failures (MTBF) becomes smaller. Expensive humidity control systems, maintenance, and other recourses offer only limited coverage. Air cooling exposes servers to risks that affect stability, while immersion cooling offers environmental defense.
Analysis
Air cooling simply can’t keep up with increasing data center demands. A proliferation of processors is becoming more power-hungry. The move to the edge exacerbates this issue, as space is even tighter at remote sites.
Liquid immersion cooling is flowing in to fix things. It not only addresses all of the key concerns with air but goes way beyond. Immersion handles far more server density with less noise and power use. In addition to its superior cooling capacity, liquid also has several other advantages like location independence and low cost.
All told, liquid immersion cooling offers enough benefits over air cooling to make this a lopsided comparison. We have moved from air cooling as the only choice to immersion cooling as the clear, smart, superior option.
Does Your Data Center Need Immersion Cooling? Let GRC Help
Liquid immersion cooling for data centers has made great strides over the last decade. The single-phase version of this technology now offers winning efficiency, price, and reliability. It enables HPC, AI, blockchain, and the other technologies that define tomorrow’s computing landscape.
For these reasons alone, it’s clear that liquid immersion cooling is the future of high-performance computing – and also of data centers in general. When we look at the evolution of computing and cooling, we can see a turning point approaching as the next generation of applications requires the efficiency, capacity, and adaptability of liquid immersion cooling
GRC’s ICEraQ system extends from a simple rack up to 4 x 42U quads that you can combine for any size data center. It works with the facilities and hardware that you have already, or you can buy efficient new buildings and servers. Immersion cooling can solve your heat issues now and in the future. It lifts the limits on density while saving you money and space while significantly expanding compute capacity. And Oh, it’ll make your data center more sustainable. Ready to take the plunge? Let GRC help!
