GRC announces details surrounding one of its largest installations to date, a 24-rack, 600kW system at CGGVeritas in Houston, Texas.
GRC announces details surrounding one of its largest installations to date, a 24-rack, 600kW system at CGGVeritas in Houston, Texas.
Paul Rubens over at Enterprise Networking Planet wrote an article last week that looks at the current state of liquid cooling technologies available to data centers. The piece, Liquid Cooling Gaining in Popularity Again, presents some key reasons that liquid cooling is gaining traction in the data center:
…thanks to spiraling energy costs, corporate green initiatives and new high-tech coolants, the concept of liquid cooling in the data center is enjoying a renaissance.
While these can be factors in the decision to switch to liquid cooling, we find customers turning to our liquid cooling solution for other reasons. For example, the CarnotJet system offers:
The article continues with a description of current dielectric fluids in use:
These liquids do not conduct electricity so they can be in direct contact with electronics without causing any damage, and since they are many times better than air at capturing and transporting heat they offer the prospect of dramatically more energy-efficient cooling than is possible using the conventional chilled air approach.
It’s no secret that GreenDEF™ dielectric fluid coolant is highly refined mineral oil. Unlike Fluorinert and other solutions of the past, our coolant is inexpensive — and because it’s essentially baby oil, it’s environmentally and human friendly. There’s also no risk of evaporation or spoiling, which allows for our easy-access open-system design.
Rubens does a nice job discussing the benefits of submersion cooling in the article:
The potential benefits of liquid cooling in are significant. For starters, a suitably designed system can capture almost all of the heat generated by a server’s components, so there is no need to power internal fans to assist airflow. That in itself can reduce server power consumption by about 30 percent. But the main savings come from reduced air cooling costs. Since heat from the servers is captured by the coolant and removed without warming the air around the server racks, there is little or no need for computer room air conditioning (CRAC) equipment. And since the electricity needed to power CRAC equipment, chilling plants and other cooling equipment may account for as much as 30 percent of data center running costs, the potential savings are enormous.
Removing server fans is an important energy-saving practice in our submersion cooling system. In a liquid environment, fans are obsolete! As GRC Founder Christiaan Best said in an interview at Texas Advanced Computing Center, submersion cooling replaces managed air flow with managed liquid flow. As Team Texas discovered at the Student Cluster Competition during SC’11, removing server fans can provide significant energy savings — typically 5-25%.
And because liquid so efficiently captures server heat and makes it accessible, it is also possible to reuse that heat. In November, Green Revolution Cooling announced heat recapture for data centers at KTH (Royal Institute of Technology) in Stockholm, Sweden. Working with GRC’s technology, researchers at KTH have achieved 50°C water output using commodity server hardware, a resource that can be useful year round.
The article offers a paragraph about Green Revolution Cooling, specifically:
Green Revolution Cooling, a Texas-based based company, offers probably the most radical departure from traditional data canter cooling. Its CarnotJet system is based on the concept of dunking — literally placing an entire server rack into a tank of its GreenDEF coolant.
It may seem like a radical departure from the norm, but the dramatically reduced costs and increased performance are a welcome change.
Green Revolution Cooling is featured on the Texas Advanced Computing Center website today.
Christiaan Best, Founder and CEO of Green Revolution Cooling, sat down for an interview to talk about the installation at TACC that was first installed in April 2010. Best also gives some insight into the backstory of GRC — including the original inspiration for the product — and why he believes the CarnotJet™ system is the most efficient and cost-effective data center cooling solution available today.
Watch the video here:
In addition to the video, the featured article reveals why GRC brought the CarnotJet™ system to market in early 2010:
“Ten years ago, no one cared,” Best explained. “Power density was very low, the cost of hardware compared to cooling was much higher, and the number of overall servers was lower. But now, people who run data centers are starting to build dedicated buildings just to move the air though the computers. Where does this madness end? You can’t just keep shoving power into a box and expecting it to be cooled by air, which is fundamentally an insulator.”
With energy costs rising and society requiring more compute power and data services, Best saw renewed opportunities for liquid cooling technology if it could be turned into a cost-saving product for data centers.
The article goes on to discuss the partnership between GRC and TACC that has produced more than eighteen months of data:
In 2009, Green Revolution Cooling was awarded a National Science Foundation (NSF) Small Business Innovation Research program grant to support the benchmarking of its CarnotJet™ dielectric fluid submersion cooling system. In April 2010, GRC deployed the system at TACC’s data center on The University of Texas at Austin’s J.J. Pickle Research campus.
More than a year and a half into the collaboration, the results of system tests have exceeded expectations on both sides. According to measurements, the installation has consistently used three to five Watts or less of cooling power for every 100 Watts of server equipment used. Moreover, the fluid submersion cooling system has shown reductions in total power usage of approximately 40 percent compared to the same hardware cooled by air, despite the fact that the GRC system was installed on an unconditioned loading dock where the ambient air temperature is sometimes well over 100°F.
Finally, the article shares Best’s hopes for the future of high performance computing and efficient fluid-submersion cooling for data centers:
“So much of the datacenter is not focused on the silicon itself,” Best said. “By reducing the amount of infrastructure, I think it will allow scientists to spend more money on the silicon itself.
“If all data centers were able to reduce the amount of energy used on the scale that we’re able to achieve in HPC, then I think the world would be a different place.”
With a rapidly growing customer base that includes five of the Top-100 supercomputing sites and an installed capacity approaching one megawatt — not to mention a new Phase II NSF Grant — the future is looking bright for GRC.
Green Revolution Cooling has been awarded a Phase-II National Science Foundation Small Business Innovation Research (SBIR) grant to support the continued development of its high-performance fluid-submersion cooling system for data centers.
The grant comes two years after the Phase I grant from NSF and the funds will be used to streamline large-scale manufacturing of the CarnotJet™ system. In addition, the funds will be used for R&D to further optimize system performance and efficiency.
This is very exciting news for GRC. The continued support from NSF signifies the foothold that GRC has gained in the data center cooling market.
You can read the full press release here.
AUSTIN, TX, January 16, 2012 – Green Revolution Cooling (GRC) today announced that it has received a Phase II Small Business Innovation Research (SBIR) grant from the National Science Foundation (NSF). The second-stage grant follows the Phase I grant received by GRC in November 2009.
Christiaan Best, CEO and Technical Founder of GRC, said: “We’re happy to be working with NSF again to continue the development of what we believe is the most powerful and efficient cooling solution available for the data center today. This funding will support advanced development of the next generation of efficient fluid-submersion cooling systems.”
Since the Phase I grant in 2009, GRC has been rapidly growing its customer base. Now approaching one megawatt of installed capacity worldwide, including installations at four of the Top-50 supercomputing sites worldwide, GRC continues to gain traction in the industry.
The Phase II grant signifies continued support for GRC’s next generation of fluid-submersion cooling systems. The majority of funding will support research and development to further increase efficiency and usability while streamlining the manufacturing process.
About NSF’s Small Businessss Innovation Research Grant
The Small Business Innovation Research (SBIR) program stimulates technological innovation in the private sector. Phase II awards are available only to recipients of Phase I funding. For more information, please visit: www.nsf.gov/eng/iip/sbir/
SC’11 wrapped up last week and the conference proved a tremendous success for Green Revolution Cooling. Trade shows are always fun for us; attendees unfamiliar with our technology get pretty excited to see servers submerged in fluid.
The exhibit featured an assortment of literature and giveaways, including coolant samples and USB drives. The main attraction, however, was the 13U evaluation unit on display. We included a few servers for demonstration purposes and enjoyed talking to interested spectators.
Unsurprisingly, there were a lot of questions directed our way. Here are some common ones:
• Is that water? [Nope, it's GreenDEF dielectric mineral oil.]
• If the coolant captures the heat, where does the heat go? [The coolant is continuously pumped through a heat exchanger near the Rack where the heat is transferred to water.]
• Why do this at all? [We can save 10-25% of server power, 95% of cooling power, and 45% of overall data center power.]
It gets really exciting when someone begins to understand the technology and the potential for leveraging fluid cooling. The questions, too, get better and better.
• So you don’t need any infrastructure, could you install the CarnotJet in a parking lot? [Yes. We've installed in a loading dock and a closet, too. With our system, data center space can be much, much cheaper than usual.]
• Capacity is really a function of coolant flow, isn’t it? [Mostly. Bigger pumps and pipes means more flow which equates to more cooling.]
• Over-clocking becomes very simple doesn’t it? [Over-clocking servers is very possible in the CarnotJet system -- ultra-efficient fluid-submersion cooling allows you to dissipate much more heat than usual.]
On Wednesday afternoon, Christiaan Best, Technical Founder of GRC, delivered a talk on lessons learned from two-and-a-half years of fluid submersion cooling experience. His presentation emphasized the importance of providing cost savings and performance increases too significant to ignore when developing a new, disruptive technology.
Christiaan also released details on two new GRC installations. The first is a new installation at Royal Institute of Technology (KTH) in Stockholm, Sweden. Also the main news point of our most recent newsletter, KTH has achieved 50°C waste water using the CarnotJet™ system.
Christiaan also unveiled some preliminary details from our largest installation to date, CGGVeritas. Check back in the coming weeks for a full case study and announcement.
Student Cluster Competition
Another highlight of the show was the Student Cluster Competition. Students from the University of Texas at Austin participated in a series of HPC events that saw eight student-led teams compete to finish five application sets with the highest compute efficiency. In an effort to increase power efficiency (the teams were limited to twenty-six amps), the UT team utilized CarnotJet system technology to cool their equipment.
When the final results were revealed, the Texas team finished third, behind two teams (Taiwan and China) that were using GPU-accelerated servers. Congratulations to the students from UT for finishing so close to the top without the help of GPUs, and for demonstrating great skill with the programming challenges — best of luck now and in the future!
One final note from SC’11: all week on display in the exhibition hall was a poster detailing GRC’s first installation: a single Rack install at Texas Advanced Computing Center completed in April 2010. If you’re interested in some GRC history, follow the link below!
Green Revolution Cooling and Texas Advanced Computing Center Research Poster Displayed at SC'11, this poster details the results of Green Revolution Cooling's first installation at Texas Advanced Computing Center in April 2010.
The CarnotJet fluid-submersion cooling system is the first to recapture nearly 100% of server heat without adding infrastructure costs. A single 20 kW rack with 42 servers can reduce heating costs by $15,000 annually.
One of the biggest barriers to efficiency in the data center industry today is the inability to recapture server heat without incurring extravagant infrastructure costs. Servers convert vast amounts of electricity to heat, most of which is wasted. Green Revolution Cooling is announcing the ability to recapture and reuse nearly 100% of server heat at a cost lower than most air cooling solutions.
A July 2011 installation at the Royal Institute of Technology (KTH) in Stockholm will reliably produce water up to 50°C (122°F) — hot enough to pump to surrounding buildings for building heat. Remarkably, this performance has been achieved with commodity servers and standard CPUs.
Unlike many cooling solutions, the ability to recapture and transport heat is a standard feature in the CarnotJet system. The system design is simple: OEM servers are installed in an open rack filled with dielectric fluid. The dielectric fluid circulates freely through the servers, removing heat to a Pump Module where it is exchanged to a standard water loop. Typically, that loop is run to an evaporative cooling tower to reject the heat to the atmosphere, but it is also possible, as KTH is proving out, to reuse the heat from the water.
As testing continues at KTH, researchers will strive to produce 70°C water, which could be used to produce hot tap water, a useful commodity all year long. And if 70°C is achieved, KTH will be able to recapture energy in addition to heat. Added to the energy savings produced by server fan removal, the system could outstrip even the most energy efficient CarnotJet system installations to date.
Testing has commenced with newer server hardware and we expect a full white paper by the end of the year.
Exhibitor: Booth #664
Green Revolution Cooling debuted the CarnotJet fluid-submersion cooling system for OEM servers at SC’09 in Portland and was named a Disruptive Technology of the Year. At SC’10, we exhibited an evaluation unit and showcased our ongoing field trials and evaluations at various sites. This year at SC’11 in Seattle we’re proud to highlight our rapidly expanding customer base, our growing number of installations approaching one megawatt of installed cooling capacity, and our successes in extreme GPU cooling and heat recapture.
Wed Nov 16: Exhibitor Forum: “Fluid Submersion Cooling Field Trial Results and Learnings“
1:30-2:00 PM – Room: WSCC 613/614
Fluid submersion cooling is being used by a growing number of the Top-500 HPC sites around the world, in trials or in larger production systems up to 300kW per site. For these users, there is a path to sustainable Exascale production where cooling energy is cut by 95%, server power is cut by 5-25%, rack densities can reach 100kW per 42U rack, and components such as GPUs can run up to 20°C cooler than in air. One recent installation site in Northern Europe is reducing net data center power to near-zero by recapturing server heat in the form of hot water at 50°C and higher. Christiaan Best, CTO of Green Revolution Cooling, will present findings from two and a half years of customers’ experiences utilizing fluid submersion technology.
Tues Nov 15th: Poster Session: “A Top-20 Supercomputing Facility Rethinks Its Cooling Efficiency with Fluid-Submersion Cooling”
5:15 – 7:00 PM – Room: WSCC North Galleria 2nd/3rd Floors
Texas Advanced Computing Center (TACC) and Green Revolution Cooling (GRC) partnered to solve two common data center inefficiencies: the amount of energy spent cooling servers (relative to powering loads) and the inability to efficiently cool high-density, high-output devices such as blades, GPUs, and servers with over-clocked CPUs. Using Green Revolution Cooling’s fluid submersion cooling solution, the CarnotJet system, TACC and GRC have been conducting efficiency and reliability studies with OEM servers with 100% uptime since April 2010. The self-contained system has demonstrated an 85% reduction in overall cooling energy while using 6 Watts or less per 100 Watts of IT. During overclocking trials, the system successfully dissipated more than 200 Watts per socket while clocking current-generation Intel Xeon processors 35-60% over the base rate. During the poster session, staff from GRC and from TACC will discuss the groundbreaking installation and the petaFLOP installation planned for early 2012.
Sat Nov 12 – Wed Nov 16: Student Cluster Competition
WSCC East Lobby
Students from the University of Texas have partnered with TACC, Dell, Intel, and Green Revolution Cooling to compete to deliver the highest sustained performance across a variety of applications. The UT students will be using GRC’s CarnotJet technology to keep their servers cool as they compete for the SC’11 crown.
For more information, visit the SC’11 website, here.
Green Revolution Cooling staff is currently installing a block of four 42U CarnotJet system racks in a customer’s data center. The installation is located in formerly air-cooled production space.
Following are a few sneak-peak pictures from the installation site. Although the tanks are in place and hooked up to the pump module, servers and coolant have yet to be added.
This picture shows four 42U racks [foreground], one pump module [back left], and a group of air conditioners [back right] left over from the air-cooled systems previously occupying this data center space.
Each pump module can support up to four racks. For each additional block of four racks added to this configuration, an additional pump module will also be installed. Pump modules contain pumps, filters, and heat exchangers.
The raised floor of the data center (although not necessary for a CarnotJet system installation) allows the hoses that connect the racks to the pump module to remain out of sight. Also shown below are the PDUs that mount to the back of the server racks.
The view between two of the racks shows the hoses that deliver coolant to and away from the racks. The hoses connect to two input/output manifolds under the pump module that consolidate (input) and disperse (output) coolant as it enters or exits the pump module.
We’ll have more photos from this installation available in the future, so be sure to check the Gallery soon.
Midas Green Technologies, our customer since January 2011, has produced some excellent videos recently featuring their GRC-powered green data center.
Here’s the video as posted on Data Center Knowledge:
Another video from Midas details the server modifications required to install any OEM server in the CarnotJet dielectric fluid cooling system:
A new mini-whitepaper produced by Green Revolution Cooling this week explains how wet-bulb temperature is related to evaporative cooling tower efficiency. Using empirical data from two of the hottest regions in the United States, we determined that it’s possible to leverage the efficiency of free cooling in our liquid-submersion cooling system all year long.
You can find the new whitepaper, “Free Cooling in the CarnotJet System: When is it Available?” right here.