image: The new HoreKa-Teal upgrade (right) with 88 hot-water-cooled graphics processing units next to the “Hochleistungsrechner Karlsruhe” (in the background), which was commissioned in June 2021. (Photo: Markus Breig, KIT)
Credit: Photo: Markus Breig, KIT
“With supercomputers, we can perform complex simulations, data analyses and calculations that are essential to scientific research and help us find answers to today’s most pressing questions,” said Professor Kora Kristof, Vice-President Digitalization and Sustainability at KIT. “But we also need to keep energy consumption and other sustainability aspects in mind. Our outstanding position in the top 10 of the global Green500 list shows that we at KIT can make performance and energy efficiency compatible.”
“Our supercomputer brings together a wide range of components and delivers extremely high calculation performance, for example in solving systems of equations or training neural networks for artificial intelligence,” said Professor Martin Frank, Director of the Scientific Computing Center (SCC) at KIT. “Efficient cooling, smart power management and optimized hardware architectures all help to minimize energy consumption and promote sustainability in research.”
New High-efficiency Accelerator Processors
HoreKa is a hybrid system consisting of 668 accelerator processors and 60,000 standard commercially available processors (CPUs). To these two groups, called HoreKa-Green and HoreKa-Blue, the KIT researchers have added HoreKa-Teal with 88 new NVIDIA H100 processors to increase HoreKa’s performance from 17 to over 20 petaFLOPS; one petaFLOP corresponds to a quadrillion operations per second. The accelerators perform calculations with much greater energy efficiency than before because they have been produced in an advanced manufacturing process with considerably smaller processor structures.
The computing performance of the new accelerators was measured with a special benchmark application, the high-performance LINPACK. This was a requirement for nominating HoreKa for the list of the most energy-efficient computers. The benchmark measures a computing system’s ability to solve a defined system of equations. The computing performance is determined from the time needed to calculate a solution, and the energy efficiency is derived from the amount of energy used. Measured at 64 gigaFLOPS per watt, HoreKa took 6th place in the global ranking.
Optimizing Various Parameters Improves Performance
To reach their impressive result, the SCC teams optimized various other parameters such as the maximum power consumption of the accelerators. “Even the smallest of adjustments can affect energy efficiency,” said Simon Raffeiner, Technical Manager of the National High Performance Computing Center at KIT (NHR@KIT). “We’ve built up a lot of expertise over the years while also raising awareness among our users. In the end, the energy efficiency according to a certain benchmark isn’t really what matters. All of the applications that actually run on the system have to be as efficient as possible.”
For nearly ten years, KIT has used an efficient hot water cooling system for its high-performance computers. “It minimizes the energy consumption for cooling the components throughout the year, and the waste heat can be used for things like heating our office buildings,” Frank said. The SCC received the German Computing Center Prize in 2017 for its groundbreaking role in this technology.
Comprehensive User Support
A special team at the SSC helps researchers optimize the performance and energy efficiency of their programs. With the Job Performance Monitoring System, they can display and analyze program runtime parameters such as energy consumption or CPU and GPU utilization.
Details about KIT Center Information · Systems · Technologies (KCIST)