Researchers use LRZ HPC resources to perform largest-ever supersonic turbulence simulation

January 11, 2021

Through the centuries, scientists and non-scientists alike have looked at the night sky and felt excitement, intrigue, and overwhelming mystery while pondering questions about how our universe came to be, and how humanity developed and thrived in this exact place and time. Early astronomers painstakingly studied stars' subtle movements in the night sky to try and determine how our planet moves in relation to other celestial bodies. As technology has increased, so too has our understanding of how the universe works and our relative position within it.

What remains a mystery, however, is a more detailed understanding of how stars and planets formed in the first place. Astrophysicists and cosmologists understand that the movement of materials across the interstellar medium (ISM) helped form planets and stars, but how this complex mixture of gas and dust--the fuel for star formation--moves across the universe is even more mysterious.

To help better understand this mystery, researchers have turned to the power of high-performance computing (HPC) to develop high-resolution recreations of phenomena in the galaxy. Much like several terrestrial challenges in engineering and fluid dynamics research, astrophysicists are focused on developing a better understanding of the role of turbulence in helping shape our universe.

Over the last several years, a multi-institution collaboration being led by Australian National University Associate Professor Christoph Federrath and Heidelberg University Professor Ralf Klessen has been using HPC resources at the Leibniz Supercomputing Centre (LRZ) in Garching near Munich to study turbulence's influence on galaxy formation. The team recently revealed the so-called "sonic scale" of astrophysical turbulence--marking the transition moving from supersonic to subsonic speeds (faster or slower than the speed of sound, respectively)--creating the largest-ever simulation of supersonic turbulence in the process. The team published its research in Nature Astronomy.

Many scales in a simulation

To simulate turbulence in their research, Federrath and his collaborators needed to solve the complex equations of gas dynamics representing a wide variety of scales. Specifically, the team needed to simulate turbulent dynamics on both sides of the sonic scale in the complex, gaseous mixture travelling across the ISM. This meant having a sufficiently large simulation to capture these large-scale phenomena happening faster than the speed of sound, while also advancing the simulation slowly and with enough detail to accurately model the smaller, slower dynamics taking place at subsonic speeds.

"Turbulent flows only occur on scales far away from the energy source that drives on large scales, and also far away from the so-called dissipation (where the kinetic energy of the turbulence turns into heat) on small scales" Federrath said. "For our particular simulation, in which we want to resolve both the supersonic and the subsonic cascade of turbulence with the sonic scale in between, this requires at least 4 orders of magnitude in spatial scales to be resolved."

In addition to scale, the complexity of the simulations is another major computational challenge. While turbulence on Earth is one of the last major unsolved mysteries of physics, researchers who are studying terrestrial turbulence have one major advantage--the majority of these fluids are incompressible or only mildly compressible, meaning that the density of terrestrial fluids stays close to constant. In the ISM, though, the gaseous mix of elements is highly compressible, meaning researchers not only have to account for the large range of scales that influences turbulence, they also have to solve equations throughout the simulation to know the gases' density before proceeding.

Understanding the influence that density near the sonic scale plays in star formation is important for Federrath and his collaborators, because modern theories of star formation suggest that the sonic scale itself serves as a "Goldilocks zone" for star formation. Astrophysicists have long used similar terms to discuss how a planet's proximity to a star determines its ability to host life, but for star formation itself, the sonic scale strikes a balance between the forces of turbulence and gravity, creating the conditions for stars to more easily form. Scales larger than the sonic scale tend to have too much turbulence, leading to sparse star formation, while in smaller, subsonic regions, gravity wins the day and leads to localized clusters of stars forming.

In order to accurately simulate the sonic scale and the supersonic and subsonic scales on either side, the team worked with LRZ to scale its application to more than 65,000 compute cores on the SuperMUC HPC system. Having so many compute cores available allowed the team to create a simulation with more than 1 trillion resolution elements, making it the largest-ever simulation of its kind.

"With this simulation, we were able to resolve the sonic scale for the first time," Federrath said. "We found its location was close to theoretical predictions, but with certain modifications that will hopefully lead to more refined star formation models and more accurate predictions of star formation rates of molecular clouds in the universe. The formation of stars powers the evolution of galaxies on large scales and sets the initial conditions for planet formation on small scales, and turbulence is playing a big role in all of this. We ultimately hope that this simulation advances our understanding of the different types of turbulence on Earth and in space."

Cosmological collaborations and computational advancements

While the team is proud of its record-breaking simulation, it is already turning its attention to adding more details into its simulations, leading toward an even more accurate picture of star formation. Federrath indicated that the team planned to start incorporating the effects of magnetic fields on the simulation, leading to a substantial increase in memory for a simulation that already requires significant memory and computing power as well as multiple petabytes of storage--the current simulation requires 131 terabytes of memory and 23 terabytes of disk space per snapshot, with the whole simulation consisting of more than 100 snapshots.

Since he was working on his doctoral degree at the University of Heidelberg, Federrath has collaborated with staff at LRZ's AstroLab to help scale his simulations to take full advantage of modern HPC systems. Running the largest-ever simulation of its type serves as validation of the merits of this long-running collaboration. During this period, Federrath has worked closely with LRZ's Dr. Luigi Iapichino, Head of LRZ's AstroLab, who was a co-author on the Nature Astronomy publication.

"I see our mission as being the interface between the ever-increasing complexity of the HPC architectures, which is a burden on the application developers, and the scientists, which don't always have the right skill set for using HPC resource in the most effective way," Iapichino said. "From this viewpoint, collaborating with Christoph was quite simple because he is very skilled in programming for HPC performance. I am glad that in this kind of collaborations, application specialists are often full-fledged partners of researchers, because it stresses the key role centres' staffs play in the evolving HPC framework."
-Eric Gedenk

Gauss Centre for Supercomputing

Related Science Articles from Brightsurf:

75 science societies urge the education department to base Title IX sexual harassment regulations on evidence and science
The American Educational Research Association (AERA) and the American Association for the Advancement of Science (AAAS) today led 75 scientific societies in submitting comments on the US Department of Education's proposed changes to Title IX regulations.

Science/Science Careers' survey ranks top biotech, biopharma, and pharma employers
The Science and Science Careers' 2018 annual Top Employers Survey polled employees in the biotechnology, biopharmaceutical, pharmaceutical, and related industries to determine the 20 best employers in these industries as well as their driving characteristics.

Science in the palm of your hand: How citizen science transforms passive learners
Citizen science projects can engage even children who previously were not interested in science.

Applied science may yield more translational research publications than basic science
While translational research can happen at any stage of the research process, a recent investigation of behavioral and social science research awards granted by the NIH between 2008 and 2014 revealed that applied science yielded a higher volume of translational research publications than basic science, according to a study published May 9, 2018 in the open-access journal PLOS ONE by Xueying Han from the Science and Technology Policy Institute, USA, and colleagues.

Prominent academics, including Salk's Thomas Albright, call for more science in forensic science
Six scientists who recently served on the National Commission on Forensic Science are calling on the scientific community at large to advocate for increased research and financial support of forensic science as well as the introduction of empirical testing requirements to ensure the validity of outcomes.

World Science Forum 2017 Jordan issues Science for Peace Declaration
On behalf of the coordinating organizations responsible for delivering the World Science Forum Jordan, the concluding Science for Peace Declaration issued at the Dead Sea represents a global call for action to science and society to build a future that promises greater equality, security and opportunity for all, and in which science plays an increasingly prominent role as an enabler of fair and sustainable development.

PETA science group promotes animal-free science at society of toxicology conference
The PETA International Science Consortium Ltd. is presenting two posters on animal-free methods for testing inhalation toxicity at the 56th annual Society of Toxicology (SOT) meeting March 12 to 16, 2017, in Baltimore, Maryland.

Citizen Science in the Digital Age: Rhetoric, Science and Public Engagement
James Wynn's timely investigation highlights scientific studies grounded in publicly gathered data and probes the rhetoric these studies employ.

Science/Science Careers' survey ranks top biotech, pharma, and biopharma employers
The Science and Science Careers' 2016 annual Top Employers Survey polled employees in the biotechnology, biopharmaceutical, pharmaceutical, and related industries to determine the 20 best employers in these industries as well as their driving characteristics.

Three natural science professors win TJ Park Science Fellowship
Professor Jung-Min Kee (Department of Chemistry, UNIST), Professor Kyudong Choi (Department of Mathematical Sciences, UNIST), and Professor Kwanpyo Kim (Department of Physics, UNIST) are the recipients of the Cheong-Am (TJ Park) Science Fellowship of the year 2016.

Read More: Science News and Science Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to