Nuclear weapon simulations show performance in molecular detail

June 05, 2012

WEST LAFAYETTE, Ind. - U.S. researchers are perfecting simulations that show a nuclear weapon's performance in precise molecular detail, tools that are becoming critical for national defense because international treaties forbid the detonation of nuclear test weapons.

The simulations must be operated on supercomputers containing thousands of processors, but doing so has posed reliability and accuracy problems, said Saurabh Bagchi, an associate professor in Purdue University's School of Electrical and Computer Engineering.

Now researchers at Purdue and high-performance computing experts at the National Nuclear Security Administration's (NNSA) Lawrence Livermore National Laboratory have solved several problems hindering the use of the ultra-precise simulations. NNSA is the quasi-independent agency within the U.S. Department of Energy that oversees the nation's nuclear security activities.

The simulations, which are needed to more efficiently certify nuclear weapons, may require 100,000 machines, a level of complexity that is essential to accurately show molecular-scale reactions taking place over milliseconds, or thousandths of a second. The same types of simulations also could be used in areas such as climate modeling and studying the dynamic changes in a protein's shape.

Such highly complex jobs must be split into many processes that execute in parallel on separate machines in large computer clusters, Bagchi said.

"Due to natural faults in the execution environment there is a high likelihood that some processing element will have an error during the application's execution, resulting in corrupted memory or failed communication between machines," Bagchi said. "There are bottlenecks in terms of communication and computation."

These errors are compounded as long as the simulation continues to run before the glitch is detected and may cause simulations to stall or crash altogether.

"We are particularly concerned with errors that corrupt data silently, possibly generating incorrect results with no indication that the error has occurred," said Bronis R. de Supinski, co-leader of the ASC Application Development Environment Performance Team at Lawrence Livermore. "Errors that significantly reduce system performance are also a major concern since the systems on which the simulations run are very expensive."

Advanced Simulation and Computing is the computational arm of NNSA's Stockpile Stewardship Program, which ensures the safety, security and reliability of the nation's nuclear deterrent without underground testing.

New findings will be detailed in a paper to be presented during the Annual IEEE/IFIP International Conference on Dependable Systems and Networks from June 25-28 in Boston. Recent research findings were detailed in two papers last year, one presented during the IEEE Supercomputing Conference and the other during the International Symposium on High-Performance Parallel and Distributed Computing.

The researchers have developed automated methods to detect a glitch soon after it occurs.

"You want the system to automatically pinpoint when and in what machine the error took place and also the part of the code that was involved," Bagchi said. "Then, a developer can come in, look at it and fix the problem."

One bottleneck arises from the fact that data are streaming to a central server.

"Streaming data to a central server works fine for a hundred machines, but it can't keep up when you are streaming data from a thousand machines," said Purdue doctoral student Ignacio Laguna, who worked with Lawrence Livermore computer scientists. "We've eliminated this central brain, so we no longer have that bottleneck."

Each machine in the supercomputer cluster contains several cores, or processors, and each core might run one "process" during simulations. The researchers created an automated method for "clustering," or grouping the large number of processes into a smaller number of "equivalence classes" with similar traits. Grouping the processes into equivalence classes makes it possible to quickly detect and pinpoint problems.

"The recent breakthrough was to be able to scale up the clustering so that it works with a large supercomputer," Bagchi said.

Lawrence Livermore computer scientist Todd Gamblin came up with the scalable clustering approach.

A lingering bottleneck in using the simulations is related to a procedure called checkpointing, or periodically storing data to prevent its loss in case a machine or application crashes. The information is saved in a file called a checkpoint and stored in a parallel system distant from the machines on which the application runs.

"The problem is that when you scale up to 10,000 machines, this parallel file system bogs down," Bagchi said. "It's about 10 times too much activity for the system to handle, and this mismatch will just become worse because we are continuing to create faster and faster computers."

Doctoral student Tanzima Zerin and Rudolf Eigenmann, a professor of electrical and computer engineering, along with Bagchi, led work to develop a method for compressing the checkpoints, similar to the compression of data for images.

"We're beginning to solve the checkpointing problem," Bagchi said. "It's not completely solved, but we are getting there."

The checkpointing bottleneck must be solved in order for researchers to create supercomputers capable of "exascale computing," or 1,000 quadrillion operations per second.

"It's the Holy Grail of supercomputing," Bagchi said.
The research has been funded by Lawrence Livermore and the National Science Foundation. The work also involves Lawrence Livermore scientists Greg Bronevetsky, Dong H. Ahn, Martin Schulz and IBM Austin researcher Mootaz Elnozahy.

Laguna was awarded a George Michael Memorial High Performance Computing Fellowship during the Supercomputing 2011 Conference in recognition of his work on the project. The Supercomputing Conference paper can be accessed online at IEEExplore or ACM Digital Library or from the research group's home page at

Purdue researchers did not work with the actual classified nuclear weapons software code, but instead used generic benchmarks, a set of programs designed to help evaluate the performance of parallel supercomputers.

Writer: Emil Venere, 765-494-4709,

Sources: Saurabh Bagchi, 765-494-3362, Donald B. Johnston, senior editor, press officer, Lawrence Livermore National Laboratory, 925-423-4902,

Related website:
Surabh Bagchi:


Employees at Lawrence Livermore National Laboratory work on a high-performance computer. Purdue researchers have collaborated with the national laboratory, using a similar high-performance computer to improve simulations that show a nuclear weapon's performance in precise molecular detail. (Photo courtesy of Lawrence Livermore National Laboratory)

A publication-quality photo is available at

Abstract on the research in this release is available at:

Purdue University

Related Data Articles from Brightsurf:

Keep the data coming
A continuous data supply ensures data-intensive simulations can run at maximum speed.

Astronomers are bulging with data
For the first time, over 250 million stars in our galaxy's bulge have been surveyed in near-ultraviolet, optical, and near-infrared light, opening the door for astronomers to reexamine key questions about the Milky Way's formation and history.

Novel method for measuring spatial dependencies turns less data into more data
Researcher makes 'little data' act big through, the application of mathematical techniques normally used for time-series, to spatial processes.

Ups and downs in COVID-19 data may be caused by data reporting practices
As data accumulates on COVID-19 cases and deaths, researchers have observed patterns of peaks and valleys that repeat on a near-weekly basis.

Data centers use less energy than you think
Using the most detailed model to date of global data center energy use, researchers found that massive efficiency gains by data centers have kept energy use roughly flat over the past decade.

Storing data in music
Researchers at ETH Zurich have developed a technique for embedding data in music and transmitting it to a smartphone.

Life data economics: calling for new models to assess the value of human data
After the collapse of the blockchain bubble a number of research organisations are developing platforms to enable individual ownership of life data and establish the data valuation and pricing models.

Geoscience data group urges all scientific disciplines to make data open and accessible
Institutions, science funders, data repositories, publishers, researchers and scientific societies from all scientific disciplines must work together to ensure all scientific data are easy to find, access and use, according to a new commentary in Nature by members of the Enabling FAIR Data Steering Committee.

Democratizing data science
MIT researchers are hoping to advance the democratization of data science with a new tool for nonstatisticians that automatically generates models for analyzing raw data.

Getting the most out of atmospheric data analysis
An international team including researchers from Kanazawa University used a new approach to analyze an atmospheric data set spanning 18 years for the investigation of new-particle formation.

Read More: Data News and Data Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to