Nav: Home

Big PanDA tackles big data for physics and other future extreme scale scientific applications

August 16, 2016

UPTON, NY-A billion times per second, particles zooming through the Large Hadron Collider (LHC) at CERN, the European Organization for Nuclear Research, smash into one another at nearly the speed of light, emitting subatomic debris that could help unravel the secrets of the universe. Collecting the data from those collisions and making it accessible to more than 6000 scientists in 45 countries, each potentially wanting to slice and analyze it in their own unique ways, is a monumental challenge that pushes the limits of the Worldwide LHC Computing Grid (WLCG), the current infrastructure for handling the LHC's computing needs. With the move to higher collision energies at the LHC, the demand just keeps growing.

To help meet this unprecedented demand and supplement the WLCG, a group of scientists working at U.S. Department of Energy (DOE) national laboratories and collaborating universities has developed a way to fit some of the LHC simulations that demand high computing power into untapped pockets of available computing time on one of the nation's most powerful supercomputers-similar to the way tiny pebbles can fill the empty spaces between larger rocks in a jar. The group-from DOE's Brookhaven National Laboratory, Oak Ridge National Laboratory (ORNL), University of Texas at Arlington, Rutgers University, and University of Tennessee, Knoxville-just received $2.1 million in funding for 2016-2017 from DOE's Advanced Scientific Computing Research (ASCR) program to enhance this "workload management system," known as Big PanDA, so it can help handle the LHC data demands and be used as a general workload management service at DOE's Oak Ridge Leadership Computing Facility (OLCF, https://www.olcf.ornl.gov/), a DOE Office of Science User Facility at ORNL.

"The implementation of these ideas in an operational-scale demonstration project at OLCF could potentially increase the use of available resources at this Leadership Computing Facility by five to ten percent," said Brookhaven physicist Alexei Klimentov, a leader on the project. "Mobilizing these previously unusable supercomputing capabilities, valued at millions of dollars per year, could quickly and effectively enable cutting-edge science in many data-intensive fields."

Proof-of-concept tests using the Titan supercomputer at Oak Ridge National Laboratory have been highly successful. This Leadership Computing Facility typically handles large jobs that are fit together to maximize its use. But even when fully subscribed, some 10 percent of Titan's computing capacity might be sitting idle-too small to take on another substantial "leadership class" job, but just right for handling smaller chunks of number crunching. The Big PanDA (for Production and Distributed Analysis) system takes advantage of these unused pockets by breaking up complex data analysis jobs and simulations for the LHC's ATLAS and ALICE experiments and "feeding" them into the "spaces" between the leadership computing jobs. When enough capacity is available to run a new big job, the smaller chunks get kicked out and reinserted to fill in any remaining idle time.

"Our team has managed to access opportunistic cycles available on Titan with no measurable negative effect on the supercomputer's ability to handle its usual workload," Klimentov said. He and his collaborators estimate that up to 30 million core hours or more per month may be harvested using the Big PanDA approach. From January through July of 2016, ATLAS detector simulation jobs ran for 32.7 million core hours on Titan, using only opportunistic, backfill resources. The results of the supercomputing calculations are shipped to and stored at the RHIC & ATLAS Computing Facility, a Tier 1 center for the WLCG located at Brookhaven Lab, so they can be made available to ATLAS researchers across the U.S. and around the globe.

The goal now is to translate the success of the Big PanDA project into operational advances that will enhance how the OLCF handles all of its data-intensive computing jobs. This approach will provide an important model for future exascale computing, increasing the coherence between the technology base used for high-performance, scalable modeling and simulation and that used for data-analytic computing.

"This is a novel and unique approach to workload management that could run on all current and future leadership computing facilities," Klimentov said.

Specifically, the new funding will help the team develop a production scale operational demonstration of the PanDA workflow within the OLCF computational and data resources; integrate OLCF and other leadership facilities with the Grid and Clouds; and help high-energy and nuclear physicists at ATLAS and ALICE-experiments that expect to collect 10 to 100 times more data during the next 3 to 5 years-achieve scientific breakthroughs at times of peak LHC demand.

As a unifying workload management system, Big PanDA will also help integrate Grid, leadership-class supercomputers, and Cloud computing into a heterogeneous computing architecture accessible to scientists all over the world as a step toward a global cyberinfrastructure.

"The integration of heterogeneous computing centers into a single federated distributed cyberinfrastructure will allow more efficient utilization of computing and disk resources for a wide range of scientific applications," said Klimentov, noting how the idea mirrors Aristotle's assertion that "the whole is greater than the sum of its parts."
-end-
This project is supported by the DOE Office of Science.

Brookhaven National Laboratory is supported by the Office of Science of the U.S. Department of Energy. The Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit science.energy.gov.

One of ten national laboratories overseen and primarily funded by the Office of Science of the U.S. Department of Energy (DOE), Brookhaven National Laboratory conducts research in the physical, biomedical, and environmental sciences, as well as in energy technologies and national security. Brookhaven Lab also builds and operates major scientific facilities available to university, industry and government researchers. Brookhaven is operated and managed for DOE's Office of Science by Brookhaven Science Associates, a limited-liability company founded by the Research Foundation for the State University of New York on behalf of Stony Brook University, the largest academic user of Laboratory facilities, and Battelle, a nonprofit applied science and technology organization.

Media contacts: Karen McNulty Walsh, (631) 344-8350, kmcnulty@bnl.gov, or Peter Genzer, (631) 344-3174, genzer@bnl.gov

DOE/Brookhaven National Laboratory

Related Large Hadron Collider Articles:

The Large Hadron Collider -- the greatest adventure in town
World Scientific's latest book, 'The Large Hadron Collider,' homes in on the ATLAs Experiment to illustrate how and why this process happens, why it has an importance well beyond traditional spin-off and how it adds new meaning to the cost of this research and to the value of international collaboration.
Why odds are against a large Zika outbreak in the US
Is the United States at risk for a large-scale outbreak of Zika or other mosquito-borne disease?
Laser R&D focuses on next-gen particle collider
A set of new laser systems and proposed upgrades at Berkeley Lab's BELLA Center will propel long-term plans for a more compact and affordable ultrahigh-energy particle collider.
Physicist offers leading theory about mysterious Large Hadron Collider excess
K.C. Kong's idea: a sequence of particles at different masses -- without a 'resonance' particle at 750 GeV -- triggered the mystery signal at the Large Hadron Collider.
The large-scale stability of chromosomes
A new study led by the SISSA of Trieste and published in PLOS Computational Biology adds detail to the theoretical models used in chromatin simulations and demonstrates that even when made up of a mixture of fibres with different properties chromatin does not alter its three-dimensional structure above a certain spatial resolution.
Syracuse physicists help restart Large Hadron Collider
After months of winter hibernation, the LHC has resumed smashing beams of protons together, in attempt to recreate conditions of the first millionth of a second of the universe, some 13.9 billion years ago.
A quasiparticle collider
Experiments prove that basic collider concepts from particle physics can be transferred to solid-state research.
Physicists offer theories to explain mysterious collision at Large Hadron Collider
Physicists around the world were puzzled recently when an unusual bump appeared in the signal of the Large Hadron Collider, the world's largest and most powerful particle accelerator, causing them to wonder if it was a new particle previously unknown, or perhaps even two new particles.
Inadequate policies for hunting large carnivores
Many policies regulating carnivore hunting do not adequately acknowledge and address the negative effects of hunting on demography and population dynamics, authors of this Policy Forum say.
ALCF helps tackle the Large Hadron Collider's big data challenge
To help tackle the considerable challenge of interpreting data, researchers from the US Department of Energy's (DOE's) Argonne National Laboratory are demonstrating the potential of simulating collision events with Mira, a 10-petaflops IBM Blue Gene/Q supercomputer at the Argonne Leadership Computing Facility (ALCF), a DOE Office of Science User Facility.

Related Large Hadron Collider Reading:

Large Hadron Collider Manual (Haynes Manuals)
by Gemma Lavender (Author)

What is the universe made of? How did it start? This Manual tells the story of how physicists are seeking answers to these questions using the world’s largest particle smasher – the Large Hadron Collider – at the CERN laboratory on the Franco-Swiss border. Beginning with the first tentative steps taken to build the machine, the digestible text, supported by color photographs of the hardware involved, along with annotated schematic diagrams of the physics experiments, covers the particle accelerator’s greatest discoveries – from both the perspective of the writer and the scientists... View Details


The Large Hadron Collider: The Extraordinary Story of the Higgs Boson and Other Stuff That Will Blow Your Mind
by Don Lincoln (Author)

Since 2008 scientists have conducted experiments in a hyperenergized, 17-mile supercollider beneath the border of France and Switzerland. The Large Hadron Collider (or what scientists call "the LHC") is one of the wonders of the modern world―a highly sophisticated scientific instrument designed to recreate in miniature the conditions of the universe as they existed in the microseconds following the big bang. Among many notable LHC discoveries, one led to the 2013 Nobel Prize in Physics for revealing evidence of the existence of the Higgs boson, the so-called God particle.

Picking... View Details


The Large Hadron Collider: The Greatest Adventure in Town and Ten Reasons Why it Matters, as Illustrated by the ATLAS Experiment
by Andrew J Millington (Author), Markus Nordberg (Editor), Thorsten Wengler (Editor), Rob Mcpherson (Editor)

When the discovery of the Higgs Boson at Cern hit the headlines in 2012, the world was stunned by this achievement of modern science. Less well appreciated, however, were the many ways in which this benefited wider society. The Large Hadron Collider - The Greatest Adventure in Town charts a path through the cultural, economic and medical gains of modern particle physics. It illustrates these messages through the Atlas experiment at Cern, one of the two big experiments which found the Higgs particle. Moving clear of in-depth physics analysis, it draws on the unparalleled curiosity about... View Details


Who Cares about Particle Physics?: Making Sense of the Higgs Boson, the Large Hadron Collider and CERN
by Pauline Gagnon (Author)

CERN, the European Laboratory for particle physics, regularly makes the news. What kind of research happens at this international laboratory and how does it impact people's daily lives? Why is the discovery of the Higgs boson so important? Particle physics describes all matter found on Earth, in stars and all galaxies but it also tries to go beyond what is known to describe dark matter, a form of matter five times more prevalent than the known, regular matter. How do we know this mysterious dark matter exists and is there a chance it will be discovered soon? About sixty countries contributed... View Details


The Large Hadron Collider
by Lyndon Evans (Editor)

This richly illustrated, full color book dissects the technology of the LHC into its component parts, showing how state-of-the-art techniques have been applied to beam control (injection, stabilization, acceleration, and dumping); cryogenics; superconducting magnet technology; and vacuum. It also describes the civil engineering and logistical challenges of the construction of the machine as well as the theoretical challenges that drove the scientific community to build the LHC. Each of the experiments is explained in terms of scientific goals, new methods of particle detection, and the... View Details


The Large Hadron Collider: Unraveling the Mysteries of the Universe (Astronomers' Universe)
by Martin Beech (Author)

It may at first seem that the world of subatomic physics is far removed from our every day lives. Isn’t it all just a waste of time and taxpayers' money? Hopefully, all who read this book will come to a different conclusion. Collider physics is all about our origins, and this aspect alone makes it worthy of our very best attention. The experiments conducted within the vast collider chambers are at the forefront of humanity’s quest to unweave the great tapestry that is the universe. Everything is connected. Within the macrocosm is the microcosm. By knowing how matter is structured, how... View Details


The Large Hadron Collider: Harvest of Run 1
by Thomas Schörner-Sadenius (Editor)

This comprehensive volume summarizes and structures the multitude of results obtained at the LHC in its first running period and draws the grand picture of today’s physics at a hadron collider. Topics covered are Standard Model measurements, Higgs and top-quark physics, flavour physics, heavy-ion physics, and searches for supersymmetry and other extensions of the Standard Model. Emphasis is placed on overview and presentation of the lessons learned. Chapters on detectors and the LHC machine and a thorough outlook into the future complement the book. The individual chapters are written by... View Details


The Large Hadron Collider Pop-up Book: Voyage to the Heart of Matter
by Anton Radevsky & Emma Sanders (author) (Author)

View Details


The Large Hadron Collider (Great Idea)
by Bonnie Juettner Fernandes (Author), Dr Don Lincoln PH.D. (Consultant Editor)

"Describes the struggles and accomplishments in building the Hadron Super Collider, the tool that scientists use to understand how the universe first began. Includes glossary, websites, and bibliography for further reading"-- View Details


Smash!: Exploring the Mysteries of the Universe with the Large Hadron Collider
by Sara L Latta (Author), Sara Latta (Author), Jeff Weigel (Illustrator)

What is the universe made of? At CERN, the European Organization for Nuclear Research, scientists have searched for answers to this question using the largest machine in the world: the Large Hadron Collider. It speeds up tiny particles, then smashes them together—and the collision gives researchers a look at the building blocks of the universe.



Nick and Sophie, two cousins, are about to visit CERN for a tour of the mysteries of the cosmos. Sophie's a physics wiz. Nick, not so much. But by the time they're through, Nick and Sophie will both feel the power of hidden... View Details

Best Science Podcasts 2018

We have hand picked the best science podcasts for 2018. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

The Person You Become
Over the course of our lives, we shed parts of our old selves, embrace new ones, and redefine who we are. This hour, TED speakers explore ideas about the experiences that shape the person we become. Guests include aerobatics pilot and public speaker Janine Shepherd, writers Roxane Gay and Taiye Selasi, activist Jackson Bird, and fashion executive Kaustav Dey.
Now Playing: Science for the People

#479 Garden of Marvels (Rebroadcast)
This week we're learning about botany and the colorful science of gardening. Author Ruth Kassinger joins us to discuss her book "A Garden of Marvels: How We Discovered that Flowers Have Sex, Leaves Eat Air, and Other Secrets of the Way Plants Work." And we'll speak to NASA researcher Gioia Massa about her work to solve the technical challenges of gardening in space.