Nav: Home

Researchers work toward systematic assessment of climate models

June 25, 2018

A research team based at the Pacific Northwest National Laboratory in Richland, Wash., has published the results of an international survey designed to assess the relative importance climate scientists assign to variables when analyzing a climate model's ability to simulate real-world climate.

The results, which have serious implications for studies using the models, were published as a cover article in Advances in Atmospheric Sciences on June 22, 2018.

"Climate modelers spend a lot of effort on calibrating certain model parameters to find a model version that does a credible job of simulating the Earth's observed climate," said Susannah Burrows, first author on the paper and a scientist at the Pacific Northwest National Laboratory who specializes in Earth systems analysis and modeling.

However, Burrows noted, there is little systematic study on how experts prioritize such variables as cloud cover or sea ice when judging the performance of climate models.

"Different people might come to slightly different assessments of how 'good' a particular model is, depending to large extent on which aspects they assign the most importance to," Burrows said.

One model, for example, may better simulate sea ice while another model excels in cloud simulation. Each scientist must strike a balance between their competing priorities and goals--a difficult thing to capture systematically in data analysis tools.

"In other words, there isn't a single, completely objective definition of what makes a 'good' climate model, and this fact is an obstacle to developing more systematic approaches and tools to assist in model evaluations and comparisons," Burrows said.

The researchers found, from a survey of 96 participants representing the climate modelling community, that experts took specific scientific objectives into consideration when rating variable importance. They found a high degree of consensus that certain variables are important in certain studies, such as rainfall and evaporation in the assessment of the Amazonian water cycle. That agreement falters on other variables, such as how important it is to accurately simulate surface winds when studying the water cycle in Asia.

Understanding these discrepancies and developing more systematic approaches to model assessment is important, according to Burrows, since each new version of a climate model must undergo significant evaluation, and calibration by multiple developers and users. The labor-intensive process can take more than a year.

The tuning, while designed to maintain a rigorous standard, requires experts to make trade-offs between competing priorities. A model may be calibrated at the expense of one scientific objective in order to achieve another.

Burrows is a member of an interdisciplinary research team at PNNL working to develop a more systematic solution to this assessment problem. The team includes Aritra Dasgupta, Lisa Bramer, and Sarah Reehl, experts in data science and visualization, and Yun Qian, Po-Lun Ma, and Phil Rasch, climate science experts.

To help climate modelers understand these trade-offs more clearly and efficiently, the visualization researchers are building interactive, intuitive visual interfaces that allow modelers to summarize and explore complex information about different aspects of model performance.

The data scientists are working to characterize expert climate model assessment in greater detail, building on the findings from the initial survey. Eventually, the researchers aim to blend a combination of metrics with human expertise to assess how well-suited climate models are for specific science objectives, as well as to predict how frequently experts will agree or disagree with that assessment.

"[We plan] to combine the best of both worlds, using computing to reduce manual effort and allowing scientists to more efficiently apply their human insight and judgment where it is most needed," Burrows said.
-end-
This work was conducted as part of a project led by Dasgupta, the principle investigator and co-author on the paper, and Bramer, co-principle investigator and co-author on the paper, under the Laboratory Directed Research and Development Program at the Pacific Northwest National Laboratory, a multi-program national laboratory operated by Battelle for the U.S. Department of Energy.

Institute of Atmospheric Physics, Chinese Academy of Sciences

Related Sea Ice Articles:

Melting sea ice may lead to more life in the sea
Every year an increasing amount of sea ice is melting in the Arctic.
Sea ice extent sinks to record lows at both poles
The Arctic sea ice maximum extent and Antarctic minimum extent are both record lows this year.
When the sea ice melts, juvenile polar cod may go hungry
Polar cod fulfill a key role in the Arctic food web, as they are a major source of food for seals, whales and seabirds alike.
NASA study improves forecasts of summer Arctic sea ice
The Arctic has been losing sea ice over the past several decades as Earth warms.
Melting sea ice may be speeding nature's clock in the Arctic
Spring is coming sooner to some plant species in the low Arctic of Greenland, while other species are delaying their emergence amid warming winters.
Sea ice hit record lows in November
Unusually high air temperatures and a warm ocean have led to a record low Arctic sea ice extent for November, according to scientists at the National Snow and Ice Data Center (NSIDC) at the University of Colorado Boulder.
My contribution to Arctic sea ice melt
Measurements reveal the relationship between individual CO2 emissions and the Arctic's shrinking summer sea ice.
See how Arctic sea ice is losing its bulwark against warming summers
Arctic sea ice, the vast sheath of frozen seawater floating on the Arctic Ocean and its neighboring seas, has been hit with a double whammy over the past decades: as its extent shrunk, the oldest and thickest ice has either thinned or melted away, leaving the sea ice cap more vulnerable to the warming ocean and atmosphere.
Tracking the amount of sea ice from the Greenland ice sheet
The Greenland ice sheet records information about Arctic climate going back more than 120.000 years.
Technique could assess historic changes to Antarctic sea ice and glaciers
Historic changes to Antarctic sea ice could be unravelled using a new technique pioneered by scientists at Plymouth University.

Related Sea Ice Reading:

Best Science Podcasts 2019

We have hand picked the best science podcasts for 2019. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Bias And Perception
How does bias distort our thinking, our listening, our beliefs... and even our search results? How can we fight it? This hour, TED speakers explore ideas about the unconscious biases that shape us. Guests include writer and broadcaster Yassmin Abdel-Magied, climatologist J. Marshall Shepherd, journalist Andreas Ekström, and experimental psychologist Tony Salvador.
Now Playing: Science for the People

#513 Dinosaur Tails
This week: dinosaurs! We're discussing dinosaur tails, bipedalism, paleontology public outreach, dinosaur MOOCs, and other neat dinosaur related things with Dr. Scott Persons from the University of Alberta, who is also the author of the book "Dinosaurs of the Alberta Badlands".