A new method to evaluate overall performance of a climate model

November 03, 2017

Many climate-related studies, such as detection and attribution of historical climate change, projections of future climate and environments, and adaptation to future climate change, heavily rely on the performance of climate models. Concisely summarizing and evaluating model performance becomes increasingly important for climate model intercomparison and application, especially when more and more climate models participate in international model intercomparison projects.

Most of current model evaluation metrics, e.g., root mean square error (RMSE), correlation coefficient, standard deviation, measure the model performance in simulating individual variable. However, one often needs to evaluate a model's overall performance in simulating multiple variables. To fill this gap, an article published in Geosci. Model Dev., presents a new multivariable integrated evaluation (MVIE) method.

"The MVIE includes three levels of statistical metrics, which can provide a comprehensive and quantitative evaluation on model performance." Says XU, the first author of the study from the Institute of Atmospheric Physics, Chinese Academy of Sciences. The first level of metrics, including the commonly used correlation coefficient, RMS value, and RMSE, measures model performance in terms of individual variables. The second level of metrics, including four newly developed statistical quantities, provides an integrated evaluation of model performance in terms of simulating multiple fields. The third level of metrics, multivariable integrated evaluation index (MIEI), further summarizes the three statistical quantities of second level of metrics into a single index and can be used to rank the performances of various climate models. Different from the commonly used RMSE-based metrics, the MIEI satisfies the criterion that a model performance index should vary monotonically as the model performance improves.

According to the study, higher level of metrics is derived from and concisely summarizes the lower level of metrics. "Inevitably, the higher level of metrics loses detailed statistical information in contrast to the lower level of metrics." XU therefore suggests, "To provide a more comprehensive and detailed evaluation of model performance, one can use all three levels of metrics."
-end-


Institute of Atmospheric Physics, Chinese Academy of Sciences

Related Climate Models Articles from Brightsurf:

Polar ice, atmospheric water vapor biggest drivers of variation among climate models
A Florida State University researcher is part of a team that has found varying projections on global warming trends put forth by climate change scientists can be explained by differing models' predictions regarding ice loss and atmospheric water vapor.

Revising climate models with new aerosol field data
Advanced field measurements of how quickly aerosol particles are pulled out of the air can help improve climate predictions - and air quality forecasts.

Simpler models may be better for determining some climate risk
Typically, computer models of climate become more and more complex as researchers strive to capture more details of our Earth's system, but according to a team of Penn State researchers, to assess risks, less complex models, with their ability to better sample uncertainties, may be a better choice.

Atmospheric scientists study fires to resolve ice question in climate models
Black carbon from fires is an important short-term climate driver because it can affect the formation and composition of clouds.

New soil models may ease atmospheric CO2, climate change
To remove carbon dioxide from the Earth's atmosphere in an effort to slow climate change, scientists must get their hands dirty and peek underground.

Patterns in permafrost soils could help climate change models
A team of scientists spent the past four summers measuring permafrost soils across a 5,000 square-mile swath of Alaska's North Slope.

Latest climate models show more intense droughts to come
An analysis of new climate model projections by Australian researchers from the ARC Centre of Excellence for Climate Extremes shows southwestern Australia and parts of southern Australia will see longer and more intense droughts due to a lack of rainfall caused by climate change.

Some of the latest climate models provide unrealistically high projections of future warming
A new study from University of Michigan climate researchers concludes that some of the latest-generation climate models may be overly sensitive to carbon dioxide increases and therefore project future warming that is unrealistically high.

A Europe covered in grasslands or forests: innovation and research on climate models
An experiment to better understand how atmospheric variables respond to land use changes.

How tiny water droplets form can have a big impact on climate models
Droplets and bubbles are formed nearly everywhere, from boiling our morning coffee, to complex industrial processes and even volcanic eruptions.

Read More: Climate Models News and Climate Models Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.