Assessing the performance of multiple influenza forecasting models

January 15, 2019

AMHERST, Mass. - In what the authors believe is the first documented comparison of several real-time infectious disease forecasting models by different teams across many seasons, five research groups report this week that a majority of models consistently showed higher accuracy than historical baseline models.

Led by biostatistician Nicholas Reich at the University of Massachusetts Amherst, the research teams formed a consortium called the FluSight Network and compared the forecast accuracy of 20 models compared to a historical baseline seasonal average. Accurately predicting the size and timing of infectious disease outbreaks helps public health officials in planning appropriate responses, they say.

Reich, at the School of Public Health and Health Sciences at UMass Amherst, says this paper, the first of several planned from this research consortium, offers "a survey of which models do well, when and why, plus a sort of meta-analysis of the state of the field right now. We have brought together some of the top flu forecasting teams in the world, and through this collaboration have enabled an apples-to-apples comparison of our different methods and results." Details appear in the current online edition of Proceedings of the National Academy of Sciences.

He and colleagues write, "Across all regions of the United States, over half of the models showed consistently better performance than the historical baseline when forecasting incidence of influenza-like illness one, two and three weeks ahead of available data and when forecasting the timing and magnitude of the seasonal peak."

Influenza infects an estimated 9 million to 35 million individuals each year in the United States and is a contributing cause for between 12,000 and 56,000 deaths annually, the researchers point out. More than 6 million Americans have already caught influenza this flu season, Reich adds, and as many as 80,000 have ended up in the hospital, federal health officials reported most recently last Friday.

In addition to comparing the contributed models, the efforts of the FluSight Network has enabled the creation of an "ensemble" model. Each group uses their own advanced analytical techniques, Reich explains, to come up with a forecasted flu trajectory for the year. These individual forecasts are combined into a single "ensemble" forecast that is sent to the CDC each week.

When compared to historical trends, these forecasts help the CDC and other public health officials more effectively plan and respond to the evolving seasonal epidemic. The FluSight Network ensemble was one of the top-performing real-time models in the 2017/2018 season. The CDC uses the FluSight Network ensemble approach in their internal and external communication and planning for the influenza season. Peer-reviewed research on the ensemble model is forthcoming, says Reich.

Overall, the authors state, "Our collaborative, team science approach highlights the ability of multiple research groups working together to uncover patterns and trends of model performance that are harder to observe in single-team studies."

"The field of infectious disease forecasting is in its infancy and we expect that innovation will spur improvements in forecasting in the coming years," they add. "Public health officials are still learning how best to integrate forecasts into real-time decision making. Close collaboration between public health policy-makers and quantitative modelers is necessary to ensure that forecasts have maximum impact and are appropriately communicated to the public and the broader public health community."

Reich worked with colleagues at Carnegie Mellon University, the University of Texas at Austin, the U.S. Centers for Disease Control and Prevention (CDC), Columbia University, Los Alamos National Laboratory and Mount Holyoke College on the project.

The researchers note that in the past 15 years, increased interest in forecasting infectious disease has been fueled in part by the promise of "big data," and the idea that new data streams could lead to measurable improvements in how disease transmission is measured, forecast and controlled. "While multi-model comparisons exist in the literature for single-outbreak performance, here we compare a consistent set of models over seven influenza seasons," 2010 through 2017.

University of Massachusetts at Amherst

Related Influenza Articles from Brightsurf:

Predicting influenza epidemics
Researchers at Linköping University, Sweden, have developed a unique method to predict influenza epidemics by combining several sources of data.

Common cold combats influenza
As the flu season approaches, a strained public health system may have a surprising ally -- the common cold virus.

Scent-sensing cells have a better way to fight influenza
Smell receptors that line the nose get hit by Influenza B just like other cells, but they are able to clear the infection without dying.

New antivirals for influenza and Zika
Leuven researchers have deployed synthetic amyloids to trigger protein misfolding as a strategy to combat the influenza A and Zika virus.

Assessment of deaths from COVID-19, seasonal influenza
Publicly available data were used to analyze the number of deaths from seasonal influenza deaths compared with deaths from COVID-19.

Obesity promotes virulence of influenza
Obesity promotes the virulence of the influenza virus, according to a study conducted in mice published in mBio, an open-access journal of the American Society for Microbiology.

Influenza: combating bacterial superinfection with the help of the microbiota
Frenc researchers and from Brazilian (Belo Horizonte), Scottish (Glasgow) and Danish (Copenhagen) laboratories have shown for the first time in mice that perturbation of the gut microbiota caused by the influenza virus favours secondary bacterial superinfection.

Chemists unveil the structure of an influenza B protein
MIT chemists have discovered the structure of an influenza B protein called BM2, a finding that could help researchers design drugs that block the protein and help prevent the virus from spreading.

How proteins help influenza A bind and slice its way to cells
Researchers have provided new insight on how two proteins help influenza A virus particles fight their way to human cells.

Eating elderberries can help minimize influenza symptoms
Conducted by Professor Fariba Deghani, Dr. Golnoosh Torabian and Dr.

Read More: Influenza News and Influenza Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to