Nav: Home

Are star players over-rated in MLB? A key statistic doesn't stand up to scrutiny

July 22, 2018

In football, soccer and basketball, player quality has one obvious and inescapable metric: scoring touch-downs, goals, points. In baseball, it's more complicated--it isn't easy to directly associate a single player's play with team wins. This is why "wins above replacement player" (WARP) has become so popular: It's a statistic that manages to aggregate a player's total contribution to their team--offense and defense and primary position--into one easily understood figure.

The key to WARP's popularity is it distills all these factors into a universal estimate of value expressed in the number of wins a player is worth to his team if they were injured and had to be replaced by a freely available minor league player (in fact, a composite baseline minor leaguer). Created by the sabermetric baseball community, WARP plays a decisive role when it comes to voting on annual awards, such the MVP, Cy Young or Silver Slugger.

But does it actually measure--or even roughly estimate--player value?

Initially, Austin Brown--a PhD student at the University of Northern Colorado in the department of applied statistics and research methods--was interested in figuring out the relationship between injuries, teams catching the injury bug throughout the course of the regular season and team performance. But as he worked on the topic with his adviser--Jay Schaffer, professor of applied statistics and research methods--they began to look at the relationship between individual players and team performance. "Our thinking was that a player of lesser quality is unlikely to impact overall team performance in the same way as a player of higher playing quality," he says. "And that led us to WARP."

"But as we looked at the 2016 MLB season, we found that the WARP statistic wasn't measuring what we thought it ought to be measuring. We kind of took a step back and thought, 'Okay, this is kind of weird--this is definitely not what we suspected.'"

They suspected the 2016 season might be an aberration. "There were a couple of really good teams that happened to have a lot of injuries," says Brown. "So, we thought that might have been throwing it off." But as they extended their analysis to cover the 2014-2017 seasons, WARP wilted. "No matter how we looked at it, even when we accounted for differing independent variable combinations, it's just not related to wins," says Brown.

"When MLB players got hurt and the real replacement players were required to come in and play for some number of games, team performance across the board was not really impacted," says Brown. "If a player was on a poor team, the team tended to still play poorly. If a player was on a great team, the team still tended to play great."

When Los Angeles Dodgers' all-star pitcher Clayton Kershaw missed a sizable part of the 2016 season between July and September due to injury, the Dodgers actually played slightly better (in terms of winning percentage) than before he was hurt. That didn't mean he was a worse player than the player replacing him. It meant the quality of the team was more indicative of team performance.

"I think the problem with these overall measures of player quality in baseball is that baseball is not like soccer, it's not like hockey, it's not like basketball," Brown explains. "Look at the NBA finals last season. LeBron James pretty much got his team to the NBA finals by himself because the rest of the players are nowhere near his level of quality. But in baseball, it's just not the same. There are great players, of course, but it's really hard for them to carry a team just on their own because of the way the game is played."

Collecting and organizing the injury data was not as easy as one might think, given the passion for baseball statistics and the impact it has had on the game. Brown and Schaffer had to draw from the MLB, Baseball Prospectus, Baseball Reference, and Sports Illustrated. Which meant a lot of time cleaning and merging data. And perhaps that's why, in part, WARP took off--getting the right data to check it out was a huge challenge.

The next step, says Brown, is to drill further down into the data and figure out a statistic that is more indicative of player quality--one that derives from player-specific factors. "This will probably not be able to scale in the way that WARP does," says Brown, "which is what makes WARP so popular. But it will be more useful. Unlike WARP, it will be actually measuring something at the player level more directly related to team performance."
JSM Talk:

For details, contact: Austin Brown


About JSM 2018

JSM 2018 is the largest gathering of statisticians and data scientists in the world, taking place July 28-August 2, 2018, in Vancouver. Occurring annually since 1974, JSM is a joint effort of the American Statistical Association, International Biometric Society (ENAR and WNAR), Institute of Mathematical Statistics, Statistical Society of Canada, International Chinese Statistical Association, International Indian Statistical Association, Korean International Statistical Society, International Society for Bayesian Analysis, Royal Statistical Society and International Statistical Institute. JSM activities include oral presentations, panel sessions, poster presentations, professional development courses, an exhibit hall, a career service, society and section business meetings, committee meetings, social activities and networking opportunities.

About the American Statistical Association

The ASA is the world's largest community of statisticians and the oldest continuously operating professional science society in the United States. Its members serve in industry, government and academia in more than 90 countries, advancing research and promoting sound statistical practice to inform public policy and improve human welfare. For additional information, please visit the ASA website at

American Statistical Association

Related Baseball Articles:

Sleep extension improves response time, reduces fatigue in professional baseball players
Preliminary results from a new study suggest that short-term sleep extension improves response time and daytime functioning of professional baseball players.
Shoulder injuries in professional baseball players: A continuing puzzle
Professional baseball players struggle to return to a high level of play after biceps tenodesis (BP) surgery, according to research presented today at the American Orthopaedic Society for Sports Medicine's (AOSSM) Specialty Day in San Diego.
Study identifies modifiable risk factors for elbow injuries in baseball pitchers
Elbow injuries continue to be on the rise in baseball players, especially pitchers, yet little is known about the actual variables that influence these injuries.
Wikipedia readers get shortchanged by copyrighted material
When Google Books digitized 40 years worth of copyrighted and out-of-copyright issues of Baseball Digest magazine, Wikipedia editors realized they had scored.
Jet lag impairs performance of Major League Baseball players
A Northwestern University study of how jet lag affects Major League Baseball players traveling across just a few time zones found that when players travel in a way that misaligns their internal 24-hour clock with the natural environment and its cycle of sunlight, they suffer negative consequences.
Heavy hitters: Obesity rate soars among professional baseball players
Major League Baseball players have become overwhelmingly overweight and obese during the last quarter century, say health researchers.
Hamstring injuries in baseball may be preventable
Creating a program to prevent hamstring injuries in minor league and major league baseball players might be a possibility say researchers presenting their work today at the American Orthopaedic Society of Sports Medicine's Annual Meeting in Colorado Springs, Colo.
No long-term 'star effect' for baseball teams on Twitter
University of Missouri researchers have analyzed the Twitter usage of Major League Baseball (MLB) teams, athletes and fans and discovered that the 'star effect' had no long-term impacts on MLB teams' Twitter following and fan engagement.
NJIT professor predicts winners of Major League 2016 Baseball season: The Mets come out on top
After being one of the few who picked the Mets to make it to the postseason in 2015, NJIT Mathematical Sciences Professor and Associate Dean Bruce Bukiet has published his projections of how the standings should look at the end of Major League Baseball's 2016 season.
Young baseball players could benefit from preseason arm injury prevention programs
Preseason prevention programs are beneficial to young baseball pitchers, according to research presented today at the American Orthopaedic Society for Sports Medicine's Specialty Day.

Related Baseball Reading:

Best Science Podcasts 2019

We have hand picked the best science podcasts for 2019. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Changing The World
What does it take to change the world for the better? This hour, TED speakers explore ideas on activism—what motivates it, why it matters, and how each of us can make a difference. Guests include civil rights activist Ruby Sales, labor leader and civil rights activist Dolores Huerta, author Jeremy Heimans, "craftivist" Sarah Corbett, and designer and futurist Angela Oguntala.
Now Playing: Science for the People

#521 The Curious Life of Krill
Krill may be one of the most abundant forms of life on our planet... but it turns out we don't know that much about them. For a create that underpins a massive ocean ecosystem and lives in our oceans in massive numbers, they're surprisingly difficult to study. We sit down and shine some light on these underappreciated crustaceans with Stephen Nicol, Adjunct Professor at the University of Tasmania, Scientific Advisor to the Association of Responsible Krill Harvesting Companies, and author of the book "The Curious Life of Krill: A Conservation Story from the Bottom of the World".