Nav: Home

Current system unlikely to pick up surgeons with above average patient death rates

October 31, 2016

Publishing the patient death rates of individual surgeons in England is unlikely to pick up those whose mortality rates are above average, because the caseload varies so much, concludes the first analysis of its kind published in the online journal BMJ Open.

Performance within the 'expected' range is too crude a measure to detect doctors whose practice might be a cause for concern, and is therefore creating a false sense of security, say the researchers.

When the patient death rates for individual surgeons were first published in June 2013, the move was hailed as a major breakthrough in transparency that would drive up standards of care in England.

But the chances of detecting a surgeon whose death rates are worse than the national average is a question of statistical power, say the researchers: in other words, the greater the caseload, the greater the ability to detect worrying trends.

To assess how reliable the available data for individual surgeons are, the researchers reviewed the outcomes for three common high risk procedures--bowel surgery, gullet surgery, and planned aortic aneurysm repair--and three common low risk procedures--hip replacement, bariatric surgery, and thyroid removal.

And they analysed every surgeon's caseload for each of the procedures, all of which were carried out between 2010 and 2014 across England.

They focused in particular on how well these data would be able to detect a surgeon whose patient death rate in hospital or within 30 or 90 days of the patient's discharge was between two and five times higher than the national average.

Unsurprisingly, the higher risk procedures were associated with a higher death rate of between 2.2-4.5% while the lower risk ones were associated with a death rate of 0.07-0.4%.

But caseload was an issue. For example, the average number of bowel surgery operations carried out by individual surgeons was 55 over three years, but ranged from just 3 to 237.

With an average national 90 day death rate of 3%, the national average of 55 cases provides 20% statistical power to detect a mortality rate three times the national average. That means that around 20 out of 100 individual surgeons with an actual death rate of 9% would fall outside the expected range.

But the caseload would have to be more than 200 to provide 90% statistical power of detecting a surgeon whose 90 day mortality rate is three times the national average.

Similar findings emerged for gullet surgery, where the average number of procedures was 23 over a two year period, but ranged from 10 to 81.

Based on national 30 day death rates of 2.4%, the average number of cases would provide less than 20% statistical power to detect a surgeon with a patient death rate four times the national average.

And a caseload of 300 procedures would be needed to provide 80% statistical power to detect a 90 day mortality rate twice as high as the national average over two years.

For low risk procedures, the national average caseload ranged from 48 to 75 per surgeon, meaning that fewer than 20 out of 100 surgeons with an actual mortality rate five times the national average would be picked up.

For hip replacements, for example, an annual caseload of more than 500 cases would be needed to provide 80% statistical power to pick up just one individual with a mortality rate five times the national average.

At these kinds of rates it is unlikely that a surgeon would ever perform enough procedures in his/her entire career for a mortality rate five times the national average to be detected, say the researchers.

"On the basis of these rates and published case volumes, surgeons with mortality rates in excess of that expected are highly unlikely to be detected," they write. "Performance within an expected mortality rate range cannot therefore be considered reliable evidence of acceptable performance."

More meaningful outcome measures are required, they say. These could include patient satisfaction, the ease with which routine daily tasks can be performed (functional health status), and other health related quality of life indicators.

And an individual's performance could be addressed by regular internal appraisal and feedback from multiple sources, they suggest.

Interpreting performance data for individual surgeons has major implications for patient care, the individual practitioner, and their employer, they emphasise.

But they conclude: "This analysis demonstrates that, for these common procedures, mortality rates are not a robust method for detecting divergent practice. It is not surprising that the performance of all but one surgeon across all six procedures was found to be acceptable."
-end-
Research: Individual surgeon mortality rates: can outliers be detected? A national utility analysis

About the journal:

BMJ Open is BMJ's first online general medical journal dedicated solely to publishing open access research. All its articles, supplementary files, and peer reviewers' reports are fully and openly available online, along with an increasing number of linked raw data sets in the Dryad repository.
http://bmjopen.bmj.com

BMJ

Related Mortality Rates Articles:

Amenable death in Europe: Health care expenditure decreases mortality rates
The June issue of Health Affairs includes an analysis of amenable mortality rates -- rates of deaths that are potentially preventable with available health care treatment options -- in 17 European countries, which found that higher health care expenditure was associated with lower amenable mortality and with smaller absolute inequalities in amenable mortality.
New drug reduces transplant and mortality rates significantly in patients with hepatitis C
Patients with hepatitis C who suffer from advanced stages of liver disease have renewed hope, thanks to findings by researchers who have discovered that a new drug significantly reduces their risk of death and need for transplantation.
Mortality rates lower at major teaching hospitals
In an analysis that included more than 21 million Medicare discharges, admission to a major teaching hospital was associated with a lower overall 30-day risk of death compared with admission to a nonteaching hospital, according to a study published by JAMA.
New analysis finds Medicare program underestimates heart attack mortality rates
New analysis of Medicare's Hospital Compare portal shows the statistical methodology used to rate and compare hospitals underestimates mortality rates of acute myocardial infarction (AMI) at small hospitals.
Cervical cancer mortality rates may be underestimated
A new analysis reveals that for most women, the risk of dying from cervical cancer is higher than previously thought.
Breast cancer mortality rates decline in many countries
Breast cancer mortality rates continue to decline in many nations, but a review of mortality trends in 47 countries around the world indicates some significant disparities, particularly in South Korea and some Latin American nations, according to results presented at the 2016 San Antonio Breast Cancer Symposium, held Dec.
Kidney disease hospitalization and mortality rates continue to decline in the US
According to an annual data report from the United States Renal Data System (USRDS), hospitalization and mortality rates for patients with chronic kidney disease continue to decline in the US.
US maternal mortality rates higher than reported, BU study finds
Despite the United Nations Millennium Development Goal of a 75 percent reduction in maternal deaths by 2015, the estimated maternal mortality rate for 48 US states and the District of Columbia actually increased by 26.6 percent from 2000 to 2014, according to a new study co-authored by Boston University School of Public Health researchers.
Eating more whole grains linked with lower mortality rates
Eating more whole grains may reduce the risk of premature death, according to a new meta-analysis by researchers from Harvard T.H.
Analysis of more than 1.5 million people finds meat consumption raises mortality rates
A review of large-scale studies involving more than 1.5 million people found all-cause mortality is higher for those who eat meat, particularly red or processed meat, on a daily basis.

Related Mortality Rates Reading:

Best Science Podcasts 2019

We have hand picked the best science podcasts for 2019. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Jumpstarting Creativity
Our greatest breakthroughs and triumphs have one thing in common: creativity. But how do you ignite it? And how do you rekindle it? This hour, TED speakers explore ideas on jumpstarting creativity. Guests include economist Tim Harford, producer Helen Marriage, artificial intelligence researcher Steve Engels, and behavioral scientist Marily Oppezzo.
Now Playing: Science for the People

#524 The Human Network
What does a network of humans look like and how does it work? How does information spread? How do decisions and opinions spread? What gets distorted as it moves through the network and why? This week we dig into the ins and outs of human networks with Matthew Jackson, Professor of Economics at Stanford University and author of the book "The Human Network: How Your Social Position Determines Your Power, Beliefs, and Behaviours".