Bias may exist in rating of medical trainees

December 04, 2012

Peter Yeates, M.B.B.S., M.Clin.Ed., of the University of Manchester, United Kingdom, and colleagues conducted a study to examine whether observations of the performance of postgraduate year 1 physicians influence raters' scores of subsequent performances.

"The usefulness of performance assessments within medical education is limited by high interrater score variability, which neither rater training nor changes in scale format have successfully ameliorated. Several factors may explain raters' score variability, including a tendency of raters to make assessments by comparing against other recently viewed learners, rather than by using an absolute standard of competence. This has the potential to result in biased judgments," according to background information in the article.

The study consisted of an internet-based randomized experiment using videos of Mini Clinical Evaluation Exercise (Mini-CEX) assessments of postgraduate year 1 trainees interviewing new internal medicine patients. Participants were 41 attending physicians from England and Wales experienced with the Mini-CEX, with 20 watching and scoring 3 good trainee performances and 21 watching and scoring 3 poor performances. All then watched and scored the same 3 borderline video performances. The study was completed between July and November 2011.

The researchers found that attending physicians exposed to videos of good medical trainee performances rated subsequent borderline performances lower than those who had been exposed to poor performances, consistent with a contrast bias. The implication is that a rater of a trainee's performance may be unconsciously influenced by the previous trainee, rather than objectively assessing the individual in isolation.

"With the movement toward competency-based models of education, assessment has largely shifted to a system that relies on judgments of performance compared with a fixed standard at which competence is achieved (criterion referencing). Although this makes conceptual sense (with its inherent ability to reassure both the profession and the public that an acceptable standard has been reached), the findings in this study, which are consistent with contrast bias, suggest that raters may not be capable of reliably judging in this way."
-end-
(JAMA. 2012;308[21]:2226-2232. Available pre-embargo to the media at http://media.jamanetwork.com)

Media Advisory: To contact Peter Yeates, M.B.B.S., M.Clin.Ed., email peter.yeates@manchester.ac.uk.

The JAMA Network Journals

Related Performance Articles from Brightsurf:

Guiding the way to improved solar cell performance
Small molecules could hold the key to enhancing the efficiency of organic solar cells.

Performance test for neural interfaces
Freiburg researchers develop guidelines to standardize analysis of electrodes.

Cognitive performance - Better than our predecessors
We employ our cognitive skills daily to assimilate and process information.

Predicting sports performance with "big data"
Smartphones and wearable devices are not simple accessories for athletes.

All of the performance, none of the fuss: Nitrile hydrogenation done right
Researchers developed a nano-cobalt phosphide catalyst (nano-Co2P) for the hydrogenation of nitriles to primary amines.

Big ideas in performance management 2.0
Industrial-era performance management paradigms and practices are outdated and ineffective in the modern VUCA work environment.

Can exercise improve video game performance?
Time spent playing video games is often seen as time stolen from physical activities.

Anticipating performance can hinder memory
Anticipating your own performance at work or school may hinder your ability to remember what happened before your presentation, a study from the University of Waterloo has found.

Want to optimize sales performance?
CATONSVILLE, MD, September 16, 2019- According to new research published in the INFORMS journal Marketing Science, companies can improve sales performance when they adjust sales commissions for the sale of more popular items.

Assessing battery performance: Compared to what?
A team from the US Department of Energy's (DOE) Argonne National Laboratory, University of Warwick, OVO Energy, Hawaii National Energy Institute, and Jaguar Land Rover reviewed the literature on the various methods used around the world to characterize the performance of lithium-ion batteries to provide insight on best practices.

Read More: Performance News and Performance Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.