Purpose

To conduct a scoping review of the timing, scope, and purpose of literature related to the United States Medical Licensing Examination (USMLE) given the recent impetus to revise USMLE scoring.

Method

The authors searched PubMed, PsycInfo, and ERIC for relevant articles published from 1990 to 2019. Articles selected for review were labeled as research or commentaries and coded by USMLE Step level, sample characteristics (eg, year (s), single/multiple institutions), how scores were used (eg, predictor/outcome/descriptor), and purpose (eg, clarification/justification/description).

Results

Of the 741 articles meeting inclusion criteria, 636 were research and 105 were commentaries. Publication totals in the past 5 years exceeded those of the first 20 years. Step 1 was the sole focus of 38%, and included in 84%, of all publications. Approximately half of all research articles used scores as a predictor or outcome measure related to other curricular/assessment efforts, with a marked increase in the use of scores as predictors in the past 10 years. The overwhelming majority of studies were classified as descriptive in purpose.

Conclusions

Nearly 30 years after the inception of the USMLE, aspirations for its predictive utility are rising faster than evidence supporting the manner in which the scores are used. A closer look is warranted to systematically review and analyze the contexts and purposes for which USMLE scores can productively be used. Future research should explore cognitive and noncognitive factors that can be used in conjunction with constrained use of USMLE results to inform evaluation of medical students and schools and to support the residency selection process.