Pass or Fail Reporting for Step 1: Is this Enough for Plastic Surgery Applicants?

by Praneeth V. Kambhampati1 and Malke Asaad, MD2

Medical Student, Baylor College of Medicine

Postdoctoral Research Fellow, Department of Plastic Surgery, MD Anderson Cancer Center


The United States Medical Licensing Examination (USMLE) Step 1 was developed by the National Board of Medical Examiners (NBME) as a preclinical licensing examination.1,2 This exam has been used as an objective measure to evaluate a medical student’s basic science knowledge and as a screening tool to filter the large pool of residency applicants.2

The National Resident Matching Program (NRMP) data show that Step 1 was the most common factor to be cited by program directors in selecting applicants to interview with an average rating of 4.1 (on a 5-point Likert score with 5 being very important).3 By looking at the data of senior allopathic students applying to integrated plastic surgery residency (PSR), almost a linear relationship exists between Step 1 score and the probability of matching, with higher Step 1 scores among students who matched in integrated PSR. Similar trends can be observed in other relatively competitive specialties such as dermatology, interventional radiology, neurological surgery, orthopedic surgery, and otolaryngology.4 The current difference between the passing score on the Step 1 exam (194) and the mean score of students who matched into PSR (249) is about 3 standard deviations (standard deviation on step 1 =19).4,5 Studies have shown conflicting results regarding the value of Step 1 in predicting future academic performance.2

On February 12, 2020, the Federation of State Medical Boards (FSMB) and the NBME announced that starting no earlier than January 1, 2022, the Step 1 exam will be reported in a binary pass/fail format.6 If Step 1 is such a powerful tool in screening applicants, one might wonder why to abandon the numeric score for a pass/fail outcome. To answer these questions, we highlight the major arguments in favor of and against the new policy.

Arguments for Pass/Fail Score Reporting:

  1. Mental health concerns: Many believe that the Step 1 climate represents physical and psychological challenges for medical students and is a major source of stress and anxiety. Reporting the exam results as pass/fail scoring could help to address the mental health issues suffered by medical students.7

  2. Diversity in the medical workforce: Some authors noted gender and racial differences in Step 1 scores.8-10 Therefore, the three-digit Step 1 score could potentially pose a structural barrier to diversity in medical fields due to the significant emphasis placed on Step 1 scores in selecting applicants.1,7

  3. Change in the medical education learning environment: Step 1 has been a huge determinant of what is considered “valuable” to students in their two years of preclinical medical education. This has raised the concern that many students are disengaged from institutional curricula and focus instead on a “parallel curriculum” that helps them to excel on the examination. This practice undervalues certain aspects of the medical education experience that could have significant ramifications on a student’s professional development.7

Arguments in Favor of Three-numeric Score:

  1. “They cannot all walk on water”: Given the heterogeneous grading system between medical schools and the subjective aspect of clinical evaluations and letters of recommendation (LORs), a pass/fail score would make applicants almost indistinguishable.2,11 Many believe that the new policy will place a significant burden on program directors to evaluate the “tsunami” of applications due to the loss of a critical standardized national objective metric.2

  2. Impact on students from small medical schools: With the lack of standardized comparison between medical students, PDs may rely on the assumption that students from top-tier medical schools have better medical education. Abandoning a numeric Step 1 score would make it harder for students from less well-known medical schools to stand apart from the crowd, especially in competitive specialties.2,12 This change will possibly impact osteopathic students and international graduates in a similar manner.

    3. Impact on education: Another concern for a pass/fail score is that students would dedicate less time for their studying due to lack of incentive, in favor of other aspects of residency application that will be of high importance to PDs.2 This could be detrimental to students’ medical knowledge given that Step 1 lays the foundation for the subsequent years of clinical education.

Going Forward:

Step 1 is well on its way to serve as a binary benchmark for professional licensure. With this shift comes the potential following changes:

  • More emphasis on holistic candidate review. However, many argue that Step 1 was not the sole criterion for applicants’ selection but used in combination with all other aspects of the application.
  • More emphasis on away rotations as a way to evaluate students’ knowledge, work ethic, and professionalism.
  • Potential change in medical school curricula with a new focus on helping students to pass rather than ace the exam.
  • Increased emphasis on Step 2 CK scores which could serve as a potential alternative to the standardized metric previously offered by the numeric Step 1 score. Students may be mandated to take Step 2 CK prior to applying to the Match. This would ultimately go back to square one in the vicious cycle of applicant comparison based on a numeric score.
  • Students might be required to take dedicated research time to establish connections with residency programs and demonstrate commitment to respective specialties.13,14 Given the disparity in research productivity between institutions across the country, this will negatively impact students from smaller medical schools and favor students working with well-known mentors from large institutions.
  • For surgical specialties, students might be required to undergo technical skills assessment during their clinical rotations or interviews.

Finally, despite the different arguments for and against the new policy, there is no doubt that the new change will have a significant impact on applicant selection and medical education in the United States.


References:

1.         Chen DR, Priest KC, Batten JN, Fragoso LE, Reinfeld BI, Laitman BM. Student Perspectives on the “Step 1 Climate” in Preclinical Medical Education. Academic Medicine. 2019;94(3):302-304.

2.         Rozenshtein A, Mullins ME, Marx MV. The USMLE Step 1 Pass/Fail Reporting Proposal: The APDR Position. Academic radiology. 2019;26(10):1400-1402.

3.         Results of the 2018 NRMP Program Director Survey June 2018. Available at: https://www.nrmp.org/wp-content/uploads/2018/07/NRMP-2018-Program-Director-Survey-for-WWW.pdf Accesssed on: Feb 17, 2020.

4.         Charting Outcomes in the Match: U.S. Allopathic Seniors 2nd edition. . National Resident Matching Program July 2018 (revised October 2019)  Available at: https://www.nrmp.org/wp-content/uploads/2018/06/Charting-Outcomes-in-the-Match-2018-Seniors.pdf Accessed at: November 29, 2019.

5.         USMLE Score Interpretation Guidelines. Updated on Jan 31, 2020 Available at: https://www.usmle.org/pdfs/transcripts/USMLE_Step_Examination_Score_Interpretation_Guidelines.pdf Accessed on Feb 18, 2020.

6.         Change to pass/fail score reporting for Step 1. InCUS. USMLE. Feb 2020. . Available at: https://www.usmle.org/incus/ Accessed on: Feb 17, 2020.

7.         Carmody JB, Sarkany D, Heitkamp DE. The USMLE Step 1 Pass/Fail Reporting Proposal: Another View. Academic radiology. 2019;26(10):1403-1406.

8.         Cuddy MM, Swanson DB, Clauser BE. A multilevel analysis of examinee gender and USMLE step 1 performance. Academic medicine : journal of the Association of American Medical Colleges. 2008;83(10 Suppl):S58-62.

9.         Edmond MB, Deschenes JL, Eckler M, Wenzel RP. Racial bias in using USMLE step 1 scores to grant internal medicine residency interviews. Academic medicine : journal of the Association of American Medical Colleges. 2001;76(12):1253-1256.

10.       Rubright JD, Jodoin M, Barone MA. Examining Demographics, Prior Academic Performance, and United States Medical Licensing Examination Scores. Academic medicine : journal of the Association of American Medical Colleges. 2019;94(3):364-370.

11.       Rajesh A, Rivera M, Asaad M, et al. What Are We REALLY Looking for in a Letter of Recommendation? Journal of surgical education. 2019;76(6):e118-e124.

12.       Cottrell S, Ferrari N. More About USMLE Step 1 Scoring. Academic Medicine. 2019;94(11):1626.

13.       Mehta K, Sinno S, Thanik V, Weichman K, Janis JE, Patel A. Matching into Integrated Plastic Surgery: The Value of Research Fellowships. Plastic and reconstructive surgery. 2019;143(2):640-645.

14.       Rajesh A, Asaad M, Nashed M, Vyas K. Matching into Integrated Plastic Surgery: The Value of Research Fellowships. Plastic and reconstructive surgery. 2019;144(5):946e-947e.

Leave a Reply