Events of 2020 have brought significant disruption to the world, as well as to medical student education and assessment [1, 2]. One major area of disruption surrounds the US Medical Licensing Examinations (USMLEs). In February 2020, the Invitational Conference on USMLE Scoring (InCUS) prompted several changes to Step 1, including the change to pass/fail scoring expected in January 2022 [3]. The intention was to reduce the current overemphasis on USMLE performance, while also retaining the use of the exam for its primary purpose of medical licensure eligibility.
Shortly after the announcement about the upcoming Step 1 score change, disruption from COVID-19 on medical school curricula, student learning, Step 1 studying, and Step 1 exam administration at Prometric sites led faculty, staff, and students at most medical schools to rethink many standard assumptions including those related to Step 1 timing [4]. In the face of these disruptions, the issue of when students should take Step 1 has come to the forefront for undergraduate medical educators and students.
Medical schools are dealing with students’ and administrations’ stressors such as global uncertainty as well as those specific to the medical school experience, such as frequent schedule revisions and compression or completely missing out on some future learning opportunities. In response to COVID-19, some schools are considering moving Step 1 after clerkships, in part to avoid exposing their students to the vulnerability of USMLE testing site shortages. Others have allowed students to use the times that students are not able to be in the clinical setting due to a “COVID pause” to study for and take Step 1, or are considering a model of student flexibility as curricula are rearranged in anticipation of potential further curricular disruption from the ongoing pandemic. There is significant uncertainty around the logistics of Step 1 examinations due to the uncertainty of current and future COVID disruptions. The purpose of this Commentary is to explore advantages and challenges for moving the timing of the USMLE Step 1 examination during the COVID-19 pandemic. This analysis can inform educators about the potential for flexibility within the medical school curriculum that may benefit learners with other individual or collective disruptions.
While there is no precedent for these enormous curricular and assessment impacts, we can look to the experiences of schools who have made changes to USMLE Step 1 timing in order to identify considerations for Step 1 timing that can optimize student learning and outcomes. While most US medical schools position Step 1 before the core clerkships, to date, more than 20 schools have transitioned or positioned the USMLE Step 1 exam after the core clerkships [5, 6]. All of these schools originally instituted the changes in a reasoned and deliberate manner, yet some of the insights may provide guidance in the current context of rapid changes. This Commentary is based on the research conducted by a collaboration of schools that have situated USMLE Step 1 after core clerkships. A related article by our collaboration group discusses Step 1 timing in the face of the exam moving to Pass / Fail [7].
Advantages of Post-clerkship Step 1
Student Motivation to Integrate Basic and Clinical Science Learning and Engage in Long-Term Learning Processes
We have previously outlined both theoretical and practical advantages to situating Step 1 after clerkships [5]. First, placing Step 1 after the clerkship year has the potential to drive integration of basic science learning while in the clinical learning environment by tapping into student motivation [8]. Calls for integration of basic and clinical sciences as part of curricular reform and better utilization of learning theory to promote conceptual understanding and long-term retention of knowledge suggest that positioning Step 1 after clerkship may be beneficial [9,10,11]. In addition, Step 1 content has increasingly included items framed as patient presentations that require application of knowledge to clinical problems rather than recall of isolated facts [12]. Medical schools implementing reform have often shortened their pre-clerkship curricula. They introduced pedagogies that emphasized case- or patient-based learning and provided students earlier opportunities for clinical immersion. In the traditional Step 1 timing model, students may prepare intensively for Step 1 but then may not deeply integrate the basic science foundations in the clinical context. Thus, the transfer of scientific knowledge from the preclerkship sciences to the clinical setting is problematic [13, 14]. Positioning Step 1 after core clerkships can incentivize students to continue to focus on foundational sciences through their clerkships.
A late Step 1 can also help schools to create an assessment system that emphasizes assessment for learning in the preclinical curriculum, to emphasize student learning, feedback and development, and to deemphasize short-term exam preparation and performance [15]. We believe the placement of Step 1 after the core clerkships encourages students to utilize and deepen their basic science knowledge while in the clinical setting, thereby fostering basic science connections beyond the exam. Further, the revisiting of science topics in the clinical year leverages spaced repetition to improve long-term learning [16, 17]. This model of placing Step 1 after clerkship may help to focus students, and the curriculum, toward prioritizing long-term learning and retention of information, instead of a binge and purge mentality.
Improved Preparation of Students for Understanding Clinical Context of Step 1 Questions
The second reason for moving the examination is to improve preparedness of students for Step 1. Enhancing the clinical relevance of foundational science in the context of clinical learning is thought to increase retention and facilitate transfer of concepts into practice [18, 19]. Because Step 1 is increasingly clinically focused, it is thought that more clinical exposure may help students read and analyze the clinical vignettes more easily and with greater comprehension. Students may also have developed illness scripts and pattern recognition through patient care that may help with answering questions [20].
A Later Step 1 Appears to Have Positive Student Outcomes
The third reason schools may consider moving Step 1 is that emerging empiric evidence supports possible benefits for student scores with moving Step 1 to after the core clerkship phase. Among four schools that moved Step 1 to after the core clerkships, students’ Step 1 scores increased modestly with a decrease in failure rates [21]. Students at risk seemed to generally benefit from a post-clerkship Step 1 exam as well [22]. In addition, while there were minimal changes on Subject examination scores (shelf exam), there was no change in Step 2 CK scores or failure rates in schools with a post-clerkship Step 1 [23, 24].
Post-clerkship Step 1 Exams Can Create More Flexible Elective Schedules if Needed
Fourth, post-clerkship Step 1 exams enable flexible schedules that allow for schools and students to push back the Step 1 date into elective months. We have found it is easier to re-arrange elective months than clerkship blocks if a student needs to delay Step 1 due to academic or personal challenges, or unanticipated disruptions [6]. As COVID-19 has appeared as a prevalent unanticipated disruption, students in schools with late Step 1 exams were able to engage in service activities or electives while waiting to sit for their exams.
In summary, the evidence of the effects of moving Step 1 are both theoretical (increased integration, retention, and transfer of knowledge to clinical practice) as well as empirical with slightly increased Step 1 scores and decreased failures in a select group of schools.
Challenges of Moving Step 1 in Response to COVID-19 and Mitigation Strategies
For schools considering moving Step 1 during the current COVID-19 disruption or in anticipation of future disruptions, some of these learning and logistical advantages may apply. However, we also discovered a series of challenges and strategies that may be helpful for schools to adapt to those challenges [6].
Curricular and Assessment Revision Planning and Implementation
Rearranging a curriculum and assessment system under the best of circumstances is a major undertaking that typically requires forethought, planning, stakeholder involvement, and a lot of communication. For our schools, moving the timing of Step 1 was related to curricular changes that began with creating shortened preclinical periods (12–18 months) and expanding basic science instruction in the clinical setting [5]. Schools looking to move Step 1 in response to COVID-19 disruptions may not have the curricular foundation in place to support the move. Of note, however, a few schools (University of Michigan and Harvard) each had one class that experienced a post-clerkship Step 1 before major curricular changes were undertaken. These classes did not experience any adverse effects and noted similar increases in Step 1 scores and reductions in failure rates. During the pandemic, schools that are able may wish to offer students flexibility about the timing of their Step 1 exam in recognition of the variation in students’ learning trajectories and preferences exacerbated by the pandemic circumstances. As noted below, this flexibility requires access to robust advising so that students are not overwhelmed with their options.
Student Preparation for Clerkship and Learning Progress
Concerns may be raised that students would not be sufficiently prepared for clinical clerkships without consolidating their knowledge by studying for Step 1, and that students with insufficient medical knowledge might not be identified and provided support early. Communication with clerkship faculty and residents is important so that they are aware that students have not yet fully prepared for and taken Step 1.
For schools that decide to move Step 1 to a later point, these challenges may be mitigated by having all students complete a remote National Board of Medical Examiners’ (NBME) Comprehensive Basic Science Self-Assessment (CBSSA) prior to clerkships. This assessment provides practice opportunities with Step 1-style questions as well as feedback in the form of a Step 1 score and content domain performance descriptors (below average, average, and above average). Because students can review the CBSSA for feedback and review incorrect answers, it can be a useful learning tool. Schools and learning specialists can also access performance reports to help identify students in need of remediation or studying support. Clerkship exams can also be used to identify and support struggling students, and to help cohorts of students to learn and utilize effective long-term studying strategies. As schools consider moving Step 1 in a short time frame in response to COVID-19, they may need to put in place additional support strategies such as review sessions, boot camps for clerkships, and intentional science integration in the clinical phase, which are outlined in the advice section.
Availability of USMLE Scores for Residency Applications
An important challenge to consider for schools with late Step 1 exams is the current requirement for completion of Step 1 and Step 2CK prior to residency application uploads, rank list certification, graduation and/or limited licensure by the Federation of State Medical Boards. The Prometric closures in March through June of 2020 caused many students not to be able to take Step 1 and Step 2 CK in a timely fashion. Uncertainties surrounding exam scheduling created an additional challenge of supporting students during extended Step 1 study periods of sometimes five months or more due to delays in ability to take the examination. The possibility of subsequent waves of COVID-19 that may cause additional delays in in-person USMLE testing loom. Under ordinary circumstances, we have found that with clear student affairs advising and procedures related to testing timing, students have been able to have scores ready for residency applications, even in the cases of students needing to repeat USMLE exams.
Advice to Support Students with Step 1 Changes Amidst COVID-19 Disruptions
We recognize that medical school administrators are working now to make decisions and plans for curricular and assessment interruptions for cohorts of students in the Classes of 2021 and 2022 and some in the Class of 2023 for which a Step 1 score is still used in residency decision making, and for subsequent classes anticipated to take a pass/fail Step 1 exam. We provide the following advice and guidance related to moving or adjusting Step 1 during the COVID-19 environment, and for enhancing the support of student learning during this disruptive time. It is important to engage stakeholders, from students to course directors and administration, in this process that is uncertain and requires flexibility.
Proactively Address Student Concerns
With the educational disruptions due to COVID, it is important to work with the students and student leadership to attempt to proactively address anticipated issues. Faculty need to have clear communication and stress the importance of flexibility since there are many rapid changes occurring as the environmental context changes.
Enhancing Science in the Clerkship
Ultimately, we want our students to be competent for practice, so integrating basic and clinical sciences throughout the curriculum is an important step [10]. Integrating basic sciences into the clinical year is important to do not just to help students prepare for a late Step 1 and Step 2 CK, but also to help them understand the sciences foundational to the practice of medicine. There are various approaches to increasing science in the curriculum including integration at the clerkship and system level [258].
Using Studying Periods and Deadlines
We advise schools to be careful about giving students too much flexibility to decide when to take Step 1, as the paradox of choice [26] might cause increased stress and diminished performance. It is advisable to set and adhere to deadlines for taking the examinations, to the extent this is possible considering testing availability. We have found that some students with high score expectations and/or high anxiety will be tempted to study for prolonged periods of time, at the expense of other learning and curricular engagement [27]. Some schools have been able to support students by implementing designated study periods and deadlines, and procedures and approvals for students requesting additional time to prepare for Step 1. Defining a beginning and end to a dedicated Step 1 studying period might be more challenging in the COVID-driven curriculum, but setting expectations and boundaries could help students to plan practical studying schedules and ideal test dates to work from [22]. Advising that students take a self-assessment like a CBSSA can help students to understand their baseline score and how far they are from their goal score, to guide their study planning or decisions about needing to delay testing.
Monitoring to Identify Students Who Might Struggle
Helping students to assess their learning progress and retention is an important support strategy to help students and schools be in tune to learning difficulties. Using testing data to help students and advisors to assess student learning progress can be helpful. Our schools found a slight decrease in the initial shelf examinations scores when Step 1 was moved to after clerkships, though the average effect was small (~1–2 points), and the decrease gradually disappeared with subsequent examinations [24]. Nonetheless, it is helpful to utilize shelf exam scores to monitor student progress and potential need for extra help during Step 1 studying time. Similarly, schools can gather information from the NBME Comprehensive Basic Science Exams, preclinical exams, and other assessments to help predict scores, and over time, develop predictive models to identify students who may need extra time or support during the Step 1 studying process [28].
Enhancing Academic Support
Schools can offer many supports for all students, from student affairs deans, learning specialists, advisory deans, wellness offices and others, and provide additional support for students with specific needs [29]. Learning specialists can help provide advice for students who are studying under less-than-ideal circumstances with more unknowns than students are accustomed to. They can advise students on how to utilize question banks for learning as well as for maintaining learning over time, how to assess progress toward and readiness for the exam using self-assessment exams such as the CBSSA, and how to plan in break time during studying and before taking Step 1. Some schools may provide supplemental commercial question banks for students to practice retrieving and applying knowledge over time, such as over the summer, or during COVID pause breaks, especially if student Step 1 exams will be delayed. Making academic advising counseling and resources more widely available to students can also support students.
Enhancing Wellness Support
While students with academic or test-taking difficulties can encounter Step 1 preparation challenges, difficulties can also stem from health or mental health stressors, social and/or situational stressors, and psychosocial or family challenges, which can all be compounded by the COVID-19 environment [30]. Working with wellness offices and resources to provide proactive support to all students and individualized support to students in need is critical now.
Expanding Remote USMLE Learning and Studying Support
Preparing for Step 1 during a normal year can be a challenging and isolating experience for many students. Especially as students are social distancing during USMLE studying, setting up frequent check-ins with students via periodic virtual office hours can be helpful. Schools might consider setting up remote studying rooms that students can log into to chat with classmates, or suggest that students use study partners where they can study in the same remote space together in order to see each other or check in on each other. Senior students can also be especially helpful resources to answer student questions and to offer guidance and encouragement.
Conclusion
In summary, COVID-19 has created new and unique challenges to medical education as well as all aspects of life. Stakeholders including students, medical schools, residency programs, and various organizations including the NBME are being pushed to explore the foundations of the process of medical training including issues of timing for examinations, criteria for graduation, residency selection, and licensing. The Coalition for Physician Accountability, consisting of 13 medical education organizations, is helping to guide the field but that path forward is uncertain [31]. This Commentary from our prior experience can provide guidance for programs considering moving Step 1 to after clerkships and supporting students preparing for USMLE exams during the pandemic. Overall, it is critical to be flexible and thoughtful regarding USMLE exam timing and support given the current situation and preparation for future COVID waves and ongoing disruptions.
References
Lucey CR, Johnston SC. The transformational effects of COVID-19 on medical education. JAMA. 2020;324(11):1033–4. https://doi.org/10.1001/jama.2020.14136.
Hauer KE, Lockspeiser TM, Chen HC. The COVID-19 pandemic as an imperative to advance medical student assessment: three areas for change. Acad Med. 2021;96(2):182–5. https://doi.org/10.1097/acm.0000000000003764.
United States Medical Licensing Examination. InCUS – Invitational Conference on USMLE Scoring. Change to pass/fail score reporting for Step 1. https://www.usmle.org/InCUS/#decision. Accessed 16 May 2020.
Whelan AJ. The change to pass/fail scoring for Step 1 in the context of COVID-19: implications for the transition to residency process. Acad Med. 2020;95(9):1305–7. https://doi.org/10.1097/ACM.0000000000003449.
Daniel M, Fleming A, Grochowski CO, Harnik V, Klimstra S, Morrison G, et al. Why not wait? Eight institutions share their experiences moving United States Medical Licensing Examination Step 1 after core clinical clerkships. Acad Med. 2017;92(11):1515–24. https://doi.org/10.1097/ACM.0000000000001714.
Pock A, Daniel M, Santen SA, Swan-Sein A, Fleming A, Harnik V. Challenges associated with moving the United States Medical Licensing Examination (USMLE) Step 1 to after the core clerkships and how to approach them. Acad Med. 2019;94(6):775–80. https://doi.org/10.1097/ACM.0000000000002651.
Daniel M, Hauer K, Chandran L, Pock A, Morrison G, Santen S. The optimal timing of Step 1 in medical education following the transition to pass / fail: a unique perspective from post-clerkship Step 1 schools Med Sci Educ. 2021;In Press.
Daniel M, Morrison G, Hauer KE, Pock A, Seibert C, Amiel J, et al. Strategies from 11 U.S. medical schools for integrating basic science into core clerkships. Acad Med. 2020;Publish Ahead of Print.https://doi.org/10.1097/ACM.0000000000003908.
Cooke M, Irby DM, Sullivan W, Ludmerer KM. American medical education 100 years after the Flexner report. N Engl J Med. 2006;355(13):1339–44. https://doi.org/10.1056/NEJMra055445.
Cooke M, Irby DM, O'Brien BC. Educating physicians: a call for reform of medical school residency. John Wiley & Sons; 2010.
Kauffman D, Mann K. Teaching and learning in medical education: How theory can inform practice. In: Swanwick T, editor. Understanding Medical Education: Evidence, Theory and Practice. 2nd ed. West Sussex, UK: Wiley-Blackwell; 2013. p. 7–29.
USMLE Content Outline. United States Medical Lisensing Exam, https://www.usmle.org/pdfs/usmlecontentoutline.pdf. 2020. Accessed 2/1/2021.
Peters S, Clarebout G, van Nuland M, Aertgeerts B, Roex A. A qualitative exploration of multiple perspectives on transfer of learning between classroom and clinical workplace. Teach Learn Med. 2018;30(1):22–32. https://doi.org/10.1080/10401334.2017.1339605.
Castillo JM, Park YS, Harris I, Cheung JJH, Sood L, Clark MD, et al. A critical narrative review of transfer of basic science knowledge in health professions education. Med Educ. 2018;52(6):592–604. https://doi.org/10.1111/medu.13519.
Swan Sein A, Rashid H, Meka J, Amiel J, Pluta W. Twelve tips for embedding assessment for and as learning practices in a programmatic assessment system. Med Teach. 2020;Online ahead of print:1–7. https://doi.org/10.1080/0142159X.2020.1789081.
Dunlosky J, Rawson KA, Marsh EJ, Nathan MJ, Willingham DT. Improving students’ learning with effective learning techniques: promising directions from cognitive and educational psychology. Psychol Sci Public Interest. 2013;14(1):4–58. https://doi.org/10.1177/1529100612453266.
Kerfoot BP, DeWolf WC, Masser BA, Church PA, Federman DD. Spaced education improves the retention of clinical knowledge by medical students: a randomised controlled trial. Med Educ. 2007;41(1):23–31. https://doi.org/10.1111/j.1365-2929.2006.02644.x.
Brauer DG, Ferguson KJ. The integrated curriculum in medical education: AMEE Guide No. 96. Med Teach. 2015;37(4):312–22. https://doi.org/10.3109/0142159X.2014.970998.
Harden RM. The integration ladder: a tool for curriculum planning and evaluation. Med Educ. 2000;34(7):551–7. https://doi.org/10.1046/j.1365-2923.2000.00697.x.
Hennrikus EF, Skolka MP, Hennrikus N. Applying metacognition through patient encounters and illness scripts to create a conceptual framework for basic science integration, storage, and retrieval. J Med Educ Curric Dev. 2018;5:2382120518777770. https://doi.org/10.1177/2382120518777770.
Jurich D, Daniel M, Paniagua M, Fleming A, Harnik V, Pock A, et al. Moving the United States Medical Licensing Examination Step 1 after core clerkships: an outcomes analysis. Acad Med. 2019;94(3):371–7. https://doi.org/10.1097/ACM.0000000000002458.
Swan Sein A, Daniel M, Fleming A, Morrison G, Christner JG, Esposito K, et al. Identifying and supporting students to prevent USMLE Step 1 failures when testing follows clerkships: insights from 9 schools. Acad Med. 2020;95(9):1338–45. https://doi.org/10.1097/ACM.0000000000003272.
Jurich D, Santen SA, Paniagua M, Fleming A, Harnik V, Pock A, et al. Effects of moving the United States Medical Licensing Examination Step 1 after core clerkships on step 2 clinical knowledge performance. Acad Med. 2020;95(1):111–21. https://doi.org/10.1097/ACM.0000000000002921.
Jurich D, Daniel M, Hauer KE, Seibert C, Chandran L, Pock AR, et al. Does delaying the United States Medical Licensing Examination Step 1 to after clerkships affect student performance on clerkship subject examinations? Teach Learn Med. 2020:1–19. https://doi.org/10.1080/10401334.2020.1860063.
Daniel MMG, Hauer K, Pock A, Seibert C, Amiel J, Poag M, Ismail N, Dalrymple, J Esposito K, Pettepher C, Santen S. Connecting basic science to clinical care: strategies for integrating basic science during core clerkships. Academic Medicine (Submitted).
Schwartz B. Incentives, choice, education and well-being. Oxford Review of Education. 2009;35(3):391–403. https://doi.org/10.1080/03054980902934993.
Harnik V, Santen SA, Fleming A, Sein AS, et al. In Reply to Green et al. Acad Med. 2020;95(5):665–6. https://doi.org/10.1097/ACM.0000000000003199.
Guiot HM, Franqui-Rivera H. Predicting performance on the United States Medical Licensing Examination Step 1 and Step 2 Clinical Knowledge using results from previous examinations. Adv Med Educ Pract. 2018;9:943–9. https://doi.org/10.2147/AMEP.S180786.
Swan Sein A, Dathatri S, Bates TA. Twelve tips on guiding preparation for both high-stakes exams and long-term learning. Med Teach. 2020;Online ahead of print:1–6. https://doi.org/10.1080/0142159X.2020.1828570.
Rajkumar RP. COVID-19 and mental health: a review of the existing literature. Asian J Psychiatr. 2020;52:102066. https://doi.org/10.1016/j.ajp.2020.102066.
Chaudhry HJ, Kirch DG, Nasca TJ, Katsufrakis PJ, McMahon GT, Shannon SC, et al. Navigating tumultuous change in the medical profession: the coalition for physician accountability. Acad Med. 2019;94(8):1103–7. https://doi.org/10.1097/acm.0000000000002801.
Funding
The University of Michigan School of Medicine and Virginia Commonwealth University School of Medicine receive funding from the American Medical Association Accelerating Change in Medical Education grant.
Author information
Authors and Affiliations
Contributions
All authors participated in the conception and design of the work and writing the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Competing Interests
The authors declare that they have no conflict of interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Swan Sein, A., Daniel, M., Hauer, K.E. et al. Educational and Practical Implications of Step 1 Timing in the Context of COVID-19. Med.Sci.Educ. 31, 911–916 (2021). https://doi.org/10.1007/s40670-021-01255-5
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40670-021-01255-5