Events of 2020 have brought significant disruption to the world, as well as to medical student education and assessment [1, 2]. One major area of disruption surrounds the US Medical Licensing Examinations (USMLEs). In February 2020, the Invitational Conference on USMLE Scoring (InCUS) prompted several changes to Step 1, including the change to pass/fail scoring expected in January 2022 [3]. The intention was to reduce the current overemphasis on USMLE performance, while also retaining the use of the exam for its primary purpose of medical licensure eligibility.

Shortly after the announcement about the upcoming Step 1 score change, disruption from COVID-19 on medical school curricula, student learning, Step 1 studying, and Step 1 exam administration at Prometric sites led faculty, staff, and students at most medical schools to rethink many standard assumptions including those related to Step 1 timing [4]. In the face of these disruptions, the issue of when students should take Step 1 has come to the forefront for undergraduate medical educators and students.

Medical schools are dealing with students’ and administrations’ stressors such as global uncertainty as well as those specific to the medical school experience, such as frequent schedule revisions and compression or completely missing out on some future learning opportunities. In response to COVID-19, some schools are considering moving Step 1 after clerkships, in part to avoid exposing their students to the vulnerability of USMLE testing site shortages. Others have allowed students to use the times that students are not able to be in the clinical setting due to a “COVID pause” to study for and take Step 1, or are considering a model of student flexibility as curricula are rearranged in anticipation of potential further curricular disruption from the ongoing pandemic. There is significant uncertainty around the logistics of Step 1 examinations due to the uncertainty of current and future COVID disruptions. The purpose of this Commentary is to explore advantages and challenges for moving the timing of the USMLE Step 1 examination during the COVID-19 pandemic. This analysis can inform educators about the potential for flexibility within the medical school curriculum that may benefit learners with other individual or collective disruptions.

While there is no precedent for these enormous curricular and assessment impacts, we can look to the experiences of schools who have made changes to USMLE Step 1 timing in order to identify considerations for Step 1 timing that can optimize student learning and outcomes. While most US medical schools position Step 1 before the core clerkships, to date, more than 20 schools have transitioned or positioned the USMLE Step 1 exam after the core clerkships [5, 6]. All of these schools originally instituted the changes in a reasoned and deliberate manner, yet some of the insights may provide guidance in the current context of rapid changes. This Commentary is based on the research conducted by a collaboration of schools that have situated USMLE Step 1 after core clerkships. A related article by our collaboration group discusses Step 1 timing in the face of the exam moving to Pass / Fail [7].

Advantages of Post-clerkship Step 1

Student Motivation to Integrate Basic and Clinical Science Learning and Engage in Long-Term Learning Processes

We have previously outlined both theoretical and practical advantages to situating Step 1 after clerkships [5]. First, placing Step 1 after the clerkship year has the potential to drive integration of basic science learning while in the clinical learning environment by tapping into student motivation [8]. Calls for integration of basic and clinical sciences as part of curricular reform and better utilization of learning theory to promote conceptual understanding and long-term retention of knowledge suggest that positioning Step 1 after clerkship may be beneficial [9,10,11]. In addition, Step 1 content has increasingly included items framed as patient presentations that require application of knowledge to clinical problems rather than recall of isolated facts [12]. Medical schools implementing reform have often shortened their pre-clerkship curricula. They introduced pedagogies that emphasized case- or patient-based learning and provided students earlier opportunities for clinical immersion. In the traditional Step 1 timing model, students may prepare intensively for Step 1 but then may not deeply integrate the basic science foundations in the clinical context. Thus, the transfer of scientific knowledge from the preclerkship sciences to the clinical setting is problematic [13, 14]. Positioning Step 1 after core clerkships can incentivize students to continue to focus on foundational sciences through their clerkships.

A late Step 1 can also help schools to create an assessment system that emphasizes assessment for learning in the preclinical curriculum, to emphasize student learning, feedback and development, and to deemphasize short-term exam preparation and performance [15]. We believe the placement of Step 1 after the core clerkships encourages students to utilize and deepen their basic science knowledge while in the clinical setting, thereby fostering basic science connections beyond the exam. Further, the revisiting of science topics in the clinical year leverages spaced repetition to improve long-term learning [16, 17]. This model of placing Step 1 after clerkship may help to focus students, and the curriculum, toward prioritizing long-term learning and retention of information, instead of a binge and purge mentality.

Improved Preparation of Students for Understanding Clinical Context of Step 1 Questions

The second reason for moving the examination is to improve preparedness of students for Step 1. Enhancing the clinical relevance of foundational science in the context of clinical learning is thought to increase retention and facilitate transfer of concepts into practice [18, 19]. Because Step 1 is increasingly clinically focused, it is thought that more clinical exposure may help students read and analyze the clinical vignettes more easily and with greater comprehension. Students may also have developed illness scripts and pattern recognition through patient care that may help with answering questions [20].

A Later Step 1 Appears to Have Positive Student Outcomes

The third reason schools may consider moving Step 1 is that emerging empiric evidence supports possible benefits for student scores with moving Step 1 to after the core clerkship phase. Among four schools that moved Step 1 to after the core clerkships, students’ Step 1 scores increased modestly with a decrease in failure rates [21]. Students at risk seemed to generally benefit from a post-clerkship Step 1 exam as well [22]. In addition, while there were minimal changes on Subject examination scores (shelf exam), there was no change in Step 2 CK scores or failure rates in schools with a post-clerkship Step 1 [23, 24].

Post-clerkship Step 1 Exams Can Create More Flexible Elective Schedules if Needed

Fourth, post-clerkship Step 1 exams enable flexible schedules that allow for schools and students to push back the Step 1 date into elective months. We have found it is easier to re-arrange elective months than clerkship blocks if a student needs to delay Step 1 due to academic or personal challenges, or unanticipated disruptions [6]. As COVID-19 has appeared as a prevalent unanticipated disruption, students in schools with late Step 1 exams were able to engage in service activities or electives while waiting to sit for their exams.

In summary, the evidence of the effects of moving Step 1 are both theoretical (increased integration, retention, and transfer of knowledge to clinical practice) as well as empirical with slightly increased Step 1 scores and decreased failures in a select group of schools.

Challenges of Moving Step 1 in Response to COVID-19 and Mitigation Strategies

For schools considering moving Step 1 during the current COVID-19 disruption or in anticipation of future disruptions, some of these learning and logistical advantages may apply. However, we also discovered a series of challenges and strategies that may be helpful for schools to adapt to those challenges [6].

Curricular and Assessment Revision Planning and Implementation

Rearranging a curriculum and assessment system under the best of circumstances is a major undertaking that typically requires forethought, planning, stakeholder involvement, and a lot of communication. For our schools, moving the timing of Step 1 was related to curricular changes that began with creating shortened preclinical periods (12–18 months) and expanding basic science instruction in the clinical setting [5]. Schools looking to move Step 1 in response to COVID-19 disruptions may not have the curricular foundation in place to support the move. Of note, however, a few schools (University of Michigan and Harvard) each had one class that experienced a post-clerkship Step 1 before major curricular changes were undertaken. These classes did not experience any adverse effects and noted similar increases in Step 1 scores and reductions in failure rates. During the pandemic, schools that are able may wish to offer students flexibility about the timing of their Step 1 exam in recognition of the variation in students’ learning trajectories and preferences exacerbated by the pandemic circumstances. As noted below, this flexibility requires access to robust advising so that students are not overwhelmed with their options.

Student Preparation for Clerkship and Learning Progress

Concerns may be raised that students would not be sufficiently prepared for clinical clerkships without consolidating their knowledge by studying for Step 1, and that students with insufficient medical knowledge might not be identified and provided support early. Communication with clerkship faculty and residents is important so that they are aware that students have not yet fully prepared for and taken Step 1.

For schools that decide to move Step 1 to a later point, these challenges may be mitigated by having all students complete a remote National Board of Medical Examiners’ (NBME) Comprehensive Basic Science Self-Assessment (CBSSA) prior to clerkships. This assessment provides practice opportunities with Step 1-style questions as well as feedback in the form of a Step 1 score and content domain performance descriptors (below average, average, and above average). Because students can review the CBSSA for feedback and review incorrect answers, it can be a useful learning tool. Schools and learning specialists can also access performance reports to help identify students in need of remediation or studying support. Clerkship exams can also be used to identify and support struggling students, and to help cohorts of students to learn and utilize effective long-term studying strategies. As schools consider moving Step 1 in a short time frame in response to COVID-19, they may need to put in place additional support strategies such as review sessions, boot camps for clerkships, and intentional science integration in the clinical phase, which are outlined in the advice section.

Availability of USMLE Scores for Residency Applications

An important challenge to consider for schools with late Step 1 exams is the current requirement for completion of Step 1 and Step 2CK prior to residency application uploads, rank list certification, graduation and/or limited licensure by the Federation of State Medical Boards. The Prometric closures in March through June of 2020 caused many students not to be able to take Step 1 and Step 2 CK in a timely fashion. Uncertainties surrounding exam scheduling created an additional challenge of supporting students during extended Step 1 study periods of sometimes five months or more due to delays in ability to take the examination. The possibility of subsequent waves of COVID-19 that may cause additional delays in in-person USMLE testing loom. Under ordinary circumstances, we have found that with clear student affairs advising and procedures related to testing timing, students have been able to have scores ready for residency applications, even in the cases of students needing to repeat USMLE exams.

Advice to Support Students with Step 1 Changes Amidst COVID-19 Disruptions

We recognize that medical school administrators are working now to make decisions and plans for curricular and assessment interruptions for cohorts of students in the Classes of 2021 and 2022 and some in the Class of 2023 for which a Step 1 score is still used in residency decision making, and for subsequent classes anticipated to take a pass/fail Step 1 exam. We provide the following advice and guidance related to moving or adjusting Step 1 during the COVID-19 environment, and for enhancing the support of student learning during this disruptive time. It is important to engage stakeholders, from students to course directors and administration, in this process that is uncertain and requires flexibility.

Proactively Address Student Concerns

With the educational disruptions due to COVID, it is important to work with the students and student leadership to attempt to proactively address anticipated issues. Faculty need to have clear communication and stress the importance of flexibility since there are many rapid changes occurring as the environmental context changes.

Enhancing Science in the Clerkship

Ultimately, we want our students to be competent for practice, so integrating basic and clinical sciences throughout the curriculum is an important step [10]. Integrating basic sciences into the clinical year is important to do not just to help students prepare for a late Step 1 and Step 2 CK, but also to help them understand the sciences foundational to the practice of medicine. There are various approaches to increasing science in the curriculum including integration at the clerkship and system level [258].

Using Studying Periods and Deadlines

We advise schools to be careful about giving students too much flexibility to decide when to take Step 1, as the paradox of choice [26] might cause increased stress and diminished performance. It is advisable to set and adhere to deadlines for taking the examinations, to the extent this is possible considering testing availability. We have found that some students with high score expectations and/or high anxiety will be tempted to study for prolonged periods of time, at the expense of other learning and curricular engagement [27]. Some schools have been able to support students by implementing designated study periods and deadlines, and procedures and approvals for students requesting additional time to prepare for Step 1. Defining a beginning and end to a dedicated Step 1 studying period might be more challenging in the COVID-driven curriculum, but setting expectations and boundaries could help students to plan practical studying schedules and ideal test dates to work from [22]. Advising that students take a self-assessment like a CBSSA can help students to understand their baseline score and how far they are from their goal score, to guide their study planning or decisions about needing to delay testing.

Monitoring to Identify Students Who Might Struggle

Helping students to assess their learning progress and retention is an important support strategy to help students and schools be in tune to learning difficulties. Using testing data to help students and advisors to assess student learning progress can be helpful. Our schools found a slight decrease in the initial shelf examinations scores when Step 1 was moved to after clerkships, though the average effect was small (~1–2 points), and the decrease gradually disappeared with subsequent examinations [24]. Nonetheless, it is helpful to utilize shelf exam scores to monitor student progress and potential need for extra help during Step 1 studying time. Similarly, schools can gather information from the NBME Comprehensive Basic Science Exams, preclinical exams, and other assessments to help predict scores, and over time, develop predictive models to identify students who may need extra time or support during the Step 1 studying process [28].

Enhancing Academic Support

Schools can offer many supports for all students, from student affairs deans, learning specialists, advisory deans, wellness offices and others, and provide additional support for students with specific needs [29]. Learning specialists can help provide advice for students who are studying under less-than-ideal circumstances with more unknowns than students are accustomed to. They can advise students on how to utilize question banks for learning as well as for maintaining learning over time, how to assess progress toward and readiness for the exam using self-assessment exams such as the CBSSA, and how to plan in break time during studying and before taking Step 1. Some schools may provide supplemental commercial question banks for students to practice retrieving and applying knowledge over time, such as over the summer, or during COVID pause breaks, especially if student Step 1 exams will be delayed. Making academic advising counseling and resources more widely available to students can also support students.

Enhancing Wellness Support

While students with academic or test-taking difficulties can encounter Step 1 preparation challenges, difficulties can also stem from health or mental health stressors, social and/or situational stressors, and psychosocial or family challenges, which can all be compounded by the COVID-19 environment [30]. Working with wellness offices and resources to provide proactive support to all students and individualized support to students in need is critical now.

Expanding Remote USMLE Learning and Studying Support

Preparing for Step 1 during a normal year can be a challenging and isolating experience for many students. Especially as students are social distancing during USMLE studying, setting up frequent check-ins with students via periodic virtual office hours can be helpful. Schools might consider setting up remote studying rooms that students can log into to chat with classmates, or suggest that students use study partners where they can study in the same remote space together in order to see each other or check in on each other. Senior students can also be especially helpful resources to answer student questions and to offer guidance and encouragement.

Conclusion

In summary, COVID-19 has created new and unique challenges to medical education as well as all aspects of life. Stakeholders including students, medical schools, residency programs, and various organizations including the NBME are being pushed to explore the foundations of the process of medical training including issues of timing for examinations, criteria for graduation, residency selection, and licensing. The Coalition for Physician Accountability, consisting of 13 medical education organizations, is helping to guide the field but that path forward is uncertain [31]. This Commentary from our prior experience can provide guidance for programs considering moving Step 1 to after clerkships and supporting students preparing for USMLE exams during the pandemic. Overall, it is critical to be flexible and thoughtful regarding USMLE exam timing and support given the current situation and preparation for future COVID waves and ongoing disruptions.