Changing Student Perception of an Online Integrated Structured Clinical Examination During the COVID-19 Pandemic [Letter]

Dear editor

We read with interest the article by Ganesananthan et al¹ which evaluated final year medical students’ perception towards novel online Integrated Structured Clinical Examinations (ISCE) in the COVID-19 pandemic to understand concerns and gain acceptability in the assessment. As final year medical students who have undertaken similar summative in-person and adapted online formative OSCEs, we understand the importance of student confidence in assessments as a factor in their validity and would like to discuss suggestions to build upon this commendable work.

We appreciate the methods used by the authors to assess students’ perception, however we feel more emphasis could have been placed on the latter section of the questionnaire allowing participants to express anxiety-related factors with more openness since they could be addressed by institutions conducting the exams. This study’s questionnaire primarily used pre-set questions with binary responses illustrating a closed-ended approach. Although this allowed quantitative analysis of outcomes, it may have attracted bias that an open-ended approach with free-text and thematic analysis of responses would have avoided.² By eliciting anxiety-related factors from students themselves, these factors could have been used as the basis for the post-ISCE binary scale questions rather than utilising the original pre-set questions therefore retaining quantitative significance. Unfortunately, the authors failed to mention how the pre-set factors were originally created so this approach would improve validity of the results obtained.

Furthermore, the pre-post study design could have been enhanced to include a control group consisting of the remaining students who had not sat a prior mock online ISCE. Formative clinical exams are already shown to be useful to improve students’ outcomes,³ therefore comparing the results of a follow-up questionnaire post-summative ISCE among both the participant and control groups could have proved useful to isolate any effect of ongoing improved perception in addition to a single pre-post cross-sectional result especially since students lacked familiarity with the format.

Moreover, Al Kadri et al⁴ showed that student perceptions towards formative and summative assessments widely vary with respect to the amount and seriousness of preparation. Having performed the study on a formative exam, the conclusions are not generalisable to a summative cohort as any confidence or acceptability in the assessment expressed has not been tested in an ideal higher stake setting. Students taking a summative exam more seriously will act as greater stakeholders in a perception questionnaire response thus eliminating confounding variables such as perceived lack of exam importance. This would be particularly interesting for the final two questions (of the questionnaire) interrogating the online format of the ISCE as a contributor in the students’ results — with possibly more blame or merit being placed on it specifically for students’ results.

With the view to incorporate summative online ISCEs as assessment tools, the preliminary findings from this paper will be worth exploration. We recommend not only addressing cohort demographic factors acknowledged by the authors but implementing questionnaire and analysis changes as well as overall and specific aforementioned study design improvements in order to further understand the online ISCE’s utility.