Ensuring Continuity of Pediatric Psychological Assessment Services During the COVID-19 Pandemic

Introduction

The field of psychology has evolved significantly over the past few decades along with the advent of technology and the need to improve accessibility of applied psychology to the wider community. Among this, the use of telemedicine within the field has gained traction, especially for the delivery of critical psychology interventions to patients in various forms. Parent education, psychotherapy, and counseling have all been delivered online with positive response from receivers as well as providers (Nelson & Bui, 2010; Palermo et al., 2016). Similarly, the use of telemedicine in psychological assessments has been emerging; specifically for certain aspects such as neuropsychological testing and cognitive testing, as well as, to a more limited extent, diagnostic testing for developmental disabilities like Autism Spectrum Disorder (Juárez et al., 2018; Schopp, Johnstone, & Merrell, 2000). Another mode of increasing accessibility in the context of psychological assessment has been the examination of the necessary rigor of “gold-standard” assessment measures especially in the diagnosing of developmental disabilities, where reducing wait-times for assessments allows for access to early intervention. The context of these has thus far largely been in the setting of rural health services, to overcome barriers in access to psychology services due to geographic settings (Campbell, Theodoros, Russell, Gillespie, & Hartley, 2019).

The ongoing novel Coronavirus disease pandemic (COVID-19) has disrupted lives worldwide across countries due to its widespread nature in this highly globalized age. Rapidly implemented pandemic-related measures embraced by most countries have emphasized the need for social distancing, restricted movement of people, and enhanced infection control measures across workplaces including within medical settings. The use of telemedicine has increased significantly over the past few months as various medical services turn to these alternative models of care to continue provision of services. The COVID-19 pandemic may thus have an unintended benefit of accelerating the implementation of telemedicine in the field of psychology as well. The examination of “essential” aspects of psychological assessment, as opposed to “gold-standard,” should then also be accelerated in view of the increasing wait-times for psychological assessment services, which in turn, heightens the barrier to access treatment.

At the same time, the increased need of teleconsultation in psychological assessment brings up the question of whether mode of assessment (in-person vs. online) may affect assessment results. For example, Patalay, Deighton, Fonagy, and Wolpert (2015) and Patalay, Hayes, Deighton, and Wolpert (2016) reported higher mental-health difficulties scores when children completed assessments by computer, compared to paper-and-pencil versions. Gremigni, Mobilio, Casu, and Catapano (2014) found that higher levels of dental anxiety were reported in an online questionnaire, versus a paper-and-pencil version completed in the dentist’s waiting room. In a recent meta-analysis of proctored versus unproctored ability assessment, Steger, Schroeders, and Gnambs (2020) found that higher scores are obtained in unproctored assessments. However, comparing paper-and-pencil versus computer assessment of reasoning ability, Schroeders and Wilhelm (2010) found small to negligible effects. Thus, current results on the effects of assessment methods seem mixed, and the topic requires further investigation, especially in times of increasing online assessments.

Implementing alternative modes of assessment entails considerable changes to traditional workflows of organizations. It also demands acceptance by both receivers and providers of the changes made and the need to demonstrate the efficacy of these changes. In this guest editorial, we describe one such pediatric psychology practice that has had to adapt to COVID-19 related national and institutional measures, making changes to its operational workflows; chief of which was to initiate the use of teleconsultation for psychological assessments. The considerations involved in these changes and measures taken to ensure the integrity of psychological assessments are also detailed. In addition, we discuss specifically the implementation of teleconsultations in a practice which previously practiced in-person only assessments, including the design of security protocols and preliminary end-user feedback from caregivers. Lastly, we reflect on how this process of re-shaping our organizational workflows for assessments led to a re-thinking of the role and essential aspects of assessments. The spurring of the evaluation of essential aspects of psychological assessments remains an ongoing process that the entire field of psychological assessment should continue in light of these times.

COVID-19 Pandemic and Implications for Singapore

The COVID-19 pandemic has spread to almost every nation in the world, with more than 6 million cases to date, and more than 350,000 deaths worldwide (World Health Organisation, 2020). This resulted in the implementation of unprecedented measures worldwide to curtail its spread. Singapore reported its first case of COVID-19 on January 23, 2020, prior to the rapid spread of the disease in Europe, USA, and the rest of the world. Over the next 2 weeks, more cases were identified, and Singapore had one of the highest numbers of reported cases outside China at that time. In response, on February 8, 2020 the nation’s Disease Outbreak Response System Condition (DORSCON) level was officially raised to orange (Government of Singapore, 2020a). This signified that the disease was severe and easily spread between people but there was no widespread community transmission. As part of the escalation of the DORSCON status, the country implemented Phase 1 of the pandemic measures. Implications of this included tightening border control and travel restrictions at the national level, instituting regular temperature screening at all workplaces, implementing strict infection control practices and visitor restrictions at healthcare institutions.

Subsequently, due to the worsening community spread, the Singapore government implemented the second major phase (Phase 2) of pandemic measures on April 5, known as the “Circuit Breaker” (Government of Singapore, 2020b). Similar to the measures initiated in many other countries, the Circuit Breaker mandated the closure of schools and all non-essential services and workplaces. In addition, residents were discouraged from leaving their homes, unless they needed to carry out essential activities. This also encompassed healthcare services providing non-essential services as defined by the Ministry of Health Singapore (Ministry of Health, Singapore, 2020).

Such services had to consider how to continue to provide support to their patients within the new restrictions whereby physical patient visits were not allowed. One such service was the Child Development Unit (CDU) at the National University Hospital (NUH), Singapore, which is a developmental and behavioral pediatric (DBP) practice.

Clinical Service

The CDU, which is part of NUH (a tertiary level academic institution), is one of the two nationally designated sites in Singapore offering multidisciplinary care for children with developmental, learning, and behavioral difficulties. The CDU is located at 3 separate sites – 1 within the main hospital and 2 sites in the community (https://www.nuh.com.sg/our-services/Specialties/Paediatrics/Pages/Developmental-and-Behavioural-Paediatrics.aspx).

The unit assesses and manages children from birth to 7 years old with a range of learning, behavioral, and developmental difficulties. The CDU supports around 13,000 outpatient visits annually, through services provided by a multidisciplinary team of about 50 staff. The unit consists of developmental pediatricians, psychologists, speech therapists, occupational therapists, learning support educators, physiotherapists, social workers, patient service assistants, therapy assistants, and an operations administrative team. Services include developmental and psychological assessments, therapy, and learning support services.

This guest editorial will focus on the CDU psychological assessment services. The CDU psychology team primarily carries out psychological assessments, which include diagnostic assessments for Autism Spectrum Disorder (ASD), Attention Deficit-Hyperactivity Disorder (ADHD) and Specific Learning Disorder. The team also provides school readiness evaluations through cognitive and adaptive behavior assessments for children, as well as developmental assessments for at-risk premature infants. Apart from these, the team also provides individual intervention services and group programs addressing internalizing and externalizing behavior problems.

Objective

The COVID-19 related nationwide pandemic measures had direct implications on the psychological services at the CDU during both Phase 1 and Phase 2 of the measures. The psychology team at the unit had to constantly adapt to the pandemic-related measures and design workflows to adhere to them, while as far as possible, continuing to deliver the service in a safe and reliable manner for our staff and patients. The aim of this guest editorial is to detail the considerations and workflows that were adapted to allow continued psychological assessments for our patients during this COVID-19 period. We also discuss the challenges faced and the long-term implications of these measures.

Workflows

Phase 1 of the Pandemic Measures

The response by our unit focused on 2 main aspects: staff safety and patient care.

Staff Safety

To ensure continuity of psychological assessment services, while protecting staff, the following measures were taken:

1. (1)
   Team segregation, whereby the psychology team was split into two teams. Each team was fully based at one community site. In-person interactions between the two teams were discouraged, and no cross-site movement was allowed. This also allowed for continuity of service in the event that any individual in one team was afflicted with COVID-19 resulting in the team having to be quarantined.
2. (2)
   Facilitating work from home arrangements for staff. This was done by rescheduling patients assigned for psychologists such that they had to physically be at the unit only for 3 days per week at the most. The remaining 2 or more work-week days were set aside to carry out work without patient-contact at home, such as meetings, research, and completion of reports following assessments. Institution-issued laptops with appropriate security configurations were used to work from home. Standard operating procedures were written to ensure security of assessment protocols (Electronic Supplementary Material, ESM 1). This was done so as to minimize the commuting required and promote social distancing measures.
3. (3)
   Personal protective equipment for staff. All staff were required to wear surgical masks as mandated by institution level guidelines.

Patient Care

The aim was to continue the provision of care while minimizing the number of patients in the clinic and thereby reducing the need for in-person encounters in the interest of patient safety. This was achieved by:

1. (1)
   A framework to decide on the continued provision of specific assessments. The various psychological assessments commonly conducted at the CDU were first evaluated based on whether they were of high, medium, or low priority. This helped in decision making of whether to proceed with, or place on hold the various assessments. The framework used is shown in Table 1.
2. (2)
   Considerations to ensure integrity of assessments while adhering to pandemic-related institutional guidelines. This included considerations on minimizing direct contact with patients and the need for staff to wear a surgical mask at all times. Psychological assessments that were slated for continuation during the pandemic were segregated into those likely to be unaffected by the partial obscuring of the psychologist’s face (due to the use of the mask), and those that may be compromised by this. It was assessed that cognitive assessments, learning assessments, and parent interview components of assessments were unlikely to be affected by the partial obscuring of the psychologist’s face. Hence, these could proceed as usual.
3. (3)
   The administration of the Autism Diagnostic Observation Schedule – Second edition (ADOS-2) was the only one likely to be affected by the partial obscuring of the examiner’s face. The ADOS-2 is a semi-structured, standardized assessment instrument that includes a number of play-based activities designed to obtain information in the areas of communication, reciprocal social interactions, and restricted and repetitive behaviors associated with a diagnosis of ASD (Lord et al., 2012).
4. (4)
   Tasks from each module were evaluated to determine whether they could continue to be administered according to standardized procedures. The only task evaluated to be directly affected was the “Reciprocal Social Smile” task on Module 1 of the ADOS-2. A decision was made to omit the level 1 prompt in this task, where the examiner was required to smile at the child. Instead, the level 2 prompt was directly administered, whereby the examiner asked the parent to get their child’s attention and smile at their child. Item B2 “Reciprocal Social Smile” was coded as missing to indicate that the task was not administered. The B2 “Reciprocal Social Smile” score does not contribute to the overall cut-off score, so a missing score did not impact the overall score. However, non-performance on the level 2 prompt of this task was not taken into account for summary items, which contributed to the overall cut-off score, i.e., B12 “Quality of Social Overtures,” B14 “Quality of Social Response,” B15 “Level of Engagement,” and B16 “Overall Quality of Rapport.” Taking into account that the examiner’s use of the mask could have affected the overall interaction between the examiner and the child, the child’s calculated ADOS-2 Comparison score, which defines the “level of autism-related symptoms,” was also not reported. To further ascertain whether the child’s overall interaction was affected by the examiner’s wearing of the mask, parents were asked at the end of the ADOS-2 session whether their child’s behavior during the assessment was typical of their child at home. If parents reported that their child’s behavior was different, home videos that demonstrated their child’s interaction with familiar and unfamiliar adults were requested from parents as supplementary information. In making an overall diagnosis for ASD, greater weight was placed on other sources of information obtained, which included parental report of the child’s behavior on the Autism Diagnostic Interview – Revised (ADI-R) (Le Couteur, Lord, & Rutter, 2003), the child’s adaptive behavior functioning as reported on the Vineland Adaptive Behavior Scales – Third edition (Vineland-3) (Sparrow, Cicchetti, & Saulnier, 2016), as well as teacher feedback. Information from all these sources was integrated into a report according to the diagnostic criteria from the Diagnostic and Statistical Manual of Mental Disorders – Fifth edition (DSM-5) to determine whether the child met the criteria for ASD (American Psychiatric Association, 2013).
5. (5)
   The use of telemedicine. Prior to this pandemic, the unit had only practiced face-to-face consultations. However, in the interest of minimizing direct patient contact for staff as well as to minimize commuting to the unit for patients, the use of teleconsultations was pioneered. This applied to the parent interview components of the assessments that did not require the administration of stimulus materials (as shown in Table 1). The team opined that the provision of video-consultation would be preferable to phone-consultation during parent interviews, so that demonstrations and nonverbal communication could be used to aid understanding for both the caregivers and the examiners. Examples of the assessments that were continued via video conferencing included the ADI-R and the Vineland-3. Teleconsultation guidelines were drawn up, which included protocols for proper administration, and to ensure security of sessions and parents’ privacy (ESM 2).
6. (6)
   The use of supplementary information. For all the assessments, as deviations from standardized protocol were made; supplementary information that previously would have been optional was made a requirement, to provide additional sources of information to corroborate the findings. These were obtained through questionnaires sent via electronic mail. Examples include a teacher questionnaire to obtain information about behavior and/or learning depending on the age of the child and the Vanderbilt or Conners-3 questionnaires for children with concerns of ADHD.

Table 1 Framework for continuity of psychological assessment services during the pandemic

				Pandemic Phase 1		Pandemic Phase 2
Assessment type	Assessment components	Priority*	Nature	Status	Changes implemented	Status	Changes implemented
Note. *Priority level is determined by time-sensitivity and implications of the assessment results. For example, cognitive assessments were high priority because they are needed to meet application datelines for special education schools.
Autism Spectrum Disorder	Adaptive behavior assessment	Medium/High	Parent Interview	Continued	Offer parents the option of teleconsultation	Continued	Conduct parent interviews only by teleconsultation
	Autism Diagnostic Observation Schedule – 2nd edition	Medium/High	In-person	Continued	Changes made to the scoring algorithm with respect to the Reciprocal social smile task. Supplementary information also mandated to be obtained from other individuals familiar with child	On Hold	Waitlist for in-person assessment, which is to be completed within 6 months from the first date of assessment
	Autism Diagnostic Interview – Revised	Medium/ High	Parent Interview	Continued	Offer parents option of conducting by teleconsultation. Place greater weight on this component of assessment in diagnosis formulation	Continued	Conduct parent interviews only by teleconsultation
School Placement	Cognitive assessment	High	In-person	Continued	Nil	On Hold	Waitlist for in-person assessment after the circuit breaker
	Adaptive behavior assessment	High	Parent Interview	Continued	Nil	Continued	Conduct parent interviews only by teleconsultation
Specific Learning Disorder	Cognitive and Learning assessment	Medium	In-person	Continued	Nil	On Hold	Waitlist for in-person assessment after the circuit breaker
Attention Deficit-Hyperactivity Disorder	Test of Everyday Attention for Children – 2nd edition	Low	In-person	Continued	Nil	On Hold	Waitlist for in-person assessment after the circuit breaker
	Conners-3	Low	Questionnaire	Continued	Nil	Continued	Place greater weight on this component of assessment in diagnosis formulation

Table 1 Framework for continuity of psychological assessment services during the pandemic

View as image HTML

Discussion

The COVID-19 pandemic situation has forced everyone around the world to (1) redefine and re-evaluate what is considered essential, and (2) re-think traditional ways of operations and service delivery. Telemedicine has proved to be a vital resource for many aspects of medical care and DBP-specific care, including physician consults and therapist-led interventions. However, the use of telemedicine is more ambiguous for psychological assessment services as assessments rely on standardized administration procedures requiring in-person contact. Whether all psychological assessments are “essential services” and hence ought to be continued during a pandemic is also debatable. The American Psychological Association has suggested that “essential” refers to assessments that are “time-sensitive, high-need and/or high-stakes” (American Psychological Association, 2020). In addition, for these essential assessments, the importance of keeping to “gold-standard” assessment practices (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2014) needs to be revaluated. Even prior to the COVID-19 situation, there was already a movement away from “gold-standard” practices to increase access to assessments to allow for early diagnosis of some disorders (Zwaigenbaum & Penner, 2018). The COVID-19 pandemic has accelerated this process of evaluating what is “essential” for diagnosis rather than rigidly adhering to what is “gold-standard.”

As a psychology service, we have rapidly developed and implemented new workflows to allow continued psychology services for our patients during the period. With these, we were able to maintain our patient numbers with no drop in numbers in March 2020 as compared to that in March 2019. In April 2020 (during the “Circuit Breaker” in the nation), our numbers dropped to 30% compared to that in April 2019. This was inevitable, given the fact that in-person physical visits were not allowed. Nevertheless, the provision of selected services continued through teleconsultations for parent interviews. Thus overall, we were able to continue provision of services during both phases of the pandemic measures. Through this period of tremendous change and workflow re-organization, we have had three major learning points through this process so far:

Firstly, as a unit, through the scrutiny of the tools used for diagnosing a child with ASD, we were able to reflect on the diagnostic process for ASD. An important takeaway was the fact that a diagnosis of ASD is made by clinical evaluation (parent-reported history and observation and bedside developmental assessments) and not necessarily only by the routine use of “gold-standard” assessment measures. The DSM-5 that guides diagnosis of ASD does not mandate the use of particular tools, but rather describes the symptom criteria for a person to be diagnosed with ASD (American Psychiatric Association, 2013). This is a strong proponent for diagnostic reports that integrate findings from multiple sources and assessment tools according to the diagnostic criteria, rather than those that report findings test-by-test. On a related note, this illustrated the need to have the flexibility to deviate from standardized protocols when necessary, and to gather information from multiple sources to supplement diagnosis.

Secondly, the usage of teleconsultations as a viable option for parent interview assessments, instead of physical visits is another important learning point. Telemedicine has been explored and in use in the field of psychology including pediatric psychology for over the past decade now. The focus of this has largely been on delivering interventions through technology including psychotherapy targeted at parents and children (Nelson & Bui, 2010; Palermo et al., 2016; Ritterband et al., 2003). Telemedicine in these settings has been shown to be a feasible mode of service delivery with acceptance by both providers and patients (Backhaus et al., 2012; Jenkins-Guarnieri, Pruitt, Luxton, & Johnson, 2015). The use of telemedicine for psychological assessments has also been described, although to a lesser extent and mainly in adults (Schopp, Johnstone, & Merrell, 2000). Challenges in interaction with children via digital platforms to undertake assessments may be a unique barrier in pediatrics. However, there is emerging literature that is demonstrating a role for assessments via telemedicine for children with developmental disabilities, especially ASD (Juárez et al., 2018; Talbott et al., 2019). During Phase 1 (March 2020), the psychology team conducted 14 of its 76 assessment sessions by teleconsultation, and during the first month of Phase 2 (April 2020), all 18 scheduled assessment sessions were conducted via teleconsultation. Prior to COVID-19, no visits were conducted by teleconsultation. The team had to adapt to this significant change in their daily practice. To gather feedback on the teleconsultation process during the implementation phase, all 14 parents who completed an assessment session by teleconsultation in Phase 1 were asked to complete a questionnaire on their experience. Eight of the 14 parents returned the completed questionnaire. All eight parents agreed that they would like to conduct sessions by teleconsultation again. Qualitative feedback included that teleconsultations were convenient and minimized travel for them. All the psychologists had positive feedback on the use of teleconsultation and reported it to be a method they would consider in the future as well. While further empirical data will be required, the positive response of parents and the relative ease of use suggest that even after the COVID-19 pandemic situation, this is a viable option as compared to physical visits for parent interview assessments. The use of telemedicine for diagnostic assessments can thus be explored further in the current pandemic context. Within our clinical context, it also increases the accessibility of some psychological assessment services to people who do not reside in Singapore.

A third learning point is the need for measures to be put in place to prioritize staff mental health during a time of anxiety, as well as massive and almost constant change. Research has shown that there is an impact on the psychological wellbeing of individuals during a pandemic (Cullen, Gulati, & Kelly, 2020; Wu et al., 2009). In addition, team morale and relationships undergo significant strain with repeated change in processes. Despite the inevitable stress on staff during pandemics, their psychological well-being is typically overlooked until signs emerge, for example signs of stress in an individual or signs of strain in team relationships. Ensuring staff well-being is of paramount importance and is an area that needs to be included as an important aspect of the change in workflows (Aiello et al., 2011; Dong & Bouey, 2020; Gavidia, 2020), so that a proactive rather than reactive approach can be taken. In terms of the approach, avenues of help should be made available to staff, and they should be encouraged to access these when they need to. However, it is important that team leaders have open communication with the team and initiate regular interactions with every member of their team. This is especially important during a time when the lack of face-to-face interaction can result in a breakdown in communication and affect work relationships (Walton, Murray, & Christian, 2020). Ensuring staff well-being will in turn facilitate the continued provision of care and services for our patients in a time of massive change, by preserving team relationships, and avoiding the need for repair work of team dynamics post-change.

Conclusion

The Psychology services at NUH’s CDU in Singapore have had to adapt considerably to multiple changes brought on by the COVID-19 pandemic. While testing our agility as a team, this has brought on new opportunities to be explored and sustained even beyond the current pandemic. Adapting to provide teleconsultations for certain psychological assessments, re-examining the traditional role of assessments in diagnosing developmental disabilities, and staff well-being have been key areas of learning for the team as a whole. However, given that we are still in the early stages of the pandemic, it is important to be prepared to refine these workflows further to ensure a responsive unit committed to continuing patient care.

Electronic Supplementary Materials