Article Text
PDF
Abstract
Objective To develop a psychometrically reliable instrument to assess psychological distress during the COVID-19 pandemic across Arab countries.
Design The new instrument was developed through the review of relevant literature. We adapted multiple items from the following tools: The Fear of COVID-19 Scale, Social Phobia Inventory, Health Anxiety Inventory, Swine Influenza Anxiety Scale and the Arabic Scale of Death Anxiety to design our new assessment tool which is called COVID-19 Psychological Distress Scale (CPDS). For psychometric analyses and validation, we conducted a cross-sectional study that solicited data through a web-based survey using the newly developed CPDS.
Setting and participants This validation study was conducted in four Arab countries, including Algeria, Kuwait, Saudi Arabia and Yemen. A total of 1337 participants from these countries have voluntarily responded to our survey questionnaire that included the newly developed scale.
Results The final version of the CPDS comprised 12 items. Participants from Algeria (n=447), Kuwait (n=437), Saudi Arabia (n=160) and Yemen (n=293) have completed the 12-item CPDS. Exploratory factor analysis (used on the Algerian sample) suggested a two-factor structure of the CPDS. The two-factor structure was then supported by the confirmatory factor analysis with an independent sample. Additionally, Rasch analyses showed that all the items fit well in their embedded construct; only one item showed somewhat substantial differential item functioning across gender and country.
Conclusion The 12-item CPDS was found to be measurement invariant across country and gender. The CPDS, with its promising psychometric properties, might help healthcare professionals to identify people with COVID-19-induced psychological distress.
- adult psychiatry
- anxiety disorders
- depression & mood disorders
- COVID-19
Data availability statement
Data are available on reasonable request. The data used in the current study are confidential and cannot be publicly shared. This was also stated in the participants’ consent. However, it is available from the corresponding author on a reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The final COVID-19 Psychological Distress Scale (CPDS) comprised 12 items to assess COVID-19-induced psychological distress.
The 12-item CPDS has been validated on a sample from four Arab countries.
Robust statistical tests were used to psychometrically validate the 12-item CPDS.
The online survey and convenience sampling may limit the generalisation of our results.
Further studies may assess the clinical utility of the 12-item CPDS in various settings and populations.
Introduction
The outbreak of novel COVID-19 which was declared by WHO as a global pandemic, has posed a significant threat to human life as well as to most life sectors.1–3 Both physical and psychological well-being were severely afflicted by this pandemic and its mitigation measures.4 As of 20 April 2021, the direct biological harm caused by the COVID-19 pandemic has resulted in more than 141 million patients, and the death toll surpassed three million deaths globally.5
Worldwide, the various impacts of the COVID-19 pandemic could not pass by without affecting the psychological status of people. The fear of infection itself, the lack of effective antiviral therapy and the lack of vaccines at an earlier stage of the pandemic can result in intense emotional and psychological responses.4 6–9 In addition, these psychological and behavioural changes may result indirectly from the confinement and control measures such as lockdown, quarantine, stay-at-home orders and physical distancing rules.8 10 The psychological impacts of such a global pandemic have afflicted different countries and populations around the world.11–13 The pandemic-induced distress can also affect people regardless of their socioeconomic status, gender or even their psychiatric history. Different societal components including the general population, students, healthcare professionals, academicians, people with a previous psychiatric history and even those who have never experienced any psychological issues, could have been impacted by a large-scale crisis such as the COVID-19 pandemic.8 11 13–22 Therefore, people in Arab countries were not excluded from being impacted by the mental health repercussions resulting from a pandemic. Besides, addressing the psychological effects of infectious diseases outbreaks can be challenging at early stages, when most efforts and services are intended to protect physical well-being and less attention is paid to the psychological side.23
The scientific community has been working eagerly to develop different psychological assessment tools specified for the COVID-19-related distress. Among the recently developed instruments were: The Coronavirus Anxiety Scale, the Fear of COVID-10 Scale, the Obsession with COVID-19 Scale, COVID-19 Stress Scale and the Questionnaire on Perception of Threat from COVID-19.24–28 To the best of our knowledge, none of these scales have been evaluated for measurement invariance or differential item functioning (DIF) across Arab countries. Measurement invariance is a crucial issue, primarily when tools are used for diagnoses and selection purposes.29 DIF detection procedures are important methodological procedures for a better understanding of the underlying constructs that are measured by items on an educational or psychological test.30
The main objectives of our present research study were (1) To develop a psychometrically reliable tool, namely, the COVID-19 Psychological Distress Scale (CPDS), that can be used in assessing the predisposition to develop COVID-19-induced psychological distress in Arab countries and (2) To test the measurement invariance of the newly developed tool across country and gender, as well as to assess the CPDS items fairness using DIF analyses.
Methods
Development of the CPDS
The CPDS was developed with items generated based on an extensive literature review of empirical studies and various published scales including, The Fear of COVID-19 Scale,26 The Social Phobia Inventory,31 The Health Anxiety Inventory,32 The Swine Influenza Anxiety Scale33 and The Arabic Scale of Death Anxiety.34 Thus, 30 relevant and possible items were pooled. The statements of these items were written in modern standard Arabic. The initial version of the CPDS consisted of 30 items with two sub-scales: physiological (i.e, shortness of breath; headache) and emotional (ie, fear; anxiety) to be answered on a five-point Likert-type intensity scale, as follows: (1) no, (2), a little (3), moderate, (4) much and 5: very Much.
Expert review
The 30 statements were presented to 10 staff members from the Psychology and Psychiatry departments at the University of Algeria, Cairo University, Qatar University Kuwait University, as well as one professor and a doctoral researcher, both from the University of Auckland in New Zealand. The referees were requested to review all items to determine the appropriate items for accurate measurement of COVID-19-induced distress. The referees classified the statements as acceptable or unacceptable. If 3 out of the 10 referees classified an item as unacceptable, that item was removed from the initial version of the CPDS. Also, the referees had to determine the adequacy of phrasing and clarity of meaning. As a result, specific revisions and corrections were carried out accordingly. Finally, 18 items were agreed on to be excluded, which resulted in the final version of our newly developed tool: the 12-Item CPDS. The 12-item CPDS is provided in both Arabic and English in online supplemental material.
Supplemental material
Validation study and participants
We conducted a web-based cross-sectional study between 26 March 2020 and 21 April 2020. Data were collected via an anonymous online questionnaire that was administered to 1337 voluntary participants from four Arab countries (Algeria n=447, Kuwait n=437, Saudi Arabia n=160 and Yemen n=293). By employing a convenience sampling approach, study participants were recruited from academic settings in the selected countries. Interested participants from different educational levels have voluntarily completed our survey questionnaire. Given the unfolding situation of the COVID-19 pandemic and the imposed control measures at the time of conducting our study, the choice was made to use Google Forms (A secure web-based survey platform provided by Google) to create and design the survey questionnaire, which was then disseminated to the participants via online platforms. Several online forums and social networking websites (eg, Facebook, Twitter and LinkedIn) were used to attract potential Internet users to participate in the present study.
Participants who were eligible to enrol in our study had to be at least 18 years old and have provided electronic written informed consent. Also, participants were assured about anonymity, confidentiality, voluntary participation and withdrawal.
Measures
Alongside the 12-item CPDS questions, our survey questionnaire solicited relevant sociodemographic data from the participants. This included age (in years), gender (female vs male), marital status (single, married, divorced and widowed), country (Algeria, Kuwait, Saudi Arabia and Yemen) and educational field/study programme.
Patient and public involvement
In this study, patients have not been involved the study design, setting, study survey and outcome measures. The electronic written informed consent was obtained from all participants acknowledging that they are willing to participate in the study. Besides, participants neither were engaged in interpreting, nor in reporting the results of the study.
Data analyses
Given that the 12-item CPDS has never been examined for its factorial structure, the Algerian sample was first used to explore the CPDS’s factorial structure. Specifically, the Algerian sample was used for exploratory factor analysis (EFA) and parallel analysis (PA). Then, all the samples from different countries (Algeria, Kuwait, Saudi Arabia and Yemen) were analysed for the following analyses: internal consistency, confirmatory factor analysis (CFA), multigroup CFA (for measurement invariance across gender and country) and Rasch analysis (for DIF analyses).
In the EFA and PA, the principal axis factoring method was used, followed by Oblimin oblique rotation. In the PA, eigenvalues were compared in two datasets: the observed dataset (ie, the Algerian sample) and a simulated dataset. The criterion used to retain a resulted factor is that its eigenvalue in the observed dataset should be larger than parallel factors derived from simulated data (having the same sample size and the number of variables in the observed dataset).35 36 When the PA suggests a factor structure with two or more factors, oblimin oblique rotation was applied. Afterward, the internal consistency of the CPDS was calculated using Cronbach’s alpha for the factors found in the EFA and the entire CPDS.
In the CFA and multigroup CFA, the diagonally weighted least squares (DWLS) method was used to estimate the data-model fit. DWLS provides more accurate parameter estimates, and the fit of the model is more robust to variable type and non-normality.37 Moreover, several fit indices were used to examine whether the data fit well with the CFA or multigroup CFA using the factor structural initially found using the EFA. The fit statistics included Comparative Fit Index (CFI), Root Mean Square Error of Approximation (RMSEA) and Standardised Root Mean Square Residual (SRMR). Values of CFI >0.95, together with those of RMSEA <0.06 and SRMR <0.08, indicate a satisfactory fit.38 39 The multigroup CFA further examined whether measurement invariance was supported across the country and/or across gender. Three nested models were constructed in the multigroup CFA: (1) a configural model; (2) a model with factor loadings constrained equal across subsamples (ie, either country subsamples or gender subsamples) and (3) a model with factor loadings and item intercepts constrained equal across subsamples.40 Then, the three nested models were compared with assess the level of measurement invariance across country and gender. When comparing two hierarchically constrained models (eg, configural invariance model vs equal-loadings invariance model), the measurement invariance is achieved when ∆CFI, ∆RMSEA and ∆SRMR are less than (0.02).41
Finally, Rasch analysis was used to assess the psychometric properties of the CPDS measure and to evaluate DIF for each item. Two partial credit models (one for each subfactor) were used to examine (1) the separation reliability and separation index of the CPDS; (2) how the CPDS items fit in their embedded factors and (3) whether any of the CPDS items have substantial DIF across the country or gender. Separation reliability greater than 0.7, and a separation index greater than 2.0 were used as criteria to indicate satisfactory properties for the CPDS.42 Information-weighted mean square (infit MnSq) and outlier-sensitive MnSq (outfit MnSq) between 0.5 and 1.5 are proposed to indicate a good fit of an item in its embedded factor.43 Moreover, an item was flagged as having substantial DIF if the absolute value of the DIF contrast statistics was larger than (0.5).40 The EFA and PA were executed using ‘psych’ package in the R software,44 CFA and multigroup CFA using ‘lavaan’ package in the R software,45 while Rasch analysis and DIF using WINSTEPS V.3.75.0 (Winsteps, Chicago, Illinois, USA).
Results
One thousand three hundred and thirty-seven participants (n=1337) have participated in our study. On average, Kuwaiti participants were the youngest, and the Saudi participants were the oldest. Nevertheless, the entire sample across the four countries comprised a relatively young population and was predominated by females. Around 17.6% of participants (n=235) were from educational programmes related to health/medical field (eg, medicine, pharmacy), while 74.0% of participants (n=990) were from non-health related educational programmes (eg, social sciences, engineering, languages and literature, computer science). The remaining participants (n=112) did not report their study fields. Table 1 demonstrates further sociodemographic characteristics of the study participants.
Participants’ characteristics
The PA results with the scree plot (figure 1) suggested a two-factor structure for the CPDS. Specifically, eigenvalues extracted from the first two extracted factors (5.58 and 0.60) were larger than the eigenvalues of the parallel factors (in the upper limit of the 95% CI) obtained from the simulated data (0.58 and 0.22). The factor loadings (table 2) also supported the two-factor solution. Moreover, items 5 and 7–12 were embedded in factor 1 (physiological symptoms); items 1–4 and 6 were embedded in factor 2 (emotional symptoms).
EFA and PA findings for the CPDS (using the Algerian sample, n=447)
The parallel analysis results with the scree plot suggested a two-factor structure for the COVID-19 Psychological Distress Scale. Specifically, eigenvalues extracted from the first two extracted factors (5.58 and 0.60) were larger than the eigenvalues of the parallel factors (in the upper limit of the 95% CI) obtained from the simulated data (0.58 and 0.22). FA, factor analysis.
The internal consistency of the CPDS was satisfactory in both factors and the entire CPDS across the four countries (factor 1: α=0.84 for Kuwait, 0.91 for Saudi Arabia, 0.88 for Algeria and 0.81 for Yemen; Factor 2: α=0.80 for Kuwait, 0.80 for Saudi Arabia, 0.84 for Algeria and 0.80 for Yemen; entire CPDS: α=0.88 for Kuwait, 0.92 for Saudi Arabia, 0.91 for Algeria, and 0.88 for Yemen). Moreover, the CFA results showed that the two-factor structure of the CPDS was supported. Specifically, except for the significant χ2 test (χ2=101.89; df=53; p<0.001).
Given the well-known sensitivity of the χ2 statistic to sample size, it is no longer relied on as a basis for acceptance or rejection, and fit indices were applied for the purpose of evaluating model fit.46 47 All global fit indices were excellent (CFI=0.996; RMSEA=0.026; SRMR=0.040). Multigroup CFA additionally suggested that measurement invariance of the CPDS was generally supported across country (∆CFI=0.008 and 0.003; ∆RMSEA=0.032 and 0.004; and ∆SRMR=0.017 and 0.004) and gender (∆CFI=0.005 and 0.004; ∆RMSEA=0.013 and 0.009; and ∆SRMR=0.007 and 0.005) (table 3). Moreover, factor loadings of the CPDS were high (range 0.48–0.84) across country and gender (table 4).
Findings of multigroup confirmatory factor analysis for the COVID-19 Psychological Distress Scale (n=1337)
Factor loadings from configural model and scalar-invariant model of multigroup confirmatory factor analysis for the COVID-19 Psychological Distress Scale (n=1337)
The Rasch analysis showed that the CPDS had excellent separation item reliability (0.99), separation item index (9.26), but not person separation reliability (0.52), and person separation index (1.04). The CPDS factor 2 had excellent separation item reliability (1.00), separation item index (23.21), person separation reliability (0.79) and person separation index (1.92). The trait levels of the CPDS items ranged between −0.49 and 0.62 for factor 1 and between −1.41 and 1.16 for factor 2. All items fit well in their embedded construct (infit MnSq=0.78 to 1.32 and outfit MnSq=0.66 to 1.27 for factor 1; infit MnSq=0.72 to 1.20 and outfit MnSq=0.71 to 1.19 for factor 2).
No substantial DIF items were displayed across gender and country, except for item 7 (DIF contrast=−0.52 across gender; DIF contrast=0.59 across countries of Kuwait and Yemen) (table 5).
Rasch analysis findings for the COVID-19 Psychological Distress Scale across countries and genders (n=1337)
Discussion
The overall purpose of this study was to develop and evaluate the psychometric properties of a screening tool that can be used to assess the predisposition to COVID-19-induced psychological distress, namely, the 12-item CPDS. The CPDS is considered the first designed Arabic measure of COVID-19-related psychopathology and was validated on medium, appropriately sized samples from four Arabic countries, reflecting diverse cultures. Two clear-cut psychometrically sound factors were found that reflect the physiological and emotional symptoms of the COVID-19 distress. These factors showed evidence of stability when tested against an independent sample using CFA. Importantly, the two-factor structure was found to be invariant across gender and country. Thus, the CPDS is applicable in most Arab cultures, and the derived scores can be validly compared without the need for specific gender adjustments.
In addition, the results of Rasch analysis showed that all the items fit well in their embedded construct; only one item showed somewhat substantial DIF across gender and country. This item was related to sleep disturbance due to so much thinking about being infected by a coronavirus. Sleep disturbance represents a common physiological symptom associated with clinically elevated fear and anxiety.48 However, a recent paper has reported significant gender differences regarding sleep disturbances in many anxiety disorders.49 Generally, researchers who intend to use the 12-item CPDS as a screening tool of COVID-19-related distress for comparison between gender or across the country should handle this only item with a specific caution.
The 12 items of the CPDS scale are loaded into two independents but highly correlated factors, that are, the physical and emotional symptoms of anxiety. It is not surprising to see a distinct emotional factor in the Arabic culture. It has been suggested that Latino and Arab collectivists tend to express emotion in a ‘rhetorical and exaggerated’ manner.50 We believe that the two-factor structure found in the current research may be a result of the collectivistic nature of the Arabic culture.51 This might suggest that the CPDS could be safely modified to apply for any pandemics in Arab cultures since it is expected that pandemics are mainly associated with lockdown, quarantine, stay-at-home orders, and physical distancing. Such confinement measures may result in intense emotional and psychological responses such as anxiety in cultures like the Arab culture.
People who reported a high frequency of emotional symptoms (eg, afraid of being infected with coronavirus, and preoccupied with the idea of being infected by coronavirus); also reported a high frequency of physical symptoms (eg, hardly sleep from thinking too much about being infected by coronavirus, and headaches when receiving news about new cases of coronavirus infection). This result provided evidence that the physical condition and emotional state of anxiety for healthy participants are interrelated. Because negative changes in emotional valence influence pain and fatigue symptoms of anxiety, one might expect that positive changes in emotional valence may be associated with improvements in the physical condition. According to our results, researchers should use the CDPS as two separate subscales. There is no evidence that these subscales could be summed into only one distress score.
An important quality of the 12-item CPDS is that its items reflect both distressing physical symptoms, as well as emotional symptoms of anxiety. The mass hysteria of the COVID-19 pandemic and the common biological symptoms of it have left many people thinking that they are infected with the virus while they are actually not. A screening tool like the CPDS may help to identify people with physiological and emotional distress aiming at providing appropriate as well as timely mental health services before symptoms get worse and without putting unnecessary load on medical facilities. On the other hand, people who test positive for COVID-19 can also benefit from the CPDS tool that is, If COVID-19 patients have high scores on the CPDS, they will be considered as candidates for psychological help; thus, raising the clinical utility of the CPDS.
Our current study sheds the light on a psychometrically reliable tool that can be used in screening for COVID-19-induced distress. Nevertheless, our study has some limitations that should be acknowledged and considered. The nature of the online survey and the sampling method employed may have affected the representativeness of our study sample (in terms of sociodemographic characteristics of the study population); thus, limiting the generalisability of results to other Arab countries. However, this was the only feasible approach considering the COVID-19 pandemic and its associated measures and lockdown. Also, due to the pandemic crisis, random sampling or stratified sampling was not feasible. Moreover, participants were recruited from academic settings; thus, their educational level may presumably be higher than that of the general population. Our rationale behind this choice was motivated by the nature of our web-based survey aiming to reach participants who most commonly use online platforms and have access to the internet during the pandemic and lockdown. The web-based survey has eliminated any geographical boundaries during the recruitment of participants who may live in different regions within the four Arab countries.
Conclusion
The 12-item CPDS was found to be measurement invariant across country and gender. The 12-item CPDS, with its promising psychometric properties, may help healthcare professionals to identify people with COVID-19-induced psychological distress in Arab contexts. The CPDS as a screening tool might need some more evidence of construct validity including correlations of the CPDS scores with other psychometrically sound scales. Our Study also focused on only four Arab countries, which might limit the results to only those countries. Although the authors deliberately chose countries that reflect the diversity of Arabic cultures, further studies may assess the clinical utility, psychometric properties, and measurement invariance of the 12-item CPDS in other Arab countries and among diverse study populations.
Data availability statement
Data are available on reasonable request. The data used in the current study are confidential and cannot be publicly shared. This was also stated in the participants’ consent. However, it is available from the corresponding author on a reasonable request.
Ethics statements
Ethics approval
The study has been approved by the research ethics committee of the Faculty of Humanities and Social Sciences at The University of Djillali Liabes Sidi Bel Abbes in Algeria (Reference# UDL/Lab RPE01/2020).
Acknowledgments
The authors offer their sincere thanks and appreciation to Dr. Christian Krägeloh, Department of Psychology, Auckland University of Technology, New Zealand, and to the doctoral candidate Mohsen Alyami, Department of Psychological Medicine, School of Medicine, University of Auckland, New Zealand for providing guidance related to this research study.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors Study conception and design: FZEA, C-YL, MLR, MAA and AA. Acquisition of data: FZEA, MLR, MAA and AA. Analysis and interpretation of data: FZEA, HFG and C-YL. Drafting of the manuscript: FZEA, ABA-T, HFG, C-YL and MAA. Critical revision and editing: FZEA, ABA-T and HFG. All authors have approved the submission of this version.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.