Back to Journals » Psychology Research and Behavior Management » Volume 15

Online Mental Health Survey for Addressing Psychosocial Condition During the COVID-19 Pandemic in Indonesia: Instrument Evaluation

Authors Sunjaya DK , Sumintono B , Gunawan E , Herawati DMD , Hidayat T

Received 2 November 2021

Accepted for publication 1 January 2022

Published 18 January 2022 Volume 2022:15 Pages 161—170

DOI https://doi.org/10.2147/PRBM.S347386

Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 2

Editor who approved publication: Professor Mei-Chun Cheung



Deni Kurniadi Sunjaya,1 Bambang Sumintono,2,3 Elvine Gunawan,1 Dewi Marhaeni Diah Herawati,1 Teddy Hidayat1

1Department of Public Health, Faculty of Medicine, Universitas Padjadjaran, Bandung, Indonesia; 2Faculty of Education, Universiti Malaya, Kuala Lumpur, Malaysia; 3Faculty of Education, Universitas Islam Internasional Indonesia, Jakarta, Indonesia

Correspondence: Deni Kurniadi Sunjaya
Department of Public Health, Faculty of Medicine, Universitas Padjadjaran, Jalan Eyckman No. 38, Bandung, Indonesia
Tel +62 82218893543
Email [email protected]

Background: Regular monitoring of the pandemic’s psychosocial impact could be conducted among the community but is limited through online media. This study aims to evaluate the self-rating questionnaire commonly used for online monitoring of the psychosocial implications of the coronavirus disease 2019 (COVID-19) pandemic.
Methods: The data were taken from the online assessment results of two groups, with a total of 765 participants. The instruments studied were Self-Rating Questionnaire (SRQ-20), post-traumatic stress disorder (PTSD), and Center for Epidemiological Studies Depression Scale-10 (CESD-10), used in the online assessment. Data analysis used Rasch modeling and Winsteps applications. Validity and reliability were tested, and data were fit with the model, rating scale, and item fit analysis.
Results: All the scales for outfit mean square (MnSq) were very close to the ideal value of 1.0, and the Chi-square test was significant. Item reliability was greater than 0.67, item separation was greater than 3, and Cronbach’s alpha was greater than 0.60; all the instruments were considered very good. The raw variance explained by measures for the SRQ-20, PTSD, and CESD-10 was 30.7%, 41.6%, and 47.6%, respectively. The unexplained eigenvalue variances in the first contrast were 2.3, 1.6, and 2.0 for the SRQ-20, PTSD, and CESD-10, respectively. All items had positive point-measure correlations.
Conclusion: The internal consistency of all the instruments was reliable. Data were fit to the model as the items were productive for measurement and had a reasonable prediction. All the scales are functionally one-dimensional.

Keywords: online assessment, psychosocial, pandemic COVID-19, instrument evaluation

Introduction

The coronavirus disease 2019 (COVID-19) attacks many countries with long incubation periods, high morbidity, and mortality rates for both the community and health workers.1,2 This pandemic had a major impact on all aspects of people’s lives and the country, both economically and public health such as international travel dropped drastically, school closure happened everywhere and remote working become common, so that monitoring and surveillance efforts were initiated.3,4 Surveillance efforts are not only related to the disease but also its effects on psychosocial and economic problems which significantly impact to the people way of life.5 Government decisions made to reduce disease spread by isolation and quarantine measures change the structure of activities that are at the core of mental health, including social connectivity, daily routine activities, access to information sources, and the duration and frequency of device use.6,7

Many monitoring and research activities have been performed on COVID-19 and its health and socio-economic effects. In the context of public health, disease surveillance, and mitigation of the impact of COVID-19 are needed to develop intervention strategies to deal with the pandemic and its effects. Monitoring and research conducted in pandemic situations face problems with direct measurements because of restrictions to reduce disease transmission, several studies show about this such as in China,8–10 Spain,8 USA,9 Poland,10 Iran,11 the Philiphines12 and Vietnam.13 Those studies inform direct challenges that has never been face before in many parts of the world. Therefore, many of these activities are conducted online. This arrangement can affect the quality of the process and the results of the research. However, this method is a safe option for both public and research interests.

This public health emergency does not only affect physical health but also affects one’s mental health.14 The assessment of psychosocial status in the general population is very important to reduce the negative impact of a pandemic. Several independent research institutes and researchers measure the state of mental disorders in the community. Psychometric standard instruments are widely used and administered online.

Self-Rating Questionnaire (SRQ) is a screening instrument that is self-administered online to screen for mental health disorders in general.15 The SRQ is a screening instrument used by the Ministry of Health of the Republic of Indonesia to conduct Basic Health Research.16,17 The measurement results with this instrument can quickly detect emotional and mental disorders.

One of the negative effects that need to be evaluated is post-traumatic stress disorder (PTSD). PTSD usually occurs after more than one month but within six months of the traumatic event and causes suffering or significant functional impairment. Traumatic stress occurs beyond the limits of ordinary human experience. This condition occurs when a person experiences serious threats to their own psychological and physical safety or their loved ones.2,18,19 PTSD has been reported as a psychological effect of SARS and virus H1N1 epidemics, and recently used in COVID-19 pandemic studies too.20,21 Therefore, PTSD should also receive attention in the COVID-19 pandemic. PTSD causes dramatic changes in mood and cognition. Based on the PTSD cognitive model, negative emotions (sadness, fear, anger) can cause a person to adopt negative judgments and negative coping styles.2,6,19

The CESD (Center for Epidemiological Studies Depression Scale) is a screening tool used to detect cases of depression in the general population.22,23 The CESD-10 is a more concise version and is effective enough to be used in surveillance by health professionals and for confirmation by specialists at a later stage. Are these instruments then reliable for measuring online? It is necessary to evaluate this method so that its sustainable use for disaster mitigation and surveillance of a pandemic’s impact can be continued. This study aims to evaluate the self-rating questionnaires commonly used for online monitoring of the psychosocial implications of the COVID-19 pandemic. The research results are expected to be useful to researchers and the public in mental health monitoring and surveillance and further online research in an epidemiological study.

Materials and Methods

Research Design

We conducted several online pandemic survey packages for different groups covering the general national population limited to specific populations. For instrument evaluation, we decided to select data from two surveys that used the same three instruments and had their data entered on approximately the same date.

There were 765 respondents who participated in this study from West Java Province, Indonesia. Convenience sampling technique was used, respondents were administered an online survey via the mailing list/online platform. With regard to ethical considerations, the participants’ consent to take part in this study was obtained before they filled in the questionnaire. On the front cover of the questionnaires’ online form, it was stated that the respondents are given a choice whether to participate in the survey. Participation was strictly voluntary and anonymous. Thus, by completing the questionnaire, the respondents provided their consent. Details of the respondent demographic profiles are shown in Table 1.

Table 1 Demographic Data of Respondent (N = 765)

Instrument

Three instruments were used in this study: the SRQ (Self-Rating Questionnaire), PTSD (Post-Traumatic Stress Disorders), and CESD-10 (Center for Epidemiological Studies Depression Scale). The Self Reporting Questionnaire has been developed by WHO to screen psychiatric disturbance. In Indonesia SRQ consist of 20 items (SRQ-20) was widely used in primary health center, hospitals, and Ministry of Health to screen psychiatric symptoms like neurotic, substance abuse, psychotic symptoms. Screening with SRQ-20 is widely used because easily to be used, short period of time for filling the questionnaire, self-reported with two choices (Yes or No).24–31

The PTSD instrument is for measuring psychological situations and consists of five items with two choices (Yes or No). In clinical area, the PTSD is used to helped health professional identified emotional disturbance in daily practice. The code for PTSD is F 43.10 with consist of persistent re-experienced symptoms, persistence avoidance of stimuli associated with trauma, persistence symptoms of increased arousal which not present before the trauma with two choices (Yes or No).32,33 Meanwhile, the CESD-10 is a screening tool for depression with a scale consisting of 10 items that provides five choices (0 to 4) each.

The Model of Measurement

Rasch model analysis was employed in this study to analyze the data collected from the three questionnaires. This study was chosen as it can provide accurate and precise latent trait measurements about measuring mental health. According to Wright and Mok, a good and valid measurement model has to follow five measurement principles for human science. They are to (a) yield a linear measure, (b) overcome missing data, (c) provide a precision estimate, (d) discover outliers or misfits, and (e) be replicable.34 The Rasch model fulfills these compared with other measurement models.

The Rasch model is a subset of a larger group of measurement models, called item response theory (IRT), which transform raw ordinal type data using probability and logarithms to become equal-interval scale data called logit (log odd unit). The Rasch model had been used widely to analyze psychometric data in many fields, such as educational research, language assessment, and health sciences.35–39 Rasch model analysis also provided an extremely effective alternative to investigate the psychometric properties of a cognitive and non-cognitive instrument to address response bias.40,41

All collected raw data was input into a Microsoft Excel file, checked by the Rasch measurement model software WINSTEPS version 3.73, for data validation and cleaning. No missing data were found. Rasch analysis was performed using the Rasch Rating Scale Model (RSM), an extension of the Rasch model for dichotomous data developed by Andrich.38

The content validity and internal consistency reliability of each scale were determined to validate the three instruments used in this study. The first stage is to identify at the instrument level, which includes data fit to the model, Chi-square test, item reliability, item separation, and Cronbach’s alpha indices were examined as well. The uni-dimensionality requirement was investigated using the Principal Components Analysis of Rasch measures and residuals. It can be asserted that the data is fundamentally one-dimensional if the Rasch measurement indicates a relatively elevated percentage of explained variance (at least 20% for dichotomous data, which were the SRQ-20 and PTSD questionnaires, and at least 40% for polytomous data applied to the CESD-10 questionnaire) and the first residual components of the unexplained variances are less than 3 eigenvalues.42

Rating scale analysis is another examination to check the effectiveness rating given to respondents. Several criteria have been suggested by Linacre to diagnose a malfunctioning empirical rating scale used in the instrument.43 A rating scale at an ideal point can be considered when: a) there are at least 10 observations in each category, b) the person’s measurements are an average by category and increase monotonically with the rating scale, c) outfit MnSq should be less than 1.5 in each step, and d) step difficulties should advance no less than 1.4 and not more than 5 logits.44

The appropriateness of the item quality was checked two indices because the number of respondents was more than 500 which are the Outfit MnSq and Point-measure correlation (Pt-Measure Corr) for each item.36,45 The Wright map was displayed to demonstrate the spread item difficulty and respondent abilities comprehensively (Figures 1,2,3).

Figure 1 Variable map of SRQ-20.

Figure 2 Variable map of PTSD.

Figure 3 Variable map of CESD-R10.

The presence of test items that functioned differently, a response bias, was explored for each respondent’s demographic variable. DIF analysis was also executed. A moderate item DIF was considered to be present if the difficulty parameters between different groups of demographic variables (such as male and female) fulfilled three criteria, which are if it had a t-value of less than −2.0 or more than 2.0, DIF contrast value of less than −0.5 or more than 0.5, and the p (Probability) value of less than 0.05 or greater than −0.05.36,37,46

Ethical Consideration

This research is in accordance with the Declaration of Helsinki and has obtained ethical clearance from the Ethics Committee of the Faculty of Medicine Universitas Padjadjaran No. 496/UN6.KEP/EC/2020. All participants gave written informed consent about the purpose of the study.

Results

Reliability and Data Fit to the Model

The content validity of the three questionnaires was identified by assessing fit validity. The Rasch instrument-fit statistics assessment was utilized to determine the degree to which data fit with the model and, therefore, fit the concept of a single attribute.38 Index of Outfit MnSq as shown in Table 2, it was found that all the scales were very close to the ideal value of 1.0 and within the acceptance ranges of 0.5 to 1.5.36 This result was also supported by the Chi-square test where all instrument results were significant. Hence, the data fit the model as items were productive for measurement and had a reasonable prediction value. For measuring reliability, three indices, which were item reliability (should be more than 0.67), item separation (more than 3), and Cronbach’s alpha (more than 0.67), demonstrated that all three instruments were considered very good. Therefore, the internal consistency of all instruments was shown to be reliable.

Table 2 Reliability and Model Fit (N = 765)

The analysis of dimensionality in the Rasch model showed that three of the scales met the assumptions of unidimensionality, both for the dichotomous data scale (SRQ-20 and PTSD) and polytomous data of CESD-10. The raw variance explained by measures for the SRQ-20, PTSD, and CESD-10 was 30.7%, 41.6%, and 47.6%, respectively. The unexplained eigenvalue variances in the first contrast were 2.3, 1.6, and 2.0, for the SRQ-20, PTSD, and CESD-10, respectively. These results show that all the scales are functionally one-dimensional.

Rating Scale Analysis

The effectiveness of the rating scale used in each questionnaire was examined. Results illustrated that there were no disordered thresholds from one scale to another (Table 3). All categories in the three questionnaires were chosen by more than 100 respondents. The average person measured by category moved up monotonically with the respective rating scale for the SRQ-20, PTSD, and CESD-10 scales. All categories had outfit mean squares close to the ideal value of 1.0 in each step, except for rating 4 on the CESD-10 scale that had a value of more than 1.5. Gaps in step difficulties are always obvious for dichotomous data, in this case, the SRQ-20 and PTSD scales.36,38 Also, for the CESD-10 scale with polytomous data (5 ratings), the steps between rating scales are greater than 1.4 and lower than 5 logits.44 Hence, all Likert scale categories in the three questionnaires used in this study were well functioning.

Table 3 Rating Scale Model Category Statistics (N = 765)

Item Fit Statistics Analysis

The fit of the data to the Rasch model was analyzed for all items in each scale in the study. The mean of item measure (logit) of three questionnaires has an ideal value of 0.0 logit, showing that all scales have an ideal starting point for measurement (Table 3), and none of the three scales has an extreme measure value. The values of the standard deviation for two scales have greater (more than 1 logit) values. This means that they are better regarding the spread of item difficulty, which was in the SRQ-20 and PTSD questionnaires; for the CESD-10 scale, the standard deviation is half (0.51 logits). This also can be seen in the range of item measures on each scale, whereas CESD-10 has less range than SRQ-20 and PTSD scale. There were three items, two in the SRQ and one in the CESD-10 scale, that were considered misfit item values of the outfit MnSq larger than 1.5. However, all items had positive point-measure correlations (there was no item has polarization), the range was greater than 0.4 for PTSD and the CESD-10 scale.

The DIF analysis was conducted to examine the item bias according to each demographic variable collected, which were gender, age, and marital status of the respondent, on each scale. As shown in Table 4, item DIF is nonexistent only for male and female respondents in the CESD-10 scale; it has item DIF for another situation. The lowest number was on the PTSD scale (three items), followed by the CESD-10 scale (4 items) and the SRQ-20 scale (15 items).

Table 4 Item Fit Statistics (N = 765)

Discussion

Measurements require a valid and reliable instrument so that they can be used in epidemiological studies. Evaluation of the three instruments show that the SRQ-20, PTSD, and CESD-10 are valid and reliable to be used as online measuring tools. In practice, health office, surveillance officers on the front line or at institutions who intend to monitor their staff’s psychological impact can use these instruments without hesitation.

We reported items showing misfit: 2 items on the SRQ-20 and 1 item on the CESD-10. When an item is found that misfits the model, it means that it is not working as expected in the model. Misfit can occur and be caused by the participants’ interpretation of the item statement or treatment of online instruments. This can happen because of the use of online instruments.

The treatment of the misfit item is to remove the item. However, this will interfere with the measurement results and is very detrimental to the measurement. These instruments have also been used in daily practice for a long time and may not align with these instruments’ developer. Is the measurement result with the misfit item still acceptable? Kohler and Hartig suggest comparing the results with inclusion and those with the deletion of misfit items. If there is a decrease in reliability, the items that are misfit are included.47 Thus, the instruments used online should be evaluated and conveyed in each measurement result.

Instrument users also need to pay attention to the occurrence of DIF in certain groups. DIF means that there is a group influence on the measurement results. DIF does not mean merely measurement bias. The DIF analysis results provide support for interpreting research results with awareness in the group where DIF occurs. The Rasch model is more convenient than other, more complex models for detecting DIF because it can determine the whole group’s item parameters without the influence of confounding parameters.48 DIF contributes to better accuracy of the final results in epidemiological studies because it increases the scale of the composite measurement.48

COVID-19 is a disease with a high infection rate and a long incubation period caused by SARS-CoV-2.49 It has had a significant effect on the global economy and public health. Although patients and the community have been affected, mental health is not prioritized, even though it imposes a formidable burden.6 In the early days of the pandemic, large parts of society experienced much fear and anxiety.50,51 Several cases of suicide and attempted suicide in Indonesia occurred in patients who were treated and doing self-isolation.52

Social restrictions are executed in the form of wearing face mask, lockdowns, quarantines, or social restrictions.10,53 This disruption creates mental pressure on individuals. These restrictions also affect economic activity, cause unemployment and lack of income, and disrupt food access, thereby ultimately causing anxiety, fear, and panic.50,54 Restricted religious activities create a religious sentiment, tension, and collective pressure in society. The implementation of the quarantine and isolation policy (lockdown) and the increasing number of cases affect the core of human mental health, including relationships in daily routine activities and adverse psychosocial effects.6,19

Studies showed 42.65% and 67.09% of participants self-reported psychiatric disorders and a high level of PTSD, respectively. At the initial phase of the disease, there was a high prevalence of depression (48.3%), anxiety (22.6%), and a combination of depression and anxiety (19.4%). The study results of COVID-19 survivors indicated that 28% for PTSD, 31% for depression, 42% for anxiety, 20% for obsessive-compulsive symptoms, and 40% for insomnia.6,47

Is it true that fear, panic, and mental disorders occur? How significant is the mental, emotional disorder? Is it necessary to intervene? Who needs to be treated? These are questions that can be answered by monitoring and surveillance through measurement. Communities, families, and governments need the information to be able to act and care. Psychosocial surveillance can help identify health status, including mental illness due to the COVID-19 pandemic, and the need for the more effective and rational distribution of resources.55 Several studies suggested that Cognitive Behavior Therapy (CBT) is provide good evidences can ameliorate this situation,56,57 which also include internet CBT.58

Psychosocial impact monitoring is conducted for surveillance and response purposes and for measuring the impact of disasters to decide a policy. The risks for face-to-face measurement are very high, apart from the restriction regulations themselves. Online surveys are the best choice since most people have access to internet59 and can afford gadgets.

This study is limited to the results of measuring secondary data. A comparison with the results of direct measurements by experts will provide better evidence of the extent to which online surveys are reliable and trustworthy.60 Other studies has informed that COVID-19 pandemic hemodynamic changes in the brain61–64 which is beyond the context of this study.

Conclusion

The SRQ-20, PTSD, and CESD-10 questionnaires can be used as online instruments for assessing the psychological impact of the COVID-19 pandemic. The use of an online self-rating questionnaire to monitor the community’s psychosocial condition due to the pandemic’s impact should be conducted with caution. Researchers analyze not only the measurement results but also the instrument itself. Analysis with Rasch modeling is very instrumental and clearer to explain other facts behind the measurement results.

Acknowledgments

The authors would like to thank the Centre for Economic Development Studies, Faculty of Economic, UNPAD for supporting the research. Also to the two anonymous reviewers that provide good suggestion to the improvement of this article, we thank you for that.

Disclosure

All authors have no financial interests or conflicts of interest in this study.

References

1. Gulati A, Pomeranz C, Qamar Z, et al. A comprehensive review of manifestations of novel coronaviruses in the context of deadly COVID-19 Global Pandemic. Am J Med Sci. 2020;360(1):5–34. doi:10.1016/j.amjms.2020.05.006

2. Liu N, Zhang F, Wei C, et al. Prevalence and predictors of PTSS during COVID-19 outbreak in China hardest-hit areas: gender differences matter. Psychiatry Res. 2020;287:112921. doi:10.1016/j.psychres.2020.112921

3. Xiong J, Lipsitz O, Nasri F, et al. Impact of COVID-19 pandemic on mental health in the general population: a systematic review. J Affective Disord. 2020;277:55–64. doi:10.1016/j.jad.2020.08.001

4. Wang C, Tee M, Roy AE, et al. The impact of COVID-19 pandemic on physical and mental health of Asians: a study of seven middle-income countries in Asia. PLoS One. 2021;16(2):e0246824. doi:10.1371/journal.pone.0246824

5. Ceban F, Nogo D, Carvalho IP, et al. Association between mood disorders and risk of COVID-19 infection, hospitalization, and death: a systematic review and meta-analysis. JAMA Psychiatry. 2021;78(10):1079–1091. doi:10.1001/jamapsychiatry.2021.1818

6. Li Q. Psychosocial and coping responses towards 2019 coronavirus diseases (COVID-19): a cross-sectional study within the Chinese general population. QJM. 2020;1139(10):):731–738. doi:10.1093/qjmed/hcaa226

7. Lee Y, Lui LMW, Chen-Li D, et al. response moderates the mental health impact of COVID-19: a systematic review and meta-analysis of depression outcomes across countries. J Affective Disord. 2021;290:364–377. doi:10.1016/j.jad.2021.04.050

8. Wang C, López-Núñez MI, Pan R, et al. The impact of the COVID-19 pandemic on physical and mental health in China and Spain: cross-sectional study. JMIR Form Res. 2021;5(5):e27818. doi:10.2196/27818

9. Wang C, Tripp C, Sears SF, et al. The impact of the COVID-19 pandemic on physical and mental health in the two largest economies in the world: a comparison between the United States and China. J Behav Med. 2021;44:741–759. doi:10.1007/s10865-021-00237-7

10. Wang C, Chudzicka-Czupała A, Grabowski D, et al. The association between physical and mental health and face mask use during the COVID-19 pandemic: a comparison of two countries with different views and practices. Front Psychiatry. 2020;9(11):569981. doi:10.3389/fpsyt.2020.569981

11. Wang C, Fardin MA, Shirazi M, et al. Mental health of the general population during the 2019 Coronavirus Disease (COVID-19) pandemic: a tale of two developing countries. Psychiatry Int. 2021;2:71–84. doi:10.3390/psychiatryint2010006

12. Tee ML, Tee CA, Anlacan JP, et al. Psychological impact of COVID-19 pandemic in the Philippines. J Affective Disord. 2020;277:379–391. doi:10.1016/j.jad.2020.08.043

13. Thanh LXT, Kim DA, Jayson T, et al. Evaluating the psychological impacts related to COVID-19 of Vietnamese people under the first nationwide partial lockdown in Vietnam. Front Psychiatry. 2020;11:824. doi:10.3389/fpsyt.2020.00824

14. Wang C, Chudzicka-Czupała A, Tee ML, et al. A chain mediation model on COVID-19 symptoms and mental health outcomes in Americans, Asians and Europeans. Sci Rep. 2021;11:6481. doi:10.1038/s41598-021-85943-7

15. Wang C, Pan R, Wan X, et al. Immediate psychological responses and associated factors during the initial stage of the 2019 Coronavirus Disease (COVID-19) epidemic among the general population in China. Int J Environ Res Public Health. 2020;17:1729. doi:10.3390/ijerph17051729

16. Stratton KJ, Aggen SH, Richardson LK, et al. Evaluation of the psychometric properties of the Self-Reporting Questionnaire (SRQ-20) in a sample of Vietnamese adults. Compr Psychiatry. 2013;54(4):398–405. doi:10.1016/j.comppsych.2012.10.011

17. Beusenberg M, John O. A User’s Guide to the Self Reporting Questionnaire. Geneva: World Health Organization; 1994.

18. Kurniawan A. Patients with reactions to severe stress or mourning (Pasien dengan reaksi terhadap stres berat atau berkabung. In: Birell S, Marwick K, editors. Psychiatry Crash Course. 1st Indonesian Edition. Singapore: Elsevier; 2015:163–172.

19. Dubey S, Biswas P, Ghosh R, et al. Psychosocial impact of COVID-19. Diabetes Metab Syndr. 2020;14(5):779–788. doi:10.1016/j.dsx.2020.05.035

20. Hao F, Tan W, Jiang L, et al. Do psychiatric patients experience more psychiatric symptoms during COVID-19 pandemic and lockdown? a case-control study with service and research implications for immunopsychiatry. Brain Behav Immun. 2020;87:100–106. doi:10.1016/j.bbi.2020.04.069

21. Hao F, Tam W, Hu X, et al. A quantitative and qualitative study on the neuropsychiatric sequelae of acutely ill COVID-19 inpatients in isolation facilities. Transl Psychiatry. 2020;10(1):355. doi:10.1038/s41398-020-01039-2

22. Baron EC, Davies T, Lund C. Validation of the 10-item Centre for Epidemiological Studies Depression Scale (CES-D-10) in Zulu, Xhosa and Afrikaans populations in South Africa. BMC Psychiatry. 2017;17:6. doi:10.1186/s12888-016-1178-x

23. Moon JR, Huh J, Song J, et al. The center for epidemiologic studies depression scale is an adequate screening instrument for depression and anxiety disorder in adults with congential heart disease. Health Qual Life Outcomes. 2017;15:176. doi:10.1186/s12955-017-0747-0

24. Ministry of Health of the Republic Indonesia, Communication and Community Service Bureau. Layanan Kesehatan Jiwa Bergerak Ramaikan Pameran Rakernas. (Mental Health Services are Moving to Enliven the National Work Meeting Exhibition). Available from: https://www.kemkes.go.id/article/view/19022000001/layanan-kesehatan-jiwa-bergerak-ramaikan-pameran-rakerkesnas.html. Accessed January 6, 2022.

25. Arini L, Syarli S. Early detection of mental disorders and psychosocial using Self Reporting Questionnaire/SRQ-29 (Deteksi dini gangguan jiwa dan masalah psikososial dengan menggunakan Self Reporting Qustioner/SRQ-29). Jurnal Keperawatan Muhammadiyah (Muhammadiyah Nursing J). 2020;3:1667–1672.

26. Uzhma LS, Sriatmi A, Jati SP, et al. Analysis of the implementation of the healthy Indonesia program with a family approach for people with severe mental disorders at public health center. Schizoprenia case study at Kedungmundu Public Health Center Semarang City (Analisis pelaksanaan program Indonesia Sehat dengan pendekatan keluarga bagi orang dengan gangguan jiwa berat di Puskesmas. Studi kasus Skizofrenia di Puskesmas Kedungmundu Kota Semarang). Jurnal Kesehatan Masyarakat (J Public Health). 2019;7:2.

27. Nurjanah S. Emotional mental disorders on clients pandemic Covid 19 at quarantine house. Jurnal Ilmu Keperawatan Jiwa (J Psy Nursing). 2020;3:3.

28. Ministry of Health of the Republic of Indonesia. Sehat Jiwa (Healthy Soul); 2017. Available from: http://sehat-jiwa.kemkes.go.id/deteksi_dini. Accessed January 6, 2022.

29. Tempo. Check Mental Condition via Android Application (Cek Kondisi Jiwa Lewat Aplikasi Android). Tempo; 2019.

30. Agiananda F, LukmanPR. Psychological Well Being of Medical Students and Health Care Workers in Covid-19 Pandemic. Oral presented at: Sharing Experience from Dr Cipto Mangunkusumo National General Hospital-Main Teaching Hospital of Faculty of Medicine Universitas Indonesia. 2020.

31. Wiguna T, Keliat BA, Matulessy A, et al. Protocol for Mental and Social Health Support for Children and Adolescent During the Adaptation Period for New Habits of the Covid-19 Pandemic (Protokol Dukungan Kesehatan Jiwa Dan Psikosial (DKJPS) Anak Dan Remaja Pada Masa Adaptasi Kebiasaan Baru Pandemi Covid-19). Directorate of Prevention and Control of Mental Health and Drug problems, Directorate General of Prevention and Disease Control, Ministry of Health of the Republic of Indonesia; 2020.

32. World Health Organization. The ICD-10 Classification of Mental and Behavioural Disorders: Clinical Descriptions and Diagnostic Guideline; 1992.

33. Ministry of Health of the Republic of Indonesia, Directorate of Community Health Development. Guidelines for Classification of Mental Disorders Diagnosis in Indonesia III (Pedoman Penggolongan Diagnosis Gangguan Jiwa Di Indonesia III/PPDGJ III); 2004.

34. Wright BD, Mok MMC. An overview of the family of Rasch Measurement Models. In: E.V. Smith Jr. & R.M. Smith, Introduction Rasch Measurement: Theory, models and applications (pp. 1-24). Maple Grove, MN: JAM Press 2014.

35. Engelhard JG. Invariant Measurement: Using Rasch Models in the Social Behavioral, and Health Sciences. New York: Routledge; 2013.

36. Bond TG, Fox CM. Applying the Rasch Model: Fundamental Measurement in the Human Sciences. New York, NY: Routledge; 2015.

37. Andrich D, Marais I. A Course in Rasch Measurement Theory, Measuring in the Educational, Social and Health Sciences. Singapore: Springer; 2019.

38. Khine MS. Objective measurement in psychometric analysis. In: Khine MS, editor. Rasch Measurement Applications in Quantitative Educational Research. Springer Nature Singapore Pte Ltd; 2020:3–7.

39. Bradley KD, Peabody MR, Akers KS, Knutson N. Rating scales in survey research: using the Rasch model to illustrate the middle category measurement flaw. Survey Practice. 2015;8(1):1–14. doi:10.29115/SP-2015-0001

40. Boone WJ, Staver JR, Yale MS. Rasch Analysis in the Human Sciences. Dordrecht, The Netherlands: Springer; 2014.

41. Linacre JM. A User’s Guide to Winsteps & Ministep: Rasch-Model Computer Programs. Chicago: Winsteps.com; 2012.

42. Linacre JM. Optimizing rating scale category effectiveness. J Appl Meas. 2002;3(1):85–106.

43. Van Zile-tamsen C. Using Rasch analysis to inform rating scale development. Res High Educ. 2017;58:922–933. doi:10.1007/s11162-017-9448-0

44. Sumintono B, Widhiarso W. Application of Rasch Modelling in Educational Measurement. Cimahi: Trimkom Publishing House; 2015.

45. Zwick R. A review of ets differential item functioning assessment procedures: flagging rules, minimum sample size requirements, and criterion refinement. ETS Res Rep Ser. 2012;2012(1):i–30. doi:10.1002/j.2333-8504.2012.tb02290.x

46. Kohler C, Hartig J. Practical significance of item misfit in educational assessments. Appl Psychol Meas. 2017;41(5):388–400. doi:10.1177/0146621617692978

47. Mazza MG, De Lorenzo R, Conte C, et al. Anxiety and depression in COVID-19 survivors: role of inflammatory and clinical predictors. Brain Behav Immun. 2020;89:594–600. doi:10.1016/j.bbi.2020.07.037

48. Rouquette A, Hardoulin JB, Vanhaesebrouck A, et al. Differential item functioning (DIF) in composite health measurements scale: recommendations for characterizing DIF with meaningful consequences within the Rasch model framework. PLoS One. 2019;14:e0215073. doi:10.1371/journal.pone.0215073

49. Fu L, Wang B, Yuan T, et al. Clinical characteristics of coronavirus disease 2019 (COVID-19) in China: a systematic review and meta-analysis. J Infect. 2020;80(6):656–665. doi:10.1016/j.jinf.2020.03.041

50. Casagrande M, Favieri F, Tambelli R, et al. The enemy who sealed the world: effects quarantine due to the COVID-19 on sleep quality, anxiety, and psychological distress in the Italian population. Sleep Med. 2020;75:12–20. doi:10.1016/j.sleep.2020.05.011

51. Cao W, Fang Z, Hou G, et al. The psychological impact of the COVID-19 epidemic on college students in China. Psychiatr Res. 2020;287:112934. doi:10.1016/j.psychres.2020.112934

52. Diduga Stress, 3 Pasien Positif Covid-19 Nekat Melakukan Bunuh Diri. (Suspected of Being Stressed, 3 Patients Positive for Covid-19 Committed Suicide). 2020. Available from: www.suarajogja.id. Accessed January 6, 2022.

53. Thi LH, Xian LAJX, Jiaqian S, et al. Anxiety and depression among people under the nationwide partial lockdown in Vietnam. Front Public Health. 2020;8:656.

54. Xuan TB, Thi NH, Thi LH, et al. Impact of COVID-19 on economic well-being and quality of life of the Vietnamese during the national social distancing. Front Psychol. 2020;11:2289.

55. Khan Y, Schwartz B, Johnson I. Surveillance and epidemiology in natural disasters: a novel framework and assessment of reliability. PloS Curr. 2014;6. doi:10.1371/currents.dis.6773eb9d5e64b733ab490f78de346003

56. Ho CS, Chee CY, Ho RC. Mental health strategies to combat the psychological impact of Coronavirus Disease 2019 (COVID-19) beyond paranoia and panic. Ann Acad Med Singap. 2020;49(3):155–160. doi:10.47102/annals-acadmedsg.202043

57. Zhang MW, Ho RC. Moodle: the cost effective solution for internet cognitive behavioral therapy (I-CBT) interventions. Technol Health Care. 2017;25(1):163–165. doi:10.3233/THC-161261

58. Soh HL, Ho RC, Ho CS, et al. Efficacy of digital cognitive behavioural therapy for insomnia: a meta-analysis of randomised controlled trials. Sleep Med. 2020;75:315–325. doi:10.1016/j.sleep.2020.08.020

59. Geldsetzer P. Use of rapid online surveys to assess people’s perceptions during infectious disease outbreaks: a cross-sectional survey on Covid-19. J Med Internet Res. 2020;22(4):e18790. doi:10.2196/18790

60. Olszewska-Guizzo A, Mukoyama A, Naganawa S, et al. Hemodynamic response to three types of urban spaces before and after lockdown during the COVID-19 pandemic. Int J Environ Res Public Health. 2021;18:6118. doi:10.3390/ijerph18116118

61. Ho RC, Sharma VK, Tan BYQ, et al. Comparison of brain activation patterns during olfactory stimuli between recovered COVID-19 patients and healthy controls: a functional near-infrared spectroscopy (fNIRS) study. Brain Sci. 2021;11:968. doi:10.3390/brainsci11080968

62. Husain SF, Yu R, Tang TB, et al. Validating a functional near-infrared spectroscopy diagnostic paradigm for Major Depressive Disorder. Sci Rep. 2020;10(1):9740. doi:10.1038/s41598-020-66784-2

63. Husain SF, Tang TB, Yu R, et al. Cortical haemodynamic response measured by functional near infrared spectroscopy during a verbal fluency task in patients with major depression and borderline personality disorder. EBioMedicine. 2020;51:102586. doi:10.1016/j.ebiom.2019.11.047

64. Ho Cyrus SH, Lim Lucas JH, Lim AQ, et al. Diagnostic and predictive applications of functional near-infrared spectroscopy for major depressive disorder: a systematic review. Front Psychiatry. 2020;11:378. doi:10.3389/fpsyt.2020.00378

Creative Commons License © 2022 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.