Motors of COVID-19 Vaccination Acceptance Scale (MoVac-COVID19S): Evidence of Measurement Invariance Across Five Countries

Plain Language Summary

The advent of the novel coronavirus disease 2019 (COVID-19) has led to the disruption of individual’s normal lives. The devastating effect of COVID-19 can still be felt individually (putting on face masks, regular washing of hands, working from home), socially (physical distancing, reducing face-to-face contact), and economically (job loss or reduced working caused by lockdowns and quarantines). These restrictions were put in place to stem the spread of the virus. With the emergence of antivirus vaccines, most countries are now trying to secure a quota for their citizens. However, the vaccination drive has been met with hesitation from some individuals. To help boost the vaccination drive, the Motors of COVID-19 Vaccination Acceptance Scale (MoVac-COVID19S) was developed to assess COVID-19 acceptance levels. However, the MoVac-COVID19S has only been previously tested in Taiwan and mainland China. Therefore, the present study examined the construct validity and measurement invariance of the MoVac-COVID19S among individuals from five countries (ie, Taiwan, mainland China, India, Ghana, and Afghanistan). Using a cross-sectional survey, a total of 6053 participants living in these countries were recruited between January and March 2021. The results indicated that the fit indices using confirmatory factor analysis were relatively good across the countries except for root mean square error of approximation (RMSEA). The four-factor model using nine MoVac-COVID19S items was a model that had measurement invariance supported for both factor loadings and item intercepts across the five countries.

Introduction

The psychosocial and economic impact of the novel coronavirus disease 2019 (COVID-19) on individuals’ lives globally has remained substantial given that its variants are still active worldwide.^1–6 Several policies and implementations have been used to control the rapid transmission of COVID-19 including city lockdowns, home working, closures of schools, and border control.^7–10 However, these changes in human life seem to be a big challenge for individuals in maintaining a high level of compliance.^11,12 Consequently, clinical guidelines proposed by the World Health Organization have emphasized the importance of getting vaccinated.¹³ Indeed, the importance of vaccination has been documented because it has been evidenced as a public health intervention that is reliable and cost-effective.^14,15

However, there is a debate concerning the adverse effects of vaccination, and vaccine hesitancy has been observed in prior studies.^16–18 The estimates of a successful herd immunity to combat COVID-19ʹs infectiousness indicate that at least 70% of the population should get COVID-19 vaccinated to have effective immunity for that community.¹⁹ Therefore, improving the vaccination rate worldwide is the key to helping control the global spread of COVID-19 efficiently and effectively.²⁰ As aforementioned, vaccine hesitancy may slow down the speed of achieving herd immunity and is a potentially serious threat to global health.²¹ Therefore, having an instrument to assess individuals’ thoughts, considerations, and attitudes regarding COVID-19 vaccination may help in understanding the underlying reasons of vaccine hesitancy.²²

Although prior research has used different health behavior theories (eg, protection motivation theory, theory of planned behavior, and health belief model) to understand the potential underlying constructs for COVID-19 vaccine hesitancy, these theories were tested without using a standardized instrument.^22–28 However, some instruments have been developed to assess COVID-19 vaccine hesitancy with the use of a theory, such as the cognitive model of empowerment (CME).^29,30 One such instrument is the Motors of COVID-19 Vaccination Acceptance Scale (MoVac-COVID19S). Moreover, the MoVac-COVID19S was developed using rigorous psychometric testing methods.^29,30 The MoVac-COVID19S was modified based on the Motors of Influenza Vaccination Acceptance Scale (MoVac-Flu Scale),³¹ an instrument that was also developed using the CME.³²

Using CME, four constructs related to vaccine acceptance have been proposed in the MoVac-COVID19S: (i) values (whether individuals care about why it is important to get COVID-19 vaccinated); (ii) impacts (whether individuals believe in the COVID-19 vaccine’s effects); (iii) knowledge (whether individuals have the knowledge regarding COVID-19 vaccination uptake); and (iv) autonomy (whether individuals have confidence in getting COVID-19 vaccinated when they want to). Moreover, current evidence concerning the MoVac-COVID19S supports the use of a one-factor structure model (ie, without considering the aforementioned four CME constructs) and a four-factor structure model (ie, using the four CME constructs). In addition, known-group validity of the MoVac-COVID19S is satisfactory and measurement invariance of the MoVac-COVID19S across different subgroups (eg, Taiwanese vs. Chinese; males vs. females) is supported.^29,30

However, to the best of the present authors’ knowledge, the psychometric properties of the MoVac-COVID19S have only been tested among East Asian populations (ie, Taiwanese and mainland Chinese). Therefore, additional psychometric evidence is needed for the MoVac-COVID19S. More specifically, only when the MoVac-COVID19S is psychometrically sound across different populations can the MoVac-COVID19S be used to investigate and compare COVID-19 vaccine acceptance between individuals from different populations. Subsequently, vaccination uptake policies can be effectively designed and implemented across countries. Therefore, the present study examined the construct validity and measurement invariance of the MoVac-COVID19S among individuals from five different countries (ie, Taiwan, mainland China, India, Ghana, and Afghanistan).

Materials and Methods

Participants and Recruitment Procedure

Taiwanese Participants

Participants in Taiwan, who were all students, were recruited online (using Google Forms) and snowballing method via posts on social media pages and social networking apps (eg, Facebook and LINE). The first page of the survey delivered information concerning the study. When a participant agreed to participate after reading the study purpose and their rights in the study, the participants pressed an “agree” icon to continue the survey. Between January 5 and February 5 in 2021, 932 responses were collected. The MoVac-COVID19S used for Taiwanese participants was written in traditional Chinese (Taiwan’s official language). The Kaohsiung Medical University Chung-Ho Memorial Hospital (IRB ref: KMUHIRB-EXEMPT(I)-20200119) approved the data collection in Taiwan.

Mainland Chinese Participants

Participants in mainland China, who were all students, were recruited online in the same way as those in Taiwan with the same instructions except the survey was hosted on Sojump and the link to the survey was posted on Chinese social networking platforms (eg, WeChat). Between January 5 and January 16 in 2021, 3145 responses were collected. The MoVac-COVID19S used for mainland Chinese participants was written in simplified Chinese (mainland China’s official language). The Jianxi Psychological Consultant Association (IRB ref: JXSXL-2020-DE22), a local independent IRB, approved the data collection in mainland China.

Indian Participants

Participants in India, who were all students, were recruited online in the same way as those in Taiwan with the same instructions (survey hosted on Google Forms and the link posted on popular social media platforms such as Facebook and WhatsApp). Between July 25 and October 5 in 2021, 508 responses were collected. The MoVac-COVID19S used for Indian participants was written in English (official language of Indian universities). The University of Kashmir’s Department of Social Work (IRB ref: F-2 (MSW) KU/2021 dated 22-6-2021) approved the data collection in India.

Ghanaian Participants

Participants in Ghana, who were all students, were recruited offline from the Kwame Nkrumah University of Science and Technology (KNUST) using a convenience sampling method. The research team sought permission from various department heads and respective lecturers whose classes were used for the data collection. Prospective participants were given adequate information concerning the study’s purpose and their rights in the study. Those who provided written informed consent were given the surveys. Sufficient time was given for them to complete all the items in the survey. This procedure was used to collect 1244 responses between January 25 and March 12, 2021. Given that English is the official language in Ghana, all the participants completed the English version of the survey. The Kwame Nkrumah University of Science and Technology (IRB ref: CHRPE/AP/283/21) approved the data collection in Ghana.

Afghan Participants

Participants in Afghanistan, who were all healthcare providers, were recruited online in the same way as those in Taiwan with the same instructions (survey hosted on Google Forms and the link posted on popular social media platforms such as Facebook). Between April 1 and July 31 in 2021, 224 questionnaires were collected. The MoVac-COVID19S used for Afghan participants was written in English. The  Kateb University Hospital Ethics Committee (IRB ref: 2305) approved the data collection in Afghanistan.

The present study was conducted according to the principles stated in the Declaration of Helsinki with all participants providing informed consent and ethical approval obtained from respective institutions in each country. It should also be noted that somewhat different approaches and sampling methods were used across the five countries because there were different technology advancements in each of the countries. For example, Ghana has a poorer internet infrastructure compared to other countries. Therefore, it was difficult to carry out an online survey. Moreover, because of the COVID-19 pandemic, an online survey was used (where possible) to collect data because this type of data collection avoids the need for human contact and decreases the chance of infection.

Measure

Motors of COVID-19 Vaccination Acceptance Scale (MoVac-COVID19S)

The MoVac-COVID19S was used to assess COVID-19 vaccination acceptance. The scale comprises nine positively worded items and three negatively worded items. Prior evidence shows that the MoVac-COVID19S has wording effects that should be taken into account when testing its factor structure.^29,30 The MoVac-COVID19S items are rated on a seven-point scale from 1 to 7. After aligning the directions of the positively worded and negatively worded items, a higher score in the MoVac-COVID19S (including the entire instrument and the four domains) indicates a higher level of acceptance to get COVID-19 vaccinated.

Demographic measures

Demographic data were also collected in the survey including age (in years), gender (male or female), education level (undergraduate or postgraduate), and study major at university (health-related and non-health-related).

Data Analysis

The participants’ characteristics (eg, their age, gender, educational background) were firstly analyzed using descriptive statistics, including means (and standard deviations) and frequencies (and percentages). Then, four structure models were used to examine the construct validity of the MoVac-COVID19S using confirmatory factor analysis (CFA) for each subsample separately (ie, Taiwanese, mainland Chinese, Indian, Ghanaian, and Afghan participants). The four structure models include (a) a one-factor structure using nine MoVac-COVID19S items; (b) a four-factor structure using nine MoVac-COVID19S items; (c) a one-factor structure using 12 MoVac-COVID19S items taking account of wording effect; and (d) a four-factor structure using 12 MoVac-COVID19S items taking account of wording effect (Figure 1). According to the prior research,³⁰ the one-factor model contains the overall concept of vaccination acceptance and the four-factor model contains four concepts of vaccination acceptance corresponding to the CME model (ie, value, impact, knowledge, and autonomy).

Figure 1 Factor structure of the Motors of COVID-19 Vaccination Acceptance Scale (MoVac-COVID19S). (A) One-factor structure on 9 MoVac-COVID19S items; (B) Four-factor structure on 9 MoVac-COVID19S items; (C) One-factor structure with wording effect on 12 MoVac-COVID19 items; (D) Four-factor structure with wording effect on 12 MoVac-COVID19 items. Abbreviations: MoVac, Motors of COVID-19 Vaccination Acceptance; Know, knowledge; Aut, autonomy; Word, wording effect.

The CFA fit indices were used to examine whether the four structure models fitted well to the data. In particular, comparative fit index (CFI), Tucker-Lewis index (TLI), root mean square error of approximation (RMSEA), and standardized root mean squared residual (SRMR) were taken into account in the model testing. According to Hu and Bentler,³³ both CFI and TLI should be larger than 0.95 together with both RMSEA and SRMR less than 0.08 to support the model fit. Moreover, a diagonally weighted least square estimator was used to fit the data with the four proposed models.

The four proposed models were then examined to understand if they were measurement invariant across different countries with the use of all five subsamples simultaneously. When a model is found to be measurement invariant, the measurement invariance evidence confirms that this specific factor structure of the MoVac-COVID19S was equivalent across the participants in different countries. In other words, individuals in different countries will interpret the MoVac-COVID19S items similarly and this provides additional psychometric evidence for construct validity of the MoVac-COVID19S. Each model proposed in Figure 1 was then tested using three nested models, including a configural model, a factor-loading-constrained model, and a factor-loading and item-intercept-constrained model. For the configural model, both factor loadings and item intercepts were freely estimated across participants in different countries. For the factor-loading-constrained model, factor loadings were constrained to be equal, and item intercepts were freely estimated across participants in different countries. For the factor-loading and item-intercept-constrained model, both factor loadings and item intercepts were constrained to be equal across participants in different countries.^34–36 The configural models for the four proposed structure models were examined using the CFI, RMSEA, and SRMR for their goodness of fit. Then, configural models were compared with the factor-loading-constrained models to examine whether factor loadings of the MoVac-COVID19S were invariant across participants in different countries. ΔCFI > −0.01, ΔRMSEA < 0.015, together with ΔSRMR < 0.03 support the invariant factor loadings. After this, factor-loading-constrained models were compared with the factor-loading and item-intercept-constrained model to examine whether factor loadings of the MoVac-COVID19S were invariant across participants in different countries. ΔCFI > −0.01, ΔRMSEA < 0.015, together with ΔSRMR < 0.01 support the invariant item intercepts.³⁷ All the CFAs were analyzed using LISREL (Scientific Software International, Lincolnwood, IL, USA) with all the other analyses performed using IBM SPSS 24.0 (IBM Corp., Armonk, NY).

Results

The present sample included Taiwanese university students (n = 932), mainland Chinese university students (n = 3145), Indian university students (n = 508), Ghanaian university students (n = 1244), and Afghan healthcare providers (n = 224). The mean ages of the subsamples were 25.51 years (SD = 6.42) for Taiwanese participants, 20.72 years (SD = 2.06) for mainland Chinese participants, 24.46 years (SD = 7.34) for Indian participants, 20.34 years (SD = 1.74) for Ghanaian participants, and 26.82 years (SD = 4.76) for Afghan participants. Regarding their university major (excluding the Afghanistan participants), just over one-third of the Taiwanese participants (37.6%), less than one-tenth of the mainland Chinese participants (7.7%), less than one-fifth of the Indian participants (14.9%), and just less than one tenth of the Ghanaian participants were majoring in a health-related program (9.7%). Table 1 presents detailed information regarding the participants’ socio-demographic characteristics and their MoVac-COVID19S item scores.

Table 1 Participants’ Characteristics and Item Score in the Motors of COVID-19 Vaccination Acceptance Scale (MoVac-COVID19S)

The fit of the data from each subsample with each of the four different model structures was first examined separately. The results showed that the fit indices of the CFA were relatively good across the five subsamples (ie, one-factor model using nine MoVac-COVID19S items, four-factor model using nine MoVac-COVID19S items, one-factor model using 12 MoVac-COVID19S items, and four-factor model using 12 MoVac-COVID19S items), except for the RMSEA. More specifically, for all the testing models across the five subsamples, CFI and TLI values were all higher than 0.95; SRMR values were all less than 0.08, except for the four-factor model using 12 items on the Ghanaian sample (0.091). RMSEA was high in two models for Taiwanese participants (0.097 in the one-factor model using nine items; 0.102 in the four-factor model using 12 items) and all the models for Afghan participants (RMSEA values = 0.092 to 0.111). Moreover, the four-factor structure (nine items or 12 items) had a better fit than the one-factor structure (Table 2).

Table 2 Confirmatory Factor Analysis Results of the Motors of COVID-19 Vaccination Acceptance Scale (MoVac-COVID19S) in Different Subsamples

The four factor structures were subsequently tested regarding whether their measurement invariance using all the five subsamples simultaneously was supported across the different subsamples (Table 3). The configural models fitted perfectly with the data (CFI = 0.985 to 0.997; RMSEA = 0.052 to 0.072; SRMR = 0.059 to 0.072) for all the factor structures. However, ΔSRMR indicated that the factor loadings were not invariant to both one-factor and four-factor structures when 12 MoVac-COVID19S items were used (0.105 for one-factor model and 0.103 for four-factor structure). Additionally, ΔSRMR indicated that the item intercepts were not invariant in both factor structures when the 12 MoVac-COVID19S items were used (0.027 for one-factor model and 0.073 for four-factor model); ΔCFI and ΔRMSEA did not support the item intercept invariant across the subsamples in either the one-factor model using nine items (ΔCFI = −0.015 and ΔRMSEA = 0.021) or the one-factor model using 12 items (ΔCFI = −0.017 and ΔRMSEA = 0.017). The only model that had measurement invariance support for both factor loadings and item intercepts across the five subsamples was the four-factor model using nine MoVac-COVID19S items (Table 3).

Table 3 Measurement Invariance Testing Across Subsamples in the Structure of the Motors of COVID-19 Vaccination Acceptance Scale (MoVac-COVID19S)

Discussion

Extending the findings of previous studies regarding the factor structure of the MoVac-COVID19S,^29,30 the results of the present study supported both one-factor and four-factor structures for the MoVac-COVID19S from two East Asian countries (ie, Taiwan and mainland China) to other countries (India in South Asia, Ghana in West Africa, and Afghanistan in Central Asia). The five countries of Taiwan, mainland China, India, Ghana, and Afghanistan were selected as participating countries in the present study because the (i) psychometric properties of the MoVac-COVID19S had been tested previously among Taiwanese, mainland Chinese, andGhanaian participants and could be replicated, and (ii) psychometric properties of the MoVac-COVID19S needed to be tested on other populations (ie, Indians and Afghans) which have a similar culture to Taiwanese and mainland Chinese. After confirming that the psychometric properties of the MoVac-COVID19S are satisfactory in the populations with similar cultures, the MoVac-COVID19S should be tested on more heterogeneous populations (eg, Europeans). Furthermore, consistent with what Chen et al²⁹ and Yeh et al³⁰ reported, the four-factor structure appeared to perform better than the one-factor structure. More specifically, the one-factor structure using the nine-item MoVac-COVID19S was measurement invariant across the five countries’ participants. Therefore, when future studies want to compare COVID-19 vaccine acceptance across these countries, the nine-item version of the MoVac-COVID19S is recommended.^29,30,43

To the best of the present authors’ knowledge, the present study is the first to examine the measurement invariance of MoVac-COVID19S across participants residing in five different countries. Therefore, although the low rate of COVID-19 vaccine acceptance remains,^38,39 researchers and healthcare authorities in the five studied countries can use the MoVac-COVID19S to assess their residents’ willingness and reasons for COVID-19 vaccine uptake. More specifically, the four factors in the MoVac-COVID19S (ie, values, impacts, knowledge, and autonomy) can help interested stakeholders to understand which factor is more important for an individual to decide when getting COVID-19 vaccinated. Subsequent actions and policies can then be implemented to deal with that specific factor. For example, if the autonomy factor was found to be low in a population, the healthcare authorities may consider increasing the vaccine dosages and improving vaccine uptake procedures to elevate autonomy among the population.

Given the dearth of psychometric evidence concerning the MoVac-COVID19S, the present study’s findings can be compared with the psychometric evidence concerning the MoVac-Flu Scale³¹ given that the only difference between MoVac-Flu and MoVac-COVID19S is that the word “flu” was changed to “COVID-19”. Therefore, the structure should arguably be the same on the CME model. The MoVac-Flu Scale was found to be a single trait-factor, although it was developed using the CME, a theoretical model containing four factors.³² The use of different factor analyses (ie, exploratory factor analysis [EFA] and CFA) is a possible reason that explains the different findings.³⁰ More specifically, the MoVac-Flu Scale has only been tested using EFA and not using CFA.³¹ Because EFA is subjective and exploratory in nature, it cannot use a priori hypothesized model to test the factor structure.⁴⁰ In contrast, CFA that was carried out in the present study and other MoVac-COVID19S psychometric testing studies compared different factor structures (ie, a one-factor structure and a four-factor structure) according to hypothesized models supported by prior evidence or theories.^29,30 Therefore, EFA and CFA may provide different findings for any existing instruments, such as the MoVac-COVID19S. Moreover, EFA cannot take into account the wording effect when identifying a potential factor structure, while CFA can. However, the CFA results indicated that the three negatively worded items were better not included in the MoVac-COVID19S to yield a better fit.

There are some limitations in the present study. First, all the participants in all the countries were recruited using a convenience sampling method. This type of sampling usually has low ability in generalizing the findings to entire populations. Moreover, the present sample across five countries were mostly university students or junior healthcare workers. Therefore, caution is advised when generalizing the present psychometric evidence to other populations and cohorts (eg, older people). Moreover, psychometric evidence of the MoVac-COVID19S among other populations is needed to increase its applicability. Second, although the present study tested the construct validity of the MoVac-COVID19S, only CFA was carried out to examine the factor structure of the MoVac-COVID19S. Other methods for assessing construct validity (eg, using external criterion instruments) were not investigated in the present study. Therefore, additional psychometric evidence for the MoVac-COVID19S is needed utilizing other validated external criterion instruments. Third, the five countries used different methods to collect the data (ie, online survey and paper-based survey). Therefore, this practice is subject to the possibility of increasing measurement biases. For example, prior research showed the differences in response rate and cost between online and paper-based surveys.⁴¹ Moreover, some researchers claim that the paper-based survey as compared to online-mode survey could enable participants to report higher health concerns and negative attitudes.⁴² Fourth, the MoVac-COVID19S is a self-report scale and therefore other confounding factors such as social desirability bias cannot be controlled in the present study.⁴³ Finally, the sample sizes were different across the five countries (eg, the Afghan subsample had a much smaller sample size). Therefore, the findings in the present study may be less representative among the Afghan subsample. Moreover, the Afghan participants were healthcare providers, which was a different cohort to the other countries’ participants (who were all university students). Therefore, the small sample size and the different cohort population in the Afghan subsample might be factors that explain the slightly worse fit indices in RMSEA when compared with the other subsamples. However, the small sample size might not seriously impact the present study’s findings because prior psychometric simulation evidence and expert opinion indicate that a sample size of 100 can achieve accurate solutions when utilizing CFA.^44–47

Conclusion

The present study extended the current empirical evidence concerning the MoVac-COVID19S from East Asian populations to other countries (ie, India, Ghana, and Afghanistan). The psychometric evidence reported in the present study supported the four CME factors across the five countries. Nevertheless, the one-factor model of the MoVac-COVID19S is also acceptable in some countries. Therefore, the present findings are helpful for healthcare providers to assess the willingness of individuals to get a COVID-19 vaccination. With the severity of COVID-19 pandemic still at a high level globally, different authorities and governments can use the MoVac-COVID19S to understand the underlying reasons for COVID-19 vaccine hesitancy. Such information can be used to implement appropriate policies to achieve national herd immunity.