1. Introduction
The global COVID pandemic has affected more than half a billion people and has so far led to around 6,332,618 deaths, driving economic, political and social changes. The protective measures against SARS-CoV-2 virus infection are currently both general and recommended in all epidemics with respiratory-transmitted pathogens (social distancing, wearing protective masks), but also specific such as testing and vaccination. The role of these measures is to reduce the number of severe forms of disease and death but also to prevent the overcapacity of hospitalization and intensive care at the national level [
1].
In a time of measures’ relaxation (the removal of the mandatory wearing of the mask in public places, the changes to the rules for testing and isolating new cases) after a period of decline in the incidence of COVID-19 cases throughout March, April and May 2022, our paper provides a warning signal based on latest World Health Organization (WHO) briefing (6 July 2022) that urges countries to readopt tried and tested public health measures and plug immunity gaps as infections new sub-lineages have risen 30% in the past two weeks (as of 6 July 2022) and increased in four out of six WHO regions [
2]. This warning comes at a time when many countries have still not reached WHO’s vaccination targets (Bulgaria 29.9%, Slovakia 50.8%, Croatia 55.6%, Slovenia 57.1%, Poland 59.6%) [
3] or gained access to antivirals. This is also the case for Romania, with a cumulative uptake of full vaccination in the total population of 42.3% as of 14 July 2022 [
3]. Moreover, even for those vaccinated, vaccine efficiency declined with the appearance of new mutations in the structure of spike protein, thus amplifying the existent hesitancy in adherence to preventive measures [
4]. Breakthrough infections in vaccinated populations with the recently emerged SARS-CoV-2 Omicron variant in the fifth wave of the pandemic exposed some of these limitations of COVID-19 vaccines and highlighted the need for other medical treatments such as drug therapy [
5]. However, antiviral drugs prove to reduce the risk for severe outcomes and death in non-hospitalized patients or when given in the first 5 days after COVID diagnosis [
6,
7,
8], but fewer data are on efficiency in hospitalized patients with severe disease [
8]. These findings reinforce the need not to underestimate general preventive measures and the role of massive testing.
Worldwide, countries with high vaccination rates validated COVID-19 knowledge and behavior questionnaires (France [
9], Spain [
10], Italy [
11], Hungary [
12], Brazil [
13], Korea [
14], India [
15], etc.). Timely insights from behavioral and attitudinal data proved to be crucial for the decision-making process in the COVID-19 international public health policies [
16,
17].
Currently, there is no validated Romanian questionnaire to assess people’s attitudes toward anti-COVID vaccination and general preventive measures. Such a tool is vital for improving people’s attitudes and compliance and reaching vaccination targets.
In this context, we considered the opportune warning from WHO, and we aligned ourselves to this call by proposing the first validated Romanian questionnaire to assess people’s level of education and attitudes towards general measures to protect against COVID-19 infection. Another objective of our study was to characterize the level of adherence and the main motivations behind acceptance or refusal to follow the imposed preventive measures. The ultimate goal was to use the proposed questionnaire within national programs to identify populational areas with low adherence and establish specific educational programs to increase the willingness to vaccinate against COVID-19 in an effort to prevent a full-blown sixth wave.
3. Results
Our sample included 194 people, of which 137 (70.6%) were women, with a mean age of 41.5 years old and a standard deviation (SD) of 12.4. Age varied between 18 and 89 years old. Of all respondents, 58.2% inhabited urban areas. Regarding the geographical area, the participants came from the north-eastern part of Romania, and almost half of them were from Iasi county: Iasi 110, Bacau 14, Botosani 24, Galati 13, Neamt 12, Suceava 14, Vaslui 7. A significant proportion of the study sample declared to work in healthcare (27.53%) (
Table 1).
The first version of the questionnaire reviewed by experts consisted of 40 items, to which the participants were instructed to respond according to their conviction (
Table 2). The score for the items M2, M5, M6, M7, M8, T2, T4, V1, V2, V4, V5, V6, V7, V8, S1, S3, S4, S5, S6, S7, S8, CV2, CV3 and CV4 was reversed.
The mean score of the questionnaire was 42.8 (95% IC: 41–44.6; SD = 12.6). Significant differences in the score were observed between subgroups regarding the area of residence, the presence of diabetes, the awareness of the potential severity of SARS-CoV2 disease and the profession (
Table 3). Vaccinated people had significantly lower scores.
According to the psychometric criteria, items SD1, M2, M3, M4, M6, V1, V2, V3, V4, V6, V10, S3, S4, S6 and S7 did not meet the recommendation that at least 5% of people should respond in the same way and were eliminated. Items M2, M3, M4, M6, V2, V3, V6, S3, S4, S6 and S7 were withdrawn due to the DI that was lower than 20%. Item V3 was removed due to the lack of ability to discriminate between people with different levels of knowledge.
The exploratory factorial analysis for the identification of latent components identified three factors that together explained 52.7% of the total variation and were, therefore, chosen as the components of our questionnaire (
Table 4,
Figure 1). T The value of the Kaiser–Meyer–Olkin coefficient (0.791) and the result of Bartlett’s test (
χ2 = 1571.865,
p < 0.001) indicated the adequacy of the sample size chosen for our analysis. The names of the factors are given according to the variables that they encompass. Factor 1 was named “Compliance with protective measures”, factor 2 was “Attitudes toward vaccination” and factor 3 was “Attitudes regarding potential COVID-19 therapies”. For factor 1, there are high correlations between items exploring social distancing, testing and the protective role of mask-wearing. For factor 2, the items selected were addressing potential harmful side effects or inefficiency of the vaccines and attitudes toward restrictions associated with the green certificate. For factor 3, the questions investigated behaviors toward potential therapies for SARS-CoV-2 infection, uncertainty toward their scientific validation and distrust in the scientific plausibility of protective measures.
Principal component analysis (PCA) simplified the complexity of high-dimensional data while retaining trends and patterns. After performing PCA and excluding the variables with loadings lower than 0.5, the final model contains 16 items (
Table 5 and
Table 6) and confirms the factorial structure of the proposed questionnaire with the three dimensions identified by the exploratory factorial analysis.
The value of the alpha Cronbach coefficient shows a good internal fidelity of the questionnaire (0.853). The Guttman Split-Half coefficient shows the probability of test-retest reproducibility. By applying the Split-Half method to show the probability of test-retest reproducibility, a value of the Guttman Split-Half coefficient of 0.859 was obtained, which indicates that the fidelity of the scale is acceptable (
Table 7).
As another argument for the survey’s ability to screen people’s attitudes toward vaccination, we identified that the test’s final score predicts the presence of vaccination for a cut-off value of 50% with an accuracy of 0.773 and an AUC of 0.793 (
Figure 2).
Table 8 illustrates the predictive power of various variables used in the factorial analysis to estimate the presence of vaccination. In total, the questionnaire score, the diagnosis of diabetes, the advice provided by healthcare workers and the medical profession proved to be significant predictors of vaccination.
4. Discussion
Our paper represents the first validation study of a Romanian questionnaire aimed at screening attitudes toward vaccination and preventive measures against SARS-CoV-2 infection. Our article contributes to the literature with a scientific method to validate a questionnaire in a non-English language. As such, our method can serve as a model to be used by researchers in other countries to develop and validate surveys in their language. Moreover, our article characterizes the level of adherence and the main motivations behind acceptance or refusal to follow the COVID-19 preventive measures of a Romanian population.
Even though we are now in a period of relaxation of the public health measures due to an apparent withdrawal of the epidemic in the first half of the year, recent publications warn about several countries entering the sixth wave of the epidemic [
24,
25,
26,
27]. According to the WHO’s latest briefing (6 July 2022), preventive measures and vaccination programs should again be implemented [
2].
Several reviews dealing with the worldwide attitudes and hesitancy towards COVID-19 vaccination were published [
28,
29,
30]. In countries with high levels of vaccination, knowledge and level of education proved to have positive associations with attitude and adherence to precautionary measures [
23]. On the contrary, in Eastern Europe, a lower overall proportion of vaccine acceptance has been reported [
30]. Increasing the vaccination rate is crucial in combating the COVID-19 pandemic, but it requires the prior identification of the underlying causes/specific determinants of hesitancy towards vaccines specific to this region.
Similar to other middle-income countries, Romania faces several barriers and difficulties in re-establishing control measures against the COVID-19 epidemic. The means of controlling the SARS-CoV-2 virus infection spreading were accompanied by economic costs such as unemployment, corporate bankruptcies and a disproportionate impact on less-skilled and less-educated workers. Social costs included increased domestic violence and damaged educational systems [
31]. Furthermore, the media attention and uncertainty about how to protect against infection generate contradictory attitudes and behaviors among the population, starting from accepting protective measures and vaccination to denying their effects and even the disease. Along with the conspiracy theories, the imposition of compulsory vaccination has increased social tensions in many countries, with vaccination being associated with the threat to individual human freedoms, such as the right to free choice of health or the right to work. This perception of the threat to freedom has provided an opportunity for various political factions to condemn vaccination, to question the effectiveness of current vaccines amid the tension generated by general protection measures seen as restrictive in order to gain electoral capital. These perspectives emphasize that while COVID-19 vaccines are becoming increasingly available, they are still met with reluctance, and thus, safety measures (e.g., face masks, personal hygiene and social distancing) are still of key importance in protecting personal and public health against COVID-19 [
32].
In order to meet the WHO’s recommendations and overcome the particular Romanian patterns of public health measures adherence and acceptance, we developed and validated a questionnaire that addresses knowledge, attitudes and practices focusing on preventive measures against COVID-19. It is divided into six categories related to social distancing, mask-wearing, testing, vaccination, use of supplements and the benefit of a green certificate.
The analysis of the completed questionnaire database offers a global picture of the adherence motivations to general preventive measures against COVID-19 in Romania. By knowing the actual reasons behind not reaching the country’s vaccination target, specific measures can be taken to increase people’s confidence and compliance. For instance, cardiovascular patients within our 194-sized sample did not have higher vaccination rates, although cardiovascular diseases are known risk factors for developing severe forms of COVID-19 [
33,
34,
35,
36]. However, the presence of diabetes was significantly associated with a better score in our questionnaire and with a higher proportion of vaccinated people. Thus, specific measures directed toward cardiovascular patients to raise awareness through educational programs could be effective in levering up vaccination rates. Other main determinants of vaccine acceptance and preventive measures compliance, as identified by our study, were related to vaccines’ safety and efficiency, the people’s trust in the government and medical system, and health literacy. These results show us that there is room for progress in COVID-19 health literacy through intensified education. Gaining people’s trust in the government and healthcare system through communication and education should also be a priority. Our results also suggest a need for targeted community awareness interventions for the most vulnerable populations, those with no school education, the elderly and people living in rural areas.
Moreover, we conducted a multivariate analysis that identified several predictors of vaccination adherence: questionnaire’s final score, medical profession, medical advice versus mass media consultation, the presence of diabetes mellitus and the fear of the disease.
In addition to proposing a Romanian survey regarding COVID-19 attitudes, we also proceeded to the scientific validation of the questionnaire in order to obtain pertinent information in a reliable and valid way [
21] and ensure that the questionnaire “
measures what is intended to be measured” [
37]. The application of improper measurement tools that are not validated can lead to inaccurate and misleading findings, resulting in a poor plan for interventions and, therefore, too unreliable efficacy.
Validity is assessed through two categories of tests, which evaluate theoretical construct and empirical construct. The theoretical construct is tested through face validity. The purpose of face validity is to ascertain that the items of the questionnaire fully represent the domain that is intended to be judged [
37]. We used literature reviews, critical incidents, direct observations and expert judgment approaches to construct a questionnaire of 40 items with acceptable face validity. The literature review did not identify, to date, any similar validated questionnaire in the Romanian language [
38]. Face validity is an important aspect related to the empirical construct of the questionnaire [
20,
21]. The next validation step was represented by the psychometric evaluation, which prompted the elimination of 15 questionnaire items due to not meeting psychometric criteria. The remaining 25 items were included in the exploratory factor analysis. The factor analysis revealed a three factors structure with 20 items. The resulting model explained 52.7% of the variance in our study. Furthermore, PCA was applied, resulting in a final model of 16 items.
In order to strengthen the evidence that the questionnaire “measures what is intended to be measured”, we proved that the test is able to discriminate between vaccinated and unvaccinated people.
Our results are similar to other resembling COVID-19 questionnaire validations in other languages. A Korean study was based on similar sample size (229) and obtained an alpha Cronbach coefficient of 0.75 for validating a survey that assesses COVID-19 knowledge, attitudes and practices among nursing students [
39]. Likewise, the validation of an Indian questionnaire to assess knowledge, attitude, practices and concerns regarding COVID-19 vaccination among the general population was based on 201 participants and resulted in an alpha Cronbach coefficient of 0.86. With the exception of a french questionnaire aimed to assess COVID-19 knowledge and behaviors that was validated on a big cohort of 8045 participants [
9], our study fits the current international scientific landscape for the population sample as well as validity results.
Limitations
Firstly, the number of survey respondents was small. This low interest may be explained by the fact that the incidence of COVID-19 cases declined during the spring of 2022, preventive measures were relaxed, and the mass media and people’s attention was focused on the war in Ukraine, with which Romania shares a significant long border. However, this number is still appropriate for factor analysis and testing of the questionnaire. Secondly, our sample might not be representative of the general population in Romania as the proportion of vaccinated people was 74,8%, much higher than that reported by the authorities of 42.3% [
3] (as of 14 July 2022). Moreover, a significant proportion were healthcare workers, thus explaining the high rate of vaccination. The percentage of women (70.6%) in our sample is also higher compared to the general population. In our future validation studies, we aim to overcome these limitations by including a larger population with similar variables’ distributions as the general population.