Next Article in Journal
Unfixed Movement Route Model, Non-Overcrowding and Social Distancing Reduce the Spread of COVID-19 in Sporting Facilities
Previous Article in Journal
Romanian Students’ Environment-Related Routines during COVID-19 Home Confinement: Water, Plastic, and Paper Consumption
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Teaching in Times of the COVID-19 Pandemic: A Pilot Study on Teachers’ Self-Esteem and Self-Efficacy in an Italian Sample

Department of Pedagogy, Psychology, Philosophy, Faculty of Humanistic Studies, University of Cagliari, 09123 Cagliari, Italy
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2021, 18(15), 8211; https://doi.org/10.3390/ijerph18158211
Submission received: 8 July 2021 / Revised: 27 July 2021 / Accepted: 30 July 2021 / Published: 3 August 2021

Abstract

:
The aim of this research was to investigate the impact of the COVID-19 pandemic on teachers, particularly on their self-esteem and self-efficacy, their difficulty in the transition to distance learning, the difficulty of students, and specially of students with learning disabilities (LDs students), as perceived by teachers. 226 teachers were invited to complete an online questionnaire. Our results showed lower self-esteem and lower self-efficacy by the teachers compared with the normative sample. Self-esteem and self-efficacy also decrease in teachers with greater service seniority at work. Teachers perceived a greater difficulty in students than in their own difficulty. The concentration of the school system’s efforts on the massive and, for long periods, exclusive organisation of distance learning risks favouring only cognitive aspects to the detriment of affective dynamics. This aspect could make teaching more complex for teachers and learning poorer for students, impoverishing the complex relational process that forms the basis of the learning process.

1. Introduction

A body of literature showed the fundamental role played by the socioemotional aspects in the school context [1]. When the school is also interested in promoting student’s socioemotional learning, apart from traditional outcomes, many educational benefits, including students’ academic achievements, affects, behaviours, and motivation, are reaped [2,3,4].
A lot of research on socioemotional issue shows the important role played by a positive quality of teacher–student relationships in influencing learning both directly and indirectly [5,6,7,8,9,10].
Some studies have contributed to the understanding of student–teacher dynamics and how this relationship in the school context results in student learning. Several interpersonal variables were identified from the literature as positively related to learning: immediacy [11,12] communicator style [13,14] affinity-seeking [15], self-disclosure [16], humour [17], job satisfaction [18], enthusiasm [19], self-esteem [20], self-efficacy [21], etc.
The importance of relationships, and in particular those in school settings, is a theme that has begun to appear in the forefront since the past year, as policy-makers and educators realise the importance of human socioemotional aspects in the relationship between teachers and students amidst the sudden shift from face-to-face to online schooling during the ongoing COVID-19 pandemic.
Teachers suddenly have to deliver their lessons using technological tools, including through specific online platforms, in order to reach out to the students. Students have been deprived of social face-to-face interaction among peers, and teachers and parents began to be more involved because of the need for monitoring school lessons at home.
Although some teachers were ready to face the situation, a large majority had to adapt their teaching in a short time without training, with insufficient capacity, and little preparation. This unexpected and rapid transition to online learning has led to a multiplication of teachers’ strategies for distance learning in lectures, tutorials, project groups, lab works, and assessments [22].
Several studies showed that the teaching profession has experienced many periods of crisis and discontent throughout history. The profession has been strongly guided by high rates of professional dissatisfaction, stress linked to the vastness of the bureaucracy, excessive workload imposed on teachers, incidents of indiscipline, precariousness and professional instability, teachers’ lack of motivation and/or interest, and the growing use of technology in teaching [23,24,25].
The current pandemic has given rise to a crisis that particularly involves the last aspect, which is the impact of digital technology on teachers’ school practice. COVID-19 has once again highlighted the problematic relationship between teaching and new technologies, both in the international than in the national context [26,27].
For example, a study [28] on the professional well-being of Portuguese teachers showed that the pandemic has led to a shift in teachers’ perception from fair positivity to concern about their professional future. Additionally, in the Italian context, this aspect of increasing technology use is emphasised on its influence on job satisfaction [26,27].
Job satisfaction involves psychological, physiological, and environmental conditions and factors that, together, guarantee positive feelings towards work [29], which, in turn, increase the rate of productivity and sense of well-being. Among the variables found in the literature, self-esteem and self-efficacy were found to play an important role in job satisfaction and in the ability to meet or address changes. The influential variables on teachers’ job satisfaction affect teacher–student and teacher–parent communication as well as the aspect of collaboration.
Rosenberg [30] describes self-esteem as individuals’ positive and negative self-perception. Self-esteem is an individual’s consideration of his/her own self as competent and important, as well as perceiving oneself as successful and valuable [31]. Several studies showed that employees with high-level of self-esteem had high-level of job satisfaction [32,33]. Mbuva [20] stated that teachers’ self-esteem is important for their success in teaching and that teachers’ positive and high esteem, in turn, positively affects students’ self-esteem and learning processes.
Another variable that was found to positively influence job satisfaction, was self-efficacy [34]. Self-efficacy is a person’s conviction in their ability to succeed in a particular situation [35]. Perceptions of teacher self-efficacy correspond to judgements about the teacher’s personal ability to achieve the desired results in terms of student engagement and learning. The concept of self-efficacy derives from Bandura’s social-cognitive theory of behavioural change, It refers to a teacher’s belief in his/her ability to successfully cope with tasks, obligations and challenges related to his/her professional role [36]. Teacher self-efficacy is also defined as teacher’s perception in motivating students to learn [37]. Another definition describes the teachers’ belief that they can influence their students’ learning process. The literature has shown that teachers with high levels of self-efficacy experience higher levels of job satisfaction and lower levels of job-related stress [38]. Teacher self-efficacy was positively related to class-average achievement and interaction quality [21].
Several studies on distance education showed that despite teachers’ limited experience in this type of education in terms of technical skills, time management, knowledge and attitude in online education, teachers can cope with the trends in distance learning [39]. Some studies highlighted the relationship difficulties frequently experienced by regular students (lack of interaction and feedback, difficulty to start and maintain communication, ambiguity of posted messages, technical problems disrupting conversations, etc.) [40,41,42] and by students with learning disabilities (LDs) (high levels of social distress and loneliness) [43,44,45].
Distance learning is a psychological process supported by e-technology, but learning is a social activity. The use of distance learning can improve the learning of students and students with LDs only where a supportive context is present [45].
The success of distance learning also depends on education management and the quality of the students’ home learning environment. Because this pandemic is still on-going, it is essential to determine how the teachers, who are the main facilitators of education, are adjusting to this transition.
At this stage, it is interesting to investigate teachers’ perceptions in relation to this sudden transition from face-to-face teaching to distance teaching; in particular, we are interested in investigating their perceptions of difficulties (both for themselves and for their students) and their self-assessments in terms of self-esteem and self-efficacy.

2. Materials and Methods

The aim of this study is to investigate the impact of the COVID-19 pandemic on teachers, particularly regarding their perspectives on the following:
  • The effect of changes because of pandemic-related changes on self-esteem and self-efficacy through comparison with the normative samples;
  • Their level of difficulty in the transition to distance learning;
  • The level of difficulty of students, particularly students with LDs, as perceived by teachers in the transition to distance learning.
We also aimed to investigate the effect of the variables “school type” and “level of service” (low <14 years vs. high >14 years) on the perception of difficulties and on self-esteem and self-efficacy.

2.1. Participants

Our participants were recruited through non-probability sampling across 28 schools in some areas of Sardinia (Italy), which produced a total of 226 respondents (Figure 1 depicts phases of recruiting): 199 females (88.1%) and 27 (11.9%) male teachers. Of the total, 23 (10.2%) participants were aged <35 years, 93 (41.2%) were between 36–45 years, 66 (29.2%) were between 46–55 years and 44 (19.5%) were age >56 years. Twenty-eight participants (12.4%) taught in kindergarten school, 80 (35.4%) participants in primary school, 49 (21.7%) in junior high school, and 69 (30.5%) in high school. A total of 143 (63.3%) participants had professional stability, while 82 (36.3%) participants had professional instability (one teacher did not give this data). Only 44.7% of teachers have received professional training on learning disability. The descriptive statistics were showed in Table 1.
The study was approved by the Ethics Committee of the University of Cagliari, Italy (Prot. n. 0073815).

2.2. Instruments

All teachers were invited via e-mail to complete an online questionnaire using Google Forms as a platform. A work session lasting about 25 minutes was carried out, organised in different sections.
The first section of the questionnaire assessed the demographic variables (age, gender, level of education, years of seniority at work, type of contract, experience in LDs field, etc.).
In order to identify specific situations and perceptions during this period of the COVID-19 pandemic, the second section included a number of relevant questions: (Q15) “During the COVID-19 emergency, did your school set up online teaching activities? (Q16) If yes, was an official platform used? (Q17) If yes, which one? Gsuite, Weschool, Adobe connect, Microsoft Teams, No platform, only e-mail, Other”; (Q18) “What is the level of difficulty you encountered in using the platform?”, (Q19) “What is the level of difficulty encountered by your students in using the platform?”, (Q20) “What is the level of difficulty encountered by your students (with LDs) in using the platform?”, (Q21) “What is the level of difficulty encountered by your students (with SEN) in using the platform?”, (Q22) “What is the level of difficulty encountered by your students (with H) in using the platform?”, (Q23) “How was the level of cooperation from parents?”, (Q24) “How was the level of student’s participation?”.
The second section of the questionnaire assessed the following variables:

2.2.1. Self-Esteem

The Rosenberg Self-Esteem Scale is a self-report questionnaire, introduced in 1965 by Rosenberg. It comprises five positive and five negative worded items (for the analyses, the five negative items were reversed). In this study, we use the Italian version by Prezza, Trombaccia and Armento [46]. The rating scale was a four-point Likert-type scale, ranging from 1 (strongly disagree) to 4 (strongly agree). Its internal consistency, analysed by the Cronbach coefficient, was 0.84.

2.2.2. Self-Efficacy

The Teacher Self-Efficacy Scale [37] was administered in its Italian adaptation by Biasi and Dominici [47]. The scale comprises 24 items organised in three factors: «Efficacy for Student Engagement» (8 items), «Efficacy for Instructional Strategies» (8 items), and «Efficacy for Classroom Management» (8 items). Regarding the scales assessed, the “teachers’ perception of self-efficacy” relates to their ability to gain students’ commitment, the ability to choose appropriate teaching strategies, and their ability to manage the classroom.
The “Efficacy for Student Engagement” sub-scale measures teachers’ sense of efficacy in motivating the students. A teacher who is confident of being able to motivate the student, would need to be involved and committed in the study in order to affect the results.
The “Efficacy for Instructional Strategies” sub-scale identifies the teacher perceptions in using appropriate teaching strategies.
The “Efficacy for Classroom Management” sub-scale evaluate the teacher perceptions in managing the class in a functional way.
Cronbach’s alpha in the Italian sample was good (0.97).

2.3. Data Analysis

With the aim of eliciting answers to research questions, an exploratory and descriptive approach was applied [48]. The descriptive analyses were applied in relation to all the investigated variables. Thereafter the non-parametric and parametric inferential analyses were conducted.
One sample t test was carried out to compare the mean scores of the participants in relation to the normative samples. The bivariate correlations (Pearson’s r coefficient) evaluated the relationships between continuous variables. The multivariate analysis of variance (MANOVA) was applied to assess the differences in relation to the level of seniority (low and high seniority—evaluated by the median split) and the type of school (pre-school, primary school, lower secondary school, upper secondary school), regarding the dimensions of self-efficacy and self-esteem. The non-parametric Friedman test and Mann–Whitney test were carried out to explore the differences in ordinal variables regarding the level of seniority and type of school.
The two-step cluster analysis was applied in order to identify the participants’ groups regarding dimensions. This statistical technique applied two phases of clustering [49]: the first pointing to grouping cases into pre-clusters, and the second to allocating all cases in clusters. The first phase aims to reduce the size of the matrix that contains distances between all possible pairs of cases. This clustering, conducted on continuous variables, makes use of the Euclidian algorithm. For this study, the individuals were the objects to be clustered, while the variables characterised attributes on which clustering was grounded. For the second step, the pre-clusters were grouped using the hierarchical clustering algorithm. Proposing a range of clustering solutions, this step also involved condensing to the best number of clusters based on the Schwarz’s Bayesian information criterion (BIC). After the identification of the cluster solution, t tests for continuous variables were carried out to examine the importance of each variable in a cluster [50,51].
The data analyses were carried out using the opensource software Jamovi (release—1.6.15; jamovi project 2021, https://www.jamovi.org (accessed on 5 June 2021)) [51] and SPSS (release 24—IBM Corporation, Armonk, NY, USA) [52].

3. Results

3.1. Comparisons with Normative Samples for Self-Esteem and Self-Efficacy Scores

In order to compare the mean scores of the participants to the scores obtained by the normative samples in the validated instruments that were applied, the one sample t test was carried out (Table 2). The data highlighted noteworthy differences regarding the aspect of self-esteem, with the participants showing significantly lower scores compared to the normative sample (t = −32.700; df = 225; p < 0.001; Cohen’s d = −2.170). A significant difference was found regarding the dimension of self-efficacy for instructional strategies, consistently revealing significantly lower scores for the teachers taking part in our research (t = −2.020; df = 225; p = 0.044; Cohen’s d = −0.134) compared with the normative sample. For the dimensions assessing self-efficacy for classroom management and self-efficacy for student engagement, no significant differences were highlighted between normative sample and participants.

3.2. Bivariate Correlations and MANOVA Regarding Type of School and Level of Seniority for Self-Esteem and Self-Efficacy Scores

In order to evaluate the linear relations between the inquired dimensions, the Pearson’s r coefficient was applied (Table 3). The data were found to emphasise the significant positive correlations between self-esteem and self-efficacy and the negative correlations between years of seniority and self-efficacy.
A MANOVA with two factors (2 × 4) was applied to evaluate the differences in relation to the level of seniority (low and high seniority, appraised by the median split. Low: less than 14 years’ seniority; high: over 15 years’ seniority) and the type of school (pre-school, primary school, lower secondary school and upper secondary school), regarding the dimensions of self-efficacy and self-esteem. The use of Wilk’s Lambda multivariate test highlighted a significant principal effect for the level of seniority (Wilks’ Lambda = 0.948; F = 2.942; df = 4; 214; p = 0.021). There were no other significant effects, found in relation to the principal effect of the type of school (Wilks’ Lambda = 0.913; F = 1.647; df = 12; 566; p = 0.075) and to the interaction between the level of seniority and type of school (Wilks’ Lambda= 0.972; F = 0.506; df = 12; 566; p = 0.911). Significant effects regarding the level of seniority were observed regarding all assessed dimensions of self-esteem and self-efficacy in teaching (see Table 4 and Table 5). Table 5 reports the descriptive values (mean, standard deviation) relating to the significant effect of the level of seniority factor, highlighting that teachers with lower level of seniority showed higher scores in all scales (self-esteem and self-efficacy).

3.3. Non-Parametric Analyses in Order to Evaluate Differences Regarding the Ordinal Variables

A specific section of the questionnaire assessed the level of difficulty perceived by teachers in using digital platforms.
In order to evaluate if the level of difficulties perceived by the teacher was different in relation to their perceptions of different actors in school (oneself, students, LDs students—Questions Q18, Q19, Q20) Friedman’s non-parametric test for repeated measures at ordinal level was applied (Table 6). In this evaluation, we excluded teachers working in pre-school, as students under the age of 6 tended not to use the platforms for structured learning activities. Subjecting the data to the Freedman test revealed a significant effect (Chi Squared = 125.00; df = 2; p < 0.001). Pairwise comparisons with the Durbin–Conover Test emphasised significant differences between all couples of questions. Teachers’ perceptions of their personal difficulties were significantly lower than their perceptions of students’ difficulties, and specifically of LD students.
The Mann–Whitney test for repeated measures was applied for the questions assessing the level of participation of student and parents (Q23, Q24). This assessment comparison emphasised a higher level of participation of the students than their parents (Wilcoxon = 1294, p < 0.001).
Thereafter, we evaluated the level of difficulties perceived in relation to the level of seniority (low, high). The Mann–Whitney test highlighted a significant difference regarding the level of difficulty experienced by teachers (Mann–Whitney U = 5010; p = 0.008; Effect size = 0.194) that was higher for persons with higher seniority.
We compared the difficulty perceived by the teachers in relation to the type of school. The Kruskal–Wallis test highlighted significant differences only regarding the level of collaboration of parents (KW = 36.019; df = 2; p < 0.00; Effect size = 0.209), i.e., that significant differences were found to be higher in the primary school than in higher types of schools.

3.4. Two Steps Cluster Analysis

Finally, two-step cluster analysis was carried out to identify similar groups or “clusters” of people within this study’s data sets [53,54]. This technique was applied on the continuous scale scores in the psychological dimensions of self-esteem and self-efficacy achieved by the teachers. A solution characterised by three clusters was identified (Schwarz Bayesian Criterion (BIC) = 1330.654; modification BIC = −189.941; ratio of modifications BIC = 0.710; ratio of distance measures = 1.736). This was also confirmed following a random subdivision of the total sample into two subgroups. Three clusters of teachers were identified: the first comprised of 47 participants (21.5%), the second by 104 participants (47.5%), and the third by 68 (31.1%) (Table 7).
As seen in Table 7, cluster 1 is characterised by teachers with a seniority of service >14 years, having a predominantly open-ended contract (68.1%) and lower levels of self-esteem and self-efficacy than in the average of the sample participants. Cluster 2 is also characterised by teachers with a seniority of service >14 years, with an indefinite duration but with levels of self-esteem and self-efficacy higher than in the average of the sample and lower than the average of cluster 3, which is characterised instead by teachers with a seniority of service <14 years and with a type of contract that was mainly fixed-term. These data suggest that a relevant difference between Clusters 1 and 2 could be recognised in the training of teachers belonging to Cluster 2, in terms of the use of technology prior to the ongoing pandemic; this aspect might have a relevant role supporting a more comfortable transition to distance learning. Furthermore, this aspect might be supported by the features of Cluster 3: indeed, the fewer difficulties encountered by younger teachers might could be attributable to their better knowledge of technology, regardless of their training.

4. Discussion

The aim of this pilot study was to investigate the impact of the COVID-19 pandemic on teachers. In particular, we investigated teachers’ perspectives on the following: the impact of pandemic-related changes on self-esteem and self-efficacy through comparison with the normative sample; their level of difficulty in the transition to distance learning; and the level of difficulty of students, and specially of students with LDs, as perceived by teachers. We also investigated the effect of the variables ‘school type’ and ‘level of service’ (low <14 years vs. high >14 years) on the perception of difficulties and on self-esteem and self-efficacy.
Our results allow some important reflections on the pressures to which schools were subjected to during this period of the COVID-19 dilemma. The observation point we have used in this first phase of the study is from the perspective of the teachers. Political decisions, aimed at managing the pandemic, forced Italian schools to reorganise their teaching homogeneously without considering the differences in the national territory. The required transition from face-to-face teaching to distance teaching in a sudden and discontinuous way has brought out the gap on the national territory both on teachers’ training and on the concrete possibility for families to support this transition to online platforms [55].
In consideration of our aim, the data highlighted differences regarding the aspects of self-esteem and self-efficacy for instructional strategies wherein we found that participants significantly scored lower than the normative sample. As for the dimensions assessing self-efficacy for classroom management and self-efficacy for student engagement, no significant differences were found between the normative samples and the participants. The use of the Wilk’s Lambda multivariate test highlighted that the teachers’ perceptions of their personal difficulties were significantly lower compared to their perceptions of students’ difficulties, particularly LDs students, while the Mann–Whitney test highlighted a significant difference regarding the level of difficulty experienced by teachers having greater seniority.
A first consideration underscores a stronger focus on self-esteem and self-efficacy for instructional strategies vs. what we may call the ‘relational/interactional’ dimensions of efficacy for classroom management and self-efficacy for student engagement. This may indicate that teachers perceive their personal difficulties more as belonging to ‘what they are’ than to what they do, or may do. This may be explained by the teachers’ lack of a clear range of strategies for interventions and management, which in turn may be explained by this never-before experienced situation of long-term online distance teaching. It seems that the teachers, lacking a clear representation of possible alternative teaching and relational strategies, failed to assess the potential solutions to this challenging teaching environment [26,27,56,57,58]. Moreover, the data on scores on these self-related dimensions (self-esteem and self-efficacy for instructional strategies) points to a possible negative development, namely the failure to form positive and constructive coping responses and the inability to find possible active solutions to frustrating tasks or predicaments.
A second consideration that seems to emerge from our data is that the finding of difficulties as increasing with seniority may indicate that, amidst a challenging new predicament, seniority, when not accompanied by proper training on interactional/relational dimensions, may be more of an obstacle than a resource. As experience brings with itself a tendency to learn and therefore repeat what has been learned, in this case, before developing new teaching strategies, the senior teachers might need to partially reconsider what they have learned throughout their career—a psychological and professional challenge that the junior teachers simply do not go through.
Teachers with greater seniority also tend to focus more on students and students with LDs, i.e., teachers tend to perceive a greater difficulty in students than inthemselves. In general, while the great pressure deriving from the particular predicament we are analysing probably directed the focus on technology and not on the relationship with their students, teachers with greater seniority mainlyfocussed on the relational components as, due to their experience, they inevitably have greater perceptions of the difficulty of others, particularly that of students.

5. Conclusions

The concentration of the school system’s efforts on the massive and, for a long time, exclusive organisation of distance learning risks favouring only cognitive aspects to the detriment of affective dynamics. This aspect could make teaching more complex for teachers and effect poorer learning in students. This raises a further reflection on the importance of social relationships. Even though distance learning allows virtual meeting for sharing time and didactic contents, it takes away many characteristics of the didactic experience that are fundamental in social relationships, especially in subjects of a developmental age [59]. The absence of relational experience space, with its verbal and non-verbal communicative characteristics, impoverishes the complex relational process that forms the basis of the learning process [60].
The results of this study allow us in part to confirm some data in the literature: data that identify in the relational variables those that can trigger a virtuous process that increases the self-esteem and self-efficacy of teachers who can thus involve students in the learning process, both affectively and cognitively, resulting in academic success.
We believe that our study results may be useful for planning possible teacher’s training based on emotional support related to the self-related dimensions, on relational/interactional field and, finally, on a specific preparation aimed at giving prerequisite necessary to reorganise the learning processes through new relational and teaching skills and strategies.
Our work has several limitations, including a small sample that needs to be extended (numbers of subjects and representative of school orders), from the perspective of teachers rather than that of students and families. We intend to continue our work in these directions.
It would be interesting in future research, also from a longitudinal perspective, to analyse the effects and changes in the relational processes underlying learning.

Author Contributions

Conceptualization, S.C., S.M.C., M.L.M., C.M., D.R.P., M.A., M.P.P.; methodology, S.C., M.L.M., M.A., M.P.P.; formal analysis, M.A.; investigation, S.C., S.M.C., M.L.M., C.M., D.R.P., M.A., M.P.P.; writing—original draft preparation, S.C., S.M.C., M.L.M., M.A., M.P.P.; writing—review and editing, S.C., S.M.C., M.L.M., M.A., M.P.P. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by ATS Sardinia with the project “Neuro-Psychological Profile and Emotional Problems in LDs: A Research-Intervention Proposal” [grant number CS181].

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Institutional Ethics Committee of the University of Cagliari, Italy (Prot. n. 0073815).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The datasets for this study are available from corresponding author on reasonable request.

Acknowledgments

The authors would like to thank all participants for their contribution to the work.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Domitrovich, C.E.; Durlak, J.A.; Staley, K.C.; Weissberg, R.P. Social-Emotional competence: An essential factor for promoting positive adjustment and reducing risk in school children. Child Dev. 2017, 88, 408–416. [Google Scholar] [CrossRef]
  2. Brinkworth, M.E.; McIntyre, J.; Juraschek, A.D.; Gehlbach, H. Teacher-student relationships: The positives and negatives of assessing both perspectives. J. Appl. Dev. Psychol. 2018, 55, 24–38. [Google Scholar] [CrossRef]
  3. Martin, A.J.; Collie, R.J. Teacher–student relationships and students’ engagement in high school: Does the number of negative and positive relationships with teachers matter? J. Educ. Psychol. 2019, 111, 861–867. [Google Scholar] [CrossRef]
  4. Quin, D. Longitudinal and Contextual Associations between Teacher–Student Relationships and Student Engagement: A Systematic Review. Rev. Educ. Res. 2017, 87, 345–387. [Google Scholar] [CrossRef]
  5. Goodboy, A.K.; Booth-Butterfield, M.; Bolkan, S.; Griffin, D.J. The Role of Instructor Humor and Students’ Educational Orientations inStudent Learning, Extra Effort, Participation, and Out-of-Class Communication. Commun. Q. 2015, 63, 44–61. [Google Scholar] [CrossRef]
  6. Frymier, A.B.; Houser, M.L. The teacher-student relationship as an interpersonal relationship. Commun. Educ. 2000, 49, 207–219. [Google Scholar] [CrossRef]
  7. Khan, A.; Khan, S.; Zia-Ul-Islam, S.; Khan, M. Communication Skills of a Teacher and Its Role in the Development of the Students’ Academic Success. J. Educ. Prac. 2017, 8, 18–21. [Google Scholar]
  8. Shokrpour, N.; Bambaeeroo, F. The impact of the teachers’ non-verbal communication on success in teaching. J. Adv. Med. Educ. Prof. 2017, 5, 51–59. [Google Scholar]
  9. Burić, I.; Moe, A. What makes teachers enthusiastic: The interplay of positive affect, self-efficacy and job satisfaction. Teach. Teach. Educ. 2020, 89, 103008. [Google Scholar] [CrossRef]
  10. Anderson, V.; Rabello, R.; Wass, R.; Golding, C.; Rangi, A.; Eteuati, E.; Bristowe, Z.; Waller, A. Good teaching as care in higher education. High. Educ. 2020, 79, 1–19. [Google Scholar] [CrossRef]
  11. Roberts, A.; Friedman, D. The Impact of Teacher Immediacy on Student Participation: An Objective Cross-Disciplinary Examination. Int. J. Teach. Learn. High. Educ. 2013, 25, 38–46. [Google Scholar]
  12. Miller, A.N.; Katt, J.A.; Brown, T.; Sivo, S.A. The Relationship of Instructor Self-Disclosure, Nonverbal Immediacy, and Credibility to Student Incivility in the College Classroom. Commun. Educ. 2014, 63, 1–16. [Google Scholar] [CrossRef]
  13. Myers, S.; Martin, M.; Mottet, T. Students’ motives for communicating with their instructors: Considering instructor socio-communicative style student socio-communicative orientation, and student gender. Commun. Educ. 2002, 51, 121–133. [Google Scholar] [CrossRef]
  14. Norton, R.W. Teacher Effectiveness as a Function of Communicator Style. Ann. Int. Commun. Assoc. 2017, 1, 525–542. [Google Scholar] [CrossRef]
  15. Frymier, A.B. A Model of Immediacy in the Classroom. Commun. Q. 1994, 42, 133–144. [Google Scholar] [CrossRef]
  16. Cayanus, J.L.; Martin, M.M. Teacher self-disclosure: Amount, relevance, and negativity. Commun. Q. 2008, 56, 325–341. [Google Scholar] [CrossRef]
  17. Wanzer, M.B.; Frymier, A.B. The relationship between student perceptions of instructor humor and students’ reports of learning. Commun. Educ. 1999, 48, 48–62. [Google Scholar] [CrossRef]
  18. Dicke, T.; Marsh, H.W.; Parker, P.D.; Guo, J.; Riley, P.; Waldeyer, J. Job satisfaction of teachers and their principals in relation to climate and student achievement. J. Educ. Psychol. 2020, 112, 1061–1073. [Google Scholar] [CrossRef]
  19. Lazarides, R.; Gaspard, H.; Dicke, A.L. Dynamics of classroom motivation: Teacher enthusiasm and the development of math interest and teacher support. Learn. Instr. 2019, 60, 126–137. [Google Scholar] [CrossRef]
  20. Mbuva, J. Exploring Teachers’ Self-Esteem and Its Effects on Teaching, Students’ Learning and Self-Esteem. J. High. Educ. Theory. Pract. 2016, 16. [Google Scholar]
  21. Perera, H.N.; John, J.E. Teachers’ self-efficacy beliefs for teaching math: Relations with teacher and student outcomes. Cont. Educ. Psychol. 2020, 61, 101842. [Google Scholar] [CrossRef]
  22. Dietrich, N.; Kentheswaran, K.; Ahmadi, A.; Teychene, J.; Bessiere, Y.; Alfenore, S.; Laborie, S.; Bastoul, D.; Loubiere, K.; Guigui, C.; et al. Attempts, successes, and failures of distance learning in the time of covid-19. J. Chem. Educ. 2020, 97, 2448–2457. [Google Scholar] [CrossRef]
  23. Garrick, A.; Mak, A.S.; Cathcart, S.; Winwood, P.C.; Bakker, A.B.; Lushington, K. Non-Work Time Activities Predicting Teachers’ Work-Related Fatigue and Engagement: An Effort-Recovery Approach. Aust. Psychol. 2018, 53, 243–252. [Google Scholar] [CrossRef]
  24. Mattern, J.; Bauer, J. Does teachers’ cognitive self-regulation increase their occupational well-being? The structure and role of self-regulation in the teaching context. Teach. Teach. Educ. 2014, 43, 58–68. [Google Scholar] [CrossRef]
  25. Skaalvik, E.M.; Skaalvik, S. Job satisfaction, stress and coping strategies in the teaching profession-what do teachers say? Int. Educ. Stud. 2015, 8, 181–192. [Google Scholar] [CrossRef] [Green Version]
  26. Toto, G.A.; Limone, P. From Resistance to Digital Technologies in the Context of the Reaction to Distance Learning in the School Context during COVID-19. Educ. Sci. 2020, 11, 163. [Google Scholar] [CrossRef]
  27. Toto, G.A.; Limone, P. Motivation, Stress and Impact of Online Teaching on Italian Teachers during COVID-19. Computers 2021, 10, 75. [Google Scholar] [CrossRef]
  28. Alves, R.; Lopes, T.; Precioso, J. Teachers well-being in times of Covid-19 pandemic: factors that explain professional well-being. Ijeri 2020, 15, 203–217. [Google Scholar] [CrossRef]
  29. Baluyos, G.R.; Rivera, H.L.; Baluyos, E.L. Teachers’ Job Satisfaction and Work Performance. Open J. Soc. Sci. 2019, 07, 206–221. [Google Scholar] [CrossRef] [Green Version]
  30. Rosenberg, M. Conceiving the Self; Basic Books: New York, NY, USA, 1979. [Google Scholar]
  31. Gini, G.; Pozzoli, T.; Hymel, S. Moral disengagement among children and youth: A meta-analytic review of links to aggressive behavior. Aggress. Behav. 2014, 40, 56–68. [Google Scholar] [CrossRef]
  32. Bono, J.E.; Judge, T.A. Core Self-Evaluations: A Review of the Trait and its Role in Job Satisfaction and Job Performance. Eur. J. Pers. 2003, 17, 5–18. [Google Scholar] [CrossRef]
  33. Piccolo, R.F.; Judge, T.A.; Takahashi, K.; Watanabe, N.; Locke, E.A. Core self-evaluations in Japan: Relative effects on job satisfaction, life satisfaction, and happiness. J. Organ. Behav. 2005, 26, 965–984. [Google Scholar] [CrossRef]
  34. Flammer, A. Self-Efficacy. Int. Encycl. Soc. Behav. Sci. Second Ed. 2015, 4, 504–508. [Google Scholar] [CrossRef]
  35. Bandura, A. Self-efficacy: Toward a unifying theory of behavioral change. Psychol. Rev. 1977, 84, 191–215. [Google Scholar] [CrossRef]
  36. Caprara, G.V.; Barbaranelli, C.; Steca, P.; Malone, P.S. Teachers’ self-efficacy beliefs as determinants of job satisfaction and students’ academic achievement: A study at the school level. J. Sch. Psychol. 2006, 44, 473–490. [Google Scholar] [CrossRef]
  37. Tschannen-Moran, M.; Hoy, A.W. Teacher efficacy: Capturing an elusive construct. Teach. Teach. Educ. 2001, 17, 783–805. [Google Scholar] [CrossRef]
  38. Caprara, G.V.; Barbaranelli, C.; Borgogni, L.; Steca, P. Efficacy Beliefs as Determinants of Teachers’ Job Satisfaction. J. Educ. Psychol. 2003, 95, 821–832. [Google Scholar] [CrossRef]
  39. Alea, L.A.; Fabrea, M.F.; Roldan, R.D.A.; Farooqi, A.Z. Teachers’ Covid-19 awareness, distance learning education experiences and perceptions towards institutional readiness and challenges. Int. J. Learn. Teach. Educ. Res. 2020, 19, 127–144. [Google Scholar] [CrossRef]
  40. Contreras-Castillo, J.; Favela, J.; Pérez-Fragoso, C.; Santamaría-del-Angel, E. Informal interactions and their implications for online courses. Comput. Educ. 2004, 42, 149–168. [Google Scholar] [CrossRef]
  41. Kim, K.J.; Liu, S.; Bonk, C.J. Online MBA students’ perceptions of online learning: Benefits, challenges, and suggestions. Internet High. Educ. 2005, 8, 335–344. [Google Scholar] [CrossRef]
  42. Mullen, G.E.; Tallent-Runnels, M.K. Student outcomes and perceptions of instructors’ demands and support in online and traditional classrooms. Internet High. Educ. 2006, 9, 257–266. [Google Scholar] [CrossRef]
  43. Cataudella, S.; Carta, S.; Mascia, M.L.; Masala, C.; Petretto, D.R.; Penna, M.P. Psychological Aspects of Students With Learning Disabilities in E-Environments: A Mini Review and Future Research Directions. Front. Psychol. 2021, 11, 3712. [Google Scholar] [CrossRef] [PubMed]
  44. Sharabi, A.; Sade, S.; Margalit, M. Virtual connections, personal resources, loneliness, and academic self-efficacy among college students with and without LD. Eur. J. Spec. Needs Educ. 2016, 31, 376–390. [Google Scholar] [CrossRef]
  45. Petretto, D.R.; Carta, S.M.; Cataudella, S.; Masala, I.; Mascia, M.L.; Penna, M.P.; Piras, P.; Pistis, I.; Masala, C. Some Lessons Learned in the Use of Distance Learning with Students with Special Educational Needs during COVID-19 Outbreak. Educ. Sci. 2021, 11, 108. [Google Scholar] [CrossRef]
  46. Prezza, M.; Trombaccia, F.; Armento, L. The Rosenberg Self-esteem scale: Italian translation and validation. Boll. Psicol. Appl. 1997, 223, 35–44. [Google Scholar]
  47. Biasi, V.; Domenici, G.; Patrizi, N.; Capobianco, R. Teacher Self-Efficacy Scale (Scala sull’auto-efficacia del Docente–SAED): Adattamento e validazione in Italia. J. Educ. Cult. Psychol. Stud. (ECPS J.) 2014, 10, 485–509. [Google Scholar] [CrossRef] [Green Version]
  48. Chiu, T.; Fang, D.; Chen, J.; Wang, Y.; Jeris, C. A robust and scalable clustering algorithm for mixed type attributes in large database environment. In Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’01), San Francisco, CA, USA, 26 August 2001; ACM Press: New York, NY, USA, 2001; pp. 263–268. [Google Scholar]
  49. Bacher, J.; Wenzig, K.; Vogler, M. SPSS TwoStep Cluster—A First Evaluation; Staedtler-Stiftung: Nürnberg, Germany, 2004; Volume 1. [Google Scholar]
  50. Benassi, M.; Garofalo, S.; Ambrosini, F.; Sant’Angelo, R.P.; Raggini, R.; De Paoli, G.; Ravani, C.; Giovagnoli, S.; Orsoni, M.; Piraccini, G. Using Two-Step Cluster Analysis and Latent Class Cluster Analysis to Classify the Cognitive Heterogeneity of Cross-Diagnostic Psychiatric Inpatients. Front. Psychol. 2020, 11, 1085. [Google Scholar] [CrossRef]
  51. The Jamovi Project 2021. Available online: https://www.jamovi.org/ (accessed on 5 June 2021).
  52. IBM Corporation. IBM SPSS 24 Statistics for Windows 2016; IBM Corporation: Armonk, NY, USA, 2016. [Google Scholar]
  53. Gelbard, R.; Goldman, O.; Spiegler, I. Investigating diversity of clustering methods: An empirical comparison. Data Knowl. Eng. 2007, 63, 155–166. [Google Scholar] [CrossRef]
  54. Kent, P.; Jensen, R.K.; Kongsted, A. A comparison of three clustering methods for finding subgroups in MRI, SMS or clinical data: SPSS TwoStep Cluster analysis, Latent Gold and SNOB. BMC Med. Res. Methodol. 2014, 14, 113. [Google Scholar] [CrossRef] [Green Version]
  55. Parmigiani, D.; Benigno, V.; Giusto, M.; Silvaggio, C.; Sperandio, S. E-inclusion: Online special education in Italy during the Covid-19 pandemic. Technol. Pedagog. Educ. 2020, 30, 111–124. [Google Scholar] [CrossRef]
  56. Baloran, E.; Hernan, J. Crisis Self-Efficacy and Work Commitment of Education Workers among Public Schools during COVID-19 Pandemic. Preprints 2020, 2020070599. [Google Scholar] [CrossRef]
  57. Ma, K.; Chutiyami, M.; Zhang, Y.; Nicoll, S. Online teaching self-efficacy during COVID-19: Changes, its associated factors and moderators. Educ. Inf. Technol. 2021, 1–23. [Google Scholar] [CrossRef]
  58. Pressley, T. Returning to teaching during COVID-19: An empirical study onelementary teachers self-efficacy. Psychol Schs. 2021, 58, 1611–1623. [Google Scholar] [CrossRef] [PubMed]
  59. Esposito, S.; Giannitto, N.; Squarcia, A.; Neglia, C.; Argentiero, A.; Minichetti, P.; Cotugno, N.; Principi, N. Development of Psychological Problems Among Adolescents During School Closures Because of the COVID-19 Lockdown Phase in Italy: A Cross-Sectional Survey. Front. Pediatr. 2021, 8, 1–12. [Google Scholar] [CrossRef] [PubMed]
  60. Niemi, H.M.; Kousa, P. A Case Study of Students’ and Teachers’ Perceptions in a Finnish High School during the COVID Pandemic. Int. J. Technol. Educ. Sci. 2020, 4, 352–369. [Google Scholar] [CrossRef]
Figure 1. Recruiting phases.
Figure 1. Recruiting phases.
Ijerph 18 08211 g001
Table 1. Descriptive statistics of the assessed variables.
Table 1. Descriptive statistics of the assessed variables.
Overall (n = 226)
Age
between 36–45 years93 (41.2%)
>56 years44 (19.5%)
between 46–55 years66 (29.2%)
<35 years23 (10.2%)
Gender
females199 (88.1%)
males27 (11.9%)
Level of School
university degree112 (49.6%)
high school diploma55 (24.3%)
university diploma7 (3.1%)
post-graduate training52 (23.0%)
Type of School You Teach In
kindergarten school28 (12.4%)
primary school80 (35.4%)
junior high school49 (21.7%)
high school69 (30.5%)
Years of Seniority at Work
n-miss1
mean (sd)14.7 (11.2)
range0.0–42.0
Levels of Years of Seniority at Work
n-miss1
<14years127 (56.4%)
>15 years98 (43.6%)
Specialisation in Student Support
no186 (82.3%)
yes40 (17.7%)
Learning Disabilities and/or Disabilities Referent
no215 (95.1%)
yes11 (4.9%)
Hours of Work
n-miss5
timeslot9 (4.1%)
part time24 (10.9%)
full time188 (85.1%)
Professional Training on Learning Disability
no125 (55.3%)
yes101 (44.7%)
Type of Contract
n-miss1
limited time82 (36.4%)
undefined time143 (63.6%)
During the COVID-19 emergency, did your school set up online teaching activities?
yes218 (96%)
If yes, was an official platform used?
yes186 (82%)
Self-Esteem
mean (sd)23.5 (3.3)
range10.0–32.0
Self-Efficacy for Student Engagement
mean (sd)6.8 (2.1)
range1.0–9.0
Self-Efficacy for Classroom Management
mean (sd)6.6 (2.1)
range1.0–9.0
Self-Efficacy for Instructional Strategies
mean (sd)6.9 (2.1)
range1.0–9.0
Table 2. Comparison between mean scores of participants and normative samples.
Table 2. Comparison between mean scores of participants and normative samples.
Self EsteemSelf-Efficacy for Instructional StrategiesSelf-Efficacy for Classroom ManagementSelf-Efficacy for Student Engagement
Meannormative sample30.617.146.86.98
Mean (sd) participants23.50 (3.26)6.86 (2.10)6.580 (2.07)7.50 (2.08)
Median participants24.007.637.2507.50
Standard Error participants0.2170.1400.1380.139
Student’s t (df)−32.700 (225)−2.02 (225)−1.63 (225)−1.29 (225)
p<0.001 ***0.044 *0.1050.200
Mean difference (95% CI)−7.100
(−7.530; −6.670)
−0.282
(−0.557; −0.008)
−0.224
(−0.495; 0.047)
−0.177
(−0.448; 0.094)
Cohen’s d effect size (95% CI)−2.170
(−2.410; −1.930)
-0.134
(−0.265; −0.003)
-0.108
(−0.239; 0.022)
-0.085
(−0.216; 0.045)
Note: * p < 0.05; *** p < 0.001.
Table 3. Linear correlations between years of seniority, self-esteem and self-efficacy scores.
Table 3. Linear correlations between years of seniority, self-esteem and self-efficacy scores.
Self-EsteemSelf-EfficacySelf-EfficacySelf-Efficacy
for Studentfor Classroomfor Instructional
EngagementManagementStrategies
Self-esteemr
p
Self-efficacy for student engagementr0.376 ***
p<0.001
Self-efficacy for classroom managementr0.354 ***0.964 ***
p<0.001<0.001
Self-efficacy for instructional strategiesr0.365 ***0.962 ***0.952 ***
p<0.001<0.001<0.001
Years of seniority at workr−0.117−0.147 *−0.139 *−0.168 *
p0.0790.0280.0380.012
Note: * p < 0.05; *** p < 0.001.
Table 4. Manova—Univariate Tests.
Table 4. Manova—Univariate Tests.
Dependent VariableSum of SquaresDf(b;w)Mean SquareFpEffect Size Partial Eta Squared
Level of schoolSelf-esteem61.033;21720.341.9690.1200.031
Self-efficacy for student engagement19.703;2176.571.5780.1960.017
Self-efficacy for classroom management23.733;2177.911.9050.1300.017
Self-efficacy for instructional strategies20.693;2176.901.6170.1860.018
Level of senioritySelf-esteem74.321;21774.327.1930.008 **0.024
Self-efficacy for student engagement30.891;21730.897.4260.007 **0.020
Self-efficacy for classroom management26.951;21726.956.4900.012 *0.021
Self-efficacy for instructional strategies35.281;21735.288.2690.004 **0.022
Level of school* level of senioritySelf-esteem18.623;2176.210.6010.6150.008
Self-efficacy for student engagement8.453;2172.820.6770.5670.009
Self-efficacy for classroom management9.323;2173.110.7480.5250.010
Self-efficacy for instructional strategies7.563;2172.520.5910.6220.008
Note: * p < 0.05; ** p < 0.01.
Table 5. Means and standard deviations regarding the levels of seniority.
Table 5. Means and standard deviations regarding the levels of seniority.
Level of Seniority at Work Self-EsteemSelf-Efficacy
for Student Engagement
Self-Efficacy
for Classroom Management
Self-Efficacy
for Instructional Strategies
<14 y
n = 127
mean23.9767.1146.8677.187
sd2.9581.9731.9621.981
>15 y
n = 98
mean22.8876.3876.1836.419
sd3.5572.1342.1532.182
Totalmean23.5026.7976.5696.852
sd3.27072.0722.0732.101
Table 6. Level of difficulty perceived by teachers in using digital platforms.
Table 6. Level of difficulty perceived by teachers in using digital platforms.
Options of ResponseQ18Q19Q20Q23Q24
none—fr(fr%)56 (25%)10 (4.4%)7 (3.1%)13 (5.8%)2 (0.9%)
low—fr(fr%)96 (42%)70 (31%)40 (18%)43 (19%)33 (15%)
medium—fr(fr%)61 (27%)98 (43%)84 (37%)111 (49%)122 (54%)
high—fr(fr%)13 (5.8%)48 (21%)38 (17%)59 (26%)69 (31%)
missing—fr(fr%)0057 (25%)00
median12222
Total sample
mean

1.14

1.81

1.91

1.97

2.19
sd0.8550.7720.7580.8300.649
MDN1.002.002.002.002.00
Low level of seniority at work (<14 years)
mean

1.02

1.78

1.85

1.94

2.16
sd0.8280.7370.7910.8260.641
MDN1.002.002.002.002.00
High level of seniority at work (>14 years)
mean

1.28

1.84

1.99

2.01

2.24
sd0.8700.8150.7120.8370.661
MDN1.002.002.002.002.00
Primary school
mean

1.24

1.94

2.02

2.33

2.29
sd0.9450.8320.6800.6890.697
MDN1.002.002.002.002.00
Junior high school
mean

1.04

1.88

1.89

2.14

2.24
sd0.7890.6340.8590.6770.560
MDN1.002.002.002.002.00
High school
mean

1.10

1.61

1.83

1.43

2.03
sd0.7890.7520.7400.8130.641
MDN1.002.002.002.002.00
Note: (Q18) “What is the level of difficulty you encountered in using the platform?”, (Q19) “What is the level of difficulty encountered by your students in using the platform?”, (Q20) “What is the level of difficulty encountered by your students with LDs in using the platform?”, (Q23) “How was the level of cooperation from parents?”, (Q24) “How was the level of student’s participation?”.
Table 7. Description of the three clusters according to the variables assessed.
Table 7. Description of the three clusters according to the variables assessed.
Center of ClustersCluster
1
(n = 47)
2
(n = 104)
3
(n = 68)
Combined
(n = 219)
F(df)
p
Self-esteem
mean (SD)
21.787
(4.422)
23.615
(3.021)
24.647
(2.057)
23.543
(3.284)
11.609 (2)
p < 0.001
Self-efficacy for student engagement
mean (SD)
3.343
(1.116)
7.713
(0.844)
7.830
(0.966)
6.812
(2.047)
402.684 (2)
p < 0.001
Self-efficacy classroom management
mean (SD)
3.263
(1.120)
7.525
(0.969)
7.446
(1.200)
6.586
(2.045)
284.141 (2)
p < 0.001
Self-efficacy for instructional strategies
mean (SD)
3.391
(1.188)
7.752
(0.911)
7.906
(0.956)
6.864
(2.070)
368.308 (2)
p < 0.001
Data description for cluster1
(n = 47)
2
(n = 104)
3
(n = 68)
Combined
(n = 219)
F(df)
p
Years of seniority at work
mean (SD)
16.980
(11.301)
19.490
(10.570)
5.760
(4.860)
14.690
(11.134)
45.897 (2)
p < 0.001
Data description for cluster1
(n = 47)
2
(n = 104)
3
(n = 68)
Combined
(n = 219)
Chi Squared (df)
p
Limited time employment contract
fr (fr%)
15 (31.9%)065 (95.6%) 162.584 (2)
p < 0.001
Indefinite time employment contract
fr (fr%)
32 (68.1%)104 (100%)3 (4.4%)
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Cataudella, S.; Carta, S.M.; Mascia, M.L.; Masala, C.; Petretto, D.R.; Agus, M.; Penna, M.P. Teaching in Times of the COVID-19 Pandemic: A Pilot Study on Teachers’ Self-Esteem and Self-Efficacy in an Italian Sample. Int. J. Environ. Res. Public Health 2021, 18, 8211. https://doi.org/10.3390/ijerph18158211

AMA Style

Cataudella S, Carta SM, Mascia ML, Masala C, Petretto DR, Agus M, Penna MP. Teaching in Times of the COVID-19 Pandemic: A Pilot Study on Teachers’ Self-Esteem and Self-Efficacy in an Italian Sample. International Journal of Environmental Research and Public Health. 2021; 18(15):8211. https://doi.org/10.3390/ijerph18158211

Chicago/Turabian Style

Cataudella, Stefania, Stefano Mariano Carta, Maria Lidia Mascia, Carmelo Masala, Donatella Rita Petretto, Mirian Agus, and Maria Pietronilla Penna. 2021. "Teaching in Times of the COVID-19 Pandemic: A Pilot Study on Teachers’ Self-Esteem and Self-Efficacy in an Italian Sample" International Journal of Environmental Research and Public Health 18, no. 15: 8211. https://doi.org/10.3390/ijerph18158211

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop