Next Article in Journal
Interactions between Geomorphology and Production Chain of High-Quality Coffee in Costa Rica
Next Article in Special Issue
Ukrainian Migrants in Poland and the Role of an Employer as the Channel of Information during the COVID-19 Pandemic
Previous Article in Journal
Efficiency Evaluation of Chinese Provincial Industrial System Based on Network DEA Method
Previous Article in Special Issue
Higher Education in and after COVID-19: The Impact of Using Social Network Applications for E-Learning on Students’ Academic Performance
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Learning Outcomes of a Hybrid Online Virtual Classroom and In-Person Traditional Classroom during the COVID-19 Pandemic

1
Department of Economics, Qingdao University of Technology, Qingdao 266525, China
2
Department of Agricultural Economics, University of Kentucky, Lexington, KY 40546, USA
*
Author to whom correspondence should be addressed.
Sustainability 2022, 14(9), 5263; https://doi.org/10.3390/su14095263
Submission received: 31 March 2022 / Revised: 23 April 2022 / Accepted: 23 April 2022 / Published: 27 April 2022
(This article belongs to the Special Issue Economic and Social Consequences of the COVID-19 Pandemic)

Abstract

:
This paper explores the effects of using a hybrid virtual/traditional classroom, a blended teaching and learning approach, on undergraduate learning outcomes during the initial stage of the COVID-19 pandemic. The study aimed to investigate the prominent problems that appeared in the process of online teaching and learning and to measure the impact of the COVID-19 pandemic on undergraduates’ learning outcomes in public universities in China. The field of specialization of the subjects of this study is under the rubric of social science and higher education. We designed and conducted a pilot survey to identify students’ perspectives on the key issues and experiences of the use of distance-learning through an online virtual classroom in the initial phase of the COVID-19 pandemic. In addition, we applied a binary logistic regression model to real data from two different economics course exams to measure the short-term impacts of using the two different learning environments on the undergraduates’ performance outcomes. The results indicate a statistically significant negative impact of using virtual classrooms on undergraduate learning outcomes. By contrast, in-person traditional classrooms had more desirable learning outcomes. Moreover, the hybrid approach proved to be more effective than the use of online virtual classrooms alone.

1. Introduction

Over two years has passed since the novel coronavirus-infected pneumonia was first identified in Wuhan City, China, in 2019 [1]. According to the World Health Organization, as of 30 March 2022, over 6 million people across the globe have died from the COVID-19 pandemic, and the number of confirmed cases is still on the rise. The world is now feeling the impact of the pandemic with a new surge of infections of the SARS-CoV-2 Omicron variant in many countries. The eradication of SARS-CoV-2 is not a simple prospect, as the world is a global village, and no individual country can move forward without working with other countries. Pandemic eradication takes the ‘weakest-link’ perspective, in which the active participants are the individual countries investing in epidemic eradication. From this point of view, a chain is as strong as its weakest link, and the overall effort concerning pandemic eradication is equal to the lowest amount of effort across different countries [2].
The pandemic has had a profound effect on education at all levels, especially in the early stages of the pandemic. In the early stages of the pandemic, schools worldwide were forced to take various measures, including social distancing measures, in order to reduce the spread of the virus and to ensure student health and safety. The use of the in-person traditional classroom approach always outperforms the online virtual classroom and distance-learning instruction when it comes to attracting and keeping student interest [3]. However, in the spring of 2020, Chinese higher education institutions were abruptly forced to move from in-person classrooms to online virtual classrooms because of the COVID-19 pandemic, despite the fact that the conventional classroom is preferred by both teachers and students for efficient communication and learning outcomes. Thus, a significant shift toward online learning was prompted by the COVID-19 pandemic, and the virtual classroom automatically became the best option to continue teaching and learning. Hence, the first research question is how a change in the learning environment affects educational activities, especially with the students’ new virtual learning experiences. We investigated this question using a pilot survey.
It is important to note that the authors frequently use the terms ‘short-term impacts’ and ‘the initial phase of the COVID-19 pandemic’. The ‘short-term impacts’ refer to students’ learning outcome variations associated with the blended teaching and learning mode in the initial phase of the COVID-19 pandemic. When the COVID-19 pandemic was first discovered in China, the country did not effectively control the spread of the virus in the first quarter of 2020, but social and economic activities were gradually restored beginning in the second quarter of 2020 [4]. Similarly, in Germany and other countries, beginning in mid-March 2020, governments announced major restrictions to fight against the rapid spread of the pandemic [5]. Hence, on the basis of these facts, we assume late-March 2020 to be the baseline, and we call the time before this baseline ‘the early or initial stages’ and the time after this baseline ‘the later stage’ of the pandemic. Not until the fall semester of 2020 did traditional classrooms gradually begin to be restored on Chinese campuses and beyond. Currently, the virtual classroom remains in use, but only as an auxiliary approach compared with its use in the early stages of the pandemic.
Furthermore, the three modes or environments employed in this research refer to the traditional face-to-face mode; the virtual online mode; and the blended mode, which is a combination of the former two modes. In the spring term (from March to mid-July) of 2020, the delivery was only by online mode, but then in the fall semester (from late-August to December), the traditional face-to-face mode was used. The third mode was the combination of the other two modes, also known as the blended mode. The blended mode here can be understood from three perspectives. First, from the perspective of one entire academic year in 2020, we used the online mode in the spring term and the traditional face-to-face mode in the fall term, which was, for the first time, a combination of the other two modes used. Second, from the perspective of one term (autumn), the above-mentioned blended mode was also applicable in some measures in that it differed from the case prior to the pandemic. Third, from the perspective of economics course teaching and learning practices in China, before the pandemic, microeconomics and macroeconomics always went hand-in-hand; namely, they were delivered in sequential order and always employed the conventional mode.
The two closely related undergraduate microeconomics and macroeconomics courses and the corresponding final exams were assessed at the end of each semester to follow up on the students’ distance-learning experiences and the resulting learning outcomes. We used exam scores to denote the learning outcomes and employed the mean value to assess the microeconomics and macroeconomics learning outcomes. The microeconomics course and its final test were completed in a virtual online classroom, while the macroeconomics course and its final test were completed in a conventional in-person classroom. Hence, a second research question arose: do the blended teaching and learning modes used in the initial stage of the pandemic affect the undergraduates’ learning performance? We further evaluated the impact of the COVID-19 pandemic on undergraduate students’ learning outcomes using a binary logistic regression model. The objective was to explore the causal relationship between the use of a hybrid virtual/traditional classroom and undergraduate learning performance.
Therefore, the objectives of this study were twofold. The first objective was to investigate the student experiences and perceptions of the online open learning environment. The second was to measure the impact of different learning modes and the associated experiences on learning outcomes; we then compared and contrasted this impact with that of the traditional in-person classroom approach. We particularly focus on the key problems associated with distance-learning through a mixed study design, i.e., one pilot survey, followed by two course exams during the 2020 academic year, and use regression model analysis to students’ grade data. To the best of our knowledge, this is the first study to investigate the nexus between learning performance and the use of a blended virtual/traditional learning environment from a holistic perspective for the early days of the pandemic. To date, there is little empirical research evaluating and comparing the relationships between the two hybrid virtual/traditional classroom platforms used during the pandemic to investigate the combined impacts on student learning-outcomes, though at the time of writing this article, a few empirical studies are available that focus on direct influences of COVID-19 pandemic on student learning performance.
This study, therefore, contributes to the literature on the impact of COVID-19 on student learning performance in a mixed hybrid teaching and learning environment. The results indicate a statistically significant negative impact of the use of a virtual classroom on the undergraduate learning-outcomes. In contrast, in-person traditional classroom has more desirable learning-outcomes. Moreover, the blended approach proved to be more effective than using only online virtual classroom.
The rest of the paper is organized as follows. Section 2 provides a literature review. Section 3 presents the details of methodology and describes the dataset and data sources. Section 4 discusses the empirical results and Section 5 offers a discussion of the results. Finally, Section 6 concludes this study.

2. Background Literature

The virtual classroom has received much attention in academia. Researchers have conducted a great many studies focused on the student experience and the learning effects associated with synchronous virtual classrooms. However, various mixed conclusions have been reached. Some studies have found that the virtual classroom environment leads to desirable learning effects. For example, Hiltz [6] reviewed and compared various features of both the traditional classroom and virtual classroom. The findings revealed that the virtual classroom can stimulate and enhance collaborative student learning. Moreover, both teachers and students reported high levels of satisfaction as a result of productive shared contributions. Cao et al. [7] stated that the virtual classroom effectively raises student satisfaction. Lietzau and Mann [8] found that students can learn more and achieve higher marks as long as they engage in synchronous virtual classrooms. Parker and Martin [9] compared the perceptions of undergraduate students using a virtual classroom in a fully online and a blended education course and found that online students rated their perceptions of the characteristics of the virtual classroom higher than students on the blended course.
A number of studies have shown that the virtual classroom is effective in terms of enhancing student learning and achievement [10,11,12,13]. Fidalgo et al. [14] found that most students were interested in taking distance-education courses, though they were somewhat apprehensive. Even during the COVID-19 pandemic, online learning tools, such as Tencent Meeting and Smart Classroom, had a significant impact on student learning performance [15,16]. In another study, Yu [17] revealed that gender, education level, and personality traits might affect student online learning outcomes in the context of the COVID-19 pandemic. In particular, postgraduate students outperformed undergraduate students in terms of online learning outcomes. Because the pandemic has not been fully eradicated worldwide, virtual classroom and distance-learning tools remain of importance in certain critical circumstances, such as during the blockade of a city, while learning or working from home. In such cases, it is vital that teachers highlight students’ perceptions of playfulness or enjoyment during online classes in order to enhance their intention to repeat the virtual learning experience [18].
In addition, course design and structure are important to students’ satisfaction and their learning-outcomes [19,20,21,22,23,24]. Betty et al. [19] found that effectiveness in an online environment is contributive to the development of relationships and the overall success of coursework. Meanwhile, the online learning environment, if it is devoid of personal communication, is inundated with approaches that help students feel a sense of partnership while meeting their learning needs. Van Wart [20] emphasizes the importance of course structure and pre-planning, organization, and consistency, and argues that students prefer virtual class organizations that are modular and repetitive. Kauffman [21] argues that course instructional design to be of great importance. Eom, Wen, and Ashill [22] surveyed 397 university students and found instructor feedback in online courses to be statistically significant; and course structure was found to be the most significant determinant of student satisfaction. Jung [23] argues that online courses are more challenging to organize; students tend to be very critical of what they perceive as any confusion or unclear structure in the distance-learning setting. McMurtry [24] states that good online teachers keep students focused by creating a structured environment that is logically organized, delivered in small chunks, and sufficiently repetitive.
Meanwhile, other scholars have insisted that the virtual classroom has certain disadvantages, such as equipment requirements, the textual and technical skills required, the motivation and regular participation required, and the potential “information overload”. In practice, a semi-virtual classroom may be a good alternative for the distance-education environment [6]. In addition to these factors, interaction difficulty may be negatively associated with student satisfaction during internet-based course learning [18]. Learning from a virtual classroom alone may lead to significantly lower grades compared with learning outcomes from a traditional classroom [25].
In the context of the COVID-19 pandemic, Rizun and Strzelecki [26] revealed that the impact of COVID-19 in terms of shifting higher education to virtual learning in Poland was seen in a positive light and was effective. Sprenger and Schwaninger [27] compared e-lectures, the classroom response system, classroom chat, and mobile virtual reality in terms of technology acceptance and found that the classroom response system had the highest level of acceptance, while mobile virtual reality saw a substantial drop in behavioral intention after 3 months of usage. Various practical problems have been commonly encountered in distance-learning during the COVID-19 pandemic. For instance, Sharma and Bumb [28] listed 25 challenges faced in online classes, including a lack of interaction with peers, interruptions in the online classroom, and mental stress resulting from the pandemic. In addition, even though students reported a moderate feeling that virtual learning tools enhanced their effectiveness and productivity, they still preferred to return to a traditional classroom [16,26].
There are a large number of studies focusing on the comparison between online virtual and traditional teaching [29,30,31,32], with mixed results. For instance, Ali Alghazo [29] used students’ final grade to examine the effectiveness of both online and traditional teaching, and concluded that there existed no significant differences in the effectiveness of online education and traditional face-to-face education. Sondoozi [30] showed that the online learning outcomes of students were similar to the performance of those using the traditional approach, while the satisfaction and attitudes of online learners were positive. Feng et al. [31] compared the effects of online teaching during COVID-19 with the pre-pandemic traditional teaching in compulsory education. They found that student’s performance before the pandemic was better than after the pandemic, with many previously high-scoring students now scoring closer to the mean. Zhao et al. [32] compared teaching efficiency between virtual reality and traditional education in medical education. Their results indicated students in the virtual reality group performed better than those in the traditional teaching group, concluding that virtual reality teaching may enhance learning-comes for medical students.
In general, the virtual classroom has gained considerable popularity since 1994. However, despite both teachers and students covering the material set forth in the textbooks and workbooks [33], and later moving from instructor-centered to student-centered curriculum [34], the traditional face-to-face approach remains key in modern education practice. Since both modes have pros and cons, it is best to synthesize the merits of the two modes in higher education. Although the hybrid virtual classroom/traditional classroom mode offers a promising channel for both teaching and learning in various contexts, little effort has been focused on the blended mode and the relationships between learning patterns and performance during the COVID-19 pandemic.
At present, we are more adaptive to the ongoing pandemic environment in terms of reduced fear and anxiety, as well as more protective ways to address this health crisis. More teachers are adapting to blended virtual classrooms in colleges and universities and some are returning to conventional classrooms. However, compared with the currently numerous research regarding the nexus between learning performance and the various learning patterns, studies focusing on the subject in the initial phase of the COVID-19 pandemic are relatively scarce. China was the first country to be shocked by the pandemic, and few empirical studies in the field of higher education were conducted. To the best of our knowledge, there is only one study that is most relevant to this paper. Hence, this research has practical significance in terms of exploring the undergraduate learning performance associated with the initial virtual online teaching and learning experience, as well as the blended learning approach in higher education.
The results of the available literature depend on the models and data used in the studies, and the impacts of the COVID-19 pandemic on higher education in China and beyond remain unclear with mixed conclusions. Additionally, there is no one-size-fits-all model to adopt for examining the effects of online teaching and learning environment on undergraduates’ learning performance. The most relevant models are regression analysis tools [10,15,35]. Chowdhury [10] using binary logistic regression to explore whether virtual classroom can improve students’ learning performance, found most participants have positive opinions about the use of virtual classroom for learning purposes, but this paper does not include the impact of COVID-19 disruption in the model analysis explicitly. In another study, Quadir and Zhou [15] used regression and showed that determinants of technology acceptance models had a positive effect on student learning outcomes in the context of the pandemic. Guse et al. [36] performed the binary logistic regression model on the association of gender, distress, and depression with serious worries in medical students during the pandemic. They investigated mental health outcomes among medical students during the initial phase of the COVID-19 pandemic, and perceptions of the students on how the learning environment had changed in a larger sample of undergraduates. They found that medical students experienced much distress and mental burden during the pandemic.
In this study, we address the following two main research questions, including but not limited to the above mentioned in the pilot survey. The first research question is from the perspective of university students: what are the main environmental determinants affecting teaching and learning activities during the initial stage of COVID-19? The second research question is whether there exists a relationship between teaching and learning environment and student learning performance.
The Null Hypothesis (H0):
The use of a virtual classroom does not affect student learning outcomes during the COVID-19 pandemic.
The contributions of this study is the use of primary data for the early stage of the COVID-19 pandemic for two successive semesters that can better represent the characteristics of the subjects of the study, and estimates of the impact of COVID-19 in its initial phase on undergraduates’ learning outcomes in a blended mode.

3. Materials and Methods

3.1. Study Design

This study consists of the design of a questionnaire and conduct of a survey to collect primary data, two successive course exams, and a binary logistic regression model. The questionnaire, as shown in Table 1, was developed to identify the outstanding problems experienced by university students when using virtual classroom during the spring semester of 2020 academic year in China. In order to assess the impact of the early stages of the COVID-19 pandemic in higher education, specifically from 21 April to 30 April 2020, an online questionnaire survey was designed and sent to students of more than 10 Chinese mainland universities, including students from Huazhong Agricultural University in Wuhan City, Hubei Province, where the COVID-19 pandemic was first detected, where participants in those regions were consequently better representing the characteristics of the sudden shock of the pandemic. The principle sampling selection of the pilot survey was using a survey platform named wenjuanxing with the website address https://www.wjx.cn/ (accessed on 21 April 2020).
A total of 689 subjects participated in the pilot survey. We had designed six primary questions, as shown in Table 1, relevant to the virtual classroom. Since the COVID-19 crisis is new, no previous model is applicable for the survey. We developed a list of questions focusing on university students, especially undergraduates. The electronic survey was conducted to investigate the main aspects of the virtual classroom being used in the context of COVID-19. For example, the second question was aimed to gather information on the learning software the respondents employed in their virtual classrooms. Key aspects regarding distance-learning were developed to identify outstanding problems, respondents’ preferences regarding virtual and traditional classroom learning, and comparing the impact of class size on learning outcomes.
Following the pilot survey, data were collected from two course exams that were held at the end of the spring semester in the 2019/20 academic year and at the end of the fall semester in the 2020/21 academic year. The microeconomics final exam was conducted in an open-book approach, while the macroeconomics exam was in a closed-book fashion.
Due to the pandemic, students were learning from home in the spring semester of 2020. The microeconomics final exam took place online on 22 July 2020. It took the form of an open-book exam for the first time because of the COVID-19 pandemic, and a total of 794 students enrolled using Tencent Meeting. For the open-book exam (online), students can bring any related printed materials with them to take the exam. Prior to the pandemic, the open-book exam was taken in traditional classrooms. The second exam, macroeconomics, took place on 23 December 2020 in the form of traditional closed-book exam, and a total of 668 students were enrolled. In contrast, a traditional closed book exam requires students not to bring any printed materials with them when entering into the classroom, and complete the exam in a physical classroom. Exam takers receive a security check before entering the classroom. For both classes, the exam format is identical: single choice, multiple choice, true or false, calculation questions, short answers, and longer essays. In total, 1462 students took part in the final microeconomics and macroeconomics exams during the COVID-19 pandemic in 2020. Finally, a binary regression analysis was used to examine the causality between students’ performance and different teaching and learning modes.

3.2. Data Collection

The pilot survey was administered to collect data on student perceptions of the virtual classroom in the early stages of the COVID-19 pandemic. The responses of 689 respondents on the usage of the virtual classroom and outstanding problems associated with distance-learning were analyzed. The exam grades data of students at Qingdao University of Technology were collected at the end of the respective spring and fall semester of 2020. There is an economics course teaching group, which has seven fixed lecturers with the same course syllabus, textbook, test content, as well as the same grading policy to judge the test papers at the end of each semester. Additionally, the seven lecturers grade and mark the exam papers together. Moreover, for this study, the researchers obtained real data, consisting of 1462 observations from students on the economics courses: 794 observations for the microeconomics course in the 2019–2020 academic year and 668 observations for the macroeconomics course in the 2020–2021 academic year. The two courses were taught over two successive semesters during the pandemic. The dataset was set up as panel data, and included variables such as students’ major, gender, and final exam grades. These are available upon request from both liberal arts and STEM major public universities on the Chinese mainland. Economics courses in this university, regardless of current major, are open to all students as long as they apply for the courses to gain credits that count towards their respective degrees. The descriptive statistical results are shown in Table 2.

3.3. Data Analysis

The survey data collected from 689 participants were mainly used for the pre-analysis of the impact of COVID-19 on student learning outcomes associated with the virtual classroom. Among the 689 participants, a total of 647 students, or 93.9%, were undergraduate students, and only 6.1% of respondents in the sample were graduate students. As can be seen in Table 2, the mean result of the microeconomics test was about seven points lower than that of the macroeconomics test. Additionally, it is important to note that the microeconomics course was conducted entirely via a virtual classroom and the final test was conducted in the form of an open-book exam, which was held 3 months after the pilot survey. In contrast, the macroeconomics course was held mostly in a traditional classroom, and the final exam was conducted in the form of a closed-book exam. Furthermore, as is often the case, the course grade in the full sample appeared to be asymptotically normally distributed [35]. Specifically, with a respective skewness and kurtosis of −0.392 and 2.739, as is shown in Table 2, the skewness was around zero and the kurtosis centered on 2, which was close to the normal bell-shaped curve. On the basis of the results shown in Table 2, it is obvious that student learning outcomes achieved via a virtual classroom were less desirable than those achieved via a traditional classroom. Thus, the virtual classroom appeared to affect student learning outcomes in the earlier stage of the pandemic. In order to capture and compare the effects of the virtual classroom and traditional classroom on student learning performance, a binary logistic regression model was applied to further measure the nexus between student performance and the use of hybrid virtual/traditional classrooms during the COVID-19 period.

3.4. The Binary Logistic Regression Analysis

On the basis of the course data analysis, as shown in Table 2, it was obvious that the results were asymptotically normally distributed. Since the student learning outcomes associated with the virtual classroom were lower than expected compared with those of the traditional classroom, we took the traditional closed-book test mean value of the macroeconomics test as the baseline, making comparisons of two respective semester grades. That is, if the actual score (from 100 points) was equal to or greater than 70, it was replaced with 1; otherwise, it was replaced with a 0. Alternatively, we treated the microeconomics grade dataset as the treated group, and the macroeconomics grade dataset as the controlled group, as is shown in Table 3.
As a result of the data transformation shown above, we were able to use the logistic model to examine whether the virtual classroom mode affected student learning performance. Given the features of the above data, it was appropriate to take the binary logistic regression model and examine the nexus among the dependent variable denoted by grade, and three explanatory variables: (1) the virtual classroom or traditional classroom denoted by mode; (2) the varieties of specialties (liberal arts and STEM) denoted by major; and (3) gender. In particular, exploring whether the blended pattern affected student learning outcomes was our primary interest. The binary logistic model is as follows:
ln ( P i 1 P i ) = X i β
where   X i   represents the vector form of the explanatory variable, β is the corresponding estimated parameter, and P i is the function of the grade conditional on mode or/and major or/and gender. Specifically,
P i = P ( grade i = 1 | mode i , major i , gender i )
Note that, similarly to the dependent variable grade, mode, major, and gender are all dummy variables by construction. We assumed that the grade variable was 1 if the grade was equal to or was greater than the mean score for macroeconomics, which was 70; and 0 otherwise. Similarly, the mode took the value of 1 if virtual classroom was adopted and 0 if not. As for major variable, 1 represents liberal arts majors and 0 represents STEM majors. Moreover, the gender variable was 1 if the respondent was female and 0 otherwise. The variables are displayed in Table 4.
We were able to further derive the marginal effect of the virtual classroom or traditional classroom mode on student learning outcomes either by using the equation below or by computing the corresponding odds ratio (OR):
P ( grade i = 1 | X i ) mode i = e X i β ( 1 + e X i β ) 2 β i
The variables in Equation (3) are the same as those in Equation (1).

4. Results

4.1. Results of the Pilot Survey

Table 5 lists key aspects concerning the use of virtual classrooms during the pandemic. For example, regarding online learning software in the early stage of the pandemic on the Chinese mainland, the top two application software was Tencent Classroom and QQ Classroom. As regards to the preference for the distance-learning environment, 43.83% of the respondents preferred the traditional classroom, only 14.22% preferred the virtual classroom, and the remaining 41.94% preferred the hybrid virtual/traditional classroom. Other prominent problems were poor internet connectivity (70.54%), reduced interaction between teacher and students (56.17%), and noise distractions during class (37.01%). The results shown in Table 5 imply that the virtual classroom environment seemed likely to affect student learning performance in the context of the COVID-19 pandemic.

4.2. Results with Controlling Covariates

A binary logistic regression analysis was conducted with the learning outcomes being denoted by the transformed student exam grades as a dependent variable and the mode, which was the virtual classroom or traditional classroom, student major, and gender as the independent variables. The results of the regression analysis are presented in Table 6 and Table 7.
Table 6 displays the logistic regression results of grades on mode with controlling covariates, inclusive of major and gender. Furthermore, all regressors shown in Table 6, other than major, are significantly different from 0 at dissimilar levels; we could thus strongly reject the null hypothesis that the virtual classroom does not affect student learning outcomes during the early stage of the COVID-19 pandemic. Furthermore, it was clear that, when the covariates of major and gender were controlled, the mode was significantly negatively correlated with student learning performance. To be specific, when controlling for major and gender, the mode had a significantly negative impact on exam grades, with a parameter of −1.117 and a significance level of 0.01. The coefficient of mode was negative, which implies the use of a virtual classroom may have lowered undergraduate student final exam scores significantly. In essence, the poor internet connectivity and reduced interaction between teachers and students, as mentioned in Table 5, may have jointly contributed to the relatively low mean of student exam grades. In contrast, the constant coefficient was positive (0.432), which means the effect of the traditional classroom was more likely to enhance student learning performance. Intuitively, this also makes sense, as there are more face-to-face communications and peer effects in a physical classroom.

4.3. Results without Controlling Covariates

Table 7 displays the logistic regression results of grade on mode without controlling covariates, inclusive of major and gender. Furthermore, all regressors shown in Table 7, other than major, are significantly different from 0 at dissimilar levels; we could thus strongly reject the null hypothesis that the virtual classroom does not affect student learning outcomes during the early state of the COVID-19 pandemic. Furthermore, it was clear that, when not controlling the covariates of major and gender, the mode was significantly negatively correlated with student learning performance. To be specific, when not controlling for major and gender, the mode had a significantly negative impact on exam grade, with a parameter of −1.117 and a significance level of 0.01. The coefficient of mode was negative, which implies the use of a virtual classroom may have lowered undergraduate student final exam scores significantly. In essence, the poor internet connectivity and reduced interaction between teachers and students, as in Table 5, may have jointly contributed to the relatively low mean of student exam grades. In contrast, the constant coefficient was positive (0.682), which means the effect of the traditional classroom was more likely to enhance student learning performance. Intuitively, this makes sense, as there are more face-to-face communications and peer effects in a physical classroom. Notice that when not controlling the major and gender variables, the mode still significantly negatively impacted the grade, with both the parameter and the significance level remaining unchanged. Additionally, as shown in Table 7, the major variable had a negative impact on learning performance, but this was insignificant. Interestingly, gender had a significantly positive impact on student learning outcomes. This may be due, in part, to the fact that over half of the undergraduates who major in liberal arts such as economics in China are female, and they are generally more diligent and are much better at taking exams.

4.4. The Marginal Effects

Furthermore, in Table 6 and Table 7, we present the aforementioned marginal effect of mode on learning performance in the form of an odds ratio. Regarding the microeconomics course that adopted a virtual teaching and learning pattern, on average, the probability of grade improvement was in the order of 0.355, while the unimproved probability was 0.665. Thus, the odds ratio of microeconomics grade improvement OR1 was about 0.504. Moreover, the odds ratio of macroeconomics grade improvement OR2 was about 1.54. In this way, we derived the ultimate OR, i.e., 0.327, which is the ratio of grade improvement of microeconomics relative to that of macroeconomics (OR1/ OR2). In other words, during the COVID-19 disruption period, the virtual teaching and learning mode was found to lower the grade improvement with a probability of approximately 67.3%, as compared with that of the traditional classroom.

5. Discussion

As mentioned earlier, we employed an online virtual classroom during the entire 2019–2020 spring term due to the pandemic and resumed a traditional classroom setting in the fall semester of the 2020–2021 academic year. It is important to note that even though we adopted a conventional classroom pattern in the autumn semester of 2020, there was still an online learning section before and after the physical lesson, involving previews and exercises associated with each chapter. Our regression results support the findings from the pilot survey that, given the choice of a traditional classroom, a virtual classroom, and a hybrid virtual/traditional classroom, 44% of the respondents chose the traditional classroom, 42% chose the blended mode, and only 14% chose the virtual classroom. Hence, given the context of the COVID-19 pandemic, with everything else being equal, the virtual classroom learning environment does have an effect on undergraduate student learning performance.
The results of the current study are similar to [32] to some extent, but are quite different from most prior studies [10,15,26,37,38,39]. For example, Chowdhury [10] found that the virtual classroom could produce improvements in student learning and performance. Rizun and Strzelecki [26] concluded that student acceptance of distance-learning during the COVID-19 pandemic in Poland was characterized by enjoyment and efficiency. In another study, Quadir and Zhou [15] identified that a virtual classroom learning environment, such as the Tencent Meeting system, had a significant effect on the student learning performance in terms of the perceived ease of use and perceived usefulness during the COVID-19 pandemic school disruption. Chen and Meng [38] found that task-oriented behaviors accounted for most online learning processes and student engagement levels during virtual classroom learning processes had a statistically significant impact on their learning outcomes. This may be, in part, because of the apprehensions or anxieties of both instructors and students resulting from COVID-19, which, in turn, may have resulted in inactive or somewhat pessimistic learning behaviors and ultimately lower-than-expected performance; or, to put it slightly differently, the emotional state of the students may contribute to their learning process and performance [37]. For instance, our results are in conflict with the findings of [29,39] who found no major differences between the virtual classroom and face-to-face classroom regarding student grades. Our findings are also in disagreement with those of [10,19,27], who indicated that most features of the virtual classroom had a positive impact on student learning. However, ironically COVID-19 that has caused a significant shift toward online learning, has also undoubtedly led to many pedagogical improvements, with instructors gaining more experience with that modality.

6. Conclusions

This study explored the effects of using different modes of learning on undergraduate learning outcomes during the initial stages of the COVID-19 pandemic in China’s mainland. We investigated the short-term impacts of the pandemic on student learning performance when using a virtual classroom, a conventional classroom, and a blended online and traditional classroom. In particular, we assessed the relationship between the use of a hybrid virtual/traditional classroom and undergraduate student learning outcomes, and examined their impact on the undergraduate learning process.
There are three outstanding problems that should be addressed to support the future use of virtual classroom technology to facilitate distance-learning. Namely, 70.54% of the respondents reported that having a better internet connection became the top priority for distance-learning. This was closely followed by the lack of interactions among teachers and students, and then by noise interference or distractions during online classes. Regarding the preference for the three aforementioned formats, the survey results indicated that the responses ranged between positive and negative: in particular, only 14% preferred the virtual classroom, which was the least preferred item among the three options. There was only one promising result in the pilot survey, namely, that 94.5% of respondents perceived that online class size had almost no impact on learning outcomes, which is dissimilar to the physical classroom in terms of a relatively fixed classroom capacity [40].
This study investigated and measured the causal effects between various learning modes and the corresponding outcomes of undergraduate students at a public university in China. Via a virtual classroom, the academic teacher can offer electronic textbooks and handouts and upload PowerPoint presentations and exercises synchronously; however, as mentioned above, as a result of poor network connection or instability and other unpredictable problems, student learning outcomes are less desirable. In comparison, the blended approach proved to be more effective than the virtual classroom alone in terms of learning outcomes. The results from the binary logistic regression model further revealed that, in contrast to many prevailing arguments, there was a statistically negative correlation between the virtual classroom environment and undergraduate student final grades in this study. Additionally, the marginal effect of mode on student learning performance in the form of an odds ratio was 0.327, which was lower than expected as compared to before the COVID-19 pandemic.
Although this research offers valuable findings regarding the causal relationship between undergraduate student learning outcomes and the associated learning environment while using a hybrid virtual/conventional classroom during the COVID-19 pandemic, it has certain limitations. First, the panel data concerning grades was obtained from one public university on the Chinese mainland during the COVID-19 pandemic. As such, the conclusions might not be representative, whether due to sample bias or the unique pandemic period. Second, reliability and validity evaluation of the pre-test survey was not performed due to the multiple-choice nature of two research questions in the pilot survey. For this reason, certain biases may exist within the results. In future studies, these aspects should be looked into.

Author Contributions

X.X. has the primary role in this manuscript. She is responsible for collecting the data, modeling of the data, and drafting the manuscript. S.S.’s contribution in the paper is revision and editing of the draft manuscript, and guidance and help with structural issues. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Data available on request from the authors. Data are not public for privacy reasons.

Acknowledgments

X.X. acknowledges the support from economics course teaching team of Business School, Qingdao University of Technology. S.S. acknowledges the support from the United States Department of Agricultural, National Institute of Food and Agriculture, Hatch project No. KY004052, under accession number 1012994. The authors would like to thank the editors and the reviewers for all their efforts.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Li, Q.; Guan, X.; Wu, P.; Wang, X.; Zhou, L.; Tong, Y.; Ren, R.; Leung, K.S.M.; Lau, E.H.Y.; Wong, J.Y.; et al. Early transmission dynamics in Wuhan, China, of novel coronavirus-infected pneumonia. N. Engl. J. Med. 2020, 382, 1199–1207. [Google Scholar] [CrossRef]
  2. Charles, D.; Kolstad. Market Failure: Public Goods, Public Bads, and Externalities. In Environmental Economics, 2nd ed.; Oxford University Press: New York, NY, USA, 2007; pp. 100–109. [Google Scholar]
  3. Garson, G.D. Evaluating implementation of web-based teaching in political science. PS Polit. Sci. Pol. 1998, 31, 585–590. [Google Scholar] [CrossRef] [Green Version]
  4. Xing, X.; Zhang, X.; Xu, M. Exploring the impact of COVID-19 on inward out circulation in the strategy of dual circulations. J. Indust. Tech. Econ. 2021, 8, 118–125. [Google Scholar]
  5. Steinmetz, H.; Batzdorfer, V.; Bosnjak, M. The ZPID Lockdown Measures Dataset for Germany. ZPID Sci. Inf. Online 2020. [Google Scholar] [CrossRef]
  6. Hiltz, S.R. The Virtual Classroom: Learning without Limits via Computer Networks; Ablex Publishing Corp: New Jersey, NJ, USA, 1994. [Google Scholar] [CrossRef]
  7. Cao, Q.; Griffin, T.E.; Bai, X. The importance of synchronous interaction for student satisfaction with course web sites. J. Inf. System. Educ. 2009, 20, 331–338. [Google Scholar]
  8. Lietzau, J.A.; Mann, B.J. Breaking out of the asynchronous box: Using web conferencing in distance learning. J. Libr. Inf. Serv. Dist. Learn. 2009, 3, 108–119. [Google Scholar] [CrossRef]
  9. Parker, M.A.; Martin, F. Using virtual classrooms: Student perceptions of features and characteristics in an online and a blended course. MERLOT J. Online Learn. Teach. 2010, 6, 135–147. [Google Scholar]
  10. Chowdhury, F. Virtual classroom: To create a digital education system in Bangladesh. Int. J. High. Educ. 2020, 9, 129–138. [Google Scholar] [CrossRef]
  11. Miltiadou, M.; Savenye, W.C. Applying social cognitive constructs of motivation to enhance student success in online distance education. Educ. Technol. Rev. 2003, 11, 78–95. [Google Scholar]
  12. Riel, M.; Harasin, L. Research perspectives on network learning. Mach. Med. Learn. 1994, 4, 91–113. [Google Scholar]
  13. Vrasidas, C.; McIsaac, M.S. Factors influencing interaction in an online course. Am. J. Dist. Educ. 1999, 13, 22–36. [Google Scholar] [CrossRef]
  14. Fidalgo, P.; Thormann, J.; Kulyk, O.; Lencastre, J.A. Students’ perceptions on distance education: A multinational study. Int. J. Educ. Technol. High. Educ. 2020, 17, 18. [Google Scholar] [CrossRef]
  15. Quadir, B.; Zhou, M. Student’s perceptions, system characteristics and online learning during the COVID-19 epidemic school disruption. Int. J. Dist. Educ. Technol. 2021, 19, 15–33. [Google Scholar] [CrossRef]
  16. Petchamé, J.; Iriondo, I.; Villegas, E.; Riu, D.; Fonseca, D. Comparing Face-to-Face, Emergency Remote Teaching and Smart Classroom: A Qualitative Exploratory Research Based on Students’ Experience during the COVID-19 Pandemic. Sustainability 2021, 13, 6625. [Google Scholar] [CrossRef]
  17. Yu, Z. The effects of gender, educational level, and personality on online learning outcomes during the COVID-19 pandemic. Int. J. Educ. Technol. High. Educ. 2021, 18, 14. [Google Scholar] [CrossRef] [PubMed]
  18. Virto, N.R.; López, M.F.B. Lessons from lockdown: Are students willing to repeat the experience of using interactive smartboards? Int. J. Emerg. Technol. Learn 2020, 15, 225. [Google Scholar] [CrossRef]
  19. Barr, B.A.; Miller, S.F. Higher Education: The Online Teaching and Learning Experience. Available online: https://files.eric.ed.gov/fulltext/ED543912.pdf (accessed on 11 April 2022).
  20. Van Wart, M.; Ni, A.; Rose, L.; McWeeney, T.; Worrell, R. A Literature Review and Model of Online Teaching Effectiveness Integrating Concerns for Learning Achievement, Student Satisfaction, Faculty Satisfaction, and Institutional Results. Pan-Pac. J. Bus. Res. 2019, 1, 1–22. [Google Scholar]
  21. Hauffman, H. A Review of Predictive Factors of Student Success in and Satisfaction with Online Learning. Res. Learn. Technol. 2015, 23, 2. [Google Scholar]
  22. Eom, S.; Wen, H.; Ashill, N. The Determinants of Students’ Perceive Learning Outcomes and Satisfaction in University Online Education: An Empirical Investigation. Decis. Sci. J. Innov. Educ. 2006, 2, 215–235. [Google Scholar] [CrossRef]
  23. Jung, I. The Dimensions of e-Learning Quality: From the Learner’s Perspective. Educ. Technol. Res. Develop. 2011, 4, 445–464. [Google Scholar] [CrossRef]
  24. McMurtry, K. Effective Teaching Practices in Online Higher Education. Ph.D. Thesis, Nova Southeastern University, Fort Lauderdale, FL, USA, 2016. [Google Scholar]
  25. Hung, J.L.; Hsu, Y.C.; Kerry, R. Integrating data mining in program evaluation of K-12 online education. J. Educ. Technol. Soc. 2012, 15, 27–41. [Google Scholar]
  26. Rizun, M.; Strzelecki, A. Students’ Acceptance of the COVID-19 Impact on Shifting Higher Education to Distance Learning in Poland. Int. J. Environ. Res. Public Health 2020, 17, 6468. [Google Scholar] [CrossRef] [PubMed]
  27. Sprenger, D.A.; Schwaninger, A. Technology acceptance of four digital learning technologies (classroom response system, classroom chat, e-lectures, and mobile virtual reality) after three months’ usage. Int. J. Educ. Technol. High. Educ. 2021, 18, 8. [Google Scholar] [CrossRef]
  28. Sharma, S.; Bumb, A. The Challenges Faced in Technology-Driven Classes During COVID-19. Int. J. Distance Educ. Technol. 2021, 19, 66–88. [Google Scholar] [CrossRef]
  29. Ali, A. Comparing Effectiveness of Online and Traditional Teaching Using Students’ Final Grade. Available online: https://core.ac.uk/download/pdf/60530022.pdf (accessed on 10 April 2022).
  30. Sondoozi, T. A Comparison of the Effectiveness of On-Line Teaching versus Traditional Teaching Methods in Higher Education: A Critical Review. 2000. Available online: https://www.learntechlib.org/p/126982 (accessed on 10 April 2022).
  31. Feng, X.; Ioan, N.; Li, Y. Comparison of the effect of online teaching during COVID-19 and pre-pandemic traditional teaching in compulsory education. J. Educ. Res. 2021, 114, 307–316. [Google Scholar] [CrossRef]
  32. Zhao, G.; Fan, M.; Yuan, Y.; Zhao, F.; Huang, H. The comparison of teaching efficiency between virtual reality and traditional education in medical education: A systematic review and meta-analysis. Ann. Transl. Med. 2021, 9, 252. [Google Scholar] [CrossRef] [PubMed]
  33. Goodlad, J. A Place Called School; McGraw-Hill Book Company: New York, NY, USA, 1984; pp. 13–14. [Google Scholar]
  34. Cuban, L. How Teachers Taught: Constancy and Change in American Classrooms, 1890–1990, 2nd ed.; Teachers College Press: New York, NY, USA, 1993. [Google Scholar]
  35. Cameron, A.C.; Trivedi, P.K. Microeconometrics Using Stata; Stata Corp LP: College Station, TX, USA, 2010; pp. 130–131. [Google Scholar]
  36. Guse, J.; Heinen, I.; Mohr, S.; Bergelt, C. Understanding mental burden and factors associated with study worries among undergraduate medical students during the COVID-19 pandemic. Front. Psychol. 2021, 12, 734264. [Google Scholar] [CrossRef] [PubMed]
  37. Alencar, M.A.D.S.; Netto, J.F.D.M. Improving Learning in Virtual Learning Environments Using Affective Pedagogical Agent. Int. J. Distance Educ. Technol. 2020, 18, 1–16. [Google Scholar] [CrossRef]
  38. Chen, C.; Meng, X. Exploring the Relationship Between Student Behavioral Patterns and Learning Outcomes in a SPOC. Int. J. Distance Educ. Technol. 2021, 19, 35–49. [Google Scholar] [CrossRef]
  39. Nicol, A.A.; Owens, S.M.; Le Coze, S.S.; MacIntyre, A.; Eastwood, C. Comparison of high-technology active learning and low-technology active learning classrooms. Act. Learn. High. Educ. 2018, 19, 253–265. [Google Scholar] [CrossRef]
  40. Xing, X.; Xi, A.; Zhao, X. Does university class size matter? Evidence from course micro data. In Proceedings of the 2019 International Conference on Education Research, Economics and Management, Singapore, 29–30 June 2019; pp. 97–100. [Google Scholar]
Table 1. Survey design.
Table 1. Survey design.
Survey Items
Q1. Devices used for distance-learning
Smartphone
Computer
Q2. Software chosen for distance-learning
Tencent Classroom
QQ Classroom
DingDing
Wisdom Tree
Rain Classroom
Other (Tencent Meeting, Webex Meet, etc.)
Q3. Impact of online class size on learning performance
Little impact
Somewhat impact
Great impact
Q4. Preference for virtual or traditional classroom
Traditional classroom
Virtual classroom
Hybrid virtual/traditional classroom
Q5. Prominent problems of distance-learning
Lack of interactions among teacher and students
Poor internet connectivity or/and instability
Noise distractions during class
Q6. Quality difference between online and traditional teaching
Little difference
Somewhat difference
Great difference
Table 2. Statistical descriptions of economics course data.
Table 2. Statistical descriptions of economics course data.
VariableObs.MeanStd. Dev.Min.Max.Skew.Kurt.
Course grade146266.46615.1871499−0.3922.739
Microeconomics in 2019–20 academic year79463.46613.1362493−0.2572.728
Macroeconomics in 2020–21 academic year66870.03116.6321499−0.7483.021
Table 3. Transformed dataset of microeconomics and macroeconomics.
Table 3. Transformed dataset of microeconomics and macroeconomics.
Treated group: Microeconomics grade dataset (N1)
Full sample: N = 1462; Subsample: N1 = 794.
Improved (grade = 1): 266;
Unimproved (grade = 0): 528
Controlled group: Macroeconomics grade dataset (N2)
Full sample: N = 1462; Subsample: N2 = 668.
Improved (grade = 1): 405;
Unimproved (grade = 0): 263
Grade SubtotalRatio
0791791/1462
1671671/1462
Table 4. Descriptions of dependent and independent variables.
Table 4. Descriptions of dependent and independent variables.
GradeModeTotal
Virtual ClassroomTraditional Classroom
0528263791
1266405671
Total7946681462
Table 5. Results of the pilot survey regarding students’ virtual classroom experiences.
Table 5. Results of the pilot survey regarding students’ virtual classroom experiences.
Survey ItemsCountPercent
Devices used for distance-learning (Q1)
Smartphone28641.51
Computer40358.49
Software chosen for distance-learning (Q2, multiple choice)
Tencent Classroom44965.17
QQ Classroom43362.84
DingDing35451.38
Wisdom Tree23233.67
Rain Classroom14821.48
Other (Tencent Meeting, Webex Meet, etc.)26738.75
Impact of online class size on learning performance (Q3)
Little impact44063.86
Somewhat impact21831.64
Great impact 31 4.5
Preference for virtual or traditional classroom (Q4)
Traditional classroom30243.83
Virtual classroom9814.22
Hybrid virtual/traditional classroom 28941.94
Prominent problems of distance-learning (Q5, multiple choice)
Lack of interactions among teacher and students38756.17
Poor internet connectivity or/and instability 48670.54
Noise distractions during class 25537.01
Quality difference between online and traditional teaching (Q6)
Little difference17825.83
Somewhat difference39757.62
Great difference11416.55
Table 6. Logistic regression results with controlling covariates.
Table 6. Logistic regression results with controlling covariates.
ModelCoefficientOdds Ratio
mode−1.117 ***0.327 ***
(0.109)(0.036)
constant0.432 ***1.540 ***
(0.079)(0.121)
Observations1462
Pseudo R square0.054
Note: *** indicates a 0.01 significance level. Robust standard errors are presented in parentheses.
Table 7. Logistic regression results without controlling covariates.
Table 7. Logistic regression results without controlling covariates.
ModelCoefficientOdds Ratio
mode−1.117 ***0.327 ***
(0.110)(0.036)
major−0.4990.607
(0.366)(0.222)
gender0.323 **1.381 **
constant0.682 *1.977 *
(0.368)(0.727)
Observations1462
Pseudo R square0.058
Note: ***, **, and * indicate a 0.01, 0.05, and 0.1 significance level, respectively. Robust standard errors are presented in parentheses.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Xing, X.; Saghaian, S. Learning Outcomes of a Hybrid Online Virtual Classroom and In-Person Traditional Classroom during the COVID-19 Pandemic. Sustainability 2022, 14, 5263. https://doi.org/10.3390/su14095263

AMA Style

Xing X, Saghaian S. Learning Outcomes of a Hybrid Online Virtual Classroom and In-Person Traditional Classroom during the COVID-19 Pandemic. Sustainability. 2022; 14(9):5263. https://doi.org/10.3390/su14095263

Chicago/Turabian Style

Xing, Xiufeng, and Sayed Saghaian. 2022. "Learning Outcomes of a Hybrid Online Virtual Classroom and In-Person Traditional Classroom during the COVID-19 Pandemic" Sustainability 14, no. 9: 5263. https://doi.org/10.3390/su14095263

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop