Cejun Cao^1*, Juan Li¹, Yuanfan Zhu¹, Ying Gong¹, Min Gao^2
1Department of Logistics Management, School of Management Science and Engineering, Chongqing Technology and Business University, Chongqing, China.
²Department of Marketing, School of Business Administration, Chongqing Technology and Business University, Chongqing, China.
DOI: 10.4236/jss.2020.87029   PDF    HTML   XML   935 Downloads   2,196 Views   Citations

Abstract

Different online teaching platforms are adopted in schools at all levels under the background of COVID-19 in China. In order to systematically analyze the factors that affect the selection of online teaching platforms, this paper establishes an online teaching platform evaluation system. Then, five main factors are identified after reviewing a series of factors that has significant influences on selection of online teaching platforms in terms of course entitled “Third-party Logistics Management”. A hierarchical structure model for online teaching platform selection is established based on Analytic Hierarchy Process. In particular, in terms of each layer, the weights of the indicators are calculated and checked so as to obtain the total ranking, thus finding the ultimately best scheme. Results indicated that the order of importance concerning evaluation indicators of online teaching platforms was as follows: internet stability, teaching performance, operational convenience, course resources and interactive effect. Additionally, WeChat Work was identified as the best platform for teachers and students.

Keywords

COVID-19, Evaluation of Online Teaching Platform, Analytic Hierarchy Process

Share and Cite:

1. Introduction

COVID-19 (Coronavirus Disease 2019) as a global epidemic quickly spreads to all over the world. In this context, the spring semester of 2020 is generally delayed by schools at all levels. In response to the slogan entitled “ensuring learning undisrupted when classes are disrupted” proposed by the Ministry of Education of the People’s Republic of China, all schools encouraged the teachers and students to implement online teaching. Nevertheless, different platforms have different advantages and disadvantages. Therefore, how to adopt an appropriate and efficient method to evaluate online teaching platforms is an urgent problem. Furthermore, it is also a hot topic in the field of online education under the background of new era.

In recent years, a plenty of scholars at home and abroad have made the meaningful exploration on online teaching problems such as evaluation, thus obtaining some achievements. For example, Terrazas-Arellanes et al. (2019) applied web-based professional development model to evaluate the performance of teaching, and upgrade online research skills of the students. Alturkistani et al. (2019) explored data collection methods to evaluate massive open data courses. Martin et al. (2019) proposed timely response and feedback on the availability and presence of online teaching process and regular communication with students, which were the effective ways to increase online teaching practice. Yeh et al. (2014) used KAP questionnaires to evaluate traditional and online teaching modes. Besides, they concluded that the integrated teaching modes would lead to a better practical teaching. Carol & Eileen (2014) clarified that peer evaluation forms could be used to evaluate teachers’ online teaching from the perspective of teachers. Besides, the form used for online assessment is considered to be relevant, reasonable, feasible and beneficial to online teaching. Bangert (2008) investigated the exploratory factors affecting online teaching from the viewpoint of online teaching. Results indicated that teacher-student interaction, active learning, task time, and cooperation among students were four interpretable factors. Ryan et al. (2005) employed dimension matrix to explain the scope of online teacher teaching in order to improve efficiency. The evaluation of online teaching is of great significance.

According to the overview of the existing literature, it is reported that there are various dimensions and methods to evaluate online teaching, such as the quality of teaching, online learning management, student learning outcomes, student satisfaction and system design. However, most of them merely concentrate on one dimension, which is not comprehensive due to the complex relationship among these factors. To make up for this gap, this paper uses Analytic Hierarchy Process (AHP) to establish a hierarchical structure model for online teaching platform selection, and analyze the hierarchical relationship among the factors that influence the output of the platform, students, and teachers. The ultimate goals of this paper are to select the best one from all online teaching platforms, provide the teachers and students with the high-quality environment in terms of online teaching, and upgrade the performance of online teaching. Finally, the course titled “Third Party Logistics Management” is considered as an example to verify the feasibility and effectiveness of the proposed research program.

The structure of this paper is as follows. Section 2 describes the research method regarding AHP. The indicators to evaluate online teaching platforms are presented in Section 3. The specific procedure of the application of AHP into assessment of online teaching plat-forms is discussed in Section 4. Section 5 concludes this paper and gives the suggestions.

2. Research Method

The focus of this paper is the evaluation of online teaching platforms. Because of a series of complex factors and their mutual relationship, this paper uses AHP that combines qualitative and quantitative approach to assess the different online teaching platforms.

2.1. Analytic Hierarchy Process

AHP is proposed by Saaty in the early 1970s who is an American operations researcher. It aims to solve complex multi-objective decision-making problems. This method decomposes such problem into target layer, criterion layer and scheme layer. By calculating the weights of indicators for the decision of the elements in the upper layer, the combined weights are finally obtained. The best one can be selected from the different online teaching platforms according to the combination of weights.

2.2. Research Procedure

The specific procedure of application of AHP into the selection of online teaching platforms includes the following two stages in this paper. At the first stage, a hierarchical structure model is established. In particular, decomposition of complex multi-objective problems, identification of main factors that affect the performance of online teaching platforms, as well as their mutual relationship are considered. Accordingly, a multi-level structural model is constructed. At the second stage, the influences of the weights of elements for each level on that of those on the upper level is calculated. Particularly, this stage can be subdivided into the following three steps.

2.2.1. Construct the Judgment Matrix at Each Level

In regard to each level, multiple factors are compared in pairs and scored according to 1 - 9 scale method (Table 1). In terms of two factors, if one (marked 1) is obviously more important than another (marked 2), and then the corresponding element (a₁₂) in the judgment matrix is filled with 5. In a similar way, all judgment matrixes can be obtained by using this rule.

2.2.2. Calculate Eigenvalues and Eigenvectors

With respect to AHP, the most important step is to find the largest eigenvalue, which provides a foundation for the weights of the indicators. In this paper, the square root method is used to calculate the weights. Let A denote the judgment matrix which can be written as follows: $A={\left(a_{ij}\right)}_{n\times n}$.

1) Calculate the weight of each index.

$M_i=a_{11}{a}_{12}{a}_{13}\cdots a_{1n}$ (1)

${\omega }_i=\sqrt[n]{M_i}$ (2)

2) Normalize ${\omega }_i$ to get the standard weight ( ${\omega }_i^0$ ) of each index.

${\omega }_i^0={\omega }_i/{\displaystyle \underset{i=1}{\overset{n}{\sum }}{\omega }_i}$ (3)

3) Calculate the largest characteristic root of the judgement matrix.

${\lambda }_i={\left(A\omega \right)}_i/{\omega }_i$ (4)

${\lambda }_{\max }=\frac{1}{n}{\displaystyle \underset{i=1}{\overset{n}{\sum }}{\lambda }_i}$ (5)

2.2.3. Consistency Check

It is noting that inconsistent results may be obtained due to the subjectivity of the experts. As a result, a consistency test is required. Especially, the judgment matrix will pass the consistency test if the consistency ratio is less than 0.1. In other words, it indicates that the weights of the indicators obtained in this way are effective. Otherwise, the judgment matrix should be adjusted until the consistency test passes. The relevant calculation formula of the consistency check is as follows.

$CI=\frac{{\lambda }_{\max }-n}{n-1}$ (6)

In addition, the value of random consistency index (RI) is always known, which is depicted in Table 2.

Finally, consistency ratio can be obtained based on the following formula.

$RC=CI/RI$ (7)

3. Construction of the Indicators of Online Teaching Platforms Selection Regarding Third-Party Logistics Management

3.1. Definition of Research Boundary

In this paper, Third-party Logistics Management in Chongqing Technology and Business University is treated as the case. According to the survey, it is found that QQ group classroom (B₁), WeChat Work (B₂), and Tencent classroom (B₃) are the mainstream online teaching platforms in Chongqing Technology and Business University. It is universal that all of the platforms include live teaching, online discussion, online interaction, and other functions. However, different platforms hold different advantages and disadvantages. Specifically, QQ group classroom relies on QQ software, which can conveniently download, save mobile phone space, and others. In terms of WeChat Work, it records all information related to all personnel including teachers, managers and students. It regards university as a whole and automatically establishes different groups which contain administrative classes and teaching ones. Such action is conducive to the acquisition of course resources, as well as interaction between teachers and students. Meanwhile, it also has a variety of teaching modes, such as live broadcast, conference, voice and video. With respect to Tencent Classroom, it has an excellent monitoring system that can supervise students, thus improving teaching effectiveness. In this paper, QQ group classroom, WeChat Work and Tencent classroom are considered as the candidates for teachers and students. And then, the establishment of an evaluation system for online teaching platforms is the focus. The aim of this paper is to choose the best platform from the alternatives.

3.2. Construction of the Indicators to Evaluate Online Teaching Platforms

It is reported that there are many factors to affect the performance of online teaching platforms. It includes the following three aspects: teachers, students, and platforms. For instance, Kumar et al. (2010) incorporated micro-course resources, teaching process, teaching performance and so on into the intelligent teaching evaluation index system. Liu & Yu (2019) clarified that interaction effect is another major factor that affects online teaching performance. Priority should be attached to a live broadcast platform with classroom real-time interaction. It is beneficial to restore the interactive scenes between teachers and students through online testing and topic discussion. In this context, it is possible to supervise the behaviors of the students, stimulate the interests of the students, and upgrade the performance of online-study. Lan et al. (2003) claimed that there was a negative correlation between teaching evaluation and the experience concerning technical issues. Technical issues exert influences on the evaluation of the students on online teaching platform selection. Similarly, both operational convenience and internet satiability also play an indispensable role in the selection of online teaching platforms. In particular, internet stability is the foundation of successful implementation of online teaching inpractice, the teaching platform should be changed in time if teachers or students have some troubles in login, caton, system crash, and others.

According to the foregoing description, five main factors from three perspectives to affect the performance of online teaching platforms are identified. It includes operational convenience (A₁), internet stability (A₂), interaction effect (A₃), course resources (A₄), teaching effect (A₅), which is depicted in Table 3.

4. The Application of AHP into the Evaluation of Online Teaching Platforms regarding Third-Party Logistics Management

4.1. A Hierarchical Structure Model for Online Teaching Platforms Assessment Concerning Third-Party Logistics Management

In this context, the evaluation indicators are summarized from the perspectives of platforms, students and teachers. And then, the task regarding the best platform selection of online teaching is selected as the goal (Z). Therefore, a hierarchical structure model is constructed. It is depicted as Figure 1.

4.2. Weights of the Indicators and Their Consistency Checks

After establishing the hierarchical structure model of online teaching platforms regarding Third-party Logistics Management, the corresponding judgement matrixes should be constructed. Particularly, the scores of each judgement matrix are obtained by experts. That is, each element of each judgement matrix is obtained by comparing the importance of one index to another on the same level. Besides, 1 - 9 scale method is considered.

According to formulas (1) to (4), the weights and characteristic roots of each indicator are obtained. It is noting that only the first indicator is regarded as an example. The corresponding values is depicted in Table 4.

And then, a consistency check is required. According to formulas (5) to (7), the largest characteristic root and the consistency ratio can be obtained.

${\lambda }_{\max }=\frac{1}{5}\times \left(5.031+5.076+5.101+5.031+5.044\right)=5.057$

$CI=\frac{5.057-5}{5-1}=0.0143$

$CR=\frac{0.0143}{1.12}=0.0128<0.1$

The aforementioned results indicate that the consistency check passed. In a similar way, the related values of all judgement matrix can be obtained, which is depicted in Table 5.

4.3. Combined Weights of the Indicators and the Sequence of Schemes

Table 4 and Table 5 present the weights and consistency ratio, and other parameters. Thus, total rankings of all the schemes can be calculated. It is shown in Table 6.

Table 6 demonstrates that WeChat Work is the best platform for teachers and students to implement online teaching activities. The second and third choice are respectively QQ group classroom and Tencent classroom. With regard to the combined weights of the indictors, consistency check is also needed. It can be calculated based on Equation (8).

$CR=\frac{{\displaystyle {\sum }_{i=1}^n{b}_{i}C{I}_i}}{{\displaystyle {\sum }_{i=1}^n{b}_{i}R{I}_i}}$ (8)

Wherein, $b_i$ is the weight of indicator $i$ on the criterion layer. $C{I}_i$ and $R{I}_i$ respectively present the consistency index and random consistency index. As a consequence, the procedure of consistency check for total rankings is as follows.

${\displaystyle \underset{i=1}{\overset{n}{\sum }}{b}_{i}C{I}_i}=\left(0,0,0,0.0045,0.0045\right){\left(0.144,0.405,0.058,0.143,0.250\right)}^T=0.0018$

${\displaystyle \underset{i=1}{\overset{n}{\sum }}{b}_{i}R{I}_i}=\left(0.52,0.52,0.52,0.52,0.52\right){\left(0.144,0.405,0.058,0.143,0.250\right)}^T=0.52$

$CR=0.0018/0.52=0.0035<0.1$

It is obvious that the results indicate that the consistency check of total rankings passed. In other words, the obtained weights and ranking of all schemes are desirable.

5. Conclusion and Suggestions

In this paper, AHP is used to select and evaluate different online teaching platforms regarding Third-party Logistics Management. Different factors that affect the performance of online teaching platforms are ranked as follows: internet stability, teaching effect, operational convenience, teaching resources and interaction effect. These weights provide teachers and managers with a basis to assess the pros and cons of the three online teaching platforms. In addition, the priority of the three online teaching platforms is as follows: WeChat Work, Tencent classroom, QQ group classroom.

5.1. Internet Stability Is the Most Important Factor to Evaluate the Performance of Online Teaching Platforms

Results show that internet stability is the most critical factor to assess the performance of online teaching platforms because of its high-level weight which reaches 40 percent. Furthermore, the weight of teaching effect is ranked as the second. In summary, total proportions reach 65 percent. It demonstrates internet stability and teaching effect play an indispensable role in the evaluation of performance of online teaching platform, which aligns with the results of survey conducted by us in practice. In terms of operational convenience and course resources, the weights of them are about 14 percent. However, interaction effect merely occupies 5.8 percent, which indicates this factor is ignored in current evaluation system of online teaching platforms. From what has been discussed above, the following conclusion can be reached. More attention or importance should be attached to the interaction effect between teachers and students on online teaching platforms. Interaction effect can not only enable students to have a better learning performance, but also make teachers bridge the gaps. It pushes teachers to create innovative teaching and interactive methods in the context COVID-19. Case introduction, game interaction, story attraction, and a series of interaction methods to improve students’ classroom participation and enthusiasm are encouraged.

With regard to online teaching, internet stability is a prerequisite for students’ learning and teachers’ teaching. Internet stability depends on not only the platform itself but also the number of real-time online participators who are difficult to control. At this time, it is extremely important for the teachers to prepare the alternatives. In order to ensure the internet is always stable, the following suggestions are given. Firstly, large-scale enterprise platforms with continuous stability are encouraged to choose so as to reduce stutters, flashbacks, and black screens. Secondly, teachers are suggested to prepare various alternatives in accordance with the internet situation when they adopt online teaching. For instance, audio, graphic and other methods to online teaching can be chosen if live forecast is unavailable. Thirdly, the integrated platform of online teaching and high-quality MOOC video is recommended so that the goals of staggered live broadcast and stable internet can be achieved.

5.2. WeChat Work Is the Best Online Teaching Platform for Third-Party Logistics Management

As mentioned above, WeChat Work is the best choice for online teaching of third-party logistics management in Chongqing Technology and Business University. In addition, QQ group live broadcast is the worst. WeChat Work that is created or developed for enterprise to manage official affairs outperforms other platforms in the aspects of internet stability, teaching effect, operational convenience, teaching resources, and interaction effect (Kim et al., 2020). At the same time, administrative classes and teaching ones are automatically established in WeChat Work, which is conducive to the interaction between students and teachers, and the sharing of teaching resources. Nevertheless, QQ group classroom is built on QQ software. Although it operates easily for both teachers and students, it is widely used to chat online for students, which gives rise to an unexpected performance of teaching. Besides, according to the recent survey, the internet stability of QQ group classroom is also relatively worse. In regard to Tencent Classroom, the biggest disadvantage is that it cannot establish a learning group. In addition, teachers and students cannot communicate with each other and share course resources. However, it has significant advantages in internet stability and teaching effect. In summary, the course entitled “Third-Party Logistics Management” is recommended as being taught on WeChat Work. Meanwhile, Tencent Classroom can be regarded as the alternative. It is noting that the proposed methodology also could be used to evaluate other courses in different universities.

Acknowledgements

This paper was supported by Education and Teaching Reform Research Project of Chongqing Municipal Education Commission Grant No. 171606, Graduate Education and Teaching Reform Research Project of Chongqing Technology and Business University Grant No. 2015YJG0106.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1]	Alturkistani, A., Majeed, A., Car, J., Brindley, D., Wells, G., & Meinert, E. (2019). Data Collection Approaches to Enable Evaluation of a Massive Open Online Course About Data Science for Continuing Education in Health Care: Case Study. JMIR Medical Education, 5, e10982. https://doi.org/10.2196/10982
[2]	Bangert, A. W. (2008). The Development and Validation of the Student Evaluation of Online Teaching Effectiveness. Computers in the Schools, 25, 25-47. https://doi.org/10.1080/07380560802157717
[3]	Carol, D. G., & Eileen, K. (2014). Development, Evaluation, and Utility of a Peer Evaluation Form for Online Teaching. Nurse Educator, 39, 22-25. https://doi.org/10.1097/NNE.0000000000000007
[4]	Kim, D., Lee, Y., Leite, W. L., & Huggins-Manley, A. C. (2020). Exploring Student and Teacher Usage Patterns Associated with Student Attrition in an Open Educational Resource-Supported Online Learning Platform. Computers & Education, 156, Article ID: 103961. https://doi.org/10.1016/j.compedu.2020.103961
[5]	Kumar, S., Perraton, L., & Machotka, Z. (2010). Development and Implementation of an Online Hybrid Model for Teaching Evidence-Based Practice to Health Professions: Processes and Outcomes from an Australian Experience. Advances in Medical Education and Practice, 1, 1-9. https://doi.org/10.2147/AMEP.S12110
[6]	Lan, W., Tallent-Runnels, M. K., Fryer, W., Thomas, J., Cooper, S., & Wang, K. (2003). An Examination of the Relationship between Technology Problems and Teaching Evaluation of Online Instruction. The Internet and Higher Education, 6, 365-375. https://doi.org/10.1016/j.iheduc.2003.07.003
[7]	Liu, Y. H., & Yu, F. Y. (2019). Supporting Active Learning and Formative Evaluation via Teaching-by-Questioning in Classrooms: Design, Development, and Preliminary Evaluation of an Online Learning System. Interactive Learning Environments, 27, 841-855. https://doi.org/10.1080/10494820.2018.1489858
[8]	Martin, F., Ritzhaupt, A., Kumar, S., & Budhrani, K. (2019). Award-Winning Faculty Online Teaching Practices: Course Design, Assessment and Evaluation, and Facilitation. The Internet and Higher Education, 42, 34-43. https://doi.org/10.1016/j.iheduc.2019.04.001
[9]	Ryan, M., Hodson-Carlton, K., & Ali, N. S. (2005). A Model for Faculty Teaching Online: Confirmation of a Dimensional Matrix. The Journal of Nursing Education, 44, 357-365. https://doi.org/10.3928/01484834-20050801-06
[10]	Terrazas-Arellanes, F. E., Strycker, L. A., & Walden, E. D. (2019). Web-Based Professional Development Model to Enhance Teaching of Strategies for Online Academic Research in Middle School. Journal of Research on Technology in Education, 51, 118-134. https://doi.org/10.1080/15391523.2018.1564637
[11]	Yeh, Y. T., Chen, H. Y., Cheng, K. J., Hou, S. A., Yen, Y. H., & Liu, C. T. (2014). Evaluating an Online Pharmaceutical Education System for Pharmacy Interns in Critical Care Settings. Computer Methods and Programs in Biomedicine, 113, 682-689. https://doi.org/10.1016/j.cmpb.2013.11.006