Determinants of Learning Management Systems during COVID-19 Pandemic for Sustainable Education

Abstract

Research has shown that effective and efficient learning management systems (LMS) were the main reasons for sustainable education in developed nations during COVID-19 pandemic. However, due to slow take-up of LMS many schools in developing countries, especially Africa were completely shut down due to COVID-19 pandemic. To fill this gap, 4 AI-based models; Support Vector Machine (SVM), Gaussian Process Regression (GPR), Artificial Neural Network (ANN), and Boosted Regression Tree (BRT) were developed for prediction of LMS determinants. Nonlinear sensitivity analysis was employed to select the key parameters of the LMS determinants data obtained from 1244 schools’ students. Five statistical indices were used to validate the models. The performance results of the four developed AI models discovered facilitating conditions, attitude towards LMS, perceived enjoyment, users’ satisfaction, perceived usefulness, and ease of use to be the most significant factors that affect educational sustainability in Nigeria during COVID-19. Further, single model’s performance results comparison proved that SVM has the highest prediction ability compared to GPR, ANN, and BRT due to its robustness in handling data uncertainties. The study results identified the factors responsible for total schools’ closure during COVID-19. Future studies should examine the application of other linear and other nonlinear AI techniques.

1. Introduction

Nowadays, emerging technologies have continued to change the face at which activities are being carried out in all aspect of human endeavours. For instance, the ways and manners in which colleges and universities now access and used various information and communication technologies (ICTs) resources in advanced and few developing nations have impacted positively on both teaching delivery and learning strategies [1,2,3]. In recent past, some colleges and other higher institutions of learning have succeeded in extending their modes of knowledge conveyance to include full online interactions between teacher and students to support the usual traditional methods of teaching and learning, i.e., face-to-face method as evidently showed in prior studies [4]. In recent years, the world has witnessed a tremendous increase on acceptance and usage of different Learning Management Systems (LMS) in tertiary institutions as colleges and universities in advanced and few developing nations have successfully deployed various online education platforms [5] that allow students to take their lectures online, collaborate with one another, and have access to learning materials using any device, anywhere, and anytime via the Internet [6,7]. These systems are usually integrated with other multimedia tools such as video, text, and audio [8], and communication tools, e.g., email, chat and discussions forums, and assessment tools [9]. Despite these benefits of LMS, hundreds of thousands of colleges and universities in most developing countries, especially in Africa were completely shut down due to COVID-19 pandemic. As of 25 May 2020, UNESCO stated that out of the total enrolled learners in 130 countries, there has been 990,324,537 affected learners across all level of learning which is (56.6%) of the world total enrolled learners due to school closures caused by COVID-19 pandemic [10]. However, tertiary institutions in countries such as US [11], UK [12], Germany [13], France [14], Cyprus [15], and Malaysia [16] have successfully migrated all their teaching and learning activities to an online environment using various LMS as the normal traditional method of face-to-face was disrupted due to schools’ closures caused by COVID-19.

The interruption of usual teaching and learning approaches (i.e., face-to-face methods) caused by COVID-19 pandemic have impacted negatively on sustainable development of the educational sector across the globe, especially in developing countries where schools were completely closed during the pandemic period. Thus, the needs for all educational institutions to adopt different eLearning technologies such as LMS as the only option for ensuring educational sustainability during pandemic periods [5]. Furthermore, the abrupt appearance of COVID-19 and other control measures imposed by governments all over the world, also posed serious challenges concerning LMS usage [5,7,15]. In addition to the usual LMS challenges, some of these unusual challenges has to do with the need to assess users’ attitude towards LMS, perceived enjoyment, satisfaction, and informed decision on users prepared LMS to be deployed for continued education.

Though, there is rapid increased on the adoption and usage of LMS across the Globe, the story is different in Nigeria [17], and other West African countries [18] as adoption and usage of various technology-based education systems in the region continued to fail. For instance, in Nigerian colleges and universities were completely closed for over six months due to COVID-19 pandemic. Just like several developing nations, the take up of LMS in Nigeria has been very slow as only few colleges and universities have incorporated LMS for academic purposes [19]. These and other problems led to system void and anomalies which in turn affect the sustainable development of educational activities in the country [20]. Furthermore, the limitation of prior studies that examined LMS success factors, failure and/or challenges in Nigerian and other developing nations, e.g., see [17,19,21,22], is the studies inability to used artificial intelligence-based (AI) techniques to investigate the determinants of LMS success for sustainable educational activities during COVID-19 pandemic.

The aim of this study is to investigate the factors that affects LMS success for sustainable education during COVID-19 pandemic in developing countries with special attention to Nigerian higher education, using four AI models, i.e., Support Vector Machine (SVM), Artificial Neural Network (ANN), Gaussian Process Regression (GPR), and Boosted Regression Tree (BRT) to obtained more accurate results. Literature have shown that different approaches may provide dissimilar results for a particular problem [23]. Thus, outputs from the study AI models (SVM, GRP, ANN, and BRT) may likely produce results with little error variance compared to other classical models used in eLearning domain [24]. Moreover, the study results will assist schools’ administrators, education authority, and other stakeholders in Nigeria particularly, and other developing nations in understanding the factors responsible for slow growth of LMS and deploy effective LMS for sustainable education during pandemic periods. Furthermore, the study ensemble model will serve as a motivation for researchers to embrace AI techniques for investigation purposes due to its reliability, flexibility, and accuracy in making prediction above other regression models [25,26].

2. Materials and Methods

2.1. Learning Management Systems (LMS) and Challenges

LMS comprises the use of web-based technologies for knowledge acquisitions and sharing [27]. It is also defined as a tool that provides e-learning podium that uses extranet, intranet, and internet networks to spread learning resources and support other administrative activities related to teaching and learning [28,29]. Additionally, LMS enable learners to establish their teaching rhythm and to acclimatize the learning process in order suit their personal needs. Furthermore, LMS aid instructors to send learning materials and follow students’ evaluation and allow learners to gain access to learning tools for their web-based education [30].

According to Dias [2] and Ghazal [31], expediency and suppleness are the two most valuable features of LMS. Though, LMS provides several opportunities and benefits to education stakeholders across the globe, there is need also to examine the issues and challenges associated with its deployment, acceptance, use, and management [32], in developing nations especially, in African nations where colleges and universities were completely shut down due to schools’ closures caused by COVID-19 pandemic [20]. Moreover, García [33] examined the factors affecting LMS implementation and use in Caribbean developing nations. The authors found individuals’ social norm, limited access to computers, and systems availability to be the major challenging factors that affect LMS implementation and use in Caribbean English-speaking nations. The authors argued that these factors significantly influenced perceived usefulness and perceived ease of use which in turn affects LMS acceptance and development. Similarly, El-Masri [34] found societal influence and effort anticipation to be the main factors affecting LMS progress in most developing nations but not in advanced nations. Furthermore, they authors argued that availability of resources that facilitate LMS usage such as internet speed and penetration, constant power supply, access to computers and systems availability were the main reasons for rapid growth of LMS in advanced nations.

Abugre [18] argued that infrastructural deficit, feebleness LMS policies, excess teaching loads, students’ congestion in academic facilities, and lack of access to computers and systems availability were the major factors responsible for slow adoption and continual usage of LMS in most African higher institutions. Thus, advocates the needs for clear LMS policies that can improve e-learning acceptance and usage in tertiary institutions. Similarly, Mtebe [35] claimed that course content and systems qualities have significance influence on students’ attitude towards LMS as well as on their intention to continual usage. Thus, the needs for tutors to develop a high-quality course contents that are in line with students’ knowledge, abilities, and skills to increase students’ satisfaction with the systems which in turn enhance LMS usage. Furthermore, Yakubu [17] stressed the importance of tutor quality and learning value on students’ acceptance and use of LMS. The authors found learning value and tutor quality to have positive and significant effects on behavioral intent to use LMS in Nigeria.

2.2. Artificial Intelligence (AI) Techniques

Factors that affect eLearning technologies (LMS) development in both developed and developing states are usually measured using empirical models such as covariance, least square, and partial least square structural equation modelling, to mention but few. However, predictions based on these approaches are time consuming, bias-based, and sometimes produce inaccurate and unreliable results [36,37]. Thus, the need to employs AI techniques such as SVM, GPR, ANN, and BRT in order to overcome the limitations of the conventional models [23]. These techniques provide opportunities for scholars in engineering, science, and social sciences to validate research models [38]. Moreover, Mohamadnejad [39] argued that AI techniques had proven to have greater performance in research. Similarly, Yang [40] emphasized the importance of AI techniques for complex nature research like peoples’ attitudes, behaviors, and emotions due to their robustness, flexibilities, and generalization abilities in terms of predictions. For this reason, in this study, four AI-based techniques, namely, SVM, GPR, ANN, and BRT were employed so that reliable and accurate prediction of LMS determinants in Nigeria can be obtained.

2.3. Technology Acceptance and Adoption Models in eLearning

In recent past, a large number of studies have been carried out, using different hypothetical models in order to have clear foresight on issues that affect users’ acceptance, use and continual usage of technology. These models include Davis [41] “Technology Acceptance Model” (TAM); Delone [42] revised model of “Information Systems Success” (D&M Model); Venkatesh [43] updated “Unified Theory of Acceptance and Use of Technology” (UTAUT2); Zhang [44] reviewed “Innovation Diffusion Theory (IDT) for IT adoption, Smith [45] revised “Theory of Planned Behaviour” (TPB); and Fishbein [46] updated “Theory of Reasoned Action (TRA). This research employs three of these models’ (i.e., D&M, UTAUT2, and TAM) constructs and recommendations from prior LMS studies to investigate factors that affect learning management systems (LMS) adoption and use in developing countries, especially in Nigerian where colleges and universities were completely shut down due to COVID-19 pandemic [20].

In the context of web-based learning, Davis [41] technology acceptance model (TAM) is among the popular models used by researchers to offer judgement on users’ perception, acceptance, and attitudes towards technologies. Several studies have used TAM to investigate user acceptance and use of LMS, e.g., see [17,47,48,49]. Venkatesh [43] revised UTAUT2 is another model commonly used to offer opinion on users’ acceptance and intention to use new technology. In relation to LMS, UTAUT2 has also been used extensively to examine the factors that affects LMS acceptance due to TAM limitations with regards to explaining people acceptance of technology. Three additional variables of habit, hedonic motivation, and price value were integrated into the UTAUT2 which makes it more user centered than TAM. However, the price value construct was later replaced with Learning value by Ain [50] to further explain individuals’ adoption of technology from LMS perspectives as monetary value could not be attributed to LMS usage [34]. Another model frequently used in eLearning domain is Delone [42] revised model of information systems success. Just like TAM and UTAUT2 models, D&M model has also been used by scholars to explain the importance and influence of systems reliability, usability, efficiency, functionality and maintainability on administrators’, students’, and tutors’ acceptance and use of various LMS [51].

2.4. Methodology

Research Model

In contrast to other studies in LMS literature, D&M, TAM, and UTAUT2 models’ constructs were used in this study to improve the robustness and applicability of the study proposed model. The study conceptual model consists of eight independent variables (Perceived Enjoyment (PE), Attitude Towards Technology (ATT), Perceived Usefulness (PU), Perceived Ease of Use (PEOU), Systems Quality (SQ), User Satisfaction (US), Facilitating Conditions (FC), and Social Influence (SI), and one mediating variable of (Behavioral Intentions, BI), while LMS actual usage represent the dependent variable as shown in Figure 1.

2.5. Hypotheses Formulation

2.5.1. Perceived Enjoyment

Perceived enjoyment (PE) as for this study, is to examine the effects of enjoyable aspects of eLearning technologies actual usage. In the context of LMS, Moorthy [52] defined perceived enjoyment as the degree to which users perceived different LMS to be enjoyable. It is considered as one of the strongest factors that influence behavioral intent to accept and use new technology [27,47]. Furthermore, Wasko [53] argued that perceived enjoyment has a positive relationship with knowledge acquisition in digital setting. Incorporation of perceived enjoyment construct into the research AI model, makes the present study more important, and to the best of our knowledge none of the prior LMS studies in Nigerian and other developing countries investigate this factor using AI techniques, as well as D&M, TAM, and UTAUT2 models did not cite it. Thus, hypothesize that

Hypothesis 1 (H1).

Perceived enjoyment will have a positive and significant influence on behavioral intent to use LMS in Nigerian context during COVID-19 pandemic.

2.5.2. Attitude towards Technology

Magen-Nagar [54] in their study examined the impact of online learning technologies in Israel. The authors found fear of using technological gadgets, lack of technological self-possession, and negative attitudes towards eLearning and other collaborative learning tools to be the main reasons for Israeli colleges students’ and tutors’ phobia for online courses. This factor, attitude towards technology (ATT), was added to the study model as per Yakubu [17] and Jacob [20]. The authors argued that deployment and use of LMS in Nigeria, just like other developing nations have continued to fail, thus the need for future LMS studies to investigate students’ attitude towards LMS acceptance and use. Thus, hypothesized:

Hypothesis 2 (H2).

Attitude towards technology will have a positive and significant influence on behavioral intent to use LMS in Nigerian context during COVID-19 pandemic.

2.5.3. Perceived Usefulness

This construct, perceived usefulness (PU), is sourced from TAM, and is similar to UTAUT construct of performance expectancy (PE). Davis [41] defined perceived usefulness as the degree to which persons believe that deploying a specific technology would increase their job performance. Similarly, Baki [55] stressed that the higher the degree of students’ belief in LMS, the better the learning performance and grades. Furthermore, prior LMS studies argued that PU may likely have positive and significant influence on users’ behavioral intent to accept and use LMS [50]. Learners will be willing to adopt and use a particular eLearning technology if they believe that the technology will assist them in achieving their educational goals. Based on prior literatures, we assume that PU will have a significant influence on learners’ behavioral intent to accept and use different LMS in Nigeria. Thus, we pose the following hypothesis:

Hypothesis 3 (H3).

Perceived usefulness will have a positive and significant influence on behavioral intention to use LMS in Nigerian context during COVID-19 pandemic.

2.5.4. Perceived Ease of Use

Just like PU, perceived ease of use (PEOU) is also sourced from TAM. In the context of web-based learning technologies, a lot of effort has been devoted towards the creation of an interface that is user-friendly, in acknowledging the importance of perceived ease of use construct [56]. PEOU is defined as the extent to which an individual believes that utilization of a particular web-based technology such as LMS would be free of mental and/or physical effort. Joo [57] argued that PEOU has significant influence on students’ behavioral intention to use LMS. Thus, LMS should be designed in an intuitive manner, and the systems should not require technical expertise before usage. Thus, hypothesized that

Hypothesis 4 (H4).

Perceived ease of use will have a positive and significant influence on behavioral intent to use LMS in Nigerian context during COVID-19 pandemic.

2.5.5. System Quality

System quality (SQ), as for this study, is to inspect the effect of system quality on LMS acceptance and use and was derived from D&M model of IS success. Ramírez-Correa [58] described the concept as the performance of a particular software (in this case, LMS) in terms of convenience, functionality, reliability, ease of use, and other software metrics. Similarly, it has been argued that enhancements in system quality would increase system usage as well as satisfaction with the system [42,59]. According to study by Syed [60] conducted in Saudi Arabia, system quality has positive relationship with behavioral intent to use LMS. Thus, the study proposes the following proposition:

Hypothesis 5 (H5).

Systems quality will have a positive and significant influence on behavioral intent to use LMS in Nigerian context during COVID-19 pandemic.

2.5.6. User Satisfaction

Yuen [61] in their study defined user satisfaction (US) as the extent to which students believe that the available information systems (IS) meet their educational and other information requirements. A lot of scholars believe that the higher the satisfaction level derived from a particular IS, the higher the usage of such system [62]. Conversely, not meeting users’ information requirements by the system will lead to dissatisfaction amongst users [63]. In the context of LMS, prior studies [33,35,50] have shown that many students, especially higher education students from developing nations are not satisfied with the online classroom due to lack of technical skills, unfamiliarity with the environment, and personality type. Thus, affecting students’ behavioral intent to use LMS. Therefore, the study hypothesized the following:

Hypothesis 6 (H6).

User satisfaction will have a positive and significant influence on behavioral intent to use LMS in Nigerian context during COVID-19 pandemic.

2.5.7. Facilitating Conditions

In the context of technology adoption and use, Venkatesh [64] defined facilitating conditions (FC) as the degree to which people perceive the availability of technological and other organizational resources needed to use a specific technology. In the context of eLearning, Tarhini [65] argued that resources such as internet, access to computers and other devices, and electricity supply significantly influenced students’ intention to use LMS. The authors discovered that lack of access to computers and other movable devices, low internet diffusion and speeds, and lack of technical support negatively effects students’ intention to use LMS in Africa. Similarly, Al Amoush [66] and Yakubu [17] stressed that students will be less interested to use LMS if these resources are not available. Thus, propose:

Hypothesis 7 (H7).

Resource facilitating conditions will have a positive and significant influence on behavioral intent to use LMS in Nigerian context during COVID-19 pandemic.

2.5.8. Social Influence

El-Masri [34] and Tarhini [67] in their studies defined social influence (SI) as the effects of social pressure on individuals’ behavior and perception towards engaging in a particular action. According to Venkatesh [64], SI refers to the degree to which a person perceives the opinion of others who believe that he/she should make use of the new system. The factor is derived from UTAUT and is similar to “social norm” factor in Theory of Planned Behavior (TPB), Theory of Reasoned Action (TRA), and updated Technology Acceptance Model (TAM2). SI as for this study, is to examine the effect of pressures coming from other people such as students from other institutions, senior lecturers, and schools’ administrators on learners’ intent to accept and use different LMS. Thus, we hypothesized:

Hypothesis 8 (H8).

Social influence will have a positive and significant influence on behavioral intent to use LMS in Nigerian context during COVID-19 pandemic.

2.5.9. Behavioral Intentions and Actual Usage

Behavioral intention (BI) is defined as the gradation of persons intent to use a particular technology for various undertakings. In the context of eLearning, Sánchez-Prieto [68] defined BI as the measure of students’ intention strength to use different LMS in achieving their educational goals. The authors argued that once students’ intention to use a system is shaped, then it would be converted into the real usage of the system. Therefore, the researchers propose that:

Hypothesis 9 (H9).

Behavioral intention will have a positive and significant influence on LMS actual usage in Nigeria during COVID-19 pandemic.

2.6. Participants

The study participants were drawn from four private and two public universities. Three universities each from Northern and Southern parts of Nigeria using convenience sampling techniques as per Chambo [69]. According to study by Yakubu [17], the six universities were the only schools that attempted to incorporate LMS into their teaching and learning activities without being successful. Thus, the needs to obtained first-hand information from these schools with regards to factors that affects LMS progress in Nigeria. The study participants comprise of 808 male (65.0%) and 436 females (35.0%). The different between the two gender groups may not be unconnected with the high ratio of male students in Nigerian tertiary institutions. As shown in

Table 1. Demographic distribution.

Variables	Demographic Variable	Frequencies	Percentage
Gender	Male	808	65.0
	Female	436	35.0
Age	18–25	506	40.7
	Above 25	738	59.3
Study Level	Undergraduate	679	54.6
	Postgraduate	565	45.4
Length of Usage	Less than one year	930	74.8
	Over one year	314	25.2

, participants above the age of 25 account for 59.3% of the study sample, while those within the age of 18–25 account for 40.7% of the total participants. With regards to participants education level, 54.6% of the respondents were undergraduate students, while the postgraduate students account for 45.4% of the participants. Similarly, 74.8% of the participants stated that, they have used LMS for a period of less than one year, while only 25.2% of the participants used it for over a year, signifying the low level of LMS usage in Nigerian higher institutions.

2.7. Data Collection Tools

The study questionnaire consists of three parts. The first part 1, contained the study consent form, which introduces the study to the participants, highlights the purpose of the research, and what is required from the participants concerning the survey. It also states the benefits and risks associated with partaking in the research, and that participation is voluntary. While the second part of the questionnaire contained respondents’ demographic information such as age, gender, length of LMS usage, and education level (i.e., postgraduate or undergraduate) as seen in Table 1. The third part of the questionnaire consists of ten (10) constructs from three different models such as D&M, TAM, and UTAUT2. To bridge gap from LMS perspectives, this present study integrated “Perceived Enjoyment” and “Attitude towards technology” constructs into the study model.

The first two constructs were integrated into the study model as recommended by Yakubu [17] and Moorthy [52] that future LMS studies should investigate people’s attitude towards new technology and perceived enjoyment in order to determine their effects on LMS growth for sustainable education. Perceived usefulness and ease of use were gotten from Davis [41] technology acceptance model (TAM). While System Quality and User Satisfaction constructs were sourced from Delone [42] revised model of information system success, and the last three constructs of Social Influence, Facilitating Conditions, and Behavioral Intentions were sourced from UTAUT2 model by Venkatesh [43]. Five-point Likert scale ranging from strongly agree (1) to strongly disagree (5) in order to evaluate the respondents’ perceptions of the constructs used in the study hypothetical model see (Appendix A). Uniformity amongst the study constructs’ dimensions was calculated using Cronbach Alpha (CA) to ascertain the consistencies level of all the theories to be used in the study proposed AI based models. As showed in

Table 2. Reliability of the study constructs.

Model	Reference (Adapted from)	Constructs	Number of Items	Cronbach Alpha
Recommended	[17,52]	Perceived Enjoyment (PE)	4	0.864
Recommended	[17,52]	Attitude (ATT)	4	0.814
TAM	[41]	Perceived Usefulness (PU)	5	0.749
TAM	[41]	Perceived Ease of Use (PEOU)	4	0.769
D&M	[42]	Systems Quality (SQ)	4	0.842
D&M	[42]	User Satisfaction (US)	4	0.828
UTAUT2	[43]	Facilitating Conditions (FC)	4	0.873
UTAUT2	[43]	Social Influence (SI)	4	0.888
UTAUT2	[43]	Behavioral Intentions (BI)	4	0.721
UTAUT2	[43]	Actual Usage	4	0.713

, the internal consistencies levels for all the constructs were above 0.70, this showed that accepted level of uniformity were achieved.

2.8. Procedure and Data Collection

Two methods of data collection (i.e., electronic and paper-based) were used in administering the study questionnaire. For private universities participants, an online survey tool (Google Form) was designed, and the link send to the participants via their university students mailing list and other social networking platforms (e.g., WhatsApp, and Facebook) utilize by the institutions. While the paper-based approach was used for public universities participants due to absence of students mailing systems and functional students’ portal. The study paper-based questionnaire was distributed by some selected administrative personnel of these universities who are conversant with administering surveys questionnaires and were informed about the study purpose to ensure that the participants read and keep a duplicate of the study consent form. The completed questionnaires were collected via a pool styled box to ensure respondents anonymity. However, the respondents have the right not to participate in the survey as participation is voluntary.

A total of 560 paper-based questionnaires were distributed in the two public universities, and 512 (91.4%) valid responses were obtained, which is an excellent returned rate [50]. Just like the paper-based, the study online survey tool produces 769 (96.1%) usable responses out of the total 800 survey links with feedback form sent to four private universities’ participants. Moreover, the online method (i.e., Google Form) was configured in such a way that participants’ details such as IP address, emails, and ID were not part of the responses captured. In total, the two methods produced a total of 1244 valid responses, the study was carried out during the pandemic period of COVID-19 (i.e., from February–December 2020). The methodological structure of the study is presented in Figure 2.

As seen in Figure 2, the structure consists of three key stages. In the first stage, data for the study were collected, categorized, pre-processed, and dominant inputs were selected. While the four nonlinear AI-based (SVM, GPR, ANN, and BRT) models were developed in the second stage. Validation activities took place in the third stage using five indices of Nash–Sutcliffe efficiency (NSE), Root mean square error (RMSE) Mean absolute error (MAE), correlation coefficient I, and Percentage bias (PBIAS). Moreover, performance of each of the developed models were compared, and the final outcomes (predictions) obtained in same stage.

2.9. Data Analysis Methods

Data collected for the study were processed and analyzed using MATLAB 2019b and SPSS version 20 software. Cronbach’s alpha (CA) was used for assessing the uniformities of the study constructs. Respondents’ demographics information such as gender, age, education level, and period of LMS usage were analyzed using descriptive analysis method. Furthermore, an AI-based technique (sensitivity analysis) was used to select the most dominants factors amongst the study variables. Finally, four AI-based models namely SVM, GPR, ANN, and BRT were employed to obtain a reliable and accurate prediction of LMS determinants in the proposed research model.

2.9.1. Ensemble Techniques

Collaborative approach, or in other words ensemble techniques is a machine learning method used by researchers to combine two or more predictions results obtained from different models to improve the performance of the separate models so that accurate and reliable results can be obtained [70]. According to Nourani [23], there are two major types of ensemble techniques: (1) nonlinear ensemble technique, e.g., ANFIS, ANN, SVM, GPR, and regression trees; and (2) linear ensemble technique. Nowadays, use of ensemble techniques in engineering and science-based disciplines has proven to be effective and provide better outcomes than the use of single modelling [71]. In this study, four nonlinear (SVM, GPR, ANN, and BRT) ensemble methods were used to enhance the performance of the four AI-based developed models for predictions of LMS determinants. The structural algorithms of the ensemble methods used were presented below.

The SVM flow diagram (Figure 3) consists of four (4) stages. The input vector contained the study input parameters, represented as X, the hidden nodes kernel functions, the weight Lagrange multipliers that adjust the network synopsis weight and determine the percentage of Bias, and the output vector (y) that determine the correlation coefficient between the variables under investigation. Flow diagram of the study GPR structure is presented in Figure 4, below.

As shown in Figure 4, Y represent the measured values of the targeted output, U represent U represent the gaussian units (GU), while 0 and K represent the mean and covariance, respectively. The study ANN algorithm is presented in Figure 5.

As shown in Figure 5, the study ANN algorithm consists of three (3) layers. The inputs layer representing the study independent parameters, the hidden layers that nasalize the association between the target and input variables, and the output class represent the study target, i.e., (forecast variable). The structure of the study BRT model is presented in Figure 6, below.

As seen in Figure 6, the structure of the study BRT model consists of three ladders. Normalized input data were fed into model so that the targeted output can be estimated from the input variables. The process continues by adjusting the network input weights until the best result (prediction) is obtained. The general structural procedure of the developed ensemble methods is presented in Figure 7.

As seen in Figure 7, the structure of the study ensemble methods comprised of six phases, namely, inputs, sensitivity examination, models outputs, ensemble units, and ensemble outputs phases.

2.9.2. Sensitivity Analysis

According to Pham [72], the efficacy of all AI-based models relies on the number of input parameters imposed on the model. Use of too much input parameters will increase the complexity of the models and can lead to model overfitting. while insufficient parameters on the other hand can lead to inaccurate model processing. Thus, neither too much nor insufficient parameters were required for each of the study AI-based nonlinear models. As seen in Figure 7 and Figure 8 input parameters were used to obtain a reliable and accurate predictions of LMS determinants in developing countries for sustainable education especially Nigeria during COVID-19 pandemic. To obtain the most relevant and dominant factors that affects LMS acceptance and continual usage in Nigeria during COVID-19 pandemic, a nonlinear Feed-forward neural network (FFNN) sensitivity analysis was used in this study instead of using the usual classical linear techniques, e.g., Pearson correlation, a linear correlation coefficient measure used in selecting key factors of LMS usage in higher education, e.g., see [73]. Use of linear measure techniques like that of Pearson correlation has been criticized by some studies, e.g., see [74]. The eight LMS determinant predictors were ranked and evaluated based on average coefficient of determination (DC) of the separate modelling using training and testing ladders of the FFNN modelling process. The closer DC value to 1, the higher the effect of such factor, and the higher the performance of the AI model vice versa [23].

2.9.3. Data Normalization for the Study Models

Study data in AI modelling and simulation are usually normalized before inputted to the AI models. Normalization is used to avoid data with higher numeric values overshadowing those in lower numeric range by bringing the study dependent and independent variables into the same range [71]. Furthermore, data normalization simplified numerical computations in AI modelling process which in turn reduced results processing time and upsurges models accuracy. Thus, the study data were normalized as per Nourani [23], ranging between 0 and 1 using Equation (1).

$$N_{norm}=\frac{N-N_{min}}{ N_{max}-N_{min}}$$

(1)

where N is the number of constructs used in the study, and N_norm represent the study constructs normalized values, while N_max and N_min in the formular represent the observed maximum and minimum values of the study constructs influence on LMS usage.

3. Results

The results of the study parameters effects on LMS acceptance and use, sensitivity analysis, testing and training, and models’ predictions of LMS determinants in Nigeria during COVID-19 were presented in the following subsections.

3.1. Data Pre-Processing and Performance Evaluation

Data for the study were first pre-processed and the performance of each of the dominant parameters was evaluated using sensitivity analysis, i.e., average coefficient of determination (DC).

Sensitivity Analysis Results

LMS predictors ranking results based on sensitivity examination is presented in

Table 3. Results of sensitivity analysis.

Parameter	DC (Average)	Rank
Facilitating conditions (FC)	0.8210	1
Attitude towards Tech. (ATT)	0.8101	2
Perceived enjoyment (PE)	0.7602	3
User satisfaction (US)	0.7563	4
Perceived ease of use (PEOU)	0.6142	5
Perceived usefulness (PU)	0.6112	6
Social influence (SI)	0.4816	7
System quality (SQ)	0.4635	8
Facilitating conditions (FC)	0.8210	1

, using Equation (2).

$$DC=1-\frac{{{\displaystyle \sum }}_{i=1}^n\left(N_{obsi}-Nprei\right)2}{{{\displaystyle \sum }}_{i=1}^n\left(N_{obsi}-N_{obsi}\right)2}$$

(2)

where n stands for the number of observations, Ns is the mean value of the observed LMS determinants, N_obs is the observed effects of LMS determinants, while N_pre is the predicted effects of LMS determinants. Though, some of the dominant predictors have low DC values (e.g., SI and SQ) compared to other dominant factors. All the eight parameters were used as inclusion of these parameters with lower DC values does not affects the performance of the AI-based models. Table 3, depicts the study sensitivity analysis results.

Having selected the most dominant inputs parameters (i.e., facilitating conditions, attitude towards technology, perceived enjoyment, user satisfaction, perceived ease of use, and perceived usefulness), four nonlinear AI-based models (SVM, GPR, ANN, and BRT) were developed to obtain predictions with regards to factors responsible for slow take-up of LMS in developing nations, especially in Nigeria during COVID-19 pandemic. Though, Cronbach Alpha (CA) was calculated to ascertain the uniformity level of the study constructs. However, the sensitivity analysis results were used in selecting the most dominants factors as the approach offers more accurate and reliable results than the CA method.

3.2. Single Modelling Results

The performance results of the four developed AI-based models were measured using five statistical indices, namely, NSE, RMSE, MAE, R, and PBIAS as shown in Table 2.

As seen in

Table 4. Single modelling results.

Training						Testing
Models	NSE	RMSE	MAE	R	PBIAS	NSE	RMSE	MAE	R	PBIAS
SVM	0.9857	0.0058	0.0004	0.9925	0.0188	0.9941	0.0113	0.0002	0.9970	0.0047
GPR	0.9999	0.0000	0.0000	1.0000	0.0000	1.0000	0.0000	0.0000	1.0000	0.0000
ANN	0.9999	0.0010	0.0000	1.0000	0.0012	0.9999	0.0017	0.0000	0.9999	0.0010
BRT	0.9700	0.0288	0.0007	0.9907	0.0491	0.9528	0.0256	0.0009	0.9960	0.0457

, GPR model has the highest performance values in terms of training and testing. The model has an NSE values of 0.9999 and 1.0000, signifying the predictive power of the model. The RMSE values of 0.0000 and MAE 0.0000 clearly showed that the model has an accurate prediction value. Similarly, R values of 1.0000 also indicated a strong correlation between the independent variables and the dependent variable. Furthermore, optimal PBIAS values was achieved in the GPR model as the values stood at 0.0000 in both training and testing. Just like the GPR model, the ANN model also has a good training and testing performance results. The model has an NSE values of 0.9999, RMSE values of 0.0010 and 0.0017 in training and testing respectively, indicating how well the observed and simulated data fits in the plot line. The MAE values for the ANN model stood at 0.0000 which is the highest accuracy level in terms of model prediction, while the R values of 1.0000 and 0.9999 indicate a strong relationship between LMS usage and the study input parameters. Additionally, the PBIAS values of 0.0012 and 0.0010 obtained in ANN model signifies the model prediction ability. As seen in Table 4, the single modelling results showed that the SVM model also has accurate predictive skill as it has an NSE values of 0.9857 and 0.9941, RMSE values of 0.0058 and 0.0113, MAE values of 0.0004 and 0.0002, R values of 0.9925 and 0.9970 in training and testing, respectively, indicating that both simulated and observed data fits well in the line plot, accuracy in predictions, and significant association between the study variables. Furthermore, the SVM model has an accurate simulation process as its PBIAS values stood at 0.0188 and 0.0047. Though, the single modelling results showed that all the four developed AI-based (SVM, GPR, ANN, and BRT) models have accurate predictive skills, and indicated a strong correlation between the study variables. Moreover, the BRT model has the lowest performance results compared to other three models (SVM, GPR, and ANN) as it has an NSE values of 0.9700 and 0.9528, RMSE values of 0.0288 and 0.0256, MAE values of 0.0007 and 0.0009, R values of 0.9907 and 0.9960, PBIAS values of 0.0491 and 0.0457 in testing and training, respectively. Training and testing scatter plots for the developed AI-based models were presented in following subsection.

3.3. Training and Testing Scatter Plots for the Developed AI-Based Models

To further examine the performance levels of the developed AI-based models (SVM, GPR, ANN, and BRT) which are based on nonlinear relationship between LMS determinants and determinants predictors. Scatter plots were used to further examine the accuracy level of the developed AI based models in terms of predicting LMS determinants in Nigeria during COVID-19 pandemic. As shown in Figure 8, the NSE results from the training scatter plots for the developed models further strengthens the projection influence of the models in terms of accurate predictions between the simulated and observed data.

As shown in Figure 8, the training scatter plots results affirms the performance results obtained in the single modelling process. The testing scatter plots is presented in Figure.

As seen in Figure 9, the training scatter plots for the developed models indicate that the study input variables (FC, ATT, PE, US, PEOU, PU, and SI,) were the main factors responsible for slow take-up of LMS in Nigeria during COVID-19 pandemic.

4. Discussion

Corroboration of the four AI nonlinear models was done with the testing and training data. The outcomes of both the predicted and targeted output values for all the developed models were pretty good as shown in Figure 8 and Figure 9, indicating a strong correlation between LMS actual usage and the experimental values. Figure 10, depicts the prediction results of the developed models.

As seen in Figure 10, the predicted values of 0.9800, 0.9600, 0.9400, and 0.9200 are for SVM, GPR, ANN, and BRT models, respectively. While 1.0000 represent the targeted values. Although, the GPR model has better prediction ability in terms of training and testing as shown in Figure 8 and Figure 9. The prediction results obtained from the four studied AI models clearly indicate a strong correlation between the simulated and observed values as the predicted values of all the developed models were close to the targeted output value of 1.0. Furthermore, the single model’s performance results comparison proved that SVM has the highest prediction ability compared to GPR, ANN, and BRT due to its robustness in handling data uncertainties. This result is in line with the findings of Pham [72] who argued that SVM has the ability to handle input–output data uncertainties compared to other AI models, e.g., ANN, GPR, and BRT.

The prediction results obtained from the four studied nonlinear AI-based models showed that the study input parameters of facilitating conditions (FC), students’ attitude towards LMS (ATT), perceived enjoyment (PE), and user satisfaction (US) were the most dominant factors that influenced students’ behavioral intent to use LMS in Nigeria, which in turn led to the total shut down of schools during the peak period of COVID-19 pandemic. The results clearly show that lack of access to computers, internet, constant power supply, and students’ negative attitude towards technology (LMS) were main the reasons for long paused of educational activities in Nigeria during COVID-19 pandemic. Similarly, Table 3, indicates that the second most dominant factors that disrupt educational activities in the study are during COVID-19 pandemic are perceived ease of use (PEOU) and perceived usefulness (PU) of LMS. The results may not be unconnected with the non-availability of web-based learning technologies in Nigerian colleges. The result indicates that only few schools in Nigerian attempted to deploy LMS for their teaching and learning activities without being successful due to several challenges. Furthermore, the result is in alignment with the findings of Baki [55], who stressed that students can only perceive a particular eLearning system to be useful (PU) and easy to use (PEOU) only if the systems were made available to them, and they found it entertaining.

Though, social influence (SI) and system quality (SQ) has lower DC values as shown in Table 3, their inclusion does not affect the performance of the developed AI nonlinear models as the final outcomes of all the developed model stood above 0.9 indicating a strong correlation between the dependent variable, in this case LMS actual usage and the independent variables. However, this result is in contrast with the findings of Yakubu [17], who argued that learners in Nigerian colleges believe that individuals they respect or considered to be important influenced their decision with regards to LMS acceptance and usage. Methods employed may likely be the reason for the results differences as authors of the prior study employed structural equation modelling (SEM), while this study employed AI-based approach, which is more reliable, robust, and accurate in making predictions that are complex in nature, e.g., individuals’ attitudes, and emotion.

The study results deliver an experimentally resulting evidence that AI-based models produce more accurate and reliable results than the classical models as all the study AI developed models predicted a strong correlation between the study variables which in turn negatively affect educational activities during COVID-19, by identifying students’ negative attitude towards LMS, satisfaction, enjoyment, perceived usefulness, and ease of use as main reasons for slow progress of LMS in Nigeria. Moreover, the study findings are meaningful in the field of web-based learning technologies for sustainable education during pandemic periods, simply because prior studies focused more on LMS adoption, and to the best of our knowledge none of the previous studies used AI-based techniques to investigate other determinants of LMS such as satisfaction, enjoyment, and attitude.

5. Conclusions

In this study, determinants of LMS success for sustainable education in Nigeria was examined and modelled using four artificial intelligence (AI) based models (SVM, GPR, ANN, and BRT). The final performance results for all the four developed AI models indicate a significant correlation coefficient between LMS usage and the study parameters (FC, ATT, PE, US, PEOU, PU, SI, and SQ) in Nigeria context. Therefore, it can be deduced that the proposed model can be used in scientific studies. However, though SI and SQ were included in training and testing of the developed AI models, the study sensitivity analysis results indicate that SI and SQ have the lowest DC values compared to other input parameters, thus indicating an insignificant correlation between the two variables and LMS acceptance and usage in Nigerian context. Therefore, further investigations are required in order to have more insight with regards to the effects of these two variables. As a result, prior to the development of the study reported model (i.e., determinants models), an analysis of dominant LMS determinants predictors was performed using nonlinear sensitivity analysis to select the most dominant parameters. Consequently, facilitating conditions, attitude towards technology, perceived enjoyment and users’ satisfaction were discovered to be the most significant factors that affect LMS take-up in Nigeria. Other factors contributing to the slow progress of LMS in Nigeria are perceived ease of use and usefulness. Moreover, single models’ performance results comparison proved that SVM has the highest prediction ability compared to GPR, ANN, and BRT due to its robustness in handling data uncertainties. Though, an ensemble model is usually developed in studies that involve two or more AI-based models to improve the performance of the separate models. However, the ensemble model was ignored as the performance of all the developed models were pretty good. Education authorities, school administrators, and other stakeholders in developing nations like Nigeria can use the study findings to provide the necessary resources that facilitate LMS usage and deploy efficient and effective learning management system to augment the conventional teaching and learning method for sustainable education during pandemic periods like that of COVID-19 pandemic.

Just like all other studies, this study too has some limitations; the study examined the factors that affect LMS take-up in developing countries especially Nigeria tertiary institutions from students’ perspectives without taken tutors, administrators, and primary/secondary schools into consideration. Another limitation of this study is the use of nonlinear AI models for all the studied models. Future studies should examine the application of linear and other nonlinear AI techniques like ANFIS, ARIMA, TDNN, and SVR to investigate other factors that affects educational activities from cultural and policies point of views in educational colleges of all kind. Furthermore, future studies should examine the design aspects of LMS with a view to identify it enjoyable components for sustainable development of educational activities.