On the economic evaluation of online learning in higher education during the COVID-19 pandemic: a willingness-to-accept approach

Salvador del Saz-Salazar (Department of Applied Economics II, Facultat d’Economia, Universitat de València, Valencia, Spain)
Salvador Gil-Pareja (Department of Applied Economics II, Facultat d’Economia, Universitat de València, Valencia, Spain)
María José García-Grande (Department of Economics, Universidad de Alcala de Henares, Alcalá de Henares (Madrid), Spain)

Applied Economic Analysis

ISSN: 2632-7627

Article publication date: 5 May 2022

Issue publication date: 28 February 2023

1879

Abstract

Purpose

This study, using a contingent valuation approach, aims to shed light on the economic evaluation of online learning during the first wave of the pandemic.

Design/methodology/approach

A sample of 959 higher education students was asked about their willingness-to-accept (WTA) a monetary compensation for the loss of well-being resulting from the unexpected and mandatory transition to the online space. In explaining WTA determinants, the authors test the appropriateness of the double-hurdle model against the alternative of a Tobit model and find that the factors affecting the participation decision are not the same as those that affect the quantity decision.

Findings

Results show that a vast majority of the respondents think that the abrupt transition to online learning is detrimental to them, while those willing to accept a monetary compensation account for 77% of the sample, being the mean WTA between €448 and €595. As expected, WTA decreases with income and age, and it increases if some member of the family unit is unemployed. By aggregating the mean WTA by the population affected, total loss of well-being is obtained.

Originality/value

To the best of the authors’ knowledge, to date, this method has not been used to value online learning in a WTA framework, much less in the particular context of the pandemic. Thus, based on the understanding that the economic evaluation of online learning could be very useful in providing guidance for decision-making, this paper contributes to the literature on the economic evaluation of higher education.

Keywords

Citation

del Saz-Salazar, S., Gil-Pareja, S. and García-Grande, M.J. (2023), "On the economic evaluation of online learning in higher education during the COVID-19 pandemic: a willingness-to-accept approach", Applied Economic Analysis, Vol. 31 No. 91, pp. 19-38. https://doi.org/10.1108/AEA-04-2021-0083

Publisher

:

Emerald Publishing Limited

Copyright © 2022, Salvador del Saz-Salazar, Salvador Gil-Pareja and María José García-Grande. Published in Applied Economic Analysis.

License

Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence maybe seen at http://creativecommons.org/licences/by/4.0/legalcode


1. Introduction

The COVID-19 outbreak and the following declaration of a pandemic by the World Health Organization (WHO) on 11th March 2020 disrupted our lives in many ways. Today, little more than one year after the onset of the disease in Wuhan (China), the COVID-19 landscape is already one of constant change and uncertainty despite the universal efforts in developing vaccines that are starting to prove their worth and safety (Häfner, 2020; Knipe et al., 2020). Indeed, it is becoming clear that COVID-19 will circulate for years since new viral variants may resist current vaccines and, therefore, the disease seems likely to become endemic (The Economist, 2021).

Following the logic that extraordinary times call for extraordinary measures (Murphy, 2020), schools, colleges and universities were shut down to keep social distance as a means to slow down the spread of the COVID-19 pandemic. This crisis prompted the need for the rapid transition from traditional face-to-face classes to distance learning systems and thus accelerating steady developments in higher education digitalization (Komljenovic, 2020). Although in many higher education institutions the shift to distance learning has been seen as an unparalleled opportunity to reshape teaching and learning in unprecedented ways (Green et al., 2020), it is also true that for many academics, the abrupt transition into online domains has been an unexpected, disconcerting and unwelcome experience, while it has revealed many of the deficiencies and vulnerabilities of the higher education sector devoid of meaningful research into digital pedagogies (Tesar, 2020; Watermeyer et al., 2021).

With these issues in mind, this paper aims to shed light on the economic evaluation of online learning during the first wave of the pandemic in a contingent valuation framework. The contingent valuation is a survey-based approach used to assign a value on public goods that are not usually traded in the market, and hence their value is unknown (Mitchell and Carson, 1989). Thus, through the use of a questionnaire, a hypothetical valuation framework is constructed in which respondents are asked about their willingness-to-pay (WTP) for the hypothetical provision of a public good (in this particular case, public higher education) that increases their well-being or the amount of monetary compensation they would require (willingness-to-accept or WTA) to give up this public good. While WTP is typically associated with a desirable change, WTA is associated with a negative change. Whether WTP or WTA is the correct measure depends on how property rights are assigned (Carson, 2000).

In the case presented in this paper, a sample of 959 college students from two Spanish public universities were asked about their WTA to compensate them for the loss in their well-being resulting from the unexpected, rapid and mandatory transition to the online space. To the best of our knowledge, to date, this method has not been used to value online learning in a WTA framework, much less in this particular context of the pandemic. Thus, based on the understanding that the economic evaluation of online learning could be very useful in providing guidance for decision-making and management, this paper contributes to the literature on higher education in the context of the global crisis caused by the COVID-19 pandemic. Considering that students were not asked to voluntary give up traditional face-to-face learning and that the tuition fees paid can be regarded as an implicit contract by which traditional teaching is guaranteed on an ongoing basis, it seems that college students have a legal entitlement to the public good and accordingly WTA is the more theoretically correct of the two measures. Indeed, this is particularly true if we acknowledge that there is a predominant sense that universities did not deliver to students what they came to these institutions for in the first place (Goedegebuure and Meek, 2021) [1]. Results show that a vast majority of the respondents (81%) think that the abrupt transition to online learning is detrimental to them, while those willing to accept a monetary compensation account for 77.2% of the sample, being the mean WTA €512 using an open-ended question format.

The outline of the paper is as follows. Section 2 summarizes the literature on online learning in higher education in the setting of COVID-19. Section 3 shows the theoretical model used for estimating the mean WTA. Section 4 describes the case study, the survey process and the main elements of the hypothetical market constructed. Section 5 presents the results obtained. Finally, Section 6 discusses the results while conclusions and policy implications are summarized.

2. Literature review

Since the COVID-19 pandemic disrupted usual on-campus activities in March 2020, the world has seen the most extensive educational system disruption in history in more than 190 countries (Almahasees et al., 2021). Consequently, over the past year and a half, researchers all over the world have been written much work to assess the impact on students of the shift from traditional in-person teaching to remote instruction. A first line of research on this change has adopted a descriptive approach trying to assess or reflect on, among other issues, the future of higher education, the positive and negative effects of the new scenario, its impact on students’ mental and physical well-being or the importance of online learning in a time of crisis and pandemics such as the COVID-19 (Bolumole, 2020; Cao et al., 2020; Dhawan, 2020; Green et al., 2020; Leask, 2020). Another important strand of research has relied on surveys that try to measure the effect of the COVID-19 lockdown on students’ performance and learning strategies (González et al., 2020; Wolfgang et al., 2021) or to capture students’ and educators’ perceptions about this shifting to online learning by higher education institutions during COVID-19 pandemic (Aristovnik, 2020; Rizun and Strzelecki, 2020; Watermeyer et al., 2021; Zalat et al., 2021). However, to the best of our knowledge, only two papers, Aucejo et al. (2021) and Wang et al. (2021), have used valuation methods to quantify in monetary terms students’ preference for alternative course delivery options and other university education attributes in the setting of COVID-19.

Aucejo et al. (2021), using a choice experiment, aimed to obtain students’ WTP for college-related activities in the absence of COVID-19. More specifically, they tried to find out the value that students assign to in-person classes (versus remote instruction) and to social activities while in college. Furthermore, they also aimed to characterize the heterogeneity in the students’ WTP based on their socio-economic and demographic characteristics. Using a sample of 1,150 undergraduate students from the Arizona State University, these authors found that the WTP for in-class instruction (relative to a remote format) represented around 4.2% of the average annual net cost of attending this college ($12,948 per year), while university-related social activities (relative to full-cancellation of campus life) were valued almost twice as much (8.1% of the average costs). Moreover, they also found that in-person instruction was less valued by students with previous experience with online learning and that students who work (who are likely to be from more disadvantaged backgrounds) derived substantially lower value from university social life, due mainly to time and resource constraints.

Wang et al. (2021) also used students’ WTP as a measurement of their preferences to quantify the value that students place on several key aspects of the college experience and, in particular, the course delivery mode (fully online, hybrid and in-person) against the background of COVID-19. For this research, the authors used data from 605 valid samples collected from an online survey sent to the 34,920 undergraduate students enrolled at Purdue University in Fall 2020. They found that students were willing to pay $1,190 for the in-person delivery option versus the fully online alternative, but results also showed that students’ preferences were heterogeneous. Freshmen preferred in-person teaching and open campus consistently higher than students in other years. However, students living far from campus and those that work during study felt the convenience of online education and would not discount it over face-to-face teaching.

3. Case study, survey instrument and data collection

As the COVID-19 virus spread quickly in Spain, on 13th March 2020, two days after the declaration of a pandemic by the WHO, the Spanish Government declared the “state of alarm” and the subsequent nationwide lockdown that was lifted on 21st June 2020. Universities were forced to close, and although were quick to replace face-to-face lectures, this closure affected learning and examinations until the reopening in September 2020. To have a better understanding of the impact that the abrupt and unexpected transition to online learning may have had on higher-education students’ well-being, 959 undergraduate students from the Spanish universities of Valencia and Alcalá de Henares were surveyed using a contingent valuation framework.

The survey was constructed and fielded using the free online survey tool Google Docs. Thus, a link to the questionnaire was emailed to participants from different majors (Humanities, Social Sciences, STEM [2], Medicine, Business and Law, etc.) who answered it on their personal computers. They were invited to assess their experience with online learning and were also informed about the anonymity and confidentiality of the study and about their right to withdraw. Participation was voluntary, and no incentives were provided. Answers were collected automatically on an Excel compatible spreadsheet, which makes Google Docs a low-cost, reliable, convenient and time-saving survey-creation tool, especially in these exceptional times. Higher internet penetration in the population and the availability of internet panels to administrate surveys will mean that Web-based surveys will become the dominant survey format of the future (Liebe et al., 2015).

However, in spite of their advantages and growing popularity, internet surveys are not without methodological problems that can lead to biased welfare estimates (Bonnichsen and Olsen, 2016; Sandorf et al., 2016). In particular, problems concerning sample coverage bias and selection bias may be expected. A coverage problem arises when access to a personal computer and to the internet is not available to everyone in a population, so people without internet access cannot be included in the survey. However, in our case study, this was not a problem because all the potential participants in the survey had internet access and a personal computer during the lockdown, and as it will be shown in the results section, they were quite satisfied with the quality of their internet connection. Additionally, when using internet surveys, people can choose whether or not to participate in the survey, i.e. there may be self-selection by the individuals since the researcher does not control the selection process (Heckman, 1979). In that case, if respondents differ from non-respondents on the variables of interest, it will not be possible to extrapolate value estimates and make inferences from the sample to the population (Messonnier et al., 2000; Bonnichsen and Olsen, 2016). This is often referred as a non-response bias. When the sample is obtained from an internet panel, “propensity weighting” is used to correct for a possible bias through the comparison of two populations: those who participate in the survey and those who do not participate (Bethlehem, 2010). In this particular study, as it is often the case with many contingent valuation studies (Loomis, 1987; Whitehead et al., 1993), such a correction is not possible because we do not have information about the distribution in the population of a set of variables that have been measured in the survey. Although non-response bias is almost impossible to eliminate completely, there are several ways to ensure that it is avoided – as much as possible – following Mitchell and Carson’s (1989) assessment that careful survey design and implementation can produce valid and reliable values. Firstly, in this study, in line with Sandorf et al. (2016), to avoid self-selection, respondents received a generic email with an invitation to participate with no clear reference to the purpose of the survey, i.e. no prior information was given about a possible compensation to those potentially harmed by the shift to online learning. So, we tried to avoid that the knowledge about a possible compensation would have made some individuals to participate, who would otherwise have chosen not to and vice versa. In fact, this particular information was provided to the respondents in the third section of the questionnaire, i.e. once they have answered 24 previous questions. Secondly, the survey instrument was thoroughly pretested to ensure that it ran smoothly regardless of the device used by the respondent. Indeed, respondents are much more likely to ignore survey requests if loading times are long and questions do not fit properly on their screens. Thirdly, the survey was kept live for a month to avoid rushed or short data collection periods that usually lead to an increase in non-response rates. Certainly, flexibility is one of the main advantages of online surveys since they do not require in-person (or phone) interviews that must be completed at a specific time of the day. Furthermore, to gather more responses, 15 days after the launching of the survey, a reminder was sent to the potential respondents. And fourthly, it is usually the case that many people refuse to respond to surveys in which personal information is required. For example, in contingent valuation studies, respondents identify family income as a sensitive and personal information. So, in this study, respondents were assured that confidentiality and anonymity would be protected.

The survey was carried out in June 2020 once students finished their exams. As it is customary in survey research, especially attention was paid to using a pre-test form of the questionnaire to detect sources of bias and to identify unclear wording. The pre-testing procedure consisted of two focus groups and a pilot study of 80 interviews, approximately 8% of the final sample. Focus-group is a research technique that collects data from group interaction on a particular topic determined by the researcher and handled by a skilled moderator (Morgan, 1988), while a pilot survey is a test of the draft survey materials and implementation process (Champ, 2017). The information gathered provided valuable insights on the clarity of the questionnaire and, more importantly, on the appropriateness of the contingent valuation scenario presented to the respondents. In light of this feedback, the questionnaire was shortened and modified to facilitate comprehension.

The questionnaire was divided into four sections. The first, apart from demographic factors such as age and gender, contained questions as the major in which students are enrolled, current year in college, number of years spent on campus and the subjective evaluation of their academic performance. These questions are important for at least two reasons (Whitehead, 2006). Firstly, they can be considered as introductory “warm-up” questions that are easy to answer, interesting for the respondents and, more importantly, they pave the way for answering tougher and more thought-provoking questions, such as those included in the valuation section. The second reason is that some of these variables can subsequently be used as predictors of the stated WTA.

The second part of the questionnaire included a series of attitudinal questions aimed at finding out the COVID-19 awareness of respondents. As shown in Table 1, these questions were in Likert scale format, and the respondents had to choose the appropriate answer from completely to disagree (1) to completely agree (5). It can be seen from the table that overall, respondents exhibit a high degree of COVID-19 concern since, for example, 85% of them agree or strongly agree with the statement that “the human and material resources used to deal with the health crisis caused by the COVID-19 have been insufficient”, 82% of them think that “even today, the vast majority of people do not act responsibly to prevent the spread or regrowth of the COVID-19 pandemic” and 75% of them believe that “the COVID-19 pandemic is a threat to humanity”.

In the third section of the questionnaire, using a Likert scale, respondents were first asked to rate both the quality of their internet access and their overall satisfaction with the online learning received. The valuation scenario was then described in detail. This section was aimed at finding out whether the unexpected transition into the online space could have resulted in a change in the quality of the learning received by the students in comparison to the situation prior to the COVID-19 outbreak. Those that stated to be adversely affected by this change were asked about their WTA a compensation for the resulting loss in their well-being. More specifically, respondents were presented with the following proposal:

As a result of the lockdown in response to the COVID-19 pandemic, in the middle of March 2020, face-to-face classes were cancelled and were replaced by online learning. For some people, this change may have been an inconvenience or even detrimental because of the lack of direct interaction between students and lecturers, the lack of motivation when there are no physical classes to attend or even the lack of internet access from home (digital divide). Considering your own experience over these past few months, do you think that the shift to online learning has negatively affected your overall performance and academic attainment? YES or NO.

For those who answered “yes”, the following questions were asked:

As negatively affected by the shift to online learning, would you be willing to accept a reduction in the tuition fee for the next academic year, 2020–2021, as a compensation for the damage caused? Before answering this question, please, bear in mind (1) that the compensation received would imply a reduction in the current availability of resources for funding other university policies and services and (2) that this proposal would be carried out only if a majority of respondents favour it.

If YES, how much?

If NO, why you are not willing to accept a compensation?

Thus, following the recommendations of the National Oceanic and Atmospheric Administration (NOAA) panel on contingent valuation (Arrow et al., 1993), respondents were given substitute and budget constraints reminders and were also informed that all the responses were consequential (Vossler and Watson, 2013). Once the compensation was offered, respondents were also asked if they were able to remember the cost of the tuition fee that they paid at the beginning of the current academic year.

The survey concluded with a set of questions that can be used later to interpret and validate WTA estimates. These involved socio-economic, attitudinal and behavioural indicators such as volunteering, membership in the neighbourhood and environmental groups, views towards the environment, family and personal income after tax and family size.

4. Theoretical framework

4.1 A measure of utility change: the compensating variation

The abrupt transition into distance learning caused by the COVID-19 outbreak may have resulted in a decrease of the quality of the learning received by higher-education students from q0 to q1. Without this change, the average student is assumed to enjoy a quality level q0, while after the referred change, the quality level experienced is only q1. Now, following Johansson (1993), let us consider an individual that maximizes her utility (or well-being) subject to a budget constraint. The individual’s indirect utility function can thus be written as follows:

(1) V=U[x(p, y, q), q]=V(p, y, q)
where x is a 1·n vector of private goods, and q is a 1·m vector of public goods. The quantity demanded of private goods is a function of prices (p), income (y) and the provision or quality of public goods (q). The indirect utility function is decreasing in prices and increasing in income and the quality of the public good. Let us now introduce a change in the quality of the learning received by the individual. Then the change in utility is:
(2) V=V(p, y, q1)V(p, y, q0)
where the subscript 0 (1) denotes initial (final) level values for the public good. As the utility function is not observable, we need a money measure to evaluate the change in utility. Then let us consider the compensating variation (CV). If the quality of the learning received deteriorates, then CV is the minimum amount of money that must be given to the individual to compensate her for the loss of learning quality, leaving her just as well off as prior to the change. Thus, CV measures the WTA compensation for the decrease in quality:
(3) V(p, y+CV, q1)=V(p, y, q0)

4.2 Willingness-to-pay versus willingness-to-accept

According to standard economic theory, as long as income effects are small, disparities between WTP and WTA should be negligible (Willig, 1976). However, empirical evidence on contingent valuation shows WTA measures of welfare to be substantially higher than that of WTP (Tuncel and Hammitt, 2014). Several explanations for the WTA-WTP gap have been proposed in the literature. Firstly, Hanemann (1991) demonstrates that both a substitution effect and an income effect determine the sign and magnitude of this disparity. The psychological theory of loss aversion, proposed by Kahneman and Tversky (1979), provides a second explanation for this disparity. Indeed, it is usually the case that people value losses more than they value equivalent gains, and accordingly, they ask for more compensation when losing a good than what they are willing to pay to obtain it. A third explanation is that in experiments and surveys when respondents are forced to make decisions with limited time and limited learning, commitment costs are greater and thus the divergence between WTP and WTA (Zhao and Kling, 2001).

On the grounds of convenience and influenced by the report of the NOAA blue ribbon panel on contingent valuation (Arrow et al., 1993), the conventional wisdom is to elicit WTP even when WTA is clearly the conceptually more appropriate measure (Lloyd-Smith and Adamowicz, 2018). The arguments against WTA are the relative lack of respondent’s experience with compensation claims compared to making purchases and the problem of obtaining unrealistically high WTA responses since they are not bounded by respondent’s income, as is the case with WTP. However, in valuing losses, the predominant use of WTP estimates may yield misleading welfare estimates and policy advice (Knetsch, 2010). Therefore, in this particular case it seems that college students have a legal entitlement to the public good (traditional face-to-face learning) and accordingly, the appropriate monetary measure of the decrease in economic well-being accompanying this loss is WTA (Nguyen et al., 2021). Indeed, it is quite obvious that most respondents would regard the existence of the situation prior to the pandemic as the reference state and basis for their feelings of loss given that they were not asked to voluntary give up traditional face-to-face learning rather the change was imposed on them. In this case, the reference state (traditional face-to-face learning) is better than the status quo (online learning) since the latter is perceived as a temporary and unfortunate condition that will return to normality (Whittington et al., 2017), something that was confirmed in focus groups and in the pre-test.

4.3 A double-hurdle model

In contingent valuation analysis, when an open-ended question is used to elicit a value, a recurrent problem is that a large number of responses cluster around the zero value. For example, in this case study, the compensation that an individual is WTA is bounded by zero, i.e. the observed data will comprise zero or some positive amount or, in other words, the dependent variable is left-censored. A straightforward way of addressing the problem of the censored nature of the data is the Tobit regression model (Tobin, 1958). However, a key limitation of the Tobit model is that the probability of a positive value and the actual value are determined by the same underlying process. Indeed, as noted by Green (2012), this model ignores that usually there are two decisions involved in these scenarios: people first decide whether to accept or not a monetary compensation (participation decision) and secondly, given that the answer to the first question was “yes”, then they decide how much are WTA (quantity decision). Empirical models with censored distributions that allow for the possibility that these two decisions are affected by separate processes are mentioned in the literature as “Double-Hurdle” models (Cragg, 1971). If both processes are the same, the appropriate censoring rule is given by the Tobit model (Maddala, 1983):

(4) yi=yi*   if  yi*>0, yi=0   otherwhise, yi*=Xiβ+εi
where yi is the stated WTA of individual i, yi* is the corresponding latent value of individual i’s actual WTA, Xi is a vector of the individual’s characteristics, β is a vector of parameters to be estimated, and εiN(0,σ2). Alternatively, if the process generating the censoring is different from the process generating the WTA response, the appropriate censoring rule is expressed as:
(5) yi=yi*   if  yi*>0 and Di>0,yi=0   otherwhise, Di=Ziθ+ui
where Di is a latent variable representing the participate/no participate decision, Zi is a vector of explanatory variables related to the discrete participate/no participate decision, θ represents a vector of parameters to be estimated and ui is the error term ∼ N(0,σ2). Thus, the double-hurdle procedure involves two models: a probit model to address the participate/no participate censoring rule (Ziθ) and a truncated model to account for WTA bids (Xiβ).

5. Results

5.1 Descriptive analysis

Mean WTA estimates are shown in Table 2. Our analysis is confined to non-protest responses, as is usual in contingent valuation analysis (Morrison et al., 2000). Protesters are the respondents who do not reveal their true WTA, and accordingly, they state a zero value because they object to some aspects of the valuation scenario, such as lack of information or even because they do not believe that the compensation will be paid due to lack of trust in public bodies (Meyerhoff and Liebe, 2006). To the respondents who stated that they were not willing to accept any compensation, a follow-up question was asked to differentiate protest responses from true zero responses. The proportion of protest responses was 26.4% of the sample, which lies in the interval (20%, 40%) that is considered acceptable in contingent valuation studies, according to Carson (1991). Those who agreed to participate and who gave a positive WTA value accounted for the 77.2% of the sample, while for the universities of Valencia and Alcalá de Henares, these proportions were, respectively, 80.1% and 73.6%. On average, the mean WTA for the entire sample was €511.6. However, students from the University of Alcalá de Henares had a higher WTA (€595.7) than those from the University of Valencia (€448.4). Figure 1 shows the distribution of WTA for the entire sample using eight €200-intervals.

To get an idea of how much is the WTA obtained, mean WTA estimates are compared with the tuition fees paid by students in the academic year 2019–2020. So, at the University of Valencia, the tuition fee ranged from a minimum of €836 (for a degree in Law) to a maximum of €1,270 (for a degree in Medicine), while at the University of Alcalá de Henares, these figures were, respectively, €1,283 and €1,568. Thus, the mean WTA obtained would mean a hypothetical reduction in tuition fees of between 35% and 54% for students enrolled at the University of Valencia and of between 38% and 46% for those enrolled at the University of Alcalá de Henares. However, if we consider that the change from face-to-face classes to online classes only affected half of the academic year, these percentages would be higher. For example, in the case of a degree in Medicine at the University of Valencia, it would mean a reduction in tuition fees of 70%.

The definition and descriptive statistics for the variables of interest are presented in Table 3. The respondents’ age ranged from 18 to 65, being the average age of 21.7. A majority of them are female (63.5%), while for the entire population, this figure is very similar (62%), so in this respect, our sample is representative of the population. The average monthly household income amounts to €1,811, and 38% of the respondents stated that some member of the family unit was unemployed.

On average, students have been on campus for 2.7 years and those that have received a scholarship amount to 37% of the sample. Overall, their level of satisfaction with the online learning received is low since, on a scale from 0 (totally unsatisfied) to 10 (totally satisfied), the average score was only 4.8. Indeed, those reporting a value lower or equal to “5” accounted for 58% of the sample, while on the other extreme, only 16% of the respondents reported a satisfaction score greater or equal to “8”. This is an expected result because a vast majority of respondents (81%) stated that the abrupt transition to online learning was detrimental to them. Nevertheless, in a study also conducted in Spain, González et al. (2020) found that the COVID-19 confinement resulted in a positive effect on higher-education students scores, although, in this respect, the possibility of cheating cannot be ruled out since it became a major concern across campuses during the spring of 2020 (Nguyen et al., 2020).

To avoid the so-called “digital-divide”, both universities of this study set up a lending program to provide laptops and capable smartphones to those students who could not afford them. In general, it can be said that access to the internet was not a major barrier since, on a scale from 1 (poor) to 5 (excellent), the average rating for “quality of internet access” was 3.7 and those respondents reporting a score greater or equal to “4” accounted for the 61% of the sample. Finally, other background characteristics of the study sample, such as COVID-19 awareness, nationality, volunteering and membership in some associations or groups, are also shown in Table 3.

5.2 Tobit regression versus double-hurdle regression

The aim of this section is two-fold. Firstly, to test the appropriateness of the double-hurdle model against the alternative of a Tobit model and secondly, to explain the determinants of the two decisions made by the respondents: whether to accept or not a compensation (participation decision) and how much to accept (quantity decision). The Tobit and double-hurdle model results are presented in Table 4. Following Green (2012) and assuming that the same set of explanatory variables appears in all the three equations, a specification test that evaluates the Tobit model against the double-hurdle model can be computed using:

(5) λ=2[lnLT(lnLP+lnLTR)]
where λ is distributed as a χ2 with degrees of freedom equal to the number of explanatory variables in the models. lnLT and lnLP are, respectively, the likelihood for the Tobit and probit models. Finally, lnLTR is the likelihood for the truncated model. The null hypothesis that the Tobit specification is correct is rejected if λ is greater than the critical χ2 value. In this particular case, the likelihood ratio statistic favours the double-hurdle model against the Tobit model since λ = 135.82 that exceeds the chi-square critical value of 34.81 at the α = 0.01 level of significance. This suggests that WTA responses are better explained by the double-hurdle model than by the Tobit model, i.e. that the participation decision and the quantity decision appear to be governed by different processes, thus making the Tobit model inappropriate to explain why some respondents state a zero WTA value. Therefore, in interpreting the regression results, our analysis will be restricted to the double-hurdle model. Thus, as can be seen from Table 4, third and fourth columns, the fact of being in a STEM major is significantly and positively associated with the probability of accepting a monetary compensation (+8.2%; p = 0.07). This is an expected result since, in these disciplines, students spend considerable part of their time doing a wide range of practical or laboratory work, something that was not possible to appropriately carry out while the university facilities were closed. Therefore, these students may have felt a greater decrease in their well-being as a result of the abrupt transition to online learning. Interestingly, remembering the tuition fee paid past year is significantly and positively associated with the probability of accepting a monetary compensation (+12.4%; p = 0.002). This result may suggest that these individuals were more aware of the real magnitude of the compensation offered to them since they had in mind a reference figure.

On the other hand, the variables that are associated with a decrease in the probability of accepting a monetary compensation are age, Spanish nationality, a dummy variable indicating whether the respondent has received or not a scholarship and the respondent’s self-reported satisfaction with the online learning received. In particular, the fact of having received a scholarship is significantly and negatively associated with the probability of accepting a monetary compensation (−11.4%; p = 0.002), while each additional year of age has also a significant and negative association with this probability (−0.8%; p = 0.02). The negative sign of the age parameter is consistent with previous findings in the contingent valuation literature (see, for example, Santagata and Signorello, 2000). An interesting result is that each additional level of self-reported satisfaction with the online learning received is significantly and negatively associated with the probability of accepting a monetary compensation (−3.4%; p = 0.001), so the more satisfied is the individual with the online learning received, the lower is the probability of accepting a compensation. Results also reveal that Spanish nationality is significantly and negatively associated with the probability of accepting a monetary compensation (−11.3%; p = 0.04), so Spanish students are less likely to be willing to accept a compensation than their European and non-European counterparts. Finally, other variables such as family income, gender, academic performance, years spent on campus, quality of internet access, COVID-19 awareness and membership in some organizations were not statistically significant.

Once the respondent has decided to participate, the truncated regression model shows the intensity of her participation, i.e. the degree of support given by those individuals with a positive WTA. As shown in the fifth and sixth columns of Table 4, it seems that the quantity decision appears to be more an economic decision than the participation decision since the variables unemployed and family income are now statistically significant. In particular, the fact of having a member of the family unit unemployed has a significant and positive association with the stated WTA (€102.8; p = 0.002), while each additional unit increase in family income is significantly and negatively associated with WTA (−€0.07; p = 0.003). As suggested by Groothuis et al. (1998), the negative sign of the income parameter is consistent with the diminishing marginal utility of income, so respondents with higher income are less influenced by the compensation offered for the loss in well-being derived from the abrupt transition to online learning. Similarly, the negative coefficient of the unemployed variable may suggest that less affluent individuals are more willing to accept compensation.

Results show that students in STEM majors are not only more likely to accept a monetary compensation but also that this fact is significantly and positively associated with the stated WTA (€135.8; p = 0.003). Likewise, as previously noted, students enrolled at the University of Valencia have a lower WTA than their counterparts from the University of Alcalá de Henares indeed to be interviewed at the University of Valencia is significantly and negatively associated with WTA (−€190.7; p = 0.000). These different WTA estimates could be explained by the fact that tuition fees are also different between the two universities. Similarly, a marginal increase in the respondent’s self-rated academic performance is also significantly and negatively associated with WTA (−€33.6; p = 0.07). Finally, an additional year of college is significantly and positively associated with the respondent’s WTA (€24.4; p = 0.03).

5.3 Aggregation

To inform decision-making, one final step of this study is to estimate in monetary terms the loss of well-being resulting from the abrupt transition to online learning. Therefore, the mean WTA estimates should be aggregated over the relevant population, i.e. the aggregate values will depend on both the loss per person and the population of those negatively affected (Bateman et al., 2006). So, if we now multiply the mean WTA estimates for the Universities of Valencia (€448.4) and Alcalá de Henares (€595.7) by the number of undergraduate students enrolled at each university during the academic year 2019–2020 (37,727 and 16,347, respectively), we obtain that the loss of well-being is estimated to be €16,905,468.7 in the case of the University of Valencia and €9,737,907.9 for the University of Alcalá de Henares (see Table 5). In any case, these estimates should be taken with caution since the possible presence of self-selection bias could lead to an upward or downward bias in value estimates depending on how many omitted variables are not included due to non-participation (Hudson et al., 2003; Giannoccaro et al., 2017).

6. Discussion and conclusions

In a context of digital disruption and abrupt change forced by the COVID-19 outbreak, this paper has provided a valuable insight into the economic evaluation of online learning during the first wave of the pandemic. We believe that our contribution is significant, at least in two aspects. Firstly, this study has shown that it is possible to value in monetary terms the loss of well-being experienced by college students as a consequence of the unexpected transition from face-to-face classes to online learning systems. Thus, the information gathered from this study could be very useful in providing guidance for decision-making and management as it evidences the view of students and serves to improve the educational system. And secondly, we have dealt with the issue of zero WTA responses applying a double-hurdle model that has proved to be superior to the Tobit model. This has allowed us to have a better understanding of the valuation process made by the respondents that involve two decisions: the participation decision and the quantity decision.

Results show that, although access to online learning from home was not a major barrier, overall, the level of satisfaction with the online learning received was low. In fact, a majority of the respondents thought that the abrupt transition to online learning was harmful to them. Thus, not surprisingly, those respondents willing to accept a monetary compensation for the loss of well-being resulting from the transition to online learning accounted for 77% of the sample. The mean WTA estimate for the entire sample was €511.6, i.e. this is the compensation that the average respondents would be willing to accept for the loss of well-being resulting from the shift from face-to-face learning to online learning. A comparison of our results with previous studies reveals some remarkable differences. Indeed, while we have used a contingent valuation framework to value the loss of well-being resulting from the transition to online learning, Aucejo et al. (2021) and Wang et al. (2021) used a different approach (choice experiment) to obtain student’s WTP for in-person classes versus the online alternative. In particular, Aucejo et al. (2021) obtained that the average student was willing to pay $544 for in-class instruction (relative to a remote format) and $1,049 for on-campus social activities, while Wang et al. (2021) obtained a value of $1,190. So, considering that, as a result of the pandemic, there is a prevailing sense that universities did not deliver to students what they came to these institutions for in the first place, our research is unique in the respect that no previous study has attempted to value in a WTA framework the loss of well-being resulting from the transition into online learning.

Expanding the sample’s values for the affected population have shown that in the case of the University of Valencia, the loss of well-being accounted for €16.9m, while for the University of Alcalá de Henares, the loss was lower (€9.7m). Although controversial (Hausman, 2012), monetization is necessary if we want to have a complete picture of the impact that the transition to online learning have had on the well-being of higher-education students; otherwise, policy decisions that ignore these values could be incomplete and misleading. In addition, these monetary estimates have passed some minimal tests of theoretical validity since results show with reasonable explanatory power that, as expected, WTA decreases with income and age, and it increases if some member of the family unit is currently unemployed or in other words, results conform to the underlying principles of economic theory (Mitchell and Carson, 1989). Indeed, as pointed out by Carson (2000), this provides evidence in support of the proposition that the survey has measured the intended construct. Results also show that, despite the fact that apparently, respondents exhibit a high degree of COVID-19 awareness, surprisingly, this variable is not statistically significant in explaining neither of the two decisions made by the respondent.

Finally, there are some limitations of this study that should be noted. Firstly, although we have combined data from two different universities to have a better understanding of this phenomenon, results are specific to this study and accordingly cannot be generalized to the entire university system since other Spanish universities can differ in size, in the number and type of majors that are offered and especially in which type of online tools and instructional strategies were used when transitioning to online learning. Secondly, the estimates obtained should be taken with caution since we cannot fully guarantee the representativeness of the sample due to the possible presence of self-selection bias. However, it does not necessarily mean that the contribution of this study is not relevant since it provides a minimum basis for informing decision-making in this context of exceptionality created by the pandemic while paving the way for future research in this area of higher education. This is especially true if we consider that, as a result of the expansion of digital infrastructure in response to the COVID-19 pandemic, hybrid forms of education are predicted to stay (Komljenovic, 2020). Besides the health crisis, this study can be also carried out under other real or fictitious circumstances that might affect the teaching-learning process. And thirdly, although this is not necessarily a limitation of the study, it is imperative to be realistic and to consider the feasibility of implementing these monetary compensations in the context of the COVID-19 crisis. In Spain, higher education is considered a public service and accordingly, the Spanish university system is mostly based on public funding from regional governments. As a result, public funding is crucial to cover the total cost of the service since tuition fees are very low. However, in recent years, austerity policies have led to a reduction in public funding, and Spanish universities have, therefore, been forced into debt. With regional governments at the forefront of the COVID-19 pandemic, prioritizing the health and safety of people, it is expected that this state of debt of the Spanish public universities will continue in the near future. Hence the outlook for such compensations seems to be bleak, at least in the short term, considering that today universities are more dependent on regional governments than ever.

Figures

Distribution of WTA in 200-Euro intervals

Figure 1.

Distribution of WTA in 200-Euro intervals

Statements for measuring COVID-19 concern and their descriptive statistics

Attitudinal questions Mean (SD) ++
The COVID-19 pandemic is a threat to humanity 4.07 (1.01) 75.4
I have followed the health crisis caused by the COVID-19 with great attention in the media 3.85 (1.05) 68.7
I am feeling sadness and pain when watching TV or reading in the newspapers about the problems caused by the COVID-19 health crisis 3.86 (1.09) 65.5
Even today, the vast majority of people do not act responsibly to prevent the spread or regrowth of the COVID-19 pandemic 4.26 (0.88) 82.1
I think that the current situation is temporary and that in autumn/winter, it is very possible that the severity of the COVID-19 pandemic will be even greater 3.73 (1.07) 62.7
Economic globalization is largely responsible for the COVID-19 pandemic because of the free movement of people and goods 3.53 (1.21) 57.5
To avoid the spread of the COVID-19 virus, restrictions on free movement of people should be implemented, even if this reduces the number of jobs in the economy 2.79 (1.18) 29.4
It is still true that politicians do much too little to protect people against the COVID-19 3.89 (1.15) 65.2
The human and material resources used to deal with the health crisis caused by the COVID-19 have been insufficient 4.29 (0.85) 84.6
Notes:

++ means percentage of respondents that answered “agree” or “strongly agree” on the five-point Likert scale

Mean WTA estimates

University Percentage WTA > 0 Mean WTA (€)
University of Valencia 80.1 448.4 (300.3)
University of Alcalá de Henares 73.6 595.7 (441.2)
Total 77.2 511.6 (374.2)
Note:

Standard deviations are in parentheses

Explanatory variables

Variable Definition Mean (SD) % of 1s
AGE Respondent’s age 21.7 (4.3)
GENDER 1 if the respondent is male, 0 otherwise 35.6
SPANISH 1 if the respondents have Spanish nationality, 0 otherwise (European Union and the rest of the world) 89.8
YEARS_CAMP Number of years spent in college 2.7 (1.5)
SCHOLARSHIP 1 if the respondent has received a scholarship, 0 otherwise 37.3
ACADEMIC_PER Respondent’s self-rated academic performance on a five-point scale (1 = very bad; 2 = bad; 3 = normal; 4 = good; 5 = very good) 3.5 (0.88)
REMEMBER_TUI 1 if the respondent remembered how much was the tuition fee that she paid the past academic year, 0 otherwise 81.7
ELEARNING_SAT Respondent’s self-reported e-learning satisfaction on an 11-point scale where “0” means “completely dissatisfied” and “10” means “completely satisfied” 4.8 (2.5)
INTERNET_QUA Respondent’s self-reported quality of internet access on a five-point scale (1 = very poor; 2 = poor; 3 = fair; 4 = good; 5 = excellent) 3.7 (1.1)
VALENCIA 1 if the respondent is enrolled at the University of Valencia, 0 otherwise 55.2
STEM 1 if the student is in a science, technology, engineering and mathematics major (STEM), 0 otherwise 25.2
INCOME Respondent’s household monthly income after taxes in 14 €300-intervals ranging from interval 1 (<€300) to interval 14 (>€3900). Net household income was calculated as the average of the midpoints of the selected income intervals 1,810.9 (775.2)
UNEMPLOYED 1 if some member of the family unit to which the respondent belongs is unemployed, 0 otherwise 37.9
COVID-19 1 if the respondent is highly COVID-19 concerned, 0 otherwise. Highly COVID-19 concerned, in this case, means that the respondents answered “agree” or “strongly agree” when asked about the different statements for measuring “covid-19 awareness” shown in Table 1 5.4
ENVIRONMENTALIST 1 if respondent is member of an environmentalist group, 0 otherwise 7.0
STUDENT_AS 1 if the respondents belong to a student association, 0 otherwise 6.5
VOLUNTEERING 1 if the respondent volunteers, 0 otherwise 27.2
COMMUNITY 1 if respondent belongs to a community association, 0 otherwise 12.5

Regressions results: double-hurdle versus Tobit

Variable Tobit regression Double-hurdle regression
Tobit coef. Marginal effects Probit coef. Marginal effects Truncated coef. Marginal effects
AGE −5.9576 (4.9023) −3.2347 (2.6607) −0.0298** (0.0131) −0.0083** (0.0027) 7.9269 (7.7470) 4.1957 (1.1982)
GENDER 30.9165 (40.2959) 16.7864 (21.8826) 0.0295 (0.1231) 0.0082 (0.0026) 44.1236 (63.1317) 23.3548 (6.6699)
YEARS_CAMP 15.4970 (13.9375) 8.4142 (7.5725) −0.0415 (0.0401) −0.0115 (0.0037) 46.0756** (21.7888) 24.3880** (6.9650)
SCHOLARSHIP −107.9179** (43.1935) −58.5948** (23.4441) −0.4103*** (0.1293) −0.1144*** (0.0372) −23.5568 67.7814 −12.4687 (3.5609)
ACADEMIC_PER −26.3064 (22.7196) −14.2833 (12.3408) 0.0188 (0.0690) 0.0052 (0.0017) −35.6512* (83.6706) −33.6432* (9.6082)
REMEMBER_TUI 118.2147** (51.5889) 64.1855** (27.9832) 0.4442*** (0.1453) 0.1239*** (0.0403) −63.5611 (35.6349) −18.8703 (5.38921)
ELEARNING_SAT −37.4662*** (8.3174) −20.3425*** (4.5127) −0.1213*** (0.0246) −0.0338*** (0.0110) −12.1721 (13.5260) −6.4427 (1.8399)
INTERNET_QUA −14.4275 (18.3125) −7.8335 (9.9425) −0.0488 (0.0570) −0.0136 (0.0044) −10.4423 (28.5609) −5.5271 (1.5785)
VALENCIA −119.1794*** (45.3820) −64.7093*** (24.7075) 0.0914 (0.1348) 0.0044 (0.0083) −360.3844*** (79.7946) −190.7531*** (54.4774)
STEM 158.1987*** (51.5600) 85.8952*** (28.0292) 0.2935* (0.1634) 0.0819* (0.0266) 256.5854*** (85.9467) 135.8118*** (38.7867)
INCOME −0.0761*** (0.0277) −0.0413*** (01506) −0.0000 (0.0000) −0.0000 (0.0000) −0.13470*** (0.0456) −0.0713*** (0.0203)
UNEMPLOYED 112.7095*** (41.1888) 61.1965*** (22.4003) 0.1444 (0.1269) 0.0402 (0.0131) 194.3363*** (63.9535) 102.8631*** (29.37681)
COVID19 3.3578 (89.2312) 1.8231 (48.4487) 0.2568 (0.2837) 0.0716 (0.0233) −19.2663 (139.0479) −10.1977 (2.9123)
ENVIRONMENTALIST −64.8473 (76.7701) 35.2093 (41.6889) −0.0975 (0.2322) −0.0272 (0.0088) −42.6747 (124.5632) −22.5879 (6.4509)
STUDENT_AS −32.2364 (80.4063) 17.5030 (43.6517) −0.2933 (0.2309) −0.0818 (0.0266) 138.8656 (118.2787) 73.5022 (20.9916)
VOLUNTEERING 3.2957 (46.2930) 7.2190 (25.1368) −0.1394 (0.1390) −0.0389 (0.0126) 65.96575 (70.5963) 34.9159 9.9717
COMMUNITY_AS 30.1332 (63.30966) 16.3610 (34.3740) −0.2933 (0.2309) 0.0367 (0.0119) −5.1545 (93.7084) −2.7283 (0.7791)
SPANISH −147.5056** (63.8586) 80.0893** (34.6770) −0.4039** (0.2006) −0.1127** (0.0367) −45.6521 (94.0999) −24.1639 (6.9010)
Constant 664.2068** (255.2362) 2.3838*** (0.4967) 664.5414*** (256.8215)
Log likelihood LR chi2(18) Prob > chi2 −3,845.4562 87.08 0.0000 −290.9045 98.16 0.0000 −3,486.6401 60.58 0.000
N 649 645 645
Notes:

Standard errors are in parentheses.

*

Significant at 10%,

**

significant at 5%,

***

significant at 1%

Aggregate loss of well-being

No. of students Mean WTA (€) Aggregate loss (€)
University of Valencia 37,727 448.4 16,905,468.7
University of Alcalá de Henares 16,347 595.7 9,737,907.9

Notes

1.

To this respect, it is worth to mention that a survey of 17,302 college students carried out in the USA and Canada (OneClass, 2020) showed that more than 93% of US students and 85% of Canadian students believed that if classes were fully held online because of COVID-19, tuition fees should be lowered. It was also found that 75% of college students were unhappy with the quality of online classes, 35% had considered withdrawing from school, and many of them were asking for their money back because of the nature of their education had so drastically changed.

2.

STEM: science, technology, engineering and mathematics majors.

References

Almahasees, Z., Mohsen, K. and Amin, M.O. (2021), “Faculty’s and students’ perceptions of online learning during covid-19”, Frontiers in Education, Vol. 6, p. 638470.

Aristovnik, A., Keržic, D., Ravšelj, D., Tomaževic, N. and Umek, L. (2020), “Impacts of the COVID-19 pandemic on life of higher education students: a global perspective”, Sustainability, Vol. 12 No. 20, p. 8438.

Arrow, K., Solow, R., Portney, P., Leamer, E., Radner, R. and Schumar, H. (1993), “Report of NOAA panel on contingent valuation”, Federal Register, Vol. 58, pp. 4601-4614.

Aucejo, E.M., French, J.F. and Zafar, B. (2021), “Estimating students’ valuation for college experiences”, National Bureau of Economic Research Working Paper No. 28511.

Bateman, I.J., Brett, H.D., Georgiou, S. and Lake, I. (2006), “The aggregation of environmental benefit values: welfare measures, distance decay and total WTP”, Ecological Economics, Vol. 60 No. 2, pp. 450-460.

Bethlehem, J. (2010), “Selection bias in web surveys”, International Statistical Review, Vol. 78 No. 2, pp. 161-188.

Bolumole, M. (2020), “Student life in the age of COVID-19”, Higher Education Research and Development, Vol. 39 No. 7, pp. 1357-1361.

Bonnichsen, O. and Olsen, S.B. (2016), “Correcting for non-response bias in contingent valuation surveys concerning environmental non-market goods: an empirical investigation using an online panel”, Journal of Environmental Planning and Management, Vol. 59 No. 2, pp. 245-262.

Cao, W., Fang, Z., Hou, G., Han, M., Xu, X., Dong, J. and Zheng, J. (2020), “The psychological impact of the COVID-19 epidemic on college students in China”, Psychiatry Research, Vol. 287, p. 112934.

Carson, R.T. (1991), “Constructed markets”, in Braden, J.B. and Kolstad, C.D. (Eds), Measuring the Demand for Environmental Quality, Elsevier, North-Holland, Amsterdam.

Carson, R.T. (2000), “Contingent valuation: a user’s guide”, Environmental Science and Technology, Vol. 34 No. 8, pp. 1413-1418.

Champ, P.A. (2017), “Collecting nonmarket valuation data”, in Champ, P., Boyle, K. and Brown, T. (Eds), A Primer on Nonmarket Valuation. The Economics of Non-Market Goods and Resources, Springer, Dordrecht.

Cragg, J. (1971), “Some statistical models for limited dependent variables with application to the demand for durable goods”, Econometrica, Vol. 39 No. 5, pp. 829-844.

Dhawan, S. (2020), “Online learning: a panacea in the time of COVID-19 crisis”, Journal of Educational Technology Systems, Vol. 49 No. 1, pp. 5-22.

Giannoccaro, C., de Gennaro, B.C., De Meo, E. and Prosperi, M. (2017), “Assessing farmers' willingness to supply biomass as energy feedstock: cereal straw in Apulia (Italy)”, Energy Economics, Vol. 61, pp. 179-185.

Goedegebuure, L. and Meek, L. (2021), “Crisis – what crisis?”, Studies in Higher Education, Vol. 46 No. 1, pp. 1-4.

González, T., de la Rubia, M.A., Hinncz, M., Comas-López, M., Subirats, L., Fort, S. and Sacha, G.M. (2020), “Influence of covid-19 confinement on students’ performance in higher education”, PLoS One, Vol. 15 No. 10, p. e023990.

Green, W., Anderson, V., Tait, K. and Tran, L.T. (2020), “Precarity, fear and hope: reflecting and imagining in higher education during a global pandemic”, Higher Education Research and Development, Vol. 39 No. 7, pp. 1309-1312.

Green, W.H. (2012), Econometric Analysis, Prentice Hall, Upper Saddle River, NJ.

Groothuis, P.A., Van Houtven, G. and Whitehead, J.C. (1998), “Using contingent valuation to measure the compensation required to gain community acceptance of a lulu: the case of a hazardous waste disposal facility”, Public Finance Review, Vol. 26 No. 3, pp. 231-249.

Häfner, S.J. (2020), “Pandemic number five – latest insights into the COVID-19 crisis”, Biomedical Journal, Vol. 43 No. 4, pp. 305-310.

Hanemann, W.M. (1991), “Willingness to pay and willingness to accept: how much can they differ?”, American Economic Review, Vol. 81 No. 3, pp. 635-647.

Hausman, J. (2012), “Contingent valuation: from dubious to hopeless”, Journal of Economic Perspectives, Vol. 26 No. 4, pp. 43-56, doi: 10.1257/jep.26.4.43.

Heckman, J.J. (1979), “Sample selection bias as a specification model”, Econometrica, Vol. 47 No. 1, pp. 153-161.

Hudson, D., Seah, L.-H., Hite, D. and Haab, T. (2003), “Telephone presurveys, self-selection and non-response bias to mail and internet surveys in economic research”, Applied Economics Letters, Vol. 11 No. 4, pp. 237-240.

Johansson, P.-O. (1993), Cost-Benefit Analysis of Environmental Change, Cambridge University Press, Cambridge.

Kahneman, D. and Tversky, A. (1979), “Prospect theory: an analysis of decision under risk”, Econometrica, Vol. 47 No. 2, pp. 263-291.

Knetsch, J.L. (2010), “Values of gains and losses: reference states and choice of measure”, Environmental and Resource Economics, Vol. 46 No. 2, pp. 179-188.

Knipe, D.M., Levy, O., Fitzgerald, K.A. and Mühlberger, E. (2020), “Ensuring vaccine safety”, Science, Vol. 370 No. 6522, pp. 1274-1275, doi: 10.1126/science.abf0357.

Komljenovic, J. (2020), “The future of value in digitalised higher education: why data privacy should not be our biggest concern”, Higher Education, Vol. 83 No. 1, pp. 119-135.

Leask, B. (2020), “Embracing the possibilities of disruption”, Higher Education Research & Development, Vol. 39 No. 7, pp. 1388-1391, doi: 10.1080/07294360.2020.1824211.

Liebe, U., Glenk, K., Oehlmann, M. and Meyerhoff, J. (2015), “Does the use of mobile devices (tablets and smartphones) affect survey quality and choice behavior in web surveys?”, Journal of Choice Modelling, Vol. 14, pp. 17-31.

Lloyd-Smith, P. and Adamowicz, W. (2018), “Can stated measures of willingness-to-accept be valid? Evidence from laboratory experiments”, Journal of Environmental Economics and Management, Vol. 91, pp. 133-149.

Loomis, J.B. (1987), “Expanding contingent value sample estimates to aggregate benefit estimates: current practices and proposed solutions”, Land Economics, Vol. 63 No. 4, pp. 396-402, doi: 10.2307/3146296.

Maddala, G.S. (1983), Limited-Dependent and Qualitative Variables in Econometrics, Cambridge University Press, Cambridge.

Messonnier, M.L., Bergstrom, J.C., Cornwell, C.M., Teasley, R.J. and Cordell, H.K. (2000), “Survey response-related biases in contingent valuation: concepts, remedies and empirical application to valuing aquatic plant management”, American Journal of Agricultural Economics, Vol. 83 No. 2, pp. 438-450.

Meyerhoff, J. and Liebe, U. (2006), “Protest beliefs in contingent valuation: explaining their motivation”, Ecological Economics, Vol. 57 No. 4, pp. 583-594.

Mitchell, R.C. and Carson, R.T. (1989), Using Surveys to Value Public Goods: The Contingent Valuation Method, Resources for the Future, Washington, DC.

Morgan, D. (1988), Focus Groups as Qualitative Research, Sage Publications, Newbury Park, CA.

Morrison, M.D., Blamey, R.K. and Bennett, J.W. (2000), “Minimizing payment vehicle bias in contingent valuation studies”, Environmental and Resource Economics, Vol. 16 No. 4, pp. 407-422.

Murphy, M.P.A. (2020), “COVID-19 and emergency eLearning: consequences of the securitization of higher education for post-pandemic pedagogy”, Contemporary Security Policy, Vol. 41 No. 3, pp. 492-505.

Nguyen, J.G., Keuseman, K.J. and Humston, J.J. (2020), “Minimize online cheating for online assessments during COVID-19 pandemic”, Journal of Chemical Education, Vol. 97 No. 9, pp. 3429-3435.

Nguyen, K.T., Knetsch, J.L. and Mahasuweerachai, P. (2021), “WTP or WTA: a means of determining the appropriate welfare measure of positive and negative changes when preferences are reference dependent”, Environmental and Resource Economics, Vol. 78 No. 4, pp. 615-633.

OneClass (2020), “Students think online classes should mean lower tuition” (OneClass.com), available at: https://oneclass.com/blog/featured/180637-students-think-online-classes-should-mean-lower-tuition.en.html) (accessed 15 July 2020).

Rizun, M. and Strzelecki, A. (2020), “Students’ acceptance of the COVID-19 impact on shifting higher education to distance learning in Poland”, International Journal of Environmental Research and Public Health, Vol. 17 No. 18, p. 6468.

Sandorf, E.D., Aanesen, M. and Navrud, S. (2016), “Valuing unfamiliar and complex environmental goods: a comparison of valuation workshops and internet panel surveys with videos”, Ecological Economics, Vol. 129, pp. 50-61, doi: 10.1016/j.ecolecon.2016.06.008.

Santagata, W. and Signorello, G. (2000), “Contingent valuation of a cultural public good and policy design: the case of Napoli Musei Aperti”, Journal of Cultural Economics, Vol. 24 No. 3, pp. 181-204.

Tesar, M. (2020), “Towards a post-Covid-19 ‘new normality?’: physical and social distancing, the move to online and higher education”, Policy Futures in Education, Vol. 18 No. 5, pp. 556-559.

The Economist (2021), “How well will vaccines work?”, (Leaders), available at: www.economist.com/leaders/2021/02/13/how-well-will-vaccines-work) (accessed 13 February 2021).

Tobin, J. (1958), “Estimation of relationships for limits dependent variables”, Econometrica, Vol. 26 No. 1, pp. 24-36.

Tuncel, T. and Hammitt, J.K. (2014), “A new Meta-analysis on the WTP/WTA disparity”, Journal of Environmental Economics and Management, Vol. 68 No. 1, pp. 175-187.

Vossler, C.A. and Watson, S.B. (2013), “Understanding the consequences of consequentiality: testing the validity of stated preferences in the field”, Journal of Economic Behavior and Organization, Vol. 86, pp. 137-147.

Wang, H.H., Hua, Y. and Wilson, C. (2021), “The impact of online teaching due to COVID-19 on students’ valuation of college education: a study on college students’ willingness-to-pay for alternative course options”, 2021 Agricultural and Applied Economics Association Annual Meeting, August 1-3.

Watermeyer, R., Crick, T., Knight, K. and Goodall, J. (2021), “COVID-19 and digital disruption in UK universities: afflictions and affordances of emergency online migration”, Higher Education, Vol. 81 No. 3, pp. 623-641.

Whitehead, J.C. (2006), “A practitioner’s primer on contingent valuation”, in Alberini, A. and Kahn, J. (Eds), Contingent Valuation Handbook, Edward Elgar, Cheltenham.

Whitehead, J.C., Groothuis, P.A. and Blomquist, G.C. (1993), “Testing for nonresponse and sample selection bias in contingent valuation: analysis of a combination phone/mail survey”, Economics Letters, Vol. 41 No. 2, pp. 215-230.

Whittington, D., Adamowicz, W. and Lloyd-Smith, P. (2017), “Asking willingness-to-accept questions in stated preference surveys: a review and research agenda”, Annual Review of Resource Economics, Vol. 9 No. 1, pp. 317-336.

Willig, R. (1976), “Consumer’s surplus without apology”, American Economic Review, Vol. 66 No. 4, pp. 589-597.

Wolfgang, M., Spitzer, H. and Musslick, S. (2021), “Academic performance of K-12 students in an online-learning environment for mathematics increased during de shutdown of schools in wake of the COVID-19 pandemic”, PLoS ONE, Vol. 16 No. 8, p. e0255629.

Zalat, M.M., Hamed, M.S. and Bolbol, S.A. (2021), “The experiences, challenges and acceptance of e-learning as a tool for teaching during the COVID-19 pandemic among university medical staff”, PLoS One, Vol. 16 No. 3, p. e0248758.

Zhao, J. and Kling, C.L. (2001), “A new explanation for the WTP/WTA disparity”, Economics Letters, Vol. 73 No. 3, pp. 293-300.

Acknowledgements

Special thanks are due to the editor and to four anonymous reviewers for their helpful comments.The authors received no financial support for the research, authorship, and publication of this article.

Corresponding author

Salvador del Saz-Salazar can be contacted at: Salvador.Saz@uv.es

Related articles