National Vaccination and Local Intervention Impacts on COVID-19 Cases

Abstract

COVID-19, as a global pandemic, has spread across Indonesia. Jakarta, as the capital of Indonesia, is the province with the most positive cases. The government has issued various guidelines, both at the central and regional levels. Since it began in 2021, the planned new measures, called ‘Pemberlakuan Pembatasan Kegiatan Masyarakat Darurat’, or PPKM emergency public activity restrictions, began with the possibility that the number of active cases might decrease. Accordingly, global vaccinations were also carried out, as they were in Indonesia. However, the first phase prioritized frontline health workers and high-risk elderly people. This study conducted a causal impact analysis to determine the effectiveness of PPKM in Jakarta and its vaccination program against the increase in daily new cases. Based on this test, PPKM showed a significant effect on the addition of daily new cases and recovered cases. Conversely, the vaccination program only had a significant impact on recovered cases. A forecast of the COVID-19 cases was conducted and indicated that the daily new cases showed a negative trend, although it fluctuated for the next 7 days, while death and recovered cases continued to increase. Hence, it can be said that the vaccination program has still not shown its effectiveness in decreasing the number of daily new cases while PPKM is quite effective in suppressing new cases.

1. Introduction

The coronavirus disease (COVID-19) has been deemed a global pandemic by World Health Organization (WHO). Coronaviruses are enveloped, positive, single-stranded large RNA viruses that infect humans and a wide range of animals [1]. The virus was called 2019-nCoV when it was first discovered in Wuhan, China, in people who had been exposed to seafood or wet markets. However, attempts to identify potential intermediate hosts appear to have been neglected in Wuhan, and resulted in rapid transmission routes emerging in various countries [2]. Previous MERS and SARS incidents, that is human-to-human transmission, occurred through droplets and contact with contaminated objects; same transmission was seen in COVID-19 [3,4,5]. In order to prevent the spread of this virus, people needed to regularly wash their hands using soap, cook meat and eggs well, and avoid close contact with or keep a distance from people who have symptoms of respiratory diseases [6]. As of 8 April 2020, 22,073 cases of COVID-19 have been documented from 52 countries; Indonesia is included, here, as its cases have been reported to the WHO by health workers [7].

On 2 March 2020, Indonesia had reported two confirmed cases of COVID-19 at first. As of 29 March 2020, thus number had increased to 1285 cases in 30 provinces. The five provinces with the highest COVID-19 cases were Jakarta (675), West Java (149), Banten (106), East Java (90), and Central Java (63) [8]. Indonesia’s health facilities were not ready to deal with COVID-19 and massive and serious preparations had to be made to reduce the spread of the COVID-19 disease [2,9,10,11,12,13].

According to the Indonesian Ministry of Health, the ratio of the number of beds to the total population is 1.21:1000, which means that per 1000 membrs of the population there are only 1.21 treatment beds in hospitals. In 2020, Indonesia had only 310,000 hospital beds for a population of around 260 million, meaning Indonesia’s health facilities were not ready to deal with COVID-19 because the ratio was still far from the WHO’s recommendation, 5:1000.

According to regulation for Law No 6 2018 on Health Quarantine, there are four types of health quarantine including home quarantine, regional quarantine, hospital quarantine, and large-scale social restrictions or ‘Pembatasan Sosial Berskala Besar’ (PSBB). PSBB was a set of restrictions carried out in certain areas where an increase in COVID-19 cases was suspected. PSBB implementation, meant to control the increase in positive cases in different cities or provinces, therefore meant economic recovery could begin immediately.

At the end of March 2020, the president of Indonesia, Joko Widodo, finally decided to implement PSBB in cities and provinces, not only in the quarantine regions. The government appealed to Indonesian citizens to stay home [8,11,14,15,16,17]. This PSBB was carried out in the hope that the rate of transmission could be slowed down and then stopped, leaving optimal health services available for patients, and allowing the recovery of the health system, economic activities, social services and other sectors from the COVID-19 pandemic [18,19].

Figure 1 depicts that COVID-19’s occurrence in Jakarta had increased rapidly from the beginning of the confirmed cases in March to early September 2020. The number of recovered cases increased, as confirmed, while the number of cumulative deaths was stable. On 26 March 2020, 56 cases of COVID-19 were found; then, a week later, the total number of new COVID-19 cases amounted to 364, with a total of 87 people having died and 37 people having recovered. Since the first phase, PSBB, beginning on 10 April in Jakarta, counted an increase in the number of COVID-19 cases from 1810 to 3399 cases by 22 April. By then, the number of deaths also jumped, to 308 from April 10’s 156 cases, and the number of people who recovered increased from 82 to 291.

From the start of COVID-19 until 1 April 2021, East Kalimantan (63,093), East Java (138,715), Central Java (167,664), West Java (248,396), and DKI Jakarta (380,706) were the five provinces with the highest COVID-19 cases in Indonesia. This means that DKI Jakarta, the capital of Indonesia, was the province with the highest positive cases of COVID-19 among the other provinces. However, the people of Jakarta had very low awareness of implemented government policies to reduce the spread rate of COVID-19. Activity policies under PSBB included: 75% of employees (private and government offices) were to WFH (work from home), learning or educational activities were to be carried out online, shopping centers that sell basic needs were only open until 8:00 p.m., and access to public areas was temporarily suspended.

An increase in COVID-19 cases proved that the PSBB has not been implemented effectively and many people still failed to comply with government regulations for preventing the spread of COVID-19. The government had also extended the PSBB period five times, including a transitional PSBB, which spanned 3–16 July 2020, 17–30 July 2020, 30 July–14 August 2020, 14–27 August 2020, and 27 August–10 September 2020. The implementation of the transitional PSBB was almost the same as that of the PSBB, only with some concessions. (1) Previously, learning activities in schools had not been allowed, but in the transition period, face-to-face schooling, with strict health protocols, was allowed; (2) before the transition, 75% employees worked from home, and after the transition, 50% of employees were allowed to work from the office; (3) church and mosque services were opened; and (4) the opening of public markets, with a capacity of 50%, was allowed, where previously, only markets that sold basic necessities could open.

On 13 September 2020, the government returned to implementing a tight PSBB, also known as the “emergency brake” for 2 weeks, from 14–27 September 2020, and was extended until 11 October 2020. Then, the Jakarta government decided to implement a transitional PSBB through December 2020 because there had still been a surge in virus transmission, with hundreds of new family clusters emerging in the city. The timeline of the implementation of the Jakarta government’s policies in preventing the spread of COVID-19 can be seen in Figure 2.

At the end of 2020, a ban on crowd-pulling events and New Year’s Eve celebrations was imposed by the government, but the increase in confirmed cases of COVID-19 was inevitable, as supported by Figure 2. In the midst of the increasing number of COVID-19 cases in Jakarta, good news emerged, regarding the start of a vaccination program in Indonesia. On 13 January 2021, the president of Indonesia was the first person to receive the Sinovac vaccine. This vaccine was prioritized for people aged 18–59 years. The first phase of vaccination prioritized health workers and support staff. The second phase of mass vaccination targeted priority groups other than medical works, such as the public workforce, including workers in transportation, the tourism sector, public ports and stations, electric companies, banks, water companies, and any officials providing community service.

Until 13 July 2021, the vaccination process was carried out evenly; it included the public, and the process for children had even started. The overall vaccination target was 208,265,720 people, from health workers, the elderly, civil servants, vulnerable communities, the general public, and ages 12–17. Based on this target, 19% received one dose of vaccine. Figure 3 represents the number of active cases after each measure. A total of 38,909,433 people received the first dose and 15,611,554 of them received the full dose. The details of the vaccination procedure for each group are as follows: (1) of 1,468,764 registered health workers, 97.55% received the full dose; (2) 14.53% of 21,553,118 elderly people received the full dose; (3) of 17,327,167 office workers, 52.64% received full doses; (4) 2,159,098 initial doses and 1,579,457 full doses were given to educators; (5) from the general public and vulnerable communities, only 1.36% received full doses, of 141,211,181 people; (6) from the age group of 12- to 17-year-olds, comprising 26,705,490 people, only 0.93% received the first dose, as the vaccination process for this group had only started in early July.

By 18 March 2021, 4.84 million Indonesians received at least one dose of the COVID-19 vaccine and 1.95 million people had been fully vaccinated against COVID-19, after completing three vaccination phases since 13 January 2021. By 13 June 2021, 20.16 million or 7.4% Indonesians had received one shot, while 4.5% of those are fully vaccinated. The start of vaccination resulted in a reduced number of COVID-19 cases in Indonesia and also in Jakarta specifically. It was also shown, in Figure 1, from January to March 2021 that the number of cases in Jakarta experienced a downward trend.

CoronaVac, by Sinovac, is an inactivated viral vaccine with an alum adjuvant, administered in two shots, at two weeks apart in general and four weeks apart for the elderly, in order to have sufficient efficacy [20]. The efficacy of the vaccine, then, appears a few weeks later [21]. Final blood tests are expected in December, however, in order to review longer-term efficacy and side effects collection may be extended in three months [12]. Sinovac showed 91.25% efficacy in a trial in Turkey, and Brazil reported over 50% efficacy, although detailed results have been delayed [22]. The start of vaccination has resulted in a reduced number of COVID-19 cases in Indonesia and also in Jakarta. It was also shown in Figure 1 that, from January to March 2021, the number of cases experienced a downward trend.

Apart from the vaccination program, the Jakarta government also introduced new measures to restrict activities; this enforcement of public activity restrictions, called ‘Pemberlakuan Pembatasan Kegiatan Masyarakat’ (PPKM), implemented from 11 to 25 January 2021, under Home Ministerial Instruction No. 1/2021, as shown in Figure 3. The PPKM mechanism is different from PSBB. Under PSBB, the initial restriction initiatives came from the local government, but PPKM was overseen by the central government.

The central government established initial restriction criteria for all regions. Areas that were included in the criteria had to apply these restrictions to their community activities. These restrictions were slightly looser than requirements had been under the PSBB policy. On 9 February 2021, the government enforced the local scale of community activity, or micro-scale, of PPKM. It would be formed at the village level, to aid health centers in handling isolated COVID-19 patients. The implementation of micro-PPKM showed significant results in reducing the number of COVID-19 cases in Jakarta, and therefore was extended to 5 April 2021.

Based on the facts above, it was somewhat important to predict future cases based on what had happened in the immediate past, especially for a novel infectious disease, in order to warrant the availability of sufficient supplies of personal equipment, deliberation about the health care workforce and other health care resources capability, and how to stabilize preventive safety guidelines while keeping businesses open, to stabilize the economy. Modelling and forecasting infectious disease epidemics are characterized by different approaches, such as (1) mechanistic models based on the SEIR (Susceptible, Exposed, Infected, and Recovered) framework; (2) time series prediction models; and (3) agent-based models (i.e. simulating individual activities for a population) [23,24]. For this study, we mainly focus on short-term predictions based on the time series model for the next week. Before performing the forecast, we try to examine the effectiveness of the PPKM implemented in Jakarta and the national vaccination program to reduce the daily new cases, as well as their effects on the daily recovered cases and daily death cases, as a novelty.

This study aims to determine the effectiveness of PPKM in Jakarta and the vaccination program against the increase in daily new cases, conducting a causal impact analysis using only the two month time period following the vaccination program; then we use neural network models that are suitable for capturing patterns in the data for prediction. In this study, we are avoiding micro-assumptions on a large number of unknown variables, such as death rates and transmissibility, due to the limitations of available data and understanding of the death rates and transmissibility. We also the predictions against actual values then evaluate the model through several error metrics, as one of the four main principles in health forecasting is evaluation and measuring errors in transmissibility [25]. Currently, Indonesia is preparing for the new normal [18,19,26,27].

2. Research Method

2.1. Data Description

The data used in this study are historical data of daily new cases, recovered cases, and death cases of COVID-19 in Jakarta, Indonesia from 1 March 2020 to 18 March 2021. The dataset, of open data, was provided by the COVID-19 task force formed by the Indonesian government. This dataset will be used in order to test whether the implementation of PPKM in Jakarta and its vaccination program were effected a decline in the active cases in Indonesia, specifically in Jakarta, which is the province with the highest case count. Two months of data were used for taking into account the efficacy of the vaccine and, if from after March, other interventions are expected, specifically from the celebration of religious holidays in Indonesia that cause changes in patterns of community mobility between regions.

We have demonstrated a causal impact analysis to test the effectiveness of government measures. This method is widely used to investigate the effectiveness of the new campaigns or programs in the business sector, using a time series dataset; therefore we wish to do the same analyses with like-structured data from different sectors. For short-term forecasting, many researchers have used simple methods to predict their variables because these methods are robust, have meaningful results, and are not challenging computationally [28]. However, we try to make a prediction using a neural network model, since this approach has gained renown, recently, due to its accuracy.

2.2. Causal Impact Analysis and Neural Networks

Causal impact analysis performs causal inference using Bayesian structural time series models. It implements an approach to estimate the causal effect of a designed intervention on a time series [29]. In particular, the model assumes that the time series of the treated unit can be explained in terms of a set of covariates that were themselves not affected by the intervention whose causal effect we are interested in. Structural time series models are state-space models for time series data [30]. They can be defined in terms of a pair of equations

$$y_t=Z_t^T{\alpha }_t+{\epsilon }_t$$

(1)

$${\alpha }_{t+1}=T_t{\alpha }_t+R_t{\eta }_t$$

(2)

where ${\epsilon }_t ~ N\left(0,{\sigma }_t^2\right)$ and ${\eta }_t ~ N\left(0,Q_t\right)$ are independent of all other unknowns. Equation (1) is the observation equation; it links the observed data $y_t$ to a latent d-dimensional state vector ${\alpha }_t$. Equation (2) is the state equation; it governs the evolution of the state vector ${\alpha }_t$ through time. In this paper, $y_t$ is a scalar observation, $Z_t$ is a d-dimensional output vector, $T_t$ is a d × d transition matrix, $R_t$ is a d × q control matrix, ${\epsilon }_t$ is a scalar observation error with noise variance ${\sigma }_t$, and ${\eta }_t$ is a q-dimensional system error with a q × q state-diffusion matrix $Q_t$, where $\mathrm{q}\le \mathrm{d}$.

In R software, this analysis is performed with the CausalImpact package [31]. The package is designed to make a counterfactual inference as easily as fitting a regression model, but with much more power, provided the assumptions above are met. The package has a single entry point. Given a response time series and a set of control time series, the function constructs a time series model and performs posterior inference on the counterfactual.

A neural network or artificial neural network is a forecasting method based on a simple mathematical model and has a network system that works like the human brain. An artificial neural network is a network consisting of a collection of processing units called “nodes” that are arranged in certain layers. In a neural network model, the predictor variable or the input is at the bottom layer, while the response variable or the output is at the top layer [32,33,34,35,36,37]. A hidden layer that also contains hidden nodes can be added between the input layer and the output layer as an intermediary for processing the nodes to produce better results [38,39,40]. Neural networks are divided into two types of architectures, namely Feed-Forward Neural Networks (FFNNs) and Recurrent Neural Networks (RNNs). The difference between them is that the RNN, which is often used for sequential modelling, has at least one feedback loop, while FFNN does not [33,34,35,41,42].

A feed-forward neural network is fitted with estimations of y as inputs and a single hidden layer of neurons. The inputs are for lags 1 to p and lags m to mP where m = freq (y). Its columns are additionally utilized as inputs if xreg is given. Though, if there are missing values in y or xreg, the involved rows are rejected from the fit. With irregular initial weights, a total revised network is fitted. Recursively, multi-step predictions are processed, although the network is prepared for a one-step prediction.

The fitted model for data with a non-seasonal pattern is inevitably NNAR (p,k), where k is the number of hidden neurons. This is corresponding to an AR (p) with non-linear functions. For data with seasonal patterns, the fitted model is indicated as NNAR (p,P,k) [m], which is similar to ARIMA (p,0,0) (P,0,0) [m] with non-linear functions. The modelled cycles are always symmetric in AR. Nonetheless, the cyclic model in the NNAR has been modelled well to facilitate the irregularity of the cycles. This is the one difference between AR and NNAR [27].

2.3. Metrics Evaluation

After generating a model from the training process, the model will evaluate using the testing data to get the accuracy of the model. The most common selection criteria is the mean absolute percentage error (MAPE) (3); because the value is in the form of a percentage, it is then appropriate to measure the accuracy of a model. Then, in the network validation phase, MAPE is used. Meanwhile, mean absolute error (MAE) (4) and root-mean-square error (RMSE) (5) are used to calculate network accuracy for different models on the same scale [43,44,45,46]. In addition, the RMSE gives a high weight to large errors because errors are squared; thus this metric is useful when large errors are particularly undesirable [47]. Where n is the number of observations, $Y_t$ is the observed value, $\widehat{Y_t}$ is the predicted value.

$$MAPE=\frac{1}{n}{\displaystyle \sum }_{t=1}^n\left|\frac{Y_t-\widehat{Y_t}}{Y_t}\right|\times 100$$

(3)

$$MAE=\frac{1}{n}{\displaystyle \sum }_{t=1}^n\left|Y_t-\widehat{Y_t}\right|$$

(4)

$$RMSE=\sqrt{\frac{1}{n}{\displaystyle \sum }_{t=1}^n{\left(Y_t-\widehat{Y_t}\right)}^2}$$

(5)

3. Results and Discussion

All analyses were performed in open-source R version 3.6.3 with the nnetar function in the forecast package for prediction, and the CausalImpact package for the causal impact analysis. We use a device with 4GB RAM supported by an AMD Ryzen 5 2500U processor. The artificial neural network method used in this study is NNAR. The networks are not based on a well-defined stochastic model, therefore, they did not require any assumptions in their use.

3.1. Causal Impact Analysis

Causal Impact analysis is also performed using R software with the CausalImpact package. This function performs causal inference through counterfactual predictions using a Bayesian structural time series model. This analysis aims to see whether there is a difference in the number of new cases and cases recovered before and after the implementation of vaccination and PPKM in Jakarta through hypothesis testing. We are not including the death cases since no direct relationship of the measures to the death cases. Vaccination started on 13 January 2021, while the Jakarta PPKM started on 9 February 2021. According to the data released by Our World in Data, on the first day, based on the 7-day average and counting by single doses, there were 13,200 doses administered to the priority targets, i.e., elderly people and health workers. On 18 March 2020, the total number of people vaccinated with at least a single dose was 4.84 million people or approximately 0.7% of the population. In addition, 1.95 million people were fully vaccinated.

The vertical grey dashed line in Figure 4 and Figure 5 shows the occurrence of the first vaccine in Jakarta and the implementation of PPKM Jakarta. It is shown that after those interventions the number of new COVID-19 cases decreased until March and the number of recovered cases increased until the end of February.

Based on the p-value in

Table 1. p-Values of each case.

Variable	Daily New Cases	Recovered Cases
Vaccination	0.16649	0.00219
PPKM Jakarta	0.00106	0.00109

with the assumption, α = 0.05, vaccination had a significant impact on the increase in daily recovered cases in Jakarta. On the other hand, although vaccination has indeed succeeded in reducing the number of daily new cases as shown in Figure 6a, the effect was not significant because of the uneven application of the vaccine. The vaccination began to have an impact in February; presently, the number of daily new cases has begun to show a downward trend. Nevertheless, PPKM in Jakarta had a significant impact on daily new cases and daily recovered cases. Figure 6c shows that daily new cases of COVID-19 have experienced a downward trend since the implementation of the PPKM in Jakarta.

In Figure 1, it can be seen that new cases started to rise again in late May due to the end of the Eid celebration period. This indicates that vaccination is not the only solution to suppress increases in COVID-19 cases. The proof is that, with a lax implementation of PSBB, the number of daily cases has continued to increase even though the vaccination program continues. Therefore, the PSBB is still very much needed to prevent cross-region transmission in Indonesia, especially as it approaches the celebration of major holidays. Currently, to suppress the daily number of cases, the regional government has implemented a local lockdown in areas with a significant increase in cases.

The efficacy of the vaccine was observed one month after the administration of the second dose. According to this study, after two months of the vaccination program, the effect was not statistically significant in regard to the addition of new cases, though it was in the increase in the number of recovered cases. However, due to data limitations, no conclusions can be drawn indicating the efficacy of the vaccination program in dealing with the COVID-19 pandemic in Indonesia. Moreover, as shown in previous research, all populations are generally susceptible to SARS-CoV-2.

Thus, this study focused on the province with the highest confirmed cases, as well as the highest population, Jakarta. In addition, the vaccination program focused on this area because, as the capital of Indonesia, Jakarta has high mobility. Since elderly patients with or without underlying comorbidities [48,49] are at high risk, due to rapid migration of the virus to the lung gas exchange units and, furthermore, across multiple organs [50], they are included in the priority group of vaccine recipient.

Recently, the entire world has cooperated, using all available resources and labor, to develop a vaccine against COVID-19. Vaccinations are believed to be an existing solution to control the COVID-19 outbreak, and many vaccines against the current pandemic are being tested in various parts of the world [51,52]. Vaccines based on the structures of the epitope-binding domain, mRNA, and protein S were the most widely explored [53]. However, scientists have predicted that the global vaccination process will take some time and will include a test phase with several safety assessments. In particular, with mass manufacturing, this entire process will take between 12 and 18 months to achieve optimal results [54]. Additionally, once the vaccine is available to the general population, human clinical trials will be needed to demonstrate its overall efficacy and safety. Based on recently discovered epidemiological facts, the nature of the virus, the immune response to the virus, and challenges in vaccine production arise every day [55]. Evidently, the efforts invested in research must be strengthened with urgent applications to combat this pandemic. With the support of this evidence, research on the effectiveness of vaccines for COVID-19 still really need further research, and over a long period of time.

3.2. NNAR Analysis

The lag values in the time series can be used as input for an artificial neural network. NNAR uses a single hidden layer for forecasting univariate time series data. Each hidden layer node performs a single sigmoid transformation of its input. The nnetar function produced a good model by averaging the results of 20 networks with linear output units, but it was not the best. A better model will be produced through a trial-and-error mechanism than the model that is automatically selected by the function itself.

The order of auto-regression indicates the number of lag values upon which the current value of the time series is dependent. Meanwhile, the order of the neural network indicates the number of hidden nodes used in the network. Each node multiplies the input signal with a weight, and each weight is adjusted with the help of the learning process. The nnetar function in R uses the feed-forward algorithm to achieve this.

Both AR and NN orders were obtained by a trial-and-error mechanism, which was validated by forecasting the training data for only one period and comparing it with the actual value of the testing data. Then the compared testing data becomes training data. The same procedure is carried out until as much as the testing data as possible has been analyzed; this process is known as evaluation on a rolling forecasting origin [56,57,58] and obtained results of the validation phase are shown in

Table 2. MAPE value of validation for daily new cases, death cases, and recovered cases.

	Number of Hidden Nodes	Number of Lags
		6	7	8	9	10	11	12	13	14
Daily New Cases	1	31.53	28.73	28.66	28.86	29.63	32.89	38.82	37.97	39.94
	2	28.22	26.97	27.46	26.66	28.03	27.97	29.92	29.26	26.67
	3	19.88	19.02	19.66	18.62	18.37	17.84	16.93	16.03	13.15
	4	14.19	11.24	11.93	11.03	9.74	9.86	8.72	9.49	7.25
	5	9.26	8.2	8.09	6.11	5.86	5.22	5.18	4.54	4.44
	6	5.85	5.9	5.19	4.37	3.99	3.18	2.63	3.58	2.83
	7	5.87	3.7	2.69	3.24	2.42	2.36	2.2	2.29	2.03
	8	3.6	2.76	2.64	1.95	1.99	1.76	1.79	1.71	1.7
	9	3.12	2.14	2.08	1.89	1.77	1.61	1.53	1.62	1.31
	10	2.57	1.8	1.63	1.42	1.46	1.43	1.41	1.29	1.25
	11	2.09	1.95	1.74	1.51	1.37	1.27	1.27	1.28	1.26
	12	1.9	1.55	1.53	1.28	1.28	1.28	1.23	1.2	1.16
	13	1.8	1.48	1.45	1.24	1.25	1.24	1.21	1.2	1.17
	14	1.72	1.48	1.27	1.22	1.21	1.28	1.15	1.14	1.13
Death	1	99.08	104.59	104.89	105.47	104.63	106.52	108.36	110.75	110.61
	2	89.23	88.9	89.96	97.03	91.16	85.81	90.94	84.49	87.03
	3	58.97	57.52	52.62	58.28	60.69	55.34	56.36	60.05	42.11
	4	39.79	38.61	32.31	37.23	36.02	33.87	28.06	26.69	24.47
	5	29.91	24.98	25.71	21.73	21.74	18.23	16.89	13.12	15.7
	6	22.56	18.76	16.32	14.18	14.92	10.89	10.69	10.31	11.8
	7	15.14	14.84	13.85	11.05	11.03	8.8	8.13	8.7	7.33
	8	13.65	12.62	12.67	8.63	8.35	7.06	7.56	7.55	7.72
	9	12.04	11.03	9.76	7.53	7.71	6.65	6.78	6.95	6.63
	10	11.31	10.23	9.42	7.34	6.98	6.36	6.46	6.15	6.3
	11	10.47	9.21	8.6	6.59	6.37	6.12	6.04	6.51	6.11
	12	10.18	9.51	8.33	6.33	6.4	6.08	5.88	6.1	5.95
	13	9.2	8.29	7.97	6.2	6.39	6.2	6.14	5.87	5.89
	14	9.13	8.06	7.72	6.15	6.16	6.12	6.1	5.86	5.98
Recovered	1	25.75	27.65	27.98	27.58	29.84	29.24	29.42	28.98	28.98
	2	29.67	29.63	29.41	29.77	30.53	31.12	30.43	32.01	32.01
	3	27	25.75	25.17	24.92	26.33	23.27	23.34	22.4	22.4
	4	24.76	21.31	20.82	22.36	16.73	15.14	15.58	15.87	15.87
	5	21.67	18.8	17.33	16.44	13.75	12.76	12.17	12.04	12.04
	6	18.97	15.47	13.93	12.9	10.54	10.45	9.89	9.25	9.25
	7	14.35	14.41	10.8	10.93	8.99	9.34	8.28	8.39	8.39
	8	13.09	11.72	10.43	9.84	8.87	8.19	7.8	7.83	7.83
	9	11.36	10.44	9.97	8.92	8	7.66	7.62	7.72	7.72
	10	10.7	9.099	8.565	8.431	7.647	7.571	7.246	7.277	7.277
	11	10.2	8.421	8.048	7.976	7.509	7.359	7.33	7.399	7.399
	12	9.56	8.068	7.96	7.862	7.402	7.378	7.279	7.238	7.238
	13	8.93	8.096	7.984	7.611	7.368	7.345	7.182	7.179	7.179
	14	8.57	7.954	7.528	7.625	7.396	7.335	7.268	7.183	7.183

for the new cases, death cases, and recovered cases. The values in the table are the MAPEs from the evaluations, on a rolling forecasting origin. The end target of the forecasting application does not produce a set of predictions.

It is fundamental to compare the result against the actual values consistently, after which the model is evaluated by rolling through the origins. This process is validated by forecasting the training data for only one period and comparing it with the actual values of the testing data. The compared testing data then becomes training data. The same procedure is carried out until as much as the testing data as possible has been analyzed.

In

Table 3. MAPE values of validation with different splits of data using NNAR (14,14).

Hidden Nodes	Split Ratio
	80/20			70/30
	Recovered	New Case	Death	Recovered	New Case	Death
1	32.66	32.69	45.00	27.63	25.96	38.76
2	22.59	20.91	36.95	19.89	17.58	32.22
3	14.90	12.42	25.86	13.99	12.39	24.40
4	9.43	7.38	17.06	9.35	8.57	17.32
5	5.87	4.26	10.76	7.11	5.78	12.90
6	3.75	2.40	8.45	4.88	3.84	9.76
7	2.54	1.19	5.12	3.41	3.02	7.60
8	2.10	0.79	3.44	2.45	2.23	6.14
9	1.71	0.65	2.82	1.97	1.73	4.68
10	1.65	0.45	1.70	1.79	1.38	3.99
11	1.51	0.33	1.56	1.42	1.17	2.96
12	1.48	0.30	0.99	1.32	1.09	2.68
13	1.42	0.28	0.96	1.19	0.91	2.20
14	1.45	0.26	0.78	1.06	0.87	1.82

the MAPE values of both daily new cases and recovered cases begins to stabilize and is included in the low MAPE category, with 14 hidden nodes on each lag. This is in contrast to the death cases; after 14 hidden nodes, the MAPE value is still not stable at each lag, even though it has resulted in a low MAPE. According to the three tables above, all networks produce a smaller MAPE value; increasing the number of lags and the number of hidden nodes also results in a MAPE value with a low error category. Therefore, in the next step, the same mechanism will be applied again to find the optimal dataset split ratio to be used in the NNAR model. In addition to the 95:5 ratios that have been applied above, 80:20 and 70:30 ratios will be applied, this time with a trial-and-error mechanism. The obtained results are shown in the table below.

In

Table 4. Network accuracy.

Cases	Model	RMSE	MAE
Daily new	NNAR (14,14)	50.74	30.64
Death	NNAR (14,14)	2.79	1.45
Recovered	NNAR (14,14)	76.48	41.16

, the MAPE values for daily new cases and daily death casess are smaller at a 70:30 split ratio, while, for daily recovered cases, the MAPE value is smaller, at an 80:20 split ratio. The NNAR model with the smallest MAPE values thereby will be used to do short-term forecasting. The proposed short-term method can be beneficial in government measures, as well as resources distribution and rapid planning of healthcare service deployment. Furthermore, medium-term and long-term forecasting is more appropriate for ensuring ventilator availability and vaccine production [28].

Forecasting will be carried out using the best network architecture that has the smallest error value in the validation phase. In this study, we will forecast the next 7 days using the one-step-ahead forecasting method. Forecasts are only carried out for the next seven days because the data will be updated approximately every seven days. In addition, forecast accuracy will decrease if longer forecast periods are carried out. This short-term forecast can be utilized for healthcare management decisions and supporting critical retailers in order to avoid stock outages. The accuracy of the network in fitting actual data is determined from the value of RMSE and MAE. The accuracy values of the best network architectures are shown in Table 4. Figure 7 illustrates the future value of daily new cases, daily death cases, and daily recovered cases, respectively, while

Table 5. Forecasting results of COVID-19 cases in Jakarta.

Date	Daily New Cases				Death Cases				Recovered Cases
	Lo 95	Point	Hi 95	Actual	Lo 95	Point	Hi 95	Actual	Lo 95	Point	Hi 95	Actual
19/03/2021	1464	1568	1678	1588	19	25	30	24	1097	1245	1388	1175
20/03/2021	1179	1282	1385	1935	15	20	26	22	1246	1401	1561	1184
21/03/2021	1103	1209	1313	1638	23	28	34	23	1297	1453	1599	1884
22/03/2021	1259	1364	1466	1474	16	22	28	8	1293	1432	1575	1715
23/03/2021	1259	1371	1481	816	26	31	37	11	1162	1314	1467	1691
24/03/2021	1219	1321	1415	890	15	21	27	11	1063	1222	1396	1986
25/03/2021	1334	1431	1533	1727	32	38	43	12	2333	2485	2643	490

shows the forecasting results of the NNAR to predict each case type for Jakarta.

Based on Figure 7a, the forecast results show that daily new cases in the next 7 days will decline, to within a range range of 1249 to 1591. As the number of daily new cases are expected to decrease over time with the expansion of vaccination, there might be a more significant effect, given the implementation of PPKM Jakarta. On the other hand, the forecast results for daily recovered cases and daily death cases still projects an increase over the next 7 days. Therefore, the government needs additional interventions to decrease the daily death cases and to increase daily recovered cases more substantially; thereafter, active cases can be controlled during the remainder of the pandemic. However, the forecast results were only accurate for the first period, ending 19 March 2021. This is because the validation process was carried out with one-step-ahead validation; it would perform better were the data updated every day and forecasts made for the next day.

4. Conclusions

The Indonesian government continues its efforts to overcome the global pandemic at both national and local levels. Jakarta, as the capital of Indonesia, is the province with the most cases in Indonesia. As a result, Jakarta’s government issued its PPKM policy to reduce the number of COVID-19 cases. Based on the results of causal impact analysis, the application of PPKM Jakarta had a significant impact on the addition of daily new cases, as well as daily recovered cases, in Jakarta. On the other hand, the Indonesian government had also begun adopting national vaccination measures, which were divided into three phases, thus far, in which the frontline guards, such as health workers and the elderly, were the top priority. Unfortunately, according to the results of our analysis, this vaccination program did not have a significant impact on the increasing number of daily new cases in Jakarta, although the number of daily recovered cases was significantly impacted.

As per the forecast, the number of daily new cases were predicted to show a decrease and an increase in recovered and death cases was predicted for the 7-day period after 19 March 2021. However, when we compared these to the actual values, predicted values were only accurate for the first day. This might have happened because of the method we implemented, i.e., one-step-ahead forecasting, not multi-step-ahead forecasting. Considering that this pandemic is new to the world, the data collected are still few and do not reflect actual reality; this lack of data can lead to inaccuracy in long-term forecast results. Additionally, the behavior of any community cannot be controlled by the government on the basis of non-strict policies alone [59,60,61,62]. This leads to the fact that the number of daily new cases fluctuates, jumping up extremely then decreasing significantly, and, therefore, not following previous patterns. In fact, a down-slope in new daily cases does not necessarily lead to a reduced number of deaths. Furthermore, we conclude that this method is better used to generate daily forecasting of COVID-19 cases and evaluate it by rolling the origins or else using other, more appropriate methods to obtain weekly, monthly, and yearly predictions.

Despite all the limitations surround it, it can be said that the PPKM program was statistically significant in reducing the number of daily new cases in Jakarta, while the vaccination program has not yet been significant. The PPKM program might also be accompanied by socialization in the future in order to raise public awareness. By that time, vaccinations will also be more numerous and equally available for all Indonesian citizens, not just priority groups. This is also be a factor in the success of a national vaccination program, as seen in other countries where almost all citizens have been vaccinated.

The success of vaccination programs must be further evaluated over a long period of time to test their effectiveness in general. Therefore, for the time being, vaccination was only evaluated on the basis of confirmed cases in this study. For future studies, it would be better if more variables were included, or if they were carried out in a cohort study. Although the results of this study suggest that vaccination’s impact in Jakarta was not statistically significant, the success of the vaccination cannot be considered a failure because the vaccination process has only been in practice for two months and has not covered the entire population.

However, most recently, these results are consistent with the increase in new cases seen during the major holidays of April and May, although a downward trend emerged by the end of March 2021. Vaccination had been implemented intensively and widely by the major holidays but still hindered the suppression of new COVID-19 cases, since PPKM was not implemented correctly. This supports the result showing that the PPKM had a statistically significant effect and that regional restrictions are, therefore, indeed necessary to reduce mobility between regions. In this study, the two interventions were not tested separately; such would have been better, but was difficult as the two interventions had been implemented at the same time.