Keywords

1 Introduction

Electricity is a form of energy that being used widely [1]. Coal, solar, and wind energy are the examples of primary sources of energy. These primary energy sources were used in electricity generation. There are two types of primary energy sources: renewable and non-renewable. The conventional way of generating electricity was from non-renewable sources. This method affects the environment in negative way by releasing carbon emission. The drawback of using non-renewable sources is overcome by the implementation of renewable energy. However, the integration of renewable energy is challenging to ensure the supply meets the demand. Electricity power generator companies can use electricity load prediction based on the consumers’ electricity usage pattern to estimate the power generation. The electricity forecasting able to predict for short-, medium, and long-term [2]. Short-term forecasting is suitable to predict the demand for less than a month [3]. As the prediction for a longer period such as monthly until decades, medium-, and long-term are more suitable. This study applied two machine learning algorithms in forecasting the electricity consumption pattern.

The research of machine learning in electricity area has been increased in recent years [4]. Supervised learning allows the machine to learn from historical electricity load data. In supervised learning, the training process learns based on the sample dataset. The training process produces a model, while the testing process evaluates the model quality using evaluation metric. The testing process uses a set of unknown data that only consists of features.

Regression is a machine learning algorithm used to predict in time-series [5]. There are many computational algorithms to solve regression problems such as Artificial Neural Network (ANN), Support Vector Machine (SVR) [3], and Regression Neural Network (RNN). These algorithms were revised into extended versions that include Deep ANN, Sequential Minimal Optimization (SMO) [6], Support Vector Regression (SVR) [7, 8], and Long Short-Term Memory (LSTM). RNN handles the sequential data and memorise previous inputs that are stored in the internal memory [7, 9]. The drawback of RNN is vanishing gradients caused by too small parameter updates. Gradients is important because it contain information used in RNN iteration. However, LSTM overcome this issue.

LSTM can memorise information for extended period because it has its own cell [10, 11]. LSTM works by repeating modules of a Neural Network (NN) with a few layers that communicate with each other [9].

The data input for machine learning represents by features and label. However, the data input for RNN and LSTM requires data transformation into features, time step, and label. The time step benefits LSTM in terms of prediction based on a specific period of time, whereby it is applied in time-series forecasting in predicting the future value derived from historical data patterns [12]. These algorithms apply in training phase to produce a model. The model is assessed using evaluation metric.

In RNN, the common evaluation metrics applied are mean absolute error (MAE), root mean square error (RMSE), and mean square error (MSE) [13, 14]. A lower evaluation metric value represents a better model [13,14,15].

The remainder of this paper is organised as follows: Sect. 2 provides a review of existing published works related to electricity load forecasting. Section 3 describes the case study applied to this paper. Section 4 describes the methodology employed in the chosen machine learning algorithm. Section 5 describes the experiment results that include the model quality and prediction values. Finally, Sect. 6 concludes the findings.

2 Review of Related Works

Liu et al. [16] applied historical electricity dataset on LSTM. The article generated two models from two machine learning algorithms, Elman Neural Network and LSTM. The article compared the prediction results made by these models for normal and special days. This article used mean absolute percentage error (MAPE) to evaluate the model. As the results, MAPE value for LSTM model is 2.13%, and Elman Neural Network is 5.19%. The model generated using LSTM achieved high precision compared to Elman Neural Network.

Hossen et al. [17] used the historical electricity load dataset with independent variables. Hossen et al. [17] used the two-year duration dataset, Almanac of Minutely Power Dataset (AMPDs) dataset. The data collected based on one-minute interval. The dataset consists of independent variables from the utility meters: power and water, and weather data. The article compared the execution between Deep Neural Network (DNN) using LSTM with Gated Recurrent Unit (GRU) and simple RNN. The result proven the lowest MAPE error rate, 24% is by the model generated from DNN using LSTM. The error rate of the model generated from GRU and simple RNN are 24.7% and 37.7% respectively.

Zheng et al. [2] used another set of independent variables in the dataset of study. Initially, the authors proposed to include independent variables such as temperature, humidity, rainfall, and wind speed. These independent variables went through the selection operator (Lasso) to identify which independent variables affect the electricity load. As the results, only average temperature added to the historical electricity load dataset. The dataset duration used was one year and 11 months. This research study applied Adaptive Moment Estimation (ADAM) optimiser in the execution. The study conducted in three categories: load dataset, load and temperature dataset, and load, time, and temperature dataset. The researcher used MAPE to evaluate the model. The best model was generated from the experiment that used the load, time, and temperature dataset with MAPE value of 6.00%. The MAPE result for the experiment that used only load feature is 8.52%. The model was used to predict electricity load in summer, autumn, winter, and spring seasons. Memarzadeh and Keynia [11] also compare the model performance on each season.

Wang et al. [18] compared the prediction performance among various traditional forecasting methods, such as Autoregressive Moving Average model (ARMA), Auto-regressive Fractionally Integrated Moving Average model (ARFIMA), and Backpropagation Neural Network (BPNN) with LSTM. The RMSE result for LSTM showed the lowest error among other traditional methods.

Karunathilake and Nagahamulla [19] studied the implementation of ANN in electricity load prediction. The researcher used RMSE, MAE, and R-squared (R2) to evaluate the model. The result of MAE value was 0.0211, RMSE value was 0.6328, and R2 value was 0.7385.

Table 1 summarises the machine learning algorithm applied, and the evaluation metrics used in the reviewed research articles.

Table 1. Summary of machine learning algorithms and evaluation metrics used in the reviewed articles.
Full size table

3 Case Study: Danish Electricity Usage Dataset

Denmark is one of the developed countries that leads the implementation of renewable energy. This article intends to assist in predicting residential consumers’ electricity demand in Denmark. This study used the unpublished Danish electric power consumption dataset. The dataset was retrieved from a smart meter data of one household unit in Denmark with an hourly sampling rate throughout four years duration, between the year 2015 and year 2018. The dataset used consists of historical electricity load, year, month, day, hour, minute, weekend, holiday type, seasons, and day length columns.

The dataset is divided into the training, testing, and prediction dataset as shown in Table 2.

Table 2. Summary of training, testing, and validation datasets.
Full size table

Table 2 summarises the total rows for each dataset. The training dataset consisted of 26,304 rows and ten columns. The target of the training data was the electricity load column. Other columns were used as features. The testing dataset had 8,760 rows that contained one-year of data. The prediction set had 24 rows that represented 24 h in a day. The prediction dataset is the subset of testing dataset.

4 Modelling Using LSTM and ANN

The prediction was performed based on a periodic of time. The data input used in ANN were samples and features. In LSTM, the data input consisted of samples, time steps, and features. The optimizer used is ADAM optimizer [20]. The programme flow is illustrated in Fig. 1.

Fig. 1.
figure 1

Experiment methodology applicable for LSTM and ANN.

Full size image

The difference between LSTM and ANN is in the input layer process. LSTM ap-plied the LSTM method with input_shape parameter values set to time steps value, 24, and number of features. The execution for ANN applied the Dense method with input_shape parameter value set based on the number of features. The ANN and LSTM algorithms applied consist of one input layer, one hidden layer, and one output layer.

The programme produced two models, one model produced by ANN algorithm, and the other model produced by LSTM algorithm. These models were used in the testing phase to evaluate the model quality based on the evaluation metrics, namely MSE and MAE. The model was used in prediction phase to predict the electricity load of a certain duration. The actual values were compared with the actual value. The results’ pattern and difference were observed.

5 Experimental Results and Discussions

This section shows the results of the implementation of the proposed methodology. The models were trained, tested, and predicted on Intel® Core™ i7-3930K CPU with 6 cores and maximum clock frequency of 3.2 MHz. The software used was Jupyter Notebook on Anaconda Navigator. The TensorFlow Keras library was applied in the execution.

5.1 Training and Testing Datasets

The dataset used was a single household unit electricity load dataset comprising 35,064 rows of samples with nine columns representing features. The initial training data shape was 26,304 rows with nine columns, while the testing dataset shape was 8,760 rows and nine columns. This two-dimensional (2D) array data was applied with ANN algorithm.

For the implementation using LSTM, the time steps applied in this experiment were 24, which represented 24 h per day. The shapes were reshaped into three-dimensional (3D) array with 24 time steps. The new 3D training dataset shape was 1,096 samples, 24 time steps, and nine columns, while the testing set reshape value had 365 samples, 24 time steps, and nine columns.

5.2 Training and Testing Activities

The input for training activity was the training dataset. The output of the training activity was a model. The model generated was tested with the testing dataset. The result of testing activity was evaluated using evaluation metrics, MSE and MAE. Then, the prediction was made on the prediction set, represented for any of the day in 2018. The default epoch value was set to 50.

Table 3 summarises the result of testing process and the epoch required to complete the training on both datasets with ANN and LSTM.

Table 3. Training and testing activities results.
Full size table

There were two models generated, modelLSTM and modelANN. The lowest error rate produced by modelLSTM with MSE values of 0.1238, and MAE values of 0.0388. The total epoch used in LSTM was 26, while the ANN algorithm completed its training phase with 12 epochs.

5.3 Prediction Activity

The prediction was made for 24-h duration on 2nd April 2018. It was on Monday, in the spring season, in the beginning of the week, and no special occasion on this day in Denmark. The prediction was in hourly, started on 0000 h and ended on 2300 h. Table 4 shows the execution result for actual and prediction values of electricity load derived from the modelANN and modelLSTM.

Table 4. Actual and prediction values generated by modelANN and modelLSTM for one day.
Full size table

Figure 2 illustrates the results of actual and predicted values by the ANN and LSTM models in Table 4.

Fig. 2.
figure 2

Comparison between actual electricity load and prediction values from ANN and LSTM models.

Full size image

Figure 2 shows the actual and prediction results by the modelANN and modelLSTM. The sample of one-day hourly electricity load was on 2nd April 2018. It falls in the spring season. The peak electricity load was identified in between 0600 h and 0900 h, and 1300 h and 1900 h. The electricity load decreased from 0000 h until 0600 h, followed by 1000 h until 1300 h.

The results by the LSTM algorithm was also proven to be better than the model generated by the ANN algorithm as stated in Table 3. This was obtained due to the ability of the LSTM algorithm in memorising the time-series input during the training session.

6 Conclusion

Electricity power generator companies can use electricity load prediction based on the consumers’ electricity usage pattern to estimate the power generation. This paper proposed the implementation of LSTM algorithm to predict electricity demand in time series. Experiments on the traditional machine learning algorithm, ANN, was executed to verify the performance of LSTM. The contribution of this paper is in the implementation with time-series prediction. The model generated using LSTM was proven to produce a lower error rate than the traditional algorithm, ANN, although independent variables were added into the input variables.

This experiment was conducted based on the historical electricity load data with independent variables of a single household unit. Future experiment should consider the implementation on Deep Learning algorithm.