Keywords

1 Introduction

Accurate forecasting of demand and supply is a critical component of operation and planning of power systems. Historically, load forecasting has played an important role in power system operation, and thus it has been a popular topic for research especially with the growing interest of integrating renewable energy sources. Load forecasting is typically divided into short term forecasting, which is less than a day ahead, and mid to long term forecasting, which is more than a day ahead. While unit commitment, economic dispatch and maintenance schedule rely on the short term forecasting, transmission planning, generation mix, and long term outage planning rely on long term load forecasting. In this paper, we will focus on short term forecasting methods, which contribute to the operation of the grid.

Generation from renewable energy sources is uncertain in nature because they are intermittent primary sources. Therefore, increased penetration of renewables has introduced high volatility and uncertainty into the system, and the need for accurate supply and demand forecasting is growing. Wind and solar energy have been the two most dominant intermittent renewable energy sources, and the forecasting methods have been developed separately based on their characteristics. Wind power generation is highly dependent on wind speed, and the solar power generation is highly dependent on solar irradiation.

This paper reviews the current demand in renewable and load forecasting for smart grid applications and existing methodologies in forecasting. We pay particular attention to the recent development in smart grid technologies to identify the current trend in their need of supply and demand forecasting. We start with Sect.2 where we formulate the general problem of load and renewable forecasting and the characteristics that distinguish the problem. In Sect. 3, we identify the applications of forecasting in smart grids and explore the current needs in terms of accuracy and duration. Section 4 reviews the state of the art forecasting methods, and the trend and future needs will be discussed in Sect. 5 followed by the conclusion.

2 Problem Formulation

The renewable and load forecasting problem is predicting the future condition \(\mathbf y =[y_1\ y_2\ ...\ y_m]\) based on input parameters \(\mathbf x =[x_1\ x_2\ ...\ x_n]\), which often involve weather indices, time, any special events, etc. The problem usually involves a set of parameters \(\mathbf w \) that characterize the predictor \(\hat{\mathbf{y }}=f(\mathbf x ,\mathbf w )\). The objective is to minimize the forecast error,

$$\begin{aligned} \min d(\mathbf y ,\hat{\mathbf{y }}) \end{aligned}$$
(1)

where \(d(y,\hat{y})\) defines the pre-defined distance between data y and \(\hat{y}\). For example, this distance can be the square of the difference or absolute value of the difference. The problem resembles the classical machine learning problem, and thus the machine learning algorithm has been a popular tool for renewable and load forecasting. There are some unique characteristics that are only inherent in energy system forecasting problems:

  1. 1.

    Energy forecasting requires a number of exogenous variables that are difficult to capture with the provided data. Weather conditions, unprecedented events and human behaviour are difficult to quantify with only few variables.

  2. 2.

    Renewable and load require time series forecasting. The generation and load change over continuous time, and the forecast has to be made in discrete time with fixed or various time steps depending on its application.

  3. 3.

    Power system is a critical infrastructure where reliability should be one of the most prioritized objectives. The forecast error should be small enough so that it does not affect the security of the system.

Forecasting, when inaccurate, can threaten the reliability of the system, and thus is important for decision making of the system operators. Smart grid brings many features that will benefit the system operation, but it will introduce greater vulnerabilities at the same time. In the next section, we will look at how the forecast fit into the smart grid applications and their impact on the grid.

3 Applications of Forecasting in Smart Grid

Smart grid introduced many applications where load and renewable forecasting is required. The observability of the state are the input variables for the control and dispatch of power systems, and therefore, the performance of applications have a strong link to the performance of the forecast.

3.1 Demand Response

Contrary to the conventional economic dispatch in transmission system, the demand response attempts to match the load to supply [1, 2]. Demand response uses planned contracts or price incentives to reduce the consumption when the supply is insufficient. Since the required capacity of demand response depends on the renewable supply and the load, the forecast is a valuable piece of information to make plans ahead of time. The main purpose of demand response is the peak reduction, which requires the forecast of the amount of deficient supply and the time when the electricity would peak. The implementation of demand response in buildings and home energy management is done at the residential level with decision support system and home automation [3]. Forecasting is often done for the aggregate load and resources, but there is a growing need for forecast at the local level [4]. In the case of controllable loads, the load control depends on the forecast, requiring the control strategy to be robust against forecast error [5]. Many of the demand response are aware that the forecasts are subject to error, and thus robustness against forecast error is an important objective in designing the demand response program. Demand response could utilize real-time pricing where load forecasting becomes critical to balance the market [6]. In addition, demand response is usually used for short-term or real-time applications, and the forecast value is required at anytime from hours to day ahead [3].

3.2 Microgrid Energy Management System

Microgrid is a low voltage power system that allows easier integration of renewable energy sources. The energy management system of the microgrid assigns real and reactive power references to the generators and controllable loads, and for long-term energy balance, the EMS requires the day-ahead forecast of renewable generation and load [7]. The main challenge in microgrid management is that the power output fluctuates with the changing weather conditions, which tampers the power quality of the grid. Reliable and economical operation of microgrids requires coordination among the distributed generators, which also accounts for the uncertainty. The uncertainty in microgrid is higher than the bulk power system, so the role of forecast becomes more crucial [8]. The forecast value is mostly used in their tertiary control, which determines appropriate unit commitment and dispatch for the resources. In fact, most energy management system for microgrid include forecast modules [9], and improved accuracy and reliability of forecast will strengthen the security and resilience of the microgrid.

3.3 Control of Energy Storage

Energy storage is identified as one of the key technologies that will enable large integration of renewable energy sources. Energy storage can compensate for the errors in the forecast and maintain a certain level of reserves [10]. The purpose of energy storage is to increase reliability and reduce the penalties from the forecast errors. The management of energy storage includes the prediction layer where the renewable and load forecast are used to estimate the status [11]. The charging and discharging depends on both the current status and the predicted load and supply profile because the capacity of energy storage is limited. The energy storage needs to forecast the time the supply is expected to be deficient, and it needs to be charged ahead of the expected time. In the next section, we review the existing forecasting methods and their appropriate applications.

4 Renewable and Load Forecasting

The load and renewable forecasting requires the researchers to deal with the unpredictability of the natural and human behaviour. Many papers have addressed these challenges in uncertainty with various methods including time series models, artificial neural network, support vector machine and hybrid models. In this section, we will explore the techniques that are recently developed and used in smart grid.

4.1 Time Series Models

Time series models use a sequence of data points over a time interval to make predictions for the future. One of the most well known time series models is the auto regressive (AR) model where the output variable depends linearly on the data previously taken over time. In [12], a method of predicting the photovoltaic (PV) generation using an Bayesian AR model is presented. The paper first chooses an analytical expression of the probability density function (pdf) of the hourly clearness index and defines an AR time series model to represent the relationship between the pdf parameters, meteorological variables, and the clearness index. While the conventional time series approach does not consider the weather data [13], the Bayesian AR model is able to incorporate weather data such as clearness index, ambient temperature, relative humidity, wind speed and cloud cover. Finally, through the Monte Carlo simulation procedure, the model generates the predicted pdf of the PV’s active power. Similarly, the method proposed in [14] predicts short term solar power generation using adaptive linear time series models based on recursive least squares (RLS). It accomplishes this by first normalizing solar power with the clear sky model and employing the AR and the AR with exogenous input (ARX) models. The paper introduces adaptivity to consider the snow cover, leaves and dirt on the panel, etc. Several works including [15, 16] have also reviewed the application of AR models for the purpose of solar forecasting.

Other methods of time series analysis include auto regressive moving average (ARMA) models, which are general cases of the AR and moving average (MA) models. ARMA models consist of both the AR and MA models and forecast future values based on the linear combination of the past values and errors. [17] presents a method of integrating the ARMA models with a Kalman filter to ensure accuracy in predicting the time distribution of solar radiation and ambient temperature. It adjusts the climatic parameters at every five minute intervals based on their acquired values and their last prediction errors. Auto regressive integrated moving average (ARIMA) models are presented in [18] to predict sub-hourly and hourly PV arrays power output.

The time series model is a dominant technique in short term solar forecasting because it is flexible in handling a wide range of different time series patterns. The short term weather is likely to be dependent on the previous weather condition, so it is especially attractive for modeling solar irradiance. However, time series models show deficiency in modeling holidays, weekends, and seasonal changing periods [13, 19], and currently, it is not widely used in load forecasting or wind forecasting, which constantly changes.

4.2 Artificial Neural Networks

Artificial neural networks (ANN) are inspired from the human brain and how it processes information. In our brain, the neurons are the basic units that receive, process and output response. Similarly, ANNs define the activation functions that receive linear combination of the data with weight \(\mathbf w \). ANNs are organized in multiple layers that use hidden variables, which are not directly observed, but are used in the computation. There has been a great success with ANNs in renewable and load forecasting [13, 20]. In [21], solar irradiation and wind velocity are taken as inputs to predict the maximum power generation of PV systems, and in [22] solar irradiation and air temperature are considered. ANN is also a popular method in short term wind power forecasting because wind often experiences high variation over a short period of time [23, 24]. However, the performance of ANNs significantly drops as the prediction lead period increases. The major application where ANNs showed most success is the load forecasting, which predicts the peak load, valley load, and total load [13, 19]. Accuracy and reliability make it very attractable for applications in smart grid [6, 25].

The main advantage of ANN is that it does not require specific model based on knowledge, and the historical data is directly linked to the output [19]. It accounts for non-linearity among data and is able to detect all possible interactions between predictor variables. However, NNs also present the problem of over-fitting, requiring much computational resources and time, missing an exact rule for setting the number of hidden neurons, having poor scalability, and lacking the ability to generate explanations for their results [26, 27]. These are classical problems in ANN for other applications, and there are methods such as cross-validation and regularization to overcome the problem of over-fitting.

4.3 Support Vector Machine

Support vector machines (SVM) are machine learning techniques of inferring a function from a set of training data associated with learning algorithms which analyze data and recognize patterns. SVMs are typically used for classification and regression analysis, and has been a popular tool for load forecasting. [28] proposes a method of using linear least squares regression and SVM with site-specific forecast data from National Weather Service to derive prediction models for solar power at individual smart homes. [29] proposes daily peak load forecasting model for smart meters based on the technique of support vector regression (SVR) using least squares. The authors mention that their method can potentially be used by utility companies in their large-scale load forecasting application. [30] reports that high accuracy can be achieved with SVM for distribution system energy forecasting. The paper also looks at the extension of their method for determining optimal demand response and anomaly detection in energy usage.

SVM accounts for non-linearity among data by finding a hyperplane that could divide the data points into different spaces. The accuracy of SVM depends on the selection of kernel function and parameters, which transform the input space to the high-dimensional feature space [28]. Although SVM may have advantages in accuracy, it requires great computational resources and time in both training and testing, as well as the parametrisation of kernel functions [26].

4.4 Hybrid Models

In order to take advantage of different methods, many researches have been done to develop and test hybrid models or a combination of known methods to improve the efficiency and accuracy of load forecasting. In [26], a post-processing method is employed to overcome the shortcomings of the conventional methods in efficiency, flexibility, and scalability. Their post-processing method attempts to identify the appropriate techniques to be applied in different circumstances for better performance. Similarly, [31] proposes a hybrid model of wavelet transform, firefly algorithm, and fuzzy ARTMAP to achieve significant improvement in lowering mean average percentage error compared to various other single or hybrid forecasting models. [32] performs similar tasks of improving accuracy by combining regression tree and Gaussian Process to accurately predict maximum daily temperatures, which are vital in handling demand response and PV.

The hybrid models address new challenges that rise in smart grid by introducing variables that were not considered conventionally. For example, as consumers gain the ability to manage their consumption, new considerations such as the effect of electricity prices on consumer behaviour require a new framework that can successfully account for this price-demand relationship. [33] proposes such a model which combines multi-input multi-output forecasting engine with data mining algorithms to make joint price and load predictions. Hybrid models consider the complex relationships among the variables, which could improve the accuracy. While hybrid models can improve the accuracy and flexibility in forecasting, they depend on the combination of methods or the overall framework and require much computational resources.

5 Discussion

We have explored smart grid applications, which make use of renewable and load forecasts, and the existing forecasting methods. Fundamentally, the applications such as demand response and energy management of microgrid depend on the forecast to plan the operation. Although the demand of accurate and reliable forecasting methods is always high, developing a single perfect forecasting method that works best in all situations is practically impossible due to the volatility of nature. This paper reviewed existing methods for forecasting in power systems and their advantages and disadvantages. Hybrid models seem to be promising in increasing accuracy by taking advantage of different methods, but they will come at the expense of simplicity.

The principles and techniques used for forecasting did not significantly change with the advent of smart grids. However, many recent papers suggest that the power system is going in the direction of local distributed network such as microgrid, and the development of forecasting methods should accompany the current needs. There is a growing demand for distributed forecasting where the forecast is required at the household level, and it will require an efficient and reliable forecasting methods.

In addition, we have seen that the performance of forecasting is fundamentally dependent on and limited by the historic data. It is important to test and evaluate the methods for the data from various case studies to overcome the issue of data-dependency. In order to overcome this issue, we have seen the research effort for the algorithm to be robust and adaptive to change in variables.

6 Conclusion

Reliable and accurate forecasting of renewables and load is an essential component in the development of smart grids. This paper presented an overview of applications and current forecasting methods to review the current demand and trend in the field. We explored the requirements and desirable characteristics of each application and also looked at the traits of the existing methods.

While many of the work showed success in various applications, there is still a need for developing forecast models at the distribution system level. In addition, we require standard test cases and numerical examples to clearly illustrate the performance difference between developed methods. A more standardized procedure will enable direct comparison of the methods and numerical characterization of the forecasting techniques.