Short Term Probabilistic Load Forecasting With Integrated Methods

. In smart grid era, electric load is becoming more stochastic and less predictable in short horizons with more intermittent energy and competitive electricity market transactions. As a result, short-term probabilistic load forecasting (STPLF) is becoming essential for energy utilities because it helps quantify the risks of decision-making for power systems operation. Currently, probabilistic load forecasts (PLF) are commonly produced from three single components, namely input, model and output. Nevertheless, whether integrating two components to represent dual uncertainties of electric load is practical and able to improve STPLF attracts little regards. To address this issue, this paper proposes three integrated methods by pairwise combination of single representative component, i.e. uniform-biased temperature scenarios (UBTS), quantile regression (QR) and logarithmic residual empirical simulation (LRES). Case study on real utility data demonstrates the superiority of the integrated methods and excavates the relationship between predictive model class and specific integrated method.


Introduction
Grid modernization with more intermittent energy and competitive electricity market transactions has exacerbated the volatility and uncertainty of electric load, making it less predictable even in short horizons than ever before. As a result, probabilistic load forecasting, offering intervals, quantiles and probability density as prediction formats, has become significant and urgent for utilities to help make risk analysis and practical decision in power systems operation and planning [1]. Practical applications of probabilistic load forecasting include probabilistic load flow [2], stochastic unit commitment [3] and energy management [4].
By far, researchers and forecasters mainly produced probabilistic load forecasts from three single components of forecasting engines [5], i.e. (1) the input component: simulating predictors to generate input scenarios; (2) the model component: constructing probabilistic forecasting models or multiple predictive models; (3) the output component: output post-processing via residual simulation or forecasting combination.
Among various generation methods of PLF, temperature scenarios, quantile regression and residual simulation are the most representative methods for the aforementioned three components, respectively. For example, in the input component, Xie and Hong [6] adopted fixed-date, shifted-date and bootstrap methods to generate temperature scenarios for probabilistic load forecasting; in the model component, Yang et al. [7] proposed Gaussian process quantile regression, a dual probabilistic model, to produce probability density for electric load; in the output component, Fan and Hyndman [8] generated day-ahead prediction intervals for electric load in Australian National Electricity Market by simulating forecasting residuals via block bootstrapping.
Since the probabilistic generation method from three single components of forecasting engines only reveals a specific uncertainty of electric load [9], a research question is naturally raised, "whether combining two kinds of uncertainties via integrating pairwise components could help improve probabilistic load forecasts?" To our best knowledge, little literature focused on this topic, which makes the integrated methods for probabilistic load forecasts a blank to industry and research.
This paper contributes to formally investigating the integrated methods for probabilistic load forecasting. We first pick out uniform-biased temperature scenarios (UBTS), quantile regression (QR) and logarithmic residual empirical simulation (LRES) as three representatives for probabilistic generation methods from three single components. Through pairwise combination of single representative, we then propose three integrated methods, a.k.a UBTS-QR, QR-LRES and LRES-UBTS. Compared with probabilistic generation method from single component, whether integrated methods help improve probabilistic load forecasts can be validated. Besides, we also excavate the relationship between a certain integrated method and predictive models.
The rest of this paper is organized as follows: Section II elaborates three representative probabilistic generation methods and then illustrates how to integrate the representative generation methods from two components of forecasting engines. In Section III, two model classes, i.e. Multi-linear regression (MLR) based and Gradient Boosting Regression Tree (GBRT) based models with different predictive powers are briefly introduced. Section IV presents the case studies on Shanghai utility data to evaluate the performances of integrated methods. Section V concludes the paper and provides future research directions.

Probabilistic generation methods from single component
Since the proposed integrated methods are derived from the probabilistic generation methods from single component, we first briefly introduce three representatives for three single components of forecasting engines, respectively.

Input component: uniform-biased temperature scenario (UBTS)
As temperature forecasts plays an important role in load forecasting \cite{fan2012short}, it is intuitive to generate temperature scenarios on the basis of temperature forecasts. To our best knowledge, day-ahead hourly temperatures can be precisely predicted with the maximum mean absolute error around 2 C° [10].
In this paper, we apply uniform-distributed noises onto temperature forecasts to generate day-ahead temperature scenarios, denoted as Uniform-biased temperature scenarios. According to Ref. [11], we set the temperature forecasting biases follow the uniform distribution, [-2, 2]( C°). Practically, we simulate the uniform biases for 99 times, then 99 day-ahead temperature scenarios and 99 point load forecasts will be generated.

Model component: quantile regression (QR)
Quantile regression (QR), capable of creating multiple predictive models by varying any quantiles, is a modelbased probabilistic generation method. Theoretically, QR-based models keep the same structure as point forecast model, but distinguish parameters' values according to certain quantiles by minimizing the pinball loss function (Eq.(9)). In other words, m quantiles will produce m different predictive models. More details about QR can be found in [11].
In this paper, quantiles q shade from 0.01 to 0.99 with an interval of 0.01. Therefore, 99 quantile forecasts can be generated from QR method.

Output component: logarithmic residual empirical simulation (LRES)
Residual simulation is an effective way in output side to move from single point forecast to probabilistic forecasts. Since no statistical evidence supports forecasting residuals generally follow a certain distribution, logarithmic residual empirical simulation (LRES) is proposed to generate day-ahead forecasting residuals.
To simulate residuals, we first apply logarithmic transformation to load data, then extract the forecasting residuals in the past seven days and take 99 percentiles from empirical distribution. Next, 99 empiricaldistributed residuals are added back to the point forecast followed by the exponent transformation to generate 99 quantile forecasts in the forecasting horizon.

Integrated probabilistic generation methods
After introducing three representative probabilistic generation methods from single component of forecasting engine, three kinds of integrated methods (see Table 1) are proposed by pairwise combination of the aforementioned representatives.

UBTS-QR
The combination of temperature scenarios and quantile regression takes both the uncertainties of temperature forecasts and load variation into account. The core of this integration is to feed the uniform-biased temperature scenarios into QR based predictive models. In details, we first train multiple predictive models following the QR method, then replace the temperature forecast (usually a deterministic value given) with uniform-biased temperature scenarios in the forecasting horizon and take these temperature scenarios as predictors for the trained QR based models. In other words, if m QR based models are trained and J temperature scenarios are generated, then m J × point forecasts will be produced. Finally, 99 quantile forecasts are derived from the empirical distribution of m J × point forecasts.

LRES-UBTS
The association between temperature scenarios with residual simulation originates from the mismatches of temperature forecasts and predictive models. Integrating uniform bias temperature scenarios and logarithmic residual empirical simulation helps make correction for temperature-scenario-based point forecasts. Specifically, residuals over past seven days are first determined for each temperature scenario based forecast, then their 99 percentiles are extracted from the empirical distribution and added back to the corresponding temperature-scenario-based point forecast. Note that all the residuals are logarithmic and the transformation should be accordingly performed.

QR-LRES
Integrating quantile regression and residual simulation takes dual uncertainties of load and predictive models into consideration. This approach attempts to correct the mismatches of the certain q th-quantile predictive model. Similar to UBTS-LRES, we first calculate the logarithmic residuals over past seven days for each QR based model, then take their 99 percentiles from the empirical distribution and adds them back to the corresponding quantile forecast as well as applying the exponent transformation.

Predictive models
Probabilistic generation methods convert point load forecasts into probabilistic forecasts. Before applying and evaluating these methods, we need underlying predictive models to produce point forecasts. To ensure the generality of our findings, Multi-Linear Regression (MLR) and Gradient Boosting Regression Tree (GBRT) model classes, with different variables input to reflect different levels of predictive powers, will be constructed.

Multi-Linear regression class
The first model in MLR class, named as Vanilla, was first proposed by Hong [12] and served as benchmarking STLF model in GEFCom2012 [13]. Vanilla, mainly featured as interacting temperature and calendar variables, can be expressed as: More details for Vanilla can be found in [12]. The second model, removing the calendar variables but adds the recent loads as predictors, is denoted as Lag-Cubic. In STLF, lagged loads are commonly used as predictors [8], [12] for their ability to capture serial correlations. Since model construction is not the point of our research, the lagged loads are heuristically selected.
Here, the lagged loads for 1 day, 2 days and 1 week are selected. It can be written as:

Gradient Boosting Regression Tree Class
GBRT, a decision-tree-based ensemble regression model, is trained by adding a new classification and regression tree (CART) in an iterative process. The optimal CART for k th iteration is determined by fitting residuals from the ( 1) k − th tree. Given a specific loss function ( , ( )) t t L y f X , the residuals for time t at k th iteration can be written as: Next, the gradient boosting tree at k th iteration can be calculated: After K iterations, the GBRT is finally determined: Where, 0 ( ) t f X denotes the initial CART, a weak learned tree, trained by data set . More details about GBRT can be found in [14].
With the framework of GBRT, three GBRT-based models can be constructed by feeding different variables. Without losing justifiability, the same variables as MLR class are selected for benchmarking GBRT (denoted as B-GBRT), lagged GBRT (denoted as Lag-GBRT) and extended GBRT (denoted as E-GBRT), respectively. In details, the input variables are listed in Table 2: Shanghai, one of the most developed cities around the world, is located in East China. Her utility provides energy and related services for 15 districts and 1 county with around 25 million people. In this paper, hourly electric load and hourly temperature information from 2013 to 2016 in Shanghai, part of the data used in [15], are utilized. Note that the first three years' data (2013-2015) are used for training the model parameters, while the last year's (2016) data intended for short term ex ante probabilistic load forecasting. The forecasting horizon is fixed at one day ahead.

Evaluation metric
To make fair evaluation for the proposed integrated methods on probabilistic load forecasts, quantile score is used to evaluate the performances of probabilistic forecasts. It is derived from the pinball loss function, which is considered as an error measurement formally carried out in the GEFCom2014. [5] For q th quantile, if the quantile forecast is ,t q y , then the pinball loss function can be expressed as: t q t tq t tq q y y y y Pinball y y q y y y y To obtain quantile score, the pinball losses are summed for all targeted quantiles and then the sum is averaged over all quantiles and samples. The quantile score can be specified as:  (10) Where, N represents the total samples of the holdout set, and M is the total number of quantiles.

Probabilistic load forecasts with MLR class
Under the MLR model class, Table 3 presents the quantile scores of the test year (2016) via applying various probabilistic generation methods from single or dual components of forecasting engine. The lowest quantile score for each predictive model is highlighted in bold. We can clearly obtain that QR-LRES method provides the best short term probabilistic load forecasts for the above MLR models. To make the comparison between dual-component integrated methods and single-component generation methods more intuitive, a color chart is drawn to indicate the degree of relative improvement as to better and worse method for integration, as Fig. 1 shows. It can be seen that QR-LRES outperforms the better method most, which might account for the fact that QR-LRES matching MLR model class can improve probabilistic load forecasts to the largest extent. An evident of why QR-LRES can beat the better method (QR) in quantile score is shown in Fig. 2, where actual loads in summer and winter peak week are more central within the intervals created by QR-LRES than those by QR based on E-Vanilla. In addition, the integrated methods override 8 out of 9 on quantile scores compared to the better singlecomponent method and outperform all compared to the worse one. Therefore, we can conclude that integrated methods by combining pairwise components help improve short term probabilistic load forecasts.

Discussion on GBRT class
To avoid drawing the conclusion specifically for one model class, we conduct the same short term ex ante probabilistic load forecasting with GBRT model class. Table 4 shows the quantile scores of the test year (2016) with the same generation methods but based on GBRTbased models. The values within the parenthesis are the relative improvement of integrated methods as to the better single-component (left) and worse method (right).
In GBRT model class, LRES-UBTS generates the best probabilistic load forecasts on average and outperforms the better method most, which is similar to QR-LRES in MLR case. Meanwhile, the integrated methods outweigh 13 out of 18 comparisons to singlecomponent. Again, the conclusion that integrated methods by combining pairwise components help improve short term probabilistic load forecasts is empirically verified. Besides, since the optimal integrated method differs as the model class changes, but stays consistent within the same model class, it might originate from the fact that the performance of a specific integrated method rely heavily on the model class. Fig. 2. Actual load (black dots) and 98% PIs (blue: QR; red: QR-LRES) in the summer (Sub- Fig.(a)) and winter (Sub- Fig.(b)) peak week. Actual loads are more central within the 98% interval of QR-LRES than QR.

Conclusion and future direction
This paper formally investigates the integrated methods for short term probabilistic load forecasting by proposing three integrated methods, a.k.a. UBTS-QR, QR-LRES and LRES-UBTS, by pairwise combining the components from the forecasting engine. The Shanghai utility case shows that integrated methods help improve short term probabilistic load forecasts. Meanwhile, the model class is the most decisive factor for the performances of a specific integrated methods. Therefore, it is practically necessary to pick out the optimal integrated method for a certain model class, which requires numerous case studies. Following a similar framework of our proposed methods, more case studies are encouraged to conduct for other regions, model classes and forecasting horizons.
Future works will take the following two aspects into account. First, the intrinsic theory for integrating two methods should be more specific and clear, as the integrated methods only outperform the better singlecomponent methods slightly, but greatly outweighs the worse one. Hence, there must be some trade-off between the two single-component methods for integration, which needs more investigation. Second, more density or quantile combination theory should be dug out, as the core of integration is the mixture of two distributions.