Multi-energy load forecasting model based on bi-directional gated recurrent unit multi-task neural network

. The complex coupling, coordination and complementarity of different energy in the integrated energy system puts forward higher requirements for the technology of multi-energy load forecasting. To this end, this paper proposes a novel multi-energy load forecasting model based on bi-directional gated recurrent unit (BiGRU) multi-task neural network. Firstly, through the correlation analysis, an effective multi-energy load input data set is constructed. Secondly, the input data set is utilized to train the BiGRU and master the evolution laws of multi-energy loads. Then, multi-task learning (MTL) is used to share the information learned by BiGRU from perspectives of different load forecasting tasks, so as to fully dig the coupling relations among various energy loads. Finally, different types of load forecasting results can be obtained. Simulation results show that BiGRU can simultaneously consider the known data of the past and the future, and it can learn more characteristic information effectively. At the same time, the proposed model utilizes MTL to carry out parallel learning and information sharing for forecasting tasks of various energy loads, which can dig the complex coupling relations among different types of loads more deeply, thus improving the forecasting accuracy of multi-energy loads.


Introduction
Nowadays, there are three major problems in the global economy: energy security, environmental pollution and climate change. Human society is also facing the continuous challenge of sustainable development. Traditional energy structure can no longer meet the needs of people for production and life, the energy structure needs to be transformed. Therefore, the integrated energy system is emerged. Integrated energy system integrates various forms of energy, such as cooling, heating and electricity, which improves the flexibility of energy system and energy utilization ratio.
In the aspect of load forecasting, relevant scholars have carried out a lot of research work. The forecasting methods based on traditional mathematical statistics include time series method [1], regression method [2] and exponential smoothing method [3]. The forecasting methods based on artificial intelligence include support vector machine [4], random forest [5] and neural network [6]. The above methods achieved high accuracy of load forecasting. However, most of the existing load forecasting researches are single-task learning (STL) and ignore the coupling among different types of loads. At present, research scholars have tried to apply nonlinear autoregressive model [7], support vector machine [8], radial basis function neural network [9], long short-term memory (LSTM) [10], etc. to forecast multi-energy loads and have achieved higher forecasting accuracy. However, further research is still needed to fully dig coupling relations among various energy loads and improve the accuracy of multi-energy load forecasting.
Artificial neural network is one of the representative methods of artificial intelligence forecasting, and it performs very well in the field of load forecasting. Among them, recurrent neural network (RNN) is a kind of artificial neural network, but it has problems such as gradient vanishing and gradient explosion. Therefore, some scholars proposed gated recurrent neural networks such as LSTM and gated recurrent unit (GRU) to enhance the usability of RNN. Compared with LSTM, GRU reduces the number of gated recurrent units from three to two [11], which can speed up training to a certain extent. However, the unidirectional GRU cannot consider the load information of future in forecasting process, therefore GRU is developed into bi-directional GRU (BiGRU) to solve this problem.
On the basis of existing research, this paper proposes a multi-energy load forecasting based on BiGRU multitask neural network. The proposed model utilizes BiGRU as the basic unit to construct a multi-energy load forecasting model, so that it can dynamically model multi-energy loads of forecasted time from the forward and backward directions. In addition, multi-task learning (MTL) is introduced into the proposed model, which makes the tasks of cooling, heating and electricity loads forecasting can be shared and learned in parallel, so as to dig the coupling relations among different types of loads more deeply and improve the forecasting accuracy of multi-energy loads. *

BiGRU multi-task neural network
The proposed multi-energy load forecasting model based on MTL and BiGRU is shown in Figure 1. It is mainly composed of following blocks: 1) The effective input data set of multi-energy load is constructed by using collected data.
2) BiGRU is trained with the constructed input data set to learn the complex characteristic information of multi-energy loads.
3) MTL is carried out for tasks of different types of load forecasting using BiGRU, so as to share characteristic and further dig the coupling relations among different types of loads.
4) The forecasting results of cooling, heating and electricity loads are output together.

2.1BiGRU
GRU is an improved neural network of RNN. It uses gated recurrent units instead of traditional recurrent units to solve the gradient problem of RNN. At the same time, GRU neural network only needs two gated recurrent units, reset gate and update gate, to solve the gradient problem, therefore the neural network with fewer parameters and complete connection is easier to train and avoid overfitting. The structure of GRU is shown in Figure 2.
However, in forecasting process, the unidirectional GRU neural network cannot consider the state of future. BiGRU can solve this problem, it adds a hidden layer on the basis of unidirectional GRU neural network, divides the forecasting process into forward and backward forecasting direction, and the hidden layers in two directions jointly determine the output result [12], so as to achieve the purpose of considering the past and future information of the forecasted time. BiGRU can be expressed as: According to experiments, the relevant parameters of BiGRU in the proposed model are set as follows: the number of BiGRU layers is 2, and the number of neurons in each layer is 32 and 64 respectively; the number of Dropout layers is 1, and the value of Dropout is 0.1; the number of fully connection layers is 2, the number of neurons in each layer is 50 and 30, and the activation functions are relu and linear, respectively.

2.2MTL
MTL is a machine learning mode that combines multiple related tasks to learn together based on shared representation. Using MTL instead of traditional STL for forecasting tasks can reduce the amount of data and the parameters scale of overall model, thereby making the model more efficient. Applying MTL to multi-energy load forecasting can make full use of specific information hidden in different types of load forecasting tasks, fully dig the coupling relations among different energy loads, and improve the generalization ability of model through weight sharing and parallel training. The general structures of STL and MTL are shown in Figure  3, where taski is the ith task (i=1, 2, ……, n).  MTL has many forms, such as joint learning, learning to learn and learning with auxiliary task. Its essence is to optimize multiple loss functions simultaneously. The model proposed in this paper adopts joint learning and uses the homoscedastic uncertainty among tasks to measure the loss in MTL [13].

3.1Data source
In order to verify the forecasting effect of the multienergy load forecasting model proposed in this paper, the multi-energy loads and related data from the main campus of the University of Texas at Austin are selected [14]. The data collected includes the historical actual cooling, heating, and electricity loads, dry bulb temperature (DBT), relative humidity (RH), the charging of thermal energy storage (TES charge) and the discharging of thermal energy storage (TES discharge) from September 2, 2011 to February 18, 2012. The sampling interval of the data set is 1h.

3.2Correlation analysis of factors
In the integrated energy system containing multiple energy forms, the correlation of different influencing factors for multi-energy load forecasting is different. It is necessary to analyze the correlation of these factors, so as to construct the input data set that can effectively forecast the multi-energy loads. Pearson correlation coefficient method is the most widely used correlation analysis method, but it focuses on the linear correlation among variables. In contrast, Spearman correlation coefficient can quantify the nonlinear correlation among variables, and it is more suitable for correlation analysis of various factors in the integrated energy system. Figure 4 is the scatter plots between any two types of loads. Meanwhile, the Spearman correlation coefficients between the cooling and heating loads, the cooling and electricity loads and the heating and electricity loads are calculated as -0.903, 0.923 and -0.801 respectively. It can be seen that there are strong coupling relations between cooling, heating and electricity loads, and there is a negative correlation between cooling and heating loads, while there is a positive correlation between cooling and electricity loads.  Table 1 shows Spearman correlation coefficient between multi-energy loads and influencing factors. It can be seen that each type of load has a strong correlation with DBT, RH and Hour. Spearman correlation coefficients between multi-energy loads, TES charge and TES discharge are all less than 0.15 that indicating weak correlations. Therefore, six factors including cooling load, heating load, electricity load, DBT, RH and Hour are selected to construct the input data set of forecasting model.

Processing of input data set
In this paper, data of the cooling, heating and electricity load 4h before forecasted time, the DBT and RH 1h before forecasted time and the Hour at forecasted time are selected to construct the input data set. Among them, divide the data from September 2, 2011 to February 11, 2012 as train set, the data from January 27 to February 11, 2012 as validation set, and the data from February 12 to 18, 2012 as test set. In addition, the input data set are normalized to avoid the adverse effects caused by dimensional differences of data for sample analysis.

Results
In order to illustrate the superiority of the proposed model, the LSTM-RNN model proposed in [17], the DBN model proposed in [7], the GRU model and the BiGRU model are selected to be comparison models.
The forecasting results of five models for cooling, heating and electricity loads are shown in Figure 5, 6 and 7 respectively. It can be seen that these five models have high fitting degrees between forecasting curves and actual curves of multi-energy loads, which indicates that they can learn the evolution laws of multi-energy loads effectively, so as to realize the forecasting of various energy loads. However, in contrast, the curves of multienergy loads forecasted by the proposed model have higher fitting degrees with actual curves, and it can better judge the different changing trends of various energy loads in the future.   The mean absolute percentage error (MAPE) and the root mean square error (RMSE) are selected as evaluation indicators to evaluate the five forecasting models more intuitively. The results are shown in Table evaluation indicators. Among them, MAPE is reduced by 9.29%, 26.54% and 10.30% respectively for forecasting cooling, heating and electricity loads. It shows that the introduction of MTL enables different tasks for forecasting various energy loads to learn jointly, obtaining more effective information, which contribute to improve the forecasting accuracy of multi-energy loads. In addition, compared with the other two models, no matter what type of load, the proposed model still has higher forecasting accuracy, which illustrates the superiority of the proposed model.

Conclusion
Integrated energy system is a complex system with muilt-energy loads and strong coupling. How to make full use of the known information and deeply dig the coupling relations among different types of loads, and construct a multi-energy load forecasting model with high accuracy is urgently to be solved. This paper proposes a multi-energy load forecasting model based on BiGRU multi-task neural network. The proposed model can consider both the known information of past and future, and it can dynamically model the multi-energy loads at forecasted time from the forward and backward directions. In addition, through the introduction of MTL, different types of load forecasting tasks can carry out parallel learning and learning results sharing, so that the proposed model can dig the coupling relations among various energy loads more fully. Simulation results verify the best fitting degree and the highest accuracy of multi-energy load forecasting for the proposed model compared with other comparison models.