New method for wind potential prediction using recurrent artificial neural networks

. The aim of the study is to find the right architecture of the NARX neural network, in order to perform the daily prediction of the maximum wind speed of Laayoune city. We relied on the Levenberg-Marquardt optimization algorithm. The RMSE error metric showed that NARX-SP outperforms NARX-P.


Introduction
The fight against climate change requires ecological solutions such as renewable energies [1][2][3]. We are interested in wind power [4], which considers itself an important source for the production of green electricity. Wind technology is totally dependent on wind, which is a very variable meteorological parameter. Its intermittence creates endless fluctuations in the production process, these variations directly influence the stability of the energy produced. The management of wind production is based on several stages, prediction at different time scales are essential.
Our study mainly focuses on short-term wind prediction [5]. Several methods can solve the wind prediction problem in a different way but we choose the statistical methods [6]. These statistical methods are also divided into several sub-methods. In the literature, we find statistical methods based on time series such as the best-known models ARMA (Auto-Regressive Moving Average) [7], ARIMA (Auto-Regressive Integrated Moving Average) is the most used model in the wind energy field which were established by box and Jenkins, ARX( Autoregressive with Exogenous Inputs)... which are linear models. While wind is characterized by a nonlinear behavior. For this reason, we propose the most recent methods which are the methods of artificial intelligence (AI) like ensemble methods, for example, random forests, decision trees, as well as other types of methods like SVR (Support Vector Regression) ...and also artificial neural network, of which there are two types of networks, those inspired by the electrical part of a biological neural network, and those inspired by its chemical part. The former are the case of our study, of which we have chosen from which we have chosen the NARX neuron network, in its two forms, NARX-SP (open loop) and NARX-P (closed loop).
The aim of our work is to compare NARX-P and NARX-SP in order to find the best model among them that will be able to predict the short-term maximum wind speed of Laayoune city. NARX-P and NARX-SP have been optimized by the Levenberg-marquadrt algorithm.

Research methods
In order to find the best model to predict wind speed on a short time scale, we collected meteorological data from the city of Laayoune [8][9], taken every day (Daily) throughout the three years 2017-2018-2019. The parameters are wind speed (WS) in m/s, wind direction (DIR) in deg, temperature (TMP) in °C, air pressure (ATM) in hPa and humidity (H) in %. We prepared the data in Excel and then trained it in Matlab.

Neural Network
A neural network is part of artificial intelligence [10] and in particularly, in deep learning sphere, which is a science, inspired by brain activities. In this study, we have chosen the NARX neural network, a feedback neural network, i.e. recurrent artificial neurons network [11]. We worked with both architectures, NARX parallel or closed loop, and NARX serial-parallel or open loop, we kept the same architecture for both networks, a threelayer architecture, an input layer with 5 inputs, an intermediate layer and an output layer containing a single output. Any artificial neural network is a multilayer perceptron (in all cases of learning, either supervised learning or unsupervised learning).

Multilayers perceptron (MLP)
The multilayer perceptron is a combination of linear separators which allows producing an overall non-linear separator. MLP is a neural network of a classical hierarchy having inputs or a single input, weights and Bias corresponding to the inputs, a linear summation function, an activation function and one or more outputs, the network is defined by the following relation: There are several types of classical neural networks such as Adaline (Adaline (Adaptive Linear Neuron), Madaline (Multi-Adaline) also RBF neural network (Radial Basis Function), the latter replaces the sigmoid activation function by radial basis function [12].

NARX Close Loop
NARX is a recurrent, i.e. cyclic or loop, neural network. NARX is based on the ARX time series. One can have open-loop NARX (NARX-SP) and closed-loop NARX (NARX-P) [13]. NARX-P is characterized by the feedback of past estimated values from the actual output to the input (an early feedback), and is a real-time configuration. The function that represents the operation of the NARX model in closed loop mode is:

NARX Open Loop
The NARX open-loop network generally has two inputs, the first input based on the inputs x(t) or u(t) which is the sign of the inputs, and the second input presents the actual output of the network y(t), it is used as it is, instead of returning the estimated output. While the regressor of the output is formed only by real values of the target series, i.e. the output of the network y(t+1) so that the (SP) mode is used for learning between the target variables and the components which are the input series.
Some studies have used NARX for forecasting wind speed, hourly solar radiation, forecasting network traffic, pollution, etc.
Annalisa Di Piazza et al used NARX to perform the hourly forecast of solar irradiation and wind speed, considering temperature as an exogenous variable. NARX was based on two techniques, the optimization technique based on a genetic algorithm (GA) and method that determine the optimal network architecture by pruning (optimal brain surgeon (OBS) strategy) [14].
Hong Thom Pham et al, chose to work with a highpowered hybrid NARX-ARMA model in order to predict the long-term condition of a machine. This forecast was based on vibration data. NARX is used to predict the deterministic component and ARMA to predict the error component [15].
As the NARX model is a type of dynamic neural network. Ines Sansa et al used a simulation to prove NARX's predictive effectiveness against static network models [16].

Input variables
The table below shows the correlation between the variables used. The correlation coefficient is used to select the important parameters. The table shows that all the weather parameters mentioned are to be used because all the correlation coefficients are low.

Results and discussion
The objective of the study is to find the most suitable network among NARX-P and NARX-SP for the daily prediction of the wind speed of Laayoune city. We optimized the learning in both networks by the Levenbrg-Marquardt optimization algorithm [17][18][19].
LM combines the advantageous techniques of the Gauss-Newton and gradient descent methods, is the derivative of the Netwon method, a second-order optimization technique, and used in the case of a continuous and differentiable function in a specific interval, it is performed to minimize functions that have sums of squares of non-linear functions.
The Levenberg-Marquardt algorithm is expressed as: First of all, choosing the right training function helps to predict the wind speed very efficiently, i.e. to learn the non-linear behavior perfectly. Secondly, The correct choice of parameters such as the number of inputs, the number of neurons, and the correct activation function in the hidden layer also plays a primary role in the correct adjustment of the weights and subsequently in obtaining a or several optimal wind prediction models. We have chosen the same architecture for the two neural networks; we have five inputs (t-1): WS (t-1), DIR (t-1), ATM (t-1), H (t-1), TMP (t-1) in the input layer, a hidden layer with 10 neurons and hyperbolic tangent activation function, and only one output in the output layer with linear activation function. Closed loop (or parallel) (18-10-1)

Fig. 2. NARX-P Representation (18, 10, 1)
The NARX-P model, with 18 inputs (5 × 3 of the exogenous inputs) and (1 × 3 of the inputs from the output), the number of inputs is under the condition of the chosen delays, [1:2], represents the exogenous series (u) at present, time (t-1) and past (t-2), as well as for the input from the output ‫ݕ‬ ො is at times (t), (t-1),(t-2), a hidden layer of 10 neurons connected by the hyperbolic tangent function, and an output activated by the linear function. With: and Then: The input to the perceptron in general for NARX-P and NARX-SP is:   Figure 4, it can be seen that the NARX-P network was unable to correctly predict the wind speed variables.  1  57  113  169  225  281  337  393  449  505  561  617  673  729  785  841  897  953  1009  1065 Actual Forecast NARX-SP keeps the same structure as NARX-P, the objective is to compare the two models with the same parameters.

NARX
With:  The NARX-SP model was able to learn and subsequently predict the wind speed series perfectly. The network architecture was chosen after several trials, we changed the number of hidden layers as well as the number of neurons in the hidden layer, but the chosen architecture presented the best results among others.
In this study, we should have predicted quantitative variables, i.e. it is a multiple linear regression problem (5 inputs and a single output). For this regression analysis, we used four types of forecast error measures, as shown in the table below. RMSE [20].

Conclusion
This study attempts to find the most appropriate network for predicting the maximum wind speed of the city of Laayoune. We chose two different architectures of the NARX recurrent neural network, closed loop and open loop, trained by the Levenberg-Marquardt algorithm. NARX-SP showed good similarity, and the estimated variables were very close to the real variables.