Optimization of Artificial Neural Networks Based Models for Wave Height Prediction

For an efficient wave energy extraction, the evolution of some specific parameters must be known. These parameters, like significant wave height and period, are mainly determined by the wind speed and influenced by some sea environment characteristics. Their evolution in time is one of the basic information necessary for designing of an accurate energy conversion system. In many scientific works the benefits of artificial neural networks based modeling are presented. These models allow the prediction and optimization of the wave parameters starting from experimentally acquired data. Due to specific calculus method of the artificial neural networks, in order to obtain accurate results, a very important step is the appropriate neural model design. If the model is optimal correlated with the data processed, the results obtained can be more significant than those coming from the mathematical formulas. The main neural models parameters that must be taken into account for an optimal design are model structure, transfer function and training algorithm. This paper presents an investigation of the results obtained with different models, proving that for a specific dataset a specific neural model offers the best results. Several models are analyzed, for a dataset corresponding to specific point in Black Sea and a comparison of results is presented.


Introduction
As the wave energy extraction efficiency is directly linked to wave heights, which are determined by wind speed, water depth and weather conditions, a tool allowing to model and predict the wave height evolution would be very useful. Formulas based models are widely used in this purpose, offering less or more accurate results, as the mathematical apparatus cannot represent all changes into the conditions governing this phenomenon. On the other hand, all the data used for modeling are experimentally acquired from buoys and satellite measurements available in free access databases. Taking this into account, other modeling tools are used -the Artificial Neural Networks (ANNs).
The main ANN based modeling benefit is the ability to compute results directly from experimentally acquired data, without the use of mathematical formulas. But a price must to be paid: a reasonable error must be accepted. Taking into account that at global scale over 1TW is available as wave energy [1] the establishing of an optimized modeling procedure capable to rise the efficiency of wave energy conversion is highly desirable.
Several neural network based models for wave characteristics prediction are presented into the literature [2,3] praising that the neural network parameters tuning is very important in order to obtain accurate results [4]. Also, taking into account the fast changes of weather condition in late years, an appropriate choosing of the processed data must be performed.

Artificial neural networks
Artificial neural networks (ANNs) are parallel computing tools offering several benefits comparing to von Neumann architecture based machines -classical computers -which works in a sequential, algorithms based way. The basic unit of a neural network is the artificial neuron, figure 1.  [5] As can be seen into the figure 1 the artificial neuron accept the data (x 1 , x 2 ...x n ) as inputs, multiplied with some values -weights (w 1 , w 2 ....w n ), process the sum with an function (f -transfer function) and pass forward the result.
An artificial neural network is composed by several artificial neurons, organized into a complex structure. Following the way in which information travel into the structure, several networks architectures can be found [6]. In wave prediction domain the most used is a layered structure Multi-Layer Percepton (MLP) where the information travel only from inputs to outputs (feedforward), figure 2. The goal of ANNs is to modify the weights such way that for known input data known output data is obtained, with a prescribed error. This is known as "training" or "learning" stage. These modifications are based on algorithms which role is to minimize the ANN output data error comparing to desired values. Several algorithms are available, following the ANN structure. After the ANN is trained, it can be used for several goals like prediction, optimization, image recognition etc.
Other essential element of ANNs is the transfer function. There are different functions but the sigmoidal one is widely used [8]. Usually the same function is used for each neuron in the ANN but also this can be different in different layers.
In order to build an ANN based model, one must perform a fine tuning, choosing the optimal structure, transfer function and training algorithm, only this way the best results can be obtained.

Wave energy formulas
In order to compute wave energy for real seas conditions the formula presented in equation 1 [9] can be used. Here, in order to take into account that the wave stream on the real sea surface is composed by a series of wave amplitudes varying in time following a frequency spectrum, the spectral estimate of significant wave height Hm0 is used. Following other authors [10] even the Hm0 values can be replaced with Hs only for very narrow spectra, rarely occurring in real life, the difference between these two values is very small in most of the cases so, the equivalence Hm0=1.05Hs can be accepted. As consequence, equation 2 can be used for wave energy estimation.
On the other hand, as several authors found [11], the wave significant height Hs is mainly determined by the wind speed value so, the most influencing factor in equation 2 is determined by wind speed over sea surface. As consequence, in order to predict wave energy the wind speed value must be taken into account. E = g(H m0 ) 2  Taking into account that for large areas the gravitational acceleration (g) value can be considered constant and sea water density () value is negligible, in order to find an optimal point for energy extraction the wave significant height must be predicted, based on wind speed values.
In [12] the equation 3 is used for wave significant height prediction. where: Based on equation 3, was found in [13] that significant differences exists between the predicted values and the measured ones. Taking into account that wave significant height is also influenced, beside wind speed, by the water depth, it was proposed a new model starting from fitting experimentally acquired data, equations 4.
for U 10  In other works [14] also the fitting of experimentally acquired data method is used for establishing the significant wave height value based on wind speed is used, equation 5. where: Other authors [15] shows that also the atmospheric conditions, have an influence on wave generation by wind speed but they don't propose an appropriate model.
Taking into all presented above, it is obvious that the best results for significant wave height -and consequently for wave energy -prediction based on wind speed are obtained by fitting the experimental acquired data. More, there is no universal valid formula, for each tested area different equation being found.

Significant wave height prediction based on ANN models
One of the most valuable property of ANN based models is the ability to compute a "black-boxed" link between input-output datasets, link that can be used for prediction of unknown outputs starting from known inputs. Based on this, taking into account the availability of large databases regarding the wind speed and significant wave height, an ANN model trained with appropriate data sets will provide results with higher precision than the fitting models or even than the numerical models like WAM (Wave Analysis Model) or SWAN (Simulating WAves Nearshore) [16].

ANN models building principles
In present study the performances of several ANN models for significant wave height prediction are analyzed, trying to find the optimal combination of type, structure, transfer function and training algorithm. Taking into account that the ANN structures are directly influenced by analyzed datasets, the corresponding models should be built and optimized for each and every studied area, here two locations in Black Sea, near Romanian shore, were chosen (figure 3). In order to build the ANN models some actions was performed: -choose the datasets: as input datasets several choices are available. Some works use only date, hour and significant wave height value [17], others take into account also the wind speed [18]. In present work, as inputs for the ANN models were used: the water depth, date, hour, wind speed and corresponding significant wave height values. In figure 4 are presented the significant wave height value evolution during year 2016 for the chosen locations. The recorded data for year 2017 were used for model's prediction error estimation.
-choose the appropriate ANN type: for wave height prediction the best choice is the MLP type.
-choose the optimal ANN structure: for each dataset there is a specific structure offering the best results [19]. In this purpose a genetic algorithm was used. -choose the optimal training algorithm: this is an optimization problem, different training algorithms offering best results for different datasets and ANN structures [20]. As consequence, several training algorithm must be tested.

ANN model structure establishing
In order to establish the optimal ANN structure, the Pythia software was used. Embedded evolutionary generator, based on genetic algorithm with settings presented in table 1, allowed to establish two optimal ANN structures, figure 5.  As can be observed into the figure 5, the best ANN structure is 5-5-9-8-1, followed by 5-5-5-5-1 on the second position. The meaning of the numbers are: 5 -input layer, with five neurons corresponding to the five input data: month, day, hour, depth, wind speed and recorded significant wave height, with leading times six hours; 5,9,8 -hidden layers, with five, nine and eight neurons respectively; 1 -output layer, predicted significant wave height.
These three structures will be tested with different training algorithms and transfer functions, in order to establish the best ANN model.

ANN model optimizing
As the performance of an ANN based model depends not just by network structure but also by transfer function and training algorithm, all these parameters must be taken into account. With this purpose, Neural Power software was chosen, as it offers the possibility to vary these parameters.
Two training algorithms were chosen: Levenberg-Marquardt and Quick Propagation, as these offers the best performances for MLP networks [21].
As transfer functions, Sigmoid and Tanh functions, figure 6, were chosen, as these are widely used in prediction MLP models [22]. In order to investigate the functions response, two slope values were tested. All layers into the ANN were subjected to the same activation function.
tanh(x) = (2/(1+e (2-x) )) -1 a) b) Figure 6 [30]. Activation functions a) sigmoid, s -slope; b) tanh As the performance criteria, allowing to choose the best ANN model in the training stage, two criteria were chosen: the sample Pearson correlation coefficient value better than 0.95 [23] (higher values leads to an over-fitted network, with low generalization capabilities) or a stagnation of correlation coefficient value. The root mean square error (RMSE) also was recorded, looking for values below 0.300 [24]. In table 2 the obtained results after ANN models training are presented.  Looking into the table 2 one can see that there are models with lower values of correlation coefficient and/or RMSE. Also, the differences between models are in number of training cycles. As consequence, the best model for studied phenomenon will be that which offer the highest CC values and lowest RMSE values. From this point of view it is obvious that the best ANN model is the 5-5-9-8-1 one, trained with Levenberg-Marquardt algorithm and using a sigmoidal transfer function with slope value 1.

ANN model validating
Before using the ANN model a validation procedure is required, allowing to evaluate the prediction error. In this purpose, several data, acquired in 2017, were used. The chosen locations were the same locations used for ANN model development, figure 3. The results are presented in table 3 As can be observed into table 3, the obtained errors values are acceptable [2] so, the model is valid and can be used.

Discussion
After the ANN model is validated, this can be used for several goals: prediction of wave significant height when the inputs values are known, establishing the most important inputs for output evolution and optimization of inputs values such way that the output have the desired values -highest or lowest.

Prediction
The ANN model allows to predict the significant wave values, based on the time coordinates, water depth and wind speed at interest location. In table 4 are presented the errors obtained for prediction of significant wave height in As can be observed into table 4, the prediction precision is acceptable, being consistent with others researchers' results [25].
The prediction error values are different for different water depth values. This means that an ANN model offers best precision for predictions performed in relatively close area around the geographic location from where the model building data were extracted. Anyway, the input values for an ANN model used for prediction cannot be outside the boundaries used for training.

Importance establishing
Other facility offered by the ANN model is the analysis of inputs' influence on outputs. Based on weights sum from a particular input to the output of the neural network, the most important input can be established, figure 8. As can be seen in figure 8, the most important factor for significant wave height value is the wind speed, followed by water depth. This is in concordance with the acquired values presented in figure 4.

Optimization
The ANN model can be used also for an optimization procedure: the establishing of desired values of significant wave height in chosen area.
The optimization can be performed with all inputs variable or just few of them. If the procedure is applied for an area, looking for the maximum significant wave values, all inputs should be variable. If the study is for one precise location, them the water depth should be fixed, with the corresponding value. Also, for the optimization procedure several algorithms can be used.
Into the table 5 are presented two examples of optimization, both for area and for point location. There were used two optimization algorithms: Force Approach (FA) and Genetic Algorithm (GA). The optimization procedure can be used in order to find if and when the Hs values are over the maximum or below the minimum accepted limits for the wave energy extraction devices.

Conclusions
Taking into account all the results presented above, some conclusions can be drawn: -In order to convert wave energy, which have an enormous potential, the prediction of the Hs values for a chosen area or location is one of the most important steps in converting system designing.
-The ANN models are very suitable for Hs values prediction, starting from previously recorded data, i.e. date, wind speed, water height. Comparing with models based on mathematic relations, which are rigidly linked to the same equations, the ANN models have the ability to adapt differently to each and every dataset acquired, offering the best analyze for the particular investigations. -However, the high versatility of ANNs can be a dangerous trap, leading to false results. As a consequence, the design of an ANN based model must be carefully approached. The main aspects of the model (network structure, transfer function and training algorithm) must be optimized for each dataset in order to obtain the best model. The usually try-and-error method for network structure establishing must be replaced with procedures based on genetic algorithms, allowing this way the identification of the most appropriate structure for a chosen dataset. Also, the use of advanced ANN software, allowing to manipulate both the transfer function and the training algorithm is highly recommendable.
-The results obtained in this work show that not just the wind speed, but also the water depth has important influence on the Hs parameter.