Comparing Single and Hybrid methods of Deep Learning for Remaining Useful Life Prediction of Lithium-ion Batteries

. The prediction lifetime of a Lithium-ion battery is able to be utilized as an early warning system to prevent the battery’s failure that makes it very significant for assuring safety and reliability. This paper represents a benchmark study that compares its RUL prediction results of single and hybrid methods with similar articles. We suggest a hybrid method, named the CNN-LSTM, which is a combination of Convolutional Neural Network (CNN) and Long Short Term Memory (LSTM), for predicting and improving the accuracy of the remaining useful life (RUL) of Lithium-ion battery. We selected three statistical indicators (MAE, R², and RMSE) to assess the results of performance prediction. Experimental validation is performed using the lithium-ion battery dataset from the NASA and results reveal that the effectiveness of the suggested hybrid method in reducing the prediction error and in achieving better RUL prediction performance compared to the other algorithms .


I. Introduction
Lithium-ion (Li-ion) batteries remain among the main sources of energy for EVs and electronic equipment and considered appropriate for the environment [1]. Thus, it considers a perfect choice for the energy storage system (ESS). In few years, they play major power sources in different areas e.g. consumer electronics, electric vehicles, aerospace electronics, and mobile communications [2]. Due to their best advantages: high-density of power and energy, suitable for the environment, low discharge rate, lightweight, speedy charge, and long lifetime [3] [4]. However, there is the gradual deterioration of the battery 1 Corresponding author: b.zraibi@uhp.ac.ma during its lifetime, i.e. resistance increase and capacity decrease, for the reason that the external environment (e.g. discharge rates, temperatures), electrochemical reactions, and physical/chemical changes of the battery.
In spite of their advantages, degradation of battery performance over time can cause some losses such as catastrophic results of the devices, battery explosion of phones and EVs, rising maintenance costs, losses economic [5]. For avoiding these disasters, it requires more effort in lifespan prediction of Li-ion batteries that are very necessary for improving the reliability and safety of the overall energy system however its future behavior prediction is a difficult task. The battery management system (BMS) is necessary for ensuring the safety of Li-ion batteries, which is generally based on three essential elements: remaining useful life (RUL), state of charge (SOC), and state of health (SOH) which have a relationship respectively to the charge of the batteries and their aging [6]. The RUL is defined as the remaining number of cycles (charge/discharge) to get to the failure threshold, i.e. 70 % of the nominal capacity, of the battery with a specific output capacity. Thus, for the RUL prediction of Li-ion battery, it can use four methods: direct measurement, model-based, data-driven, and hybrid methods [7].
The first method is the direct measurement, which uses to calculate the capacity and impedance of battery cells using the open-circuit voltage and Electrochemical Impedance Spectroscopy, respectively. The second method is model-based that uses different models such as equivalent circuit, electrochemical, or empirical models (Kalman filter, unscented Kalman filter, particle filter, etc.). The third method is data-driven prediction methods like artificial neural networks (ANN), support vector machines (SVM), and relevance vector machine (RVM), etc. [8], the advantage of this method is using the historical data that help for avoiding the necessity of complex physical or mathematical models for battery capacity degradation [9]. While the fourth method is the hybrid methods which combine the previous methods with each other [3] [10].
Several researchers have depended on hybrid neural network methods to predict the RUL of Li-ion batteries. Their performance results showed a high prediction accuracy compared to single methods. Wu et al. presented a method composed of the feedforward neural network (FFNN) and importance sampling (IS). Young et al. applied ELM in two approaches, which combine with MPSO, i.e. MPSO-ELM [9], and combine with HKA, i.e. HKA-ELM [10], for improving the stochastic parameters of the ELM to achieve a good prediction accuracy. While Chen et al. [11] presented a method called ELM-BSASVM, which fuse ELM with a backtracking spiral algorithm (BSA) and support vector machines (SVM). Ren et al. [12] present a combined Autoencoder with Deep Neural Network (ADNN). Cadini et al. [13] presented a hybrid method that combines multi-layer perceptron and particle filter (MLP-PF). Fan et al. [3] proposed a mixedmethod named HA-FOSELM, which combines the Forgetting Online Sequential (FOS), Extreme Learning Machine (ELM), and the Hybrid Grey Wolf Optimizer (HGWO) algorithm. Wang et al. [14] used a method of the ensemble empirical mode decomposition EEMD with nonlinear autoregressive neural networks NARNN to predict RUL battery. Zhang et al. [15] propose a method fusing the partial incremental capacity and ANN. Cui et al. [16] present a hybrid method that combines unscented Kalman filter UKF, LSTM, and NN model. Yang et al. [17] proposed a mixed-method named CNN-BiLSTM, which combines CNN and bidirectional Long Short Term Memory (BiLSTM). Jia et al. [18] used a Wavelet neural network (WNN) with an unscented particle filter (UPF) to predict the RUL battery. Li et al. [20] and Ma et al. propose a hybrid neural network method, the first combines the Elman with LSTM [19], and the second combines CNN with LSTM [4].
The main contributions of this paper are to build a forecast model for RUL prediction of Li-ion batteries based on a hybridization technique by combining CNN, LSTM. To the best authors' knowledge, this is the first attempt to predict RUL Li-ion battery using CNN-LSTM based on univariate time series. Additionally, this work is to provide deeper insights on the single and hybrid methods for predicting the RUL of Li-ion battery by the comparison between methods of our work i.e., LSTM, CNN-LSTM, and methods in other papers. The proposed method obtained excellent results and achieved high predictive accuracy for the RUL estimation. Thus, it can help that to improve the lifetime control strategies and safety monitoring function of the battery for avoiding catastrophes.
The other sections of this paper are prepared as follows: Section II presents the architecture of CNN-LSTM. Section III introduces the RUL estimation techniques using the different methods. Section IV shows the experiment results and comparative study. Finally, a conclusion is given.

Related work of CNN-LSTM method
Deep learning methods have already been used in many papers, which have known success in many areas especially in time-series prediction, where they gave a good performance due to their good advantages e.g. selfadaptive, and capability dependence of nonlinear. Recently, the CNN-LSTM hybrid method has used in many studies because has realized good results when it applied in various fields. Among the fields that contributed is the medicine domain, ( e.g. forecast of haemorrhage into the cranium [20]), the plant domain, (e.g. estimating the characteristics of the plant [21] and recognizing the different headlines of Clickbait and classify it [22]), the pollution of air domain, (e.g. achieving a good prediction accuracy of air quality [23]), the financial domain, (e.g. estimating the volatility of gold [24]), the traffic domain, (e.g. avoiding the overcrowding of traffic [25]), the energy domain, (e.g. predicting the structure of energy at the following years [26], estimating the RUL and SOC of the batteries [27]), and more application domains. Depending on the good results shown by the CNN-LSTM hybrid method in previous works mentioned above. We propose to use it in our work to predict the RUL of Li-ion batteries for improving the performance of prediction accuracy of the same task.

The CNN-LSTM method
Every one of the CNN and LSTM has many advantages, thus, the combination of these networks in unite framework able to obtain good results of the RUL prediction of Li-ion batteries.
CNN is capable to capture the spatial relationship, extract local features and reduce the amount of the weights using the shared weights structure, and additional advantages such as the local dependency and scale invariance. Its structure includes the convolutional layer, which extracts different features of input through the convolution process, which contains several feature planes and neurons. The pooling layer extracts the secondary features, where it reduces the feature resolution and its surface dimension to obtain constant spatial features. The fully connected layers, and output layers. Both previous layers are congruous with each other since the pooling layer inputs are the outputs of the convolution layer. The fully connected layers are able to fuse the information from the previous layers. The final layer is the output layer that receives outputs of fully connected. The CNN is implemented using a kernel "K" or filter for obtaining a feature map S from the input vector A. The formula is: Fig.1. The architecture of the CNN Recurrent Neural Networks (RNNs) are a category of ANN where they have an advantage different from DNNs that is the internal memory, which leads to allowing information to continue and to remember past information. It profits the temporal correlations between neurons and is utilized to treat the tasks that include the sequence of the features. Thus discovering the best way to make the next estimation reasonable. Nevertheless, it has a problem with the long-distance dependencies that lead to disappear gradients and vanishing. To avoid these problems, LSTM is used to control the propagation of gradients information and remembering the parameters as input during the long term and it also has the addition operation that leads to solving the problem of disappearing gradient. The LSTM architecture is containing three gates i.e. the input (i), forget (f), and output (o), as well as a memory unit.
The LSTM is the cell of the LSTM consists of a longterm state of Ct and a short-term state ht. The calculation of the hidden layer nodes depends on the input of the current layer and the activation values of nodes at the previous moment. LSTM's equations can be defined as follows: = tanh( [ℎ −1 , ] + ) = ( [ℎ −1 , ] + = * −1 + * ℎ = * tanh( ) W is the weight matrices, b is the bias, is the sigmoid function, xt is the unit input at time t, ht-1 is the unit output of the previous LSTM cell, ct and ct-1 are the cell states at time t and t-1, respectively, qt is the hyperbolic function. The combination CNN-LSTM algorithm profit the advantages of CNN and LSTM. It able to extract two types of features are the spatial and temporal. The first is the interrelations within current inputs, while the second is the correlations between current RUL and past inputs. Thus, the proposed method is designed to take advantage of all of them; this architecture of the proposed method shown in fig.3 is designed to profit from all previous advantages mentioned above.

III. RUL PREDICTION
In this section, we will present the results of ours experimentations to predict RUL with three algorithms: RNN, LSTM, and CNN-LSTM. The experimental data of the NASA Prognostics Center of Excellence [31] is used in this paper for validation. It consists of aging data for 18650 Li-ion batteries of 2Ah rated capacity. Table.1 introduce the information about this battery as follow: The B0006 battery dataset, shown in Figure 4, contains 168 cycles, 80 of them are utilized for training and the rest for validation. The hardware and software environment shown in Table 2 was used to implement the three methods, i.e. RNN, LSTM, and CNN-LSTM, The rectified linear unit (ReLU) activation function is used along with Adam optimizer. Huber loss is also employed. Besides, to evaluate the RUL prediction performance of the algorithms, we use the mean absolute error (MAE) [28], root mean square error (RMSE) and R square (R²) [29]. They are defined in equations 3, 4 and 5.
Where is the true value battery capacity, while ̂ is the estimated value one, ̅̅̅ represents the average of actual one. When the MAE and RMSE it is close to zero, the capacity prediction accuracy is higher. As for R², a value close to one yields better accurate RUL prediction results
Below, the RUL prediction results are presented for two algorithms the real value is represented in blue color, forecast validation in green color that is starting in the 80 cycle, and new forecast results of these algorithms in red color.

RUL estimation with the RNN method
First algorithm is RNN, which is used to predict the RUL of Li-ion battery. Fig. 5 reveals RUL estimation performance for the B0006 Li-ion battery. It can be observed that the curve of validation is near the curve of true value, which makes this algorithm good learned. Nevertheless, it has also a bad new prediction. The MAE decreases gradually with the number of epochs. The MAE and RMSE values of RUL estimation are equal to 0.01594 and 0.02903; In addition, the R² value is equal to 0.917.

RUL estimation with the LSTM method
A second algorithm, namely LSTM, is used for the same task. Fig. 6 reveals RUL estimation performance. A better accuracy is obtained in the validation phase. The MAE decreases gradually with the number of epochs. The MAE and RMSE values of RUL estimation are equal 0.01299 and 0.02240, respectively, which is lower than those obtained with the RNN. While, the R² value is 0.950 that is high with respect to the RNN. However, it overcome to RNN, the prediction performance of new forecasts is still not satisfactory.  Next, hybrid algorithm combining the CNN, and LSTM, which are proposed to improve further the estimation performance and more importantly to achieve acceptable prediction for new forecasts.

RUL estimation with the CNN-LSTM method
The CNN-LSTM hybrid algorithm is introduced as a combination between CNN and LSTM for the RUL prediction of Li-ion battery.  It is clear from Fig. 7 that there is better consistency between the estimates and the true values. Moreover, a good performance is finally obtained for the prediction of new forecasts. In addition, the MAE curve does converge to the loss curve that remains close to zero in spite the high fluctuations. The MAE, R², and RMSE values are equal 0.0090, 0.957 and 0.02092, respectively, which are the lowest values obtained compared to previous methods.
This experiment demonstrates clearly the superiority of hybrid method over single methods in achieving high estimation accuracy and more importantly in predicting new forecasts. Thus, CNN-LSTM achieve the highest accuracy in the RUL prediction of the Li-ion battery. Next section summarizes the numeric RUL prediction errors and put them in perspective with other the best performant methods in literature.

RUL validation and the evaluation criteria
The above experiments reveal that hybrid method has successfully learned the dynamic nature of Li-ion batteries. Table 3 summarizes the three indicators used to evaluate prediction performance, which are the MAE, the R² and the RMSE.  Table 3 shows the performance of the proposed algorithms with the same starting point for testing sets for battery B0006, where the MAE and RMSE values with hybrid method are lower than that with single ones and the R² values with hybrid method are higher than that of single ones. This confirms that proposed hybrid method improve substantially the RUL estimation. In addition, the improvement from CNN-LSTM to LSTM is 30.5 % of MAE and 6.6 % of RMSE.

Results analysis and comparison
This section presents a comparative analysis of the RUL estimation accuracy between various comparable methods in literature among the single and hybrid methods. The single seem to be not adequate for time series data compared to the hybrid. This is revealed by the findings summarized in Table 3 and in the comparative analysis presented in this section. In order to compare more generally with other types of neural networks of prediction methods, we extract some results of performance methods from other papers, where their methods have the same dataset, i.e. NASA, and the same indicators of performance.  Table 4 reveals that the accuracy of all hybrid methods exceeds the one obtained with their single counterparts. This shows the power of hybridization in achieving better results and confirms the findings presented in this manuscript. In fact, hybridization is observed to decrease MAE, RMSE, and R².
According to the above analysis, we can deduce that the proposed CNN-LSTM RUL prediction approach is an excellent estimator with its high accuracy.

V. Conclusion
In this paper, a hybrid CNN-LSTM algorithm is suggested by combining two well-known algorithms, i.e., Convolutional Neural Networks (CNNs) and Long Short Term Memory (LSTM) to predict the remaining useful life (RUL) and improve the prediction of the Li-ion batteries. The proposed method is experimentally validated on a dataset obtained from NASA. Experimental results demonstrate the high RUL prediction capability of the Liion battery. Moreover, the prognostic of the proposed hybrid method is more accurate than single ones. Generally, three prediction performance indices reveal the highest accuracy of CNN-LSTM compared to RNN, LSTM, and other existing methods.