SOC Estimation of Li-ion Battery using on Variational Mode Decomposition and Transformer-Generative Adversarial Network

.


Introduction
State of charge (SOC) is one of the core parameters of lithium power batteries, the accurate estimation of its value not only ensures the safe and reliable operation of the battery, but also improves the battery range and service life [1] .There are four common approaches for estimating the SOC of Li-ion batteries, which are modelbased method [2] , fusion model-based method [3] , experimental-based method [4] and data-driven method [5] .The experiment-based method is simple in principle, easy to implement, and limited in practical application [6] .With the rapid development and improvement of machine learning, the data-driven based method is widely used in the estimation of SOC [7][8][9] , this method has strong generalization and robustness. [7]applies the least squares support vector machine (LSSVM) combined with the distribution estimation algorithm to the estimation of lithium battery SOC, and verifies its effectiveness in simulation experiments.Neural networks, especially deep neural networks, have highprecision generalized approximation ability and have gradually become the mainstream method for estimating battery SOC in recent years. [8]used back propagation neural network (BP) for SOC estimation, and this method can effectively reduce the estimation error of SOC. [9]uses the long and short-term memory network (LSTM) in recurrent neural network (RNN) for SOC estimation, which can encode the data correlation to achieve accurate estimation of SOC.
In this paper, VMD method is used to decompose the collected SOC signal into six intrinsic mode functions (IMFs) to reduce the non-stationarity of the data, and then the obtained six sub-modal components are used as training data.The GAN is used to predict the value of SOC, and the Transformer model is used as the generation module of GAN in the estimation process.It can improve the quality of GAN generated data.

Variational Mode Decomposition (VMD)
VMD is often used to process nonlinear signals.It can decompose complex original data to obtain a series of modal components [10] , which can effectively extract the characteristics of SOC data and reduce the influence of its nonlinearity and non-stationarity on the estimation results [11] .

Generating Adversarial Networks (GAN)
The GAN network simultaneously trains the generator G and discriminator D networks [12] .The two models (G and D) are learned alternately until D is unable to determine whether it is real data, that is, Nash equilibrium is reached [13] .The G and D loss functions are defined as: For the original GAN model, the phenomenon of G gradient disappearance and optimization target confusion is easy to occur during the training process, which makes the generated data features unevenly distributed and leads to the collapse of the model.

Transformer model
Transformer is widely used in the field of natural language processing (NLP) [14] .The self-attention mechanism in the Transformer model is good at capturing the common law of the internal correlation of data or features.The core process of the attention mechanism is to use the query vector matrix Q and the key vector matrix K to calculate the weight of attention, and then apply it to the value vector matrix V to obtain the whole weight and output.The calculation formula of the output vector is as fellow : The feedforward neural network is composed of two fully connected networks and an activation function Relu.The operation formula is as fellow: ( ) In the formula,  is the input vector data with d dimension, 1 b and 2 b are the bias parameter, 1 W and 2 W are the parameter matrix of the linear mapping.

SOC estimation model of Transformer-GAN
As characteristics of SOC data of lithium-ion battery is the strong nonlinear, the deep learning method can automatically learn the nonlinear relationship between measurable parameters and SOC without the knowledge of battery working principle or complex mathematical model.The traditional back propagation ( BP ) neural network has low accuracy in SOC estimation, and the recurrent neural network ( RNN ) is also easy to fall into local optimum.Therefore, this paper uses VMD to decompose the SOC signal to obtain six intrinsic mode functions and increase the characteristics of the input signal.At the same time, there are the problem of model instability in the SOC estimation of the original GAN model, the combination of the powerful feature extraction ability of the transformer network and the generative adversarial network (Transformer-GAN) with gradient penalty is used to deeply mine the internal information of the SOC data.
In this paper, a estimation model of GAN based on transformer is proposed.The proposed estimation model combines different techniques such as sample generation, feature extraction and objective function optimization.The G of the Transformer-GAN network is based on the Transformer model.Combined with the powerful correlation information modeling ability of Transformer for time series, the information of time series data is completely presented, which not only improves the estimation accuracy, but also shortens the estimation time.The coding part extracts the hidden layer vector representation of SOC time series data, and then restores the original feature sequence through the decoding network to realize the self-reconstruction ability of time series feature data.The network input part contains the encoding of the position information of each time point in the original feature sequence and the time series fragment.The encoding network consists of three layers of stacked modules containing the self-attention mechanism, and the feedforward neural network contains an independent hidden layer.After the input is encoded, the coding vector of the important feature information implied in the input time series is obtained.After obtaining the coding vector, the vector is combined with the feature sequence and the position coding information to input the decoding network at the same time, and the original sequence is reconstructed by the decoding network.The coding network is also composed of three layers of stacked modules including selfattention mechanism, and the coding vector is the input signal of the middle layer.After obtaining the decoding vector, the reconstructed time series is obtained by linear layer output.
The supervision information of network training is the original time series data, and finally the Transformer model learns the correct coding mode through the correct time series data.
When Transformer model are used for sequence reconstruction, it cannot correctly obtain time series features from normal time series and reduce the interference of abnormal sequence.Therefore, the completed Transformer codec network needs to be used in conjunction with the GAN training architecture.In this paper, the Transformer model is used as the generator network, and for the discriminator, only the binary classification task needs to be completed.Therefore, the discriminator network is realized by constructing a simple multi-layer perceptron.

Estimation process based on VMD and Transformer-GAN
Combining VMD with Transformer-GAN, a SOC estimation model based on VMD-Transformer-GAN is obtained.VMD is used to decompose the original SOC data to obtain six components, and the training set components are input into the Transformer-GAN model for training, as Fig. 1.
The specific steps of constructing the VMD-Transformer-GAN model are as follows: 1) VMD is used to decompose the SOC data to obtain six modal components, which are normalized.At the same time, the first 70 % is divided into training set and the last 30 % is divided into test set.
2) The SOC data is input into the Transformer-GAN model as the training object, and the number of hidden layer nodes, learning rate and training times are set.

Experiments
The test data in this section are based on the lithium battery test data of the University of Maryland in the United States [14] .In this test, the Arbin BT200 battery test system was used to test the Samsung INR 18650-20R battery.The specific battery parameters are shown in Table 1.

Fig.2 SOC data
The SOC data of the battery is obtained by using the ampere-hour integral method.The SOC curves are shown in Fig. 2.

Model training
Firstly, VMD is performed on the data of the monitoring points, and k is set as 6.The data is decomposed into 6 sets of IMF components ( 1 IMF , 2 IMF ,……, 6 IMF ) with different frequency scales, as shown in the following Fig. 3.

Fig.3 VMD decomposition results
Then the obtained data is input into the Transformer-GAN model, in which the epochs of the cycle number is 50, the batch size is 10, and the learning rate is 0.002.Finally, 70 % of the SOC data is used as the training set, and 30 % of the data is used as the test set.

Model estimation and analysis
Fig. 4 The estimation results of different algorithms It can be seen from Fig. 4 that in the single estimation model, compared with the the rest of model, the estimation effect based on the VMD-Transformer-GAN model is the best.Although the predicted values on some mutation data are different from the actual values, the overall estimation results are consistent with the actual values, and the fluctuation period is also similar, which can truly reflect the deformation of the monitoring point displacement.
It can be seen from Table 2 that the GRU model has the largest deviation.Comparing the single model with the VMD-Transformer-GAN model, it can be found that the VMD-Transformer-GAN model has better estimation effect, and the three evaluation indexes have decreased, which reflects the characteristics of the combined estimation model to make the estimation process more detailed and accurate.It shows that this method can reduce the influence of nonlinearity and non-stationarity of lithium battery SOC data on estimation accuracy.Comparing the three models, it can be seen that the accuracy of VMD-Transformer-GAN is higher.

Conclusion
In order to improve the estimation performance of lithium battery SOC, alleviate the problems of gradient disappearance and gradient explosion that are easy to occur in neural networks, and effectively characterize the dynamic physical and electrochemical characteristics inside the battery, this paper introduces a VMD-Transformer-GAN model.The data of battery voltage, current and temperature, which are relatively easy to observe, are used as the input of the network model, which reduces the difficulty of data acquisition and improves the practical operability.The proposed network model maps the input observation data to the SOC estimation of the battery through its own learning ability, and realizes the effective characterization of the nonlinear characteristics inside the battery based on datadriven without direct physical or electrochemical models.The sufficient experiments on the Maryland lithium battery benchmark dataset show that the bidirectional learning strategy can effectively improve the SOC estimation performance.Compared with the GRU network, the MAE value of the VMD-Transformer-GAN network is reduced by 82%.Compared with the LSTM network, the MAE value of the VMD-Transformer-GAN network is reduced by 44.74 %.The two-way learning strategy proposed in this paper is pure data-driven.In the later stage, the relevant physical model or electrochemical characteristics of lithium battery will be further combined to construct an effective SOC estimation model in a more practical working environment, so as to improve the estimation performance and enhance the interpretability of SOC estimation method.

Table 2 .
Estimation error statistics of each algorithm The original data is used as input, 40 hidden layer nodes and the same number of model training times are set, and RMSE is used as the evaluation index of model estimation effect.The test results of each model are shown in Fig. 4, and the evaluation index of each model is shown in Table 2.