Optimal Energy Allocation Algorithm of Li-Battery/Super capacitor Hybrid Energy Storage System Based on Dynamic Programming Algorithm

The establishment of an integrated fast charging station for photovoltaic storage and charging is an effective solution to fast charging of electric vehicles. For the li-battery/Super capacitor hybrid energy storage system, it is an effective method to reduce the cost of the system by extending the life of the li-batteries. This paper establishes the li-battery cycle life estimation model with irregular discharge and proposes an optimal energy allocation algorithm of li-battery/super capacitor hybrid energy storage system is proposed based on dynamic programming algorithm. Simulation results are presented to validate the theoretical analysis.


Introduction
To meet the growing demand for the fast charging stations of electric vehicles, it is necessary to build fast charging stations on a large scale. Among a variety of rechargeable batteries, li-batteries have relatively high power density and energy density, and are widely used and mature in technology. In addition, the international society is promoting the construction of the new energy vehicle power battery recycling system. As a power battery for electric vehicles, li-batteries need to be replaced when the battery capacity decays to 80% of the rated capacity. However, the retired li-batteries still have a relatively high capacity. Li-batteries are selected as the energy storage device of the hybrid energy storage system of the charging station for echelon utilization [1].
The charging station load has the characteristics of randomness and high power. Irregular charging and discharging caused by frequent charging and discharging of li-batteries and large-scale fluctuations of load power will greatly shorten the service life of the battery. The super capacitor unit in the hybrid energy storage system can respond to the high frequency part and high power part, which can effectively extend the service life of the lithium battery, reduce the peak load, smooth the load curve, and reduce the negative impact of the load on the distribution network.
When solving the capacity optimization configuration problem of the hybrid energy storage system, the capacity configuration optimization problem [2][3] is highly coupled with the energy allocation problem. In the parameter optimization problem of a hybrid energy storage system, the optimal configuration solution is the number of energy storage devices when a performance indicator is optimal, which is an optimal design. And the energy allocation problem is an optimal control problem. The two are coupled with each other. Therefore, capacity optimization and energy allocation strategy optimization should be considered simultaneously in parameter optimization. At present, most researches on hybrid energy storage systems focus on the optimization of energy allocation strategies under fixed parameters.
Reference [4] selects the minimum total cost of the hybrid power system and the minimum capacity loss of the Li-battery as the optimization goals for the hybrid energy storage system of electric vehicles, and uses the non-dominant sorting multi-objective genetic algorithm (NSGA-II) [5] to solve the optimal configuration scheme under the multi-objective.
For the hybrid energy storage system of mediumsized electric vehicles, reference [6] selects the maximum battery life and the minimum overall size as the optimization goals, uses dividing rectangles (DIRECT) algorithm to find the optimal configuration combination, and verifying that the DIRECT algorithm can solve the Pareto front of this multi-objective optimization problem.
For the hybrid energy storage system of trams, reference [7] adopts an adaptive energy management strategy based on fuzzy logic, and selects the cost of absorbing energy from the catenary and the operating cost of the hybrid ESS (including investment and cycle costs) as the optimization target, to optimize the optimal size of trams through multi-objective genetic algorithm (GA).
For the hybrid energy storage system of electric vehicles, reference [8][9] uses an integrated optimization method to find the optimal capacity combination globally. When optimizing capacity parameters, the maximum energy storage capacity is selected as the optimization objective. When optimizing the energy allocation strategy, the minimum total energy consumption is selected as the optimization objective to carry out double-layer optimization. And the simulation proves that the capacity configuration combination obtained by this method can effectively improve the energy storage capacity of the hybrid energy storage system and reduce the energy consumption of the system. This paper proposes an optimal energy allocation algorithm of Li-Battery/super capacitor hybrid energy storage system based on dynamic programming algorithm. The system structure of the hybrid energy storage system is selected according to the application scenarios of the fast charging station, and the dynamic planning model of the hybrid energy storage system is established. The optimization goal is to minimize the lithium battery life attenuation increment. Then the energy allocation scheme of the hybrid energy storage system with the least li-battery life attenuation is obtained.
The rest of the paper is organized as follows. Section 2 introduces Li-battery cycle life estimation model with irregular discharge. Dynamic programming algorithm is presented in Section 3. Then, Section 4 introduces simulation results based on MATLAB. A conclusion is presented in Section 5.  [10], the durability model based on the increment of battery internal resistance [11], and the durability model based on the growth mechanism of internal SEI [12], these models are based on the internal degradation excitation of the battery for model analysis and life prediction. The calculation process is cumbersome and cannot be compared with the life impact factors at the system level (such as the depth of discharge, charge and discharge rate, Temperature, etc.) to establish a quantitative relationship. This paper uses a semi-empirical model based on the Arrhenius degradation model [4], which is described in (1).
where Q loss represents the li-battery capacity loss, A the preexponential factor, B the discharge rate correction factor, C_Rate the charge-discharge rate, E a the activation energy, T en the absolute temperature, and R the gas constant of 8.314 J mol −1 K −1 . The parameters in the battery capacity loss estimation model A, E a , and z are obtained based on the empirical fitting of a large experimental data set. The Ah-throughput A h represents the amount of charge delivered by the battery during cycling. It is assumed that the semi-empirical model of equation (1) is suitable for the dynamic decay process. During the analysis, the discharge rate is regarded as a constant. Change equation (1) to (2).
where Q loss,p and Q loss,p+1 are the accumulated li-battery capacity loss during the previous p and p+1 respectively. ∆A h is the total charge in and out of the battery from p to p+1, and it is also called the total ampere-hours, which is described in (5).

Model with actual parameters
According to the obtained experimental data, the least square method is used to fit the parameters, and the dynamic battery attenuation model of the selected single lithium iron phosphate battery is obtained [2], which is described in (6) Under this semi-empirical model, the influence of discharge rate, depth of discharge, and ambient temperature on the life attenuation of li-batteries can be quantitatively analyzed. However, (7) can only calculate the capacity attenuation calculation at a fixed discharge rate, depth of discharge, and ambient temperature. If the parameters change during the period, the capacity loss increase cannot be calculated.
Put  (8) can quantitatively calculate the influence on the capacity loss of the li-battery. It is confirmed that the capacity attenuation increment of the li-battery in the pth sampling period is not only related to electric charge,charge and discharge rate, and ambient temperature in the pth period, but also to the cumulative capacity attenuation value of the previous p-1 period Q loss,p-1 . So the same charge and discharge process will have different effects on li-batteries with different initial capacity decay values, and the capacity decay increment will gradually slow down as the cumulative capacity decay value increases. In summary, the calculation formula for the capacity attenuation increment ∆Q loss,p of the li-battery is adjusted to (9).
When calculating the life of the li-battery cell of the hybrid energy storage system in the station, the initial state of charge is SOC 0 , and the whole day is divided into X sampling periods, and the li-battery power P HESS,bat(n) of X sampling periods is obtained through the energy distribution strategy. The power P bat (n) of the libattery cell in each sampling period, calculate the attenuation increment ∆Q loss,p of each sampling period, and perform cyclic calculation until the cumulative capacity loss reaches the limit Q loss_limit . At this time the cumulative working time is L HESS,bat (in years) is the life value of the li-battery in this state.

Dynamic programming algorithm
The low-frequency/high-frequency filtering algorithm smoothes the curve of li-battery power demand by filtering the power demand of the hybrid energy storage system, but this method cannot judge the current situation, and the super capacitor only acts on the shortterm after the load changes. The following will discuss the optimization goal of minimizing the increment in the life of the li-battery, and the dynamic programming algorithm is used to obtain the optimal energy distribution scheme under the fixed configuration parameters.
Dynamic programming is often used to solve problems with overlapping sub-problems and optimal sub-structures. The multi-level decision-making problem is converted to multiple single-level decision-making problems. The optimal solution is obtained from the final state merge sub-problems, and the calculation time is much less than naive algorithm [13][14][15][16].
The solution process of dynamic programming is mainly divided into the following steps: (1) Determine the objective function; (2) Select state variables and establish the state transition equation; (3) Establish initial conditions and boundary conditions; (4) Determine the solution sequence, whether to calculate from front to back or from back to front.

Determine the objective function
The goal is to obtain the energy allocation strategy of the minimum capacity attenuation increment of the libattery through dynamic programming. From the libattery cycle life estimation model in the previous section, the capacity attenuation increment ∆Q loss (k) of the kth sampling period accords to (10) .
In the process of calculating the increment of libattery life attenuation, the individual power of the libattery is taken into the calculation of P bat (k), ignoring the influence of the initial capacity loss limit. η d is set to 0.95, the ambient temperature is set to 15°C , and the relevant parameters of li-battery cell life attenuation are taken into, which is described in (11).
In the process of calculating the increment of libattery life attenuation, the individual power of the libattery is taken into the calculation of P bat (k), ignoring the influence of the initial capacity loss limit. η d is set to 0.95, the ambient temperature is set to 15°C , and the relevant parameters of li-battery cell life attenuation are taken into, which is described in (12).  (12) where P bat (k) is the average output power of the libattery cell in the kth optimized cycle. The goal of dynamic programming is shown in (13) , (14):

Establish the state transition equation
The state of charge of the supercapacitor is selected as the state variable. When the state of the supercapacitor changes, the power of the supercapacitor and the libattery can be calculated, as shown in (15 where SOC(k-1,i) is the supercapacitor SOC value of state i in the (k-1)th period, SOC(k,j) is the supercapacitor SOC value of state j in the kth period, P sc (k,j,i) is the energy released by the supercapacitor energy storage system from state i in the (k-1) th period to state j in the kth period, P bat (k,j,i) is the energy released by the li-battery cell in the process. Through state transition equation can get the power output status of the li-battery cell, and then calculate the increment of the li-battery capacity loss in this process.

Establish initial conditions and boundary conditions
Set the initial state : set the ambient temperature as 15°C, the initial battery loss value as 0.1, the initial SOC of the supercapacitor as 0.9, and the end state of the supercapacitor SOC the same as the initial state to avoid the unfairness caused by different end states. During the process, the SOC of the supercapacitor is within the fluctuation limit range.

Determine the solution sequence
Because the same charge and discharge process will have different effects on li-batteries with different initial capacity decay values, the solution sequence must be from front to back, which is shown in Fig.1. For the state j when k=3, traverse all the states when k=2, and calculate the increment in li-battery cell capacity loss and cumulative state loss caused by the transition from k=2 to state j when k=3 When calculating the optimal path to k=k max , traverse all states of k=k max -1, and calculate the increase of lithium battery cell capacity loss from different states when k=k max -1 to the end SOC state of k=k max Select the smallest accumulated loss value and record it as Q loss (k max ，1), and record the state i Of k= k max-1 as the optimal path to the end state at the time of k= k max .
Obtain the optimal path Select the minimum cumulative loss value to reach the state j of k=3 and record it as Q loss (3，j). And record the state i of k=2 at this time as the optimal path to the state j of k=3.

Fig. 1. Flowchart of the dynamic programming algorithm
At first, calculate the loss value increment and cumulative loss value of each state from the initial state to the next layer, then calculate the optimal path to each state of each layer, until (k max-1 )th layer. Finally, calculate the minimum cumulative loss of each state of the (k max-1 )th layer to the end state, and obtain the optimal path to the end state.

Simulation results
According to the load model of the fast charging stations and the sampling period of 1s, the plot of the load sampling point from 11 o'clock to 12 o'clock is shown in Fig.2. 11 o'clock to 12 o'clock is the time period when the load model changes most drastically during the day. The load is only considered for power supply by the hybrid energy storage system, ignoring grid power supply and photovoltaic power supply temporarily [17][18].
For the li-battery/supercapacitor hybrid energy storage system, compared to the underlying control algorithm, the energy allocation algorithm belongs to the upperlevel algorithm. The power distribution is performed according to the charging load demand and the state of charge of the supercapacitor at different times. It reduces the capacity decay of li-batteries by the energy allocation between li-batteries and supercapacity, thereby extending the life of li-batteries. The workflow of the energy allocation algorithm in the hybrid energy storage system is shown in Fig.3. Next, use 4000 li-batteryies and 120 supercapacitor cells as the optimal configuration combination to obtain the optimal energy allocation scheme through the dynamic programming algorithm. The simulation results of optimal energy allocation algorithm based on dynamic programming algorithm is shown in Fig.4. Observing the energy allocation of li-batteries and supercapacitors in Fig.4, the supercapacitor is responsible for supplementing or absorbing the power difference between the load and the stable value of the libatteries. After adding the super capacitor, the output waveform of the li-battery becomes significantly smoother than the load waveform, the calculated standard deviation is reduced from 336 to 330, and the standard deviation of the li-battery output is reduced by about 1.8%.
According to the principle of the low-pass/high-pass filtering algorithm in the section 3, the simulation of the filtering energy allocation scheme is carried out. When the T f is set to 25, the cumulative li-battery capacity attenuation increment is the smallest, and the allocation result waveform is shown in Fig.5.
The attenuation cost of li-battery refers to the replacement cost of the energy storage system due to performance degradation during operation [19][20][21], which is calculated as follows.
If the capacity loss of li-battery exceeds Q loss_limit , the battery array needs to be replaced, and calculate the number of replaced batteries n bat .   (20) The life of super capacitors and DC/DC converters is longer than that of lithium batteries, so the hybrid energy storage system rarely needs replacing super capacitors and DC/DC converters during its lifetime. Table 1 shows the the life decay of li-battery with different scheme. Based on no super capacitor shceme, the li-battery capacity attenuation reduction rate dynamic programing scheme is much higher than that of filter allocation scheme.In other words, the energy allocation algorithm can extend the life of the li-batteries. According to Table 1, dynamic programming scheme can minimum cumulative li-battery life attenuation increment, which can decrease replacement cost of the energy storage system and improve economic efficiency.

Conclusion
In this paper, an optimal energy allocation algorithm of li-battery/super capacitor hybrid energy storage system is proposed. The li-battery cycle life estimation model is established which is used in the objective function. Then according to the solution process of dynamic programming algorithm, to realize the proposed energy allocation algorithm. The simulation result validated that the he proposed energy allocation algorithm can extend the life of the li-batteries.