Bidding strategy of pumped storage power station in spot market considering different optimization periods

. Pumped storage power station has multiple functions, such as alleviating the contradiction between peak and valley, to ensure the safe and economic operation of power grid. In the non market stage, pumped storage power stations mainly obey the system operator’s scheduling. In the market stage, pumped storage power stations in China are likely to participate in the competitive power market and provide peak power, frequency regualtion and recovery services. The paper studies the bidding strategies of the pumped storage power stations participating in the power market, and provides decision support for the pumped storage power stations to maximize market returns..


Introduction
Since the launch of a new round of power system reform in 2015, China's electricity market construction has been rapidly promoted in all provinces. In August 2017, the National Development and Reform Commission and the State Energy Administration issued the notice on carrying out the electric power spot pilot market construction, selecting eight regions in the South (starting from Guangdong), Mengxi, Zhejiang, Shanxi, Shandong, Fujian, Sichuan and Gansu as the first batch of electric power spot pilot markets. In the market stage, the market price is uncertain, so are the benefits and risks obtained by each subject participating in the market. Pumped storage power stations need to establish a set of bidding decision-making system to adapt the market environment, and ensure the optimal economic arrangement of production within the feasible range.
So far, scholars at home and abroad have done a lot of research on the bidding strategy of the power generation competitors. In Literature [1], evolutionary game theory is applied to the bidding strategy of generators, so that a stable optimal bidding strategy can be obtained through dynamic evolution in uncertain environment. In the power market, Pumped storage power stations in the electric energy market can track the market electricity prices in real time, and rationally arrange the period of pumping power purchase and water discharge power generation. Not only can they obtain huge electrical energy benefits [2] , but also have a huge impact on maintaining the stability of the grid system, the smooth operation of power plants, and energy saving positive externality [3][4][5] .Literature [6] establishs a secondary optimization model for pumped storage power station, first determine the operation plan of the pumped storage power station based on the prediction of the future electricity price of the electricity market, further consider the storage capacity constraints, and adjust the operation plan. This model can optimize different time The periodic operation plan of the pumped storage power station has good adaptability, but it does not consider the impact of the pumped storage power plant's behavior on the market. Literature [7] establishes a multi-input decision factor model that can simulate the bidding behavior of power generators in the previous market, applies Ver-learning reinforcement learning algorithm, and conducts overall design from the three dimensions of power trading operation simulation, market member decision behavior simulation and grid operation simulation Simulate the bidding behavior of market members on a 5-node test system. This method can effectively simulate the bidding behavior of power generators in the previous market, which is conducive to the rapid establishment of the initial mechanism and rules of the electricity market. Literature [8] gets the best price for power generation enterprises in the dark auction model. The results show that the bid price is positively correlated with the average price, the unit cost of power generation and the bidding electricity, and the optimal price is negatively correlated with the risk aversion factor. Literature [9] introduces current research of pumped storage participating in energy market, and then establishes a quadratic optimization model of pumped storage when participating in energy market． At present, only 15% of pumped storage power stations in the world participate in the market, one of the important reasons is that the benefits of pumped storage participating in the market are difficult to cover the investment and operation costs. Therefore, in order to maximize the economy of pumped storage palnts, it is an urgent task to design reasonably the bidding strategy model for pumped storage stations and get the optiaml bidding strategy with accuracy and applicability.

Bidding Strategy Analysis for Pumped Storage Power Station
In China, the electricity market is divided into "medium and long-term market" and "spot market" based on the idea of "risk aversion in medium and long-term trading and price discovery in spot trading". China's electric power market construction follows the law of scale from small to large, trading varieties gradually increasing, trading mechanism gradually improving, and market model continuously optimizing. Different market mechanisms will affect generators' bidding behaviors. Due to space limitation, the paper only studies the bidding strategy of pumped storage power stations in the spot market of electric energy.
There are five common bidding strategies for generators: (1) Bid based on the marginal cost The strategy is to quote the marginal generation cost of the generating unit. The increased production cost of unit is called marginal cost. When the cost of generate unit is linear, its marginal cost equals to the variable cost. When the cost of generate unit is non-linear, its marginal cost increases with the power output. The marginal cost is independent from the fixed cost, so the marginal cost can be used as the basis for generaters to quote.
(2) Bid based on "cost+revenue" Bidding based on cost plus revenue is the most popular strategy for power generation enterprises to participate in the market. The core idea is to define the marginal cost of power generation unit first, and then add a certain profit margin to form its quotation.
(3) Bid based on forecasting market clearing price The strategy is to refer to the prediction of the future market clearing price as the quotation. If the predicted market clearing price is higher than the unit's own cost, the bidding price only needs to be slightly lower than the market clearing price, which can guarantee the bid winning rate and certain incremental income for generators. (

4) Bid strategy based on prediction of competitors' behaviors
The strategy is to estimate the bidding behaviors of other competitors firstly. On this basis, the quotation is defined. Electricity market competition belongs to incomplete information competition. Each party has limited information, so it is unable to accurately predict the quotations of other market entities. And for some power generation enterprises with small market power, they do not need to spend a lot of time to pursue highprecision clearing price prediction. They can predict the bidding behavior of several competitors with large market power, combining their own costs, and form the quotation bidding.
(5) Bid based on reinforcement learning algorithm With the development of computer technology, artificial intelligence, machine learning and other methods provide an effective solution for the complex problem of power generators bidding in the electricity market. The strategy's core idea is to "learn" in the bidding process, and power generators will adjust their bidding behaviors under the incentive of return.
These are the bidding strategies commonly used by power generators in the power market. The first three ones are static bidding strategies, which mainly consider cost recovery. The third one and the fourth one belong to dynamic bidding strategy. The third one considers the game behavior of market competitors. The fourth one, with the help of modern intelligent learning algorithm, imitates the mechanism of biological "natural selection", uses the learning and evolution characteristics of the algorithm itself, can theoretically calculate better bidding results.

Bidding model of pumped storage power station considering different optimization periods
In this section, reinforcement learning algorithms are used to simulate the competitive behaviors of pumped storage stations participating in the electricity market. As the operation of pumped storage station is divided into two working conditions: pumping and power generation. From pumping to full storage, and then from full storage to empty storage, is a cycle of working conditions. If the cycle lasts one day and night, the pump station is called daily adjusted station. If it lasts one week, it is called weekly adjusted station. If the cycle lasts longer, it can be called seasonally adjusted station. Therefore, this section compares the influences that different optimization periods do on bidding strategies for pumped storage station with a certain storage capacity.
Before the market bidding, the pumped storage station predicts the future market price, and then makes preliminary arrangements for the future operation plan based on its own pump-discharge technical characteristics. Due to the limited space, this article does not elaborate on how to make the self-dispatching plan of pumped storage. On the basis of the self-dispatching plan made by the pumped storage station, optimize the bidding strategy of the pumped storage station by the method of reinforcement learning algorithms. Power generators, including pumped storage stations, are regarded as agents with adaptive learning capabilities. They can continuously adjust their quotations according to their own bidding strategies during the dynamic interaction between their actions and the environment, and ultimately obtain or approach an action strategy that maximizes the return.
Pumped storage stations will adjust their own price strategies under the reward (positive and negative) incentives. When a pumped storage station is awarded the positive incentive, the adjustment coefficient is greater than 1, the quotation will be appropriately increased; when the pumped energy storage is not successful, the negative incentive is obtained, and the adjustment coefficient is less than 1, the quotation will be appropriately reduced. The quotation adjustment formula is as follows: Where, It can be seen from equation (3) that the cost per electricity of pumped storage stations is related to the cost of purchasing electricity and the actual energy storage. Therefore, the marginal cost of the pumped storage station is zero before the pumped storage purchases electricity, and it changes continuously with each trading round of the electricity market.
The overall process of bidding strategy optimization is shown in Fig 1.

Data Preparation
Quote the market's highest limit price Modify strategy table and quotes based on trading results

Example calculation
Market conditions simulated in the experiment: there are 20 thermal power plants in the market, among which 8 power plants adopt the "marginal cost+20% return" fixed quotation strategy, and the remaining 12 power plants adopt a combination of "marginal cost+10% return" and reinforcement learning bidding strategy. There are two conventional hydropower, two wind powers and two photovoltaic power plants, and one nuclear power plant. All of them adopt a quotation strategy based on the marginal cost. The energy storage capacity of the pumped storage power station in this experiment is 10460MWh, the installed capacity is 1800MW, the overall efficiency is 0.75, and the initial energy storage is 0. The transaction is set in spot market of electrical energy, cleared every hour, and settled using a coincident marginal clearing price. Firstly, forecast the future market price. The daily optimization is to predict the price of the next day to arrange the future day's operation plan. The weekly optimization is to predict the price of the next week to arrange the next week's operation plan. From the actual prediction results, the accuracy rate of the daily price prediction curve is better and closer to the real-time price curve, while the error between the weekly prediction curve and the actual price curve is gradually increasing, as shown in  With the help of predicted prices in Fig 2, the daily operation optimization and the weekly operation optimization of the pumped storage power station can be calculated out, which are shown as followed.
(1)The daily optimized operating condition and the weekly optimized operating condition of the pumped storage power station are shown in Fig 3 and Fig.3 and Fig.4, the line segment of the operating curve less than 0 represents pumping, and the line segment of·the operating curve greater than 0 represents power generation.
(2)The daily optimized bidding price curve and the weekly optimized bidding price curve of the pumped storage power station are shown in Fig 5 and   It can be seen from the bidding price curve curve that the bidding strategies brought by different operating conditions are also different.
(3)Benefit comparsion with two optimization periods of the pumped storage power station.
The benefit of daily optimized operating condition is ￥ 6987475.9, while the benefit of weekly optimized operating condition is ￥10531628.2. It can be found that the benefit of daily optimized operating condition is higher than that of weekly optimized operating condition.
In summary, the pumped storage power station with the storage capacity of 15000MWh should choose the weekly optimized operation, which can bring about more objective economic benefits.

Conclusions
With the further promotion of China's electricity marketization, it is more and more important to explore how pumped storage power stations can develop economically and healthily in the market environment. The paper studies the bidding strategy model based on different optimization periods, which can solve the problem of how to choose a more reasonable optimization period and bidding strategy to achieve the optimal economic benefit for a pumped storage power station with a specific storage capacity. Through the analysis of calculation examples, the following conclusions are drawn: choosing a longer optimization period can achieve global optimization, but also faces the problem of slightly lower accuracy of long-term price prediction in the future. Choosing a shorter optimization period can not achieve global optimization, which may bring about the problem of profit loss. The bid strategy model in this paper has good operability. However, due to the lack of data and space limitations, the scientificity and adaptability of the model still need to be tested further.