Multi-agent optimization in solving the problem of optimal power system control and reliability assessment

. The paper proposes a method for distributed optimal control of EPS with the presence of stochastic elements. The proposed technique is based on distributed optimization methods. The forecast of electric mode parameters is used as input data for optimization. The forecast horizon is divided into separate time steps at equal intervals. Distributed optimization methods are extended to the dynamic optimization problem, when not just independent states for each moment of time, but the process of their change in time is considered.


Introduction
This paper is a continuation of the work on automated optimal control of power system [1].In the software complex for optimal control of voltage and reactive power in the automatic control cycle developed in ISEM SB RAS, forecasting and dynamic optimization of electrical modes are performed.The control algorithm is realized according to the hierarchical principle.Data from all substations are collected in a single control center, where state estimation and forecasting are automatically performed, based on which the optimal control actions and the time of their application are calculated.
The electric power system (EPS) is a complex dynamic object, the control of which requires the use of complex automatic systems.The developed hierarchical system of organization of automatic optimal control of EPS can be improved, if only because fully centralized control becomes ineffective due to the presence of large flows of information and significant time costs for its transfer to the center to calculate the control action.This issue is especially relevant in a power system with a significant number of sources of small generation or active load.The peculiarity of such systems is the stochastic nature of both load and generation.Optimal control in systems with a large share of uncertainty becomes even more complex, and hierarchical control, when it is required in a single center to quickly collect a model to optimize the entire network may no longer be appropriate.Multi-agent control can be a possible variant of optimal control system realization under the condition of stochasticity of EPS parameters.
The multi-agent approach is based on the concept of a mobile software agent, which is implemented and functions as an independent specialized computer program or an element of artificial intelligence.The paper proposes a method of distributed optimal control of EPS with the presence of stochastic elements.The proposed control system is aimed at application in systems with small and distributed generation, where the introduction of electricity storage systems is required, so it is important to take into account the fact that it is not always possible to organize a centralized optimal control system.Thus, when developing the model, the possibility of building a distributed multi-agent system is investigated at each stage.

Proposed solution
The proposed technique is based on distributed optimization methods [2,3].As input data for optimization, the forecast of changes in the regime parameters for a given time is used.The forecast horizon is divided into separate time steps at equal intervals.The global target function of the distributed optimization problem can be formulated as the sum of target functions  for subsystems represented as individual computational agents.
Each of the functions  is known to only one agent.The admissible region for the values of the control vector  must take into account the constraints of each agent.Therefore, it is defined as the intersection of the corresponding admissible regions.
The agents are connected by a directed graph  〈, 〉 representing the available communication channels.A node  ∈  corresponds to an agent that computes a function  .Each edge ,  ∈  corresponds to an element  of the incident matrix of the graph .
The solution of the distributed optimization problem by the subgradient projection method for the case of convex functions  • is presented in [4].At each  step at node , the value of the vector of control actions is refined as follows: where  is the projection of the computed value of the control vector at node  onto its admissible area;   is the coefficient of the incident matrix;   -list of neighboring nodes for the i-th node;   -value of the vector of control actions of the neighboring node j;   -subgradient of the local target function   ;  -step size at the calculation step k.
To ensure convergence of the distributed optimization problem, a necessary condition is that the switching graph is balanced, i.e.,   0 and ∑   ∈ 1.When agents interact in the process of power system control, delays in information transmission are inevitable.Data received from neighboring control nodes may be outdated.In the presence of communication delays, the balance and symmetry of communication graphs will be broken, which makes algorithms that do not take into account such delays inapplicable.The loss or corruption of individual data can also be expressed in the form of data transmission delays.To solve the problem taking into account possible delays in information transmission, the following algorithm for computing the vector of control actions is proposed in [3]: The idea behind the algorithm is to use each agent's own state and information having a delay of size  to balance the impact of communication delays assuming an initially balanced graph.The coefficient  is chosen such that the condition min ∈      is satisfied.
If we consider the problem of optimal control in an integral formulation, it is transformed into a dynamic optimization problem, where, unlike static optimization of a single mode, it is important to take into account the "cost" of control actions, depending not only on the system state vector, but also on the time of their implementation.
The cost of controlling one or another equipment depends on such factors as: the residual resource of the equipment; the priority of using the control action; the minimum permissible time between commutations by the same device.The optimization problem taking into account the cost of control action is written as: where  is the control actions available at time t;  -static optimization function of each single state for time t;  -monotonically decreasing function of the cost of the control action  , depending on the time of the actions that have been performed up to time t.
In the multi-agent formulation, the dynamic optimization problem is written as:   ,    ,  .Since time can be represented as discrete samples, each representing the time of data acquisition by one of the agents,  ,  can be rewritten as: Having the sum of agents' target functions by time steps, the dynamic optimization problem is reduced to the problem (1), for the solution of which we use the subgradient method taking into account the information transmission delays (4).In this case, the communication graph  is a union of communication graphs  of the multi-agent interaction for each time step, supplemented by links between the corresponding agents at different time steps.In Fig. 1, these links are represented by dashed lines.
The component of the cost of the control action  that depends on the time of its application can be represented in the form of information delay .However, it will already be time dependent.
The multi-agent dynamic optimization cycle uses multiple static optimization calculations.Therefore, they require fast electrical optimization software, which can be embedded in an external software package in the form of a library.
The paper presents such a library.The main optimization method implemented in it is the interior point method.The calculation of the Hessian in the optimization cycle is the most costly procedure, which must be performed at each iteration.To reduce the computation time, the Hessian is replaced by its approximation, refined iteratively according to the LBFGS algorithm.The use of the quasi-Newtonian algorithm speeds up the optimization calculation for each of the modes in the dynamic optimization process.
The interior point method [5 -8] can be applied to solve the optimization problem both in the formulation (3 -5) and in the formulation (7,4,5).
In general, the optimization problem for each time instant can be written as: This problem is solved by Newton's method, where a linearized system of equations is solved at each step: where Hessian  ∇   ∑  ∇ ℎ  ∑  ∇   .
,    , Λ   , Λ   ,  -unit matrix.The calculation of the Hessian is the most costly procedure, which must be performed at each iteration.To reduce the computation time, the Hessian is replaced by its approximation, refined iteratively according to the LBFGS algorithm [9 -14].
, where  1/  и      ,  is the change in the argument vector at step k:   ,  ,  ,   ,  ,  ,   is the change in the Lagrangian gradient.
The use of the quasi-Newton algorithm allows us to speed up the optimization calculation for each of the time steep in the dynamic optimization process.The optimization problems to be solved are actually not quite independent of each other.Indeed, the change of electrical parameters in time does not occur in a jumplike manner.Additional time speeding up of the calculation time is possible due to the use of the vector of control actions obtained as a result of optimization of the state for the previous moment of time as a starting point of optimization.At the same time, it is necessary to control the proximity of the previous state.For example, during switching operations, significant changes in the state may occur and the application of past optimal actions will not be effective.The proximity of the steady state can be estimated by the total deviation of active power flows on branches: .
The value of the relative value of ρ is determined for each power system individually.

Implementation
The presented methodology is implemented as a C++ library and included in the ANARES software package.At the same time, an interface in Python language is provided, which allows using this library both in commercial solutions and in research projects.For use in research projects, the library is planned to be made publicly available in 2023 The proposed optimization algorithm was compared with the existing ones in the ANARES complex (gradient descent and LBFGSB).The calculation was performed on the Irkutsk power system model.The considered model of the power system includes power grids of the Irkutsk region with voltage of 500 kV -110 kV.Generators in the model are mainly specified by nodes of the voltage at which the corresponding generators actually operate (6 kV -15 kV).Some generation is represented as equivalent generators on high or medium voltage busbars.The load is mainly specified on the low side of transformers or on the tapping side of transmission lines.There are 486 generator nodes in the model and 485 load nodes.The total number of nodes in the node/branch model for this system is 1248, with 1481 branches.The calculations were performed on a workstation with the following parameters: Platform based on Intel Xeon W-2125 4 GHz processor with 32 GB DCU.
The transformation coefficients, as well as the change of reactive power generation of stations and synchronous compensators were considered as control parameters.The calculation was performed on the archive data obtained from the ANARES state estimation subsystem.
In a series of calculations from different initial conditions corresponding to different states from the archive, in comparison with the quasi-Newtonian algorithm LBFGSB showed the acceleration of calculations in the amount from 28% to 55%.These figures will also include data input from the model.On average, the optimization time per mode ranged from 212 ms to 437 ms.Whereas, in the LBFGSB implementation, the computation time ranged from 296 ms to 983 ms.

Conclusion
The presented optimization methodology was implemented in the research version of the ANARES software-computing complex.The calculations have shown high speed of the proposed methods with provision of the necessary accuracy of calculations.
The algorithm performance has been tested on the data of a real sufficiently large power system.The library for optimization of EPS modes with reformulated target function can be used in the cycle of mode reliability analysis.

Fig. 1 .
Fig. 1.Communication graph of dynamic multi-agent optimization.1 -vertex corresponding to the agent at different moments of time; 2 -the edge corresponding to the relationship between different times of the same agent; 3 -the edge corresponding to the connection between agents.