Optimisation for Coalitions Formation Considering the Fairness in Flexibility Market Participation

. This paper proposes a coalitional game-theoretical model for consumers’ flexibility coalition formation, supported by an optimization model based on differential evolution. Traditionally, the participation in conventional electricity markets used to be limited to large producers and consumers. The final end-users contract their energy supply with retailers, since due to the smaller quantity available for trading, they cannot participate in electricity market transactions. Nowadays, the growing concept of local electricity market brings many advantages to the end-users. The flexibility negotiation considering local areas is an important procedure for network operators and it is incorporating a local electricity market opportunity. A coalition formation model to facilitate small players participation in the flexibility market proposed by the network operator is addressed in this work. The inclusion of Shapley value in the proposed model enables finding the best coalition structures considering the fairness of the coalitions in addition to the potential income achieved by the consumers when selling their flexibility. An optimization model based on differential evolution is also proposed as the way to find the optimal coalition structures based on the multi-criteria specifications.


Introduction
Renewable energy sources' large-scale integration in power and energy systems has been heavily verified in recent years. Several changes in the planning and operation of the power systems have been introduced due the volatility of the distributed renewable sources generators (namely solar and wind sources). The network operators that are in charge of electrical network operation and planning should take advantage of the positive impact that the high penetration of renewable generators can make [1].
The transmission system operators (TSO) and distribution system operators (DSO) are the electrical network operators, for transmission networks and distributions networks respectively. The massive integration of renewable energy sources makes networks operators search for solutions for the new challenges. Consumption flexibility is seen as one of the most promising solutions to overcome the variability from the generation side and mitigate various of the problems that are emerging [2]. The coordination between TSO and DSO is needed in order to capture the great flexibility potential. The end consumers are considered power systems agents with great potential in provide flexibility. TSO was been used the flexibility provided by large utilities in order to adapt the consumption in a certain local to the needs of the electricity network [3]. The DSO is following the same approach but considering the end-users connected to distribution network, namely households, residential buildings, and some small industries [4].
Currently the negotiation of flexibility is a hot research topic and the actual application of flexibility trading uses prices incentives [5] or dynamic tariff [6] to incentive the potential flexibility providers to deliver their flexibility. Usually, the flexibility providers are rewarded considering its potential for flexibility delivered, so if the provider has a great potential in terms of volume it is rewarded in a better way. The participation of small end-users (households) is affected by large end-users, thus those that negotiate large volumes have greater influence in the decision.
The problem of coalitions in the power system area has been applied in order to organise groups in energy communities. In [7], a cooperative game theory model is used to create coalitional groups minimising the energy costs of the coalitions. Another relevant application is presented in [8] which considers the incentives by optimisation of energy storage system control for prosumer coalitions. The objective of optimisation is to get the highest monetary profits for the participants through collaborative operation of multiple energy storage systems. This paper proposes an optimisation problem for coalition formation for flexibility provision but considering the fairness in the coalition formation. The fairness is measured considering the standard deviation of the coalition Shapley values. In [9] the Shapley value is used in other way, to obtain a fair remuneration in demand response provision, the Shapley value is obtained considering the individual contribution of each prosumer. In this study the Shapley value is obtained considering different attributes for each prosumer. The optimisation process is performed considering an evolutionary algorithm hybrid-adaptive differential algorithm (HyDE). The results show a fairness coalitions formation with a maximisation of incomes for all considered agents.
The rest of this paper is structured as follows: Section 1 introduces the concept of flexibility markets and the necessities of flexibility for network operators, the coalition problem and the use of Shapely value is also present, Section 2 is presented the proposed methodology with the presentation of problem formulation and the method used for solve the problem. Section 3 the case studies and the results of the paper is presented and discussed. Section 4 concludes the paper and present some future works.

Proposed Methodology
This section presents the proposed methodology for the coalition formation in flexibility participation considering the fairness. Figure 1 present the overview of the problem.

Fig. 1. Coalition Formation Example
As can seen in figure 1, there is a distributed system operator (DSO) that makes a request for the end-users. The request that is made considers the flexibility in end-users, which is used by the DSO to avoid possible contingencies that can appear during the electrical network operation. For the problem resolution we consider that DSO and end-users are different agents, and within end-user agents we can have different types, e.g. households, commercial and residential buildings or industries. Each of these end-users agents has the ability to provide flexibility when the DSO makes a request, and DSO remunerates each end-users for the provision. Each end-user agent has different attributes that the DSO agent takes into account for the coalition formation. The attributes are: price desired, amount available, participation rate, location, facility type and comfort affect. The remuneration for each agent depends on the coalition in which is placed because the price for remuneration is different from coalition to coalition. The number of the coalition in total is equal to the number of agents, which means that each agent is a coalition. The proposed problem is to try to form coalitions in order to obtain a solution that is fairness in the distribution of incomes.

Mathematical Formulation
The present section shows the mathematical formulation of the proposed problem. In equation 1 is presented the objective function of the maximisation problem.
where, represents the incomes received by agent , std c represents the standard deviation of the remuneration received by agents belonging to a coalition , represents the number of agents and the number of coalitions. The term ∑ ( ) ∈ gives the total incomes of all agents in all coalitions , meaning that maximising this term will result in a maximal gain for all agents. The second term, i.e. ∑ std c ∈ provides the sum of standard deviation in relation to the distribution of remuneration based in the shapely values in each coalition ∈ . The negative sign is used to optimise the minimisation of such term, which will result in a fairness state for all possible coalitions within all agents. Equation 2 presents the incomes calculation for each agent .

= × , ∀ ∈
(2) where, represents the weighed average price for each coalition and represents the available flexibility of agent in a given coalition .
The weighed average price is calculated considering the importance of each agent has in the coalition. This importance is measured considering the available flexibility of each agent, equation 3 shows the weighed average price for each coalition.
where, the term ∑ ∈ gives the sum of the prices of agents in coalition ; ∑ ∈ represents the total flexibility available in coalition c. Equation 3 gives a price for each coalition. In equation 4 is presented the calculation of the standard deviation of each coalition.
where, represents the Shapley value of agent in coalition , represent the Shapley value average in coalition , and | | represent the cardinality of . The presented formula in equation 4 is the traditional formula of standard deviation calculation.
With the use of standard deviation in each coalition, present in equation 4, we try to obtain a metric for measure the stability of the coalitions. Considering a coalition game, the Shapley value divides playoffs among players according to equation 5.
where, is the total number of players and the sum extends over all subsets of not containing player . The Shapley value captures the "marginal contributions" of each agent , averaging over all the different sequences according to which the grand coalition cloud be built up. The value of characteristic function for each agent is given by the application of equation 6.
where, the represent the participation rate in last events of flexibility of agent is used for benefit the players that contribute regularly to this type of programs, and thus incentivize players participation. represent the importance of agent location in the network i.e. if the location of the customer is more or less beneficial to the system (if the reduction of consumption in the specific location contributes to effective power flow).
represent the facility classification of agent and the represent the effort placed by the customer in the flexibility provision, measures by the relative amount of flexibility provides in relation to the total amount of consumption of the player, in order to reward players that make bigger efforts (provide a larger percent-age of relative flexibility). The values of ′ represent a wight for each attribute present in the equation 6. Equation 7 present the coalition formation constraint.
where, ∑ ∈ represent the sum of the flexibility available in coalition and ∑ is the available flexibility sum of all agents. With this constraint we force coalitions to provide at least 1 4 of the flexibility available on all agents. This constraint is considered because in order to keep coalitions balanced in terms of total available flexibility. This is important for the DSO for management and operation reasons, in order to assure that a relevant amount of flexibility is available from all coalitions.

Hybrid-adaptive Differential Evolution
Optimal coalition formation is known to be an NP-complete problem [10]. Thus, we advocate the use of evolutionary computation to find near-optimal coalition structures in acceptable times. In particular, we apply a hybrid-adaptive differential evolution with a decay function (HyDE-DF) algorithm recently proposed in [11]. HyDE-DF is a new variation of the well-known differential evolution (DE) algorithm [12], and has proven an excelled performance solving a variety of problems ranging from benchmark function optimisation to smart grid applications.
As . HyDE-DF follows the general iterative process from of evolutionary algorithms, namely the creating of new solutions by means of mutation and recombination operators, and performing elitist selection (e.g., solutions with superior performance survive into the next generation).
HyDE-DF uses a mutation operator known as "DE/target-to-perturbed-best/1" that modifies the well-known DE/target-to-best/1 strategy [12] with a perturbation of the best individual (inspired by the EPSO [13]), and the self-adaptive mechanism of jDE [14]. The main operator of HyDE-DF is defined as follows: where 1 and 2 , are scale factors in the range [0,1] independent for each individual , and = ( 3 , 1) is a random perturbation factor taken from a normal distribution with mean 3 and standard deviation 1. 1 , 2 and 3 are updated each iteration following the same rule of jDE algorithm (see Sect. III.B of [15]). HyDE-DF incorporates a decay function to perform a transition in the iteration process from the main operator of HyDE-DF (Eq. 8) to the basic operator of DE. This transition allows an enhance phase of exploration in the early stage of evolution and stress the exploitation in later stages of the optimisation [11].
After creating the mutant vector, a recombination operator combines the mutant individual ⃗⃗⃗ , with the target vector ⃗ , giving place to a new trial ⃗ , : Finally, a simple rule of elitist selection is applied comparing the fitness between the trial vector ⃗ , , and the target vector ⃗ , , in the objective function: where , +1 is the population of the next generation, that changes by accepting or rejecting new individuals, and (. ) is the fitness function used to measure the performance of an individual (i.e., Eq. (1)).
After the description of the algorithm, HyDE-DF can be applied easily by defining an encoding of solutions (typically as vectors or a numerical string) and a fitness function to evaluate such solutions. Thus, to capture all the information required by the encoding of a solution in our problem, we define a vector: where { 1 , … , } ∈ ⃗ are variables used to represent the coalition structures Ω Agg . The encoding of a coalition structure follows a simple label-based clustering encoding [16], in which the position of the element indicates the player ∈ , while the value in the position represents the coalition that the player belongs to. This particular encoding was designed to operate over integer values. However, since HyDE-DF operates over real values, we define the bounds of variables in the range [1, + 1], being the maximum number of coalitions allowed. Then, for the decoding process, we use the floor function, which maps the real values to integers in the range [1, ].

Fig. 2. Coalition Formation Example
After the decoding process, solutions can be evaluated in the formulation provided in Section 2.1.

Numerical Simulations
This section is divided into two different subsections: firstly, it is presented the case study of the problem and all scenarios tested. Secondly, the results of the proposed methodology applied to the case study are shown.

Case study
The case study is composed by one DSO agent, and eleven end-user agents, as can seen by table 1, which shows the attributes for all end-users agents.  In this case study the end-user agents are classified into two different categories, the households agents (Agent 1 to Agent 9) and residential buildings agents (Agent 10 and Agent 11). Considering table 1 it is possible to characterise all end-users agents that participate in the coalition formation. By the attribute flexibility, measured in KWh, it is possible to see that the Agents 10 and 11 have a large flexibility amount to provide, and it is expected by the formula of equation 3 the agents with more capacity to provide flexibility have bigger influence on coalition price determination. Considering the column of price attribute, measured in EUR/kWh, the tendency is that the agents with more flexibility is expected that desired a bigger price. In the participation attribute the agents Agent 10 and 11 are the most participating agents in the last DSO agent request. This fact can be justified as this agents is a residential building and possibly there is a building manager who is in charge of participating in the request. The location attribute, , represents the relative distance in electrical network of all agents related to the connection on the main electrical network. Agent 10 and 2 are the highest ranked agents in this category. The facility type attribute is used to benefit the agents that have the ability to provide less flexibility. In this case the Agent 7, 8 and 9 have the highest classification. For the comfort affect attribute the Agent 10 and 11 is the agents with the highest classification, which means that is the agents that are able to reduce more consumption compared to their overall consumption. Table 2 specifies the weights attributes for the different considered scenarios.  The values of Table 2 are used for calculating the characteristic function in equation 6. As can seen by the same table there are eight scenarios. In total we will perform eight different simulations. The sum of all weights in all scenario is equal to 1. For scenario 1 the weight is the same in all attributes. The other seven scenarios are used to understand the weight each attribute will have on the formation of coalitions. In scenarios 2 and 3 it is given the maximum importance to the flexibility and price respectively. The scenario number 4 presents a division of the importance between 1 and 2 , but always considering the other attributes with smaller importance. The scenario 5 is a balance scenario similar to baseline, although the 1 and 2 have a little higher importance. The scenario 6 is very similar to scenario 1 but the 3 to 6 have a higher minimal value difference compared to 1 and 2 . Scenario 7 consider a higher value of 4 , which means that the location attribute is the most important. The facility type attribute has the higher importance on scenario 8 with the 6 as a higher weight value.
Regarding the algorithm setting for HyDE-DF † , the parameters were chosen according to other studies [17]. In fact, HyDE-DF is a self-adaptive parameter version but initial values for and where set to 0.5. The size of population was set to = 10 and 500 iteration were selected in each experiment.

Results
This subsection presents the results. Considering the behaviour of the metaheuristics we perform 30 trials for each scenario presented in 2 in order to obtain statistical significance in the results. Table 3 presents the results for objective function presented in equation 1  considering all scenarios presented in table 2. Table 3. Optimization results.  As can be seen, in table 4 the results for the number of members of each coalition number is always 2. There are differences in the results as can be seen by the coalition structures presented. In a baseline where the std c is not included in the objective function, the optimal solution considers one coalition with Agents 1 and 11, and the other coalition aggregates the rest of the agents. The coalitions structures of scenario 1, 5, 6, 7 and 8 are the same, the Agent 10 constitutes one single coalition and the others all agents are the other coalition. In scenario 2 and 4 the coalition structures results are equal, the Agent 11 is one single coalition and the other agents constitute the other coalition. In scenario 3 there is a coalition with Agent 3 and 11 and the rest of the agents is the other coalition. With the results of the table 4 it is possible to observe that the Agent 10 and 11 have strong impact on the coalition results, due the constraint of equation 7. Therefore, the agents with lower flexibility quantity (Agent 1 to 9) have to merge with one of other two agents. By the results of the scenarios when is given more importance (greater attribute value ) to the flexibility ( ) or price ( ) the coalition present differences in the structures (scenario 2, 3 and 4). Table 5 presents the incomes for all agents grouped by the coalition structures present in table 4. As can seen by table 5 the sum of all incomes is the same in all scenarios, but the value of remuneration per each agent is different when the coalition structure changes. When comparing scenario 1 when the is the same for all attributes with scenario 2 when only considering the attribute the Agent 11 that have the higher value of is alone in one coalition and the other are together, but the Agent 11 receives more incomes compared with the incomes received in scenario 1. Considering now the scenario 3 when considering a value of 2 = 1, which means only the price attribute is considered for Shapley value calculation, the agents with higher price (Agent 3 and 11) are in one coalition and both receive higher income values when compared with baseline and scenario 1 when the 's are the same for all attributes.

Max
The incomes distribution in scenario 4 is the same of scenario 2, although the 's have different values, the coalition structures is the same. This is possible because the price of flexibility for each coalition depends on the flexibility and price attributes ( , ) of each agent. The incomes of scenarios 1, 5, 6, 7 and 8 are equal, due to the same reason explained above. Table 6 presents the results values for each of each scenario, the std c for each coalition and the ∑ std c ∈ . The values of Shapley are different for each agent and for each scenario. The standard deviation for coalition (std c ) with only one agent is 0. Scenario 8 has the higher sum of std c , which represents a great difference in agents Shapley value.

Conclusions
This paper proposes an optimisation methodology for coalition formation considering the fairness in flexibility market participation. The fairness of the coalition is measured taking into account the standard deviation of Shapley values within each coalition. The optimization problem is solved considering the Hybrid-adaptive Differential Evolution algorithm. With the analyses of the presented results we can see the influence of the attributes in coalition formation. The scenarios 1, 2, and 3 definition have a positive impact for results analyses, because the differences are visible, and the impact of attributes is easy to identify. The definition of the other scenarios could not draw significant conclusions. As future work we intend to explore measures of coalitions stability (e.g., Core) and compare with the results of this paper.