Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

The transition towards an electricity production based on renewable energy, challenges the electricity infrastructure. Maintaining a balance between supply and demand is crucial, and fluctuations from electricity production must be handled. To this end, consumer participation in Demand Response (DR) programs offers a cheap and carbon friendly solution [3]. In DR programs, consumers are offered incentive, in the form of variable electricity prices or direct payments, to change their consumption pattern [7]. To overcome capacity constraints on DR markets, the coordinated actions of multiple consumers can be exposed through a Virtual Power Plant (VPP) [9].

Literature contain numerous proposals for VPP designs. In general, these approaches can be categorized either as centralized- or distributed approaches. Centralized approaches such as the ones proposed by Ruiz et al. [11], Binding et al. [1] and Molderink et al. [8] rely on direct load control of consumers, which scales poorly for larger virtual power plants. Further, this deprives consumers of their autonomy. To overcome these limitations, distributed approaches coordinate consumers represented as autonomous agents. Kok et al. [5], Kulasekara et al. [6], Zhang et al. [13] and Ramchurn et al. [10] contains examples of such approaches. But to our knowledge, existing literature does not account for the fact, that consumers are often complex entities, with several objectives negotiating over several issues. Under these circumstances, several local objectives must be pursued by each consumer while adhering to global requirements presented by DR events.

To this end, we propose a multi-agent based VPP design, which uses a novel inter-agent multi-objective multi-issue (MOMI) negotiation mechanism to achieve a group objective among self-interested agents. Agents represent complex consumer domains, and the group objective is defined by DR events received by a VPP entity. The design has been implemented in a software application used to simulate a VPP containing three greenhouses, which are coordinated towards various DR events. The results show that the electricity consumption of the greenhouses can be coordinated in accordance with received DR events, without violating their production requirements, when sufficient flexibility is available in the greenhouses.

The remainder of the paper is organized as follows. In Sect. 2 we present the proposed multi-agent based VPP design. This is followed by a description of a VPP scenario in Sect. 3. Section 4 describes experimental results and finally Sect. 5 draws conclusions.

2 Model for an Multi-agent Based VPP

We propose a multi-agent based VPP design, in which the complex nature of consumer domains is recognized. The design consists of an Aggregator-agent who negotiates with a number of Load-agents. The purpose of the negotiation is to agree on an electricity profile for each of the Load-agents. The electricity profile describes planned consumption (or, production) for a consumer domain across a VPP-specific period of time. When combined, the electricity profiles of all consumer domains should adhere to requests for DR services received as DR events. In this context, the Aggregator-agent is a coordinating entity that receives DR events, and the Load-agents represent consumer domains. The Aggregator-agent may form bilateral negotiations with Load-agents. When this happens, the Load-agent will provide an electricity demand profile to the Aggregator-agent. The Aggregator-agent initiates negotiation with the Load-agents by calculating electricity allocation profiles. These form a combined electricity profile that adheres to the received DR event. Further, the allocations for each Load-agent is as close to the electricity demand profiles as possible. The Aggregator-agent will then inform the Load-agents of the suggestions for electricity allocation profiles, and Load-agents will respond with a revised electricity demand profile. This process is repeated until the Aggregator-agent and Load-agents reach consensus on allocation - or the negotiation reaches a specified time-limit.

The agent decision logic is modeled as a multi-objective, multi-issue (MOMI) negotiation using a genetic algorithm (GA). Agents may have several objectives and address several issues, and the GA will query each objective with suggestions for issue values. This can be perceived as a mediator-based negotiation, where the GA assumes a man-in-the-middle position towards a set of objectives, represented by concerns, who negotiate over a set of issues. The mediator proposes values for issues, in the form of contracts, to the concerns. Each concern defines a cost function, which will yield a cost as a function of a contract and a target vector. This cost describes the degree to which a proposed contract adheres to the objective represented by the concern, at the time of the proposal. The use of cost functions means that concerns will seek to reduce the distance between a contract and its target vector. Here, the concern may select a subset of values from the contract and manipulate them, in order to make them comparable to its target vector. This means that the mediator has incomplete information: The mediator has knowledge of the concerns and - after querying - the cost they associate with a specific contract. It does not have knowledge of the cost function of the concern or the value of its target vector. The use of a GA means that contract proposals will initially be random within limits defined by the issues. The mediator will create a vector for each contract containing costs for each concern. Using the Pareto criteria, the mediator then creates and evolves a Pareto set of contract proposals based on their cost vectors. Evolution happens through genetic operations (crossover and mutation) on the contracts. The evolution of the Pareto set continues, until a time-limit, or a given number of evolutions, is reached. This concludes local optimization and enables the mediator to select a contract from the Pareto set, in order to provide a proposal in the inter-agent negotiation. To this end, mediators use concern’s priority: A concern’s priority is given by a value of either 0, 1 or 2. The lower the value, the higher the priority. These priorities are used to classify mission critical Domain Concerns (DCs) in priority 0, Representation Concerns (RCs) in priority 1, and non-mission critical DCs in priority 2. DCs are used by the agent to represent local objectives where RCs represent objectives of agents, with which it is negotiating. The concept is illustrated in Fig. 1.

Fig. 1.
figure 1

Concept of bilateral negotiation between agents in agent decision logic

Full size image

A selection criteria based on maximizing social benefit [4] is then used recursively on the priority groups to select the best contract. This means that all contracts from priority 0 that yield the same social benefit, are evaluated for all concerns in priority 1 and so on. The priority based selection scheme will guarantee the selection of a contract, that satisfies concerns with a higher priority before it considers concerns with lower priority. After selecting a contract, a mediator will update target vectors of all RCs representing its agent. This initiates the next round of inter-agent negotiation, and the process continues until a final number of negotiation rounds is reached or the allocated time for inter-agent negotiation expires. This design enures, that agents may mutually influence each other without violating mission-critical, local objectives. This design enables a differentiation between DCs and RCs: The mediator can prioritize local objectives over objectives, which represent external agents. This is necessary to avoid a self-enforcing feedback loop between an opponent agent and its RCs, during inter-agent negotiation. In agent terminology, this ensures that agents act truthful, and not based on external bias. Once the inter-agent negotiation terminates, the Pareto set in each agent contains one or more contracts, with maximum social benefit for both mission-critical DCs and RCs, iff there are no conflicts between the Aggregator-agent and Load-agents. However, in case of a conflict, no such contract exist. To solve this conflict, a choice can be made to prioritize the Aggregator-agent, in order to guarantee compliance with DR events, or to prioritize Load-agents, to ensure successful operation of consumer domains (or compromises in between these extremes). This topic is outside the scope of this paper.

3 Case Study

In our case study, we have created a VPP using the design presented in section Sect. 2. The VPP is implemented with a single Aggregator-agent and four Load-agents who simulate consumers. These consumers resemble commercial greenhouse growers, who play an important role in the production of ornamental pot plants and vegetables in Northern Europe. These are ideal candidates for VPPs, as they constitute complex process domains, where maintaining an optimal growth climate for plants requires intensive energy use and continuous supervision of different climate parameters such as \(CO_2\) levels, temperature and humidity.

3.1 Load-Agent Decision Logic

Each Load-agent has been configured with a single issue, namely the Light Plan-issue. Further, each Load-agent has a mission-critical Production DC (PDC) at priority 0, which reflects production needs, as well as a non-mission critical Energy Reducing DC (ERDC) at priority 2. Various other concerns could have been included to reflect that the photosynthesis process is influenced by various climate concerns such as \(CO_2\), air temperature, and humidity. However, as the MOMI negotiation concept used has previously been shown to handle such scenarios [12], we have chosen to include only the PDC, in order to simplify the explanation of our approach. Finally, each Load-agent has a single RC, the Aggregator Agent RC (AARC), which represents the Aggregator-agent.

The Light Plan-issue defines values on the form \({\varvec{l}}_n=[l_{1,n},l_{2,n},\cdots ,l_{t,n}]\) where n is the Load-agent. This means that each contract in the Load-agent will contain a vector with values on this form. The values of elements in this vector depend on the nature of the Load-agent - or rather, the load that the issue represents in the Load-agent. For simplicity, loads are assumed to be either on or off in the following, yielding \(l_{x,n} \in \left\{ 0,1\right\} \), which means that this value maps to the state of the actuator - not its consumption.

The PDC reflects the need for artificial growth light to the plants. Specifically, the target vector of the PDC represents the amount of photo-synthetically active radiation (PAR) in MolSqrMeter, required to achieve the production goal. In this case then, the target vector is a scalar defined as \(g_n \in \mathbb {R}\).

The contribution from artificial growth light varies across the day. That is, the contribution is marginal when the sun shines whereas the contribution is significant in hours with cloudy weather - or after sunset/before sunrise. To this end, the PDC is supplied with a coefficient vector defined as \({\varvec{v}}_n = [v_{1,n},v_{2,n},\cdots ,v_{t,n}]\) where \(v_{x,n} in \mathbb {R}\). The coefficient vector represents the estimated amount of photosynthesis obtained from artificial light in each hour. This is used by the PDC to calculate the PAR contribution for a given contract C as \(h_n=\sum _i^t(v_{i,n}*l_{i,n})\), where \({\varvec{l}}_n \in C\). The PDC then returns a cost, \(q_n\), defined by the absolute difference between \(g_n\) and \(h_n\), \(q_n=|g_n-h_n|\). The coefficient vector changes over time as a result of the underlying estimates changing. This means that the nature of the cost space created as a function of contracts and the PDC target vector, changes over time. This again means that the relation between the PDC and other concerns changes over time, as this relation is forged solely based on costs returned by concerns to contracts. Hence, no assumption about relations between concerns can be made.

The AARC compares suggestions for allocations made by the VPP towards contracts suggested by the mediator in the Load-agent. Each AARC has a target vector \({\varvec{f}}_n=[f_{1,n},f_{2,n},\cdots ,f_{t,n}]\), which is the current electricity allocation made in the Aggregator-agent for that particular Load-agent. This means, that the target vector of the AARC changes as the inter-agent negotiation progresses. As the Load Plan-issues represents an actuator, the task of the AARC is to convert actuation values into an electricity consumption, which can be compared towards its target vector. The AARC maps off-hours to a consumption of 0 MWh and on-hours to a consumption of 1 MWh. The 1 MWh then corresponds to the installed effect of the lamps in the greenhouse, in which the Load-agent reside. Essentially, this means that the AARC has a coefficient vector of 1’s, which are multiplied on value suggestions for the Light Plan-issue. Had the installed effect been different, this would be reflected by the coefficient vector of the AARC. Multiplying the coefficient vector of the AARC with values for the Light Plan-issue, yields a vector \({\varvec{d}}_n=[d_{1,n},d_{2,n},\cdots ,d_{t,n}]\). The AARC calculates the cost, \(q_n\) of a contract as the absolute difference of each element in \(f_n\) and \(d_n\), - \(q_n=\sum _{i}^t|f_{i,n}-d_{i,n}|\).

The ERDC is included to have Load-agents generate baselines, which adhere to the assumption of being energy efficient. The cost function of ERDC returns a value, which corresponds to the sum of elements in \(l_n\), defined as \(q_n=\sum _i^tl_{i,n}\), which means that use of more artificial light returns a higher cost. Implementing the ERDC as a non-mission critical DC means that the Load-agent will favor adhering to requests of the Aggregator-agent over achieving energy efficiency.

3.2 Aggregator-Agent Decision Logic

The Aggregator-agent contains three Electricity Allocation-issues, which represent allocations for Load-agents as well as seven concerns: one mission critical Load Management DC (LMDC) which represents DR events in the MOMI negotiation, and two Load Agent RCs (LARCs) for each Load-agent. The logic behind using two RCs for each Load-agent is that allocations of electricity do not attribute the same contribution to the Load-agent, if it is moved from one slot to another. By having two RCs representing a single Load-agent, a distinction between temporal allocation of electricity and the amount of electricity allocated can be made. This means that a profile, in which electricity allocation has been shifted, is preferred over one, in which the electricity allocation has been reduced. If the Load-agent was represented by only one RC in the Aggregator-agent, a profile which time shifts an allocation would yield a higher cost, than one which simply reduces the allocation made to the Load-agent.

Values for the Electricity Allocation-issues represent hourly electricity allocation, for each of the Load-agents across a day. The issue values are defined as vectors, on the form \({\varvec{e}}_n=[e_{1,n},e_{2,n},\cdots ,e_{t,n}]\), where t is the number of slots, n is the Load-agent, to which the allocation maps and \(e_{x,n} in \left\{ 0,1\right\} \) is the allocation size. The definition of the values that \(e_{x,n}\) can take on depends on the minimum and maximum consumption of the industrial process as well as the steps in which the consumption of a given process can be regulated.

The LMDC enables the VPP to offer Load Management (LM) actions towards external parties. LM encompass the actions load shedding, valley filling and load shifting [2]. In peak clipping, load is reduced in one slot of an electricity profile, as opposed to valley filling, in which load is increased in a slot. Finally, load shifting is a combination of these actions, where load is moved from one slot to one (or several) other(s). The LMDC negotiates over the summarized value of all Electricity Allocation-issues, defined as \(y=\sum _n^t e_n\), to ensure that the combined allocation adheres to incoming DR events. It has a target vector \({\varvec{p}}=[p_1,p_2,\cdots ,p_t]\) which is defined by the DR event. That is, the DR event contains a target vector for the LMDC. As the Aggregator-agent has knowledge of the electricity profile of each Load-agent, before starting negotiation with these, it is fair to assume, that the Aggregator-agent can propagate this as an aggregated baseline electricity profile to an external party. This enables external parties to create DR events with target vectors as altered versions of the initial, aggregated baseline electricity profile of the VPP. The cost function of the LMDC returns a value, which corresponds to the absolute difference of each element in y and p, formally defined as \(q_n = \sum _{i}^t|{y_i-p_i}|\).

The concern \(LARC _time \) negotiates over the Electricity Allocation-issue, representing allocation for its Load-agent. Each \(LARC _time \) has a target vector \({\varvec{d}}_n=[d_{1,n},d_{2,n},\cdots ,d_{t,n}]\), which represents the electricity demand profile of the Load-agent. The value of this vector changes over time, as the inter-agent negotiation progresses. Again, this is due to the bilateral negotiation between the Aggregator-agent and the Load-agent. The cost function of each \(LARC _time \) returns a value, which corresponds to the absolute difference of each element in \(e_n\) and \(d_n\), defined as \(q_n=\sum _{i}^t|e_{i,n}-d_{i,n}|\). The concern \(LARC _sum \) works in a similar way, except this concern is solely concerned with the amount of electricity allocation - and not the time at which it is allocated. The cost function of each \(LARC _sum \) returns a value, which corresponds to absolute difference of the accumulated values of elements in \(e_n\) and \(d_n\), defined as \(q_n=|\sum _{i}^t e_{i,n}-\sum _{i}^t d_{i,n}|\).

4 Experiments and Results

We consider 4 sets of experiments based on the scenario in Sect. 3. The first experiment serves as a baseline case, to validate that Load-agents will be allocated their requested demand, in case of no DR event. The three subsequent experiments map to each of the LM actions described in Sect. 3. Each of the PDCs were given a target vector of 400 MolSqrMeter. This number reflects that each Load-agent needs to reach a certain light sum, in order to achieve the required plant growth. Each PDC x is assigned a coefficient vector \(v_x\) as shown in (1), (2), and (3), which varies between the Load-agents, to reflect varying conditions in each of the domains. For the purpose of illustration, the coefficients vectors share the property that they yield a single optimal schedule with respect to energy efficiency, in order to reach the goal of 400 MolSqrMeter.

$$\begin{aligned} v_1=[25~0~25~25~25~25~50~50~50~100~50~25~50~25~25~25~25~25~25~50~25~25~25~25] \end{aligned}$$
(1)
$$\begin{aligned} v_2=[25~0~25~25~25~25~25~25~25~25~50~100~50~50~50~25~50~50~25~25~25~25~25~25] \end{aligned}$$
(2)
$$\begin{aligned} v_3=[25~0~25~25~50~50~25~25~50~100~50~50~25~25~25~25~25~25~50~25~25~25~25~25] \end{aligned}$$
(3)

The most energy efficient schedules for the Load-agents are used as baselines, which are shown in (4),(5) and (6) and the aggregated baseline profile of the three Load-agents is shown in (7). In a real-life scenario, the baseline would be deducted live, as this knowledge is present in the target vectors of the LARC’s in the VPP.

$$\begin{aligned} d_1=\,\,\,[0~0~0~0~0~0~1~1~1~1~1~0~1~0~0~0~0~0~0~1~0~0~0~0] \quad \end{aligned}$$
(4)
$$\begin{aligned} d_2=\,\,\,[0~0~0~0~0~0~0~0~0~0~1~1~1~1~1~0~1~1~0~0~0~0~0~0] \quad \end{aligned}$$
(5)
$$\begin{aligned} d_3=\,\,\,[0~0~0~0~1~1~0~0~1~1~1~1~0~0~0~0~0~0~1~0~0~0~0~0] \quad \end{aligned}$$
(6)
$$\begin{aligned} d_{agg}=[0~0~0~0~1~1~1~1~2~2~3~2~2~1~1~0~1~1~1~1~0~0~0~0] \quad \end{aligned}$$
(7)
Fig. 2.
figure 2

Result of baseline experiment. Numbers behind legend map to Load-agents

Full size image
Fig. 3.
figure 3

Result of peak clipping experiment. Numbers behind legend map to Load-agents

Full size image

To simulate DR events, three target vectors were created for the LMDC, based on the aggregated baseline profile in (7). These can be seen in (9), (10) and (11) respectively. The baseline target vector shown in (8) is equal to \(d_{agg}\), as we want each Load-agent to be provided with the most energy efficient allocation, in case of no LM request. The subsequent target vectors were defined by altering the baseline goal according to the LM actions peak clipping, load shifting and valley filling. Obviously, the target vector of the LMDC is defined by the DR event in a real life setting.

$$\begin{aligned} p_{baseline}=\,\,[0~0~0~0~1~1~1~1~2~2~3~2~2~1~1~0~1~1~1~1~0~0~0~0] \quad \end{aligned}$$
(8)
$$\begin{aligned} p_{peak\_clip}=\,[0~0~0~0~1~1~1~1~2~2~2~2~2~1~1~0~1~1~1~1~0~0~0~0] \quad \end{aligned}$$
(9)
$$\begin{aligned} p_{valley\_fill}=\,[0~0~0~0~1~1~1~1~2~2~3~2~2~1~1~1~1~1~1~1~0~0~0~0] \quad \end{aligned}$$
(10)
$$\begin{aligned} p_{load\_shift}=\,[0~0~0~0~1~1~1~1~2~2~2~2~2~1~1~1~1~1~1~1~1~0~0~0] \quad \end{aligned}$$
(11)
Fig. 4.
figure 4

Result of valley filling experiment. Numbers behind legend map to Load-agents

Full size image
Fig. 5.
figure 5

Result of load shifting experiment. Numbers behind legend map to Load-agents

Full size image

Each experiment was executed with a generation limit of 500, which was empirically found to ensure convergence in the local negotiation. Each experiment was repeated 30 times to remove influence of randomness from the GA, as recommended by [14]. In order for the design of the proposed mechanism to be successful, the experiments should show that adherence to incoming DR events is made with minimal impact on the Load-agents. That is, each Load-agent should be allocated their demand - or as close to it as possible. When the correct amount of allocation is available (in case of a load shift), no conflict should exist between Load-agents and the Aggregator-agent.

The results of the baseline experiment is shown in Fig. 2. Here we see that the aggregated demand profile corresponds to the one in (7). Figure 3 shows the result of a peak clipping experiment, in which the LMDC in the Aggregator-agent will enforce a peak clipping request of 1 MW. To do so, a single Load-agent must be forced to compromise its electricity demand by 1 MW. As can be seen, one Load-agent is allocated 1 MW less than it demands in slot 11. This leads to an inter-agent negotiation conflict as expected. The Load-agent, which is being forced to compromise, is observed to change over the 30 experiments. However, the aggregated profiles remain the same for all experiments. In the valley filling experiment, a single Load-agent must increase its consumption. As in the peak clipping experiments, this leads to a conflict, as it forces one of the Load-agents to exceed the goal of its PDC in time slot 16. The results are shown in Fig. 4. Again, the Load-agent selected for an increase varies throughout the experiments. Lastly, Fig. 5 shows the results of the load shifting experiment, where a Load-agent must shift its consumption. We observed that the aggregated load is shifted as expected, increasing its consumption in slots 16 and 21 in return for a reduction in slot 11. An important note here is, that the Load-agent making a shift, is compensated in a way, which ensures that it reaches its production goal. In a real-life scenario this could mean, that the Aggregator-agent would first have to shed load, and then gradually increase compensation, until the conflict is solved.

5 Conclusion

Flexibility in electricity consumption on the consumer side can help to maintain balance between supply and demand, in electricity grids with high penetration of wind- and solar power. In this paper, we propose a multi-agent based design of a VPP that is able to integrate energy-intensive industrial loads in DR programs. We propose a novel inter-agent multi-objective multi-issue (MOMI) negotiation mechanism to coordinate the individual electricity consumption of several industrial loads towards received DR events. In this way, a VPP may expose an aggregated load profile which conforms to electricity market requirements. Our design handles DR events by performing LM in the form of peak clipping, valley filling, and load shifting on aggregates of industrial loads. We illustrate how the proposed design will adhere to DR events, without violating production requirements of the industrial domains, in cases where sufficient electricity is available. Further we show, how allocation of electricity follows demand from the industrial domains, to the best degree possible. Based on the present proof of concept, we plan to extend the model to handle conflicts which can arise in cases of insufficient electricity.