Co-optimization of electricity transmission and generation resources for planning and policy analysis: review of concepts and modeling approaches

Abstract

The recognition of transmission’s interaction with other resources has motivated the development of co-optimization methods to optimize transmission investment while simultaneously considering tradeoffs with investments in electricity supply, demand, and storage resources. For a given set of constraints, co-optimized planning models provide solutions that have lower costs than solutions obtained from decoupled optimization (transmission-only, generation-only, or iterations between them). This paper describes co-optimization and provides an overview of approaches to co-optimizing transmission options, supply-side resources, demand-side resources, and natural gas pipelines. In particular, the paper provides an up-to-date assessment of the present and potential capabilities of existing co-optimization tools, and it discusses needs and challenges for developing advanced co-optimization models.

1 Introduction

Optimization models have long been used by utilities to inform decisions about electricity supply and transmission investments, and by public agencies to evaluate policy designs. While the established practice has been to plan generation resources first, and then to plan transmission to deliver that supply, in co-optimization both are assessed simultaneously to identify potentially attractive integrated solutions that may be overlooked under the conventional approach.

Use of co-optimization is important for two reasons. First, generation and transmission solutions are often substitutes: loads can be met either with local resources or transmission of remote supplies. Second, siting of new generation, including renewable sources, is influenced by the availability of transmission, so that transmission additions will impact future patterns and mixes of generation investment. Consequently, benefits calculations for transmission expansion should consider not only fuel savings resulting from reduced transmission congestion, as is traditionally done in production costing studies, but also capital cost savings from more efficient generation investment. By modeling interactions between transmission and generation economics, co-optimization models promise solutions that are less expensive in total compared to decoupled optimization (transmission-only, generation-only, or iteration between the two) [1–3].

This rationale for co-optimization is recognized not only by researchers (e.g., [4]) but also by regional transmission operators (RTOs)/independent system operators (ISOs) who mandate consideration of resource interactions in transmission planning (such as the California ISO [5]). The recognition of transmission-generation interactions has motivated the development of co-optimization models and demonstration applications to transmission planning and policy problems (such as [6–11]). In particular, to provide better decision support for planners and regulators, planning models are needed that optimize transmission investment while simultaneously recognizing potential bottlenecks in natural gas supplies and considering tradeoffs with investments in electricity supply as well as demand-side/storage resources.

Recently, advances in solution techniques and computation have made it practical to formulate and solve realistically sized co-optimization models.¹ In a vertically integrated planning environment, such models can be used to expand the scope of integrated resource planning (IRP) in order to capture the value provided by transmission as well as the diverse resources that are the traditional concern of IRP [12, 13]. In an unbundled electric system, these models would instead be used by transmission planners for what we call “anticipative planning,” in which those planners project how network enhancements change incentives for independent generators and other resource companies to invest [14, 15].

The goal of this paper is to provide a review of co-optimization models to complement existing summaries of the formulation and application of models for generation expansion planning (GEP) (e.g., [16–19]), electricity market simulation [20], and transmission expansion planning (TEP) (e.g., [21, 22]). Given the growing recognition of the benefits of co-optimization and the expanding literature and methods in that area, an overview is needed of approaches to co-optimizing transmission, generation (including distributed and variable renewable resources), loads (including demand response (DR) and more traditional demand-side management programs), and natural gas pipelines. Our purpose is to provide an up-to-date assessment of the present and potential capabilities of existing co-optimization models relative to alternative transmission planning approaches.

Fig. 1

A 3-node example

This paper begins with a motivating example to provide readers some concrete ideas of the potential benefits of the co-optimization planning approach. It then reviews the tools presently used for resource planning and co-optimization in Sect. 3. Section 4 provides a summary of implementation issues and approaches for co-optimization model including data requirements, formulations of particular system features and solvers. The paper closes with conclusions in Sect. 5. The work in this paper is a modified version of a small section of work that the authors presented in their whitepaper on the “co-optimization of transmission and other supply resources” [23].

2 A motivating example

In the white paper written by the authors [23], several numerical examples (in Chapter 4 using small as well as large scale electric systems) are presented to demonstrate the benefits of co-optimization. To make the current paper stand-alone, we provide one of such small-scale examples in this section.

Consider a 3-node network as shown in Fig. 1 below. In the network, there are three existing generators (G1, G2 and G4); while demand only exists at Bus 3. In addition, a new generator can be built at Bus 3 (denoted as G3), along with the options of building three new transmission lines (as those red dotted lines in Fig. 1.) The data corresponding to the generators and transmission lines are summarized in Tables 1 and 2, respectively. Note that generator G4 is an existing wind plant, with an assumed capacity factor of 30 %. Furthermore, for simplicity, we assume that the planning horizon is only of one year, with an hourly demand at Bus 3 (D3) of 250 MW for each of the 8760 h.

Table 1 Generator data for the 3-node example

Table 2 Transmission line data for the 3-node example

All the planning problems (generation-only, transmission-only and co-optimization) are solved by the GENTEP model, which is able to solve capacity expansion problems with detailed dispatch modeling and DC-approximation of transmission lines. (See [24], for example, for more detailed account of GENTEP.) Note that although a wind plant (G4) is in the capacity mix, the stochastic variation of energy output from the wind plant is not simulated in this example purely for simplicity’s sake. Another form of the uncertainty, namely, the forced outages of generators and transmission lines, is indeed considered in all the simulated runs. As a result, there is a positive probability that the projected load may not be met in certain time, and the metric “expected energy not served (EENS)” is used as a measurement of reliability in all the three different planning approaches.

The solution to the generation-only planning is straightforward in this example, as building plant G3 at Bus 3 is the only feasible option to meet the future demand. (The capacity of the two power plants at Bus 1 cannot be utilized due to the capacity constraint of transmission line L2, which is a direct effect of the Kirchhoff’s current and voltage laws.) For transmission-only planning, new transmission lines are built, as expected, to alleviate the severe congestion on the line connecting Bus 1 and 3, so that the two (cheaper) power plants at Bus 1 can be used to meet the demand at Bus 3. For the co-optimization planning, the resulting optimization process chooses to build both a line connecting Bus 1 and 3, and a power plant (G3) at Bus 3. Such a solution results in 2 % lower system cost than the transmission-only solution and 7 % lower than the generation-only solution (as shown in the solution summary table below). Moreover, system reliability (as reflected by the metric EENS) is also improved in the co-optimized solution, since all four power plants are dispatched to meet the demand, and the transmission congestion between Bus 1 and 3 is alleviated as well. Consequently, the chance that the demand is not met due to an outage of any of the assets is much reduced, as compared to the generation-only or transmission-only approach. The solutions for the three planning methods are summarized in Table 3 below.

Table 3 Solutions of the three different planning approaches for the 3-node example

In addition to the above simple example, a case study performed by Sauma and Oren [1] can further motivate the need for co-optimization, in which they compare the co-optimization planning with a sequential planning approach. The case study considers a 30-bus, 39-transmission-line, and 6-generation-firm network, and it is demonstrated for the particular network example that co-optimization results in an approximately 17 % total system cost savings, compared to the sequential planning of determining generation investment first, followed by transmission investment decisions.

3 Review of existing co-optimization models for policy and planning

The example in the previous section provides a motivation to investigate further the co-optimization planning approach. Various forms of such an approach have indeed been proposed in the literature. In this section we begin with a historical perspective on the concept of co-optimization in Sect. 3.1. In Sects. 3.2 and 3.3, we summarize existing co-optimization models, differentiating between ones used for national- or regional-scale policy analysis, and ones used for detailed transmission planning. These discussions provide context for our discussion of particular modeling and implementation issues in Sect. 4.

3.1 Historical perspective

Early versions of co-optimization models were proposed as linear programs, and versions of these models were developed in the 1960s and documented by Turvey and Anderson in their classic book on electricity economics [25]. However these early models used transshipment (“pipes and bubbles”) formulations of load flows and thus did not address the effect of Kirchhoff’s voltage law (KVL) on transmission (i.e., parallel flows).² Therefore they implicitly assumed more ability to reroute flows to avoid congestion than is actually possible in the absence of flexible alternating current (AC) transmission system (FACTS) devices such as phase shifters. Consequently such models that disregard KVL generally overstate the existing transmission capacity and under-estimate the additional transmission capacity required to accommodate a given transfer of power from resources to load.

Developed separately, power plant siting models choose specific locations for power plants and transmission lines, subject to assumptions concerning generation mix. A flurry of activity in this area was the result of President Carter’s National Coal Utilization Assessment, and is summarized in [26, 27]. These models tended to treat generation and transmission investment as continuous variables (where any nonnegative MW value can be chosen, disregarding the fact that generators and transmission lines come in discrete sizes), and also ignore Kirchhoff’s voltage law.

More recently, several generation-transmission co-optimization models, described in more detail below, have been applied at two scales. One scale involves detailed representations of AC load flows, or linearized DC load flow approximations, of actual high voltage transmission facilities as they interact with potential generation facilities within a single utility service area or other (relatively) small regions. The other scale encompasses large regions (e.g., the Western Electricity Coordinating Council (WECC), Eastern Interconnection, or the European Union) and uses linearized DC load flow approximations of aggregations of transmission facilities, as well as simplified representations of generation options as classes of technologies rather than individual generating units with unique operating characteristics. The latter scale can be viewed either as a simplification of planning methods that are applied to larger regions, or, alternatively, as an improvement upon regional siting models and other long-used tools (such as ICF’s integrated planning model (IPM) [27]) in terms of replacing transshipment representations of transmission with more realistic linearized DC load flows.

There are three key differences between fine-scaled expansion planning models and national/regional policy tools. First is the level of spatial aggregation. National planning tools have low spatial resolution. For example, in some global versions of MARKAL, Europe is represented as a single node, while in US Environmental Protection Agency’s (USEPA) MARKAL 9 regions are specified for the US. However, regional planning tools normally contain much more detail. For example, PLEXOS can operate at 3 types of geographical units: regional, zonal, or nodal. The second key difference is the user. National planning tools are mainly used by regulatory bodies and governments, while regional planning tools are widely used among utilities, ISOs, and consulting firms. It is to be noted that some of these tools have the capability of performing both national and regional scale planning studies, being flexible in their spatial resolution. The third key difference is that national tools often explicitly consider price formation in fuel markets, such as the coal and natural gas supply curves considered in IPM. In contrast, regional tools assume fuel prices as fixed inputs, and instead treat the electric power system itself in much more detail.

3.2 Co-optimization models for national/regional policy analysis

Co-optimization tools that are employed by governments and regulatory bodies usually address the power system on an inter-regional, interconnection, or even national level. For instance, the USEPA and US Department of Energy (USDOE) use national models with simplified representations of interregional transmission constraints. Some of these models are used to assess the potential profitability of generation investments in different sub-regions considering the reaction of the rest of the market. Table 4 summarizes several national policy tools and their characteristics. Several additional models that can be used for co-optimization at the national or large region level are summarized later in Sect. 3.3; models in that section tend to have more detail on power system transmission and generation, and less on other energy forms than some of the comprehensive models shown in Table 4 (especially MARKAL and NEMS).

Table 4 National policy and planning tools

3.3 Co-optimization models for detailed transmission planning

The transmission expansion planning problem is typically regarded as a nonlinear and highly complicated problem, which justifies application of advanced optimization models and algorithms such as mixed-integer programming, Benders decomposition, and heuristics (e.g., genetic algorithms). Co-optimization models that attempt to tackle the computationally difficult aspects of transmission planning date back to 1970s where linear mixed-integer programming models were proposed [28]. Nearly all of these were research efforts and have not been implemented as commercial software. Later, decomposition approaches were used to break down the problem into more manageable pieces. For instance, a Benders decomposition-based approach was used in [29] to separate and coordinate the investment problem and operating sub-problems [29]. Other models moved beyond investment and operations cost minimization to include customer interruption penalties [30], allowing consideration of tradeoffs between costs and reliability. However, earlier models were highly simplified and thus insufficient for actual generation and transmission expansion planning.

Over the last two decades, additional research efforts have been made to address the co-optimization problem in electricity markets, while considering more operation details. Table 5 summarizes several existing co-optimization methods for planning, including what they model (such as which energy sectors are included, and how transmission is represented, etc.), their treatment of time (which is crucial for long-lived investments), and finally what types of optimization problems are solved. Next, we discuss some particular areas of application of co-optimization for planning.

Table 5 Summary of existing co-optimization models for planning

A common approach to accommodate more operation detail is to formulate co-optimization as a bi-level optimization problem for generation and transmission, sometimes using iterative approaches to coordinate the two planning problems [31–34]. Baringo and Conejo [35] presented a bi-level stochastic co-optimization model but instead of iterating, they transform it into a single-level mathematical program with equilibrium constraints. They showed that transmission expansion decisions significantly impact wind power capacity expansion even though investment costs in transmission expansion are much lower than those in wind power capacity.

As other examples of issues addressed in co-optimization models for planning, a recent study in [36] used co-optimization to show that distributed generation could mitigate congestion and defer transmission investments. Follow-up studies accounted for incentives offered to independent power producers [37] and for the possibility of building microgrids [24]. The latter analysis showed that considering microgrid investments in co-optimization could provide significant reliability and economic benefits. Head et al. [38] introduced a multi-area co-optimization model and demonstrated that the proposed model could offer considerable economic benefits in power pools. Finally, [8] proposed a capacity payment mechanism in a co-optimization model for transmission and generation facilities.

4 Modeling approaches for co-optimization

This section lays out basic choices associated with designing a co-optimization model and also summarizes advantages and disadvantages of each choice. Section 4.1 presents the choices associated with transmission network representation in terms of modeling fidelity, while Sect. 4.2 discusses network representation in terms of modeling coverage. The sections also review the associated data requirements, investment decision options and features of model optimizers for each modeling choice. Section 4.3 discusses a few important modeling issues such as end effects, optimization time periods, and uncertainties.

4.1 Network representation in co-optimization: choice of model fidelity

In this section, we summarize the pros and cons of the choices in representing transmission networks used in a co-optimization model. In the order of decreasing fidelity, these include the alternating current (AC) model (Sect. 4.1.1), the direct current (DC) model (Sect. 4.1.2), and the transshipment model (“pipes-and-bubbles” or network flow) (Sect. 4.1.3). However, with enhanced fidelity comes increased computational challenges in the resulting optimization problem. Hybrid models (Sect. 4.1.4) attempt to combine fidelity advantages of some types of network models with computational advantages of others. For each type of the network model, we summarize its formulation, solution, investment alternatives, and data requirements

4.1.1 AC power flow model

Formulation An AC model consists of a complete representation of real and reactive power flows in the transmission network governed by electrical laws, which are expressed in terms of a non-linear function of network states, namely bus voltages and angles, and network parameters (impedances). The AC optimal power flow (ACOPF) problem is formulated as an economic generation dispatch problem with network flow constraints. The comprehensive formulation enables simultaneous management of real power (P) demand with voltage (V) and reactive power (Q) requirements. Along with the non-linear power flow relations in the ACOPF, the use of integer transmission expansion variables make the co-optimization a Mixed Integer Non-Linear Programming (MINLP) problem, a very challenging optimization problem to solve.

A complete mathematical formulation of an ACOPF-based generation and transmission planning model (ACOPF-GTEP) is presented as Eqs. (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12) in Sect. A.1 of the appendix. The integer variables and non-linear constraints that render the problem a MINLP are shown in Eqs. (2, 3, 4, 5, 8). If a strategy for dividing the problem into investment and operating (OPF) subproblems is pursued, then the OPF is a continuous nonlinear optimization problems (NLPs). The OPF can be solved by iterative methods involving basic line-search steps [39], where an initial solution $x_{0}$ at $k=0$ is chosen, and then a search direction $d_{k }$and an appropriate step size $s_{k}$ are selected to update the solution vector $x_{k}$; this search is repeated until the convergence criterion is met. There are numerous methods to implement each of these steps, which thereby differentiate the many available solvers or algorithms. When integer variables are included in the NLP, the problems are rendered even more difficult; attempts to solve a combined investment-OPF problem use variations of branch and bound and cutting plane methods in combination with NLP methods for continuous problems. Commercially available solvers for MINLP and NLP problems are summarized elsewhere [40].

In the context of contemporary solvers and proven computational techniques, the convergence and computational challenges in solving ACOPF-GTEP problem make it impractical to apply for real world applications [39]. However though challenging, such models are not unsolvable and studies are being carried out to develop techniques that relax the MINLP formulation to solve the optimization model in time-efficient manner, thereby preserving the higher fidelity such model offers and convincingly document the added value of considering voltage and reactive power flows along with real power flows for better resource and transmission investment decisions. One commonly known way to simplify the model is by using a decoupled power flow formulation or removing the reactive power flow parts altogether (corresponds to Eqs. (5), (10) in the Appendix), while still capturing the interactions between bus voltage magnitude and real power transfers. Another simplifying technique is to use binary variables instead of integer decision variables, thereby changing the decision from how many transmission lines are to be built to the simpler problem of whether or not a candidate transmission project is to be built. This replaces the integer variable with multiple stages to a variable with two states (0 or 1), thereby reducing the problem complexity. A relaxation of the full ACOPF-GTEP model using binary decision variables is shown using Eqs. (13, 14, 15, 16, 17, 18) in Appendix Sect. A.1. A further relaxation of the model uses continuous decision variables for transmission investment, as shown by Eq. (19) in Sect. A.1 of the Appendix. Likewise, there are several other ways to relax the MINLP formulation [41–44], and it still remains a challenging research issue to understand the various pros. and cons. of each in terms of their degree of computational efficiency and ability to preserve model fidelity, when applied to practical real world systems.

Investment options The investment options in a multi-period ACOPF-GTEP model are:

• 1. Generation: where, when and how much of different technologies to be invested, usually represented as continuous (MW) decision variables.

• 2. Transmission: where, when, how many transmission lines to be invested (if integer), should there be investment in a particular line (if binary).

• 3. AC transmission technologies: the ability to choose between different voltage levels for AC transmission can also be embedded in the model by designing candidate arcs with appropriate arc operational and investment characteristics (i.e., cost, losses, capacity) for respective kV levels, and each will its own binary decision variable.

• 4. Shunt compensation: investments in shunt compensation can be considered in ACOPF formulation, as shown by Eq. (20) in Appendix Sect. A.1. These devices help provide the required reactive power and regulate the system voltage within specified security limits. Long distance real power transmission over AC lines involves a commensurately high reactive power transfer, which causes a significant decrease in bus voltage at the receiving end (load centers). A drop in voltage inhibits the power transfer capability and causes voltage stability issues. Therefore considering shunt compensation within the formulation considers tradeoffs between investing in more transmission or reinforcing the existing transmission using these devices at the load side [44, 45].

Data requirements The data required to run a co-optimization model basically includes data pertaining to system topology, historical and forecasted system conditions, operational and physical characteristics of existing and planned electric infrastructures, and finally the scenario descriptions. These are presented in Table 6. With an AC formulation, additional data required will pertain to reactive power and voltage, which include generator capability curves, reactive power limits, and voltage set points; transmission line apparent power rating and complex impedances; bus voltage limits; and costs and characteristics of FACTS devices.

Table 6 gives a synopsis of the kind of data required to run a co-optimization model, for both the AC and DC network representations.

Table 6 Synopsis of data and information needed to run AC- and DC-based co-optimization models

4.1.2 DC power flow model

Formulation The DCOPF problem is a linearized approximation of the power injection formulations of the ACOPF problem; it has been heavily used in industry operating and planning applications. The DC model basically consists of two relations, real power flow that is directly proportional to angle difference (in radians) and reactive power flow that is directly proportional to bus voltage difference (shown by Eqs. (21, 22) in Appendix Sect. A.2). Typically, because the goal of power system design is to minimize costs associated with real power production, and because circuit flows for heavily loaded circuits are dominated by their real power component, only equations pertaining to real power flows (also known as the B-theta model) are used in practical market and planning applications.

Though the DCOPF model in itself is a LP (an optimization realm that is advanced in terms of solution techniques and available stable solvers [40]), the DCOPF-based generation and transmission expansion problem (DCOPF-GTEP) is actually a MINLP, a non-linear and non-convex problem. This is because of its use of transmission investment integer variables in the formulation, and that voltage angles are divided by line reactances, both of which are decision variables. The constraints of this problem are shown using Eqs. (23, 24, 25, 26, 27, 28, 29) in Appendix Sect. A.2, along with which Eqs. (7, 9, and 12) in Appendix Sect. A.1 related to voltage angle and generation capacity should also be included. Fortunately, with the use of binary decision variables for transmission investments and a disjunctive formulation, the MINLP model can be transformed to a MILP. This disjunctive formulation based on the big “M” method [41] is shown in Appendix Sect. A.2 using Eqs. (30, 31, 32). Still, problems of larger size, though solvable, are computationally intensive.

The disadvantage of the DC approximation of non-linear AC relations is loss of model fidelity, since it does not incorporate voltage variables. For normal operating conditions, the inaccuracy is usually small for circuit flows. However, violations of bus voltage criteria cannot be detected. The corresponding need for reactive support can be partially captured by accounting for the effect of line-length on voltage drop and thus on line capacity for EHV-AC transmission, which was done in [46] using St. Clair curves [47]. Therefore, though the resulting expansion solution may be a good starting point, it should be subsequently studied with an AC power flow to ensure that real and reactive power flows are within limits and address any voltage violations.

An important extension of the DCOPF-GTEP is to add so-called security constraints for contingencies, resulting in the security-constrained problem. This can be done for circuit overloads via use of linear sensitivities (called generation shift factors and line outage distribution factors). Related constraint equations can be incorporated directly within the optimization, but at a computational price. Alternatively, the optimization problem may be decomposed to a master problem and contingency-related sub-problems. When violations of security constraints are encountered in the sub-problems, the master is rerun with added constraints, and the process repeats until no further violations are found in the sub-problems. In this approach, one can account for voltage violations by using a AC representation in the sub-problems, but at increased computational expense.

Investment options The investment options in multi-period DCOPF-GTEP model are:

• 1. Generation: same as AC model.

• 2. Transmission: same as AC model, except simplified based on the DC linearization.

• 3. Transmission technologies: the ability to choose between different kV levels of AC transmission can be embedded by designing separate candidate arcs for each voltage with appropriate arc operational and investment characteristics. Options for different HVDC technologies may also be provided.

Data requirements A DCOPF-GTEP model has fewer data requirements than an ACOPF-GTEP model. Of the complete set of typical network data required for an AC model, data for network resistances, bus voltage limits, generator reactive power limits and voltage set points, transmission line reactive power limits and FACTS data (in most cases) are not required for a DC model.

4.1.3 Transshipment model

Formulation In this model, the transmission network is represented similarly to transportation pipelines, which move a commodity between nodes in a network subject to an efficiency parameter representing transportation losses. As noted earlier, such a model respects the nodal balance constraint at every node and transmission flow limits, but not Kirchhoff’s voltage law. The general transshipment model is the basis of early regional siting models summarized in Sect. 3.1, and there are several variants of such models. Figure 2 shows a particular type of transshipment model called a generalized network flow representation [48] using a one-line diagram for two connected periods t and $t+1$. It is to be noted that generation (modeled by arc EG_EL and EW_EL), transmission (modeled by arc EL_EL) and demand (modeled by arc EL_L) are all represented as arcs, with appropriate values for its operational and investment cost, bounds, and efficiency properties. The two nodes of the generation arc depict typical fuel input to the generation on one side and the generation output on the other side. Cost of power flows across all arcs are subject to efficiencies and capacity bounds, and the required arc capacity expansions are minimized by the network flow optimization.

The network flow-based model’s formulation as a linear programming cost minimization problem is shown in Eqs. (33, 34, 35, 36) of Appendix Sect. A.3, which minimizes the operating and investment costs. Since, both generation and transmission arcs are considered as transportation pipelines (with different properties), the only physical relation that governs this model is the nodal power flow balance Eq. (34). We highlight the network-flow-based version because of its potential to be solved very efficiently by specialized algorithms. There is tremendous scope to speed up the solution of such problems using decomposition and parallelization methods [49], and advancements are being made in solving larger sized linear network flow problems using high performance computing [50].

Fig. 2

Power system represented using network flow model

The downside of the transshipment approximation is increased loss of model fidelity, in that it does not incorporate voltage variables and the relationship of real power transfers with the bus angle difference and line impedance. The solution may see inaccuracies in transmission flows relative to what could actually take place. Therefore, the resulting expansion solution has to be checked using DC or, preferably, full AC models in order to assess its feasibility with respect to Kirchhoff’s voltage law and network security constraints. If there are flow or voltage magnitude violations, then the expansion problem could be iteratively solved with proxy constraints for enforcing security as discussed in Sect. 4.1.2, until the expansion solution results in no violations.

Investment options The investment options in multi-period network flow based GTEP model are:

• 1. Generation: as in the AC and DC models.

• 2. Transmission: where, when, and how much transmission is to be built (continuous variable).

• 3. Transmission technologies: the ability to choose between different transmission technologies can be embedded by designing separate candidate arcs for each. However the differentiation can be made only with respect to operational and investment characteristics, and not based on equations governed by realistic electrical laws, as can be done using AC and to some extent with DC power flow models.

Data requirements A network flow model has even fewer data requirements than a DCOPF-GTEP model. It will not require network impedances, bus voltage magnitude and angle limits, generator reactive power limits and voltage set points, transmission line reactive power limits and FACTS data.

4.1.4 Hybrid models

A hybrid model is one that represents transmission lines with mixture of the above three described models. An example hybrid representation includes consideration of both AC and DC transmission technologies in the model [51]. In this case, DC lines are modeled as real power injections (positive and negative, as shown in Eq. (37) of the Appendix) at both the ends of the lines, which effectively translates to modeling it as a transportation pipeline. Therefore the resulting model will be either a hybrid of AC power and network flow models or a hybrid of DC power and network flow models.

Situations in which a hybrid model of transmission lines can be used are:

• 1. Study area emphasis If a particular area alone is of interest within an interconnected power system, the transmission lines within that area may be modeled with high fidelity, while the lines external to the area may be approximated as a transshipment model with power injections into and out of the area.

• 2. Modeling interconnections if a larger geographic region is analyzed, where interconnections across regions with different frequencies are assessed together, then a transportation model may be used to assess the DC interties connecting these interconnections, while the transmission within each interconnection may be modeled using DC or AC power flows.

All the discussions pertaining to formulation, investment options and data requirements can apply to the hybrid model based on the modeling choices made. A high level summary of the advantages and disadvantages of different network representations is presented in Table 7. Depending on the choices, a hybrid model can possess any of these advantages and disadvantages.

Table 7 Pros and cons of network representation: model fidelity

4.2 Coverage of the model

There are at least three kinds of choices concerning the scope of a model’s network and resources:

• 1. Sector and resource coverage Whether to represent fuel (gas and coal) networks, storage, and demand side resources

• 2. Geographical coverage How much of the electric network to represent (e.g., a subregion or the entire interconnection)

• 3. Voltage level How much of the lower voltage network to represent (e.g., some studies may represent only 765, 500, 345, and 230 kV equipment whereas others may also include sub-transmission equipment at 161, 138, 115, and 69 kV as well).

The benefit of increased model coverage is the ability to account for more operational and investment options, which can result in better estimates of costs as well as lower-cost plans. The drawback of the added detail is increased data preparation and computational burdens.

4.2.1 Sector/resource coverage

In this subsection, we describe some important modeling choices concerning the range of operational and investment alternatives represented within generation/transmission co-optimization planning, including fuel network (coal and gas), demand response and storage. In Table 8, we summarize the advantages and disadvantages of expanding a co-optimization model’s scope to include these features and alternatives.

Table 8 Pros and cons of expanding model coverage

Fuel network  Fuel networks are an integral part of the energy sector. If one considers electric power networks as the end-users in the energy sector, then fuel networks could be considered as the source. Figure 2 of reference [2] shows a high level schematic diagram of an energy supply chain, where three different yet interconnected subsystems are represented: coal, natural gas, and electricity sub-systems.

In addition to generation and transmission investment options, such a model may include decision variables for where, when and how much to invest in gas pipelines, gas storage, and coal and oil transportation. Including all sectors and options captures important interactions, such as the effect of increased natural gas-fueled generation capacity upon the need for gas pipelines, and the resulting delivered cost of gas. For instance, Fig. 3 shows a case where expansion options are available for generation, pipelines, and transmission lines. Solution could include siting natural gas generation close to the load with the associated expansion in pipeline, or alternatively siting it far away from the load center, thus requiring more power transmission capacity, or a combination of both (such as the 70/30 split shown in the figure). Investments in transmission and natural gas units may also provide the flexibility needed to economically integrate intermittent renewables. Therefore, a co-optimization tool that considers all these inter-related options may find more efficient and sustainable expansion solutions compared to a piece-meal approach.

Like transmission networks, fuel networks may be represented at different levels of fidelity. For example, one may represent gas pipelines using a transshipment model, or one may represent compressors together with pipeline flow-pressure relationships. In the latter case, there are various modeling choices, including Panhandle A and B, White, and Weymouth equations [52].

Fig. 3

Tradeoff between pipeline and transmission investments

A model with fuel sectors can be formulated either as an economic equilibrium model or as a linear programming-based cost-minimization model.

An economic equilibrium model is one that represents each market party’s decisions and their interactions [53, 54]. Usually, each market party is assumed to maximize their profit or net benefits, and the equilibrium problem concatenates the first-order conditions for each of those maximization problems with a market clearing condition (supply = demand). In essence, demand functions from end-users interact with supply functions from sources (which reflect production and transport technologies along with input costs) to determine equilibrium prices and quantities. Depending on the modeling of demand, production functions, and market bids, the model may be linear or non-linear. Typically such models are solved for hourly market operating conditions, and investments are made in yearly time steps. In such models, market signals, including marginal prices of a particular commodity and network congestion costs, together with input costs and investment criteria, such as required payback periods, determine whether the investments are made.

Linear programming-based fuel models instead integrate the different sub-systems of the energy sector (namely electric, coal, gas and petroleum) using a network flow model, and has an objective of minimizing the overall cost of meeting energy requirements in each sector. This is equivalent to a competitive equilibrium under certain restrictive conditions, such as zero price elasticity and perfect competition [54]. Usually for such multi-commodity models, each sector’s operations may be optimized at different time steps, typically dictated by the degree of variability in the respective commodity’s value. By having infrastructure investments as decision variables, such optimization models provide perspectives on what investments are needed to achieve particular policy goals in an efficient manner. If energy transport media such as natural gas pipelines and transmission lines are modeled using non-linear equations or integer variables for higher fidelity, then the optimization problem becomes a NLP, MILP, or MINLP.

In either case, multi-sector modeling will increase computational effort. This effort can be decreased through some degree of component/spatial aggregation (regions) within each sector and reduced temporal granularity for optimization. Linear programming models are easier to solve than equilibrium models, which require specialized complementarity or other algorithms [54].

Including fuel networks will increase the burden of data preparation. In addition to electric network data, information will be needed on fuel network topology along with geographical characterizations of investment cost, fuel cost and capacities. For instance,

• 1. The availability and quality of coal differs geographically, and so the availability, cost and transportation links for several varieties of coal may be required.

• 2. The gas imports/exports, pipeline and storage capacities need to be spatially specified.

Similar to having additional sectors on the source side (fuel network as discussed above), one could develop models that consider multiple sectors on the end-user side. One example of such a an integrated infrastructure system model is modeling electricity sector together with transportation [55, 56] and heating sectors [57, 58]. This will allow modeling and studying the interdependencies between these end-use sectors, and allow the co-optimization model to choose investments in those technologies that would allow meeting the energy demand across all the sectors in a sustainable and cost-effective manner.

Storage technologies and demand response Storage technologies can be modeled by representing the three operations of a typical unit, namely, charging, discharging, and reservoir dynamics. Conceptually a storage technology’s discharge operation is similar to a generator operation, and charge operation is tantamount to loading the system. The reservoir dynamics of storage must update the stored energy status periodically based on the current period’s injections and withdrawal, subject to the charging/discharging (or round-trip) efficiencies and energy stored from the previous period. Also, charging and discharging costs need to be considered, including perhaps the effects of depth of charging cycle on device life. The grid services provided by storage differ on the type of storage technology (based on their storage capacity), i.e., bulk storage (such as compressed air energy storage, pumped hydro, and large batteries) or short-term storage (including flywheels, superconducting magnetic energy storage, and batteries). Bulk storage technology is able to provide a wide range of grid services, such as day-night arbitrage, peak shaving, regulation, and spinning and non-spinning reserves. Short-term storage technologies are generally used for providing shorter term regulation services to accommodate load and renewable variations on a time scale of seconds to 15 min [59]. Therefore, depending upon the grid services that are required in a system, a co-optimization model should be able to appropriately site and size the storage technologies in strategic locations for grid’s as well as storage owner’s benefit [60].

Demand response competes against generation and transmission projects for addressing peak system needs, and also is capable of providing ancillary services. The modeling of demand response can be thought of as modeling a dispatchable load (or a dispatchable generator with negative generation), in some cases with storage-like characteristics (for instance, deferrable or schedule-able loads). Demand-side services can be provided from loads that are price sensitive or emergency interruptible, and from certain kind of storage systems such as ice-storage.

Demand response and storage can provide energy services to the grid, for instance displacing peaking units at the top of the generation stack. The competitive ramping rates of storage may obviate the need to invest in fast ramping combustion turbine units under increasing renewable penetration. Bulk storage may also act as a virtual transmission access, when optimally sited close to demand along the congested transmission path [61]. Both storage and demand response can provide very significant benefits in terms of flexible response to changing operating conditions as well as fuel and capacity cost savings, in part because they can help defer or replace any generation or transmission expansion plans. Hence their consideration within the overall co-optimization tool provides a wider set of potential solutions to the long-term planning problem, and thus has potential to significantly lower costs.

The specific optimization model formulation used to accommodate storage and demand response depends on the type of storage and demand response (DR) technology being integrated and the goal of the study. In general, a planning tool that models operational reserve requirements and quantifies the expense to generation owners of cycling conventional fossil-fuel fired units will increase the value of storage and demand response among the available resource options [62].

Two important modeling considerations in including storage are inter-temporal coupling and time steps, described below:

• 1. Inter-temporal constraints and simultaneous multi-period optimization  A storage technology’s reservoir modeling introduces constraints that couple multiple time periods, thus requiring simultaneous operation over a multi-interval time horizon [63] in order to economically manage reservoir status while providing grid services through charge/discharge operations. The basic storage relation needed is that the stored energy at period t must comprise of energy stored up until period t-1 less any leakage, plus (less) the energy to be charged (discharged) at period t. Including multiple periods increases the problem size, which can be controlled by assuming a reasonable operating cycle (2-day or weekly) for storage with boundary conditions on reservoir energy status at the start and end of the period.

• 2. Time steps  The appropriate optimization time step depends on the technologies considered and the purpose of the study. If the purpose of the planning study is to assess the economic benefits of bulk storage, then the tool should be able to model the ability of storage to profit from inter-temporal energy arbitrage opportunities [64, 65]. In this case, an hourly (or even a sub-hourly) time step for optimizing the operations will be ideal to capture the strategic dispatch of storage to take advantage of price differences. However, if economic assessment of storage is not the main focus of a study, then a simpler model that captures the basic storage relation mentioned above for a few samples of hourly system conditions (including peaks) may suffice. This will enable co-optimizing storage along with generation and transmission resources, while also avoiding making the model too large.

  If instead the emphasis is on short-term storage technology with very limited energy capacity and ability to make very fast transitions, an assessment at smaller time scales (sub-hourly, say multi-second or 5-min dispatches) is needed to assess their participation and profitability for providing regulation services. Though the problem remains an LP, it increases the problem size and may not be practical to include in a long-term generation-transmission co-optimization model. However, if an off-line assessment of the required MW capacity for ancillary services is estimated and represented in the long-term investment planning tool [66], then by assuming a pre-specified capacity factor, such devices’ capacity [67] could be considered as decision variables in the overall portfolio planning problem.

The above discussion of time steps is also applicable to DR modeling. An hourly time step usually suffices when the DR is dispatched based on system prices or capacity shortage conditions. It is also necessary to represent constraints on the number of times that DR can be called (e.g., 5 times per month).

Integrating storage and demand side options will increase data requirements. Data related to technology investment cost, life, and other operational attributes such as variable cost, capacities, efficiency, ramp rate, grid services and unit commitment related data (if needed) need to be prepared. For short-term storage, a good estimation of their long-term average utilization is necessary. Data for bulk storage unit’s cycling and costs are also useful [62]. The pros and cons of modeling these additional sector and resources have been provided in Table 8.

4.2.2 Spatial granularity and geographical coverage

The spatial granularity (i.e., whether nodes are defined at the plant level, substation level, or regionally) and geographical extent (i.e., sub-region, inter-regional, national) that need to be included in a model depends on the infrastructure-related problems to be investigated. Table 9 summarizes the advantages and disadvantages of detailed (plant-level) network representations vs broad geographical models, which much necessarily be more aggregate.

There is a tradeoff between granularity and extent. When optimized at a plant-level granularity, the geographical scope may be limited by the model size, required data and the associated computational complexities. However, detailed spatial granularity allows some operational strategies to be considered that otherwise cannot be represented (e.g., such as finding the individual unit retirements from an aggregated model’s capacity retirement output) or requires additional computation (e.g., optimal transmission switching [68] for mitigating operational violations—a complex MILP optimization problem providing operational solutions that might help defer the investments in generation and transmission).

Table 9 Pros and cons of network coverage: geography

On the other hand, the advantage of a larger geographic scope is that a larger variety of solution options can be included because of geographical variations in resource availability and economics. For instance, because of wind and load diversity over space, interregional transmission additions can be an effective way to provide system flexibility. However, to reduce the model size and computational burden involved in investigating infrastructure planning at a national or interconnection-wide scale, some aggregation of system components is necessary.

4.3 Additional modeling choices

In this section, three additional modeling choices are described which are significant to any planning study: end effects modeling, optimization time intervals and handling uncertainties.

Table 10 Pros and cons of the various approaches to mitigate end effects

4.3.1 End effects

It is best if a planning problem have a very long simulation time. The infrastructure assets typically have long lifetimes, and hence in making an assessment among alternative choices, one must consider the operational value of assets over their entire lifespan, as well as their overnight investment cost. However, in addition to being computationally intensive, model decisions in the later years are untrustworthy and impractical due to future uncertainties; they also are of less concern because of the time value of money. Hence optimization of long-term planning problems is truncated at some future year, such as year 20 or 30. This can cause distortions in model solutions which are termed “end effects.” A major effect of truncation is that near the end of the simulation time, there will be a bias against investment in assets with higher capital costs and longer lives because the cost savings they would provide in later years are not considered in a truncated objective function.

There are at least four well-known approaches to mitigate this issue [69, 70], for which pros and cons are summarized in Table 10:

• 1. Extended simulation  The most straight-forward, but computationally expensive way to avoid end effects is to simulate for a longer duration, but report the results only for the desired, shorter time horizon. The additional evaluation period is usually known as extension period. Many applications typically choose about 30–40 years for the extension periods [71]. This results a larger, perhaps more difficult to solve model. To simplify the evaluation, applications assume stationary conditions during these years, i.e., no increase in load and no additional investments or retirements.

• 2. Salvage value   The method truncates the optimization to the desired planning horizon, but places a proxy value on all assets whose lifetimes extend beyond that horizon. One common estimate is the undepreciated (or book) value of the asset at the final year of the simulation, for which present worth is then subtracted from the cost objective of the optimization problem.

• 3. Primal equilibrium   This method imposes an equilibrium condition on primal variables (x) after the desired planning horizon. It assumes, for an infinite horizon problem, there will be a finite horizon T beyond which the decision variables attain equilibrium, i.e., they increase at the rate of growth in demand ($\lambda )$, so that x(T+t) = $\lambda$ x(T+t-1), where t=1... $\infty$. The planning horizon ends at T, while additional constraints impose that the implied present worth cost for years following T are added to the objective function. This kind of model is most suited for problems for which resource availability would not limit system expansion in the foreseeable future.

• 4. Dual equilibrium   This method imposes an equilibrium condition on dual variables ($\mu$) after the desired end period. It assumes that the dual variables of the T stage problem increase in proportion to the assumed discount factor ($\alpha$), i.e., $\mu (T+t)= \alpha \quad \mu (T+t-1), t=1\ldots \infty$. The Lagrangian of the primal optimization reveals that all the constraints from T+1 until $\infty$ can be combined into a single term with a common multiplier $\mu (T)$. This type of model is advantageous when resource availability is limited (which is the case in most of the expansion problems in reality) and prices are expected to rise over time at an inflation rate.

4.4 Time interval definitions

The problem of planning capital-intensive infrastructure that operates for multiple decades generally must consider long-term impacts on economics, the environment, and resilience. At the same time, the impacts on short-term operation must also be considered, because many of the benefits of infrastructure arise from more efficient operations, which progressively lead towards the long-term cost and environmental benefits. Three kinds of time intervals are associated with long-term planning models (going from shortest interval to longest):

• 1. Operational time steps  This is the time interval at which operations within the power sector are modeled, i.e., hourly (like a day-ahead market) or monthly or yearly time steps. Inter-temporal operational relationships are modeled by relating successive time steps. For power systems, this may be hourly for chronological models or dozens to hundreds of hours if a multi-block load duration curve representation is used. Table 11 summarizes the advantages of different levels of aggregation of operational time steps.

  A model representing multiple sectors or sub-systems can have different operational time steps to represent the operations of different sectors. For instance, for natural gas, prices are often differentiated seasonally, so a few multi-month operational time step might be considered. For some studies, intraday price variations may not need to be considered, because of the large amount of storage as well as line pack that can absorb intraday variations. The operations in a coal network with reasonably stable prices may be optimized at yearly intervals. However, in such models it is necessary to define constraints among the decision variables so that these various time intervals are appropriately coordinated. For instance, the sum of gas consumed by power plants in all operating hours in a season should equal the seasonal amount of gas provided by the gas network.

• 2. Optimization period  This is the time period over which the operations are optimized, and investment decisions are made. As mentioned previously, for a multi-commodity model (e.g., fuel and electricity), within one optimization period the operations for each sector may be modeled at different time steps, typically dictated by the degree of variability in each respective commodity’s value.

• 3. Planning horizon  This is the period over which the planning program is executed to identify the overall long-term infrastructure investments. A planning horizon may have one or multiple optimization periods over which network investments are optimized.

Table 11 Pros and cons of operational time step choices

Figure 4 shows these three kinds of intervals using the arrows at the bottom, and three kinds of planning frameworks using rectangular boxes. While each framework has a particular operational time step, the differences are in optimization periods and planning horizons. The simplest planning model, depicted by framework 1, provides that the operational time step, the optimization period, and the planning horizon are all a single period of the same duration. Here, the decision of how much, where, and what to invest are made once for the entire planning horizon. Such a model is considered to be static because the solution provides no temporal information of when to invest. Alternatively, over a multi-decadal time horizon, there can be multiple optimization periods at yearly intervals or, in order to shrink the model size, every 2–5 years. Planning frameworks 2 and 3 adopt this strategy, with the difference being in the manner in which the multiple optimization periods are solved over the planning horizon. Framework 2 performs optimization over the entire planning horizon by considering all the optimization periods simultaneously, wherein the inter-optimization period relationships are built-in (denoted by solid lines between successive optimization periods) making it one large multi-period optimization problem. Although such modeling can illuminate very attractive strategies, it does so assuming information characterizing all optimization periods is known apriori; and it is also computationally expensive. As a result, the multiple optimization periods are often solved sequentially in a rolling manner, i.e., sequentially feeding the results of one to its successor optimization problem. For example, a 30-year planning horizon may be divided into ten 3-year optimization periods. This strategy is depicted using framework 3 which adopts a recursive solution procedure in which investments are evaluated for one optimization period, and then the model rolls forward to the next optimization period sequentially, taking the previous investments as a boundary condition (denoted by the dashed arrow feeding optimization period t solution to optimization period $t+1)$. Framework 3 is also known as dynamic or rolling planning, since it offers the possibility of accommodating dynamic changes in futures and parameter inputs between rolling optimization periods.

Fig. 4

Optimization time intervals and planning frameworks

4.4.1 Short- and long-run uncertainties

Power system operational and investment planning problems typically face many uncertainties. Some are long run, such as economic growth, technology advancements, and regulatory developments. Others are more relevant to an operating time scale, such as wind and load variability or component outages. At the operational time frame, planning against uncertainty is done through generation scheduling and commitment processes that ensure enough supply and/or demand side reserves to cover possible short run conditions that are not predicted by day- or hour-ahead forecasts. Because different types of investments have different effects on system flexibility, a long-term planning problem must also account for such operational reserve allocations when evaluating those investments [9, 66, 72]. Stochastic programming was applied in [73, 74] to simulate random outages of system components.

The problem of how to make investment decisions under long-term uncertainty has been a focus of much recent research in co-optimization. Longer-term uncertainties may be classified as local and global uncertainties [75]. Local uncertainties can be parameterized by probability distributions or uncertainty sets around a point defined by a scenario. Examples of local uncertainties include shifts in load growth and investment costs or fluctuations of fuel prices. In contrast, global uncertainties are those that cause a significant impact on the evolution of the system. Examples of global uncertainties are the implementation of emissions policies, emerging new technologies, dramatic shifts in demand, or public rejection of a certain type of resource. A specific set of realizations on global uncertainties is referred to as a future scenario. In long-term planning, it desirable to define a number of economic, technology, and policy scenarios and then develop plans which are robust and perform acceptably in most or all scenarios.

Stochastic programming is one approach that has been used to for this purpose [10]. In those applications, investments made in the first decade are assumed to be made in ignorance of which of the several scenarios might occur; then in the second decade, recourse investments are made to modify the system with the knowledge of which scenario has occurred. Each scenario is weighted by a probability so that the near-term investments are chosen to minimize probability-weighted costs over the full range of scenarios, accounting for how those investments restricted or expanded the flexibility of the system to adapt later.

Another aspect of planning under uncertainties is to plan for flexibility. Existing planning methods cannot quantify the economic value of flexibility and adaptability of future plans. As an example, some transmission investments might leave more options open than other investments for resource interconnection in the future because the regions they access might have a larger variety of resources. The option value associated with such flexibility can be important in transmission planning, but is not considered by present planning models, whether co-optimized or not. It is useful for co-optimization model formulations to explicitly consider multiple future scenarios, and how future decisions might anticipate or adapt to them.

4.4.2 Anticipatory planning

One of the main results from a co-optimization model, particularly when used by a vertically integrated utility, is the identification of joint transmission-generation expansion plans that are lower in cost than expansion plans when transmission and generation plans were developed separately. However, when the co-optimization model is used by utilities that are unbundled and where planning for transmission infrastructure is performed by one entity while planning for generation investments is performed by another set of competing entities; such scenarios may impose some pragmatic cross-organizational constraints that preclude simultaneous optimization and will necessitate a different co-optimization paradigm called “transmission planning accounting for market response” or “anticipatory transmission planning.” Algorithmically as shown in Fig. 5, this paradigm requires iteratively running generation and transmission expansion models, unlike a simultaneous optimization model. While an iterative approach to co-optimize may reduce computational burden compared to a simultaneous optimization model, it may not converge to a global optimal solution (Figs. 4–41 in [23]).

Fig. 5

Co-optimization paradigms: a simultaneous optimization (left); b iterative anticipatory optimization (right)

Pozo et al. [15, 76] demonstrated the anticipatory transmission planning using a three-level MILP model that integrated transmission planning (level 1), generation investment (level 2), and market operation decisions (level 3). In the first iteration, one can obtain the transmission expansion plans that anticipate both generation investment and dispatch decisions. Subsequently in the next iteration, equilibrium among generation side competitors will be achieved in response to the changes in transmission capacity, access, and congestion. Such a co-optimization paradigm for unbundled market structure will capture how anticipated decisions in transmission could incent changes in generation mix and siting decisions (and consequently the generation dispatch, grid congestion and emissions), and subsequently how transmission planners can account for such anticipated reactions from generation planners in their decision making. Such models will also benefit from anticipating the reactions from other market participants, including distribution or behind-the-meter consumers and their adoption of distributed resources such as solar PV or hybrid electric vehicles [77, 78].

5 Conclusion

This paper has defined the concept of co-optimization of transmission options, supply-side resources, demand-side resources, and fuel supply networks, and contrasted it to more traditional planning methods, which tended to either address generation or transmission planning, but not both. The co-optimization approach promises economic, environmental, and resource utilization related benefits compared to a traditional decoupled approach to resource optimization. A review is provided of several existing and emerging co-optimization models, outlining alternative general approaches to joint optimization of generation and transmission, including their optimizers, modeling fidelity, and data and computational requirements. Building co-optimization models is also a data-intensive task requiring significant effort to collect, maintain and share data with the multiple parties who participate in regional planning processes while maintaining necessary information security and confidentiality.

In conclusion, although various research-grade co-optimization tools already exist, none have all of the features necessary to satisfy the long-term needs of the power system, which have been discussed in detail in this paper. Demonstrations of more advanced co-optimization methods that have been proposed by researchers should be undertaken, using detailed regional data to quantify the benefits of co-optimization in a realistic setting. Such demonstrations will provide more precise estimates of the potential economic benefits of co-optimization, as well as insights on the effort and practical challenges involved in applying co-optimization.

Appendices

Appendix A.1: Full ACOPF-GTEP model

This appendix provides a detailed description of a general ACOPF based multiyear Generation-Transmission Expansion Problem (ACOPF-GTEP), which is a minimization problem defined by the following Eqs. (1)–(12). The objective function is the present worth of operation and annualized investment costs over several years t. The investment variables are the number of new lines that are built in each corridor (integer variables) and new generation capacity (modelled as being continuous). More detailed formulations can include unit commitment costs, maintenance expenses, and environmental damages in the objective; pollution limits, ramping limits, and other operational constraints in the constraint set; and integer variables for generation investments and start-ups. Constraints (2)–(3) model the nodal real and reactive power balances, which are affected by integer transmission expansion variables of candidate corridors. Constraints (4) – (5) model non-linear AC power flow relations across line ij. Constraints (6)–(8) model the network security limits for voltage magnitude, voltage angle and apparent line power flow, the latter bounds being a function of line expansion decisions. Constraints (9)–(10) model generation power bounds, which depend on generation expansion variables. Constraints (11) and (12) model the bounds on the number of investments in transmission lines and generation size respectively within a given investment time period.

$$\begin{aligned} {\mathbf{Minimize}}&T_t \sum \limits _t {\sum \limits _{(i,j)} {C_i (t)P_{gi} (t)} +\sum \limits _t {\sum \limits _i {I_i (t)PI_{gi} (t)} +\sum \limits _t {\sum \limits _{(i,j)} {I_{(i,j)} (t)z_{(i,j)} (t)} } } }\nonumber \\\end{aligned}$$

(1)

$$\begin{aligned} {\mathbf{subject}}\,\mathrm{\mathbf{to}}&P_{gi} (t)-P_{di} (t)=\sum \limits _j {P_{(i,j)} ( {V,\theta ,t})\left( {z_{(i,j)} (0)+\sum \limits _{\tau =start}^t {z_{(i,j)} (\tau )} }\right) ,\forall i}\end{aligned}$$

(2)

$$\begin{aligned}&Q_{gi} (t)-Q_{di} (t)=\sum \limits _j {Q_{(i,j)} ( {V,\theta ,t})\,\left( {z_{(i,j)} (0)+\sum \limits _{\tau =start}^t {z_{(i,j)} (\tau )} }\right) ,\;\forall i}\nonumber \\\end{aligned}$$

(3)

$$\begin{aligned}&P_{(i,j)} (V,\theta ,t)=V_i^2 (t)G_{(i,i)} +\sum \limits _{j=1,j\ne 1}^N {\vert V_i (t)\vert \left| {V_j (t)} \right| } ( G_{(i,j)} cos \theta _{ij}\nonumber \\&\quad \quad \quad \quad \quad \quad \quad \quad +B_{(i,j)} sin \theta _{ij} )\end{aligned}$$

(4)

$$\begin{aligned}&Q_{(i,j)} (V,\theta ,t)\;=\;-V_i ^2( t){}{}B_{(i,i)} +\sum \limits _{j=1,j\ne i}^N {\left| {V_i ( t)} \right| } \left| {V_j ( t)} \right| ( G_{(i,j)} sin\theta _{ij}\nonumber \\&\qquad \qquad \qquad \qquad \qquad -B_{(i,j)} cos\theta _{ij} )\end{aligned}$$

(5)

$$\begin{aligned}&V_i^{min} \le V_i (t)\le V_i^{max}\end{aligned}$$

(6)

$$\begin{aligned}&-\pi \le \theta _i (t)\le +\pi \end{aligned}$$

(7)

$$\begin{aligned}&0\;\le \;P^2_{(i,j)} (t)+Q^2_{(i,j)} (t)\;\le \;S^2_{(i,j)} {}^{max}\,\left( {z_{(i,j)} (0)+\sum \limits _{\tau =start}^t {z_{(i,j)} (\tau )} }\right) \nonumber \\\end{aligned}$$

(8)

$$\begin{aligned}&P_{gi} ^{min}\;\le \;P_{gi} (t)\;\le \;P_{gi}^{max}+\sum \limits _{\tau =start}^t {PI_{gi} (\tau )}\end{aligned}$$

(9)

$$\begin{aligned}&Q_{gi} ^{min}\;\le \;Q_{gi} (t)\;\le \;Q_{gi}^{max}+\beta \sum \limits _{\tau =start}^t {PI_{gi} (\tau )}\end{aligned}$$

(10)

$$\begin{aligned}&0\le z_{(i,j)} (t)\le n_{(i,j)}^{max}\end{aligned}$$

(11)

$$\begin{aligned}&0\le PI_{gi} (t)\le PI_{gi}^{max} \end{aligned}$$

(12)

In this model, t and $\tau$ are time periods, start is the first year in which investments can be made, $T_{t}$ is the number of hours in an individual operating time interval, $C_{i}$ is the operational cost of generation in $/MWh, $I_{i}$ is the investment cost of generation in $/MW, $I_{(i,j)}$ is the investment cost of transmission line in $/MW, $P_{gi}$ is the real power generation in MW (decision variable), $Q_{gi}$ is the reactive power generation in MVar (decision variable), $P_{gi}^{min}$ and $P_{gi}^{max}$ are the minimum and maximum real power generation in MW, $Q_{gi}^{min }$ and $Q_{gi}^{max}$ are the minimum and maximum reactive power generation in MVar, $PI_{gi}$ is the generation capacity investment (continuous decision variable), $PI_{gi}^{max}$ is the maximum allowable generation capacity investment, $z_{(i,j)}(0)$ is number of existing transmission lines in a corridor (i, j),$z_{(i,j)}$ is the transmission investment (integer decision variable), $n_{(i,j)}^{max}$ is the maximum allowable transmission lines across a corridor, $P_{(i,j)}$ is the real power flow across line ij (decision variable), $Q_{(i,j)}$ is the reactive power flow across line ij (decision variable), $S_{(i,j)}^{max}$ is the maximum allowed apparent power flow across line ij, $P_{di}$ is the real power demand, $Q_{di}$ is the reactive power demand, $V_{i}$ is the bus voltage magnitude (decision variable), $V_{i}^{max}$ is the maximum limit on bus voltage magnitude, $\theta _{i}$ is the bus voltage angle (decision variable), $\theta _{ij}$ is the bus voltage angle difference, $G_{(i,j)}$ is the line conductance (Y-bus element), $B_{(i,j)}$ is the line susceptance (Y-bus element), and $\beta$ is the maximum real to reactive power conversion constant at rated voltage based on generator capability curve.

1.1 Relaxed ACOPF-GTEP model using binary variables

Instead of the integer decision variable in (11) for optimizing the total number of lines to be built across a transmission corridor, a binary decision variable can be used to decide if a certain candidate line across a corridor should be built at any time ($s_{b}(t))$ or not? This replaces the integer variable with multi-stage to a variable with two stages (0 or 1), thereby reducing the problem complexity. In such a case, the existing arc (by Eq. (15)) and a candidate arc (by Eq. (16)) are represented individually in the network, and Eqs. (13–18) are used in the model instead of Eqs. (2–11). The presented formulation can be further improvised by giving it the ability to consider investments in multiple lines across a corridor. This can be modeled by designing many candidate arcs across that corridor, each having its own binary decision variable. Though this MINLP formulation will have a larger number of constraints and variables due to the inclusion of many candidate arcs, it will have reduced problem solving complexity due to the removal of multi-stage integer variables.

$$\begin{aligned}&P_{gi} (t)-P_{di} (t)\;=\;\sum \limits _{j} {P_{(i,j)} ( {V,\theta ,t})\;z_{(i,j)} (0)\;+\;P_{(i,j)} ( {V,\theta ,t})\,s_{b_{(i,j)}} (t)}\end{aligned}$$

(13)

$$\begin{aligned}&{Q_{gi}}(t) - {Q_{di}}(t)\; = \;\sum \limits _j {{Q_{(i,j)}}\left( {V,\theta ,t} \right) \;{z_{(i,j)}}(0)\; + \;{Q_{(i,j)}}\left( {V,\theta ,t} \right) \,{s_b}_{(i,j)}(t)}\qquad \quad \end{aligned}$$

(14)

$$\begin{aligned}&0\le P^2_{(i,j)} (t)+Q^2_{(i,j)} (t)\le z_{(i,j)} (0) S^2_{(i,j)}{}^{max}\end{aligned}$$

(15)

$$\begin{aligned}&0\le P^2_{(i,j)} (t)+Q^2_{(i,j)} (t)\le s_{b(i,j)} (t)S^2_{(i,j)}{} ^{max}\end{aligned}$$

(16)

$$\begin{aligned}&0\le z_{(i,j)} (t)\le 1\end{aligned}$$

(17)

$$\begin{aligned}&0\;\le \;s_{b _{(i,j)}} (t)=\sum \limits _{\tau =start}^t {z_{(i,j)} (\tau )} \;\le \;1 \end{aligned}$$

(18)

where, $s_{b(i,j)}(t)$ is a binary decision variable used to decide if a certain candidate line across the corridor (i, j) should be built at any time or not. In the above model, Eqs. (13) and (14), which have binary variables multiplying non-linear power flow equations, can be further relaxed by considering a disjunctive formulation of MINLP problem using the big “M” method [41]. The MINLP or MILP formulation can be further relaxed to NLP or LP problem by assuming the transmission investment variable as continuous, instead of binary. The continuous variable can be constrained close to discrete 0 or 1 value by using a binding constraint relaxed using $\varepsilon$, as shown in (19).

$$\begin{aligned} z_{(i,j)} (t)( {1-z_{(i,j)} (t)})\le \varepsilon \end{aligned}$$

(19)

The AC formulation also allows shunt devices such as MSCs (Mechanically switched capacitors) and SVCs (Static Var Compensators) as investment options. Their influence can be accounted within Eq. (5), which has bus shunt susceptance $b_{i}(t)$ as shown in (20).

$$\begin{aligned} B_{(i,i)} (t)=\sum \limits _j {b_{(i,j)} (t)} +b_i (t) \end{aligned}$$

(20)

where $b_{(i,j)}$ is the line susceptance and $b_{i}$ is the bus shunt susceptance.

Appendix A.2: DC optimal power flow based generation-transmission expansion planning model

The DCOPF formulation is based on the following simplifications to ACOPF model:

• 1. R$<<<$X: The resistance of transmission circuits is significantly less than the reactance.

• 2. Voltage angle differences very small: For typical operating conditions, the difference in voltage angles for two buses is very low (at the max, 10$^{\circ }$–15$^{\circ }$). For smaller angle differences, the cosine function approaches 1.0 and the sine function is the angle difference itself (expressed in radians).

• 3. Voltage magnitudes are assumed to be 1.0 in the per-unit system.³

The resulting power flow model has two equations, a real power flow Eq. (21) which is directly proportional to angle difference (in radians) and reactive power flow Eq. (22) which is directly proportional to bus voltage difference.

$$\begin{aligned}&P_{(i,j)} (\theta ,t)=B_{(i,j)} ( {\theta _i (t)-\theta _j (t)})\end{aligned}$$

(21)

$$\begin{aligned}&Q_{(i,j)} (V,t)=-b_i +\sum \limits _{j=1,j\ne 1}^N {\left| {b_{(i,j)} } \right| \left( {\left| {V_i (t)} \right| -\left| {V_j (t)} \right| }\right) } \end{aligned}$$

(22)

The DCOPF-GTEP problem has the following constraints as shown in Eqs. (23–29) and Eqs. (7, 9, and 12) in Appendix Sect. A.1. It should be noted that in the formulation of (23–29), the transmission investment decision variable is binary, and hence the network expansion problem is formulated using arcs representing existing (by Eqs. (24) and (25)) and candidate lines (by Eqs. (26) and (27)) individually, so if $z_{(i,j)}(t) =$1, this indicates that a certain candidate line across a corridor is to be built at time t.

$$\begin{aligned}&\sum \limits _i {P_{gi} ( t)=\sum \limits _i {P_{di} ( t)} }\end{aligned}$$

(23)

$$\begin{aligned}&P_{(i,j)} (t)=B_{(i,j)} ( {\theta _i (t)-\theta _j (t)})z_{(i,j)} (0)\end{aligned}$$

(24)

$$\begin{aligned}&-z_{(i,j)} (0)P_{(i,j)} ^{max}\le P_{(i,j)} (t)\le P_{(i,j)} ^{max}z_{(i,j)} (0)\end{aligned}$$

(25)

$$\begin{aligned}&P_{(i,j)} (t)=B_{(i,j)} ( {\theta _i (t)-\theta _j (t)})s_{b_{(i,j)} } (t)\end{aligned}$$

(26)

$$\begin{aligned}&-s_{b_{(i,j)} } (t)P_{(i,j)} ^{max}\le P_{(i,j)} (t)\le P_{(i,j)} {}^{max}s_{b_{(i,j)} } (t)\end{aligned}$$

(27)

$$\begin{aligned}&0\le z_{(i,j)} (t)\le 1\end{aligned}$$

(28)

$$\begin{aligned}&0\;\le \;s_{b _{(i,j)}} (t)=\sum \limits _{\tau =start}^t {z_{(i,j)} (\tau )} \;\le \;1 \end{aligned}$$

(29)

The above MINLP model can be relaxed to a MILP using a disjunctive formulation, sometimes referred to as the big “M” method, for candidate branches as shown in (30–32), instead of (26).

$$\begin{aligned}&P_{(i,j)} (t)=B_{(i,j)} ( {\theta _i (t)-\theta _j (t)})+( {s_b (t)-1})M+U_b (t)\end{aligned}$$

(30)

$$\begin{aligned}&U_b (t)\le 2( {1-s_b (t)})M\end{aligned}$$

(31)

$$\begin{aligned}&U_b (t)>0 \end{aligned}$$

(32)

Appendix A.3: network flow based generation-transmission expansion planning model

The network flow model-based linear programming cost minimization formulation is shown in Eqs. (33–36), where the operational arc flows and investments are minimized in (33). Since, both generation and transmission are represented as arcs (refer Fig. 2), i.e., as transportation pipelines (with different properties), the only equation that governs this model is (34), the nodal power flow balance equation. The efficiency term $\eta _{(i,j)}$ in Eq. (34) for a generation arc represents its capacity factor and for a transmission arc its losses. Equation (35) represents the capacity constraint for both generation and transmission arcs.

$$\begin{aligned} {\mathbf{Minimize}}&\sum \limits _t {\sum \limits _{(i,j)} {C_{(i,j)} } (t)P_{(i,j)} (t)+\sum \limits _i {\sum \limits _{(i,j)} {I_{(i,j)} (t)} PI_{(i,j)} } (t)}\end{aligned}$$

(33)

$$\begin{aligned} {\mathbf{Subject}\, \mathbf{to}}&\sum \limits _i {\eta _{(i,j)} (t)P_{(i,j)} (t)-\sum \limits _k {P_{(j,k)} (t)} =d_j (t)}\end{aligned}$$

(34)

$$\begin{aligned}&P_{(i,j)} ^{min}\;\le \;P_{(i,j)} (t)\;\le \;P_{(i,j)} ^{max}+\sum \limits _{\tau =start}^t {PI_{(i,j)} (\tau )}\end{aligned}$$

(35)

$$\begin{aligned}&0\le PI_{(i,j)} (t)\le PI_{(i,j)} ^{max} \end{aligned}$$

(36)

DC lines are modeled as real power injections (positive and negative) at both the ends of the lines, which effectively translate to modeling it as a transportation pipeline. Equation (37) shows the inclusion of power injection from a DC line into nodal real power balance equation. To consider DC lines among the transmission investment options, candidate arcs for DC lines are created separately from AC lines with appropriate cost and operational characteristics. The cost may also include the power electronics component costs at both the terminals.

$$\begin{aligned} P_{gi} (t)-P_{di} (t)\;=\;P_{(i,j)} ( {V,\theta ,t})\;z_{(i,j)} (0)\;+P_{(i,j)} ( {V,\theta ,t})\,s_{b _{(i,j)}} (t)+P_{(i,j)} ^{HVDC}(t) \end{aligned}$$

(37)