1 Introduction

Network revenue management models have traditionally been developed under the independent demand assumption. In the independent demand setting, customers arrive into the system with the intention to purchase a particular product. If this product is available, they purchase it. Otherwise, they leave the system. This model is reasonable when products are well differentiated so that customers do not substitute between products. The independent demand model is harder to justify when there are few differences, other than price, between fares. Indeed, a more general setting is needed when the demand for each product depends heavily on whether or not other products are available for sale. This setting gives the firms the opportunity to shape the demand for each product by adjusting the offer set made available to the customer.

In this chapter, we start by giving a dynamic programming formulation of the network revenue management problem under dependent demands. Similar to the other dynamic programming formulations that we give for network revenue management problems, the state variable in this dynamic program is high dimensional. Thus, solving the dynamic programming formulation is intractable for realistic networks. We show that a large scale deterministic linear program provides an upper bound on the value function. This linear program has one decision variable for each subset of origin-destination-fares (ODF’s) that can be offered to customers. Therefore, the number of decision variables can get large and it is common to solve the linear program by using column generation. We show how to extend some of the heuristics from the independent demand to the dependent demand setting. In particular, we show how to extract heuristic control policies from the solution to the linear program that are similar in spirit to those derived for the independent demand model. As an example, we can use the dual variables to approximate the displacement costs to drive a bid-price heuristic. However, unlike the independent demand model, an assortment problem needs to be solved to determine which ODF’s to offer at any state of the system. As in the independent demand model, we can obtain more refined heuristics by solving single-resource dynamic programs either by netting the fares using the dual variables of the capacity constraint or by allocating the fares among the resources. These refined heuristics often result in tighter upper bounds and better performance. For the basic attraction model (BAM) and the Markov chain (MC) choice model, we show that it is actually possible to reduce the size of the deterministic linear program to roughly the same size as the independent demand model.

In Sect. 7.2, we give a dynamic programming formulation of the network revenue management problem with dependent demands. In Sect. 7.3, we give a deterministic linear program that provides an upper bound on the optimal total expected revenue. In Sect. 7.4, we discuss how to obtain approximate solutions to the dynamic programming formulation by decomposing the formulation by the resources. In Sect. 7.3.2, we show that if the customer choices are governed by the BAM or the MC choice model, then the size of the deterministic linear program can be reduced to the size of the linear program in the independent demand case.

2 Formulations

There are m resources in the network, and the capacities of the resources are given by c = (c 1, …, c m). We let M := {1, …, m} denote the set of resources. As in our work for network revenue management with independent demands, we will use the single index model to capture the ODF’s and the set of ODF’s is N := {1, …, n}. For each ODF j ∈ N, we associate a fare p j and a resource consumption vector A j := (a 1j, …, a mj). We let A be the m × n matrix whose j-th column is A j. The length of the sales horizon is T. Time is measured backwards, so the time-to-go is t = T at the beginning of the sales horizon and t = 0 at the end. Customers arrive over the sales horizon according to a Poisson process with rate {λ t : 0 ≤ t ≤ T}. For economy of notation, we will assume that the choice model is time invariant. However, most of the results are easily extended to the case of time-variant choice models.

To capture our decisions at any time point, we use {u j : j ∈ N}∈{0, 1}n, where u j = 1 if ODF j is offered and u j = 0 otherwise. Given u = (u 1, …, u n), an arriving customer purchases ODF j with probability π j(u). The choice probability π j(u) may be governed by any choice model, including the ones that are discussed in the chapter on introduction to choice modeling.

For any time-to-go t, we let (t, x) to represent the state of the system, where x = (x 1, …, x m) denotes the vector of remaining capacities. The set of feasible decisions are given by \({\mathcal U}(x) := \{ u \in \{0,1\}^n : A_j {\,} u_j \leq x\}\), indicating that we cannot offer ODF’s for which we do not have sufficient capacity.

Let V (t, x) be the maximum total expected revenue that can be collected starting from state (t, x). To compute V (t, x), consider a time increment δt that is small enough so that we can approximate the probability of a customer arrival over this time increment by λ t δt. If the ODF offer decisions are given by u, then a customer arrives with probability λ t δt and is interested in ODF j ∈ N with probability π j(u). For a customer who selects product j, with u j = 1 we collect p j in revenues and provide them A j in resources. With probability 1 −∑j ∈ N π j(u), the arriving customer leaves without purchasing any of the ODF’s. Finally, with probability 1 − λ t δt there are no arrivals. Following this argument, we have the dynamic program

$$\displaystyle \begin{aligned} & V(t,x) = \max_{u \in {\mathcal U}(x)} \Bigg\{ \sum_{j \in N} \lambda_t {\,} \delta t {\,} \pi_{j}(u) \Big[ p_j + V(t - \delta t , x - A_j) \Big] \\ {} & \quad + \Big( 1 - \lambda_t {\,} \delta t+ \lambda_t {\,} \delta t {\,} \Big( 1 - \sum_{j \in N} \pi_{j}(u) \Big) \Big) V(t-\delta t,x) \Bigg\} + o(\delta t) \\ {} & = V(t-\delta t,x) \\ {} & \quad + \max_{u \in {\mathcal U}(x)} \Bigg\{ \sum_{j \in N} \lambda_t {\,} \delta t {\,} \pi_j(u) \Big[ p_j + V(t - \delta t , x - A_j) - V(t-\delta t,x) \Big] \Bigg\} + o(\delta t),\end{aligned} $$

where the equality follows by arranging the terms. Subtracting V (t − δt, x), dividing by δt, and taking the limit as δt → 0, we obtain the Hamilton–Jacobi–Bellman (HJB) equation

$$\displaystyle \begin{aligned} \frac{\partial V(t,x)}{\partial t} = \max_{u \in {\mathcal U}(x)} \sum_{j \in N} \lambda_t {\,} \pi_{j}(u) {\,} ( p_{j} - \varDelta_j V(t , x ) )\end{aligned} $$

with the boundary conditions V (t, 0) = V (0, x) = 0 or all t ≥ 0 and x ≥ 0. Similar to our notation in the chapter on network revenue management with independent demands, Δ j V (t, x) denotes the displacement cost V (t, x) − V (t, x − A j) when x ≥ A j, and we set Δ j V (t, x) = ∞ otherwise.

The right side of the HJB equation is an assortment problem of the type studied in the chapter on assortment optimization. Notice that we can write any assortment either as a subset, say S ⊆ N, or as the incidence vector u ∈{0, 1}n. Thus, when necessary, it is reasonable to abuse the notation and write π j(S) as the probability of selecting j when the set S is offered.

For any vector \(\theta \in \Re ^n_+\), let

$$\displaystyle \begin{aligned}R_t(S,\theta) := \lambda_t \sum_{j \in S} (p_j - \theta_j) {\,} \pi_j(S),\end{aligned} $$

and

$$\displaystyle \begin{aligned}{\mathcal{R}}_t (\theta) := \max_{S \subseteq N} R_t(S,\theta).\end{aligned} $$

We can write the HJB equation as

$$\displaystyle \begin{aligned} \frac{\partial V(t,x)}{\partial t} = {\mathcal{R}}_t(\varDelta V(t,x)) \end{aligned} $$
(7.1)

with boundary conditions V (t, 0) = V (0, x) = 0, where we let ΔV (t, x) = (Δ 1 V (t, x), …, Δ n V (t, x)). Notice that if S ∗ is an optimal assortment to offer at state (t, x) and j ∈ S ∗, then x ≥ A j, because otherwise \(j \notin {\mathcal U}(x)\). We can also handle this by defining Δ j V (t, x) = ∞ whenever \(j \notin {\mathcal U}(x)\) as this avoids the inclusion of j in the assortment.

We close this section by providing a discrete-time dynamic programming formulation that will be useful in numerous places in this chapter. The approach that we use to obtain this discrete-time dynamic programming formulation closely mirrors the one in the chapter with independent demands. In particular, the discrete-time formulation is given by

$$\displaystyle \begin{aligned} V(t,x) = V(t-1,x) + {\mathcal{R}}_{t}(\varDelta V(t-1,x)), \end{aligned} $$
(7.2)

with the boundary conditions V (t, 0) = V (0, x) = 0 for all t ≥ 0 and all x ≥ 0. As before, the discrete-time formulation requires rescaling the arrival rates and the sales horizon so that the probability that two or more customers arrive in a single period is negligible.

Both in the continuous and discrete-time formulations, the state variable (t, x) is a high-dimensional vector and it is difficult to compute the value functions, so we focus on building tractable approximations. For any approximation, say \(\tilde {V}(t,x)\) of V (t, x), there is a natural heuristic, which consists of solving the assortment problem

$$\displaystyle \begin{aligned} {\mathcal{R}}_t (\varDelta \tilde{V}(t,x)) = \max_{S \subseteq N}R_t(S, \varDelta \tilde{V}(t,x)), \end{aligned} $$
(7.3)

for the continuous time formulation (7.1), or solving the assortment problem

$$\displaystyle \begin{aligned} {\mathcal{R}}_{t-1}(\varDelta \tilde{V}(t-1,x)) = \max_{S \subseteq N}R_{t}(S, \varDelta \tilde{V}(t-1,x)), \end{aligned} $$
(7.4)

for the discrete-time formulation (7.2). Thus, for any approximation \(\tilde {V}(t,x)\), solving problem (7.3) or (7.4) yields heuristic policies for the continuous or discrete-time formulation. As a rule of thumb, tighter approximations lead to better heuristics. Thus, we will seek for progressively tighter upper bounds on V (t, x) in our quest for heuristics.

3 Linear Programming-Based Upper Bound on V (T, c)

In this section, we give a linear program to obtain an upper bound \(\bar {V}(T,c)\) on V (T, c). The upper bound is similar in spirit to that obtained for the independent demand model, but requires an exponential number of variables. We show how column generation can be used to solve the linear program for general choice models. The column generation step requires solving an assortment problem. Later in this section, we will present some formulations for discrete choice models that do not require an exponential number of variables.

To formulate the linear program, we let \(\varLambda := \int _0^T \lambda _t {\,} dt\) denote the expected number of customer arrivals over the sales horizon. We will also use the notation

$$\displaystyle \begin{aligned} R(S) := \sum_{j \in S} p_j \pi_j(S)~~\mbox{and}~~A(S) := \sum_{j \in S}A_j \pi_j(S)~~\forall {\,} S \subseteq N. \end{aligned} $$

These quantities represent the gross revenue and the expected consumption rate associated with offering set S.

Let τ(S) be the proportion of customers offered assortment S over the sales horizon. We consider the linear program

$$\displaystyle \begin{aligned}\begin{array}{r*{20}l} \bar{V}(T,c) ~:=~ \max ~~~~ & \varLambda \sum_{S \subseteq N} R(S) {\,} \tau(S) {} \\ \mbox{s.t.} ~~~~ & \varLambda \sum_{S \subseteq N} A(S) {\,} \tau(S) \leq c \\ & ~~\sum_{S \subseteq N} \tau(S) = 1 \\ & ~~~ \tau(S) \geq 0 \qquad \forall {\,} S \subseteq N. \end{array}\end{aligned} $$
(7.5)

The linear program above is commonly known as the choice-based deterministic linear program. The objective function accumulates the total expected revenue over all customers and over all assortments. The first set of constraints ensures that the expected consumptions of the resources do not exceed the capacities. The second constraint ensures that a subset is offered to each customer. In the next theorem, we show that the optimal objective value of the linear program above is an upper bound on the optimal total expected revenue V (T, c).

Theorem 7.1

\(V(T,c) \leq \bar {V}(T,c).\)

The proof of the theorem above is based on constructing a feasible solution to the linear program in (7.5) by using the decisions of the optimal policy.

3.1 Column Generation Procedure

There are 2n decision variables in problem (7.5). Solving this problem directly by using linear programming software can be difficult or impossible if n is large. In practice, problem (7.5) is commonly solved by using column generation. The idea behind column generation is to iteratively solve a master problem that has the same objective function and constraints as problem (7.5), but the master problem includes only a small fraction of the decision variables {τ(S) : S ⊆ N}. Let \(\{ {\hat z}_i : i \in M\}\) and \(\hat {\beta }\) denote, respectively, the optimal dual variables associated with the first and second set of constraints in the master problem. The reduced cost of the decision variable τ(S) is given by \(R_t(S,A'\hat {z}) - \hat {\beta }\). Suppose we can efficiently solve the assortment optimization problem

$$\displaystyle \begin{aligned} {\mathcal{R}}_t(A'\hat{z}) ~=~ \max_{S \subseteq N} R_t(S, A'\hat{z}) ~=~ \max_{S \subseteq N} \sum_{j \in N} \pi_j(S) \left( p_j - \sum_{i \in N} a_{ij} {\,} {\hat z}_i \right) \end{aligned} $$

If \({\mathcal {R}}_t(A'\hat {z}) \leq \hat {\beta }\), then all the reduced costs are non-positive, so there are no decision variables {τ(S) : S ⊆ N} that can improve the current solution to the master problem. Consequently, the optimal solution to the master problem is optimal for problem (7.5) and we can stop. On the other hand, if there is a subset \({\hat S}\) such that \(R_t({\hat S}, A'\hat {z}) - \hat {\beta } > 0\), then adding the column associated with \({\hat S}\) will improve the objective value of the master problem. We add this decision variable to the master problem and resolve the master problem. Consequently, a critical step in the column generation idea is to check whether there is some subset S such that the corresponding reduced cost above is strictly positive.

3.2 Sales-Based Linear Program

The linear program (7.5) involves one decision variable for each subset of ODF’s, corresponding to the frequency of offering each subset of ODF’s. As a result, the number of decision variables can be large and we propose solving the linear program by using column generation. We now show that if the customers choose under the BAM or the MC choice model, then the linear program (7.5) can be reduced to an equivalent linear program with a manageable number of decision variables. This result eliminates the need for using column generation when the customers choose according to the BAM or the MC choice model.

3.2.1 Basic Attraction Model

Under the BAM, a customer associates the attraction value v j with ODF j and the attraction value v 0 with the no-purchase alternative. A customer then selects ODF j ∈ S with probability

$$\displaystyle \begin{aligned} \pi_j(S) = \frac{v_j}{v_0 + V(S)}, \end{aligned} $$

where V (S) =∑j ∈ S v j. Naturally, we have π j(S) = 0 when j∉S. Here, we show that if the customers choose according to the BAM model, then problem (7.5) is equivalent to the linear program

$$\displaystyle \begin{aligned}\begin{array}{r*{20}l} {\tilde V}(T,c) ~=~ \max ~~~\varLambda & \sum_{j \in N} p_j {\,} x_j {} \\ {} \mbox{s.t.} ~~~~ \varLambda & \sum_{j \in N} a_{ij} {\,} x_j \leq c_i \qquad && \forall {\,} i \in M \\ {} & \sum_{j \in N} x_j + x_0 = 1 \\ {} & \frac{x_j}{v_j} \leq \frac{x_0}{v_0} && \forall {\,} j \in N \\ {} & x_j, x_0 \geq 0 && \forall {\,} j \in N. \end{array}\end{aligned} $$
(7.6)

In the problem above, we interpret the decision variable x j as the fraction of customers that purchase ODF j, whereas we interpret the decision variable x 0 as the fraction of customers that leave without making a purchase. The objective function accumulates the total expected revenue over the sales horizon. The first set of constraints ensures that the expected capacity consumption of each resource does not violate the capacity available of the resource. The second constraint ensures that each customer either purchases an ODF or leaves without a purchase. The third set of constraints scales the purchase probability of each ODF and the no-purchase probability to ensure that they are consistent with the BAM. Note that the parameters of the BAM only appear in these constraints.

We emphasize that the decision variables in problem (7.6) are the fractions of customers that purchase different ODF’s, rather than the probabilities of offering different subsets of ODF’s. In other words, the sales for different ODF’s are decision variables in problem (7.6). For this reason, problem (7.6) is sometimes referred to as the sales-based linear program. The appealing aspect of problem (7.6) is that this problem has n + 1 decision variables, whereas problem (7.5) has 2n decision variables. Thus, problem (7.6) can be solved directly through linear programming software without using column generation, even for relatively large-scale airline networks.

In the next theorem, we establish that problem (7.6) is equivalent to problem (7.5). Note that \({\tilde V}(T,c)\) corresponds to the optimal objective value of problem (7.6).

Theorem 7.2

\({\tilde V}(T,c) = {\bar V}(T,c)\).

One of the useful aspects of the proof of Theorem 7.2 is that it shows how to recover an optimal solution to problem (7.5) by using an optimal solution to problem (7.6). This is helpful if primal-based heuristics are to be used. As will be discussed later in this chapter, the dual variables \(z^* =\{ z_i^* : i \in M\}\) of the capacity constraints can be used to develop bid-price heuristics. The dual variables of the capacity constraints in the sales-based linear program and in the original linear program match.

3.2.2 Markov Chain Choice Model

We now provide a compact formulation of problem (7.5) for the MC choice model. Under the MC choice model, a customer arriving into the system is interested in ODF j with probability γ j. If this ODF is available, then the customer purchases it. Otherwise, the customer transitions from ODF j to ODF k with probability ρ jk. In this way, the customer transitions between the ODF’s, until she reaches an ODF that is offered or she reaches the no-purchase option. In this section, we show that if the customers choose according to the MC choice model, then problem (7.5) is equivalent to the linear program

$$\displaystyle \begin{aligned}\begin{array}{r*{20}l} {\tilde V}(T,c) ~=~ \max ~~~ \varLambda & \sum_{j \in N} p_j {\,} x_j {} \\ \mbox{s.t.} ~~~~ \varLambda & \sum_{j \in N} a_{ij} {\,} x_j \leq c_i \qquad && \forall {\,} i \in M \\ & x_j + y_j = \gamma_j + \sum_{k \in N}\rho_{kj} {\,} y_k \quad && \forall {\,} j \in N \\ & x_j, y_j \geq 0 && \forall {\,} j \in N. \end{array}\end{aligned} $$
(7.7)

In the problem above, we interpret the decision variable x j as the fraction of customers that consider purchasing product j during the course of their choice process and purchase this product. On the other hand, we interpret the decision variable y j as the fraction of customers that consider purchase product j during the course of their choice process but do not purchase this product because of the unavailability of this product. The objective function accounts for the total expected revenue over the sales horizon. The first set of constraints ensures that the expected capacity consumption of each resource does not violate the capacity of the resource. The second set of constraints can be interpreted as a balance constraint. Noting the definitions of the decision variables x j and y j, x j + y j corresponds to the fraction of customers that consider product j during the course of their choice process. For a customer to consider product j, either she should arrive with the intention of purchasing product j or she should consider purchasing some product i, not purchase this product and transition from product i to product j.

In the next theorem, we establish that problem (7.7) is equivalent to problem (7.5) when customers choose according to the MC choice model.

Theorem 7.3

\({\tilde V}(T,c) = {\bar V}(T,c)\).

By Theorem 7.3, problems (7.5) and (7.7) have the same optimal objective value. It is straightforward to show that the optimal values of the dual variables associated with the first set of constraints in the two problems are the same. Thus, letting \(z^* =\{ z_i^* : i \in M\}\) be the optimal values of the dual variables associated with the first set of constraints in problem (7.7), we can use these dual variables as input to the heuristics we will discuss next.

3.3 Heuristics Based on the Linear Program

We can use the solution to the linear program (7.5) to obtain heuristic control policies. One approach for doing this is to use the primal solution to problem (7.5). In this heuristic, letting {τ ∗(S) : S ⊆ N} be an optimal solution to problem (7.5), we offer assortment S to a random fraction τ ∗(S) of customers, subject to capacity availability. Another approach is to use the dual solution to problem (7.5). In particular, letting \(z^* = (z_1^*,\ldots ,z_m^*)\) be the optimal values of the dual variables associated with the first set of constraints, the idea is to use \(z_i^*\) as an opportunity cost of resource i. This is equivalent to using the approximate value function \(\tilde V(t,x) = x'z^*\) and results in \(\tilde V(t,x) - \tilde V(t,x - A_j) = A^{\prime }_j z^*\). Consequently, the continuous time heuristic (7.3) solves the assortment problem \({\mathcal {R}}_t(A'z^*) = \max _{S \subseteq N}R_t(S,A'z^*)\), and the discrete-time heuristic (7.4) solves the assortment problem \({\mathcal {R}}_{t-1}(A'z^*) = \max _{S \subseteq N}R_{t-1}(S,A'z^*)\). Notice that unlike the independent demand model, it is not necessarily optimal to offer all ODF’s j with non-negative net contributions \(p_j - A^{\prime }_jz^*\).

Example 7.4

Consider a network with two connecting resources and assume that two fares are available for each origin-destination pair. Thus, there are six ODF’s with capacity consumption vectors

$$\displaystyle \begin{aligned} A_1 = A_2 = \begin{pmatrix} 1 \\ 0 \end{pmatrix} \quad A_3 = A_4 = \begin{pmatrix} 0 \\ 1 \end{pmatrix} \quad A_5 = A_6 = \begin{pmatrix} 1 \\ 1 \end{pmatrix}. \end{aligned} $$

The fares for the different ODF’s are given by

$$\displaystyle \begin{aligned}(p_1,\ldots,p_6) = (30, 150, 60, 120, 100, 200). \end{aligned}$$

The length of the sales horizon is T = 100. The customer arrival rate is constant at λ t = 1 for 0 ≤ t ≤ T. The capacities on the resources are c = (30, 30). To specify the choice process of the customers, we assume that there are three classes of customers. Customers of class 1 are interested in ODF’s 1 and 2, customers of class 2 are interested in ODF’s 3 and 4, and customers of class 3 are interested in ODF’s 5 and 6. An arriving customer is of class 1, 2, and 3, respectively, with probabilities 0.3, 0.3, and 0.4. The probability that a customer of each class chooses a particular ODF as a function of the subset of ODF’s offered to this customer is given in Table 7.1. For example, if both ODF’s 1 and 2 are available, then a customer of class 1 chooses ODF 1 with probability 0.64 and ODF 2 with probability 0.03. With probability 0.33, this customer leaves without a purchase. For this problem instance, the optimal objective value of problem (7.5) is $5740. Thus, the total expected revenue obtained by the optimal policy is upper bounded by $5740. The dual solution is given by z ∗ = (18, 120).

Table 7.1 Choice probabilities for the three customer classes
Full size table

Given this vector of dual variables, we can solve the assortment problem for each of the three customer classes. For example, the net fares for customer class one are \(p_1 - z^*_1 = 12\) and \(p_2 - z^*_1 = 132\). Given this net fares, it is optimal to offer only ODF 2 to this customer class. For customer class two, all ODF’s have non-positive contribution, so it is not optimal to offer ODF 3 or 4. Finally, for customer class three, the net fares are \(p_5 - z^*_1 -z^*_2 = -38\) and \(p_6 - z^*_1 - z^*_2 = 62\), so it is optimal to offer only ODF 6. In summary, the bid-price heuristic offers ODF’s 2 and 6 only, even if ODF 1 has a positive net contribution.

In practice, the performance of the policy obtained from problem (7.5) tends to improve if this problem is periodically resolved over the sales horizon. In particular, we can resolve the linear program (7.5) at state (t, x) after replacing Λ with \(\int _0^t \lambda _sds\) and c i with x i for all i ∈ M. If we want to use the policy from the primal solution, then letting {τ ∗(S) : S ⊆ N} be the optimal solution, we offer the subset S of ODF’s with probability τ ∗(S), subject to capacity availability. If, on the other hand, we want to use the policy from the dual solution, then letting \(z^* = (z_1^*,\ldots ,z_m^*)\) be the optimal values of the dual variables associated with the capacity constraints, we offer the assortment that maximizes R t(S, A′z ∗).

Similar to our analysis for network revenue management problems with independent demands, it is possible to show that several policies extracted from the linear program (7.5) are asymptotically optimal.

4 Dynamic Programming Decomposition

In this section, we describe two approaches for obtaining tractable approximations to the value functions in the dynamic programming formulation of the network revenue management problem. These approaches are based on decomposing the network revenue management problem by resources. To make our exposition simpler, we work with the discrete-time dynamic program given in (7.2).

4.1 Exploiting the Deterministic Linear Program

In the chapter on network revenue management with independent demands, we showed how we can leverage the linear programming approximation to decompose the dynamic programming formulation of the network revenue management problem by the resources. In this section, we extend the idea to the dependent demand setting. Assume that we solve problem (7.5) and let \((z_1^*,\ldots ,z_m^*)\) be the optimal values of the dual variables associated with the first set of constraints. We choose an arbitrary resource i and relax the first set of constraints for all other resources by associating the dual multipliers \(z^*_k, k \neq i\) with them. Thus, the objective function of problem (7.5) reads

$$\displaystyle \begin{aligned} \begin{array}{rcl} &\displaystyle &\displaystyle \varLambda \sum_{S \subseteq N} \sum_{j \in N} p_j {\,} \pi_j(S) {\,} \tau(S) + \sum_{k \neq i} z_k^* {\,} \Big[ c_k - \varLambda \sum_{S\subseteq N} \sum_{j \in N} a_{kj} {\,} \pi_j(S) {\,} \tau(S) \Big] \\ &\displaystyle &\displaystyle \qquad \qquad \qquad \qquad = \varLambda \sum_{S \subseteq N} \sum_{j \in N} \Big[ p_j - \sum_{k \neq i} a_{kj} {\,} z_k^* \Big] \pi_j(S) {\,} \tau(S) + \sum_{k \neq i} c_k {\,} z_k^*. \end{array} \end{aligned} $$

Since we relax the first set of constraints by using the optimal values of the dual variables associated with these constraints as multipliers, problem (7.5) has the same optimal objective value as the problem

$$\displaystyle \begin{aligned}\begin{array}{r*{20}l} \bar V(T,c) ~~~=~~~ \max ~~~~~ & \varLambda \sum_{S \subseteq N} \sum_{j \in N} \Big[ p_j - \sum_{k \neq i} a_{kj} {\,} z_k^* \Big] \pi_j(S) {\,} \tau(S) + \sum_{k \neq i} c_k {\,} z_k^* \\ \mbox{s.t.} ~~~~ & \varLambda \sum_{S\subseteq N} \sum_{j \in N} a_{ij} {\,} \pi_j(S) {\,} \tau(S) \leq c_i \qquad \\ & ~~\sum_{S \subseteq N} \tau(S) = 1 \\ & ~~~ \tau(S) \geq 0 ~~ \qquad \forall {\,} S \subseteq N. \end{array}\end{aligned} $$

If we ignore for the moment the constant term \(\sum _{k \neq i} c_k {\,} z_k^*\) in the objective function, then the problem above is the linear programming approximation for a single-resource revenue management problem that takes place over resource i only. In this single-resource problem, product j generates net revenue \(p_j(z^*) = p_j - \sum _{k \neq i} a_{kj} {\,} z_k^* \). By Theorem 7.1, the optimal objective value of the linear programming approximation provides an upper bound on the optimal total expected revenue. Therefore, \(\bar V(T,c) - \sum _{k \neq i} c_k {\,} z_k^*\) is an upper bound on the optimal total expected revenue in the single-resource revenue management problem that takes place over resource i.

We can solve a dynamic program to compute the optimal total expected revenue in the single-resource revenue management problem that takes place over resource i with revenues p j(z ∗) for each ODF j ∈ N. Let v i(t, x i) denote the optimal expected revenue for this dynamic program for resource i when the time-to-go is t and the remaining inventory of resource i is x i. Then, the discrete time version of the dynamic program is given by

$$\displaystyle \begin{aligned} v_i(t,x_i) = v_i(t-1,x_i) + \lambda_t \max_{S \in {\mathcal U}_i(x_i)} \sum_{j \in N} {\,} \pi_j(S) {\,} ( p_j(z^*) - \varDelta_jv_i(t-1,x_i) ) {} \end{aligned} $$
(7.8)

with boundary condition v i(0, x i) = v i(t, 0) = 0, where Δ j v i(t, x i) = v i(t, x i) − v i(t, x i − a ij) if x i ≥ a ij and Δ j v i(t, x i) = ∞ otherwise. In the dynamic program (7.8), \({\mathcal U}_i(x_i) = \{ u \in \{0,1\}^n: a_{ij}u_j \leq x_i ~~\forall {\,} j \in N\}\) corresponds to the set of feasible subsets of ODF’s to offer given that the remaining capacity on resource i is x i. By the discussion at the end of the previous paragraph, \(v_i(T,c_i) \leq \bar V(T,c) - \sum _{k \neq i} c_k {\,} z_k^*\).

On the other hand, by using induction over time, it is possible to show that \(v_i(t,x_i) + \sum _{k \neq i} z_k^* {\,} x_k\) provides an upper bound on the exact value function V (t, x). The proof of this result is identical to the corresponding result under independent demands. Computing this upper bound at the beginning of the sales horizon with the initial capacities, we obtain

$$\displaystyle \begin{aligned} V(T,c) \leq \min_{i \in M} \Big\{v_i(T,c_i) + \sum_{k \neq i} z_k^* {\,} c_k \Big\} \leq \bar V(T,c). \end{aligned} $$

By solving the single-resource dynamic program for all choices of i ∈ M, we can approximate V (t, x) by v(t, x) =∑i ∈ M v i(t, x i). This allow us to approximate Δ j V (t, x) by Δ j v(t, x) =∑i ∈ M[v i(t, x i) − v i(t, x i − a ij)] on the right side of (7.8). This is equivalent to solving the assortment problem maxS⊆N R t(S, Δv(t, x)) corresponding to the continuous time heuristic (7.3) with a similar counterpart for the discrete-time heuristic (7.4).

Example 7.5

For the problem instance in Example 7.4, we compute the value functions {v i(t, ⋅) : t = 1, …, T} by using the dynamic program in (7.8). Thus, we can obtain an upper bound on the optimal total expected revenue by using \(\min _{i \in M} \{ v_i(T,c_i) + \sum _{k \neq i} z_k^* {\,} c_k \}\) = $5622, which tightens the upper bound of $5740 provided by the linear program in (7.5).

4.2 Decomposition by Fare Allocation

In this section, we provide an alternative approach for obtaining approximate solutions to the dynamic programming formulation of the network revenue management problem. The main idea behind this approach is to allocate the fare of an ODF over the different resources. Once we allocate the fare of each ODF over the different resources, we solve a single-resource revenue management problem for each resource in the network with the allocated fares. We show that the sum of the value functions over all of the resources is an upper bound on the optimal total expected revenue. We convert the bound into a heuristic by using the bound as an approximation to the value function. The approach that we develop in this section can be viewed as an analogue of Lagrangian relaxation under independent demands.

We use α ij to denote the fare allocation of ODF j over resource i. We do not yet specify how the fare allocations are chosen. For the moment, we only assume that ∑i ∈ M α ij = p j, so that the total fare allocation of ODF j over all resources is equal to the original fare for ODF j. Using the fare allocations α = {α ij : i ∈ M, j ∈ N}, we solve a single-resource revenue management problem for each resource i ∈ M. The dynamic programming formulation of the single-resource revenue management problem that takes place over resource i is given by

$$\displaystyle \begin{aligned} v_i^\alpha(t,x_i) = v_i^\alpha (t-1,x_i) + \lambda_t {\,} \max_{u \in {\mathcal U}_i(x_i)} \sum_{j \in N} \pi_j(u) {\,} (\alpha_{ij} - \varDelta_jv_i^\alpha(t-1,x_i )), {} \end{aligned} $$
(7.9)

where \(\varDelta _jv^{\alpha }(t,x_i) = v^{\alpha }_i(t,x_i) - v^{\alpha }_i(t,x_i - a_{ij})\) if x i ≥ a ij and \(\varDelta _j v^{\alpha }_i(t,x_i) = \infty \) otherwise. The superscript α in the value functions emphasizes that the value functions obtained from this dynamic program depends on our choice of the fare allocations. In the single-resource revenue management problem above, which takes place over resource i, if we sell ODF j, then the revenue that we obtain is α ij. In the next theorem, we show that we can obtain upper bounds on the exact value functions by solving the dynamic program above for all resources.

Theorem 7.6

If the fare allocations α = {α ij : i ∈ M , j ∈ N} satisfy ∑i ∈ M α ij = p j for all j ∈ N, then \(V(t,x) \leq \sum _{i \in M} v_i^\alpha (t,x_i)\).

By Theorem 7.6, for any fare allocations α = {α ij : i ∈ M, j ∈ N} that satisfy ∑i ∈ M α ij = p j for all j ∈ N, \(\sum _{i \in M} v_i^\alpha (t,x_i)\) provides an upper bound on the exact value function V (t, x). Computing this upper bound with the initial capacity at the beginning of the sales horizon, it follows that \(\sum _{i \in M} v_i^\alpha (T,c_i)\) provides an upper bound on the optimal total expected revenue. To obtain the tightest possible upper bound on the optimal total expected revenue, we can solve the problem

$$\displaystyle \begin{aligned} \min ~~~ & ~~~ \sum_{i \in M} v_i^\alpha(T,c_i) {} \\ \mbox{s.t.}~~~ & ~~~ \sum_{i \in M} \alpha_{ij} = p_j \qquad \forall {\,} j \in N, \end{aligned} $$
(7.10)

where the decision variables are the fare allocations α = {α ij : i ∈ M, j ∈ N}. It is possible to show that \(v_i^\alpha (T,c_i)\) is a convex function of α. Thus, the problem above has a convex objective function and linear constraints, so we can solve this problem using standard convex optimization methods. We can use problem (7.10) not only to obtain the tightest possible upper bound on the optimal expected revenue, but also to choose the fare allocations. Let α ∗ be the optimal solution to problem (7.10), and assume we solve the dynamic program (7.9) by using these fare allocations. Then, we can replace Δ j V (t, x) with \(\varDelta _j v^{\alpha ^*}(t,x) = \sum _{i \in M} [v_i^{\alpha ^*}(t,x_i) - v_i^{\alpha ^*}(t,x_i - a_{ij})]\) to solve the assortment problem (7.3) or (7.4) to pick the assortment of ODF’s to offer.

Example 7.7

Consider the problem instance in Example 7.4. We solve problem (7.10) to obtain the optimal solution α ∗. Then, \(\sum _{i \in M} v_i^{\alpha ^*}(T,c_i)\) provides an upper bound on the optimal total expected revenue. This upper bound comes out to be $5606. This is the tightest upper bound among the upper bounds provided by the different solution methods in this chapter.

We described two approaches to obtain approximate solutions to the dynamic programming formulation. Both approaches are based on solving a sequence of single-resource revenue management problems. The first approach exploits the linear program, whereas the second approach is based on allocating the fare of an ODF over different resources. Both approaches can benefit from periodic resolving. For the first approach, if we are at time t with remaining capacities x, then we can solve the linear program in (7.5) after replacing Λ with \(\int _0^t \lambda _s {\,} ds\) and c i with x i. Let \(z^* = (z_1^*,\ldots ,z_m^*)\) be the optimal values of the dual variables associated with the first set of constraints. Using these dual variables in (7.8), we can obtain a heuristic solution to the dynamic programming formulation by approximating V (t, x) by ∑i ∈ M v i(t, x i) until the time point where we carry out the next periodic resolve. For the second approach, if we are at time t with remaining capacities x, then we solve a variant of problem (7.10), where we minimize \(\sum _{i \in M} v_i^\alpha (t,x)\) subject to the constraint that ∑i ∈ M α ij = p j for all j ∈ N. Letting α ∗ be the optimal solution to this problem, we solve the dynamic program in (7.9) with these fare allocations. We then use \(\sum _{i \in M} v_i^{\alpha ^*}(t,x_i)\) as an approximation to V (t, x) until the time point where we carry out the next periodic resolve.

Example 7.8

Table 7.2 provides the total expected revenues obtained by the different solution strategies discussed in this chapter, namely the bid-price heuristic based on linear programming and the two decomposition methods: (a) by netting fares via the dual of the linear program and (b) by selecting a fare allocation. We use 1, 4, and 10 resolves over the sales horizon. The first line corresponds to the bid-price policy obtained from the dual solution to the linear program in (7.5). The second line corresponds to the value function approximation approach that exploits the linear program. The third line corresponds to decomposition by fare allocation. The performance of the bid-price policy improves significantly when we resolve the linear program. On the other hand, the value function approximation methods can provide good revenues without resolving too many times. Furthermore, Example 7.7 shows that the optimal total expected revenue is no larger than $5606. The largest total expected revenue in Table 7.2 is quite close to $5606, indicating that the performance of the best policy is near-optimal.

Table 7.2 Performance of the solution methods considered in this chapter
Full size table

As mentioned at the beginning of this chapter, almost every revenue management model built under independent demands has an analogue under dependent demands. So far, we focused our attention on the linear programming formulation and two approaches that approximate the value functions by decomposing the dynamic programming formulation by the resources. There are other solution approaches under dependent demands. For example, we can generalize the extended linear programming formulation under independent demands to cover dependent demands. To see how this works, we discretize time so that there is at most one customer arrival at each time period. Assuming that there are T time periods in the selling horizon, we use the decision variable α t(S) to capture the probability of offering the subset S of ODF’s at time period t. Assuming there is a customer arrival at time period t with probability λ t, it is possible to show that the optimal objective value of the linear program

(7.11)

provides an upper bound on the optimal total expected revenue. In the linear program above, we have \(\varTheta _i(S) = \max \{ a_{ij} : i \in M,~j \in S \}\), corresponding to the maximum capacity consumption on resource i by one of the ODF’s in the subset S. In addition to obtaining upper bounds on the optimal total expected revenue, if we let \(z_t^* = \{z_{it}^* : i \in M,~t=1,\ldots ,T\}\) be the optimal values of the dual variables associated with the first two sets of constraints, then we can use \(\sum _{i \in M} z_{it}^* {\,} x_i\) as an approximation to the value function V (t, x). In this case, we can replace ΔV (t, x) with \(A'z^*_t\) to decide which subset of ODF’s to make available at state (t, x).

Last, it is possible to show that a heuristics solution based on the primal solution where assortment S is offered with probability τ(S) is asymptotically optimal. However, all of the heuristics presented here have better performance particularly when the number of times the heuristic is updated during the sales horizon is large.

4.3 Overbooking

Here, we consider a sales-based network revenue management model that allows for overbooking under the MC choice model. We assume a discrete-time model over T time periods, where λ t is the probability of a customer arrival at time period t. A customer that arrives at time-to-go t makes a choice among the offered ODF’s according to the MC choice model with first choice probabilities {γ tj : j ∈ N} and transition probabilities \(\{ \rho ^t_{ij}: i, j \in N \}\). If we make a sale for ODF j at time-to-go t, then this sale is not canceled by the departure time, i.e. retained, with probability Q tj. The net expected revenue obtained for each booking of ODF j at time-to-go t is p tj. As an example, if a refund of c with 0 ≤ c ≤ p j is given to bookings that do not survive until the departure time, then we have p jt = p j − c(1 − Q jt). Finally, we let θ j to be the overbooking cost for each unit of ODF j that we cannot accommodate at the departure time. We use the decision variable x tj to capture the probability that a customer arriving at time period t visits ODF j during the course of her choice process and purchases this ODF, whereas we use the decision variable y tj to capture the probability that a customer arriving at time period t visits ODF j during the course of her choice process and does not purchases this ODF due to unavailability. Also, we use the decision variable z j to capture the number of ODF j bookings that are denied during boarding. In this case, we can obtain an upper bound on the optimal total expected profit by using the optimal objective value of the linear program

5 End of Chapter Problems

  1. 1.

    An airline operates a flight network among three locations A, B, and C. There are two flights. The first flight is from A to B, and the second flight is from B to C. There are six ODF’s. The fares associated with the ODF’s are given in Table 7.3. A customer interested in flying between a certain origin-destination pair chooses among the two ODF’s that serve that destination pair according to the basic attraction model. The attraction values of the six ODF’s are, respectively, 5, 3, 1, 0.5, 6, and 2. The attraction value of the no-purchase option is always 1. For example, a customer interested in going from location A to B chooses among the first two ODF’s according to the basic attraction model with the attraction values 5 and 3, with the attraction value of the no-purchase option being 1. Assume that we are at time period t and the remaining capacities on the flights are (1, 1). The value function at the next time period is given in Table 7.4. For example, if the remaining capacities at the next time period are given by (1, 0), then the value function takes value 25.

    1. (a)

      For each ODF, decide whether it is optimal to open or close it.

    2. (b)

      At time period t, it is equally likely to observe a customer wanting to go from each origin to each destination, with each probability being 1∕3. If the capacities at time period t are given by x = (1, 1), then compute the value function at time period t at the current capacities. That is, compute V (t, x) at x = (1, 1).

  2. 2.

    In this problem, you will show that a policy obtained from problem (7.5) is asymptotically optimal. Consider the revenue management problem discussed in Sect. 7.2 but under the assumption that there is a single resource so c is a scalar and the arrival rates of the customers are stationary so λ t = λ for all t ∈ [0, T] for some fixed λ. Let {τ ∗(S) : S ⊆ N} be an optimal solution to problem (7.5). Consider a heuristic policy where we offer subset S with probability τ ∗(S) at each time point, as long as we have capacity on the resource. As soon as we run out of capacity of the resource, we stop offering any products.

    Let H k be the total expected revenue obtained by the heuristic policy above when the capacity of the resource is kc and the length of the selling horizon is kT. Similarly, let V k be the total expected revenue obtained by the optimal policy. Show that

    $$\displaystyle \begin{aligned} \lim_{k \rightarrow \infty} \frac{H^k}{V^k} = 1. \end{aligned} $$

    (Hint: Follow an approach similar to the one that we used in the asymptotic optimality result under the independent demand model.)

  3. 3.

    Show that the optimal objective value of problem (7.11) is an upper bound on the optimal total expected revenue.

  4. 4.

    Consider the random consideration set model discussed in Chap. 2.

    1. (a)

      Show that this choice model is a special case of the Markov chain choice model. In other words, assume that we are given a random consideration set model with n products, where the products are labeled such that 1 ≺ 2 ≺… ≺ n and the attention probabilities are {β i : i = 1, …, n}. Show that we can construct a single Markov chain choice model such that the choice probability for each product out of each assortment is the same under the two choice models.

    2. (b)

      Building on Part a gives a sales-based linear programming formulation under the random consideration set model.

  5. 5.

    We want to solve an assortment optimization problem when customers choose according to the basic attraction model, each product occupies a certain amount of space and we want to ensure that the total amount of space occupied by the offered assortment does not violate the space availability in the store. There are n products indexed by 1, …, n. The revenue and the attraction value associated with product j are, respectively, r j and v j. The attraction value of the no-purchase option is normalized to 1. Product j occupies c j units of space. The total space consumed by the offered assortment cannot exceed C. Show that we can obtain an upper bound on the optimal expected revenue from a customer by using the optimal objective value of the linear program

    $$\displaystyle \begin{aligned} \max ~~~ & \sum_{j=1}^n r_j {\,} x_j \\ \mbox{s.t.} ~~~ & \sum_{j=1}^n x_j + x_0 = 1 \\ & \frac{x_j}{v_j} \leq x_0 \qquad \quad ~~ \forall {\,} j =1,\ldots,n \\ & \sum_{j =1}^n c_j {\,} \frac{x_j}{v_j} \leq C {\,} x_0 \\ & x_j \geq 0,~ x_0 \geq 0 \quad \forall {\,} j=1,\ldots,n. \end{aligned} $$
Table 7.3 ODF’s and fares
Full size table
Table 7.4 Value function at the next time period
Full size table

6 Bibliographical Remarks

The linear program in (7.5) is from Gallego et al. (2004). The authors show that the optimal objective value of this linear program provides an upper bound on the optimal total expected revenue. Liu and van Ryzin (2008a) show that a heuristic resulting from the linear program is asymptotically optimal. Balseiro et al. (2010) use the linear program for pricing tickets in tournaments, where the tickets are sold before knowing which teams will advance to the final. Jasin and Kumar (2012) show that resolving the linear program yields bounded revenue loss. Bront et al. (2009) study the same linear program when there are multiple customer segments choosing according to the BAM with different parameters and show that the column generation subproblem is NP-hard. Meissner et al. (2013), Talluri (2014) and Strauss and Talluri (2017) give tractable relaxations of the same linear program that avoids an NP-hard column generation subproblem.

Zhang and Adelman (2009) extend the approximate dynamic programming approach described for independent demands to the dependent demand case. The authors of that paper use linear value function approximations. Kunnumkal and Talluri (2015b) also construct linear approximations to the value functions. Zhang (2011) gives an approach to obtain nonseparable approximations to the value functions. Kunnumkal and Topaloglu (2011a) and Kunnumkal (2014) give variants of the linear programming formulation that are directed towards capturing the randomness in customer choice process more accurately. Kunnumkal and Topaloglu (2008) and Zhang and Vossen (2015) generalize the extended linear programming formulation that we discussed for the independent demands to cover the case with dependent demands.

The idea behind the dynamic programming decomposition method described in Sect. 7.4.1 rests on Liu and van Ryzin (2008a). Zhang and Adelman (2009) establish that this idea provides upper bounds on the exact value functions. The dynamic programming decomposition approach in Sect. 7.4.2 is based on Kunnumkal and Topaloglu (2010a). Kunnumkal and Talluri (2015a) give a dynamic programming decomposition approach that provides tighter upper bounds on the optimal total expected revenue. The sales-based linear program under the BAM appears in Gallego et al. (2015), whereas the sales-based linear program under the MC choice model appears in Feldman and Topaloglu (2017). The papers by van Ryzin and Vulcano (2008b) and Chaneton and Vulcano (2011) use stochastic approximation methods to tune bid-price and protection-level policies under customer choice behavior. Vulcano et al. (2010) and Dai et al. (2015) discuss applications of choice-based network revenue management in practice.

There are several other models for network revenue management problems with dependent demands. Zhang and Cooper (2005) consider the case where customers choose among single-resource ODF’s that operate between the same origin-destination pair. They show how to obtain upper bounds on the value functions in the dynamic programming formulation of the problem. Golrezaei et al. (2014), Bernstein et al. (2015), Gallego et al. (2016b) and Chen et al. (2016d) consider related problems arising in the retail and resource allocation settings, where the firm dynamically adjusts the assortment of products or resources offered to its customers.

Another related class of problems occur when the firm chooses the assortment of products to offer to the customers, along with their initial inventory levels and the customers arriving over time make a choice among the offered products. For this stream of literature, we point the reader to van Ryzin and Mahajan (1999), Mahajan and van Ryzin (2001), Gaur and Honhon (2006), Honhon et al. (2010), Topaloglu (2013), Honhon and Seshadri (2013), Goyal et al. (2016), Aouad et al. (2018c), Segev (2019) and Aouad et al. (2019).