Applications to Production and Inventory

Abstract

Applications of optimization methods to production and inventory problems date back at least to the classical EOQ (Economic Order Quantity) model or the lot size formula of Harris (1913) . The EOQ is essentially a static model in the sense that the demand is constant and only a stationary solution is sought. A dynamic version of the lot size model was analyzed by Wagner and Whitin (1958) . The solution methodology used there was dynamic programming.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Applications of optimization methods to production and inventory problems date back at least to the classical EOQ (Economic Order Quantity) model or the lot size formula of Harris (1913) . The EOQ is essentially a static model in the sense that the demand is constant and only a stationary solution is sought. A dynamic version of the lot size model was analyzed by Wagner and Whitin (1958) . The solution methodology used there was dynamic programming.

An important dynamic production planning model was developed by Holt et al. (1960) . In their model, referred to as the HMMS model , they considered both production costs and inventory holding costs over time. They used calculus of variations techniques to solve the continuous-time version of their model. In Sect. 6.1, a model of Thompson and Sethi (1980) , similar to the HMMS model, is formulated and completely solved using optimal control theory. The turnpike solution is also obtained when the horizon is infinite.

In Sect. 6.2, we introduce the wheat trading model of Ijiri and Thompson (1970) , in which a wheat speculator must buy and sell wheat in an optimal way in order to take advantage of changes in the price of wheat over time. In Sects. 6.2.1–6.2.3, we solve the model when the short-selling of wheat is allowed. In Sect. 6.2.4, we follow Norström (1978) to solve a simple example that disallows short-selling.

In Sect. 6.3, we introduce a warehousing constraint, i.e., an upper bound on the amount of wheat that can be stored, in the wheat trading model. In addition to being realistic, the introduction of the warehousing constraint helps us to illustrate the concepts of decision and forecast horizons by means of examples. This section is expository in nature, but theoretical developments of these ideas are available in the literature.

6.1 Production-Inventory Systems

Many manufacturing enterprises use a production-inventory system to manage fluctuations in consumer demand for their products. Such a system consists of a manufacturing plant and a finished goods warehouse to store products which are manufactured but not immediately sold. Once a product is made and put into inventory, it incurs inventory holding costs of two kinds: (1) costs of physically storing the product, insuring it, etc.; and (2) opportunity cost of having the firm’s money invested or tied up in the unsold inventory. The advantages of having products in inventory are: first, that they are immediately available to meet demand; second, that excess production during low demand periods can be stored in the warehouse so it will be available for sale during high demand periods. This usually permits the use of a smaller manufacturing plant than would otherwise be necessary, and also reduces the difficulties of managing the system.

The optimization problem is to balance the benefits of production smoothing versus the costs of holding inventory. Works that apply control theory to production and inventory problems have been reviewed in Sethi (1978a, 1984) .

6.1.1 The Production-Inventory Model

We consider a factory producing a single homogeneous good and having a finished goods warehouse. To state the model we define the following quantities:

$$\displaystyle\begin{array}{rcl} I(t)& =& \mbox{ the inventory level at time }t\mbox{ (state variable)}, {}\\ P(t)& =& \mbox{ the production rate at time }t\mbox{ (control variable)}, {}\\ S(t)& =& \mbox{ the exogenously given sales rate at time }t; {}\\ & & \mbox{ assumed to be bounded and differentiable for}\;t \geq 0, {}\\ T& =& \mbox{ the length of the planning period, } {}\\ \hat{I}& =& \mbox{ the inventory goal level,} {}\\ I_{0}& =& \mbox{ the initial inventory level,} {}\\ \hat{P}& =& \mbox{ the production goal level,} {}\\ h& =& \mbox{ the inventory holding cost coefficient;}\;h > 0, {}\\ c& =& \mbox{ the production cost coefficient;}\;c \geq 0, {}\\ \rho & =& \mbox{ the constant nonnegative discount rate};\rho \geq 0. {}\\ \end{array}$$

The interpretation of the inventory goal level $\hat{I}$ is that it is a safety stock that the company wants to keep on hand. For example, $\hat{I}$ could be 2 months of average sales or $\hat{I}$ could be 100 units of the finished goods. Similarly, the production goal level $\hat{P}$ can be interpreted as the most efficient level at which it is desired to run the factory.

With this notation, the state equation is given by the stock-flow differential equation

$$\displaystyle{ \dot{I}(t) = P(t) - S(t),\;I(0) = I_{0}, }$$

(6.1)

which says that the inventory at time t is increased by the production rate and decreased by the sales rate. The objective function of the model is:

$$\displaystyle{ \min \left \{J =\int _{ 0}^{T}e^{-\rho t}[\frac{h} {2}(I -\hat{ I})^{2} + \frac{c} {2}(P -\hat{ P})^{2}]dt\right \}. }$$

(6.2)

The interpretation of the objective function is that we want to keep the inventory as close as possible to its goal level $\hat{I},$ and also to keep the production rate P as close as possible to its goal level $\hat{P}.$ The quadratic terms $(h/2)(I -\hat{ I})^{2}$ and $(c/2)(P -\hat{ P})^{2}$ impose “penalties” for having either I or P not being close to its corresponding goal level.

Next we apply the maximum principle to solve the optimal control problem specified by (6.1) and (6.2). A stochastic extension of this problem will be carried out in Sect. 12.2.

6.1.2 Solution by the Maximum Principle

We now associate an adjoint function λ with Eq. (6.1) and can write the current-value Hamiltonian function as

$$\displaystyle{ H =\lambda (P - S) -\frac{h} {2}(I -\hat{ I})^{2} -\frac{c} {2}(P -\hat{ P})^{2}. }$$

(6.3)

In (6.3), we have used the negative of the (undiscounted) integrand in (6.2), since the minimization of J in (6.2) is equivalent to the maximization of − J. 

To apply the Pontryagin maximum principle, we differentiate (6.3) and set the resulting expression equal to 0, which gives

$$\displaystyle{ \frac{\partial H} {\partial P} =\lambda -c(P -\hat{ P}) = 0. }$$

(6.4)

From this we obtain the optimal production rate

$$\displaystyle{ P^{{\ast}}(t) =\hat{ P} +\lambda (t)/c. }$$

(6.5)

We should mention that in writing (6.5), we are allowing negative production (or disposal). Of course, the situation of a disposal will not arise if we assume a sufficiently large $\hat{P}$ and a sufficiently small I ₀. 

Remark 6.1

If P is constrained to be nonnegative, then the form of the optimal control will be

$$\displaystyle{ P^{{\ast}}(t) =\max \{\hat{ P} +\lambda (t)/c,0\}. }$$

(6.6)

This case will be treated in Sect. 6.1.6.

By substituting (6.5) into (6.1), we obtain

$$\displaystyle{ \dot{I} =\hat{ P} +\lambda /c - S,\;I(0) = I_{0}. }$$

(6.7)

The equation for the adjoint variable is easily found to be

$$\displaystyle{ \dot{\lambda }=\rho \lambda -\frac{\partial H} {\partial I} =\rho \lambda +h(I -\hat{ I}),\ \lambda (T) = 0. }$$

(6.8)

We see that (6.7) has the initial boundary specified and (6.8) has the terminal boundary specified, so together these give a two-point boundary value problem. We will employ a method to solve these two equations simultaneously, which works only in some special cases including the present case. The method is the well-known trick used to solve simultaneous differential equations by differentiation and substitution until one of the variables is eliminated. Specifically, we differentiate (6.7) with respect to t, which creates an equation with $\dot{\lambda }$ in it. We then use (6.8) to eliminate $\dot{\lambda }$ and (6.7) to eliminate λ from the resulting equation as follows:

$$\displaystyle{\ddot{I} =\dot{\lambda } /c -\dot{ S} =\rho (\lambda /c) + (h/c)(I -\hat{ I}) -\dot{ S}}$$

$$\displaystyle{=\rho (\dot{I} -\hat{ P} + S) + (h/c)(I -\hat{ I}) -\dot{ S}.}$$

We rewrite this as

$$\displaystyle{ \ddot{I} -\rho \dot{ I} -\alpha ^{2}I = -\alpha ^{2}\hat{I} -\dot{ S} -\rho (\hat{P} - S), }$$

(6.9)

where the constant α is given by

$$\displaystyle{ \alpha = \sqrt{h/c}. }$$

(6.10)

We can now solve (6.9) by using the standard method described in Appendix A. The auxiliary equation for (6.9) is

$$\displaystyle{m^{2} -\rho m -\alpha ^{2} = 0,}$$

which has the two real roots

$$\displaystyle{ m_{1} = (\rho -\sqrt{\rho ^{2 } + 4\alpha ^{2}})/2,\;\;m_{2} = (\rho +\sqrt{\rho ^{2 } + 4\alpha ^{2}})/2; }$$

(6.11)

note that m ₁ < 0 and m ₂ > 0. We can therefore write the general solution to (6.9) as

$$\displaystyle{ I(t) = a_{1}e^{m_{1} t} + a_{ 2}e^{m_{2} t} + Q(t),\;I(0) = I_{ 0}, }$$

(6.12)

where Q(t) is a particular integral of (6.9).

We will say that Q(t) is a special particular integral of (6.9) if it has no additive terms involving $e^{m_{1}t}$ and $e^{m_{2}t}.$ From now on we will always assume that Q(t) is a special particular integral.

Although (6.12) has two arbitrary constants a ₁ and a ₂, it has only one boundary condition. To get the other boundary condition we differentiate (6.12), substitute the result into (6.7), and solve for λ. We obtain

$$\displaystyle{ \lambda (t) = c(m_{1}a_{1}e^{m_{1} t} + m_{ 2}a_{2}e^{m_{2} t} +\dot{ Q} + S -\hat{ P}),\;\lambda (T) = 0. }$$

(6.13)

Note that we have imposed the boundary condition on λ so that we can determine the constants a ₁ and a ₂. 

For ease of expressing a ₁ and a ₂, let us define two constants

$$\displaystyle\begin{array}{rcl} b_{1}& =& I_{0} - Q(0),{}\end{array}$$

(6.14)

$$\displaystyle\begin{array}{rcl} b_{2}& =& \hat{P} -\dot{ Q}(T) - S(T).{}\end{array}$$

(6.15)

We now impose the boundary conditions in (6.12) and (6.13) and solve for a ₁ and a ₂ as follows:

$$\displaystyle\begin{array}{rcl} a_{1}& =& \frac{b_{2}e^{m_{1}T} - m_{2}b_{1}e^{(m_{1}+m_{2})T}} {m_{1}e^{2m_{1}T} - m_{2}e^{(m_{1}+m_{2})T}},{}\end{array}$$

(6.16)

$$\displaystyle\begin{array}{rcl} a_{2}& =& \frac{b_{1}m_{1}e^{2m_{1}T} - b_{2}e^{m_{1}T}} {m_{1}e^{2m_{1}T} - m_{2}e^{(m_{1}+m_{2})T}}.{}\end{array}$$

(6.17)

If we recall that m ₁ is negative and m ₂ is positive, then when T is sufficiently large so that $e^{m_{1}T}$ and $e^{2m_{1}T}$ are negligible, we can write

$$\displaystyle\begin{array}{rcl} a_{1}& \approx & b_{1},{}\end{array}$$

(6.18)

$$\displaystyle\begin{array}{rcl} a_{2}& \approx & \frac{b_{2}} {m_{2}}e^{-m_{2}T}.{}\end{array}$$

(6.19)

Note that for a large T, $e^{-m_{2}T}$ is close to zero and, therefore, a ₂ is close to zero. However, the reason for retaining the exponential term in (6.19) is that a ₂ is multiplied by $e^{m_{2}t}$ in (6.13), which, while small when t is small, becomes large and important when t is close to T. 

With these values of a ₁ and a ₂ and with (6.5), (6.12), and (6.13), we now write the expressions for I ^∗,  P ^∗, and λ. We will break each expression into three parts: the first part labeled Starting Correction is important only when t is small; the second part labeled Turnpike Expression is significant for all values of t; and the third part labeled Ending Correction is important only when t is close to T. 

$$\displaystyle\begin{array}{rcl} \underline{\mbox{ Starting Correction}}& \underline{\mbox{ Turnpike Expression}}& \underline{\mbox{ Ending Correction}} \\ I^{{\ast}} = (b_{ 1}e^{m_{1}t})+& (Q)+ & \left ( \frac{b_{2}} {m_{2}}e^{m_{2}(t-T)}\right ) {}\end{array}$$

(6.20)

$$\displaystyle\begin{array}{rcl} \begin{array}{lll} P^{{\ast}} = (m_{1}b_{1}e^{m_{1}t})+&(\dot{Q} + S)+&\left (b_{2}e^{m_{2}(t-T)}\right )\end{array} & &{}\end{array}$$

(6.21)

$$\displaystyle\begin{array}{rcl} \begin{array}{lll} \lambda = c(m_{1}b_{1}e^{m_{1}t})+&c\left (\dot{Q} + S -\hat{ P}\right )+&c\left (b_{2}e^{m_{2}(t-T)}\right )\end{array} & &{}\end{array}$$

(6.22)

Note that if b ₁ = 0, which by (6.14) means I ₀ = Q(0), then there is no starting correction. In other words, I ₀ = Q(0) is a starting inventory that causes the solution to be on the turnpike initially. In the same way, if b ₂ = 0, then the ending correction vanishes in each of these formulas, and the solution stays on the turnpike until the end.

Expressions (6.20) and (6.21) represent approximate closed-form solutions for the optimal inventory and production functions I ^∗ and P ^∗ as long as S is such that the special particular integral Q can be found explicitly. For such examples of S; see Sect. 6.1.4.

6.1.3 The Infinite Horizon Solution

It is important to show that this solution also makes sense when T → ∞. In this case it is usual to assume that the discount rate ρ > 0 and the sales rate S does not grow too fast so that the objective function (6.2) remains finite. One can then show that the limit of the finite horizon solution as T → ∞ also solves the infinite horizon problem. Note that as T → ∞, the ending correction terms in (6.20)–(6.22) disappear because $e^{-m_{2}T}$ goes to 0. We now have

$$\displaystyle{ \lambda (t) = c[m_{1}b_{1}e^{m_{1}t} +\dot{ Q} + S -\hat{ P}]. }$$

(6.23)

Since we would like

$$\displaystyle{ \lim _{t\rightarrow \infty }e^{-\rho t}\lambda (t) = 0, }$$

(6.24)

we would require that $S +\dot{ Q}$ grows slower asymptotically than the discount rate ρ. One can easily verify that this condition holds for the demand terms discussed in Sect. 6.1.4 that follows. Moreover, the condition is easy to check for any given specific demand S(t) for which the particular integral Q(t) is known.

By the sufficiency of the maximum principle conditions (Sect. 2.4), it can be verified that the limiting solution

$$\displaystyle{ I^{{\ast}}(t) = b_{ 1}e^{m_{1}t} + Q,\ P^{{\ast}}(t) = m_{ 1}b_{1}e^{m_{1}t} +\dot{ Q} + S }$$

(6.25)

is optimal. If I(0) = Q(0), the solution is always on the turnpike. Note that the triple $\{\bar{I},\bar{P},\bar{\lambda }\}=\{ Q,\dot{Q} + S,c(\dot{Q} + S -\hat{ P})\}$ represents a nonstationary turnpike. If I(0) ≠ Q(0), then b ₁ ≠ 0 and the expressions (6.25) imply that the paths of inventory and production only approach the turnpike but never attain it.

6.1.4 Special Cases of Time Varying Demands

In this section, we provide some important cases of time varying demands including seasonal demands. These involve polynomial or sinusoidal demand functions. We then solve some numerical examples of the model described in Sect. 6.1.1 for ρ = 0 and T < ∞. 

For the first example, we assume that S(t) is a polynomial of degree 2p or 2p − 1 so that S ^(2p+1) = 0, where S ^(k) denotes the kth time derivative of S with respect to t. In other words,

$$\displaystyle{ S(t) = C_{0}t^{2p} + C_{ 1}t^{2p-1} +\ldots +C_{ 2p}, }$$

(6.26)

where at least one of C ₀ and C ₁ is not zero. Then, from Zwillinger (2003) , a particular integral of (6.9) is

$$\displaystyle{ Q(t) =\hat{ I} + \frac{1} {\alpha ^{2}} S^{(1)} + \frac{1} {\alpha ^{4}} S^{(3)} +\ldots + \frac{1} {\alpha ^{2p}}S^{(2p-1)}. }$$

(6.27)

In Exercise 6.2 the reader is asked to verify this by direct substitution.

For the second example, we assume that S(t) is a sinusoidal demand function of form

$$\displaystyle{ S(t) = A\sin (\pi Bt + C) + D, }$$

(6.28)

where A, B, C, and D are constants. In Exercise 6.3 you are asked to verify that a particular integral of (6.9) for S in (6.28) is

$$\displaystyle{ Q(t) =\hat{ I} + \frac{\pi AB} {\alpha ^{2} +\pi ^{2}B^{2}}\cos (\pi Bt + C). }$$

(6.29)

It is well known in the theory of differential equations that demands that are sums of functions of the form (6.26) and/or (6.28) give rise to solutions that are sums of functions of form (6.27) and/or (6.29).

Example 6.1

Assume $\hat{P} = 30,\ \hat{I} = 15,\ T = 8,\ \rho = 0,\mbox{ and }\ h = c = 1$ so that α = 1,  m ₁ = −1, and m ₂ = 1. Assume

$$\displaystyle{S(t) = t(t - 4)(t - 8) + 30 = t^{3} - 12t^{2} + 32t + 30.}$$

Solution It is then easy to show from (6.27) that

$$\displaystyle{Q(t) = 3t^{2} - 24t + 53\mbox{ and }\dot{Q}(t) = 6t - 24.}$$

Also from (6.14), (6.15), and (6.16), we have a ₁ ≈ b ₁ = I ₀ − 53 and b ₂ = −24. Then, from (6.20) and (6.21),

$$\displaystyle{I^{{\ast}}(t) = (I_{ 0} - 53)e^{-t} + Q(t) - 24e^{t-8},}$$

$$\displaystyle{P^{{\ast}}(t) = -(I_{ 0} - 53)e^{-t} +\dot{ Q}(t) + S(t) - 24e^{t-8}.}$$

In Fig. 6.1 the graphs of sales, production, and inventory are drawn with I ₀ = 10 (a small starting inventory), which makes b ₁ = −43. In Fig. 6.2 the same graphs are drawn with I ₀ = 50 (a large starting inventory), which makes b ₁ = −3. In Fig. 6.3 the same graphs are drawn with I ₀ = 30, which makes b ₁ = −23. Note that initially during the time from 0 to 4, the three cases are quite different, but during the time from 4 to 8, they are nearly identical. The ending inventory ends up being 29 in all three cases.

Figure 6.1

Solution of Example 6.1 with I ₀ = 10

Figure 6.2

Solution of Example 6.1 with I ₀ = 50

Figure 6.3

Solution of Example 6.1 with I ₀ = 30

Example 6.2

Assume that

$$\displaystyle{ S(t) = A + Bt +\sum _{ k=1}^{K}C_{ k}\sin (\pi D_{k}t + E_{k}), }$$

(6.30)

where the constants A, B, C _k, D _k, and E _k are estimated from future demand data by means of one of the standard forecasting techniques such as those in Brown (1959, 1963) .

Solution By using formulas (6.27) and (6.29), we obtain the particular integral

$$\displaystyle{ Q(t) =\hat{ I} + \frac{1} {\alpha ^{2}} B +\sum _{ k=1}^{K} \frac{\pi C_{k}D_{k}} {\alpha ^{2} + (\pi D_{k})^{2}}\cos (\pi D_{k}t + E_{k}). }$$

(6.31)

6.1.5 Optimality of a Linear Decision Rule

In Sect. 6.1.2, our emphasis was to explore the turnpike nature of the solution of the inventory model of Sect. 6.1.1. For this purpose, we made some asymptotic approximations when solving the state and adjoint differential equations under the assumption that the horizon is long. Here our focus is to solve the undiscounted version (i.e., ρ = 0) of the model exactly to find its optimal feedback solution, and show that it is a linear decision rule as reported in the classical work of Holt et al. (1960) .

Since the two-point boundary value problem given by (6.7) and (6.8) is a linear system of differential equations, it is known via its fundamental solution matrix that λ can be expressed in terms of I in a linear way as follows:

$$\displaystyle{ \lambda (t) =\psi (t) - s(t)I(t), }$$

(6.32)

where ψ(t) and s(t) are continuously differentiable in t. Differentiating (6.32) with respect to t and substituting for $\dot{I}$ and $\dot{\lambda }$ from (6.7) and (6.8) with ρ = 0, respectively, we obtain

$$\displaystyle{I(h - s^{2}/c +\dot{ s}) + (\hat{P} +\psi /c - S)s - h\hat{I}-\dot{\psi } = 0.}$$

Since the above relation must hold for any value of the initial inventory I ₀, we must have

$$\displaystyle{ \dot{s} = s^{2}/c - h\mbox{ and }\dot{\psi } = (\hat{P} +\psi /c - S)s - h\hat{I}. }$$

(6.33)

Also from λ(T) = 0 in (6.8) and (6.32), we have 0 = ψ(T) − s(T)I(T), a relation that must hold regardless of the value of I(T). Thus, we can conclude that

$$\displaystyle{ s(T) = 0\mbox{ and }\psi (T) = 0. }$$

(6.34)

Clearly, the solution of the differential equation given by (6.33) and (6.34) will give us the optimal control (6.5) in terms of S(t) and ψ(t). In particular, the differential equation

$$\displaystyle{ \dot{s} = s^{2}/c - h,\;\;s(T) = 0 }$$

(6.35)

is known as the Riccati equation, whose solution is given by

$$\displaystyle{ s(t) = \sqrt{hc}\tanh (\sqrt{\frac{h} {c}} (T - t)). }$$

(6.36)

Using (6.32) and (6.36) in (6.5), the optimal production rate P ^∗(t) is

$$\displaystyle{ P^{{\ast}}(t) =\hat{ P} -\sqrt{\frac{h} {c}}\tanh \left (\sqrt{\frac{h} {c}} (T - t)\right )I^{{\ast}}(t) + \frac{\psi (t)} {c}. }$$

(6.37)

This says that the optimal production rate equals the production goal level $\hat{P}$ plus two adjustment terms. The first term implies ceteris paribus that the higher the current inventory level, the lower the production rate is. Furthermore, this dependence is linear with the linear effect decreasing as t increases, reaching zero at t = T. The second term depends on all the model parameters including the demand rate from time t to T. 

Because of the linear dependence of the optimal production rate on the inventory level in (6.37), this rule is known as a linear decision rule as reported by Holt et al. (1960) . More generally, this rule can be extended to linear quadratic problems as listed in Table 3.3(c). In Appendix D.4, we derive this rule for the problems given in Table 3.3(c), but without the forcing function d. Furthermore, the rule can be extended to a class of stochastic linear-quadratic problems that include the stochastic production planning problem treated in Sect. 12.2.

6.1.6 Analysis with a Nonnegative Production Constraint

Thus far in this chapter, we have ignored the production constraint P ≥ 0 and used (6.5) and (6.37) as the optimal decision rules. Here we will solve the production-inventory problem subject to P ≥ 0, and use (6.6) as the optimal production rule. For simplicity of analysis and exposition, we will assume also that S is a positive constant, T = ∞, and ρ > 0. These specifications make $\dot{S} = 0,$ making the right hand side $-\alpha \hat{I} -\rho (\hat{P} - S)$ a constant, a ₁ = b ₁ in (6.16), and a ₂ = 0 in (6.17).

In view of its constant right-hand side, we can use Row (3) of Table A.2 to obtain its particular integral as

$$\displaystyle{ Q = \frac{\rho } {\alpha ^{2}}(\hat{P} - S) +\hat{ I}, }$$

(6.38)

which is a constant and thus $\dot{Q} = 0.$ From (6.14) and (6.15), we now have

$$\displaystyle{b_{1} = I_{0} - Q = I_{0} -\hat{ I} - (\rho /\alpha ^{2})(\hat{P} - S)\mbox{ and }b_{ 2} =\hat{ P} - S.}$$

The turnpike is defined by the triple $\{\bar{I},\bar{P},\bar{\lambda }\}=\{ (\rho /\alpha ^{2})(\hat{P} - S) +\hat{ I},S,c(S -\hat{ P})\}$ formed from the turnpike expressions in (6.20), (6.21), and (6.22), respectively. Note that we could have obtained the turnpike levels directly by applying the conditions (3.108), which in this case are

$$\displaystyle{ \dot{\bar{I}} = 0,\;\dot{\bar{\lambda }}= 0,\mbox{ and }\bar{P} =\hat{ P} +\bar{\lambda } /c = S. }$$

(6.39)

If I ₀ = Q, then the optimal solution stays on the turnpike. If I ₀ ≠ Q, we must obtain the transient solution. It should be clear that the control in (6.25) may become negative, especially when the initial inventory is high. Let us complete the solution of the problem by considering three cases: I ₀ ≤ Q, Q < I ₀ ≤ Q − S∕m ₁, and I ₀ > Q − S∕m ₁. 

If I ₀ ≤ Q, then the control in (6.25) with b ₁ = I ₀ − Q ₀ is clearly positive. Thus, the optimal production rate is given by

$$\displaystyle{ P^{{\ast}}(t) = m_{ 1}b_{1}e^{m_{1}t} + S = m_{ 1}(I_{0} - Q)e^{m_{1}t} + S \geq 0. }$$

(6.40)

Moreover, from the state in (6.25), we can obtain the corresponding I ^∗(t) as

$$\displaystyle{ I^{{\ast}}(t) = (I_{ 0} - Q)e^{m_{1}t} + Q. }$$

(6.41)

It is easy to see that I ^∗(t) increases monotonically to Q as t → ∞, as shown in Fig. 6.4.

Figure 6.4

Optimal production rate and inventory level with different initial inventories

If Q < I ₀ ≤ Q − S∕m ₁, we can easily see from (6.40) that P ^∗(0) ≥ 0. Furthermore, $\dot{P}^{{\ast}}(t) \geq 0,$ and therefore the optimal production rate is once again given by (6.40). We also have I ^∗(t) as in (6.41) and conclude that I ^∗(t) → Q monotonically as t → ∞, as shown in Fig. 6.4.

Finally, if I ₀ > Q − S∕m ₁, (6.40) would have a negative value for the initial production which is infeasible. By (6.6), P ^∗(0) = 0. We can now depict this situation in Fig. 6.4. The time $\hat{t}$ shown in the figure is the time at which $P^{{\ast}}(\hat{t}) =\hat{ P} +\lambda (\hat{t})/c = 0.$ We already know from (6.40) that in the case when I ₀ = Q − S∕m ₁, P ^∗(0) = 0. This suggests that

$$\displaystyle{ I^{{\ast}}(\hat{t}) = Q - \frac{S} {m_{1}}. }$$

(6.42)

For $t \leq \hat{ t},$ we have P ^∗(t) = 0 so that $\dot{I}^{{\ast}} = -S,$ which gives

$$\displaystyle{ I^{{\ast}}(t) = I_{ 0} - St,\;t \leq \hat{ t}. }$$

(6.43)

As for the adjoint equation (6.7), we now need the boundary condition at $\hat{t}.$ For this, we can use (6.4) to obtain $\lambda (\hat{t}) = -c\hat{P}.$ Thus, the adjoint equation in the interval $[0,\hat{t}\;]$ is

$$\displaystyle{ \dot{\lambda }=\rho \lambda +h(I -\hat{ I}),\ \lambda (\hat{t}) = -c\hat{P}. }$$

(6.44)

We can substitute I ₀ − St for I in Eq. (6.44) and solve for λ. Note that we can easily obtain $\hat{t}$ as

$$\displaystyle{ I_{0} - S\hat{t} = Q - \frac{S} {m_{1}} \Rightarrow \hat{ t} = \frac{I_{0} - Q} {S} + \frac{1} {m_{1}}. }$$

(6.45)

We can now specify the complete solution in the case when I ₀ > Q − S∕m ₁. With $\hat{t}$ specified in (6.45), the solution is as follows.

For $0 \leq t \leq \hat{ t}: P^{{\ast}}(t) = 0,I^{{\ast}}(t) = I_{0} - St,$ and λ(t) is the solution of

$$\displaystyle{\dot{\lambda }=\rho \lambda +h(I_{0} - St -\hat{ I}),\lambda (\hat{t}) = -c\hat{P}.}$$

For $t >\hat{ t}:$ we replace I ₀ by Q − S∕m ₁ and t by $t -\hat{ t}$ on the right hand side of (6.40) to obtain $P^{{\ast}}(t) = -Se^{m_{1}(t-\hat{t})}.$ The same replacements in (6.41) gives us the corresponding $I^{{\ast}}(t) = -(S/m_{1})e^{m_{1}t}.$ Finally, λ(t) can be obtained by solving

$$\displaystyle{\dot{\lambda }=\rho \lambda -h\left ( \frac{S} {m_{1}}e^{m_{1}t} +\hat{ I}\right ),\lambda (\hat{t}) = -c\hat{P}.}$$

We have thus solved the problem in every case of the initial condition I ₀. These solutions are sketched in Fig. 6.4 for $\hat{I} = 8,\hat{P} = 5,S = 6,h = 1,c = 4,$ and ρ = 0. 1, for three different values of I ₀, namely, 25, 15, and 1. In Exercise 6.7, you are asked to solve the problem for these values and obtain Fig. 6.4.

6.2 The Wheat Trading Model

Consider a firm that buys and sells wheat. The firm’s only assets are cash and wheat, and the price of wheat over time is known with certainty. The objective of this firm is to buy and sell wheat in order to maximize the total value of its assets at the horizon time T. The problem here is similar to the simple cash balance model of Sect. 5.1 except that there are nonlinear holding costs associated with storing wheat. An extension of this model to one having two control variables appears in Ijiri and Thompson (1972) .

6.2.1 The Model

We introduce the following notation:

$$\displaystyle\begin{array}{rcl} T& =& \mbox{ the horizon time, } {}\\ x(t)& =& \mbox{ the cash balance in dollars at time $t,$ } {}\\ y(t)& =& \mbox{ the wheat balance in bushels at time $t,$ } {}\\ v(t)& =& \mbox{ the rate of purchase of wheat in bushels per unit time; a negative purchase means a sale, } {}\\ p(t)& =& \mbox{ the price of wheat in dollars per bushel at time $t,$ } {}\\ r& =& \mbox{ the constant positive interest rate earned on the cash balance, } {}\\ h(y)& =& \mbox{ the cost of holding $y$ bushels per unit time. } {}\\ \end{array}$$

In this section we permit x and y to go negative, meaning that borrowing money and short-selling wheat are both allowed. In the next section we disallow the short-selling of wheat.

The state equations are:

$$\displaystyle\begin{array}{rcl} \dot{x}& =& rx - h(y) - pv,\;\;x(0) = x_{0},{}\end{array}$$

(6.46)

$$\displaystyle\begin{array}{rcl} \dot{y}& =& v,\;\;y(0) = y_{0},{}\end{array}$$

(6.47)

and the control constraints are

$$\displaystyle{ -V _{2} \leq v(t) \leq V _{1}, }$$

(6.48)

where V ₁ and V ₂ are nonnegative constants. The objective function is:

$$\displaystyle{ \max \{J = x(T) + p(T)y(T)\} }$$

(6.49)

subject to (6.46)–(6.48). Note that the problem is in the linear Mayer form.

6.2.2 Solution by the Maximum Principle

Introduce the adjoint variables λ ₁ and λ ₂ and define the Hamiltonian function

$$\displaystyle{ H =\lambda _{1}[rx - h(y) - pv] +\lambda _{2}v. }$$

(6.50)

The adjoint equations are:

$$\displaystyle\begin{array}{rcl} \dot{\lambda }_{1}& =& -\lambda _{1}r,\;\;\lambda _{1}(T) = 1,{}\end{array}$$

(6.51)

$$\displaystyle\begin{array}{rcl} \dot{\lambda }_{2}& =& h'(y)\lambda _{1},\;\;\lambda _{2}(T) = p(T).{}\end{array}$$

(6.52)

It is easy to solve (6.51) as

$$\displaystyle{ \lambda _{1}(t) = e^{r(T-t)} }$$

(6.53)

and (6.52) as

$$\displaystyle{ \lambda _{2}(t) = p(T) -\int _{t}^{T}h'(y(\tau ))e^{r(T-\tau )}d\tau. }$$

(6.54)

The interpretation of λ ₁(t) is that it is the future value (at time T) of one dollar held as cash from t to T. The interpretation of λ ₂(t) is the price at time T of a bushel of wheat less the total future value (at time T) of the stream of storage costs incurred to store that bushel of wheat from t to T. 

From (6.50) the optimal control is

$$\displaystyle{ v^{{\ast}}(t) = \mbox{ bang}[-V _{ 2},V _{1};\lambda _{2}(t) -\lambda _{1}(t)p(t)]. }$$

(6.55)

In Exercise 6.8 you are asked to provide the interpretation of this optimal policy.

Equations (6.46), (6.47), (6.54), and (6.55) determine the two-point boundary value problem which usually requires a numerical solution procedure. In the next section we assume a special form for the storage function h(y) to be able to obtain a closed-form solution.

6.2.3 Solution of a Special Case

For this special case we assume $h(y) = \frac{1} {2}\vert y\vert,\;r = 0,\;x(0) = 10,\;y(0) = 0,$ V ₁ = V ₂ = 1,  T = 6, and

$$\displaystyle{ p(t) = \left \{\begin{array}{cl} 3&\mbox{ for }0 \leq t \leq 3,\\ 4 &\mbox{ for } 3 < t \leq 6. \end{array} \right. }$$

(6.56)

We will apply the maximum principle (2.31) developed in Chap. 2 to this problem even though h(y) is not differentiable at y = 0. The answer can be obtained rigorously by using the maximum principle for models involving nondifferentiable functions discussed, e.g., in Clarke (1989, Chapter 4) and Feichtinger and Hartl (1985b) .

For this case with r = 0, we have λ ₁(t) = 1 for all t from (6.53) so that the TPBVP is

$$\displaystyle\begin{array}{rcl} \dot{x}& =& -\frac{1} {2}\vert y\vert - pv,\;x(0) = 10,{}\end{array}$$

(6.57)

$$\displaystyle\begin{array}{rcl} \dot{y}& =& v,\;y(0) = 0,{}\end{array}$$

(6.58)

$$\displaystyle\begin{array}{rcl} \dot{\lambda }_{2}(t)& =& \frac{1} {2}\mbox{ sgn}(y),\;\lambda _{2}(6) = 4.{}\end{array}$$

(6.59)

For this simple problem it is easy to guess a solution. From the fact that λ ₁ = 1, the optimal policy (6.55) reduces to

$$\displaystyle{ v^{{\ast}}(t) = \mbox{ bang}[-1,1;\lambda _{ 2}(t) - p(t)]. }$$

(6.60)

The graph of the price function is shown in Fig. 6.5. Since p(t) is increasing, short-selling is never optimal. Since the storage cost is 1/2 per unit per unit time and the wheat price jumps by 1 unit at t = 3, it never pays to store wheat for more than 2 time units. Because y(0) = 0, we have v ^∗(t) = 0 for 0 ≤ t ≤ 1. This obviously must be a singular control. Suppose we start buying wheat at t ^∗ > 1. From (6.60) the rate of buying is 1; clearly buying will continue at this rate until t = 3, and not longer. In order to not lose money on the storage of wheat, it must be sold within 2 time units of its purchase. Clearly we should start selling at t = 3⁺ at the maximum rate of 1, and continue until a last sale time t ^∗∗. In order to sell exactly all of the wheat purchased, we must have

$$\displaystyle{ 3 - t^{{\ast}} = t^{{\ast}{\ast}}- 3. }$$

(6.61)

Figure 6.5

The price trajectory (6.56)

Thus, v ^∗(t) = 0 in the interval [t ^∗∗, 6], which is also a singular control. With this policy, y(t) > 0 for all t ∈ (t ^∗, t ^∗∗). From (6.59), $\dot{\lambda }_{2} = 1/2$ in the interval (t ^∗, t ^∗∗). In order to have a singular control in the interval [t ^∗∗, 6], we must have λ ₂(t) = 4 in that interval. Also, in order to have a singular control in [0, t ^∗], we must have λ ₂(t) = 3 in that interval. Thus, λ ₂(t ^∗∗) −λ ₂(t ^∗) = 1, which with $\dot{\lambda }_{2} = 1/2$ allows us to conclude that

$$\displaystyle{ t^{{\ast}{\ast}}- t^{{\ast}} = 2, }$$

(6.62)

and therefore t ^∗ = 2 and t ^∗∗ = 4. Thus from (6.59) and (6.60),

$$\displaystyle{ \lambda _{2}(t) = \left \{\begin{array}{ll} 3, &0 \leq t \leq 2, \\ 2 + t/2,&2 \leq t \leq 4, \\ 4, &4 \leq t \leq 6. \end{array} \right. }$$

(6.63)

We can now sketch graphs for λ ₂(t),  v ^∗(t), and y ^∗(t) as shown in Fig. 6.6. In Exercise 6.13 you are asked to show that these trajectories are optimal by verifying that the maximum principle necessary conditions hold and that they are also sufficient.

Figure 6.6

Adjoint variable, optimal policy and inventory in the wheat trading model

6.2.4 The Wheat Trading Model with No Short-Selling

We next consider the wheat trading problem in the case when short-selling is not permitted, i.e., we impose the state constraint y ≥ 0. Moreover, for simplicity in exposition we consider the following special case of Norström (1978) . Specifically, we assume h(y) = y∕2,  r = 0,  x(0) = 10, y(0) = 1, V ₁ = V ₂ = 1, T = 3, and

$$\displaystyle{ p(t) = \left \{\begin{array}{rl} - 2t + 7&\mbox{ for }0 \leq t < 2,\\ t + 1 &\mbox{ for } 2 \leq t \leq 3. \end{array} \right. }$$

(6.64)

The statement of the problem is:

$$\displaystyle{ \left \{\begin{array}{ll} \max \left \{J = x(3) + p(3)y(3) = x(3) + 4y(3)\right \} \\ \text{subject to} \\ \dot{x} = -\frac{1} {2}y - pv,\ x(0) = 10, \\ \dot{y} = v,\ y(0) = 1, \\ v + 1 \geq 0,\ 1 - v \geq 0,\ y \geq 0. \end{array} \right. }$$

(6.65)

To solve this problem, we use the Lagrangian form of the indirect maximum principle given in (4.29). The Hamiltonian is

$$\displaystyle{ H =\lambda _{1}(-y/2 - pv) +\lambda _{2}v. }$$

(6.66)

The optimal control is

$$\displaystyle{ v^{{\ast}}(t) = \mbox{ bang}[-1,1;\lambda _{ 2}(t) -\lambda _{1}(t)p(t)]\mbox{ when }y > 0. }$$

(6.67)

Whenever y = 0 we must impose $\dot{y} = v \geq 0$ in order to insure that no short-selling occurs. Therefore,

$$\displaystyle{ v^{{\ast}}(t) = \mbox{ bang}[0,1;\lambda _{ 2}(t) -\lambda _{1}(t)p(t)]\mbox{ when }y = 0. }$$

(6.68)

Next we form the Lagrangian

$$\displaystyle{ L = H +\mu _{1}(v + 1) +\mu _{2}(1 - v) +\eta v, }$$

(6.69)

where μ ₁,  μ ₂, and η satisfy the complementary slackness conditions:

$$\displaystyle\begin{array}{rcl} \mu _{1} \geq 0,& \mu _{1}(v + 1) = 0,&{}\end{array}$$

(6.70)

$$\displaystyle\begin{array}{rcl} \mu _{2} \geq 0,& \mu _{2}(1 - v) = 0,&{}\end{array}$$

(6.71)

$$\displaystyle\begin{array}{rcl} \eta \geq 0,& \eta y = 0.&{}\end{array}$$

(6.72)

Furthermore, the optimal trajectory must satisfy

$$\displaystyle{ \frac{\partial L} {\partial v} =\lambda _{2} - p\lambda _{1} +\mu _{1} -\mu _{2}+\eta = 0. }$$

(6.73)

With r = 0, we get λ ₁ = 1 as before, and

$$\displaystyle{ \dot{\lambda }_{2} = -\frac{\partial L} {\partial y} = 1/2,\;\lambda _{2}(3^{-}) = 4+\gamma, }$$

(6.74)

with

$$\displaystyle{ \gamma \geq 0,\gamma y(3) = 0. }$$

(6.75)

Let us first try γ = 0. Then λ ₂(3⁻) = 4, and if we let $\hat{t}$ denote the time of the last jump before the terminal time, then there is no jump in the interval $(\hat{t},3)$. Then, from (6.74) we have

$$\displaystyle{ \lambda _{2}(t) = t/2 + 5/2\mbox{ for }\hat{t} \leq t < 3, }$$

(6.76)

and the optimal control from (6.67) or (6.68) is v ^∗ = 1, i.e., buy wheat at the maximum rate of 1, so long as λ ₂(t) > p(t). Also, this will give y(3) > 0, so that (6.75) holds. Let us next find the time $\hat{t}$ of the last jump before the terminal time. Clearly, this value will not be larger than the time at which λ ₂(t) = p(t). Thus,

$$\displaystyle{ \hat{t} \leq \left \{t\vert t/2 + 5/2 = -2t + 7\right \} = 1.8. }$$

(6.77)

Since p(t) is decreasing at the start of the problem, it appears that selling at the maximum rate of 1, i.e., v ^∗ = −1, should be optimal at the start. Since the beginning inventory is y(0) = 1, selling at the rate of 1 can continue only until t = 1, at which time the inventory y(1) becomes 0. Suppose that we do nothing, i.e., v ^∗(t) = 0 in the interval (1, 1. 8]. Then, t = 1 is an entry time (see Sect. 4.2) and t = 1. 8 is not an entry time, and $\hat{t} = 1$. Hence, according to the maximum principle (4.29), λ ₂(t) is continuous at t = 1. 8, and therefore λ ₂(t) is given by (6.76) in the interval [1, 3), i.e.,

$$\displaystyle{ \lambda _{2}(t) = t/2 + 5/2\mbox{ for }1 \leq t < 3. }$$

(6.78)

Using (6.73) with λ ₁ = 1 in the interval (1, 1. 8] and v ^∗ = 0 so that μ ₁ = μ ₂ = 0, we have

$$\displaystyle{\lambda _{2} - p +\mu _{1} -\mu _{2}+\eta =\lambda _{2} - p+\eta = 0,}$$

and consequently

$$\displaystyle{ \eta (t) = p(t) -\lambda _{2}(t) = -5t/2 + 9/2,\;t \in (1,1.8]. }$$

(6.79)

Since h _t = 0, the jump condition in (4.29) for the Hamiltonian at τ = 1 reduces to

$$\displaystyle{H[x^{{\ast}}(1),\,u^{{\ast}}(1^{-}),\,\lambda (1^{-}),\,1] = H[x^{{\ast}}(1),\,u^{{\ast}}(1^{+}),\,\lambda (1^{+}),\,1].}$$

From the definition of the Hamiltonian H in (6.66), we can rewrite the condition as

$$\displaystyle{\begin{array}{cc} \lambda _{1}(1^{-})[-y(1)/2 - p(1^{-})v^{{\ast}}(1^{-})] +\lambda _{2}(1^{-})v^{{\ast}}(1^{-}) = \\ \lambda _{1}(1^{+})[-y(1)/2 - p(1^{+})v^{{\ast}}(1^{+})] +\lambda _{2}(1^{+})v^{{\ast}}(1^{+}). \end{array} }$$

Since λ ₁(t) = 1 for all t, the above condition reduces to

$$\displaystyle{-p(1^{-})v^{{\ast}}(1^{-}) +\lambda _{ 2}(1^{-})v^{{\ast}}(1^{-}) = -p(1^{+})v^{{\ast}}(1^{+}) +\lambda _{ 2}(1^{+})v^{{\ast}}(1^{+}).}$$

Substituting the values of p(1⁻) = p(1⁺) = 5 from (6.64), λ ₂(1⁺) = 3 from (6.78), and v ^∗(1⁺) = 0 and v ^∗(1⁻) = −1 from the above discussion, we obtain

$$\displaystyle{ -5(-1) +\lambda _{2}(1^{-})(-1) = -5(0) + 3(0) = 0 \Rightarrow \lambda _{ 2}(1^{-}) = 5. }$$

(6.80)

We can now use the jump condition in (4.29) on the adjoint variables to obtain

$$\displaystyle{\lambda _{2}(1^{-}) =\lambda _{ 2}(1^{+}) +\zeta (1) \Rightarrow \zeta (1) =\lambda _{ 2}(1^{-}) -\lambda _{ 2}(1^{+}) = 5 - 3 = 2 \geq 0.}$$

It is important to note that in the interval [1, 1. 8], the optimal control condition (6.68) holds, justifying our supposition that v ^∗ = 0 in this interval. Furthermore, using (6.80) and (6.74),

$$\displaystyle{ \lambda _{2}(t) = t/2 + 9/2\mbox{ for }t \in [0,1), }$$

(6.81)

and the optimal control condition (6.67) holds, justifying our supposition that v ^∗ = −1 in this interval. Also, we can conclude that our guess γ = 0 is correct. The graphs of λ ₂(t),  p(t), and v ^∗(t) are displayed in Fig. 6.7. To complete the solution of the problem, you are asked to determine the values of μ ₁,  μ ₂, and η in these various intervals.

Figure 6.7

Adjoint trajectory and optimal policy for the wheat trading model

6.3 Decision Horizons and Forecast Horizons

In some dynamic problems it is possible to show that the optimal decisions during an initial positive time interval are either partially or wholly independent of the data from some future time onwards. In such cases, a forecast of the future data needs to be made only as far as that time to make optimal decisions in the initial time interval. The initial time interval is called the decision horizon and the time up to which data is required to make the optimal decisions during the decision horizon is called the forecast horizon; see Bes and Sethi (1988) , Bensoussan et al. (1983) , and Haurie and Sethi (1984) for details on these concepts. Whenever they exist, these horizons naturally decompose the problem into a series of smaller problems.

If the optimal decisions during the decision horizon are completely independent of the data beyond the forecast horizon, then the latter is called a strong forecast horizon . If, on the other hand, some mild restrictions on the data after the forecast horizon are required in order to keep the optimal decisions during the decision horizon unaffected, then it is called a weak forecast horizon .

In this section we demonstrate these concepts in the context of the wheat trading model of the previous section. In Sect. 6.3.1 we obtain a decision horizon for the model of Sect. 6.2.4 which is also a weak forecast horizon. In Sect. 6.3.2 we modify the wheat trading model by adding a warehousing constraint. For the new problem we obtain a decision horizon and a strong forecast horizon. See also Sethi and Thompson (1982) , Rempala (1986) and Hartl (1986a, 1988a) for further research in the context of the wheat trading model.

In what follows we obtain these horizons and verify them for some examples with different forecast data. For more details and proofs in other situations including more general ones, see Modigliani and Hohn (1955) , Lieber (1973) , Pekelman (1974, 1975, 1979) , Kleindorfer and Lieber (1979) , Vanthienen (1975) , Morton (1978) , Lundin and Morton (1975) , Rempala and Sethi (1988, 1992) , Hartl (1989a) , and Sethi (1990) .

6.3.1 Horizons for the Wheat Trading Model with No Short-Selling

For the model of Sect. 6.2.4, we will demonstrate that t = 1 is a decision horizon as well as a weak forecast horizon. In Fig. 6.8 we have redrawn Fig. 6.7 with a new price trajectory in the time interval [1, 3]. Also in the figure, we have extended the initial λ ₂ trajectory and labeled it the price shield . Its significance is that, as long as the new price trajectory in the interval [1, 3] stays below the price shield, the optimal solution in the interval [0, 1], which is the decision horizon, remains unchanged. That is, it is optimal to sell throughout the interval. The restriction that p(t) must stay below the price shield in [1, 3] is the reason that t = 1 is a weak forecast horizon. The optimality of the control shown in Fig. 6.8 can be concluded by obtaining the adjoint trajectory in the interval [1, 3] as a straight line with slope 1∕2 and the terminal value λ ₂(3⁻) = p(3). This way of drawing the adjoint trajectory is correct as long as the corresponding policy does not violate the inventory constraint y(t) ≥ 0 in the interval [1, 3]. For example, this will be the case if the buy interval in Fig. 6.8 is shorter than the sell interval at the end. On the other hand, if the inventory constraint is violated, then the λ ₂(t) trajectory may jump in the interval [1, 3), and it will be more complicated to obtain it. Nevertheless, the decision horizon and weak forecast horizon still occur at t = 1. Moreover, if we let T > 1 be any finite horizon and assume that p(t) in the interval [1, T] is always below the price shield line of Fig. 6.8 extended to T, then the policy of selling at the maximum rate in the interval [0, 1] remains optimal.

Figure 6.8

Decision horizon and optimal policy for the wheat trading model

6.3.2 Horizons for the Wheat Trading Model with No Short-Selling and a Warehousing Constraint

In order to give an example in which a strong forecast horizon occurs, we modify the example of Sect. 6.2.4 by adding the warehousing constraint y ≤ 1 or

$$\displaystyle{ 1 - y \geq 0, }$$

(6.82)

changing the terminal time to T = 4, and defining the price trajectory to be

$$\displaystyle{ p(t) = \left \{\begin{array}{rl} - 2t + 7&\mbox{ for }t \in [0,2),\\ t + 1 &\mbox{ for } t \in [2, 4]. \end{array} \right. }$$

(6.83)

The Hamiltonian of the new problem is unchanged and is given in (6.66). Furthermore, λ ₁ = 1. The optimal control is defined in three parts as:

$$\displaystyle\begin{array}{rcl} v^{{\ast}}(t)& =& \mbox{ bang}[-1,1;\lambda _{ 2}(t) - p(t)]\mbox{ when }0 < y < 1,{}\end{array}$$

(6.84)

$$\displaystyle\begin{array}{rcl} v^{{\ast}}(t)& =& \mbox{ bang }[0,1;\lambda _{ 2}(t) - p(t)]\mbox{ when }y = 0,{}\end{array}$$

(6.85)

$$\displaystyle\begin{array}{rcl} v^{{\ast}}(t)& =& \mbox{ bang}[-1,0;\lambda _{ 2}(t) - p(t)]\mbox{ when }y = 1.{}\end{array}$$

(6.86)

Defining a Lagrange multiplier η ₁ for the derivative of (6.82), i.e., for $-\dot{y} = -v \geq 0$, we form the Lagrangian

$$\displaystyle{ L = H +\mu _{1}(v + 1) +\mu _{2}(1 - v) +\eta v +\eta _{1}(-v), }$$

(6.87)

where μ ₁, μ ₂, and η satisfy (6.70)–(6.72) and η ₁ satisfies

$$\displaystyle{ \eta _{1} \geq 0,\;\eta _{1}(1 - y) = 0,\;\dot{\eta }_{1} \leq 0. }$$

(6.88)

Furthermore, the optimal trajectory must satisfy

$$\displaystyle{ \frac{\partial L} {\partial v} =\lambda _{2} - p +\mu _{1} -\mu _{2} +\eta -\eta _{1} = 0. }$$

(6.89)

As before, λ ₁ = 1 and λ ₂ satisfies

$$\displaystyle{ \dot{\lambda _{2}} = 1/2,\;\lambda _{2}(4^{-}) = p(4) +\gamma _{ 1} -\gamma _{2} = 5 +\gamma _{1} -\gamma _{2}, }$$

(6.90)

where

$$\displaystyle{ \gamma _{1} \geq 0,\;\gamma _{1}y(4) = 0,\;\gamma _{2} \geq 0,\;\gamma _{2}(1 - y(4)) = 0. }$$

(6.91)

Let us first try γ ₁ = γ ₂ = 0. Let $\hat{t}$ be the time of the last jump of the adjoint function λ ₂(t) before the terminal time T = 4. Then,

$$\displaystyle{ \lambda _{2}(t) = t/2 + 3\mbox{ for }\hat{t} \leq t < 4. }$$

(6.92)

The graph of (6.92) intersects the price trajectory at t = 8∕5 as shown in Fig. 6.9. It also stays above the price trajectory in the interval [8∕5, 4] so that, if there were no warehousing constraint (6.82), the optimal decision in this interval would be to buy at the maximum rate. However, with the constraint (6.82), this is not possible. Thus $\hat{t} > 8/5$, since λ ₂ will have a jump in the interval [8∕5, 4].

Figure 6.9

Optimal policy and horizons for the wheat trading model with no short-selling and a warehouse constraint

To find the actual value of $\hat{t}$ we must insert a line of slope 1∕2 above the minimum price at t = 2 in such a way that its two intersection points with the price trajectory are exactly one time unit (the time required to fill up the warehouse) apart. Thus using (6.83), $\hat{t}$ must satisfy

$$\displaystyle{-2(\hat{t} - 1) + 7 + (1/2)(1) =\hat{ t} + 1,}$$

which yields $\hat{t} = 17/6$.

The rest of the analysis for determining λ ₂ including the jump conditions is similar to that given in Sect. 6.2.4. Thus,

$$\displaystyle{ \lambda _{2}(t) = \left \{\begin{array}{ll} t/2 + 9/2 &\mbox{ for }t \in [0,1), \\ t/2 + 29/12&\mbox{ for }t \in [1,17/6), \\ t/2 + 3 &\mbox{ for }t \in [17/6,4]. \end{array} \right. }$$

(6.93)

This makes γ ₁ = γ ₂ = 0 the correct guess.

Given (6.93), the optimal policy is given by (6.84)–(6.86) and is shown in Fig. 6.9. To complete the maximum principle we must derive expressions for the Lagrange multipliers in the four intervals shown in Fig. 6.9.

Interval [0, 1):   μ ₂ = η = η ₁ = 0,  μ ₁ = p −λ ₂ > 0; 

$$\displaystyle{v^{{\ast}} = -1,\;0 < y^{{\ast}} < 1.}$$

Interval [1, 11∕6):   μ ₁ = μ ₂ = η ₁ = 0,  η = p −λ ₂ > 0,   $\dot{\eta }\leq 0$;

$$\displaystyle{v^{{\ast}} = 0,\;y^{{\ast}} = 0.}$$

Interval [11∕16, 17∕6):   μ ₁ = η = η ₁ = 0,  μ ₂ = λ ₂ − p > 0; 

$$\displaystyle{v^{{\ast}} = 1,\;0 < y^{{\ast}} < 1.}$$

Interval [17∕6, 4]:   μ ₁ = μ ₂ = η = 0,  η ₁ = λ ₂ − p > 0,  $\dot{\eta }_{1} \leq 0$, γ ₁ = γ ₂ = 0; 

$$\displaystyle{v^{{\ast}} = 0,\;y^{{\ast}} = 1.}$$

In Exercise 6.17 you are asked to solve another variant of this problem.

For the example in Fig. 6.9 we have labeled t = 1 as a decision horizon and $\hat{t} = 17/6$ as a strong forecast horizon. By this we mean that the optimal decision in [0, 1] continues to be to sell at the maximum rate regardless of the price trajectory p(t) for t > 17∕6. Because $\hat{t} = 17/6$ is a strong forecast horizon, we can terminate the price shield at that time as shown in the figure.

In order to illustrate the statements in the previous paragraph, we consider two examples of price changes after $\hat{t} = 17/6.$

Example 6.3

Assume the price trajectory to be

$$\displaystyle{p(t) = \left \{\begin{array}{ll} - 2t + 7 &\mbox{ for }t \in [0,2), \\ t + 1 &\mbox{ for }t \in [2,17/6), \\ 25t/7 - 44/7&\mbox{ for }t \in [17/6,4], \end{array} \right.}$$

which is sketched in Fig. 6.10. Note that the price trajectory up to time 17∕6 is the same as before, and the price after time 17∕6 goes above the extension of the price shield in Fig. 6.9.

Figure 6.10

Optimal policy and horizons for Example 6.3

Solution The new λ ₂ trajectory is shown in Fig. 6.10, which is the same as before for t < 17∕6, and after that it is λ ₂(t) = t∕2 + 6 for t ∈ [17∕6,  4]. The optimal policy is as shown in Fig. 6.10, and as previously asserted, the optimal policy in [0,1) remains unchanged. In Exercise 6.17 you are asked to verify the maximum principle for the solution of Fig. 6.10.

Example 6.4

Assume the price trajectory to be

$$\displaystyle{p(t) = \left \{\begin{array}{ll} - 2t + 7 &\mbox{ for }t \in [0,2), \\ t + 1 &\mbox{ for }t \in [2,17/6), \\ - t/2 + 21/4&\mbox{ for }t \in [17/6,4], \end{array} \right.}$$

which is sketched in Fig. 6.11.

Figure 6.11

Optimal policy and horizons for Example 6.4

Solution Again the price trajectory is the same up to time 17∕6, but the price after time 17∕6 is declining. This changes the optimal policy in the time interval [1, 17∕6), but the optimal policy will still be to sell in [0, 1). 

As in the beginning of the section, we solve (6.90) to obtain λ ₂(t) = t∕2 + 5∕4 for $\hat{t}_{1} \leq t \leq 4,$ where $\hat{t}_{1} \geq 1$ is the time of the last jump which is to be determined. It is intuitively clear that some profit can be made by buying and selling to take advantage of the price rise between t = 2 and t = 17∕6. For this, the λ ₂(t) trajectory must cross the price trajectory between times 2 and 17∕6 as shown in Fig. 6.11, and the inventory y must go to 0 between times 17∕6 and 4 so that λ ₂ can jump downward to satisfy the ending condition λ ₂(4⁻) = p(4) = 13∕4. Since we must buy and sell equal amounts, the point of intersection of the λ ₂ trajectory with the rising price segment, i.e., $\hat{t}_{1}-\alpha$, must be exactly in the middle of the two other intersection points, $\hat{t}_{1}$ and $\hat{t}_{1} - 2\alpha$, of λ ₂ with the two declining price trajectories. Thus, $\hat{t}_{1}$ and α must satisfy:

$$\displaystyle\begin{array}{rcl} -2(\hat{t}_{1} - 2\alpha ) + 7 +\alpha /2& =& (\hat{t}_{1}-\alpha ) + 1, {}\\ (\hat{t}_{1}-\alpha ) + 1 +\alpha /2& =& -\hat{t}_{1}/2 + 21/4. {}\\ \end{array}$$

These can be solved to yield $\hat{t}_{1} = 163/54$ and α = 5∕9. The times $\hat{t}_{1},\hat{t}_{1}-\alpha,$ and $\hat{t}_{1} - 2\alpha$ are shown in Fig. 6.11. The λ ₂ trajectory is given by

$$\displaystyle{\lambda _{2}(t) = \left \{\begin{array}{ll} t/2 + 9/2 &\mbox{ for }t \in [0,1), \\ t/2 + 241/108&\mbox{ for }t \in [1,163/54), \\ t/2 + 5/4 &\mbox{ for }t \in [163/54,4]. \end{array} \right.}$$

Evaluation of the Lagrange multipliers and verification of the maximum principle is similar to that for the case in Fig. 6.9.

In Sect. 6.3 we have given several examples of decision horizons and weak and strong forecast horizons. In Sect. 6.3.1 we found a decision horizon which was also a weak forecast horizon, and it occurred exactly when y(t) = 0. We also introduced the idea of a price shield in that section. In Sect. 6.3.2 we imposed a warehousing constraint and obtained the same decision horizon and a strong forecast horizon, which occurred when y(t) = 1. 

Note that if we had solved the problem with T = 1, then y ^∗(1) = 0; and if we had solved the problem with T = 17∕6, then y ^∗(1) = 0 and y ^∗(17∕6) = 1. The latter problem has the smallest T such that both y ^∗ = 0 and y ^∗ = 1 occur for t > 0, given the price trajectory. This is one of the ways that time t = 17∕6 can be found to be a forecast horizon along with the decision horizon at time t = 1. There are other ways to find strong forecast horizons. For a survey of the literature, see Chand et al. (2002) .

Exercises for Chapter 6

E 6.1

Verify the expressions for a ₁ and a ₂ given in (6.16) and (6.17).

E 6.2

Verify (6.27). Note that ρ = 0 is assumed in Sect. 6.1.4.

E 6.3

Verify (6.29). Again assume ρ = 0. 

E 6.4

Given the demand function

$$\displaystyle{S = t(t - 4)(t - 8)(t - 12)(t - 16) + 30,}$$

$\rho = 0,\hat{I} = 15,T = 16,\mbox{ and }\alpha = 1,$ obtain Q(t) from (6.27).

E 6.5

Complete the solution of Example 6.2 in Sect. 6.1.4.

E 6.6

For the model of Sect. 6.1.6, derive the turnpike triple by using the conditions in (6.39).

E 6.7

Solve the production-inventory model of Sect. 6.1.6 for the parameter values listed on Fig. 6.4, and draw the figure using MATLAB or another suitable software.

E 6.8

Give an intuitive interpretation of (6.55).

E 6.9

Assume that there is a transaction cost cv ² when v units of wheat are bought or sold in the model of Sect. 6.2.1. Derive the form of the optimal policy.

E 6.10

In Exercise 6.9, assume T = 10, x(0) = 10, y(0) = 0, c = 1∕18, h(y) = (1∕2)y ², V ₁ = V ₂ = ∞, r = 0, and p(t) = 10 + t. Solve the resulting TPBVP to obtain the optimal control in closed form.

E 6.11

Set up the two-point boundary value problem for Exercise 6.9 with c = 0. 05,  h(y) = (1∕2)y ², and the remaining values of parameters as in the model of Sect. 6.2.3.

E 6.12

Use Excel, as illustrated in Sect. 2.5, to solve the TPBVP of Exercise 6.11.

E 6.13

Show that the solution obtained for the problem in Sect. 6.2.3 satisfies the necessary conditions of the maximum principle. Conclude the optimality of the solution by showing that the maximum principle conditions are also sufficient.

E 6.14

Re-solve the problem of Sect. 6.2.3 with V ₁ = 2 and V ₂ = 1. 

E 6.15

Compute the optimal trajectories for μ ₁,  μ ₂, and η for the model in Sect. 6.2.4.

E 6.16

Solve the model in Sect. 6.2.4 with each of the following conditions:

$$\displaystyle{\begin{array}{ll} (a)&y(0) = 2.\\ (b)&T = 10\mbox{ and }p(t) = 2t - 2\mbox{ for }3 \leq t \leq 10. \end{array} }$$

E 6.17

Verify that the solutions shown in Figs. 6.10 and 6.11 satisfy the maximum principle.

E 6.18

Re-solve the model of Sect. 6.3.2 with y(0) = 1∕2 and with the warehousing constraint y ≤ 1∕2 in place of (6.82).

E 6.19

Solve and interpret the following production planning problem with linear inventory holding costs:

$$\displaystyle{ \left \{\begin{array}{ll} \max \left \{J =\int _{ 0}^{T} - [hI + \frac{c} {2}P^{2}]dt\;\right \} \\ \mbox{ subject to } \\ \dot{I} = P,\;I(0) = 0,\;I(T) = B;\;0 < B < hT^{2}/2c, \\ P \geq 0\mbox{ and }I \geq 0.\end{array} \right. }$$

(6.94)

E 6.20

Re-solve Exercise 6.19 with the state equation $\dot{I}(t) = P(t) - S(t),$ where I(0) = I ₀ ≥ 0 and I(T) is not fixed. Assume the demand S(t) to be continuous in t and non-negative. Keep the state constraint I ≥ 0, but drop the production constraint P ≥ 0 for simplicity. For specificity, you may assume S = −sinπt + C with the constant C ≥ 1 and T = 4. (Note that negative production can and will occur when initial inventory I ₀ is too large. Specifically, how large is too large depends on the parameters of the problem.)

E 6.21

Re-solve Exercise 6.19 with the state equation $\dot{I}(t) = P(t) - S,$ where S > 0 and h > 0 are constants, I(0) = I ₀ > cS ²∕2h, and I(T) is not fixed. Assume that T is sufficiently large. Also, graph the optimal P ^∗(t) and I ^∗(t),  t ∈ [0, T].