Optimal Control and Economic Dynamics

Abstract

Optimal control methods and the related methods of dynamic programming and the calculus of variations are ubiquitous in the analysis of dynamic economic systems. This is so because the serious modeller of dynamic economic phenomena in positive economics or in welfare economics, in capitalistic economies or in socialist economies is forced to do four things (i) model the restraints that absence of intertemporal arbitrage opportunities places upon the evolution of the economy over time, (ii) relate expectations of future prices to actual past prices and present prices in a useful notion of equilibrium, (iii) model the learning by the economy’s participants of relevant parameters in an evolving economy (iv) design the models so they lead naturally to the implementation of received methods of econometrics in order to confront their predictions with data.

Optimal control methods and the related methods of dynamic programming and the calculus of variations are ubiquitous in the analysis of dynamic economic systems. This is so because the serious modeller of dynamic economic phenomena in positive economics or in welfare economics, in capitalistic economies or in socialist economies is forced to do four things (i) model the restraints that absence of intertemporal arbitrage opportunities places upon the evolution of the economy over time, (ii) relate expectations of future prices to actual past prices and present prices in a useful notion of equilibrium, (iii) model the learning by the economy’s participants of relevant parameters in an evolving economy (iv) design the models so they lead naturally to the implementation of received methods of econometrics in order to confront their predictions with data.

For the positive economist the objective is to achieve an analytically tractable framework to explain and organize data.

For the normative economist the objective is to achieve an analytically tractable framework to analyse the following issues detailed below which are central to economics. In order that the welfare conclusions carry conviction with scientists as well as with philosophers, this framework should be compatible with that designed by the positive economist who is disciplined by confrontation with data. Some issues are: (i) Is capitalism inherently unstable or inherently stable? What forces determine the speed of adjustment to (or divergence from) steady state evolution? (ii) Is it possible to decentralize a planned economy with prices or with some other signals? Is decentralization possible with the micro agents needing to know only a finite number of prices or other signals at each point in time? (iii) Does speculation serve any socially useful purpose?

Section “The Framework” of this entry exposits an optimal control framework to deal with these issues. The section develops notions of stability that are used in economic dynamics, while section “The Case δ Near Zero” develops the proposition that if agents do not discount the future very much then a centrally planned multisector economy is asymptotically stable under general conditions, that is any two trajectories come together rather than diverge as time progresses. The notions of bliss and overtaking criteria are exposited in these two sections. These notions play a key role in asymptotic stability theory of optimal control.

Section “Some Economic Applications of the Theory” contains a brief exposition of the modern theory of speculative bubbles, manias, and hyperinflations. This theory uses the necessity of the transversality condition of optimal control to investigate possible market forces that may temper the inherent instability displayed by the equations for the myopic perfect foresight asset market equations.

Section “Equilibrium Dynamics” reviews an approach to adjustment dynamics and Samuelson’s correspondence principle inspired by optimal control methods. The basic idea is to use optimal control and rational expectations to endogenize the adjustment dynamics with respect to (wrt) which the hypothesis of stability is used to place restrictions on comparative statics. In this way one can push the correspondence principle further than the original version, where the dynamics were ad hoc. This is so because endogenized dynamics contain more restrictions linked to tastes and technology than ad hoc dynamics. Finally section “A Summing Up” presents a brief summing up.

The Framework

In continuous time the general optimal control problem is stated thus:

$$ V\left(y,{t}_0\right)\equiv \max {\int}_{t_0}^Tv\left(x,u,s\right)\mathrm{d}s+B\left[x(T),T\right], $$

(1.1)

$$ \mathrm{s}.\mathrm{t}.\dot{x}=f\left(x,u,t\right),x\left({t}_0\right)=y. $$

(1.2)

where V: Rⁿ × R → R; f: Rⁿ × R^m × R → Rⁿ; v: Rⁿ × R^m × R → R; B: Rⁿ × R → R. Here V is the state valuation function, also called the indirect utility function, starting at state y at time t₀; υ is the instantaneous utility or payoff when the system is in state x=x(s)∈Rⁿ at time s, and control u=u(s)∈R^m is applied at date s; B is a bequest or scrap value function giving the value of the state x (T) at date T; and $\dot{x}$ ≡ dx/dt = f (x,u,t) gives the law of motion of the state. The discrete time version of step size h of (1.1) and (1.2) is analogous, with $\dot{x}$ replaced by (x (t + h) − x(t))/h, ∫ replaced by Σ. Under modest regularity conditions the solution to the discrete time problem converges to the solution to the continuous time problem as h → 0. The horizon T may be finite or infinite.

Under regularity assumptions, by dynamic programming the value function V satisfies the Hamilton–Jacobi–Bellman (HJB) equation; furthermore the co-state–state necessary conditions must be satisfied with p ≡ V_x:

$$ -{V}_t=\underset{u}{\max }{H}^{\ast}\left(p,x,u,t\right)\equiv {H}^{\ast 0}\left(p,x,t\right),\left(\mathrm{HJB}\ \mathrm{equation}\right) $$

(1.3)

$$ {H}^{\ast}\left(p,x,u,t\right)\equiv v+ pf,\left(\mathrm{Hamiltonian}\ \mathrm{definition}\right) $$

(1.4)

$$ \dot{p}=-{H}_x^{\ast 0},\dot{x}={H}_p^{\ast 0},x\left({t}_0\right)=y,\left(\mathrm{co}-\mathrm{state}\ \mathrm{equations}\right) $$

(1.5)

$$ V\left(x,T\right)=B\left(x,T\right),p(T)={B}_x\left(x,T\right),\left(\mathrm{transversality}\ \mathrm{conditions}\right) $$

(1.6)

The variable p is called the costate variable, adjoint variable, or dual variable; and the function H^* is called the Hamiltonian. These variables are introduced for the same reasons and have the same interpretation that Lagrange–Kuhn–Tucker multipliers are introduced in nonlinear programming. The terminal conditions (1.6) are sometimes called transversality conditions.

Equations (1.3)–(1.6) are the workhorses of optimal control theory. We briefly explain their derivation and meaning here.

Equation (1.1) may be written:

$$ {\displaystyle \begin{array}{ll}\hfill & V\left(y,{t}_0\right)=\max \left\{{\int}_{t_0}^{t_0+h}\upsilon \left(x,u,s\right)\mathrm{d}s+{\int}_{t_0+h}^T\upsilon \left(x,u,s\right)\mathrm{d}s+B\left[x(T),T\right]\right\}\\ {}& =\max \left[{\int}_{t_0}^{t_0+h}\upsilon \left(x,u,s\right)\mathrm{d}s+\max \left\{{\int}_{t_0+h}^T\upsilon \left(x.u,s\right)\mathrm{d}s+B\left[x(T),T\right]\right\}\right]\hfill \\ {}& =\max \left\{{\int}_{t_0}^{t_0+h}\upsilon \left(x,u,s\right)\mathrm{d}s+V\left[x\left({t}_0+h\right),{t}_0+h\right]\right\}\hfill \\ {}& =\max \left\{\upsilon \left(y,u,{t}_0\right)h+V\left(y,{t}_0\right)+{V}_x\left(y,{t}_0\right)\Delta x+{V}_t\left(y,{t}_0\right)h+o(h)\right\}\hfill \\ {}& =\max \left\{\upsilon h+V\left(y,{t}_0\right)+{V}_x fh+{V}_th+o(h)\right\}.\hfill \end{array}} $$

(1.7)

The first equation is obvious; the second follows from the following principle called the ‘principle of optimality’: to maximize a total sum of payoffs from x(t0) = y over [t0, T] you must maximize the subtotal of the sum of payoffs from x(t0 + h) over [t0 + h, T]; the third follows from the definition of the state valuation function; the fourth follows from the integral mean value theorem and expansion of V(x (t0 + h), t0 + h ) in a Taylor series about x(t0) = y, t0; and the fifth follows from Δx ≡ x(t0 + h) − x(t0) = fh + o(h). Here o(h) is any function of h that satisfies

$$ \underset{h\to 0}{\lim }o(h)/h=0. $$

Subtract V(y, t₀) from the LHS and the extreme RHS of the above equation; divide by h and take limits to get (1.3). So (1.3) is nothing but the principle of optimality in differential form. That is all there is to the HJB equation.

Equation (1.4) is just a definition. To motivate this definition rewrite equation (1.7), thus putting p ≡ V_x.

$$ -{V}_t=\max \left\{v\left(y,u,{t}_0\right)+ pf\left(y,u,{t}_0\right)+o(h)/h\right\}. $$

(1.8)

The function H^*, called the Hamiltonian function, just collects the terms that contain the control u. The control u must be chosen to maximize H^* along an optimum path. This follows directly from equation (1.7).

The principle that the optimal control u⁰ must maximize H^* is important. It is called the maximum principle. This principle squares with commmon sense: you should choose the control to maximize the sum of current instantaneous payoff u(y,u,t₀) and future instantaneous value $p\dot{x}= pf\left(y,u,{t}_0\right),p\equiv {V}_x$. The quantity p, called the costate variable, is the marginal value of the state variable. It measures the incremental sum of payoffs from an extra unit of state variable. Equations (1.5) are easy to derive. The relation $\dot{x}={H}_p^{\ast 0}$ follows from $\dot{x}=f\left(x,{u}^0,t\right)$ and the envelope theorem. The relation $\dot{p}=-{H}_x^{\ast 0}$ follows from substitution of the derivative of (1.3) wrt x into the expression for dp/dt=(d/dt)V_x.

Finally (1.6) is obvious. If there is an inequality constraint x(t) ≧ 0 for all t, but B≡0, then, the transversality condition, p(T) = B_x(x, T) takes the form p(T)x(T) = 0. The condition p(T)x(T) = 0 means that nothing of value is left over at the terminal date T. When T is infinite, for a large class of problems the condition takes the form

$$ \underset{T\to \infty }{\lim }p(T)x(T)=0 $$

(1.9)

and is called the transversality conditions at infinity. Benveniste and Scheinkman (1982), Araujo and Scheinkman (1983), and Weitzman (1973) show that (1.9) is necessary and sufficient for optimality for a large class of problems.

Let me give a very rough heuristic argument to motivate why (1.9) might be necessary for optimality. For any date T with terminal date in (1.1) set equal to infinity, assume the state valuation function V(y, T) is concave in y. (Note that ‘t₀’ is replaced with ‘T’ and ‘T’ is replaced by ‘∞’ in (1.1) here.) Use concavity and p(T) ≡ V_x(x(T), T) to get the bound

$$ V\left(x(T),T\right)-V\left(x(T)/2,T\right)\geqq {V}_x\left(x(T),T\right)x(T)/2=p(T)x(T)/2 $$

(1.10)

Now suppose that the distant future is insignificant in the sense that V(z(T), T) → 0, T ∞ ∞ for any state path z. Then it is plausible to expect that the LHS of (1.10) will go to 0 as T ∞ ∞. If x(T) ≧ 0 and p(T) ≧ 0 (more x is better than less) then

$$ \underset{T\to \infty }{\lim }p(T)x(T)=0 $$

which is (1.9).

Examples exist where (1.9) is not necessary for optimality. The idea is that if the distant future is ‘significant’ then there is no reason to expect the value of ‘leftovers’ p(T)x(T) to be forced to zero along an optimum path. See Benveniste and Scheinkman (1982), and Araujo and Scheinkman (1983) for the details and references.

In the same manner and for the same reasons as a time series analyst transforms his time series to render it time stationary the dynamic economic modeller searches for a change of units so that (abusing notation to economize on clutter) problem (1.1) may be written in the time stationary form

$$ V\left(y,{t}_0\right)={\int}_{t_0}^T{\mathrm{e}}^{-\delta t}v\left(x,u\right)\mathrm{d}s+{\mathrm{e}}^{-\delta t}B\left[x(T)\right] $$

(1.11)

$$ \dot{x}=f\left(x,u\right),x\left({t}_0\right)=y. $$

(1.12)

By the change of units W(y, t₀) = e^δtV(y, t₀), q = e^δtp, H = e^δtH^* and we may write the optimality conditions (1.3)–(1.6) in the form:

$$ \delta W-{W}_t=\underset{u}{\max }H\left(q,x,u\right)={H}^0\left(q,x\right) $$

(1.13)

$$ H\left(q,x,u\right)\equiv v\left(x,u\right)+ qf $$

(1.14)

$$ \dot{q}=\delta q-{H}_x^0,\dot{x}={H}_q^0,x\left({t}_0\right)=y $$

(1.15)

$$ W\left(x,T\right)=B(x),q(T)={B}_x(x). $$

(1.16)

When the horizon T = ∞, W becomes independent of T so that W_t = 0; the transversality condition becomes (cf. Benveniste and Scheinkman 1982)

$$ \underset{t\to \infty }{\lim }{\mathrm{e}}^{-\delta t}q(t)x(t)=0, $$

(1.17)

and (1.17) is necessary as well as sufficient, for a solution of (1.15) to be optimal. The condition (1.17) determines q₀.

Equipped with the framework (1.11) and (1.12) together with the optimality conditions (1.13)–(1.17) we are now ready to discuss the economic questions mentioned in the introduction.

Stability

We now have a framework in which to discuss stability of an ideal centrally planned economy. After we do that we will show that the same framework can be used to study related issues in an ideal capitalist economy.

There are five basic notions of stability:

• (i) stability of the optimum path with respect to small changes in the horizon and target stocks;

• (ii) stability of the optimum path with respect to small changes in υ, f;

• (iii) existence of an optimum steady state ($\overline{x}$, ū) and asymptotic stability of optimum paths wrt ($\overline{x}$, ū);

• (iv) asymptotic stability of (x(t, u(t)) wrt ($\overline{x}$(t), ū(t)) for any two optimum paths (x(t), u(t)), ($\overline{x}$ (t), ū(t));

• (v) asymptotic stability of optimal paths x(t) towards a general attractor set Λ.

  First, there is an extensive literature (e.g., Mitra 1979, 1983; Majumdar and Zilcha (1987), and their references) that studies the conditions that one must impose upon υ υ, f in order that

  $$ \underset{T\to \infty }{\lim }x\left(t,{x}_0,T\right)=x\left(t,{x}_0,\infty \right) $$

  (2.1)

where x(t, x₀, T), x(t, x₀, ∞) denote solutions to problem (1.1) with T finite and infinite respectively. Here x(t₀, x₀, T) = x(t₀, x₀, ∞) = x₀. Sufficient conditions on υ, f needed to obtain the insensitivity result (2.1) are very weak. The result (2.1) is important because it shows that the choice of the terminal time T is unimportant for the initial segment of an optimal plan provided that T is large. We do not have space here to discuss the ‘insensitivity’ literature any further.

The second notion of stability requires that optimal solutions do not change much when the functions υ, f do not change much. We shall not treat this type of stability in this entry. It is a standard topic in the mathematical theory of optimal control and can be found in many textbooks on the subject. In many economic applications the conditions sufficient for this type of stability are automatically imposed. This kind of stability is a minimal requirement to impose on a problem in order that it be ‘well posed’.

The third notion of stability is ubiquitous in economic analysis. The basic notions are easy to explain.

Definitions

The pair of vectors ($\overline{q}$,$\overline{x}$) ∈ R²ⁿ is an optimal steady state (OSS) if ($\overline{q}$,$\overline{x}$) solves (1.15) while $\dot{q}$ = 0, $\dot{x}$= 0. The optimal steady state $\overline{x}$ is said to be locally (globally) asymptotically stable if the solution x(t, y) of the optimal dynamic system

$$ \dot{x}={H}_q^0\left(q,x\right)={H}_q^0\left({W}_x(x),x\right)\equiv h(x), x\left({t}_0\right)=y $$

converges to $\overline{x}$ as t → ∞ for initial conditions y near $\overline{x}$ (for all initial conditions y).

The Case δ Near Zero

We will show in this case that a centrally planned multisector economy is asymptotically stable under modest concavity assumptions. The case δ = 0 is the case where the central planner does not discount the future. F. P. Ramsey’s famous paper (1928) on one sector optimal growth introduced the notion of bliss in order to deal with the possibly non-convergent integral in (1.7) for the infinite horizon case. That is to say Ramsey put B equal to the maximum obtainable rate of utility or enjoyment and minimized ${\int}_0^{\infty}\left(B-\upsilon \right)\mathrm{d}t\equiv R\left({x}_0\right)$ and his famous rule: B − υ = $\dot{x}$u′ follows directly from the HJB equation for R.

The desire to treat utility functions that did not satiate, to treat multiple sectors, and to treat classes of problems where Ramsey’s integral ${\int}_0^{\infty}\left(B-\upsilon \right)\mathrm{d}t$ was not well defined led later investigators (von Weiszäcker 1965; Gale 1967; Brock 1970) to replace

$$ B\;\mathrm{by}\;\overline{\upsilon}\equiv \underset{x,u}{\max}\upsilon \left(x,u\right) \mathrm{s}.\mathrm{t}.f\left(x,u\right)\ge 0, $$

and to introduce the overtaking ordering (von Weiszäcker 1965) in various guises. We explain two common versions of overtaking type orderings and their corresponding notions of optimality here. McKenzie’s (1976), (1981), syntax is used.

Definitions

let Z ≡ (x,u),Z ≡ (x′,u′) be two paths. We say that Z catches up to Z′ if

$$ \overline{\underset{T\to \infty }{\lim }}{\int}_0^T\left[\upsilon \left({Z}^{\prime}\right)-\upsilon (Z)\right]\mathrm{d}t\le 0. $$

(3.1)

Here $\overline{\lim}\;{a}_T$ denotes the larges cluster point (i.e., the limit superior) of the sequence a_T as T → ∞. Inequality (3.1) states that the accrued utility along Z eventually exceeds the accured utility along Z′. as T → ∞. This defines a partial ordering of paths Z, Z′. An optimal path (Gale 1967) catches up to every other path that starts from the same initial in conditions x₀. We say that Z′overtakesZ if there is ε > 0 such that

$$ \underset{T\to \infty }{\underline{\lim}}{\int}_0^T\left[\upsilon \left({Z}^{\prime}\right)-\upsilon (Z)\mathrm{d}t\ge \varepsilon .\right. $$

(3.2)

A weakly maximal path (Brock 1970) is not overtaken by any other path that starts from the same initial condition x_o. an optimal path beats every other path. A weakly maximal path is not beaten by any other path.

Under the assumption of strict concavity of the payoff and convexity of the constraint set Gale (1967) proved for a discrete time model that a unique optimal path existed and the unique optimal steady state was globally asymptotically stable. For the same model Brock (1970) replaced Gale’s strict concavity assumption on the payoff with the weaker assumptions of concavity of the payoff, uniqueness of the optimal steady state, and convexity of the technology, and, under these weaker assumptions, shortened the proof of Gale’s existence theorem, proved existence of weakly maximal programmes, gave an example where the optimal steady state failed to be optimal in the class of all paths starting from it, and proved that time averages of weakly maximal paths converged to the optimal steady state even though the paths themselves may not converge. Continuous time versions of these theorems are in Brock and Haurie (1976). The assumptions needed in the continuous time case basically amount to concavity of H⁰(q, x) in x.

Theorems of this type are useful for the stability question because they show the truth of the following proposition.

Proposition

If you do not discount the future and you make the usual concavity and convexity assumptions of diminishing marginal rates of substitution and nonincreasing returns on utility and technology then all optimal paths converge to a unique optimal steady state.

This is a strong result. It is independent of the number of sectors. A similar result holds for δ near zero (Scheinkman 1976). These results may be motivated as follows. Linearize (1.15) about the optimal steady state ($\overline{q}$,$\overline{x}$) to obtain, putting

$$ \Delta z=\left[\begin{array}{l}\Delta q\\ {}\Delta x\end{array}\right],\Delta \dot{z}=J\Delta z,\Delta x(0)={x}_0-\overline{x}, $$

(3.3)

where J is defined by

$$ J=\left[\begin{array}{cc}\hfill \delta -{H}_{xq}^0\hfill & \hfill -{H}_{xx}^0\hfill \\ {}\hfill {H}_{qq}\hfill & \hfill {H}_{qx}^0\hfill \end{array}\right]. $$

(3.4)

It is known (see Levhari and Leviatan 1972, for the discrete time analogue) that if λ is an eigenvalue of J so is −λ + δ.

In the case δ = 0 we see that eigenvalues of J came in pairs −λ, λ so that, except for hairline cases, exactly n of the eigenvalues have negative real parts and exactly half of the eigenvalues have positive real parts. Hence, except for hairline cases, the stable manifold LW_s of (3.3), which is called the local stable manifold of (1.15) (i.e., the set of (Δq(0), Δx(0)) such that the solution of (3.3) starting from (Δq(0), Δx(0)) converges to (0, 0)) is an n-dimensional vector space embedded in R²ⁿ whose projection on x-space is n-dimensional. In the ‘nondegenerate case the space LW_s is the linear vector space in R²ⁿ that is spanned by the n eigenvectors corresponding to the n eigenvalues with negative real parts. To put it another way, except for hairline cases, to each Δx(0) there is a unique Δq(0) such that (Δx(0), Δq(0)) ∈ LW_s. Unstable manifolds are defined the same way by reversing the flow of time.

Now the stable manifold W_s of (1.15) at (q, x), which is defined by W_s ≡ {(q₀,x₀)| the solution of (1.15) starting from (q₀,x₀) converges to ($\overline{q}$,$\overline{x}$) as t → ∞} is tangent to LW_s at ($\overline{q}$,$\overline{x}$). The existence and stability theorems for δ = 0 show that the initial costate q₀ must be chosen so that (q₀,x₀)∈W_s for each initial state x₀.

Scheinkman’s result (1976) may be interpreted intuitively as continuity of W_s in δ at δ = 0, so global asymptotic stability of an optimal steady state holds provided that δ is near zero. That is to say, in nondegenerate cases, the manifold W_s does not change much when δ does not change much. There is another way to see the role a small δ plays in ensuring stability of a multisector economy.

Differentiate the function

$$ V={\dot{q}}^T\dot{x}={\dot{x}}^T{W}^{{\prime\prime}}\dot{x}\le 0 $$

(3.5)

along solutions of (1.15) that satisfy the transversality condition (1.17) [which by Benveniste–Scheinkman (1982) is necessary for optimum] to obtain

$$ \dot{V}={\dot{z}}^TQ\dot{z} $$

(3.6)

where

$$ Q=\left[\begin{array}{cc}\hfill \delta /2{I}_n\hfill & \hfill -{H}_{xx}^0\hfill \\ {}\hfill {H}_{qq}^0\hfill & \hfill \delta /2{I}_n\hfill \end{array}\right] $$

(3.7)

Equation (3.6) is easy to derive. Differentiate (1.15) wrt t and substitute the results into $\dot{V}={\ddot{q}}^T\dot{x}+{\dot{q}}^T\ddot{x}$. Let α, β denote the smallest eigenvalue of $-{H}_{xx}^0,{H}_{qq}^0$ respectively. Brock and Scheinkman (1976) show that

$$ 4\alpha \beta >{\delta}^2 $$

(3.8)

implies Q is positive definite so V increases and, hence, global asymptotic stability (G.A.S.) holds. This is so because V is always negative (cf. (3.5)) and is zero only at $\overline{x}$ where $\dot{x}$ = 0. It can be shown that (3.8) implies that the optimal steadly state $\overline{x}$ is unique. Hence V increasing in time forces convergence of x(t) to $\overline{x}$ as t → ∞. Since, except for hairline cases, $-{H}_{xx}^0,{H}_{qq}^0$ are positive definite for problems with H⁰ concave in the state x, therefore G.A.S. holds provided that δ is small enough.

Finally there is yet one more way to see why a small δ forces global asymptotic stability of optimal paths. Put δ = 0 and look at the objective.

$$ {}^{`}{\mathrm{max}}^{'}{\int}_0^{\infty}\left[v\left(x,u\right)-v\left(\overline{x},\overline{u}\right)\right]\mathrm{d}t, \mathrm{s}.\mathrm{t}.\dot{x}=f\left(x,u\right), $$

(3.9)

Here ‘max’ means weak maximality. Now under strict concavity of υ, f in (x, u) and natural monotonicity usually assumed in economic applications ($\overline{x}$, ū) is the unique solution to the nonlinear programming problem.

$$ \max v\left(x,u\right)\mathrm{s}.\mathrm{t}.f\left(x,u\right)\ge 0. $$

(3.10)

Hence, intuitively (x(t), u(t)) must converge to ($\overline{x}$, ū) otherwise (3.9) would blow up since the future is not discounted. See Brock and Haurie (1976) for the details. So if δ is close to zero, by continuity of W_s in δ, global asymptotic stability to a unique steady state is preserved. McKenzie (1974) treats the case where ($\overline{x}$, ū) depends on t.

We have focused on asymptotic stability in the foregoing. It is natural to ask what economic forces cause instability in a centrally planned economy. Intuitively, instability is present when the underlying dynamics $\dot{x}$ = f (x, u) are unstable when no control u is applied, when control is ineffective (∂f/∂u is ‘small’ in ‘absolute value’), when control is expensive, when it is not costly to be out of equilibrium in the state, and when the discount δ, on the future is large. This seems clear. Why spend a lot of resources now in ineffective expensive control to push an economy back into state equilibrium when it currently costs little to be out of equilibrium and benefits arrive in the future which is deeply discounted? A discussion on instability and alternative sufficient conditions for asymptotic stability to those presented here is in Brock (1977). We have no more space to discuss it here. In any event the notions of ‘overtaking’ and ‘bliss’ were introduced mainly to resolve issues of existence of optimum paths (Magill 1981) and to investigate asymptotic stability of optimum paths when the future is not discounted.

It is possible for trajectories of centrally planned economies to converge to a limit set Λ that is not a steady state or even a limit cycle. There are more complicated limit sets called ‘strange’ attractors: they have the property that each pair of nearby trajectories starting in Λ locally diverge at an exponential rate and each trajectory in Λ moves in an apparently ‘random’ manner. But as we have seen above such ‘unstable’ phenomena cannot appear when future payoffs are worth almost as much as present payoffs. See Grandmont (1986) for literature on strange attractors in economics as well as literature on empirically testing economic time series for the presence of strange attractors.

Since, as we shall see in section “Some Economic Applications of the Theory” below, each model of a centrally planned economy has a rational expectations market model analogue; therefore the stability literature discussed above applies directly to market models. The strategy of turning optimal growth models into market models and borrowing results from optimal growth theory is at the heart of much of modern macroeconomics and real theories of the business cycle (Kydland and Prescott 1982; Long and Plosser 1983). This kind of application has made the analytical techniques discussed above an essential element of the modern economist’s tool-box. We turn now to some of the applications mentioned in the introduction.

Some Economic Applications of the Theory

Are Asset Markets Inherently Unstable?

Rewrite equations (1.15) as

$$ {\dot{q}}_t/{q}_i+{H}_{x_i}^0/{q}_i=\delta, $$

(4.1)

$$ {\dot{x}}_i={H}_{q_i}^0,i=1,2,\dots, n,{x}_0 \mathrm{given}, $$

(4.2)

and interpret (4.1) as ‘capital gains on asset i plus net yield on asset i = a common rate of return δ’, and (4.2) as ‘demand for investment in i = supply of investment in i’. The system (4.1), (4.2) has similar mathematical structure to the system of equations describing a market for n assets under myopic perfect foresight analysed by F. Hahn (1966). One may view Hahn’s paper as an attempt to formalize the idea held by many people that asset markets are inherently unstable. Indeed Hahn noticed that the linearization of a set of equations much like (4.1), (4.2) around a steady state ($\overline{q}$,$\overline{x}$) displayed a saddle point structure, so that unless q₀ was chosen ‘just right’ (i.e., on the stable manifold at ($\overline{q}$,$\overline{x}$)), then solutions of (4.1), (4.2) starting at (q₀,x₀) would diverge.

The knife-edge problem noticed by Hahn is ubiquitous in models of intertemporal equilibrium in asset markets. See, for example, Gray (1984). Obstfeld and Rogoff (1983, 1986) and references, However, market participants might be expected, knowing the structure of the system (4.1), (4.2), to attempt to forecast the future evolution of earnings of each asset along the solution of the system starting from (q₀, x₀). If capitalized earnings were less than q₀ one would expect traders to bid down q₀, if greater to bid up q₀. Only when q₀ is equal to the present value of anticipated earnings of the asset would one expect no pressure for change of q₀ in the market. Dechert (1978) solves the dynamic integrability problem of when intertemporal equilibrium equations solve some optimal control problem.

The intuitive solution to the knife-edge instability problem given above can be made rigorous for rational expections asset pricing models. See Benveniste and Scheinkman (1982) and references for the deterministic case and Brock (1982, p. 17) for the stochastic case.

To exposit how this line of argument goes, look at the neoclassical one-sector optimal growth model.

$$ W\left({x}_0\right)\equiv \max {\int}_0^{\infty }{\mathrm{e}}^{-\delta t}u(c)\mathrm{d}t,\mathrm{s}.\mathrm{t}.c+\dot{x}=f(x) $$

(4.3)

where u′ > 0, u′(0) = +∞, u′(∞) = 0, u″ < 0, f(0) = 0, f′(0) = +∞, f′ < 0, δ > 0 are the maintained assumptions on utility u and production frunction f. Make an asset pricing model out of this by introducing a representative consumer who faces a, r, π parametrically and solves.

$$ \max {\int}_0^{\infty }{\mathrm{e}}^{-\delta t}u(c)\mathrm{d}t,\mathrm{s}.\mathrm{t}.c+a\dot{z}+\dot{x}= rx+\pi x+\pi z,z(0)=1, x(0)={x}_0 $$

(4.4)

and a representative firm who leases capital from consumers at rate r to solve.

$$ \pi \equiv \underset{x}{\max}\left[f(x)- rx\right] $$

(4.5)

Here a, r, π, z, c, x denote asset price, interest or rental rate, profits, quantity of asset, consumption, and quantity of capital respectively. There is one perfectly divisible share of the asset available at each point in time. General multisector control planning models may be turned into market models in the same way as the single sector model treated here. For example, such a multisector market model is fitted to data and used to explain business cycles in Long and Plosser (1983).

The collection a, r, π, z, c, x an equilibrium if facing a, r, π the solutions of (4.4) and (4.5) agree and z = 1 so that all markets clear at all points in time. The necessary conditions of optimality of c, z, x from (4.4) are

$$ \delta -{\dot{u}}^{\prime }/{u}^{\prime }=r=\dot{a}/a+\pi /a \left(\mathrm{simple}\ \mathrm{control}\ \mathrm{theory}\right) $$

(4.6)

$$ {\displaystyle \begin{array}{ll}\lim {\mathrm{e}}^{-\delta t}{u}^{\prime }x& =\underset{t\to \infty }{\lim \limits }{\mathrm{e}}^{-\delta t}{u}^{\prime } az\hfill \\ {}& =0 \left(\mathrm{Benveniste}-\mathrm{Scheinkman},1982\right)\hfill \end{array}} $$

(4.7)

Equations (4.7) state that the present value of capital and asset stocks must go to zero as t → ∞. Since (4.5) implies r = f′ and z = 1 in equilibrium we must have setting q = u′, c(q) ≡ u′⁻¹,

$$ \dot{q}=\delta q=q{f}^{\prime },\dot{x}=f(x)-c(q),x(0)={x}_0. $$

(4.8)

The system (4.8) which is the dynamics of the standard neoclassical one sector optimal growth model dramatically displays the knife-edge instability discussed by Hahn (1966) when phase diagrammed. We come to the main substantive point of this section:

Proposition: The necessity of the transversality condition at infinity for the consumer’s problem determines the initial value of q₀ and a₀ To put it another way equilibrium c, x are characterized by the solution to (4.3). Furthermore for each t the equilibrium asset price is given by

$$ {\displaystyle \begin{array}{ll}a(t)=& {\int}_0^{\infty}\exp \left[-{\int}_t^s\left(\delta -{\dot{u}}^{\prime }/{u}^{\prime}\right)\mathrm{d}t\right]\hfill \\ {}& \left[f(x)-{f}^{\prime }(x)x\right]\mathrm{d}s\hfill \end{array}} $$

evaluated along the solution to (4.3).

A detailed discussion of this kind of result for the case of uncertainty is in Brock (1982).

At an abstract theoretical level this proposition is a resolution of the classical knife-edge instability problem of capital asset markets but how relevant is such a resolution in practice? The assumption of the absence of arbitrage profits and correct expectations over the short period embodied in (4.6) probably captures a central tendency in well developed asset markets like stock exchanges. It is the long term fundamentalist rationality embodied in (4.7) that is more problematic. A more thorough discussion of the economic plausibility of (4.7) is contained in Gray (1984), Obstfeld and Rogoff (1986), and references. Furthermore there is no allowance for short-term or long-term learning and forecasting in the framework.

The study of learning and disequilibrium adjustment mechanisms in capital asset markets is still in its infancy. The literature has not progressed much beyond the work discussed by Blume et al. (1982).

Nevertheless optimal control theoretic intertemporal general equilibrium models much like the one articulated here have had a large impact on the scientific study of asset market bubbles and speculative manias both theoretical (e.g., Gray (1984), Obstfeld and Rogoff (1983), (1986), and empirical (e.g., Flood and Garber 1980; Meese 1986). Indeed, one might say that such methods launched the modern empirical study of bubbles, hyperinflations and speculative manias.

The ‘theoretical resolution’ of the short-run instability of myopic perfect foresight asset markets has a family resemblance to the problem of decentralization of an infinitely lived economy with the microagents using only a finite number of prices or other signals at each point in time. For example, in the model discussed above, the presence of a stock market forced Pareto optimality of all equilibria. This conclusion is also true in many cases for models where individuals have finite lives (Tirole 1985). Hence, in a sense, decentralizability can be achieved by a finite number of markets at each point in time even though the economy is infinitely lived. To put it another way, in Samuelson–Diamond overlapping generations models where competitive equilibria may be inefficient the mere addition of a stock market eliminates the inefficient equilibria. See Tirole’s (1985) discussion of unpublished work by J. Scheinkman for the argument.

Equilibrium Dynamics

We have seen how the notion of transversality condition at infinity contributed to the theoretical and empirical investigation of instability and bubbles in markets for speculative assets. Turn now to a contribution of the asymptotic stability theory of optimal control to the modelling of adjustment dynamics.

Critical articles such as Gordon and Hines (1970) and Lucas (1976) have made many economists wary of ‘ad hoc’ dynamic models such as the Walrasian tatonnement ent $\dot{p}$ = E(p) where p is price and E is excess demand, as well as techniques such as Samuelson’s Corespondence Principle that rule out ‘unstable’ equilibria wrt such ad hoc dynamics. We exposit here a framework, using optimal control, that gets around the objection that the dynamics are ‘ad hoc’ under adjustment costs.

Suppose that a vector x of goods is produced with convex cost function B(x). Suppose that demand is integrable in the sense that there is a social benefit function B(x) such that Bx = D(x) ≡ p. Then intertemporal competitive equilibrium is characterized by the solution to the surplus maximization problem.

$$ \max {\int}_0^{\infty }{\mathrm{e}}^{-\delta t}\left[B(x)-C\left(x,\dot{x}\right)\right]\mathrm{d}t\equiv W\left({x}_0\right) $$

(5.1)

which yields the necessary conditions

$$ \dot{q}= rq-{H}_x^0,\dot{x}={H}_q^0,x(0)={x}_0,{H}^0\left(q,x\right)\equiv \max \left[B(x)-C\left(x,\dot{x}\right)+q\dot{x}\right]. $$

(5.2)

This is easy to see. For let a representative firm face p parametrically and solve

$$ \max {\int}_0^{\infty }{\mathrm{e}}^{- rt}\left[ px-C\left(x,\dot{x}\right)\mathrm{d}t\right] $$

(5.3)

to yield necessary conditions

$$ \dot{\lambda}= r\lambda -{G}_x^0,\dot{x}={G}_q^0, x(0)={x}_0,{G}^0\left(\lambda, x\right)\equiv \underset{x}{\max}\left[ px-C\left(x,\dot{x}\right)+\lambda \dot{x}\right]. $$

(5.4)

Equilibrium requires

$$ p=D(x). $$

(5.5)

Note that ${H}_x^0={B}_x-{C}_x=D-{C}_x\equiv p-{C}_x={G}_x^0$. Identify λ with q and use Benveniste-Scheinkman’s (1982) theorem on the necessity of the transversality condition at infinity to finish the proof.

Does $\dot{p}$ in the ‘new’ framework where the dynamics are endogenous relate naturally to any notion of ‘excess demand’ as in the traditional but ad hoc Walrasian tatonnement? Differentiate (5.5) along the solution of (5.1) to obtain, denoting the optimal value of $\dot{x}$ by $h(x)\equiv {H}_q^0\left({W}_x(x),x\right)$,

$$ \dot{p}={D}_x\dot{x}={D}_xh(x)\equiv K(x)=K\left({D}^{-1}(p)\right)\equiv L(p). $$

(5.6)

Notice that in the one good case, p moves opposite to x if D_x<0. But there is little relationship between the function L(p) and any obvious notion of ‘excess demand’. This is as it should be, because the optimal dynamics h(x) embodies future information whereas static excess demand depends only upon current information (or, in distributed lag models, past information).

The optimal control framework laid out here can be used to make four points.

First, although the issue of learning is begged, this framework suggests what actors in the model should be learning about in a useful model. That is they should be modelled as learning about the function h(x). See Blume et al. (1982) and their references for literature on learning.

Second, this framework gets around the Gordon–Hines–Lucas objection to ‘ad hoc’ dynamic modelling like the Walrasian tatonnement. No agent in the model, knowing h(x), can make money on this knowledge. Hence the ‘equilibrium’ adjustment dynamics $\dot{x}$ = h(x) are ‘stable’ against profit-seeking behaviour. This shows that it is logically possible to write down models of adjustment dynamics that are immune to the famous ‘Lucas Critique’ (Lucas 1976).

Third, this framework suggests a reformulation of the Samuelson correspondence principle (Brock 1976) that gets around two fundamental objections to Samuelson’s original version: (i) the dynamics were ad hoc and not linked to self-interested purposive behaviour by agents in the model, (ii) the principle had no content because any continuous function can be an excess demand function (the Sonnenschein–Mantel–Debreu Theorem; Debreu 1974). Dynamics (5.6) are equilibrium rational expectations dynamics so objection (i) is met.

Objection (ii) is that the original correspondence principle was contentless since excess demand functions are arbitrary. Although when r is small (5.1) imposes many restrictions on $\dot{x}$ = h(x), it can be shown that there are few restrictions on h provided that r is large enough (Grandmont 1986). Nevertheless the structure of (5.1) has been used to formulate versions of the correspondence principle that exhibit restrictions on comparative statics imposed by global asymptotic stability of $\dot{x}$ =h(x). Perhaps the most important thing to realize is that the results of section “The Case δ Near Zero” imply that the adjustment dynamics $\dot{x}$ = h(x) possess a unique steady state which is globally asymptotically stable when the real interest rate, r, is close enough to 0. This is a very strong restriction on the dynamics $\dot{x}$ = h(x) for the empirically relevant case of small real interest rate. See Brock (1976), Magill and Scheinkman (1979), and McKenzie (1981) for results along this line.

Fourth, quadratic versions of (5.1) with the addition of uncertainty generate a large class of empirically useful and econometrically tractable models. See Sargent (1981) for this development.

A Summing Up

In the applications section of this entry we have shown how optimal control methods have contributed to the investigation of basic economic questions such as inherent stability or instability of capitalism, and in centrally planned economies determination of the strength of forces for and against stability, and decentralizability of economies that last foreover. For an example, myopic perfect-foresight asset market equations display a similar saddle point knifeedge instability to that found in the costate–state equations of optimal control (which are necessary for optimum). The corrective force in optimal control theory is the transversality condition at infinity, which motivates search for market forces that are analogous to it; the modern literature on speculative manias emerged from this search.