Lyapunov Stability Theory

Abstract

Stability of nonlinear systems are discussed in this chapter. Lyapunov stability, asymptotic stability, and exponential stability of an equilibrium point of a nonlinear system are defined. The Lyapunov’s direct method is introduced as an indispensable tool for analyzing stability of nonlinear systems. The Barbashin–Krasovskii theorem provides a method for global stability analysis. The LaSalle’s invariant set theorem provides a method for analyzing autonomous systems with invariant sets. Stability of non-autonomous systems involves the concepts of uniform stability, uniform boundedness, and uniform ultimate boundedness. The Barbalat’s lemma is an important mathematical tool for analyzing asymptotic stability of adaptive control systems in connection with the concept of uniform continuity of a real-valued function.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Stability is an important consideration of any dynamical systems with feedback control. Stability for LTI systems can be analyzed by many well-established methods such as eigenvalue analysis, root locus, phase and gain margins, etc. For nonlinear systems, Lyapunov stability theory provides a powerful technique for stability analysis of such systems. The Lyapunov stability theory is central to the study of nonlinear adaptive control [1,2,3,4]. In this chapter, the learning objectives are to develop a basic understanding of:

• Various stability concepts for autonomous and non-autonomous systems, such as local stability, asymptotic stability, exponential stability, uniform stability, and uniform boundedness;

• Lyapunov’s direct method and LaSalle’s invariant set theorem for analyzing stability of nonlinear systems; and

• Uniform continuity concept and Barbalat’s lemma for analyzing stability of non-autonomous systems.

4.1 Stability Concepts

Consider an autonomous system

$$\begin{aligned} \dot{x}=f\left( x\right) \end{aligned}$$

(4.1)

with an initial condition $x\left( t_{0}\right) =x_{0}$, where $f\left( x\right)$ is locally Lipschitz in some subset $\mathscr {D}$ of $\mathbb {R}^{n}$ and the solution $x\left( t;t_{0,}x_{0}\right)$ exists and is unique in a region $B_{R}=\left\{ x\left( t\right) \in \mathbb {R}^{n}:\left\| x\right\| <R\right\} \subset \mathscr {D}$ of an equilibrium point $x^{*}$. The region $B_{R}$ can be thought of as a hypersphere in $\mathbb {R}^{n}$ with the origin at $x=x^{*}$. Colloquially, it is often referred to in the literature as a ball $B_{R}$. Since $x^{*}$ is a constant vector, for convenience, the autonomous system can be transformed by shifting the equilibrium point to the origin at $x=0$. Let $y\left( t\right) =x\left( t\right) -x^{*}$, then

$$\begin{aligned} \dot{y}=f\left( y+x^{*}\right) \triangleq g\left( y\right) \end{aligned}$$

(4.2)

whose equilibrium is the origin $y^{*}=0$.

Thus, for convenience, the equilibrium point for autonomous systems described by Eq. (4.1) is understood to be $x^{*}=0$.

Example 4.1 The system

$$ \left[ \begin{array}{c} \dot{x}_{1}\\ \dot{x}_{2} \end{array}\right] =\left[ \begin{array}{c} -x_{1}+x_{1}x_{2}\\ x_{2}-x_{1}x_{2} \end{array}\right] $$

has an equilibrium at $x_{1}^{*}=1$ and $x_{2}^{*}=1$.

The system can be transformed by letting $y_{1}\left( t\right) =x_{1}\left( t\right) -1$ and $y_{2}\left( t\right) =x_{2}\left( t\right) -1$ which yields

$$ \left[ \begin{array}{c} \dot{y}_{1}\\ \dot{y}_{2} \end{array}\right] =\left[ \begin{array}{c} -\left( y_{1}+1\right) +\left( y_{1}+1\right) \left( y_{2}+1\right) \\ \left( y_{2}+1\right) -\left( y_{1}+1\right) \left( y_{2}+1\right) \end{array}\right] =\left[ \begin{array}{c} y_{1}y_{2}+y_{2}\\ -y_{1}y_{2}-y_{1} \end{array}\right] $$

4.1.1 Stability Definition

Definition 4.1

The equilibrium point $x^{*}=0$ of a system starting at an initial condition $x\left( t_{0}\right) =x_{0}$ is said to be stable (in the sense of Lyapunov) if, for any $R>0$, there exists some $r\left( R\right) >0$ such that

$$\begin{aligned} \left\| x_{0}\right\|<r\Rightarrow \left\| x\right\| <R,\,\forall t\ge t_{0} \end{aligned}$$

(4.3)

Otherwise, the equilibrium point is unstable.

$\blacksquare$

Stability concept essentially implies that, given a system with an initial condition close to the origin, the trajectory of the system can be kept arbitrarily close to it. Figure 4.1 illustrates the stability concept.

Fig. 4.1

Stability concept

Note that instability for linear systems means that the solution grows exponentially as $t\rightarrow \infty$ due to unstable poles in the right half plane, resulting in unbounded signals. For nonlinear systems, instability of an equilibrium does not always lead to unbounded signals. For example, the Van der Pol oscillator in Example 2.6 has a stable limit cycle that encloses an unstable equilibrium point at the origin. So, the equilibrium point in theory is unstable and the system cannot stay arbitrarily close to it. If we choose any arbitrary circle $B_{R}$ to be completely inside the limit cycle, then no matter how close the initial condition is to the origin, the trajectory of the system will eventually escape the circle $B_{R}$ as illustrated in Fig. 4.2. However, the trajectory tends to the limit cycle and remains there as $t\rightarrow \infty$.

Fig. 4.2

Unstable origin of Van der Pol Oscillator

4.1.2 Asymptotic Stability

The stability concept in the Lyapunov sense does not explicitly imply that the trajectory of a nonlinear system will eventually converge to the origin. For example, an ideal spring-mass system without friction will display a sinusoidal motion forever if it is subject to a disturbance. So, the system is stable in the Lyapunov sense but does not converge to the origin. Asymptotic stability is a stronger stability concept than the Lyapunov stability concept and is defined as follows:

Definition 4.2

The equilibrium point $x^{*}=0$ is said to be asymptotically stable if there exists some $r>0$ such that

$$\begin{aligned} \left\| x_{0}\right\| <r\Rightarrow \lim _{t\rightarrow \infty }\left\| x\right\| =0 \end{aligned}$$

(4.4)

$\blacksquare$

All trajectories starting within $B_{R}$ will eventually converge to the origin. The origin is then said to be attractive. For a second-order system, both stable focus and stable node are attractive. The largest such region is called a region of attraction, defined as

$$\begin{aligned} \mathscr {R}_{A}=\left\{ x\left( t\right) \in \mathscr {D}:\lim _{t\rightarrow \infty }x\left( t\right) =0\right\} \end{aligned}$$

(4.5)

It is noted that the asymptotic stability concept in the definition above is a local concept for any initial condition that lies within the ball $B_{R}$. If an equilibrium point of a system is asymptotically stable for all initial conditions $x_{0}\in \mathbb {R}^{n}$, then the equilibrium point is said to be asymptotically stable in the large. This notion is equivalent to global asymptotic stability.

Example 4.2 The equilibrium point of the system

$$ \dot{x}=-x^{2} $$

with $x\left( 0\right) =x_{0}>0$ is asymptotically stable since the solution

$$ x\left( t\right) =\frac{x_{0}}{x_{0}t+1} $$

tends to zero as $t\rightarrow \infty$. The region of attraction is

$$ \mathscr {R}_{A}=\left\{ x\left( t\right) \in \mathbb {R}^{+}:x\left( t\right) =\frac{x_{0}}{x_{0}t+1},x_{0}>0\right\} $$

Note that the equilibrium point is unstable if $x_{0}<0$ and has a finite escape time at $t=-1/x_{0}$. So, the equilibrium is asymptotically stable for all $x\left( t\right) \in \mathbb {R}^{+}$ but not asymptotically stable in the large.

4.1.3 Exponential Stability

The rate of convergence of a solution of a nonlinear differential equation can be estimated by comparing its solution to an exponential decay function [2, 4]. This gives rise to a notion of exponential stability which is defined as follows:

Definition 4.3

The equilibrium point $x^{*}=0$ is said to be exponentially stable if there exist two strictly positive constants $\alpha$ and $\beta$ such that

$$\begin{aligned} \left\| x\right\| \le \alpha \left\| x_{0}\right\| e^{-\beta \left( t-t_{0}\right) },\,\forall x\in B_{R},\,t\ge t_{0} \end{aligned}$$

(4.6)

$\blacksquare$

This definition gives a local version of the exponential stability concept for some initial condition $x_{0}$ close to the origin. If the origin is exponentially stable for all initial conditions $x_{0}\in \mathbb {R}^{n}$, then the equilibrium point is said to be exponentially stable in the large. The constant $\beta$ is called the rate of convergence.

It is noted that exponential stability implies asymptotic stability, but the converse is not true.

Example 4.3 The differential equation

$$ \dot{x}=-x\left( 1+\sin ^{2}x\right) $$

subject to $x\left( 0\right) =1$ is bounded from below and above by

$$ -2\left| x\right| \le \left| \dot{x}\right| \le -\left| x\right| $$

if $x\left( t\right) >0$.

The solution is bounded from below and above as shown in Fig. 4.3 by

$$ e^{-2t}\le \left| x\left( t\right) \right| \le e^{-t} $$

Therefore, the equilibrium is exponentially stable and the rate of convergence is 1.

Fig. 4.3

Exponential stability

4.2 Lyapunov’s Direct Method

4.2.1 Motivation

Consider a spring-mass-damper system with friction as shown in Fig. 4.4.

Fig. 4.4

Spring-mass system

The equation of motion without external forces is described by

$$\begin{aligned} m\ddot{x}+c\dot{x}+kx=0 \end{aligned}$$

(4.7)

where m is the mass, $c>0$ is the viscous friction coefficient, and k is the spring constant.

The system in the state-space form is expressed as

$$\begin{aligned} \left[ \begin{array}{c} \dot{x}_{1}\\ \dot{x}_{2} \end{array}\right] =\left[ \begin{array}{c} x_{2}\\ -\frac{c}{m}x_{2}-\frac{k}{m}x_{1} \end{array}\right] \end{aligned}$$

(4.8)

where $x_{1}\left( t\right) =x\left( t\right)$ and $x_{2}\left( t\right) =\dot{x}\left( t\right)$. The system has an equilibrium point at $\left( 0,0\right)$, that is, it is at rest with zero displacement and velocity

The spring-mass-damper system possesses two types of energy: (1) kinetic energy and (2) potential energy. The kinetic energy of any moving point mass is given by

$$\begin{aligned} T=\frac{1}{2}mv^{2}=\frac{1}{2}m\dot{x}^{2}=\frac{1}{2}mx_{2}^{2} \end{aligned}$$

(4.9)

The potential energy for a spring is given by

$$\begin{aligned} U=\frac{1}{2}kx^{2}=\frac{1}{2}kx_{1}^{2} \end{aligned}$$

(4.10)

The energy of the spring-mass-damper system is the sum of the kinetic energy and potential energy. Thus, the energy function of the system is defined as

$$\begin{aligned} E=T+U=\frac{1}{2}mx_{2}^{2}+\frac{1}{2}kx_{1}^{2} \end{aligned}$$

(4.11)

Note that the energy function is a quadratic positive-definite function.

The friction force also does work on the mass. This type of work is called a non-conservative work which is usually due to a dissipative force, as opposed to a conservative work of which potential energy is one form. The work function is defined in general as

$$\begin{aligned} W=\oint F.dx \end{aligned}$$

(4.12)

where F is a force acting on a mass that displaces it by an infinitesimal distance dx, and the integral is evaluated over a path that the mass traverses.

For a viscous friction force, the work done is evaluated as

$$\begin{aligned} W=\int c\dot{x}dx=\int c\dot{x}^{2}dt=\int cx_{2}^{2}dt \end{aligned}$$

(4.13)

The total energy of the system is the sum of the energy and the work done. Thus,

$$\begin{aligned} E+W=\frac{1}{2}mx_{2}^{2}+\frac{1}{2}kx_{1}^{2}+\int cx_{2}^{2}dt \end{aligned}$$

(4.14)

According to the first law of thermodynamics, the total energy of a closed system is neither created or destroyed. In other words, the total energy is conserved and is equal to a constant. Thus,

$$\begin{aligned} E+W=\text {const} \end{aligned}$$

(4.15)

or equivalently

$$\begin{aligned} \dot{E}+\dot{W}=0 \end{aligned}$$

(4.16)

This energy conservation law can be easily verified for the spring-mass-damper system as

$$\begin{aligned} \dot{E}+\dot{W}=mx_{2}\dot{x}_{2}+kx_{1}\dot{x}_{1}+\frac{d}{dt}\int cx_{2}^{2}dt=mx_{2}\left( -\frac{c}{m}x_{2}-\frac{k}{m}x_{1}\right) +kx_{1}x_{2}+cx_{2}^{2}=0 \end{aligned}$$

(4.17)

The time derivative of the energy function is evaluated as

$$\begin{aligned} \dot{E}=mx_{2}\dot{x}_{2}+kx_{1}\dot{x}_{1}=mx_{2}\left( -\frac{c}{m}x_{2}-\frac{k}{m}x_{1}\right) +kx_{1}x_{2}=-cx_{2}^{2}\le 0 \end{aligned}$$

(4.18)

for $c>0$.

The reason $\dot{E}$ is only negative semi-definite is because $\dot{E}$ can be zero for any $x_{1}\ne 0$.

Thus, for a dissipative system, the time derivative of the positive definite energy function is a negative semi-definite function. That is,

$$\begin{aligned} \dot{E}\le 0 \end{aligned}$$

(4.19)

The equilibrium point is then stable. Thus, stability of a dynamical system can be studied by examining the time derivative of the energy function. The Lyapunov stability theory is motivated by the concept of energy. In fact, the energy function is a Lyapunov function. Whereas the energy function is unique for a given physical system, a Lyapunov function can be any positive-definite function that satisfies the negative (semi-)definiteness of its time derivative.

Lyapunov’s direct method is a powerful tool for assessing stability of an equilibrium of a nonlinear system directly without solving the system’s dynamical equation. The motivation of the method is based on the energy concept of a mechanical system. From the spring-mass-damper example, the following observations are made:

• The energy function is positive definite.

• The time rate of the energy function is negative semi-definite in which case the equilibrium is stable.

Aleksandr Mikhailovich Lyapunov (1857–1918) recognized that stability of a system can be proven without developing a true knowledge of the system energy using a class of positive-definite functions, known as Lyapunov functions, provided they can be found.

Fig. 4.5

Illustration of Lyapunov function

Definition 4.4

A function $V\left( x\right)$ is said to be a Lyapunov function if it satisfies the following conditions:

• $V\left( x\right)$ is positive definite; i.e.,

  $$\begin{aligned} V\left( x\right) >0 \end{aligned}$$

  (4.20)

  and has a continuous first partial derivative.

• $\dot{V}\left( x\right)$ is at least negative semi-definite; i.e.,

  $$\begin{aligned} \dot{V}\left( x\right) =\frac{\partial V}{\partial x}\dot{x}=\frac{\partial V}{\partial x}f\left( x\right) \le 0 \end{aligned}$$

  (4.21)

  or

  $$\begin{aligned} \dot{V}\left( x\right) <0 \end{aligned}$$

  (4.22)

$\blacksquare$

Geometrically, a Lyapunov function may be illustrated by a bowl-shaped surface as shown in Fig. 4.5. The Lyapunov function is a level curve on the bowl starting at the top and progressing downward toward the bottom of the bowl. The value of the Lyapunov function thus decreases toward zero at the bottom of the bowl which represents a stable equilibrium.

Example 4.4 For the spring-mass-damper system, the energy function is clearly a Lyapunov function. Suppose one chooses another Lyapunov candidate function

$$ V\left( x\right) =x_{1}^{2}+x_{2}^{2}>0 $$

Then,

$$ \dot{V}\left( x\right) =2x_{1}\dot{x}_{1}+2x_{2}\dot{x}_{2}=2x_{1}x_{2}+2x_{2}\left( -\frac{c}{m}x_{2}-\frac{k}{m}x_{1}\right) =2x_{1}x_{2}\left( 1-\frac{k}{m}\right) -2\frac{c}{m}x_{2}^{2} $$

Note that $\dot{V}\left( x\right)$ is not negative (semi-)definite because of the term with the product $x_{1}x_{2}$. So, this candidate function is not a Lyapunov function.

In many systems, finding a Lyapunov function is not trivial. Unfortunately, there is no straightforward way to obtain a Lyapunov function. Perhaps, the most obvious Lyapunov function for any system is the energy function, but it is not always easy to identify such a function for a nonlinear system unless one knows the physics of the system.

Example 4.5

Consider a pendulum with viscous friction whose motion is described by

$$ ml^{2}\ddot{\theta }+c\dot{\theta }+mgl\sin \theta =0 $$

which is expressed in a state-space form as

$$ \left[ \begin{array}{c} \dot{x}_{1}\\ \dot{x}_{2} \end{array}\right] =\left[ \begin{array}{c} x_{2}\\ -\frac{c}{ml^{2}}x_{2}-\frac{g}{l}\sin x_{1} \end{array}\right] $$

with $x_{1}\left( t\right) =\theta \left( t\right)$ and $x_{2}\left( t\right) =\dot{\theta }\left( t\right)$.

It is not obvious what a Lyapunov candidate function would look like for this system. One can try

$$ V\left( x\right) =x_{1}^{2}+x_{2}^{2} $$

but that would not yield $\dot{V}\left( x\right) \le 0$ since

$$ \dot{V}\left( x\right) =2x_{1}\dot{x}_{1}+2x_{2}\dot{x}_{2}=2x_{1}x_{2}+2x_{2}\left( -\frac{c}{ml^{2}}x_{2}-\frac{g}{l}\sin x_{1}\right) \nleq 0 $$

The kinetic energy and potential energy for the pendulum are known to be

$$ T=\frac{1}{2}ml^{2}x_{2}^{2} $$

$$ U=mgl\left( 1-\cos x_{1}\right) $$

So, the energy function is

$$ E=T+U=\frac{1}{2}ml^{2}x_{2}^{2}+mgl\left( 1-\cos x_{1}\right) >0 $$

Using this as a Lyapunov function, then evaluating $\dot{V}\left( x\right)$ gives

$$ \dot{V}\left( x\right) =ml^{2}x_{2}\dot{x}_{2}+mgl\sin x_{1}\dot{x}_{1}=ml^{2}x_{2}\left( -\frac{c}{ml^{2}}x_{2}-\frac{g}{l}\sin x_{1}\right) +mgl\sin x_{1}x_{2}=-cx_{2}^{2}\le 0 $$

Thus, it is not surprising that the energy function can always be used for a Lyapunov function, provided such a function can be found.

$\blacksquare$

In summary, it can be seen that the Lyapunov’s direct method is a powerful technique for studying stability of an equilibrium point of a nonlinear system.

4.2.2 Lyapunov Theorem for Local Stability

Theorem 4.1

Let $x^{*}=0$ be an equilibrium point and if there exists a Lyapunov function $V\left( x\right) >0$ for all $x\left( t\right) \in B_{R}$ such that $\dot{V}\left( x\right) \le 0$ for all $x\left( t\right) \in B_{R}$, then the equilibrium is locally stable in a Lyapunov sense. Moreover, if $\dot{V}\left( x\right) <0$ for all $x\left( t\right) \in B_{R}$, then the equilibrium is locally asymptotically stable.

$\blacksquare$

It is important to note that the Lyapunov’s direct method only gives a sufficient condition for stability. Failure of a Lyapunov candidate function to satisfy the stability condition does not imply that the equilibrium is unstable. It simply means that a good Lyapunov candidate function may not have been identified. An exception to this rule is the energy function which provides both the necessary and sufficient conditions for stability.

Example 4.6 For the pendulum in Example 4.5, $\dot{V}\left( x\right)$ is negative semi-definite, so the equilibrium is locally stable.

Example 4.7 Consider the system

$$ \left[ \begin{array}{c} \dot{x}_{1}\\ \dot{x}_{2} \end{array}\right] =\left[ \begin{array}{c} x_{1}^{3}+x_{1}x_{2}^{2}-x_{1}\\ x_{2}^{3}+x_{1}^{2}x_{2}-x_{2} \end{array}\right] $$

Choose a Lyapunov candidate function

$$ V\left( x\right) =x_{1}^{2}+x_{2}^{2} $$

Then, $\dot{V}\left( x\right)$ is evaluated as

$$\begin{aligned} \dot{V}\left( x\right)&=2x_{1}\dot{x}_{1}+2x_{2}\dot{x}_{2}=2x_{1}\left( x_{1}^{3}+x_{1}x_{2}^{2}-x_{1}\right) +2x_{2}\left( x_{2}^{3}+x_{1}^{2}x_{2}-x_{2}\right) \\&=2\left( x_{1}^{2}+x_{2}^{2}\right) \left( x_{1}^{2}+x_{2}^{2}-1\right) \end{aligned}$$

Observing $\dot{V}\left( x\right)$, one can conclude that $\dot{V}\left( x\right) <0$ for all $x\in B_{R}$ where

$$ B_{R}=\left\{ x\left( t\right) \in \mathscr {D}\subset \mathbb {R}^{2}:x_{1}^{2}+x_{2}^{2}<1\right\} $$

The equilibrium is asymptotically stable. The region of attraction is $B_{R}$ within which all the trajectories converge to the equilibrium.

Example 4.8 For the spring-mass-damper system, consider a Lyapunov candidate function

$$ V\left( x\right) =x^{\top }Px>0 $$

where $x\left( t\right) =\left[ \begin{array}{cc} x_{1}\left( t\right)&x_{2}\left( t\right) \end{array}\right] ^{\top }$ and $P=P^{\top }>0$ to be determined such that $\dot{V}\left( x\right) <0$ for asymptotic stability.

Expressing $V\left( x\right)$ as

$$ V\left( x\right) =\left[ \begin{array}{cc} x_{1}&x_{2}\end{array}\right] \left[ \begin{array}{cc} p_{11} &{} p_{12}\\ p_{12} &{} p_{22} \end{array}\right] \left[ \begin{array}{c} x_{1}\\ x_{2} \end{array}\right] =p_{11}x_{1}^{2}+2p_{12}x_{1}x_{2}+p_{22}x_{2}^{2} $$

where $p_{ij}$ are elements of P, then evaluating $\dot{V}\left( x\right)$ yields

$$\begin{aligned} \dot{V}\left( x\right)&=2p_{11}x_{1}\dot{x}_{1}+2p_{12}\left( x_{1}\dot{x}_{2}+\dot{x}_{1}x_{2}\right) +2p_{22}x_{2}\dot{x}_{2}\\&=2p_{11}x_{1}x_{2}+2p_{12}x_{1}\left( -\frac{c}{m}x_{2}-\frac{k}{m}x_{1}\right) +2p_{12}x_{2}^{2}+2p_{22}x_{2}\left( -\frac{c}{m}x_{2}-\frac{k}{m}x_{1}\right) \\&=-2p_{12}\frac{k}{m}x_{1}^{2}+2\left( p_{11}-p_{12}\frac{c}{m}-p_{22}\frac{k}{m}\right) x_{1}x_{2}+2\left( p_{12}-p_{22}\frac{c}{m}\right) x_{2}^{2} \end{aligned}$$

Since $\dot{V}\left( x\right) <0$, one can choose

$$ \dot{V}\left( x\right) =-2x_{1}^{2}-2x_{2}^{2} $$

Equating terms then yields

$$ p_{12}\frac{k}{m}=1\Rightarrow p_{12}=\frac{m}{k} $$

$$ p_{12}-p_{22}\frac{c}{m}=-1\Rightarrow p_{22}=\frac{m}{c}\left( p_{12}+1\right) =\frac{m}{c}\left( \frac{m}{k}+1\right) $$

$$ p_{11}-p_{12}\frac{c}{m}-p_{22}\frac{k}{m}=0\Rightarrow p_{11}=p_{12}\frac{c}{m}+p_{22}\frac{k}{m}=\frac{c}{k}+\frac{m}{c}+\frac{k}{c} $$

The matrix

$$ P=\left[ \begin{array}{cc} \frac{c}{k}+\frac{m}{c}+\frac{k}{c} &{} \frac{m}{k}\\ \frac{m}{k} &{} \frac{m}{c}\left( \frac{m}{k}+1\right) \end{array}\right] $$

can be verified to be positive definite for $m>0$, $c>0$, and $k>0$.

The system is then asymptotically stable. It is noted that since the system is linear, stability is always referred to in a global context.

Another approach to be considered is as follows:

The system can be expressed as

$$ \dot{x}=Ax $$

where

$$ A=\left[ \begin{array}{cc} 0 &{} 1\\ -\frac{k}{m} &{} -\frac{c}{m} \end{array}\right] $$

is a Hurwitz matrix with negative real part eigenvalues.

Proceeding to evaluate $\dot{V}\left( x\right)$ yields

$$ \dot{V}\left( x\right) =\dot{x}^{\top }Px+x^{\top }P\dot{x} $$

which upon substitution yields

$$ \dot{V}\left( x\right) =x^{\top }A^{\top }Px+x^{\top }PAx=x^{\top }\left( A^{\top }P+PA\right) x<0 $$

This inequality is satisfied if and only if

$$ A^{\top }P+PA<0 $$

which is called a linear matrix inequality (LMI) that can be solved for P.

Alternatively, one can write the LMI as a linear matrix equation

$$ A^{\top }P+PA=-Q $$

where $Q=Q^{\top }>0$ is a positive-definite matrix. This equation is known as the algebraic Lyapunov equation.

Thus, setting $Q=2I$, where I is an identity matrix, which in this case is a 2-by-2 matrix, then yields

$$ \dot{V}\left( x\right) =-2x^{\top }x=-2x_{1}^{2}-2x_{2}^{2} $$

The matrix P is then solved from

$$ A^{\top }P+PA=-2I $$

There are numerical methods that can be used to solve the Lyapunov equation. For a low matrix dimension less than 4, the equation can be solved analytically. For this example, one can set up

$$ \left[ \begin{array}{cc} 0 &{} -\frac{k}{m}\\ 1 &{} -\frac{c}{m} \end{array}\right] \left[ \begin{array}{cc} p_{11} &{} p_{12}\\ p_{12} &{} p_{22} \end{array}\right] +\left[ \begin{array}{cc} p_{11} &{} p_{12}\\ p_{12} &{} p_{22} \end{array}\right] \left[ \begin{array}{cc} 0 &{} 1\\ -\frac{k}{m} &{} -\frac{c}{m} \end{array}\right] =\left[ \begin{array}{cc} -2 &{} 0\\ 0 &{} -2 \end{array}\right] $$

then expand

$$ \left[ \begin{array}{cc} -2p_{12}\frac{k}{m} &{} p_{11}-p_{12}\frac{c}{m}-p_{22}\frac{k}{m}\\ p_{11}-p_{12}\frac{c}{m}-p_{22}\frac{k}{m} &{} 2\left( p_{12}-p_{22}\frac{c}{m}\right) \end{array}\right] =\left[ \begin{array}{cc} -2 &{} 0\\ 0 &{} -2 \end{array}\right] $$

and equate terms to solve for $p_{ij}$.

4.2.3 Lyapunov Theorem for Exponential Stability

Theorem 4.2

Let $x^{*}=0$ be an equilibrium point and if there exists a Lyapunov function $V\left( x\right) >0$ for all $x\left( t\right) \in B_{R}$ such that $\dot{V}\left( x\right) <0$ for all $x\left( t\right) \in B_{R}$ and there also exist two positive constants $\eta$ and $\beta$ such that

$$\begin{aligned} V\left( x\right) \le \eta \left\| x\right\| ^{2} \end{aligned}$$

(4.23)

and

$$\begin{aligned} \dot{V}\left( x\right) \le -\beta V\left( x\right) \end{aligned}$$

(4.24)

then the equilibrium is locally exponentially stable.

Example 4.9 Consider

$$ \dot{x}=-x\left( 1+\sin ^{2}x\right) $$

subject to $x\left( 0\right) =1$.

Choose a Lyapunov candidate function

$$ V\left( x\right) =x^{2}=\left\| x\right\| ^{2} $$

So,

$$ V\left( 0\right) =x_{0}^{2}=1 $$

$\dot{V}\left( x\right)$ is computed as

$$ \dot{V}\left( x\right) =2x\dot{x}=-2x^{2}\left( 1+\sin ^{2}x\right) \le -2\left\| x\right\| ^{2}=-2V\left( x\right) <0 $$

Integrating $\dot{V}\left( x\right)$ yields

$$ V\left( t\right) \le V\left( 0\right) e^{-2t} $$

or

$$ x^{2}\le e^{-2t} $$

which is equivalent to

$$ \left| x\right| \le e^{-t} $$

4.2.4 Radially Unbounded Functions

Definition 4.5

A continuous, positive-valued function $\varphi \left( x\right) \in \mathbb {R}^{+}$ is said to belong to class $\mathscr {K}$; i.e., $\varphi \left( x\right) \in \mathscr {K}$, if

• $\varphi \left( 0\right) =0$

• $\varphi \left( x\right)$ is strictly increasing for all $x\left( t\right) \le R$ or $x\left( t\right) <\infty$

Definition 4.6

A continuous, positive-valued function $\varphi \left( x\right) \in \mathbb {R}^{+}$ is said to belong to class $\mathscr {K}\mathscr {R}$; i.e., $\varphi \left( x\right) \in \mathscr {K}\mathscr {R}$, if

• $\varphi \left( 0\right) =0$

• $\varphi \left( x\right)$ is strictly increasing for all $x\left( t\right) <\infty$

• $\lim _{x\rightarrow \infty }\varphi \left( x\right) =\infty$

Definition 4.7

A continuous, positive-valued function $V\left( x\right) \in \mathbb {R}^{+}$ with $V\left( 0\right) =0$ is said to be a radially unbounded function if there exists a function $\varphi \left( \left\| x\right\| \right) \in \mathscr {K}\mathscr {R}$ such that $V\left( x\right) \ge \varphi \left( \left\| x\right\| \right)$ for all $x\left( t\right) \in \mathbb {R}^{n}$. Thus, $V\left( x\right)$ must be infinitely large when $\left\| x\right\|$ tends to infinity. That is,

$$\begin{aligned} V\left( x\right) \rightarrow \infty \;\text {as}\;\left\| x\right\| \rightarrow \infty \end{aligned}$$

(4.25)

4.2.5 Barbashin–Krasovskii Theorem for Global Asymptotic Stability

The asymptotic stability concept in the Lyapunov sense of an equilibrium point is a local concept such that there exists $V\left( x\right) >0$ for all $x\in B_{R}$, where $B_{R}$ is a finite region in $\mathscr {D}\subset \mathbb {R}^{n}$ for which the function $f\left( x\right)$ is locally Lipschitz, then $\dot{V}\left( x\right) <0$. There exists a region of attraction $\mathscr {R}_{A}\subset \mathscr {D}\subset \mathbb {R}^{n}$ in which all trajectories will converge to the origin. On the other hand, asymptotic stability in the large is a global concept that requires the region of attraction to extend to the entire Euclidean space $\mathbb {R}^{n}$. As a result, $V\left( x\right) >0$ must be defined for all $x\in \mathbb {R}^{n}$.

There is an additional requirement imposed on $V\left( x\right)$ for stability in the large. That is, $V\left( x\right)$ is required to be a radially unbounded function. The condition of the radial unboundedness of $V\left( x\right)$ ensures that all trajectories in the large when $\left\| x\right\| \rightarrow \infty$ will be attracted to the origin. The global stability Lyapunov condition can be stated by the following Barbashin–Krasovskii theorem [2, 4]:

Theorem 4.3

The equilibrium point $x^{*}=0$ is asymptotically stable in the large if there exists a radially unbounded Lyapunov function $V\left( x\right) >0$ for all $x\left( t\right) \in \mathbb {R}^{n}$ such that $\dot{V}\left( x\right) <0$ for all $x\left( t\right) \in \mathbb {R}^{n}$.

Example 4.10 Consider a scalar linear system

$$ \dot{x}=-ax $$

where $a>0$, whose equilibrium point at the origin is asymptotically stable in the large. Suppose a Lyapunov candidate function is chosen as

$$ V_{1}\left( x\right) =\frac{x^{2}}{1+x^{2}}>0 $$

$V_{1}\left( x\right) \in \mathscr {K}$ but $V_{1}\left( x\right) \notin \mathscr {K}\mathscr {R}$ since $V_{1}\left( 0\right) =0$ and $V_{1}\left( x\right)$ is strictly increasing for all $x\left( t\right) <\infty$, but $\lim _{x\rightarrow \infty }V_{1}\left( x\right) =1$. This means that one cannot analyze global asymptotic stability of the origin of this system using this Lyapunov candidate function since

$$ \dot{V}_{1}\left( x\right) =\frac{2x\dot{x}}{1+x^{2}}-\frac{2x^{3}\dot{x}}{\left( 1+x^{2}\right) ^{2}}=\frac{-2ax^{2}}{\left( 1+x^{2}\right) ^{2}}\rightarrow 0 $$

as $\left\| x\right\| \rightarrow \infty$, which implies that the origin is not asymptotically stable as $\left\| x\right\| \rightarrow \infty$. Therefore, the origin is stable but not asymptotically stable in the large, which is a contradiction.

Now, suppose another Lyapunov candidate function is chosen as

$$ V_{2}\left( x\right) =x^{2}>0 $$

$V_{2}\left( x\right)$ is a radially unbounded function since there exists a function $\varphi \left( \left\| x\right\| \right) =\alpha x^{2}\in \mathscr {K}\mathscr {R}$, where $\alpha <1$, such that $V_{2}\left( x\right) \ge \varphi \left( \left\| x\right\| \right)$ for all $x\left( t\right) <\infty$. Global asymptotic stability of the origin can be analyzed using this radially unbounded Lyapunov candidate function. Evaluating $\dot{V}_{2}\left( x\right)$ yields

$$ \dot{V}_{2}\left( x\right) =2x\dot{x}=-2ax^{2}<0 $$

which implies that the origin is indeed asymptotically stable in the large.

4.2.6 LaSalle’s Invariant Set Theorem

Asymptotic stability requires that $\dot{V}\left( x\right) <0$. Yet, for the spring-mass-damper system, if the energy function is chosen as a Lyapunov function, then $\dot{V}\left( x\right) \le 0$, even though the solution of the system is clearly asymptotic in the presence of a dissipative force due to the viscous friction. The LaSalle’s invariant set theorem can resolve this apparent contradiction when an asymptotically stable equilibrium point of an autonomous system only satisfies the Lyapunov condition $\dot{V}\left( x\right) \le 0$.

Definition 4.8

For an autonomous system, a set $\mathscr {M}$ is said to be invariant if every trajectory that starts from a point in $\mathscr {M}$ will remain in $\mathscr {M}$ for all future time [2, 4]. That is,

$$\begin{aligned} x\left( 0\right) \in \mathscr {M}\Rightarrow x\left( t\right) \in \mathscr {M},\,\forall t\ge t_{0} \end{aligned}$$

(4.26)

Example 4.11

• An equilibrium point is an invariant set because by definition $x\left( t\right) =x^{*}$ is a constant solution of an autonomous system so that

  $$ x\left( t\right) =x\left( t_{0}\right) =x^{*}\in \mathscr {M},\,\forall t\ge t_{0} $$

• A region of attraction $\mathscr {R}_{A}$ is an invariant set since all trajectories in $\mathscr {R}_{A}$ will remain in $\mathscr {R}_{A}$ for all future time and converge to the origin as $t\rightarrow \infty$

  $$ \mathscr {R}_{A}=\left\{ x\left( t\right) \in \mathscr {M}:\lim _{t\rightarrow \infty }x\left( t\right) =0\right\} $$

• The limit cycle of the Van der Pol oscillator is an invariant set since any point on the limit cycle will remain on it for all future time.

• The entire Euclidean space $\mathbb {R}^{n}$ is a trivial invariant set since all trajectories must be in some subspace that belongs in $\mathbb {R}^{n}$.

$\blacksquare$

The LaSalle’s invariant set theorem is stated as follows:

Fig. 4.6

Bounded solution set

Theorem 4.4

Given an autonomous system, let $V\left( x\right) >0$ be a positive-definite function with a continuous first partial derivative such that $\dot{V}\left( x\right) \le 0$ in some finite region $B_{R}\subset \mathscr {D}$. Let $\mathscr {R}$ be a set of all points where $\dot{V}\left( x\right) =0$. Let $\mathscr {M}$ be the largest invariant set in $\mathscr {R}$. Then, every solution $x\left( t\right)$ starting in $B_{R}$ approaches $\mathscr {M}$ as $t\rightarrow \infty$.

Example 4.12 Consider

$$ \left[ \begin{array}{c} \dot{x}_{1}\\ \dot{x}_{2} \end{array}\right] =\left[ \begin{array}{c} -x_{1}^{3}-x_{1}x_{2}^{2}+x_{1}\\ -x_{2}^{3}-x_{1}^{2}x_{2}+x_{2} \end{array}\right] $$

Choose a Lyapunov candidate function

$$ V\left( x\right) =x_{1}^{2}+x_{2}^{2} $$

Then, $\dot{V}\left( x\right)$ is evaluated as

$$\begin{aligned} \dot{V}\left( x\right)&=2x_{1}\left( -x_{1}^{3}-x_{1}x_{2}^{2}+x_{1}\right) +2x_{2}\left( -x_{2}^{3}-x_{1}^{2}x_{2}+x_{2}\right) \\&=-2x_{1}^{2}\left( x_{1}^{2}+x_{2}^{2}-1\right) -2x_{2}^{2}\left( x_{1}^{2}+x_{2}^{2}-1\right) \\&=-2\left( x_{1}^{2}+x_{2}^{2}\right) \left( x_{1}^{2}+x_{2}^{2}-1\right) \end{aligned}$$

$\dot{V}\left( x\right) <0$ in a set $\mathscr {S}$ where

$$ \mathscr {S}=\left\{ x\left( t\right) \in B_{R}:x_{1}^{2}+x_{2}^{2}-1>0\right\} $$

but $\dot{V}\left( x\right) \ge 0$ in the complementary set

$$ \mathscr {S}^{c}=\left\{ x\left( t\right) \in B_{R}:x_{1}^{2}+x_{2}^{2}-1\le 0\right\} $$

which represents a circular region that includes the origin. Therefore, the origin is unstable.

Let $\mathscr {R}$ be a set of all points where $\dot{V}\left( x\right) =0$. Then,

$$ \mathscr {R}=\left\{ x\left( t\right) \in B_{R}:g\left( x\right) =x_{1}^{2}+x_{2}^{2}-1=V\left( x\right) -1=0\right\} $$

in fact represents a bounded solution $x\left( t\right)$ since all trajectories either inside or outside of $\mathscr {S}^{c}$ will move toward $\mathscr {R}$ and remain in $\mathscr {R}$ as illustrated in Fig. 4.6. Thus, $\mathscr {R}$ is an invariant set. One can also verify this by taking the time derivative of the function $g\left( x\right)$ that represents the trajectories in the set $\mathscr {R}$

$$ \dot{g}\left( x\right) =\frac{d}{dt}\left( V-1\right) =\dot{V}\left( x\right) =0,\,\forall x\left( t\right) \in \mathscr {R} $$

Fig. 4.7

Trajectories of Lyapunov function tending to an invariant set

The Lyapunov function can be solved analytically by noting that

$$ \dot{V}=-2V\left( V-1\right) $$

which leads to

$$ \frac{dV}{V\left( V-1\right) }=-2dt $$

Using the partial fraction, this can be expressed as

$$ \left( \frac{1}{V-1}-\frac{1}{V}\right) dV=-2dt $$

which yields the following general solution:

$$ V\left( t\right) =\frac{V_{0}}{V_{0}-\left( V_{0}-1\right) e^{-2t}} $$

As $t\rightarrow \infty$, $V\left( t\right)$ tends to a constant solution as shown in Fig. 4.7 with

$$ \lim _{t\rightarrow \infty }V\left( t\right) =1 $$

which is in fact the set $\mathscr {R}$. Therefore, the set $\mathscr {R}$ is a bounding set of all possible solutions $x\left( t\right)$. Thus, $x\left( t\right) \in \mathscr {L}_{\infty }$; i.e., $x\left( t\right)$ is bounded.

Example 4.13 Consider the spring-mass-damper system. Choose the Lyapunov function to be the energy function

$$ V\left( x\right) =\frac{1}{2}mx_{2}^{2}+\frac{1}{2}kx_{1}^{2} $$

Then,

$$ \dot{V}\left( x\right) =mx_{2}\dot{x}_{2}+kx_{1}\dot{x}_{1}=-cx_{2}^{2}\le 0 $$

Since $\dot{V}\left( x\right)$ is negative semi-definite, then the origin is only stable in the Lyapunov sense, but not asymptotically stable as one would expect for a spring-mass-damper system with friction. Let $\mathscr {R}$ be a set of all points where $\dot{V}\left( x\right) =0$. Then,

$$ \mathscr {R}=\left\{ x\left( t\right) \in \mathbb {R}^{2}:\dot{V}\left( x\right) =0\Rightarrow x_{2}=0\right\} $$

is a collection of all points that lie on the $x_{1}$-axis. It follows that any point on this axis must satisfy

$$ m\dot{x}_{2}+kx_{1}=0 $$

or

$$ \dot{x}_{2}=\ddot{x}_{1}=-\frac{k}{m}x_{1} $$

If $x_{1}\left( t\right) \ne 0$, then $\ddot{x}_{1}\left( t\right) \ne 0$ with $\text {sgn}\left( \ddot{x}_{1}\right) =-\text {sgn}\left( x_{1}\right)$ in $\mathscr {R}$, where $\text {sgn}\left( \right)$ is the sign function which returns 1 if the argument is positive, 0 if the argument is zero, or -1 if the argument is negative. This means that a point on this axis cannot remain in $\mathscr {R}$ because the acceleration $\ddot{x}_{1}$ causes the point to move toward the origin, unless it is already at the origin.

Another way to find an invariant set is to evaluate the derivative of the function that describes $\mathscr {R}$ and set it equal to zero. Hence,

$$ \dot{x}_{2}=0 $$

which is satisfied if and only if $x_{1}\left( t\right) =0$.

Thus, the invariant set $\mathscr {M}\subset \mathscr {R}$ is a set that contains only the origin. Then, according to the LaSalle’s invariant set theorem, all trajectories will converge to the origin as $t\rightarrow \infty$. The origin then is asymptotically stable.

$\blacksquare$

This example brings up an interesting observation that can be stated in the following corollary of the LaSalle’s invariant set theorem:

Corollary 4.1

Let $V\left( x\right) >0$ be a positive definite function with a continuous first partial derivative such that $\dot{V}\left( x\right) \le 0$ in some finite region $B_{R}\subset \mathscr {D}$. Let $\mathscr {R}=\left\{ x\left( t\right) \in B_{R}:\dot{V}\left( x\right) =0\right\}$ and suppose that no solution can stay in $\mathscr {R}$ except the trivial solution $x=0$. Then, the origin is asymptotically stable. Moreover, if $V\left( x\right) >0$ is a positive definite radially unbounded function and $\mathscr {R}=\left\{ x\left( t\right) \in \mathbb {R}^{n}:\dot{V}\left( x\right) =0\right\}$, then the origin is asymptotically stable in the large.

4.2.7 Differential Lyapunov Equation

The Lyapunov equation has a connection to the optimal control theory. In particular, the Lyapunov equation can be viewed as a special case of the Riccati equation for a Linear Quadratic Regulator (LQR) optimal control problem. Consider the following LTI system:

$$\begin{aligned} \dot{x}=Ax \end{aligned}$$

(4.27)

subject to $x\left( t_{0}\right) =x_{0}$, where $x\left( t\right) \in \mathbb {R}^{n}$ and $A\in \mathbb {R}^{n}\times \mathbb {R}^{n}$.

It is of interest to find a condition that minimizes the following quadratic cost function:

$$\begin{aligned} \min \,J=\int _{t_{0}}^{t_{f}}x^{\top }Qxdt \end{aligned}$$

(4.28)

where $Q>0\in \mathbb {R}^{n}\times \mathbb {R}^{n}$ is a positive definite matrix.

The solution can be established by the Pontryagin’s maximum principle in the optimal control theory [5, 6]. The Hamiltonian function of this system is defined as

$$\begin{aligned} H=x^{\top }Qx+\lambda ^{\top }\left( Ax+Bu\right) \end{aligned}$$

(4.29)

where $\lambda \left( t\right) \in \mathbb {R}^{n}$ is called an adjoint or co-state vector.

The adjoint equation is given by

$$\begin{aligned} \dot{\lambda }=-\frac{\partial H^{\top }}{\partial x}=-Qx-A^{\top }\lambda \end{aligned}$$

(4.30)

subject to the transversality (terminal time) condition

$$\begin{aligned} \lambda \left( t_{f}\right) =0 \end{aligned}$$

(4.31)

Choose a solution of $\lambda \left( t\right)$ in the form of

$$\begin{aligned} \lambda \left( t\right) =P\left( t\right) x \end{aligned}$$

(4.32)

where $P\left( t\right) \in \mathbb {R}^{n}\times \mathbb {R}^{n}$ is a time-varying matrix.

Then, the adjoint equation is evaluated with the system dynamics as

$$\begin{aligned} \dot{P}x+PAx=-Qx-A^{\top }Px \end{aligned}$$

(4.33)

By factoring out $x\left( t\right)$, the differential Lyapunov equation is obtained as

$$\begin{aligned} \dot{P}+PA+A^{\top }P+Q=0 \end{aligned}$$

(4.34)

subject to $P\left( t_{f}\right) =0$.

Contrasting this with the differential Riccati equation

$$\begin{aligned} \dot{P}+PA+A^{\top }P-PBR^{-1}B^{\top }P+Q=0 \end{aligned}$$

(4.35)

the differential Lyapunov equation is a special case of the differential Riccati equation for $R\rightarrow \infty$.

Note that the differential Lyapunov equation is defined backward in time with the transversality condition given at the final time. By transforming into a time-to-go variable, $\tau =t_{f}-t$, then

$$\begin{aligned} \frac{dP}{d\tau }=PA+A^{\top }P+Q \end{aligned}$$

(4.36)

subject to $P\left( 0\right) =0$ in the time-to-go coordinate.

If A is Hurwitz and let $t_{f}\rightarrow \infty$ which corresponds to an infinite time horizon solution, then the time-varying solution of the differential Lyapunov equation tends to a constant solution of the algebraic Lyapunov equation

$$\begin{aligned} PA+A^{\top }P+Q=0 \end{aligned}$$

(4.37)

The constant solution of P is given by

$$\begin{aligned} P=\lim _{\tau \rightarrow \infty }\int _{0}^{\tau }e^{A^{\top }\tau }Qe^{A\tau }d\tau \end{aligned}$$

(4.38)

which is positive definite for $Q>0$ and requires that A be Hurwitz since the solution must be a stable solution such that

$$\begin{aligned} \lim _{\tau \rightarrow \infty }e^{A\tau }=0 \end{aligned}$$

(4.39)

Example 4.14 Numerically compute P, given

$$ A=\left[ \begin{array}{cc} 0 &{} 1\\ -4 &{} -4 \end{array}\right] ,\;Q=\left[ \begin{array}{cc} 1 &{} 0\\ 0 &{} 1 \end{array}\right] $$

The differential Lyapunov equation in time-to-go can be solved using any numerical technique for solving differential equations such as the Euler or Runge–Kutta method. For example, the Euler method for solving the Lyapunov equation is as follows:

$$ P_{i+1}=P_{i}+\Delta \tau \left( P_{i}A+A^{\top }P_{i}+Q\right) $$

where i denotes the value of $P\left( \tau \right)$ at a time $\tau _{i}=i\Delta \tau$.

The equation can be integrated until the solution converges to within a specified error. Figure 4.8 illustrates the solution convergence of the differential Lyapunov equation. The result is obtained as

$$ P=\left[ \begin{array}{cc} 1.125 &{} 0.125\\ 0.125 &{} 0.15625 \end{array}\right] $$

Fig. 4.8

Numerical solution of Lyapunov equation

4.3 Stability of Non-Autonomous Systems

Most of the concepts for Lyapunov stability for autonomous systems can be applied to non-autonomous systems with some additional considerations [2, 4].

Consider a non-autonomous system

$$\begin{aligned} \dot{x}=f\left( x,t\right) \end{aligned}$$

(4.40)

subject to $x\left( t_{0}\right) =x_{0}$, where $f\left( x,t\right)$ is locally Lipschitz in $\mathscr {D}\times \left[ 0,\infty \right)$ and $\mathscr {D}\subset \mathbb {R}^{n}$.

The notion of the origin as an equilibrium point now takes on a different meaning in that the equilibrium point $x^{*}$ must be time-invariant for all $t\ge t_{0}$ and satisfies

$$\begin{aligned} f\left( x^{*},t\right) =0 \end{aligned}$$

(4.41)

Otherwise, the system does not have a “true” equilibrium point in the Lyapunov sense.

Example 4.15

• The system

  $$ \dot{x}=g\left( t\right) h\left( x\right) $$

  has an equilibrium point $x^{*}$ where

  $$ h\left( x^{*}\right) =0 $$

• The system

  $$ \dot{x}=g\left( t\right) h\left( x\right) +d\left( t\right) $$

  does not have a true equilibrium point since $h\left( x\right)$ would have been a function of t to satisfy $\dot{x}\left( t\right) =0$, which is a contradiction unless $g\left( t\right) =\alpha d\left( t\right)$ for some constant $\alpha$.

4.3.1 Uniform Stability

The Lyapunov stability definition for a non-autonomous system is defined as follows:

Definition 4.9

The equilibrium point $x^{*}=0$ is said to be stable (in the sense of Lyapunov) if, for any $R>0$, there exists some $r\left( R,t_{0}\right) >0$ such that

$$\begin{aligned} \left\| x_{0}\right\|<r\Rightarrow \left\| x\right\| <R,\,\forall t\ge t_{0} \end{aligned}$$

(4.42)

$\blacksquare$

Note that the difference in this definition as compared to that for an autonomous system is the ball of radius r that encloses $x_{0}$ now may depend on the initial time $t_{0}$. Thus, the stability of the origin may also be dependent on the initial time.

The concept of uniform stability is an additional consideration for non-autonomous systems. Uniform stability implies that the radius $r=r\left( R\right)$ is not dependent on the initial time, and so are the stability properties of the equilibrium point. For autonomous systems, stability is independent of the initial time. This property is highly desirable since it eliminates the need for examining the effect of the initial time on the stability of a non-autonomous system.

Definition 4.10

Given a Lyapunov function $V\left( x,t\right)$ for a non-autonomous system that satisfies the following conditions:

$$\begin{aligned} V\left( 0,t\right) =0 \end{aligned}$$

(4.43)

and

$$\begin{aligned} 0 < V\left( x,t\right) \le W\left( x\right) \end{aligned}$$

(4.44)

where $W\left( x\right) >0$ is a positive-definite function, then $V\left( x,t\right)$ is said to be a positive definite, decrescent function.

Example 4.16 The Lyapunov candidate function

$$ V\left( x,t\right) =\left( 1+\sin ^{2}t\right) \left( x_{1}^{2}+x_{2}^{2}\right) $$

is bounded from above by

$$ 0<V\left( x,t\right) \le 2\left( x_{1}^{2}+x_{2}^{2}\right) =W\left( x\right) $$

Since $W\left( x\right) >0$, then $V\left( x,t\right)$ is a positive definite, decrescent function.

$\blacksquare$

The Lyapunov’s direct method for a non-autonomous system is stated in the following theorem:

Theorem 4.5

If there exists a positive definite, decrescent Lyapunov function $V\left( x,t\right)$ for all $x\left( t\right) \in B_{R}$ and $t\ge 0$ such that

$$\begin{aligned} \dot{V}\left( x,t\right) =\frac{\partial V}{\partial x}f\left( x,t\right) +\frac{\partial V}{\partial t}\le 0 \end{aligned}$$

(4.45)

then the origin is said to be uniformly stable in the Lyapunov sense. Moreover, if $\dot{V}\left( x,t\right) <0$, then the origin is said to be uniformly asymptotically stable, and additionally, if the region $B_{R}$ is extended to the entire Euclidean space $\mathbb {R}^{n}$, then the origin is said to be uniformly asymptotically stable in the large.

4.3.2 Uniform Boundedness

When a non-autonomous system does not have an equilibrium point, stability of such a system is defined by the notion of boundedness [2, 4].

Definition 4.11

The solution of a non-autonomous system is said to be uniformly bounded if, for any $R>0$, there exists some $r\left( R\right) >0$ independent of the initial time $t_{0}$ such that

$$\begin{aligned} \left\| x_{0}\right\| <r\Rightarrow \left\| x\right\| \le R,\,\forall t\ge t_{0} \end{aligned}$$

(4.46)

Moreover, the solution is said to be uniformly ultimately bounded if, for any $R>0$, there exists some $r>0$ independent of R and the initial time $t_{0}$ such that

$$\begin{aligned} \left\| x_{0}\right\| <r\Rightarrow \left\| x\right\| \le R,\,\forall t\ge t_{0}+T \end{aligned}$$

(4.47)

where $T=T\left( r\right)$ is some time interval after the initial time $t_{0}$.

$\blacksquare$

The uniform ultimate boundedness concept simply means that the solution may not be uniformly bounded initially according to Definition 4.11 but eventually becomes uniformly ultimately bounded after some time has passed. The constant R is called a bound if the solution is uniformly bounded or an ultimate bound if the solution is uniformly ultimately bounded.

The Lyapunov’s direct method can be applied to a non-autonomous system according to the following theorem:

Theorem 4.6

Given a Lyapunov function $V\left( x,t\right)$ for all $\left\| x\right\| \ge R$ and $t\in \left[ 0,\infty \right)$, then the solution of the non-autonomous system (4.40) is said to be uniformly bounded if there exist functions $\varphi _{1}\left( \left\| x\right\| \right) \in \mathscr {K}\mathscr {R}$ and $\varphi _{2}\left( \left\| x\right\| \right) \in \mathscr {K}\mathscr {R}$ such that [2]

• $\varphi _{1}\left( \left\| x\right\| \right) \le V\left( x,t\right) \le \varphi _{2}\left( \left\| x\right\| \right)$

• $\dot{V}\left( x,t\right) \le 0$

for all $\left\| x\right\| \ge R$ and $t\in \left[ 0,\infty \right)$. In addition, if there exists a function $\varphi _{3}\left( \left\| x\right\| \right) \in \mathscr {K}\mathscr {R}$ such that

• $\dot{V}\left( x,t\right) \le -\varphi _{3}\left( \left\| x\right\| \right)$

for all $\left\| x\right\| \ge R$ and $t\in \left[ 0,\infty \right)$, then the solution is said to be uniformly ultimately bounded.

Example 4.17 Consider

$$ \dot{x}=-x+2\sin t $$

subject to $x\left( 0\right) =x_{0}$.

The system does not have an equilibrium. The solution is

$$ x=\left( x_{0}+1\right) e^{-t}+\sin t-\cos t $$

If $\left\| x_{0}\right\| <r$ and recognizing that $e^{-t}\le 1$ and $\left| \sin t-\cos t\right| \le \sqrt{2}$, then

$$ \left\| x\right\| \le \left\| x_{0}+1\right\| +\sqrt{2}<r+1+\sqrt{2}=R $$

Thus, one can choose $r\left( R\right) =R-1-\sqrt{2}$ according to Definition 4.11. So, the solution is uniformly bounded. Suppose $x_{0}=1$, then the bound is $R=2+\sqrt{2}$.

Moreover, as $t\rightarrow \infty$, then the solution tends to

$$ x\rightarrow \sin t-\cos t $$

so that

$$ \left\| x\right\| \le \sqrt{2}=R $$

independent of r. The solution then is also uniformly ultimately bounded with an ultimate bound of $\sqrt{2}$ as shown in Fig. 4.9.

Fig. 4.9

Uniform ultimate boundedness

The Lyapunov’s direct method is now applied to determine the uniform boundedness of a solution. Consider a Lyapunov candidate function for this system

$$ V\left( x\right) =x^{2}>0 $$

Note that one can always find $\varphi _{1}\left( \left\| x\right\| \right) \in \mathscr {K}\mathscr {R}$ and $\varphi _{2}\left( \left\| x\right\| \right) \in \mathscr {K}\mathscr {R}$ for this Lyapunov candidate function such that $\varphi _{1}\left( \left\| x\right\| \right) \le V\left( x\right) \le \varphi _{2}\left( \left\| x\right\| \right)$, for example, $\varphi _{1}\left( \left\| x\right\| \right) =ax^{2}$ with $a<1$ and $\varphi _{2}\left( \left\| x\right\| \right) =bx^{2}$ with $b>1$.

Then,

$$ \dot{V}\left( x\right) =2x\dot{x}=2x\left( -x+2\sin t\right) \le -2x^{2}+4\left\| x \right\| $$

We see that

$$ \dot{V}\left( x\right) \le -2V\left( x\right) +4\sqrt{V\left( x\right) } $$

Let $W (t)=\sqrt{V (t)}=\left\| x\right\|$. Then,

$$ \dot{W}=\frac{\dot{V}}{2\sqrt{V}}=-\sqrt{V}+2\le -W+2 $$

The solution of $W\left( t\right)$ is

$$ W\le \left( \left\| x_{0}\right\| -2\right) e^{-t}+2 $$

Thus,

$$ \lim _{t\rightarrow \infty }\left\| x\right\| =\lim _{t\rightarrow \infty }W\le 2=R $$

Choose

$$ \varphi _{3}\left( \left\| x\right\| \right) =2\left\| x\right\| ^{2}-4\left\| x\right\| $$

Note that $\varphi _{3}\left( \left\| x\right\| \right) \in \mathscr {K}\mathscr {R}$ (verify!). Then, it follows that

$$ \dot{V}\left( x\right) \le -\varphi _{3}\left( \left\| x\right\| \right) $$

Then, $\dot{V}\left( x\right) \le 0$ if $-2x^{2}+4\left\| x\right\| \le 0$ or $\left\| x\right\| \ge 2$. Therefore, according to Theorem 4.6, the solution $x\left( t\right)$ is uniformly ultimately bounded with a Lyapunov ultimate bound of 2. It is noted that the Lyapunov ultimate bound is always more conservative or greater than or equal to the ultimate bound derived from the actual solution as illustrated in Fig. 4.9.

There is another way of showing that $x\left( t\right)$ is uniformly ultimately bounded. By completing the square, $\dot{V}\left( x\right)$ can also be expressed as

$$ \dot{V}\left( x\right) \le -2\left( \left\| x\right\| -1\right) ^{2}+2 $$

Then, $\dot{V}\left( x\right) \le 0$ if $-2\left( \left\| x\right\| -1\right) ^{2}+2\le 0$ or $\left\| x\right\| \ge 2$. Since $\dot{V}\left( x\right) \le 0$ outside the compact set $\left\| x\right\| \le 2$, but $\dot{V} (x)>0$ inside it, therefore the solution $x\left( t\right)$ is uniformly ultimately bounded. Any trajectory that starts outside the compact set will reach the ultimate bound $\left\| x\right\| =2$ because $\dot{V}\left( x\right) \le 0$ outside the compact set. Any trajectory that starts inside the compact set will move away from the origin because $\dot{V}(x)>0$ inside the compact set, but will eventually be attracted to the ultimate bound $\left\| x\right\| =2$.

4.3.3 Barbalat’s Lemma

The LaSalle’s invariant set theorem can be used to show asymptotic stability of an equilibrium point for an autonomous system when $\dot{V}\left( x\right)$ is only negative semi-definite. This theorem cannot be used for non-autonomous systems. Therefore, it can be much more difficult to show asymptotic stability for a non-autonomous system than for an autonomous system. Barbalat’s lemma is a mathematical tool that can be used to address this situation to some extent [4].

Firstly, the concept of uniform continuity needs to be introduced. A mathematical formal definition of uniform continuity of a function is given as follows:

Definition 4.12

The function $f\left( t\right) \in \mathbb {R}$ is uniformly continuous on a set $\mathscr {D}$ if, for any $\epsilon >0$, there exists some $\delta \left( \epsilon \right) >0$ such that

$$\begin{aligned} \left| t_{2}-t_{1}\right|<\delta \Rightarrow \left| f\left( t_{2}\right) -f\left( t_{1}\right) \right| <\epsilon ,\,\forall t_{1},t_{2} \end{aligned}$$

(4.48)

$\blacksquare$

The following statements are equivalent to the definition of uniform continuity:

• Suppose a function $f\left( t\right)$ is continuous on a closed finite interval $t\in \left[ t_{1},t_{2}\right]$. Then, $f\left( t\right)$ is uniformly continuous on $t\in \left[ t_{1},t_{2}\right]$.

• Suppose a function $f\left( t\right)$ is differentiable on a set $\mathscr {D}$, and there exists a constant $M>0$ such that $\left| \dot{f}\left( t\right) \right| <M$ for all t. Then, $f\left( t\right)$ is uniformly continuous on $\mathscr {D}$.

In the simplest term, uniform continuity of a differentiable function $f\left( t\right)$ requires its derivative $\dot{f}\left( t\right)$ to exist and be bounded.

Example 4.18

• The function $f\left( t\right) =t^{2}$ for all $t\in \left[ 0,\infty \right)$ is continuous but is not uniformly continuous since $\dot{f}\left( t\right)$ is not bounded for all $t\in \left[ 0,\infty \right)$.

• The function $f\left( t\right) =t^{2}$ for $t\in \left[ 0,1\right]$ is uniformly continuous since $f\left( t\right)$ is continuous for $t\in \left[ 0,1\right]$.

• The function $f\left( t\right) =\sqrt{t}$ for all $t\in \left[ 0,\infty \right)$ does not have a bounded derivative $\dot{f}\left( t\right) =\frac{1}{2\sqrt{t}}$ for all $t\in \left[ 0,\infty \right)$, and since the interval is semi-open and infinite, one cannot readily conclude that $f\left( t\right)$ is not uniformly continuous on $t\in \left[ 0,\infty \right)$. However, this function is actually uniformly continuous on $t\in \left[ 0,\infty \right)$. To see this, we note that the interval can be divided into two subintervals $t\in \left[ 0,a\right]$ and $t\in \left[ a,\infty \right)$ where $a>0$. Then, $f\left( t\right)$ is uniformly continuous on $t\in \left[ 0,a\right]$ since $f\left( t\right)$ is continuous on $t\in \left[ 0,a\right]$, and furthermore $f\left( t\right)$ is also uniformly continuous on $t\in \left[ a,\infty \right)$ since $\dot{f}\left( t\right) =\frac{1}{2\sqrt{t}}$ is bounded on $t\in \left[ a,\infty \right)$. Therefore, in totality, $f\left( t\right)$ is uniformly continuous on $t\in \left[ 0,\infty \right)$.

• Consider a stable LTI system

  $$ \dot{x}=Ax+Bu $$

  with $x (t_0) = x_0$ and a continuous bounded input $u\left( t\right)$. The system is exponentially stable with the solution

  $$ x=e^{-A\left( t-t_{0}\right) }x_{0}+\int _{t_{0}}^{t}e^{A\left( t-\tau \right) }Bu\left( \tau \right) d\tau $$

  Thus, $x\left( t\right)$ is a continuous bounded signal with a bounded derivative $\dot{x}\left( t\right)$ for all $t\in \left[ 0,\infty \right)$. Therefore, $x\left( t\right)$ is uniformly continuous. Any output signal

  $$ y=Cx+Bu $$

  is also uniformly continuous if $u\left( t\right)$ has a bounded derivative. The system is then said to be bounded-input-bounded-output (BIBO) stable.

$\blacksquare$

The Barbalat’s lemma is now stated as follows:

Lemma 4.1

If the limit of a differentiable function $f\left( t\right)$ as $t\rightarrow \infty$ exists and is finite, and if $\dot{f}\left( t\right)$ is uniformly continuous, then $\lim _{t\rightarrow \infty }\dot{f}\left( t\right) =0$.

Example 4.19

• The function $f\left( t\right) =e^{-t^{2}}$ has a finite limit as $t\rightarrow \infty$. To determine the uniform continuity of the first derivative $\dot{f}\left( t\right) =-2te^{-t^{2}}$, we need to determine whether or not the second derivative $\ddot{f}\left( t\right)$ is bounded for all $t\in \left[ 0,\infty \right)$. The second derivative $\ddot{f}\left( t\right) =-2e^{-t^{2}}+4t^{2}e^{-t^{2}}$, in fact, is bounded because the exponential term $e^{-t^{2}}$ decreases at a much faster rate than the power term $t^{2}$. Therefore, $\lim _{t\rightarrow \infty }\dot{f}\left( t\right) =0$. In fact, one can verify $\lim _{t\rightarrow \infty }-2te^{-t^{2}}=0$ using the L’Hospital rule.

• The function $f\left( t\right) =\frac{1}{t}\sin \left( t^{2}\right)$ which tends to zero as $t\rightarrow \infty$ but whose derivative $\dot{f}\left( t\right) =-\frac{1}{t^{2}}\sin \left( t^{2}\right) +2\cos \left( t^{2}\right)$ does not have a limit as $t\rightarrow \infty$. Thus, it can be seen that even if the limit of a differentiable function $f\left( t\right)$ exists and is finite as $t\rightarrow \infty$, it does not necessarily imply that $\lim _{t\rightarrow \infty }\dot{f}\left( t\right) =0$ since $\dot{f}\left( t\right)$ may not be uniformly continuous, that is, $\dot{f}\left( t\right)$ has a bounded derivative or equivalently $\ddot{f}\left( t\right)$ is bounded. Therefore, the function $f\left( t\right) =\frac{1}{t}\sin \left( t^{2}\right)$ does not satisfy the Barbalat’s lemma.

• The function $f\left( t\right) =\sin \left( \ln t\right)$ whose derivative $\dot{f}\left( t\right) =\frac{1}{t}\cos \left( \ln t\right)$ tends to zero but $f\left( t\right)$ does not have a finite limit as $t\rightarrow \infty$. Thus, $\lim _{t\rightarrow \infty }\dot{f}\left( t\right) =0$ does not necessarily imply that the limit of a differentiable function $f\left( t\right)$ exists and is finite. Therefore, the converse of the Barbalat’s lemma is not true.

$\blacksquare$

The Barbalat’s lemma is now extended to the Lyapunov’s direct method to examine the asymptotic stability of a non-autonomous system by the following Lyapunov-like lemma [4]:

Lemma 4.2

If a positive-definite function $V\left( x,t\right)$ has a finite limit as $t\rightarrow \infty$, and if $\dot{V}\left( x,t\right)$ is negative semi-definite and uniformly continuous for all $t\in \left[ 0,\infty \right)$, then $\dot{V}\left( x,t\right) \rightarrow 0$ as $t\rightarrow \infty$.

Example 4.20 Consider a simple adaptive control system

$$ \dot{x}=-ax+b\left[ u+\theta ^{*}w\left( t\right) \right] $$

where $a>0$, $w\left( t\right) \in \mathscr {L}_{\infty }$ is a bounded time-varying disturbance, and $\theta ^{*}$ is an unknown constant parameter.

To cancel out the effect of the time-varying disturbance, an adaptive controller is designed as

$$ u=-\theta \left( t\right) w\left( t\right) $$

where $\theta \left( t\right)$ is an adaptive parameter that estimates $\theta ^{*}$.

If $\theta \left( t\right) \rightarrow \theta ^{*}$ as $t\rightarrow \infty$, then the adaptive controller perfectly cancels out the disturbance and the closed-loop system tends to an ideal reference model

$$ \dot{x}_{m}=-ax_{m} $$

where $x_{m}\left( t\right)$ is the desired response of $x\left( t\right)$.

The adaptive parameter is computed as

$$ \dot{\theta }=-bew\left( t\right) $$

where $e\left( t\right) =x_{m}\left( t\right) -x\left( t\right)$ is called a tracking error, described by the tracking error equation

$$ \dot{e}=\dot{x}_{m}-\dot{x}=-ae+b\tilde{\theta }w\left( t\right) $$

where $\tilde{\theta }\left( t\right) =\theta \left( t\right) -\theta ^{*}$ is the parameter estimation error.

The combined system is non-autonomous due to $w\left( t\right)$. Both the variables $e\left( t\right)$ and $\theta \left( t\right)$ are influenced by the tracking error equation $\dot{e}\left( t\right)$ and the adaptive law $\dot{\theta }\left( t\right)$. To show that the system is stable, choose the following Lyapunov candidate function that includes both the variables $e\left( t\right)$ and $\tilde{\theta }\left( t\right)$:

$$ V\left( e,\theta \right) =e^{2}+\tilde{\theta }^{2} $$

Then,

$$ \dot{V}\left( e,\tilde{\theta }\right) =2e\left[ -ae+b\tilde{\theta }w\left( t\right) \right] +2\tilde{\theta }\left[ -bew\left( t\right) \right] =-2ae^{2}\le 0 $$

Since $\dot{V}\left( e,\theta \right)$ is negative semi-definite, $e\left( t\right) \in \mathscr {L}_{\infty }$ and $\theta \left( t\right) \in \mathscr {L}_{\infty }$, i.e., they are bounded, but the LaSalle’s invariant set theorem cannot be used to show that the tracking error $e\left( t\right)$ converges to zero. This is where the Barbalat’s lemma comes in handy. Firstly, $V\left( e,\tilde{\theta }\right)$ must be shown to have a finite limit as $t\rightarrow \infty$. Since $\dot{V}\left( e,\tilde{\theta }\right) \le 0$, then

$$\begin{aligned} V\left( e\left( t\rightarrow \infty \right) ,\tilde{\theta }\left( t\rightarrow \infty \right) \right) -V\left( e\left( t_{0}\right) ,\tilde{\theta }\left( t_{0}\right) \right)&=\int _{t_{0}}^{\infty }\dot{V}\left( e,\tilde{\theta }\right) dt\\&=-2a\int _{t_{0}}^{\infty }e^{2}\left( t\right) dt=-2a\left\| e\right\| _{2}^{2} \end{aligned}$$

$$\begin{aligned} V\left( e\left( t\rightarrow \infty \right) ,\tilde{\theta }\left( t\rightarrow \infty \right) \right)&=V\left( e\left( t_{0}\right) ,\tilde{\theta }\left( t_{0}\right) \right) -2a\left\| e\right\| _{2}^{2}\\&=e^{2}\left( t_{0}\right) +\tilde{\theta }^{2}\left( t_{0}\right) -2\left\| e\right\| _{2}^{2}<\infty \end{aligned}$$

So, $V\left( e,\tilde{\theta }\right)$ has a finite limit as $t\rightarrow \infty$. Since $\left\| e\right\| _{2}$ exists, therefore $e\left( t\right) \in \mathscr {L}_{2}\cap \mathscr {L}_{\infty }$.

Next, $\dot{V}\left( e,\tilde{\theta }\right)$ must be shown to be uniformly continuous. This can be done by examining the derivative of $\dot{V}\left( e,\tilde{\theta }\right)$ to see if it is bounded. $\ddot{V}\left( e,\tilde{\theta }\right)$ is computed as

$$ \ddot{V}\left( e,\tilde{\theta }\right) =-4ae\left[ -ae+b\tilde{\theta }w\left( t\right) \right] $$

Since $e\left( t\right) \in \mathscr {L}_{2}\cap \mathscr {L}_{\infty }$ and $\tilde{\theta }\left( t\right) \in \mathscr {L}_{\infty }$ by the virtue of $\dot{V}\left( e,\tilde{\theta }\right) \le 0$, and $w\left( t\right) \in \mathscr {L}_{\infty }$ by assumption, then $\ddot{V}\left( e,\tilde{\theta }\right) \in \mathscr {L}_{\infty }$. Therefore, $\dot{V}\left( e,\tilde{\theta }\right)$ is uniformly continuous. It follows from the Barbalat’s lemma that $\dot{V}\left( e,\tilde{\theta }\right) \rightarrow 0$ and hence $e\left( t\right) \rightarrow 0$ as $t\rightarrow \infty$. Note that one cannot conclude that the system is asymptotically stable since only $e\left( t\right) \rightarrow 0$ as $t\rightarrow \infty$, but $\tilde{\theta }\left( t\right)$ is only bounded.

4.4 Summary

The Lyapunov stability theory is the foundation of nonlinear systems and adaptive control theory. Various stability concepts for autonomous and non-autonomous systems are introduced. The Lyapunov’s direct method is an indispensable tool for analyzing stability of nonlinear systems. Barbashin–Krasovskii theorem provides a method for global stability analysis. LaSalle’s invariant set theorem provides another complementary tool for analyzing systems with invariant sets. Stability of non-autonomous systems involves the concepts of uniform stability, uniform boundedness, and uniform ultimate boundedness. Barbalat’s lemma is an important mathematical tool for analyzing stability of adaptive control systems in connection with the concept of uniform continuity of a real-valued function.

4.5 Exercises

• 1. Given

  $$ \left[ \begin{array}{c} \dot{x}_{1}\\ \dot{x}_{2} \end{array}\right] =\left[ \begin{array}{c} x_{1}\left( x_{1}^{2}+x_{2}^{2}-1\right) -x_{2}\\ x_{1}+x_{2}\left( x_{1}^{2}+x_{2}^{2}-1\right) \end{array}\right] $$

  • a. Determine all the equilibrium points of the system and linearize the system about the equilibrium points to classify the types of equilibrium points.

  • b. Use the Lyapunov candidate function

    $$ V\left( x\right) =x_{1}^{2}+x_{2}^{2} $$

    to determine the types of Lyapunov stability of the equilibrium points and their corresponding regions of attraction, if any.

• 2. Given

  $$ \dot{x}=x\left( -1+\frac{1}{2}\sin x\right) $$

  subject to $x\left( 0\right) =1.$

  • a. Determine the upper and lower bound solutions.

  • b. Use the Lyapunov candidate function

    $$ V\left( x\right) =x^{2} $$

    to determine the type of Lyapunov stability and the upper bound of $V\left( x\right)$ as an explicit function of time.

• 3. Use the Lyapunov candidate function

  $$ V\left( x\right) =x_{1}^{2}+x_{2}^{2} $$

  to study stability of the origin of the system

  $$ \left[ \begin{array}{c} \dot{x}_{1}\\ \dot{x}_{2} \end{array}\right] =\left[ \begin{array}{c} \left( x_{2}-x_{1}\right) \left( x_{1}^{2}+x_{2}^{2}\right) \\ \left( x_{1}+x_{2}\right) \left( x_{1}^{2}+x_{2}^{2}\right) \end{array}\right] $$

• 4. Given

  $$ \dot{x}=Ax $$

  • a. Calculate analytically P that solves

    $$ A^{\top }P+PA=-2I $$

    where

    $$ A=\left[ \begin{array}{cc} 0 &{} 1\\ -4 &{} 4 \end{array}\right] $$

    and verify the result using the MATLAB function “lyap.”

  • b. Determine if P is positive or negative (semi-)definite. What can be said about stability of the origin of this system.

• 5. Given

  $$ \left[ \begin{array}{c} \dot{x}_{1}\\ \dot{x}_{2} \end{array}\right] =\left[ \begin{array}{c} x_{1}\left( 1-x_{1}^{2}-x_{2}^{2}\right) +x_{2}\\ -x_{1}+x_{2}\left( 1-x_{1}^{2}-x_{2}^{2}\right) \end{array}\right] $$

  • a. Use the Lyapunov candidate function

    $$ V\left( x\right) =x_{1}^{2}+x_{2}^{2} $$

    to determine the type of Lyapunov stability of the origin.

  • b. Find an invariant set.

  • c. Solve for $V\left( t\right)$ as an explicit function of time and plot the trajectories of $V\left( t\right)$ for $V\left( 0\right) =0.01,\,0.5,\,1,\,1.5,\,2$.

• 6. Given

  $$ A=\left[ \begin{array}{ccc} 0 &{} 1 &{} 0\\ -1 &{} -1 &{} -2\\ 1 &{} 0 &{} -1 \end{array}\right] $$

  Determine whether or not A is Hurwitz. If so, compute P using the Euler method to integrate the differential Lyapunov equation

  $$ \frac{dP}{d\tau }=PA+A^{\top }P+I $$

  subject to $P\left( 0\right) =0$, where $\tau$ is time-to-go. Plot all six elements of P on the same plot and verify the result at the final time-to-go with the MATLAB function “lyap.”

• 7. Use the Lyapunov’s direct method to determine an ultimate bound of the solution $x\left( t\right)$ for the following equation:

  $$ \dot{x}=-x+\cos t\sin t $$

  subject to $x\left( 0\right) =1$. Plot the solution $x\left( t\right)$ for $0\le t\le 20$.

• 8. Given a non-autonomous system

  $$ \dot{x}=\left( -2+\sin t\right) x-\cos t $$

  • a. Show that the system is uniformly ultimately bounded by the Lyapunov theorem for non-autonomous systems. Also determine the ultimate bound of $\left\| x\right\|$.

  • b. Plot the solution by numerically integrating the differential equation and show that it satisfies the ultimate bound.

• 9. Given

  $$ \dot{x}=-\left( 1+\sin ^{2}t\right) x+\cos t $$

  • a. Use the Lyapunov candidate function

    $$ V\left( x\right) =x^{2} $$

    to determine the upper bound of $\dot{V}\left( x\right)$ as a function of $V\left( x\right)$.

  • b. Let $W=\sqrt{V}$. Solve for the inequality $W\left( t\right)$ as an explicit function of time and determine the ultimate bound of the system.

  • c. Show that the system is uniformly ultimately bounded.

• 10. For the following functions:

  • a. $f\left( t\right) =\sin \left( e^{-t^{2}}\right)$

  • b. $f\left( t\right) =e^{-\sin ^{2}t}$

  Plot $f\left( t\right)$ for $t\in \left[ 0,5\right]$. Determine whether or not the limit of $f\left( t\right)$ exists as $t\rightarrow \infty$ and $\dot{f}\left( t\right)$ is uniformly continuous. If so, use the Barbalat’s lemma to show that $\dot{f}\left( t\right) \rightarrow 0$ as $t\rightarrow \infty$ and verify by taking the limit of $\dot{f}\left( t\right)$ as $t\rightarrow \infty$.

• 11. Consider the following adaptive control system:

  $$ \dot{e}=-e+\theta x $$
  $$ \dot{\theta }=-xe $$

  where $e\left( t\right) =x_{m}\left( t\right) -x\left( t\right)$ is defined as the tracking error between a given explicit reference time signal $x_{m}\left( t\right)$ which is assumed to be bounded, i.e., $x_{m}\left( t\right) \in \mathscr {L}_{\infty }$, and the state variable $x\left( t\right)$. Show that the adaptive system is stable and that $e\left( t\right) \rightarrow 0$ as $t\rightarrow \infty$.