Combined Inverse and Gradient Based Design

Abstract

The inverse and gradient approaches to Iterative Control are brought together in new compensator structures that offer the prospect of improved performance and robustness. The ideas of bi-directional filtering and stable inverse algorithms are introduced and the negative effects of non-minimum-phase zeros are identified and parameterized.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preceding chapters have shown the theoretical power of inverse model-based and gradient-based Iterative Control algorithms for linear systems using an operator formulation that highlights the great generality of the concepts. The general theory inevitably needs more detail to be added depending on the form of plant model to be used. Linear, time invariant state space models (and their transfer function matrix equivalents) formed the focus of the discussion with emphasis on the case of discrete/sampled data systems as the finite dimensional nature of the resultant problem simplified the presentation considerably. The purpose of this chapter is to explore the links between inverse model and gradient approaches and bring them together in the form of a structured design approach. It will not be claimed that the presentation answers all the questions posed by the wide range of applications but it is hoped that the issues described and answers considered will help the reader in the process of tailoring the ideas to meet his or her needs.

The benefits of bringing inverse model and gradient concepts together is first described by a section on filtering to improve the robustness of inverse model algorithms. This is followed by a summary of issues that may be faced in design and a range of compensation strategies.

8.1 Inverse Algorithms: Robustness and Bi-directional Filtering

The purpose of this section is use gradient concepts to improve the robustness of the inverse model algorithms discussed in Chap. 6 using bi-directional (“zero phase change”) filtering techniques. The approach is capable of achieving some compensation of the effects of modelling error gains without the introduction of the phase lag associated with “uni-directional” filters. Controlling phase lag is essential as it could lead to the possible consequence that the required positivity conditions for the modelling error are violated.

The operator notation is used throughout and specific detailed results for linear, time invariant, discrete time state space models stated in some detail.

The Case of $m \le \ell$ Suppose that $G_R$ is a right inverse of a model G of a plant UG with left multiplicative modelling error U. Then the inverse model Algorithm 6.1 has update formula $u_{k+1}=u_k + \beta G_Re_k$ and is monotonically convergent if U is asymptotically stable and the relevant operator or frequency domain conditions are satisfied (Theorem 6.6). Compensation for any difficult gain characteristics of U(z) such as high frequency parasitic resonances clearly must take into account the need to avoid introducing additional phase lag into the system whilst simultaneously providing appropriate gain changes over chosen frequency bands. In what follows this is achieved using an algorithm based on the use of a filter $F: \mathscr {F} \rightarrow \mathscr {Y}$ where, as used previously, $\mathscr {Y}= \mathscr {R}^{m(N+1)}$ is regarded as a Hilbert space created from time series in $\mathscr {R}^{m}$ of length $N+1$ in supervector form. The chosen inner product and norm are

$$\begin{aligned} \langle y,w \rangle _{\mathscr {Y}} = \sum _{t=0}^N~y^T(t)Qw(t) \quad and \quad \Vert y\Vert _{\mathscr {Y}} = \sqrt{\langle y,y \rangle _{\mathscr {Y}}} \quad with \quad Q=Q^T>0. \end{aligned}$$

(8.1)

The space $\mathscr {F}= \mathscr {R}^{m_F(N+1)}$ is regarded as a Hilbert space created from time series in $\mathscr {R}^{m_F}$ of length $N+1$ in supervector form. The chosen inner product and norm are written as follows

$$\begin{aligned} \langle f,g \rangle _{\mathscr {F}} = \sum _{t=0}^N~f^T(t)R_Fg(t) \quad and \quad \Vert f\Vert _{\mathscr {F}} = \sqrt{\langle f,f \rangle _{\mathscr {F}}}\quad where \quad R_F=R_F^T>0. \end{aligned}$$

(8.2)

As usual, the filter is assumed to have a state space model $S(A_F,B_F,C_F,D_F)$. The following algorithm is bi-directional (or bi-causal) as it includes the operation $FF^*e_k$ which is realized by two simulations. The first filtering computation, $\psi _k=F^*e_k$, is approached using the “adjoint model”

$$\begin{aligned} S(A^T_F,C^T_FQ,R_F^{-1}B^T_F,R_F^{-1}D^T_FQ) \end{aligned}$$

and time reversal operator techniques and takes the form

$$\begin{aligned} \psi _k = \mathscr {T}(m_F,N)G(A^T_F,C^T_FQ,R_F^{-1}B^T_F,R_F^{-1}D^T_FQ)\mathscr {T}(m,N)e_k, \end{aligned}$$

(8.3)

An equivalent statement is as follows,

$$\begin{aligned} \begin{array}{rcl} z_k(t+1) &{} = &{} A_F^Tz_k(t) + C_F^T Q e_k(N-t), \quad z_k(0)=0, \quad \quad ~and \\ \psi _k(t) &{} = &{} R_F^{-1}B_F^T z(N-t) + R_F^{-1}D_F^TQ e_k(t),~ 0 \le t \le N. \end{array} \end{aligned}$$

(8.4)

The second simulation, $F(F^*e_k) = F \psi _k$ uses the state space model $S(A_F,B_F,C_F, D_F)$ to compute its response, from zero initial conditions, to the time series $\psi _k(t),~0 \le t \le N$.

Algorithm 8.1

(Right Inverse Algorithm with Bi-directional Filtering) Suppose that a model G of the plant UG has a right inverse $G_R$ and U is invertible. Suppose also that the design process has specified an $m \times m_F$ filter F(z) that is asymptotically stable and minimum-phase with the property that $ker[F^*]=\{0\}$ (that is, $rank[D_F]=m$ and hence $m \le m_F$ and $\mathscr {R}[F] = \mathscr {Y}$). The Right Inverse Model Algorithm with Bi-directional Filtering is defined by the input update formula

$$\begin{aligned} u_{k+1}=u_k + \beta G_RFF^*e_k, \quad k \ge 0. \end{aligned}$$

(8.5)

The resultant error evolution is described by the equation

$$\begin{aligned} e_{k+1}=(I-\beta UFF^*)e_k, \quad k \ge 0. \end{aligned}$$

(8.6)

The error evolution equation has the formal structure of a perturbed steepest descent algorithm with G replaced by F and hence

• 1. A sufficient condition for convergence to be achieved is obtained from Theorem 7.16. In terms of the proof of that Theorem, the error was written as $e=UG(u_{\infty }-u)$ and monotonic convergence defined in terms of the subsidiary time series $w_k = u_{\infty } - u_k,~~ k \ge 0$, which has an obvious interpretation. For Algorithm 8.1, the variable $w_k$ defined by $e_k=UFw_k$ does not have this interpretation but it can still be used in the proof as it is uniquely defined if values are taken to be in the closed (Hilbert) subspace $\mathscr {R}[F^*]$. With this understanding, a sufficient condition for monotonic convergence is that U(z) is minimum-phase and also that,

  $$\begin{aligned} \hat{U}^T(z) Q + Q \hat{U}(z^{-1}) > \beta QF(z^{-1})R_F^{-1}F^T(z)Q \quad whenever ~~|z|=1~. \end{aligned}$$

  (8.7)

  The left-hand-side of this expression depends primarily on the modelling error U(z) whilst the right-hand-side depends on the chosen filter F(z) and the gain $\beta$. This observation is a key link between the properties of U and the choice of filter. Conversely, for a given filter, the expression provides data on the class of modelling errors U that can be tolerated whilst retaining monotonic convergence.

• 2. In the “nominal” case of no modelling error, $U=I$ and the condition reduces to $2Q > \beta QF(z^{-1})R_F^{-1}F^T(z)Q$ whenever $|z|=1$, which, in operator terms, is just $0 < \beta \Vert F^*\Vert ^2 < 2$ if N is large.

• 3. Both of the previous two observations have meaning if $\varepsilon$-weighted models and norms are used as described in Sect. 7.5.5, simply by replacing U(z) by $U_{\varepsilon }(z)=U(\varepsilon ^{-1}z)$ and F(z) by $F_{\varepsilon }(z)=F(\varepsilon ^{-1}z)$ from which monotonic convergence with respect to the weighted norm is ensured if

  $$\begin{aligned} \hat{U}^T(z) Q + Q \hat{U}(\overline{z}) > \beta QF(\overline{z})R_F^{-1}F^T(z)Q \quad whenever ~~|z|=\varepsilon ^{-1}. \end{aligned}$$

  (8.8)

The choice of F(z) is an open question that is strongly related to the design issues pertinent to the problem/application being considered. The need for a form of positive real modelling errors is still essential but, using the techniques of Sect. 7.2.3, an analysis of the first iteration $e_1=(I-\beta FF^*)e_0$ where $e_0$ is the supervector corresponding to the time series $\{e_0(t,z,\alpha )\}_{0 \le t \le N} = \{z^t\alpha \}_{0 \le t \le N}$ gives the approximate time series

$$\begin{aligned} e_1(t) \approx (I - \beta U(z)F(z)R_F^{-1}F^T(z^{-1})Q)e_0(t,z,\alpha ), \quad 0 \le t \le N,\,{ if}\,N\,is\,{ large.} \end{aligned}$$

(8.9)

This indicates that (a) the use of low pass filters with a specified bandwidth will help to suppress the effects of significant high frequency properties of U(z) or (b) the use of a notch filter could reduce the effects of problematic frequency ranges. The choice should reflect the need to reduce problems in algorithm performance and also allow increased values of gain $\beta$ to improve convergence rates.

Special Case of $m=\ell =1$: Insight into possibilities is clearer in the SISO case. Take, for example,$Q=1$ and $R_F=I_{m_F}$, and set F(z) to be the row vector

$$\begin{aligned} F(z) = \left[ F_1(z), \ldots ,~F_{m_F}(z) \right] \end{aligned}$$

(8.10)

where $F_j(z),~ 1 \le j \le m_F$, are scalar, asymptotically stable filters. The condition $ker[F^*]=\{0\}$ simply requires that at least one such filter has a non-zero “D” term in its state space model. With this construction, the convergence condition becomes

$$\begin{aligned} \left| \frac{1}{\beta } - U(z)\sum _{j=1}^{m_F}|F_j(z)|^2 \right| < \frac{1}{\beta } \quad whenever \quad |z|=1~. \end{aligned}$$

(8.11)

This clearly reveals the role of F in shaping the gain characteristics of U and their consequences for convergence and gain selection. The flexibility implicit in the role of $m_F$ is also revealed. Note the separate effects of each filter in the summation. This structure opens up the possibility of filter design using simple component elements $F_j$ designed for different purposes over different bandwidths. For example, let $m_F=2$ and consider the $1 \times 2$ filter structure

$$\begin{aligned} F(z) = \left[ F_1(z), F_2(z) \right] = \left[ (1-\lambda )^{0.5}, \lambda ^{0.5} \left( \frac{1-p}{z-p}\right) ~\right] , \quad with ~~ \lambda \in (0,1), \end{aligned}$$

(8.12)

gives

$$\begin{aligned} \sum _{j=1}^{m_F}|F_j(z)|^2 = (1-\lambda ) + \lambda \frac{(1-p)^2}{|z-p|^2}~ \end{aligned}$$

(8.13)

and provides separate influence over high frequency and low frequencies (by choice of $\lambda$) and bandwidth compensation (by choice of p).

The Case of $m \ge \ell$: The algorithm in this case takes the form,

Algorithm 8.2

(Left Inverse Model Algorithm with Bi-directional Filtering) Suppose that $m \ge \ell$ and that the model G of the plant GU has a left inverse $G_L$ (and hence that $ker[G]=\{0\}$ and $\mathscr {Y} = ker[G_L]\oplus \mathscr {R}[G]$). Suppose also that U is invertible and that the design process has specified an $\ell \times m_F$ filter F(z) that is asymptotically stable with the property that $ker[F^*]=\{0\}$ (that is $rank[D_F]=\ell$). The Left Inverse Model Algorithm with Bi-directional Filtering is defined by the update formula

$$\begin{aligned} u_{k+1}=u_k + \beta FF^*G_Le_k, \quad k \ge 0. \end{aligned}$$

(8.14)

The resultant error evolution is described by $e_{k+1}=(I-\beta GUFF^*G_L)e_k, ~~ k \ge 0~$.

Each error $e_k$ can be written in the unique form $e_k = Gw_k + \xi _k$ with $\xi _k \in ker[G_L]$. A simple calculation gives the result that $\xi _k = \xi _0$ for all $k \ge 0$, and hence that the component of $e_0$ in $ker[G_L]$ remains unchanged from iteration to iteration. Significant error evolution only occurs, therefore, in $\mathscr {R}[G]$ and is described by the formula

$$\begin{aligned} w_{k+1}=(I-\beta UFF^*)w_k, \quad k \ge 0~. \end{aligned}$$

(8.15)

Again, the equation has the structure of a perturbed steepest descent algorithm with G replaced by F and hence a sufficient condition for convergence of $\{w_k\}_{k \ge 0}$ to a limit $w_{\infty }=0$ is that given for the case of $m \le \ell$. The corresponding error evolution satisfies the condition

$$\begin{aligned} \lim _{k \rightarrow \infty } e_k = \xi _0~. \end{aligned}$$

(8.16)

Conclusions and Comments: The material described above provides a partial unification of the inverse model and gradient paradigms for Iterative Control. The outcome is aimed at providing some control over the robustness of the inverse model approach using bi-directional filtering operations that mimic the structure of gradient methodologies. In the form described, the need for positivity conditions to be satisfied is retained but, where this causes difficulties, the use of $\varepsilon$-weighted signals can help considerably. With these constraints in mind however, the approach is suitable for implementation as a conditioning tool for inverse model algorithms although the choice of filter F will need careful consideration to meet the needs and concerns of the particular application being considered.

8.2 General Issues in Design

Although much of the general theory and underlying concepts and ideas presented in the text applies to a wide class of dynamical systems, the detailed analysis of the case of linear, time invariant, discrete time, state space systems provides some tools for analysis and design and indicates that the properties of the plant model G and the form of any multiplicative modelling error U play an important role in the design process and behaviours. A review of the material of previous chapters suggests some general design guidelines. For example, specific issues that can be identified include:

• 1. The application of fixed (iteration independent) control loops to “pre-condition” the plant may be essential as a means of improving the potential for successful Iterative Control. The resultant controller is implemented on the plant. The original plant model is then replaced by a model of the pre-conditioned plant (and again denoted by G).

• 2. A preliminary analysis of the resultant conditioned model is useful and perhaps essential.

• 3. The design of of compensation schemes for the pre-conditioned plant to achieve the desired Iterative Control performance and robustness are the final off-line design stage.

8.2.1 Pre-conditioning Control Loops

Given an $\ell$-input, m-output, state space model S(A, B, C, D) (and associated transfer function matrix G(z)) of the plant to be controlled, the stability and robustness conditions for gradient-based Iterative Control depend explicitly on the the transfer function matrix G(z) of the plant model and model error U(z). In particular there is a need to ensure stability of the plant (essential for feedforward Iterative Control algorithms) and, where possible to eliminate or, at least reduce, the effects of non-minimum-phase zeros, resonances (small damping) and zeros close to the stability boundary. With this in mind, it is reasonable to assert that control loops should be added to the plant before Iterative algorithms are considered.

There are a large number of candidates for such pre-conditioning control loops and the pre-conditioning can be applied as “analogue control” to the underlying continuous system before sampling or to the sampled system itself. They, of course, include standard output feedback and feedforward controllers designed for stability, robustness and applications specific reasons. From a scientific point of view however, it is of interest to assess the potential full power of feedback by investigating what the use of full state feedback can offer. The full analysis of such an idea is beyond the scope of this text but an insight into possibilities can be obtained from the following examples.

Deadbeat Control Suppose that $m = \ell$, that $D=0$ and that the model has uniform rank $k^* \ge 1$. That is, $CA^{j-1}B = 0$ for $1 \le j < k^*$ with $\det [CA^{k^*-1}B] \ne 0$. Then the state feedback control law

$$\begin{aligned} u(t)=v(t) - (CA^{k^*-1}B)^{-1}CA^{k^*}x(t), \quad t \ge 0 \end{aligned}$$

(8.17)

replaces the input u by the new input v and generates the closed loop plant model $S(A-B(CA^{k^*-1}B)^{-1}CA^{k^*}, B,C)$. A simple calculation then yields the following simple equation for closed loop output dynamics,

$$\begin{aligned} y(t+k^*)=CA^{k^*-1}Bv(t), \quad t \ge 0. \end{aligned}$$

(8.18)

This model has a minimal realization of state dimension $k^*m$ defined by the matrices

$$\begin{aligned} A= \left[ \begin{array}{cccccc} 0 &{} I_m &{} 0 &{} \cdots &{} \cdots &{} 0 \\ 0 &{} 0 &{} I_m &{} 0 &{} \cdots &{} 0 \\ \vdots &{} &{} &{} &{} &{} \vdots \\ \vdots &{} &{} &{} &{} &{} I_m \\ 0 &{} 0 &{} 0 &{} \cdots &{} 0 &{} 0 \end{array} \right] ~, \quad B= \left[ \begin{array}{c} 0 \\ 0 \\ \vdots \\ 0 \\ CA^{k^*-1}B \end{array} \right] ~, \quad C= \left[ I_m ~~ 0 ~~ \cdots 0 \right] ~. \end{aligned}$$

(8.19)

The state vector is $x(t) = \left[ y^T(t+k^*), y^T(t+k^*-1) , \ldots , y^T(t) \right] ^T$. The remaining $n - k^*m$ unobservable states are associated with poles equal to the zeros of the plant and hence are stable if, and only if, the original plant was minimum-phase.

State feedback can simplify the nature of plant dynamics although the range of its successful application may be influenced by the position of plant zeros. An interesting observation here is that the resultant closed loop system has transfer function matrix equal to $z^{-k^*}CA^{k^*-1}B$ and hence, assuming zero initial conditions, that the shifted output $y^e(t)$ defined by $y^e(t) = y(t+k^*)$ for $t \ge 0$ is described by the simple algebraic expression $y^e(t)=CA^{k^*-1}Bv(t)$. It is a simple matter now to show that the update rule $v_{k+1}(t) = v_k(t) + \beta (CA^{k^*-1}B)^{-1}e^e_k(t)$ where $e^e_k(t)= r^e(t) -y^e_k(t)$ and $r^e_k(t)=r(t+k^*)$ gives the recursion $e_{k+1}^e(t) = (1 - \beta ) e^e_k(t), t \ge 0$. This converges if $\beta \in (0,2)$, a direct parallel to the expressions appearing in the inverse model algorithm. Clearly the use of state feedback can create inverse model behaviour exactly. More generally, it may create useful approximations to inverse algorithms.

Pole Allocation Loops: State feedback can also be used to allocate the poles of the preconditioned plant using the more general state feedback rule

$$\begin{aligned} u(t) = v(t) - Fx(t), \quad t \ge 0, \end{aligned}$$

(8.20)

where F denotes an $\ell \times n$ state feedback matrix. If S(A, B, C) is controllable, the eigenvalues/poles of the closed loop system $S(A-BF, B,C)$ can be allocated to any desired locations. The deadbeat control example is just one choice but leads to an unobservable closed loop system. Consider the special case of a single-input, single-output system ($m=\ell =1$) of relative degree $k^*=1$ and with $n-1$, distinct, real zeros $0 < z_j < 1$ ordered as follows $z_1 > z_2 >\cdots > z_{n-1}$. Choose F to allocate the closed loop poles to real values $0 < p_j < 1, \ \le j \le n$ satisfying the interlacing condition

$$\begin{aligned} p_n < z_{n-1} < p_{n-1} < \cdots < z_1 < p_1. \end{aligned}$$

(8.21)

The interesting properties of such “interlaced systems” include the facts that

• 1. The residues of the poles of G(z) in its partial fraction expansion all take the same sign. Physically, if they are positive for example, the step response from zero initial conditions is monotonically increasing in the sense that $y(t_2) \ge y(t_1)$ whenever $t_2 > t_1$.

• 2. Also the $H_{\infty }$ norm $\Vert G(z)\Vert _{\infty }=\sup _{|z|=1} |G(z)|$ of the closed loop transfer function G(z) (and its shifted version zG(z)) is exactly the steady state value |G(1)|. This value can be used to compute a good upper bound for the induced norm of the matrix operator in the supervector description of $S(A-BF, B, C)$ and its shifted version $S(A-BF, B, CA, CB)$.

Within the monotonicity property, there is an infinity of design opportunities to influence closed loop response speed and bandwidth. Once achieved, the computation of a suitable range of $\beta$ for use in steepest descent-based Iterative Control can then be computed from steady state step response characteristics. The approach also creates opportunities for simplification of the algorithm. For example, if the interlacing condition is satisfied and the differences $|z_j - p_j|, 1 \le j \le n-1$ are small, then the closed loop system can be thought of as being the product of a first order model G(z) and a model error U(z) described by,

$$\begin{aligned} G(z) = G(1)\frac{(1-p_n)}{(z-p_n)} \quad and \quad U(z) = \prod _{j=1}^{n-1}\left( \frac{ (z-z_j)(1-p_j)}{(z-p_j)(1-z_j)}\right) . \end{aligned}$$

(8.22)

Noting that $U(1)=1$, it is a simple matter to prove that U(z) is also strictly positive real. The use of the first order model (in shifted form) as the model to be used for steepest descent Iterative Control will therefore be successful with a suitable value of $\beta$. Using the techniques of Sect. 7.5.3, the permissible range of $\beta$ is computed from the graphical condition that, whenever $|z|=1$,

$$\begin{aligned} \left| \frac{1}{\beta } - U(z)\left( zG(z)\right) \left( z^{-1}G(z^{-1})\right) \right| = \left| \frac{1}{\beta } - U(z) \frac{|G(1)|^2(1-p_n)^2}{(z-p_n)(z^{-1} - p_n)}\right| < \frac{1}{\beta } \end{aligned}$$

(8.23)

8.2.2 Compensator Structures

Given the final, pre-conditioned, asymptotically stable, plant model G, the design of steepest descent algorithms opens up a wide range of possibilities. Based on the successful analysis of inverse and gradient-based algorithms, the following general structures could form the basis of systematic design. The two compensator characterizations considered below contain elements of approximate inversion aimed at producing rapid and controlled convergence rates but also make use of the “zero-phase” aspects of steepest descent. The details are as follows and are based on the use of an “approximate right inverse” $G_{aR}$ of a discrete state space system G,

Algorithm 8.3

(Combined Inverse and Gradient-based Control when $m \le \ell$) Suppose that $m \le \ell$ with $\mathscr {R}[G]= \mathscr {Y}$. Suppose also that $\mathscr {R}[G_{aR}]\cap ker[G]=\{0\}$ and $ker[G_{aR}]=\{0\}$. In particular it follows then that $\mathscr {U} = \mathscr {R}[G_{aR}]\oplus ker[G]$. A combined inverse and gradient-based input update rule could have the form

$$\begin{aligned} u_{k+1} = u_k + \beta G_{aR}(GG_{aR})^*e_k, \quad k \ge 0, \end{aligned}$$

(8.24)

where $G_{aR}$ is an asymptotically stable compensation network. Conceptually, it can be regarded as an approximate right inverse of G. The idea that $G_{aR}$ is an approximate right inverse is not interpreted in a narrow sense and allows the design engineer freedom to roam over a wide range of possibilities.

The resultant error evolution is represented by the supervector recursion

$$\begin{aligned} e_{k+1}=(I-\beta GG_{aR}(GG_{aR})^*)e_k \end{aligned}$$

(8.25)

and the choice of $\beta$ can be based on the conditions for the steepest descent algorithm for the m-input, m-output system $GG_{aR}$. Analysis of robustness to left multiplicative model errors can be approached using the techniques of Sect. 7.5.3.

The following is a similar result using “approximate left inverse” systems $G_{aL}$ of G.

Algorithm 8.4

(Combined Inverse and Gradient-based Control when $m \ge \ell$) Suppose that $m \ge \ell$, that $ker[G]=\{0\}$, that $\mathscr {R}[G_{aL}]= \mathscr {U}$ and that $ker[G_{aL}] \cap \mathscr {R}[G]=\{0\}$. In particular, it follows that $\mathscr {Y} = ker[G_{aL}]\oplus \mathscr {R}[G]$. The compensator structure naturally takes the form

$$\begin{aligned} u_{k+1} = u_k + \beta (G_{aL}G)^*G_{aL}e_k, \quad k \ge 0, \end{aligned}$$

(8.26)

where $G_{aL}$ represents an approximate left inverse of G. The error evolution is

$$\begin{aligned} e_{k+1}=((I-\beta G(G_{aL}G)^*G_{aL}))e_k \end{aligned}$$

(8.27)

Defining $w_k = G_{aL}e_k, ~~ k \ge 0$, then

$$\begin{aligned} w_{k+1}=(I-\beta G_{aL}G(G_{aL}G)^*)w_k \end{aligned}$$

(8.28)

and choice of $\beta$ can be based on the conditions for the steepest descent algorithm for the $\ell$-input, $\ell$-output system $G_{aL}G$. Monotonic convergence of $\{w_k\}_{k \ge 0}$ to zero then ensures that $e_k \rightarrow e_0^{(2)}$ where the initial error has the unique decomposition $e_0 = e_0^{(1)} + e_0^{(2)}$ with $e_0^{(1)} \in \mathscr {R}[G]$ and $e_0^{(2)} \in ker[G_{aL}]$.

Finally, analysis of robustness to right multiplicative model errors can be approached using the techniques of Sect. 7.5.3.

Note that in the case when the approximate inverse is an exact inverse, the algorithms reduce to the simple inverse model algorithms described in Chap. 6.

8.2.3 Stable Inversion Algorithms

Combined approximate inverse and gradient concepts by adding bi-directional filtering offers a possible means of coping with inverse algorithms when the plant model G is non-minimum-phase.

The Case of $m=\ell =1$: Consider the case of SISO systems with $n_z$ zeros $z_1, \ldots , z_{n_z}$, the first $n_+$ of which, $\{z_j\}_{1 \le j \le n_+}$, have a modulus $|z| > 1$ with all remaining zeros satisfying $|z|<1$. The transfer function of this system can be written as a product $G=G_{ap}G_m$. That is,

$$\begin{aligned} G(z) = G_{ap}(z)G_m(z) \quad where \quad G_{ap}(z) = \prod _j^{n_+}\left( \frac{z-z_j}{1-zz_j}\right) ~. \end{aligned}$$

(8.29)

Here $G_m$ is minimum-phase as the zeros of G at the points $z=z_j, ~~ 1 \le j \le n_+$ have been replaced by zeros at points $z=z_j^{-1}, ~~ 1 \le j \le n_+$. The factor $G_{ap}(z)$ is an asymptotically stable, all-pass network in the sense that

$$\begin{aligned} G_{ap}(z)G_{ap}(z^{-1}) \equiv 1, ~\quad and~~hence \quad |G_{ap}(z)|\ \equiv 1 \quad whenever \quad |z|=1. \end{aligned}$$

(8.30)

Following the application of the relevant shift, the system $G_m$ has a stable inverse $G_m^{-1}$. Consider the Iterative Algorithm

Algorithm 8.5

(Stable Inverse Algorithm when $m=\ell =1$) Using the notation defined above, suppose that the input and output spaces have the Euclidean norms $\Vert f\Vert =\sqrt{f^Tf}$ (so that $Q=R=1$). Suppose also that the plant has dynamics represented by UG where the multiplicative modelling error U(z) is asymptotically stable. Consider the Iterative Algorithm input update rule

$$\begin{aligned} u_{k+1} = u_k + \beta G_m^{-1}FF^*G^*_{ap}e_k \end{aligned}$$

(8.31)

where F is a filter as described in Sect. 8.1. The resultant error evolution is described by

$$\begin{aligned} e_{k+1} = (1- \beta U G_{ap}FF^*G^*_{ap})e_k, \quad k \ge 0. \end{aligned}$$

(8.32)

The error sequence hence has the structure of steepest descent control for a process with model $G_{ap}F$ and modelling error U(z). Taking $m_F=1$, application of the robustness conditions for steepest descent iterations then proves that $\{e_k\}_{k \ge 0}$ converges to zero for all choice of $\beta$ satisfying the graphical condition

$$\begin{aligned} \left| \frac{1}{\beta } - U(z)|F(z)|^2 \right| < \frac{1}{\beta }, \quad whenever \quad |z|=1. \end{aligned}$$

(8.33)

The algorithm properties follow from discussion of robustness of steepest descent algorithms in Sect. 7.5.3. In particular it follows as the all-pass structure of $G_{ap}$ implies that $U(z)|G_{ap}(z)F(z)|^2 \equiv |F(z)|^2U(z)$. The interesting conclusion is that the convergence condition is precisely the condition derived for robustness of the inverse algorithm using the full (right) inverse $G^{-1}$ of the model G of UG. The replacement of this inverse by the defined stable inverse hence has no effect on the choice of gain required for monotonic convergence.

Note: There is a significant algorithm behaviour hidden behind this analysis. In the next subsection, an eigenvalue analysis indicates that convergence rates have special properties that can severely limit convergence rates in certain subspaces.

The Case of $m=\ell \ge 1$: Finally, the supervector descriptions may apply , in principle, to the case of multi-input, multi-output systems. The theoretical and computational aspects of this observation are beyond the scope of this text but it is worthwhile stating the natural structures that could be needed. More precisely, assuming $m=\ell$ and the topologies in $\mathscr {Y}$ and $\mathscr {U}$ defined by weighting matrices Q and R respectively, the natural model G should have a decomposition $G=G_{ap}G_m$ where both elements have asymptotically stable, discrete, state space models. The model of $G_m$ should also be minimum phase whilst that of $G_{ap}$ should be all-pass in the sense that its transfer function matrix $G_{ap}(z)$ satisfies the identity

$$\begin{aligned} G_{ap}(z)Q^{-1}G_{ap}^T(z^{-1})Q \equiv I_m. \end{aligned}$$

(8.34)

In operator terms, this simply a condition that ensures that $G_{ap}G_{ap}^*=I$.

8.2.4 All-Pass Networks and Non-minimum-phase Systems

In what follows, the mathematical structure of $G^*_{ap}G_{ap}$ and $G_{ap}G^*_{ap}$ is described and related to slow algorithm convergence in certain subspaces of $\mathscr {Y}$. The SISO case of $m = \ell =1$ is considered using eigenvalue analysis. Without loss of generality, it is assumed that $Q=R=1$. The main theorem is now stated,

Theorem 8.1

(All-pass System Singular Value Distribution)

Suppose that an all pass system $G_{ap}(z)$ has the form

$$\begin{aligned} G_{ap}(z) = \frac{{\prod \limits _{j = 1}^{n_+ } {(z-z_j )}}}{{\prod \limits _{j = 1}^{n_+ } {(1-z_jz )}}}=\frac{a_0+a_1z+\dots +a_{n_+}z^{n_+}}{{a}_{n_+}+{a}_{n_+-1}z+\dots +{a_0}z^{n_+}}, \ \end{aligned}$$

(8.35)

where $|z_j|>1, i=1, \ldots , n_+$, $a_{n_+}=1$ and $a_0 \ne 0$. If the system operates on the time interval $0 \le t \le N$, then there are two cases possible for the distribution of the singular values of the supervector model matrix operator $~G_{ap}: \mathscr {R}^{N+1}~\rightarrow \mathscr {R}^{N+1}$:

• 1. if the time interval length $N+1 \le n_+$, then all singular values $\sigma _j<1$.

• 2. Alternatively, there are $n_+$ singular values $\sigma _j<1, ~~1 \le j \le n_+$ whilst all remaining singular values take the value $\sigma = 1$.

Note: the eigenvalues of $G^T_{ap}G_{ap}$ and $G_{ap}G^T_{ap}$ are identical.

Proof

For any vector $u = [u(0), u(1), \ldots , u(N)]^T$ define $u(z)=\sum _{j=0}^{N}u(j)z^{-j},$ and $y(z)=G_{ap}(z)u(z)$. Note that

$$\begin{aligned} u^TG^T_{ap}G_{ap}u \le \frac{1}{2\pi i}\oint _{|z|=1}\left| G_{ap}(z)u(z)\right| ^2\frac{dz}{z} = \frac{1}{2\pi i}\oint _{|z|=1}\left| u(z)\right| ^2\frac{dz}{z}=u^Tu.\, \end{aligned}$$

(8.36)

so that $G_{ap}^TG_{ap} \le I$ and all the eigenvalues of $G_{ap}^TG_{ap}$ are less than or equal to one. That is, all the singular values of $G_{ap}$ are less than or equal to one. Now, consider the equation

$$\begin{aligned} u^T(I-G_{ap}^TG_{ap})u= 0. \end{aligned}$$

(8.37)

All the non-zero solutions u are eigenvectors of $G_{ap}^TG_{ap}$ corresponding to eigenvalues equal to one, and they span a subspace of $\mathscr {R}^{N+1}$ whose dimension is identical to the number of singular values satisfying $\sigma _i=1$. Examination of the frequency domain inequality indicates that the equality condition holds if, and only if, in the expansion $y(z) = \sum _{j=0}^{\infty }y(j)z^{-j}$, the coefficients y(j) are zero for $j > N$. That is y(z) must be a polynomial in $z^{-1}$ of degree $\le N$. There are two cases to consider, the first being the most important as, typically, N is large.

Case 1: Suppose that $N+1 \ge n_+$. Consider the choice $u(z)=a_0+a_1z^{-1}+\dots +a_{n_+}z^{-n_+},$ then

$$\begin{aligned} y(z)=G_{ap}(z)u(z)={a}_{n_+}+{a}_{(n_+-1)}z^{-1}+\dots +{a}_0z^{-n_+}. \end{aligned}$$

(8.38)

Clearly $y(j)=0$ for $j > n_+$ and hence for $j > N$. Therefore the equality in Eq. (8.36) holds, which means $u=[a_0, a_1, \ldots , a_{n_+}, 0 , \ldots 0]^T$ is a solution of (8.37). There are $N+1-n_+$ solutions of this type defined by the “shifted” polynomials

$$\begin{aligned} \begin{array}{rcccl} u_{(n_+ p)}(z) &{} = &{} z^{-(p-1)}u(z) &{} = &{} {a}_0z^{-(p-1)}+{a}_{1}z^{-(p-1)-1}+\cdots +{a}_{n_+}z^{-(p-1)-n_+}, \\ &{} &{} &{} &{} \quad where \quad p=1,\ldots ,N+1-n_+, \end{array} \end{aligned}$$

(8.39)

that also yield equality in Eq. (8.36) and the corresponding supervectors,

$$\begin{aligned} \begin{array}{rclccccccc} u_{n_+ +1}&{} = &{} [ a_0,&{} a_1,&{} \cdots ,&{} a_{n_+},&{} 0 , &{} \cdots ,&{} 0 , &{}0 ]^T \\ u_{n_+ +2}&{} = &{} [ ~~ 0, &{} a_0,&{} a_1,&{} \cdots ,&{} a_{n_+},&{} 0, &{} \cdots , &{}0 ]^T \\ \vdots &{} &{} &{} &{} &{} &{} \vdots &{} &{} &{} \\ u_{N+1}&{} = &{} [~~0, &{} 0,&{} \cdots &{} 0,&{} a_0,&{} a_1,&{} \cdots ,&{} a_{n_+}]^T \end{array} \end{aligned}$$

(8.40)

are also solutions of (8.37). This procedure generates $N+1-n_+$ linearly independent supervector solutions $\{u_p\}$ of (8.37). There are no other linearly independent solutions. To prove this, suppose there exists such a solution supervector $v= [v(0), \ldots , v(N)]^T$ represented by $v(z)=\sum _{j=0}^{N}v(j)z^{-j}$. Then there must exist a non-zero $v_0(z)=\sum _{j=N-n_+ +1}^{N}v_0(j)z^{-j}$ which is also a solution and can be constructed by subtracting linear combinations of $u_p$ from v. Note that

$$\begin{aligned} G_{ap}(z)v_0(z)=\left( \frac{a_0+a_1z+\dots +a_{n_+}z^{n_+}}{{a}_{n_+}+{a}_{(n_+-1)}+\dots +{a_0}z^{n_+}}\right) \sum _{j=N-n_++1}^{N}v_0(j)z^{-j} \end{aligned}$$

(8.41)

is a polynomial, if and only if, $v_0(z)$ cancels all the poles of $G_{ap}(z).$ This, however, cannot be satisfied because $v_0(z)$ has $n_+-1$ non-zero roots whilst $G_{ap}(z)$ has $n_+$ poles. Hence, there are exactly $N-n_+$ linearly independent solutions of (8.37) and these correspond to $N-n_+$ singular values equal to one. This also proves that all other singular values are strictly less than unity.

Case 2: $N+1<n_+$ In this case, it can be easily seen from the discussion above that there are no solutions satisfying (8.37). Hence all the singular values should be less than one. That completes the proof of the Theorem. $\square$

Theorem 8.2

(Eigenvectors of $G_{ap}G_{ap}^T$) Assume the notation of the preceding result and suppose that $N+1>n_+$. Then the eigenstructure of $G_{ap}G_{ap}^T$ has the form of

• 1. $N+1-n_+$ linear independent eigenvectors corresponding to the $N+1-n_+$ eigenvalues $\sigma ^2_j =1,~ n_+ +1 \le j \le N+1$:

  $$\begin{aligned} \begin{array}{rclccccccc} \tilde{u}_{n_+ +1}&{} = &{} [ a_{n_+},&{} a_{(n_+ - 1)},&{} \cdots &{} a_0 ,&{} 0 , &{} \cdots ,&{} 0 , &{}0 ]^T \\ \tilde{u}_{n_+ +2}&{} = &{} [ ~~ 0, &{} a_{n_+},&{} \cdots ,&{} a_1, &{} a_0, &{} 0, &{} \cdots , &{}0 ]^T \\ \vdots &{} &{} &{} &{} &{} &{} \vdots &{} &{} &{} \\ \tilde{u}_{N}&{} = &{} [~~0, &{} 0,&{} \cdots &{} 0,&{} a_{n_+},&{} a_{(n_+ - 1)}, &{} \cdots , &{} a_0 ]^T \end{array} \end{aligned}$$

  (8.42)

  The span of these vectors, denoted $\mathscr {E}_1$, has the property that $G_{ap}G^T_{ap}y = y$ whenever $y \in \mathscr {E}_1$.

• 2. The $n_+$ eigenvectors of $G_aG_a^T$ corresponding to the $n_+$ eigenvalues that are less than one span the subspace $\mathscr {E}_{a+} = \mathscr {E}_1^{\perp }$.

Proof

Using time reversal matrices, note that $G_{ap}G_{ap}^T=\mathscr {T}(1,N)G^T_{ap}G_{ap}\mathscr {T}(1,N)$ and hence the eigenvectors $u_p$ of $G_{ap}^TG_{ap}$ and $\tilde{u}_p$ of $G_{ap}G_{ap}^T$ are related by $\mathscr {T}(1,N)$. The proof of Theorem 8.1 then completes the proof of the first part of the result as a simple calculation gives $\tilde{u}_p = \mathscr {T}(1,N)u_{N+2+n_+-p}, p = n_++1, \ldots ,N+1$. The form of $\mathscr {E}_{a+}$ follows as all other eigenvalues are less than unity and the eigenspaces of symmetric matrices corresponding to different eigenvalues are orthogonal. $\square$

The two issues of importance to what follows are the magnitude of the non-unit eigenvalues and the structure of the subspace $\mathscr {E}_{a+}$. The simplest case is that when $n_+ =1$ when the zero $z_1$ is real. In this case $\mathscr {E}_{a+}$ has dimension equal to one. Using the orthogonality property of $\mathscr {E}_{a+}$ and the definition of the zero, a simple calculation shows that eigenvector is expressed simply as

$$\begin{aligned} \alpha _1 = \left[ 1,z_1^{-1}, z_1^{-2}, \ldots ,z_1^{-N} \right] ^T \end{aligned}$$

(8.43)

as $\tilde{u}_p^T\alpha _1 = 0,~~n_++1 \le p \le N+1$. Noting that, in any state space realization of U(z), the “D” matrix is simply the scalar $-z_1^{-1}$, and that the determinant is the product of the eigenvalues, write

$$\begin{aligned} \det [G_{ap}G^T_{ap}] = \det [G_{ap}]^2 = z_1^{-2(N+1)} = \prod _{j=1}^{N+1}~\sigma _j^2 = \sigma ^2_1. \end{aligned}$$

(8.44)

which proves the result that

Theorem 8.3

(Non-unit Singular Values and Eigenvectors when $n_+ = 1$) If  $n_+=1$, then the only non-unit eigenvalue of $G_{ap}G^T_{ap}$ is $\sigma _1^2 = z_1^{-2(N+1)}$ with eigenvector $\alpha _1$. In particular, as $|z_1|>1$,

$$\begin{aligned} \lim _{N \rightarrow \infty }~\sigma _1^2 = 0 \end{aligned}$$

(8.45)

and hence this eigenvalue can be made to be arbitrarily small if $z_1^{2(N+1)}$ is sufficiently large.

Note: For a given value of $z_1$, increasing the length of the interval reduces the magnitude of the eigenvalue.

It is useful at this stage to discuss the implications of this result for the stable inverse Algorithm of Sect. 8.2.3 in the special case of $F=I$ (no filtering) and the use of Euclidean norms. The resultant error evolution $e_{k+1}=(I-\beta G_{ap}G^T_{ap})e_k$ can then be described by decomposing $e_0$ into the sum $e_0=e_0^{(1)} + e_0^{(2)}$ where $e_0^{(1)} \in \mathscr {E}_{a+}$ and $e_0^{(2)} \in \mathscr {E}_1$, from which

$$\begin{aligned} e_k = (1-\beta \sigma _1^2)^k e_0^{(1)} + (1 - \beta )^k e_0^{(2)}, \quad k \ge 0~. \end{aligned}$$

(8.46)

If $z_1^N$ is large, then, whereas the second term in $\mathscr {E}_1$ becomes small quickly (depending on the choice of $\beta$), it is clear that the first term in $\mathscr {E}_{a+}$ decays infinitesimally slowly. This is clearly demonstrated in the special case of $\beta = 1$ when the second term is removed in one iteration. Suppose that $z_1 = 1.1$ and $N=127$, then $\sigma _1^2 \approx 0.5 \times 10^{-4}$ and it is clear that $(1-\beta 0.5 \times 10^{-4})^k$ reduces very slowly. This situation is further worsened as N increases. The theoretical convergence to the limit $e_{\infty }=0$ remains true but the practical implication of the zero $z_1$ is that the user, over the finite number of iterations possible in the real world, will observe an apparent convergence to a “pseudo-limit” vector

$$\begin{aligned} e^{pseudo}_{\infty } = e_0^{(1)} = \frac{\langle \alpha _1, e_0 \rangle }{\Vert \alpha _1\Vert ^2}\alpha _1 \quad (The~ Apparent~Limit~Error). \end{aligned}$$

(8.47)

Graphically, this will observed as monotonic reductions in error norm until an apparent “plateauing” or “flat-lining” of the plot of $\Vert e_k\Vert$ against k will appear at a value very close to that computed from the formula

$$\begin{aligned} \begin{array}{rcl} \Vert e^{pseudo}_{\infty }\Vert &{} = &{} \Vert e_0^{(1)}\Vert = \frac{\left| \langle \alpha _1, e_0 \rangle \right| }{\Vert \alpha _1\Vert } = \sqrt{\frac{1 - z_1^{-2}}{1-z_1^{-2(N+1)}}}\left| \langle \alpha _1, e_0 \rangle \right| \\ &{} = &{} \sqrt{\frac{1 - z_1^{-2}}{1-z_1^{-2(N+1)}}}\left| \sum _{t=0}^N z_1^{-t} e_0(t)\right| \quad (The~Apparent~Limit~Norm). \end{array} \end{aligned}$$

(8.48)

These equations shed light on the relationship between the magnitude of the apparent limit, the initial error and the value of the zero. In particular,

• 1. Error norms continue to reduce monotonically from iteration to iteration.

• 2. The apparent limit error is proportional to $\alpha _1$ which can be identified with the time series $\{1, z_1^{-1}, z_1^{-2}, \ldots , z_1^{-(N-1)}, z_1^{-N} \}$.

  • a. This signal decays geometrically with the most significant values being at the beginning of the time interval.

  • b. The larger the value of $z_1$, the smaller the size of this interval.

  • c. If $|z_1|$ is very large, the decay $z_1^{-t}$ indicates that the apparent limit has significant values only in the first few samples.

• 3. The Cauchy-Schwarz inequality proves that $\Vert e^{pseudo}_{\infty }\Vert \le \Vert e_0\Vert$. As a consequence, the smaller the magnitude of the initial error is, the smaller the magnitude of the apparent limit. In particular, the magnitude can be reduced by ensuring that the initial errors $e_0(t)$ are very small at the beginning of the time interval.

• 4. For a given norm of $e_0$, the norm of the apparent converged error is maximized when it is proportional to $\alpha _1$. If $e_0 = \alpha _1$, then $e_0^{pseudo} = e_0$ and the convergence observed in practice will immediately become problematic.

• 5. Given the above and assuming zero initial conditions, the choice of $u_0=0$ will produce the initial error $e_0 = r$. This suggests that

  • a. a reference signal r(t) that is small or zero at sample points when $z_1^{-t}$ is significant relative to unity will produce small apparent limit vector magnitudes.

  • b. If, however, $|z_1|$ is close to unity then $e_0^{pseudo}$ can still be small but the value will depend on the details of the time series $\{r(t)\}$. As a guide to what may be required, consider the case where $z_1$ is sufficiently close to unity so that $z_1^t \approx 1$ for $0 \le t \le N$. In this circumstance, the plateau will occur at a small value of the norm only if the arithmetical mean of the elements of r is around zero.

The final observation suggests that highly “active” reference signals will tend to reduce the plateau value. Also, when made possible by the conditions required by the application, it might be a good design strategy to lengthen the interval $0 \le t \le N$ to include the addition of $N_I$ sample points to form an initial period where the reference signal is taken to be zero. The tracking of r on the original interval may then be replaced by the tracking of the new reference signal $\{0,0, \ldots , 0 ,r(0), r(1) , \ldots , r(N) \}$ on $0 \le t \le N+N_I$. The simple calculation with $e_0$ equal to the new reference gives

$$\begin{aligned} \sum _{t=0}^{N+N_I+1} ~z_1^{-t}e_0(t) = z_1^{-N_I}\left( \sum _{t=0}^{N+1} ~z_1^{-t}r(t) \right) ~, \end{aligned}$$

(8.49)

which indicates that the observed limit is reduced by a factor of approximately $z_1^{-N_I}$, a factor that can be small if $N_I$ is large enough. For example, if $N_I = 8$ and $z_1=2$, then the reduction factor is $2^{-8} \approx 4 \times 10^{-3}$ which is substantial.

Intuitively, the addition of this interval adds degrees of freedom that the algorithm can use to create inputs that reduce the pseudo-limit error to smaller magnitudes more rapidly. This will be manifested by input activity on $0 \le t \le N_I$. In effect, a non-zero “initial condition” for the remaining $N+1$ samples is created to produce the practical error reduction sought.

The general ideas described above also apply when $n_+ > 1$. The following results consider the eigenvalues and eigenvectors as N increases.

Theorem 8.4

(Distinct Zeros and a Basis for $\mathscr {E}_{a+}$) Suppose that the zeros $\{z_j\}_{1 \le j \le n_+}$ are distinct and that $N \ge n_++1$. Then $\mathscr {E}_{a+}$ is spanned by the vectors $\alpha _j = \alpha (z_j^{-1}), ~1 \le j \le n_+$, where

$$\begin{aligned} \alpha (z) = \left[ 1,z, z^2, \ldots ,z^N \right] ^T, \quad 1 \le j \le n_+. \end{aligned}$$

(8.50)

Moreover, the Euclidean norm of each $\alpha _j$ remains bounded as $N \rightarrow \infty$.

Note: This defines $\alpha _j$ if $z_j$ is real but, if complex with $z_k=z_j$, the pair $\alpha _j$ and $\alpha _k$ should be replaced by the real and imaginary parts of $\alpha _j$.

Proof

From Theorem 8.2 and the definition of the zeros, it is a simple matter to demonstrate that $\tilde{u}^T_p \alpha _j = 0$ so that $\alpha _j \in \mathscr {E}_{a+}$. That the $\{\alpha _j \}$ span $\mathscr {E}_{a+}$ follows as they are linearly independent when the zeros are distinct. Finally, a simple calculation gives $\overline{\alpha }_j^T \alpha _j < (1-|z_j|^{-2})^{-1}$ for all N. $\square$

Theorem 8.5

(Non-unit Eigenvalues when $n_+{ \ge }1$) Suppose that the zeros $\{z_j\}_{1 \le j \le n_+}$ of the all-pass system $G_{ap}$ are distinct. Then, all non-unit eigenvalues $\sigma _j^2, 1 \le j \le n_+$ of $G_{ap}G^T_{ap}$ (and hence $G^T_{ap}G_{ap}$) converge to zero as $N \rightarrow \infty$. In particular, defining $z_0 = \min \{|z_1|, \ldots , |z_{n_+}|\}$, there exists a scalar $M_{(+)}$ such that

$$\begin{aligned} \max \{\sigma _1^2 , \ldots , \sigma _{n_+}^2\} \le M_{(+)} z_0^{-2N}~\quad for~~all \quad N \ge n_++1. \end{aligned}$$

(8.51)

Proof

It is sufficient to show that $\alpha _j^T G_{ap}G^T_{ap} \alpha _j \le (|z_j|-1)^{-1} |z_j|^{-2N}, 1 \le j \le n_+$. Without loss of generality, take $j=1$ and consider the input time series defined by $\alpha _1 = \{ 1,z_1^{-1}, \ldots , z_1^{-N} \}$. Using time reversal operators $\mathscr {T}(1,N)$, $\alpha _1^T G_{ap}G^T_{ap} \alpha _1 = (\mathscr {T}\alpha _1)^T G^T_{ap}G_{ap}(\mathscr {T} \alpha _1)$. The $\mathscr {Z}$-transform of the time series $\mathscr {T} \alpha _1$is just

$$\begin{aligned} \ (\mathscr {T}\alpha _1)(z) = \sum _{j=0}^N~ z^{-j}z_1^{j-N} = z_1^{-N}\sum _{j=0}^N~ z^{-j}z_1^j = z_1^{-N} \left( \frac{(1- (\frac{z_1}{z})^{N+1})}{1-\frac{z_1}{z}} \right) . \end{aligned}$$

(8.52)

and $G_{ap}\mathscr {T} \alpha _1$ is the restriction of the infinite time series on $0 \le t \le \infty$ generated by $G_{ap}(z)(\mathscr {T}\alpha _1))(z)$ to the interval $0 \le t \le N$. Defining $\tilde{G}_{ap}(z) = G_{ap}(z) \frac{z}{z-z_1}$, the pole-zero cancelation produces a stable transfer function with $H_{\infty }$ norm

$$\begin{aligned} \Vert \tilde{G}_{ap}(z)\Vert _{\infty } = \sup _{|z|=1}\left| \tilde{G}_{ap}(z) \right| = \sup _{|z|=1}\left| \frac{z}{z-z_1} \right| = (|z_1|-1)^{-1}. \end{aligned}$$

(8.53)

Noting that the term $(\frac{z_1}{z})^{N+1}$ in $(\mathscr {T}\alpha _1)(z)$ does not contribute to the time series on $0 \le t \le N$, the use of discrete time reversal operators gives

$$\begin{aligned} \begin{array}{rcl} \alpha _1^T G_{ap}G^T_{ap} \alpha _1 &{} ~=~ &{} (\mathscr {T}\alpha _1)^T G^T_{ap}G_{ap}(\mathscr {T} \alpha _1) \\ &{} ~\le ~ &{} z_1^{-2N}\frac{1}{2\pi i}\oint _{|z|=1}~ \left| \tilde{G}_{ap}(z) \right| ^2 \frac{dz}{z} \le \Vert \tilde{G}_{ap}(z)\Vert ^2_{\infty } z_1^{-2N} \end{array} \end{aligned}$$

(8.54)

which is the desired result. $\square$

The immediate conclusion from the above is that the consequences, for convergence, of having $n_+ > 1$ distinct zeros are the same as those already described for the case of $n_+=1$. The changes are in the technical details, but the general conclusion, for the stable inverse algorithm, is that, for large values of N (measured by the need for $z_0^{-2N}$ to be very small), the algorithm will appear to converge to the orthogonal projection $e_0^{(1)}$ of $e_0$ onto $\mathscr {E}_{a+}$. If $E_+$ is an $(N+1) \times n_+$ real matrix whose columns span $\mathscr {E}_{a+}$, then

$$\begin{aligned} e_0^{pseudo} = e_0^{(1)} = E_+ (E_+^TE_+)^{-1}E^T_+e_0 \quad with \quad \Vert e_0^{(1)}\Vert ^2 = e_0^T E_+(E_+^TE_+)^{-1}E_+^Te_0. \end{aligned}$$

(8.55)

Using the vectors $\{\alpha _j\}$ as models of such a basis, the reader can easily verify that initial errors $e_0$ that take small values on some interval close to the origin will have apparent convergence but to a plateau where norm values are, potentially, small.

The section is concluded by two observations

• 1. Repeated Zeros: The most useful conclusion from the analysis is expected to carry over to the more general case when some or all of the zeros in an all-pass system are repeated. It is left as an exercise for the reader to verify that, if the system has q distinct zeros $z_j$, one or more of which has multiplicity $n_j \ge 1$, then $\mathscr {E}_{a+}$ is spanned by vectors, $1 \le p \le n_j$, $1 \le j \le q$, computed from the (scaled) time reversal $z_j^{-N}\mathscr {T}(1,N)\gamma _{jp}$ of the supervectors

  $$\begin{aligned} \gamma _{jp} = \frac{d^{p-1}}{dz^{p-1}}\alpha (z) \quad evaluated ~~ at \quad z=z_j. \end{aligned}$$

  (8.56)

• 2. Gradient Algorithms: Many of the results have implications for other algorithms where the plant has non-minimum-phase zeros. For example, if $m=\ell =1$, write the transfer function $G(z) = G_m(z)G_{ap}(z)$ where $G_m$ is minimum-phase and $G_{ap}$ is all-pass. In the supervector form, $G=G_mG_{ap}$. Write $GG^T = G_m G_{ap}G^T_{ap}G_m^T$ and note that, if $v_0\in \mathscr {E}_{a+}$ and $e_0 = (G_m^T)^{-1}v_0$, then, if N is large, the norm of the product $GG^Te_0$ will be infinitesimally small relative to $\Vert e_0\Vert$. As a consequence $e_1 = (I - \beta GG^T)e_0 \approx e_0$ and convergence will be infinitesimally slow. Note: The issue of slow convergence in $(G_m^T)^{-1}\mathscr {E}_{a+}$ will be compounded, for low pass systems by the inevitable slow convergence of the high frequency components of the error. However, whereas high frequencies are associated with rapidly varying time series such as $\{1,-1,1,-1, \ldots \}$, the consequences of non-minimum-phase zeros are associated with slowly varying signals such as $\alpha _1$ which contains substantial low (but still slowly converging) frequency components.

8.3 Gradients, Compensation and Feedback Design Methods

The previous sections have indicated the potentially powerful role of compensation in steepest descent algorithms and the inhibiting role of non-minimum-phase zeros on convergence rates. It also demonstrated the natural place of inverse models in Iterative Control and has suggested that good choices of compensator will, in some way, have a role that mirrors that of good approximate models of an inverse of G. Feedback design methodologies can be regarded as methods of designing structured approximations to inverse systems by ensuring good tracking of a specified reference signal. By their very nature, they must recognize the very significant place of non-minimum-phase zeros in the design process.

The following discussion is formulated using language relevant to applications to linear, time-invariant, discrete time state space systems S(A, B, C, D) and, in the following section, to the continuous time case. It describes Iterative Control design as a two stage process beginning with feedback system design for the plant followed by its conversion into a steepest descent-like algorithm. Note, however, that, being formulated in operator notation, it also applies more generally although the design of feedback systems in such cases may need further development.

8.3.1 Feedback Design: The Discrete Time Case

Algorithm 8.6

(Compensators based on Feedback Design Principles) Suppose that Q is a constant matrix (independent of time) defining the topology in $\mathscr {Y}$. Suppose also that $K_c$ is the forward path compensator in a unity negative feedback system for the plant G. The output from this closed-loop system is the plant output y whilst its inputs (equivalent to reference signals) are denoted by $\tilde{u}$.

The design criteria for $K_c$ is assumed to include closed loop stability and familiar performance objectives in the frequency domain with the aim of providing some ability to track, albeit approximately, the reference signal r. Implicitly, it is assumed that the tracking is good and that the compensator ameliorates the effects of plant properties such as oscillation and loop interaction, the effects of non-minimum-phase zeros and, where thought to be of significance, to incorporate issues such as sensitivity, robustness and disturbance rejection.

Let $\mathscr {V}$ denote the input space for the closed loop system and assume that this is the Hilbert space $\mathscr {R}^{m(N+1)}$ with the same inner product as $\mathscr {Y}$ but with Q replaced by a constant, $m \times m$, symmetric, positive definite matrix $Q_{\textsf {T}}$. The closed loop operator $\textsf {T}$ and sensitivity operator $\textsf {S}$ of the resultant closed loop system will be denoted by

$$\begin{aligned} \textsf {T} = (I+GK_c)^{-1}GK_c: \mathscr {V} \rightarrow \mathscr {Y} \quad and \quad \textsf {S}=I- \textsf {T} = (I+GK_c)^{-1}: \mathscr {V} \rightarrow \mathscr {Y}. \end{aligned}$$

(8.57)

The compensator for the compensated steepest descent algorithm is then constructed using

$$\begin{aligned} K = K_c \textsf {S} ~and~ GK = GK_c \textsf {S}= \textsf {T}, \end{aligned}$$

(8.58)

and the input update formula

$$\begin{aligned} u_{k+1}= u_k + \beta K_c \textsf {S} \textsf {T}^{*} e_k, \quad k \ge 0. \end{aligned}$$

(8.59)

The resultant error evolution has the form

$$\begin{aligned} e_{k+1} = (I-\beta \textsf {T} \textsf {T}^{*})e_k, \quad k \ge 0, \end{aligned}$$

(8.60)

which takes the form of the error evolution obtained when the steepest descent algorithm is applied to the “plant” model $y=\textsf {T}\tilde{u}+d$ tracking the reference r with initial values $\tilde{u}_0$. The consequent Iterative algorithm is convergent for all $\beta$ in the range $0 < \beta \Vert T^*\Vert ^2 < 2$.

The interpretation of the symbol $\textsf {T}^{*}$ contains additional flexibility in the design process in the form of $Q_{\textsf {T}}$ which plays the role of R as it was used in previous sections. With this in mind, $\Vert \textsf {T}\Vert = \Vert \textsf {T}^{*}\Vert$ and it can be bounded by

$$\begin{aligned} \Vert T\Vert \le \Vert Q^{1/2}\textsf {T}(z)Q_{\textsf {T}}^{-1/2}\Vert _{\infty } = \sqrt{\sup _{|z|=1} r(\textsf {T}(z)Q_{\textsf {T}}^{-1}\textsf {T}^{~T}(z^{-1})Q)}. \end{aligned}$$

(8.61)

The upper bound is an increasingly accurate estimate of the norm as N increases.

Design Notes: An insight into the potential benefits of a combination of feedback design and gradient iteration can be obtained by a consideration of the input-output equation $y= \textsf {T}r$ of the feedback system. Perfect tracking requires that $\textsf {T}=I$ which is unattainable using feedback but a well-designed feedback system typically ensures that y tracks r well except, possibly, at high frequencies. Expressed in terms of bandwidth for discrete time systems where G and $K_c$ have state space models and associated transfer function matrices G(z) and $K_c(z)$, the transfer function matrix associated with $\textsf {T}$ can be denoted by $\textsf {T}(z)= (I+G(z)K_c(z))^{-1}G(z)K_c(z)$. Good tracking often means that $\textsf {T}(z)$ is close to the identity I over a desired bandwidth and, in the absence of inhibiting factors such as substantial high frequency resonances, it is likely that the associated operator norm $\Vert \textsf {T}\Vert \approx 1$. A clearer picture of what is required is obtained using the frequency domain descriptions of Sect. 7.2.3. More precisely, the supervector response $(I-\beta \textsf {T} \textsf {T}^{~*})W_j(z_k)$ to the time series $z_k^t w_j(z_k), ~~ 0 \le t \le N$, represents the approximate time series

$$\begin{aligned} (I - \beta \textsf {T}(z) Q_{\textsf {T}}^{-1} \textsf {T}^T(z^{-1}) Q )z_k^t w_j(z_k), \quad 0 \le t \le N. \end{aligned}$$

(8.62)

The effect of the operator $\textsf {T}\textsf {T}^*$ is closely linked to the values of transfer function matrix $\textsf {T}(z) Q_{\textsf {T}}^{-1} \textsf {T}^T(z^{-1})Q$. The closer this matrix is to the identity and the wider the bandwidth over which this property can be achieved, the more likely it will be that frequency components in that bandwidth will be reduced by a factor of $1-\beta$.

The discussion therefore suggests that, if $K_c$ provides excellent feedback control of G and the dominant frequency content of the reference r lies in the bandwidth of the closed loop system $\textsf {T}$, then rapid convergence could be attained with a choice of gain $\beta \approx 1$.

Finally, the introduction of the use of feedback design potentially puts the proposed iterative compensation technique into the familiar area of classical multi-loop control systems design. Using the notation of discrete systems, the approach to algorithm design then includes the computational stages:

• 1. Firstly, consider the properties of G(z) and any potential benefits of replacing it by and $\varepsilon$-weighted model.

• 2. Next, design a forward path compensator $K_c(z)$ for the plant G(z) based on perceived needs for stability, the desired bandwidth and robustness.

• 3. Next, construct state space models of the systems $K(z)\,=\,K_c(z)(I\,+\,G(z)K_c(z))^{-1}$ and $\textsf {T}(z) = G(z)K(z) = (I+G(z)K_c(z))^{-1}GK_c(z)$ and, adding in the choices of Q and $Q_{\textsf {T}}$, obtain a state space model of $\textsf {T}^*$.

• 4. Analyze the properties of $\textsf {T}(z)$ and, if appropriate, re-assess the benefits of using an $\varepsilon$-weighted model in algorithm construction.

• 5. If required, assess the monotone robustness of the anticipated Iterative Control algorithm using the techniques of Sect. 7.5.3 and known, or assumed, data on anticipated multiplicative modelling errors U.

• 6. Finally, choose a value of $\beta$ and construct the input update rule as a computational procedure that computes the response $T^*e_k$ using reverse time simulation from zero terminal conditions and then sets $u_{k+1}-u_k$ equal to the response, from zero initial conditions, of $\beta K$ to that output.

Note: The summary above could also include the use of relaxation factors as discussed in Sect. 5.2.3.

8.3.2 Feedback Design: The Continuous Time Case

The observant reader will note that the basic idea of feedback design in Iterative Control generalizes quite naturally to the case of continuous time, state space systems. The changes required lie in the detail of the feedback design and the need to use the more general Theorem 7.1 to assess convergence properties. The main issue here is that, for convergence of errors to zero, the closure $\overline{\mathscr {R}[TT^*]}$ of the range of $TT^*$ must contain the reference signal otherwise a residual error equal to the orthogonal projection of the reference signal  r onto $\overline{\mathscr {R}[TT^*]}^{\perp } = ker[T^*]$ will remain.The relevant modifications needed to include relaxation factors can be deduced from Algorithm 7.7.

8.4 Discussion and Further Reading

The concepts of inversion and gradient-based Iterative Control, although seemingly based on different basic principles, have in common the monotonic reduction of chosen signal norms. It is not surprising, therefore, that there is the possibility of bringing them together in the development of feedforward algorithms (see, for example, [85]). The key is the use of gradient methodologies as applied to compensated plant models, the benefits being that the convergence and robustness results available for steepest descent algorithms can be used on the compensated process. This compensation has two general forms: the first is the pre-conditioning of the plant by control loops that address issues of plant input-output performance such as resonance whilst the second is the (series) compensator in the Iterative Control algorithm.

Pre-conditioning seems to be essential even from the point of view of ensuring acceptable plant behaviour during each iteration, plant operation and test. However, taking the characteristics of the reference signal r into account may help in guiding the process. The form of the pre-conditioning is not specified precisely by algorithmic considerations so almost all of the many control systems design techniques developed over the past eight decades could, in principle, be used. They include classical Nyquist, Bode and root-locus analysis tools for output feedforward and feedback design but also include state feedback design approaches and many of the frequency domain approaches to the design of MIMO (multi-input, multi-output, multi-loop, multivariable, multi-channel) feedback systems. Such approaches should be influenced by the nature of the reference signal r which may allow the application of the Internal Model Principle to improve tracking performance.

The pre-conditioned plant becomes the new plant to which Iterative Control will be applied. Inverse algorithms could be applied to these without modification but could suffer from robustness issues and/or instability of the inverse (and associated sensitivity and computational problems) if the plant is non-minimum-phase. For minimum-phase systems, the simple modification described in Sect. 8.1 shows how bi-directional filtering can be included in the inverse algorithm. This leads to error evolution which has the structure of a gradient algorithm for the filter. As the filter will, typically, have simple structure, the convergence conditions will be simple to deduce as $\Vert F\Vert$ will be computable exactly off-line. The filter provides a mechanism for affecting and improving the robustness of the inverse algorithm.

Simple filters offer the potential for substantial benefits but are only the tip of the iceberg when it comes to discussing the choice of feedforward compensator. The next easily constructed step in complexity considers non-minimum-phase systems. The computational aspects for multi-loop systems is not included in the text but the benefits are clearly seen by considering the single-input, single-output case. By factoring the model transfer function into the series connection of a minimum-phase system and an all-pass network, the inverse algorithm can be modified as in Sect. 8.2.3 (see also [61]). More precisely, by only inverting the minimum-phase factor, the algorithm reduces to that of a steepest descent algorithm for the all-pass system (plus any filter included). This change removes unstable computations from the algorithm and, in the absence of filtering, leaves the range of permissible gains $\beta$ unchanged. The hidden issue here is that of the rate of convergence and its relationship to the zeros and the reference signal. Section 8.2.4 provides theoretical evidence that there is, indeed, a significant effect on convergence rates. The analysis takes the form of proving that the space of error signals can be decomposed into two components: in one, the error converges to zero as expected from the inverse algorithm but, in a subspace of dimension $n_+$ defined by the $n_+$ non-minimum-phase zeros, error convergence in practice will be infinitesimally slow if N is large. This may or may not be a problem because the error norm reduces to what appears, in practice, as a plateau beyond which further reductions are so slow as to be virtually unseen. This effective minimum achievable tracking error depends on the initial error $e_0$ and time series/vectors defined by the zeros. This relationship throws light on the form of reference signal that reduces the norm value on the plateau and hence makes improved tracking possible. It suggests that a good strategy could be to use, when possible, reference signals augmented by extending the time interval to include an initial “lead-in phase”. This adds degrees of freedom whose objective is to attain reduced plateau errors.

The non-minimum-phase analysis in this chapter has focussed on applications to stable inverse algorithms but was originally addressed for Norm Optimal Iterative Learning Control (see Chap. 9). The approach used is based on results in the text [51] on Toeplitz matrices and applied to discrete systems in [84]. A generalization to continuous systems is also available in [86] although this paper is technically more complex as the underlying spaces are infinite dimensional. The papers include comprehensive simulation and experimental results which provide a convincing verification of the physical reality of the phenomenon in practice.

More general compensation structures may suit or be necessary in some applications and further research may be needed in each case. The general principle of using compensators that are structured approximations to inverse systems naturally suggests the use of feedback methodologies. The general class of algorithms described in Sect. 8.3 have potential value as they link closely to the output feedback design techniques that may have been used in the pre-conditioning phase. In effect, the compensator is constructed from a feedback controller designed for the pre-conditioned plant. The reader is reminded that this model will often be the original plant model modified by output shifts.

MIMO control systems design methodologies are many [71, 81] and the author emphasizes his belief that there is no answer to the question “which is the best?”. Each reader will have their own favorite ranging from simple process controllers, [65], to the complexities of $H_{\infty }$ control [21, 23]. The development of good practice in this area will depend on the applications sector and the education, skills and preferences of the design team.