Abstract
Norm optimal iterative learning control (NOILC) has recently been applied to iterative learning control (ILC) problems in which tracking is only required at a subset of isolated time points along the trial duration. This problem addresses the practical needs of many applications, including industrial automation, crane control, satellite positioning and motion control within a medical stroke rehabilitation context. This paper provides a substantial generalization of this framework by providing a solution to the problem of convergence at intermediate points with simultaneous tracking of subsets of outputs to reference trajectories on subintervals. This formulation enables the NOILC paradigm to tackle tasks which mix “point to point” movements with linear tracking requirements and hence substantially broadens the application domain to include automation tasks which include welding or cutting movements, or human motion control where the movement is restricted by the task to straight line and/or planar segments. A solution to the problem is presented in the framework of NOILC and inherits NOILC’s well-defined convergence properties. Design guidelines and supporting experimental results are included.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
1 Introduction
The objective of iterative learning control (ILC) is to follow a motion profile defined over a fixed finite time interval, 0 ≤ t ≤ T, using data from previous executions of this task to suitably update the control input until perfect tracking of a defined reference, r(t), t ∈ [0, T]is achieved. Often in combination with current-trial information, the aim is to sequentially improve the performance of the operation as the trial/iteration/repetition index, k, increases. Since its formal conception 30 years ago[1], ILC has been an area of intense research interest in both theoretical and application domains[2,3]. A body of recent work has succeeded in removing the stipulation that tracking of a reference signal is required at all points of [0, T]. The framework that has been developed enforces tracking at only a subset of points {t j }1≤j≤M [3–7] and hence addresses applications such as production line automation, crane control, satellite positioning, and robotic “pick and place” tasks in which the system output (e.g., payload position) is only critical at a finite set of prescribed time instants. It has also been used to model human motor control[8] and within stroke rehabilitation to assist patients performing reaching and object manipulation tasks[9].
This paper expands the framework to tackle more general tracking problems that encompass the requirement
-
1)
that a defined output signal passes through specified points at a finite number of times {t j }⊂ [0, T],
-
2)
that, on defined subintervals \([{\hat t_j},{\hat t_{j + 1}})\), defined combinations of subsets of the outputs track required signals,
-
3)
that, on defined subintervals, the output vector is unspecified but lies on a plane in the output space and,
-
4)
no tracking of any kind is required elsewhere on [0, T].
These tasks fit many requirements needed in automation tasks which include welding or cutting movements, or human motion control where the movement is restricted by the task to straight line and/or planar segments. This expansion of the framework also enables it to tackle a special case of the “spatial ILC” problem[10–12] in which a path is specified in space (or more generally, as a functional constraint between outputs), but no temporal constraint is imposed on how the system moves along this path (or plane). Spatial ILC presents substantial novelty, but no framework currently exists which provides widely applicable algorithmic solutions and performance analysis. The inclusion of aspects of spatial ILC hence addresses applications requiring a predefined path (or plane) to be specified between intermediate point locations, but opens up the possibility of moving along the path (or plane) in a way that achieves some other objective (e.g., control effort minimisation).
Solutions are developed in this paper using the framework of norm optimal ILC (NOILC), which is a prominent member of the class of gradient based ILC algorithms whose convergence and robustness properties have been extensively studied by many groups [13–15]. Appearing originally in [16], NOILC embeds precise control over error and input norm evolution and has received significant attention in the ILC community[17–19], with experimental applications including an accelerator based free electron laser[20], multi-axis laser facility[19] and within stroke rehabilitation[20]. Theoretical developments to NOILC include extension to discrete time[21], an N-iteration ahead predictive solution[22], acceleration mechanisms[23], the inclusion of convex input constraints[24] and analysis of the effect of non-minimum phase zeros[25, 26]. To address 1)–4), a substantial generalization of NOILC is required to combine a general point-to-point tracking problem with partial tracking requirements on subintervals. It is achieved in this paper using an operator formulation of the plant dynamics and the application of NOILC concepts to construct an iterative algorithm that converges to a control input signal that ensures that specified values of system outputs y(t j ) or linear combinations of outputs F j y(t j ) are achieved at specified intermediate times {t j }1≤j≤M, whilst simultaneously ensuring that specified, and possibly different, linear combinations \({P_j}y(t),1 \le j \le \hat M\) of outputs track specified signals on (possibly different) subintervals \([{\hat t_{j - 1}},{\hat t_j}]\). Such combinations could be simple tracking tasks for a subset of the outputs or they could represent constraints on output behaviour that are necessary for effective operation. If no continuous tracking is required on an interval [t q −1, t q ], then this is simply described by setting P q = 0. The problem therefore combines the requirement to pass through defined points with the need to track defined signals on disjoint sub-intervals. The formulation is extremely general but can be realized by implementable feedback/feed forward ILC algorithms.
This paper is structured as follows. In Section 2, the intermediate weighting problem with partial tracking requirements for continuous state space systems is modelled in a general product Hilbert space setting and applied to formulate the ILC problem in Section 3 as a NOILC problem. Experimental validation results are given in Section 4 on a non-minimum-phase electromechanical test facility and their relationship to theoretical predictions is discussed. Conclusions and future work appear in Section 5.
2 Modelling in a product Hilbert space
The problem can be interpreted as a NOILC objective with emphasis placed on rapid and accurate convergence of all (or a subset) of outputs (or linear combination of outputs) at specified intermediate times {t j }1≤j≤M with effective tracking of defined signals constructed from (linear combinations of) outputs on subintervals \([{\hat t_{j - 1}},{\hat t_j}],1 \le j \le \hat M\). Through the introduction of parameters P j , Q j , 1 ≤ j ≤ M and \({\hat Q_j},1 \le j \le \hat M\), the problem can hence be expressed as a combination of the tracking of a standard reference r(t), t ∈ [0, T], and intermediate point tracking. This enables NOILC solutions to be derived using the relevant operators, signals and underlying signal spaces.
2.1 Input/output notation
Let S(A, B, C)be an m-output, l-input, state dimension n linear, time-invariant system of the form
written in the operator form
where convolution operator G and signal d are defined by
and T < ∞ is fixed. Note that all functions in the range of G are continuous and differentiable.
Let 0 = t0 < t1 < ⋯ < T M = T (respectively \(0 = {\hat t_0} < {\hat t_1} < \cdots < {\hat T_{\hat M}} = T\)) be M (respectively \(\hat M\)) distinct points in [0, T] and, for any \(f \in L_2^m[0,t]\), consider the linear map f ↦ fe
where F j is f j × m and of full row rank, 1 ≤ j ≤ M. The inclusion of F j is a generalization that enables only selected elements or linear combinations of elements of f that are important in the problem to be specified at t = t j .
The operator P is defined by the functional relations:
where
and either
-
1)
each P j is a p j × m matrix of full row rank (representing the tracking requirement \({P_j}y(t) \equiv r_j^P(t)\) on [t j −1, t j ])
-
2)
or P j = 0 (no tracking requirement on that interval).
They can be regarded as “partial projections” of the signal f on the domain \([{\hat t_{j - 1}},{\hat t_j}]\) representing combinations of f of control interest in these intervals.
With this notation, the “extended output” ye from the plant is defined to be a “partially projected” output signal (P y )(t) plus the values F j y(t j ), 1 ≤ j ≤ M, at intermediate points, i.e., dynamics can be modelled by the form
with \({G_e}:L_2^\ell [0,T] \to {R^{{f_1}}} \times \cdots \times {R^{{f_M}}} \times L_2^{{p_1}}[0,{\hat t_1}] \times \cdots \times L_2^{{p_{\hat M}}}[{\hat t_{\hat M - 1}},{\hat t_{\hat M}}]\) as a linear bounded operator.
A technical note: If G e does not have range dense in \(L_2^{{p_1}}[0,{\hat t_1}] \times \cdots \times L_2^{{p_{\hat M}}}[{\hat t_{\hat M - 1}},{\hat t_{\hat M}}]\), then \(L_2^{{p_1}}[0,{\hat t_1}] \times \cdots \times L_2^{{p_{\hat M}}}[{\hat t_{\hat M - 1}},{\hat t_{\hat M}}]\) is replaced by the closure of the range of G e as the output Hilbert space with the natural inherited topology. This has no effect on the theory which incorporates this easily by using the virtual reference \(r \in L_2^m[0,T]\).
Here, \({G_j}:L_2^\ell [0,T] \to {R^{{f_j}}}\) is defined by the relation
and \({(PG)_j}:L_2^\ell [0,T] \to L_2^{{p_j}}[{\hat t_{j - 1}},{\hat t_j}]\) is defined by
A reference signal re is constructed by specifying the desired values of F j y(t j ), 1 ≤ j ≤ M and the desired signals \(r_j^P(t)\) for \({P_j}y(t),t \in [{\hat t_{j - 1}},{\hat t_j}],1 \le j \le \hat M\). The attainability of this reference by the system requires consistency between these characterizations, and is addressed as follows.
Tracking signal characterization: It is assumed that re can be generated using a (possibly non-unique, but continuous) virtual reference r for the plant output y such that r − d lies in the range of G. In this case, the tracking error ee = re − ye can be associated with the “usual” tracking error e(t)= r(t) − y(t) used in NOILC as
It is useful to note the following:
-
1)
The virtual reference r and the associated virtual error e will play a role in the simplification of the theory and relating the optimization problems used in the algorithm to, more traditional, optimal control problems. Note that neither r nor e are needed in the final computations as they are always multiplied by operations that convert them into components of the extended signals such as P j e that can be computed from the specified signal P j r and the measured plant output.
-
2)
The pure intermediate point problem is obtained by removing P f from the definition of fe whilst NOILC is obtained by removing the intermediate points and looking at the case of \(\hat M = 1,{\hat t_1} = T\) and P = I. In terms of defining fe, this is just (respectively)
$${f^e} = \left[ {\matrix{{{F_1}f({t_1})} \cr\vdots \cr{{F_M}f({t_M})} \cr}} \right] \in {R^{{f_1}}} \times \cdots \times {R^{{f_1}}},\;{f^e} = f \in L_2^m[0,T].$$(9)It is noted that the first is obtained in what follows by setting \({P_j} = 0,1 \le j \le \hat M\) whilst the second is obtained using F j = 0, 1 ≤ j ≤ M and \({\hat Q_j} = Q,1 \le j \le \hat M\) and P j = I.
-
3)
The extension of the problem to the case of all t j < T is achieved by increasing M by one and then defining an additional point at t M = T with the corresponding F M = 0.
-
4)
If there are no tracking requirements on some subinterval \([{\hat t_{q - 1}},{\hat t_q}]\), then simply set P q = 0.
To illustrate the wide range of tracking signals that the construction allows, consider the following example of a reference signal for the case of m = 3 and control over a time interval [0, 12]:
-
1)
The intermediate points are defined by the requirement that output y1(t) passes through the points y1 = 1, 2, 3, 2, 1, 0attimest = 2, 4, 6, 8, 10, 12 respectively.
-
2)
Dynamics of all outputs are constrained during the subinterval [4, 8] ⊂ [0, 12], to satisfy the planar/spatial constraint y1(t)+ y2(t)+ y3(t) ≡ 2.
The resultant data set is t1 = 2, t2 =4, t3 = 6, t4 = 8, t5 = 10, t6 = 12 with M = 6 and F1 = F2 = ⋯ = F6 = [1, 0, 0]. The remaining data sets are \({\hat t_1} = 4,{\hat t_2} = 8,{\hat t_3} = 12\) with \(\hat M = 3\). Hence, P1 = P3 =0 and P2 = [1, 1, 1] with reference signals \(r_1^P(t) \equiv 0,{r^P}(t) \equiv 0\) and \(r_2^P(t) \equiv 2\) on the intervals [0, 4), [4, 8], (8, 12].
2.2 Hilbert space structures and operators
The product space \({R^{{f_1}}} \times \cdots \times {R^{{f_1}}} \times L_2^{{p_1}}[0,{\hat t_1}] \times \cdots \times L_2^{{p_{\hat M}}}[{\hat t_{M - 1}},T]\) is a Hilbert space with inner product
where the f j × f j matrices Q j , 1 ≤ j ≤ M and the p j × p j matrices \({\hat Q_j},1 \le j \le \hat M\) are all symmetric and positive definite. The simple notation [Q] is used to denote the data set \([Q] = \{{Q_1}, \cdots ,{Q_M},{\hat Q_1}, \cdots ,{\hat Q_{\hat M}}\}\).
Using the normal definition of the associated norm, namely \(q = \sqrt {\langle q,q\rangle}\), the squared norm of the error ee, expressed in terms of the “underlying” error e, is simply
Finally, note that
-
1)
\(L_2^\ell [0,T]\) is also a Hilbert space with inner product and norm defined by
$$\matrix{{{{\langle {{u_1},{u_2}} \rangle}_R} = \int_0^{\rm{T}} {u_1^{\rm{T}}(t)R{u_2}(t){\rm{d}}t}} \hfill \cr{\quad \Vert u\Vert_R^2 = \int_0^{\rm{T}} {{u^{\rm{T}}}(t)Ru(t){\rm{d}}t}} \hfill \cr}$$(12)where ℓ × ℓ matrix R is symmetric and positive definite.
-
2)
Equation (11) suggests that any objective function containing the term
$$\sum\limits_{j = 1}^M {{e^{\rm{T}}}({t_j}){H_j}e({t_j}) + \sum\limits_{j = 1}^{\hat M} {\int_{{{\hat t}_{j - 1}}}^{{{\hat t}_j}} {{e^{\rm{T}}}(t)Q(t)e(t){\rm{d}}t}}}$$with Q(t) symmetric, positive semi-definite and piece-wise constant on [0, T] and each H j symmetric and positive semi-definite, can be associated with an ILC problem of the type discussed in this paper by identifying the intervals \(({\hat t_{j - 1}},{\hat t_j}),1 \le j \le \hat M\) where either Q(t)=0or Q(t)is constant with rank p j ≤ m and setting
$$\matrix{{Q(t) = P_j^{\rm{T}}{{\hat Q}_j}{P_j},\;t \in [{{\hat t}_{j - 1}},{{\hat t}_j}],{{\hat Q}_j} = \hat Q_j^{\rm{T}} > 0,\;1 \leq j \leq \hat M} \hfill \cr{\;{H_j} = F_j^{\rm{T}}{Q_j}{F_j},\;\;{Q_j} = Q_j^{\rm{T}} > 0,\;1 \leq j \leq \hat M} \hfill \cr}$$(13)where each F j has full row rank and either P j has full row rank or is zero.
3 Formulation of the ILC problem
In its original form, NOILC for linear, time-invariant, state-space systems is based on the minimization of an objective function
It is clear that the new mixed problem introduced in the previous section can be expressed in this form by identifying G with G e and the norms with the new norms defined. It follows that the iterative procedure defined by constructing u k +1 on iteration k + 1 by minimizing the objective function
subject to the state space dynamics of S(A, B, C) is precisely a NOILC iteration for G e . Note that the criterion is expressed in terms of the virtual error ek+1 rather than the extended error \(e_{k + 1}^e\).
In the next sections, the solution is found in its Hilbert space setting and then formulated as the solution of a two-point boundary value problem with M “jump” conditions.
3.1 Formal solution of the intermediate point algorithm with subinterval weighting
The new algorithm can be written, in the notation of the previous section, as
This has identical structure to the NOILC problem originally proposed in [16], the only differences being the definition of operators, signals and the underlying Hilbert spaces. The iterative solution to (16) is hence written formally as
where \(G_e^*\) is the adjoint operator of G e in the topology introduced in Section 2. This yields the monotonicity and convergence properties summarised in the following theorem. Note that this guarantees monotonic convergence of \(e_k^e{_{[Q]}}\) for all possible choices of symmetric positive definite R, [Q], and furthermore that \(e_k^e{_{[Q]}} \to 0\) implies \(e_k^e \to 0\).
-
Theorem 1. Under the conditions defined above and given an initial choice of input u0, the extended error norm is monotonically decreasing
$$\Vert e_{k + 1}^e\Vert{_{[Q]}}\;\; \leq \;\;\Vert e_k^e\Vert{_{[Q]}},\quad \forall k \geq 0$$(18)with equality holding if and only if u k +1 = u k .As a consequence, there exists a real number E∞ ≤ 0 such that
$$\mathop {\lim}\limits_{k \rightarrow \infty} \Vert e_k^e\Vert{_{[Q]}} = {E_\infty}$$(19)and E∞ = 0 if and only if
$$\mathop {\lim}\limits_{k \rightarrow \infty} e_k^e = 0.$$(20)This condition is satisfied if the extended reference re lies in the closure of the range of G e . Finally,
-
1)
the resultant input sequence satisfies
$$\sum\limits_{k = 1}^\infty {\Vert{u_{k + 1}} - {u_k}\Vert_R^2 < \infty}$$(21) -
2)
if the kernel of \(G_e^*\) is {0}, the initial input u0 = 0 and the signal re − d e lies in the range of \({G_e}G_e^*\), then the input sequence {u k }k≥0 converges to the unique stationary point, u∞, of the Lagrangian associated with the optimization problem
$${u_\infty} = \arg \min \{J(u) = \Vert u\Vert{^2}:{r^e} = {G_e}u + {d_e}\} .$$(22)In other word, it converges to the input achieving the tracking requirement that has the smallest energy as represented by its norm in \({\mathcal U}\).
-
1)
-
Proof. The result follows from the work in [16] apart from the last statement. To show this, note that the Lagrangian associated with the minimum energy problem is just, with Lagrange multiplier \(\lambda \in {{\cal R}_e}\),
$${{\cal L}}(u,\lambda) = \Vert u\Vert_{{\cal U}}^2 + 2 < \lambda ,\quad {r^e} - {G_e}u - {d_e} > {{{\cal R}}_e}$$(23)which has a stationary point when \({u_\infty} = G_e^*\lambda\) and re = G e u∞ + d e , i.e., \({r^e} - {d_e} = {G_e}G_e^*\lambda\). The stationary point solution for λ exists and is unique as is the defined input solution u∞. This solution solves the minimum energy problem follows from the projection theorem in Hilbert space and the fact that the linear variety {u: re = G e u + d e } is non-empty and closed. Next, we write the iterations for the ILC algorithm in the form
$$\matrix{{{u_{k + 1}} = {u_k} + G_e^{\ast}{e_{k + 1}},\quad {e_{k + 1}} = {{(I + {G_e}G_e^{\ast})}^{- 1}}e_k^e,\quad k \geq 0} \hfill \cr{\quad \quad \quad \quad \;\;\mathop {\lim}\limits_{k \rightarrow \infty} e_k^e = 0.} \hfill \cr}$$(24)Using the notation \(X = {(I + {G_e}G_e^*)^{- 1}}\), implies that \({u_k} = G_e^*\sum _{j = 1}^k{X^j}e_0^e\). Writing \({r^e} - {d_e} = e_0^e = {G_e}G_e^*{\lambda _0}\) and noting that \(X{G_e}G_e^* = I - X\) gives
$$\matrix{{{u_k} = G_e^{\ast}\left({\sum\limits_{j = 1}^k {{X^j}e_0^e}} \right) = G_e^{\ast}\left({\sum\limits_{j = 0}^{(k - 1)} {{X^j}(I - X){\lambda _0}}} \right)} \hfill \cr{\quad \quad \quad \quad \quad \quad \quad \;\;G_e^{\ast}(I - {X^k}){\lambda _0}.} \hfill \cr}$$(25)It follows that u k converges in norm to an input û∞ satisfying \({\hat u_\infty} = G_e^*{\lambda _0}\) and re = G e û∞ + d e , i.e., it converges to a stationary point of the Lagrangian which is unique. Clearly û∞ = u∞ and λ0 = λ. □
It is useful to note the following observations:
-
1)
Convergence to the minimum energy controller distinguishes the proposed solution from any solution computed using NOILC and an arbitrarily chosen reference signal r(t) defined on the whole time interval [0, T] but satisfying the intermediate point and partial tracking requirements.
-
2)
If all P j = 0, the output space is finite dimensional and \({G_e}G_e^*\) is a square matrix, self adjoint in the defined topology. The approach then reduces to the intermediate point tracking problem[5]. In addition, G is right-invertible, then the convergence is geometric as
$$\Vert e_{k + 1}^e\Vert{_{[Q]}} \leq {1 \over {1 + {{\underline \sigma}^2}}}\Vert e_k^e\Vert{_{[Q]}},\quad \forall k \geq 0$$(26)where \(\underline {{\sigma ^2}}\) is the smallest eigenvalue of the matrix \({G_e}G_e^*\). In this case, invertibility of \({G_e}G_e^*\). In this case, invertibility of \({G_e}G_e^*\) ensures convergence to zero error.
-
3)
In addition, taking the case of f j = m and F j = I m for example, the choice of Q j is available to the designer to influence the nature and form of the convergence, e.g., choosing Q j = λj−1Q0 for some symmetric positive definite matrix Q0 and some choice of λ ∈ (0, 1) will prioritise the reduction of errors at t j −1 before t j .
-
4)
In a similar manner, choosing different values for \({\hat Q_j},1 \le j \le \hat M\) places different priorities on convergence on the subintervals \([{\hat t_j},{\hat t_{j + 1}})\).
In the discussion that follows, it is assumed that at least one \({\hat Q_j}\) is non-zero.
3.2 The two-point boundary value problem on iteration k + 1:
The solution (17) of NOILC problem (16) requires operator \(G_e^*\), which is computed in the Appendix. Substituting this operator into (17) converts the solution, u k +1, into the form of a two-point boundary value problem (TPBVP) with jump conditions, given by
where
. The next subsection converts this characterization into a feedback/feedforward realization using Riccati techniques.
3.3 A causal algorithm
The formal solution as a boundary value problem can be expressed either as a causal feedforward solution when Q(t)=0[5] or more generally, as a causal feedback plus feedforward solution. The details are described below.
The approach to constructing a state feedback plus predictive/feedforward term causal solution is familiar and closely related to the classical linear quadratic tracking problem. The solution provided below is not unique but all solutions are equivalent in the sense that they generate the same u k +1. The approach is illustrated by writing
where K(t), ξ k +1(t) are assumed to be continuous and differentiable in each open segment (t j , t j +1) but possibly discontinuous at the points t = t j and K(t) is symmetric. The state vector is continuous on [0, T].
A causal feedback solution for u k +1 is constructed as follows:
-
Step 1. Compute off-line the Riccati feedback matrix from the matrix differential equation, terminal condition and jumps
$$\matrix{{Q(t) = P_j^{\rm{T}}{{\hat Q}_j}{P_j},\;\;t \in ({{\hat t}_{j - 1}},{{\hat t}_j}],\quad 1 \leq j \leq \hat M} \hfill \cr{\dot K(t) + {A^{\rm{T}}}K(t) + K(t)A - K(t)B{R^{- 1}}{B^{\rm{T}}}K(t) +} \hfill \cr{\quad {C^{\rm{T}}}Q(t)C = 0} \hfill \cr}$$(29)with
$$\left\{{\matrix{{K(T) = {C^{\rm{T}}}F_M^{\rm{T}}{Q_M}{F_M}C} \hfill \cr{K({t_j} -) = K({t_j} +) + {C^{\rm{T}}}F_j^{\rm{T}}{Q_j}{F_j}C,\quad 1 \leq j \leq M.} \hfill \cr}} \right.$$.
-
Step 2. On the (k + 1)-th iteration, compute off-line the predictive feedforward term from the differential equation, terminal condition and jumps
$$\matrix{{{{\dot \xi}_{k + 1}}(t) + {A^{\rm{T}}}{\xi _{k + 1}}(t) - K(t)B{R^{- 1}}{B^{\rm{T}}}{\xi _{k + 1}}(t) -} \hfill \cr{\quad K(t)B{u_k}(t) + {C^{\rm{T}}}Q(t)r(t) = 0} \hfill \cr{{\xi _{k + 1}}(T) = {C^{\rm{T}}}F_M^{\rm{T}}{Q_M}{F_M}r(T)} \hfill \cr{{\xi _{k + 1}}({t_j} -) = {\xi _{k + 1}}({t_j} +) + {C^{\rm{T}}}F_j^{\rm{T}}{Q_j}{F_j}r({t_j}),1 \leq j \leq M.} \hfill \cr}$$(30)Although these equations contain the virtual reference r(t), note that F j r(t j ), 1 ≤ j ≤ M is specified in the tracking problem and that the product \(Q(t)r(t) = P_j^{\rm{T}}{\hat Q_j}{P_j}r(t),t \in [{t_{j - 1}},{t_j}]\) is known as P j r(t) is specified on [tj−1, t j ].
-
Step 3. Implement the control law
$${u_{k + 1}}(t) = {u_k}(t) + {R^{- 1}}{B^{\rm{T}}}[{\xi _{k + 1}}(t) - K(t){x_{k + 1}}(t)]$$(31)to achieve the predicted monotonic convergence.
3.4 Performance and design observations
Given the tracking objectives, the choice of R,\(R,{\hat Q_j},1 \le j \le \hat M,{Q_j},1 \le j \le M\), and t j , 1 ≤ j < M are open to the designer of the system. As with the choice of Q and R in the traditional linear quadratic regulator (LQR) problem, there are no strict rules for their selection but the following comments can be made.
-
1)
The case when \({\hat Q_j} \ne 0\) is equivalent, intuitively, to a requirement that the output tracks reference P j r(t) on subintervals of [0, T]. Changing the relative values of \({\hat Q_j}\) will change the relative convergence rates on the intervals with the general rule that, increasing \({\hat Q_q}\) will improve convergence rates on the subinterval \([{\hat t_{q - 1}},{\hat t_q}]\). Also decreasing the values of the \({\hat Q_j}\) relative to the Q j will place increasing emphasis on the specific intermediate points at times t j , 1 ≤ j ≤ M leading to an expected faster reduction of errors at the intermediate points and slower convergence elsewhere. Details will depend upon plant dynamics and the choice of R.
-
2)
In general terms, reducing the value of R is expected, intuitively to tend to accelerate algorithm convergence.
4 Experimental results
The approach has been tested on a six degree of freedom anthropomorphic robotic arm (see Fig. 1) whose five rotary joints are composed of PowerCubes (Schunk GmbH & Co.) incorporating brushless servomotors with integrated power electronics and transmission. These communicate with a dSPACE ds1103 control board via a CAN bus at a rate of 500 kbit/s, and each is controlled in torque mode. The task considered involves planar movement using only two actuated joints. Starting from an initial location, the end-effector moves to a specified location (e.g., to pick up a tool head), moving along a path (replicating a cutting or welding action), returning to another specified location (e.g., to replace/clean a tool head), and returning to the initial location.
The plant model has been identified using frequency response tests and is given by transfer function matrix (32). Cascaded current and velocity control loops are used, with inputs and outputs measured in radians.
The task requires the output to pass through selected points at specified times plus the additional requirement that a linear combination of outputs takes a specified functional value on a proper subinterval. More precisely, zero initial conditions are assumed for each iteration and tracking is represented by using intermediate point locations, together with an output constraint that enforces a path following component on a subinterval. The path following component does not specify output values but does specify their functional relationship.
The intermediate output values are given by the M =5 time points
with the corresponding outputs
where F j = I, 1 ≤ j ≤ 5. In addition, tracking is also defined by the constraint in the interval [4, 10].
representing a coordinated spatial movement of the two joints in terms of relative values. Note that this does not specify numerical values of y1(t) or y2(t) individually at any point in [4, 10]. To achieve the task, the control action (31) has been implemented using the state feedback matrix (29) and predictive feedforward term (30). As the states are not measured directly, a Kalman filter provides the state estimates using process and measurement disturbance co-variance weights of 10I and I respectively.
.
To provide a comparative approach to quantify convergence rates, the uniquely defined iteration index, k*, corresponding to the first index where the tracking criterion
is met is noted. A value of ε = 0.01 is used in all the results that follow corresponding to an improvement in tracking by a factor of 100 relative to the extended reference.
Fig. 2 shows the converged paths after 200 trials using R = I, Q j = 50I, 1 ≤ j ≤ 5 and \({\hat Q}_{j} = 2, 1 \leq j \leq 3\). Corresponding output angles and control input signals are shown in Fig. 3, together with signals on trials 1, 3 and 5. Norm results over all trials are shown in Fig. 4 for a variety of Q j and \({\hat Q_j}\).
Table 1 shows summary data, confirming that the both the intermediate point tracking and the spatial tracking components are achieved to high accuracy. As Q j increases more emphasis is placed on minimizing the intermediate point error, compared to the spatial tracking error. In practice, an optimum compromise can be found between Q j and \({\hat Q_j}\) values which provides convergence to low values of each objective. At the same time, there is clearly a trade-off between rapid convergence to low error, and the control effort required to achieve it. Lower values of Q j and \({\hat Q_j}\) typically enable less energy to be expended, but k* increases.
The combined intermediate point and spatial tracking approach is now compared to the standard ILC framework. As is shown in Theorem 1, in the absence of plant model mismatch this method converges to the minimum energy solution to the problem, and hence the most appropriate comparison is to use standard NOILC with r(t) calculated in advance using the nominal plant model to achieve the intermediate point and spatial tracking tasks whilst also requiring minimum input energy. This signal is shown in Fig. 3 where it is labeled \(\tilde r(t)\). Using this predefined reference, standard NOILC corresponds to removing the intermediate points and using \(\hat M = 1,{\hat t_1} = T,P = I\).
Results in Table 2 show that standard NOILC requires a significantly larger control effort norm to obtain similar levels of error, while requiring a greater number of trials.
5 Conclusions
A well-defined NOILC problem which places specific emphasis on two objectives, namely, simultaneous fast accurate tracking at a finite number of intermediate points of time within the interval [0, T] plus tracking of defined linear combinations of outputs on sub-intervals of [0, T], can be approached and solved using the operator methods underpinning those of NOILC in [16]. The resultant algorithm inherits the original convergence properties and monotonicity properties that makes NOILC so attractive. The formulation is quite general for linear time invariant state space systems and can be represented in a feedforward structure (when Q(t) ≡ 0), but more generally, has the form of familiar current trial Riccati state feedback and predictive/feedforward terms generated from off-line, reverse-time simulations of differential equations with defined jump conditions at the intermediate points selected. An analysis of the effect of the choice of weighting matrices provides guidelines to their selection in terms of convergence rates and the effect of removing intermediate points from the problem.
Experimental results indicate that the predictions of error monotonicity and rapid convergence can apply in practice with great accuracy and with robustness to modelling errors. Further research is needed to address more general issues of parameter choice, a formal characterization of the algorithm robustness to modelling errors and the effects of varying initial condition.
References
S. Arimoto, F. Miyazaki, S. Kawamura. Bettering operation of robots by learning. Journal of Robotic Systems, vol.1, no. 2, pp. 123–140, 1984.
D. A. Bristow, M. Tharayil, A. G. Alleyne. A survey of iterative learning control: A learning-based method for highperformance tracking control. IEEE Control Systems Magazine, vol. 26, no. 3, pp. 96–114, 2006.
H. S. Ahn, Y. Chen, K. L. Moore. Iterative learning control: Brief survey and categorization. IEEE Transactions on Systems, Man, and Cybernetics, Part C, vol. 37, no. 6, 1099–1121, 2007.
D. H. Owens, C. T. Freeman, B. Chu. Multivariable norm optimal iterative learning control with auxiliary optimization. International Journal of Control, vol. 86, no. 6, pp. 1026–1045, 2013.
D. H. Owens, C. T. Freeman, T. Van Dinh. Norm optimal iterative learning control with intermediate point weighting: Theory, algorithms, and experimental evaluation. IEEE Transactions on Control Systems Technology, vol. 21, no. 3, pp. 999–1007, 2013.
G. Pipeleers, J. Swevers. A data-driven constrained normoptimal iterative learning control framework for LTI systems. IEEE Transactions on Control Systems Technology, vol. 21, no. 2, pp. 546–551, 2013.
T. D. Son, H. S. Ahn, K. L. Moore. Iterative learning control in optimal tracking problems with specified data points. Automatica, vol. 49, no. 5, pp. 1465–1472, 2013.
S. H. Zhou, Y. Tan, D. Oetomo, C. T. Freeman, E. Burdet, I. Mareels. Point-to-point learning in human motor systems. In Proceedings of the American Control Conference, IEEE, Washington, USA, pp. 5923–5928, 2013.
C. T. Freeman, T. Exell, K. L. Meadmore, E. Hallewell, A. M. Hughes. Computational models of upper-limb motion during functional reaching tasks for application in FESbased stroke rehabilitation. Biomedical Engineering, to be published.
K. L. Moore, M. Ghosh, Y. Q. Chen. Spatial-based iterative learning control for motion control applications. Meccanica, vol. 42, no. 2, pp. 167–175, 2007.
S. K. Sahoo, S. K. Panda, J. X. Xu. Application of spatial iterative learning control for direct torque control of switched reluctance motor drive. In Proceedings of IEEE Power Engineering Society General Meeting, IEEE, Tampa, USA, pp. 1–7, 2007.
Y. H. Yang, C. L. Chen. Spatial-based adaptive iterative learning control of nonlinear rotary systems with spatially periodic parametric variation. International Journal of Innovative Computing, Information and Control, vol. 7, no. 6, pp. 3407–3417, 2011.
K. Furuta, M. Yamakita. The design of a learning control system for multivariable systems. In Proceedings of IEEE International Symposium on Intelligent Control, IEEE, Philadelphia, USA, pp. 371–376, 1987.
K. Kinosita, T. Sogo, N. Adachi. Iterative learning control using adjoint systems and stable inversion. Asian Journal of Control, vol. 4, no. 1, pp. 60–67, 2002.
D. H. Owens, J. J. Hatonen, S. Daley. Robust monotone gradient-based discrete-time iterative learning control. International Journal of Robust and Nonlinear Control, vol. 19, no. 6, pp. 634–661, 2009.
N. Amann, D. H. Owens, E. Rogers. Iterative learning control using optimal feedback and feed-forward actions. International Journal of Control, vol. 65, no. 2, pp. 277–293, 1996.
S. Gunnarsson, M. Norrlöf. On the design of ILC algorithms using optimization. Automatica, vol. 37, no. 12, pp. 2011–2016, 2001.
J. H. Lee, K. S. Lee, W. C. Kim. Model-based iterative learning control with a quadratic criterion for timevarying linear systems. Automatica, vol. 36, no. 5, pp. 641–657, 2000.
K. L. Barton, A. G. Alleyne. A norm optimal approach to time-varying ILC with application to a multi-axis robotic testbed. IEEE Transactions on Control Systems Technology, vol. 19, no. 1, pp. 166–180, 2011.
E. Rogers, D. H. Owens, H. Werner, C. T. Freeman, P. L. Lewin, S. Kichhoff, C. Schmidt, G. Lichtenberg. Norm optimal iterative learning control with application to problems in accelerator based free electron lasers and rehabilitation robotics. European Journal of Control, vol. 16, no. 5, pp. 497–524, 2010.
N. Amann, D. H. Owens, E. Rogers. Iterative learning control for discrete-time systems with exponential rate of convergence. IEE Proceedings of Control Theory and Applications, vol. 143, no. 2, 217–224, 1996.
N. Amann, D. H. Owens, E. Rogers. Predictive optimal iterative learning control. International Journal of Control, vol. 69, no. 2, pp. 203–226, 1998.
B. Chu, D. H. Owens. Accelerated norm-optimal iterative learning control algorithms using successive projection. International Journal of Control, vol. 82, no. 8, pp. 1469–1484, 2009.
B. Chu, D. H. Owens. Iterative learning control for constrained linear systems. International Journal of Control, vol. 83, no. 7, pp. 1397–1413, 2010.
D. H. Owens, B. Chu, E. Rogers, C. T. Freeman, and P. L. Lewin. Influence of nonminimum phase zeros on the performance of optimal continuous-time iterative learning control. IEEE Transactions on Control Systems Technology, vol. 22, no. 3, pp. 1151–1158, 2014.
D. H. Owens, B. Chu. Modelling of non-minimum phase effects in discrete-time norm optimal iterative learning control. International Journal of Control, vol. 83, no. 10, pp. 2012–2027, 2010.
Author information
Authors and Affiliations
Corresponding author
Additional information
Recommended by Guest Editor Rong-Hu Chi
Appendix (Computation of the adjoint operator \(G_e^*\))
Appendix (Computation of the adjoint operator \(G_e^*\))
At the centre of NOILC is the characterization[6] of the adjoint operator mapping inputs into defined outputs. Therefore, purpose of this section, is to compute the relevant adjoint operator \(G_e^*\) of G e given the Hilbert space topology defined above. By definition,
then the adjoint is computed from adjoints of G j , 1 ≤ j ≤ M and (PG).
-
1)
Adjoint operator of G j : First look at G j via
$$\matrix{{w_j^{\rm{T}}{Q_j}{F_j}\int_0^{{t_j}} {C{{\rm{e}}^{A({t_j} - t)}}Bu(t){\rm{d}}t =}} \hfill \cr{\quad \;\int_0^{{t_j}} {{{({R^{- 1}}{B^{\rm{T}}}{{\rm{e}}^{{A^{\rm{T}}}({t_j} - t)}}{C^{\rm{T}}}F_j^{\rm{T}}{Q_j}{w_j})}^{\rm{T}}}Ru(t){\rm{d}}t}} \hfill \cr}$$(36)and hence deduce that
$$\left( {G_j^* w_j } \right)\left( t \right) = \left\{ {\begin{array}{*{20}c} {R^{ - 1} B^T e^{A^{T(_{t_j - t)} } } C^T F_j^T Q_j w_j ,} & {0 \leqslant t \leqslant t_j } \\ {0,} & {t > t_j } \\ \end{array} } \right.$$(37)which can be written in the form
$$(G_e^{\ast}{w_j})(t) = {R^{- 1}}{B^{\rm{T}}}{p_j}(t)$$(38)where on [0, t j ),
$$\matrix{{{{\dot p}_j}(t) = - {A^{\rm{T}}}{p_j}(t)} \hfill \cr{{p_j}({t_j} -) = {C^{\rm{T}}}F_j^{\rm{T}}{Q_j}{w_j}} \hfill \cr}$$(39)and p j (t) = 0 on (t j , T].
-
2)
Adjoint operator of PG: Suppose the Hilbert topology in the range of G is defined (with Q arbitrary as it plays no role in the final algorithm) by the inner product
$$\matrix{{\langle {y,w} \rangle = \int_0^{\rm{T}} {{y^{\rm{T}}}(t)Qw(t){\rm{d}}t}} \hfill \cr{Q = {Q^{\rm{T}}} > 0.} \hfill \cr}$$(40)Firstly, the adjoint of G is computed as the map w = G*v as
$$\matrix{{w(t) = {R^{- 1}}{B^{\rm{T}}}{p_{M + 1}}(t)} \hfill \cr{{{\dot p}_{M + 1}}(t) = - {A^{\rm{T}}}{p_{M + 1}}(t) - {C^{\rm{T}}}Qv(t)} \hfill \cr{{p_{M + 1}}(T) = 0.} \hfill \cr}$$(41)Note now that (PG) j is simply the composite map \(G:L_2^\ell [0,T] \to L_2^m[0,T]\) and the map
$$y(t) \rightarrow {P_j}y(t),\quad t \in [{t_{j - 1}},{t_j}]$$(42)which has adjoint
$$v_j (t)\left[ {\begin{array}{*{20}c} {Q^{ - 1} P_j^T \hat Q_j v_j (t),t \in \left[ {\hat t_{j - 1} ,\hat t_j } \right]} \\ {0, otherwise} \\ \end{array} } \right] \in L_2^m [0,T].$$(43)It follows directly that the adjoint operator P* is defined by the relation \({P^*}({v_1}, \cdots ,{v_{\hat M}}) = \hat w\), where ŵ can be identified almost everywhere with the function
$$\hat w(t) = {Q^{- 1}}P_j^{\rm{T}}{\hat Q_j}{v_j}(t),t \in [{\hat t_{j - 1}},{\hat t_j}),\;\;t \in [0,T].$$(44)Hence, (PG)* = G*P* is defined by a relation of the form \(w = {(PG)^*}({v_1}, \cdots ,{v_{\hat M}})\) as the continuous solution of the costate equation
$$\matrix{{w(t) = {R^{- 1}}{B^{\rm{T}}}{p_{M + 1}}(t)} \hfill \cr{{{\dot p}_{M + 1}}(t) = - {A^{\rm{T}}}{p_{M + 1}}(t) - {C^{\rm{T}}}P_j^{\rm{T}}{{\hat Q}_j}{v_j}(t),\;\;t \in [{{\hat t}_{j - 1}},{{\hat t}_j})} \hfill \cr{{p_{M + 1}}(T) = 0.} \hfill \cr}$$(45) -
3)
Adjoint operator of G e : Using these representations, the adjoint operator \(G_e^*\) of G e is the map \(({w_1}, \cdots ,{w_M},{v_1}, \cdots ,{v_{\hat M}}) \mapsto u\) defined by
$$\matrix{{u(t) = \sum\limits_{j = 1}^M {(G_j^{\ast}{w_j})(t) + ({G^{\ast}}{P^{\ast}}({v_1}, \cdots ,{v_{\hat M}}))(t) =}} \hfill \cr{\quad {R^{- 1}}{B^{\rm{T}}}\sum\limits_{j = 1}^{M + 1} {{p_j}(t) = {R^{- 1}}{B^{\rm{T}}}p(t)}} \hfill \cr}$$(46)where
$$p(t) = \sum\limits_{j = 1}^{M + 1} {{p_j}(t) \in {{\bf{R}}^n}.}$$(47)
Using linearity, these can be brought together as
This is a familiar equation similar to the costate equations of optimal control. It is computed by backwards integration from the terminal boundary condition at t = T with defined discontinuities (or “jumps”) at times t = t j , 1 ≤ j ≤ M and the specified terminal boundary condition. Note that
-
1)
The case created by putting all P j = 0 can be included in the above as it is equivalent to omitting the term in the inner products and norms and redefining the extended output ye to omit the term Py(·). The output space is then \({R^{f1}} \times , \cdots ,{R^{fM}}\) which is finite dimensional of dimension \([f] = \sum _{j = 1}^M{f_j}\). The input space remains infinite dimensional so G e is not invertible.
-
2)
It is easily seen that the above theory also works with time varying A, B, C, \({\hat Q_j}\), R simply by replacing these matrices by time varying versions. The details are omitted for brevity.
Substituting the operator form (48) into (17) leads directly to the TPBVP of (27).
Rights and permissions
About this article
Cite this article
Owens, D.H., Freeman, C.T. & Chu, B. Generalized norm optimal iterative learning control with intermediate point and sub-interval tracking. Int. J. Autom. Comput. 12, 243–253 (2015). https://doi.org/10.1007/s11633-015-0888-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11633-015-0888-8