An Adaptive Suboptimal Control Algorithm with Variable Design Parameter for Systems with Stochastic Parameters

Abstract

A suboptimal dual controller is presented for a class of linear systems with unknown and randomly varying parameters. This work concerns a refinement of an existent suboptimal dual controller for discrete-time systems. The loss function that is minimized contains two parts. The first part is the variance of the output one step ahead of time, and the second part is a function of the covariance matrix of the parameter estimates. The idea is to add simple terms depending on the covariance matrix of the parameter estimates two steps ahead. An algorithm is used for the adaptive adjustment of the adjustable parameter lambda, for each step of the way. The behavior of the proposed controller is evaluated through a Monte Carlo simulations method by three examples.

1 Introduction

It is an interesting and desirable challenge to control systems with unknown and time-varying parameters. The uncertainty of the estimates may be incorporated in different ways; in this paper, we discuss adaptive control algorithms with dual features. The optimal controller for this problem has two objectives that must balance two conflicting purposes: control and estimation. Starting from the unprecedented work presented by Feldbaum (1960–1961), several surveys were stimulated with the purpose of investigating and clarifying the dual control problem (Åström and Wittenmark 2013; Filatov and Unbehauen 2000; Lindoff et al. 1999).

Due to the computational and numerical intractability of the optimal solution to the dual control problem, many researchers resorted to suboptimal solutions in which the optimization problem is redesigned, to include cautious and probing features in simple controllers. Wittenmark (1975) proposed an active suboptimal dual (ASOD) controller, which uses the objective function one step ahead, including the covariance matrix of the estimated parameters. Next, Wittenmark and Elevitch (1985) presented an analytic solution by making a series expansion of the loss function, with respect to the control signal. Hughes and Jacobs (1974) proposed appending an additional excitation to the control signal through a defined minimum value. The variance of the innovation process, through unknown but constant coefficients, is used in Milito et al. (1982) to optimize the system performance. Ishihara et al. (1988) extended the innovation process to suit models with input delays and stochastic coefficients containing white-noise components.

Maitelli and Yoneyama (1999) proposed a novel suboptimal dual controller, using optimal predictions of the output. This controller minimizes the output deviations during M steps ahead in time. Lee and Lee (2009) applied an approximate dynamic programming (ADP)-based strategy to the dual adaptive control problem. This ADP approach could derive a superior control policy, which actively reduces the parameter uncertainty, leading to a significant performance improvement. Flidr and Simandl (2013) show a new implicit dual control method based on Bellman optimization. The stochastic integration rule is employed to determine the control. Kral and Simandl (2011) proposed and discussed a predictive dual control for a nonlinear system with functional uncertainty based on the bicriterial approach.

While several suboptimal dual control solutions were proposed and simulated in recent decades, the application in industrial processes is still restricted. Bar-Shalom and Wall (1980) presented a method for the quantitative assessment of the effects of uncertainty in macroeconomic policy optimization problems. Ismail et al. (2003) successfully implemented a suboptimal dual controller in the paper-coating industry. The article shows the advantages of including probing in the control signal and the improvement of the quality of the process. The dual controller yielded substantial quality improvement and was able to control the process throughout the entire blade life.

In this work, a modification in the ASOD controller is presented. Originally, there is a fixed design parameter $\lambda$ that is associated with the function $f(P(k+2))$, which is intended to ensure a good parameter estimation. Therefore, it is proposed to use an adaptive adjustment of parameter $\lambda$ at every step, in order to improve the performance and robustness of the controller.

2 Problem Formulation

The dynamic system to be controlled is assumed to be a time-varying single-input/single-output ARX (AutoRegressive with eXogenous input) model with stochastic parameters. Consider the discrete-time single-input/single-output system described by

$$\begin{aligned}&y(k) + a_{1}(k)y(k-1) + \cdots + a_{m}(k)y(k-m)\nonumber \\&\quad =\,b_{1}(k)u(k-1)+\cdots +b_{n}(k)u(k-n) + e(k) \end{aligned}$$

(1)

where $y(k)$ is the output, $u(k)$ is the input, and $e(k)$ is a sequence of independent, identically distributed Gaussian variables, with zero mean and variance $\sigma ^2$. It is also supposed that $b_1(k)\ne 0$ and $e(k)$ is independent of ${y(j), u(j), }$ ${a_i (j), b_i (j), j\le k}$ for each instant $k$.

The time-varying parameters

$$\begin{aligned} x(k)=[\;a_{1}(k)\;\cdots \; a_{m}(k)\;\vert \; b_{1}(k)\;\cdots \; b_{n}(k)]^{T} \end{aligned}$$

(2)

are modeled by a Gauss–Markov process, which satisfies the stochastic difference equation

$$\begin{aligned} x(k+1)=\varPhi x(k)+v(k) \end{aligned}$$

(3)

where $\varPhi$ is a known $(m+n)*(m+n)$ stable matrix and $v(k)$ is a sequence of independent, identically distributed Gaussian random variables, with zero mean and variance matrix $R_{v}$. Moreover, it is assumed that $e(k)$ is independent of $v(k)$ and $x(0)$.

Defining the row vector

$$\begin{aligned} \theta (k)= & {} [-y(k-1)\quad -y(k-2)\quad \cdots \quad -y(k-m)\;\vert \nonumber \\&\;u(k-1)\quad u(k-2)\quad \cdots \quad u(k-n)] \end{aligned}$$

(4)

Eq. (1) can be written as

$$\begin{aligned} y(k)=\theta (k)x(k)+e(k) \end{aligned}$$

(5)

Thus, the model is defined in the compact form by (3) and (5).

The purpose of the control action is to keep the system output as close as possible around a reference sequence value, $y_{r}(k+1)$, during $H$ steps of control. The deviation is measured by the criterion

$$\begin{aligned} V_{H}(u)=E\left\{ \frac{1}{H}\sum _{i=1}^{H}\left( y(k+i)-y_{r}(k+1)\right) ^2\right\} \end{aligned}$$

(6)

where $u=(u_{k+H-1,\cdots ,u_k})^{T}$ represents the actual and future control signals and the expected value $E$ refers to all underlying random variables. The controller design problem is formulated by selecting the input sequence $u(i), i=k, \cdots , k+H-1$ that minimizes the cost function (6), subject to (3), (5) and by a causality constraint. At each instant $k$, the control signal $u(k)$ can only depend on the initial information, and on past inputs and outputs, i.e.,

$$\begin{aligned} \varUpsilon _{k}= & {} \left\{ y(k),y(k-1),\cdots , y(0),u(k-1), u(k-2), \cdots , u(0)\right\} \nonumber \\ \end{aligned}$$

(7)

In principle, the optimal solutions could be determined by direct application of dynamic programming (Kumar and Varaiya 1986). In many practical applications, however, the mathematical complexity makes the optimal solution difficult to obtain and the prohibitive amount of storage and computational effort required may preclude the use of numerical methods to solve the problem. One way of overcoming this difficulty is to search for suboptimal solutions. In this context, this work proposed a change in ASOD loss function (Wittenmark 1975) in order to achieve a better controller.

In order to control the system (1), it is necessary to make a real-time estimation of the parameters. The estimates of the unknown parameters are thus given by the standard Kalman filter equations that can be applied to (3) and (5). It can be seen that the conditional distribution of $x(k+1)$, given $\varUpsilon _{k}$, is Gaussian with mean $\hat{x}(k+1)$ and covariance $P(k+1)$, where $\hat{x}$ and $P$ satisfy the difference equations

$$\begin{aligned}&\hat{x}(k+1) = \varPhi \hat{x}(k)+K(k)[y(k)-\theta (k)\hat{x}(k)]\end{aligned}$$

(8)

$$\begin{aligned}&K(k) = \varPhi P(k)\theta ^T(k)[\theta (k)P(k)\theta ^T(k) +\sigma ^2]^{-1}\end{aligned}$$

(9)

$$\begin{aligned}&P(k+1) = [\varPhi - K(k)\theta (k)]P(k)\varPhi ^T + R_{v} \end{aligned}$$

(10)

with initial conditions $P(0)$ and $\hat{x}(0)$.

3 ASOD Controller

In order to acquire a good system performance (5), Wittenmark (1975) proposed an ASOD controller, minimizing the output variance one step ahead and adding a term to the loss function in the criterion (6), which reflects the quality of the estimates. By attaching simple terms, this will prevent the cautious controller from turning off the control. It is important to add simple terms to make it easy to numerically be able to find the resulting controller.

$$\begin{aligned} V_\mathrm{{ASOD}}(u_{k})= & {} E\lbrace (y(k+1)-y_{r}(k+1))^2\nonumber \\&+\,\lambda f(P(k+2)\vert \varUpsilon _k\rbrace \end{aligned}$$

(11)

The minimization of (11) is done with respect to $u(k)$. $P(k+2)$ is the variance of the errors of the parameter estimates and is the first time at which the covariance is influenced by $u(k)$. The function $f(.)$ is assumed to be positive, increasing monotonically, and twice continuously differentiable. Using (11), the control signal must guarantee a good trade-off between control and estimation. The second term in (11) can be regarded as a constraint which is added to the loss function.

In general, it is not possible to minimize (11) analytically, since $P(k+2)$ is a nonlinear function of $u(k)$. The solution must be obtained through an iterative procedure. Alternatively, Wittenmark and Elevitch (1985) proposed a further approximation by making a series expansion of (11) around a nominal control and keeping first- and second-order terms to obtain an analytical expression for the control signal.

The second term in equation (11) also has a parameter design $\lambda$ which needs to be defined in advance and remains fixed throughout the steps.

4 Innovations Dual Controller

An innovations approach to dual control was presented in Milito et al. (1982) to optimize the performance of the system. The cost function contains two parts with the first reflecting regulation of the output and second reflecting the need to gather as much information as possible about the parameters of the system.

The innovations dual controller is defined as the controller that minimizes

$$\begin{aligned} V_\mathrm{{IDC}}(u_{k})= & {} E\lbrace (y(k+1)-y_{r}(k+1))^2\vert \varUpsilon _k\nonumber \\&-\,\lambda (k+1)\nu ^2(k+1)\rbrace \end{aligned}$$

(12)

where $\lambda (k+1)>0$ and

$$\begin{aligned} \nu (k+1)=y(k+1)- \varPhi ^T(k)\hat{x}(k) \end{aligned}$$

(13)

The innovations controller is very easy to implement, and its performance is directly related to a $\lambda$ factor properly calibrated.

5 Two-Stage Controller with Prediction

In Maitelli and Yoneyama (1994), an approximate two-step adaptive predictive controller was derived. This two steps ahead dual suboptimal controller tries to minimize the variance of the output, around a reference value, considering simultaneously one and two prediction steps ahead. The control signal, in this case, requires the estimates of the future system output, and hence, the evaluation of third- and fourth-order statistical moments. Some simplifications were introduced to replace these moments by approximations using an optimal predictor for the value of the future outputs.

For $N=2$, (6) can be written as

$$\begin{aligned} V_\mathrm{{modAPC}}(u_{k})= & {} E\lbrace (y(k+1)-y_{r}(k+1))^2\vert \varUpsilon _k\nonumber \\&+\,\lbrace (y(k+2)-y_{r}(k+1))^2\vert \varUpsilon _k\rbrace \end{aligned}$$

(14)

The main idea is to replace the future output $y(k+1)$ by its optimal prediction.

$$\begin{aligned} y^p(k+1) = E\lbrace (y(k+1))\vert \varUpsilon _k \end{aligned}$$

(15)

Using (3) and (5), we have

$$\begin{aligned} y^p(k+1) = \theta (k+1)\hat{x}(k+1) \end{aligned}$$

(16)

The loss function (14) will be minimized with respect to $u(k)$ and $u(k+1)$. The first term in (14) can be calculated by invoking standard formulas and equations. The second term presents some difficulties because it involves future outputs and some approximations. These demonstrations are found in Maitelli and Yoneyama (1999).

6 ASOD Controller with Variable Design Parameter

In this section, the change in the ASOD controller proposed by Wittenmark (1975) will be presented. In order to get a good performance of the system, we want to minimize the output variance one step ahead and at same time ensure good estimates. If the estimates are good, then the control law will result in a good accomplishment. If the estimates are deficient, then the controller needs to make an effort to get better estimates of the parameters to improve the control in future steps.

The intended solution is to obtain a weight variable factor lambda ($\lambda$) that minimizes the loss function (11). A variable factor $\lambda (k)$ will allow to reward the estimation of the parameters in accordance with the progress of the simulation. The modification proposed attempts to improve the occurrence of poor estimates throughout the simulation due to stochastic parameters. The control signal must compromises between a good control and a good estimation.

The loss function (11) is changed to

$$\begin{aligned} V^\mathrm{{GV}}_\mathrm{{ASOD}}(u_{k})= & {} E\lbrace (y(k+1)-y_{r}(k+1))^2\nonumber \\&+\,\lambda (k) f(P(k+2)\vert \varUpsilon _k\rbrace \end{aligned}$$

(17)

where $\lambda (k)$ represents how much the estimated parameter accuracy is important compared to the control action at each step. An appropriate choice of a fixed $\lambda$ value can involve extensive system simulations. A weight variable factor is necessary, owing to stochastic variables in the system. Once the parameters and the noise interfere randomly at each step in the system, the weight factor $\lambda (k)$ must also keep up with these changes, to conduct better control and estimation.

The weight factor $\lambda (k)$ definition is performed using an algorithm that takes into account the following information: the estimated vector $\hat{x}(k+1)$, the Kalman gain vector $K(k+1)$, the noise parameters vector $v(k)$, and the certainty equivalence control $u_\mathrm{{CE}}(k)$

$$\begin{aligned} \lambda (k) \propto \left\{ \begin{array}{ll} \;\hat{x}(k+1) \\ K(k+1) \\ \;\;\;\;R_v(k)\\ \;\;u_\mathrm{{CE}}(k) \end{array} \right. \end{aligned}$$

where

$$\begin{aligned} K(k+1)= & {} \varPhi P(k+1)\theta ^T(k+1)[\theta (k+1)\nonumber \\&P(k+1)\theta ^T(k+1)+\sigma ^2]^{-1}\end{aligned}$$

(18)

$$\begin{aligned} u_\mathrm{{CE}}(k)= & {} \frac{y_{r}(k+1)-\tilde{\theta }(k)\hat{x}(k+1)}{\hat{b_{1}}(k+1)} \end{aligned}$$

(19)

and

$$\begin{aligned} \tilde{\theta }(k)= & {} [-y(k-1)\quad \cdots \quad -y(k-m) \vert \nonumber \\&0\quad u(k-2)\quad \cdots \quad u(k-n)]\nonumber \\ \hat{b_{1}}(k)\ne & {} 0 \end{aligned}$$

(20)

The algorithm 1 presents the sequence of actions made to set the $u(k)$ value at each step. The most important points in the algorithm are the ways to find out which parameter $\lambda _i(k)$ provides the best control signal $u_i(k)$ at each step. The control signal selection is detailed below.

Defining the weight factor vector, we have

$$\begin{aligned} \varphi _\lambda (k)=[\lambda _1(k)\quad \lambda _2(k) \quad \cdots \quad \lambda _i(k)] \end{aligned}$$

(21)

Using (21), an input vector $U_{\lambda }$ is calculated. The calculating procedure is performed as described in Wittenmark and Elevitch (1985).

$$\begin{aligned} U_{\lambda }=[u_1(k)\quad u_2(k)\quad \cdots \quad u_i(k)] \end{aligned}$$

(22)

The choice of $u_i(k)$ influences directly the actual loss function (17), as well as the future cost, since $u_i(k)$ influences $\theta (k+1)$, which also influences $\hat{x}(k+2)$, $P(k+2)$, and $\tilde{\theta }(k+2)$ (Åström and Wittenmark 1971).

The $u_i(k)$ selection starts by examining $\hat{x}(k+1)$. From the stochastic process (5), we know that the conditional distribution of $y(k)\vert \varUpsilon _{(k-1)}$ is normal, with mean value given by

$$\begin{aligned} E[y(k)\vert \varUpsilon _{(k-1)}]=\theta (k-1)\hat{x}(k) \end{aligned}$$

(23)

and variance

$$\begin{aligned} \sigma ^2_y(k) = \theta (k-1)P(k)\theta ^T(k-1)+\sigma ^2 \end{aligned}$$

(24)

Moreover, we know that the conditional distribution of $\hat{x}(k+1)\vert \varUpsilon _{(k-1)}$ is normal, with mean value given by

$$\begin{aligned} E[\hat{x}(k+1)\vert \varUpsilon _{(k-1)}]=\varPhi \hat{x}(k) \end{aligned}$$

(25)

and covariance

$$\begin{aligned}&{\hbox {cov}}[\hat{x}(k+1),\hat{x}(k+1)\vert \varUpsilon _{(k-1)}]\nonumber \\&\quad =\,K(k)[\sigma ^2+\theta (k)P(k)\theta ^T(k)]K^T(k) \end{aligned}$$

(26)

Åström and Wittenmark (1971) reported that the minimum value for the quadratic criterion in time-varying systems with known parameters is

$$\begin{aligned} \mathrm{{min}}\sum ^H_{i=1}[y(k+i)-y_r]^2 = H\,\sigma ^2 \end{aligned}$$

(27)

The control law obtained by minimizing (11) with respect to $u(k)$ is called the one-stage control law. Thus, the minimum expected value in (27) is $\sigma$. So when the parameter vector estimate value is lower than (27), it is not possible to ensure that the estimated calculation represents a good or a bad estimation. It is predicted that the estimated vector $\hat{x}(k+1)$ cannot be sensitive to changes in parameters when

$$\begin{aligned} \left| -\frac{\sigma ^2_v}{2}\right| \le \hat{x}(k+1)\le \left| \frac{\sigma ^2_v}{2}\right| \end{aligned}$$

(28)

Thus, the selection mode of $u_i(k)$ is directly linked to the condition above. If the $\hat{x}(k+1)$ value is between the condition limits (28), the suitable $u_i(k)$ selection is done based on the Kalman gain (8). If the $\hat{x}(k+1)$ value is not between the condition limits (28), the appropriate $u_i(k)$ selection is performed, taking into account the certainty equivalence control signal (19).

Under definitions (8) and (9), it is seen that if the estimated vector meets the condition (28), then the chosen control signal $u_i(k)$ will be the one to apply a higher gain in the future estimated vector (step forward).

Introducing the gain vector

$$\begin{aligned} K_\lambda (k+1)=[K_1(k+1)\;K_2(k+1) \cdots \;K_i(k+1)] \end{aligned}$$

(29)

where

$$\begin{aligned} K_i(k+1)=\frac{\varPhi P(k+1)\theta ^T(k+1)}{\sigma ^2_y(k+1)} \end{aligned}$$

(30)

and

$$\begin{aligned} \sigma ^2_y(k+1)= & {} \theta (k+1)P(k+1)\theta ^T(k+1)+\sigma ^2 \end{aligned}$$

(31)

It selects the largest absolute value of the gain vector (29),

$$\begin{aligned} K_\mathrm{{max}}(k+1)={\hbox {max}}\vert K_{\lambda }(k+1)\vert \end{aligned}$$

(32)

Therefore, the $u_i(k)\in U_{\lambda }(k)$ selected matches $u_i(k)$, associated with the maximum $K_i(k+1)$ value.

If the requirement (28) is false, the chosen control signal $u_i(k)$ will be that which is, in absolute terms, the closest to $u_\mathrm{{CE}}(k)$.

Considering the values of input vector (22) subtracted from (19), we have

$$\begin{aligned} U_{\lambda }(k)-u_\mathrm{{CE}}(k)= & {} [(u_1(k)-u_\mathrm{{CE}}(k))\quad (u_2(k)\nonumber \\&-u_\mathrm{{CE}}(k))\quad \cdots \quad (u_i(k)\nonumber \\&-u_\mathrm{{CE}}(k))] \end{aligned}$$

(33)

Consequently, it selects the lowest absolute value of equation (32).

7 Simulation Results

The performance of the suboptimal dual controller with variable parameter $\lambda$ proposed is illustrated through three examples. The system presented in example 1 and in example 2 has often been used, see Maitelli and Yoneyama (1999), to evaluate the performance of different adaptive controllers. Comparisons will be made with three suboptimal dual controllers: the innovations dual controller presented by Milito et al. (1982), the ASOD controller presented by Wittenmark and Elevitch (1985), as well as the two-stage controller with prediction presented by Maitelli and Yoneyama (1994). The evaluation of the controller performance is made from NS simulations, where each one consists of NP steps. Thus, the average loss per step, which evaluates the control quality, is given by

$$\begin{aligned} \overline{V}=\frac{1}{\hbox {NS}}\left\{ \sum ^\mathrm{{NS}}_{j=1}\frac{1}{\hbox {NP}}\left[ \sum ^\mathrm{{NP}}_{i=1}(y(k+i)-y_r(k+1))^2\right] \right\} \end{aligned}$$

(34)

and the standard deviation is given by

$$\begin{aligned} \sigma _V= & {} \sqrt{\frac{1}{\hbox {NS}}\left\{ \sum ^\mathrm{{NS}}_{j=1}\left( \frac{1}{\hbox {NP}}\left[ \sum ^\mathrm{{NP}}_{i=1}a^2\right] -\overline{V}\right) ^2\right\} } \end{aligned}$$

(35)

where

$$\begin{aligned} a=y(k+i)-y_r(k+1) \end{aligned}$$

(36)

The estimation quality can be measured by the mean value of $p_\mathrm{{bb}}$, the parameter variance, which is the main parameter of interest.

$$\begin{aligned} \overline{P}_\mathrm{{bb}}= & {} \frac{1}{\hbox {NS}}\left\{ \sum ^\mathrm{{NS}}_{j=1}\frac{1}{\hbox {NP}}\left[ \sum ^\mathrm{{NP}}_{i=1}p_\mathrm{{bb}}(i)\right] \right\} \end{aligned}$$

(37)

Fig. 1

Simulation of Example 1 with graphics: output y and control signal

Fig. 2

Simulation of Example 1 with graphics: noises and $\varphi _\lambda$ vector

Fig. 3

Three-dimensional lambda vector map over 100 simulations in Example 1

Fig. 4

Simulation of Example 2 with graphics: output y and control signal

Fig. 5

Simulation of Example 2 with graphics: parameter b and $\varphi _\lambda$ vector

Fig. 6

Three-dimensional lambda vector map over 100 simulations in Example 2

Fig. 7

Simulation of Example 3 with graphics: output y and control signal

Fig. 8

Simulation of Example 3 with graphics: variance and $\varphi _\lambda$ vector

Fig. 9

Three-dimensional lambda vector map over 100 simulations in Example 3

The changes between different realizations can be very large, and it can be difficult to know which suboptimal solution is best for each example. So, pairwise comparisons of the average loss per step for each simulation were made, and the statistical hypothesis test was used to decide the controller ranking. The following three examples were simulated (Figs. 1, 2, 3, 4, 5):

Example 1

Consider the first-order system with the reference signal as $y_r=1$. This system has often been used in to evaluate the performance of different adaptive suboptimal controllers (Wittenmark and Elevitch 1985) (Figs. 6, 7, 8, 9).

$$\begin{aligned} y(k)+a(k)y(k-1)=b(k)u(k-1)+e(k) \end{aligned}$$

(38)

where

$$\begin{aligned} a(k)= & {} -0.9\\ b(k+1)= & {} 0.9b(k)+v(k) \end{aligned}$$

Example 2

Consider the first-order system with the reference signal as $y_r=0$

$$\begin{aligned} y(k)+a(k)y(k-1)=b(k)u(k-1)+e(k) \end{aligned}$$

(39)

where

$$\begin{aligned} a(k+1)= & {} -1\\ b(k+1)= & {} 0.8b(k)+v(k) \end{aligned}$$

Example 3

Consider an $ARX(2,2)$ system with the reference signal as $y_r=0$.

$$\begin{aligned}&y(k)+a_1(k)y(k-1)+a_2(k)y(k-2)\nonumber \\&\quad =\,b_1(k)u(k-1)+b_2(k)u(k-2)+e(k) \end{aligned}$$

(40)

where

$$\begin{aligned} a_1(k+1)= & {} -1.2\\ a_2(k+1)= & {} 0.7\\ b_1(k+1)= & {} 0.9b(k)+v(k)\\ b_2(k+1)= & {} 1 \end{aligned}$$

For each system, 200 Monte Carlo simulations $(NS=200)$ were done, where each simulation consisted of 400 time steps $(NP=400)$. In all simulations, the noise $e(k)$ and $v(k)$ had a mean value zero and standard deviation of $0.5$ and $1.0$, respectively. The initial time of the simulation $(k<20)$ is disregarded to avoid the influence of initial effects. The weighting factor $\lambda$ in the ASOD controller and in the innovations controller was chosen to be $0.5$ in all examples, since this value has been shown to give a good exchange between control action and estimation [see Milito et al. (1982) and Wittenmark and Elevitch (1985)]. The two-stage controller using prediction does not require adjustable parameters.

The simulation results are presented in Tables 1, 2, 3, 4, 5, and 6. Figures 3, 6, and 9 show the performance of the parameter $\lambda$ during the simulation, and Figs. 1, 2, 4, 5, 7, and 8 show the performance of the proposed controller for one realization of each system.

Table 1 Simulation results—Example 1

Table 2 Pairwise comparison between controllers—Example 1

Table 3 Simulation results—Example 2

Table 4 Pairwise comparison between controllers—Example 2

Table 5 Simulation result—Example 3

Table 6 Pairwise comparison between controllers—Example 3

The $\varphi _\lambda$ vector used in simulations is given by

$$\begin{aligned} \varphi _\lambda (k)=[0.1\;\;0.3\;\;0.5\;\;0.7\;\;0.9] \end{aligned}$$

(41)

In all examples, the modified ASOD controller achieved results that indicate a better performance compared to the fixed $\lambda$ ASOD controller, as well as the innovations controller. The average loss per step obtained was $20\,\%$ lower, and moreover, as shown in Tables 2, 4, and 6, there is also often an average loss smaller in each simulation. The modified ASOD controller achieved a better performance compared to the two-stage controller with prediction in example 1 and 2.

Figures 3, 6, and 9 show the results associated with each $\varphi _\lambda$ vector during each step, for 100 simulations of examples $1--3$, respectively. Comparing Figs.  3, 6, and 9, it is possible to distinguish how the lambda ($\lambda$) selection algorithm adapts, according to the system, the estimation and control conditions. In Fig. 3, there is a greater balance in choosing lambdas, by equalizing the control action and estimate.

In Fig. 6, the lambda selection is more defined at the ends emphasizing control or estimation in each step. It is also important to note that, in Example 3, the simulations with the innovations controller presented the turn-off phenomenon, while the other controllers avoided this behavior of the control signal. Owing to this phenomenon, the control signal remained turned off, and the pairwise comparison between the innovations controller and the others in Table 6 showed unusual results.

The general behavior of the vector selection $\varphi _\lambda (k)$ will depend on the plant characteristics $\theta (k)$ and $x(k)$, of the associated stochastic variables $e(k)$ and $v(k)$, of the initial plant parameter $P(0)$ and of the known matrix $\varPhi$.

8 Conclusion

The proposed controller achieved good performance without requiring great computational effort. The controller features dual characteristics, i.e., the controller does not care only with the immediate control, but also with the future parameters estimation of the system, resulting in a better control.

The controller avoids the necessity for setting the lambda design parameter, which constitutes an advantage, since this adjustment can be critical in some cases. An algorithm for the adaptive adjustment of the lambda parameter is employed at each step. Thus, the controller performs a better action control, as well as a better estimation, according to the plant characteristics, and the stochastic variables that affect the system. The proposed controller performed satisfactorily better than the dual innovations controller by Milito et al. (1982), the ASOD controller by Wittenmark (1975), and the two-stage controller with prediction controller by Maitelli and Yoneyama (1994).