The Reiterated Neural Network Parametric Identification of Nonlinear Dynamic Models of Objects

Abstract

We posed a problem and developed an algorithm for the neural network parametric identification of nonlinear dynamic models of objects with a computational experiment, the formation of training samples on its basis and the subsequent sequential training of two neural networks. Neural networks perform bijective mapping of object parameters into the original model to the output variables of the second neural network. Sequential training or sequential bijective identification of a neural network consists in preliminary training of the first neural network based on experimental data and using its synaptic coefficients to train the second neural network. An example of parametric identification of a nonlinear dynamic model will be a 600 W high pressure sodium lamp. The computational experiment was carried out in the MATLAB environment. The computational experiment technique and the results of experimental studies are presented in the article. Taking into account the good approximating ability of neural networks, the proposed algorithm can be considered as an effective method for parametric identification of nonlinear models.

1 Introduction

Quite a lot of approaches and methods have been developed for the identification of nonlinear objects [1, 2]. However, they do not allow us to directly evaluate the parameters of a nonlinear dynamic model. Modern heuristic methods for minimizing the discrepancy between the calculated and experimental data on the parameters of the model [3] make it possible to quickly obtain results of acceptable quality but do not guarantee the finding of an unambiguous solution to the identification problem. The use of neural networks for this purpose [4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19] shows that parametric identification is in principle possible. However, in the case of nonlinear objects, an “individual” approach is required in choosing the type, structure, composition, and algorithm for training the network which in general is a rather complex problem.

2 Statement of the Identification Problem

A mathematical model of a dynamic nonlinear system is given:

$$ F\left( {x,\dot{x},y,u,\theta } \right) = 0, $$

(1)

where x, (x) ̇ are vectors of state variables and their derivatives, y, u are vectors of output and input variables; θ is the vector of model parameters.

Experimental waveforms of system variables are given (1)

$$ x_{e} = x\left( t \right); \, y_{e} = y\left( t \right);u_{e} = u\left( t \right). $$

(2)

Using the model of a dynamic nonlinear object or system (1) and experimental data (2), it is necessary to find the mapping

$$ \left[ {x_{e} ,y_{e} ,u_{e} } \right] \Rightarrow \hat{\theta }, $$

(3)

where ${\hat{\uptheta }}$ is an estimate of the model parameters that minimizes the discrepancy E_m parameters θ taken relative to their nominal values θ_n: $\theta_{o} = \theta /\theta_{n}$

$$ E_{m} = \min \left| {\theta_{o} - \hat{\theta }_{o} } \right| $$

(4)

3 Two-Dimensional Neural Network Approximation of the Inverse Operator of Parametric Identification

The parametric identification algorithm consists in mapping a set of model parameters θ into an estimate of these parameters ${\hat{\uptheta }}$ through a particular identification procedure. Traditionally, this procedure is carried out in three stages [20, 21]. At the first stage, an array of input variables of the U model is formed which has statistical characteristics close to the characteristics of white noise. This condition is necessary to improve the convergence of the model estimates during identification. At the second stage, the initial parameters of the model are mapped into a set of model state variables X and output variables Y using a physical (on the object) or computational (on the model) experiment. In the third stage, the sets of input U and output X, Y variables are mapped using the identification procedure to the estimate of the set of model parameters θ.

Without reducing the generality of reasoning we will assume that the dynamic model under study is completely observable, otherwise an array of state variables X should be used as the set of output variables.

In this formulation, the identification problem is the inverse of the problem of modeling the dynamic model under study. The primal problem is to find an array of output variables Y from the given arrays of input variables U and parameters θ of the model. The inverse problem, the identification problem on the contrary is to find an array of model parameter estimates ${(}{\hat{\uptheta }}{)}$ from the arrays of its input U and output Y variables that can be represented by the following structures (Fig. 1).

Fig. 1.

Block diagrams of modeling and identification of dynamic models.

Obviously, when solving any inverse problem questions arise about the existence, correctness, and physical realizability of the inverse operator that provides an array of estimates of the parameters of the model $\uptheta$. In solving such problems, the method of regularization of A. N. Tikhonov is widely used.

In this case, to improve the accuracy and convergence of estimates it is proposed to use the method of two-dimensional neural network approximation of the inverse operator that implements the identification procedure.

The first neural network based on a training sample of output U, output Y variables, and state X variables grouped into a training sample.

$$ \begin{aligned} {\text{P}}_{r} =\, & [X,Y,U]; \\ T_{r} =\, & Y; \\ X =\, & X(t);Y = Y(t);U = U(t), \\ \end{aligned} $$

(5)

where X, Y, U are arrays of calculated data including all the model variables obtained in each of the N experiments of the experiment plan. The resulting training arrays P_r T_r are mapped to the array of synaptic and weight coefficients of the first neural network W_k.

$$ \left[ {P_{r} ,T_{r} } \right]\mathop{\longrightarrow}\limits^{Training}W_{k} $$

(6)

The second neural network based on a training sample consisting of an array of synaptic coefficients and an array of model parameters ${\uptheta }$ obtained as a result of a computational experiment (6) calculates an estimate of the model parameters ${\hat{\Theta }}$.

$$ \begin{gathered} P_{c} = W_{k} ; \hfill \\ T_{c} = \Theta , \hfill \\ \end{gathered} $$

(7)

Taking into account the powerful approximating capabilities of neural networks, we should expect high accuracy of parametric identification.

4 Algorithm of Parametric Identification

The following algorithm is proposed for obtaining a mapping (3). On the mathematical model (1) by varying the parameters of θ is conducted an experiment (e.g. full factorial experiment 2N where N is dimension of vector θ) and a sample of the parameters of the Θ model is formed and samples with an array of inputs Pr and an array of outputs Tr s formed to build simulated models:

$$ \begin{aligned} P_{c} & = [X,Y,U]; \\ T_{c} & = Y; \\ X & = X(t);Y = Y(t);U = U(t), \\ \end{aligned} $$

(8)

where X, Y, U are arrays of calculated data that include all the model variables obtained in each of the N experiments of the experiment plan.

On the sample P_r, T_r the simulated model is constructed and the matrix of its parameters W_r is formed. As a simulated model a neural network is used in which the synaptic coefficients W_r play the role of identifiable parameters although any dynamic model can be used.

A new training sample is formed for training a static neural network:

$$ \begin{gathered} P_{c} = W_{k} ; \hfill \\ T_{c} = \Theta , \hfill \\ \end{gathered} $$

(9)

• 1. The second static network is being trained and the estimation of the vector of parameters θ model (1) is calculated in each of the N experiments.

• 2. Experimental waveforms (2) hat did not participate in the training experiments are fed to the input of the obtained simulated model (the first trained neural network) and the vector of parameters of the simulated model (synaptic coefficients of the dynamic network) W_r for the current state of the object is calculated.

• 3. The obtained coefficients are fed to the input of a static neural network and the estimation of the vectors of model parameters for the current state of the real object ${\hat{\Theta }}$ is being calculated.

5 Forming a Model of a Nonlinear Object

As an example, the parametric identification of a high-pressure sodium lamp of the type high-pressure sodium arc lamp with a power of 600 W is considered [8, 9, 12, 13, 15,16,17].

$$ \left\{ {\begin{array}{*{20}l} {\frac{{dx_{1} }}{dt} = \frac{1}{L}\left[ {U_{s} - \left( {\frac{1}{{x_{2} x_{3} }} + R} \right)x_{1} } \right],} \hfill \\ {\frac{{dx_{2} }}{dt} = A_{l} U_{0}^{2} x_{2}^{2} \frac{{\left( {\frac{{x_{1} }}{{U_{0} x_{2} x_{3} }}} \right)^{2} - 1}}{{1 + k_{1} \left( {\frac{{\left| {x_{1} } \right|}}{{U_{0} x_{2} x_{3} }} - 1} \right)}},} \hfill \\ {\frac{{dx_{3} }}{dt} = \left[ {k_{2} + k_{3} \left( {\frac{{\left| {x{}_{1}} \right|}}{{U_{0} x_{2} x_{3} }}} \right)^{{k_{4} }} } \right]\left[ {1 + k_{1} \left( {\frac{{\left| {x_{1} } \right|}}{{U_{0} x_{2} x_{3} }} - 1} \right) - x_{3} } \right],} \hfill \\ \end{array} } \right. $$

(10)

where x₁ is the lamp current; x₂ is reduced conductance of lamp that takes into account the average electron concentration; x₃ is a dimensionless quantity that takes into account the electron mobility; L, R are respectively the inductance and active resistance of the limiting choke; U_s, U₀ are respectively the supply voltage and the rated voltage on the lamp; A_l is a factor determined by the design of the lamp; k₁–k₄ are electrical coefficients determined for a particular type of lamp.

The lamp parameters are given in Table 3, the choke parameters R = 14 $\Omega$; L = 0.062 H [8, 10,11,12].

The solution of Eqs. (9) was carried out in the MATLAB environment. The calculated and experimental waveforms of the lamp voltage and current are shown in Fig. 2.

Fig. 2.

The calculated and experimental waveforms of the lamp voltage and current.

A close agreement between the calculated and experimental data confirms an adequate description of the object by the system of Eqs. (9). Therefore, to demonstrate the effectiveness of the proposed notification method, we will later use experimental waveforms (Fig. 2) obtained by numerical solution of the system of Eqs. (9).

6 Parametric Identification of a Nonlinear Object Model

Parametric identification was carried out in accordance with the proposed algorithm:

An experiment was conducted in which the parameters of the model (7) were randomly changed in a limited range (Table 1).

Table 1. An experiment plan.

A numerical solution of the system (7) was carried out in each experiment and a training sample (5) was formed based on the calculated voltage U_l and current I_l of the lamp, which in this example has the form:

$$ \begin{aligned} {\text{P}}_{r} & = [U_{l} ,I_{l} ]; \\ T_{r} & = I_{l} ; \\ \end{aligned} $$

(11)

where U_l, I_l are arrays of waveforms of currents and voltages on the lamp obtained as a result of solving system (9) in each of the experiments specified in Table 1.

Sample (10) was used to train a two-layer forward neural network with 2 neurons in a hidden layer and with linear activation functions in each layer. Thus, the first neural network has 2 synatic connections in the hidden layer, 1 in the output layer, and 2 shift coefficients. The network was trained using the Levenberg-Markwart method with Bayesian regularization. The learning error in each experiment was almost zero. The maximum error value did not exceed 2 × 10^–10 A.

A new training sample is formed for training the static neural network from the obtained synaptic coefficients of the dynamic neural network. Table 2 shows the synaptic coefficients of the network obtained as a result of training a dynamic neural network according to Table 1 and the subsequent numerical solution (7).

Table 2 was used for training a static neural network which was chosen as a three-layer direct transmission network with linear activation functions in each layer. The network was also trained similarly using the Levenberg-Markwart method with Bayesian regularization. The relative learning errors in each experiment are shown in Fig. 3.

Table 2. Training sample of a static neural network (synaptic coefficients of a dynamic neural network).

Fig. 3.

The errors in calculating the parameters of the analytical model (7) obtained after training.

The experimental values of the lamp current and voltage were fed to the input of the first neural network and the vector of its parameters was calculated (the last column of Table 2).

Calculated vector of parameters of the dynamic neural network was fed to the input of the trained neural static, and calculated estimation of the model parameter vector ${\hat{\uptheta }}$ of the real object (Table 3) [20,21,22].

Table 3. Estimates of the nonlinear model parameters obtained as a result of identification.

We can note a good agreement between the calculated data of the model (7) and the data obtained as a result of identification. In addition, the root-mean-square deviation from the experimental values of current and voltage is less during identification.

7 Summary

• 1. The problem of parametric identification of nonlinear models of an object is posed, which consists in obtaining the mapping of experimental data of an object in the parameters of its model in its parameters using neural networks.

• 2. The algorithm of parametric identification is developed. It consists of carrying a computational experiment on a given nonlinear model, forming training samples based on the results of the experiment, then training dynamic and static neural networks, and calculating estimates of the parameters of the nonlinear model based on experimental data using trained networks.

• 3. A combination of two neural networks is proposed in which, during training and subsequent work, the synaptic coefficients of the first neural network are fed to the input of the second neural network.

• 4. In such a network the experimental data is displayed in the model parameters sequentially. The experimental data is first displayed in the synaptic coefficients of the first neural network. Then the synaptic coefficients of this network are fed to the inputs of the second neural network the output of which is the required parameter of the nonlinear model.

• 5. Experimental validation of the proposed method of neural network parametric identification on the example of a high-pressure sodium lamp model showed that the root-mean-square deviation of current and voltage from the nominal values does not exceed 4% for voltage and 10% for current.

• 6. Taking into account the good approximating ability of neural networks the proposed algorithm and neural networks can be considered as an effective identification method.