Keywords

1 Introduction

In recent years, great attention has been paid to the modeling and identification of non-linear systems, because all real processes present non-linearities to some degree. The classical approach to this problem for years was to obtain linear models that were quite representative of the systems to be modeled, but there are cases in which non-linearities can cause significant errors in the identification problem. For this reason, several machine learning techniques and especially neural networks have been an important tool in recent years for the modeling of non-linear systems. In this work, the case of study is the Tennessee Eastman (TE) plant. This plant has been the object of study for 27 years and numerous investigations of various kinds have been developed on it. In [3] linear ARX, impulse response and state space models such as N4SID and CVA were identified, the latter being the best results. In [4] models of the plant reactor based on different structures of artificial neural networks trained by regression and the Levenberg-Marquardt (LM) algorithm were identified to compare their effectiveness when used in Model-based Predictive Controllers (MPC). In [5] state space models such as MOESP, N4SID and ORT were identified and compared. In [6] Genetic Programming was used to obtain a model of the plant reactor. In [7] a gray model of ET was obtained by modeling some variables by basic principles and others by identifying HAMMERSTEIN-WIENER models. In [8] a model based on a multilayer perceptron-type neural network was identified using the Swarm of Birds algorithm. In [9] a model based on neural networks trained with bio-inspired algorithms such as the Bat Algorithm, Firefly Algorithm, and Bee Colony Algorithm was identified. In [10] a Digital Twin of the complete plant is obtained for monitoring and control. In [2, 11] and [12] decentralized control strategies are developed to stabilize and control the entire plant. In [13] a model of the reactor-separator of the plant is developed in the object-oriented modeling language Modelica.

Although in [4, 8] and [9] good models based on neural networks are obtained for the reactor of the TE plant, there is a lack of clarity in the obtaining process. For example, there is no explanation about the selection of input and output variables, and some that contribute to the dynamics of the system are ignored, such as reactant flows. According to [1] the TE plant is open-loop unstable, which is why an experiment cannot be carried out without its prior stabilization. This topic will be better explained in the next section, but it represents an important step in the identification process. In [4, 8] and [9] the authors are able to obtain a large data set, without giving an explanation of the topic, which leads to a lack of clarity in the assumptions, such as the operating conditions of the plant, this analysis is very important, because they are the conditions in which the obtained models will be valid for use. In addition, questions about the design of the experiment such as the type of signals used or their parameters are not specified, it is not taken into account whether the data sufficiently reflect the transient and stable dynamics of the plant. No explanation is offered on the selected neural network structure and regarding the validation of the obtained models, according to [15], it is necessary to perform a residual analysis, in addition to the fit indices such as the MSE and others. In the field of modeling and systems identification, the quantitative validity given by the adjustment of the obtained models is as important as the qualitative validity reflected in the preparation and rigor of the obtaining method. For these reasons, the objective of this work is to obtain models based on neural networks in a case study process, the TE plant, using the systems identification methodology to detail the process and guarantee the validity of the models.

2 TE Process

The Tennessee Eastman plant, was published in [1] as a simulation process for academic research. The TE process is a highly non-linear chemical process, created by a real system, with slight changes made to protect the identity of reactants and products, and features up to 6 modes of operation. This system is a reference problem for the process control community, providing a realistic simulation of a chemical industrial process of interest for analysis, control, monitoring and identification of systems. The process shown in Fig. 1 consists mainly of five operating units: a two-phase reactor, a condenser, a recycle compressor, a vapor-liquid separator, and a distillation column. The process has eight components, including four gaseous reactants (A, C, D, and E), two liquid products (G and H), an inert component (B), and a byproduct (F). As explained in [1] all reactions are irreversible and exothermic. Reaction rates are a function of temperature through the Arrhenius expression. Furthermore, the reactions are approximately first order with respect to reagent concentrations. The gaseous reactants feed into the reactor, where they react to form the two products. Gas phase reactions are catalyzed by a non-volatile catalyst dissolved in the liquid phase. The reactor has an internal cooling system to remove the heat of the reaction. The products leave the reactor as vapor along with the reactants that did not react while the catalyst remains in the reactor. The product stream from the reactor passes through a condenser and from there to a vapor-liquid separator. The non-condensed components are recycled through a centrifugal compressor to the reactor feed. The condensed components are moved towards a distillation column to remove the reagent residues, distilling them together with the reagent C feed. Products G and H exit the column towards a refining section that is not part of the process. Byproduct F is primarily purged from the system as steam in the separator. The complete plant has 12 manipulated variables, and 41 measured, continuous and discrete variables. All measurements have noise added to simulate the behavior of real sensors.

Fig. 1.
figure 1

Flow diagram of the Tennessee Eastman plant.

Full size image

For this work, only a model of the plant’s two-phase reactor will be identified, which is the sub-process responsible for the highest non-linearities of the process. In order to generate an input-output data set to identify a process model, it is first necessary to stabilize the plant, since it is open-loop unstable and has restrictions that when violated, the process stops immediately. The control strategy must prevent these restrictions from being violated while the plant is disturbed by excitation at the inputs. This model can then be used as the basis for a Model-based Predictive Controller, which would act at the supervisory level. However, to preserve as much process dynamics as possible, the number of controllers added to the plant should be kept to a minimum and the controllers are used solely to prevent the plant from violating its limitations. The control strategy is the one proposed in [5], which is a simplification of “Stage 1” and “Stage 2” proposed in [11] and [12]. As a result of the stabilization process, not all the manipulated variables mentioned in [1] are accessible. On the other hand, the controller references constitute additional manipulated variables of the stable process.

The measured variables that represent the outputs of the model are shown in Table 1, it should be clarified that the reactor temperature was not considered as a model output because with the implemented control strategy it becomes the variable manipulated by the pressure controller, in addition, as long as the pressure restriction is met, the temperature restriction is also met. The manipulated variables that will be considered as the inputs of the system for identification and their stable values for Mode 1 of operation of the plant are represented in Table 2. The flow of reactant E is not taken into account because it is the variable manipulated by the level controller. In addition to the manipulated variables, there is a measured variable that influences the behavior of the model outputs, the feed flow to the reactor, which is composed of the flow of A, D, E and the recycling flow of the plant, this variable will also be included in the model inputs as a measurable disturbance.

Table 1. Model outputs.
Full size table
Table 2. Model inputs.
Full size table

3 Methodology

Obtaining the model will be based on the systems identification methodology shown in Fig. 2.

Fig. 2.
figure 2

System identification methodology.

Full size image

3.1 Experiment Design

In order to carry out the experiment for identification, it is necessary to know the excitation limits of the inputs, as well as the time constants of the system. To do this, a simple experiment is carried out, in which one input is varied at a time with a positive and a negative step, each with a duration of 5 h. In order not to violate the plant’s restrictions, the inputs will be varied to ±10% of their value in Mode 1, except for the pressure reference in the reactor, which is the most critical, this will be varied by ±100 kPa. Figure 3 shows the response of the system outputs to a stimulus in the D and A flows, and in the pressure and level references. The experiment illustrates the variety in terms of time constants of the system. It is concluded that a pulse width of 5 h is sufficient to capture the transient and stable dynamics of the system and that, for the chosen variations in the inputs, the outputs are far from violating any restriction. There is a risk that by stimulating all the inputs at the same time the variations in the outputs will be much larger, for this reason values relatively close to the operating point were chosen. It is very difficult to incorporate knowledge into a neural network, therefore, the quality of the network is dependent on the data used to train it. The training data must cover the entire range of inputs for which the network will be used, because neural networks, like other nonlinear black box models, do not extrapolate well. To comply with the above, the data for identification were generated by stimulating the manipulated variables with ‘skyline’ functions, with the maximum variations mentioned above and pulse width of 5 h.

Fig. 3.
figure 3

Output response for a stimulus of + -10% in the flow of D and A and level reference, + -100 kPa in the pressure reference.

Full size image

3.2 Data Collection and Pretreatment

The main objective of data pre-treatment is to facilitate network training. The TE process has incorporated in its programming a random noise additive to all measurable variables, therefore, it is necessary to filter the signals to eliminate this noise. In this work, the data were filtered by a second-order Butterworth low-pass filter with a cut-off frequency of 10 rad/s to eliminate noise from high frequencies. A standard practice in data pretreatment to train neural networks is data normalization. As the inputs of the process differ in magnitude, there is an undesired effect on the weights of the network, which try to compensate for the magnitude of the input so as not to saturate activation functions. A 500-h experiment was performed. The data obtained were filtered and normalized, remaining as shown in Fig. 4 and Fig. 5.

Fig. 4.
figure 4

Pretreated input data for the four manipulated variables and the measurable disturbance.

Full size image
Fig. 5.
figure 5

Pretreated output data of the two measured variables.

Full size image

3.3 Selection of the Type and Structure of the Model

There is a great variety of neural network structures, each with the objective of solving a specific problem, as explain in chapter 6 of [14]. It is a great task to study in depth the characteristics, advantages and disadvantages of each structure in each situation. According to Chapter 14 of [15] neural networks can be classified into static and dynamic. Static are networks in which the output is calculated directly from the inputs through direct connections. In dynamic networks, the output depends not only on the current input of the network, but also on the previous inputs, outputs or states of the network. Dynamic networks are networks that contain ‘delays’, or integrators in the case of continuous inputs, and operate with an input sequence. These dynamic networks can have only direct connections, or they can also have feedback connections, the latter type are known as recurrent neural networks. As dynamic networks have ‘memory’ they can be trained to learn patterns that vary over time, that is, they can approximate dynamic systems. This type of problem in the machine learning branch of study is known as time series analysis, which, taken to the field of control theory, is the systems identification. There is a great variety of architectures for recurrent neural networks, depending on their direct and feedback connections, the number of internal layers, etc. Among them the Hopfield network explained in Chapter 21 of [15], the fully connected recurring networks, the short and long-term memory network widely used in the field of speech recognition, the Elman network, used in [16] to model an internal combustion engine, the nonlinear autoregressive network with external inputs NARX, which is explained in chapter 27 of [15] and in [17] used to identify the model of a distillation column. The NARXs are recurrent neural networks with delays in the input connections and in the feedback connections of the output, therefore, they have a structure that facilitates learning the behavior of dynamic systems. In this work, two NARXs will be used to model the pressure and level of the reactor. Both with the same parameters of 10 neurons in the input layer and 20 delays for both the inputs and the feedback output, with sigmoid activation function for the inner layer and linear for the output layer. The parameters were obtained after performing several iterations of the methodology in Fig. 2 until satisfactory results were obtained. In [4, 8] and [9] recurrent networks are used where only the last output is fed back. The structure selected in this work is superior because dynamic systems depend on both past outputs and past inputs, and by using several delays in both, better performance is expected in the time series analysis problem.

3.4 Estimation of Parameters, Application of Algorithms

The networks were trained with the Levenberg-Marquardt algorithm using the Neural Networks ToolBox of the MATLAB version R2015a software. To train each network, a 200-h fragment of the data set obtained and pretreated was chosen. Each network was trained a total of 5 times to avoid the local optimum problem and training was stopped when the performance of the network calculated by the least squared error stopped improving significantly.

3.5 Validation

For model identification or prediction problems, it is convenient to perform an analysis of prediction residuals or errors. The residues cannot be related to each other, nor to the sequence of the entries. For this, the autocorrelation function and the cross-correlation function are used. Figure 6 shows the autocorrelation graphs of the residuals for the two networks and in Fig. 7 the cross-correlation of the residuals with the inputs. For the residuals to be unrelated, the autocorrelation function must give an impulse at the instant Ƭ = 0 and the rest of the values be within the confidence interval denoted by the dashed lines, in Fig. 6 it is observed that this is not fulfilled at all, although it is an acceptable result for this type of problem. For the cross correlation, the residuals must be within the confidence interval at all times. Figure 7 shows that this is true for the two networks. In order to validate whether a black box model is representative of the system that was identified, it is vital to perform a cross-validation, that is, to analyze the behavior of the model in the face of a different data set than the one used for training. Figure 8 and Fig. 9 show the behavior of the two networks for 100 h of data, different from those used during training. Table 3 shows the calculation of the percentage of fit (FIT) and the mean square error (MSE) of the models and their analogues with the best results in [8] and [9], the BSA and the ANN-BA respectively. As can be seen, the models obtained in this work have competitive results, despite the global optimization techniques used for training the networks in [8] and [9]. This is mainly due to the work carried out throughout the identification process, from the selection of the input variables, the design of the experiment and the selection of a better neural network structure.

Fig. 6.
figure 6

Autocorrelation of the residuals. a) Reactor pressure, b) Reactor level.

Full size image
Fig. 7.
figure 7

Cross correlation of the residuals and the inputs. a) Reactor pressure, b) Reactor level.

Full size image
Fig. 8.
figure 8

Response of the reactor pressure network for the validation data.

Full size image
Fig. 9.
figure 9

Response of the reactor level network for the validation data.

Full size image
Table 3. Validation indices calculated for the models.
Full size table

4 Conclusions and Future Work

In the present investigation, a model based on NARX-type neural networks was obtained for the chemical reactor of the Tennessee Eastman plant using the systems identification methodology, the main objective of the work. Specifically, a decentralized control was implemented to stabilize the plant, an identification experiment was carried out to obtain the data with which two recurrent neural networks of the NARX type were trained to model the level and pressure in the reactor. The models obtained present good results. This work shows that rigor and preparation in the system identification process not only gives qualitative validity to the resulting models, but also directly influences their performance. This paper will serve as the basis, for the authors, in future studies on the main problems when using neural networks as models of dynamic systems: the lack of generalization, local minimums in the training process and the lack of tools to incorporate the physical knowledge of the system in the models.