Similarity of Sensitivity Functions

Abstract

If a model is strongly autocatalytic and very different timescales are present, both of which are characteristic features of many reaction kinetic models, then the calculated local sensitivity functions are usually similar to each other. An implication of this is that in many cases, by changing a number of input parameters simultaneously according to certain ratios, almost identical simulation results can be obtained for output variables of kinetic models, over quite wide ranges of concentrations or reaction conditions. The similarity relations can be sorted into categories of local similarity, scaling relationships and global similarity. Such similarity relations have been found in models of combustion systems (explosions and flames) and molecular biological models. The theory of the origin of all these similarity relations is discussed in this chapter. The similarity of sensitivity functions is related to several important topics, such as discrimination between models, uniqueness of a model and robustness of biological systems.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

8.1 Introduction and Basic Definitions

Solutions of models using detailed reaction mechanisms are nonlinear functions of parameters. In nonlinear models, we might intuitively expect that each parameter plays a different role in driving the predicted outputs. In such cases, when one of the parameters is changed, it is not possible to return all variables back to their original values at all times simply by changing the values of other parameters. However, in some cases, similarities exist between the sensitivities of model outputs to different parameters, and hence, the influence of modifying one parameter may be counterbalanced by tuning others. This has important implications for situations where attempts are made to tune parameter sets in order to improve agreement between model simulations and experiment. For example, when certain similarities exist, it may be possible to choose different parameter sets that lead to exactly the same numerical solution of the model equations. In these situations, using experimental results to constrain the values of certain parameters may not be possible. Such similarities may also have implications for the dynamical dimension of the equation system as discussed in Sect. 6.5. For these reasons, it is valuable to study the similarity relations between sensitivity functions in a model, and this will be covered in the present chapter .

According to the kinetic system of differential equations, the production rates are linear functions of the rate coefficients, and therefore, changing a rate coefficient during a short period of time linearly changes the calculated concentrations and temperature . However, the changes in concentrations and temperature interact with each other, causing nonlinear deviations. If a reaction mechanism contains only first-order reactions, then the concentration–time functions are sums of exponential functions (as discussed in Sect. 6.3). If at least one of the reaction steps is second-order, then the analytical solution may contain very complicated concentration–time functions, even if the rate coefficients are constant. In systems with changing temperature and pressure, the temperature dependence (Arrhenius equation, modified Arrhenius equation; see Sect. 2.2.1) and pressure dependence (e.g. equations of Lindemann and Troe; see Sect. 2.2.2) of the rate coefficients have to be calculated, and these are also nonlinear functions of the parameters. Therefore, it might be expected that the simulated outputs of reaction kinetic models are usually nonlinear functions of the parameter values.

However, there are other features of the kinetic system of differential equations that may simplify the situation. The application of kinetic simplification principles (see Sect. 2.3) may result in the situation where it is not that the individual parameters have an influence on the solution, but only some combinations of these parameters. A simple example occurs when species B is a QSS-species within the A → B → C reaction system, and its concentration depends only on ratio k ₁/k ₂. Also, when the production rate of species C is calculated using the pre-equilibrium approximation (see Sect. 2.3.2) within reaction system A ⇄ B → C, it depends only on equilibrium constant K = k ₁/k ₂ and does not depend on the individual values of k ₁ and k ₂.

In addition to these simple examples, several studies have suggested that by changing a number of input parameters simultaneously according to certain ratios may result in almost identical simulation results for all output variables of kinetic models over quite wide ranges of variable concentrations. Given the nonlinearity of the models, this is perhaps surprising and implies that a highly nonlinear chemical kinetic model can behave linearly for some parameter changes. This feature, which occurs for some reaction kinetic models, is linked to the existence of relationships between the local sensitivity functions. Rabitz et al. (Reuven et al. 1986; Smooke et al. 1988; Rabitz and Smooke 1988; Vajda et al. 1990; Vajda and Rabitz 1992; Mishra et al. 1994) calculated the sensitivity–distance functions of stationary flame models and discovered several interesting relationships. Zsély et al. (Zsély et al. 2003; Zsély and Turányi 2003; Zádor et al. 2004; Zsély et al. 2005; Lovrics et al. 2008) detected the similarities of sensitivity functions in other chemical systems and provided an interpretation of these features.

The similarity of sensitivity functions has been detected in one-dimensional stationary flames (where the independent variable is distance) and in spatially homogeneous systems (where the independent variable is time). To provide a unified notation of these two types of systems, the independent variable (distance or time) will be represented by z. The local sensitivity function is given as follows:

$$ {s}_{ij}(z)=\frac{\partial {y}_i}{\partial {x}_j}. $$

(8.1)

Similarities of the sensitivity functions can then be sorted into the following categories:

• 1. Local similarity

  The ratio

  $$ {\lambda}_{ij}(z)=\frac{s_{ik}(z)}{s_{jk}(z)}, $$

  (8.2)

  changes with independent variable z (time or distance); λ _ij depends on the selection of model outputs Y _i and Y _j , but it is independent of which parameter x _k is changed.

• 2. Scaling law

  The equation

  $$ \frac{\left(\mathrm{d}{Y}_i/\mathrm{d}z\right)}{\left(\mathrm{d}{Y}_j/\mathrm{d}z\right)}=\frac{s_{ik}(z)}{s_{jk}(z)}, $$

  (8.3)

  is valid for all parameters x _k . Since the derivatives of concentrations with respect to z are always independent of the parameters, the local similarity condition is always valid if the scaling law is valid.

• 3. Global similarity

  The ratio

  $$ {\mu}_{km}=\frac{s_{ik}(z)}{s_{im}(z)}, $$

  (8.4)

  does not depend on the independent variable z (time or distance) in the interval (z ₁, z ₂), and it is also independent of the selection of the parameter.

An example of a reaction kinetic model that exhibits all the laws described above is the adiabatic explosion of hydrogen–air mixtures (Zsély et al. 2003). Figures 8.1, 8.2 and 8.3 are related to the adiabatic explosion of stoichiometric hydrogen–air mixtures with an initial temperature of T ₀ = 800 K and a constant pressure of p = 1 atm.

Fig. 8.1

An example of the local similarity of sensitivity functions. Figure (a) shows the local sensitivity coefficients belonging to the calculated H-atom concentration as a function of T, where the investigated parameters are Arrhenius parameters A of the reaction steps. Figure (b) shows similar results belonging to the sensitivity functions of the H₂O concentrations. In figures (a) and (b), the two largest sensitivity functions are indicated by red dashed and green dotted lines. Figure (c) presents the ratios of the sensitivity functions belonging to the same pair of variables, but to different parameters (e.g. the red dashed “a” curve is divided by the red dashed “b” curve, the green dotted “a” curve is divided by the green dotted “b” curve, etc.). It is well visible that all ratios of these sensitivity functions coincide [see Eq. (8.2)]. Adapted with permission from Zsély et al. (2003). Copyright (2003) American Chemical Society

Fig. 8.2

An example of a scaling law in a model of the adiabatic explosion of hydrogen–air mixtures. Figure (a) shows the ratios of sensitivity functions belonging to the concentration of OH and temperature as a function of temperature. All ratios coincide; therefore, local similarity is valid. Figure (b) shows the ratio of the production rate of OH and the time derivative of temperature, also as a function of temperature. The two curves coincide; therefore, the scaling relation is also valid [see Eq. (8.3)]. Adapted with permission from Zsély et al. (2003). Copyright (2003) American Chemical Society

Fig. 8.3

An example of the global similarity of sensitivity functions. Figure (a) shows the sensitivity functions of the H₂O concentrations, when the investigated parameters are Arrhenius parameters A of the reaction steps. Figure (b) shows that if each sensitivity function is divided by the sensitivity function having the highest maximum, then the ratios will be different from each other, but these ratios are constant across a wide range of the independent variable, which means that the global similarity criterion is valid [see Eq. (8.4)]. Adapted with permission from Zsély et al. (2003). Copyright (2003) American Chemical Society

8.2 The Origins of Local Similarity and Scaling Relationships

In this section, we show that the scaling relation emerges in situations where there is a 1D manifold in the space of variables and where changing a parameter may change the speed of the trajectory along the manifold, but negligibly shifts its location .

The 1D manifold can be defined (at least locally) by a function F _i that gives the values of all variables in the system as a function of an arbitrary variable Y ₁:

$$ {Y}_i\left(z,\mathbf{x}\right)={F}_i\left({Y}_1\left(z,\mathbf{x}\right)\right), $$

(8.5)

where z is the independent variable (time or distance). The particular case of slow manifolds was discussed in Sect. 6.5 . If the system of differential equations is autonomous (which is valid for most chemical kinetic systems) , then the function F _i does not depend directly on z. If we assume that the manifold is not shifted as a result of changing a parameter, then we may claim that F _i does not depend directly on the parameter vector x. If we differentiate Eq. (8.5) first with respect to z and then independently with respect top _k , we get

$$ \frac{\partial {Y}_i\left(z,\mathbf{x}\right)}{\partial z}=\frac{\partial {F}_i}{\partial {Y}_1}\frac{\partial {Y}_1\left(z,\mathbf{x}\right)}{\partial z}, $$

(8.6)

$$ \frac{\partial {Y}_i\left(z,\mathbf{x}\right)}{\partial {x}_k}=\frac{\partial {F}_i}{\partial {Y}_1}\frac{\partial {Y}_1\left(z,\mathbf{x}\right)}{\partial {x}_k}. $$

(8.7)

A comparison of the two equations yields

$$ \frac{\partial {Y}_i(z)}{\partial {x}_k}=\frac{\partial {Y}_1(z)}{\partial {x}_k}\frac{\partial {Y}_i}{\partial z}{\left(\frac{\partial {Y}_1}{\partial z}\right)}^{-1}. $$

(8.8)

This equation is valid for both time-dependent spatially homogeneous and spatially 1D stationary systems. Equation (8.8) leads to the emergence of the scaling law, since by applying it to Y _j , it can be easily converted to Eq. (8.3). Equation (8.3) does not contain variable Y ₁, which emphasises that the selection of variable Y ₁ is arbitrary. Equation (8.3) also means that any row of the sensitivity matrix can be obtained by multiplying any other row containing nonzero values with a scalar. This means that the rank of the sensitivity matrix is one, if the state of the system is close to a one-dimensional manifold. This relation makes a close connection between the dimension of the manifold of the dynamical systems and the rank of the sensitivity matrices.

It can be demonstrated in a similar way that the dimension of the slow manifold sets an upper limit on the rank of the sensitivity matrix. An n-dimensional manifold can be parameterised with n variables:

$$ {Y}_i\left(z,\mathbf{x}\right)={F}_i\left({Y}_1\left(z,\mathbf{x}\right),{Y}_2\left(z,\mathbf{x}\right),\dots, {Y}_n\left(z,\mathbf{x}\right)\right). $$

(8.9)

Differentiating both sides of the equation with respect to p _j gives

$$ \frac{\partial {Y}_i}{\partial {x}_j}=\left(\frac{\partial {F}_i}{\partial {Y}_1}\right)\left(\frac{\partial {Y}_1}{\partial {x}_j}\right)+\left(\frac{\partial {F}_i}{\partial {Y}_2}\right)\left(\frac{\partial {Y}_2}{\partial {x}_j}\right)+\cdots +\left(\frac{\partial {F}_i}{\partial {Y}_n}\right)\left(\frac{\partial {Y}_n}{\partial {x}_j}\right). $$

(8.10)

The multiplying factors ∂F _i /∂Y ₁, ∂F _i /∂Y ₂, … are identical for each parameter x _j ; therefore, Eq. (8.10) can be written in the following vector equation form:

$$ {\mathbf{s}}_i = {\lambda}_{i1}{\mathbf{s}}_1+{\lambda}_{i2}{\mathbf{s}}_2+\cdots +{\lambda}_{in}{\mathbf{s}}_n. $$

(8.11)

This means that if the trajectory of a simulation is close to an n-dimensional manifold, and the perturbation of the parameters negligibly shifts the location of the manifold, then the rank of the local sensitivity matrix is not higher than n. Even if the rank of the sensitivity matrix n is lower than the number of species, it does not mean that local similarity is valid for any pairs of the sensitivity vectors. The other extreme case is when all sensitivity vectors are locally similar except for n vectors.

The relationships among the dimension of the slow manifold, the rank of the sensitivity matrix, the local similarity and the scaling relations can also be demonstrated using geometric reasoning. Figure 8.4a shows a schematic drawing of a 1D manifold in a closed, adiabatic system. The full space of variables of a chemical reaction system is usually multidimensional. For example, that of a homogeneous adiabatic explosion of hydrogen–air mixtures is 10 dimensional, since the independent variables are the concentrations of nine species and temperature. The variable space in Fig. 8.4a is depicted in three dimensions for ease of visualisation. Point C denotes the actual state of the system, and point E ₀ denotes the equilibrium point belonging to a specific enthalpy h ₀. Point C moves in the space of variables with velocity $\dot{\mathbf{Y}}$. The projections of this velocity vector onto the axes are equal to the production rates of the species or the time derivative of temperature. It is clear that the direction of the velocity vector is equal to the direction of the tangent of the slow manifold at point C. We assume that a small change of parameter x _k does not change the location of the slow manifold in the space of variables, but changes the location of the system along the manifold. This means that after time t, the system will not be at point C, but at a nearby point C′. The direction of vector $\mathbf{C}{\mathbf{C}}^{\mathbf{\prime}}\to$ is along the tangent of the manifold at point C and is identical for the perturbation of any parameter x _k . The direction of this vector is identical to the direction of all sensitivity vectors ∂Y/∂x _k , and the projections of this vector onto the axes are the sensitivity coefficients (see Fig. 8.4b). If the directions of two vectors are identical, then the ratios of their projections onto the axes are identical, even if the lengths of the vectors are different. This explains the scaling law and also why any sensitivity vector can be obtained by multiplying any other nonzero sensitivity vector belonging to a different parameter by an appropriate scalar.

Fig. 8.4

(a) A one-dimensional manifold (solid curve) belonging to specific enthalpy h ₀ in the space of variables. E ₀ is the equilibrium point and point C shows the actual state of the system; its velocity is $\dot{\mathbf{Y}}$. Projections of the velocity vector on the axes are the right-hand sides of the system of differential equations (in reaction kinetics, these are the production rates). (b) Points C and C′ represent the state of the system after a given elapsed time since the beginning of the simulation using the original set of parameters and when the value of parameter x _k has been changed, respectively. Since the system may evolve only along the 1D manifold, the directions of vectors $\dot{\mathbf{Y}}$, $\mathbf{C}{\mathbf{C}}^{\mathbf{\prime}}\to$ and ∂Y/∂x _k are identical, and hence, the ratios of the coordinates of these vectors are identical for any parameter x _k and for any pair of variables Y _i and Y _j . (c) One-dimensional manifolds, belonging to different specific enthalpies h ₀, h ₁ and h ₂. If a parameter change includes the change of the specific enthalpies of the reacting mixture, then the directions of vectors ∂Y/∂x _k will be different for the different parameters. (d) A 2D manifold belonging to specific enthalpy h ₀. Point C represents the actual state of the system, and $\dot{\mathbf{Y}}$ is its velocity. If two parameters are changed without changing the specific enthalpy of the system, then after some time, the state of the system can be represented by points C′ and C″. In this case, the direction of the velocity vector $\dot{\mathbf{Y}}$ does not coincide with the directions of the sensitivity vectors, but all the three vectors are on the tangent plane of the 2D manifold. Adapted with permission from Zsély et al. (2003). Copyright (2003) American Chemical Society

For the simulation of adiabatic systems, the enthalpy of the system is always constant, even if the parameters of the kinetic model are changed. On the other hand, when changing the kinetic parameters for the simulation of fixed temperature profile systems, the calculated enthalpy of the system may change. At time t, we denote the specific enthalpy for the adiabatic model and the model with a fixed temperature profile as h ₀ and that for the model with a fixed temperature profile but modified parameters as h ₁. At time t, the modified system is at point C′, that is, near to the 1D manifold belonging to specific enthalpy h ₁. On changing another parameter, the system will be at point C″, near to the 1D manifold belonging to specific enthalpy h ₂ (see Fig. 8.4c). It is clear that the scaling relation will not emerge in calculations that apply a fixed temperature profile.

If the dimension of the manifold is two, then any row of the sensitivity matrix can be obtained as a linear combination of two independent sensitivity vectors:

$$ {\mathbf{s}}_i(t)={\lambda}_{ij}(t){\mathbf{s}}_j(t)+{\lambda}_{il}(t){\mathbf{s}}_l(t). $$

(8.12)

This means that two independent sensitivity vectors determine the tangent plane of the manifold belonging to point C (see Fig. 8.4d).

The relation between the dimension of the manifold and the rank of the sensitivity matrix was also discovered later by Ren and Pope (2006). They suggested that the minimum dimension of the attracting manifold can be determined by the investigation of the sensitivity matrices.

If local similarity exists among the sensitivity vectors, then any sensitivity vector can be obtained by multiplying another nonzero sensitivity vector with a nonzero scalar:

$$ {\mathbf{s}}_i(t)={\lambda}_{ij}(t){\mathbf{s}}_j(t), $$

(8.13)

where s _i (t) and s _j (t) are the sensitivity vectors at a given time. This means that local similarity implies the correlation of the elements of vectors s _i (t) and s _j (t). The correlation of the elements of two vectors can be calculated (Zádor et al. 2004) by the following equation:

$$ {\tilde{\rho}}_{\mathbf{xy}}=\frac{\mathbf{xy}}{\left\Vert \mathbf{x}\right\Vert \kern0.5em \left\Vert \mathbf{y}\right\Vert }, $$

(8.14)

where ‖x‖ and ‖y‖ are the Euclidean lengths of the two vectors. The calculated value ${\tilde{\rho}}_{\mathbf{xy}}$ is the cosine of the angle θ _xy between the two vectors:

$$ {\tilde{\rho}}_{\mathbf{xy}}= \cos {\theta}_{\mathbf{xy}} $$

(8.15)

Thus, $-1\le \tilde{\rho}\le +1$, as expected from a correlation function.

The correlation function $\tilde{\rho}$ defined by Eq. (8.14) is a good measure (Zádor et al. 2004) of the similarity of the sensitivity functions. Two sensitivity functions are locally similar, if they point in the same direction (or exactly the opposite direction) in the space of parameters. In this case, the angle of the two vectors is 0° (or 180°) corresponding to the case of ${\tilde{\rho}}_{ij}=+1$ (or ${\tilde{\rho}}_{ij}=-1$). If the value of ${\tilde{\rho}}_{ij}$ is not close to ±1, then the sensitivity vectors are not locally similar.

The advantage of the correlation function (8.14) is that it characterises the local similarity of two sensitivity functions with a single number and it allows the investigation of the extent of local similarity as a function of the independent variable.

Zádor et al. (2004) investigated the local similarity of the sensitivity functions in a model of the adiabatic explosion of hydrogen–air mixtures at several equivalence ratios. The correlation of the sensitivity vector of H₂O with all other sensitivity vectors [belonging to temperature (T) and the concentrations of species H, O₂, H₂O₂, H, O, OH, HO₂ and N₂] was studied. The results, presented in Fig. 8.5, show the almost perfect local similarity of the sensitivity functions.

Fig. 8.5

Correlation between the sensitivity vector of the concentration of H₂O and the sensitivity vectors of the other variables as a function of temperature in a model of the adiabatic explosion of stoichiometric hydrogen–air mixtures (Zádor et al. 2004)

8.3 The Origin of Global Similarity

It was shown in Sect. 5.2 that the sensitivity functions can also be calculated via the Green function:

$$ \frac{\partial \mathbf{Y}}{\partial {x}_k}(t)={\displaystyle {\int}_0^t\mathbf{G}\left(t,{t}^{\prime}\right)\frac{\partial \mathbf{f}}{\partial {x}_k}\left({t}^{\prime}\right)\mathrm{d}{t}^{\prime }}. $$

(8.16)

Let us calculate the sensitivity of variables Y in time intervals (0, t ₁) and (t ₁, t) using the relationship G(t, t′) = G(t, t ₁)G(t ₁, t′):

$$ \frac{\partial \mathbf{Y}}{\partial {x}_k}(t)={\displaystyle {\int}_0^{t_1}\mathbf{G}\left(t,{t}_1\right)\mathbf{G}\left({t}_1,{t}^{\prime}\right)\frac{\partial \mathbf{f}}{\partial {x}_k}\left({t}^{\prime}\right)\mathrm{d}{t}^{\prime }}+{\displaystyle {\int}_{t_1}^t\mathbf{G}\left(t,{t}_1\right)\mathbf{G}\left({t}_1,{t}^{\prime}\right)\frac{\partial \mathbf{f}}{\partial {x}_k}\left({t}^{\prime}\right)\mathrm{d}{t}^{\prime }}. $$

(8.17)

The local sensitivity matrix can be calculated using the following initial value problem [see Eq. (5.7)]:

$$ \dot{\mathbf{S}}=\mathbf{J}\kern0.1em \mathbf{S}+\mathbf{F},\kern1em \mathbf{S}(0)=\mathbf{0}, $$

(8.18)

where J = ∂f/∂Y is the Jacobian and F = ∂f/∂x. Assume that ∂f/∂x _k  ≈ 0 in the time interval (t ₁, t), which means that this equation is pseudo-homogeneous in this time interval, that is, the second term on the right-hand side of Eq. (8.18) can be neglected compared to the first one. As a consequence, the second term on the right-hand side of Eq. (8.17) is also negligible compared to the first one. The matrix G(t, t ₁) is not a function of variable t′; therefore, for any t > t ₁

$$ \frac{\partial \mathbf{Y}}{\partial {x}_k}(t)=\mathbf{G}\left(t,{t}_1\right){\displaystyle {\int}_0^{t_1}\mathbf{G}\left({t}_1,{t}^{\prime}\right)\frac{\partial \mathbf{f}}{\partial {x}_k}\left({t}^{\prime}\right)\mathrm{d}{t}^{\prime }=\mathbf{G}\left(t,{t}_1\right)}\frac{\partial \mathbf{Y}}{\partial {x}_k}\left({t}_1\right). $$

(8.19)

The sensitivity of variable Y _i with respect to parameter x _k can be calculated in the following way:

$$ \frac{\partial {Y}_i}{\partial {x}_k}(t)={\displaystyle \sum_{j=1}^N{g}_{ij}\left(t,{t}_1\right)\frac{\partial {Y}_j}{\partial {x}_k}\left({t}_1\right)}. $$

(8.20)

If the sensitivity functions are locally similar at time t ₁, then the ratios of any two sensitivity coefficients are independent of the selection of the modified parameter. Let us select another variable Y _h and substitute Eq. (8.2) that defines the local similarity into Eq. (8.20):

$$ \frac{\partial {Y}_i}{\partial {x}_k}(t)=\frac{\partial {Y}_h}{\partial {x}_k}\left({t}_1\right){\displaystyle \sum_{j=1}^N{g}_{ij}\left(t,{t}_1\right)}{\lambda}_{jh}\left({t}_1\right), $$

(8.21)

$$ \left(\frac{\partial {Y}_i}{\partial {x}_k}(t)\right)/\left(\frac{\partial {Y}_h}{\partial {x}_k}\left({t}_1\right)\right)={\displaystyle \sum_{j=1}^N{g}_{ij}\left(t,{t}_1\right)}{\lambda}_{jh}\left({t}_1\right). $$

(8.22)

A similar equation can be obtained for parameter x _m :

$$ \left(\frac{\partial {Y}_i}{\partial {x}_m}(t)\right)/\left(\frac{\partial {Y}_h}{\partial {x}_m}\left({t}_1\right)\right)={\displaystyle \sum_{j=1}^N{g}_{ij}\left(t,{t}_1\right)}{\lambda}_{jh}\left({t}_1\right). $$

(8.23)

The right-hand sides of Eqs. (8.22) and (8.23) are identical, and the combination of these two equations yields

$$ \frac{\frac{\partial {Y}_i}{\partial {x}_k}(t)}{\frac{\partial {Y}_i}{\partial {x}_m}(t)}=\frac{\frac{\partial {Y}_h}{\partial {x}_k}\left({t}_1\right)}{\frac{\partial {Y}_h}{\partial {x}_m}\left({t}_1\right)}={\mu}_{km}. $$

(8.24)

Equation (8.24) shows that the ratio of two sensitivity coefficients at any time t > t ₁ is independent of the selection of the model result Y _i and time. Therefore, the corresponding sensitivity functions are globally similar. The meaning of Eqs. (8.16) to (8.24) can be summarised as follows. If the sensitivity differential equations are pseudo-homogeneous in the time interval (t ₁, t ₂ ) and the sensitivity coefficients are locally similar at time t ₁, then the sensitivity functions are globally similar in the time interval (t ₁, t ₂). The ratio μ _km is independent of the selection of model output Y _i and therefore Eq. (8.21) implies the presence of global and local similarity at the same time .

The proof above is based on the derivation of Vajda and Rabitz (1992), which was generalised by Zsély et al. (2003) for an arbitrary number of variables. The main difference between the two derivations is that Vajda and Rabitz assumed that one of the variables is dominant. If variable Y _h is dominant, then

$$ {g}_{ih}\left(t,{t}_1\right)\frac{\partial {Y}_h}{\partial {x}_k}\left({t}_1\right)\gg {\displaystyle \sum_{j=1,j\ne \mathrm{h}}^{N+1}{g}_{ij}\left(t,{t}_1\right)\frac{\partial {Y}_j}{\partial {x}_k}\left({t}_1\right)}. $$

(8.25)

This means that all terms but the one belonging to the dominant variable can be neglected in Eq. (8.16):

$$ \frac{\partial {Y}_i}{\partial {x}_k}(t)={g}_{ih}\left(t,{t}_1\right)\frac{\partial {Y}_h}{\partial {x}_k}\left({t}_1\right). $$

(8.26)

If the derivation is also applied for parameter x _m , then the combination of the two equations yields again Eq. (8.21):

$$ \frac{\frac{\partial {Y}_i}{\partial {x}_k}(t)}{\frac{\partial {Y}_i}{\partial {x}_m}(t)}=\frac{\frac{\partial {Y}_h}{\partial {x}_k}\left({t}_1\right)}{\frac{\partial {Y}_h}{\partial {x}_m}\left({t}_1\right)}={\mu}_{km}. $$

(8.27)

Thus, the result of this derivation is identical to the previous one.

According to the reasoning in Zsély et al. (2003), if in the time interval (t ₁, t ₂ ), the system of sensitivity differential equations is pseudo-homogeneous and local similarity is present, then the sensitivity functions are globally similar. According to the derivation of Vajda and Rabitz (1992), if in the time interval (t ₁, t ₂ ) the system of sensitivity differential equations is pseudo-homogeneous and one of the variables is dominant, then the sensitivity functions are locally and globally similar. Vajda and Rabitz considered that temperature is a dominant variable in ignition systems. Zsély et al. (2003) investigated the reason behind the global similarity of sensitivity functions for simulations of the adiabatic explosion of hydrogen–air mixtures. Figure 8.6 shows that in the region of global similarity, the inhomogeneous term of the sensitivity differential equation is negligible compared to the homogeneous term. That means that the sensitivity system of differential equations (8.18) is pseudo-homogeneous. They also demonstrated that in this system, none of the variables are dominant.

Fig. 8.6

The inhomogeneous term on the right-hand side of differential equation (8.18) is much smaller than the homogeneous term between temperatures 900 K and 2,000 K in a model of the adiabatic explosion of hydrogen–air mixtures. The ratio of these two terms is near zero in this region of temperature. Adapted with permission from Zsély et al. (2003). Copyright (2003) American Chemical Society

Derivation of the condition of global similarity for spatially one-dimensional, stationary systems is similar, but not identical. Due to causality, in temporal systems, parameter changes affect only later events. In 1D reaction–diffusion systems, a parameter change may modify the concentrations in both spatial directions. The adaption of the derivation above to reaction–diffusion systems is discussed in the article of Zsély et al. (2003).

8.4 Similarity of the Sensitivity Functions of Biological Models

Rabitz et al. in their first articles assumed (Reuven et al. 1986; Smooke et al. 1988; Rabitz and Smooke 1988; Vajda et al. 1990; Vajda and Rabitz 1992; Mishra et al. 1994) that the similarity of sensitivity functions is characteristic for flame models. Zsély et al. (Zsély and Turányi 2003; Zsély et al. 2003, 2005; Zádor et al. 2004) also found the similarity of sensitivity functions for models of homogeneous explosions for several chemical systems. More recently, the similarity of sensitivity functions was detected in several biological models. Lovrics et al. (2008), for example, found such similarities in the Chen et al. (2000) model of the cell cycle of budding yeast. Danis and Turányi (2011) found such similarities in the Rao et al. (2004) model of the chemotaxis of bacteria E. coli and B. subtilis. In the following, the results of Lovrics et al. will be discussed in detail.

The cell cycle of budding yeast (Saccharomyces cerevisiae) is the best understood among the eukaryotes. The main events during a cell cycle are the duplication of the DNA content, the division of the nucleus, the migration of the nuclei towards opposite corners of the cell and the splitting of the cell. The cell cycle is a highly regulated process, since one event (like the duplication of the DNA) has to end before the start of the next process (e.g. spindle formation). The Cdk (cyclin-dependent protein kinase) molecules regulate DNA synthesis, bud formation, the separation of the nucleus and the separation of the cells. At first, the new cell is just growing (phase G1), and the next phase is DNA synthesis (phase S/G2). Finally, two identical nuclei and then two new cells are formed (mitosis, phase M).

The Chen model of the budding yeast cell cycle (Chen et al. 2000) consists of a system of ordinary differential equations with 13 variables and coupled algebraic equations. One of the variables is the mass of the cell that increases exponentially between two cell divisions. Three of the variables define the state of the cell, whilst the other nine variables are the concentrations of nine proteins. The Chen model is basically a reaction kinetic model, since the core of the model describes the synthesis and interactions of proteins. The model has 73 parameters.

Lovrics et al. (2006) carried out a timescale and dimension analysis of the Chen model as presented in Fig. 6.8. The cell mass continuously grows, and therefore, one of the real parts of the eigenvalues Re(λ _i ) of the Jacobian is always positive. The other Re(λ _i ) eigenvalues are usually negative, except for during certain time domains of the cell cycle. If at least one of the other Re(λ _i ) eigenvalues is positive (the time domain is indicated by the grey shading in Fig. 6.8), then the dynamical dimension of the model is increasing; otherwise, it is decreasing. Lovrics et al. (2006) gave a detailed explanation as to the biological background of these grey excitation periods .

Timescale and dimension analysis revealed that during several time domains of the cell cycle, the dynamic dimension of the model is low and that during the cell cycle excitation and relaxation periods, it alternates between higher and lower dimensions. According to Sect. 8.3, these two features together may trigger the global similarity of the sensitivity functions. Therefore, Lovrics et al. investigated the sensitivity functions of the Chen model (2008) .

Figure 8.7 shows the sensitivity functions of the concentration of protein Cln2 during a full cycle. Similar functions were obtained for the sensitivity functions of all other variables. It is clear that the sensitivity functions of Cln2 usually increase in the excitation (grey) periods. The reason is that a parameter change after a certain time causes a shift in the values of variables, and this shift becomes amplified, thus increasing the sensitivity functions. In the relaxation (white) periods, all Re(λ _i ) eigenvalues are negative; therefore, the difference between the original and the perturbed solution decreases and the sensitivity functions tend to zero.

Fig. 8.7

Sensitivity functions of enzyme Cln2 (Lovrics et al. 2008). The excitation periods are denoted by grey shading. Time zero is the time of cell division. The sensitivity functions are labelled with the names of the parameters; these parameter names are identified in the article of Chen et al. (2000)

Figure 8.7 demonstrates that the shapes of some sensitivity functions are similar to those of others. Obviously, several groups of sensitivity functions can be separated so that each group contains functions of a similar shape. Within each group, any sensitivity function can be obtained by multiplying any other function by a positive or negative scalar, that is, the sensitivity functions are globally similar.

The sensitivity functions were sorted in the following way. First, each function was divided by its maximum with the result that similar functions almost coincided. Figure 8.8 shows such normalised sensitivity functions for the species Cln2. It is clear that most of the functions follow either the shape indicated by the solid line (“shape A”) or that indicated by the dotted line (“shape B”). For the model simulating the explosion of hydrogen–air mixtures, all sensitivity functions were globally similar. On the other hand, for the cell cycle model, most of the sensitivity functions can be sorted into one of the two groups, but several functions do not follow either of these two shapes.

Fig. 8.8

Sensitivity functions of enzyme Cln2 normalised to unit maximum. The thick solid line indicates 10 coinciding functions having shape A, whilst the dotted line shows 38 coinciding functions having shape B. The shapes of other 9 sensitivity functions (thin solid line) are not similar to either shapes A or B (Lovrics et al. 2008)

Similar sorting of all 73 sensitivity functions belonging to each of the 13 variables was automated using cluster analysis. The shapes of the sensitivity functions were investigated between two cell divisions, that is, in the time interval [t ₁, t ₂]. As discussed above, the sensitivity functions were normalised first to unit maximum:

$$ {\overset{\frown }{s}}_{ik}(t)={s}_{ik}(t)/ \max \left|{s}_{ik}(t)\right|. $$

(8.28)

Next, the integral of the square of the difference of two sensitivity functions was calculated during the interval [t ₁, t ₂]. Mathematically speaking, the L ² distance of the normalised sensitivity functions was calculated:

$$ {C}_i\left(k,l\right)={\displaystyle \underset{t_1}{\overset{t_2}{\int }}{\left({{\overset{\frown }{s}}_i}_k(t)-{\overset{\frown }{s}}_{il}(t)\right)}^2\mathrm{d}t}, $$

(8.29)

C _i (k, l) is a non-negative value that shows the distance of the shapes of two sensitivity functions belonging to variable i and parameters k and l. If there is a perfect global similarity between the two sensitivity functions, then C _i (k, l) = 0. Non-similar sensitivity functions are related to large C _i (k, l) values. Values of C _i (k, l) can be arranged into a matrix C _i , and this distance matrix can be investigated using cluster analysis.

Cluster analysis (Everitt et al. 2001) is a tool for grouping various objects on the basis of their distance in a multidimensional space. In chemistry, cluster analysis is used for the interpretation of analytical results. For example, in food or drink samples, the concentrations of many chemicals are measured, and the question is which of the samples are similar on the basis of the analytical results. The first step is always the transformation of the raw measurement data into a distance matrix. The general features of a distance matrix are that the diagonal elements are zero (everything is at zero distance from itself), all matrix elements are non-negative (negative distance cannot be interpreted) and the matrix is symmetrical (to and from distances are identical). It is clear that the distance matrix defined by Eq. (8.29) fulfils these requirements.

One of the cluster analysis methods is the agglomerative method (Everitt et al. 2001). Using this approach, a distance threshold parameter is continuously increased. At the first step, the two nearest objects are identified, and at this stage, only their distance is below the threshold. These two objects are united and the location of the unified object is the arithmetic mean of the coordinates. By increasing the threshold further and further, objects are united until only one object remains. The similarity of the objects is indicated by the order of the aggregations.

Figure 8.9 shows the result of cluster analysis for the grouping of sensitivity functions of the enzyme Sic1T. The two main groups are labelled as 30 and 15. The cluster analysis also identifies two groups (labelled X) containing functions that are not similar to the of groups 30 or 15, but show some qualitative similarity to these functions. The fifth group found contains constant zero sensitivity functions (group 0). The corresponding parameters in this group therefore have no effect on the calculated concentration of Sic1T.

Fig. 8.9

Grouping of the sensitivity functions of the enzyme Sic1T using cluster analysis. The two main shapes are labelled with 30 and 15; the shape of many sensitivity functions is not globally similar (label X) to any of these. Sensitivity functions with label 0 are constantly zero

The local similarity of the sensitivity functions was also investigated in this study. Not all parameters exhibited local similarity, but a local similarity group did exist that was composed of parameters kasbf, kisbf′, esbfn3, BCK0, CLN3MAX, Dn3 and Jn3. The correlation between the sensitivity vectors of all species with those of species Cln2 was investigated using Eq. (8.14), where only parameters in the above group were included. Figure 8.10 shows that the calculated cos θ is close to ±1 for all pairs of sensitivity vectors, confirming the presence of local similarity.

Fig. 8.10

The correlation of the sensitivity vector of enzyme Cln2 with the sensitivity vectors of all other variables of the cell cycle model . The investigated parameters were the following: kasbf, kisbf′, esbfn3, BCK0, CLN3MAX, Dn3 and Jn3. It is clear that cos θ is close to ±1 during most of the time period of the simulated cycle and for most variables, indicating that these sensitivity vectors are locally similar (Lovrics et al. 2008)

8.5 The Importance of the Similarity of Sensitivity Functions

At the start of the chapter, we suggested that in nonlinear models, we might expect that each parameter plays a different role in driving the predicted outputs. However, we have demonstrated through examples that there are many cases when sensitivity functions are globally similar. A consequence of the global similarity of sensitivity functions is that the effect of changing one of several parameters can be counterbalanced by changing a different sensitive parameter. This means that by modifying a second parameter, the temporal (or spatial) profile of all variables can be shifted back to the original trajectory. If the global similarity relation is valid for the sensitivity functions of only some of the variables, then only these concentration profiles can be shifted back by changing the appropriate parameters.

Zsély et al. (2003) performed numerical experiments to investigate this consequence of global similarity. Initially the concentration profiles were calculated for simulations of the adiabatic explosion of a stoichiometric hydrogen–air mixture using a nominal parameter set based on the values recommended by Baulch et al. (2005). Local sensitivity analysis was then used to select those parameters with the largest influence on the simulated species concentrations based on a study of A-factors for the reaction rate coefficients. Five reactions were selected as dominating the influence on the calculated concentrations. At the next stage, the A-factors of four reaction steps (O₂ + H + M → HO₂ + M, H + HO₂ → H₂ + O₂, O₂ + H → OH + O, H₂O + H → H₂ + OH) were increased by 1 % and the concentration–time curves were recalculated. When the A-factor of the fifth reaction step (H + HO₂ → 2OH) was also increased by 0.5 %, then the concentration–time profiles of all species returned to the original trajectories. Figure 8.11 shows the concentration profiles of species H and H₂O obtained using the original mechanism, the modified mechanism, and after tuning the A-factor of the fifth reaction.

Fig. 8.11

Calculated mass fraction–time profiles of species H and H₂O for the simulation of the adiabatic explosion of hydrogen–air mixtures. Solid line: profiles calculated using the original parameter set. Dashed line: calculated concentrations when the rate parameters of four important reactions are changed. Dotted line, usually not visible under the solid line: calculated concentrations when the rate parameter of a fifth reaction is also changed in an optimal way. Adapted with permission from Zsély et al. (2003). Copyright (2003) American Chemical Society

Lovrics et al. (2008) carried out a similar numerical experiment for the cell cycle model discussed above . The calculations indicated that the sensitivity functions of the parameters kisbf′ and BCK0 are globally similar for all variables. The ratio of the maxima of the two sensitivity functions was −1.10/1.22. Both were important parameters, i.e. a small change of any of the two, significantly changed the concentration–time curves. When parameter kisbf′ was increased by 10 %, then the calculated concentration profile of species Cln2 changed significantly. Subsequently the value of BCK0 was increased by 22 %, resulting in almost identical Cln2 curves when compared to the original model. As expected, all concentration–time curves were almost identical after the dual parameter changes (see Figs. 8.12 and 8.13).

Fig. 8.12

Calculated concentration profiles of protein Cln2 in the original model (solid line), when parameter kisbf′ is increased by 10 % (×signs) and when parameters kisbf′ and BCK0 are increased simultaneously by 10 % and 22 %, respectively (dots) (Lovrics et al. 2008)

Fig. 8.13

The simulated concentration–time curves of all proteins during a cell cycle (solid lines) and the curves simulated by a modified model (×legends) when parameters kisbf′ and BCK0 were increased by 10 % and 22 %, respectively (Lovrics et al. 2008)

There are important implications of this type of behaviour for the development of models. The main aim of empirical models is the accurate description of experimental observations. The parameters of these models may not have any physical meaning and have usually been derived by fitting to limited sets of experimental observations. If the sensitivity functions of such a model are globally similar, this means that several parameter sets may give an equivalent description of the same experimental data. If the model is to be applied only under conditions where the original fitting was achieved, then this may not present too many problems. However, if the aim is to develop a model which can be extrapolated to conditions where no experimental data exists, then problems could arise. Therefore, the parameter values that have been fitted or tuned under limited sets of conditions may not be able to be extrapolated to new situations. Ideally for a model to be general, it should be able to be extrapolated, and therefore, more physically based approaches to model development are becoming common as opposed to purely empirical models.

Physical models contain parameters that are developed at a more fundamental level and are thought to have “real” physical meaning. They are often derived from different sources, e.g. experimental measurements that attempt to isolate a single parameter, or theoretical calculations (see Chap. 3). A physical model is usually considered to be “validated” if the model reproduces all experimental data within their uncertainty limits across a wide range of values of the independent variables. Assuming there is perfect agreement between the model simulations and experimental data, we may be tempted to interpret the parameters of the “validated” model as physically correct values. However, the results of the numerical experiments above show that if one or some of the parameter values in a physical model are incorrect, this may be disguised by setting other parameters to incorrect values. If the errors are perfectly balanced, the model may still reproduce all experimental data quite well. Therefore, it is dangerous to determine rate coefficients within a complex mechanism, by fitting one or several rate parameters to experimental data, whilst fixing the other parameters within the model at their literature values, when in fact these fixed parameters may be uncertain to varying degrees (see Sect. 5.6.1). Small inaccuracies in the fixed values may result in large deviations in the fitted values, whilst the model still describes the experimental data well. This could be one reason why complex chemical kinetic models suggested by different authors provide descriptions of experimental data with similar accuracy, even though the applied rate coefficients are very different. Parameters of globally similar sensitivity functions are in a kind of cooperative relationship, since if the value of one parameter is changed, then its effect can be compensated by an appropriate change in the other parameter. The identification of such cooperative parameters promotes a better understanding of the model.

The comments above are valid for all types of models. However, there are some aspects of the global similarity of sensitivity functions that are especially interesting and important for biological models. Gutenkunst et al. (2007) highlighted that many systems biology models have sensitivity coefficients of similar magnitude and fitting all these parameters simultaneously to the experimental data results in unrealistically large parameter uncertainties. They investigated 17 published models and called such sensitivities as “universally sloppy parameter sensitivities”. Gutenkunst et al. identified that the main reason for this behaviour is that only parameter groups, and not the individual parameters, influence the model solution in most systems biology models. Note that such parameter groups can be a result of global similarity and that these parameter groups can be identified by the principal component analysis of the local sensitivity matrix (see Sect. 5.3).

A general feature of living organisms is that the error of a part of an organism can often be compensated for by another part. Evolution has promoted the emergence of such features, and hence, this error-correcting feature of living organisms is general and is present in the anatomy of several organs (Wagner 2013). Also, most regulating mechanisms contain parallel pathways. In this way, a failure of one pathway can be compensated for by a backup system. There are also similar parallel pathways in cell cycle regulation, and if one route is eliminated by a mutation, the other pathways may take over its role. However, this may not account for all error correction mechanisms.

The global similarity of sensitivity functions indicates the possibility for a novel error correction mechanism. A change of activity of an important enzyme can be fully compensated for by the change of activity of another enzyme, thereby restoring the concentration profiles of important species at all times. This error correction mechanism can be used not only once, but unlimited times, since later there are further possibilities for small adjustments. This feature is quantitatively different to those of other error correction mechanisms when, e.g., backup parallel pathways are used. The groups of enzymes that can be partners in this process can be identified by the inspection of the sensitivity functions of detailed chemical kinetics (systems biology) models of biological systems with implications for the treatment of disease.

The cause of some diseases is that the parameters of certain chemical reactions become different from those parameters which are characteristic for a healthy body. A possible aim of treatment is to restore the original parameters using medical drugs, but this can be difficult in some cases. However, if parameters which are globally similar to the original parameters are changed using drug therapy, then healthy functioning could be restored in a different way. This second option offers wider possibilities, and the rates of other biological processes can be influenced in an easier way. Therefore, an emerging trend in the pharmaceutical industry is to apply drug therapies to fix not the direct cause of the disease, but to restore healthy functioning in an indirect way. As more and more detailed models are developed for biological systems, the investigation of similarities in the sensitivity functions may provide a theoretical background for this new approach to the development of medical drugs.

The similarity of sensitivity functions may also have a role in genetic error correction. Let us assume that in a biochemical regulatory system, the protein concentration profiles have been refined by evolution and are nearly ideal for a given task. However, errors may occur during DNA replication, for example, resulting in lower enzyme activity. This error can either be lethal or can result in damaged functioning of the cell in the surviving organism. In the latter case, a further mutation may correct the previous error. It is very unlikely that the next mutation exactly corrects the functioning of the same enzyme. However, if the sensitivity functions of the regulatory system are globally similar, then within a group of enzymes, the activity change of any other enzyme may correct the functioning of the regulation. If a second mutation of this type yields a fit cell, it is evolutionary advantageous, and therefore, such a correction may remain permanent.

It is clear from these discussions that the development of detailed models of biochemical processes and the investigation of their sensitivity relationships may have important applications in improving our understanding of disease and in developing treatments. Not surprisingly, therefore, a great detail of effort is being invested into the development of systems biology models with increasing levels of detail as discussed in Chap. 3. The types of behaviour indicated by the models discussed in this chapter indicate that this will be a fruitful area of research.