Model Equations: Restoration of Equivalent Characteristics

Abstract

In the case called “grey box” in Sect. 5.2, a researcher has partial knowledge about the structure of model equations ${{\mathbf{x}}_{n + 1}} = {\mathbf{f}}({{\mathbf{x}}_n},{\mathbf{c}})$ or ${{{\mathrm{d}}{\mathbf{x}}} \mathord{\left/ {\vphantom {{{\mathrm{d}}{\mathbf{x}}} {{\mathrm{d}}t}}} \right. \kern-\nulldelimiterspace} {{\mathrm{d}}t}} = {\mathbf{f}}({\mathbf{x}},{\mathbf{c}})$. More concretely, some components of the function f are unknown. Then, the problem gets more complicated, than just parameter estimation discussed in Chap. 8, and more interesting from a practical viewpoint.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

In the case called “grey box” in Sect. 5.2, a researcher has partial knowledge about the structure of model equations ${{\mathbf{x}}_{n + 1}} = {\mathbf{f}}({{\mathbf{x}}_n},{\mathbf{c}})$ or ${{{\mathrm{d}}{\mathbf{x}}} \mathord{\left/ {\vphantom {{{\mathrm{d}}{\mathbf{x}}} {{\mathrm{d}}t}}} \right. \kern-\nulldelimiterspace} {{\mathrm{d}}t}} = {\mathbf{f}}({\mathbf{x}},{\mathbf{c}})$. More concretely, some components of the function f are unknown. Then, the problem gets more complicated, than just parameter estimation discussed in Chap. 8, and more interesting from a practical viewpoint.

To illustrate its peculiarities, let us consider modelling of a harmonically driven non-linear dissipative oscillator described by the equations

$$\begin{array}{l} {{{\mathrm{d}}{x_1}} \mathord{\left/ {\vphantom{{{\mathrm{d}}{x_1}}{{\mathrm{d}}t}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}t}}={x_2}, \\ {{{\mathrm{d}}{x_2}} \mathord{\left/ {\vphantom{{{\mathrm{d}}{x_2}}{{\mathrm{d}}t}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}t}}=-{\gamma_0}{x_2}+F({x_1})+{A_0}\,\mathrm{cos(}{\omega_0}t+{\phi_0}), \\\end{array}$$

((9.1))

where $\gamma_0, A_0, \omega_0, \phi_0$ are parameters and F is a non-linear restoring force, whose form is unknown. Let the variable x ₁ be an observable, i.e. $\eta=x_1$. Note that the function F is only a component of the entire dynamical system (9.1): the function F together with the other terms in the right-hand side specifies the phase velocity field. In practice, F is a characteristic of the object, which makes clear physical sense and can be of significant interest by itself. Its values may be unavailable for direct measurements due to experimental conditions, i.e. it may be impossible to get experimental data points directly on the plane $(x_1, F)$. However, information about the function F is contained in the time series, since F influences the dynamics. One can “extract” the values of F indirectly, i.e. via construction of an empirical model whose structure includes a model function corresponding to F. Namely, one should construct a model in the form

$$\begin{array}{l} {{{\mathrm{d}}{x_1}} \mathord{\left/ {\vphantom{{{\mathrm{d}}{x_1}}{{\mathrm{d}}t}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}t}}={x_2}, \\ {{{\mathrm{d}}{x_2}} \mathord{\left/ {\vphantom{{{\mathrm{d}}{x_2}}{{\mathrm{d}}t}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}t}}=-\gamma{x_2}+f({x_1},{\mathbf{c}})+A\,{\mathrm{cos(}}\omega t+\phi),\\\end{array}$$

((9.2))

where $f(x_{1},\textbf{c})$ should approximate F. Approximation of a one-variable function is a much more feasible task in practice, than a general problem of multivariable function approximation arising in “black box” reconstruction (Chap. 10). If one manages to get a “good” model, the ideas behind the model structure (9.2) are validated and the characteristic F is restored in the form $f(x_{1},\hat{\textbf{c}})$. We stress that it can be the only way to get the characteristic F.

Due to the importance of information about characteristics of non-linear elements inaccessible for direct measurements, we call the considered modelling problem “restoration of equivalent characteristics”. Components of a model function f can make different physical sense: a restoring force, non-linear friction, etc. Opportunities to extract such information arise if physical sense is introduced into a model structure in advance. More often, this is achieved with differential equations, since many laws of nature are formulated in such a form.

As in Chap. 8, the models considered here determine either a dependency “a future state $\textbf{x}_{n+1}$ versus a current state x _n ” or “a phase velocity ${\mathrm{ d}}\textbf{x}/{\mathrm{ d}}t$ versus a state x”. The difference from Chap. 8 is that one must specify a functional form of the characteristics to be restored before the stage of parameter estimation (Sect. 9.1). Hence, it gets more important to optimise a model structure (Sect. 9.2) and even to select it in a specific way for a certain object (Sect. 9.3) or a class of objects (Sect. 9.4).

1 Restoration Procedure and Peculiarities of the Problem

1.1 Discrete Maps

Let an original be a one-dimensional map $x_{n+1}=F(x_{n})$, where an observable is $\eta_{n}=x_{n}$. Let the dimension of the system be known and the form of the function F unknown.

A model is constructed as a one-dimensional map $x_{n+1}=f(x_{n},\textbf{c})$. For this simple example, data points on the plane $(\eta_{n},\eta_{n+1})$ represent the plot of F. One should just select the form of $f(x,\textbf{c})$ and find the values of c so as to approximate the data points in the best way (Fig. 9.1). Only the entire function $f(x,\hat{\textbf{c}})$ makes physical sense, rather than each single parameter, that is typical under the “grey box” setting.

Fig. 9.1

Construction of a model map: finding a dependence of the next value of an observable on the previous one from experimental data (circles)

The problem is almost the same as in Chap. 7, see, e.g., Fig. 7.1b. The difference is that the quantities $(\eta_{n},\eta_{n+1})$ are shown along the coordinate axes rather than the quantities $(t,\eta)$. Therefore, one can use the techniques discussed in Sect. 7.2.1 by replacing the pair of quantities $(t,\eta)$ by $(\eta_{n},\eta_{n+1})$. Thus, it is convenient to use a low-order algebraic polynomial or cubic splines (Sect. 7.2.4) to approximate a one-variable function shown in Fig. 9.1. Parameters can be estimated with the ordinary LS technique (8.3).

If a dynamical noise is present in the original map as $x_{n+1}=F(x_{n})+\xi_{n}$, nothing changes in the model construction procedure. A measurement noise, i.e. $\eta_{n}=x_{n}+\zeta_{n}$, makes the parameter estimation more complicated (Sect. 8.1.2). If its level is low, the ordinary LS technique is still suitable. For higher levels of the measurement noise, it would be desirable to use more sophisticated techniques (Sects. 8.1.2 and 8.2.1), but under the “grey box” setting, a model typically contains many parameters to be estimated, which makes the use of those techniques much more troublesome.

1.2 Ordinary Differential Equations

To describe complex motions, including chaotic ones, one uses non-linear model ODEs ${\mathrm{ d}}\textbf{x}/{\mathrm{ d}}t=\textbf{f}(\textbf{x,c})$ with at least three dynamical variables. Some components of the velocity field f can be unknown as in the examples (9.1) and (9.2). Those “equivalent characteristics” are built into the model structure and one can find them via the construction of the entire model. Let us consider some details.

The first case is when all the dynamical variables x _k are observed: $\eta_{k}(t_{i})=x_{k}(t_{i})+\zeta_{k}(t_{i}),\ k=1,\ldots,D$. To construct a model, one approximates a dependence of the derivative ${\mathrm{ d}}x_{k}(t_{i})/{\mathrm{ d}}t$ on $\textbf{x}(t_{i})$ with a function $f_{k}(\textbf{x,c}_{k})$ for each k. The values of ${\mathrm{ d}}x_{k}(t_{i})/{\mathrm{ d}}t$ are usually obtained from the observed data $\eta_{k}(t_{i})$ via numerical differentiation (Sect. 7.4.2). Let us denote their estimates ${\mathrm{ d}}\hat{x}_{k}(t_{i})/{\mathrm{ d}}t$. “Smoothed” values of the dynamical variables $\hat{x}_{k}(t_{i})$ emerge as a by-product of the differentiation procedure. From the values $\hat{x}_{k}(t_{i}),\ {\mathrm{ d}}\hat{x}_{k}(t_{i})/{\mathrm{ d}}t$, one estimates model parameters c _k with the ordinary LS technique:

$$S({{\textbf{c}}_k})=\sum\limits_i{{{\left({{{{\textrm{d}}{{\hat x}_k}({t_i})} \mathord{\left/ {\vphantom{{{\textrm{d}}{{\hat x}_k}({t_i})}{{\textrm{d}}t}}} \right. \kern-\nulldelimiterspace}{{\textrm{d}}t}}-{f_k}(\hat{\textbf{x}}({t_i}),{{\textbf{c}}_k})} \right)}^2}} \to{\textrm{min}}, k=1,\ldots,D.$$

((9.3))

The functions $f_{k}(\textbf{x,c}_{k})$ contain, in particular, sought equivalent characteristics.

The second typical case is when one observes a single dynamical variable: $\eta(t_{i})=x_{1}(t_{i})+\zeta(t_{i})$. The dimension D of the system is known. Successful modelling gets more probable if the dynamical equations for the original take the standard form (3.27), where the components of the state vector x are successive derivatives of the variable x ₁. A model is constructed in the corresponding form

$${{{{\mathrm{d}}^D}x(t)} \mathord{\left/ {\vphantom{{{{\mathrm{d}}^D}x(t)}{{\mathrm{d}}{t^D}}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}{t^D}}}=f(x(t),{{\,{\mathrm{d}}x(t)} \mathord{\left/ {\vphantom{{\,{\mathrm{d}}x(t)}{{\mathrm{d}}t}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}t}},\,{{{{\mathrm{d}}^{D-1}}x(t)} \mathord{\left/ {\vphantom{{{{\mathrm{d}}^{D-1}}x(t)}{{\mathrm{d}}{t^{D-1}}}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}{t^{D-1}}}},{\mathbf{c}}).$$

((9.4))

Firstly, one gets the time series of $\hat{x},\hat{x}^{(1)},\hat{x}^{(2)},\ldots,\hat{x}^{(D-1)}$, where superscript denotes an order of the derivative, via numerical differentiation of the observable $\eta(t_{i})$. Parameters of the function f, which includes equivalent characteristics to be restored, are estimated with the ordinary LS technique:

$$S({\textbf{c}})=\sum\limits_i{{{\left({{{\hat x}^{(D)}}({t_i})-f(\hat x({t_i}),{{\hat x}^{(1)}}({t_i}),\ldots,{{\hat x}^{(D-1)}}({t_i}),{\textbf{c}})} \right)}^2}} \to{\textrm{min}}.$$

((9.5))

The described techniques perform well under sufficiently low levels of the measurement noise. Moreover, the results are more reliable if the functions f _k in Eq. (9.3) and f in Eq. (9.5) depend on the parameters in a linear way, i.e. represent pseudo-linear models (Sect. 7.2.4). For higher noise levels, the modelling gets much more difficult, since numerical differentiation amplifies any noise, especially when $D>1$ derivatives are computed. Then, the ordinary LS technique becomes unsuitable, while the use of more sophisticated techniques (Sect. 8.2.1) in the case of multidimensional models with many unknown parameters is also unrealistic.

1.3 Stochastic Differential Equations

A more general idea, which is seemingly advantageous in the case of a multiplicative dynamical noise, is suggested in Friedrich et al. (2000) and Siegert et al. (1998). It is based on the estimation of parameters in the Fokker – Planck equation for a considered non-linear system, i.e. the estimation of the drift and diffusion coefficients (Sects. 4.3 and Sect. 4.5). Thus, let an object of modelling be given by the Langevin equations

$${{{\mathrm{d}}{x_k}}\mathord{\left/ {\vphantom{{{\mathrm{d}}{x_k}}{{\mathrm{d}}t}}}\right. \kern-\nulldelimiterspace}{{\mathrm{d}}t}}={F_k}({\mathbf{x}})+{G_k}({\mathbf{x}}){\xi_k}(t), k=1,\ldots,D,$$

where independent zero-mean white noises $\xi_{k}(t)$ have auto-covariance functions $\langle\xi_{k}(t)\xi_{k}(t^{\prime})\rangle=\delta(t-t^{\prime})$. One assumes that the dimension D and noise properties are known, and all D state variables are observed. Only concrete functional forms of $F_{k}(\textbf{x}),\ G_{k}(\textbf{x})$ are unknown so that these functions are to be determined from a time series.

Let us consider the case of $D=1$ for simplicity of notations, i.e. the system ${\mathrm{ d}}x/{\mathrm{ d}}t=F(x)+G(x)\xi(t)$. Recall that the Fokker – Planck equation (4.8) is defined as

$$\frac{{\partial p(x,t)}} {{\partial t}}=-\frac{\partial} {{\partial x}}\left({{c_1}(x,t)p(x,t)}\right)+\frac{1} {2}\frac{{{\partial^2}}} {{\partial{x^2}}}\left({{c_2}(x,t)p(x,t)}\right) ,$$

where the drift coefficient is

$${c_1}(x,t)=\mathop{{\mathrm{lim}}}\limits_{\tau\to 0}\frac{1} {\tau}\int\limits_{-\infty}^\infty{(x^{\prime}-x)p(x^{\prime},t+\tau |x,t){\mathrm{d}}x^{\prime}}$$

and the diffusion coefficient is

$${c_2}(x,t)=\mathop{{\mathrm{lim}}}\limits_{\tau\to 0}\frac{1} {\tau}\int\limits_{-\infty}^\infty{{{(x^{\prime}-x)}^2}p(x^{\prime},t+\tau |x,t){\mathrm{d}}x^{\prime}}$$

indent The functions F, G are related to these coefficients as

$${c_1}(x)=F(x)+\frac{1} {2}\frac{{{\mathrm{d}}G(x)}} {{{\mathrm{d}}x}}G(x)$$

and $c_{2}(x)=G^{2}(x)$ (Sect. 4.3). If $c_{1}(x),c_{2}(x)$ are known, the functions F, G can be restored from them (e.g., if one requires positivity of $G(x)$). Moreover, to answer many questions, it is possible to use the Fokker – Planck equation directly, rather than the original stochastic DE with the functions F, G.

As one can see from the above definitions, $c_{1}(x)$ and $c_{2}(x)$ are directly related to the conditional mean and conditional variance of the next value of x, given the current value of x. The conditional mean and variance can be estimated from data just as the sample mean and sample variance (Sect. 2.2.1) over all observed states close to a given state x. Having the estimates of the conditional mean and variance for different intervals τ (the smallest possible value of τ being equal to the sampling interval Δt), one can estimate the limits $\tau\to 0$ by extrapolation (Friedrich et al., 2000). Thereby, the estimates $\hat{c}_{1}(x)$ and $\hat{c}_{2}(x)$ are obtained. They are reliable at a given state x if an observed orbit passes near this state many times. Thus, the estimates $\hat{c}_{1}(x)$ and $\hat{c}_{2}(x)$ are more accurate for the often visited regions in the state space. They are poorly defined for “rarely populated” regions.

The estimates $\hat{c}_{1}(x)$ and $\hat{c}_{2}(x)$ are obtained in a non-parametric form (just as tables of numerical values). However, one may approximate the obtained dependencies $\hat{c}_{1}(x)$ and $\hat{c}_{2}(x)$ with any smooth functions if necessary, e.g. with a polynomial (Friedrich et al., 2000). The obtained functions $\hat{c}_{1}(x)$ and $\hat{c}_{2}(x)$ can also be considered as (non-linear) characteristics of the system under study. Deriving the estimates of the functions F and G, entering the original stochastic equation, from the estimates $\hat{c}_{1}(x)$ and $\hat{c}_{2}(x)$ is possible under some conditions on the function G assuring uniqueness of the relationship. Getting the functions F and G is of a specific practical interest if they have a clearer physical interpretation compared to the coefficients $\hat{c}_{1}(x)$ and $\hat{c}_{2}(x)$.

Several examples of successful applications of the approach to numerically simulated time series, electronic experiments and physiological data, as well as a detailed discussion are given in Friedrich et al. (2000), Ragwitz (2001); and Siegert et al. (1998). The approach directly applies if there is no measurement noise. Its generalisation to the case of measurement noise is presented in Siefert et al. (2003).

2 Model Structure Optimisation

Model structure selection is as important for the restoration of equivalent characteristics as for the problem considered in Sect. 7.2.1. To choose a model size, e.g. a polynomial order, one can use the criteria described in Sect. 7.2.3. However, the question remains: How to choose small subset of function terms from a large set of basis functions to provide the best model of a given size?

Let us consider an efficient approach (Bezruchko et al., 2001a) with the example of the reconstruction of equations for the van der Pol – Toda oscillator

$$\begin{array}{l} {{{\mathrm{d}}{x_1}} \mathord{\left/ {\vphantom{{{\mathrm{d}}{x_1}}{{\mathrm{d}}t}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}t}}={x_2}, \\ {{{\mathrm{d}}{x_2}} \mathord{\left/ {\vphantom{{{\mathrm{d}}{x_2}}{{\mathrm{d}}t}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}t}}=(1-x_1^2){x_2}-1+{{\mathrm{e}}^{-{x_1}}}. \\\end{array}$$

((9.6))

A time series of the observable is $\eta=x_{1}$ is supposed to be available. The corresponding phase orbit, containing a transient process, is shown in Fig. 9.2. A model is constructed in the form

$$\begin{array}{l} {{{\mathrm{d}}{x_1}} \mathord{\left/ {\vphantom{{{\mathrm{d}}{x_1}}{{\mathrm{d}}t}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}t}}={x_2}, \\ {{{\mathrm{d}}{x_2}} \mathord{\left/ {\vphantom{{{\mathrm{d}}{x_2}}{{\mathrm{d}}t}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}t}}=f({x_1},{x_2},{\mathbf{c}}), \\\end{array}$$

((9.7))

where

$$f({x_1},{x_2},{\mathbf{c}}) = \sum\limits_{i,j = 0}^K {{c_{i,j}}x_1^ix_2^j} ,\,\,i + j \leq K .$$

Fig. 9.2

A phase orbit of the van der Pol – Toda oscillator (9.6) which contains a transient process. The attractor is a limit cycle. The numbers are temporal indices of some data points in the time series, which is recorded with a sampling interval of 0.01. There are about 600 data points per basic period of oscillations

Many terms in the polynomial are “superfluous” since they have no analogues in Eq. (9.6), e.g. the terms $c_{0,0},\ c_{1,1}x_{1}x_{2},\ c_{0,2}x_{2}^{2}$ and others. Estimates of the coefficients corresponding to the superfluous terms can appear non-zero due to various errors and fluctuations. This circumstance can strongly reduce model quality. Thus, it is desirable to exclude the superfluous terms from the model equations.

Superfluous terms can be identified via the estimation of the model coefficients from different segments of the time series, i.e. from the data points lying in different domains of the phase space. The estimates of the “necessary” coefficients must not depend on a time series segment used. In contrast, the estimates of the coefficients corresponding to superfluous terms are expected to exhibit considerable variations. Such variations are stronger if a time series contains a transient process (Anishchenko et al., 1998; Bezruchko et al., 2001a), since the phase orbit then explores different domains in the phase space (Fig. 9.2).

We constructed the model (9.7) with a polynomial of a high-order K from subsequent time series segments of length W: $\{\eta_{(k-1)W+1},\ldots,\eta_{(k-1)W+W}\},\ k=1,2,\ldots,L$. Thus, we obtained a set of estimates for each coefficient $\hat{c}_{i,j}^{(k)}$ (Fig. 9.3a). A degree of stability of each coefficient estimate $\hat{\textit{c}}_{i,j}$ is defined as $\left|\left\langle\hat{{\textit{c}}}_{i,j}\right\rangle\right|\left/\sigma_{i,j}\right.$, where

$$\left\langle {\hat{\textit{{c}}} _{i,j}^{}} \right\rangle = \left( {{1 / L}} \right)\sum\limits_{k = 1}^L {\hat{{\textit{{c}}}} _{i,j}^{(k)}}$$

and

$${\sigma_{i,j}}=\sqrt{\left({{1\mathord{\left/ {\vphantom{1 L}}\right. \kern-\nulldelimiterspace} L}}\right)\sum\limits_{k=1}^L{{{\left({\hat c_{i,j}^{(k)}-\left\langle{{c_{i,j}}}\right\rangle}\right)}^2}}}.$$

Fig. 9.3

Construction of a model for the van der Pol – Toda oscillator (9.6) starting with a full two-variable polynomial of order $K=7$ in Eq. (9.7) from time series segments of length $W=2000$ data points: (a) estimates of the coefficients, corresponding to the indicated terms, versus the starting time instant of a time series segment (in units of sampling interval), the thick lines; (b) the model error versus the number of excluded terms, it is minimal for 20 terms excluded

The term corresponding to the least stable coefficient was excluded. The entire procedure was repeated for the simplified model structure. By repeating the exclusion procedure many times, we sequentially removed the “unstable terms”.

The procedure was stopped when the model quality did no longer improve. The criterion of quality was a minimum of the approximation error over a wide area V in the phase space (shown in Fig. 9.3b)

$${\sigma^2}=\iint\limits_V{{{\left\{{f({x_1},{x_2},{\hat{\textbf{c}}})-\left[{(1-x_1^2){x_2}-1+{{\mathrm{e}}^{-{x_1}}}}\right]}\right\}}^2}{\mathrm{d}}{x_1}\,{\mathrm{d}}{x_2}}.$$

After excluding 20 terms from an initial polynomial with $K=7$, the error was reduced by an order of magnitude in comparison with its starting value (Bezruchko et al., 2001a). At that, the final model reproduced much more accurately the dynamics of an object (including transient processes starting from different initial conditions) in the entire square region of the phase space shown in Fig. 9.2. This is because the described procedure of the model structure optimisation allows better generalisation of some essential dynamical features by reducing the danger of overfitting.

3 Equivalent Characteristics for Two Real-World Oscillators

In this section, we describe our results on the restoration of equivalent characteristics of oscillatory processes from the fields of physiology (Stoop et al., 2006) and electronics (Bezruchko et al., 1999a).

3.1 Physiological Oscillator

The cochlear amplifier is a fundamental, generally accepted concept in cochlear mechanics, having a large impact on our understanding of how hearing works. The concept, first brought forward by Gold in 1948 (Gold, 1948), posits that an active mechanical process improves the mechanical performance of the ear (Robles, and Ruggero, 2001). Until recently, the study of this amplifying process has been restricted to the ears of vertebrates, where the high complexity and the limited accessibility of the auditory system complicate the in situ investigation of the mechanisms involved. This limitation has hampered the validation of cochlear models that have been devised (Dallos et al., 1996; Kern and Stoop, 2003). The hearing organs of certain insects have recently been shown to exhibit signal-processing characteristics similar to the mammalian cochlea by using active amplification (Goepfert and Robert, 2001; 2003; Goepfert et al., 2005); the ears of these insects are able to actively amplify incoming stimuli, display a pronounced compressive non-linearity, exhibit power gain and generate self-sustained oscillations in the absence of sound. In both vertebrates and insects, the mechanism that promotes this amplification resides in the motility of auditory mechanosensory cells, i.e. vertebrate hair cells and insect chordotonal neurons. Both types of cells are developmentally derived by homologous genes and share similar transduction machineries, pointing to a common evolutionary origin (Boekhoff-Falk, 2005). In line with such an evolutionary scenario, it seems possible that the fundamental mechanism of active amplification in the ears of insects and vertebrates is also evolutionarily conserved (Robert and Goepfert, 2002).

Since insect’s hearing organs are located on the body surface, they are accessible to non-invasive examination. Moreover, because the external sound receiver is often directly coupled to the auditory sense cells, insect auditory systems can be expected to provide profound experimental and theoretical insights into the in situ mechanics of motile sense cells and their impact on the mechanical performance of the ear. Such information is technically relevant: providing natural examples of refined active sensors, the minuscule ears of insects promise inspiration for the design of nanoscale artificial analogues. Here, we present the results of modelling self-sustained oscillations of the antennal ear of the fruit fly Drosophila melanogaster with non-linear oscillator equation (9.7) and restoring its equivalent characteristics (Stoop et al., 2006).

In Drosophila, hearing is mediated by mechanosensory neurons that directly connect to an external sound receiver formed by the distal part of the antennas (Goepfert and Robert, 2000). These neurons actively modulate the receiver mechanics and, occasionally, give rise to self-sustained receiver oscillations (SOs). SOs occur spontaneously and are reliably induced by thoracic injection of dimethyl sulphoxide (DMSO), a local analgesic known to affect insect’s auditory transduction. The precise action of DMSO on the auditory neurons remains unclear. However, as spontaneous and DMSO-induced SOs are both physiologically vulnerable and display similar temporal patterns, the latter can be used to probe the nature of the amplification mechanism in the fly’s antennal ear (Goepfert and Robert, 2001). As revealed by measurements of the receiver vibrations (Fig. 9.4), about 20 min after the administration of DMSO, fully developed SOs are observed (Fig. 9.4b). They exhibit the temporal profile of relaxation oscillations with a characteristic frequency of about 100 Hz (Goepfert and Robert, 2003). About 10 min later, the SOs start to decrease in amplitude (Fig. 9.4c) and finally converge to a sinusoidal profile (Fig. 9.4d). The evoked SOs may last for up to 1–1.5 h.

Fig. 9.4

Self-sustained oscillations of the Drosophila hearing sensor (velocity measurements): (a) 10 min, (b) 20 min, (c) 30 min, (d) 34 min after DMSO injection

The temporal profile of these oscillations is reminiscent of limit cycle oscillations generated by the van der Pol oscillator

$$\begin{array}{l} {{{\mathrm{d}}{x_1}} \mathord{\left/ {\vphantom{{{\mathrm{d}}{x_1}}{{\mathrm{d}}t}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}t}}={x_2}, \\ {{{\mathrm{d}}{x_2}} \mathord{\left/ {\vphantom{{{\mathrm{d}}{x_2}}{{\mathrm{d}}t}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}t}}=(\mu-x_1^2){x_2}-{x_1}, \\\end{array}$$

((9.8))

where x ₁ is identified with the receiver’s vibrational position and the control parameter $\mu>0$ is slowly decreased in order to account for the changes in the SO shape during time. It is well known that at $\mu=0$, the van der Pol oscillator undergoes the Andronov – Hopf bifurcation; for $\mu>0$, a stable limit cycle emerges that can be interpreted as undamping (i.e. amplification). A more detailed examination of the experimental data reveals a pronounced asymmetry (see Fig. 9.4b) by comparing the onsets and extents of the upward and downward excursions within one period, which requires a more general model for the SO generation than the standard van der Pol system.

In order to capture this asymmetry, we construct a model in the form of the generalised van der Pol oscillator (9.7) with $f(x_{1},x_{2})=f_{1}(x_{1})x_{2}-f_{2}(x_{1})$, where $f_{1}(x_{1})$ and $f_{2}(x_{1})$ describe polynomials of the orders n and m, respectively. From the viewpoint of physics, $-f_{1}(x_{1})$ describes a non-linear and possibly negative friction, whereas $-f_{2}(x_{1})$ describes a non-linear restoring force. It is necessary to determine the orders n and m and polynomial coefficients that yield the optimal reproduction of the experimental data. One can expect that for a proper model, the polynomial orders n and m are unambiguously determined and only variations in the coefficients account for the observed changes in the SO temporal profile over time.

From the measurements with the sampling interval $\Delta t=0.08\,{\mathrm{ ms}}$, we are provided with a time series of the receiver’s vibration velocities v which is described by the variable x ₂ in the model. The values of the displacement and the acceleration are determined via numerical integration and differentiation, respectively. The latter is performed by applying the first-order Savitzky – Golay filter (Sect. 7.4.2). Quasi-stationary segments of the original data of lengths $N=4000$ data points (i.e. the duration of 0.32 s) are used for the model construction. In order to determine the optimal polynomial orders n and m, we use the criterion of the training error saturation (Sect. 7.2.3):

$${\hat \varepsilon^2}=\mathop{{\mathrm{min}}}\limits_{{{\mathbf{c}}_1},{{\mathbf{c}}_2}} \frac{1} {N}\sum\limits_{i=1}^N{{{\left({{{{\mathrm{d}}{{\hat x}_2}({t_i})} \mathord{\left/ {\vphantom{{{\mathrm{d}}{{\hat x}_2}({t_i})}{{\mathrm{d}}t}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}t}}-{f_1}({{\hat x}_1}({t_i}),{{\mathbf{c}}_1}){{\hat x}_2}({t_i})+{f_2}({{\hat x}_1}({t_i}),{{\mathbf{c}}_2})} \right)}^2}}.$$

((9.9))

The error $\hat{\varepsilon}$ saturates for $n=2$ and $m=5$ (Fig. 9.5). A further increase in n and m does not reduce $\hat{\varepsilon}$. The emergence of such a conspicuous saturation point is a rare case in practice and indicates that the model structure (9.7) faithfully reproduces the auditory data of Drosophila.

Fig. 9.5

Mean-squared error $\hat{\varepsilon}$ of the model fitting (9.9) showing a precipitous decay and saturation around the orders $n=2$ and $m=5$

A comparison between realisations of the model and the measurements corroborates the validity of our modelling. For the fully developed SOs (after 20 min, Fig. 9.6), the comparison reveals that the measured velocities are faithfully reproduced. This is further illustrated in Fig. 9.7, where the modelled and the measured data are compared on the phase plane $(x_{1},x_{2})$. Similar observations take place for the time series recorded 10, 30 and 34 min after DMSO injection, respectively.

Fig. 9.6

Diagnostic check: (a) experimentally measured receiver’s vibration velocity observed 20 min after DMSO injection, i.e. fully developed SOs; (b) a corresponding time series generated by the model (9.7) with $n=2$ and $m=5$

Fig. 9.7

Phase-space representation of the measured (dots) and model (the solid line) receiver vibrations in the case of fully developed SOs

The shapes of the polynomials $f_{1}(x_{1})$ and $f_{2}(x_{1})$ reflect the asymmetry of the observed receiver oscillations, specifically when the SOs are fully developed (Fig. 9.4b). The asymmetry of $f_{1}(x_{1})$ (Fig. 9.8a) and, in particular, $f_{2}(x_{1})$ (Fig. 9.8b) becomes effective at large displacements and may have its origin in structural-mechanical properties of the antenna. An enlightening interpretation of the amplification dynamics can be given for the behaviour around zero displacement position $x_{1}\approx 0$, where $f_{1}(x_{1})$ attains positive values for small displacements x ₁ (Fig. 9.8a). Since $-f_{1}(x_{1})$ represents friction, the inequality $f_{1}(x_{1})>0$ implies that energy is injected into the system. This is a characteristic feature of an active amplification process. Around $x_{1}=0$, the non-linear restoring force $-f_{2}(x_{1})$ and its first and second derivatives are relatively small. This implies that for small receiver displacements, virtually no restoring force is present. By means of the negative friction term, the system is thus easily driven out to relatively large amplitudes.

Fig. 9.8

Restored non-linear characteristics of the receiver for the fully developed SOs 20 min after DMSO injection: (a) the second-order polynomial $f_{1}(x_{1})$, which means a non-linear friction with the opposite sign and shows the undamping $f_{1}(x_{1})>0$ (see the dashed line); (b) the fifth-order polynomial $f_{2}(x_{1})$, which means a non-linear restoring force with the opposite sign and displays a noticeable asymmetry

In the course of time, i.e. with decreasing DMSO concentration, the non-linear contributions to friction and restoring force decay. In particular, the range, where the friction is negative, gradually decreases and finally vanishes in agreement with the observed reduction in SO amplitude (see Fig. 9.4). When the SO starts to disappear, the restoring force function $f_{2}(x_{1})$ gets approximately linear with a very small slope. At the same time, the friction term remains to be very small. As a consequence, weak stimuli will be sufficient to elicit considerable antennal vibrations. Although the amplifier has now returned into a stable state, where limit cycles do not occur, it remains very sensitive. Only small parameter variations are necessary in order to render the friction term negative and to lead to an amplification of incoming vibrations.

Thus, the model obtained captures several characteristics of the antennal ear oscillations, indicating that the empirical modelling may be useful for analysing the physics of the cochlear amplifier as well (Stoop et al., 2006).

3.2 Electronic Oscillator

An illustrative example from the field of electronics refers to the case when a chaotic motion of a non-linear system is successfully modelled under the “grey box” setting. An object is an RLC circuit with switched capacitors under an external sinusoidal driving with the amplitude U ₀ and the angular frequency ω ₀. Its scheme is presented in Fig. 9.9a, where K is an electronic key, a micro-scheme containing dozens of transistors and other passive elements, which is fed from a special source of direct voltage. Under small values of the voltage U on the capacity C ₁, linear oscillations in the circuit $\mathit{RLC}_{1}$ take place, since the resistance of the key is very high. When U reaches a threshold value $U_{\mathrm{ thr}}$, resistance of the key reduces abruptly so that it closes the circuit and connects the capacity C ₂. Reverse switching occurs at the value of U somewhat lower than $U_{\mathrm{ thr}}$, i.e. the key exhibits a hysteresis. It is the presence of non-linearity that leads to the possibility of chaotic oscillations in the circuit.

Fig. 9.9

Modelling of an electronic oscillator: (a) the scheme of an experimental set-up; (b) an experimental chaotic orbit on the plane “charge versus current” in dimensionless units (Bezruchko et al., 1999a); the values of I are measured with a 12-bit ADC at $\Delta t=4\,\upmu{\mathrm{ s}},\ C_{1}=0.1\,\upmu{\mathrm{ F}},\ C_{2}=4.4\,\upmu{\mathrm{ F}},\ L=20\,{\mathrm{ mH}},\ R=10\ \Omega,\ U_{\mathrm{ thr}}=-0.2\,{\mathrm{ V}}$, the driving period $T\approx 84.02\ \Delta t$, and $U_{0}=2.344\,{\mathrm{ V}}$; (c) an orbit of the reconstructed model (9.2) with the polynomial f of the ninth order; (d) a plot for the function – f (thick line) and an expected piecewise linear dependence (thin line)

A model of this system derived from Kirchhoff’s laws takes the form of the non-autonomous non-linear oscillator (9.2). The dimensionless variables read $t=t^{\prime}/\sqrt{\mathit{LC}_{1}}$ and $x_{1}=q/C_{2}|U_{\mathrm{ thr}}|$, where t ^′ is the physical time and q is the total charge on the capacities C ₁ and C ₂. It is expected that the original function F is piecewise linear due to such voltage – capacity characteristic of the non-linear element represented by the switched capacitors.

Experimental measurements provide us with a chaotic time series of the current I through the resistor R, which corresponds to the quantity x ₂ in Eq. (9.2). The time series of the variable x ₁ is obtained via numerical integration of the observed signal and the time series of the variable ${\mathrm{ d}}x_{2}/{\mathrm{ d}}t$ is obtained via numerical differentiation. We do not use information about the piecewise linear form of F, especially recalling that it is a theoretical approximation which ignores hysteresis of the key and other realistic features. Models are constructed in the form (9.2) with polynomials f of different orders K. Figure 9.9c shows the results for the best model with $K=9$, which reproduces well the observed chaotic motion illustrated in Fig. 9.9b. The theoretical piecewise linear “restoring force” and the model polynomial f coincide with a good accuracy in the observed range of the x ₁ values bounded by the dashed line in Fig. 9.9d. We note that without prior information about the model structure (9.2), it is impossible to get an adequate empirical model making physical sense (Bezruchko et al., 1999a).

This example illustrates restoring equivalent characteristics of a non-linear element via empirical modelling even in regimes of large amplitudes and chaos, where such characteristics may be inaccessible to direct measurements with ordinary tools. The empirical modelling has been successfully used to study dynamical characteristics of a ferroelectric capacitor (Hegger et al., 1998), semiconductor diodes (Sysoev et al., 2004) and optical fibre systems (Voss and Kurths, 1999).

4 Specific Choice of Model Structure

Uncertainty with respect to a model structure may not be so small as in the above examples. The “box” can be “dark grey” rather than “light grey” (Fig. 5.2) which makes empirical modelling much more difficult. However, in some cases, even small amount of a priori information along with a preliminary analysis of an observed time series can lead to a success if it is properly taken into account in a model structure. This is illustrated below with two wide classes of objects: systems under regular external driving and time-delay systems.

4.1 Systems Under Regular External Driving

If the presence of regular (periodic or quasi-periodic) driving is known a priori or assumed from a preliminary data analysis (e.g., strong discrete components in a power spectrum), then it is fruitful to include functions explicitly depending on time into model equations to describe such driving.

Thus, to describe an additive harmonic driving, one can reasonably use the model structure

$${{{{\mathrm{d}}^D}x} \mathord{\left/ {\vphantom{{{{\mathrm{d}}^D}x}{{\mathrm{d}}{t^D}}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}{t^D}}}=f(x,{{{\mathrm{d}}x} \mathord{\left/ {\vphantom{{{\mathrm{d}}x}{{\mathrm{d}}t}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}t}},\ldots,{{{{\mathrm{d}}^{D-1}}x} \mathord{\left/ {\vphantom{{{{\mathrm{d}}^{D-1}}x}{{\mathrm{d}}{t^{D-1}}}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}{t^{D-1}}}},{\mathbf{c}})+a\,{\mathrm{cos}}\,\omega t+b\,{\mathrm{sin}}\,\omega t$$

((9.10))

where x is an observable and f is an algebraic polynomial (Bezruchko and Smirnov, 2001; Bezruchko et al., 1999a). One may use smaller number of variables D in Eq. (9.10) than it would be necessary for the autonomous standard model (9.4). This circumstance determines advantages of the special model structure (9.10). The oscillator equation (9.2) is a particular case of Eq. (9.10) for $D=2$ and an incomplete two-variable polynomial.

Along with the choice of the model structure, one should overcome a specific technical problem. It consists of the estimation of the driving frequency ω, which enters the model equations (9.10) in a non-linear way. As usual, one makes a starting guess and solves a minimisation problem for a cost function like Eq. (9.5) with an iterative technique. However, the right-hand side of Eq. (9.10) is very sensitive with respect to ω at large t analogous to example (7.19) in Sect. 7.1.2. Therefore, the cost function S of the type (9.5) is sensitive with respect to ω for a large time series length N. It implies that the variance of the resulting estimator of the frequency ω rapidly decreases with the time series length if one manages to find the global minimum of S. Namely, the variance scales as $N^{-3}$ analogous to example (7.19). On the one hand, it gives an opportunity to determine ω to a very high accuracy. On the other hand, it is more difficult to find the global minimum since one needs a very lucky starting guess for ω. Taking it into account, one should carefully try multiple starting guesses for ω.

If ω is known a priori to a certain error, it is important to remember that for a very long time series, a small error in ω can lead to a bad description of the “true” driving with the corresponding terms in Eq. (9.10) due to the increase in “phase difference” between them over time (Bezruchko et al., 1999a). Then, the model structure (9.10) would get useless. Therefore, it is reasonable to consider the a priori known value as a starting guess for ω and determine the value of ω more accurately from the observation data. This discussion applies to other non-autonomous systems considered below.

For an arbitrary additive regular driving (complex periodic or quasi-periodic), a more appropriate model structure is

$${{{{\mathrm{d}}^D}x} \mathord{\left/ {\vphantom{{{{\mathrm{d}}^D}x}{{\mathrm{d}}{t^D}}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}{t^D}}}=f(x,{{{\mathrm{d}}x} \mathord{\left/ {\vphantom{{{\mathrm{d}}x}{{\mathrm{d}}t}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}t}}, \ldots,{{{{\mathrm{d}}^{D-1}}x} \mathord{\left/ {\vphantom{{{{\mathrm{d}}^{D-1}}x}{{\mathrm{d}}{t^{D-1}}}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}{t^{D-1}}}},{\mathbf{c}})+g(t,{\mathbf{c}}) ,$$

((9.11))

where the function $g(t)$ describes the driving and can be represented as a trigonometric polynomial (Smirnov et al., 2003):

$$g(t)=\sum\limits_{i=1}^k{\sum\limits_{j=1}^{{K_i}}{{c_{i,j}}\,{\mathrm{cos}}({\mathrm{j}}{\omega_i}t+{\phi_{i,j}})}} .$$

((9.12))

One can get good models with trigonometric polynomials of very high orders K _i , while approximation with a high-order algebraic polynomial typically leads to globally unstable model orbits.

To allow for multiplicative or even more complicated forms of driving, an explicit temporal dependence can be introduced into the coefficients of the polynomial f in Eq. (9.10) (Bezruchko and Smirnov, 2001). Thus, Fig. 9.10 shows an example of modelling of the non-autonomous Toda oscillator under combined harmonic driving

$${{{{\mathrm{d}}^2}x} \mathord{\left/ {\vphantom{{{{\mathrm{d}}^2}x}{{\mathrm{d}}{t^2}}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}{t^2}}}=-0.45{{{\mathrm{d}}x} \mathord{\left/ {\vphantom{{{\mathrm{d}}x}{{\mathrm{d}}t}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}t}}+(5+4\,{\mathrm{cos}}\,t)({{\mathrm{e}}^{-x}}-1)+7\,{\mathrm{sin}}\,t .$$

((9.13))

Fig. 9.10

Reconstruction of the equations for the non-autonomous Toda oscillator (9.13) from a time series of the variable x: (a) an original attractor; (b) an attractor of the non-autonomous polynomial model whose coefficients depend on time $(D=2,\ K=9)$; (c) an orbit of the standard model (9.4) with $D=4,\ K=6$

A model is constructed in the form (9.4) with an explicit temporal dependence introduced into all the coefficients of the polynomial f, i.e. one replaces all $c_{k}$ in the model structure with $c_{k}+a_{k}\cos\omega t+b_{k}\cos\omega t$. The best model is obtained for $D=2$ and $K=9$. Its phase orbit looks very similar to the original one (Fig. 9.10a,b). At that, the dimension of the standard model (9.4) without explicit temporal dependence should be not less than 3 to describe a chaotic regime. However, the standard model typically exhibits divergent orbits for $D>2$ (Fig. 9.10c).

Efficiency of the special choice of a model structure is demonstrated in a similar way for the periodic pulse driving, periodic driving with suharmonics and quasi-periodic driving in Smirnov et al. (2003).

4.2 Time-Delay Systems

Modelling of time-delay systems has been actively considered in the last years (Bunner et al., 1996, 2000; Bezruchko et al, 2001b; Horbelt et al, 2002; Ponomarenko and Prokhorov, 2004; Ponomarenko et al., 2005; Prokhorov et al., 2005; Voss and Kurths, 1997, 1999). Despite such systems being infinite-dimensional, many of the above techniques are suitable to model them with some technical complications, e.g., the multiple shooting approach (Horbelt et al., 2002). Some principal differences (Bunner et al., 2000) are beyond the scope of our discussion.

Let us consider an example where modelling of a time-delay system corresponds to the “grey box” setting and can be performed with the techniques similar to those described above. We deal with the systems of the form

$${\varepsilon_0}\,{{{\mathrm{d}}x(t)} \mathord{\left/ {\vphantom{{{\mathrm{d}}x(t)}{{\mathrm{d}}t}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}t}}=-x(t)+F(x(t-\tau)) ,$$

((9.14))

where an observable is $\eta=x$. Let us illustrate a modelling procedure with the reconstruction from a chaotic time realisation of the Ikeda equation (Fig. 9.11a):

$${\varepsilon_0}\,{{{\mathrm{d}}x(t)} \mathord{\left/ {\vphantom{{{\mathrm{d}}x(t)}{{\mathrm{d}}t}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}t}}=-x(t)+{\mu_0}\,{\mathrm{sin}}(x(t-{\tau_0})-{x_0}) ,$$

((9.15))

which describes the dynamics of a passive optical resonator.

Fig. 9.11

Reconstruction of a time-delay system: (a) a time realisation of the Ikeda equation (9.15) with $x_{0}=\pi/3,\ \varepsilon_{0}=1.0,\ \mu_{0}=20.0,\ \tau_{0}=2.0$; (b) the number of pairs of extrema $M(\tau)$ divided by the total number of extrema in the time series, $M_{\min}(\tau)=M(2.0)$; (c) a restored non-linear function. Numerical experiments with measurement noise show that modelling is successful for the ratio of the standard deviations of noise and signal up to 20%

Models are constructed in the form

$$\varepsilon \,{{{\mathrm{d}}x(t)} \mathord{\left/ {\vphantom{{{\mathrm{d}}x(t)}{{\mathrm{d}}t}}} \right. \kern-\nulldelimiterspace}{{\mathrm{d}}t}}=-x(t)+f(x(t-\tau),{\mathbf{c}}) .$$

((9.16))

Similar to the above examples, one may solve the minimisation problem $\sum\limits_{n}(\varepsilon{\mathrm{ d}}x(t_{n})/{\mathrm{ d}}t+x(t_{n})-f(x(t_{n}-\tau),\textbf{c}))^{2}\to\min$, where the response constant ε and the delay time τ are considered as additional unknown parameters (Bunner et al., 2000). However, there is a special efficient approach (Bezruchko et al., 2001b), which is based on the statistical analysis of the time intervals separating extrema in a time series of the time-delay system (9.14). It appears that the number of pairs of extrema M separated by a given interval τ exhibits a clear minimum as a function of τ at τ equal to the true delay time of the system (9.14), Fig. 9.11b. This observation gives an opportunity to estimate the delay time and diagnose that a system under study belongs to the class of time-delay systems (9.14). Having an estimate $\hat{\tau}\approx\tau_{0}$, one can assess a response characteristic ε by checking different trial values of ε and selecting such value $\hat{\varepsilon}$ for which experimental data points on the plane $(x(t-\hat{\tau}),\hat{\varepsilon}{\mathrm{ d}}x(t)/{\mathrm{ d}}t+x(t))$ lie on a smooth one-dimensional curve. This curve is a plot of the sought function f, which is an approximation to F. Figure 9.11c illustrates such a restoration of the “true” function F for the system (9.15). Having such a plot, one can find an approximating function f using an expansion in a certain functional basis or a special formula.

The described approach to the determination of the delay time and reconstruction of the entire equation can be extended to the delay differential equations of higher orders and to the systems with several delay times. It is parsimonious with respect to the computation time and not highly sensitive to the measurement noise (Ponomarenko et al., 2005; Prokhorov et al., 2005).

Thus, as illustrated by several examples in this chapter, special selection of model structure based on the preliminary analysis of data and some (even rather general) a priori information about an object under study can essentially improve an empirical model quality and make possible meaningful interpretations of the modelling results.