Keywords

1 Closed-Loop Turbulence Control—Applications and Challenges

Features of turbulent flows have a large effect on the performance of engineering applications, like ground, maritime or airborne transport and energy systems. In numerous studies, feedback control has been shown to change these features employing small modern actuators and corresponding sensors. Thereby the performance has been significantly improved. Examples include drag reduction of cars and trucks, lift increase of airplanes, gust mitigation of wind turbines and NOX reduction in combustors—just to name a few.

A key challenge for control design is the inherent nonlinearity of turbulence dynamics. For instance, excitation at a given frequency will affect also other frequencies. Dominant vortex shedding at one frequency may be mitigated by a low- or high-frequency actuation [1]. This frequency crosstalk is not accessible in any linear control framework and has many qualitatively different characteristics [2]. Yet, virtually all turbulence control experiments exploit frequency crosstalk [3].

This contribution addresses these nonlinear actuation mechanisms in three parts. First (Sect. 2), success stories of turbulence control are presented in which the nonlinear actuation mechanism has been modelled. Second (Sect. 3), we discuss machine learning control (MLC) and selected applications. MLC detects and exploits the winning actuation mechanisms in the experiment in an unsupervised model-free manner. Finally (Sect. 4), future developments of turbulence control are outlined.

2 Control with Human Learning—Building Blocks of Understanding

Myriad control design method have been proposed for flow control. These methods can be attributed to numerous categories: model-free, model-based with black, gray and white box models, and open-loop versus adaptive feedback versus in-time response—just to name a few. Yet, only few actuation mechanisms have been exploited in the vast majority of flow control applications and the chosen control design is typically a corollary to that mechanism:

  • Opposition control (direct, in-time): This mechanism is encapsulated in the simple example

    $$\begin{aligned} da/dt=a+b, \end{aligned}$$
    (1)

    where a represents the state with unstable fixed point \(a_s=0\) and b is the control. Obviously \(b=-2a\) stabilizes the plant. Several skin friction reductions [4] and Tollmien-Schlichting wave suppressions [5] are based on this mechanism: the wall-normal velocity is counteracted by a membrane or by suction/blowing.

  • Phasor control (direct, in-time): Let us consider following dynamical system coupling a self-amplified amplitude limited oscillator \((a_1,a_2)\) and a stable linear oscillator at 10-fold frequency \((a_3,a_4)\):

    $$\begin{aligned} \begin{array}{l@{\quad }l} da_1/dt =\sigma a_1-a_2, &{} da_3/dt =-0.1 a_3-10 a_4, \\ da_2/dt =\sigma a_2+a_1+b_1, &{} da_4/dt =-0.1 a_4+10 a_3 + b_3, \\ \sigma = 0.1-a_1^2-a_2^2-a_3^2-a_4^2-b_2. \end{array} \end{aligned}$$
    (2)

    Without control, \(b_1=b_2=b_3 \equiv 0\), the first oscillator converges to the limit cycle \(a_1^2+a_2^2=0.1\) with unit frequency, while the second one vanishes, \(a_3=a_4=0\). We will ignore the stable oscillator \((a_3,a_4)\) unless it is excited with \(b_3\). The first oscillator can be stabilized with phasor control \(b_1=-0.4 \>a_2\). This can be considered as opposition control for the evolution equation of the energy \(E=(a_1^2+a_2^2)/2\). Many cavity noise mitigations [6] and low-Reynolds number wake stabilizations [7, 8] belong to this category.

  • Constant forcing (indirect, open-loop): The first oscillator of (2) may be stabilized with \(b_2=0.2\). Physically, this corresponds to an enforced change of the baseflow towards a more stable regime. Such a stabilization has, for instance, been realized for mitigating vortex shedding behind high-lift airfoil via Coanda blowing [9].

  • Periodic forcing at high or low frequency (indirect, open-loop): The first oscillator of (2) can also be stabilized exploiting the frequency crosstalk with the stable one: Now \(b=\cos 10 t\) can be seen to excite the second oscillator which stabilizes the first one via \(\sigma \). Physically, this corresponds to baseflow change via an induced Reynolds stress at a new frequency, ideally exploiting a weakly damped instability. The actuation frequency may be higher or lower than the natural one. Such high- and low-frequency stabilization of vortex shedding has been observed numerous times for wakes, jets and shear-layers [1]. Figure 1 illustrates the drag reduction by low-frequency forcing of a D-shaped cylinder and high-frequency actuation of an Ahmed body.

Fig. 1
figure 1

The subfigures are adapted from [3, 11]

Flow visualization behind bluff bodies with significant drag reductions in wind-tunnel experiments. Left: D-shaped cylinder (\(Re_D=40,000\)) with symmetric low-frequency forcing [10]. The D-shaped body is indicated in gray, the red squares mark the location of the pressure sensors and the blue arrows indicated the employed zero-net-mass-flux actuators. The flow is visualized with smoke without forcing (top) and with forcing (bottom). Note the delayed vortex shedding by actuation. Right: Ahmed body (\(Re_H=3 \times 10^5\)) with high-frequency Coanda blowing at all four rearward edges [11]. The mean flow is displayed in the near-wake symmetry plane from PIV data without forcing (top) and with forcing (bottom). The top right figure indicates the displayed flow region (brown) and the Ahmed body (gray). Note the more streamlined form of the forced wake region (fluidic boat tailing).

Full size image

We shall not pause to list obvious variations, like a closed-loop destabilization in the last example. In addition, adaptive generalizations, e.g. extremum or slope seeking, may allow to adjust open- and closed-loop control to slowly varying operating conditions. Once, the actuation mechanism and corresponding model is identified, the nonlinear control design is typically straightforward. We refer to the literature for the Navier-Stokes based derivation of such models and examples, e.g. our review article [3] and our books [12, 13]. We also refer to exquisite reviews on specific topics, like linear control [6, 14, 15], wake control [16] and actuators [17].

The above discussion focusses on nonlinear dynamical models. An alternative approach follows Brockett’s idea of control design for the equivalent linearLiouville equation for the probability distribution [18]. A practical data-driven realization is based on clustering of snapshot data and the resulting Markov transition model [19, 20].

3 Control with Machine Learning—Often Faster, More Flexible and Better

The bald eagle can perform impressive flight maneuvers under gusty wind conditions with closed-loop control—yet without apparent knowledge of Navier-Stokes equations and control design. Nature has found another strategy: control design by evolution with smart trial and error. This approach has been pioneered by Rechenberg [21] and Schwefel [22] for shape optimization at the TU Berlin over 50 years ago. Nature’s way of control design is mimicked in the recently discovered Machine Learning Control (MLC). Control design is framed as regression problem of second kind: Find the control law \(\varvec{b}=\varvec{K} (\varvec{s}, t)\) which minimizes the cost function J, \( \varvec{K}^{\star } = \arg \min _{\varvec{K}} J \left[ \varvec{K} (\varvec{s},t) \right] \). Here, t represents time, \(\varvec{s}\) comprises the sensor signals and \(\varvec{b}\) analogously the actuation commands. The chosen form of the control law includes periodic excitation \(b=B \cos \Omega t\), multi-frequency forcing, sensor-based feedback \(\varvec{b}=\varvec{K}(\varvec{s})\) and combinations thereof.

Fig. 2
figure 2

Principle sketch of MLC using genetic programming. MLC consists of a fast inner feedback loop for testing the performance J and a slow outer learning loop for evolving the control law \(\varvec{K}\). After the learning phase, the best individual of the last generation is taken as control law

Full size image

Genetic programming [23] is chosen as a powerful regression technique for general nonlinear control laws of unknown structure. It is particularly suited for exploring a complex J landscape with several local minima. MLC with genetic programming as regression method consists of following steps (see also Fig. 2):

  • First generation: Let \(\varvec{b}=\varvec{K}_i^{(1)}\), \(i=1,\ldots ,N_i\) be random control laws, also called individuals. A typical population size is \(N_i=100\). These control laws are tested and graded in the plant \(J_i^{(1)}\). Without loss of generality, the individuals are sorted after testing, \(J_1^{(1)} \le J_2^{(1)} \le \ldots \le J_{N_i}^{(1)}\). The first step of genetic programming is a simple Monte-Carlo search.

  • Second generation: The second generation \(\varvec{b}=\varvec{K}_i^{(2)}\), \(i=1,\ldots ,N_i\) is created from the first by three stochastic genetic operations mimicking natural selection: Crossover of two individuals shall produce better individuals; mutations shall explore potentially new unpopulated minima; and replication shall memorize successful individuals. In addition, elitism copies the best \(N_e\) individuals directly in the new generation. Typically, only the best individual (\(N_e=1\)) is saved. The selection probabilities of crossover, mutation and replication are typically chosen to be \(P_c=0.7\), \(P_m=0.2\) and \(P_r=0.1\).

  • Next generations and termination: Analogously, more generations are produced until a convergence or another stop criterion is reached. In all experiments, convergence is reached before \(N_g=10\) generations. The best individual \(\varvec{K}_1^{N_g}\) of the last generation yields the desired MLC law.

Genetic programming has numerous parameters. Fortunately, a single set of parameters was found to produce the winning control in over a dozen different experiments [13]. In addition, the MLC laws have been found to be reproducible modulo small unavoidable uncertainty.

MLC has been studied in a number of experiments.

  • Mixing layer energetization (TUCOROM wind tunnel, France [24]). In this first MLC experiment, a turbulent mixing layer is forced upstream with 96 synchronously operated streamwise fluid jets (b) in the separating plate and monitored downstream with a vertical array of 24 hot-wire probes \(\varvec{s}\). The Reynolds number based on initial mixing layer thickness is 500, yielding a laminar boundary layer for the learning phase. The testing for off-design conditions is done at \(Re=2000\) with a turbulent boundary layer. The goal J is to increase the mixing layer width as measured by the hot-wire rag. The best periodic forcing yields a 55% increase of shear-layer width. MLC yields reproducibly a direct sensor feedback \(b=K(\varvec{s}^{\prime })\) which increases this width by 67% and at only 54% of cost of optimal periodic forcing. In addition, MLC is more robust to large changes of the freestream velocities as compared to the periodic benchmark. An important enabler of this control is the feedback of the velocity fluctuations to reduce sensitivity to slow baseflow changes. MLC converges after few generations with 100 individuals. The actuation mechanism is based on phase synchronization (phasor control). Despite the simple actuation mechanism, we could not identify a linear ERA-OKID model for the actuation response—even in a narrow frequency range: Sensor response could hardly be correlated with the actuation command.

  • Reduction of a circulation bubble behind a backward facing step (PMMH water tunnel, France [25]). In this MLC experiment, a flow over backward-facing step is forced upstream with slot actuator (blowing and suction) and monitored in the dead-water region with in-time PIV. The sensor signal is the size of the reverse flow region, which is ‘blind’ to the Kelvin-Helmholtz shedding phase. The Reynolds number based on step height is 1350 for the learning phase and is varied between 900 and 1800 for testing off-design conditions. The goal is to minimize the recirculation zone with an actuation penalty. MLC yields a direct sensor feedback \(b=K(s)\) which excites a low-frequency flapping mode. The cost is similar to an optimal periodic forcing exciting the Kelvin-Helmholtz shedding for the reference condition. However, MLC is much more robust for a range of (untested) oncoming velocities resulting in cumulative cost \(J=0.43\) (MLC) versus \(J=0.77\) for the periodic benchmark. \(J=1\) would correspond to the unforced flow at reference conditions. MLC is converged after 8 generations with 500 individuals. The actuation mechanism exploits frequency crosstalk with feedback destabilization of the flapping mode at low frequency.

  • Drag reduction of a car model (wind tunnel Beton, PPRIME, France [26]). The drag of a wall-mounted Ahmed body at \(Re_H=3 \times 10^5\) is reduced with 4 independent on/off Coanda jets at the rear edges (\(\varvec{b} \in R^4\)). The flow is sensed with 12 pressure sensors distributed over the rear side (\(\varvec{s} \in R^{12}\)). The optimal periodic forcing is found to be at high frequency and low duty cycle, yielding 19% drag reduction at an estimated actuation power of only 1/7 of the saved dragpower [11]. MLC rapidly converges with a population size \(N_i=50\) in \(N_g=5\) generations (250 runs). MLC depends on the assumed control law. Sensor-based feedback \(\varvec{b}=\varvec{K}(\overline{\varvec{s}},\varvec{s}^{\prime })\) reproduces high-frequency forcing with similar drag reduction. Note that the sensor signals have been decomposed into a short-term mean \(\overline{\varvec{s}}\) (not used by MLC) and a fluctuation \(\varvec{s}^{\prime }\). The performance is remarkable since the time delay from actuation to sensing is two (!) periods and since the signature of high-frequency actuation is below the noise level in most sensors. MLC chooses the only sensor with good high-frequency signal to noise level, i.e. performs sensor selection for the optimal control law. With the ansatz \(\varvec{b}=\varvec{K}(\varvec{s}^{\prime },h(t))\), h being a harmonic signal at optimal high-frequency forcing, MLC chooses the slightly better performing open-loop control. Given an open-loop multi-frequency ansatz \(\varvec{b}=\varvec{K}(\varvec{h}(t))\), MLC improves drag reduction by 22% with a 2-frequency forcing. The employed actuation mechanism is based on frequency crosstalk in all cases.

  • Separation mitigation of a turbulent boundary layer (wind tunnel, LML and PRISME, France [13, 27]). The separation of a turbulent boundary layer over a smooth ramp is mitigated by an upstream array of synchronously operated jets in two geometrically similar wind-tunnel experiments at different Reynolds numbers based on step height: 13,000 at LML and 130,000 at PRISME. The flow is monitored with skin-friction and pressure sensors. The goal is to mitigate the separation, as measured by a cost function J with actuation penalization. The optimal open-loop forcing laws are found to be constant blowing (LML) and periodic actuation (PRISME). The ansatz for the control laws is sensor feedback \(\varvec{b}=K(\varvec{s})\). The optimal MLC laws found intermittent forcing (LML) and high-frequency actuation (PRISME)—outperforming the optimized open-loop benchmark.

  • Other MLC studies. Applications to other plants with the author include:

    • Stabilization of a noisy unstable oscillator [13]. Here the optimal control solution is almost exactly reproduced using random filters.

    • Chaos maximization of a Lorenz system [28].

    • Stabilization of two coupled oscillators, Eq. (2) [13].

    • Stabilization of three nonlinearly coupled oscillators [26].

    • Energetization of a 2D mixing layer in a direct numerical simulation.

    • Lift-increase of a NACA0015 airfoil with an emulated plasma actuator in a direct numerical simulation. Intriguingly, MLC found intermittent actuation leading to a series of small vortices moving over the top side of the airfoil. This actuation increases lift beyond constant and periodic forcing while reducing the actuation power [Joint work with H. Fukumoto and A. Oyama].

    • Stabilization of a fluidic pinball in a wind-tunnel experiment [29].

    • Mixing increase behind a backward-facing step in a wind-tunnel experiment [31].

    • Drag reduction of a D-shaped cylinder in a wind-tunnel experiment [30].

    • Symmetrization of the Ahmed body wake to mitigate bimodal behavior. The problem and benchmark feedback controller are described in [32].

    In all cases, MLC has reproduced or outperformed the optimized benchmark control in some 1,000 test runs using the same parameters of genetic programming.

In the mentioned studies, we have used genetic programming as powerful regression method for arbitrary nonlinear control laws. MLC may also be performed with other regression techniques. Starting from an assumed linear control law, the gains may also be optimized with a simpler genetic algorithm [33]. A further simplification is achieved in the rare case that the optimal actuation for given flow state can be computed with a full-state simulation. In this case, the search for a mapping from known sensor signals to know actuation commands constitutes a regression problem of the first kind. A sensor-based feedback law may, for instance, be obtained with a neural network [34].

4 Conquering Terra Incognita—Paradigm Shifts by Machine Learning

Presented results indicate that methods of machine learning will dramatical change and accelerate progress in turbulence control in the coming years. Here are few expected future directions.

  • The control laws to be explored will go far beyond constant and periodic actuation (or small variations thereof). MLC makes it easy to explore control laws of the form

    $$ \varvec{b} = \varvec{K} ( \varvec{s}, \varvec{h}, \varvec{n}) $$

    where \(\varvec{h}\) comprises harmonic functions and \(\varvec{n}\) noise terms for stochastic forcing.

  • The traditional paradigm: ‘From understanding to modeling to control’ will largely be replaced by the new paradigm ‘From control to modeling to understanding’. The winning control tends to be too complex to be predicted by any model. Moreover, identified models will be much more powerful and predictive if they incorporate many different control laws.

  • Also modeling will be strongly affected. Modern data-driven regression techniques may allow to derive simple human-interpretable dynamical models from data. For instance, SINDy [35] has been shown to derive first-principle based generalized mean-field models—similar to (2)—from properly prepared simulation data. The traditional paradigm ‘From first principles to modes to dynamics’ is likely to be replaced by the new paradigm ‘From (controlled) dynamics to modes (or more general flow estimators) to a first-principles based understanding’. The old paradigm assumes, for instance, that the optimal actuation mechanism is correctly predicted. However, the mentioned MLC studies have, more oven than not, surprised with unexpected better mechanisms. We show the potential of the new paradigm for a transient wake flow [36].

  • As consequence of machine-learned models, we can expect many new qualitatively different nonlinear dynamics models enabling more a powerful control.

  • A key task of turbulence control includes to estimate achievable performance with control for a new configuration and to predict the right choice and location ofactuators. These are likely to remain a challenging topic in the first principles domain until the performance data of actuators in different configurations is becoming very rich.