1 Introduction

The most common imaging-based velocity measurement technique is particle image velocimetry (PIV), which computes velocity fields from the cross-correlation of consecutive tracer particle images. While a discussion of PIV is outside the scope of the present letter, an inherent property of PIV is the sparsity (or lower resolution) of the estimated velocity field due to the correlation-based analysis. For more details concerning PIV specifics, including advanced analysis approaches, the reader is referred to reference texts such as that by Raffel and coworkers (2018). An alternative to cross-correlation-based PIV algorithms is optical flow velocimetry (OFV), which extends the more general class of optical flow techniques from the computer vision community (e.g., Horn and Schunck 1981) to experimental fluid mechanics. Many advanced OFV methods have demonstrated an increase in accuracy and spatial resolution compared to state-of-the-art correlation-based PIV (Corpetti et al. 2002; Yuan et al. 2007; Kadri-Harouna et al. 2013; Héas et al. 2014; Chen et al. 2015; Cai et al. 2018; Schmidt and Sutton 2020).

The solution of the optical flow problem starts with the so-called conservation of intensity which is expressed as an advection equation

$$\begin{aligned} \frac{\partial I\left( \underline{x},t\right) }{\partial t} + \underline{v} \left( \underline{x},t \right) \cdot \nabla I \left( \underline{x},t \right) = 0, \end{aligned}$$
(1)

where \(I\left( \underline{x},t \right)\) is the image intensity and \(\underline{v} \left( \underline{x},t \right)\) is the fluid velocity. Previous works (Liu and Shen 2008; Schmidt and Sutton 2019) have shown that Eq. (1) is equivalent to the transport equation for a passive scalar in fluid mechanics if \(\underline{v}\) is the path-averaged fluid velocity within the imaging plane defined by the laser sheet, and it is assumed that there is no transport of tracer particles in or out of the laser sheet. The form of Eq. (1) also assumes that the projected fluid flow in the visualization plane is divergence-free, or equivalently, that flow divergence has no direct impact on the appearance of particle images. If the velocity is constant between images, Eq. (1) can be integrated in time to yield the displaced frame difference (DFD) equation

$$\begin{aligned} I_0 \left( \underline{x} \right) - I_1 \left( \underline{x} + \underline{v} \left( \underline{x} \right) {\Delta } t \right) = 0 . \end{aligned}$$
(2)

The majority of OFV algorithms solve Eq. (2) through a minimization problem that balances a data term, \(J_\mathrm {D}\), against a regularization term \(J_\mathrm {R}\) as

$$\begin{aligned} \hat{\underline{v}} = \underset{\underline{v}}{\mathrm {argmin}} \; \; J_\mathrm {D}\left( I_0,I_1,\underline{v} \right) + \lambda J_\mathrm {R}\left( \underline{v} \right) . \end{aligned}$$
(3)

In Eq. (3), \(J_\mathrm {D}\) penalizes the mismatch between a pair of images: \(I_0\) and the motion-compensated image, \(I_1\); the regularization term \(J_\mathrm {R}\) enforces smoothness on the estimated velocity field \(\underline{v}\); and \(\lambda\) is a scalar parameter that controls the relative influence of \(J_\mathrm {D}\) and \(J_\mathrm {R}\). The interested reader can find further information on the construction of \(J_\mathrm {D}\) and the solution to Eq. (3) in any of the OFV papers cited above. The concern of the present Letter is the form of the regularization term \(J_\mathrm {R}\), its physical interpretation, and its impact on the resulting velocity field.

2 Motivation

The choice of the regularization term \(J_\mathrm {R}\) in the variational formulation of the optical flow problem (Eq. (3)) is very important for providing the proper closure and smoothing for an estimation of a velocity field. It is noted that there are other classes of OFV methods that seek to reduce or remove the dependence on the regularization and use statistical methods involving stochastic transport and turbulence models to estimate the velocity field (Héas et al. 2014; Chen et al. 2015; Cai et al. 2018). While these formulations require less explicit regularization terms, they result in a considerably more complex form of Eq. (3) and are subject to the user’s choice of turbulence model, similar to the choice of subgrid model in large eddy simulations (LES). These implementations are beyond the scope of the present manuscript and will not be discussed further here.

Various forms of \(J_\mathrm {R}\) exist in the literature, but the majority are largely mathematical constructs whose primary purpose is to constrain/smooth the estimated velocity field. The most commonly used regularization is the original scheme (or similar variants) introduced by Horn and Schunck (1981), which is equivalent to first-order Tikhonov regularization:

$$\begin{aligned} J_\mathrm {R}^{\mathrm {HS}} = \int \limits _\varOmega ||\underline{\nabla }v_1 ||^2 + ||\underline{\nabla }v_2||^2 \mathrm {d}\underline{x} . \end{aligned}$$
(4)

As shown by Corpetti et al. (2006) the Horn and Schunck regularization penalizes the curl of a velocity field (i.e., vorticity), which is inappropriate for turbulent fluid flows since vorticity is a key characteristic that should be calculated accurately. Alternatively, Corpetti et al. (2002) proposed the second-order div-curl regularization because it forces the divergence and vorticity of a flow into coherent “blobs,” which is more physically sound for fluid flows:

$$\begin{aligned} J_\mathrm {R}^\text {d-c} = \int \limits _\varOmega ||\underline{\nabla }\left( \underline{\nabla } \cdot \underline{v} \right) ||^2 + ||\underline{\nabla }\left( \underline{\nabla } \times \underline{v} \right) ||^2 \mathrm {d}\underline{x} . \end{aligned}$$
(5)

This form of regularization has been shown to produce very good results by a number of researchers (Corpetti et al. 2002; Yuan et al. 2007; Schmidt and Sutton 2020).

It is clear that choosing \(J_\mathrm {R}\) is an opportunity to enforce fluid flow physics on the estimated velocity field, since \(J_\mathrm {D}\) is determined solely from the acquired images and includes no physical constraints on the estimated motion. While the second-order div-curl term leads to velocity field behavior that is qualitatively similar to that of real fluids, it has no firm basis in the equations which govern fluid motion, i.e., the Navier–Stokes equations, as it is based only on empirical observations of qualitative turbulent flow behavior. Methods such as the second-order div-curl and the penalization of the divergence proposed by Chen et al. (2015) solely focus on better preservation of vorticity with no particular constraint on the form of the regularization. Preservation of vorticity certainly is necessary for turbulent flows, but may not be sufficient to achieve optimal results. In the current work, we seek a form of \(J_\mathrm {R}\) that is directly motivated by the Navier–Stokes equations and hence is the best form of \(J_\mathrm {R}\) applicable for OFV in a broad range of fluid flows.

3 Regularization term

We begin by developing an intuitive understanding of the role of \(J_\mathrm {R}\) during the velocity estimation process. The discussion is generally applicable to all OFV methods, although it is particularly insightful in the context of multiresolution strategies as illustrated below. Equation (3) represents a balance of the data term \(J_\mathrm {D}\), which attempts to match images \(I_0\) and \(I_1\) as closely as possible with no constraints on the displacement (i.e., velocity) field, against a regularization \(J_\mathrm {R}\) which introduces constraints on the velocity field.

Wavelet-based optical flow velocimetry (wOFV) is one type of multiresolution optical flow analysis that will be used as an example (see Kadri-Harouna et al. 2013; Héas et al. 2014; Schmidt and Sutton 2020, 2019). In wOFV, finer-scale motions are permitted as more wavelet scales are successively included, and \(J_\mathrm {D}\) causes the estimated velocity field to fluctuate greatly over very small spatial scales. While the exact match of the images improves very slightly, the velocity field quickly becomes nonphysical, with much more energy contained at high wave numbers (fine scales) than is physically possible for a real fluid. While wavelet sparsity in the velocity field provides closure of the ill-posed inverse problem, it cannot adequately enforce realistic fluid physics on the velocity field. This is why \(J_\mathrm {R}\) becomes important in wOFV methods as it provides both closure and physical smoothness.

Overall, the role of \(J_\mathrm {R}\) in any OFV method is to impose smoothness (i.e., regularity) on the velocity field by penalizing strong fluctuations across fine spatial scales either explicitly for multiresolution approaches or implicitly for other methods. In real fluid flows, this same role, namely enforcing smoothness on the velocity field at fine scales, is carried out by viscosity. Therefore, the physically motivated form of \(J_\mathrm {R}\) that we seek should act on the estimated velocity field in Eq. (3) in the same way as viscosity acts on velocity in the Navier–Stokes equations. This is clearly seen in the solution of the optical flow problem in Eq. (3). The data term \(J_\mathrm {D}\) only is constrained by permissible motion from an advection equation (Eq. (1)), which is the same functional form as the advection operator of the momentum equation for constant density flows. To faithfully mimic fluid behavior, the transient and advective terms should be balanced by a diffusive term, which for velocity is the diffusion of momentum or viscosity. Thus, the remaining task is to determine the resulting form of \(J_\mathrm {R}\) that mimics the role of viscosity. It is important to note that we do not explicitly seek to solve the full Navier–Stokes equations using optical flow nor constrain the solution of \(\hat{\underline{v}}\) to satisfy the Navier–Stokes equations. Our objective only is to determine the form of \(J_\mathrm {R}\) that will enforce smoothness on \(\hat{\underline{v}}\) in the same way as viscosity acts to diffuse momentum in physical fluid flows, thus appropriately balancing the purely advective behavior of \(J_\mathrm {D}\).

3.1 Derivation

We begin with the compressible form of the Navier–Stokes equations. Equations (6) and (7) give the conservation of mass and momentum, respectively, for a flow with variable density, negligible body forces, and constant viscosity:

$$\begin{aligned}&\frac{\partial \rho }{\partial t}+\underline{\nabla } \cdot \left( \rho \underline{v} \right) = 0 \end{aligned}$$
(6)
$$\begin{aligned}&\frac{\partial \left( \rho \underline{v}\right) }{\partial t}+\underline{\nabla } \cdot \left( \rho \underline{v} \otimes \underline{v} \right) \nonumber \\&\quad = - \underline{\nabla }p + \mu \left( \underline{\nabla }^2\underline{v} + \frac{1}{3} \underline{\nabla }\left( \underline{\nabla } \cdot \underline{v} \right) \right) \end{aligned}$$
(7)

It is important to use the compressible form of the Navier–Stokes equations for two reasons. First, this form is more general and is more applicable to compressible and reacting flows that have variable density. Second, and more importantly, many velocimetry applications, including OFV methods, involve planar images, which are two-dimensional slices through three-dimensional flows. In this case, even if a flow has constant density, the projection of the velocity field in two dimensions is not divergence-free (Liu and Shen 2008). To see this effect, constant density is assumed for simplicity and Eq. (6) is split into its various components:

$$\begin{aligned} \left( \underline{\nabla } \cdot \underline{v} \right) _{\mathrm {12}} = \frac{\partial v_1}{\partial x_1} + \frac{\partial v_2}{\partial x_2} = - \frac{\partial v_3}{\partial x_3} \end{aligned}$$
(8)

Since no information about the out-of-plane component of velocity \(v_3\) nor its variation in the out-of-plane dimension \(x_3\) can be determined from a planar image, a divergence-free, three-dimensional flow can be treated equivalently as a two- or three-dimensional flow with divergence. This is due to the fact that the term \(\frac{\partial v_3}{\partial x_3}\) becomes arbitrary in the same manner as \(\left( \underline{v} \cdot \frac{\underline{\nabla }\rho }{\rho } \right)\) and \(\left( \frac{\partial v_3}{\partial x_3} + \underline{v} \cdot \frac{\underline{\nabla }\rho }{\rho } \right)\) in 2D and 3D flows with divergence, respectively (as \(\rho\) cannot be determined from planar imaging).

By inspection of Eq. (7), it is noted that \(J_\mathrm {R}\) will smooth the motion estimation in an equivalent manner as viscosity smooths an actual flow if it penalizes the quantity

$$\begin{aligned} \underline{\nabla }^2\underline{v} + \frac{1}{3} \underline{\nabla }\left( \underline{\nabla } \cdot \underline{v} \right) . \end{aligned}$$
(9)

A regularization term that penalizes the quantity shown in Eq. (9) specifically will enforce smoothness on the velocity field by reducing the magnitude of second derivatives of \(\underline{v}\), in a similar manner to the div-curl regularization term shown in Eq. (5). However, unlike the div-curl regularization term, the proposed viscosity-inspired regularization term will penalize the derivatives of \(\underline{v}\) and hence enforce smoothness, in the same way as viscosity imposes smoothness on a physical fluid flow according to the Navier–Stokes equations. Typically, \(J_\mathrm {R}\) penalizes the square of the \(L^2\) norm of a vector expression (for example, see Eqs. (4) and (5)) such that new regularization parameter takes on the form of

$$\begin{aligned} J_\mathrm {R}^\mu = \int \limits _\varOmega ||\underline{\nabla }^2\underline{v} + \frac{1}{3} \underline{\nabla }\left( \underline{\nabla } \cdot \underline{v} \right) ||^2 \mathrm {d}\underline{x} \end{aligned}$$
(10)

or in terms of partial derivatives:

$$\begin{aligned} \begin{aligned} J_\mathrm {R}^\mu&= \frac{1}{9} \int \limits _\varOmega \left( 4 \frac{\partial ^2 v_1}{\partial x_1^2} + 3 \frac{\partial ^2 v_1}{\partial x_2^2} + \frac{\partial ^2 v_2}{\partial x_1 \partial x_2} \right) ^2 \\&\quad +\left( 4 \frac{\partial ^2 v_2}{\partial x_2^2} + 3 \frac{\partial ^2 v_2}{\partial x_1^2} + \frac{\partial ^2 v_1}{\partial x_1 \partial x_2} \right) ^2 \mathrm {d}\underline{x}. \end{aligned} \end{aligned}$$
(11)

It is noted here that the quantity in Eq. (10) bears resemblance to the strict enforcement of Stokes flow in the fluid-based image registration method of Christensen et al. (1996). Image registration and optical flow are closely related inverse problems in image processing. Furthermore, if divergence-free conditions are assumed, the derived form of \(J_\mathrm {R}\) will take on the form of Laplacian regularization:

$$\begin{aligned} J_\mathrm {R}^{\mathrm {Lap}} = \int \limits _\varOmega ||\underline{\nabla }^2\underline{v} ||^2 \mathrm {d}\underline{x}. \end{aligned}$$
(12)

Laplacian penalization is widely used in image processing for filtering and smoothing Paris et al. (2011). To the authors’ knowledge, a Laplacian regularization scheme has not appeared in the OFV literature with the exception of the work by Kadri-Harouna et al. (2013), but no physical justification was given for doing so.

3.2 Implementation

For the remainder of this Letter, a wOFV framework will be used, although implementation of Eq. (10) in any OFV method should be possible with schemes likely similar to those used for second-order div-curl regularization Corpetti et al. (2002, 2006). The viscosity-inspired regularization term \(J_{\mathrm {R}}^\mu\) is implemented in a straightforward manner in wOFV methods without the need to develop a stable numerical scheme to handle the associated Euler–Lagrange equations. This is performed using the results of Beylkin (1992), as first implemented by Kadri-Harouna et al. (2013). As shown by Schmidt and Sutton (2020), the implementation can be performed efficiently using matrix multiplications by matrices \(N^{\left( n \right) }\), which enact differentiation in the wavelet domain according to the chosen wavelet basis. Using this approach, \(J_\mathrm {R}\) is derived in the context of wOFV as

$$\begin{aligned} \begin{aligned} J_\mathrm {R}^\mu&= \frac{1}{9} \underline{\psi }_1 : \left( 16 \underline{N}^{\left( 4 \right) } \underline{\psi }_1 \underline{N}^{\left( 0 \right) {\mathrm {T}}} + 25 \underline{N}^{\left( 2 \right) } \underline{\psi }_1 \underline{N}^{\left( 2 \right) {\mathrm {T}}} \right. \\&\quad + 9 \underline{N}^{\left( 0 \right) } \underline{\psi }_1 \underline{N}^{\left( 4 \right) {\mathrm {T}}} \\&\quad \left. +\,8 \underline{N}^{\left( 1 \right) } \underline{\psi }_2 \underline{N}^{\left( 3 \right) {\mathrm {T}}} + 6 \underline{N}^{\left( 3 \right) } \underline{\psi }_2 \underline{N}^{\left( 1 \right) {\mathrm {T}}} \right) \\&\quad + \frac{1}{9} \underline{\psi }_2 : \left( 16 \underline{N}^{\left( 0 \right) } \underline{\psi }_2 \underline{N}^{\left( 4 \right) {\mathrm {T}}} \right. \\&\quad +25 \underline{N}^{\left( 2 \right) } \underline{\psi }_2 \underline{N}^{\left( 2 \right) {\mathrm {T}}} + 9 \underline{N}^{\left( 4 \right) } \underline{\psi }_2 \underline{N}^{\left( 0 \right) {\mathrm {T}}} \\&\quad + 8\underline{N}^{\left( 3 \right) } \underline{\psi }_1 \underline{N}^{\left( 1 \right) {\mathrm {T}}} \\&\quad \left. +6 \underline{N}^{\left( 1 \right) } \underline{\psi }_1 \underline{N}^{\left( 3 \right) {\mathrm {T}}} \right) , \end{aligned} \end{aligned}$$
(13)

where \(\underline{\psi }_i\) is the wavelet transform of the ith component of the velocity field \(\underline{v}\) and  :  denotes the Frobenius inner product.

4 Sample results

Performance of the wOFV method utilizing the regularization scheme derived in this Letter (denoted \(J_{\mathrm {R}}^\mu\)) is evaluated using a hierarchy of simulated data. Four additional common regularization schemes, including the three identified above and a second-order Tikhonov regularization scheme previously used by the authors Schmidt and Sutton (2019), also are utilized and compared to results using \(J_{\mathrm {R}}^\mu\). First, the schemes are evaluated using the two-dimensional direct numerical simulation (DNS) of isotropic, incompressible turbulence from Carlier and Wieneke (2005). This simulation is commonly used in the OFV community to benchmark algorithm performance and has synthetic particle images associated with the velocity fields that can be processed with various methods. Further evaluation of the proposed regularization scheme involves 3D flows with divergence (either projected or physical and projected). First, we use a three-dimensional DNS of homogeneous isotropic turbulence with constant density (denoted 3D-NR). The DNS was constructed with Lagrangian flow tracers, which are used to create synthetic particle images using the approach described in Schmidt et al. (2019). Unlike the 2D DNS of Carlier and Wieneke, the 2D velocity fields extracted from the 3D DNS contain significant amounts of projected divergence in the 2D planes, even though the 3D flow is physically divergence-free. Finally, the regularization schemes are evaluated using a 3D DNS of a reacting non-premixed hydrogen–air system in a homogeneous isotropic turbulent flow (denoted 3D-R), which is described in further detail in Schmidt et al. (2019). As in the 3D-NR case, the DNS was constructed with Lagrangian flow tracers to create synthetic tracer particle images. The reacting flow has three-dimensional flow divergence in addition to projected 2D divergence. It is noted that the magnitude of the three-dimensional divergence is only 9% of the projected 2D divergence in this flow. This indicates that in many flows of interest the projected 2D divergence may be of greater importance when considering the effects of divergence in planar velocimetry applications.

The calculated 2D vorticity (\(\omega _3 = \frac{\partial {u_2}}{\partial {1}}-\frac{\partial {u_1}}{\partial {x_2}}\)) from the “true” DNS velocity fields at a single instance in time is shown for each flow in the left column of Fig. 1. For each of the three-dimensional simulations (3D-NR and 3D-R), only a sample plane from the full 3D domain is shown. The computed vorticity from the estimated velocity fields using wOFV with no explicit regularization term \(\left( \lambda = 0 \right)\) is shown in the center column, and with the new viscosity-based regularization in the right column. It is clear by comparing the vorticity fields across the columns that explicit regularization \(J_\mathrm {R}\) acts to smooth the flow computed from the data term \(J_\mathrm {D}\), removing nonphysical microscale fluctuations (i.e., noise) and generating velocity and vorticity fields are more faithful representations of the true fields.

Fig. 1
figure 1

Example 2D vorticity field calculated from the DNS velocity (left column); vorticity computed from wOFV velocity estimation with no explicit regularization (center column); and vorticity computed from wOFV velocity estimation with viscosity-based regularization (right column) for the two-dimensional, nonreacting (top), three-dimensional, nonreacting (center), and three-dimensional, reacting (bottom) flows

Full size image

The error in the estimated velocity fields using the various regularization methods is quantified using the normalized root-mean-square error, \(\varepsilon _v\), defined as

$$\begin{aligned} \varepsilon _v = \sqrt{\frac{\sum || \hat{\underline{v}}_i - \underline{v}_i ||^2}{\sum ||\underline{v}_i ||^2}} , \end{aligned}$$
(14)

where \(\hat{\underline{v}}\) is the estimated velocity and \(\underline{v}\) is the true velocity, defined as the DNS result for the synthetic data. Figure 2 shows \(\varepsilon _v\) as a function of the regularization parameter \(\lambda\) for the three instantaneous velocity fields used to calculate the 2D vorticity fields shown in Fig. 1. The calculated value of \(\varepsilon _v\) for a given value of \(\lambda\) is similar for all instantaneous velocity fields (and corresponding synthetic particle images), and thus only one is used to report the normalized error as a function of \(\lambda\). The error using the wOFV method with no explicit regularization term (\(\lambda = 0\)) and error using a commercial cross-correlation-based PIV algorithm (TSI Insight4G) are shown for comparison. The velocity results and associated error using the current PIV software are very similar to results using other commercial packages (e.g., LaVision DaVis).

Fig. 2
figure 2

Normalized RMS error (\(\varepsilon _v\)) as a function of \(\lambda\) for several regularization schemes as applied to a two-dimensional, nonreacting b three-dimensional, nonreacting, c three-dimensional, reacting flows

Full size image

The results show that the new viscosity-inspired regularization scheme produces the lowest error of the seven schemes used to estimate velocity. Although the absolute improvement in \(\varepsilon _v\) using \(J_{\mathrm {R}}^\mu\) is modest (\(\approx \, 3\)–\(4\%\)), (1) it achieves the lowest error across all three flows which show different flow physics (i.e., mixing, heat release, divergence) and topology (range of length/timescales) and (2) the error for \(J_{\mathrm {R}}^\mu\) using nonoptimal values of \(\lambda\) is still lower than the minimum error for the other schemes even when the value of \(\lambda\) is \(\pm \,40\%\) away from its optimal value. Hence, the proposed regularization not only gives an improvement in the absolute value of the error compared to other schemes, but the results are significantly more robust and less sensitive to the choice of \(\lambda\). The latter result is consistent across the three turbulent flows considered. It should be noted that previous regularization schemes are largely mathematical expressions, predominately constructed to minimize error. The current regularization scheme outperforms the previous schemes, achieving a reduction in error and increase in robustness while using a physically meaningful form of the regularization. This further strengthens the case that \(J_{\mathrm {R}}^\mu\) is currently the best form of regularization for OFV in fluid flows.

5 Summary and conclusions

The focus of this Letter is to establish a physical interpretation of the regularization process that occurs as part of the solution to an optical flow problem within fluids. In doing so, a novel, physically sound regularization scheme for OFV methods has been developed using the Navier–Stokes equations as direct guidance. Inspection of the Navier–Stokes equations shows that a new regularization scheme that enforces smoothness on calculated velocity fields in a way that mimics the role of viscosity in fluid flows is physically meaningful and likely applicable across a broad range of fluid problems. While the new regularization term does not constrain the solution of the optical flow problem to conform to the Navier–Stokes equations, it does enforce smoothness on the solution in a physically sound way using the viscous term of the Navier–Stokes equations. The new, viscosity-inspired regularization was implemented within the context of a wavelet-based optical flow velocimetry (wOFV) framework for testing, but it is expected to be applicable to a broad range of OFV methods. The new regularization approach was evaluated on three sets of synthetic tracer particle data from a series of DNS test cases that cover 2D, incompressible turbulent flow; 3D, nonreacting turbulent flow; and 3D, reacting flow. For all test cases, the new regularization scheme leads to lower overall error in the velocity estimation. While the improvement in absolute error for three DNS cases is small, the error for the viscosity-based regularization scheme is less sensitive to the choice of the weighting parameter, \(\lambda\), which is very important for the use of OFV methods on experimental data, since the optimal value of \(\lambda\) cannot be precisely determined a priori for a given data set. The primary strength of the new regularization scheme is that, to the authors’ knowledge, it is the only one that is rigorously based on flow physics using the Navier–Stokes equations. Thus, we recommend that this form of the regularization parameter be applied broadly to fluid dynamics problems as opposed to arbitrary mathematical forms.