1 Introduction

In recent years, the development of active fluid control methods is important for improvement of the aircraft flight performance. Plasma actuators (Corke et al. 2007; Aono et al. 2017; Komuro et al. 2019) are active fluid control devices that have attracted attention because of their relatively simple structure, installation method, and fast responsiveness, and their application in combination with feedback control is desirable for more efficient fluid control. Thus far, feedback control suppressing flow separation around an airfoil using a plasma actuator and pressure sensors was investigated by Benard et al. (2011) and Post and Corke (2006) in the conditions of constant-angle-of-attack and pitching-angle-of-attack, respectively. Also, the feedback control using a fiber Bragg grating sensor was investigated by Segawa et al. (2016) and its effectiveness was shown. The model-free flow control based on machine learning was also reported (Shimomura et al. 2020), and effective input–output functions were determined for a feedback control system.

It is necessary for feedback control to observe the flow field in real time by various measurements. The pressure sensor sounds the reasonable choice, but it suffers from weak correlation with the flow away from the wall if the target of flow control is away from the wall. The hot-wire anemometer which can directly measure the velocity fluctuations away from the wall is another choice, but it intrudes the flow fields and suffers from the simultaneous measurements of at multiple locations due to its complexity. Thus, among those measurements, particle image velocimetry (PIV), which can nonintrusively measure the entire velocity field appropriate for the real-time measurement of feedback control, is considered for feedback control. However, the fluctuation of aerodynamic phenomena is generally very fast, making real-time measurements extremely difficult. Since the computational cost for processing all of the flow velocity field from the particle images is high, it is difficult to straightforwardly adopt it as an observer in feedback control of flow fields.

Several approaches for the real-time PIV measurement (Lelong et al. 2003; Yu et al. 2006; Kreizer et al. 2010) have been proposed and validated, though those approaches suffer from its computational costs. Recently, Gautier and Aider (2015a) showed the capabilities of the real-time estimations of flow fields by using an optical-flow algorithm accelerated by a graphics processor unit. They utilized the real-time PIV for feedback control of the flow over a backward-facing step of a hydrodynamic experiments (Gautier and Aider 2013, 2015b; Varon et al. 2019). Besides, data assimilation and predictions of shear flows are conducted using the time-resolved flow fields obtained by the method previously proposed as input by Giannopoulos and Aider 2020a, b. However, these studies adopted relatively low sampling rates (20–224 fps) for measurement of the flow fields. This might be because of long processing time for PIV analyses. The low-cost real-time PIV systems enable us to estimate flow fields in real time with the sampling rate of several kilohertz and to conduct feedback control for flows owning high-frequency behaviors which include the separated airflow around airfoils of our target. Therefore, we would like to reduce the computational cost further for the real-time PIV measurement. It is effective to adopt the sparse sensing (Manohar et al. 2018) that estimates the entire fields only from a small number of processing points which reduces the computational cost, whereas this technique is applied to not only fluid flow problems (Kaneko et al. 2021; Inoba et al. 2022) but also problems in vast science and engineering fields (Li et al. 2021; Nagata et al. 2022).

Kanda et al. proposed a sparse processing PIV (SPPIV) based on the concept of sparse sensing, which estimates the entire structure with a small number of interrogation windows, and proper orthogonal decomposition (POD), which describes data by extracting low-dimensional features (Kanda et al. 2021, 2022). In this method, the velocity field is estimated using a discrete linear model that estimates the time evolution of the velocity field and the measurement data at processing interrogation windows predetermined by training data, and a real-time measurement is realized by modifying the estimated values sequentially using a Kalman filter (KF). The velocity field of \(29 \times 61\) was finally obtained from a \(256 \times 512\) pixel\(^2\) particle image with a interrogation window size of \(32\times 32\;\text {pixel}^2\) and a 75% overlap of the search area in the PIV. Increase in spatial resolution is favorable, but smaller interrogation window size of \(16\times 16\) or \(8\times 8\;\text {pixel}^2\) gives a larger number of outliers than that of a standard size of \(32\times 32\;\text {pixel}^2\). However, the conventional method eliminating outliers cannot be straightforwardly adopted for SPPIV because such a method usually uses velocity vectors at neighbors but SPPIV does not have such a information due to its sparsity. For instance, the outlier detection by taking a median of neighbor velocity vectors around the target vector and by comparing the median vector with the target vector is usually used as by Westerweel (1997), but this method is clearly not applicable to SPPIV because the neighbor velocity vectors around the target are not computed in the process of SPPIV. Of course, we can calculate the neighbor velocity vectors together, but it clearly leads to increase in the computational time and degrade the advantage of SPPIV of fast computation.

The purpose of this study is to extend SPPIV to a more robust measurement system. We reduce the size of the interrogation window and increase the resolution of resultant PIV and demonstrate that it can be accurately estimated under conditions where the quantity of particles per window is small and harsh to measurement. The following two methods were proposed and investigated for real-time measurements. One is robust SPPIV which observes in a interrogation window corresponding to a high spatial resolution, and removes outliers in the real-time data step by step using a robust KF (RKF) (Mattingley and Boyd 2010). The other is multiresolution SPPIV which calculates the velocity field at low spatial resolution to avoid outliers in real-time measurements, and projects the velocity field using a predefined relationship between the high-resolution and low-resolution velocity fields. These two methods are implemented and compared for the development of high-resolution SPPIV. In addition to the evaluation of the new methods above, the effects of the sensor selection method, which was not addressed in the previous study, are also investigated and evaluated.

2 Sparse processing PIV and its extension

As discussed in the previous section, SPPIV estimates the entire flow field only from the sparse velocity vectors calculated by local PIV for the sparse interrogation windows in the particle image (Kanda et al. 2021, 2022). Figure 1 shows the flowchart of this method. A series of study proposed to use linear KF (hereafter, standard KF in this paper) for SPPIV together with the proper orthogonal decomposition (POD). Therefore, the POD mode expression and the state-space approximation are described in Sects. 2.1 and 2.2, respectively. Although Sects. 2.1 and 2.2 explain the common technique for all SPPIV used in this paper, while Sect. 2.3 explains the standard KF for original and multiresolution SPPIV and Sect. 2.4 explains RKF for robust SPPIV. It should be noted that though SPPIV can be realized without using KF, the better performance of SPPIV with KF for especially a limited number of interrogation windows processed was reported in the previous study (Kanda et al. 2022) and SPPIV with KF and its extension was only considered in the present study. Section 2.5 explains the projection (regression) technique for multiresolution SPPIV. Table 1 shows the Sections which explain the elements used in each SPPIV for clarity because some of the techniques explained in the coming subsections are not necessarily used for one of implementations.

Table 1 Section which explains the elements used in each SPPIV
Full size table
Fig. 1
figure 1

The flowchart of improved SPPIV

Full size image

2.1 Proper orthogonal decomposition

POD is one of the methods that extract a low-dimensional model, and it gives the bases which have the largest energy under a limited number of bases (Berkooz et al. 1993; Taira et al. 2017). These bases are called POD modes. The POD analysis enables us to reduce the dimension of large flow field data such as that by PIV measurement, while keeping important feature of flow field (Taira et al. 2017). Equation (1) describes the formula in which the data matrix \(\varvec{X}\) is decomposed into POD modes. In this research, a data matrix is constructed as follows: The snapshots of time series data are reshaped into one-dimensional column vectors, and then, they are arranged in the row direction. The resultants of POD, \(\varvec{U}_r\), \(\varvec{\Sigma }_r\), and \(\varvec{V}_r\), correspond to the spatial mode, the amplitude, and the temporal mode, respectively, whereas r is the number of truncated POD modes. Usually, r is optimized as a minimum number which satisfies the requirement, and in this study, r is optimized in wider sense in terms of both reconstruction ratio and smaller computation times. Usually, \(\varvec{\Sigma }_r\) is referred to magnitudes (Chatterjee 2000) but is referred as amplitudes in the present study, while square of amplitudes, \(\varvec{\Sigma }^2_r\), is so-called energies. The flow velocity field reconstructed by POD is represented by the spatial modes multiplied by the time coefficients of \(\varvec{\Sigma }_r\varvec{V}_r^{\textsf{T}}\). It describes that the estimation of flow velocity field could be reduced into the estimation of \(\varvec{\Sigma }_r\varvec{V}_r^{\textsf{T}}\), if the training data gives spatial modes. Hereafter, the POD time coefficients \(\varvec{Z}=\varvec{\Sigma }_r\varvec{V}_r^{\textsf{T}}\) would be used.

$$\begin{aligned} \varvec{X}=\varvec{U}\Sigma \varvec{V}^{\textsf{T}}\approx \varvec{U}_r\Sigma _r\varvec{V}_r^{\textsf{T}}=\varvec{U}_r\varvec{Z}. \end{aligned}$$
(1)

2.2 Model construction and sensor selection

In this study, the data matrix of the flow velocities \(\varvec{X}=[\varvec{X}_u^{\textsf{T}}~\varvec{X}_v^{\textsf{T}}]^{\textsf{T}}\) is employed. Here, \(\varvec{X}_u^{\textsf{T}}\) and \(\varvec{X}_v^{\textsf{T}}\) correspond to the data for the freestream and vertical velocity components, respectively. Here, POD applied to data matrix \(\varvec{X}\) gives us the spatial mode \(\varvec{U} = [\varvec{U}_u^{\textsf{T}}\;\varvec{U}_v^{\textsf{T}}]^{\textsf{T}}\) whereas \(\varvec{U}_u^{\textsf{T}}\) and \(\varvec{U}_v^{\textsf{T}}\) correspond to the spatial modes of freestream and vertical velocity components, respectively. The training data were then constructed according to the following procedure.

2.2.1 Making training data with high spatial resolution

In the preprocess of SPPIV, the velocity field obtained by PIV is used as training data: The positions of processing interrogation windows are optimized and various KF parameters are calculated. The following two methods were used to obtain flow velocity fields with PIV interrogation windows of \(16\times 16\;\text {pixel}^2\) and \(8\times 8\;\text {pixel}^2\).

  1. (A)

    Recursive local-correlation PIV (Hart 1999).

  2. (B)

    Robust principal component analysis (RPCA) with gappy data (Scherl et al. 2020; Willcox 2006; Kaneko et al. 2023).

In (A), a regular PIV analysis was first conducted at \(32\times 32\;\text {pixel}^2\) interrogation window and was reduced from \(16\times 16\;\text {pixel}^2\) to \(8\times 8\;\text {pixel}^2\) by predicting the destination of particles within the image pair based on the results of the previous step of PIV analysis. In this case, certain criteria are set, and the corresponding vectors are stored as outliers. In (B), the components considered as outliers are excluded, and the missing data are complemented by gappy-POD and RPCA is performed. RPCA is a method for extracting a low-dimensional matrix \(\varvec{L}\) from a data matrix \(\varvec{X}\), excluding a matrix \(\varvec{S}\) that contains outliers. The two steps (A) and (B) are applied alternately to obtain a velocity field with fewer outliers. The detailed algorithm was presented in the other paper (Kaneko et al. 2023)

2.2.2 Processing interrogation window (sensor) optimization

In this paper, sparse sensors are corresponding to the sparse processing interrogation windows. The term ‘sensor’ is used when we refer the sensor optimization technique of the previous study, while the term ‘processing interrogation windows’ is used when we refer it in the SPPIV analysis. Please reread the technical word of ‘sensor’ as ‘processing interrogation windows’ when applying the sensor optimization technique to the SPPIV.

For the location determination of the processing interrogation windows, we compared the snapshot-to-snapshot-based sensor location optimization method in a vector field (Saito et al. 2020, 2021a, b) and that based on the KF index (Zhang et al. 2017; Takahashi et al. 2023; Nagata et al. 2024) with extension to the vector field though details of extension is not addressed in the present paper for brevity. In this study, processing interrogation windows obtained by the former method are called determinant-based-greedy sensor optimization (DG-sensor) method and those using the latter are called KF greedy sensor optimization (KF-sensor) method. (Here, the word ‘sensor’ is used for brevity.) Here, sensor location by the DG-sensor method is optimized for the inverse problem of POD mode coefficients only from the snapshot observation, while that by the KF-sensor method is for the inverse problem of time series POD mode coefficients up to the present state. Both methods employ a greedy method which optimizes the location of the sensors in order, acquiring multiple components at a point. The former aims to minimize the expected error variance in the estimation of mode strength and takes the maximization of the determinant of the Fisher information matrix as the objective function, while the latter aims to minimize the estimation error by KF and takes the minimization of the determinant of the error covariance matrix of KF as the objective function.

The sensor location matrix \(\varvec{H}_s\) is obtained by the methods above. Here, a component of a selected sensor location of \(\varvec{H}_s\) is unity and other entries are zeros. Based on the sensors selected by \(\varvec{H}_s\), the observation equation which presents the relationship between the observation value \(\varvec{y}\) and the state vector \(\varvec{z}\) is written as follows:

$$\begin{aligned} \varvec{y}=\varvec{H}_s\varvec{U}_{r}\varvec{z}=\varvec{C}\varvec{z} \end{aligned}$$
(2)

where \(\varvec{C}\) is an observation matrix. Here, the dimension of \(\varvec{z}\) is r as it is a column vector of \(\varvec{Z}\), and r should be defined for each experiment. See references Saito et al. (2021a), Zhang et al. (2017), Takahashi et al. (2023), and Nagata et al. (2024), for more details of both greedy methods. With regard to the structure of the \(\varvec{H}_s\) matrix for vector sensor extension, refer (Joshi and Boyd 2009; Saito et al. 2020, 2021b).

2.2.3 Linear-reduced-order model

This study employs the low-dimensional discrete linear dynamical model proposed by Nankai et al. (2019). The system matrix \(\varvec{F}\) in Eq. (10) is constructed based on the database of the state variables \(\varvec{Z}\), which results in the linear model approximating time variation of POD modes \(\varvec{Z}\). Two matrices in Eqs. (3) and (4) are employed for the construction of \(\varvec{F}\). Equation (5) gives the system matrix \(\varvec{F}\), whereas \(\varvec{Z}_{m-1}^{+}\) denotes the Moore–Penrose pseudoinverse of an arbitrary matrix \(\varvec{Z}_{m-1}\).

$$\begin{aligned} \varvec{Z}_{m-1}&=[\varvec{z}_1 \; \varvec{z}_2 \cdots \varvec{z}_{m-2} \; \varvec{z}_{m-1}], \end{aligned}$$
(3)
$$\begin{aligned} \varvec{Z}_{m}&=[\varvec{z}_2 \; \varvec{z}_3 \cdots \varvec{z}_{m-1} \; \varvec{z}_m], \end{aligned}$$
(4)
$$\begin{aligned} \varvec{F}&=\varvec{Z}_m(\varvec{Z}_{m-1})^{+}. \end{aligned}$$
(5)

2.3 Original SPPIV: State estimation using KF

In SPPIV, KF is employed as a core algorithm because KF gives us much better estimation than snapshot estimation demonstrated by Kanda et al. (2022). In this process, state vectors are sequentially updated by the least squares estimate based on the observations in the linear system which includes the Gaussian observation and system noises (Kalman 1960). The state and the observed values are described by Eqs. (6) and (7), respectively. Here, \(\varvec{z}\), \(\varvec{y}\), and \(\varvec{v}_n\) and \(\varvec{w}_n\) denote the estimated state vector, the observed value, and the system noise and observation noise, respectively.

$$\begin{aligned} \varvec{z}_{k+1 \mid k}&=\varvec{F} \varvec{z}_{k \mid k} + \varvec{v}_k, \end{aligned}$$
(6)
$$\begin{aligned} \varvec{y}_{k}&=\varvec{C} \varvec{z}_k + \varvec{w}_k, \end{aligned}$$
(7)

Equations (8) and (9) present the time series data matrix constructed by arranging the estimated value \(\varvec{z}\) and the observed value \(\varvec{y}\), respectively. Here, \(\varvec{\Sigma }_r\) and \(\varvec{V}_r\) are lower-dimensional amplitude and temporal POD modes in the leading r mode, respectively, obtained by the truncation at the leading r modes. Here, the term ‘leading’ which expresses the larger amplitude POD modes and can be rephrased by ‘top’ or ‘optimized’ is used hereafter in the present paper. \(\varvec{H}\) in Eq. (9) represents the sensor location matrix determined by the observation position optimization method described in Sect. 2.2.2. In addition, the hyperparameters represent the noise covariance will be calculated by the following training data:

$$\begin{aligned} \varvec{Z}&=[\varvec{z}_1 \; \varvec{z}_2 \cdots \varvec{z}_m]=\varvec{\Sigma }_r\varvec{V}_r^\textsf{T}, \end{aligned}$$
(8)
$$\begin{aligned} \varvec{Y}&=[\varvec{y}_1 \; \varvec{y}_2 \cdots \varvec{y}_m]=\varvec{H}\varvec{X}, \end{aligned}$$
(9)

whereas m is the number of time in these equations.

Here, KF consists of an observation update step of Eqs. (10) and (11) and a time update step of Eqs. (12)–(14):

$$\begin{aligned} \hat{\varvec{z}}_{k+1 \mid k}&=\varvec{F} \hat{\varvec{z}}_{k \mid k}, \end{aligned}$$
(10)
$$\begin{aligned} \varvec{P}_{k+1 \mid k}&=\varvec{F} \varvec{P}_{k \mid k}\varvec{F}^{\textsf{T}} + \varvec{Q}, \end{aligned}$$
(11)
$$\begin{aligned} \varvec{K}_k&=\varvec{P}_{k \mid k-1}\varvec{C}^{\textsf{T}}(\varvec{C}_k\varvec{P}_{k \mid k-1}\varvec{C}^{\textsf{T}}+\varvec{R})^{-1}, \end{aligned}$$
(12)
$$\begin{aligned} \hat{\varvec{z}}_{k \mid k}&=\hat{\varvec{z}}_{k \mid k-1}+\varvec{K}_k\varvec{e}_k, \end{aligned}$$
(13)
$$\begin{aligned} \varvec{P}_{k \mid k}&=\varvec{P}_{k \mid k-1}-\varvec{K}_k\varvec{C}\varvec{P}_{k \mid k-1}, \end{aligned}$$
(14)

where \(\varvec{e}_k=\varvec{y}_k-\varvec{C}\hat{\varvec{z}}_{k \mid k-1}\) is the difference between observation and estimation. Here, \(\varvec{P}\), \(\varvec{K}\) and \(\varvec{Q}\) and \(\varvec{R}\) represent the error covariance matrix, the Kalman gain matrix, and noise covariance matrices for the system and the observation, respectively. In addition, \(\varvec{P}_{k \mid k-1}\) presents the kth estimated covariance matrix from the data up to the \((k-1)\)th, and \(\varvec{P}_{k \mid k}\) does that from the data up to the kth.

Meanwhile, the noise covariance matrices were also simply estimated based on the training data, as described by Eqs. (6), (7), (15), and (16). Also, the present study assumes that \(\varvec{Q}\) and \(\varvec{R}\) obtained from the training data are time invariant in the present study.

$$\begin{aligned} \varvec{Q}&=\frac{1}{m-1}\sum _{k=1}^{m-1}\varvec{v}_{k}\varvec{v}^{\textsf{T}}_{k}, \end{aligned}$$
(15)
$$\begin{aligned} {R_{ij}}&=\frac{1}{m}\sum _{k=1}^{m}\varvec{w}_{k}\varvec{w}^{\textsf{T}}_{k}. \end{aligned}$$
(16)

Here, substitution of Eq. (5) into Eqs. (6) and (7) gives estimation of \(\varvec{v}_k\) and \(\varvec{w}_k\), respectively, in the training data.

In this study, the initial values of the error covariance matrix \(\varvec{P}\) and the estimated state vector z are the identity matrix and the zero vector, respectively.

2.4 Robust SPPIV: state estimation using RKF

When the PIV interrogation window is reduced in real-time measurements, the velocity field at the location of the observation point, which is obtained as the observed value, is disturbed by more of outliers. In this study, we use an RKF, Mattingley and Boyd (2010), which performs state estimation for a system that is assumed to contain outliers other than Gaussian noise. We add the assumed non-Gaussian outlier \(\varvec{t}\) into the system of Eq. (18).

$$\begin{aligned} \varvec{z}_{k+1 \mid k}&=\varvec{F} \varvec{z}_{k \mid k} + \varvec{v}_k, \end{aligned}$$
(17)
$$\begin{aligned} \varvec{y}_{k}&=\varvec{C} \varvec{z}_k + \varvec{w}_k +\varvec{t}_k. \end{aligned}$$
(18)

In this case, a filtering procedure becomes as follows:

$$\begin{aligned} \hat{\varvec{z}}_{k+1 \mid k}&=\varvec{F} \hat{\varvec{z}}_{k \mid k}, \end{aligned}$$
(19)
$$\begin{aligned} \varvec{P}_{k+1 \mid k}&=\varvec{F} \varvec{P}_{k \mid k}\varvec{F}^{\textsf{T}} + \varvec{Q}, \end{aligned}$$
(20)
$$\begin{aligned} \varvec{K}_k&=\varvec{P}_{k \mid k-1}\varvec{C}^{\textsf{T}}(\varvec{C}_k\varvec{P}_{k \mid k-1}\varvec{C}^{\textsf{T}}+\varvec{R})^{-1}, \end{aligned}$$
(21)
$$\begin{aligned} \hat{\varvec{z}}_{k \mid k}&=\hat{\varvec{z}}_{k \mid k-1}+\varvec{K}_k(\varvec{e}_k-{\varvec{t}}_k), \end{aligned}$$
(22)
$$\begin{aligned} \varvec{P}_{k \mid k}&=\varvec{P}_{k \mid k-1}-\varvec{K}_k\varvec{C}\varvec{P}_{k \mid k-1}, \end{aligned}$$
(23)

where Eq. (22) includes the term for outlier \(\varvec{t}_k\). Here, the outliers \(\varvec{t}_k\) were calculated by solving the following \(l_1\)-norm optimization problem each time within an RKF step.

$$\begin{aligned} \min _{\varvec{t}_k} (\varvec{e}_k - \varvec{t}_k)^{\textsf{T}} \varvec{W}_k(\varvec{e}_k - \varvec{t}_k) + \lambda ||\varvec{t}_k||_1, \end{aligned}$$
(24)

Here, \(\varvec{W}\) satisfies \(\varvec{W}=(\varvec{C}\varvec{P}_{k \mid k-1}\varvec{C}^{\textsf{T}}+\varvec{R})^{-1}\) and \(\lambda\) is a hyperparameter. Note that the estimation method that does not solve Eq. (24) with \(\varvec{t} = 0\) is called standard KF of the previous subsection, which is used in the original SPPIV work (Kanda et al. 2021, 2022).

Finally, it should be again noted that usually the outliers in PIV are removed by spatial information as described in Sect. 1 but SPPIV cannot use spatial information owing to its spatial sparsity. Therefore, this technique removes the outlier based temporal information using RKF. Therefore, the mechanism of removal of outliers is much different from those of conventional techniques.

2.5 Multiresolution SPPIV: projection of low-resolution data into high-resolution training data

In this study, as another method to improve the spatial resolution of the velocity field obtained by SPPIV, test data acquired with a large interrogation window size in real-time measurements are projected onto training data created with a small interrogation window size. The large interrogation area reduces a number of outliers, and accurate estimation is expected if the projection below works well.

When creating the training data, we take advantage of the fact that the recursive local-correlation PIV of the \(32\times 32\;\text {pixel}^2\), \(16\times 16\;\text {pixel}^2\) and \(8\times 8\;\text {pixel}^2\) can provide PIV results for all interrogation window sizes. The projection matrix is constructed in the following method. The outliers in each interrogation window sizes are removed by recursive local-correlation PIV and RPCA with gappy data. Here, \(\varvec{L}_{32}\), \(\varvec{L}_{16}\), and \(\varvec{L}_{8}\) are the velocity field data matrices after the removal of the outliers in the interrogation window sizes of \(32\times 32\;\text {pixel}^2\), \(16\times 16\;\text {pixel}^2\) and \(8\times 8\;\text {pixel}^2\). These matrices can be decomposed by POD in the leading r mode as follows.

$$\begin{aligned} \varvec{L}_{32}&\approx \varvec{U}_{32r}\varvec{\Sigma }_{32r}\varvec{V}_{32r}^{\textsf{T}}, \end{aligned}$$
(25)
$$\begin{aligned} \varvec{L}_{16}&\approx \varvec{U}_{16r}\varvec{\Sigma }_{16r}\varvec{V}_{16r}^{\textsf{T}}, \end{aligned}$$
(26)
$$\begin{aligned} \varvec{L}_{8}&\approx \varvec{U}_{8r}\varvec{\Sigma }_{8r}\varvec{V}_{8r}^{\textsf{T}}, \end{aligned}$$
(27)

Now, multiresolution SPPIV measures POD coefficients with a interrogation window size of \(32\times 32\;\text {pixel}^2\) and obtains a time evolution \(\varvec{Z}_{32}^{\text {test}}\) that follows \(\varvec{U}_{32r}\) in real time. Using each mode of the training data matrix, the relationship between the \(\varvec{Z}_{32r}\) and \(\varvec{Z}_{16r}\) or \(\varvec{Z}_{8r}\) by projection (regression).

$$\begin{aligned} {\varvec{Z}}_{16r}^{\text {proj}}&=\mathbb {T}_{32\rightarrow 16} \varvec{Z}_{32r}^{\text {test}}, \end{aligned}$$
(28)
$$\begin{aligned} {\varvec{Z}}_{8r}^{\text {proj}}&=\mathbb {T}_{32\rightarrow 8} \varvec{Z}_{32r}^{\text {test}}, \end{aligned}$$
(29)

whereas \(\mathbb {T}\) is the transformation matrix based on projection (regression) as follows:

$$\begin{aligned} \mathbb {T}_{32\rightarrow 16}&=\varvec{\Sigma }_{16r}\varvec{V}_{16r}^{\textsf{T}}\varvec{V}_{32r}\varvec{\Sigma }_{32r}^{-1}, \end{aligned}$$
(30)
$$\begin{aligned} \mathbb {T}_{32\rightarrow 8}&=\varvec{\Sigma }_{8r}\varvec{V}_{8r}^{\textsf{T}}\varvec{V}_{32r}\varvec{\Sigma }_{32r}^{-1}. \end{aligned}$$
(31)

Here, \({\varvec{z}}^{\text {proj}}\) represents the estimation of \(\varvec{z}\) by projection.

Using the relationship of Eqs. (28) and (29), \(\varvec{U}_{16r}{\varvec{Z}}_{16r}^{\text {proj}}\) and \(\varvec{U}_{8r}{\varvec{Z}}_{8r}^{\text {proj}}\) can be reconstructed as the high-resolution velocity field. In real-time measurements, the transformation matrices above are created from the training data in advance, and the vectors \(\varvec{z}_{32k}\) estimated at each step of the standard KF are multiplied by the transformation matrices and output sequentially, as follows:

$$\begin{aligned} \varvec{z}_{16k}^{\text {proj}}&=\mathbb {T}_{32\rightarrow 16} \varvec{z}_{32k} \end{aligned}$$
(32)
$$\begin{aligned} \varvec{z}_{8k}^{\text {proj}}&=\mathbb {T}_{32\rightarrow 8} \varvec{z}_{32k} \end{aligned}$$
(33)

In this way, sparse processing including projection is realized with suppressing the outliers by adopting relatively large interrogation windows for the sparse analysis side.

3 Wind tunnel test

3.1 Experimental equipment

The wind tunnel experiment was conducted using the Small Low-Turbulence Wind Tunnel at the Institute of Fluid Science, Tohoku University. An NACA0015 airfoil of chord length and span width of 100 and 300 mm, respectively, was installed in the test section. The PIV measurement system is constructed by combining a real-time high-speed camera (IDP-Express R2000, Photron), a double-pulse Nd:YLF laser (DM30-527-DH, Photonics Industries), and a function generator (WF1948, NF). A Laskin nozzle was employed, and dioctyl sebacate is granulated as the tracer particles. The experimental setup of the wind tunnel testing was the almost same as that in the previous study (see Figure 3 of Kanda et al. 2022).

3.2 Test condition and computational condition

The freestream velocity and the angle of attack were set to be 10 m/s and 18 deg, respectively. The resulting Reynolds number was \(6.9 \times 10^4\). The sampling rate of PIV was set to be 2000 fps. The image resolution is \(512 \times 256\) pixel\(^2\). The sampling rate of images and the laser interval for the PIV measurement were set to be 4000 fps and \(\Delta t = 70\) µs with the frame straddling technique. Interrogation window sizes are changed from \(32\times 32\;\text {pixel}^2\) to \(8\times 8\;\text {pixel}^2\), and those windows are 75% overlap with each other. The particle density is estimated to be 0.131 1/pixel\(^2\) by counting the extrema of image intensity fields, which corresponds to 134.0, 33.5, and 8.4 particles in \(32 \times 32\), \(16 \times 16\), and \(8\times 8\) interrogation windows, respectively. Figure 2 shows the raw particle images obtained and processed in the present study. Table 2 summarizes the specification of the computer employed in this study, which was noted for repeatability of the experiment. The computational time for the computer employed in the present study is approximately 6–10 µs per interrogation window in the process of SPPIV as shown later, whereas multiple interrogation windows are processed in parallel by multiple cores in the computational process unit. The PIV system above produces time series data of the flow velocity field. The in-house code written in MATLAB was employed and the sensor location matrix \(\varvec{H}\) and noise covariance matrices \(\varvec{Q}\) and \(\varvec{R}\) used for the training data were calculated.

Fig. 2
figure 2

Raw particle images obtained and processed in the present study. Intensity was adjusted for visualization

Full size image
Table 2 The specification of the computer
Full size table

An off-line analysis was conducted using the particle images obtained in the wind tunnel test, and the performance of robust/multiresolution SPPIV was investigated in this study. In the off-line analysis, the effects of three different parameters on the change in the performance of SPPIV were investigated. The PIV analysis was performed on the 1000 pairs of particle images acquired, and the velocity field and time series data matrices were obtained using the method. The data were decomposed by POD, and the leading 10 modes were used (\(r = 10\)) in this study. Here, 10 modes reconstruction is approximately 60 % reconstruction as shown by Kanda et al. (2022), and it is selected for demonstrate the real-time observation. In the future application of SPPIV, the number r of modes should be carefully chosen based on not only the complexity of flow fields but also the capability of real-time observation. The observation point locations were determined as described in Sect. 2.2.2, and the noise covariance matrix was estimated. All the training data were generated using in-house codes created in MATLAB. The number of processing points p was set to 5, 10, 15, 20, 25, and 30, and the estimation accuracy of original SPPIV, robust SPPIV, and multiresolution SPPIV were analyzed and the effects of the two sensor selection methods described in Sect. 2.2.2 were investigated. The all estimation accuracy were evaluated by fivefold cross validation.

In robust SPPIV, the interrogation window of PIV at the selected processing points was \(16\times 16\;\text {pixel}^2\), and \(8\times 8\;\text {pixel}^2\). These are the same size as the interrogation window of the velocity field used as the training data. Here, \(\lambda\) was, respectively, predetermined to be 0.40 and 0.10 in SPPIV at \(16\times 16\;\text {pixel}^2\) and \(8\times 8\;\text {pixel}^2\) interrogation window except for the investigation of the effect of the hyperparameter. The outlier \(\varvec{t}\) is obtained by iteration using fast iterative shrinkage thresholding algorithm (FISTA) (Beck and Teboulle 2009). In multiresolution SPPIV, the interrogation window of PIV at the selected processing points was \(32\times 32\;\text {pixel}^2\) and projected to high-resolution spatial modes of the training data.

Here, real-time measurements were performed using code written in C++. The conditions were the same as that in the off-line parametric study, and the feasibility of a real-time measurement and the processing time were investigated. The success or failure of the real-time measurement should be judged in the present state by comprehensively considering the estimation accuracy and comparing the number of images taken and the number of images for which SPPIV was conducted. It should be noted that the judgment above is strongly related to the naive implementation of the processing code as discussed by Kanda et al. (2022), because the error becomes very large if the first and second images of image pairs were swapped due to the delayed computation of naive implementation of image processing. This error could be relaxed by the sophisticated processing code with including error handling, but the success or failure of the real-time measurement does not essentially change. Therefore, we use the judgment based on error when using naive implementation of the processing code, similar to Kanda et al. (2022).

4 Results and discussions

The former two subsection in this section describes the results of an off-line parametric study for evaluating the characteristics of SPPIV in the real-time measurements. We confirm that the off-line analyses give the almost the same characteristics as the real-time measurement. All results except processing time are the result of fivefold cross validation. The third subsection shows the real-time measurement results to demonstrate the real-time measurements. Finally, the two improved SPPIV in the present paper are discussed and compared in the last subsection.

4.1 Parametric study of robust SPPIV

Figure 3 shows the reconstruction of the freestream direction velocity field from the SPPIV results obtained in this test using the spatial modes of the training data. The number of processing interrogation windows is 5 and 30 (processing interrogation window locations are indicated by black circles), and the estimation method is a case applying RKF. The images explained as ‘Regular PIV’ in the figure are the velocity fields at the same time when PIV was applied to the particle image on the test data side of SPPIV, using the same procedure as that used to generate the training data. Figure 4 shows the time histories of the first POD mode in the data matrix with the test data for SPPIV with 5 and 30 processing interrogation windows selected by the KF-sensor methods. The blue line is the result of SPPIV estimation, and the red line is the data from PIV on the test data, with the resulting velocity field projected onto the spatial mode of the training data. The PIV results are qualitatively well reproduced with robust SPPIV and original SPPIV in the case of \(16\times 16\;\text {pixel}^2\) interrogation windows. On the other hand, original SPPIV fails in the case of \(8\times 8\;\text {pixel}^2\) interrogation windows, while robust SPPIV eliminates noise and improves the qualitative estimation situation. The degradation in the original SPPIV is caused by the outliers that appear due to the small interrogation windows and relatively larger \(\Delta t\). It should be noted that the size of \(8 \times 8\;\text {pixel}^2\) is not a quantitative index because it seems to be affected by the particle density of experiments. Meanwhile, robust SPPIV successfully works well for the velocity fields including a lot of outliers compared with the original SPPIV, as expected. This implicates that robust SPPIV can work more robustly with larger \(\Delta t\).

Here, \(Z^{\text {ref}}\) in Eq. (35) is obtained by projecting the data matrix of the time series flow velocity field obtained by PIV into the spatial mode of the training data used in SPPIV by using Eq. 34.

$$\begin{aligned} \varvec{Z}^{\text {ref}}&= \varvec{U}_r{^\textsf{T}}\varvec{X} \end{aligned}$$
(34)
$$\begin{aligned} \text {Error}&= \frac{\Vert \varvec{Z}^{\text {ref}}-\tilde{\varvec{Z}}\Vert _{\text {Fro}}}{\Vert \varvec{Z}_{i,j}^{\text {ref}}\Vert _{\text {Fro}}} \end{aligned}$$
(35)

Here, \(\Vert \circ \Vert _{\text {Fro}}\) denots the Frobenius norm of \(\circ\).

Fig. 3
figure 3

The snapshots of the freestream direction velocity fields reconstructed by POD modes obtained from robust SPPIV at \(tU_{\infty }/c = 50\) in the case of \(p = 5\) and 30. Here, circles show the sparse processing interrogation windows

Full size image
Fig. 4
figure 4

The time histories of the first mode amplitude obtained using PIV and estimated using SPPIV in the case of \(p = 5\) and 30

Full size image
Fig. 5
figure 5

The error of robust SPPIV when changing hyperparameter \(\lambda\). These are the results with the processing interrogation windows selected by the KF-sensor method

Full size image

The error was reduced by applying RKF when measurements were taken at the same processing point. This effect is particularly large in the case of the \(8\times 8\;\text {pixel}^2\) interrogation window. The error decreases as the number of processing interrogation windows increases in all the cases except the two cases of original SPPIV of the interrogation window of \(8\times 8\;\text {pixel}^2\). On the other hand, the estimation accuracy does not decrease when the number of processing interrogation windows is increased in these two cases of original SPPIV of interrogation window of \(8\times 8\;\text {pixel}^2\). It might be because of generation of outliers in the SPPIV analyses.

Fig. 6
figure 6

The error of robust SPPIV

Full size image

Figure 5 shows the estimation errors for the leading ten modes for each interrogation window size when \(\lambda\) is changed from 0.01 to 1.00 in 0.01 increments. The point indicated by the red circle in the graph for each observation is the point where the error is minimized, and is referred to as the ‘appropriate \(\lambda\)’ in the following discussion. Equation (24) describes that \(\lambda\) is a constant that determines the magnitude of the contribution of the \(l_1\) norm term. The parameter \(\lambda\) changes the strength of soft thresholding which removes outliers. The solution \(\varvec{t}\) of Eq. (24) approaches \(\varvec{t} = \varvec{e} = \varvec{y} - \varvec{Cz}\) when \(\lambda\) is decreased. Since SPPIV starts the estimation with \(\varvec{z}\) initially as a zero vector, \(\varvec{z}\) would keep to be a zero vector at all times when \(\lambda = 0\) and the estimation error would become 100%. On the other hand, the solution \(\varvec{t}\) of Eq. (24) approaches \(\varvec{t} = \varvec{0}\) as \(\lambda\) is increased. If \(\lambda \rightarrow +\infty\), the outlier removal of RKF does not work and the estimation result is equal to that of the standard KF.

Figure 5 shows that the error is convex downward when the number of processing interrogation windows is large. The valley shape of the graph is even more pronounced for an interrogation window size of \(8\times 8\;\text {pixel}^2\), and the effect of RKF is clearly small when \(\lambda\) is not properly set. For all interrogation window sizes, the larger the number of processing interrogation windows, the smaller the appropriate \(\lambda\) value tends to be. Here, decreasing \(\lambda\) can be interpreted as increasing the sensitivity of detection of outliers in RKF. Therefore, more erroneous vectors are expected to be introduced as the number of processing interrogation windows increases, and it is necessary to decrease \(\lambda\) and to remove more erroneous vectors. In other words, it is necessary to perform parameter tuning for each case with a different number of processing interrogation windows.

Finally, the sensor selection methods are discussed. The processing interrogation windows selected by the KF-sensor method is more scattered than those selected by the DG-sensor method, as shown in Fig. 4. The processing interrogation windows selected by the KF-sensor method give more accurate results than those by the DG-sensor method when the number of processing interrogation windows was increased. Figure 6 shows that the estimation error in the case using processing interrogation windows selected by the DG-sensor method increases with 30 processing interrogation windows while that by the KF-sensor method decreases even with 30 processing interrogation windows. The degradation of performance of the former case might be caused by contamination of noise due to processing interrogation window selected somehow inappropriately. This result shows that use of the KF-sensor method is advantageous when the number of processing interrogation windows is large.

4.2 Parametric study of multiresolution SPPIV

Figure 7 shows the freestream direction velocity field projection from multiresolution SPPIV results obtained in this test using the spatial modes of the higher-resolution training data. Similar to Fig. 3, the images explained as ‘Regular PIV’ in the figure are the velocity fields at the same time when PIV was applied to the particle image on the test data side of SPPIV, using the same procedure as that used to generate the training data. Figure 8 shows the time histories of the first POD mode in the data matrix with the test data for SPPIV with 5 and 30 KF sensors. These POD modes are after projection, i.e., \(\varvec{Z}_{16r}^{\text {proj}}\) and \(\varvec{Z}_{8r}^{\text {proj}}\).

Fig. 7
figure 7

The snapshots of the freestream direction velocity fields reconstructed by POD modes obtained from robust SPPIV at \(tU_{\infty }/c = 50\) in the case of \(p = 5\) and 30

Full size image
Fig. 8
figure 8

The time histories of the first mode amplitude obtained using PIV and estimated using SPPIV in the case of \(p = 5\) and 30

Full size image

These figures show that projection onto a high-resolution velocity field can successfully reproduce the large-scale structure of the flow field. In addition, the time histories of the POD mode are well tracked, and no noise appears. Therefore, it appears that multiresolution SPPIV which adopts projection is an effective method for estimation of high-resolution velocity fields by sparse processing. The relatively larger interrogation windows suppress the error vector, and it is expected to be robust for further larger \(\Delta t\). Furthermore, quantitative errors are shown and the effectiveness of the method was confirmed.

The error of multiresolution SPPIV is shown in Fig. 9. The error was calculated using Eqs. (34) and (35) for the POD mode of the velocity field corresponding to the high-resolution data. In addition, the estimation errors for the POD mode, where both training and test data are estimated with an interrogation window size of \(32 \times 32\), are also shown for comparison.

Fig. 9
figure 9

The error of multiresolution SPPIV

Full size image

Figure 9 shows that the estimation accuracy of multiresolution SPPIV is much higher than that of the original SPPIV by reducing the interrogation window of both training and test data which is labeled as ‘\(16 \times 16\) SPPIV’ or ‘\(8 \times 8\) SPPIV,’ similar to the previous subsection. The original SPPIV at high-resolution suffers from the outliers that frequently appear. It also does not significantly vary from the estimation by standard SPPIV with an interrogation window size of \(32\times 32\;\text {pixel}^2\). The error value for multiresolution SPPIV with the processing interrogation windows selected by the KF- and DG-sensor methods is slightly smaller and larger than those for the standard SPPIV of a \(32\times 32\;\text {pixel}^2\) interrogation window size, respectively. A possible reason for the slight increase in error due to the projection is the shift of the time histories corresponding to each spatial mode. In other words, some of the time histories that should correspond to the leading modes of \(\varvec{U}_r\) exist below the (\(r + 1\))th mode of the velocity field obtained by the \(32\times 32\;\text {pixel}^2\) interrogation window size. They are considered to vanish during the estimation. The cause of the slight decrease in error due to projection can be interpreted in the same way. Time histories containing outliers, such as noise, might be modified by the projection to corresponding spatial modes with smaller energies. The details of this minor effect of the shift of each component by projection should be discussed more in detail in the future.

Finally, the sensor selection methods are again discussed using the results of multiresolution SPPIV. The processing interrogation windows selected by the KF-sensor method are more scattered than those selected by the DG-sensor method, as shown in Fig. 8, similar to the previous subsection. In addition, the processing interrogation windows selected by the KF-sensor method give more accurate results than those by the DG-sensor method similar to the case of robust SPPIV when the number of processing interrogation windows was increased. Figure 9 shows the trend and suggests the KF-sensor method gives better results.

4.3 Real-time measurements using two improved SPPIV methods

Figure 10 shows the average processing time per step of the real-time measurements of estimation of the velocity field. It must be less than \(500\,\upmu \hbox {s}\) since the sampling rate of PIV is 2000 Hz. Even if outlier removal of RKF is performed in all steps, the computation time does not increase significantly. The real-time measurement is possible even when RKF is incorporated. In addition, decreasing the interrogation window size does not lead to reduction in the processing time. In general, decreasing the PIV interrogation window size reduces the amount of data, which is expected to reduce the processing time to calculate the cross correlation. However, the processing time to calculate the PIV correlation in the C++ program used in this study is not considered to depend on the size of the interrogation window much in the range we investigated. This point was confirmed by the fact that the computational time of robust SPPIV of \(16\times 16\) is almost the same as that of \(8\times 8\) by the same C++ code, by comparing Fig. 10a, b. Therefore, the clear reduction in computational time was not observed. This is considered to be because the definition of variables and memory accesses account for a large ratio of the total computational time.

Fig. 10
figure 10

The average processing time per step of real-time measurements using robust SPPIV. These are the results with the processing interrogation windows selected by KF-sensor

Full size image

Meanwhile, the processing time of multiresolution SPPIV is almost the same as original SPPIV of \(32 \times 32 \;\text {pixel}^2\) as expected because the sparse processing interrogation windows of multiresolution SPPIV has \(32 \times 32\;\text {pixel}^2\) and the additional computational costs of projection calculation is very small.

Therefore, unexpectedly, the processing speeds of robust SPPIV and multiresolution SPPIV are almost the same as each other in the present situation. Now, the numbers of POD mode estimated and the sparse interrogation windows are only limited to \(\mathcal {O}(10)\), and the processing speeds does not seem to change in such a condition. Thus, the selection of robust SPPIV or multiresolution SPPIV should only depend on the accuracy.

The measurement with a number of processing interrogation windows greater than 30 fails because processing is not in time due to a large number of interrogation windows to be processed in short time. Similar to the previous study by Kanda et al. (2022), the measurement with processing time less than 200 µs with a number of processing interrogation windows of 30 is only capable in the present study though 500 µs is maximum processing time that allows us to conduct real-time measurements of 2000 Hz. This is because it takes longer time than 500 µs in one of iterations in the case with a number of processing interrogation windows greater than 30. Especially, the processing time of the initial step was longer than the average, and there were cases where measurement failed. Improvement of the program, such as reducing the effect of thread startup, is still needed.

4.4 Comparison of robust and multiresolution SPPIV

Robust and multiresolution SPPIV are compared with each other based on the results above. First, errors of robust and multiresolution SPPIV are compared and discussed. Comparison of Figs. 6 and 9 illustrates that the error of robust SPPIV for \(16 \times 16\;\text {pixel}^2\) is slightly larger than that of multiresoltion SPPIV, while it shows that the error of robust SPPIV for \(8 \times 8\; \text {pixel}^2\) is further larger than that of multiresoltion SPPIV. These results show that multiresolution SPPIV which avoids the outliers by taking large interrogation window is more robust in the present situation. Second, the processing times of robust SPPIV and multiresolution SPPIV are almost the same as that of original SPPIV, and the real-time measurements are successfully achieved by both methods. Therefore, those comparison presents that multiresolution SPPIV is better to implement in the present situation in which the numbers of modes and processing interrogation windows are only \(\mathcal {O}(10)\). We expected the superiority of multiresolution SPPIV do not change until the POD mode which contains high wave number fluctuation that can be expressed not in the \(32\times 32\) interrogation window resolution but in the \(16\times 16\) interrogation window resolution is estimated. However, the performance of robust SPPIV and multiresolution SPPIV possibly alters when those parameters are changed, and further discussion will be required, which is out of scope of the present study.

5 Conclusions

In this paper, we conducted development of two methods of high-resolution SPPIV for the real-time measurements of flow field. One is robust SPPIV which incorporates with RKF to eliminate the outliers, while the other is multiresolution SPPIV which adopts the large interrogation area for the real-time measurements and project it into the high-resolution velocity fields. Both methods are successfully constructed and implemented, and we succeeded in the real-time measurements using both methods, respectively. The results of new SPPIV compared with the original SPPIV in the range we investigated are summarized as follows:

  • Robust and multiresolution SPPIV can estimate the velocity fields more accurately than high-resolution standard SPPIV with \(16 \times 16\) or \(8 \times 8\;\text {pixel}^2\) interrogation windows. This is because the standard SPPIV suffers from outliers in relatively small size of interrogation windows while both new methods work well for suppression of errors from the outliers, as expected.

  • The error of robust SPPIV is larger than that of multiresolution SPPIV, especially in the case of \(8 \times 8\;\text {pixel}^2\) interrogation windows, while robust SPPIV requires tuning of the hyperparameter.

  • The additional computational costs of robust and multiresoltion SPPIV are almost negligible.

  • In the present setup of experiments, multiresolution SPPIV is recommended.

Throughout the paper, the sensor optimization technique is also discussed. The result is summarized:

  • The processing interrogation windows selected by the KF index-based sensor optimization method give better results than those by the snapshot-to-snapshot DG method.