Multi-dimensional confocal laser scanning microscopy image correlation for nanoparticle flow velocimetry

Abstract

We present a new multi-dimensional confocal laser scanning microscopy (CLSM) image correlation for nanoparticle flow velocimetry that is robust to sources of decorrelating errors. Random and bias errors from nanoparticle flow measurements exacerbate with increased dimensionality in CLSM images, rendering measurements unusable. Our new algorithm tackles these measurement limitations in twofold. First, we model and correct for the bias errors introduced by the effects of the volumetric laser scanning image acquisition. Second, we developed a new spectral filter using a phase-quality masking technique that optimizes its size for the spectral content of CLSM images, without requiring a priori knowledge of displacement fields or flow tracer properties. We validated our algorithm using synthetic images and experimentally obtained 2D and 3D CLSM images of nanoparticle flow through a micro-channel. We show that our technique significantly outperforms the standard cross-correlation (SCC) in reducing both the random and bias errors and accelerated the convergence of ensemble correlation velocity measurements from CLSM images.

1 Introduction

The correlation-based velocity measurements from confocal laser scanning microscopy (CLSM) images are subject to random error due to the Brownian motion of nanometer-sized tracer particles, and a bias error due to the formation of images by raster scanning (Jun 2016). In our recent work on the development of the scanning laser image correlation–robust phase correlation (SLICR), the results demonstrated improved measurement accuracy by establishing a method that applies an optimal spectral filter diameter combined with an analytical model of the CLSM measurement bias error (Jun 2016). Our analysis verified that these errors primarily depend both on the particle size and fluid to scanner velocity ratio (FSR), rather than on either parameter alone. The work presented the first successful attempt to quantify and mitigate the errors of CLSM-based flow velocimetry using diffusion-dominated nanoparticles as flow tracers. While the SLICR processing algorithm was designed for one-dimensional measurements, in practice CLSM is mostly used to visualize two- or three-dimensional microscopic fields. The objective of this work is to identify and understand the effects of both random and bias errors in 2D and 3D CLSM measurements, and subsequently expand and improve the SLICR algorithm to mitigate the errors in multi-dimensional imaging with CLSM.

The CLSM has been a popular and useful tool for life science researchers over the past several decades, primarily due to its ability to remove blur from outside of the focal plane of the image (Digman et al. 2005, 2013; Raben et al. 2013; Rossow et al. 2010a, b). Nowadays, CLSM is commonplace in essentially all biomedical research institutions (Jonkman and Brown 2015). However, CLSM’s strength in high contrast and spatial resolution comes at the expense of low temporal resolution (Jun 2016; Digman et al. 2013; Rossow et al. 2010a, b; Jonkman and Brown 2015). CLSM is slower than widefield imaging because an image is built up point by point by a focused laser beam scanning across the sample. This drawback causes significant random and bias errors when imaging flow tracers to measure particle velocity of interest. These errors increase with respect to the added dimensionality in CLSM images due to the longer time it takes to acquire multi-dimension scans. As a result, performing particle image velocimetry (PIV) technique with CLSM measurements requires an understanding of the effects of three parameters: the diffusion of the tracer particles, the laser scanning speed, and the velocity of the flow (Jun 2016).

First, a random error due to the Brownian motion of small tracer particles increases with respect to higher dimensional CLSM measurements. As the image acquisition time increases, the particles’ positions will deviate further from those of the fluid pathlines (Einstein 1905; Olsen and Adrian 2000). In the SLICR algorithm, we optimized the width of the Gaussian RPC filter using the Monte Carlo error analysis of unidirectional flows of tracer particles that were computer-generated 1D CLSM images to improve the correlation SNR and reduce random errors (Jun 2016; Eckstein and Vlachos 2009a, b; Eckstein et al. 2008). However, in practice, each measurement has a unique SNR that varies based on many factors such as a particle size, the nature of the flow field, and the characteristics of the image noise. Applying a single-sized RPC filter to multiple measurements acquired under different conditions is a limitation itself, and there is a need for developing a dynamic filter that changes to the optimal size for each measurement. To date, no study has examined these problems or demonstrated a means to deal with them in 2D or 3D CLSM velocimetry.

Second, the FSR is the primary driver of the bias error. In our previous work, we showed that the bias error increases with respect to the ratio of the mean velocity of the tracer particles to that of the laser scanner. The bias error depends only on the FSR; therefore, the bias error correction is still required under the absence of any Brownian motion in the imaging domain (Jun 2016). If the FSR is high, the positions of the particles within the CLSM images will be distorted due to the longer delay between pixels, as each pixel is recorded sequentially in time (Jun 2016). Accordingly, the FSR will have multiple components for each dimension added to the CLSM measurements. This added complexity is due to CLSM’s raster scanning pattern, in which a digital image is built-up point by point as a small focused laser beam is scanned across the specimen. The three-dimensional volume is constructed after scanning the two-dimensional x–y (in-plane) domain in multiple z (out-of-plane) locations, as illustrated in Fig. 1. Accordingly, the effective scanner velocity will be slowest along the z-plane and fastest along the x-plane. If the fluid velocity has one component, that is aligned with the scanning axis of the CLSM, then the 1D analytical bias correction model developed from our previous work (Jun 2016) can be applied to the measured velocity to correct for the bias error. However, in practice, flow occurs with multiple velocity components irrespective of the alignment with the scanning axis of the CLSM. Furthermore, the velocity measurement will have varying magnitudes of bias error in different velocity components, which is contributed by the FSR. However, no protocol exists to mitigate these errors.

Fig. 1

a Illustration of 2D raster scanning pattern from CLSM, reconstructing 2D image coordinated by scanning mirrors. A 2D image is formed by scanning multiple lines row by row along the scanning axis, b 3D scan from CLSM, reconstructing a volume from optical sectioning using the piezo-electric positioning stage. A 3D volume scan is formed by scanning multiple 2D images along the direction of the piezo stage scan and c the presence of the bias error in each component of the velocity measurement. The dotted area represents the measured three-dimensional displacement including the bias error (yellow colored area) while the actual particle displacement is represented by the red colored area

Despite this shortcoming, scanning laser image correlation (SLIC) technique used with CLSM images (first introduced by Rossow et al. 2010a, b) has become a popular tool for biomedical engineering researchers investigating micro-channel flow and micro-circulation within live samples (Malone et al. 2007; Pan et al. 2009; Sironi 2014) in the last few years. More recently, Sironi et al. used a SLIC-like algorithm called flow image correlation spectroscopy (FLICS) to measure the velocities of flow tracer particles from a single 2D CLSM image (Sironi 2014). In this study, the researchers captured 2D CLSM images of 1-µm particles flowing in a microfluidic channel, and in vivo blood flow measurements in the circulatory systems of zebrafish embryos and mouse livers. In this case, information regarding the two-dimensional velocity was quantified by computing the cross-correlation of the fluorescence fluctuations detected in pairs of columns of a selected region of interest of the 2D CLSM image where diagonal lines appear (Sironi 2014). However, without addressing the random and bias errors in which correlating CLSM images are subject to, FLICS will yield less accurate measurements. First, the stretched particle shape that appears as diagonal lines in their 2D CLSM images were caused due to the high FSR which will indeed contribute to the bias error. Second, the velocity measurement quantified from a single image for nanometer-sized tracer particles will probably have a significant presence of random errors caused by Brownian motion and image noise. Although the particle size used by Sironi et al. was considerably large (1-µm microsphere and red blood cells in the range between 3.0 and 10 µm) with a lower diffusion coefficient than nanoparticles, the velocities of these tracers within the circulatory systems change over time. Therefore, the velocity measurement from a single image does not represent the time-varying flow field. Furthermore, the stretched-out shape of the particles in the CLSM image will cause the intensity peak shape to broaden in the standard cross-correlation matrix, which can further increase the random error of the measurements. We addressed this issue previously with 1D CLSM images by minimizing the uncertainties associated with the particle shape on the displacement measurements using the phase-only filtering techniques such as RPC (Jun 2016). Nevertheless, the aforementioned limitations with measurement accuracies are still observed among many published studies utilizing CLSM velocimetry (Digman et al. 2005, 2013; Rossow et al. 2010a, b; Sironi 2014).

A processing algorithm for mitigating measurement errors to which multi-dimensional CLSM images are subject to will enable development of more robust systems for using CLSM velocimetry to characterize micro- and nanoscale flow kinematics. Here, we propose a processing method that combines a dynamic spectral filter using the phase-quality map (Ghiglia and Pritt 1998) and expanded SLICR algorithm (Jun 2016) that mitigates both random and bias errors for 2D and 3D CLSM velocity measurements.

2 Methodology

2.1 Phase-quality masking

Eckstein et al. (Eckstein and Vlachos 2009b) showed that the signal-to-noise ratio (SNR) of the cross-power spectrum between PIV interrogation regions decreased with increasing spectral wave number. They showed that this SNR was Gaussian shaped when the images of the particles were themselves Gaussian, and derived the standard deviation of the SNR envelope as a function of the particles’ diameters. While the assumption of Gaussian-shaped particles is valid in traditional PIV images (Olsen and Adrian 2001), the shapes of the particles in CLSM images highly depend on the interrogated flow and the microscope parameters, and cannot be estimated a priori. This incongruence suggests that Eckstein’s “RPC” filter cannot model the SNR in CLSM images and applying such a pre-computed spectral filter to CLSM cross-correlations will yield suboptimal results. To address this, we developed an algorithm to measure the variation of SNR across spectral wave numbers in the complex cross-correlation and suppress regions where it is low. Our algorithm is based on insight into the behavior of the cross-correlation in the Fourier domain and is described below.

Recall that the Fourier Shift Theorem illustrates the relationship between the Fourier transform of an image and that of a shifted copy of itself (Westerweel 1997):

$$\mathcal{F}\left\{ {f\left( {x - {s_x},y - {s_y}} \right)} \right\}\left( {k,m} \right)={{\text{e}}^{ - i2\pi \left( {\frac{{{s_x}k}}{w}+\frac{{{s_y}m}}{h}} \right)}}F(k,m),$$

(1)

where ${s_x}$ and ${s_y}$ are the horizontal and vertical shifts of the pattern in pixels, $k$ and $m$ are the horizontal and vertical wave numbers in pixels⁻¹, w and h are the width and height of the image in pixels, and $F(k,m)$ is the Fourier transform of the pattern $f\left( {x,y} \right)$. From the cross-correlation theorem,

$$C\left( {k,m} \right)=~\mathcal{F}\left\{ {f*g} \right\}\left( {k,m} \right)=F \cdot {G^*}~,$$

(2)

where $C\left( {k,m} \right)$ is the cross-correlation of $f\left( {x,y} \right)$ and $g\left( {x,y} \right);~G\left( {k,m} \right)$ is the Fourier transform of $g\left( {x,y} \right)$, and ${G^*}\left( {k,m} \right)$ is the complex conjugate of $G$. If $g\left( {x,y} \right)$ is a shifted copy of $f\left( {x,y} \right)$ (that is, $g\left( {x,y} \right)~f\left( {x - {s_x},y - {s_y}} \right)$) then by Eq. (1),

$$C\left( {k,m} \right)={{\text{e}}^{i2\pi \left( {\frac{{{s_x}k}}{w}+\frac{{{s_y}m}}{h}} \right)}}F \cdot {F^*}.$$

(3)

The phase of the cross-correlation is given by

$$R\left( {k,m} \right)=\frac{C}{{\sqrt {C \cdot {C^*}} }}={{\text{e}}^{i2\pi \left( {\frac{{{s_x}k}}{w}+\frac{{{s_y}m}}{h}} \right)}}.$$

(4)

By Euler’s theorem, $R\left( {k,m} \right)$ is a complex sinusoid whose phase angle $\phi \left( {m,n} \right)$ can be represented in the Fourier domain as a plane of slope $\phi \left( {k,m} \right)=2\pi \left( {{s_x}k/w+{s_y}m/h} \right)$ “wrapped” to the range $\pm \pi$. In practice, $\phi \left( {k,m} \right)$ is calculated as

$$\phi \left( {m,n} \right)={\tan ^{ - 1}}\left( {\frac{{\mathcal{I}(R)}}{{\mathcal{R}(R)}}} \right)~,$$

(5)

where $\mathcal{R}(R)$ and $\mathcal{I}(R)$ are the real and imaginary components of $R\left( {k,m} \right)$, and ${\tan ^{ - 1}}$ is the two-dimensional inverse tangent. Figure 2 illustrates the theoretical relationship between displacements relating pairs of noise-free, perfectly correlated patterns in the spatial domain and the corresponding phase angle of their cross-correlations in the two-dimensional Fourier domain. Because $\phi \left( {k,m} \right)$ is a (wrapped) plane, we refer to it as the “phase-angle plane.”

Fig. 2

Phase-angle planes $\phi \left( {k,m} \right)$ derived from Eqs. (1)–(5) for two different two-dimensional displacements over a grid of 64 × 64 pixels

In practice, the phase-angle-relating PIV image pairs are not a perfect plane, but exhibit the effects of noise and the variable SNR due to the shapes of the particles as explained by Eckstein et al. (Eckstein and Vlachos 2009a, b). The top row of Fig. 3 illustrates this, and shows phase-angle planes derived from CLSM ensemble cross-correlations (Meinhart et al. 1999, 2000; Raffel 2007) 3- and 100-nm particles. The regions of these phase-angle planes that appear “smooth” and qualitatively resemble those of Fig. 2 contribute energy to the correlation peak in the spatial domain that corresponds to the best estimate of the image pattern displacement, and the regions that appear “noisy” contribute to noise in the spatial cross-correlation and to error in the displacement estimate. We, therefore, developed an algorithm to automatically identify and preferentially weigh the smooth regions of the phase-angle plane to the cross correlation in the spatial domain, and concomitantly to suppress the contribution of the “noisy” regions.

Fig. 3

Top row: phase-angle planes calculated from experimental CLSM images using 3- and 100-nm diameter particles. $\phi \left( {k,m} \right)$ is smoother toward the center of the domain (i.e., the lower wave numbers) and the prominence of the noise increases at higher wave numbers. Bottom row: phase-quality arrays calculated from the above phase-angle planes. Lower values in the phase quality correspond to smoother regions of $\phi \left( {k,m} \right)$, and thus to areas of higher SNR of the cross-correlation in the Fourier domain

Our algorithm is based on the fact that because $\phi \left( {k,m} \right)$ is theoretically a plane, its gradient is constant except where it wraps to $\pm \pi$. The gradient of $\phi \left( {k,m} \right)$ can be calculated to account for its wrapping behavior using the “wrapped phase difference” described by Ghiglia and Pritt (1998):

$${\Delta ^k}\left( {k,m} \right)={\tan ^{ - 1}}\left( {\frac{{\sin \left( {\phi \left( {k,m} \right) - \phi \left( {k - 1,m} \right)} \right)}}{{\cos \left( {\phi \left( {k,m} \right) - \phi \left( {k - 1,m} \right)} \right)}}} \right)~\quad {\Delta ^m}\left( {k,m} \right)={\tan ^{ - 1}}\left( {\frac{{\sin \left( {\phi \left( {k,m} \right) - \phi \left( {k,m - 1} \right)} \right)}}{{\cos \left( {\phi \left( {k,m} \right) - \phi \left( {k,m - 1} \right)} \right)}}} \right).$$

(6)

Here, ${\Delta ^k}$ and ${\Delta ^m}$ are constant across the Fourier domain when $\phi \left( {k,m} \right)$ is free of noise, and their variances are, therefore, identically zero. Accordingly, non-zero variance in either ${\Delta ^k}$ or ${\Delta ^m}$ indicates the presence of noise in the phase plane. We use a “phase-quality” metric $q\left( {k,m} \right)$ (Ghiglia and Pritt 1998) to quantify the local variance of $\nabla \phi \left( {k,m} \right)$ as

$$q\left( {k,m} \right)=\frac{1}{{{{(2r+1)}^2}}}\left( {\sqrt {\mathop \sum \limits_{{i=k - r}}^{{k+r}} \mathop \sum \limits_{{j=m - r}}^{{m+r}} {{\left( {{\Delta ^k}_{{i,j}} - {{\overline {{{\Delta ^k}}} }_{k,m}}} \right)}^2}} +\sqrt {\mathop \sum \limits_{{i=k - r}}^{{k+r}} \mathop \sum \limits_{{j=m - r}}^{{m+r}} {{\left( {{\Delta ^m}_{{i,j}} - {{\overline {{{\Delta ^m}}} }_{k,m}}} \right)}^2}} } \right)~.$$

(7)

Here, $\Delta _{{i,j}}^{k}$ and $\Delta _{{i,j}}^{m}$ are the scalar wrapped phase differences evaluated at the position $k=i,~\quad m=j$, and the overbars denote averaged quantities within a square region of width and height$~2r+1$. Because $q\left( {k,m} \right)$ measures deviation from smoothness of $\phi \left( {k,m} \right)$, larger values of $q\left( {k,m} \right)$ correspond to regions of $\phi \left( {k,m} \right)$ of higher noise and lower values correspond to regions of lower noise. Figure 3 illustrates the relationship between phase-quality and noise in the phase plane for representative CLSM PIV ensemble cross-correlations.

Our algorithm based on the phase quality of the cross-correlation is called phase-quality masking (PQM), and is depicted in Fig. 4. First, the cross-correlation between image pairs is calculated in the typical fashion of conjugate multiplying their Fourier transforms. If the “ensemble” correlation is used, then the summation of cross-correlations is performed in the Fourier domain. The phase quality of the cross-correlation is then calculated according to Eq. (7). Next, the phase quality is binarized using a thresholding operation, inverted, and then segmented into eight-connected regions (Efford 2000). The result of using eight-connected regions is that each pixel in the binarized mask is assigned to the largest group of pixels that is closest to the center of the array, then the shape of the smallest ellipse that can encircle that group is calculated, as shown from Fig. 4.

Fig. 4

Algorithm for constructing the PQM

For each region, we calculate a median-weighted centroid $\tilde {p}$ as the product of the radial coordinate of its centroid with the median radial coordinate of its constituent pixels, i.e.,

$${\tilde {p}_i}={p_i} \cdot {\text{median}}({r_i})~,$$

(8)

where ${p_i}$ is the radial coordinate of the ith region’s centroid and ${r_i}$ is the vector of radial coordinates of the set of pixels that constitute the ith region. We chose the region for which $\tilde {p}$ is the minimum as the one corresponding to the “high-quality” (i.e., low noise) region of the cross-correlation. We then fit an ellipse to this region, and construct a radially symmetric Gaussian spectral filter whose standard deviation is equal to one-fourth of the minor axis of that ellipse. We refer to this Gaussian function as the PQM, which is then used in lieu of the pre-computed “robust phase correlation” filter of Eckstein and Vlachos (2009b). The complex phase correlation $R\left( {k,m} \right)$ is thus multiplied by our PQM filter prior to computing its inverse Fourier transform and estimating the pattern displacement by locating its maximum value in the spatial domain.

2.2 Bias error correction in three dimensions

The particle image positions in CLSM are influenced by three primary factors: three-dimensional fluid velocity $({U_{\text{f}}},~{V_{\text{f}}},~{W_{\text{f}}})$, scanner velocity $({U_{\text{s}}},~{V_{\text{s}}},~{W_{\text{s}}})$, and random displacement $({X^\prime },{Y^\prime },{Z^\prime })$ caused by Brownian motion. It should be noted that the 3D coordinate axes X, Y, and Z are specifically assigned to the scanning axes of the CLSM. The coordinate axis X represents the primary scanning axis illustrated as a horizontal line (yellow arrow) in Fig. 1a. The primary scanning axis (X) alone can generate 1D CLSM images. The coordinate axis Y represents the secondary scanning axis used to generate 2D CLSM images, which repositions the primary scanning axis (X) as illustrated in Fig. 1a. The coordinate axis Z represents the third (piezo stage) scanning axis which repositions the primary and secondary scanning axes (X and Y) in different plane, constructing 3D volumes consists of stacked 2D CLSM images (Fig. 1a). Equation (9) represents the positions of 3-D confocal images of a tracer particle in a uniform, unidirectional flow. The ${X_{f,p}}$ is the position in the image of the p^th particle from the recorded confocal volume scan (fth frame). $X_{{f,p}}^{0}$ refers to the initial position of the pth particle in space at the beginning of the fth frame, and $t_{{f,p}}^{X}$ is the time elapsed between the beginning of the fth frame and the moment the scanner arrives at the position of the pth particle (i.e., ${X_{f,p}}/{U_{\text{s}}}$) along X-coordinate. $X_{{f,p}}^{\prime }$ represents the random displacement of the p^th particle during the interval ${t_{f,p}}$. The same notations apply for other two dimensions (Y- and Z-coordinates) of the particle position. Subsequently, the same definition is assigned for the next consecutive position and time variables used for each consecutive frame (f = 1, 2, 3 …). It should be noted that due to the raster scanning pattern of the CLSM, the elapsed scanning time along the Y-direction depends on the total elapsed scanning time along X-direction, since the secondary scanning axis (Y) moves once the primary scanning axis (X) completes scanning a line (Fig. 1). Subsequently, the elapsed scanning time along the Z-direction is dependent on both the total scanning time along X- and Y-directions, respectively, as the third scanning axis (Z) will not move while 2D images are being constructed. This means that the particle position in X-coordinate will involve a number of additional displacements due to the time elapsed while scanner is moving in Y and Z-directions as well. Therefore, the X-coordinate position (${X_{f,p}}$) of the particle recorded in the image consists of the elapsed scanning time along all axes X, Y, and Z ($t_{{f,p}}^{X}$, $t_{{f,p}}^{Y}$ and $t_{{f,p}}^{Z}$), as represented in Eq. (9). Accordingly, the Y-coordinate position (${Y_{f,p}}$) of the particle recorded in the image will consist of the elapsed scanning time along Y- and Z-axes (Eq. 10) and the Z-coordinate position $({Z_{f,p}})$ will involve only the elapsed scanning time along Z-axis (Eq. 11).

$${X_{f,p}}=X_{{f+1,p}}^{0}+{U_f} \cdot t_{{f,p}}^{X}+{U_f} \cdot t_{{f,p}}^{Y}+{U_f} \cdot t_{{f,p}}^{Z}+X_{{f,p}}^{\prime },$$

(9)

$${Y_{f,p}}=Y_{{f+1,p}}^{0}+{V_f} \cdot t_{{f,p}}^{Y}+{V_f} \cdot t_{{f,p}}^{Z}+Y_{{f,p}}^{\prime },$$

(10)

$${Z_{f,p}}=Z_{{f+1,p}}^{0}+{W_f} \cdot t_{{f,p}}^{Z}+Z_{{f,p}}^{\prime }.$$

(11)

Subsequently, the displacement of particles between consecutive frames is given by the following equations.

$${X_{f+1,p}} - {X_{f,p}}=X_{{f+1,p}}^{0} - X_{{f,p}}^{0}+{U_f} \cdot \left( {t_{{f+1,p}}^{X}+t_{{f+1,p}}^{Y}+t_{{f+1,p}}^{Z} - t_{{f,p}}^{X} - t_{{f,p}}^{Y} - t_{{f,p}}^{Z}} \right)+~X_{{f+1,p}}^{\prime } - X_{{f,p}}^{\prime }~,$$

(12)

$${Y_{f+1,p}} - {Y_{f,p}}=Y_{{f+1,p}}^{0} - Y_{{f,p}}^{0}+{V_f} \cdot \left( {t_{{f+1,p}}^{Y}+t_{{f+1,p}}^{Z} - t_{{f,p}}^{Y} - t_{{f,p}}^{Z}} \right)+~Y_{{f+1,p}}^{\prime } - Y_{{f,p}}^{\prime }~,$$

(13)

$${Z_{f+1,p}} - {Z_{f,p}}=Z_{{f+1,p}}^{0} - Z_{{f,p}}^{0}+{W_f} \cdot \left( {t_{{f+1,p}}^{Z} - t_{{f,p}}^{Z}} \right)+~Z_{{f+1,p}}^{\prime } - Z_{{f,p}}^{\prime },$$

(14)

The difference ${X_{f+1,p}} - {X_{f,p}}$ will be referred to as $\Delta {X_{f,p}}$ and $\Delta t_{{f,p}}^{X}=t_{{f+1,p}}^{X} - t_{{f,p}}^{X}$, which is the interval between the times at which the scanner reaches the position of the pth particle in subsequent frames $\left( {\frac{{\Delta {X_{f,p}}~}}{{{U_{\text{s}}}}}+\frac{{\Delta {Y_{f,p}}~}}{{{V_{\text{s}}}}}+\frac{{\Delta {Z_{f,p}}~}}{{{W_{\text{s}}}}}} \right)$. Initial positions $X_{{f+1,p}}^{0} - X_{{f,p}}^{0}$ (referred as $\Delta X_{{f,p}}^{0}$) can be expressed alternatively as Eq. (15), $\Delta t$ is the recorded elapsed time per frame from CLSM and $~X_{{f,p}}^{{0\prime }}~$is the random displacement that occurred between the beginnings of frames f and f + 1.

$$\Delta X_{{f,p}}^{0}={U_f} \cdot \Delta t+\Delta X_{{f,p}}^{{0\prime }}.$$

(15)

Substituting Eq. (15) in Eq. (12), we get

$$\Delta {X_{f,p}}={U_f} \cdot \Delta t+{U_f}\cdot\left( {\frac{{\Delta {X_{f,p}}~}}{{{U_s}}}+\frac{{\Delta {Y_{f,p}}~}}{{{V_s}}}+\frac{{\Delta {Z_{f,p}}~}}{{{W_s}}}} \right)+X_{{f+1,p}}^{\prime } - X_{{f,p}}^{\prime }+X_{{f,p}}^{{0\prime }}.$$

(16)

Additionally, the particle displacement in the image can also be expressed in terms of the laser scanning velocities $({U_{\text{s}}},~{V_{\text{s}}},~{W_{\text{s}}})$ which are represented below.

$$\Delta {X_{f,p}}={U_s} \cdot \Delta t_{{f,p}}^{X}+{V_s} \cdot \Delta t_{{f,p}}^{Y}+{W_s} \cdot \Delta t_{{f,p}}^{Z}.$$

(17)

The unknown variables from Eq. (16) are ${U_f}$ and the combined random displacement, $X_{{f+1,p}}^{\prime } - X_{{f,p}}^{\prime }+X_{{f,p}}^{{0\prime }}~~,$which occurred over three different time instances. In terms of measurements, we can regard the ensemble-averaged $\left\langle {\Delta {X_{f,p}}} \right\rangle$ as the most probable displacement of particles estimated by averaging cross-correlation of a confocal scanned image pair over total number of particles (q) per each frame with a sufficiently large total number of frames (m), given as follows:

$$\left\langle {\Delta {X_{f,p}}} \right\rangle ~=\frac{1}{m}\mathop \sum \limits_{{f=1}}^{m} \frac{1}{q}\mathop \sum \limits_{{p=1}}^{q} \Delta {X_{f,p}}.$$

(18)

Equation below represents the ensemble average of each term.

$$\left\langle {\Delta {X_{f,p}}} \right\rangle ={U_f} \cdot \Delta t+{U_f} \cdot \frac{{\left\langle {\Delta {X_{f,p}}} \right\rangle }}{{{U_s}}}+{V_f} \cdot \frac{{\left\langle {\Delta {Y_{f,p}}~} \right\rangle }}{{{V_s}}}+{W_f} \cdot \frac{{\left\langle {\Delta {Z_{f,p}}} \right\rangle }}{{{W_s}}}+X_{{f+1,p}}^{\prime } - X_{{f,p}}^{\prime }+X_{{f,p}}^{{0\prime }}.$$

(19)

The ensemble average of the random displacements $\left\langle {X_{{f+1,p}}^{\prime } - X_{{f,p}}^{\prime }+X_{{f,p}}^{{0\prime }}} \right\rangle$ will yield zero, due to Brownian motion which can be modeled as normally distributed variable with zero mean (Einstein 1905). Subsequently, we get the following equation with three unknown variables ${U_f}$, ${V_f}$ and ${W_f}$:

$$\left\langle {\Delta {X_{f,p}}} \right\rangle ={U_f} \cdot \Delta t+{U_f} \cdot \frac{{\left\langle {\Delta {X_{f,p}}~} \right\rangle }}{{{U_s}}}+{V_f} \cdot \frac{{\left\langle {\Delta {Y_{f,p}}~} \right\rangle }}{{{V_s}}}+{W_f} \cdot \frac{{\left\langle {\Delta {Z_{f,p}}~} \right\rangle }}{{{W_s}}}~.$$

(20)

After rearranging, the fluid velocity ${U_f}$ can now be decoupled from the measured ensemble-averaged displacement $\left\langle {\Delta {X_{f,p}}} \right\rangle$, which consists of the bias error due to the effect of scanning, given as follows:

$${U_f}=\frac{{\left\langle {\Delta {X_{f,p}}} \right\rangle }}{{\left( {\Delta t+\left\langle {\Delta {X_{f,p}}} \right\rangle /{U_s}+\left\langle {\Delta {Y_{f,p}}} \right\rangle /{V_s}+\left\langle {\Delta {Z_{f,p}}} \right\rangle /{W_s}} \right)}}~~$$

(21)

Finally, applying the same steps used to derive Eq. (21) for other two dimensions, we yield the following equations, to solve for three unknown variables ${U_f}$, ${V_f}$ and ${W_f}$:

$${V_f}=\frac{{\left\langle {\Delta {V_{f,p}}} \right\rangle }}{{\left( {\Delta t+\left\langle {\Delta {Y_{f,p}}} \right\rangle /{V_s}+\left\langle {\Delta {Z_{f,p}}} \right\rangle /{W_s}} \right)}}~~,$$

(22)

$${W_f}=\frac{{\left\langle {\Delta {X_{f,p}}} \right\rangle }}{{\left( {\Delta t+\left\langle {\Delta {Z_{f,p}}} \right\rangle /{W_s}} \right)}}~~.$$

(23)

2.3 Processing algorithm

Figure 5 introduces the overall processing algorithm called scanning laser image correlation–phase quality (SLICQ). After filtering the phase correlation with the PQM (described in detail from Figs. 2, 3, 4), the next step calculates its inverse FT and apply the sub-pixel fit to identify the most probable displacement of the image pattern. Finally, the multi-dimensional bias corrector is applied to the sub-pixel displacement, which yields our best estimate of the time-averaged displacement of the particles that were imaged.

Fig. 5

SLICQ processing algorithm for measuring time-averaged velocity of particles imaged by multi-dimensional CLSM

2.4 Synthetic image generation

CLSM images of particles under unidirectional flow were rendered by sampling synthetic particle flow fields pixel-by-pixel, scanning under raster scanning pattern across multiple planes (Fig. 1), while continuously advecting the underlying tracer particles. The interrogated flow field was simulated over a 2D and 3D domain, and was seeded with tracer particles (3 and 100 nm in size) with random initial vertical and horizontal coordinates. All the synthetic images were generated with the seeding density of 10⁻² particles/pix², which was chosen to prevent particle aggregation during the experiments. The domain was discretized into interrogation regions with size of 512 × 64 pixels for 2D images and 512 × 64 × 64 pixels for 3D images, each of which represented a single station at which the CLSM scanner sampled the field. In our simulations, the FSR was varied from 0.0 to 3.3 × 10⁻¹. The diffusion was modeled based on the Stokes–Einstein equation with the statistics of Brownian motion. The specific details of the image formation process and modeling particle diffusion with advection can be found in Jun et al. (Jun 2016).

2.5 Micro-channel flow experiments

To evaluate the performance of our SLICQ algorithm on real data, we collected 2D and 3D confocal images of nanometer-sized tracer particles suspended in water flowing through a plastic microfluidic channel of rectangular cross-section (µ–Slides I Luer, ibidi Inc) in the middle and circular cross-section at the inlet. Figure 6 illustrates the overall experimental system and imaging location. The dimensions of the rectangular cross-section were 0.1 mm (depth) × 5.0 mm (width) and 4.0 mm for the diameter of the circular cross-section (x–y coordinates) at the inlet. Polystyrene fluorescent microspheres (0.1 µm diameter; Fisher Scientific) and mCherry fluorescent protein (3.0 nm average hydrodynamic diameter; BioVision, 694,614) were used as tracer particles. The channel was filled with a suspension of particles in water, and the concentration and seeding density for both 3 and 100 nm particles were 10⁻² mg/mL and 10⁻² particles/pix², respectively. The volumetric flow rate through the channel was controlled by a syringe pump (Harvard Apparatus), and ranged from 2.0 to 6.0 µL/s. The interrogation region for the rectangular cross-section involved ten different positions spaced along Z-axis at three different angles of rotations 0°, 45° and 90° with respect to the scanner direction (Fig. 6) while for the circular section involved one center position along Z-axis at three different flow rates (2.0, 4.0 and 6.0 µL/s). The inlet region was selected to measure the out-of-plane velocity (W) component along the Z-axis, which cannot be measured within the rectangular channel (U and V components only). The expected flow velocities at the interrogation spots ranged from 2.0 to 1.5 × 10² µm/s.

Fig. 6

a Schematic of the experimental setup used to obtain CLSM images of nanometer-sized tracer particles flowing in water through a micro-channel. b Illustration of the orientation of the micro-channel with respect to the scanning direction

A Nikon A1R scanning laser confocal microscope (Nikon Corporation, Tokyo, Japan) was used to image the flow through the microfluidic channel. The channel was viewed through a 40x objective lens (numerical aperture NA = 1.25, working distance of 0.61 mm), and illuminated by an argon ion laser (561-nm wavelength). The image sizes for 2D and 3D scans were 512 × 64 pixels and 512 × 64 × 64 pixels, respectively. Dwell time at each pixel was 0.1–2.2 µs, with an image magnification of 1.6 × 10⁻¹ µm per pixel. Each trial consisted of 5.0 × 10³ consecutive frames along the same image region. Total scan time per frame for 2D and 3D images ranged from 9.0 to 2.2 × 10² ms, which includes additional time pausing at the beginning and end of each line. Spatial resolution of the automated piezo scan (Z step size) was 3.0 × 10⁻² µm. The apparent CLSM particle image diameters (2.3 and 4.8 pixels for 3- and 100-nm particles, respectively) were estimated using the auto-correlation diameter measured from the experimental images. As discussed in the introduction, the shape and size of the nanoparticles recorded in the CLSM images are different from that of traditional PIV images. First, because of the finite scanning velocity of CLSM, the motion of diffusion-dominated particles is often significant on the time scale of the acquisition of point by point scanning. This introduces an imaging artifact similar to motion blur in traditional cameras. Second, the nanoparticles are imaged under the diffraction limit of the optics. Therefore, the CLSM images cannot accurately locate a single particle and distinguish close particles as separate from one another. As a result, it should be noted that the CLSM particle image diameters we are reporting is an apparent image size of one or more group of particles with imaging artifacts caused by the aforementioned limitations.

3 Results

3.1 Convergence of measurements

Figure 7 and Table 1 show a comparison of ensemble SLICQ and SCC velocity estimates convergence for 2D and 3D synthetic and experimental images. The metric used to quantify convergence was based on the change of the displacement between ensemble increments. The convergence criterion was that displacement changed by less than 0.1 pixels for a step increase in the ensemble pairs. An upper bound limit of 0.1 pixels was referenced from the standard deviation of 1000 displacement estimates (instantaneously cross-correlated) from synthetic CLSM images generated with no effect of diffusion. For both synthetic and experimental CLSM images, we quantified the number of ensemble image pairs required for convergence of the displacement estimate using our SLICQ compared to the SCC. The results show that the convergence was faster for the 100-nm than 3-nm particles for 2D images estimated with both SLICQ and SCC, due to the increased contribution of diffusion to the displacements of the smaller particles. Because of the significant random motions caused by the small particle sizes with longer time scale of the measurement in 2D and 3D images, the SCC failed to reach convergence for the expected velocity, and instead reached near-zero displacement caused by a strong auto-correlation peak in the SCC plane for the 3-nm particles (2D images) and both the 3- and 100-nm particles (3D images).

Fig. 7

Convergence behavior of SLICQ and SCC algorithms for a 2D and b 3D synthetic images of 3- and 100-nm flow tracer particles for the U component velocity measurement normalized by the expected value, corresponding to the measurement with the input velocity of 60 µm s⁻¹ with dwell time of 0.1 µs (22 data points per case are shown in the plot to distinguish markers)

Table 1 Convergence estimate for the measurement with CLSM images

3.2 Bias correction: synthetic images

Figure 8a shows the accuracy improvement of our bias correction model on the converged ensemble SLICQ velocity estimates in synthetic 2D images. This case represents the range of different FSR for X-direction while maintaining a fixed fluid to scanner velocity for Y-direction. The analytical errors (two theoretical lines in Fig. 8a) represent computed errors of the uncorrected theoretical measurements, purely based in Eqs. (21)–(23). The bias error increases with respect to a higher magnitude of the FSR with a maximum error magnitude of about 5 pixels. Conversely, low FSR decreased the bias error, approaching the behavior of traditional “snapshot” imaging. The filled markers in Fig. 8a show the remaining errors after the application of our bias correction model to both the X and Y components of the velocity measurements.

Fig. 8

Absolute velocity errors with respect to a the U velocity and b the V velocity ratio (fluid/scanning) measured from the converged ensemble SLICQ measurements from 2D CLSM synthetic images (20 data points per case are shown in the plot to distinguish markers)

Figure 8b shows the bias correction model with different FSR for the Y-direction. Despite the fixed FSR assigned on the X-direction, the uncorrected error magnitude (U) increased with respect to the higher magnitude of the FSR on the Y-direction. The uncorrected error magnitude (V) has a similar trend with lower values than the uncorrected error magnitude (U). Such a trend is expected based on the derivation from Eqs. (21)–(23) for the bias error correction, in which the X component of the fluid velocity will have increased bias error when there is a non-zero Y component of the fluid velocity.

Figure 9 shows the bias correction model for 3D CLSM images with different FSR for the Z-direction. The fluid to scanner velocities in the X and Y-directions are fixed. The uncorrected errors, the U, V and W components behave similarly, as seen in Fig. 8. Subsequently, higher bias errors were observed from the U and V components of the fluid velocities due to the increased elapsed time which was caused primarily by the W component of the fluid velocity, according to Eqs. (21)–(23).

Fig. 9

Absolute velocity errors with respect to the W velocity ratio (fluid/scanning) measured from the converged ensemble SLICQ measurements from 3D CLSM synthetic images of flowing 3- and 100-nm particles (20 data points per case are shown in the plot to distinguish markers)

These results from Figs. 8 and 9 indicate that applying our bias correction yielded 98% reductions in the mean and RMS errors, going from 3.3 to 4.9 × 10⁻² pixels and 3.5 to 5.8 × 10⁻² pixels.

3.3 Bias correction: experimental images

Figure 10a compares the theoretical and measured velocity profiles within the micro-channel using the converged SLICQ and un-converged SCC for both tracer particles. For experimental data, we estimated the ground-truth velocity analytically using the equation for fully developed Poiseuille flow, evaluated at the measurement locations that we interrogated. The theoretical confidence interval band was estimated based on the uncertainty of the calculated error by propagating (via the Taylor series expansion method) the known elemental sources of error in our experiment, and the RMS of the measured velocity, through the error equation. The elemental sources of error we considered were the volumetric flow rate delivered by the syringe pump, the physical location of the interrogation region, and the dimensions of the microfluidic channel, whose values were used to calculate the Poiseuille flow velocity profile.

Fig. 10

Comparison of different cases of 2D velocity measurements across the depth of the channel for a 0°, b 45° and c 90° angle between the scanning and input flow directions, compared to the theoretical solution for plane Poiseuille flow (flow rate 2 µl s⁻¹), FSR between 1.0 × 10⁻² and 3.0 × 10⁻¹. Whiskers indicate the 95% confidence interval about the mean velocity

In this case (a), the micro-channel’s length (X-coordinate in Fig. 6) was aligned with the scanning direction in the X-coordinate which would primarily induce the fluid velocity to be along the X-direction. For consistent comparison between methods, the SCC measurements were ensemble averaged using the same number of image pairs required to converge the SLICQ measurements. As mentioned previously in Fig. 7 and Table 1, the SCC measurements did not themselves converge. As shown in Fig. 10a, nearly all the U component velocity measurements for both 3- and 100-nm particles with SLICQ fell within the 95% confidence interval about the nominal theoretical velocity profile. The V component velocity measurements from SLICQ ranged from − 8.7 × 10⁻¹ to 1.8 pixels with a mean displacement of 3.4 × 10⁻¹ pixels among collected measurement planes. The effect of the bias error correction from the U component velocities seems marginal, which is due to the negligible presence of the V component velocities as discussed previously with the analytical bias error Eqs. (21)–(23). The presence of both the U and V component velocities were imposed by rotating the micro-channel’s width 45° with respect to the scanning direction in the X-coordinate (Fig. 10b). In principle, this particular orientation should result in an equal magnitude for both the U and V component velocities, as represented by the theoretical velocity profile in Fig. 10b. The uncorrected U and V component velocity measurements are well below the nominal theoretical velocity profile. Subsequently, the bias corrections decreased the error magnitude up to 10 pixels for both the U and V components. The un-converged SCC measurements remain near-zero displacement across all measurement planes.

Figure 10c represents the final case with 2D CLSM images, in which the micro-channel’s width was rotated 90° with respect to the scanning direction in X-coordinate. With this orientation, the U component velocities were near-zero displacement, as the scanning direction in X-coordinate is perpendicular to the direction of the flow. Theoretically, the V component velocities should show a similar trend to Fig. 10a, as for the one component velocity with the same flow rate. However, the measured V component velocities have lower magnitude across all measurement planes than the U component velocity profile from Fig. 10a, as a result of the bias error along Y-direction. After the bias error corrections, the V component velocities changed in magnitude by up to 20 pixels which brought the velocity profile much closer to the theoretical velocity profile.

Figure 11a represents the three-component velocity profiles measured from 3D CLSM images with the 3-nm mCherry protein and 100-nm particles in water flowing through the micro-channel (0° angle rotation with respect to the scanning direction in X-coordinate). The general trends for all cases for both SLICQ and SCC measurements look similar to Fig. 10a, since only the U component velocities are dominant with this orientation of the micro-channel. As a result, the effect of the bias error corrections is also negligible. The velocity magnitude for the U component velocities is higher than the 2D cases due to the longer acquisition time with volume scans.

Fig. 11

Comparison of different cases of 3D velocity measurements with 3-nm mCherry protein and 100-nm particles across. a The depth of the channel, compared to the theoretical solution for plane Poiseuille flow (flow rate 2 µl s⁻¹), FSR between 1.0 × 10⁻¹ and 3.0 × 10⁻¹ and b the center of the circular inlet of the channel (out-of-plane flow direction), compared to the theoretical centerline displacement (flow rate 2–6 µl s⁻¹, FSR between 1.0 × 10-22.5 × 10 − 1). Whiskers indicate the 95% confidence interval about the mean velocity

Figure 11b shows the three-component velocity measurements at the center of the circular inlet location, which is illustrated in Fig. 6. This specific location was chosen to capture the W component velocity inside the micro-channel since the flow direction across the rectangular cross-section will be predominantly U component alone. The center location of the circular cross-section was volume scanned to capture the strongest out-of-plane motion of the flow. The selected flow rates were 2.0, 4.0 and 6.0 µL/s, and measurements were compared to the theoretical solution for Poiseuille flow through a constant circular cross-section.

The three measured W component velocities with ensemble SLICQ are all outside the 95% confidence interval about the nominal theoretical velocity profile. After applying the bias error corrections, the measurements improved with a reduction in error magnitude by up to 7.0 pixels in W component velocities and 1.0 pixel in U and V component velocities. As with all the previous experimental cases, the measurements with SCC resulted in near-zero displacements for all three components of the velocity.

4 Discussion

Diffusion of the tracer particles, laser scanning speed, and velocity of the flow impact the accuracy of 2D and 3D CLSM-based flow velocimetry (Jun 2016). In this study, we tackle all these issues through a new processing method that combines use of the dynamic spectral filter and 3D bias correction model.

The Brownian motion of tracer particles in multi-dimensional CLSM images and the effects of their random errors were suppressed with the use of the PQM with ensemble averaging. The SLICQ demonstrated vastly improved efficiency and versatility as a dynamic spectral filter not requiring a priori knowledge of displacement fields such as diffusivity and particle diameter, which was not capable with SLICR algorithm (Jun 2016). The superior performance of our SLICQ compared to the SCC is evident from the failure of the SCC measurements for 2D and 3D CLSM images with 3- and 100-nm particles, respectively, from both synthetic and experimental results.

Figure 12 illustrates three main types of spatial correlation shapes. The first, Fig. 12a, represents converged SCC measurements with 100-nm particles captured in 2D images. Even though the correlation magnitude ratio from the first peak to the secondary broad Gaussian peak is less than 1.1 for all 2D measurements with 100-nm particles, ensemble SCC measurements with bias error corrections can yield as accurate displacement as SLICQ which is represented in Fig. 7.

Fig. 12

Representative spatial cross-correlation types observed from both synthetic and experimental images illustrating, a identified displacement peak from SCC (2D 100-nm particle images), b failed SCC without displacement peak (2D and 3D) 3-nm particle images and 3D 100-nm particle images and c identified displacement peak from SLICQ (all the CLSM images)

The second shown in Fig. 12b represents correlation shapes of all the other SCC measurements including 2D and 3D 3-nm particle images and 3D 100-nm particle images. As discussed in our previous SLICR study (Jun 2016), the spatial and temporal resolutions of CLSM devices are not optimal to resolve kinematics of such cases with use of SCC alone. Such limitations manifest as a broad Gaussian-shaped correlation in the ensemble-averaged SCC as shown in Fig. 12b. Even though the magnitude of the broad Gaussian-shaped peak shown in Fig. 12b is large (> 10¹⁰) due to ensemble correlation, the location of the peak is always centered near-zero displacement regardless of the flow velocity magnitude. The third type shows correlation shape from SLICQ after filtering with PQM in which the broad Gaussian shape peak no longer exists and maintains peak-to-peak ratio greater than 2.2 for all the cases. Unlike traditional pair-wise PIV with use of particle sizes larger than 1 µm, where well-known SNR metrics (such as correlation peak-to-peak ratio and height of the tallest correlation peak) can predict the measurement accuracy, these metrics do not apply well when the spatial domain is contaminated under decorrelating effects of diffusion and noise (Fig. 12a, b). As a result, we used measurement errors as a more rigorous metric representing the measurement robustness.

The main contributor that caused the SCC to fail is the rate of image acquisition being much longer than 1D acquisition of CLSM images which were assessed in our previous work (Jun 2016), more than the small tracer particle sizes. As the time allowed for nanoparticles to move gets longer, Brownian motion manifests as a broad Gaussian-shaped correlation in the ensemble-averaged SCC. This degrades the ability of peak-searching algorithms to identify the small correlation peak corresponding to the mean background velocity of the flow. In contrast to SCC, the RPC algorithm (Eckstein and Vlachos 2009b; Eckstein et al. 2008) recognized that the correlation SNR decreases with an increasing wavenumber. Subsequently, the PQM technique from SLICQ measures the SNR across the Fourier domain and suppresses the correlation where the SNR is measured to be low. Ultimately, the SLICQ’s ability to decouple the lower and high wave numbers representing true mean displacement and diffusion-dominated displacement, respectively, allows a search for the true peak even with the longer time scale of the acquisition used in this study.

The analysis of the propagation of bias errors in multi-dimensional CLSM measurements revealed that measurement with both larger magnitude and number of velocity component under the finite scanning velocity of CLSM involves higher bias error. With the damaging effects of the bias error as shown from our results, there are three parameters to consider selectively for CLSM imaging to minimize the FSR. First, it will be advantageous to use the fastest scanning velocity for CLSM recording regardless of the image dimension which will lower the FSR. Second, the image size must be chosen with respect to the expected velocity displacement. The most dependent scanning displacement (Y- and Z-dimensions for the 2D and 3D CLSM images, respectively) should be at least 20 times larger than the expected displacement for mitigating the error magnitude below 1.0 × 10⁻¹ pixel, according to Figs. 8 and 9. The experimental results from Figs. 11 and 12 also demonstrated the effects of image dimensions (scanned displacements) on the bias error, in which the extreme value of the FSR (maximum up to of 3.0 × 10⁻¹) was chosen to increase the uncorrected error magnitude. Finally, the fluid velocity can be optimized by controlling the input flow devices or geometry of the channel being used when there are limited choices of different scanner speeds.

Overall, the process of multi-dimensional image formation by CLSM and the use of diffusion-dominated nanoparticles as flow tracers further exacerbated the difficulties dealing with the random and bias error that we experienced with 1D SLIC measurements (Jun 2016). Our new algorithm (SLICQ) overcomes these limitations by expanding our 1D analytical model of CLSM imaging and replacing the single-sized RPC filter with the PQM that dynamically changes the filter size most optimal for the range of conditions tested in this study.

The primary limitation in this study was the flow condition being steady unidirectional and that our measurements required ensemble averaging. Additionally, the point-scanner nature of the CLSM system is disadvantageous when exploring more complex global flow fields that interest most PIV users. Nevertheless, CLSM having low temporal resolution (its main disadvantage), it is still considered the most popular choice and versatile 3D imaging system for a wide range of imaging applications (Rossow et al. 2010a, b; Jonkman and Brown 2015; Sironi 2014). Our method to yield reliable multi-dimensional scale velocity measurements with CLSM will provide many researchers with guidance in the designing of micro- and nanoscale flow experiments using these instruments.