Signal attenuation maps for needle enhancement and localization in 2D ultrasound

Abstract

Purpose

We propose a novel framework for enhancement and localization of steeply inserted hand-held needles under in-plane 2D ultrasound guidance.

Methods

Depth-dependent attenuation and non-axial specular reflection hinder visibility of steeply inserted needles. Here, we model signal transmission maps representative of the attenuation probability within the image domain. The maps are employed in a contextual regularization framework to recover needle shaft and tip information. The needle tip is automatically localized by line-fitting along the local-phase-directed trajectory, followed by statistical optimization.

Results

The proposed method was tested on 300 ex vivo ultrasound scans collected during insertion of an epidural needle into freshly excised porcine and bovine tissue. A tip localization accuracy of $0.55\pm 0.06\,\hbox {mm}$ was achieved.

Conclusion

The proposed method could be useful in challenging procedures where needle shaft and tip are inconspicuous. Improved needle localization results compared to previously proposed methods suggest that the proposed method is promising for further clinical evaluation.

Introduction

Many percutaneous interventional procedures such as biopsies [1] and regional anesthesia [2] involve insertion of a needle and directing it toward target anatomy under image guidance. Ultrasound (US) has emerged as a popular imaging modality in these procedures due to its low-cost, radiation-free and real-time imaging capabilities. However, at steep trajectories, non-axial specular reflection and signal attenuation cause needle shaft discontinuity and/or tip imperceptibility. An invisible needle affects procedure efficacy and may lead to injury. Therefore, solutions for localization or tracking of needles in US have been proposed. Broadly, these can be categorized as either hardware or software based.

Examples of hardware-based methods include mechanical needle guides [3], electromagnetic/optical tracking systems [4, 5] and specialized needles or probes [6,7,8]. Needle guides restrict needle redirection and are thus not suitable in procedures where fine needle trajectory adjustment is required. Electromagnetic/optical tracking enhancements increase overall cost of the imaging system and disrupt the normal clinical workflow. Moreover, electromagnetic systems are affected by metal objects in the operating environment. Needle or probe modification increases cost and complexity of the imaging system.

Concerning software-based approaches, we will limit our discussion to methods applied to 2D US due to wide availability of these systems. Methods based on the Hough transform (HT) have been proposed [9], but this approach assumes the needle will appear as the longest line-like feature with high intensity in the US image. At steep insertion angles, this is often violated due to attenuation and reflection of US waves away from the transducer. In [10], a learning-based framework for detection of an imperceptible needle was presented. Although the method facilitates localization of the needle trajectory, tip enhancement and localization were not investigated. In [11], a learning-based method for needle localization using 2D beam-steered US is proposed. Although a mean targeting error of 0.48 mm is achieved, US data for validation are collected within a 4.5 cm depth setting and shallow insertion angles. An approach for localizing discontinuous needles using local-phase-based projections was recently proposed by Hacihaliloglu et al. [12], but it fails when the tip is inconspicuous. For shaft enhancement, a method that relies on vibration of a transducer-actuated needle has been proposed [13], but it necessitates specialized needles in which the needle’s resonance frequency must match that of the transducer.

Here, we propose a robust, accurate and automatic algorithm for enhancement of needle shaft and tip in 2D US. Our main novelty is incorporation of US signal modeling into an optimization problem to estimate an unknown signal transmission map which is used for enhancement of the needle shaft and tip while considering US-specific signal propagation constraints. We present qualitative and quantitative validation results on scans collected from bovine and porcine phantoms. A preliminary investigation of our approach was reported in [14]. This extension focuses on steep needle insertions typical of challenging US-guided lumbar spinal injections. We also address the challenge of high-intensity artifacts that may reduce needle localization accuracy. The proposed algorithm improves visibility of the needle shaft and tip and achieves superior tip localization to previous approaches reported in [12, 14].

Methods

The proposed method is based on our prior experience with in-plane US guidance of lumbar injections in vivo, in which (i) the needle is inserted in-plane and the insertion side (left or right) is known; (ii) the needle tip appears as a characteristic, but variable intensity; and (iii) a portion of the needle shaft near the transducer surface is visible. In our approach, we focus on enhancement of needles under curvilinear 2D US guidance at insertion angles of $40^{\circ }{-}80^{\circ }$ to the skin surface. Owing to the transducer’s convex shape, only part of the shaft is present in the US image. An overview of the proposed is shown in Fig. 1. In the next section, we describe how these image features are used in our proposed method.

Fig. 1

Overview of the proposed framework, consisting of three main processes: (1–4) tip enhancement using a linear interpolation model that utilizes US signal transmission maps, (5) local-phase-based tip localization, and (6) shaft enhancement. Inset is the pseudocode, whose parameters are described in detail hereafter

Needle restoration model

The incident US waves traveling through tissue are attenuated along their transmission paths and scattered in other directions. To recover needle information, we must perform image restoration. Considering a 2D B-mode US image I(x, y) in which the needle tip and shaft are inconspicuous, we propose a linear interpolation model to account for the effect of attenuation and scattering:

$$\begin{aligned} {{\varvec{I}}}\left( {x,y} \right) =t\left( {x,y} \right) {{\varvec{I}}}_{\varvec{e}} \left( {x,y} \right) +\left( {1-t\left( {x,y} \right) } \right) {\varvec{\nu }} , \end{aligned}$$

(1)

where t(x, y) is the depth-dependent signal transmission map function (representing response of a loss field in the transmission medium), ${\varvec{I}}_{\varvec{e}} ({x,y})$ is the US image intensity to be recovered and ${\varvec{\nu }}$ is a constant intensity equal to echogenicity of the tissue confining the needle. Throughout the text, we will use a bold notation to denote matrices, and otherwise for scalars. If $\rho$ is the arc length parametrization of the signal path, then, for each short distance $\mathrm{d}\rho$ along this path, the fraction of US signal lost is given by $\propto \times \mathrm{d}\rho$, where $\propto$ is the attenuation coefficient. A spatially varying $\propto$ yields $t({x,y} )=\exp ( {- \int \nolimits _0^{\mathrm{d}({x,y} )} \propto ( {\rho ( \tau )} )\mathrm{d}\tau } ).$ Therefore, $t({x,y} ):0\le t({x,y} )\le 1$ denotes the fraction of US signal that reaches a target point d(x, y ) without attenuation.

Referring to (1), the enhanced image is obtained using:

$$\begin{aligned} {\varvec{I}}_{\varvec{e}} \left( {x,y} \right) =\frac{{\varvec{I}}\left( {x,y} \right) -{\varvec{\nu }} }{\left[ {\max \left( {t\left( {x,y} \right) ,\kappa } \right) } \right] ^{\propto }}+{\varvec{\nu }} , \end{aligned}$$

(2)

where $\kappa =0.001$, a small constant prevents division by zero. The proposed solution is severely under constrained because the number of unknowns is greater than the number of equations. The approach we devise to achieve a solution is to first model a patch-wise transmission map, $\psi ({x,y}),$ that reflects the boundary constraints imposed on the needle by attenuation and the needle trajectory, for all $({x,y})\in \Omega$, the set of pixel locations in ${\varvec{I}}({x,y})$. $\psi ({x,y})$, is then optimized to generate t(x, y) through a regularization framework we will describe later. Next, we describe how we compute different $\psi ({x,y})$ and thereafter, t(x, y) for tip and shaft enhancement, and localization of the needle tip.

Patch-wise transmission maps for tip enhancement

Ideally, derivation of the patch-wise transmission map would involve partitioning the image into grids specified by a patch size and knowledge of image depth information. Karamalis et al. [15] previously proposed an approach to estimate uncertainty in US images caused by depth-dependent attenuation and shadowing, referred to as a confidence map. Leveraging this approach to estimate the patch-wise transmission map automatically includes depth information and ensures that the transmission in a local image patch, at the same depth, will always be almost constant. Therefore, we do not have to specify image patch sizes.

Fig. 2

Effect of $\gamma$ on the patch-wise transmission map, $\psi ({x,y} )$, while fixing $\propto =2$ and $\beta =90$. $\gamma =0.03$ achieves a distinct function with minimum horizontal discontinuities

A confidence map, ${\varvec{I}}_{\varvec{c}}({x,y})$, results from a probability density function that assigns to each pixel in ${\varvec{I}}({x,y})$ a probability that a random walk [16] emanating from that pixel would be able to reach virtual transducer elements at the top of the image, given US-specific constraints. To calculate the confidence map, ${\varvec{I}}({x,y})$ is represented as a weighted graph in an 8-connected lattice. Random walks originating from virtual transducers at the top of the image are used to calculate expected signal strengths throughout the image [15]. The weighting function for the random walks is given by:

$$\begin{aligned} x_ij= \left\{ \begin{array}{ll} x_{ij}^\mathrm{H}=exp\left( -\beta \big |c_{i}-c_{j}\big |+y\right) , &{} \quad e_{ij} \in E_\mathrm{H}\\ x_{ij}^\mathrm{V}=exp\left( -\beta \big |c_i-c_j\big |\right) , &{} \quad e_{ij} \in E_\mathrm{V}\\ x_{ij}^\mathrm{D}=exp\left( -\beta \big |c_{i}-c_{j}\big |+\sqrt{2}y\right) , &{} \quad e_{ij} \in E_\mathrm{D}\\ 0, &{} \quad \mathrm{otherwise}. \end{array} \right. \nonumber \\ \end{aligned}$$

(3)

Here, $e_{ij}$ represents the edge between nodes i and j, $E_\mathrm{H}, E_\mathrm{V}$ and $E_\mathrm{D}$ are the horizontal, vertical and diagonal edges on the graph, $c_i ={\varvec{I}}({x,y})_i \exp \, ( {-\propto l_i } ),$ where ${\varvec{I}}({x,y})_i$ is the image intensity at node i and $l_i$ is the normalized closest distance from the node to the virtual transducer elements. The behavior of the confidence map is controlled by three free parameters: $\propto ,\beta$ and $\gamma$. $\propto$ is the attenuation coefficient which controls the depth-dependent attenuation rate. It is expected that pixels close to the top of the image will exhibit high confidence and vice versa. $\beta$ is an algorithmic constant which affects the robustness and accuracy of segmentation. $\gamma$ models the beam width, imposing a penalty on random walks crossing a horizontal/diagonal edge in the graph with increasing corresponding distance from the starting scanline. Recalling that ${\varvec{I}}_{\varvec{c}}({x,y} )$ expresses probability that a random walk emanating from a pixel in the image reaches the top of the image, yet we are interested in the likelihood that a signal sent from the transducer will reach a point of interest in the image (needle shaft and tip); we accordingly model the patch-wise transmission map as $\psi ({x,y})={\varvec{I}}_{\varvec{c}}({x,y} )^{*}$, where * denotes the complement. The parameters of $\psi ({x,y} )$ are determined as follows: $\propto$ and $\beta$ were previously optimized for US data in [15, 16]. Therefore, we use similar values: $\propto =2$ and $\beta =90$ for all the experiments. $\gamma$ should achieve a balance between a distinct map and minimal horizontal discontinuities. In Fig. 2, we show the effect of $\gamma \in [ {0\,1} ]$ on the derived $\psi ({x,y} )$. Note that $\gamma =0.03$ yields the best balance; this value was determined from several test images and kept constant throughout the validation experiments.

Fig. 3

Effect of the regularization parameter $\lambda$ on derivation of t(x, y) from $\psi ({x,y}). \psi ({x,y} )$ is derived with $\propto =2,\beta =90$ and $\gamma =0.03. t({x,y})$ displays lower intensities near the top of the image, where attenuation and scattering are minimal, and higher intensities with increasing depth. Note that as $\lambda$ becomes larger, t(x, y) tends to $\psi ({x,y})$. $\lambda =1$ ensures t(x, y) with the smoothest attenuation density estimate and the best mutual enhancement of the shaft and tip

Signal transmission maps for tip enhancement

The solution for (1) should be less sensitive to noise and abrupt changes in image intensity. To achieve this, we formulate a regularization problem for deriving the signal transmission map, t(x, y). Our approach is like that proposed by Tikhonov [17], but instead of using only the $L_2$ norm, we include the $L_1$ norm, which has in some cases been reported to give more robust solutions in image restoration [18]. Specific to our problem, since we expect features that have edges in our optimization data and hence possible outliers, the use of $L_1$ is prudent. Therefore, we obtain t(x, y) by minimizing the following objective function:

$$\begin{aligned} \frac{\lambda }{2}\big \Vert t\left( {x,y} \right) -\psi \left( {x,y} \right) \big \Vert _2^2 +\mathop \sum \nolimits _{i\in \Omega }\big \Vert W_i \circ \left( {G_i *t\left( {x,y} \right) } \right) \big \Vert _1 . \end{aligned}$$

(4)

This equation has two components: The first is the data which measure the closeness of t(x, y) to $\psi ({x,y})$, while the second introduces additional contextual constraints on t(x, y). The regularization parameter, $\lambda$, is used to balance the two parts. $G_i$ is a bank of high-order differential operators consisting of 8 Kirsch filters [19] and a Laplacian operator. The 8 Kirch filters consist of the same kernel mask rotated in 45-degree increments through all 8 compass directions. Combining the first-order derivative Kirch filters with a second-order derivative Laplacian mask preserves edge features associated with the needle. In (4), we introduce a weighting function W to further constrain t(x, y) in a local image patch. Considering two neighboring pixels, the weighting function is such that $W( {t_2 ({x,y})-t_1 ({x,y})})\approx 0$. If the two pixels are far apart, then W should be small, and vice versa. Consequently, we compute the weighting function from $W_i(\varrho )=\exp (-\big |(G_i*{\varvec{I}}(x, y))_\varrho \big |^{2}$, where $\varrho$ is a given location in the image. When $t_2(x,y)=t_1(x,y), W=0$, Conversely, when $W=0$ the constraint on t(x, y) between neighboring pixels is eliminated.

Like in [20], the optimization of (4) is achieved using variable splitting where several auxiliary variables are introduced to construct a sequence of simple subproblems, the solutions of which finally converge to the optimal solution of the original problem. In Fig. 3, we show the result of deriving the signal transmission map t(x, y) from $\psi ({x,y})$ using various $\lambda$ . In all cases, t(x, y) exhibits low intensities at the top, with depth dependent increase. However, we desire t(x, y) with the smoothest attenuation density estimate for use in the needle restoration model. Setting $\lambda =1$ ensured that the needle shaft and tip were enhanced mutually on a number of test images, and this value was kept constant throughout the validation experiments. Knowing t(x, y), the tip-enhanced image is determined using (2). Notice that the mean intensity of the local region in t(x, y) is always less than the echogenicity of the tissue confining the needle. From (2), the tip will be represented by a local average of the surrounding points, yielding a high-intensity feature corresponding to the tip in the enhanced image.

In (2), the choice of ${\varvec{\nu }}$ affects the accuracy of the enhancement result. It is imperative that ${\varvec{\nu }}$ results in partial enhancement of the shaft alongside the tip. As we will see later, shaft information is essential for the tip localization process. If ${\varvec{\nu }}$ is brighter than most pixels in ${\varvec{I}}({x,y})$, including the needle, the output of (2) will produce no needle information. Alternatively, a low ${\varvec{\nu }}$ creates more high-intensity artifacts. To maximize contrast of the needle, a value of ${\varvec{\nu }} =0.3\times \max ({\varvec{I}}({x,y})$ where $\max ({\varvec{I}}({x,y})$ denotes maximum intensity in ${\varvec{I}}({x,y})$ is chosen.

In [14], the tip enhancement process also enhanced hyperechoic artifacts present along the needle trajectory. With such artifacts in the enhanced image, it is difficult to distinguish needle features. Therefore, we need a mechanism to remove them. When these artifacts appear linearly, it is easy to eliminate them using morphological filters. Specifically, we use a top-hat filter (TF), which computes the morphological opening of ${\varvec{I}}_{\varvec{e}} ({x,y} )$ and subtracts it from ${\varvec{I}}_{\varvec{e}} ({x,y} ):TF ( {{\varvec{I}}_{\varvec{e}} ({x,y} )} )={\varvec{I}}_{\varvec{e}} ({x,y} )-D_L [ {E_L ( {{\varvec{I}}_{\varvec{e}} ({x,y} )} )} ]$. Here, L is a linear structuring element, while $D_L$ and $E_L$ denote dilation and erosion operations, respectively. The output of the top-hat filter yields the final tip-enhanced image, ${\varvec{I}}_{\varvec{e}}({x,y} )_\mathrm{tip}.$ In Fig. 4, we show the output of tip enhancement process for various ${\varvec{\nu }}$. It is observed that in the optimum case, the high-intensity artifact at the bottom of the image is suppressed.

Fig. 4

Example of needle tip enhancement at various ${\varvec{\nu }}$. Note that ${\varvec{\nu }}$ influences whether sufficient tip and shaft information is present in the tip-enhanced image, ${\varvec{I}}_{\varvec{e}}({x,y} )_\mathrm{tip}$, to facilitate the tip localization process. In the desired case, the tip (enclosed by red circle) is enhanced together with part of the shaft (enclosed by red rectangle)

Tip localization

The first step in the tip localization process is estimation of the needle trajectory. To achieve this, we need to extract shaft information from the enhanced image, ${\varvec{I}}_{\varvec{e}} ({x,y} )_\mathrm{tip} .$ For a $500\times 500\,\, I_e ({x,y} )_\mathrm{tip}$ image, a fixed $100\times 100$ region of interest (ROI), ${\varvec{I}}_{\varvec{e}} {\varvec{ROI}}({x,y}),$ is defined on the insertion side of the needle as shown in Fig. 5a. ${\varvec{I}}_{\varvec{e}} {\varvec{ROI}}({x,y})$ must contain part of the shaft, although it need not contain the tip. To extract shaft information from ${\varvec{I}}_{\varvec{e}} {\varvec{ROI}}({x,y})$, we use a bank of orientation-tuned band-pass 2D Log-Gabor filters. We choose these filters because they facilitate local-phase-based processing, from which the resulting image descriptor is intensity invariant and therefore insensitive to US imaging variables such as tissue type [16]. The resulting phase symmetry image, ${\varvec{PS}}({x,y})$, contains distinct local phase features for the needle shaft. The filter parameters were tuned to the optimized parameters in [12]. Limiting the calculation to ${\varvec{I}}_{\varvec{e}} {\varvec{ROI}}({x,y})$ minimizes the effect of any residual high-intensity artifacts that may be present elsewhere in ${\varvec{I}}_{\varvec{e}}({x,y})_\mathrm{tip}$. In Fig. 6, we show the result of extracting ${\varvec{PS}}({x,y})$ for scans obtained from different tissue types. Independence of our method from tissue type is augmented by calculating the image from a binary tip-enhanced image.

From ${\varvec{PS}}({x,y})$, the needle trajectory is estimated using the Radon transform (RT) with an angular range of $0^{\circ }-179^{\circ }$. To obtain a region where the needle shaft and tip certainly lie, the estimated trajectory is expanded over the whole ${\varvec{I}}_{\varvec{e}} ({x,y})_\mathrm{tip}$ image as shown in Fig. 5(b). Knowledge of the trajectory region helps us to extract only data lying along the trajectory in ${\varvec{I}}_{\varvec{e}} ({x,y})_\mathrm{tip}$ by convolution. These data are trimmed using the maximum likelihood estimation sample consensus (MLESAC) algorithm [21], which performs inlier detection and geometrical optimization. The resulting image, ${\varvec{I}}_{{\varvec{MLESAC}}} ({x,y})$, is shown in Fig. 5(c). Following the approach described in [12], the resulting colinear candidate intensities lying along a line L are distributed among a set of line segments, each defined by a set of points or knots denoted as $\mu _1 \ldots ..\mu _n$ . The needle tip is extracted using:

$$\begin{aligned} {\varvec{I}}_{{\varvec{needle}}} \left( {{\varvec{I}}_{{\varvec{BP}}} \left( {x,y} \right) } \right) =\frac{ \int \nolimits _{\mu _i}^{\mu _i +1} {\varvec{I}}_{{\varvec{BP}}} \left( {x,y} \right) \mathrm{d}\mu }{\big \Vert L_{\left( {\mu _i +1} \right) } -L_\mu \big \Vert _{2} };\mu \in \left[ {\mu _i ,\mu _{i+1} } \right] .\nonumber \\ \end{aligned}$$

(5)

Here, ${\varvec{I}}_{{\varvec{BP}}} ({x,y})$ is obtained by applying a Log-Gabor filter without orientation selectivity to ${\varvec{I}}_{\varvec{e}}({x,y} )_\mathrm{tip}$, whereas $\mu _i$ and $\mu _{i+1}$ are successive knots. The function in (5) assigns to pixels between knots $\mu _i$ and $\mu _{i+1}$, a mean intensity value along L. The result of this operation, ${\varvec{I}} _{{\varvec{needle}}} ({x,y})$, is shown in Fig. 5(d). From ${\varvec{I}}_{{\varvec{needle}}} ({x,y})$ , the needle tip is localized as the farthest maximum intensity pixel at the distal end of the needle trajectory (Fig. 5(e)).

Fig. 5

Framework for automatic tip localization. a Region of interest ${\varvec{I}}_{\varvec{e}} {\varvec{ROI}}( {x,y} )$ on enhanced tip image ${\varvec{I}}_{\varvec{e}} ({x,y})_\mathrm{tip}$, indicated by white rectangle. b Estimated trajectory (green) overlayed on ${\varvec{I}}_{\varvec{e}} ({x,y})_\mathrm{tip}$. c ${\varvec{I}}_{{\varvec{MLESAC}}} ({x,y})$, the output of the MLESAC algorithm. d ${\varvec{I}}_{{\varvec{needle}}} ({x,y})$ image obtained from (5). From ${\varvec{I}}_{{\varvec{needle}}} ( {x,y} )$, needle tip localization is achieved by selecting the first maximum intensity pixel lying along the calculated needle trajectory. e Enhanced needle tip (red dot) marked on the US image ${\varvec{I}}({x,y})$

Fig. 6

Illustrating tissue independence of PS(x, y) image for four different tissue types: bovine, porcine, kidney and chicken. Row 1: US image ${\varvec{I}}({x,y})$. Row 2: Tip-enhanced image ${\varvec{I}}_{\varvec{e}} ({x,y})_\mathrm{tip}$. The enhanced tip is surrounded by a red circle. Row 3: PS(x, y) image. The use of PS(x, y) in calculating needle trajectory ensures accuracy through orientation tuning and independence from tissue type

Shaft enhancement

The final step in our proposed framework is needle shaft enhancement using (2) and (4). However, we desire different patch-wise and signal transmission maps pertinent to the shaft restoration problem. Suppose along the needle trajectory, N represents a set of pixels belonging to the needle and B represents a set of pixels belonging to the background in ${\varvec{I}}({x,y})$. The patch-wise transmission map $\psi ({x,y})_\mathrm{shaft}$ is derived using the Euclidean distance transform of all N, i.e., a measure of the minimal distance between N and B. If we denote the Euclidean distance by $\sigma ({x,y})$, then:

$$\begin{aligned} \psi \left( {x,y} \right) _\mathrm{shaft} =\frac{\left| {\sigma \left( {x,y} \right) -max\left( {\sigma \left( {x,y} \right) } \right) } \right| }{max\left( {\sigma \left( {x,y} \right) } \right) }. \end{aligned}$$

(6)

Since we know the tip location from the previous step, we constrain $\psi ({x,y})_\mathrm{shaft}$ not to exceed the tip position. The signal transmission map for the shaft, $t({x,y})_\mathrm{shaft}$, is obtained from $\psi ({x,y})_\mathrm{shaft}$ using (4), and shaft restoration is performed using (2), with ${\varvec{\nu }} =max({\varvec{I}}({x,y}).$ The result of shaft enhancement is illustrated in Fig. 7. Notice that $t({x,y})_\mathrm{shaft}$ has low intensities along the needle axis and higher intensities for image regions away from the axis. The enhanced shaft arises from a local average of pixels belonging to the shaft along the trajectory.

Fig. 7

Needle shaft enhancement process. The signal transmission map $t({x,y})_\mathrm{shaft}$ derived from the patch-wise transmission map $\psi ({x,y})_\mathrm{shaft}$ shows higher intensities with increasing distance from the central axis of the estimated trajectory. In the shaft-enhanced image, the automatically localized tip is indicated by the red dot

Fig. 8

Qualitative results showing shaft enhancement and tip localization at steep insertion angles. Column A: US image ${\varvec{I}}({x,y})$. B: Tip-enhanced image ${\varvec{I}}_{\varvec{e}} ( {x,y} )_\mathrm{tip} .\hbox { C}$: ${\varvec{I}}_{{\varvec{needle}}} ( {x,y} )$ image, from which the needle tip is determined as the first bright intensity pixel at the distal end of the needle. D: Shaft-enhanced image. The automatically localized tip is indicated by a red dot. The proposed method restores needle information when the shaft is discontinuous, the tip is inconspicuous or high-intensity artifacts are present along the trajectory

Experimental validation

US images for this study were obtained using a SonixGPS system (Analogic Corporation, Peabody, MA, USA) equipped with a C5-2/60 curvilinear probe. A 17-gauge (1.5 mm diameter, 90 mm length) Tuohy epidural needle (Arrow International, Reading, PA, USA) was inserted by hand into freshly excised bovine and porcine tissue at varying angles ($40^{\circ }{-}80^{\circ }$) and depths (up to 9 cm). A total of 300 scans (150 from each tissue type) were collected with the needle in-plane.

The proposed method was implemented in MATLAB 2017a (Mathworks, Natick, Massachusetts, USA) on a 4.2 GHz Intel(R) $\hbox {Core}^{\mathrm{TM}}$ i7 CPU, 16GB RAM Windows PC (DELL, Round Rock, Texas, USA). After automatically determining the tip localization accuracy, an expert manually localized the tip on corresponding images. We then determined localization error as the Euclidean distance (ED) between the automatically localized tip and the manually localized position. The root-mean-square (RMS) error and 95% confidence interval (CI) for localization error were also calculated. The same dataset used in evaluating the proposed method was used to assess the performance of the methods proposed in [12, 14]. Similarly, the RMS error and the associated 95% CI were calculated. Further, we performed a one-tailed paired t-test to determine whether the differences between localization errors from the various methods were statistically significant.

Results

Qualitative results for tip and shaft enhancement (Fig. 8) show that the proposed method efficiently enhances the tip and shaft at steep insertion angles when the shaft is broken and the tip is inconspicuous. Our method works for different visibility profiles of the shaft and tip in the US image. Figure 9 shows a qualitative comparison of the proposed method and the methods in [12, 14]. In the top row, we notice that when the tip intensity is high compared to surrounding tissue and there are no high-intensity artifacts along the trajectory, all 3 methods give accurate tip localization. However, investigating the middle row shows that low tip intensity affects tip localization by the method in [12]. The last row illustrates some case where high-intensity artifacts lead to inaccurate localization by the methods in [12, 14]. These artifacts do not affect the proposed method.

Quantitative results from the proposed method are shown in Table 1. Since the localization error is invariant to tissue type, we report aggregate errors for bovine and porcine tissue. The overall localization error from the proposed method was $0.55\pm 0.06\hbox { mm}$. In comparison, the localization error from the method in [14] was $0.74\pm 0.07\hbox { mm}$, while [12] yielded $0.88\pm 0.08\hbox { mm}$. For the methods in [12, 14], we excluded outliers (localization error $>2\hbox { mm}$). For the proposed method, all localization errors were less than 2 mm. With [14], only 68% of the dataset was retained, and this dropped to 56% with [12]. A one-tailed paired $t-$test shows that the differences between the localization errors reported in [12, 14] and the proposed method are statistically significant ($p<0.005$).

Further, recall that ${\varvec{\nu }}$ is optimally fixed at $0.3\times \max ({\varvec{I}}( {x,y} )$. We show the effect of ${\varvec{\nu }}$ on tip localization accuracy in Table 2. Values of ${\varvec{\nu }}\ge 0.5\times \max ({\varvec{I}}({x,y})$ are excluded since they result in complete loss of shaft information. The results are in consonance with qualitative results (Fig. 4) regarding optimal choice for ${\varvec{\nu }}$ . In Table 3, we show effect of the regularization parameter, $\lambda$ , on tip localization accuracy. It is observed that minimum error occurs at $\lambda =1,$ in agreement with qualitative results in Fig. 3, where $\lambda =1$ gave the smoothest estimate of the signal transmission map. Also, recall that we chose a $100\times 100$ ROI during the tip localization process. In Table 4, we show the effect of ROI size on tip localization accuracy. Note that small and larger ROI sizes result in inaccurate results because they lead to inclusion of inadequate shaft information or interfering artifacts, respectively, thus reducing clarity of trajectory estimation. For a $500\times 500$ 2D image, the tip and shaft enhancement processes executed for 0.4 and 0.27 seconds, respectively, while tip localization took 1.1 seconds.

Fig. 9

Comparing the proposed method with methods in [12, 14]. Column a US image ${\varvec{I}}({x,y})$. The arrow points to the expert localized tip. b Tip localization from the proposed method. c Tip localization by method in [14]. d Tip localization by method in [12]. A green dot shows correct tip localization, while a red dot shows inaccurate localization

Fig. 10

Examples of cases where the proposed method fails. A) US image ${\varvec{I}}( {x,y} )$ . B) tip-enhanced image ${\varvec{I}}_{\varvec{e}} ( {x,y} )_\mathrm{tip}$. C) ${\varvec{I}}_{{\varvec{needle}}} ( {x,y} )$ image. D) Localized tip (red) marked on original image. In the top row, ${\varvec{I}}( {x,y} )$ contains insufficient shaft information, and thus ${\varvec{I}}_{\varvec{e}} ( {x,y} )_\mathrm{tip}$ contains no conspicuous shaft information. Therefore, we can’t proceed with the tip localization process. In the middle and bottom rows, ${\varvec{I}}( {x,y} )$ contains sufficient shaft information, but tip information is unclear. Presence of an enhanced shaft means we can attempt tip localization, but absence of an enhanced tip leads to incorrect tip localization

Table 1 Tip localization error (mm) using the proposed method and localization approaches in [12, 14]

Table 2 Effect of ${\varvec{\nu }}$ on tip localization error. % values are of $max({\varvec{I}}( {x,y} ))$, the maximum intensity in ${\varvec{I}}( {x,y} )$. Minimum error is obtained at 30%

Table 3 Effect of $\lambda$ on localization error. Minimum error is obtained at $\lambda =1$. For $\lambda \ge 10$, the increase in localization error with increase in $\lambda$ is not statistically significant, because $t({x,y})\rightarrow \psi ({x,y})$.

Discussion and conclusions

We have presented a solution to the challenge of poor needle tip and shaft visibility in 2D curvilinear US at steep insertion angles. The proposed method utilizes US signal transmission maps derived from a contextual regularization framework while integrating US-specific signal propagation constraints. The needle tip localization accuracy achieved was 25% and 37% better than that reported in [12, 14], respectively. Further, considering the diameter of the needle used in this study (1.5 mm) and the resolution of state-of-the-art US machines, a tip localization error of $< 1\hbox { mm}$ is clinically acceptable. Different than [12, 14], we demonstrate needle enhancement at steeper insertion angles ($40^{\circ }-80^{\circ }$) and minimize the effect of high-intensity artifacts along the needle trajectory.

Table 4 Effect of ROI size on tip localization error. The optimum ROI size was 100x100 pixels

Suffice to note, lumbar spinal injections sometimes involve the use of bending needles, yet the epidural needle used in this study exhibits negligible bending. For bending needles, bending information can be integrated in the proposed method. Further, our method requires shaft and tip information to be partially available. In Fig. 10, we illustrate cases where these assumptions are violated, leading to failure of the method. In Fig. 10 (top row), there is hardly any shaft information in the original US image; the enhanced image also contains insufficient shaft information. As such, we can’t proceed with the tip localization process. In Fig. 10 (middle and bottom rows), shaft information is available, but tip information is absent. Therefore, although the enhanced image facilitates the tip localization process, the tip location obtained is inaccurate.

Nevertheless, for instances where the shaft and tip are somehow visible, the achieved tip localization accuracy and shaft enhancement make our method appropriate for further investigation in vivo and is valuable to all previously proposed state-of-the-art needle localization methods. Future work will focus on validation of the proposed method on clinical data.