Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

1.1 Purpose

Magnetic particles which can be controlled by an external magnetic field have been explored as a method for precise and efficient drug delivery. With the increased usage of microbubbles as a system for drug delivery using ultrasound imaging, there is an interest to develop a tracking algorithm to locate the in vivo position of the microbubbles once delivered. In this chapter, ultrasound imaging of magnetic particles with encapsulated microbubbles is further studied. The objective of this chapter is to optimize the tracking of microbubbles from the ultrasound images and to perform image analysis to identify the location of the microbubbles.

1.2 Problem

Currently, clinical use of microbubbles are being visualized by methods such as ultrasound Particle Image Velocity or Echo particle image velocimetry, which return velocity vectors of the particles under flow conditions. The literature review shows a limited number of studies on microbubble localization under ultrasound imaging when using external controlled manipulation of the magnetic particles. As such, this chapter aims to create a tracking algorithm for tracking of magnetic particles with encapsulated microbubbles under controlled conditions.

1.3 Scope

Figure 1 shows the overview of this chapter. First, a suitable technique to fabricate the required microbubble encapsulated magnetic particles for imaging under ultrasound was investigated. Second, the most suitable experimental setup for obtaining the required ultrasound images was used. Third, the tracking algorithm to locate the microbubbles was applied. After which, the results from the experiments and future works for the chapter are discussed.

Fig. 1
figure 1

Overview of chapter.

Full size image

2 Literature Survey

2.1 Microbubbles as Ultrasound Contrast Agents

Microbubbles as contrast agents for ultrasound imaging [1] typically consist of a gas core (filling gas) encapsulated by a coating (shell) normally made from a protein, lipid or polymer [2]. When the compressible gas bubbles are smaller than the wavelength of incidental ultrasound signals, the microbubbles oscillate in the ultrasound acoustic field, causing the resulting backscattered ultrasound signal to be detected and differentiated from tissue [3]. In addition, ultrasound imaging is advantageous as ultrasound imaging allows capturing of dynamic moving targets inside human body noninvasively [4].

Recent growing researches shed lights on the fabrication of magnetic microbubbles by infusing magnetic nanoparticles into the microbubble shell. Research done by Yang et al. (2009) on superparamagnetic iron oxide (SPIO) particles incorporated into the microbubble polymer shell demonstrated that embedding magnetic nanoparticles into bubble shells enhanced the stability of microbubbles, preventing premature destruction of microbubbles during ultrasound imaging [5]. Magnetic particles may be further developed for diagnostic and therapeutic drug delivery.

2.2 Clinical Applications of Magnetic Microbubbles

2.2.1 Diagnostic Applications: Magnetic Microbubbles as Dual Contrast Agents

Research done by Stride et al. (2008) showed that addition of magnetic nanoparticles to the microbubble encapsulating layer enhanced ultrasound imaging contrast due to increased asymmetric bubble oscillations [6]. Furthermore, the close way of packing the magnetic particles limits bubble compression, causing increased harmonic components of the scattered ultrasound signal. Consequently, the microbubbles can be imaged at lower ultrasound power levels, minimizing the risk of high-power complications. In addition, lower ultrasound intensity used for imaging reduces unwanted premature bubble break, which is advantageous for drug or gene delivery [7]. Specific applications of microbubbles for ultrasound diagnostic include cardiac ultrasonography. Microbubbles allow for improved visualization of beating heat chambers [8] and also for the evaluation of microvascular perfusion as its similar rheology to red blood cells [9].

2.2.2 Therapeutic Applications: Targeted Drug Delivery

Magnetic robotic microbubbles can be designed as a drug delivery vehicle and delivered to the target site driven by external magnetic fields. Upon reaching the target site, robotic microbubbles can be destroyed by local application of high transmitted ultrasound acoustic powers [10], releasing the loaded drug. Targeted drug delivery by magnetic microbubbles prevents exposure of surrounding tissue to harmful side effects, and may be especially useful for cancer treatment [11]. Furthermore, microbubbles may improve effectiveness of drug delivery due to the phenomenon of sonoporation [12]. The transient cell permeability during low-intensity ultrasound was found enhanced with microbubbles [13] and thus regulating the drug release rate.

2.2.3 Therapeutic Applications: Gene Delivery

Both ultrasonically and magnetically-mediated transfection improves uptake of DNA by target cells interacting with phospholipid coated microbubbles with magnetic nanoparticles, which enhanced the transfection of the target cells [14].

2.3 Ultrasound Microbubble Tracking

Microbubbles currently used in clinical applications such as echocardiography or imaging of angiogenesis are visualized by contrast pulse sequencing [15]. The velocity of the microbubble under ultrasound imaging can be estimated by Echo Particle Image Velocimetry (EPIV), which has been utilized for measurement of local hemodynamics and wall shear rate [16] and intracavitary blood flow 2-D velocity vectors [17].

2.4 Optical Tracking of Microbubbles

The literature review here compares three methods for moving object detection: background subtraction, optical flow and frame difference [18], as indicated in Table 1. Furthermore, studies on moving object detection demonstrate that there is no fixed method in developing a tracking algorithm, as the approach taken depends largely on the application, imaging setup and contrast agents being used [19].

Table 1 Comparison of moving object detection methods.
Full size table

3 Methodology

3.1 Fabrication of Encapsulated Microbubbles

The methods of producing gas microbubbles include sonication, high shear emulsification, inkjet printing, and microfluidic processing and the later two can better control over stability, uniformity, size and composition of microbubbles [23]. Dharmakumar et al. (2005) characterized three different constructs of magnetic microbubbles [24]. Fabrication techniques which can be used to achieve the three different magnetic microbubble constructs have been classified accordingly as shown in Table 2.

Table 2 Schematic diagram of three magnetic microbubble constructs.
Full size table

Of the three magnetic microbubble structures, the second structure (B) was chosen for fabrication due to ease of reproducibility as additional gas source and equipment were not required. The magnetic microbubbles were fabricated using gas foaming; a simple, inexpensive and easy to operate method to generate a porous structure [27].

3.1.1 Materials

Polymer was chosen as the encapsulating material due to its advantage of mechanical stability. Alginate was the chosen polymer as alginate is biocompatible, biodegradable and is easily fabricated to gelated capsules by crosslinking [37]. Other polymers which have been used to fabricate microbubbles, such as poly (lactic-glycolic) acid (PLGA) [38], polyvinyl alcohol (PVA) and poly (DL-lactide) (PLA) [2] require a more complex multiple emulsion method. Furthermore, alginate allows for the fabrication of more ‘air-tight’ biopolymer shells, reducing the outward diffusion of air-filled microbubbles.

3.1.2 Methods

The encapsulated microbubbles were prepared according to a modified protocol by Huang et al. (2013) [39]. The solution was first prepared by mixing sodium bicarbonate (4%) homogeneously in Na-alginate (low viscosity, 2% (w/v)) solution. 4% sodium bicarbonate concentration was chosen as it gave the most desired structure of porous spheres. 1 ml of the homogeneous Na-Alginate mixture was mixed with 0.05 g of carbonyl iron powder and loaded in a syringe (TERUMOR® Syringe, 1 cc/mL). The mixture was manually extruded from the needle tip (25G, 0.50 × 25 mm). The drops of Na-alginate solution were collected in a solution of 25% w/v calcium chloride solution mixed with H2O2 (32 wt %) for gelation of the alginate.

The collected Na-alginate-iron spheres measured in the range of about 2–3 mm in diameter. The Na-alginate-iron spheres were left in the collecting solution for 10 min, undergoing the chemical reactions. Figure 2 illustrates the diffusion of hydrogen peroxide into the Na-alginate spheres to react with sodium bicarbonate, creating the resultant porous structure.

Fig. 2
figure 2

Porous structure formation by inward diffusion of H2O2 to react with NaHCO3.

Full size image

After the reaction went on for 10 min, the desired porous Na-alginate-iron spheres (group B) were removed and rinsed with distilled water. The procedure was repeated to prepare two more sets of spheres: (group A) Na-alginate porous spheres with no iron particles and (group C) control solid magnetic spheres with no encapsulated microbubbles as distilled water was used in place of hydrogen peroxide.

3.2 Imaging Setup

After the encapsulated microbubbles were fabricated, an imaging setup as shown in Fig. 3 was used to capture the preliminary videos for image processing. Imaging was done using an Ultrasonix machine, with a 4DL14-5/38 Linear 4D probe. The probe allows for applications such as abdominal, MSK, nerve block, small parts and vascular imaging. The bandwidth for the probe is 14–5 MHz with a depth range of 2–9 cm and a geometric focus of 28 mm [40]. The ultrasound probe was immersed in a water tank, to minimize attenuation of the ultrasound waves in air.

Fig. 3
figure 3

Stabilized ultrasound probe immersed in water tank setup.

Full size image

The magnetic spheres were aligned on a MISSION® ExpressMag® Super Magnetic Plate measuring 127.8 by 85.5 cm. The ultrasound probe was placed directly above the tube to be imaged and was kept in a fixed position by exerting force via an external structure. Stabilizing the ultrasound probe was critical in obtaining consistent images and prevent swaying of the probe as the magnetic field from the moving magnetic plate interfered with the magnetic field of the probe. Stabilization also prevented false detections caused by disturbances in the tracking algorithm [41]. The ultrasound videos were captured by moving the Super Magnetic Plate in the x direction, as shown in Fig. 4.

Fig. 4
figure 4

Experimental setup for capturing of ultrasound videos.

Full size image

3.3 Image Analysis by Frame Difference Algorithm

After the ultrasound videos of the moving microbubbles were captured, the tracking algorithm used to analyze the videos was created in C++, using libraries from Open Source Computer Vision (OpenCV). OpenCV contains a library of programming functions specifically for the use for real-time computer vision.

From Table 1 in Sect. 2.4, the frame difference algorithm was found to be the most suitable in detecting the moving microbubbles. Figure 5 shows an overview of the frame difference algorithm. First, the tracking algorithm reads in the input video, which can also be programmed to allow for live input from the ultrasound probe. Second, the ultrasound video is converted to grayscale. Third, a difference image is computed by using two consecutive frames from the video. Fourth, the frame difference images are thresholded by binary threshold, where the resultant image is converted to either black or white. Fifth, the thresholded image is smoothened to increase whitespace between the two consecutive frames. Last, the function to detect the centroids of the microbubbles is called, finding the contours in the image. The output of the total number of microbubbles detected in each frame is given, with the respective (x, y) coordinates of each microbubble and the centre of the microbubbles highlighted with a crosshair and circle.

Fig. 5
figure 5

Outline of tracking analysis of ultrasound videos.

Full size image

Table 3 details the OpenCV functions that were used in the tracking algorithm.

Table 3 Overview of OpenCV functions used in tracking algorithm.
Full size table

3.4 Ground Truth Validation

For the purposes of this chapter, the ground truth for the (x, y) coordinates of the centroids of the moving spheres were manually selected. The following adjustments were made to the tracking algorithm: First, the user mouse was adjusted to allow the (x, y) coordinates to be the output upon clicking (Fig. 6). Second, the crosshair and circle labelling were removed to prevent prejudice during the manual selection of the centre of the spheres (Fig. 7). Third, the video playback speed was reduced by increasing the waitKey switch to 400 (Fig. 8), to allow for a sufficient pause duration between each frame. Lastly, the video was enlarged for clearer selection of the centroid (Fig. 9). The ground truth readings were taken when all the moving spheres were in the frame.

Fig. 6
figure 6

Using CallBackFunction for manual selection of microbubble centroids.

Full size image
Fig. 7
figure 7

Crosshairs and circle labelling of code commented away.

Full size image
Fig. 8
figure 8

Playback speed of ultrasound videos slowed down via waitKey value.

Full size image
Fig. 9
figure 9

Code to expand window size of video.

Full size image

The Ground Truths were calculated using Pythagoras’ Theorem. By calculating the pixel difference in both the x and y coordinates, the distance between the actual and detected centroid locations was calculated using \( \sqrt {A^{2} + B^{2} } \) (Fig. 10).

Fig. 10
figure 10

Calculation of distance between detected and manually selected centroids by Pythagoras’ Theorem.

Full size image

3.5 Pixel Resolution Calculation

The resolution of the ultrasound probe was calculated by taking physical measurements of the distance between the magnetic spheres and the corresponding pixel distance in the resultant ultrasound image obtained. Figure 11 shows the physical measurement of the distance between the edges of the spheres and the corresponding measurement of pixel distance through manual selection of the edges via the manual mouse selections.

Fig. 11
figure 11

Corresponding ultrasound image of 10 aligned magnetic spheres and measured physical distances.

Full size image

From the results in Table 4, the average calculated resolution was 0.0106 cm/pixel for the ultrasound probe. This value would be used in Sect. 4.4 to calculate the tracking accuracy of the algorithm.

Table 4 Resolution calculation based on pixel and corresponding physical distance.
Full size table

4 Results

4.1 Properties of Magnetic Microbubbles

Following the methodology in Sect. 3.1, the three sets of collected 2–3 mm spheres were observed under the microscope (x4 magnification). Figure 12 shows group A, the control Na-alginate only spheres with distinct microbubbles but without iron particles. Figure 13 shows group B, the desired Na-alginate-iron spheres with encapsulated microbubbles. Figure 14 shows group C, the control solid spheres, with no encapsulated microbubbles.

Fig. 12
figure 12

(Group A) Na-alginate spheres with encapsulated microbubbles, no carbonyl iron.

Full size image
Fig. 13
figure 13

(Group B) Na-alginate-iron spheres with encapsulated microbubbles.

Full size image
Fig. 14
figure 14

(Group C) Control solid iron spheres with no encapsulated microbubbles.

Full size image

Furthermore, to verify that the fabricated magnetic particles contained encapsulated microbubbles, the Na-alginate-iron spheres with encapsulated microbubbles (group B) and the solid iron spheres (group C) were placed inside eppendorf tubes filled with water. Figure 15 shows that the magnetic microbubbles (group B) floated to the surface, while the control solid spheres (group C) sunk to the bottom. In addition, Fig. 16 shows that the magnetic microbubbles were attracted to a strong magnet.

Fig. 15
figure 15

Magnetic Microbubbles floated to the surface while solid control spheres sunk.

Full size image
Fig. 16
figure 16

Fabricated magnetic microbubbles attracted to a magnet.

Full size image

The microscopic images and experimental tests clearly prove the magnetic properties and presence of encapsulated microbubbles in the desired group B spheres.

4.2 Ultrasound Contrast Results

Upon successful encapsulation of the microbubbles, the effects of the encapsulated microbubbles as ultrasound contrast agents was tested using the setup as shown in Fig. 17. Four of the microbubble encapsulated spheres (group B) and four of the solid magnetic spheres (group C) were lined on the MISSION® ExpressMag® Super Magnetic Plate. The magnet was placed at the bottom of the water tank with the ultrasound probe positioned vertically above the magnetic particles.

Fig. 17
figure 17

Experimental setup to compare the contrast enhancement properties of microbubbles.

Full size image

The obtained ultrasound images in Fig. 18 show that at equal distance from the ultrasound probe, the encapsulated microbubbles (group B) were distinctively clearer, appearing as bright white spots. However, the solid magnetic spheres (group C) appeared duller. Hence, the experimental results obtained verify the expected contrast enhancing properties of microbubbles as explained in Sect. 2.1.

Fig. 18
figure 18

Encapsulated microbubbles compared with solid spheres under ultrasound imaging.

Full size image

4.3 Tracking Results by Frame Difference Algorithm

From the experiments conducted, it was found that the tracking algorithm worked best for up to five magnetic spheres. For six magnetic spheres and onwards, the algorithm could not track each sphere as consistently. Figure 19 shows the results of the applied tracking algorithm for 1 sphere (a), 2 (b), 3 (c), 4 (d) and 5 (e) spheres, with complete windows shown in Appendix 1. The results show that the algorithm could successfully detect the movement of the microbubbles, and give the (x, y) coordinates of the centroids of the microbubbles with a and a crosshair and circle used to label the position of the microbubble.

Fig. 19
figure 19

Results of applied frame difference algorithm on captured ultrasound videos for 1 sphere (a), 2 (b), 3 (c), 4 (d) and 5 (e) spheres.

Full size image

Furthermore, the algorithm returns the total number of particles detected at each frame and the respective (x, y) coordinates of each particle, as shown in Fig. 20.

Fig. 20
figure 20

Tracking algorithm returns total number of particles detected and respective (x, y) coordinates.

Full size image

4.4 Tracking Accuracy

Table 5 shows the calculated difference in pixels and actual distance between the detected centroids from the algorithm and the manually selected centroids, based on the method described in Sect. 3.5.

Table 5 Calculated difference in pixels and actual distance between detected and manually selected centroids.
Full size table

From Table 4, the total average difference between the algorithm detected and manually selected centroids of the five tracking experiments was 3.33 pixels, or 0.0354 cm.

4.5 Tracking Results of Solid Spheres Compared to Microbubbles

Furthermore, when the tracking algorithm was applied to both the solid sphere and microbubble encapsulated sphere under the same conditions, only the microbubble encapsulated sphere was detected by the tracking algorithm. Figure 21 shows the crosshair labelled microbubble sphere whereas the solid sphere was barely visible and was left undetected.

Fig. 21
figure 21

Comparison of tracking results of moving solid and microbubble encapsulated spheres.

Full size image

5 Discussion

The goals of this study were to fabricate magnetic spheres with encapsulated contrast enhancing microbubbles and to create an experimental setup to capture ultrasound videos to analyze the position of the spheres using a tracking algorithm. The results obtained above demonstrate an alternative to the current state of the art of ultrasound microbubble imaging and the application of OpenCV functions for biomedical tracking purposes.

5.1 Ultrasound Imaging Setup

From the experiments conducted, it was found that the resultant OpenCV tracking approach used depended largely on the ultrasound imaging setup used to obtain the ultrasound videos. Appendix 2 shows a comparison of the various imaging setups that were tested to obtain the ultrasound videos.

After experimental trials with the different setups, it was found that setup 5, using a large planar magnet, was the most suitable setup for the purposes of this chapter. Furthermore, the purpose of this chapter was to track the particles in controlled motion for targeted drug delivery, and not in specific flow conditions such as that used for vascular applications. However, additional precautions have to be taken.

First, it was also observed that the large magnet created magnetic fields that would interfere with the ultrasound video, as shown in Fig. 22. As such, the tracking algorithm used had to be able to remove the interfering magnetic field in order to track the moving microbubbles.

Fig. 22
figure 22

Interference of magnetic field with ultrasound image.

Full size image

Second, the magnetic field created also interfered with the ultrasound probe, causing the ultrasound probe to swing with the change in magnetic field caused by movement of the magnet. As such an external force had to be exerted to stabilize and keep the ultrasound probe in a fixed position.

Last, it was found that the hollow magnetic spheres were fragile and had to be handled with care. When positioning on top of the magnet, the spheres were easily crushed and destroyed. A dropper was used to position the spheres on the magnet linearly.

5.2 Tracking Algorithm

As the purpose of the tracking moudule is to return the coordinates of the microbubble location, the EPIV or PIV methods discussed in Sect. 2.3, which incorporates velocity of the microbubbles, were not applied.

One of the challenges faced in developing the tracking algorithm was the irregular shape of the magnetic spheres. Due to its hollow structure, the magnetic spheres were easily deformed. Figure 23 shows a flattened magnetic particle on the planar magnet with the corresponding ultrasound image above. Therefore, under ultrasound imaging, the magnetic particles did not always appear as a spherical white spot and at times had irregular shapes. As such the frame difference tracking algorithm used allowed for more robust detection, as compared to methods such as Hough transform which detects more spherical shaped artefacts.

Fig. 23
figure 23

Irregular shape of magnetic particles affects resultant ultrasound image.

Full size image

5.3 Future Works

Towards integrating the tracking approach here for further drug delivery, future tests would have to be done using a magnetic system, such as an actuation system consisting of a pair of Helmholtz and Maxwell coils [42]. By using an external magnetic system, the capturing of the ultrasound videos could be more realistic and allow for tracking as well in the x, y and z directions. Furthermore, the experimental conditions can be adjusted to include more in vivo like conditions, such as the presence of red blood cells, white blood cells to test the versatility of the methods in more realistic conditions.

6 Conclusion

In this project, the use of ultrasound contrast enhancing microbubbles was tested in an experimental setup inducing the motion of microbubbles through moving a large planar magnet in the x direction. The tracking algorithm used, based on frame difference, proved capable of tracking and returning the (x, y) coordinates of the moving microbubbles. The optimal number of particles to be tracked was up to five particles with an accuracy of 3.33 pixels, or 0.0354 cm, between the algorithm detected and the manually selected (x, y) coordinates of the centroids. The limitations of the project include the lack of use of in vivo like conditions, such as the presence of other particles, for example, red blood cells. The fabricated magnetic microbubbles could be further used as test particles for external manipulation systems for drug delivery studies.