Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

12.1 Introduction

A sperm is a male reproductive cell consisting of an ellipsoidal or spherical head, a short midpiece, and a thin motile tail. The sperm head contains the nucleus with genetic materials, surrounded anteriorly by a cap-like acrosome that contains digestive enzymes. These enzymes can break down the outer membrane of ovum during the fertilization process, allowing the haploid nucleus in the sperm cell to join with the haploid nucleus in the ovum. The midpiece of the sperm cell has a central filamentous core with many mitochondria spiralled around it, generating energy for the sperm’s motion. The sperm tail executes the swing movement that propel the sperm to move. In humans, the average total length of sperm tail and the short midpiece is approximately 50 μm, and the sperm head dimensions are 4. 4 ×2. 8 μm [1]. Figure 12.1 is an image of sperms on a Petri dish taken under 20 ×magnification and a schematic diagram of a human sperm cell.

Fig. 12.1
figure 1

Human sperm cells: (a) Image taken under 20 × magnification. (b) A diagram of human sperm cell

Full size image

In natural conception, a healthy sperm overcomes the physiological and biological selection barriers, actively seeks out and fertilizes an egg. Sperm selection occurs naturally in this procedure. However, for couples having infertility issues, assisted reproduction technologies are required to address their reproductive needs. For example, in intracytoplasmic sperm injection (ICSI), an embryologist selects a single sperm cell and injects it directly into an oocyte (i.e., egg cell) to overcome issues such as male infertility [2]. These assisted reproduction technologies bypass the natural sperm selection barriers and demands the operator to select high-quality sperms. The criteria for sperm quality assessment provided by the World Health Organization are vitality, morphology, and motility [3]. A widely used method for sperm selection is motile sperm organelle morphology examination (MSOME) [4–7]. Sperm motility is also a commonly used criterion for sperm quality assessment. A motility grade is often used as a specified measure and classified into four grades:

  1. Grade 1:

    Sperm with fast progressive movements

  2. Grade 2:

    Sperm with slow progressive movements

  3. Grade 3:

    Sperm with slow non-progressive movements (i.e., with curved motion)

  4. Grade 4:

    Sperm are immotile and fail to move at all

Besides sperm assessment based on morphology and motility, another method for selecting a healthy sperm is based on the analysis of sperm DNA integrity. Some of the DNA analysis methods assess sperm DNA quality directly, such as the TUNEL assay and the sperm chromatin structure assay. Some other systems indirectly measure sperm DNA quality. For instance, Huszar’s group recently proposed a hyaluronic acid (HA) assay [8]. Among a consecutive series of studies on HA-based sperm assay, Huszar’s group indicated that the HA assay permits the selection of healthy sperm with no DNA damage [9, 10]. In their studies, sperms that bind to HA microdots are proven to have a higher level of DNA integrity compared to those unbound sperm. In clinical HA-based sperm selection, a number of healthy sperm’s head bind to the HA microdot and lose their progressive movement despite vigorous tail beating. In this case, the sperm tail beating movement becomes the only indicator to differentiate the HA bound sperm from each other.

The past few decades have witnessed the development of computer-assisted sperm analysis (CASA) methods for measuring both sperm morphology and motility [11]. CASA utilizes an automated system to digitize successive images of sperm, process, and analyze the information and provide the accurate and objective value for individual sperm cell. Since 1970s, many algorithms have been developed to track sperm trajectories, measure sperm velocities, and analyze sperm morphology. Shi et al. reported a robust single-sperm tracking algorithm based on a four-class thresholding method to extract a single sperm in a region of interest (ROI) [12]. The nearest neighbor method is complemented with a speed-check feature to aid tracking in the presence of additional sperm or other particles. In another study, Nafisi et al. demonstrated a template matching algorithm for sperm tracking. The algorithm is insensitive to image acquisition conditions [13]. Existing algorithms for sperm tracking are largely limited to sperm head tracking. The small size ( ≤ 1 μm in thickness) and low contrast of sperm tails under optical microscopy make sperm tail tracking challenging. In a recent study [14], a maximum intensity region algorithm was developed for sperm tail detection and tracking.

In this chapter, we first discuss methods for tracking sperm head and tail via real-time image processing. We then discuss sperm analysis on the basis of sperm motility, morphology, and DNA quality. Recent developments in automated sperm manipulation systems will then be introduced. We finally discuss future research directions.

12.2 Sperm Tracking

12.2.1 Sperm Head Tracking

Compared with sperm tail, sperm head has clearer contrast under microscopy. Therefore, most CASA systems were designed for tracking sperm head only. To successfully track sperm heads, the algorithms should take into account sperms’ three-dimensional movements, which can cause targets to be out of focus. A target sperm can also be occluded by other sperms or by foreign particles present in the Petri dish.

Sperm head tracking often begins with filtering noise and image enhancements. Filters (e.g., median filters and Gaussian filters) are used in most of the sperm tracking systems. For image enhancements, Nafisi et al. proposed a two-step enhancement method for noise reduction [13]. The first step is to remove completely stationary objects via frame subtraction. The second step is to remove or reduce the effect of those objects with vague boundaries (due to out-of-focus) by using wavelet transform.

After noise reduction, sperm head detection is performed by using template matching, fitting ellipse, and thresholding approach. Template matching is robust to different imaging modes (e.g., bright field, phase contrast, and DIC). However, the high computing cost could make this method unsuitable for real-time sperm tracking tasks. Besides for sperm head tracking, fitting ellipse can also be used for sperm tail tracking. Through analyzing the shape of sperm head, this method can provide the direction of sperm tail for sperm tail detection. It was reported that sperm tail can be found along the major axis of the ellipse [15]. However, this method is less effective for detecting sperm heads that do not have regular elliptical shapes.

The most commonly used method for sperm detection is the thresholding approach [14, 16, 17]. This approach applies thresholding inside a ROI. The ROI for sperm head can be initiated by a human operator who selects a desired sperm head to track via computer mouse click on the sperm head (Fig. 12.2). The ROI image can be binarized by applying Otsu’s adaptive thresholding algorithm (Fig. 12.2c). Within the binarized ROI image, the contour of the sperm head is computed. The sperm head position is obtained by calculating the moments of the contour. The ROI is then updated to be another ROI centered at the sperm head’s centroid. For subsequent frames, a similar process is performed to track the sperm head position.

Fig. 12.2
figure 2

Single sperm head tracking: (a) Sperm in the original image. (b) The sperm head ROI image is extracted. (c) The sperm head is found through adaptive thresholding

Full size image

When a sperm is occluded by other sperms or particles, the sperm tracking algorithms should be able to differentiate between the sperm of interest and interfering sperms/particles that are present in the ROI. The swimming direction vector of the sperm of interest can be used as a unique identifier to discriminate the sperm of interest from other sperms/particles. In the situation where only one sperm is present in the ROI, the sperm’s current direction vector D(i) in the current frame, represented by (12.1), is found by subtracting the sperm centroid position in the previous frame P(i − 1) from the sperm centroid position in the current frame P(i).

$$\displaystyle{ D(i) = P(i) - P(i - 1) }$$
(12.1)

When more than one sperm or object is present in the sperm head ROI (SHROI), (12.1) is extended to

$$\displaystyle{ D(i,s) = P(i,s) - P(i - 1,s_{\mathrm{prev}}) }$$
(12.2)

where s represents each sperm in the SHROI and s prev is the sperm of interest in the previous frame. The candidate sperm s that produces the minimum Euclidean distance value is considered the sperm of interest s soi

$$\displaystyle{ s_{\mathrm{soi}} =\min _{s\in [1,N]}\|D(i,s) - D(i - 1,s_{\mathrm{prev}})\| }$$
(12.3)

where N is the total number of sperm and objects inside the SHROI. Specifically, the nearest neighbor approach is applied to determine the sperm of interest s soi at frame i, with the knowledge of the sperm of interest s prev at frame i − 1. The s soi is updated for every frame using this nearest neighbor computation.

12.2.2 Sperm Tail Tracking

Tracking sperm tail can also be useful, such as for the manipulation of sperm and for selecting sperms that are bound to HA dots in the HA assay. After the sperm head position is detected, a sperm tail region of interest (STROI) is extracted by using the sperm head position and the average direction vector of its movement, as shown in Fig. 12.3b. The average direction vector \(\overline{D}\) is used instead of the direction vector D(i) because the sperm may exhibit abrupt changes in movement direction between two consecutive frames. By averaging the direction vectors of the sperm across a number of frames (e.g., 30 frames), the effect of abrupt changes in the sperm moving direction between frames is mitigated and the extraction of STROI becomes more robust.

Fig. 12.3
figure 3

Sperm tail tracking. (a) Sperm head position is found. (b) STROI (sperm tail region of interest) is determined. (c) 5 ×5 windows are scanned to locate the section with the highest intensity sum in the flicker image. The center point (blue dot in figure) of the section is considered the tail location. (d) Based on the blue dot position found in (c), Kalman filter is applied to improve the accuracy of located sperm tail position (©IEEE 2013), reprinted with permission

Full size image

The STROI’s center position in the i frame, T(i), is determined by subtracting a scaled value of the direction vector from the sperm head’s centroid

$$\displaystyle{ T(i) = P(i) - a \cdot \frac{\overline{D}} {\|\overline{D}\|} }$$
(12.4)

where a is a scalar value determined by the human sperm length. Under the 20 ×magnification, the average length of human sperms is approximately 90 pixels (i.e., a = 90). After the center position is found, a 25 ×25 ROI is taken as the STROI. The size of 25 ×25 provides a sufficient tail search area that takes into consideration a range of sperm tail length variations and sperm tail beating amplitudes.

After finding the STROI, the algorithm verifies that a tail is present in the STROI. The fundamental feature of flicker is extracted by taking the absolute difference between six consecutive inverted grayscale image frames.

$$\displaystyle{ f(i) =\mathop{\sum _{k=0}}\limits^{5}\vert I(i - k) - I(i - k - 1)\vert }$$
(12.5)

where f(i) is the flicker image extracted at frame i and I represents the grayscale images containing the sperm of interest in frame i to frame i − 5. Each pixel in the flicker image is squared to enhance the pixel values of areas in which the tail is present. The sum of pixel value in the STROI of the f(i) image is used as a measure to determine the presence of a sperm tail. If the pixel sum is above a specified threshold value, a tail is considered present. The threshold value was found experimentally by comparing the pixel sum values of STROI images in which a tail exists against cases where no tail exists. An example flicker image is shown in Fig. 12.3c. If the pixel sum is below a threshold value, no tail is found inside the STROI. This situation can occur when the sperm of interest moves out of focus, resulting in the disappearance of the sperm tail.

Once the sperm tail is determined to exist within the STROI, the maximum intensity region algorithm will locate a point on the sperm tail by using the flicker image. This flicker image approach overcomes the challenges that arise from the low-contrast image of the sperm tail in a single frame. The algorithm first finds the location of maximum intensity within the STROI of the flicker image. This is accomplished by evaluating the sum of the intensity values inside a 5 ×5 window at a spatial sampling interval of 5 pixels in both the x and the y coordinates of the STROI flicker image. The center position of the 5 ×5 window with the highest intensity is considered the tail location (i.e., a point on the sperm tail). In order to obtain an accurate sperm tail position, a Kalman filter can be applied to optimize the tracked results. Figure 12.3d shows the sperm tail tracking result and Kalman filter optimized result.

12.3 Sperm Analysis

12.3.1 Sperm Motility Analysis

Sperm motility is a basic criterion for assessing sperm quality. The most commonly used criteria for sperm motility analysis are sperm’s curvilinear velocity (VCL), straight line velocity (VSL), and movement linearity (LIN). Additionally, sperm tail beating amplitude is also a motility value that reflects the sperm’s locomotive behavior. These criteria can be calculated from the sperm tracking results.

Assume a sperm enters the field of view at frame i and swims out of the field of view at frame i + N. With the sperm position detected in each frame, the travel distance of the sperm between two consecutive frames can be determined from its direction vector, D(i). The VCL, which is the average velocity of the sperm head along its actual curvilinear path, is

$$\displaystyle{ \mathrm{VCL} = \frac{1} {N}\mathop{\sum _{k=0}}\limits^{N - 1}D(i) }$$
(12.6)

The VSL, which is the average velocity of the sperm head along the straight line between its first and last detected position, is

$$\displaystyle{ \mathrm{VSL} = \frac{P(i + N) - P(i)} {N} }$$
(12.7)

where P(i) represents the position of the sperm at frame i. The linearity (LIN) of the sperm’s curvilinear path is

$$\displaystyle{ \mathrm{LIN} = \frac{\mathrm{VSL}} {\mathrm{VCL}} }$$
(12.8)

where LIN is the linearity measure (0 ≤ LIN ≤ 1). A higher LIN value means that the sperm’s moving path is more linear. In sperm selection, healthy energetic sperms with progressive/linear movement are desired (vs. those traveling in circles for instance).

With the sperm tail’s position, the sperm tail beating amplitude inside the STROI is computed. The relative position inside the STROI in frame i is denoted by PT(i). The sperm tail beating amplitude is

$$\displaystyle{ A = \frac{1} {N}\mathop{\sum _{i=1}}\limits^{N}\|PT(i) -\overline{PT}\| }$$
(12.9)

where \(\overline{PT}\) is the sperm tail’s average position inside the STROI and N is the number of frames until when the sperm tail is successfully detected. Generally speaking, a higher sperm tail’s beating amplitude will result in a faster sperm head movement. Thus, sperm tail’s beating amplitude can possibly be used as an indicator to reflect sperm’s motility when the sperms lose their head motion (e.g., in HA-based sperm selection).

12.3.2 Sperm Morphology Analysis

Sperm morphology has been recognized as a powerful predictor of the outcome of natural conception, intrauterine insemination, and conventional IVF therapies [18, 19]. For sperm morphology analysis, a method of unstained, real-time, high-magnification motile sperm organellar morphology examination (MSOME) was developed [20, 21].

In MSOME, motile sperms are transferred to an observation microdroplet of sperm medium containing PVP solution. PVP solution is added to slow down the sperm moving speed. To reduce the toxicity of PVP, a low concentration of the PVP is typically used ( < 8%). In order to assess the morphological state of the sperm nucleus, the motile sperm cell must be kept inside the field of view for at least 20 s. According to Berkovitz’s study [20], the success of MSOME is dependent on: (1) optical resolution, which depends on microscope optics and on the light source of the microscope; (2) image contrast, which is enhanced by Nomarski differential interference contrast optics; (3) maximal optical magnification, which is commonly higher than 100 ×; and (4) magnification of the video system, which is equal to the ratio of TV monitor diagonal dimension to the CCD chip diagonal dimension.

A sperm is considered morphologically normal when it exhibits a normal nucleus as well as acrosome, post-acrosomal lamina, neck, tail, and it does not possess a cytoplasmic droplet or cytoplasm around the head [21]. A normal nucleus reveals a smooth, symmetric, and oval shape. The normal length and width of the nucleus are estimated as 4.75 ± 2.8 μm and 3.28 ± 0.2 μm, respectively.

12.3.3 Sperm DNA Integrity Analysis

A number of studies have indicated that infertile men have a higher level of DNA strand breaks and other types of DNA damage than fertile sperm donors [22–24]. In order to detect DNA damage, several sperm analysis methods have been developed, including TUNEL [25], comet [26], in situ nick translation [27], DNA breakage detection fluorescence in situ hybridization [28], sperm chromatin dispersion test [29], and sperm chromatin structure assay [30]. Some of these sperm analysis systems assess DNA quality directly, such as TUNEL or comet at neutral pH while others measure DNA damage levels after denaturation steps, such as the sperm chromatin structure assay, sperm chromatin dispersion test, and comet at acid or alkaline pH.

Indirect methods measure DNA susceptibility to denaturation after exposure to acid conditions [31]. However, it has been reported that these methods can only evaluate acid-labile sites and would not have a significant impact on the formation of male pronucleus because the intracellular pH of the oocyte is approximately 7.0 [32]. Another indirect approach checks sperm’s DNA integrity by using the hyaluronic acid (HA) coated Petri dishes [8]. HA is a linear polysaccharide in the extracellular matrix of cumulus oophorus around the oocyte that seems to play an important role in natural human fertilization [33]. Huszar et al. proved that sperms that bind the head to the HA have a higher level of DNA integrity than those sperms that cannot bind to HA. After a sperm binds the head to a HA microdot, it loses the head movement but reveals more vigorous tail motion.

12.4 Sperm Manipulation

Sperm manipulation is routinely performed by embryologists in the ICSI procedure in IVF clinics. Sperm manipulation includes immobilizing a sperm, aspirating it into a micropipette, and positioning it inside the micropipette.

12.4.1 Sperm Immobilization

Sperm immobilization must be performed before injecting the sperm into the egg to increase the chance of fertilization since sperm tail movement can cause damage to the intracellular structure of the egg [34]. In sperm immobilization, a micropipette is used to press (tap) the sperm tail against a surface (e.g., the bottom of a Petri dish).

Sperm manipulation requires dexterous operation due to the motile nature of sperms and their small size. Recent study has shown that automated sperm immobilization using a robotic approach is feasible [14]. In automated sperm immobilization, a vision-based contact detection was first performed to determine the vertical depth between the micropipette and the surface of Petri dish. Single sperm head and tail tracking is then conducted to locate the sperm head and tail’s positions. The sperm head tracking algorithm enables the system to visually servo a motorized X–Y stage to keep the moving sperm at the center of the field of view. A micropipette is then controlled to tap the midpoint of the sperm tail against the dish bottom to immobilize the sperm (Fig. 12.4). The midpoint of the sperm tail is found by averaging the tracked sperm head and tail position.

Fig. 12.4
figure 4

Automated sperm immobilization

Full size image

Two conditions must be met for tapping the sperm tail: (1) sperm tail needs to be at appropriate orientation and (2) sperm moves within a depth of 25 μm above the dish bottom. In the automated sperm immobilization system, the micropipette is placed on the left side of field of view. Thus, the target sperm should move near-vertically (i.e., within − 45 to 45  ∘  with respect to the y-axis). For some sperms that do not have the appropriate orientation, a rotational stage can possibly be used to adjust the sperm tail orientation. To immobilize a sperm, the micropipette needs to tap the sperm tail against the dish bottom. If a sperm is too far away from the dish bottom (e.g., > 25 μm), the system will fail to tap the sperm tail. The automated immobilization system computes the focus measure by using the normalized variance [35] of ROI image of the sperm head. During sperm immobilization, the microscope is focused on the dish bottom. A sperm with higher normalized variance is close to the focal plane and is preferred for immobilization.

Trials of the automated sperm immobilization system on 1,000 sperms indicate the success rate of sperm immobilization is 88.2%. The failure cases include (1) the turbulent flow caused by micropipette movement displacing the sperm’s original position; (2) the sperm moving at 25 μm above the dish surface; (3) the sperm tail not staying in the required orientation; (4) sperm tail changing the orientation during the taping step; (5) sperm increasing its speed during the taping step.

12.4.2 Sperm Aspiration and Positioning Inside Micropipette

After the sperm is immobilized on the Petri dish bottom, the next step is to aspirate the sperm into the micropipette and position it to a desired location inside the micropipette. Aspirating a single sperm into a micropipette and precisely controlling the sperm’s position within the micropipette is challenging, due to the small volume of a sperm (picoliter) and the nonlinear dynamics involved in the process (e.g., varying mass of culture medium entering the micropipette in cell aspiration).

In manual operation, an operator looks through the eyepieces of a microscope and operates multiple devices (microscope stage, micromanipulator, pump, etc.). When the micropipette approaches a target sperm cell, a small negative pressure is applied to aspirate the sperm into the micropipette. Once the sperm enters the micropipette, which is a rapid event, the operator must quickly apply a positive pressure to stop the sperm movement so that the sperm does not enter too far into the micropipette and disappears. In order to position the sperm to a desired location inside the micropipette, the operator must repeatedly adjust the application of negative and positive pressure skillfully.

In automated sperm aspiration and positioning [36], the sperm aspiration and positioning tasks are achieved via computer vision microscopy and closed-loop motion control. The major components in the micropipette aspiration system are a syringe, a micropipette, and a connecting tube. The syringe and connecting tube are filled with mineral oil. The inner space of micropipette typically consists of three segments: mineral oil, air, and culture medium. A sperm cell moves together with the culture medium segment (no relative motion within the culture medium). Figure 12.5 is a schematic diagram of a sperm moving inside a micropipette.

Fig. 12.5
figure 5

A schematic diagram of a sperm moving inside a micropipette [36] (©IEEE 2012), reprinted with permission

Full size image

In the automated system, the micropipette is first controlled to approach the immobilized sperm. The sperm is then aspirated into the micropipette by applying a negative pressure. In this process, the target sperm is detected and tracked. After the sperm is aspired into the micropipette, a robust controller quickly positions the sperm to a desired position in the micropipette. In the study of the sperm positioning in the micropipette, it was reported that the robust controller significantly outperforms PD controllers in terms of efficiency, overshoot, and accuracy.

12.5 Conclusion

Sperm analysis, which involves the measurement of sperm’s velocity, morphology, and DNA integrity, is necessary for male infertility diagnostics, sperm quality assessment, sperm selection, and other IVF related issues. In the past few decades, CASA systems were developed and used for providing objective and accurate results for sperm analysis. The core of CASA systems is computer vision-based sperm tracking. Most of the sperm tracking algorithms are focused on sperm head tracking. Due to the emerging HA-based sperm assay and the automation of sperm manipulation, sperm tail tracking becomes necessary. This chapter introduced sperm head and tail tracking methods as well as sperm manipulation techniques, including immobilization, aspiration, and positioning in a micropipette.

Further studies using these sperm tracking, analysis, and manipulation technique as well biochemical approaches will answer presently open questions, such as: (1) Does a sperm that moves faster than others have a higher level of DNA integrity? (2) Does a sperm possessing perfect morphologies have less DNA defects? (3) Does a sperm showing faster tail motion, among the HA bound sperms, have higher reproductive quality? The engineering techniques discussed in this chapter will prove instrumental in addressing these questions and will prove useful in both biology research and reproductive medicine.