Computational imaging of sperm locomotion

Abstract

Not only essential for scientific research, but also in the analysis of male fertility and for animal husbandry, sperm tracking and characterization techniques have been greatly benefiting from computational imaging. Digital image sensors, in combination with optical microscopy tools and powerful computers, have enabled the use of advanced detection and tracking algorithms that automatically map sperm trajectories and calculate various motility parameters across large data sets. Computational techniques are driving the field even further, facilitating the development of unconventional sperm imaging and tracking methods that do not rely on standard optical microscopes and objective lenses, which limit the field of view and volume of the semen sample that can be imaged. As an example, a holographic on-chip sperm imaging platform, only composed of a light-emitting diode and an opto-electronic image sensor, has emerged as a high-throughput, low-cost and portable alternative to lens-based traditional sperm imaging and tracking methods. In this approach, the sample is placed very close to the image sensor chip, which captures lensfree holograms generated by the interference of the background illumination with the light scattered from sperm cells. These holographic patterns are then digitally processed to extract both the amplitude and phase information of the spermatozoa, effectively replacing the microscope objective lens with computation. This platform has further enabled high-throughput 3D imaging of spermatozoa with submicron 3D positioning accuracy in large sample volumes, revealing various rare locomotion patterns. We believe that computational chip-scale sperm imaging and 3D tracking techniques will find numerous opportunities in both sperm related research and commercial applications.

Introduction

Sperm cells are assigned a very difficult task of finding the egg, where only one in a million spermatozoa is able to even enter the Fallopian tubes in humans [1]. Providing a better understanding of this complex phenomenon is of great scientific interest, and could help mankind both in medical [2] and technological [3] advances. Multiple aspects of this journey of the spermatozoa toward the egg [4] have been of interest for researchers, including the biophysics of the sperm's swimming patterns, both from a kinematics [5–14] and a fluid dynamics [15–18] perspective. Chemical signaling [19–25] and control, along with sperm guidance mechanisms [1, 26–31], form another critical piece of the puzzle and have been active areas of research. Among all these different directions of sperm research, one common need is to image and quantify the locomotion of sperm cells.

Not only for scientific research, but also for medical diagnostics and animal husbandry-related applications, imaging of sperm cells is critical. Imaging of semen samples with computer-aided sperm analysis (CASA) platforms that are composed of conventional optical microscopes, digital cameras, and specialized semen analysis software has already been an integral part in the diagnosis and analysis of male infertility [32–34]. It is estimated that there are 48.5 million couples suffering from infertility worldwide, where 20%–30% of the cases are caused by male-related factors [35], and it is expected that the global male infertility market will be valued at $301.5 million by 2020 [36]. As another related industry, the in vitro fertilization (IVF) market alone is expected to reach $27 billion by 2022 [37], further highlighting the value of automated sperm imaging and characterization platforms.

Starting with the early sperm imaging experiments that involved capturing sequences of microscopic images onto photographic films and hand tracing the cells, advancements in computational technologies have transformed the state of the art in semen analysis and characterization. Current CASA platforms heavily rely on advanced computational capabilities, which are leading the field toward unconventional computational imaging methods [38].

Sperm imaging work before digital microscopy

There was a strong interest in sperm imaging and tracking by researchers, even before video cameras were available. Researchers used objective lenses and projected microscopic images of sperm cells onto photographic films, capturing images at around 50–200 frames per second [39–41]. The sperm heads and even their 2D flagellar beating patterns were then hand traced across a stack of photographic films, revealing their trajectories. This earlier work provided the first clues on the swimming dynamics of sperm cells [39–42], pioneering the field of sperm imaging and tracking, despite being time consuming and very limited in throughput and sample volume.

It is not surprising that computer-interfaced video cameras that can be attached to microscopes were immediately adapted by the sperm imaging community. Instead of manual tracking on photographic films, microscopic images of sperm cells were digitally processed using computers with earlier versions of sperm detection and tracking software [43]. With some of these rapid developments, CASA quickly became the tool of choice for sperm imaging and characterization also helping with the standardization of sperm handling and quantification methods [44]. In fact, the critical computational task of multiple-target detection and spatio-temporal tracking in these stack of digital images immediately became a research topic of its own [45], leading to more robust and efficient algorithms that have been widely used by the community.

Conventional sperm tracking and computer-aided sperm analysis

A conventional CASA system that is widely used and commercially available consists of an optical microscope (typically using a ×10–×20 objective lens) with a digital camera attachment that records the microscopic images of the sperm samples [33], which are further processed with custom sperm analysis software (Figure 1a). The sperm cells are typically placed in a chamber that is 20 μm deep [46], confining their motion vertically, given that human sperm head is ∼4–5 μm wide and the flagellum is ∼40–50 μm long [47]. A stack of digital images are recorded, with the camera operating at a frame rate that is sufficient to capture the locomotion of the sperm cells without temporal undersampling, e.g. >60 fps for tracking human spermatozoa [32].

Figure 1.

(a) A standard CASA system [33]. Reprinted from Ref. 33, with permission from Elsevier. (b) Reconstructed sperm trajectories over ∼1 s, sampled at 60 Hz. The purple tracks are identified to be hyperactive [34]. (c) A zoomed-in trajectory of a sperm with the corresponding motility parameters also listed [34].

Sophisticated computational tools have been made available to process a stack of microscopic images to identify the sperm cells and connect their trajectories. Thresholding and image segmentation techniques [48] are widely used to find the sperm heads in digital frames, while advanced multi-object particle or cell tracking algorithms [49, 50] help register the positions of these sperm cells across the frame stack, building fully connected spatio-temporal trajectories of the spermatozoa (Figure 1b). Once these trajectories are determined, additional processing is applied to calculate several parameters used to assess sperm health and motility (Figure 1c), which include e.g. curvilinear velocity (VCL), average path velocity (VAP), the straightline velocity (VSL), linearity (LIN), straightness (STR), wobble (WOB), amplitude of lateral head displacement (ALH), beat-cross frequency (BCF) [32], and the recently included fractal dimension (D), among others [34].

These motility parameters have formed a better indicator of sperm functionality compared to conventional semen profile parameters, which only include the number of spermatozoa in a semen sample and the percentages of progressively motile and morphologically normal sperm [51–53]. Investigated as part of a donor insemination program, it was observed that VAP was larger in semen samples that were able to achieve pregnancy compared to the ones which failed to do so [53]. Along with VAP, ALH has been found to be another valuable fertility parameter, which has been linked to the cervical mucus penetration capability of spermatozoa, an important sperm functionality indicator itself [14, 32, 51, 53]. VCL has also been found to be an important indicator of sperm functionality for IVF applications [54], among other parameters calculated by modern CASA platforms.

Improving the performance of these computational tools is an active research topic for the CASA community, where advanced concepts like particle and Kalman filters [55] have been used for multi-object tracking of human sperm cells. One of the problems that current CASA systems face is when two or more spermatozoa collide spatially (or come very close to each other) during the imaging process [56, 57], confusing the trajectory calculation. A technology from air traffic control systems, i.e. the joint probabilistic data association filter, has been recently employed [57] to better handle such cell collisions. Probabilistic models have also been used to trace the low-visibility sperm flagellum effectively [58], along with maximum intensity region and optical flow algorithms [59]. With all these efforts in further improving the computational tools for CASA, much more robust and versatile systems should be available in the near future.

Holographic imaging of sperm locomotion

There are other limitations of conventional CASA systems that are directly related to optical imaging hardware. A conventional optical microscope with objective lenses is used to image the sperm cells in CASA systems, where there is an inherent trade-off between spatial resolution and field of view (FOV or sample volume). Furthermore, such microscopes are relatively bulky and expensive, limiting the CASA setups to laboratory settings. As a cost-effective and portable alternative to lens-based optical microscopy tools, computational on-chip holography [60–67] offers a unique opportunity to image and track sperm cells without using any lenses or other bulky optical components [68]. This alternative computational imaging platform only consists of an inexpensive complementary metal–oxide–semiconductor (CMOS) imaging sensor, which is widely used in consumer electronics products including mobile phones, and a simple partially coherent light source such as a light-emitting diode (LED) (Figure 2a). The semen sample is placed very close to the CMOS image sensor (<1 mm vertical distance between the sample and the sensor plane) and the light source is placed further above (e.g. ∼4–5 cm) with the use of a large diameter (100 μm) aperture/pinhole (Figure 2b) that improves the spatial coherence of the light illuminating the sample [68].

Figure 2.

(a) A lightweight (∼46 g) and cost-effective on-chip holographic microscope composed of a CMOS image sensor and an LED that is filtered by a pinhole [68]. (b) The schematics of the device in panel a. The sample FOV is equal to the CMOS active area, e.g. 20–30 mm². (c) The holograms generated by the interference of the background light with the scattered light from individual sperm cells. (d) A bright-field image of the target sperm cells, used for comparison purposes. (e and f) The amplitude and phase images obtained through digital reconstruction of the acquired holograms. The latter is especially valuable for visualizing sperm flagella. Reprinted from Ref. 68 with permission from American Chemical Society. (g and h) Pseudo 3D representation of the sperm thicknesses obtained from bovine spermatozoa with a cytoplasmic droplet along the flagellum and a broken acrosome, respectively, obtained via a different form of digital holographic microscope that uses lenses [69]. The colorbar represents the phase difference values in radians. Copyright (2010) IEEE. Reprinted from Ref. 69 with permission from IEEE.

The scattered light from each sperm cell interferes with the background light, creating holographic interference patterns (Figure 2c) that are captured by the CMOS image sensor. These interference patterns (or holograms) encode both the amplitude (Figure 2e) and the phase information (Figure 2f) of the object (Figure 2d), which could be digitally reconstructed through various back-propagation techniques [60, 68], effectively replacing the objective lens in a conventional optical microscope with rapid computation. Such a simple imaging platform not only offers a very robust, field-portable, and cost-effective alternative to CASA platforms, but it also allows high-throughput imaging and tracking of spermatozoa over a large FOV (e.g. ∼24–30 mm²) (Figure 2b). Unlike its lens-based counterparts, for a lensfree on-chip microscope the imaging FOV is decoupled from spatial resolution, and is only limited by the active area of the image sensor chip, which in our mobile phone cameras typically range between 20 and 30 mm², more than 20-fold larger than the FOV of a typical ×10–×20 objective lens.

Using this on-chip holographic imaging framework, an automated semen analysis platform that only weighs 46 g (Figure 2a) was created to measure the concentrations of both motile and immotile human sperm cells, along with the VSL distributions of the cells and the flagellum morphology of the immotile cells [68]. With a USB-based interface used to connect to a laptop computer, this portable and cost-effective computational microscope is a powerful alternative to CASA systems in resource-limited or field settings, relevant for animal husbandry applications, where it would be quite challenging to operate a conventional optical microscope.

Lens-based digital holography has also been used for computational sperm imaging, where the detailed morphology of individual sperm cells can be extracted (Figure 2g and h) [69, 70] along with the 3D volumetric imaging of the sperm head [71] without using contact-based imaging methods. A similar lens-based holographic imaging platform has also been used to track individual sperm cells in 3D [72, 73], however, with limited throughput due to the use of objective lenses which present a tradeoff in sample volume. As an alternative, lensfree on-chip holographic imaging also offers a unique opportunity to track large numbers of sperm cells in 3D with submicron positioning accuracy, which will be detailed next.

High-throughput 3D sperm tracking on a chip

Although 2D sperm tracking has been very valuable to assess sperm health and motility, the 3D nature of sperm kinematics has been relatively underexplored, mostly due to the limitations of lens-based optical microscopes, including poor depth resolution and the trade-off between lateral resolution and FOV, which also limits the imaging throughput. On-chip holographic imaging offers a high-throughput method to reconstruct the 3D trajectories of sperm cells with submicron 3D positioning accuracy [74], by only adding a second light source at a different wavelength and a known oblique angle (Figure 3a) in addition to the vertical illumination source [11].

Figure 3.

(a) Dual-angle on-chip holographic microscope for 3D imaging and tracking of sperm locomotion. A pair of holograms is simultaneously recorded for each sperm cell to accurately triangulate the vertical position of the sperm head, in addition to its lateral position. (b) High-throughput 3D tracking of spermatozoa with submicron 3D positioning accuracy. (c) The helical swimming pattern of human spermatozoa, observed in 4%–5% of the captured trajectories [11]. (d and e) Chiral ribbon trajectories of horse sperm, a twisted ribbon and a helical ribbon, respectively [12].

With this modification, a pair of holograms is simultaneously recorded for each sperm swimming in a deep chamber (e.g. ∼500 μm deep) that does not restrict the sperm motion in any direction. The acquired holograms are then back propagated separately with the corresponding illumination wavelength of each angle, and result in sperm reconstructions only if the correct set of parameters (angle and wavelength combination) are used, which helps the algorithm to correctly identify the sperm pairs and distinguish the vertical and oblique illumination perspectives of the sample volume. Note that although two different wavelengths (i.e. 625 nm for vertical illumination and 470 nm for oblique illumination) are simultaneously used for illumination, the CMOS image sensor chip is color blind, i.e. monochrome. This means that, at any frame, both the vertical and oblique holograms of a given sperm head will be detected at the same time, and without any immediately visible marker or sign to discern the two from each other. The choice of these two different illumination wavelengths is related to the reconstruction process, where the reconstructed image at one illumination wavelength will broaden the sperm head corresponding to the hologram that is acquired with the other wavelength, and diffuse it away to the background. This way, the correct reconstruction pairs for each sperm head can be obtained, and the distance between the centroids of the two reconstructions is used to accurately triangulate the vertical position of the target sperm head (Figure 3a) [11]. Finally, by extracting the horizontal position of the sperm head from the centroid of the vertical reconstruction, an overall submicron 3D positioning accuracy can be achieved in tracking the sperm cells over a large sample volume, ∼8–10 μL (Figure 3b).

This dual-angle on-chip holographic imaging device with two fiber-coupled LEDs and a CMOS image sensor operating at >90–140 frames per second [11–13] has been used to track over 24,000 human spermatozoa and over 9600 horse spermatozoa, revealing detailed 3D trajectories of individual spermatozoa. These large numbers of spermatozoa that are imaged allowed the observation of statistically rare swimming patterns, including the 3D helical trajectories of human spermatozoa (Figure 3c), which were observed 4%–5% of the time in vitro [11]. Using the same platform, other swimming patterns that define chiral and twisted ribbons in 3D (Figure 3d and e) have also been recorded in horse sperm samples, observed for the first time for any microswimmer type [12].

While the biological reasons of these peculiar 3D patterns in sperm locomotion are not well understood, the dual-angle on-chip holographic imaging platform offers a high-throughput capability for 3D tracking of microswimmers in large sample volumes, which could be especially valuable to better understand the biophysics of sperm locomotion. For fertility and sperm health analysis, CASA systems have been traditionally relying on 2D sperm trajectories, and the benefits of adding the third dimension into this analysis currently remain unclear. With further studies that investigate the relationship between the 3D swimming behavior and sperm health, 3D semen analysis might provide a critical improvement over currently existing 2D-based methods used in the assessment of sperm health and fertility.

Future directions

Being the driving force of sperm locomotion, there is no doubt that the flagellar beating mechanism is a crucial piece of the puzzle in complete understanding of sperm locomotion. As with the swimming behavior of the sperm head, there is no reason to expect that the flagellar beating should be confined to an approximate plane. The same limitations of conventional lens-based optical microscopes described in the previous sections do not allow for direct observation of 3D flagellum beating patterns. Only by rapidly oscillating a microscope objective [75, 76] (e.g. at 100 Hz), data on 3D flagellar beating of individual sperm cells have been recorded with limited throughput over a depth of ∼16 μm, which is smaller than the length of a human sperm flagellum.

Dual-angle on-chip holography could potentially offer a high-throughput computational solution to image the flagellar beating of sperm cells in 3D. Imaging the sperm flagellum is much more challenging, however, as the flagellum is an optically weaker object compared to the sperm head, creating a much weaker interference pattern on the sensor chip. Additionally, the flagellar beating requires a much higher frame capture rate (>200 fps [32]), which might require specialized high frame readout hardware. With potential improvements in the signal-to-noise ratio of the on-chip holographic imaging platform, along with a boosted frame rate and additional reconstruction and tracking algorithms, computational 3D imaging of complete sperm locomotion, including the flagellum and the head, could open new frontiers in sperm imaging and analysis.

Acknowledgments

The Ozcan Research Group at UCLA gratefully acknowledges the support of the Presidential Early Career Award for Scientists and Engineers (PECASE), the Army Research Office (ARO; W911NF-13-1-0419 and W911NF-13-1-0197), the ARO Life Sciences Division, the National Science Foundation (NSF) CBET Division Biophotonics Program, the NSF Emerging Frontiers in Research and Innovation (EFRI) Award, the NSF EAGER Award, NSF INSPIRE Award, NSF Partnerships for Innovation: Building Innovation Capacity (PFI:BIC) Program, Office of Naval Research (ONR), the National Institutes of Health (NIH), the Howard Hughes Medical Institute (HHMI), Vodafone Americas Foundation, the Mary Kay Foundation, Steven & Alexandra Cohen Foundation, and KAUST. This work is based upon research performed in a laboratory renovated by the National Science Foundation under Grant No. 0963183, which is an award funded under the American Recovery and Reinvestment Act of 2009 (ARRA). The authors also thank Dr. David Mortimer for sharing a high resolution image for our first figure.