Multi-Target Tracking for sperm motility measurement using the Kalman Filter and JPDAF: Preliminary Results

Abstract

The kinematic parameters calculated by a CASA system are the description of a series of geometrical measurements that depend on time and show information about the speed of movement of the sperm, as well as the beat frequency [Hz], changes of direction and amplitude of the trajectory described by the sperm head. Because the frequency of oscillation of the flagellum is around 80 Hz it is necessary to capture the images at least twice the frequency (160 Hz) to obtain its kinematics. [...]the measure of least variance is used as the representative value of that window (Sarabia & Munuce, 2011). [...]a comparison was made between the manual measurement and the algorithm implemented in order to find from the confusion matrix the accuracy of the counting classifier. [...]for motility, the percentage of reliability was established above 90.9% for categories A and B. It was possible to demonstrate the efficiency for at least 200 individual trajectories, by non-parametric Wicolxon correlation coefficient in a value greater than 86%.

Headnote

Abstract: The determination of sperm motility characteristics is of great importance for the specification of fertility in men. The semengram is the main diagnostic test to confirm semen quality. Currently, many fertility laboratories use visual assistance techniques to evaluate by using the Makler counting chamber, where motility and sperm count analysis can be performed. This research project proposes a method that allows the quantification of motility through the use of the probabilistic filter (JPDAF) based on the Kalman filter. This research requires the stages of segmentation, feature extraction and development of tracking algorithms for the association of sperm trajectories when there are multiple objectives. A total of 200 individual sperm were selected and the effectiveness for sperm classification was determined according to the mobility categories established by the WHO, obtaining an average value of 93.5% for the categories (A, B, C and D).

Keywords: Kalman Filter, JPDAF, morphology, motility, spermatozoa.

(ProQuest: ... denotes formulae omitted.)

1.Introduction

The spermogram is the most important and simple diagnostic test to start the study of male fertility in which physical aspects of semen are evaluated, such as volume, pH, viscosity, color and other aspects that require more specialized techniques such as concentration, motility and morphology (Avendaño, Mata, Sanchez Sarmiento, & Doncel, 2012; Blomberg Jensen et al., 2011). The development of methods that allow to determine the degree of fertility in man, is one of the fields that has been widely studied worldwide, even though the manual procedure is still one of the most used, it is carried out by experts in specialized clinical laboratories (Asociación Espanola de Urología. & SPARC (Organization), 2010; Wang, Fan, Behr, & Quake, 2012). This is where computer-aided systems (CASA) must respond to market needs in terms of accessibility and price (Mortimer, van der Horst, & Mortimer, 2015; Sharma, Harlev, Agarwal, & Esteves, 2016). These systems arose due to the need to perform a quantitative evaluation of the characteristics of the sperm samples based on the criteria established by the WHO (World Health Organization), which establishes the minimum amount of sperm and visual fields to be evaluated (Kobori, Pfanner, Prins, & Niederberger, 2016; Verón et al., 2018). This research work aims to develop an alternative method of human sperm analysis using image processing techniques and predictive algorithms based on probability filters, and thus overcome some limitations of current methods in relation to the monitoring of multiple sperm (Arasteh, Vosoughi Vahdat, & Salman Yazdi, 2018). This article describes the design, implementation and experimental validation of an artificial vision system for the kinematic determination and monitoring of multiple individual sperm using a Kalman predictor. Given the difficulties of label association when there are multiple candidates, the technique used is based in JPDAF (The joint probabilistic data-association filter), a probabilistic filter that makes it possible to optimally obtain the most likely candidate given a combination of potential candidates (Urbano, Masson, VerMilyea, & Kam, 2017). The contributions of this work are:

* Adjustment specifications on tracking algorithm in order to improve robustness and reduce processing time.

* Implementation of the Kalman Filter and JPDAF for label association and error reduction in trajectory prediction.

* Performance of algorithms for tracking and calculation of parameters in real time through efficiency study.

2.Materials and methods

2.1.Sample Preparation

To carry out this study, a total of 5 sperm samples were collected through a clinical laboratory, from which the age range was between [20-35] years. For the collection it is necessary to follow the WHO standard procedure (Bungum, Bungum, & Giwercman, 2011; Lu, Huang, & Lü, 2010; Pant et al., 2011):

* The patient is given a clearly written instruction sheet on how to collect the semen and transport it.

* The sample must be collected after 48 hours and no more than seven days of sexual abstinence.

* To make the initial evaluation, two samples of semen should be studied. These should not be older than 7 days.

* The sample must be obtained through masturbation and ejaculate in a widemouth plastic container at a temperature between 20°- 40°C.

The samples were processed by the Microscopy Unit of the University of Cauca and each sample was liquefied at an ambient temperature of 20 ° C and separated from the seminal plasma in three cycles of centrifugation. The evaluation technique was optical microscopy and it was necessary to increase the contrast of the samples by means of the simple staining method, whose preparation consists in adding a mixture of eosin at 5% and two drops of nigrosin on 3mL of semen sample (Esteves, 2014). To guarantee the reliability of the analysis, two samples must be prepared given the previous procedure and once they are evaluated the results are compared with tables that allow establishing the degree of reliability, which must be higher than 95%. If it is less than this value, the procedure for sample preparation must be repeated (Esteves, 2014; Imani, Teyfouri, Ahmadzadeh, & Golabbakhsh, 2014; Sharma et al., 2016).

2.2. Image acquisition

Because it requires counting, morphological analysis and mobility, it is necessary to capture 20-second video sequences in two different fields in order to guarantee the reliability of the process. For this, the Nikon-Eclipse reference optical microscope with 10x and 40x lenses was used, by controlling the temperature of the sample at 37 ° C. The videos were stored and sent to the computer in MJPG format, at a rate of 30 FPS (frames per second), with a resolution of 1920 x 1080 pixels. The process of calibration and adjustment of the distances used the standardized 100mm grids used in the Makler counting chamber (Kobori et al., 2016).

2.3. Calculation of mobility parameters

The World Health Organization (WHO) has defined male fertility based on concentration, percentage of normality and sperm motility. In the evaluation of the reproductive capacity, mobility is a determining criterion for its evaluation, it is necessary to analyze a total of 200 spermatozoa in 2 different samples in order to make the examination reliable (Verón et al., 2018). WHO classifies mobility into four categories, which are described by the values referenced in the table 1. The kinematic parameters calculated by a CASA system are the description of a series of geometrical measurements that depend on time and show information about the speed of movement of the sperm, as well as the beat frequency [Hz], changes of direction and amplitude of the trajectory described by the sperm head.

Because the frequency of oscillation of the flagellum is around 80 Hz it is necessary to capture the images at least twice the frequency (160 Hz) to obtain its kinematics. The head is taken as geometrical center The parameters to calculate are shown below (Urbano et al., 2017):

* Curvilinear velocity (VCL): Velocity, in pm /s, of the sperm in its curvilinear trajectory, this being the two-dimensional projection of the real threedimensional trajectory of the same defined as the ratio between travel and time.

* Linear velocity (VSL): Speed, in pm /s, of the sperm in its rectilinear trajectory, this being the result of joining the first and last point of the curvilinear trajectory during the observation period.

* Average speed (VAP): Speed, in pm /s, of the sperm in its average trajectory during the observation period; the average trajectory being indicative of the average direction of displacement of the sperm cell, its points are obtained from the average value of the coordinates of the points of the adjacent circular path.

* Linearity Index (LIN): The percentage relationship between the VSL and the VCL given by the relationship LIN= (VSL/VCL) ·100 indicates how close the circular path of the sperm is to a straight line. The circular paths have a low LIN, since the circular path will be much greater than the net space gained.

The mobility parameters are estimated from a set of points and time measures associated with each of the positions. Algorithms are implemented to act under changes of sperm direction, as well as oscillation around trajectory.

2.4.Image Processing

For implementation and description of the proposed model, the OpenCV library was used in the C ++ programming language. The stages of system development are shown below:

Morphometric calibration: For this stage we proceeded to make the measurement of the reference rules in order to determine the pixel-distance relationship and its disposition was on the horizontal and vertical axis of the microscope because the resolution can differ by a small amount (Arasteh et al., 2018).

Image filtering: One of the main stages in the systems of artificial vision is the improvement of the characteristics to be determined, because this can simplify the processing techniques to be applied. One of the characteristics observed on the images is the high contrast. This factor may be convenient, but in excess it incorporates artifacts for further processing. For this type of noise, it is essential to apply a smoothing filter, but given the high contrast, linear smoothing filters tend to "blur the axes" because the high frequencies of an image are attenuated (Guerrero González, Cardona Maya, & Morantes Guzmán, 2007). When the objective is to have greater noise reduction without affecting the edges of the image, non-linear filters are used, which represents an alternative. In this case, the filter used is called Kuwahara, which consists of the use of a window divided into 4 sectors to which the value of mean and variance is calculated. Finally, the measure of least variance is used as the representative value of that window (Sarabia & Munuce, 2011).

Segmentation of the images: Once the improvement of the characteristics has been made, the segmentation process is carried out, which consists of separating the sperm from the bottom. Given the changes of illumination in the images, it is decided to use the Mixtures of Gaussians method, a process that consists in accessing the intensity level values of each pixel in each channel (Giaretta et al., 2017; Ravanfar et al., 2014). Given a value of random X, it is possible to determine the probability that a pixel belongs to the object of interest (different from the background) and is it given by the following distribution:

... (1)

where from the equation (1) it is established that κ is the number of regions to use, and R > o and where the sum of all the probabilities will be determined by ...

... (2)

Equation 2 shows the probability distribution function, where ¡ii y ai correspond to mean and standard deviation respectively. For an image 1(¾ y) the data is determined on the basis of the model, the number of regions to divide the image in relation to the value of the histogram.

Extraction of morphological parameters: After the segmentation process, each object found is then labeled for subsequent monitoring. The next process is to determine those candidates that have the characteristics of the object of interest. For this, color and texture characteristics are extracted to allow the differentiation of live sperm from dead sperm. After the analysis of the color planes, the conversion to the HLS plane (Hue, Luminance, Saturation) is established, since the staining allows to reliably determine the tonality changes of the live and dead sperm. The texture characteristics such as (entropy and similarity) allow the separation of sperm from the other objects present in the sample.

Tracking and label association based on Kalman and JPDAF: This section shows the development of mathematical modeling and notations developed by JPDAF that denote the position of an individual sperm in a frame k given by the prediction of the state, prediction of the covariance and prediction of the measurement, determined by the following equations (3), (4), (5) and (6) :

... (3)

... (4)

... (5)

... (6)

The index tj denotes the objective t that associates a measure j. For the realization of the speed measurement validation, the noise measure of the covariance given by the equations (7) and (8) is used:

... (7)

... (8)

where Ci, C2 and C3 are the memory decay coefficients. Finally, a comparison was made between the manual measurement and the algorithm implemented in order to find from the confusion matrix the accuracy of the counting classifier. The statistical analysis was carried out using software Gnu PSPP 0.10.12. Normality tests were applied applying Ryan-Joiner's statistical test for the analysis variables. The following calculations were made:

* Mean and standard deviation for the parameters VCL, VSL, VAP, LIN, STR y WOB.

* Dispersion charts (VSL vs VCL), (LIN vs ALH), (VSL vs WOB) y (MAD vs LIN).

* Confusion analysis for comparison between classification data calculated by the mobility algorithm (A, B, C and D) regarding those classified by expert.

To establish the differences of the trajectory points obtained by algorithm tracking and association of labels (Kalman and JPDAF) regarding those extracted by expert using a manual tool, a Wilcoxon statistical test was used for related samples and a value of p<0.05 was considered as a value of statistical significance.

3.Results and Analysis

In this section, we present the results obtained in the process of segmentation and development of the individual sperm tracking algorithm, where predictive and probability techniques were used to associate trajectories. Once obtained, the motility parameters established by the WHO are extracted. One of the main achievements was to improve the effectiveness of the tracking algorithm, which allows the association of multi-objective tags, that is, it allows solving the cases of intersection of trajectories described by sperm.

3.1. Reliability of the tracking algorithm

In this section we present the monitoring results obtained by applying the algorithm of JPDAF and Kalman for multiple objective scenarios. From the analysis of 200 spermatozoa from samples A and B, a total of 10 intersections of 2 spermatozoa and 6 intersections of 3 spermatozoa were identified. Figure 2 shows some trajectories taken from the videos captured. For these scenarios the efficiency of the algorithm is estimated.

The results show a small improvement to solve individual sperm crosses. In some cases where the trajectories present a very large variability, the JPDAF algorithm does not solve the crosses. Table 2 shows the percentage of reliability for each of the methods used for the identification of intersections.

The kinematic parameters analyzed are part of 2 samples collected from 200 sperm for a ioox lens, the measurement was validated by inspection of the samples duplicated by the expert in order to guarantee reliability of the test. In addition, it was necessary to perform a normality test on the data to determine the values of mean and standard deviation. The statistical test used was that of Ryan-Joiner where the correlation value of (p<0.05,ES=0.825) is close to 1 is approved for the parameters VCL, VSL, LIN, ALH and WOB were calculated and correlated in scatter plots, these are indicated in Fig. 3 and Figure 4 for two samples A and B respectively.

As it is possible to observe in Figure 3, there is a high consistency between the VSL values vs. VCL, unlike VSL vs. data. WOB where data is more dispersed. Figure 4 shows a greater consistency in the VSL vs. data. VC and VSL vs. WOB These graphs give us indications about the behavior of these variables over time.

The mobility parameters for sample A, such as VCL, VSL, LIN, ALH y WOB and were calculated and plotted. The expert using the Makler counter obtained average concentration values of 30.6x10-6 sperm / ml. A total of 200 sperm were counted in two samples (A and B) and identification and monitoring software was used. Table 3 shows the average values and the standard deviation of the speed of sample A. While the data measured for sample B is 20.6x10-6 sperm / ml, the average values and the standard deviation of velocity can be seen in Table 4.

The data obtained for sample A are larger than for sample B, after the sperm identification process, the follow-up algorithm based on minimum distance criteria is applied.

It was applied for a sequence of 10 seconds and the extraction of the kinematic parameters was performed, resulting in a curve that describes the trajectory of the sperm around a line that determines the degree of deviation. This behavior can be observed in figure 5.

Once the kinematic parameters are obtained, this information is correlated with the data obtained in the manual assessment by the expert in the confusion matrix in Table 5 and Table 6, for samples A and B respectively.

The data obtained allow finding reliability percentages above the values referenced by the WHO, and it is also possible to observe from the confusion matrix that errors are established for underlying categories. Category A is confused with B for some classifications and this may be due to a hysteresis problem since most of the errors are those that are at a speed that oscillates around the defined threshold within the uncertainty of the measurement value. Regarding the comparison of the manual method with respect to that achieved with the tracking algorithm using the Wicolson coefficient, it was possible to find a correlation of (p <0.05, ES = 92), values that allowed to establish the degree of reliability in the tag assignment and demonstrate the reliability in determining trajectories when there are bifurcations, given the overlap of spermatozoa in the seminal fluid.

4.Conclusions

The system allowed tracking up to 200 individual sperms in real time by means of the JPDAF algorithm, and measuring the average speed, displacement, distance traveled and frequency with a resolution level of 10 microns. The preliminary results of the system allowed the identification and classification of the categories defined by the WHO based on an expert system, obtaining an accuracy in the results for the identification of sperm of 95.5%, and 95.7% for the dead sperm count (data that can be deduced from D motility category). Finally, for motility, the percentage of reliability was established above 90.9% for categories A and B. It was possible to demonstrate the efficiency for at least 200 individual trajectories, by non-parametric Wicolxon correlation coefficient in a value greater than 86%. At least for the 12 trajectories of a total of 16 that presented the crossing of two and three spermatozoids, separation was achieved in an adequate manner. It was possible to find correlations between the trajectory descriptive data allowing to understand the behavior of the variables when the fertility index is higher, a criterion that allows to give clues about the behavior of the sperm given the conditions of normality. The software showed a good performance in relation to the data obtained by the expert. However, for a more efficient validation it is necessary to take a larger number of samples for study in specialized centers. The main objective, which is the implementation of a low-cost and user-friendly support tool is achieved.

Acknowledgements

This research work is supported by the Mechatronic Engineering research Group of the Mariana University. Also, the authors are very grateful for the valuable support given by SDAS Research Group (www.sdas-group.com).

References

References

Amann, R. P., & Waberski, D. (2014). Computer-assisted sperm analysis (CASA): Capabilities and potential developments. Theriogenology, 81(1), 5-17.e3. https:// doi.org/10.1016/j.theriogenology.2013.09.004

Arasteh, A., Vosoughi Vahdat, B., & Salman Yazdi, R. (2018). Multi-Target Tracking of Human Spermatozoa in Phase-Contrast Microscopy Image Sequences using a Hybrid Dynamic Bayesian Network. Scientific Reports, 8(1), 5068. https://doi. org/10.1038/s41598-018-23435-x

Asociación Española de Urología, W., & SPARC (Organization). (2010). Actas Urológicas Españolas (Vol. 34). Retrieved from: http://scielo.isciii.es/scielo. php?script=sci_arttext&pid=S0210-48062010000700001

Avendaño, C., Mata, A., Sanchez Sarmiento, C. A., & Doncel, G. F. (2012). Use of laptop computers connected to internet through Wi-Fi decreases human sperm motility and increases sperm DNA fragmentation. Fertility and Sterility, 97(1), 39-45^2. https://doi.org/10.1016/jfertnstert.2011.10.012

Beresford-Smith, B., & Van Helden, D. F. (1994). Applications of radar tracking algorithms to motion analysis in biomedical images. In Proceedings of 1st International Conference on Image Processing (Vol. 1, pp. 411-415). IEEE Comput. Soc. Press. https://doi.org/10.1109/ICIP.1994.413346

Blomberg Jensen, M., Bjerrum, P. J., Jessen, T. E., Nielsen, J. E., Joensen, U. N., Olesen, I. A., Jorgensen, N. (2011). Vitamin D is positively associated with sperm motility and increases intracellular calcium in human spermatozoa. Human Reproduction, 26(6), 1307-1317.

Bungum, M., Bungum, L., & Giwercman, A. (2011). Sperm chromatin structure assay (SCSA): a tool in diagnosis and treatment of infertility. Asian Journal of Andrology, 13(1), 69-75. https://doi.org/10.1038/aja.2010.73

Esteves, S. C. (2014). Clinical relevance of routine semen analysis and controversies surrounding the 2010 World Health Organization criteria for semen examination. International braz j urol, 40(4), 433-453. https://doi.org/10.1590/S1677-5538. IBJU.2014.04.02

García-Fernández, Á. F., Svensson, L., & Morelande, M. R. (2016). Multiple target tracking based on sets of trajectories. Retrieved from http://arxiv.org/abs/1605.08163

Giaretta, E., Munerato, M., Yeste, M., Galeati, G., Spinaci, M., Tamanini, C., ... Bucci, D. (2017). Implementing an open-access CASA software for the assessment of stallion sperm motility: Relationship with other sperm quality parameters. Animal Reproduction Science, 176, 11-19. https://doi.org/io.ioi6/j. anireprosci.2016.11.003

Guerrero, E. R., Mancera, F. C., González, N. G., Maya, W. C., & Guzmán, L. M. (2013). Evaluación asistida por computador de la viabilidad espermática en humanos. Revista Ingeniería Biomédica, 6(12), 17-28.

Imani, Y., Teyfouri, N., Ahmadzadeh, M. R., & Golabbakhsh, M. (2014). A new method for multiple sperm cells tracking. Journal of Medical Signals and Sensors, 4(1), 35-42. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/24696807

Kobori, Y., Pfanner, P., Prins, G. S., & Niederberger, C. (2016). Novel device for male infertility screening with single-ball lens microscope and smartphone. Fertility and Sterility, 106(3), 574-578. https://doi.org/10.1016/j.fertnstert.2016.05.027

Lu, J.-C., Huang, Y.-F., & Lü, N.-Q. (2010). [WHO Laboratory Manual for the Examination and Processing of Human Semen]. Zhonghua Nan Ke Xue = National Journal of Andrology, 16(10), 867-871. Retrieved from http://www.ncbi.nlm.nih. gov/pubmed/21243747

Mortimer, S. T., van der Horst, G., & Mortimer, D. (2015). The future of computeraided sperm analysis. Asian Journal of Andrology, 17(4), 545-553. https://doi. org/10.4103/1008-682X.154312

Pant, N., Pant, A., Shukla, M., Mathur, N., Gupta, Y., & Saxena, D. (2011). Environmental and experimental exposure of phthalate esters: The toxicological consequence on human sperm. Human & Experimental Toxicology, 30(6), 507-514. https://doi. org/10.1177/0960327110374205

Ravanfar, M., Azinfar, L., Moradi, M. H., Fazel-Rezai, R., Ravanfar, M., Azinfar, L., Fazel-Rezai, R. (2014). Occlusion Robust Low-Contrast Sperm Tracking Using Switchable Weight Particle Filtering. Advances in Sexual Medicine, 04(03), 42-54. https://doi.org/10.4236/asm.2014.43008

Rivera-Acosta, M., Ortega-Cisneros, S., Gongora, M. C., Biswas, R., Rios, Y. Y., Sanchez, E. N., & Garcia, F. J. (2017). Identification of the morphological defects present in the pattern of spermatozoa using a reconfigurable device. In 2017 IEEE International Autumn Meeting on Power, Electronics and Computing (ROPEC) (pp. 1-5). IEEE. https://doi.org/10.1109/ROPEC.2017.8261599

Sarabia, L., & Munuce, M. J. (2011). Nuevos valores para el espermiograma OMS 2010. Revista Médica de Chile, 139(4), 548-549. https://doi.org/10.4067/S003498872011000400020

Shaker, F., Monadjemi, S. A., & Naghsh-Nilchi, A. R. (2016). Automatic detection and segmentation of sperm head, acrosome and nucleus in microscopic images of human semen smears. Computer Methods and Programs in Biomedicine, 132, 1120. https://doi.org/10.1016/j.cmpb.2016.04.026

Sharma, R., Harlev, A., Agarwal, A., & Esteves, S. C. (2016). Cigarette Smoking and Semen Quality: A New Meta-analysis Examining the Effect of the 2010 World Health Organization Laboratory Methods for the Examination of Human Semen. European Urology, 70(4), 635-645. https://doi.org/10.1016/j.eururo.2016.04.010

Sikka, S. C., & Hellstrom, W. J. (2016). Current updates on laboratory techniques for the diagnosis of male reproductive failure. Asian Journal of Andrology, 18(3), 392. https://doi.org/10.4103/1008-682X.179161

Urbano, L. F. (2014). Robust Automatic Multi-Sperm Tracking in Time-Lapse Images. Drexel University, (May).

Urbano, L. F., Masson, P., VerMilyea, M., & Kam, M. (2017). Automatic Tracking and Motility Analysis of Human Sperm in Time-Lapse Images. IEEE Transactions on Medical Imaging, 36(3), 792-801. https://doi.org/10.1109/TMI.2016.2630720

Verón, G. L., Tissera, A. D., Bello, R., Beltramone, F., Estofan, G., Molina, R. I., & Vazquez-Levin, M. H. (2018). Impact of age, clinical conditions, and lifestyle on routine semen parameters and sperm kinematics. Fertility and Sterility, 110(1), 6875.e4. https://doi.org/10.1016/j.fertnstert.2018.03.016

Wang, J., Fan, H. C., Behr, B., & Quake, S. R. (2012). Genome-wide Single-Cell Analysis of Recombination Activity and De Novo Mutation Rates in Human Sperm. Cell, 150(2), 402-412. https://doi.org/10.1016/j.cell.2012.06.030