Abstract
Video multi-object tracking is one of the important research topics in the field of computer vision, which is widely used in military and civil areas. At present, the research of single object tracking algorithm is quite mature, however the research of multi-object tracking is still ongoing. This paper focuses on four important stages in the multi-object tracking process: feature extraction, detector, data association and the tracker. The feature extraction part introduces the current methods of feature extraction, as well as the merits and demerits of each method; In the stage of detection, the tracking effect of the object appearance model in specific applications is described, and then the paper analyze the multi-object tracking algorithm based on detection and tracking as well as the multi-object tracking algorithm based on deep learning; In the tracking stage, the establishment of object motion model and multi-object tracking with different tracker hybrid algorithm are introduced; During the stage of data association, the paper introduce the multi-object tracking based on energy minimization and commonly used data association algorithm, respectively. Then the current mainstream datasets and evaluation methods are introduced. Finally, the future development of the multi-object tracking is discussed and forecasted.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Wu, Y., Lim, J., Yang, M.H.: Online object tracking: a benchmark. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2411–2418. IEEE (2013)
Huang, K.Q., Chen, X.T., Kang, Y.F., Tan, T.N.: Intelligent visual surveillance: a review. Chin. J. Comput. 38, 1 (2015)
Zeng, Q., Wen, G., Li, D.: Multi-target tracking by detection. In: 2016 International Conference on Audio, Language and Image Processing (ICALIP), pp. 370–374. IEEE (2016)
Nam, H., Han, B.: Learning multi-domain convolutional neural networks for visual tracking. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4293–4302. IEEE (2016)
Huang, K.Q., Ren, W.Q., Tan, T.N.: A review on image object classification and detection. Chin. J. Comput. 37(6), 1225–1240 (2014)
Viola, P., Jones, M. J., Snow, D.: Detecting pedestrians using patterns of motion and appearance. In: Null, p. 734. IEEE (2003)
Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: Exploiting the circulant structure of tracking-by-detection with kernels. In: European Conference on Computer Vision, pp. 702–715. Springer, Berlin (2012)
He, S., Yang, Q., Lau, R.W., Wang, J., Yang, M.H.: Visual tracking via locality sensitive histograms. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2427–2434. IEEE (2013)
Zhang, K., Zhang, L., Yang, M.H.: Real-time compressive tracking. In: European Conference on Computer Vision, pp. 864–877. Springer, Berlin (2012)
Chen, J., Liu, Y., Li, N., Guo, Z.: Compressive tracking based on random channel haar-like feature. In: 2015 Sixth International Conference on Intelligent Control and Information Processing (ICICIP), pp. 151–154. IEEE (2015)
Possegger, H., Mauthner, T., Bischof, H.: In defense of color-based model-free tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2113–2120 (2015)
Danelljan, M., Shahbaz Khan, F., Felsberg, M., Van de Weijer, J.: Adaptive color attributes for real-time visual tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, Ohio, USA, 24–27 June 2014, pp. 1090–1097. IEEE Computer Society (2014)
Wang, Y., Liu, J., Wang, J., Li, Y., Lu, H.: Color names learning using convolutional neural networks. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 217–221. IEEE (2015)
Wang, Y., Wang, Q., Chen, D.: Research on target recognition based on edge features. In: 2012 5th International Congress on Image and Signal Processing (CISP), pp. 1312–1315. IEEE (2012)
Wu, J.Z., Huang, Y.D.: Image denoising algorithm based on edge feature extraction in curvelet domain. In: 2012 International Conference on Wavelet Analysis and Pattern Recognition (ICWAPR), pp. 7–10. IEEE (2012)
Ojala, T., Pietikainen, M., Harwood, D.: Performance evaluation of texture measures with classification based on Kullback discrimination of distributions. In: Proceedings of the 12th IAPR International Conference on Pattern Recognition, 1994. Vol. 1-Conference A: Computer Vision and Image Processing, vol. 1, pp. 582–585. IEEE (1994)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005, vol. 1, pp. 886–893. IEEE (2005)
Hayakawa, Y., Oonuma, T., Kobayashi, H., Takahashi, A., Chiba, S., Fujiki, N.M.: Feature extraction of video using deep neural network. In: 2016 IEEE 15th International Conference on Cognitive Informatics and Cognitive Computing (ICCI* CC), pp. 465–470. IEEE (2016)
Zhang, H.L., Hu, S., Yang, G.: Video object tracking based on appearance models learning. J. Comput. Res. Dev. 52(1), 177–190 (2015)
Supancic, J.S., Ramanan, D.: Self-paced learning for long-term tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2379–2386 (2013)
Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587 (2014)
Girshick, R.: Fast R-CNN. In: IEEE International Conference on Computer Vision, pp. 1440–1448. IEEE Computer Society (2015)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017)
Yuan, G.W., Chen, Z.Q., Gong, J., Xu, D., Liao, R.J., He, J.Y.: A moving object detection algorithm based on a combination of optical flow and three-frame difference. J. Chin. Comput. Syst. 34(3), 668–671 (2013)
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., van der Smagt, P., Cremer, D., Brox, T.: Flownet: learning optical flow with convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2758–2766 (2015)
Bing-Bing, W., Zhi-Xin, C., Jia, W., Liquan, Z.: Pedestrian detection based on the combination of HOG and background subtraction method. In: 2011 International Conference on Transportation, Mechanical, and Electrical Engineering (TMEE), pp. 527–531. IEEE (2011)
Suresh, S., Deepak, P., Chitra, K.: An efficient low cost background subtraction method to extract foreground object during human tracking. In: 2014 International Conference on Circuit, Power and Computing Technologies (ICCPCT), pp. 1432–1436. IEEE (2014)
Han, X., Gao, Y., Lu, Z., Zhang, Z., Niu, D.: Research on moving object detection algorithm based on improved three frame difference method and optical flow. In: 2015 Fifth International Conference on Instrumentation and Measurement, Computer, Communication and Control (IMCCC), pp. 580–584. IEEE (2015)
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., Berg, A.C.: SSD: single shot multibox detector. In: European conference on computer vision, pp. 21–37. Springer, Cham (2016)
Fard, M.K., Yazdi, M., MasnadiShirazi, M.: A block matching based method for moving object detection in active camera. In: 2013 5th Conference on Information and Knowledge Technology (IKT), pp. 443–446. IEEE (2013)
Li, W., Yao, J., Dong, T., Li, H.: Object tracking based on fragment template and multi-feature adaptive fusion. In: 2015 8th International Symposium on Computational Intelligence and Design (ISCID), vol. 2, pp. 481–484. IEEE (2015)
Zhang, J., Presti, L.L., Sclaroff, S.: Online multi-person tracking by tracker hierarchy. In: 2012 IEEE 9th International Conference on Advanced Video and Signal-Based Surveillance (AVSS), pp. 379–385. IEEE (2012)
Guan, Y., Chen, X., Yang, D., Wu, Y.: Multi-person tracking-by-detection with local particle filtering and global occlusion handling. In: 2014 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2014)
Gan, M., Cheng, Y., Wang, Y., Chen, J.: Hierarchical particle filter tracking algorithm based on multi-feature fusion. J. Syst. Eng. Electron. 27(1), 51–62 (2016)
Wang, N., Yeung, D.Y.: Learning a deep compact image representation for visual tracking. In: Advances in Neural Information Processing Systems, pp. 809–817 (2013)
Jin, Z., Liu, C.: Accelerated TLD algorithm and its application in multiple target tracking. Comput. Syst. Appl. 25(6), 196–201 (2016)
Sharma, S., Khachane, A., Motwani, D.: Real time multi-object tracking using TLD framework. In: International Conference on Inventive Computation Technologies (ICICT), vol. 2, pp. 1–6. IEEE (2016)
Lu, Y., Wu, T., Chun Zhu, S.: Online object tracking, learning and parsing with and-or graphs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3462–3469 (2014)
Leal-Taixé, L., Canton-Ferrer, C., Schindler, K.: Learning by tracking: Siamese CNN for robust target association. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 33–40 (2016)
Yu, F., Li, W., Li, Q., Liu, Y., Shi, X., Yan, J.: POI: multiple object tracking with high performance detection and appearance feature. In: European Conference on Computer Vision, pp. 36–42. Springer, Cham (2016)
Qi, Y., Zhang, S., Qin, L., Yao, H., Huang, Q., Lim, J., Yang, M.H.: Hedged deep tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4303–4311 (2016)
Chu, Q., Ouyang, W., Li, H., Wang, X., Liu, B., Yu, N.: Online multi-object tracking using CNN-based single object tracker with spatial-temporal attention mechanism. arXiv preprint arXiv:1708.02843 (2017)
Milan, A., Rezatofighi, S.H., Dick, A.R., Reid, I.D., Schindler, K.: Online multi-target tracking using recurrent neural networks. In: AAAI, pp. 4225–4232 (2017)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Xiang, J., Zhang, G., Hou, J., Sang, N., Huang, R.: Multiple target tracking by learning feature representation and distance metric jointly. arXiv preprint arXiv:1802.03252 (2018)
Munkres, J.: Algorithms for the assignment and transportation problems. J. Soc. Ind. Appl. Math. 5(1), 32–38 (1957)
Hare, S., Golodetz, S., Saffari, A., Vineet, V., Cheng, M.M., Hicks, S.L., Torr, P.H.: Struck: structured output tracking with kernels. IEEE Trans. Pattern Anal. Mach. Intell. 38(10), 2096–2109 (2016)
Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 583–596 (2015)
Kalman, R.E., Bucy, R.S.: New results in linear filtering and prediction theory. J. Basic Eng. 83(1), 95–108 (1961)
Julier, S.J., Uhlmann, J.K.: New extension of the Kalman filter to nonlinear systems. In: Signal processing, sensor fusion, and target recognition VI, vol. 3068, pp. 182–194. International Society for Optics and Photonics (1997)
Wan, E.A., Van Der Merwe, R.: The unscented Kalman filter for nonlinear estimation. In: Adaptive Systems for Signal Processing, Communications, and Control Symposium 2000. AS-SPCC. The IEEE 2000, pp. 153–158. IEEE (2000)
Yang, S., Li, H.: Application of EKF and UKF in target tracking problem. In: 2016 8th International Conference on Intelligent Human–Machine Systems and Cybernetics (IHMSC), vol. 1, pp. 116–120. IEEE (2016)
Gordon, N., Ristic, B., Arulampalam, S.: Beyond the Kalman Filter: Particle Filters for Tracking Applications, vol. 3, pp. 1077–2626. Artech House, London (2004)
Kokul, T., Ramanan, A., Pinidiyaarachchi, U.A.J.: Online multi-person tracking-by-detection method using ACF and particle filter. In: 2015 IEEE Seventh International Conference on Intelligent Computing and Information Systems (ICICIS), pp. 529–536. IEEE (2015)
Milan, A., Leal-Taixé, L., Schindler, K., Reid, I.: Joint tracking and segmentation of multiple targets. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5397–5406. IEEE (2015)
Yu, S.I., Meng, D., Zuo, W., Hauptmann, A.: The solution path algorithm for identity-aware multi-object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3871–3879 (2016)
Maksai, A., Wang, X., Fleuret, F., Fua, P.: Non-Markovian globally consistent multi-object tracking. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2563–2573. IEEE (2017)
Milan, A., Roth, S., Schindler, K.: Continuous energy minimization for multitarget tracking. IEEE Trans. Pattern Anal. Mach. Intell. 36(1), 58–72 (2014)
Milan, A., Schindler, K., Roth, S.: Multi-target tracking by discrete-continuous energy minimization. IEEE Trans. Pattern Anal. Mach. Intell. 38(10), 2054–2068 (2016)
Wei, L., Xingwei, L.: Multiple object tracking based on energy minimization. In: 2016 4th International Symposium on Computational and Business Intelligence (ISCBI), pp. 206–211. IEEE (2016)
Singer, R.A., Stein, J.J.: An optimal tracking filter for processing sensor data of imprecisely determined origin in surveillance systems. In: 1971 IEEE Conference on Decision and Control, vol. 10, pp. 171–175. IEEE (1971)
Schulter, S., Vernaza, P., Choi, W., Chandraker, M.: Deep network flow for multi-object tracking. IEEE Conference on Computer Vision and Pattern Recognition, pp. 2730–2739. IEEE Computer Society (2017)
Kim, C., Li, F., Ciptadi, A., Rehg, J.M.: Multiple hypothesis tracking revisited. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4696–4704 (2015)
Bar-Shalom, Y., Tse, E.: Tracking in a cluttered environment with probabilistic data association. Automatica 11(5), 451–460 (1975)
Cox, I.J.: A review of statistical data association techniques for motion correspondence. Int. J. Comput. Vision 10(1), 53–66 (1993)
Hamid Rezatofighi, S., Milan, A., Zhang, Z., Shi, Q., Dick, A., Reid, I.: Joint probabilistic data association revisited. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3047–3055 (2015)
Shi, X., Song, Y. Q., Yang, Z., Chen, J.: Multiple target tracking under occlusions using modified Joint Probabilistic Data Association. In: 2015 IEEE International Conference on Communications (ICC), pp. 6615–6620. IEEE (2015)
Chen, S.L., Xu, Y.B., Zhu, M.: K nearest neighbor joint possibility data association algorithm. In: 2010 2nd International Conference on Information Engineering and Computer Science (ICIECS), pp. 1–4. IEEE (2010)
Son, J., Baek, M., Cho, M., Han, B.: Multi-object tracking with quadruplet convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5620–5629 (2017)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009. pp. 248–255. IEEE (2009)
Everingham, M., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)
Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, D., Zitnick, C.L.: Microsoft coco: common objects in context. In: European Conference on Computer Vision, pp. 740–755. Springer, Cham (2014)
Kristan, M., Pflugfelder, R., Leonardis, A., Matas, J., Porikli, F., Cehovin, L., Nebehay, G., Fernandez, G., Vojir, T., Gatt, A., Khajenezhad, A.: The visual object tracking vot2013 challenge results. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 98–111 (2013)
Leal-Taixé, L., Milan, A., Reid, I., Roth, S., Schindler, K.: Motchallenge 2015: towards a benchmark for multi-target tracking. arXiv preprint arXiv:1504.01942 (2015)
Milan, A., Leal-Taixé, L., Reid, I., Roth, S., Schindler, K.: MOT16: a benchmark for multi-object tracking. arXiv preprint arXiv:1603.00831 (2016)
Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: the CLEAR MOT metrics. J. Image Video Process. 2008, 1 (2008)
Zhang, Z., Wu, J., Zhang, X., Zhang, C.: Multi-target, multi-camera tracking by hierarchical clustering: recent progress on DukeMTMC Project. arXiv preprint arXiv:1712.09531 (2017)
Acknowledgments
This work was supported by the Scientific Research Fund of Hunan Provincial Education Department of China (Project No. 17A007); and the Teaching Reform and Research Project of Hunan Province of China (Project No. JG1615).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhou, S., Ke, M., Qiu, J., Wang, J. (2019). A Survey of Multi-object Video Tracking Algorithms. In: Abawajy, J., Choo, KK., Islam, R., Xu, Z., Atiquzzaman, M. (eds) International Conference on Applications and Techniques in Cyber Security and Intelligence ATCI 2018. ATCI 2018. Advances in Intelligent Systems and Computing, vol 842. Springer, Cham. https://doi.org/10.1007/978-3-319-98776-7_38
Download citation
DOI: https://doi.org/10.1007/978-3-319-98776-7_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-98775-0
Online ISBN: 978-3-319-98776-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)