Skip to main content
SpringerLink
Log in

Video Analytics in Urban Environments: Challenges and Approaches

  • Chapter
  • First Online:
ICT Applications for Smart Cities

Part of the book series: Intelligent Systems Reference Library ((ISRL,volume 224))

  • 212 Accesses

Abstract

This chapter reviews state-of-the-art approaches generally present in the pipeline of video analytics on urban scenarios. A typical pipeline is used to cluster approaches in the literature, including image preprocessing, object detection, object classification, and object tracking modules. Then, a review of recent approaches for each module is given. Additionally, applications and datasets generally used for training and evaluating the performance of these approaches are included. This chapter does not pretend to be an exhaustive review of state-of-the-art video analytics in urban environments but rather an illustration of some of the different recent contributions. The chapter concludes by presenting current trends in video analytics in the urban scenario field.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.statista.com/statistics/864838/video-surveillance-market-size-worldwide/.

  2. 2.

    http://www.cvlibs.net/datasets/kitti/eval_tracking.php.

  3. 3.

    https://cocodataset.org/#home.

  4. 4.

    http://host.robots.ox.ac.uk/pascal/VOC/voc2007/.

  5. 5.

    https://www.image-net.org/.

  6. 6.

    https://detrac-db.rit.albany.edu/.

  7. 7.

    https://vehiclereid.github.io/VeRi/.

  8. 8.

    http://www.nlpr.ia.ac.cn/iva/homepage/jqwang/Vehicle1M.htm.

  9. 9.

    http://www.elec.qmul.ac.uk/staffinfo/andrea/avss2007.html.

  10. 10.

    http://www.changedetection.net/.

  11. 11.

    http://www.panda-dataset.com/.

References

  1. Ahmed, I., Ahmad, M., Rodrigues, J.J., Jeon, G., Din, S.: A deep learning-based social distance monitoring framework for covid-19. Sustain. Cities Soc. 65, 1–12 (2021)

    Google Scholar 

  2. Md Arafat, M.Y., Khairuddin, A.S.M., Paramesran, R.: Connected component analysis integrated edge based technique for automatic vehicular license plate recognition framework. Intell. Transp. Syst. 14(7), 712–723 (2020)

    Google Scholar 

  3. Arafat, M.Y., Khairuddin, A.S.M., Paramesran, R.: Detection and classification of vehicles for traffic video analytics. Procedia Comput. Sci. 144, 259–268 (2018)

    Google Scholar 

  4. Aslani, S., Mahdavi-Nasab, H.: Optical flow based moving object detection and tracking for traffic surveillance. Int. J. Electr., Comput., Energ., Electron. Commun. Eng. 7(9), 1252–1256 (2013)

    Google Scholar 

  5. Tariq, S., Farooq, H., Jaleel, A., Wasif, S.M.: Anomaly detection with particle filtering for online video surveillance. IEEE Access 9, 19457–19468 (2021)

    Google Scholar 

  6. Avola, D., Foresti, G.L., Martinel, N., Micheloni, C., Pannone, D., Piciarelli, C.: Aerial video surveillance system for small-scale uav environment monitoring. In: 14th International Conference on Advanced Video and Signal Based Surveillance, pp. 1–6. IEEE (2017)

    Google Scholar 

  7. Bewley, A., Ge, Z., Ott, L., Ramos, F., Upcroft, B.: Simple online and realtime tracking. In: International Conference on Image Processing, pp. 3464–3468. IEEE (2016)

    Google Scholar 

  8. Blom, H.A.P., Bar-Shalom, Y.: The interacting multiple model algorithm for systems with markovian switching coefficients. Trans. Autom. Control 33(8), 780–783 (1988)

    Article  Google Scholar 

  9. Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: YOLOv4: optimal speed and accuracy of object detection (2020)

    Google Scholar 

  10. Bose, B., Grimson, E.: Improving object classification in far-field video. In: Proceedings of Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 1–8. IEEE (2004)

    Google Scholar 

  11. Bouwmans, T., Javed, S., Sultana, M., Jung, S.K.: Deep neural network concepts for background subtraction: a systematic review and comparative evaluation. Neural Netw. 117, 8–66 (2019)

    Google Scholar 

  12. Buch, N., Cracknell, M., Orwell, J., Velastin, S.A.: Vehicle localisation and classification in urban CCTV streams. In: ITS World Congress, pp. 1–8 (2009)

    Google Scholar 

  13. Canel, C., Kim, T., Zhou, G., Li, C., Lim, H., Andersen, D.G., Kaminsky, M., Dulloor, S.: Scaling video analytics on constrained edge nodes (2019). arXiv:1905.13536

  14. Cao, Mingwei, Zheng, Liping, Jia, Wei, Liu, Xiaoping: Joint 3D reconstruction and object tracking for traffic video analysis under Jov environment. Trans. Intell. Transp. Syst. 22(6), 3577–3591 (2020)

    Article  Google Scholar 

  15. Cao, X., Changxia, W., Lan, J., Yan, P., Li, X.: Vehicle detection and motion analysis in low-altitude airborne video under urban environment. Trans. Circuits Syst. Video Technol. 21(10), 1522–1533 (2011)

    Article  Google Scholar 

  16. Caprile, B., Torre, V.: Using vanishing points for camera calibration. Int. J. Comput. Vision 4(2), 127–139 (1990)

    Article  Google Scholar 

  17. Chandrakar, R., Raja, R., Miri, R., Sinha, U., Kushwaha, A.K.S., Raja, H.: Enhanced the moving object detection and object tracking for traffic surveillance using RBF-FDLNN and CBF algorithm. Expert Syst. Appl. 191, 1–15 (2022)

    Article  Google Scholar 

  18. Chen, K., Wang, Z., Wang, X., Gong, D., Yu, L., Guo, Y., Ding, G.: Towards real-time object detection in gigapixel-level video. Neurocomputing (2021)

    Google Scholar 

  19. Chen, Z., Ellis, T., Velastin, S.A.: Vehicle detection, tracking and classification in urban traffic. In: 15th International Conference on Intelligent Transportation Systems, pp. 951–956. IEEE (2012)

    Google Scholar 

  20. Cho, H., Seo, Y.W., Kumar, B.V., Rajkumar, R.R.: A multi-sensor fusion system for moving object detection and tracking in urban driving environments. In: International Conference on Robotics and Automation, pp. 1836–1843. IEEE (2014)

    Google Scholar 

  21. Cucchiara, R., Grana, C., Piccardi, M., Prati, A., Sirotti, S.: Improving shadow suppression in moving object detection with HSV color information. In: Intelligent Transportation Systems, pp. 334–339. IEEE (2001)

    Google Scholar 

  22. Pino, I.D., Vaquero, V., Masini, B., Sola, J., Moreno-Noguer, F., Sanfeliu, A., Andrade-Cetto, J.: Low resolution lidar-based multi-object tracking for driving applications. In: Iberian Robotics Conference, pp. 287–298. Springer (2017)

    Google Scholar 

  23. Deutscher, J., Isard, M., MacCormick, J.: Automatic camera calibration from a single manhattan image. In: European Conference on Computer Vision, pp. 175–188. Springer (2002)

    Google Scholar 

  24. Dey, B., Kundu, M.K.: Turning video into traffic data-an application to urban intersection analysis using transfer learning. IET Image Process. 13(4), 673–679 (2019)

    Google Scholar 

  25. Dyckmanns, H., Matthaei, R., Maurer, M., Lichte, B., Effertz, J., Stüker, D.: Object tracking in urban intersections based on active use of a priori knowledge: active interacting multi model filter. In: Intelligent Vehicles Symposium, pp. 625–630. IEEE (2011)

    Google Scholar 

  26. Silva, R.R., Aires, K.R., Veras, R.D.: Detection of helmets on motorcyclists. Multimed. Tools Appl. 77(5), 5659–5683 (2018)

    Google Scholar 

  27. Fan, Q., Pankanti, S.: Modeling of temporarily static objects for robust abandoned object detection in urban surveillance. In: 8th International Conference on Advanced Video and Signal Based Surveillance, pp. 36–41. IEEE (2011)

    Google Scholar 

  28. Frome, A., Cheung, G., Abdulkader, A., Zennaro, M., Wu, B., Bissacco, A., Adam, H., Neven, H., Vincent, L.: Large-scale privacy protection in google street view. In: 12th International Conference on Computer Vision, pp. 2373–2380. IEEE (2009)

    Google Scholar 

  29. Gaddigoudar, P.K., Balihalli, T.R., Ijantkar, S.S., Iyer, N.C., Maralappanavar, S.: Pedestrian detection and tracking using particle filtering. In: International Conference on Computing, Communication and Automation, pp. 110–115 (2017)

    Google Scholar 

  30. Gao, C., Li, P., Zhang, Y., Liu, J., Wang, L.: People counting based on head detection combining Adaboost and CNN in crowded surveillance environment. Neurocomputing 208, 108–116 (2016)

    Article  Google Scholar 

  31. Gautam, K.S., Thangavel, S.K.: Video analytics-based intelligent surveillance system for smart buildings. Soft. Comput. 23(8), 2813–2837 (2019)

    Article  Google Scholar 

  32. Gavrilescu, R., Zet, C., Foşalău, C., Skoczylas, M., Cotovanu, D.: Faster R-CNN: an approach to real-time object detection. In: International Conference and Exposition on Electrical and Power Engineering, pp. 165–168 (2018)

    Google Scholar 

  33. Grassi, G., Jamieson, K., Bahl, P., Pau, G.: Parkmaster: an in-vehicle, edge-based video analytics service for detecting open parking spaces in urban environments. In: Proceedings of Symposium on Edge Computing, pp. 1–14 (2017)

    Google Scholar 

  34. Graszka, P.: Median mixture model for background–foreground segmentation in video sequences 103–110 (2014)

    Google Scholar 

  35. Grents, A., Varkentin, V., Goryaev, N.: Determining vehicle speed based on video using convolutional neural network. Transp. Res. Procedia 50, 192–200 (2020)

    Article  Google Scholar 

  36. Guo, H., Zhao, C., Liu, Z., Wang, J., Hanqing, L.: Learning coarse-to-fine structured feature embedding for vehicle re-identification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, pp. 1–8 (2018)

    Google Scholar 

  37. Gupte, S., Masoud, O., Martin, R.F., Papanikolopoulos, N.P.: Detection and classification of vehicles. Trans. Intell. Transp. Syst. 3(1), 37–47 (2002)

    Google Scholar 

  38. Hamida, A.B., Koubaa, M., Amar, C.B., Nicolas, H.: Toward scalable application-oriented video surveillance systems. In: Science and Information Conference, pp. 384–388. IEEE (2014)

    Google Scholar 

  39. Jodoin, J.P., Bilodeau, G.A., Saunier, N.: Urban tracker: multiple object tracking in urban mixed traffic. In: Winter Conference on Applications of Computer Vision, pp. 885–892. IEEE (2014)

    Google Scholar 

  40. Junos, M.H., Khairuddin, A.S.M., Dahari, M.: Automated object detection on aerial images for limited capacity embedded device using a lightweight CNN model. Alex. Eng. J. (2021)

    Google Scholar 

  41. Kalman, R.E.: A new approach to linear filtering and prediction problems. J. Basic Eng. 82(1), 35–45 (1960)

    Google Scholar 

  42. Kuhn, H.W.: The hungarian method for the assignment problem. Nav. Res. Logist. Q. 2(1–2), 83–97 (1955)

    Google Scholar 

  43. Kumar, T.S.: Video based traffic forecasting using convolution neural network model and transfer learning techniques. J. Innov. Image Process. 2(03), 128–134 (2020)

    Google Scholar 

  44. Lee, B., Hedley, M.: Background estimation for video surveillance. Image Vis. Comput. N. Z. 315–320 (2002)

    Google Scholar 

  45. Li, C., Dobler, G., Feng, X., Wang, Y.: Tracknet: simultaneous object detection and tracking and its application in traffic video analysis, pp. 1–10 (2019). arXiv:1902.01466

  46. Li, Y., Padmanabhan, A., Zhao, P., Wang, Y., Xu, G.H., Netravali, R.: Reducto: on-camera filtering for resource-efficient real-time video analytics. In: Proceedings of the Annual Conference of the ACM Special Interest Group on Data Communication on the Applications, Technologies, Architectures, and Protocols for Computer Communication, pp. 359–376 (2020)

    Google Scholar 

  47. Lim, K., Jang, W.D., Kim, C.S.: Background subtraction using encoder-decoder structured convolutional neural network. In: 14th International Conference on Advanced Video and Signal Based Surveillance, pp. 1–6. IEEE (2017)

    Google Scholar 

  48. Ling, X., Sheng, J., Baiocchi, O., Liu, X., Tolentino, M.E.: Identifying parking spaces & detecting occupancy using vision-based IoT devices. In: Global Internet of Things Summit, pp. 1–6. IEEE (2017)

    Google Scholar 

  49. Liu, C., Huynh, D.Q., Sun, Y., Reynolds, M., Atkinson, S.: A vision-based pipeline for vehicle counting, speed estimation, and classification. Trans. Intell. Transp. Syst. (2020)

    Google Scholar 

  50. Liu, X., Liu, W., Mei, T., Ma, H.: A deep learning-based approach to progressive vehicle re-identification for urban surveillance. In: European Conference on Computer Vision, pp. 869–884. Springer (2016)

    Google Scholar 

  51. Liu, X., Sang, J., Weiqun, W., Liu, K., Liu, Q., Xia, X.: Density-aware and background-aware network for crowd counting via multi-task learning. Pattern Recogn. Lett. 150, 221–227 (2021)

    Article  Google Scholar 

  52. Makhmutova, A., Anikin, I.V., Dagaeva, M.: Object tracking method for videomonitoring in intelligent transport systems. In: International Russian Automation Conference, pp. 535–540. IEEE (2020)

    Google Scholar 

  53. Naik, U.P., Rajesh, V., Kumar, R., et al.: Implementation of YOLOv4 algorithm for multiple object detection in image and video dataset using deep learning and artificial intelligence for urban traffic video surveillance application. In: International Conference on Electrical, Computer and Communication Technologies, pp. 1–6. IEEE (2021)

    Google Scholar 

  54. Nguyen, T.-N., Michaelis, B., Al-Hamadi, A., Tornow, M., Meinecke, M.-M.: Stereo-camera-based urban environment perception using occupancy grid and object tracking. Trans. Intell. Transp. Syst. 13(1), 154–165 (2011)

    Article  Google Scholar 

  55. Noh, B., No, W., Lee, J., Lee, D.: Vision-based potential pedestrian risk analysis on unsignalized crosswalk using data mining techniques. Appl. Sci. 10(3), 1–21 (2020)

    Article  Google Scholar 

  56. Praveenkumar, S.M., Patil, P., Hiremath, P.S.: Real-time multi-object tracking of pedestrians in a video using convolution neural network and Deep SORT. In: ICT Systems and Sustainability, pp. 725–736. Springer (2022)

    Google Scholar 

  57. Qu, H., Yuan, T., Sheng, Z., Zhang, Y.: A pedestrian detection method based on YOLOv3 model and image enhanced by retinex. In: 11th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics, pp. 1–5. IEEE (2018)

    Google Scholar 

  58. Ridel, D., Rehder, E., Lauer, M., Stiller, C., Wolf, D.: A literature review on the prediction of pedestrian behavior in urban scenarios. In: 21st International Conference on Intelligent Transportation Systems, pp. 3105–3112. IEEE (2018)

    Google Scholar 

  59. Santos, A.M., Bastos-Filho, C.J., Maciel, A.M., Lima, E.. Counting vehicle with high-precision in Brazilian roads using YOLOv3 and Deep SORT. In: 33rd Conference on Graphics, Patterns and Images, pp. 69–76. IEEE (2020)

    Google Scholar 

  60. Yuguang Shi, Yu., Guo, Z.M., Li, X.: Stereo CenterNet-based 3D object detection for autonomous driving. Neurocomputing 471, 219–229 (2022)

    Article  Google Scholar 

  61. Shi, Z., Guo, B., Zhao, M., Zhang, C., et al.: Nighttime low illumination image enhancement with single image using bright/dark channel prior. J. Image Video Process. 2018(1), 1–15 (2018)

    Article  Google Scholar 

  62. Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: Proceedings of Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 246–252. IEEE (1999)

    Google Scholar 

  63. Tu, N.A., Wong, K.S., Demirci, M.F., Lee, Y.K., et al.: Toward efficient and intelligent video analytics with visual privacy protection for large-scale surveillance. J. Supercomput. 1–31 (2021)

    Google Scholar 

  64. Velesaca, H.O., Araujo, S., Suárez, P.L., Sánchez, A., Sappa, A.D.: Off-the-shelf based system for urban environment video analytics. In: International Conference on Systems, Signals and Image Processing, pp. 459–464 (2020)

    Google Scholar 

  65. Vishnu, C., Singh, D., Mohan, C.K., Babu, S.: Detection of motorcyclists without helmet in videos using convolutional neural network. In: International Joint Conference on Neural Networks, pp. 3036–3041. IEEE (2017)

    Google Scholar 

  66. Wang, C., Cheng, M., Sohel, F., Bennamoun, M., Li, J.: NormalNet: a voxel-based CNN for 3D object classification and retrieval. Neurocomputing 323, 139–147 (2019)

    Article  Google Scholar 

  67. Wei, H., Laszewski, M., Kehtarnavaz, N.: Deep learning-based person detection and classification for far field video surveillance. In: 13th Dallas Circuits and Systems Conference, pp. 1–4. IEEE (2018)

    Google Scholar 

  68. Wildenauer, H., Micusik, B.: Closed form solution for radial distortion estimation from a single vanishing point. In: BMVC, vol. 1, pp. 1–11 (2013)

    Google Scholar 

  69. Wojke, N., Bewley, A., Paulus, D.: Simple online and realtime tracking with a deep association metric. In: International Conference on Image Processing, pp. 3645–3649. IEEE (2017)

    Google Scholar 

  70. Xu, R., Nikouei, S.Y., Chen, Y., Polunchenko, A., Song, S., Deng, C., Faughnan, T.R.: Real-time human objects tracking for smart surveillance at the edge. In: International Conference on Communications, pp. 1–6. IEEE (2018)

    Google Scholar 

  71. Zhang, H., Wang, K., Tian, Y., Gou, C., Wang, F.-Y.: MFR-CNN: incorporating multi-scale features and global information for traffic object detection. Trans. Veh. Technol. 67(9), 8019–8030 (2018)

    Article  Google Scholar 

  72. Zhang, M., Yao, J., Xia, M., Li, K., Zhang, Y., Liu, Y.: Line-based multi-label energy optimization for fisheye image rectification and calibration. In: Proceedings of Conference on Computer Vision and Pattern Recognition, pp. 4137–4145. IEEE Computer Society (2015)

    Google Scholar 

  73. Zhu, J., Sun, K., Jia, S., Li, Q., Hou, X., Lin, W., Liu, B., Qiu, G.: Urban traffic density estimation based on ultrahigh-resolution UAV video and deep neural network. J. Sel. Top. Appl. Earth Obs. Remote. Sens. 11(12), 4968–4981 (2018)

    Article  Google Scholar 

  74. Zotin, A.: Fast algorithm of image enhancement based on multi-scale retinex. Procedia Comput. Sci. 131, 6–14 (2018)

    Article  Google Scholar 

  75. Zou, Y., Zhang, Y., Yan, J., Jiang, X., Huang, T., Fan, H., Cui, Z.: Zhongwei: license plate detection and recognition based on YOLOv3 and ILPRNET, pp. 1–8. Signal, Image and Video Processing (2021)

    Google Scholar 

Download references

Acknowledgements

This work has been partially supported by the ESPOL projects TICs4CI (FIEC-16-2018) and PhysicalDistancing (CIDIS-56-2020); and the “CERCA Programme/Generalitat de Catalunya”. The authors acknowledge the support of CYTED Network: “Ibero-American Thematic Network on ICT Applications for Smart Cities” (REF-518RT0559).

Author information

Authors and Affiliations

  1. Escuela Superior Politécnica del Litoral, ESPOL, Campus Gustavo Galindo Km., 30.5 Vía Perimetral, P.O. Box 09-01-5863, Guayaquil, Ecuador

    Henry O. Velesaca, Patricia L. Suárez, Dario Carpio, Rafael E. Rivadeneira & Angel D. Sappa

  2. E.T.S. Ingeniería Informática, Universidad Rey Juan Carlos, 28933, Móstoles, Madrid, Spain

    Ángel Sánchez

  3. Computer Vision Center, Campus UAB, Bellaterra, 08193, Barcelona, Spain

    Angel D. Sappa

Authors
  1. Henry O. Velesaca
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Patricia L. Suárez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dario Carpio
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rafael E. Rivadeneira
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ángel Sánchez
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Angel D. Sappa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Henry O. Velesaca .

Editor information

Editors and Affiliations

  1. Campus Gustavo Galindo, Escuela Superior Politecnica del Litoral, Guayaquil, Ecuador

    Angel D. Sappa

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Velesaca, H.O., Suárez, P.L., Carpio, D., Rivadeneira, R.E., Sánchez, Á., Sappa, A.D. (2022). Video Analytics in Urban Environments: Challenges and Approaches. In: Sappa, A.D. (eds) ICT Applications for Smart Cities. Intelligent Systems Reference Library, vol 224. Springer, Cham. https://doi.org/10.1007/978-3-031-06307-7_6

Download citation

Publish with us

Policies and ethics

Search

Navigation