Abstract
This paper presents the design of an embedded automated digital video surveillance system with real-time performance. Hardware accelerators for video segmentation, morphological operations, labeling and feature extraction are required to achieve the real-time performance while tracking will be handled in software in an embedded processor. By implementing a complete embedded system, bottlenecks in computational complexity and memory requirements can be identified and addressed. Accordingly, a memory reduction scheme for the video segmentation unit, reducing bandwidth with more than 70%, and a low complexity morphology architecture that only requires memory proportional to the input image width, have been developed. On a system level, it is shown that a labeling unit based on a contour tracing technique does not require unique labels, resulting in more than 50% memory reduction. The hardware accelerators provide the tracking software with image objects properties, i.e. features, thereby decoupling the tracking algorithm from the image stream. A prototype of the embedded system is running in real-time, 25 fps, on a field programmable gate array development board. Furthermore, the system scalability for higher image resolution is evaluated.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
AXIS Communications, “Official AXIS Communications Homepage,” 2007, http://www.axis.com.
Sony, “Official Sony Homepage,” 2007, http://bssc.sel.sony.com/.
IBM, “Official IBM homepage,” 2007, http://www.research.ibm.com/peoplevision/.
I. Haritaoglu, D. Harwood, and L. Davis, “W4: Real-time Surveillance of People and their Activities,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, 2000, pp. 809–830.
C. Stauffer and W. E. L. Grimson, “Learning Patterns of Activity Using Real-time Tracking,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, 2000, pp. 747–758.
R. Collins, A. Lipton, H. Fujiyoshi, and T. Kanade, “Algorithms for Cooperative Multisensor Surveillance,” Proc. I.E.E.E., vol. 89, no. 10, 2001, pp. 1456–1477.
T. Zhao and R. Nevatia, “Tracking Multiple Humans in Complex Situations,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 9, 2004, pp. 1208–1222.
W. Hu, T. Tan, L. Wang, and S. Maybank, “A Survey on Visual Surveillance of Object Motion and Behaviors,” IEEE Trans. Syst. Man Cybern. Part C Appl. Rev., vol. 34, no. 3, 2004, pp. 334–353.
R. Aguilar-Ponce, J. Tessier, A. Baker, C. Emmela, J. Das, J. Tecpanecatl-Xihuitl, A. Kumar, and M. Bayoumi, “VLSI Architecture for an Object Change Detector for Visual Sensors,” in Proceeding of IEEE Workshop on Signal Processing Systems Design and Implementation, SIPS, Athens, 2005, pp. 290–295.
S. Fahmy, P. Cheung, and W. Luk, “Novel FPGA-based Implementation of Median and Weighted Median Filters for Image Processing,” in Field Programmable Logic and Applications, 2005. International Conference, 2005, pp. 142–147.
P. Schumacher, K. Denolf, A. Chilira-RUs, R. Turney, N. Fedele, K. Vissers, and J. Bormans, “A Scalable, Multi-stream MPEG-4 Video Decoder for Conferencing and Surveillance Applications,” in Image Processing, 2005. ICIP 2005. IEEE International Conference, vol. II, 2006, pp. 886–889.
R. Kordasiewicz and S. Shirani, “ASIC and FPGA Implementations of H.264 DCT and Quantization Blocks,” in Image Processing, 2005. ICIP 2005. IEEE International Conference, vol. III, 2006, pp. 1020–1023.
R. Kleihorst, B. Schueler, A. Danilin, and M. Heijligers, “Smart Cameras Mote with High Performance Vision System,” in Workshop on Distributed Smart Cameras (DSC 06), Boulder, 2006.
E. Ljung, E. Simmons, and R. Kleihorst, “Distributed Vision with Multiple Uncalibrated Smart Cameras,” in Workshop on Distributed Smart Cameras (DSC 06), Boulder, 2006.
E. Ljung, E. Simmons, A. Danilin, R. Kleihorst, and B. Schueler, “802.15.4 Powered Distributed Wireless Smart Cameras Network,” in Workshop on Distributed Smart Cameras (DSC 06), Boulder, 2006.
R. Gonzalez and R. Woods, Digital Image Processing, 2nd ed. Prentice-Hall, 2002.
J. Toyama, B. Brumitt, and B. Meyers, “Wallflower: Principles and Practice of Background Maintenance,” in Proc. IEEE International Conference on Computer Vision and Pattern Recognition, 1999.
C. Stauffer and W. E. L. Grimson, “Adaptive Background Mixture Models for Real-time Tracking,” in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’99), Ft. Collins, 1999.
G. Russo and M. Russo, “A Novel Class of Sorting Networks,” IEEE Trans. Circuits Syst. 1 Fundam. Theory Appl., 1996.
H. Jiang, H. Ardö, and V. Öwall, “Real-time Video Segmentation with VGA Resolution and Memory Bandwidth Reduction,” in Proc. of AVSS, 2006.
F. Kristensen, P. Nilsson, and V. Öwall, “Background Segmentation Beyond RGB,” in Proc. of Asien Conference on Computer Vision (ACCV’06), Hyderabad, 2006.
J. Serra, Image Analysis and Mathematical Morpohology, Vol 1. Academic Press, 1982.
M. von Herk, “A Fast Algorithm for Local Minimum and Maximum Filters on Rectangular and Octagonal Kernels,” Pattern Recogn. Lett. vol. 13, no. 7, 1992, pp. 517–521.
H. Hedberg, F. Kristensen, P. Nilsson, and V. Öwall, “A Low Complexity Architecture for Binary Image Erosion and Dilation Structuring Element Decomposition,” in Proc. of IEEE International Symposium on Circuits and Systems (ISCAS’05), Kobe, 2005.
H. Park and R. Chin, “Decomposition of Arbitrarily Shaped Morphological Structuring Elements,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 17, 1995, pp. 2–15.
G. Anelli and A. Broggi, “Decomposition of Arbitrarily Shaped Binary Morphological Structuring Elements Using Genetic Algorithms,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, 1998, pp. 217–224.
S. Fejes and F. Vajda, “A Data-driven Algorithm and Systolic Architecture for Image Morphology,” in Proc. of IEEE Image Processing (ICIP ’94), Austin, 1994, pp. 550–554.
J. Velten and A. Kummert, “FPGA-based implementation of variable sized structuring elements for {2D} binary morphological operations,” in Proc. of IEEE International Symposium on Circuits and Systems (ISCAS’03), Bangkok, 2003, pp. 706–709.
A. Rosenfeld and J. Pfaltz, “Sequential Operations in Digital Picture Processing,” J. ACM, vol. 13, no. 1, 1966, pp. 471–494.
M. B. Dillencourt, H. Samet, and M. Tamminen, “A General Approach to Connected-component Labeling for Aritrary Image Representations,” J. ACM, vol. 39, no. 2, 1992, pp. 253–280.
W. Kesheng, O. Ekow, and S. Arie, “Optimizing Connected Components Labeling Algorithms,” in SPIE International Symposium on Medical Imaging, San Diego, 2005.
K. Suzuki, H. Isao, and S. Noboru, “Linear-time Connected-component Labeling Based on Sequential Local Operations,” Comput. Vis. Image Underst., vol. 89, 2003, pp. 1–23.
F. Chang, C. J. Chen, and C. J. Lu, “A Linear-time Component-labeling Algorithm Using Contour Tracing Technique,” Comput. Vis. Image Underst., vol. 93, 2004, pp. 206–220.
PETS, “The PETS 2001 Data Set, Sequence from Camera 1,” 2001, http://www.cvg.cs.rdg.ac.uk/cgi-bin/PETSMETRICS/page.cgi?dataset.
L. Yang and F. Algbregtsen, “Discrete Green’s theorem and its application in moment computation,” in Int. Conf. on Electronics and Information Technology, Beijing, 1994.
H. Hedberg, F. Kristensen, and V. Öwall, “Implementation of Labeling Algorithm Based on Contour Tracing with Feature Extraction,” in Proc. of IEEE ISCAS’07, New Orleans, 2007.
E. Brookner, Tracking and Kalman Filtering Made Easy, 1st ed. Wiley-Interscience, 1998.
Lund University, “Project homepage,” 2007, http://www.es.lth.se/Digital_Surveillance.
D. Magee, “Tracking Multiple Vehicles Using Foreground, Background and Motion Models,” Image Vis. Comput., vol. 22, 2004, pp. 143–155.
JBIG Committee, “Official JBIG homepage,” 2007, http://www.jpeg.org/jbig/index.html.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kristensen, F., Hedberg, H., Jiang, H. et al. An Embedded Real-Time Surveillance System: Implementation and Evaluation. J Sign Process Syst Sign Image Video Technol 52, 75–94 (2008). https://doi.org/10.1007/s11265-007-0100-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11265-007-0100-7