Abstract
The frame rate of conventional vision systems is restricted to the video signal formats (e.g., NTSC 30 fps and PAL 25 fps) that are designed on the basis of the characteristics of the human eye, which implies that the processing speed of these systems is limited to the recognition speed of the human eye. However, there is a strong demand for real-time high-speed vision sensors in many application fields, such as factory automation, biomedicine, and robotics, where high-speed operations are carried out. These high-speed operations can be tracked and inspected by using high-speed vision systems with intelligent sensors that work at hundreds of Hertz or more, especially when the operation is difficult to observe with the human eye. This paper reviews advances in developing real-time high speed vision systems and their applications in various fields, such as intelligent logging systems, vibration dynamics sensing, vision-based mechanical control, three-dimensional measurement/automated visual inspection, vision-based human interface, and biomedical applications.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
P. Read, M. P. Meyer. Restoration of Motion Picture Film, Oxford, UK: Butterworth-Heinemann, pp. 24–26, 2000.
E. Altug, J. P. Ostrowski, R. Mahony. Control of a quadrotor helicopter using visual feedback. In Proceedings of IEEE International Conference on Robotics and Automation, IEEE, Philadelphia, USA, pp. 72–77, 2002.
P. C. Zhang, D. Xu. Tracking and guiding multiple Laser beams for beam and target alignment. International Journal of Automation and Computing, vol. 12, no. 6, pp. 600–610, 2015.
Y. Watanabe, H. Oku, M. Ishikawa. Architectures and applications of high-speed vision. Optical Review, vol. 21, no. 6, pp. 875–882, 2014.
T. Senoo, Y. Yamakawa, Y. Watanabe, H. Oku, M. Ishikawa. High-speed vision and its application systems. Journal of Robotics and Mechatronics, vol. 26, no. 3, pp. 287–301, 2014.
T. M. Bernard, B. Y. Zavidovique, F. J. Devos. A programmable artificial retina. IEEE Journal of Solid-State Circuits, vol. 28, no. 7, pp. 789–797, 1993.
J. E. Eklund, C. Svensson, A. Astrom. VLSI implementation of a focal plane image processor-A realization of the near-sensor image processing concept. IEEE Transactions on Very Large Scale Integration Systems, vol. 4, no. 3, pp. 322–335, 1996.
Y. Nakabo, M. Ishikawa, H. Toyoda, S. Mizuno. 1ms column parallel vision system and its application of high speed target tracking. In Proceedings of IEEE International Conference on Robotics and Automation, IEEE, San Francisco, USA, pp. 650–655, 2000.
S. Kleinfelder, S. Lim, X. Liu, A. E. Gamal. A 10000 frames/s CMOS digital pixel sensor. IEEE Journal of Solid- State Circuits, vol. 36, no. 12, pp. 2049–2058, 2001.
T. Komuro, S. Kagami, M. Ishikawa. A dynamically reconfigurable SIMD processor for a vision chip. IEEE Journal of Solid-State Circuits, vol. 39, no. 1, pp. 265–268, 2004.
I. Ishii, K. Yamamoto, M. Kubozono. Higher order autocorrelation vision chip. IEEE Transactions on Electron Devices, vol. 53, no. 8, pp. 1797–1804, 2006.
W. C. Zhang, Q. Y. Fu, N. J. Wu. A programmable vision chip based on multiple levels of parallel processors. IEEE Journal of Solid-State Circuits, vol. 46, no. 9, pp. 2132–2147, 2011.
K. Shimizu, S. Hirai. CMOS+FPGA vision system for visual feedback of mechanical systems. In Proceedings of IEEE International Conference on Robotics and Automation, IEEE, Orlando, USA, pp. 2060–2065, 2006.
S. Hirai, M. Zakoji, A. Masubuchi, T. Tsuboi. Realtime FPGA-based vision system. Journal of Robotics and Mechatronics, vol. 17, no. 4, pp. 401–409, 2005.
Y. Watanabe, T. Komuro, M. Ishikawa. A high-speed vision system for moment-based analysis of numerous objects. In Proceedings of IEEE International Conference on Image Processing, IEEE, San Antonio, USA, pp. V177–V180, 2007.
I. Ishii, R. Sukenobe, T. Taniguchi, K. Yamamoto. Development of high-speed and real-time vision platform, H3 Vision. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, St. Louis, USA, pp. 3671–3678, 2009.
Q. Y. Gu, T. Takaki, I. Ishii. Fast FPGA-based multi-object feature extraction. IEEE Transactions on Circuits and Systems for Video Technology, vol. 23, no. 1, pp. 30–45, 2013.
I. Ishii, R. Sukenobe, K. Yamamoto, T. Takaki. Real-time image recognition using HLAC features at 1000 fps. In Proceedings of IEEE International Conference on Robotics and Biomimetics, IEEE, St. Louis, USA, pp. 954–959, 2009.
Q. Y. Gu, S. Raut, K. Okumura, T. Aoyama, T. Takaki, I. Ishii. Real-time image mosaicing system using a high-framerate video sequence. Journal of Robotics and Mechatronics, vol. 27, no. 1, pp. 12–23, 2015.
I. Ishii, T. Tatebe, Q. Y. Gu, T. Takaki. Color-histogrambased tracking at 2000 fps. Journal of Electronic Imaging, vol. 21, no. 1, Article number 013010, 2012.
Q. Y. Gu, T. Aoyama, T. Takaki, I. Ishii. High framerate tracking of multiple color-patterned objects. Journal of Real-time Image Processing, vol. 11, no. 2, pp. 251–269, 2016.
T. Komuro, T. Tabata, M. Ishikawa. A reconfigurable embedded system for 1000 f/s real-time vision. IEEE Transactions on Circuits and Systems for Video Technology, vol. 20, no. 4, pp. 496–504, 2010.
I. Ishii, T. Tatebe, Q. Y. Gu, Y. Moriue, T. Takaki, K. Tajima. 2000 fps real-time vision system with highframerate video recording. In Proceedings of IEEE International Conference on Robotics and Automation, IEEE, Anchorage, USA, pp. 1536–1541, 2010.
Y. Watanabe, T. Komuro, S. Kagami, M. Ishikawa. Realtime visual measurements using high-speed vision. In Proceedings of SPIE 5603, Machine Vision and Its Optomechatronic Applications, SPIE, Philadelphia, USA, pp. 234–242, 2004.
Q. Y. Gu, T. Takaki, I. Ishii. A fast multi-object extraction algorithm based on cell-based connected components labeling. IEICE Transactions on Information and Systems, vol. E95-D, no. 2, pp. 636–645, 2012.
Q. Y. Gu, T. Aoyama, T. Takaki, I. Ishii. Fast tracking system for multi-colored pie-shaped markers. International Journal of Optomechatronics, vol. 7, no. 3, pp. 60–80, 2013.
I. Ishii, T. Taniguchi, K. Yamamoto, T. Takaki. High-framerate optical flow system. IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 1, pp. 105–112, 2012.
H. Yang, T. Takaki, I. Ishii. Simultaneous dynamics-based visual inspection using modal parameter estimation. Journal of Robotics and Mechatronics, vol. 23, no. 1, pp. 180–195, 2011.
H. Yang, T. Takaki, I. Ishii. A structural damage quantification method for HFR-video-based modal testing. Journal of System Design and Dynamics, vol. 5, no. 4, pp. 624–641, 2011.
H. Yang, Q. Y. Gu, T. Aoyama, T. Takaki, I. Ishii. Dynamics-based stereo visual inspection using multidimensional modal analysis. IEEE Sensors Journal, vol. 13, no. 12, pp. 4831–4843, 2013.
J. Chen, Q. Y. Gu, T. Aoyama, T. Takaki, I. Ishii. Blink-spot projection method for fast three-dimensional shape measurement. Journal of Robotics and Mechatronics, vol. 27, no. 4, pp. 430–443, 2015.
Q. Y. Gu, T. Aoyama, T. Takaki, I. Ishii. Simultaneous vision-Based shape and motion analysis of cells fast-flowing in a microchannel. IEEE Transactions on Automation Science and Engineering, vol. 12, no. 1, pp. 204–215, 2015.
Q. Y. Gu, T. Kawahara, T. Aoyama, T. Takaki, I. Ishii, A. Takemoto, N. Sakamoto. LOC-Based high-throughput cell morphology analysis system. IEEE Transactions on Automation Science and Engineering, vol. 12, no. 4, pp. 1346–1356, 2015.
Y. D. Wang, I. Ishii, T. Takaki. HFR-video-based machinery surveillance for high-speed periodic operations. Journal of System Design and Dynamics, vol. 5, no. 6, pp. 1310–1325, 2011.
I. Ishii, I. Ohara, T. Tatebe, T. Takaki. 1000 fps target tracking using vibration-based image features. In Proceedings of IEEE International Conference on Robotics and Automation, IEEE, Shanghai, China, pp. 1837–1842, 2011.
A. Namiki, Y. Imai, M. Ishikawa, M. Kaneko. Development of a high-speed multifingered hand system and its application to catching. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, Las Vegas, USA, pp. 2666–2671, 2003.
A. Namiki, S. Matsushita, T. Ozeki, K. Nonami. Hierarchical processing architecture for an air-hockey robot system. In Proceedings of IEEE International Conference on Robotics and Automation, IEEE, Karlsruhe, Germany, pp. 1187–1192, 2013.
T. Senoo, D. Yoneyama, A. Namiki, M. Ishikawa. Tweezers manipulation using high-speed visual servoing based on contact analysis. In Proceedings of IEEE International Conference on Robotics and Biomimetics, IEEE, Karon Beach, Thailand, pp. 1936–1941, 2011.
T. Aoyama, Y. Harada, Q. Y. Gu, T. Takaki, I. Ishii. Cyclic motion design and analysis for a passive object manipulation using an active plate. Advanced Robotics, vol. 29, no. 7, pp. 493–503, 2015.
K. Okumura, H. Oku, M. Ishikawa. High-speed gaze controller for millisecond-order pan/tilt camera. In Proceedings of IEEE International Conference on Robotics and Automation, IEEE, Shanghai, China, pp. 6186–6191, 2011.
J. Takei, S. Kagami, K. Hashimoto. 3,000 fps 3D shape measurement using a high-speed camera-projector system. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, San Diego, USA, pp. 3211–3216, 2007.
H. Gao, Q. Y. Gu, T. Aoyama, T. Takaki, I. Ishii. A selfprojected light-section method for fast three-dimensional shape inspection. International Journal of Optomechatronics, vol. 6, no. 4, pp. 289–303, 2012.
H. Gao, T. Aoyama, T. Takaki, I. Ishii. Self-projected structured light method for fast depth-based image inspection. In Proceedings of International Conference on Quality Control by Artificial Vision, Fukuoka, Japan, pp. 175–180, 2013.
Y. J. Liu, H. Gao, Q. Y. Gu, T. Aoyama, T. Takaki, I. Ishii. High-frame-rate structured light 3D vision for fast moving objects. Journal of Robotics and Mechatronics, vol. 26, no. 3, pp. 311–320, 2014.
Y. Watanabe, T. Komuro, M. Ishikawa. 955-fps real-time shape measurement of a moving/deforming object using high-speed vision for numerous-point analysis. In Proceedings of IEEE International Conference on Robotics and Automation, IEEE, Roma, Italy, pp. 3192–3197, 2007.
Microsoft. Kinect, [Online], Available: https://en.wikipedia.org/wiki/Kinect, June 29, 2015.
S. Tabata, S. Noguchi, Y. Watanabe, M. Ishikawa. Highspeed 3D sensing with three-view geometry using a segmented pattern. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, Hamburg, Germany, pp. 3900–3907, 2015.
A. Cassinelli, M. Ishikawa. Khronos projector, emerging technologies. In Proceedings of SIGGRAPH, ACM, Los Angeles, USA, pp. 4506–4511, 2004.
Y. Watanabe, A. Cassinelli, T. Komuro, M. Ishikawa. The deformable workspace: A membrane between real and virtual space. In Proceedings of IEEE International Workshop on Horizontal Interactive Human-Computer Systems, IEEE, Amsterdam, Netherlands, pp. 145–152, 2008.
J. Chen, T. Yamamoto, T. Aoyama, T. Takaki, I. Ishii. Simultaneous projection mapping using high-frame-rate depth vision. In Proceedings of IEEE International Conference on Robotics and Automation, IEEE, Hong Kong, China, pp. 4506–4511, 2014.
T. Sueishi, H. Oku, M. Ishikawa. Robust high-speed tracking against illumination changes for dynamic projection mapping. In Proceedings of IEEE Conference on Virtual Reality, IEEE, Arles, France, pp. 97–104, 2015.
I. Ishii, T. Ichida, Q. Y. Gu, T. Takaki. 500-fps face tracking system. Journal of Real-Time Image Processing, vol. 8, no. 4, pp. 379–388, 2013.
Y. M. Nie, T. Takaki, I. Ishii, H. Matsuda. Algorithm for automatic behavior quantification of laboratory mice using high-frame-rate videos. SICE Journal of Control, Measurement, and System Integration, vol. 4, no. 5, pp. 322–331, 2011.
I. Ishii, S. Takemoto, T. Takaki, M. Takamoto, K. Imon, K. Hirakawa. Real-time laryngoscopic measurements of vocal fold vibrations. In Proceedings of IEEE International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, Boston, USA, pp. 6623–6626, 2011.
A. Drochon. Use of cell transit analyzer pulse height to study the deformation of erythrocytes in microchannels. Medical Engineering & Physics, vol. 27, no. 2, pp. 157–165, 2005.
Y. Katsumoto, K. Tatsumi, T. Doi, K. Nakabe. Electrical classification of single red blood cell deformability in highshear microchannel flows. International Journal of Heat and Fluid Flow, vol. 31, no. 6, pp. 985–995, 2010.
S. Sakuma, K. Kuroda, C. Tsai, W. Fukui, F. Arai, M, Kaneko. Red blood cell fatigue evaluation based on the close-encountering point between extensibility and recoverability. Lab on a Chip, vol. 14, no. 6, pp. 1135–1141, 2014.
Author information
Authors and Affiliations
Corresponding author
Additional information
Recommended by Associate Editor Jangmyung Lee
Qing-Yi Gu received the B.Eng. degree in electronic and information engineering from Xi’an Jiaotong University, China in 2005. He received the M. Eng. degree, and Ph.D. degree in engineering, from Hiroshima University, Japan in 2010 and 2013, respectively. He is currently a designated associate professor in Graduate School of Engineering, Hiroshima University, Japan.
His research interests include high-speed image feature extraction, and applications in industry and biomedicine.
ORCID iD: 0000-0001-8332-5350
Idaku Ishii received the B. Eng. the M. Eng. and the Ph.D. degrees from the University of Tokyo, Japan in 1992, 1994, and 2000, respectively. He is currently a professor in Graduate School of Engineering, Hiroshima University, Japan.
His research interests include high-speed robot vision, sensory information processing, sensor-based robot manipulation, and applications in industry and biomedicine.
Rights and permissions
About this article
Cite this article
Gu, QY., Ishii, I. Review of some advances and applications in real-time high-speed vision: Our views and experiences. Int. J. Autom. Comput. 13, 305–318 (2016). https://doi.org/10.1007/s11633-016-1024-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11633-016-1024-0