Abstract
This chapter reviews neuromorphic silicon retinas and cochleas that are based on the structure and operation of their biological counterparts. These devices are built using conventional chip fabrication technologies, using transistor circuits that emulate neural computations from biology. In first generation sensors, the analog outputs of every cell were read out serially at fixed sample rates. The new generation of sensors reports only the outputs of active cells through digital events (spikes) that are communicated asynchronously. Such sensors respond more quickly with reduced power consumption. Their digital “address-event” outputs rapidly convey precise timing information about the scene that is only attained from conventional sensors if they are continuously sampled at high rates. The sparseness, low latency, and spatio-temporal structure of this new form of sensor output data can benefit subsequent post-processing algorithms. Tradeoffs in the design of neuromorphic visual and auditory sensors are discussed. Examples are given of vision algorithms that process the address-events, using their spatio-temporal coherence, for low-level feature extraction and for object tracking.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Abdalla H, Horiuchi T (2005) An ultrasonic filterbank with spiking neurons, IEEE Intl. Symp. on Circuits and Systems (ISCAS 2005), pp. 4201–4204
Barbaro M, Burgi PY, Mortara A, Nussbaum P, Heitger F (2002) A 100 × 100 pixel silicon retina for gradient extraction with steering filter capabilities and temporal output coding. IEEE J. of Solid-State Circuits 37: 160–172
Bauer D, Belbachir AN, Donath N, Gritsch G, Kohn B, et al. (2007) Embedded vehicle speed estimation system using an asynchronous temporal contrast vision sensor. EURASIP J Embedded Syst 2007 (art. ID 82 174): 1–12
Belbachir AN, Litzenberger M, Posch C, Schon P (2007) Real-time vision using a smart sensor system. IEEE Intl Symp Industrial Electronics 2007. ISIE 2007, pp. 1968–1973.
Berner R, Lichtsteiner P, Delbruck T (2008) Self-timed vertacolor dichromatic vision sensor for low power face detection. IEEE Intl Symp Circuits and Systems (ISCAS 2008), pp. 1032–1035
Boerlin M, Delbruck T, Eng K (2009) Getting to know your neighbors: Unsupervised learning of topography from real-world, event-based input. Neural Computation 21: 216–238
Chan V, Liu S-C, van Schaik A (2007) AER EAR: A matched silicon cochlea pair with address event representation interface. IEEE Trans Circuits and Systems I: Regular Papers 54: 48–59
Chicca E, Whatley AM, Lichtsteiner P, Dante V, Delbruck T, et al. (2006) A multi-chip pulse-based neuromorphic infrastructure and its application to a model of orientation selectivity. IEEE Trans Circuits and Systems I: Regular Papers 54: 981–993
Choi TYW, Merolla PA, Arthur JV, Boahen KA, Shi BE (2005) Neuromorphic implementation of orientation hypercolumns. IEEE Trans Circuits and Systems I: Regular Papers 52: 1049–1060
Conradt J, Berner C. M., Delbruck T (2009) An embedded AER dynamic vision sensor for low-latency pole balancing. 5th IEEE Workshop on Embedded Computer Vision (in conjunction with ICCV 2009), Kyoto, Japan
Delbruck T, Lichtsteiner P (2007) Fast sensory motor control based on event-based hybrid neuromorphic-procedural system. IEEE Intl Symp Circuits and Systems (ISCAS 2007), pp. 845–848
Delbruck T (2008) Frame-free dynamic digital vision. Proc Intl Symp Secure-Life Electronics, Advanced Electronics for Quality Life and Society, pp. 21–26. Tokyo: University of Tokyo
Douglas R, Mahowald M, Mead C (1995) Neuromorphic Analog VLSI. Ann Rev Neurosci 18: 255–281
Fasnacht D, Delbruck T (2007) Dichromatic spectral measurement circuit in vanilla CMOS. IEEE Intl Symp Circuits and Systems (ISCAS 2007), pp. 3091–3094
Fossum ER (1997) CMOS image sensors: electronic camera-on-a-chip. IEEE Trans Electron Devices 44: 1689–1698
Fragniere E (2005) A 100-channel analog CMOS auditory filter bank for speech recognition. IEEE ISSCC Dig of Tech Papers, pp. 140–589
Fu Z, Delbruck T, Lichtsteiner P, Culurciello E (2008) An address-event fall detector for assisted living applications. IEEE Trans Biomed Circuits and Systems 2: 88–96.
Georgiou J, Toumazou C (2005) A 126-uW cochlear chip for a totally implantable system. IEEE J. Solid-State Circuits 40: 430–443
Gold B, Morgan N (2000) Speech and audio signal processing: John Wiley and Sons, Inc. New York, NY
Gritsch G, Litzenberger M, Donath N, Kohn B (2008) Real-time vehicle classification using a smart embedded device with a’ silicon retina’ optical sensor. ITSC08, pp. 534–538. Bejing, China
Hamilton T, Tapson J, Jin CT, van Schaik A (2008) An active 2-D silicon cochlea. IEEE Trans Biomed Circuits and Systems 2: 30–43
Indiveri G, Liu S-C, Delbruck T, Douglas R (2009) Neuromorphic systems. In: L Squire (ed) Encyclopedia of neuroscience, pp. 521–528: Academic Press
jAER (2007) jAER Real time sensory-motor processing for spike based address-event representation (AER) sensors and systems available: http://jaer.wiki.sourceforge.net
Katsiamis A, Drakakis E, Lyon R (2009) A biomimetic, 4.5uW, 120+ dB, log-domain cochlea channel with AGC. IEEE J Solid-State Circuits 44: 1006–1022
Kramer J (2002) An ON/OFFtransient imager with event-driven, asynchronous read-out. IEEE Intl Symp Circuits and Systems (ISCAS 2002), pp. 165–168
Lazzaro J, Wawrzynek J, Mahowald M, Sivilotti M, Gillespie D (1993) Silicon auditory processors as computer peripherals. IEEE Trans Neural Networks 4: 523–528
Lennie P (2003) The cost of cortical computation. Current Biology 13: 493–497
Lenoro-Bardallo JA, Serrano-Gotarredona T, Linares-Barranco B (2010) A spatial calibrated contrast AER vision sensor with adjustable contrast threshold. IEEE Intl Symp Circuits and Systems (ISCAS 2010), pp. 2426–2429
Lichtsteiner P, Posch C, Delbruck T (2006) A 128×128 120 dB 30 mW asynchronous vision sensor that responds to relative intensity change. ISSCC Dig Tech. Papers, pp. 508–509 (27.9). San Francisco
Lichtsteiner P, Posch C, Delbruck T (2008) A 128×128 120 dB 15us latency asynchronous temporal contrast vision sensor. IEEE J Solid State Circuits 43: 566–576
Linares-Barranco A, Gómez-Rodríguez F, Jiménez A, Delbruck T, Lichtsteiner P (2007) Using FPGA for visuo-motor control with a silicon retina and a humanoid robot. IEEE Intl Symp Circuits and Systems (ISCAS 2007), pp. 1192–1195
Litzenberger M, Posch C, Bauer D, Schön P, Kohn B, et al. (2006) Embedded vision system for real-time object tracking using an asynchronous transient vision sensor. IEEE Digital Signal Proc Workshop 2006, pp. 173–178. Grand Teton, Wyoming
Liu SC, Kramer J, Indiveri G, Delbruck T, Douglas R (2002) Analog VLSI: circuits and principles: MIT Press, Cambridge, MA
Liu SC and Delbruck T (2010) Neuromorphic sensory systems. Curr. Opin. in Neurobiol 20: 288–295
Liu SC, van Schaik A, Minch BA, Delbruck T (2010a) Event-based 64-channel binaural silicon cochlea with Q enhancement mechanisms. IEEE Intl Symp Circuits and Systems 2010 (ISCAS 2010), pp. 2027–2030
Liu SC, Mesgarani N, Harris, J, Hermansky, H (2010b) The use of spike-based representations for hardware audition systems. IEEE Intl Symp Circuits and Systems 2010 (ISCAS 2010), pp. 505–508
Lyon RF, Mead C (1988) An analog electronic cochlea. IEEE Trans Acoustics Speech and Signal Processing 36: 1119–1134
Mahowald MA (1992) VLSI analogs of neuronal visual processing: a synthesis of form and function. Computation and neural systems, Caltech, Pasadena, California
Mahowald MA (1994) An analog VLSI system for stereoscopic vision: Kluwer, Boston, MA
Mahowald MA, Mead C (1991) The silicon retina. Sci Am 264: 76–82
Mallik U, Clapp M, Choi E, Cauwenberghs G, Etienne-Cummings R (2005) Temporal change threshold detection imager. IEEE ISSCC Dig Tech. Papers, pp. 362–363
Martignoli S, van der Vyver J-J, Kern A, Uwate Y, Stoop R (2007) Analog electronic cochlea with mammalian hearing characteristics. Applied Physics Letters 91 (064 108)
Massari N, Gottardi M, Jawed S (2008) A 100uW 64 × 128-pixel contrast-based asynchronous bin ary vision sensor for wireless sensor networks. IEEE ISSCC Dig Tech Papers, pp. 588–638
Mead C (1990) Neuromorphic electronic systems. Proc IEEE 78: 1629–1636
Olsson JAM, Hafliger P (2009) Live demonstration of an asynchronous integrate-and-fire pixel-event vision sensor. IEEE Intl Symp Circuits and Systems (ISCAS 2009), pp. 774 774
Pelgrom M, Tuinhout H, Vertregt M (1998) Transistor matching in analog CMOS applications. IEDM Tech Dig: 915–918
Posch C, Hofstatter M, Matolin D, Vanstraelen G, Schon P, et al. (2007) A dual-line optical transient sensor with on-chip precision time-stamp generation. IEEE ISSCC Dig Tech Papers, pp. 500–618
Posch C, Matolin D, Wohlgenannt R (2010) A QVGA 143 dB DR asynchronous address-event PWM dynamic image sensor with lossless pixel-level video compression. IEEE ISSCC Dig Tech. Papers, pp. 400–401
Posch C, Matolin D, Wohlgenannt R, Maier T, Litzenberger M (2009) A microbolometer asynchronous dynamic vision sensor for LWIR. IEEE Sensors Journal 9: 654–664
Rodieck R (1998) The first steps in seeing: Sinauer Associates, Sunderland, MA
Ruedi PF, Heim P, Gyger S, Kaess F, Arm C, et al. (2009) An SoC combining a 132 dB QVGA pixel array and a 32 b DSP/MCU processor for vision applications. IEEE ISSCC Dig Tech Papers, pp. 46–47
Ruedi PF, Heim P, Kaess F, Grenet E, Heitger F, et al. (2003) A 128 × 128, pixel 120-dB dynamic-range vision-sensor chip for image contrast and orientation extraction. IEEE J. Solid-State Circuits 38: 2325–2333
Sarpeshkar R (1998) Analog versus digital: Extrapolating from electronics to neurobiology. Neural Computation 10: 1601–38
Sarpeshkar R, Baker MS C, Sit JJ, Turicchia L, Zhak S (2005) An analog bionic ear processor with zero-crossing detection. IEEE ISSCC Dig Tech Papers, pp. 78–79
Sarpeshkar R, Lyon RF (1998) A low-power wide-dynamic-range analog VLSI cochlea. Analog Integrated Circuits and Signal Processing 16: 245–274
Serrano-Gotarredona R, Oster M, Lichtsteiner P, Linares-Barranco A, Paz-Vicente R, et al. (2009) CAVIAR: A 45 k neuron, 5 M synapse, 12 G connects/s AER hardware sensory-processing-learning-actuating system for high-speed visual object recognition and tracking. IEEE Trans Neur al Networks 20: 1417–1438
Uysal I, Sathyendra H, Harris JG (2006) A biologically plausible system approach for noise robust vowel recognition. IEEE Proc Midwest Symp Circuits and Systems, pp. 245–249
van Schaik A, Shamma S (2004) A neuromorphic sound localizer. Analog Integrated Circuits and Signal Processing 39: 267–273
Watts L, Kerns DA, Lyon RF, Mead CA (1992) Improved implementation of the silicon cochlea. IEEE J. of Solid State Circuits 27: 692–700
Wen B, Boahen K (2006) A 360-channel speech preprocessor that emulates the cochlear amplifier. IEEE ISSCC Dig Tech Papers, pp. 556–557
Zaghloul KA, Boahen K (2004) Optic nerve signals in a neuromorphic chip II: Testing and results. IEEE Trans Biomed Engineering 51: 667–675
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag/Wien
About this chapter
Cite this chapter
Delbruck, T., Liu, SC. (2012). Event-based silicon retinas and cochleas. In: Frontiers in Sensing. Springer, Vienna. https://doi.org/10.1007/978-3-211-99749-9_6
Download citation
DOI: https://doi.org/10.1007/978-3-211-99749-9_6
Publisher Name: Springer, Vienna
Print ISBN: 978-3-211-99748-2
Online ISBN: 978-3-211-99749-9
eBook Packages: Biomedical and Life SciencesBiomedical and Life Sciences (R0)