Videos and Presentations

Neuromorphic Object Recognition

Imagine having a car that is able to drive itself or a robot that is able to perform tasks ranging from the tedious (e.g. housekeeping) to the dangerous (flying military aircrafts) and even to the difficult (building space stations) without human supervision. This is the stuff of science fiction. Today’s robots make certain tasks easier but still require remote supervision and control by humans. Intelligent robots need to be able to interact with objects in their surroundings without human involvement. The first step in this process is recognition – determining whether an object is an obstacle to be avoided, an item to be retrieved, or perhaps a tool required for a particular task [1]. People are able to categorize objects broadly as a person, a dog, a table etc. They are also able to identify those objects more precisely as my friend, my dog, my table etc. Can we design a system capable of this level of intelligence?

Visual object recognition is a fundamental cognitive task performed by the primate brain. The visual system is able to rapidly and effortlessly recognize – identify and categorize – diverse objects in cluttered scenes under widely varying viewing conditions, such as changes in position, rotation and illumination [2]. However, object recognition is a computationally difficult task and engineered systems are unable to match the level of proficiency and speed of human visual systems. Many modern object recognition devices are capable of recognizing one object category such as faces, cars, or pedestrians [3-5]. Some others can recognize many objects one at a time relying on probabilistic shape models of the desired objects [6]. The most impressive one is that of Serre et al. [7] which is based on the model of the visual cortex. However, it is implemented in software and thus unable to operate in real time.

This project involves the development of an autonomous, continuous-time visual system that emulates object recognition in the primate visual cortex. This multi-stage object recognition system will utilize large-scale arrays of identical silicon neurons (the IFAT) to build a biologically-plausible model of visual information processing. It will receive its inputs from neuromorphic retinas and implement silicon facsimiles of cortical simple cells, complex cells, composite feature cells, complex composite cells and finally, view-tuned cells according to the hierarchical model of primate visual cortex of Riesenhuber and Poggio [8].


  1. L. Stark, “Recognizing object function through reasoning about 3D shape and dynamic physical properties,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 546-553, 1994.
  2. T. Serre, M. Kouh, C. Cadieu, U. Knoblich, G. Kreiman, and T. Poggio, “ A theory of object recognition: computations and circuits in the feedforward path of the ventral stream in primate visual cortex,” AI Memo, MIT, Cambridge 2005.
  3. H. Schneiderman and T. Kanade, “A statistical method for 3D object detection applied to faces and cars,” CVPR, pp. 746-751, 2000.
  4. P. Viola and M. Jones, “Robust real-time face detection,” CVPR,pp. 1254-1259, 2001.
  5. A. Mohan, C. Papageorgiou, and T. Poggio, “Example-based object detection in images by components,” PAMI, pp. 249-261, 2001.
  6. R. Fergus, P. Perona, and A. Zisserman, “Object class recognition by unsupervised scale-invariant learning,” CVPR, pp. 264-271, 2003.
  7. T. Serre, L. Wolf, S. Bileschi, M. Riesenhuber, and T. Poggio, “ Robust object recognition with cortex-like mechanisms,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 29, pp. 411-426, 2007a.
  8. M. Riesenhuber, T. Poggio, “Models of object recognition,” Nature Neuroscience 2000a