Abstract
The problem of automatically learning object models for recognition and pose estimation is addressed. In contrast to the traditional approach, the recognition problem is formulated as one of matching appearance rather than shape. The appearance of an object in a two-dimensional image depends on its shape, reflectance properties, pose in the scene, and the illumination conditions. While shape and reflectance are intrinsic properties and constant for a rigid object, pose and illumination vary from scene to scene. A compact representation of object appearance is proposed that is parametrized by pose and illumination. For each object of interest, a large set of images is obtained by automatically varying pose and illumination. This image set is compressed to obtain a low-dimensional subspace, called the eigenspace, in which the object is represented as a manifold. Given an unknown input image, the recognition system projects the image to eigenspace. The object is recognized based on the manifold it lies on. The exact position of the projection on the manifold determines the object's pose in the image.
A variety of experiments are conducted using objects with complex appearance characteristics. The performance of the recognition and pose estimation algorithms is studied using over a thousand input images of sample objects. Sensitivity of recognition to the number of eigenspace dimensions and the number of learning samples is analyzed. For the objects used, appearance representation in eigenspaces with less than 20 dimensions produces accurate recognition results with an average pose estimation error of about 1.0 degree. A near real-time recognition system with 20 complex objects in the database has been developed. The paper is concluded with a discussion on various issues related to the proposed learning and recognition methodology.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Besl, P.J., and Jain, R.C. 1985. “Three-Dimensional Object Recognition,”ACM Computing Surveys, Vol. 17, No. 1, pp. 75–145.
Chen, W.H., Smith, H., and Fralick, S.C. 1977. “A Fast Computational Algorithm for the Discrete Cosine Transform,”IEEE Transactions on Communications, Vol.25, pp.1004–1009 Vol. 18, No. 1, pp. 67–108.
Edelman, S. and Weinshall, D. 1991. “A self-organizing multiple-view representation of 3D objects,”Biological Cybernetics Vol. 64, pp. 209–219.
Edelman, S., Bulthoff, H., and Weinshall, D. 1989. “Stimulus familiarity determines recognition strategy for novel 3D objects,” A.I. Memo No. 1138, AI Lab, MIT.
Fan, T.J., Medioni, G., and Nevatia, R. 1988. “Recognizing 3-D Objects Using Surface Descriptions,”Proc. of Intl. Conference on Computer Vision, pp. 474–481, Florida.
Fukunaga, K. 1990.Introduction to Statistical Pattern Recognition, Academic Press, London.
Householder, A.S. 1964.The theory of matrices in numerical analysis, Dover Publications, New York.
Huttenlocher, D.P., and Ullman, S. 1990. “Recognizing solid objects by alignment with an image,”International Journal of Computer Vision, Vol. 5, No. 2, pp. 195–212.
Ikeuchi, K., and Suehiro, T. 1992. “Recognizing Assembly Tasks using Face-Contact Relations,”Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 154–160.
Koenderink, J.J., and van Doorn, A.J. 1979. “The internal representation of solid shape with respect to vision,”Biological Cybernetics Vol. 32, pp. 211–216.
Mukherjee, S., and Nayar, S.K. 1994. “Appearance Based Recognition of 3D Objects using RBF Networks,” Technical Report, Department of Computer Science, Columbia University (in preparation).
Murakami, H., and Kumar, V. 1982. “Efficient Calculation of Primary Images from a Set of Images,”IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 4, No. 5, pp. 511–515.
Murase, H., Kimura, E., Yoshimura, M., and Miyake, Y. 1981. “An Improvement of the Auto-Correlation Matrix in Pattern Matching Method and Its Application to Handprinted ‘HI-RAGANA’,”Trans. IECE, Vol. J64-D, No. 3, pp. 276–283.
Murase, H., and Lindenbaum, M. 1992. “Spatial Temporal Adaptive Method for Partial Eigenstructure Decomposition of Large Images,”NTT Technical Report No. 6527. AlsoIEEE Transactions on Image Processing (in press).
Murase, H., and Nayar, S.K., 1994. “Illumination Planning for Object Recognition in Structured Environments,”Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 31–38.
Nayar, S.K., Murase, H., and Nene, S.A. 1994. “Learning, Positioning, and Tracking Visual Appearance,”Proc. IEEE Conf. on Robotics and Automation, pp. 3237–3244.
Nene, S.A., and Nayar, S.K. 1994. “Binary Search Through Multiple Dimensions,” Technical Report CUCS-018-94, Department of Computer Science, Columbia University.
Oja, E. 1983.Subspace methods of Pattern Recognition, Research Studies Press, Hertfordshire.
Poggio, T., and Girosi, F. 1990. “Networks for Approximation and Learning,”Proceedings of the IEEE, Vol. 78, No. 9, pp. 1481–1497.
Poggio, T., and Edelman, S. 1990. “A networks that learns to recognize three-dimensional objects,”Nature, Vol. 343, pp. 263–266.
Press, W., Flannery, B.P., Teukolsky, S.A., and Vetterling, W.T. 1988.Numerical Recipes in C, Cambridge University Press, Cambridge.
Sirovich, L., and Kirby, M. 1987. “Low dimensional procedure for the characterization of human faces,” Journal of Optical Society of America, Vol. 4, No. 3, pp. 519–524.
Tarr, M., and Pinker, S. 1989. “Mental rotation and orientation- dependence in shape recognition,”Cognitive Psychology, Vol. 21, pp. 233–282.
Turk, M.A., and Pentland, A.P. 1991. “Face Recognition Using Eigenfaces,”Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 586–591.
Ullman, S., and Basri, R. 1991. “Recognition by Linear Combination of Models,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 13, No. 10, pp. 992–1006.
Yang, X., Sarkar, T.K., and Arvas, E. 1989. “A Survey of Conjugate Gradient Algorithms for Solution of Extreme Eigen-Problems of a Symmetric Matrix,”IEEE Transactions on Acoustics, Speech and Signal Processing, Vol. 37, No. 10, pp. 1550–1555.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Murase, H., Nayar, S.K. Visual learning and recognition of 3-d objects from appearance. Int J Comput Vision 14, 5–24 (1995). https://doi.org/10.1007/BF01421486
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01421486