Abstract
This paper develops a semi-supervised learning algorithm called convolutional deep networks (CDN), to address the image classification problem with deep learning. First, we construct the previous several hidden layers using convolutional restricted Boltzmann machines, which can reduce the dimension and abstract the information of the images effectively. Second, we construct the following hidden layers using restricted Boltzmann machines, which can abstract the information of images quickly. Third, the constructed deep architecture is fine-tuned by gradient-descent based supervised learning with an exponential loss function. CDN can reduce the dimension and abstract the information of the images at the same time efficiently. More importantly, the abstraction and classification procedure of CDN use the same deep architecture to optimize the same parameter in different steps continuously, which can improve the learning ability effectively. We did several experiments on two standard image datasets, and show that CDN are competitive with both representative semi-supervised classifiers and existing deep learning techniques.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Chapelle O, Scholkopf B, Zien A (2006) Semi-supervised learning. MIT Press, Cambridge
Collobert R, Sinz F, Weston J, Bottou L (2006) Large scale transductive SVMs. J Mach Learn Res 7: 1687–1712
Desjardins G, Bengio Y (2008) Empirical evaluation of convolutional rbms for vision. Tech. rep
Feng GY, Hu DW, Zhou ZT (2008) A direct locality preserving projections (DLPP) algorithm for image recognition. Neural Process Lett 27(3): 247–255
Hinton GE (2002) Training products of experts by minimizing contrastive divergence. Neural Comput 14(8): 1771–1800
Hinton GE (2010) Learning to represent visual input. Philos Trans R Soc B-Biol Sci 365(1537): 177–184
Hinton GE, Salakhutdinov R (2006) Reducing the dimensionality of data with neural networks. Science 313: 504–507
Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18: 1527–1554
LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4): 541–551
Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11): 2278–2324
Lee H, Grosse R, Ranganath R, Ng A (2009) Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: International conference on machine learning. ACM, Montreal, Canada, pp 609–616
Li F, Fergus R, Perona P (2004) Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories. In: CVPR workshop on generative-model based vision. Elsevier Science Inc., New York, pp 1–9
Liu Y, Zhou S, Chen Q (2011) Discriminative deep belief networks for visual data classification. Pattern Recognit 44(10–11): 2287–2296
Machajdik J, Hanbury A (2010) Affective image classification using features inspired by psychology and art theory. In: International conference on multimedia. ACM, New York, pp 83–92
Salakhutdinov R, Hinton GE (2007) Learning a nonlinear embedding by preserving class neighbourhood structure. J Mach Learn Res 2: 412–419
Weston J, Ratle F, Collobert R (2008) Deep learning via semi-supervised embedding. In: International conference on machine learning. ACM, Helsinki, pp 1168–1175
Zhu X (2007) Semi-supervised learning literature survey. Tech. rep.. University of Wisconsin Madison, Madison
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhou, S., Chen, Q. & Wang, X. Convolutional Deep Networks for Visual Data Classification. Neural Process Lett 38, 17–27 (2013). https://doi.org/10.1007/s11063-012-9260-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-012-9260-y