Pattern classification methods based on learning-from-examples have been widely applied to character recognition from the 1990s and have brought forth significant improvements of recognition accuracies. This kind of methods include statistical methods, artificial neural networks, support vector machines, multiple classifier combination, etc. In this chapter, we briefly review the learning-based classification methods that have been successfully applied to character recognition, with a special section devoted to the classification of large category set. We then discuss the characteristics of these methods, and discuss the remaining problems in character recognition that can be potentially solved by machine learning methods.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
- Radial Basis Function Network
- Character Recognition
- Convolutional Neural Network
- Learn Vector Quantization
- Support Vector Data Description
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Mori, S., Suen, C.Y., Yamamoto, K.: Historical review of OCR research and development. Proc. IEEE 80(7) (1992) 1029-1058
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11) (1998) 2278-2324
Suen, C.Y., Liu, K., Strathy, N.W.: Sorting and recognizing cheques and fi-nancial documents. In: Lee SW, Nakano Y (eds) Document Analysis Systems: Theory and Practice. Springer, LNCS 1655 (1999) 173-187
Liu, C.L., Nakashima, K., Sako, H., Fujisawa, H.: Handwritten digit recognition: Benchmarking of state-of-the-art techniques. Pattern Recognition 36(10) (2003) 2271-2285
Marinai, S., Gori, M., Soda, G.: Artificial neural networks for document analysis and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 27(1) (2005) 23-35
Jain, A.K., Duin, R.P.W., Mao, J.: Statistical pattern recognition: A review. IEEE Trans. Pattern Anal. Mach. Intell. 22(1) (2000) 4-37
Fukunaga, K.: Introduction to Statistical Pattern Recognition. Academic Press, 2nd edition (1990)
Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press (1995)
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification. Wiley Interscience, 2nd edition (2001)
Friedman, J.H.: Regularized discriminant analysis. J. Am. Statist. Ass. 84(405) (1989) 165-175
Kimura, F., Takashina, K., Tsuruoka, S., Miyake, Y.: Modified quadratic dis-criminant functions and the application to Chinese character recognition. IEEE Trans. Pattern Anal. Mach. Intell. 9(1) (1987) 149-153
Kimura, F., Wakabayashi, T., Tsuruoka, S., Miyake, Y.: Improvement of hand-written Japanese character recognition using weighted direction code histogram. Pattern Recognition 30(8) (1997) 1329-1337
Ikeda, M., Tanaka, H., Motooka, T.: Projection distance method of recognition of handwritten characters. Trans. IPS Japan 24(1) (1983) 106-112
Nakajima, T., Wakabayashi, T., Kimura, F., Miyake, Y.: Accuracy improve- ment by compound discriminant functions for resembling character recognition. Trans. IEICE Japan J83-D-II(2) (2000) 623-633
Hinton, G.E., Dayan, P., Revow, M.: Modeling the manifolds of images of hand-written digits. IEEE Trans. Neural Networks 8(1) (1997) 65-74
Kim, H.C., Kim, D., Bang, S.Y.: A numeral character recognition using the PCA mixture model. Pattern Recognition Letters 23 (2002) 103-111
Tsay, M.K., Shyu, K.H., Chang, P.C.: Feature transformation with generalized LVQ for handwritten Chinese character recognition. IEICE Trans. Information and Systems E82-D(3) (1999) 687-92
Zhang, P., Bui, T., Suen, C.Y.: Hybrid feature extraction and feature selec- tion for improving recognition accuracy of handwritten numerals. In: Proc. 8th ICDAR, Seoul, Korea, 1 (2005) 136-140
Kawatani, T., Shimizu, H.: Handwritten Kanji recognition with the LDA method. In: Proc. 14th ICPR, Brisbane, 2 (1998) 1031-1035
Wakabayashi, T., Shi, M., Ohyama, W., Kimura, F.: Accuracy improvement of handwritten numeral recognition by mirror image learning. In: Proc. 6th ICDAR, Seattle (2001) 338-343
Simard, P.Y., Steinkraus, D., Platt, J.C.: Best practices for convolutional neural networks applied to visual document analysis. In: Proc. 7th ICDAR, Edinburgh, UK, 2 (2003) 958-962
Oh, I.S., Suen, C.Y.: A class-modular feedforward neural network for handwrit-ing recognition. Pattern Recognition 35(1) (2002) 229-244
Pao, Y.H.: Adaptive Pattern Recognition and Neural Networks. Addison- Wesley, MA (1989)
Schürmann, J.: Pattern Classification: A Unified View of Statistical and Neural Approaches. Wiley Interscience (1996)
Kreßel, U., Schürmann, J.: Pattern classification techniques based on function approximation. In: Bunke H, Wang PSP (eds) Handbook of Character Recog-nition and Document Image Analysis, World Scientific (1997) 49-78
Franke, J.: Isolated handprinted digit recognition. In: Bunke H, Wang PSP (eds) Handbook of Character Recognition and Document Image Analysis, World Scientific (1997) 103-121
Liu, C.L., Sako, H.: Class-specific feature polynomial classifier for pattern classi- fication and its application to handwritten numeral recognition. Pattern Recognition 39(4) (2006) 669-681
Kimura, F., Inoue, S., Wakabayashi, T., Tsuruoka, S., Miyake, Y.: Handwritten numeral recognition using autoassociative neural networks.In: Proc. 14th ICPR, Brisbane, 1 (1998) 166-171
Zhang, B., Fu, M., Yang, H.: A nonlinear neural network model of mixture of lo-cal principal component analysis: Application to handwritten digits recognition. Pattern Recognition 34(2) (2001) 203-214
Kohonen, T.: The self-organizing map. Proc. IEEE 78(9) (1990) 1464-1480
Liu, C.L., Nakagawa, M.: Evaluation of prototype learning algorithms for nearest neighbor classifier in application to handwritten character recognition. Pattern Recognition 34(3) (2001) 601-615
Liu, C.L., Sako, H., Fujisawa, H.: Discriminative learning quadratic discriminant function for handwriting recognition. IEEE Trans. Neural Networks 15(2) (2004) 430-444
Vapnik, V.: The Nature of Statistical Learning Theory. Springer-Verlag, New Work (1995)
Burges, C.J.C.: A tutorial on support vector machines for pattern recognition. Knowledge Discovery and Data Mining 2(2) (1998) 1-43
Kressel, U.: Pairwise classification and support vector machines. In: Schölkopf B, Burges CJC, Smola AJ (eds) Advances in Kernel Methods: Support Vector Learning, MIT Press (1999) 255-268
Bellili, A., Gilloux, M., Gallinari, P.: An MLP-SVM combination architecture for offline handwritten digit recognition: Reduction of recognition errors by support vector machines rejection mechanisms. Int. J. Document Analysis and Recogni-tion 5(4) (2003) 244-252
Dong, J.X., Krzyzak, A., Suen, C.Y.: An improved handwritten Chinese char- acter recognition system using support vector machine. Pattern Recognition Letters 26(12) (2005) 1849-1856
Xu, L., Krzyzak, A., Suen, C.Y.: Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Trans. System Man Cyber-net. 22(3) (1992) 418-435
Ho, T.K., Hull, J., Srihari, S.N.: Decision combination in multiple classifier systems. IEEE Trans. Pattern Anal. Mach. Intell. 16(1) (1994) 66-75
Rahman, A.F.R., Fairhurst, M.C.: Multiple classifier decision combination strategies for character recognition: A review. Int. J. Document Analysis and Recognition 5(4) (2003) 166-194
Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20(3) (1998) 226-239
Duin, R.P.W.: The combining classifiers: To train or not to train. Proc. 16th ICPR, Quebec, Canada, 2 (2002) 765-770
Liu, C.L.: Classifier combination based on confidence transformation. Pattern Recognition, 38(1) (2005) 11-28
Suen, C.Y., Lam, L.: Multiple classifier combination methodologies for different output levels. In: Kittler J, Roli F (eds) Multiple Classifier Systems, Springer, LNCS 1857 (2000) 52-66
Breiman, L.: Bagging predictors. Machine Learning 24(2) (1996) 123-140
Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learn-ing and an application to boosting. J. Computer and System Sciences 55(1) (1997) 119-139
Ha, T., Bunke, H.: Off-line handwritten numeral recognition by perturbation method. IEEE Trans. Pattern Anal. Mach. Intell. 19(5) (1997) 535-539
Dahmen, J., Keysers, D., Ney, H.: Combined classification of handwritten digits using the virtual test sample method. In : Kittler J, Roli F (eds) Multiple Classifier Systems, Springer, LNCS 2096 (2001) 99-108
Tang, Y.Y., et al.: Offline recognition of Chinese handwriting by multifeature and multilevel classification. IEEE Trans. Pattern Anal. Mach. Intell. 20(5) (1998) 556-561
Wang, Q.R., Suen, C.Y.: Analysis and design of a decision tree based on entropy reduction and its application to large character set recognition. IEEE Trans. Pattern Anal. Mach. Intell. 6(4) (1984) 406-417
Suzuki, M., Omachi, S., Kato, N., Aso, H., Nemoto, Y.: A discrimination method of similar characters using compound Mahalanobis function. Trans. IE-ICE Japan J80-D-II(10) (1997) 2752-2760
Kato, N., Suzuki, M., Omachi, S., Aso, H., Nemoto, Y.: A handwritten charac-ter recognition system using directional element feature and asymmetric Maha-lanobis distance. IEEE Trans. Pattern Anal. Mach. Intell. 21(3) (1999) 258-262
Fu, H.C., Xu, Y.Y.: Multilinguistic handwritten character recognition by Bayesian decision-based neural networks. IEEE Trans. Signal Processing 46(10) (1998) 2781-2789
Kimura, Y., Wakahara, T., Tomono, A.: Combination of statistical and neural classifiers for a high-accuracy recognition of large character sets. Trans. IEICE Japan J83-D-II(10) (2000) 1986-1994
Saruta, K., Kato, N., Abe, M., Nemoto, Y.: High accuracy recognition of ETL9B using exclusive learning neural network-II (ELNET-II). IEICE Trans. Informa-tion and Systems 79-D(5) (1996) 516-521
Fukumoto, T., Wakabayashi, T., Kumura, F., Miyake, Y.: Accuracy improve-ment of handwritten character recognition by GLVQ. In: Proc. 7th IWFHR, Amsterdam (2000) 271-280
Liu, C.L.: High accuracy handwritten Chinese character recognition using quadratic classifiers with discriminative feature extraction. In: Proc. 18th ICPR, Hong Kong, 2 (2006) 942-945
Liu, H., Ding, X.: Handwritten character recognition using gradient feature and quadratic classifier with multiple discrimination schemes. In: Proc. 8th ICDAR, Seoul, Korea, 1 (2005) 19-23
Liu, H., Ding, X.: Handwritten Chinese character recognition based on mirror image learning and the compound Mahalanobis distance. J. Tsinghua Univ. (Sci & Tech) 46(7) (2006) 1239-1242 (in Chinese)
Liu, H., Ding, X.: Improve handwritten character recognition performance by heteroscedastic linear discriminant analysis, In: Proc. 18th ICPR, Hong Kong, 1(2006) 880-883
Biem, A., Katagiri, S., Juang, B.H.: Pattern recognition using discriminative feature extraction. IEEE Trans. Signal Processing 45(2) (1997) 500-504
Huo, Q., Ge, Y., Feng, Z.D.: High performance Chinese OCR based on Ga-bor features, discriminative feature extraction and model training. In: Proc. ICASSP’01, Salt Lake City, Utah, 3 (2001) 1517-1520
Liu, C.L., Mine, R., Koga, M.: Building compact classifier for large character set recognition using discriminative feature extraction. In: Proc. 8th ICDAR, Seoul, Korea (2005) 846-850
Suen, C.Y., Nadal, C., Legault, R., Mai, T.A., Lam, L.: Computer recognition of unconstrained handwritten numerals. Proc. IEEE 80(7) (1982) 1162-1180
Liu, C.L., Nakashima, K., Sako, H., Fujisawa, H.: Handwritten digit recogni-tion: investigation of normalization and feature extraction techniques. Pattern Recognition 37(2) (2004) 265-279
Teow, L.N., Loe, K.F.: Robust vision-based features and classification schemes for off-line handwritten digit recognition. Pattern Recognition 35(11) (2002) 2355-2364
Holmström, L., Koistinen, P., Laaksonen, J., Oja, E.: Neural and statistical classifiers—taxonomy and two case studies. IEEE Trans. Neural Networks 8(1) (1997) 5-17
Liu, C.L., Sako, H., Fujisawa, H.: Performance evaluation of pattern classifiers for handwritten character recognition. Int. J. Document Analysis and Recogni-tion 4(3) (2002) 191-204
Tsukumo, J., Tanaka, H.: Classification of handprinted Chinese characters using non-linear normalization and correlation methods. In: Proc. 9th ICPR, Rome (1988) 168-171
Yamada, H., Yamamoto, K., Saito, T.: A nonlinear normalization method for hanprinted Kanji character recognition—line density equalization. Pattern Recognition 23(9) (1990) 1023-1029
Gori, M., Scarselli, F.: Are multilayer perceptrons adequate for pattern recog-nition and verification? IEEE Trans. Pattern Anal. Mach. Intell. 20(11) (1998) 1121-1132
Liu, C.L., Sako, H., Fujisawa, H.: Effects of classifier structures and training regimes on integrated segmentation and recognition of handwritten numeral strings. IEEE Trans. Pattern Anal. Mach. Intell. 26(11) (2004) 1395-1407
Liu, C.L., Marukawa, K.: Handwritten numeral string recognition: Character-level training vs. string-level training. In: Proc. 17th ICPR, Cambridge, UK, 1 (2004) 405-408
Raina, R., Shen, Y., Ng, A.Y., McCallum, A.: Classification with hybrid gen-erative/discriminative models. In: Advances in Neural Information Processing System 16 (2003)
Dahmen, J., Schluter, R., Ney, H.: Discriminative training of Gaussian mixtures for image object recognition. In: Proc. 21st Symposium of German Association for Pattern Recognition, Bonn, Germany (1999) 205-212
Tax, D.M.J., Duin, R.P.W.: Support vector data description. Machine Learning 54(1)(2004) 45-66
Suen, C.Y., Tan, J.: Analysis of error of handwritten digits made by a multitude of classifiers. Pattern Recognition Letters 26(3) (2005) 369-379
Polikar, R., Udpa, L., Udpa, A.S., Honavar, V.: Learn++: An incremental learn-ing algorithm for supervised neural networks. IEEE Trans. System Man Cyber-net. Part C 31(4) (2001) 497-508
Chawla, N.V., Karakoulas, G.: Learning from labeled and unlabeled data: An empirical study across techniques and domains. J. Artificial Intelligence Re-search 23 (2005) 331-366
Duin, R.P.W.: A note on comparing classifiers. Pattern Recognition Letters, 17 (1996) 529-536
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Liu, CL., Fujisawa, H. (2008). Classification and Learning Methods for Character Recognition: Advances and Remaining Problems. In: Marinai, S., Fujisawa, H. (eds) Machine Learning in Document Analysis and Recognition. Studies in Computational Intelligence, vol 90. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-76280-5_6
Download citation
DOI: https://doi.org/10.1007/978-3-540-76280-5_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-76279-9
Online ISBN: 978-3-540-76280-5
eBook Packages: EngineeringEngineering (R0)