Abstract
The performance of the error backpropagation (BP) and ID3 learning algorithms was compared on the task of mapping English text to phonemes and stresses. Under the distributed output code developed by Sejnowski and Rosenberg, it is shown that BP consistently out-performs ID3 on this task by several percentage points. Three hypotheses explaining this difference were explored: (a) ID3 is overfitting the training data, (b) BP is able to share hidden units across several output units and hence can learn the output units better, and (c) BP captures statistical information that ID3 does not. We conclude that only hypothesis (c) is correct. By augmenting ID3 with a simple statistical learning procedure, the performance of BP can be closely matched. More complex statistical procedures can improve the performance of both BP and ID3 substantially in this domain.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Bakiri, G. (1991). Converting English text to speech: A machine learning approach. Doctoral Dissertation (Technical Report 91-30-1). Corvallis, OR: Oregon State University, Department of Computer Science.
Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and regression trees. Monterey, CA: Wadsworth and Brooks.
Buntine, W. (1990). A theory of learning classification rules. Doctoral dissertation. School of Computing Science, University of Technology, Sydney, Australia.
Dietterich, T. G. (1989). Limits of inductive learning. In Proceedings of the Sixth International Conference on Machine Learning (pp. 124–128). Ithaca, NY. San Mateo, CA: Morgan Kaufmann.
Dietterich, T. G., & Bakiri, G. (1991). Error-correcting output codes: A general method for improving multiclass inductive learning programs. Proceedings of the Ninth National Conference on Artificial Intelligence, Anaheim, CA: AAAI Press.
Dietterich, T. G., Hild, H., & Bakiri, G. (1990). A comparative study of ID3 and backpropagation for English text-to-speech mapping. Proceedings of the Seventh International Conference on Machine Learning (pp. 24–31). Austin, TX: Morgan Kaufmann.
Klatt, D. (1987). Review of text-to-speech conversion for English. J. Acoust. Soc. Am., 82, 737–793.
Lucassen, J. M., & Mercer, R. L. (1984). An information theoretic approach to the automatic determination of phonemic base forms. Proc. Int. Conf. Acoust. Speech Signal Process. ICASSP-84, 42.5.1–42.5.4.
Lang, K. J., Waibel, A. H., & Hinton, G. E. (1990). A time-delay neural network architecture for isolated word recognition. Neural Networks, 3, 33–43.
Martin, G. L., & Pittman, J. A. (1990). Recognizing hand-printed letters and digits. In D. Touretzky (Ed.), Advances in Neural Information Processing Systems 2, 405–414. San Mateo, CA: Morgan Kaufmann.
McClelland, J. L., & Rumelhart, D. E. (1988). Explorations in parallel distributed processing, Cambridge, MA: MIT Press.
Mingers, J. (1989). An empirical comparison of pruning methods for decision tree induction. Machine Learning, 4, 227–243.
Mooney, R., Shavlik, J., Towell, G., & Gove, A. (1989). An experimental comparison of symbolic and connectionist learning algorithms. IJCAI-89: Eleventh International Joint Conference on Artificial Intelligence, (pp. 775–80).
Quinlan, J. R. (1983). Learning efficient classification procedures and their application to chess endgames. In R. S. Michalski, J. Carbonell, & T. M. Mitchell, (eds.), Machine learning: An artificial intelligence approach, 1, San Mateo, CA: Morgan Kaufmann.
Quinlan, J. R. (1986a). The effect of noise on concept learning. In R. S. Michalski, J. Carbonell, & T. M. Mitchell, (eds.), Machine learning: An artificial intelligence approach, 1, San Mateo, CA: Morgan Kaufmann.
Quinlan, J. R. (1986b). Induction of decision trees, Machine Learning, 1, 81–106.
Quinlan, J. R. (1987). Simplifying decision trees. International Journal of Man-Machine Studies, 27, 221–234.
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representations by error propagation. In D. E. Rumelhart, & J. L. McClelland (Eds.), Parallel distributed processing, (Vol 1). Cambridge, MA: MIT Press.
Rosenberg, C. R. (1988). Learning the connection between spelling and sound: A network model of oral reading. Doctoral Dissertation. (CSL Report 18). Princeton, NJ: Princeton University, Cognitive Science Laboratory.
Sejnowski, T. J., & Rosenberg, C. R. (1987). Parallel networks that learn to pronounce English text. Complex Systems, 1, 145–168.
Touretzky, D. S. (Ed.) (1989). Advances in neural information processing systems 1. San Mateo, CA: Morgan Kaufmann.
Touretzky, D. S. (Ed.) (1990). Advances in neural information processing systems 2. San Mateo, CA: Morgan Kaufmann.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Dietterich, T.G., Hild, H. & Bakiri, G. A Comparison of ID3 and Backpropagation for English Text-To-Speech Mapping. Machine Learning 18, 51–80 (1995). https://doi.org/10.1023/A:1022822623726
Issue Date:
DOI: https://doi.org/10.1023/A:1022822623726