Abstract
An unknown process is generating a sequence of symbols, drawn from an alphabet, \(\mathcal{A}\). Given an initial segment of the sequence, how can one predict the next symbol? Ray Solomonoff’s theory of inductive reasoning rests on the idea that a useful estimate of a sequence’s true probability of being outputted by the unknown process is provided by its algorithmic probability (its probability of being outputted by a species of probabilistic Turing machine). However algorithmic probability is a “semimeasure”: i.e., the sum, over all \(x\in \mathcal{A}\), of the conditional algorithmic probabilities of the next symbol being x, may be less than 1. Solomonoff thought that algorithmic probability must be normalized, to eradicate this semimeasure property, before it can yield acceptable probability estimates. This paper argues, to the contrary, that the semimeasure property contributes substantially, in its own right, to the power of an algorithmic-probability-based theory of induction, and that normalization is unnecessary.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
References
Wallace, C., Dowe, D.: Minimum message length and Kolmogorov complexity. Computer Journal 42, 270–283 (1999)
Wallace, C.: Statistical and Inductive Inference by Minimum Message Length. Information Science and Statistics. Springer (2005)
Solomonoff, R.: A formal theory of inductive inference: Parts 1 and 2. Information and Control 7, 1–22, 224–254 (1964)
Solomonoff, R.: Complexity-based induction systems: Comparisons and convergence theorems. IEEE Trans. Infor. Theory IT-24, 422–432 (1978)
Wallace, C., Boulton, D.: An information measure for classification. Computer Journal 11(2), 185–194 (1968)
Boulton, D., Wallace, C.: The information content of a multistate distribution. J. Theoret. Biol. 23, 269–278 (1969)
Boulton, D., Wallace, C.: A program for numerical classification. The Computer Journal 13(1), 63–69 (1970)
Boulton, D., Wallace, C.: An information measure for hierarchic classification. The Computer Journal 16(3), 254–261 (1973)
Wallace, C., Boulton, D.: An invariant Bayes method for point estimation. Classification Society Bull. 3(3), 11–34 (1975)
Boulton, D., Wallace, C.: An information measure for single-link classification. The Computer Journal 18(3), 236–238 (1975)
Dowe, D.L.: MML, hybrid Bayesian network graphical models, statistical consistency, invariance and uniqueness. In: Bandyopadhyay, P., Forster, M. (eds.) Philosophy of Statistics. Handbook of the Philosophy of Science, vol. 7, Elsevier (2011)
Li, M., Vitányi, P.: An introduction to Kolmogorov complexity and its applications, 2nd edn. Springer (1997)
Solomonoff, R.: Three kinds of probabilistic induction: universal distributions and convergence theorems. The Computer Journal: Christopher Stewart Wallace (1933-2004); Memorial Special Issue 51, 566–570 (2008)
Solomonoff, R.: The Kolmogorov lecture: The universal distribution and machine learning. Computer Journal 46, 598–601 (2003)
Solomonoff, R.: The discovery of algorithmic probability. Journal of Computer and System Sciences 55, 73–88 (1997)
Solomonoff, R.: Does algorithmic probability solve the problem of induction? Technical Report (1997)
Hutter, M.: On generalized computable universal priors and their convergence, IDSIA-05-05 (2005)
Solomonoff, R.: The probability of “undefined” (non-converging) output in generating the universal probability distribution. Inf. Process. Lett. 106, 238–240 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Campbell, D. (2013). The Semimeasure Property of Algorithmic Probability – “Feature” or “Bug”?. In: Dowe, D.L. (eds) Algorithmic Probability and Friends. Bayesian Prediction and Artificial Intelligence. Lecture Notes in Computer Science, vol 7070. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-44958-1_6
Download citation
DOI: https://doi.org/10.1007/978-3-642-44958-1_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-44957-4
Online ISBN: 978-3-642-44958-1
eBook Packages: Computer ScienceComputer Science (R0)