Abstract
Probabilistic inductive logic programming aka. statistical relational learning addresses one of the central questions of artificial intelligence: the integration of probabilistic reasoning with machine learning and first order and relational logic representations. A rich variety of different formalisms and learning techniques have been developed. A unifying characterization of the underlying learning settings, however, is missing so far.
In this chapter, we start from inductive logic programming and sketch how the inductive logic programming formalisms, settings and techniques can be extended to the statistical case. More precisely, we outline three classical settings for inductive logic programming, namely learning from entailment, learning from interpretations, and learning from proofs or traces, and show how they can be adapted to cover state-of-the-art statistical relational learning approaches.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
- Bayesian Network
- Logic Program
- Inductive Logic
- Inductive Logic Programming
- Conditional Probability Distribution
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Abney, S.P.: Stochastic Attribute-Value Grammars. Computational Linguistics 23(4), 597–618 (1997)
Anderson, C.R., Domingos, P., Weld, D.S.: Relational Markov Models and their Application to Adaptive Web Navigation. In: Hand, D., Keim, D., Ng, R. (eds.) Proceedings of the Eighth International Conference on Knowledge Discovery and Data Mining (KDD 2002), Edmonton, Canada, July 2002, pp. 143–152. ACM Press, New York (2002)
Bangsø, O., Langseth, H., Nielsen, T.D.: Structural learning in object oriented domains. In: Russell, I., Kolen, J. (eds.) Proceedings of the Fourteenth International Florida Artificial Intelligence Research Society Conference (FLAIRS 2001), Key West, Florida, USA, pp. 340–344. AAAI Press, Menlo Park (2001)
Cussens, J.: Loglinear models for first-order probabilistic reasoning. In: Laskey, K.B., Prade, H. (eds.) Proceedings of the Fifteenth Annual Conference on Uncertainty in Artificial Intelligence (UAI 1999), Stockholm, Sweden, pp. 126–133. Morgan Kaufmann, San Francisco (1999)
Cussens, J.: Parameter estimation in stochastic logic programs. Machine Learning Journal 44(3), 245–271 (2001)
Cussens, J.: Integrating by separating: Combining probability and logic with ICL, PRISM and SLPs. Technical report, APrIL Projetc (January 2005)
De Raedt, L.: Logical settings for concept-learning. Artificial Intelligence Journal 95(1), 197–201 (1997)
De Raedt, L., Džeroski, S.: First-Order jk-Clausal Theories are PAC-Learnable. Artificial Intelligence Journal 70(1-2), 375–392 (1994)
De Raedt, L., Kersting, K.: Probabilistic Logic Learning. ACM-SIGKDD Explorations: Special issue on Multi-Relational Data Mining 5(1), 31–48 (2003)
De Raedt, L., Kersting, K., Torge, S.: Towards learning stochastic logic programs from proof-banks. In: Veloso, M., Kambhampati, S. (eds.) Proceedings of the Twentieth National Conference on Artificial Intelligence (AAAI 2005), Pittsburgh, Pennsylvania, USA, July 9–13, 2005, pp. 752–757. AAAI (2005)
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B 39, 1–39 (1977)
Domingos, P., Richardson, M.: Markov Logic: A Unifying Framework for Statistical Relational Learning. In: Dietterich, T.G., Getoor, L., Murphy, K. (eds.) Proceedings of the ICML-2004 Workshop on Statistical Relational Learning and its Connections to Other Fields (SRL 2004), Banff, Alberta, Canada, July 8, 2004, pp. 49–54 (2004)
Džeroski, S., Lavrač, N. (eds.): Relational data mining. Springer, Berlin (2001)
Eisele, A.: Towards Probabilistic Extensions of Contraint-based Grammars. In: Dörne, J. (ed.) Computational Aspects of Constraint-Based Linguistics Decription-II, DYNA-2 deliverable R1.2.B (1994)
Flach, P.: Simply Logical: Intelligent Reasoning by Example. John Wiley, Chichester (1994)
Flach, P.A., Lachiche, N.: Naive Bayesian classification of structured data. Machine Learning Journal 57(3), 233–269 (2004)
Getoor, L.: Learning Statistical Models from Relational Data. PhD thesis, Stanford University, USA (June 2001)
Getoor, L., Friedman, N., Koller, D., Taskar, B.: Learning Probabilistic Models of Link Structure. Journal of Machine Leaning Research (JMLR) 3, 679–707 (2002)
Heckerman, D.: A Tutorial on Learning with Bayesian Networks. Technical Report MSR-TR-95-06, Microsoft Research (1995)
Helft, N.: Induction as nonmonotonic inference. In: Brachman, R.J., Levesque, H.J. (eds.) Proceedings of the First International Conference on Principles of Knowledge Representation and Reasoning(KR 1989), Toronto, Canada, May 15-18, 1989, pp. 149–156. Morgan Kaufmann, San Francisco (1989)
Jäger, M.: Relational Bayesian Networks. In: Laskey, K.B., Prade, H. (eds.) Proceedings of the Thirteenth Conference on Uncertainty in Artificial Intelligence (UAI 1997), Stockholm, Sweden, July 30–August 1, 1997, pp. 266–273. Morgan Kaufmann, San Francisco (1997)
Jensen, F.V.: Bayesian networks and decision graphs. Springer, Heidelberg (2001)
Kameya, Y., Sato, T., Zhou, N.-G.: Yet more efficient EM learning for parameterized logic programs by inter goal sharing. In: de Mantaras, R.L., Saitta, L. (eds.) Proceedings of the 16th European Conference on Artificial Intelligence (ECAI 2004), Valencia, Spain, August 22-27, 2004, pp. 490–494. IOS Press, Amsterdam (2004)
Kersting, K.: Bayes’sche-logische Programme. Master’s thesis, Institute for Computer Science, University of Freiburg (2000)
Kersting, K., De Raedt, L.: Bayesian logic programs. Technical Report 151, Institute for Computer Science, University of Freiburg, Freiburg, Germany (April 2001)
Kersting, K., De Raedt, L.: Towards Combining Inductive Logic Programming with Bayesian Networks. In: Rouveirol, C., Sebag, M. (eds.) ILP 2001. LNCS (LNAI), vol. 2157, pp. 118–131. Springer, Heidelberg (2001)
Kersting, K., De Raedt, L.: Bayesian Logic Programming: Theory and Tool. In: Getoor, L., Taskar, B. (eds.) An Introduction to Statistical Relational Learning, pp. 291–321. MIT Press, Cambridge (2007)
Kersting, K., De Raedt, L., Raiko, T.: Logial Hidden Markov Models. Journal of Artificial Intelligence Research (JAIR) 25, 425–456 (2006)
Kersting, K., Raiko, T.: ’Say EM’ for Selecting Probabilistic Models for Logical Sequences. In: Bacchus, F., Jaakkola, T. (eds.) Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence (UAI 2005), Edinburgh, Scotland, July 26-29, 2005, pp. 300–307 (2005)
Koller, D., Pfeffer, A.: Learning probabilities for noisy first order rules. In: Pollack, M. (ed.) Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence (IJCAI 1997), Nagoya, Japan, pp. 1316–1321. Morgan Kaufmann, San Francisco (1997)
Landwehr, N., Kersting, K., De Raedt, L.: nFOIL: Integrating Naïve Bayes and Foil. In: Veloso, M., Kambhampati, S. (eds.) Proceedings of the Twentieth National Conference on Artificial Intelligence (AAAI 2005), Pittsburgh, Pennsylvania, USA, July 9–13, 2005, pp. 795–800. AAAI Press, Menlo Park (2005)
Lavrač, N., Džeroski, S.: Inductive Logic Programming. Ellis Horwood (1994)
Lloyd, J.W.: Foundations of Logic Programming, 2nd edn. Springer, Berlin (1989)
Manning, C.H., Schütze, H.: Foundations of Statistical Natural Language Processing. MIT Press, Cambridge (1999)
Marcus, M., Kim, G., Marcinkiewicz, M.A., MacIntyre, R., Bies, A., Ferguson, M., Katz, K., Schasberger, B.: The Penn treebank: Annotating predicate argument structure. In: Weinstein, C.J. (ed.) In ARPA Human Language Technology Workshop, Plainsboro, NJ, USA, March 8–11, 1994, pp. 114–119 (1994)
McLachlan, G., Krishnan, T.: The EM Algorithm and Extensions. Wiley, New York (1997)
Muggleton, S.H.: Inverse Entailment and Progol. New Generation Computing Journal, 245–286 (1995)
Muggleton, S.H.: Stochastic logic programs. In: De Raedt, L. (ed.) Advances in Inductive Logic Programming, pp. 254–264. IOS Press, Amsterdam (1996)
Muggleton, S.H.: Learning Stochastic Logic Programs. Electronic Transactions in Artificial Intelligence 4(041) (2000)
Muggleton, S.H.: Learning stochastic logic programs. In: Getoor, L., Jensen, D. (eds.) Working Notes of the AAAI-2000 Workshop on Learning Statistical Models from Relational Data (SRL 2000), Austin, Texas, July 31, 2000, pp. 36–41. AAAI Press, Menlo Park (2000)
Muggleton, S.H.: Learning Structure and Parameters of Stochastic Logic Programs. In: Matwin, S., Sammut, C. (eds.) ILP 2002. LNCS (LNAI), vol. 2583, Springer, Heidelberg (2003)
Muggleton, S.H., Feng, C.: Efficient Induction of Logic Programs. In: Muggleton, S.H. (ed.) Inductive Logic Programming, Acadamic Press (1992)
Muggleton, S.H., De Raedt, L.: Inductive Logic Programming: Theory and Methods. Journal of Logic Programming 19(20), 629–679 (1994)
Neville, J., Jensen, D.: Dependency Networks for Relational Data. In: Rastogi, R., Morik, K., Bramer, M., Wu, X. (eds.) Proceedings of The Fourth IEEE International Conference on Data Mining (ICDM 2004), Brighton, UK, November 1–4, 2004, pp. 170–177. IEEE Computer Society Press, Los Alamitos (2004)
Ngo, L., Haddawy, P.: Answering Queries from Context-Sensitive Probabilistic Knowledge Bases. Theoretical Computer Science 171, 147–177 (1997)
Pearl, J.: Reasoning in Intelligent Systems: Networks of Plausible Inference, 2nd edn. Morgan Kaufmann, San Francisco (1991)
Pfeffer, A.J.: Probabilistic Reasoning for Complex Systems. PhD thesis, Computer Science Department, Stanford University (December 2000)
Plotkin, G.D.: A note on inductive generalization. In: Machine Intelligence, vol. 5, pp. 153–163. Edinburgh University Press (1970)
Poole, D.: Probabilistic Horn abduction and Bayesian networks. Artificial Intelligence Journal 64, 81–129 (1993)
Quinlan, J.R., Cameron-Jones, R.M.: Induction of logic programs: FOIL and related systems. New Generation Computing, 287–312 (1995)
Riguzzi, F.: Learning logic programs with annotated disjunctions. In: Camacho, R., King, R., Srinivasan, A. (eds.) ILP 2004. LNCS (LNAI), vol. 3194, pp. 270–287. Springer, Heidelberg (2004)
Russell, S.J., Norvig, P.: Artificial Intelligence: A Modern Approach, 2nd edn. Prentice-Hall, Inc., Englewood Cliffs (2002)
Sato, T.: A Statistical Learning Method for Logic Programs with Distribution Semantics. In: Sterling, L. (ed.) Proceedings of the Twelfth International Conference on Logic Programming (ICLP 1995), Tokyo, Japan, pp. 715–729. MIT Press, Cambridge (1995)
Sato, T., Kameya, Y.: Parameter Learning of Logic Programs for Symbolic-Statistical Modeling. Journal of Artificial Intelligence Research (JAIR) 15, 391–454 (2001)
Shapiro, E.: Algorithmic Program Debugging. MIT Press, Cambridge (1983)
Srinivasan, A.: The Aleph Manual (1999), Available at: http://www.comlab.ox.ac.uk/oucl/~research/areas/machlearn/Aleph/
Srinivasan, A., Muggleton, S.H., King, R.D., Sternberg, M.J.E.: Theories for Mutagenicity: A Study of First-Order and Feature based Induction. Artificial Intelligence Journal 85, 277–299 (1996)
Sterling, L., Shapiro, E.: The Art of Prolog: Advanced Programming Techniques. MIT Press, Cambridge (1986)
Stolcke, A., Omohundro, S.: Hidden Markov model induction by Bayesian model merging. In: Hanson, S.J., Cowan, J.D., Giles, C.L. (eds.) Advances in Neural Information Processing Systems, vol. 5, pp. 11–18. Morgan Kaufmann, San Francisco (1993); (Proceedings of NIPS-92, Denver, Colorado, USA, November 30–December 3 (1992)
Taskar, B., Abbeel, P., Koller, D.: Discriminative Probabilistic Models for Relational Data. In: Darwiche, A., Friedman, N. (eds.) Proceedings of the Eighteenth Conference on Uncertainty in Artificial Intelligence (UAI 2002), Edmonton, Alberta, Canada, August 1-4, 2002, pp. 485–492 (2002)
Valiant, L.G.: A theory of the Learnable. Communications of the ACM 27(11), 1134–1142 (1984)
Vennekens, J., Verbaeten, S., Bruynooghe, M.: Logic Programs with Annotated Disjunctions. In: Demoen, B., Lifschitz, V. (eds.) Proceedings of 20th International Conference on Logic Programming (ICLP 2004), Saint-Malo, France, September 6-10, 2004, pp. 431–445 (2004)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
De Raedt, L., Kersting, K. (2008). Probabilistic Inductive Logic Programming. In: De Raedt, L., Frasconi, P., Kersting, K., Muggleton, S. (eds) Probabilistic Inductive Logic Programming. Lecture Notes in Computer Science(), vol 4911. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-78652-8_1
Download citation
DOI: https://doi.org/10.1007/978-3-540-78652-8_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-78651-1
Online ISBN: 978-3-540-78652-8
eBook Packages: Computer ScienceComputer Science (R0)