Abstract
This versatile topic goes back to the inventions of Gauss, Markov, and Gibbs, whose ideas are incorporated in graphical models and regression graphs. Later, the geneticist S. Wright (1923–1934) and the philosopher and computer scientist J. Pearl (1986–1987) developed the tools, but their notation is too complicated to formulate the mathematical background. Here we mainly follow the up-to-date discussion of statisticians S. Lauritzen and N. Wermuth, and try to juxtapose the directed–undirected and discrete–continuous cases.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Abbreviations
- An(A):
-
ancestral set of the vertex-set A, which is the smallest possible vertex-set (including A) containing all vertices from where a directed path emanates to vertices of A in a directed graph
- ant(i):
-
anteriors of the vertex i in a directed graphs (non-descendants except its parents)
- bd(i):
-
boundary of the vertex i (its neighbors in the undirected, and its parents in the directed case)
- BN:
-
Bayesian Network
- CG:
-
Conditional Gaussian
- cl(i):
-
closure of the vertex i (it and its biundary)
- DAG:
-
Directed Acyclic Graph
- DF:
-
Directed Factorization Property
- DG:
-
Directed Global Markov Property
- DL:
-
Directed Local Markov Property
- DP:
-
Directed Pairwise Markov Property
- EDHS:
-
Egypt Demographic and Health Survey
- iid:
-
independent identically distributed
- IPS:
-
Iterative Proportional Scaling
- JT:
-
Junction Tree
- MCS:
-
Maximal Cardinality Search
- ML:
-
Maximum Likelihood
- MRF:
-
Markov Random Field
- par(i):
-
parents of the vertex i (from where directed edge shows to it) in a directed graph
- pdf:
-
Probability Density Function
- pmf:
-
Probability Mass Function
- RCF:
-
Recursive Casual Factorization
- RIP:
-
Running Intersection Property
- rv:
-
random variable
- RZP:
-
Reducible Zero Pattern
- SEM:
-
Structural Equation Modeling
- UF:
-
Undirected Factorization Property
- UG:
-
Undirected Global Markov Property
- UL:
-
Undirected Local Markov Property
- UP:
-
Undirected Pairwise Markov Property
References
M. Bolla, Spectral Clustering and Biclustering. Learning Large Graphs and Contingency Tables, Wiley, 2013.
L. Breiman and J. H. Friedman, Estimating optimal transformations for multiple regression and correlation, J. Am. Stat. Assoc., 80 (1985), 580–619.
C. K. Chow and C. N. Liu, Approximating discrete probability distributions with dependence trees, IEE. Trans. Inf. Theory, IT-14 (1968), 452–467.
W. G. Cochran, The omission or addition of an independent variate in multiple linear regression, J. R. Statist. Soc., suppl. 5 (1938), 171–176.
D. R. Cox and N. Wermuth, Multivariate Dependencies. Models, analysis and interpretation, Chapman&Hall/CRC, 1996.
J. N. Darroch, S. L. Lauritzen and T. P. Speed, Markov Fields and Log- Linear Interaction Models for Contingency tables, The Annals of Statistics, 8 (1980), 522–539.
A. P. Dempster, Covariance selection, Biometrics, 28 (1972), 157–175.
L. Győrfi, et al., A distribution-free theory of nonparametric regression, Springer Science & Business Media, 2006.
K. G. Jöreskog, Structural equation models in the social sciences, Specification, estimation and testing (1977), Applications of statistics, ed.: P. R. Krishnaiah, North-Holland Publishing Co., 265–287.
H. Kiiveri, T. P. Speed and J. B. Carlin, Recursive casual models, J. Austral. Math. Soc. (Ser. A), 36 (1984), 30–52.
D. Koller and N. Friedman, Probabilistic Graphical Models. Principles and Techniques, MI. Press, 2009.
S. L. Lauritzen, T. P. Speed and K. Vijayan, Decomposable graphs and hypergraphs, J. Austral. Math. Soc. (Ser A), 36 (1984), 12–29.
S. L. Lauritzen and D. Spiegelhalter, Local computations with probabilities on graphical structures and their application in expert systems, J. R. Statist. Soc. B, 50 (1988), 157–224.
S. L. Lauritzen, Graphical Models, Oxford Univ. Press, 1995.
L. Lovász, Combinatorial problems and exercises, ACM. Akademiai Kiado–North Holland, Budapest–Amsterdam, 1993.
R. Lnenicka and F. Matus, On Gaussian conditional independence structutes, Kybernetika, 43 (2007), 323–342.
F. T. Móri and J. G. Székely, Többváltozós statisztikai analízis, Műszaki Konyvkiado, Budapest, 1986 (in Hungarian, Chapter XI. by T. Rudas).
E. A. Nadaraya, On nonparametric estimates of regression functions and regression curves, Theory of Applied Probability, 10 (1965), 186–190.
J. Pearl, Causality: Models, Reasoning and Inference, Cambridge Univ. Press, 2000.
D. J. Rose, R. E. Tarjan and G. S. Lueker, Algorithmic aspects of vertex elimination on graphs, SIA. Journal of Computing, 5 (1976), 266–283.
R. Sundberg, Some results about decomposable (or Markov-type) models for multidimensional contingency tables: distribution of marginals and partitioning of tests, Scandinavian Journal of Statistics, 2 (1975), 71–79.
T. Szántai and E. Kovács, Application of t-cherry trees in pattern recognition, Broad Research in Artificial Intelligence and Neuroscience (BRAIN), eds.: B. Iantovics, D. Radoiu, M. Marusteri, M. Dehmer, special issue on Complexity in Sciences and Artificial Intelligence, 2010, 40–45.
R. E. Tarjan and M. Yannakakis, Simple linear-time algorithms to test chordality of graphs, test acyclicity of hypergraphs and selectively reduce acyclic hypergraphs, Siam J. Computing, 13 (1984), 566–579.
M. Tenenhaus, V. Esposito Vinzi, Y.-L. Chatelinc and C. Lauro, PL. path modeling, Computational Statistics & Data Analysis, 48 (2005), 159–205.
M. J. Wainwright, Graphical Models and Message-Passing Algorithms: Some Introductory Lectures, Mathematical Foundations of Complex Networked Iformation Systems, Lecture Notes in Mathematics 2141, Springer, 2015.
M. Wainwright and M. I. Jordan, Graphical models, exponential families, and variational inference, Foundations and Trends in Machine Learning, 1 (2008), 1–305.
G. S. Watson, Smooth regression analysis, Sankhya, 26 (1964), 359–372.
N. Wermuth, Recursive equations, covariance selection, and path analysis, J. Amer. Stat. Assoc., 75 (1980), 963–972.
N. Wermuth, Traceable regressions, International Statistical Review, 80 (2012), 415–438.
N. Wermuth and K. Sadeghi, Sequences of regressions and their independences, TEST, 21 (2012), 215–279.
N. Wermuth, Graphical Markov models, unifying results and their interpretation, Wiley StatsRef: Statistics Reference Online, eds.: N. Balakrishnan, et al., 2015, also available at arXiv: 1505. 02456.
N. Wermuth and D. R. Cox, Graphical Markov models: Overview, International Encyclopedia of the Social and Behavioral Sciences 10, 2nd ed., (eds.: J. Wright), Elsevier, Oxford, 2015, 341–350; also available at arXiv: 1407. 7783.
S. Wright, The method of path coefficients, Ann. Math. Stat., 5 (1934), 161–215.
Author information
Authors and Affiliations
Corresponding author
Additional information
Dedicated to the memory of András Krámli
Communicated by Gy. Pap
Acknowledgment.
The first author is indebted to NannyWermuth for her valuable explanations.
Rights and permissions
About this article
Cite this article
Bolla, M., Abdelkhalek, F. & Baranyi, M. Graphical models, regression graphs, and recursive linear regression in a unified way. ActaSci.Math. 85, 9–57 (2019). https://doi.org/10.14232/actasm-018-331-4
Received:
Published:
Issue Date:
DOI: https://doi.org/10.14232/actasm-018-331-4