Abstract
The main goal of both Bayesian model selection and classical hypotheses testing is to make inferences with respect to the state of affairs in a population of interest. The main differences between both approaches are the explicit use of prior information by Bayesians, and the explicit use of null distributions by the classicists. Formalization of prior information in prior distributions is often difficult. In this paper two practical approaches (encompassing priors and training data) to specify prior distributions will be presented. The computation of null distributions is relatively easy. However, as will be illustrated, a straightforward interpretation of the resulting p-values is not always easy. Bayesian model selection can be used to compute posterior probabilities for each of a number of competing models. This provides an alternative for the currently prevalent testing of hypotheses using p-values. Both approaches will be compared and illustrated using case studies. Each case study fits in the framework of the normal linear model, that is, analysis of variance and multiple regression.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
M.J. Bayarri J.O. Berger (2000) ArticleTitle P-values for composite null models Journal of the American Statistical Association 95 1127–1142 Occurrence Handle10.2307/2669749
J.O. Berger L. Perricchi (1996) ArticleTitleThe intrinsic bayes factor for model selection prediction Journal of the American Statistical Association 9 109–122 Occurrence Handle10.2307/2291387
J.O. Berger T. Sellke (1987) ArticleTitleTesting a point null hypothesis: the irreconcilability of p-values and evidence Journal of the American Statistical Association 82 112–122 Occurrence Handle10.2307/2289131
A. Camstra A. Boomsma (1992) ArticleTitleCross-validation in regression and covariance structure analysis Sociological Methods and Research 21 89–95
B.P. Carlin S. Chib (1995) ArticleTitleBayesian model choice via Markov Chain Monte Carlo methods Journal of the Royal Statistical Society, B 57 473–484
P. Congdon (2001) Bayesian Statistical Modelling John Wiley and Sons New York
J. Cohen (1994) ArticleTitleThe earth is round (p < 0.05) American Psychologist 12 997–1003 Occurrence Handle10.1037/0003-066X.49.12.997
C.M. Dayton (2003) ArticleTitleInformation criteria for pairwise comparisons Psychological Methods 8 61–71 Occurrence Handle10.1037/1082-989X.8.1.61
R.W. Frick (1996) ArticleTitleThe appropriate use of null hypothesis testing Psychological Methods 1 379–390 Occurrence Handle10.1037/1082-989X.1.4.379
A. Gelman J.B. Carlin H.S. Stern D.B. Rubin (1995) Bayesian Data Analysis Chapmann and Hall London
C. Howson (2002) Bayesianism in statistics R. Swinburne (Eds) Bayes Theorem Oxford University Press Oxford 39–69
R.E. Kass A.E. Raftery (1995) ArticleTitleBayes factors Journal of the American Statistical Association 90 773–795 Occurrence Handle10.2307/2291091
I. Klugkist B. Kato H. Hoijtink (2005) ArticleTitleBayesian model selection using encompassing priors Statistica Neerlandica 59 57–69 Occurrence Handle10.1111/j.1467-9574.2005.00279.x
M.A. Newton A.E. Raftery (1994) ArticleTitleApproximate Bayesian inference by the weighted likelihood bootstrap Journal of the Royal Statistical Society B 56 3–48
P.H. Ramsey (2002) ArticleTitleComparison of closed testing procedures for pairwise testing of means Psychological Methods 7 504–523 Occurrence Handle10.1037/1082-989X.7.4.504
T. Robertson F.T. Wright R.L. Dykstra (1988) Order Restricted Statistical Inference John Wiley and Sons New York
T. Sellke M.J. Bayarri J.O. Berger (2001) ArticleTitleCalibration of p values for testing precise null hypotheses The American Statistician 55 62–71 Occurrence Handle10.1198/000313001300339950
E. Sober (2002) Bayesianism–its scope and limits R. Swinburne (Eds) Bayes Theorem Oxford University Press Oxford 21–38
J. Stevens (1992) Applied Multivariate Statistics for the Social Sciences Lawrence Erlbaum London
B.G. Tabachnick L.S. Fidell (2001) Using Multivariate Statistics Allyn and Bacon London
L.E. Toothaker (1993) Multiple Comparison Procedures SAGE London
H. Wainer (1999) ArticleTitleOne cheer for null hypothesis significance testing Psychological Methods 4 212–213 Occurrence Handle10.1037/1082-989X.4.2.212
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hoijtink, H., Klugkist, I. Comparison of Hypothesis Testing and Bayesian Model Selection. Qual Quant 41, 73–91 (2007). https://doi.org/10.1007/s11135-005-6224-6
Issue Date:
DOI: https://doi.org/10.1007/s11135-005-6224-6