Abstract
Traditionally, when applying the two-sample t test, some pre-testing occurs. That is, the theory-based assumptions of normal distributions as well as of homogeneity of the variances are often tested in applied sciences in advance of the tried-for t test. But this paper shows that such pre-testing leads to unknown final type-I- and type-II-risks if the respective statistical tests are performed using the same set of observations. In order to get an impression of the extension of the resulting misinterpreted risks, some theoretical deductions are given and, in particular, a systematic simulation study is done. As a result, we propose that it is preferable to apply no pre-tests for the t test and no t test at all, but instead to use the Welch-test as a standard test: its power comes close to that of the t test when the variances are homogeneous, and for unequal variances and skewness values |γ 1| < 3, it keeps the so called 20% robustness whereas the t test as well as Wilcoxon’s U test cannot be recommended for most cases.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Easterling RG, Anderson HE (1978) The effect of preliminary normality goodness of fit tests on subsequent inference. J Stat Comput Simul 8: 1–11
Fleishman AJ (1978) A method for simulating non-normal distributions. Psychometrika 43: 521–532
Kolmogorov AV (1933) Sulla determinazione empirica di una legge di distribuzione. Inst Ital Attuari Gorn 4: 1–11
Levene H (1960) Robust tests for equality of variances. In: Olkin I (eds) Contributions to probability and statistics. Essays in honor of Harold Hotelling. University Press, Stanford, pp 278–292
Mann HB, Whitney DR (1947) On a test whether one of two random variables is stochastically larger than the other. Ann Math Stat 18: 50–60
Moser BK, Stevens GR (1992) Homogeneity of variance in the two-sample means test. Am Stat 46: 19–21
Rasch D, Guiard V (2004) The robustness of parametric statistical methods. Psychol Sci 46: 175–208
Rasch D, Teuscher F, Guiard V (2007a) How robust are tests for two independent samples?. J Stat Plan Inference 137: 2706–2720
Rasch D, Verdooren LR, Gowers JI (2007b) Design and analysis of experiments and surveys (2nd edn.). Oldenbourg, München
Schucany WR, Ng HKT (2006) Preliminary goodness-of-fit tests for normality do not validate the one-sample Student t. Commun Stat Theory Methods 35: 2275–2286
Smirnov VI (1939) On the estimation of the discrepancy between empirical curves of distribution for two independent samples. Bull Math Univ Moscou 2: 3–14
Welch BL (1947) The generalisation of “Student’s” problem when several different population variances are involves. Biometrika 34: 28–35
Wilcoxon F (1945) Individual comparisons by ranking methods. Biometrics 1: 80–82
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Rasch, D., Kubinger, K.D. & Moder, K. The two-sample t test: pre-testing its assumptions does not pay off. Stat Papers 52, 219–231 (2011). https://doi.org/10.1007/s00362-009-0224-x
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00362-009-0224-x