Abstract
We investigate extreme studentized and normalized residuals as test statistics for outlier detection in the Gauss–Markov model possibly not of full rank. We show how critical values (quantile values) of such test statistics are derived from the probability distribution of a single studentized or normalized residual by dividing the level of error probability by the number of residuals. This derivation neglects dependencies between the residuals. We suggest improving this by a procedure based on the Monte Carlo method for the numerical computation of such critical values up to arbitrary precision. Results for free leveling networks reveal significant differences to the values used so far. We also show how to compute those critical values for non-normal error distributions. The results prove that the critical values are very sensitive to the type of error distribution.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Aliosmanoglu S, Akyilmaz O (2001) A comparison between statistical and fuzzy techniques in outlier detection. International association of geodesy symposia symposia, vol 125. Springer, Berlin, pp 382–387
Alkhatib H, Neumann I, Kutterer H (2009) Uncertainty modeling of random and systematic errors by means of Monte Carlo and fuzzy techniques. J Appl Geod 3(2): 67–79
Andersen R (2008) Modern methods for robust regression. Sage Publications, London
Baarda W (1968) A testing procedure for use in geodetic networks. Netherlands Geodetic Commission, Publication on Geodesy, vol 2, No. 5, Delft, The Netherlands
Box GEP, Muller ME (1958) A note on the generation of random normal deviates. Ann. Math. Stat. 29(2): 610–611
Hawkins D (1980) Identification of outliers. Chapman and Hall, London
Koch KR (1999) Parameter estimation and hypothesis testing in linear models. Springer, Berlin
Koch KR (2007) Outlier detection in observations including leverage points by Monte Carlo simulations. Allgemeine Vermessungsnachrichten. VDE Verlag Berlin Offenbach
Lemeshko BY, Lemeshko SB (2005) Extending the application of Grubbs-type tests in rejecting anomalous measurements. Meas Tech 48(6): 536–547
Lehmann R (1994) Adjustment in non-linear models by means of the adaptive Monte-Carlo-Integration. Allgemeine Vermessungsnachrichten, vol 7/1994. Herbert Wichmann Verlag GmbH Heidelberg (in German)
Lehmann R (2010) Normalized residuals—how large is too large? Allgemeine Vermessungsnachrichten, vol 2/2010. VDE Verlag Berlin Offenbach (in German)
Lehmann R (2012) Geodetic error calculus by the scale contaminated normal distribution. Allgemeine Vermessungsnachrichten. VDE Verlag Berlin Offenbach (in German) (in press)
Lehmann R, Scheffler T (2011) Monte Carlo based data snooping with application to a geodetic network. J Appl Geod 5(3–4): 123–134
Pope AJ (1976) The statistics of residuals and the detection of outliers. NOAA Technical Report NOS65 NGS1, US Department of Commerce, National Geodetic Survey Rockville, Maryland
Stefansky W (1972) Rejecting outliers in factorial designs. Technometrics 14: 469–479
Tanizaki H (2004) Computational methods in statistics and econometrics. Marcel Dekker, New York
Verma SP, Quiroz-Ruiz A, Díaz-González L (2008) Critical values for 33 discordancy test variants for outliers in normal samples up to sizes 1000, and applications in quality control in Earth Sciences. Revista Mexicana de Ciencias Geológicas 25(1): 82–96
Wieser A (2001) Robust and fuzzy techniques for parameter estimation and quality assessment in GPS. Ph.D. dissertation, Graz University of Technology, Austria
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lehmann, R. Improved critical values for extreme normalized and studentized residuals in Gauss–Markov models. J Geod 86, 1137–1146 (2012). https://doi.org/10.1007/s00190-012-0569-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00190-012-0569-0