Bias Correction

Abstract

Bias correction is a statistical technique used to remove the bias of an estimator. An unbiased estimator is such that its expectation is equal to the parameter of interest. Many introductory statistics textbooks discuss the desirability of having an unbiased estimator, although it is quickly pointed out that unbiasedness alone cannot be a good criterion for an estimator. This is usually illustrated by comparing two estimators with the use of a concrete loss function, where it is noted that an unbiased estimator with a large variance may be inferior to a biased estimator with a small variance.

Bias correction is a statistical technique used to remove the bias of an estimator. An unbiased estimator is such that its expectation is equal to the parameter of interest. Many introductory statistics textbooks discuss the desirability of having an unbiased estimator, although it is quickly pointed out that unbiasedness alone cannot be a good criterion for an estimator. This is usually illustrated by comparing two estimators with the use of a concrete loss function, where it is noted that an unbiased estimator with a large variance may be inferior to a biased estimator with a small variance.

Analysis of exact finite sample theory is difficult, or impossible, for many estimators. Therefore, sampling properties of econometric estimators are usually discussed in the context of asymptotic approximation. Many estimators used in econometrics are consistent and asymptotically efficient, so the bias is usually a non-issue in such first-order asymptotic theory. On the other hand, the first-order asymptotic theory may fail to provide a good approximation to the exact finite sample distribution of an estimator, and even an asymptotically unbiased estimator may have a significant bias under small sample sizes. Higher-order asymptotic approximation may then be used to understand the finite sample properties, including the approximate bias. To be more specific, suppose that we use an estimator $\widehat{\theta}$ to estimate the parameter of interest θ₀. For many cases, $\widehat{\theta}$ allows a three term stochastic expansion

$$ \sqrt{n}\left(\widehat{\theta}-{\theta}_0\right)={\widehat{T}}_1+{\widehat{T}}_2/\sqrt{n}+{\widehat{T}}_3/n+{O}_p\left({n}^{-3/2}\right), $$

where n is the sample size. The higher-order asymptotic bias of $\widehat{\theta}$ is given by b0/n, where

$$ {b}_0=\underset{n\to \infty }{\lim }E\left[{\widehat{T}}_2\right]. $$

In the recent literature, bias correction is usually understood to be a method of removing such approximate bias b₀/n. These methods include analytical corrections such as the standard textbook expansion for functions of sample means, and the more complicated formulas required for other estimators. They also include jackknife and bootstrap bias corrections. Correction of approximate bias is usually accompanied by increase of variance, and early literature such as Pfanzagl and Wefelmeyer (1978) focused on the efficiency aspects of bias correction. In general, bias correction cannot be always advocated on efficiency grounds.

Bias correction has received renewed attention in the more recent literature. When there are many nuisance parameters, the parameters of interest are typically estimated with significant biases. The biases are often so severe that removal of such biases almost always results in efficiency gain. Two strands of literature deal with models with many nuisance parameters. First, when a parameter of interest is estimated with many instruments, the resultant estimator may be quite biased. For example, the two-stage least squares estimator (2SLS) tends to be severely biased when there are many first-stage coefficients to be estimated; see for example Bekker (1994). It has been noted that some estimators are not sensitive to the presence of such nuisance parameters, and the instrumental variables literature is focused on developing such robust estimators. For linear simultaneous equations models, the limited information maximum likelihood estimator (LIML) was shown to have very little bias for linear models. For nonlinear models, it was shown that the empirical likelihood (EL) estimator tends to be less biased than the generalized method of moments estimator (GMM) when there are many moment restrictions; see Newey and Smith (2004).

The second strand of literature in which bias correction has played an important role is concerned with panel models. Parameters of interest in panel models are usually estimated with substantial bias when fixed effects are estimated; see Neyman and Scott (1948). The literature examined methods of removing such bias. Hahn and Newey (2004) proposed that the bias be estimated and subtracted from the estimator itself. Arellano (2003) and Woutersen (2002) proposed that the moment equation be modified.

See Also

• Two-Stage Least Squares and The K-Class Estimator