In statistics, the jackknife (jackknife cross-validation) is a cross-validation technique and, therefore, a form of resampling. It is especially useful for bias and variance estimation. The jackknife pre-dates other common resampling methods such as the bootstrap, a jackknife estimator obtained by omitting one observation. The jackknife is a linear approximation of the bootstrap.

The jackknife technique was developed by Maurice Quenouille (1924–1973) from 1949 and refined in 1956. John Tukey expanded on the technique in 1958 and proposed the name "jackknife" because, like a physical jack-knife (a compact folding knife), it is a rough-and-ready tool that can improvise a solution for a variety of problems even though specific problems may be more efficiently solved with a purpose-designed tool.

A simple example: mean estimation

The jackknife estimator of a parameter is found by systematically leaving out each observation from a dataset and calculating the parameter estimate over the remaining observations and then aggregating these calculations.

For example, if the parameter to be estimated is the population mean of random variable , then for a given set of i.i.d..

is more involved, since the jackknife replicates are not independent.

. However, these properties do not generally hold for parameters other than the mean.

This simple example for the case of mean estimation is just to illustrate the construction of a jackknife estimator, while the real subtleties (and the usefulness) emerge for the case of estimating other parameters, such as higher moments than the mean or other functionals of the distribution.

, so this construction does not add any meaningful knowledge, but it gives the correct estimation of the bias (which is zero).

:.

Estimating the bias of an estimator

The jackknife technique can be used to estimate (and correct) the bias of an estimator calculated over the entire sample.

, which is assumed to consist of i.i.d. is constructed:

is sample-dependent, so this value will change from one random sample to another.

is as follows:

. In this kind of situation the jackknife resampling technique may be of help.

We construct the jackknife replicates:

where each replicate is a "leave-one-out" estimate based on the jackknife subsample consisting of all but one of the data points:

Then we define their average:

is given by:

is given by:

in other cases.

Estimating the variance of an estimator

The jackknife technique can be also used to estimate the variance of an estimator calculated over the entire sample.

Literature

• Shao, J. and Tu, D. (1995). The Jackknife and Bootstrap. Springer-Verlag, Inc.

Notes

References