In statistics, Cook's distance or Cook's D is a commonly used estimate of the influence of a data point when performing a least-squares regression analysis. In a practical ordinary least squares analysis, Cook's distance can be used in several ways: to indicate influential data points that are particularly worth checking for validity; or to indicate regions of the design space where it would be good to be able to obtain more data points. It is named after the American statistician R. Dennis Cook, who introduced the concept in 1977.

Definition

Data points with large residuals (outliers) and/or high leverage may distort the outcome and accuracy of a regression. Cook's distance measures the effect of deleting a given observation. Points with a large Cook's distance are considered to merit closer examination in the analysis.

For the algebraic expression, first define

is the error term is the coefficient matrix is the design matrix including a constant. The least squares are

is the projection matrix, is known as the leverage.

is removed from it

where p is the mean squared error of the regression model.

Equivalently, it can be expressed using the leverage):

Detecting highly influential observations

There are different opinions regarding what cut-off values to use for spotting highly influential points. Since Cook's distance is in the metric of an F distribution) can be used as a cut-off. has been suggested.

Note that in general the Cook's distance does not measure correctly the influence of observations.

Relationship to other influence measures (and interpretation)

can be expressed using the leverage) and the square of the internally Studentized residual), as follows:

).

is related to DFFITS is the externally are defined here):

can be interpreted as the distance one's estimates move within the confidence ellipsoid that represents a region of plausible values for the parameters. This is shown by an alternative but equivalent representation of Cook's distance in terms of changes to the estimates of the regression parameters between the cases, where the particular observation is either included or excluded from the regression analysis.

-th observation is to the deletion of each observation in the original data set. It can be formulated as a weighted linear combination's of all data points. Again, the projection matrix is involved in the calculation to obtain the required weights:

. is asymptotically normal for large sample sizes and models with many predictors. In absence of outliers the expected value. An influential observation can be identified if

as the median as the median absolute deviation-values within the original data set, i.e., a robust measure of location and a robust measure of scale. The factor 4.5 covers approx. 3 standard deviations around its centre. failed.-th prediction:

is given by:

is symmetric and idempotent, which is not necessarily the case can be calculated as:

extracts the main diagonal resembles the so-called sensitivity matrix. An eigenvector analysis - which both share the same eigenvalues – serves as a tool in outlier detection, although the eigenvectors of the sensitivity matrix are more powerful.

Software implementations

Many programs and statistics packages, such as R, Python, Julia, etc., include implementations of Cook's distance.

Extensions

(i.e., when there are more predictors than observations). While the Cook's distance quantifies the individual observation's influence on the least squares regression coefficient estimate, the HIM measures the influence of an observation on the marginal correlations.

A general class of influence measures for multivariate linear models with several response variable has been given by Barrett & Ling (1992). These are implemented for R in the mvinfluence package

See also

• Outlier

• Leverage (statistics)

• Partial leverage

• DFFITS

• Studentized residual

Notes

References

Further reading