↑ arvau

Wikipedia / Chi-squared distribution

In probability theory and statistics, the -distribution degrees of freedom independent standard normal random variables.

is a special case of the gamma distribution and the univariate Wishart distribution.

The scaled chi-squared distribution is a reparametrization of the gamma distribution and the univariate Wishart distribution.

The chi-squared distribution is one of the most widely used probability distributions in inferential statistics, notably in hypothesis testing and in construction of confidence intervals. This distribution is sometimes called the central chi-squared distribution, a special case of the more general noncentral chi-squared distribution.

The chi-squared distribution is used in the common chi-squared tests for goodness of fit of an observed distribution to a theoretical one, the independence of two criteria of classification of qualitative data, and in finding the confidence interval for estimating the population standard deviation of a normal distribution from a sample standard deviation. Many other statistical tests also use this distribution, such as Friedman's analysis of variance by ranks.

Definitions

If are independent, standard normal is distributed according to the chi-squared distribution with degrees of freedom.

The chi-squared distribution has one parameter: a positive integer that specifies the number of degrees of freedom (the number of random variables Zi being summed).

Introduction

The chi-squared distribution is used primarily in hypothesis testing, and to a lesser extent for confidence intervals for population variance when the underlying distribution is normal. Unlike more widely known distributions such as the normal distribution and the exponential distribution, the chi-squared distribution is not as often applied in the direct modeling of natural phenomena. It arises in the following hypothesis tests, among others:

It is also a component of the definition of the t-distribution and the F-distribution used in t-tests, analysis of variance, and regression analysis.

The primary reason for which the chi-squared distribution is extensively used in hypothesis testing is its relationship to the normal distribution. Many hypothesis tests use a test statistic, such as the t-statistic in a t-test. For these hypothesis tests, as the sample size, , increases, the sampling distribution of the test statistic approaches the normal distribution (central limit theorem). Because the test statistic (such as ) is asymptotically normally distributed, provided the sample size is sufficiently large, the distribution used for hypothesis testing may be approximated by a normal distribution. Testing hypotheses using a normal distribution is well understood and relatively easy. The simplest chi-squared distribution is the square of a standard normal distribution. So wherever a normal distribution could be used for a hypothesis test, a chi-squared distribution could be used.

. The subscript 1 indicates that this particular chi-squared distribution is constructed from only 1 standard normal distribution. A chi-squared distribution constructed by squaring a single standard normal distribution is said to have 1 degree of freedom. Thus, as the sample size for a hypothesis test increases, the distribution of the test statistic approaches a normal distribution. Just as extreme values of the normal distribution have low probability (and give small p-values), extreme values of the chi-squared distribution have low probability.

An additional reason that the chi-squared distribution is widely used is that it turns up as the large sample distribution of generalized likelihood ratio tests (LRT). LRTs have several desirable properties; in particular, simple LRTs commonly provide the highest power to reject the null hypothesis (Neyman–Pearson lemma) and this leads also to optimality properties of generalised LRTs. However, the normal and chi-squared approximations are only valid asymptotically. For this reason, it is preferable to use the t distribution rather than the normal approximation or the chi-squared approximation for a small sample size. Similarly, in analyses of contingency tables, the chi-squared approximation will be poor for a small sample size, and it is preferable to use Fisher's exact test. Ramsey shows that the exact binomial test is always more powerful than the normal approximation.

Lancaster shows the connections among the binomial, normal, and chi-squared distributions, as follows. De Moivre and Laplace established that a binomial distribution could be approximated by a normal distribution. Specifically they showed the asymptotic normality of the random variable

.

Squaring both sides of the equation gives

, this equation can be rewritten as

The expression on the right is of the form that Karl Pearson would generalize to the form

where

  • distribution;

  • ;

  • ; and

  • = the number of cells in the table.

). Because the square of a standard normal distribution is the chi-squared distribution with one degree of freedom, the probability of a result such as 1 heads in 10 trials can be approximated either by using the normal distribution directly, or the chi-squared distribution for the normalised, squared difference between observed and expected value. However, many problems involve more than the two possible outcomes of a binomial, and instead require 3 or more categories, which leads to the multinomial distribution. Just as de Moivre and Laplace sought for and found the normal approximation to the binomial, Pearson sought for and found a degenerate multivariate normal approximation to the multinomial distribution (the numbers in each category add up to the total sample size, which is considered fixed). Pearson showed that the chi-squared distribution arose from such a multivariate normal approximation to the multinomial distribution, taking careful account of the statistical dependence (negative correlations) between numbers of observations in different categories.

Probability density function

The probability density function denotes the gamma function, which has k.

degrees of freedom, see Proofs related to chi-squared distribution.

Cumulative distribution function

Its cumulative distribution function is the lower incomplete gamma function is the regularized gamma function.

.

Tables of the chi-squared cumulative distribution function are widely available and the function is included in many spreadsheets and all statistical packages.

, Chernoff bounds on the lower and upper tails of the CDF may be obtained.

For another approximation for the CDF modeled after the cube of a Gaussian, see under Noncentral chi-squared distribution.

Properties

Cochran's theorem

The following is a special case of Cochran's theorem.

Theorem. are independent identically distributed (i.i.d.), standard normal

Proof. which proves the claim.

Additivity

degrees of freedom.

Sample mean

i.i.d.

Asymptotically and variance the sample mean converges towards:

Note that we would have obtained the same result invoking instead the central limit theorem the expectation is ).

Entropy

The differential entropy is the Digamma function.

The chi-squared distribution is the maximum entropy probability distribution are fixed. Since the chi-squared is in the family of gamma distributions, this can be derived by substituting appropriate values in the Expectation of the log moment of gamma. For derivation from more basic principles, see the derivation in moment-generating function of the sufficient statistic.

Noncentral moments

degrees of freedom are given by

Cumulants

The cumulants are readily obtained by a power series with cumulant generating function.

Concentration

The chi-squared distribution exhibits strong concentration around its mean. The standard Laurent-Massart.

, and its convex dual, the standard Chernoff boundThis result is used in proving the Johnson–Lindenstrauss lemma.

Asymptotic properties

By the central limit theorem the distribution is sufficiently close to a normal distribution, so the difference is ignorable. tends to a standard normal distribution. However, convergence is slow as the skewness and the excess kurtosis.

, as the logarithmic transform removes much of the asymmetry.

Other functions of the chi-squared distribution converge more rapidly to a normal distribution. Some examples are:

  • and unit variance (1922, by R. A. Fisher, see (18.23), p.426 of Johnson).

This is known as the Wilson–Hilferty transformation, see (18.24), p.426 of Johnson.

Related distributions

independent standard normal random variables.

degrees of freedom.

The sum of squares of statistically independent unit-variance Gaussian variables which do not have mean zero yields a generalization of the chi-squared distribution called the noncentral chi-squared distribution.

i.i.d. symmetric, idempotent matrix with rank, then the quadratic form degrees of freedom.

then

.

The chi-squared distribution is also naturally related to other distributions arising from the Gaussian. In particular,

  • is F-distributed are statistically independent.

  • is not chi-square distributed.

Generalizations

The chi-squared distribution is obtained as the sum of the squares of independent, zero-mean, unit-variance Gaussian random variables. Generalizations of this distribution can be obtained by summing the squares of other types of Gaussian random variables. Several such distributions are described below.

Linear combination

is a special case of the generalized chi-squared distribution. A closed expression for this distribution is not known. It may be, however, approximated efficiently using the property of characteristic functions of chi-square random variables.

Chi-squared distributions

Noncentral chi-squared distribution

The noncentral chi-squared distribution is obtained from the sum of the squares of independent Gaussian random variables having unit variance and nonzero means.

Generalized chi-squared distribution

The generalized chi-squared distribution is obtained from the quadratic form where is a zero-mean Gaussian vector having an arbitrary covariance matrix, and is an arbitrary matrix.

is a special case of the gamma distribution using the scale parameterization of the gamma distribution) where is an integer.

Because the exponential distribution is an exponential distribution.

The Erlang distribution.

Occurrence and applicationsApplications

The chi-squared distribution has numerous applications in inferential statistics, for instance in chi-squared tests and in estimating variances. It enters the problem of estimating the mean of a normally distributed population and the problem of estimating the slope of a regression line via its role in Student's t-distribution. It enters all analysis of variance problems via its role in the F-distribution, which is the distribution of the ratio of two independent chi-squared random variables, each divided by their respective degrees of freedom.

Following are some of the most common situations in which the chi-squared distribution arises from a Gaussian-distributed sample.

  • are i.i.d. random variables.

  • The box below shows some statistics independent random variables that have probability distributions related to the chi-squared distribution:

The chi-squared distribution is also often encountered in magnetic resonance imaging.

Computational methods

Table of χ2 values vs p-values

The p-value is the probability of observing a test statistic at least as extreme in a chi-squared distribution. Accordingly, since the cumulative distribution function (CDF) for the appropriate degrees of freedom (df) gives the probability of having obtained a value less extreme than this point, subtracting the CDF value from 1 gives the p-value. A low p-value, below the chosen significance level, indicates statistical significance, i.e., sufficient evidence to reject the null hypothesis. A significance level of 0.05 is often used as the cutoff between significant and non-significant results.

The table below gives a number of p for the first 10 degrees of freedom.These values can be calculated evaluating the quantile function (also known as "inverse CDF" or "ICDF") of the chi-squared distribution; e. g., the ICDF for and yields as in the table above, noticing that is the p-value from the table.

History

This distribution was first described by the German geodesist and statistician Friedrich Robert Helmert in papers of 1875–6, where he computed the sampling distribution of the sample variance of a normal population. Thus in German this was traditionally known as the Helmert'sche ("Helmertian") or "Helmert distribution".

The distribution was independently rediscovered by the English mathematician Karl Pearson in the context of goodness of fit, for which he developed his Pearson's chi-squared test, published in 1900, with computed table of values published in , collected in . The name "chi-square" ultimately derives from Pearson's shorthand for the exponent in a multivariate normal distribution with the Greek letter Chi, writing for what would appear in modern notation as ( being the covariance matrix). The idea of a family of "chi-squared distributions", however, is not due to Pearson but arose as a further development due to Fisher in the 1920s.

See also

References

Sources

Further reading