In probability and statistics, the Hellinger distance (closely related to, although different from, the Bhattacharyya distance) is used to quantify the similarity between two probability distributions. It is a type of f-divergence. The Hellinger distance is defined in terms of the Hellinger integral, which was introduced by Ernst Hellinger in 1909.

It is sometimes called the Jeffreys distance.

Definition

Measure theory

To define the Hellinger distance in terms of measure theory denote two probability measures that are absolutely continuous is defined as the quantity

are the Radon–Nikodym derivatives of P and Q, i.e. the Hellinger distance between P and Q is replaced with a different probability measure with respect to which both P and Q are absolutely continuous. For compactness, the above formula is often written as

Probability theory using Lebesgue measure

To define the Hellinger distance in terms of elementary probability theory, we take to be the Lebesgue measure, so that dP/d and dQ/d are simply probability density functions. If we denote the densities as f and g, respectively, the squared Hellinger distance can be expressed as a standard calculus integral

where the second form can be obtained by expanding the square and using the fact that the integral of a probability density over its domain equals 1.

The Hellinger distance H(P,Q) satisfies the property (derivable from the Cauchy–Schwarz inequality)

Discrete distributions

, their Hellinger distance is defined as

which is directly related to the Euclidean norm of the difference of the square root vectors, i.e.

Properties

The Hellinger distance forms a bounded metric on the space of probability distributions over a given probability space.

The maximum distance 1 is achieved when P assigns probability zero to every set to which Q assigns a positive probability, and vice versa.

in front of the integral is omitted, in which case the Hellinger distance ranges from zero to the square root of two.

The Hellinger distance is related to the Bhattacharyya coefficient as it can be defined as

Hellinger distances are used in the theory of sequential and asymptotic statistics.

The squared Hellinger distance between two normal distributions is:

The squared Hellinger distance between two multivariate normal distributions is

The squared Hellinger distance between two exponential distributions is:

The squared Hellinger distance between two Weibull distributions is a common shape parameter are the scale parameters respectively):

The squared Hellinger distance between two Poisson distributions, is:

The squared Hellinger distance between two beta distributions is:

is the beta function.

The squared Hellinger distance between two gamma distributions is:

is the gamma function.

Connection with total variation distance

and the total variation distance are related as follows:

The constants in this inequality may change depending on which renormalization).

These inequalities follow immediately from the inequalities between the 1-norm and the 2-norm.

See also

• Statistical distance

• Kullback–Leibler divergence

• Bhattacharyya distance

• Total variation distance

• Fisher information metric

Notes

References