In statistics, an empirical distribution function ( an empirical cumulative distribution function, eCDF) is the distribution function associated with the empirical measure of a sample. This cumulative distribution function is a step function that jumps up by at each of the data points. Its value at any specified value of the measured variable is the fraction of observations of the measured variable that are less than or equal to the specified value.

The empirical distribution function is an estimate of the cumulative distribution function that generated the points in the sample. It converges with probability 1 to that underlying distribution, according to the Glivenko–Cantelli theorem. A number of results exist to quantify the rate of convergence of the empirical distribution function to the underlying cumulative distribution function. A simple empirical estimate is the frequency distribution.

Definition

Let be independent, identically distributed real random variables with the common cumulative distribution function . Then the empirical distribution function is defined as is the indicator of event . For a fixed is a Bernoulli random variable with parameter is a binomial random variable with mean and variance is an unbiased estimator for .

In some textbooks, the empirical distribution function is defined as However, since the ratio approaches 1 as goes to infinity, the asymptotic properties of the two definitions are the same.

Asymptotic properties

By the strong law of large numbers converges to as almost surely, for every value of : is consistent. This expression asserts the pointwise convergence of the empirical distribution function to the true cumulative distribution function. There is a stronger result, called the Glivenko–Cantelli theorem, which states that the convergence in fact happens uniformly over : The sup-norm in this expression is called the Kolmogorov–Smirnov statistic and the assumed true cumulative distribution function . Other norm functionss may be reasonably used here instead of the sup-norm. For example, the L2-norm gives rise to the Cramér–von Mises statistic.

The asymptotic distribution can be further characterized in several different ways. First, the central limit theorem states that pointwise rate of convergence: This result is extended by the Donsker’s theorem, which asserts that the empirical process, converges in distribution in the Skorokhod space to the mean-zero Gaussian process, where is the standard Brownian bridge. The uniform rate of convergence in Donsker’s theorem can be quantified by the result known as the Hungarian embedding:

can also be quantified in terms of the asymptotic behavior of the sup-norm of this expression. Number of results exist in this venue, for example the Dvoretzky–Kiefer–Wolfowitz inequality: In fact, Kolmogorov has shown that if the cumulative distribution function , which has the Kolmogorov distribution that does not depend on the form of .

Another result, which follows from the law of the iterated logarithm, is that

Confidence intervals

As per Dvoretzky–Kiefer–Wolfowitz inequality is specified as

As per the above bounds, we can plot the Empirical CDF, CDF and confidence intervals for different distributions by using any one of the statistical implementations.

Statistical implementation

A non-exhaustive list of software implementations of Empirical Distribution function includes:

• In R software, we compute an empirical cumulative distribution function, with several methods for plotting, printing and computing with such an “ecdf” object.

• In GNU Octave or MATLAB we can use Empirical cumulative distribution function (cdf) plot

• jmp from SAS, the CDF plot creates a plot of the empirical cumulative distribution function.

• Minitab, create an Empirical CDF

• Mathwave, we can fit probability distribution to our data

• Dataplot, we can plot Empirical CDF plot

• Scipy, we can use scipy.stats.ecdf

• Statsmodels, we can use statsmodels.distributions.empirical_distribution.ECDF

• Matplotlib, using the matplotlib.pyplot.ecdf function (new in version 3.8.0)

• Seaborn, using the seaborn.ecdfplot function

• Plotly, using the plotly.express.ecdf function

• Excel, we can plot Empirical CDF plot

• ArviZ, using the az.plot_dist function

See also

• Càdlàg functions

• Count data

• Cumulative frequency analysis

• Distribution fitting

• Dvoretzky–Kiefer–Wolfowitz inequality

• Empirical probability

• Empirical process

• Estimating quantiles from a sample

• Frequency (statistics)

• Empirical likelihood

• Kaplan–Meier estimator for censored processes

• Survival function

• Q–Q plot

References

Further reading

External links