Wikipedia / Kendall rank correlation coefficient
Wikipedia / Kendall rank correlation coefficient
In statistics, the Kendall rank correlation coefficient, commonly referred to as Kendall's τ coefficient (after the Greek letter τ, tau), is a statistic used to measure the ordinal association between two measured quantities. A τ test is a non-parametric hypothesis test for statistical dependence based on the τ coefficient. It is a measure of rank correlation: the similarity of the orderings of the data when ranked by each of the quantities. It is named after Maurice Kendall, who developed it in 1938, though Gustav Fechner had proposed a similar measure in the context of time series in 1897.
Intuitively, the Kendall correlation between two variables will be high when observations have a similar or identical rank (i.e. relative position label of the observations within the variable: 1st, 2nd, 3rd, etc.) between the two variables, and low when observations have a dissimilar or fully reversed rank between the two variables.
and \rho can be formulated as special cases of a more general correlation coefficient. Its notions of concordance and discordance also appear in other areas of statistics, like the Rand index in cluster analysis.
Definition
be a set of observations of the joint random variables X and Y) are unique. (See the section Accounting for ties, are said to be concordant and ; otherwise they are said to be discordant.
In the absence of ties, the Kendall τ coefficient is defined as:
is the binomial coefficient for the number of ways to choose two items from n items.
The number of discordant pairs is equal to the inversion number that permutes the y-sequence into the same order as the x-sequence.
Properties
The denominator is the total number of pair combinations, so the coefficient must be in the range −1≤τ≤1.
If the agreement between the two rankings is perfect (i.e., the two rankings are the same) the coefficient has value 1.
If the disagreement between the two rankings is perfect (i.e., one ranking is the reverse of the other) the coefficient has value −1.
If X and Y are independent random variables and not constant, then the expectation of the coefficient is zero.
.
Hypothesis test
The Kendall rank coefficient is often used as a test statistic in a statistical hypothesis test to establish whether two variables may be regarded as statistically dependent. This test is non-parametric, as it does not rely on any assumptions on the distributions of X or Y or the distribution of (X,Y).
Under the null hypothesis of independence of X and Y, the sampling distribution of τ has an expected value of zero. The precise distribution cannot be characterized in terms of common distributions, but may be calculated exactly for small samples; for larger samples, it is common to use an approximation to the normal distribution.
Theorem..
Case of standard normal distributions
are independent and identically distributed samples from the same jointly normal distribution with a known Pearson correlation coefficient, then the expectation of Kendall rank correlation has a closed-form formula.
The name is credited to Richard Greiner (1909) by P. A. P. Moran.
Accounting for ties
is said to be tied; a tied pair is neither concordant nor discordant. When tied pairs arise in the data, the coefficient may be modified in a number of ways to keep it in the range [−1,1]:
Tau-a
The Tau statistic defined by Kendall in 1938 was retrospectively renamed Tau-a. It represents the strength of positive or negative association of two quantitative or ordinal variables without any adjustment for ties. It is defined as:
where nc, nd and n0 are defined as in the next section.
, and the coefficient can never be equal to +1 or −1. Even a perfect equality of the two variables (X=Y) leads to a Tau-a < 1.
Tau-b
The Tau-b statistic, unlike Tau-a, makes adjustments for ties. This Tau-b was first described by Kendall in 1945 under the name Tau-w as an extension of the original Tau statistic supporting ties. Values of Tau-b range from −1 (100% negative association, or perfect disagreement) to +1 (100% positive association, or perfect agreement). In case of the absence of association, Tau-b is equal to zero.
The Kendall Tau-b coefficient is defined as :
where
A simple algorithm developed in BASIC computes Tau-b coefficient using an alternative formula.
Be aware that some statistical packages, e.g. SPSS, use alternative formulas for computational efficiency, with double the 'usual' number of concordant and discordant pairs.
Tau-c
Tau-c (also called Stuart-Kendall Tau-c) was first defined by Stuart in 1953. Contrary to Tau-b, Tau-c can be equal to +1 or −1 for non-square (i.e. rectangular) contingency tables, i.e. when the underlying scales of both variables have different number of possible values. For instance, if the variable X has a continuous uniform distribution between 0 and 100 and Y is a dichotomous variable equal to 1 if X ≥ 50 and 0 if X < 50, the Tau-c statistic of X and Y is equal to 1 while Tau-b is equal to 0.707. A Tau-c equal to 1 can be interpreted as the best possible positive correlation conditional to marginal distributions while a Tau-b equal to 1 can be interpreted as the perfect positive monotonic correlation where the distribution of X conditional to Y has zero variance and the distribution of Y conditional to X has zero variance so that a bijective function f with f(X)=Y exists.
The Stuart-Kendall Tau-c coefficient is defined as:
where
Significance tests
, is approximately distributed as a standard normal when the variables are statistically independent:
.
. For a 2-tailed test, multiply that number by two to obtain the p-value. If the p-value is below a given significance level, one rejects the null hypothesis (at that significance level) that the quantities are statistically independent.
distribution, and is again approximately equal to a standard normal distribution when the quantities are statistically independent:
where
This is sometimes referred to as the Mann-Kendall test.
Algorithms
, involves two nested iterations, as characterized by the following pseudocode:
numer := 0
for i := 2..N do
for j := 1..(i − 1) do
numer := numer + sign(x[i] − x[j]) × sign(y[i] − y[j])
return numerin complexity and becomes very slow on large samples. A more sophisticated algorithm built upon the Merge Sort time.
is not sorted, and the core of the algorithm consists of computing how many steps a Bubble Sort. An enhanced Merge Sort, that would be required by a Bubble Sort is computed as:
.
A Merge Sort, then sorts each half recursively, and then merges the two sorted halves into a fully sorted vector. The number of Bubble Sort swaps is equal to:
characterizes the Bubble Sort is computed as depicted in the following pseudo-code:
function M(L[1..n], R[1..m]) is
i := 1
j := 1
nSwaps := 0
while i ≤ n and j ≤ m do
if R[j] < L[i] then
nSwaps := nSwaps + n − i + 1
j := j + 1
else
i := i + 1
return nSwapsare easily obtained in a single linear-time pass through the sorted arrays.
Approximating Kendall rank correlation from a stream
time complexity. However, these algorithms necessitate the availability of all data to determine observation ranks, posing a challenge in sequential data settings where observations are revealed incrementally. Fortunately, algorithms do exist to estimate the Kendall rank correlation coefficient in sequential settings..
The first such algorithm presents an approximation to the Kendall rank correlation coefficient based on coarsening the joint distribution of the random variables. Non-stationary data is treated via a moving window approach. This algorithm is simple and is able to handle discrete random variables along with continuous random variables without modification.
The second algorithm is based on Hermite series estimators and utilizes an alternative estimator for the exact Kendall rank correlation coefficient i.e. for the probability of concordance minus the probability of discordance of pairs of bivariate observations. This alternative estimator also serves as an approximation to the standard estimator. This algorithm is only applicable to continuous random variables, but it has demonstrated superior accuracy and potential speed gains compared to the first algorithm described, along with the capability to handle non-stationary data without relying on sliding windows. An efficient implementation of the Hermite series based approach is contained in the R package package hermiter.
Software implementations
R in its "stats" package (also cor(x, y, method = "kendall") will work, but the latter does not return the p-value). All three versions of the coefficient are available in the "DescTools" package along with the confidence intervals: KendallTauA(x,y,conf.level=0.95), KendallTauB(x,y,conf.level=0.95), StuartTauC(x,y,conf.level=0.95). Fast batch estimates of the Kendall rank correlation coefficient along with sequential estimates are provided for in the package hermiter.
In Stata is implemented as ktau varlist.
See also
Mann–Whitney U test - it is equivalent to Kendall's tau correlation coefficient if one of the variables is binary.
References
Further reading
External links