A whitening transformation or sphering transformation is a linear transformation that transforms a vector of random variables with a known covariance matrix into a set of new variables whose covariance is the identity matrix, meaning that they are uncorrelated and each have variance 1. The transformation is called "whitening" because it changes the input vector into a white noise vector.

Several other transformations are closely related to whitening:

1. the decorrelation transform removes only the correlations but leaves variances intact,

2. the standardization transform sets variances to 1 but leaves correlations intact,

3. a coloring transformation transforms a vector of white random variables into a random vector with a specified covariance matrix.

Definition

is a random (column) vector with a whitening matrix with unit diagonal covariance.

.

is the Cholesky decomposition (Cholesky whitening), (PCA whitening).

. the diagonal variance matrix.

Whitening a data matrix

Whitening a data matrix follows the same transformation as for random variables. An empirical whitening transform is obtained by estimating the covariance (e.g. by maximum likelihood) and subsequently constructing a corresponding estimated whitening matrix (e.g. by Cholesky decomposition).

High-dimensional whitening

is usually assumed to be a random function or other random objects in a Hilbert space. One of the main issues of extending whitening to infinite dimensions is that the covariance operator, therefore only partial standardization is possible in infinite dimensions. A whitening operator can be then defined from the factorization of a degenerated covariance operator. High-dimensional features of the data can be exploited through kernel regressors or basis function systems.

R implementation

An implementation of several whitening procedures in R, including ZCA-whitening and PCA whitening but also CCA whitening, is available in the "whitening" R package published on CRAN. The R package "pfica" allows the computation of high-dimensional whitening representations using basis function systems (B-splines, Fourier basis, etc.).

See also

• Decorrelation

• Principal component analysis

• Weighted least squares

• Canonical correlation

• Mahalanobis distance (is Euclidean after W. transformation).

References

External links

• https://courses.media.mit.edu/2010fall/mas622j/whiten.pdf

• The ZCA whitening transformation. Appendix A of Learning Multiple Layers of Features from Tiny Images by A. Krizhevsky.