↑ arvau

Wikipedia / Matrix decomposition

In the mathematical discipline of linear algebra, a matrix decomposition or matrix factorization is a factorization of a matrix into a product of matrices. There are many different matrix decompositions; each finds use among a particular class of problems.

Example

In numerical analysis, different decompositions are used to implement efficient matrix algorithms.

For example, when solving a system of linear equations, the matrix A can be decomposed via the LU decomposition. The LU decomposition factorizes a matrix into a lower triangular matrix L and an upper triangular matrix U, though one might require significantly more digits in inexact arithmetic such as floating point.

Similarly, the QR decomposition expresses A as QR with Q an orthogonal matrix and R an upper triangular matrix. The system Q(Rx) = b is solved by Rx = QTb = c, and the system Rx = c is solved by 'back substitution'. The number of additions and multiplications required is about twice that of using the LU solver, but no more digits are required in inexact arithmetic because the QR decomposition is numerically stable.

LU decomposition

LU reduction

Block LU decomposition

Rank factorization

Cholesky decomposition

  • Applicable to: square, hermitian, positive definite

  • is upper triangular with real positive diagonal entries

  • are allowed to be zero

  • Uniqueness: for positive definite matrices Cholesky decomposition is unique. However, it is not unique in the positive semi-definite case.

  • has all real elements

  • Comment: An alternative is the LDL decomposition, which can avoid extracting square roots.

QR decomposition

RRQR factorization

Interpolative decomposition

Eigendecomposition

  • Also called spectral decomposition.

  • Applicable to: square matrix A with linearly independent eigenvectors (not necessarily distinct eigenvalues).

  • , where D is a diagonal matrix formed from the eigenvalues of A, and the columns of V are the corresponding eigenvectors of A.

  • Existence: An n-by-n matrix A always has n (complex) eigenvalues, which can be ordered (in more than one way) to form an n-by-n diagonal matrix D and a corresponding matrix of nonzero columns V that satisfies the eigenvalue equation is invertible if and only if the n eigenvectors are linearly independent (that is, each eigenvalue has geometric multiplicity equal to its algebraic multiplicity). A sufficient (but not necessary) condition for this to happen is that all the eigenvalues are different (in this case geometric and algebraic multiplicity are equal to 1)

  • Comment: One can always normalize the eigenvectors to have length one (see the definition of the eigenvalue equation)

  • Comment: Every normal matrix A is a conjugate transpose) can be eigendecomposed. For a normal matrix A. In particular all unitary, Hermitian, or skew-Hermitian (in the real-valued case, all orthogonal, symmetric, or skew-symmetric, respectively) matrices are normal and therefore possess this property.

  • Comment: For any real symmetric matrix A, where both D and V are real-valued.

  • , where V and D are the matrices formed from the eigenvectors and eigenvalues of A. Since D, just involves raising each element on the diagonal to the power t. This is much easier to do and understand than raising A to power t, since A is usually not diagonal.

Jordan decomposition

The Jordan normal form and the Jordan–Chevalley decomposition

  • Applicable to: square matrix A

  • Comment: the Jordan normal form generalizes the eigendecomposition to cases where there are repeated eigenvalues and cannot be diagonalized, the Jordan–Chevalley decomposition does this without choosing a basis.

Schur decomposition

Real Schur decomposition

QZ decomposition

  • Also called: generalized Schur decomposition

  • Applicable to: square matrices A and B

  • Comment: there are two versions of this decomposition: complex and real.

  • where Q and Z are unitary matrices, the * superscript represents conjugate transpose, and S and T are upper triangular matrices.

  • Comment: in the complex QZ decomposition, the ratios of the diagonal elements of S to the corresponding diagonal elements of T, are the generalized eigenvalues that solve the generalized eigenvalue problem is an unknown scalar and v is an unknown nonzero vector).

  • where A, B, Q, Z, S, and T are matrices containing real numbers only. In this case Q and Z are orthogonal matrices, the T superscript represents transposition, and S and T are block upper triangular matrices. The blocks on the diagonal of S and T are of size 1×1 or 2×2.

Takagi's factorization

  • Applicable to: square, complex, symmetric matrix A.

  • , where D is a real nonnegative diagonal matrix, and V is unitary denotes the matrix transpose of V.

  • Comment: The diagonal elements of D.

  • Comment: V may be complex even if A is real.

  • . Moreover, if A also does not apply.

Singular value decomposition

  • Applicable to: m-by-n matrix A.

  • , where D is a nonnegative diagonal matrix, and U and V are unitary is the conjugate transpose of V (or simply the transpose, if V contains real numbers only).

  • Comment: The diagonal elements of D are called the singular values of A.

  • Comment: Like the eigendecomposition above, the singular value decomposition involves finding basis directions along which matrix multiplication is equivalent to scalar multiplication, but it has greater generality since the matrix under consideration need not be square.

  • need not to be unique in general.

Scale-invariant decompositions

Refers to variants of existing matrix decompositions, such as the SVD, that are invariant with respect to diagonal scaling.

  • Applicable to: m-by-n matrix A.

  • , where S is a unique nonnegative diagonal matrix of scale-invariant singular values, U and V are unitary matrices is the conjugate transpose of V, and positive diagonal matrices D and E.

  • Comment: Is analogous to the SVD except that the diagonal elements of S are invariant with respect to left and/or right multiplication of A by arbitrary nonsingular diagonal matrices, as opposed to the standard SVD for which the singular values are invariant with respect to left and/or right multiplication of A by arbitrary unitary matrices.

  • Comment: Is an alternative to the standard SVD when invariance is required with respect to diagonal rather than unitary transformations of A.

  • (given by the diagonal elements of S) are always uniquely determined. Diagonal matrices D and E, and unitary U and V, are not necessarily unique in general.

  • Comment: U and V matrices are not the same as those from the SVD.

Analogous scale-invariant decompositions can be derived from other matrix decompositions; for example, to obtain scale-invariant eigenvalues.

Hessenberg decomposition

Complete orthogonal decomposition

  • Also known as: UTV decomposition, ULV decomposition, URV decomposition.

  • Applicable to: m-by-n matrix A.

  • , where T is a triangular matrix, and U and V are unitary matrices.

  • Comment: Similar to the singular value decomposition and to the Schur decomposition.

Other decompositions

Polar decomposition

  • Applicable to: any square complex matrix A.

  • (left polar decomposition), where U is a unitary matrix and P and P are positive semidefinite Hermitian matrices.

  • is unique.

  • which is the singular value decomposition. Hence, the existence of the polar decomposition is equivalent to the existence of the singular value decomposition.

Algebraic polar decomposition

  • Applicable to: square, complex, non-singular matrix A.

  • , where Q is a complex orthogonal matrix and S is complex symmetric matrix.

  • has no negative real eigenvalues, then the decomposition is unique.

  • .

  • , where R is a real matrix and C is a circular matrix.

Mostow's decomposition

  • Applicable to: square, complex, non-singular matrix A.

  • , where U is unitary, M is real anti-symmetric and S is real symmetric.

  • Comment: The matrix A, where U2 is unitary, M2 is real anti-symmetric and S2 is real symmetric.

Sinkhorn normal form

  • Applicable to: square real matrix A with strictly positive elements.

  • , where S is doubly stochastic and D1 and D2 are real diagonal matrices with strictly positive elements.

Sectoral decomposition

Williamson's normal form

Matrix square root

  • , not unique in general.

  • .

Generalizations

There exist analogues of the SVD, QR, LU and Cholesky factorizations for quasimatrices and cmatrices or continuous matrices. A "quasimatrix" is, like a matrix, a rectangular scheme whose elements are indexed, but one discrete index is replaced by a continuous index. Likewise, a "cmatrix", is continuous in both indices. As an example of a cmatrix, one can think of the kernel of an integral operator.

These factorizations are based on early work by , , and . For an account and a translation to English of the seminal papers, see .

See also

References

Notes

Citations

Bibliography