↑ arvau

Wikipedia / Matrix norm

In mathematics, a norm is a function from a vector space to non-negative real numbers that satisfies certain axioms. A matrix norm is a norm defined on a vector space of matrices. As for every norm, a matrix norm defines a distance, the distance between two matrices being the norm of their difference. The specificity of matrix norms is that they may often be defined either directly as for every norm, or by mean of their properties as linear operators.

There are many specific matrix norms. Most arise from the following three perspectives, though different perspectives may sometimes yield the same norm.

  • Consider the matrix as a linear operator; then a matrix norm may describe how much the operator can stretch vectors. Such matrix norms induced by vector norms are called operator norms.

  • Consider the matrix as a rectangular array of numbers; then a matrix norm may be defined as a function of the entries of the matrix. Such matrix norms are sometimes called "entry-wise" norms.

  • The singular value decomposition is useful in analyzing matrices. A vector norm of the singular values of a matrix may be taken as a matrix norm. Such norms are called Schatten norms.

Matrix norms are often denoted by double vertical bars). However, the meaning of the subscript may vary, since matrix norms in different perspectives relate to \ell^p-norms in different ways.

Preliminaries

Given a field of either real or complex numbers be the -vector space A matrix norm is a norm

The matrix norm is a function that must satisfy the following properties:

  • (positive-valued)

  • (definite)

  • (absolutely homogeneous)

  • (sub-additive or satisfying the triangle inequality)

The only feature distinguishing matrices from rearranged vectors is multiplication. Matrix norms are particularly useful if they are also sub-multiplicative:

can be rescaled to be sub-multiplicative; in some books, the terminology matrix norm is reserved for sub-multiplicative norms.

Possible properties

Unitary invariance

.

A symmetric gauge function is an absolute vector norm for any permutation matrix. That is:

  • Non-negativity:.

  • Positive homogeneity:.

  • Triangle inequality:.

  • Symmetry:.

A norm is a unitarily invariant matrix norm if and only if it is a symmetric gauge function on the vector of singular values.

Consistent and compatible norms

is called consistent. In the special case of is also called compatible.

.

Monotone norms

is called monotone if it is monotonic with respect to the Loewner order. Thus, a matrix norm is increasing if

The Frobenius norm and spectral norm are examples of monotone norms.

Matrix norms induced by vector norms

Suppose a vector norm matrix with respect to the standard basis, and one defines the corresponding induced norm or operator norm or subordinate norm denotes the supremum can be used for the operator norm.

Matrix norms induced by vector p-norms

If the p-norm for vectors then the corresponding operator norm is: These induced norms are different from the "entry-wise" p-norms and the Schatten p-norms

Geometrically speaking, one can imagine a p measures the longest "radius" of the distorted convex shape. In other words, we must take a p.

p = 1 or ∞

we have simple formulas.

which is simply the maximum absolute row sum of the matrix.

Spectral norm (p = 2)

(the Euclidean norm-norm for vectors), the induced matrix norm is the spectral norm. The spectral norm should not be confused with the spectral radius. The two values do not coincide in infinite dimensions see Spectral radius is the largest singular value, i.e., the square root of the largest eigenvalue denotes the conjugate transpose:

There are further properties:

Matrix norms induced by vector α- and β-norms

.

.

.

are the maximum row and column 2-norm of the matrix, respectively.

Properties

Any operator norm is consistent

. Then,

Square matrices

Moreover, any such norm satisfies the inequality for all positive integers r, where is the spectral radius of . For symmetric or hermitian , we have equality in () for the 2-norm, since in this case the 2-norm is precisely the spectral radius of which has vanishing spectral radius. In any case, for any matrix norm, we have the spectral radius formula

Energy norms

are given in terms of energy norms based on symmetric positive definite

Using the symmetric matrix square roots respectively, the operator norm can be expressed as the spectral norm of a modified matrix:

"Entry-wise" matrix norms

, and use one of the familiar vector norms. For example, using the p-norm for vectors, , we get:

This is a different norm from the induced p-norm (see above) and the Schatten p-norm (see below), but the notation is the same.

The special case p = 2 is the Frobenius norm, and p = yields the maximum norm.

L2,1 and Lp,q norms

be the dimension presents data points in an norm is the sum of the Euclidean norms of the columns of the matrix:

norm as a loss function is more robust, since the error for each data point (a column) is not squared. It is used in robust data analysis and sparse coding.

For norm as follows:

Frobenius norm

When norm, it is called the Frobenius norm or the Hilbert–Schmidt norm, though the latter term is used more frequently in the context of operators on (possibly infinite-dimensional) Hilbert space. This norm can be defined in various ways:

where the trace are the singular values. The third equality is proven by singular value decomposition, and the fact that the trace is invariant under circular shifts.

and comes from the Frobenius inner product on the space of all matrices.

The Frobenius norm is sub-multiplicative and is very useful for numerical linear algebra. The sub-multiplicativity of Frobenius norm can be proved using the Cauchy–Schwarz inequality.

Frobenius norm is often easier to compute than induced norms, and has the useful property of being invariant under rotations (and unitary):

and analogously:

).

It also satisfies

and

is the Frobenius inner product, and Re is the real part of a complex number (irrelevant for real matrices)

Max norm

The max norm is the elementwise norm in the limit as goes to infinity:

This norm is not sub-multiplicative makes it so.

Note that in some literature (such as Communication complexity-norm, refers to the factorization norm:

Schatten norms

The Schatten p-norms arise when applying the p-norm to the vector of singular values of a matrix. are denoted by i, then the Schatten p-norm is defined by

These norms again share the notation with the induced and entry-wise p-norms, but they are different.

and all unitary matrices.

The most familiar cases are p = 1, 2, . The case p = 2 yields the Frobenius norm, introduced before. The case p = yields the spectral norm, which is the operator norm induced by the vector 2-norm (see above). Finally, p = 1 yields the nuclear norm (also known as the trace norm, or the Ky Fan 'n'-norm), defined as:

is a positive semidefinite matrix, its square root is a convex envelope, so it is often used in mathematical optimization to search for low-rank matrices.

Combining von Neumann's trace inequality with Hölder's inequality for Euclidean space yields a version of Hölder's inequality:

In particular, this implies the Schatten norm inequality

Cut norms

Another source of inspiration for matrix norms arises from considering a matrix as the adjacency matrix of a weighted, directed graph. The so-called "cut norm" measures how close the associated graph is to being bipartite where . Equivalent definitions (up to a constant factor) impose the conditions ; ; or .

The cut-norm is equivalent to the induced operator norm , which is itself equivalent to another norm, called the Grothendieck norm.

To define the Grothendieck norm, first note that a linear operator is just a scalar, and thus extends to a linear operator on any . Moreover, given any choice of basis for and , any linear operator extends to a linear operator , by letting each matrix element on elements of via scalar multiplication. The Grothendieck norm is the norm of that extended operator; in symbols:

The Grothendieck norm depends on choice of basis (usually taken to be the standard basis) and .

Equivalence of norms

, we have that:

for some positive numbers r and s are equivalent; they induce the same topology has the finite dimension.

; to wit,

is said to be minimal.

Examples of norm equivalence

once again refer to the norm induced by the vector p-norm (as above in the Induced norm section).

of rank, the following inequalities hold:

See also

Notes

References

Bibliography