8/23/23, 9:33 AM Matrix decomposition - Wikipedia

Matrix decomposition
In the mathematical discipline of linear algebra, a matrix decomposition or matrix factorization is a
factorization of a matrix into a product of matrices. There are many different matrix decompositions; each
finds use among a particular class of problems.

Example
In numerical analysis, different decompositions are used to implement efficient matrix algorithms.

For instance, when solving a system of linear equations , the matrix A can be decomposed via the
LU decomposition. The LU decomposition factorizes a matrix into a lower triangular matrix L and an upper
triangular matrix U. The systems and require fewer additions and multiplications
to solve, compared with the original system , though one might require significantly more digits in
inexact arithmetic such as floating point.

Similarly, the QR decomposition expresses A as QR with Q an orthogonal matrix and R an upper triangular
matrix. The system Q(Rx) = b is solved by Rx = QTb = c, and the system Rx = c is solved by 'back
substitution'. The number of additions and multiplications required is about twice that of using the LU
solver, but no more digits are required in inexact arithmetic because the QR decomposition is numerically
stable.

Decompositions related to solving systems of linear equations

LU decomposition
Traditionally applicable to: square matrix A, although rectangular matrices can be
applicable.[1][nb 1]
Decomposition: , where L is lower triangular and U is upper triangular
Related: the LDU decomposition is , where L is lower triangular with ones on the
diagonal, U is upper triangular with ones on the diagonal, and D is a diagonal matrix.
Related: the LUP decomposition is , where L is lower triangular, U is upper
triangular, and P is a permutation matrix.
Existence: An LUP decomposition exists for any square matrix A. When P is an identity matrix,
the LUP decomposition reduces to the LU decomposition.
Comments: The LUP and LU decompositions are useful in solving an n-by-n system of linear
equations . These decompositions summarize the process of Gaussian elimination in
matrix form. Matrix P represents any row interchanges carried out in the process of Gaussian

https://en.wikipedia.org/wiki/Matrix_decomposition#:~:text=In the mathematical discipline of,a particular class of problems. 1/9

8/23/23, 9:33 AM Matrix decomposition - Wikipedia

elimination. If Gaussian elimination produces the row echelon form without requiring any row
interchanges, then P = I, so an LU decomposition exists.

LU reduction

Block LU decomposition

Rank factorization
Applicable to: m-by-n matrix A of rank r
Decomposition: where C is an m-by-r full column rank matrix and F is an r-by-n full
row rank matrix
Comment: The rank factorization can be used to compute the Moore–Penrose pseudoinverse
of A,[2] which one can apply to obtain all solutions of the linear system .

Cholesky decomposition
Applicable to: square, hermitian, positive definite matrix
Decomposition: , where is upper triangular with real positive diagonal entries
Comment: if the matrix is Hermitian and positive semi-definite, then it has a decomposition
of the form if the diagonal entries of are allowed to be zero
Uniqueness: for positive definite matrices Cholesky decomposition is unique. However, it is not
unique in the positive semi-definite case.
Comment: if is real and symmetric, has all real elements
Comment: An alternative is the LDL decomposition, which can avoid extracting square roots.

QR decomposition
Applicable to: m-by-n matrix A with linearly independent columns
Decomposition: where is a unitary matrix of size m-by-m, and is an upper
triangular matrix of size m-by-n
Uniqueness: In general it is not unique, but if is of full rank, then there exists a single that
has all positive diagonal elements. If is square, also is unique.

https://en.wikipedia.org/wiki/Matrix_decomposition#:~:text=In the mathematical discipline of,a particular class of problems. 2/9

8/23/23, 9:33 AM Matrix decomposition - Wikipedia

Comment: The QR decomposition provides an effective way to solve the system of equations
. The fact that is orthogonal means that , so that is equivalent to
, which is very easy to solve since is triangular.

RRQR factorization

Interpolative decomposition

Decompositions based on eigenvalues and related concepts

Eigendecomposition
Also called spectral decomposition.
Applicable to: square matrix A with linearly independent eigenvectors (not necessarily distinct
eigenvalues).
Decomposition: , where D is a diagonal matrix formed from the eigenvalues of A,
and the columns of V are the corresponding eigenvectors of A.
Existence: An n-by-n matrix A always has n (complex) eigenvalues, which can be ordered (in
more than one way) to form an n-by-n diagonal matrix D and a corresponding matrix of
nonzero columns V that satisfies the eigenvalue equation . is invertible if and only
if the n eigenvectors are linearly independent (that is, each eigenvalue has geometric
multiplicity equal to its algebraic multiplicity). A sufficient (but not necessary) condition for this
to happen is that all the eigenvalues are different (in this case geometric and algebraic
multiplicity are equal to 1)
Comment: One can always normalize the eigenvectors to have length one (see the definition
of the eigenvalue equation)
Comment: Every normal matrix A (that is, matrix for which , where is a
conjugate transpose) can be eigendecomposed. For a normal matrix A (and only for a normal
matrix), the eigenvectors can also be made orthonormal ( ) and the
eigendecomposition reads as . In particular all unitary, Hermitian, or skew-
Hermitian (in the real-valued case, all orthogonal, symmetric, or skew-symmetric, respectively)
matrices are normal and therefore possess this property.
Comment: For any real symmetric matrix A, the eigendecomposition always exists and can be
written as , where both D and V are real-valued.
Comment: The eigendecomposition is useful for understanding the solution of a system of
linear ordinary differential equations or linear difference equations. For example, the difference
equation starting from the initial condition is solved by , which is
equivalent to , where V and D are the matrices formed from the eigenvectors
and eigenvalues of A. Since D is diagonal, raising it to power , just involves raising each
element on the diagonal to the power t. This is much easier to do and understand than raising
A to power t, since A is usually not diagonal.

Jordan decomposition

The Jordan normal form and the Jordan–Chevalley decomposition

Applicable to: square matrix A

Comment: the Jordan normal form generalizes the eigendecomposition to cases where there
are repeated eigenvalues and cannot be diagonalized, the Jordan–Chevalley decomposition
https://en.wikipedia.org/wiki/Matrix_decomposition#:~:text=In the mathematical discipline of,a particular class of problems. 3/9
8/23/23, 9:33 AM Matrix decomposition - Wikipedia

does this without choosing a basis.

Schur decomposition
Applicable to: square matrix A
Decomposition (complex version): , where U is a unitary matrix, is the
conjugate transpose of U, and T is an upper triangular matrix called the complex Schur form
which has the eigenvalues of A along its diagonal.
Comment: if A is a normal matrix, then T is diagonal and the Schur decomposition coincides
with the spectral decomposition.

Real Schur decomposition

Applicable to: square matrix A
Decomposition: This is a version of Schur decomposition where and only contain real
numbers. One can always write where V is a real orthogonal matrix, is the
transpose of V, and S is a block upper triangular matrix called the real Schur form. The blocks
on the diagonal of S are of size 1×1 (in which case they represent real eigenvalues) or 2×2 (in
which case they are derived from complex conjugate eigenvalue pairs).

QZ decomposition
Also called: generalized Schur decomposition
Applicable to: square matrices A and B
Comment: there are two versions of this decomposition: complex and real.
Decomposition (complex version): and where Q and Z are unitary
matrices, the * superscript represents conjugate transpose, and S and T are upper triangular
matrices.
Comment: in the complex QZ decomposition, the ratios of the diagonal elements of S to the
corresponding diagonal elements of T, , are the generalized eigenvalues that
solve the generalized eigenvalue problem (where is an unknown scalar and v is
an unknown nonzero vector).
Decomposition (real version): and where A, B, Q, Z, S, and T are
matrices containing real numbers only. In this case Q and Z are orthogonal matrices, the T
superscript represents transposition, and S and T are block upper triangular matrices. The
blocks on the diagonal of S and T are of size 1×1 or 2×2.

Takagi's factorization
Applicable to: square, complex, symmetric matrix A.
Decomposition: , where D is a real nonnegative diagonal matrix, and V is unitary.
denotes the matrix transpose of V.
Comment: The diagonal elements of D are the nonnegative square roots of the eigenvalues of
.
Comment: V may be complex even if A is real.

https://en.wikipedia.org/wiki/Matrix_decomposition#:~:text=In the mathematical discipline of,a particular class of problems. 4/9

8/23/23, 9:33 AM Matrix decomposition - Wikipedia

Comment: This is not a special case of the eigendecomposition (see above), which uses
instead of . Moreover, if A is not real, it is not Hermitian and the form using also does
not apply.

Singular value decomposition

Applicable to: m-by-n matrix A.
Decomposition: , where D is a nonnegative diagonal matrix, and U and V satisfy
. Here is the conjugate transpose of V (or simply the transpose, if V
contains real numbers only), and I denotes the identity matrix (of some dimension).
Comment: The diagonal elements of D are called the singular values of A.
Comment: Like the eigendecomposition above, the singular value decomposition involves
finding basis directions along which matrix multiplication is equivalent to scalar multiplication,
but it has greater generality since the matrix under consideration need not be square.
Uniqueness: the singular values of are always uniquely determined. and need not to be
unique in general.

Scale-invariant decompositions

Refers to variants of existing matrix decompositions, such as the SVD, that are invariant with respect to
diagonal scaling.

Applicable to: m-by-n matrix A.

Unit-Scale-Invariant Singular-Value Decomposition: , where S is a unique
nonnegative diagonal matrix of scale-invariant singular values, U and V are unitary matrices,
is the conjugate transpose of V, and positive diagonal matrices D and E.
Comment: Is analogous to the SVD except that the diagonal elements of S are invariant with
respect to left and/or right multiplication of A by arbitrary nonsingular diagonal matrices, as
opposed to the standard SVD for which the singular values are invariant with respect to left
and/or right multiplication of A by arbitrary unitary matrices.
Comment: Is an alternative to the standard SVD when invariance is required with respect to
diagonal rather than unitary transformations of A.
Uniqueness: The scale-invariant singular values of (given by the diagonal elements of S) are
always uniquely determined. Diagonal matrices D and E, and unitary U and V, are not
necessarily unique in general.
Comment: U and V matrices are not the same as those from the SVD.

Analogous scale-invariant decompositions can be derived from other matrix decompositions; for example,
to obtain scale-invariant eigenvalues.[3][4]

Hessenberg decomposition
Applicable to: square matrix A.
Decomposition: where is the Hessenberg matrix and is a unitary matrix.
Comment: often the first step in the Schur decomposition.

Complete orthogonal decomposition

Also known as: UTV decomposition, ULV decomposition, URV decomposition.
Applicable to: m-by-n matrix A.
https://en.wikipedia.org/wiki/Matrix_decomposition#:~:text=In the mathematical discipline of,a particular class of problems. 5/9
8/23/23, 9:33 AM Matrix decomposition - Wikipedia

Decomposition: , where T is a triangular matrix, and U and V are unitary matrices.

Comment: Similar to the singular value decomposition and to the Schur decomposition.

Other decompositions

Polar decomposition
Applicable to: any square complex matrix A.
Decomposition: (right polar decomposition) or (left polar decomposition),
where U is a unitary matrix and P and P' are positive semidefinite Hermitian matrices.
Uniqueness: is always unique and equal to (which is always hermitian and positive
semidefinite). If is invertible, then is unique.
Comment: Since any Hermitian matrix admits a spectral decomposition with a unitary matrix,
can be written as . Since is positive semidefinite, all elements in are non-
negative. Since the product of two unitary matrices is unitary, taking one can write
which is the singular value decomposition. Hence, the existence of
the polar decomposition is equivalent to the existence of the singular value decomposition.

Algebraic polar decomposition

Applicable to: square, complex, non-singular matrix A.[5]
Decomposition: , where Q is a complex orthogonal matrix and S is complex symmetric
matrix.
Uniqueness: If has no negative real eigenvalues, then the decomposition is unique.[6]
Comment: The existence of this decomposition is equivalent to being similar to .[7]
Comment: A variant of this decomposition is , where R is a real matrix and C is a
[6]
circular matrix.

Mostow's decomposition
Applicable to: square, complex, non-singular matrix A.[8][9]
Decomposition: , where U is unitary, M is real anti-symmetric and S is real
symmetric.
Comment: The matrix A can also be decomposed as , where U2 is unitary, M2
is real anti-symmetric and S2 is real symmetric.[6]

Sinkhorn normal form

Applicable to: square real matrix A with strictly positive elements.
Decomposition: , where S is doubly stochastic and D1 and D2 are real diagonal
matrices with strictly positive elements.

Sectoral decomposition
Applicable to: square, complex matrix A with numerical range contained in the sector
.

https://en.wikipedia.org/wiki/Matrix_decomposition#:~:text=In the mathematical discipline of,a particular class of problems. 6/9

8/23/23, 9:33 AM Matrix decomposition - Wikipedia

Decomposition: , where C is an invertible complex matrix and

with all .[10][11]

Williamson's normal form

Applicable to: square, positive-definite real matrix A with order 2n×2n.
Decomposition: , where is a symplectic matrix and D is a
nonnegative n-by-n diagonal matrix.[12]

Matrix square root

Decomposition: , not unique in general.
In the case of positive semidefinite , there is a unique positive semidefinite such that
.

Generalizations
There exist analogues of the SVD, QR, LU and Cholesky factorizations for quasimatrices and cmatrices or
continuous matrices.[13] A ‘quasimatrix’ is, like a matrix, a rectangular scheme whose elements are
indexed, but one discrete index is replaced by a continuous index. Likewise, a ‘cmatrix’, is continuous in
both indices. As an example of a cmatrix, one can think of the kernel of an integral operator.

These factorizations are based on early work by Fredholm (1903), Hilbert (1904) and Schmidt (1907). For
an account, and a translation to English of the seminal papers, see Stewart (2011).

See also
Matrix splitting
Non-negative matrix factorization
Principal component analysis

References

Notes
1. If a non-square matrix is used, however, then the matrix U will also have the same rectangular
shape as the original matrix A. And so, calling the matrix U would be incorrect as the correct
term would be that U is the 'row echelon form' of A. Other than this, there are no differences in
LU factorization for square and non-square matrices.

Citations
1. Lay, David C. (2016). Linear algebra and its applications (https://www.worldcat.org/oclc/920463
015). Steven R. Lay, Judith McDonald (Fifth Global ed.). Harlow. p. 142. ISBN 978-1-292-
09223-2. OCLC 920463015 (https://www.worldcat.org/oclc/920463015).
2. Piziak, R.; Odell, P. L. (1 June 1999). "Full Rank Factorization of Matrices". Mathematics
Magazine. 72 (3): 193. doi:10.2307/2690882 (https://doi.org/10.2307%2F2690882).
JSTOR 2690882 (https://www.jstor.org/stable/2690882).

https://en.wikipedia.org/wiki/Matrix_decomposition#:~:text=In the mathematical discipline of,a particular class of problems. 7/9

8/23/23, 9:33 AM Matrix decomposition - Wikipedia

3. Uhlmann, J.K. (2018), "A Generalized Matrix Inverse that is Consistent with Respect to
Diagonal Transformations", SIAM Journal on Matrix Analysis and Applications, 239 (2): 781–
800, doi:10.1137/17M113890X (https://doi.org/10.1137%2F17M113890X)
4. Uhlmann, J.K. (2018), "A Rank-Preserving Generalized Matrix Inverse for Consistency with
Respect to Similarity", IEEE Control Systems Letters, 3: 91–95, arXiv:1804.07334 (https://arxi
v.org/abs/1804.07334), doi:10.1109/LCSYS.2018.2854240 (https://doi.org/10.1109%2FLCSY
S.2018.2854240), ISSN 2475-1456 (https://www.worldcat.org/issn/2475-1456),
S2CID 5031440 (https://api.semanticscholar.org/CorpusID:5031440)
5. Choudhury & Horn 1987, pp. 219–225
6. Bhatia, Rajendra (2013-11-15). "The bipolar decomposition". Linear Algebra and Its
Applications. 439 (10): 3031–3037. doi:10.1016/j.laa.2013.09.006 (https://doi.org/10.1016%2F
j.laa.2013.09.006).
7. Horn & Merino 1995, pp. 43–92
8. Mostow, G. D. (1955), Some new decomposition theorems for semi-simple groups (https://arch
ive.org/details/liealgebrasandli029541mbp), Mem. Amer. Math. Soc., vol. 14, American
Mathematical Society, pp. 31–54
9. Nielsen, Frank; Bhatia, Rajendra (2012). Matrix Information Geometry. Springer. p. 224.
arXiv:1007.4402 (https://arxiv.org/abs/1007.4402). doi:10.1007/978-3-642-30232-9 (https://doi.
org/10.1007%2F978-3-642-30232-9). ISBN 9783642302329. S2CID 118466496 (https://api.se
manticscholar.org/CorpusID:118466496).
10. Zhang, Fuzhen (30 June 2014). "A matrix decomposition and its applications" (https://zenodo.o
rg/record/851661). Linear and Multilinear Algebra. 63 (10): 2033–2042.
doi:10.1080/03081087.2014.933219 (https://doi.org/10.1080%2F03081087.2014.933219).
S2CID 19437967 (https://api.semanticscholar.org/CorpusID:19437967).
11. Drury, S.W. (November 2013). "Fischer determinantal inequalities and Highamʼs Conjecture" (h
ttps://doi.org/10.1016%2Fj.laa.2013.08.031). Linear Algebra and Its Applications. 439 (10):
3129–3133. doi:10.1016/j.laa.2013.08.031 (https://doi.org/10.1016%2Fj.laa.2013.08.031).
12. Idel, Martin; Soto Gaona, Sebastián; Wolf, Michael M. (2017-07-15). "Perturbation bounds for
Williamson's symplectic normal form". Linear Algebra and Its Applications. 525: 45–58.
arXiv:1609.01338 (https://arxiv.org/abs/1609.01338). doi:10.1016/j.laa.2017.03.013 (https://doi.
org/10.1016%2Fj.laa.2017.03.013). S2CID 119578994 (https://api.semanticscholar.org/Corpus
ID:119578994).
13. Townsend & Trefethen 2015

Bibliography
Choudhury, Dipa; Horn, Roger A. (April 1987). "A Complex Orthogonal-Symmetric Analog of
the Polar Decomposition". SIAM Journal on Algebraic and Discrete Methods. 8 (2): 219–225.
doi:10.1137/0608019 (https://doi.org/10.1137%2F0608019).
Fredholm, I. (1903), "Sur une classe d'´equations fonctionnelles", Acta Mathematica (in
French), 27: 365–390, doi:10.1007/bf02421317 (https://doi.org/10.1007%2Fbf02421317)
Hilbert, D. (1904), "Grundzüge einer allgemeinen Theorie der linearen Integralgleichungen",
Nachr. Königl. Ges. Gött (in German), 1904: 49–91
Horn, Roger A.; Merino, Dennis I. (January 1995). "Contragredient equivalence: A canonical
form and some applications" (https://doi.org/10.1016%2F0024-3795%2893%2900056-6).
Linear Algebra and Its Applications. 214: 43–92. doi:10.1016/0024-3795(93)00056-6 (https://do
i.org/10.1016%2F0024-3795%2893%2900056-6).
Meyer, C. D. (2000), Matrix Analysis and Applied Linear Algebra (http://www.matrixanalysis.co
m/), SIAM, ISBN 978-0-89871-454-8
Schmidt, E. (1907), "Zur Theorie der linearen und nichtlinearen Integralgleichungen. I Teil.
Entwicklung willkürlichen Funktionen nach System vorgeschriebener" (https://zenodo.org/recor

https://en.wikipedia.org/wiki/Matrix_decomposition#:~:text=In the mathematical discipline of,a particular class of problems. 8/9

8/23/23, 9:33 AM Matrix decomposition - Wikipedia

d/1428258), Mathematische Annalen (in German), 63 (4): 433–476, doi:10.1007/bf01449770

(https://doi.org/10.1007%2Fbf01449770)
Simon, C.; Blume, L. (1994). Mathematics for Economists. Norton. ISBN 978-0-393-95733-4.
Stewart, G. W. (2011), Fredholm, Hilbert, Schmidt: three fundamental papers on integral
equations (http://www.cs.umd.edu/~stewart/FHS.pdf) (PDF), retrieved 2015-01-06
Townsend, A.; Trefethen, L. N. (2015), "Continuous analogues of matrix factorizations", Proc.
R. Soc. A, 471 (2173): 20140585, Bibcode:2014RSPSA.47140585T (https://ui.adsabs.harvard.
edu/abs/2014RSPSA.47140585T), doi:10.1098/rspa.2014.0585 (https://doi.org/10.1098%2Frs
pa.2014.0585), PMC 4277194 (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4277194),
PMID 25568618 (https://pubmed.ncbi.nlm.nih.gov/25568618)
Jun, Lu (2021), Numerical matrix decomposition and its modern applications: A rigorous first
course, arXiv:2107.02579 (https://arxiv.org/abs/2107.02579)

External links
Online Matrix Calculator (http://www.bluebit.gr/matrix-calculator/)
Wolfram Alpha Matrix Decomposition Computation » LU and QR Decomposition (http://www.w
olframalpha.com/input/?i=matrix+decomposition&rawformassumption={%22C%22,+%22matrix
+decomposition%22}+-%3E+{%22Calculator%22}&rawformassumption={%22MC%22,%22%2
2}-%3E{%22Formula%22})
Springer Encyclopaedia of Mathematics » Matrix factorization (https://encyclopediaofmath.org/
wiki/Matrix_factorization)
GraphLab (https://web.archive.org/web/20110314171151/http://www.graphlab.ml.cmu.edu/pmf.
html) GraphLab collaborative filtering library, large scale parallel implementation of matrix
decomposition methods (in C++) for multicore.

Retrieved from "https://en.wikipedia.org/w/index.php?title=Matrix_decomposition&oldid=1171079414"

https://en.wikipedia.org/wiki/Matrix_decomposition#:~:text=In the mathematical discipline of,a particular class of problems. 9/9