Trace (linear algebra)

This article needs additional citations for verification. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed. Find sources: "Trace" linear algebra – news · newspapers · books · scholar · JSTOR (November 2023) (Learn how and when to remove this template message)

In linear algebra, the trace of a square matrix A, denoted tr(A),^[1] is defined to be the sum of elements on the main diagonal (from the upper left to the lower right) of A. The trace is only defined for a square matrix (n × n).

It can be proven that the trace of a matrix is the sum of its

into itself, since all matrices describing such an operator with respect to a basis are similar.

The trace is related to the derivative of the determinant (see Jacobi's formula).

Definition

The trace of an n × n square matrix A is defined as^{[1][2][3]: 34}

where a_ii denotes the entry on the ith row and ith column of A. The entries of A can be F. The trace is not defined for non-square matrices.

Example

Let A be a matrix, with

$${\displaystyle \mathbf {A} ={\begin{pmatrix}a_{11}&a_{12}&a_{13}\\a_{21}&a_{22}&a_{23}\\a_{31}&a_{32}&a_{33}\end{pmatrix}}={\begin{pmatrix}1&0&3\\11&5&2\\6&12&-5\end{pmatrix}}}$$

Then

$${\displaystyle \operatorname {tr} (\mathbf {A} )=\sum _{i=1}^{3}a_{ii}=a_{11}+a_{22}+a_{33}=1+5+(-5)=1}$$

Properties

Basic properties

The trace is a

$${\displaystyle {\begin{aligned}\operatorname {tr} (\mathbf {A} +\mathbf {B} )&=\operatorname {tr} (\mathbf {A} )+\operatorname {tr} (\mathbf {B} )\\\operatorname {tr} (c\mathbf {A} )&=c\operatorname {tr} (\mathbf {A} )\end{aligned}}}$$

for all square matrices A and B, and all scalars c.^[3]: 34

A matrix and its transpose have the same trace:^{[1][2][3]: 34}

This follows immediately from the fact that transposing a square matrix does not affect elements along the main diagonal.

Trace of a product

The trace of a square matrix which is the product of two matrices can be rewritten as the sum of entry-wise products of their elements, i.e. as the sum of all elements of their Hadamard product. Phrased directly, if A and B are two m × n matrices, then:

If one views any real m × n matrix as a vector of length mn (an operation called

:

$${\displaystyle 0\leq \left[\operatorname {tr} (\mathbf {A} \mathbf {B} )\right]^{2}\leq \operatorname {tr} \left(\mathbf {A} ^{2}\right)\operatorname {tr} \left(\mathbf {B} ^{2}\right)\leq \left[\operatorname {tr} (\mathbf {A} )\right]^{2}\left[\operatorname {tr} (\mathbf {B} )\right]^{2}\ ,}$$

if A and B are real .

The Frobenius inner product may be extended to a

.

The symmetry of the Frobenius inner product may be phrased more directly as follows: the matrices in the trace of a product can be switched without changing the result. If A and B are m × n and n × m real or complex matrices, respectively, then[1]^{[2][3]: 34 [note 1]}

This is notable both for the fact that AB does not usually equal BA, and also since the trace of either does not usually equal tr(A)tr(B).^{[note 2]} The similarity-invariance of the trace, meaning that tr(A) = tr(P⁻¹AP) for any square matrix A and any invertible matrix P of the same dimensions, is a fundamental consequence. This is proved by

Similarity invariance is the crucial property of the trace in order to discuss traces of as below.

Additionally, for real column vectors ${\displaystyle \mathbf {a} \in \mathbb {R} ^{n}}$ and ${\displaystyle \mathbf {b} \in \mathbb {R} ^{n}}$, the trace of the outer product is equivalent to the inner product:

Cyclic property

More generally, the trace is invariant under circular shifts, that is,

This is known as the cyclic property.

Arbitrary permutations are not allowed: in general,

However, if products of three symmetric matrices are considered, any permutation is allowed, since:

where the first equality is because the traces of a matrix and its transpose are equal. Note that this is not true in general for more than three factors.

Trace of a Kronecker product

The trace of the Kronecker product of two matrices is the product of their traces:

Characterization of the trace

The following three properties:

characterize the trace up to a scalar multiple in the following sense: If ${\displaystyle f}$ is a on the space of square matrices that satisfies ${\displaystyle f(xy)=f(yx),}$ then ${\displaystyle f}$ and ${\displaystyle \operatorname {tr} }$ are proportional.^{[note 3]}

For ${\displaystyle n\times n}$ matrices, imposing the normalization ${\displaystyle f(\mathbf {I} )=n}$ makes ${\displaystyle f}$ equal to the trace.

Trace as the sum of eigenvalues

Given any n × n matrix A, there is

where λ₁, ..., λ_n are the

, together with the similarity-invariance of the trace discussed above.

Trace of commutator

When both A and B are n × n matrices, the trace of the (ring-theoretic)

). In particular, using similarity invariance, it follows that the identity matrix is never similar to the commutator of any pair of matrices.

Conversely, any square matrix with zero trace is a linear combination of the commutators of pairs of matrices.^{[note 4]} Moreover, any square matrix with zero trace is unitarily equivalent to a square matrix with diagonal consisting of all zeros.

Traces of special kinds of matrices

Relationship to the characteristic polynomial

The trace of an ${\displaystyle n\times n}$ matrix ${\displaystyle A}$ is the coefficient of ${\displaystyle t^{n-1}}$ in the characteristic polynomial, possibly changed of sign, according to the convention in the definition of the characteristic polynomial.

Relationship to eigenvalues

If A is a linear operator represented by a square matrix with

), then

This follows from the fact that A is always

of A is the product of its eigenvalues; that is,

$${\displaystyle \det(\mathbf {A} )=\prod _{i}\lambda _{i}.}$$

Everything in the present section applies as well to any square matrix with coefficients in an algebraically closed field.

Derivative relationships

If ΔA is a square matrix with small entries and I denotes the identity matrix, then we have approximately

Precisely this means that the trace is the derivative of the determinant function at the identity matrix. Jacobi's formula

is more general and describes the

of the matrix.

From this (or from the connection between the trace and the eigenvalues), one can derive a relation between the trace function, the matrix exponential function, and the determinant:

A related characterization of the trace applies to linear vector fields. Given a matrix A, define a vector field F on Rⁿ by F(x) = Ax. The components of this vector field are linear functions (given by the rows of A). Its divergence div F is a constant function, whose value is equal to tr(A).

By the divergence theorem, one can interpret this in terms of flows: if F(x) represents the velocity of a fluid at location x and U is a region in Rⁿ, the net flow of the fluid out of U is given by tr(A) · vol(U), where vol(U) is the volume of U.

The trace is a linear operator, hence it commutes with the derivative:

Trace of a linear operator

In general, given some linear map f : V → V (where V is a finite-

, allowing for the possibility of a basis-independent definition for the trace of a linear map.

Such a definition can be given using the

of V. Let v be in V and let g be in V*. Then the trace of the indecomposable element v ⊗ g is defined to be g(v); the trace of a general element is defined by linearity. The trace of a linear map f : V → V can then be defined as the trace, in the above sense, of the element of V ⊗ V* corresponding to f under the above mentioned canonical isomorphism. Using an explicit basis for V and the corresponding dual basis for V*, one can show that this gives the same definition of the trace as given above.

Numerical algorithms

Stochastic estimator

The trace can be estimated unbiasedly by "Hutchinson's trick":[5]

Given any matrix ${\displaystyle W\in \mathbb {R} ^{n\times n}}$, and any random ${\displaystyle u\in \mathbb {R} ^{n}}$ with ${\displaystyle E[uu^{T}]=I}$, we have ${\displaystyle E[u^{T}Wu]=tr(W)}$. (Proof: expand the expectation directly.)

Usually, the random vector is sampled from ${\displaystyle N(0,I)}$ (normal distribution) or ${\displaystyle \{\pm n^{-1/2}\}^{n}}$ (Rademacher distribution).

More sophisticated stochastic estimators of trace have been developed.^[6]

Applications

If a 2 x 2 real matrix has zero trace, its square is a diagonal matrix.

The trace of a 2 × 2

.

The trace is used to define characters of group representations. Two representations A, B : G → GL(V) of a group G are equivalent (up to change of basis on V) if tr(A(g)) = tr(B(g)) for all g ∈ G.

The trace also plays a central role in the distribution of quadratic forms.

Lie algebra

The trace is a map of Lie algebras ${\displaystyle \operatorname {tr} :{\mathfrak {gl}}_{n}\to K}$ from the Lie algebra ${\displaystyle {\mathfrak {gl}}_{n}}$ of linear operators on an n-dimensional space (n × n matrices with entries in ${\displaystyle K}$) to the Lie algebra K of scalars; as K is Abelian (the Lie bracket vanishes), the fact that this is a map of Lie algebras is exactly the statement that the trace of a bracket vanishes:

The kernel of this map, a matrix whose trace is

${\displaystyle {\mathfrak {sl}}_{n}}$, which is the Lie algebra of the special linear group of matrices with determinant 1. The special linear group consists of the matrices which do not change volume, while the special linear Lie algebra is the matrices which do not alter volume of infinitesimal sets.

In fact, there is an internal

decomposition ${\displaystyle {\mathfrak {gl}}_{n}={\mathfrak {sl}}_{n}\oplus K}$ of operators/matrices into traceless operators/matrices and scalars operators/matrices. The projection map onto scalar operators can be expressed in terms of the trace, concretely as:

Formally, one can compose the trace (the

map) with the unit map ${\displaystyle K\to {\mathfrak {gl}}_{n}}$ of "inclusion of " to obtain a map ${\displaystyle {\mathfrak {gl}}_{n}\to {\mathfrak {gl}}_{n}}$ mapping onto scalars, and multiplying by n. Dividing by n makes this a projection, yielding the formula above.

In terms of

, one has which is analogous to (where ${\displaystyle K^{*}=K\setminus \{0\}}$) for Lie groups. However, the trace splits naturally (via ${\displaystyle 1/n}$ times scalars) so ${\displaystyle {\mathfrak {gl}}_{n}={\mathfrak {sl}}_{n}\oplus K}$, but the splitting of the determinant would be as the nth root times scalars, and this does not in general define a function, so the determinant does not split and the general linear group does not decompose:

Bilinear forms

The bilinear form (where X, Y are square matrices)

is called the Killing form, which is used for the classification of Lie algebras.

The trace defines a bilinear form:

The form is symmetric, non-degenerate^{[note 5]} and associative in the sense that:

For a complex simple Lie algebra (such as ${\displaystyle {\mathfrak {sl}}}$_n), every such bilinear form is proportional to each other; in particular, to the Killing form^{[citation needed]}.

Two matrices X and Y are said to be trace orthogonal if

There is a generalization to a general representation ${\displaystyle (\rho ,{\mathfrak {g}},V)}$ of a Lie algebra ${\displaystyle {\mathfrak {g}}}$, such that ${\displaystyle \rho }$ is a homomorphism of Lie algebras ${\displaystyle \rho :{\mathfrak {g}}\rightarrow {\text{End}}(V).}$ The trace form ${\displaystyle {\text{tr}}_{V}}$ on ${\displaystyle {\text{End}}(V)}$ is defined as above. The bilinear form

is symmetric and invariant due to cyclicity.

Generalizations

The concept of trace of a matrix is generalized to the

norm.

If K is a trace-class operator, then for any orthonormal basis ${\displaystyle (e_{n})_{n}}$, the trace is given by

and is finite and independent of the orthonormal basis.^[7]

The partial trace is another generalization of the trace that is operator-valued. The trace of a linear operator Z which lives on a product space A ⊗ B is equal to the partial traces over A and B:

For more properties and a generalization of the partial trace, see traced monoidal categories.

If A is a general associative algebra over a field k, then a trace on A is often defined to be any map tr : A ↦ k which vanishes on commutators; tr([a,b]) = 0 for all a, b ∈ A. Such a trace is not uniquely defined; it can always at least be modified by multiplication by a nonzero scalar.

A supertrace is the generalization of a trace to the setting of superalgebras.

The operation of tensor contraction generalizes the trace to arbitrary tensors.

Traces in the language of tensor products

Given a vector space V, there is a natural bilinear map V × V^∗ → F given by sending (v, φ) to the scalar φ(v). The universal property of the tensor product V ⊗ V^∗ automatically implies that this bilinear map is induced by a linear functional on V ⊗ V^∗.^[8]

Similarly, there is a natural bilinear map V × V^∗ → Hom(V, V) given by sending (v, φ) to the linear map w ↦ φ(w)v. The universal property of the tensor product, just as used previously, says that this bilinear map is induced by a linear map V ⊗ V^∗ → Hom(V, V). If V is finite-dimensional, then this linear map is a

This fundamental fact is a straightforward consequence of the existence of a (finite) basis of V, and can also be phrased as saying that any linear map V → V can be written as the sum of (finitely many) rank-one linear maps. Composing the inverse of the isomorphism with the linear functional obtained above results in a linear functional on Hom(V, V). This linear functional is exactly the same as the trace.

Using the definition of trace as the sum of diagonal elements, the matrix formula tr(AB) = tr(BA) is straightforward to prove, and was given above. In the present perspective, one is considering linear maps S and T, and viewing them as sums of rank-one maps, so that there are linear functionals φ_i and ψ_j and nonzero vectors v_i and w_j such that S(u) = Σφ_i(u)v_i and T(u) = Σψ_j(u)w_j for any u in V. Then

${\displaystyle (S\circ T)(u)=\sum _{i}\varphi _{i}\left(\sum _{j}\psi _{j}(u)w_{j}\right)v_{i}=\sum _{i}\sum _{j}\psi _{j}(u)\varphi _{i}(w_{j})v_{i}}$

for any u in V. The rank-one linear map u ↦ ψ_j(u)φ_i(w_j)v_i has trace ψ_j(v_i)φ_i(w_j) and so

${\displaystyle \operatorname {tr} (S\circ T)=\sum _{i}\sum _{j}\psi _{j}(v_{i})\varphi _{i}(w_{j})=\sum _{j}\sum _{i}\varphi _{i}(w_{j})\psi _{j}(v_{i}).}$

Following the same procedure with S and T reversed, one finds exactly the same formula, proving that tr(S ∘ T) equals tr(T ∘ S).

The above proof can be regarded as being based upon tensor products, given that the fundamental identity of End(V) with V ⊗ V^∗ is equivalent to the expressibility of any linear map as the sum of rank-one linear maps. As such, the proof may be written in the notation of tensor products. Then one may consider the multilinear map V × V^∗ × V × V^∗ → V ⊗ V^∗ given by sending (v, φ, w, ψ) to φ(w)v ⊗ ψ. Further composition with the trace map then results in φ(w)ψ(v), and this is unchanged if one were to have started with (w, ψ, v, φ) instead. One may also consider the bilinear map End(V) × End(V) → End(V) given by sending (f, g) to the composition f ∘ g, which is then induced by a linear map End(V) ⊗ End(V) → End(V). It can be seen that this coincides with the linear map V ⊗ V^∗ ⊗ V ⊗ V^∗ → V ⊗ V^∗. The established symmetry upon composition with the trace map then establishes the equality of the two traces.[8]

For any finite dimensional vector space V, there is a natural linear map F → V ⊗ V'; in the language of linear maps, it assigns to a scalar c the linear map c⋅id_V. Sometimes this is called coevaluation map, and the trace V ⊗ V' → F is called evaluation map.^[8] These structures can be axiomatized to define categorical traces in the abstract setting of category theory.

See also

• Trace of a tensor with respect to a metric tensor
• Characteristic function
• Field trace
• Golden–Thompson inequality
• Singular trace
• Specht's theorem
• Trace class
• Trace identity
• Trace inequalities
• von Neumann's trace inequality

Notes

References

1. ^ ^{a b c d e} "Rank, trace, determinant, transpose, and inverse of matrices". fourier.eng.hmc.edu. Retrieved 2020-09-09.
2. ^
   Zbl 1079.00009
   . Retrieved 2020-09-09.
3. ^
   ISBN 9780070605022
   .
4. ISBN 9780521839402
   .
5. ISSN 0361-0918
   .
6. S2CID 5827717
   .
7. ISBN 978-1470417048
   .
8. ^
   Zbl 0808.17003
   .

• MR 0107649
  .
• MR 2978290
  .
• ISBN 978-0030105678
  .

External links

• "Trace of a square matrix", Encyclopedia of Mathematics, EMS Press, 2001 [1994]