Angle Between Subspaces
Angle Between Subspaces
Angle Between Subspaces
Abstract
We first review the definition of the angle between subspaces and how it is computed using matrix algebra.
Then we introduce the Grassmann and Clifford algebra description of subspaces. The geometric product of
two subspaces yields the full relative angular information in an explicit manner. We explain and interpret the
result of the geometric product of subspaces gaining thus full access to the relative orientation information.
Keywords: Clifford geometric algebra, subspaces, relative angle, principal angles, principal vectors.
AMS Subj. Class.: 15A66.
• Two lines given by two unit vectors
g y lines A, B, • or enclose a finite angle 0 < θk ≤ π/2.
Figure 1: Angle θA,B between two
Angles between Subspaces A,B
g
spanned by unit p
vectors a, b, respectively.
– Line A: spanned by vector a In addition the pairs of vectors {a k , b k }, 1 ≤ k ≤ r
span mutually orthogonal lines (for θk = 0) and
– Line B: spanned by vector b
Line B: spanned by vector b (principal) planes ik (for 0 < θk ≤ π/2). These mu-
• Angle 0 ≤ A,B ≤ between lines A,B tually orthogonal planes ik are indicated in Fig. 2.
1 b b
Therefore if a k ∦ b k and a l ∦ b l for 1 ≤ k 6= l ≤ r,
3cos A,B = a·b
b r
2 b b then plane ik is orthogonal to plane il . The cosines
B
of the socalled principal angles θk may therefore
1
be cos θk = 1 (for a k = b k ), or cos θk = 0 (for
a 2 3 r
a k ⊥ b k ), or any value 0 < cos θk < 1. The total
a
1 r angle between the two subspaces A, B is defined as
A i1 a2 a3 the product
ir
i3 cos θA,B = cos θ1 cos θ2 . . . cos θr . (4)
i2
In this definition cos θA,B will automatically be zero
A span(a1,a2,arelationship
A = span(a
Figure 2: Angular 3,…,ar), span(b1,b2A,
), B=span(b
of two subspaces
,, bb3, …, br)pair of principal vectors {a k , b k }, 1 ≤ k ≤ r is
if any
…, b
perpendicular. Then the two subspaces are said to
B, spanned cos byA,Btwo= sets 1 vectors
cos of cos 2 {a
cos1 ,. 3. .… cos r
, a r }, and be perpendicular A ⊥ B, a familiar notion from three
{b 1 , . . . , b r }, respectively. dimensions, where two perpendicular planes A, B
share a common line spanned by a 1 = b 1 , and have
two mutually orthogonal principal vectors a 2 ⊥ b 2 ,
The angle 0 ≤ θA,B ≤ π/2 between lines A and B is which are both in turn orthogonal to the common
simply given by line vector a 1 . It is further possible to choose the
indexes of the vector pairs {a k , b k }, 1 ≤ k ≤ r such
cos θA,B = a · b. (2) that the principle angles θk appear ordered by mag-
nitude
θ1 ≥ θ2 ≥ . . . ≥ θr . (5)
3 Angles between two sub-
spaces (described by princi- 4 Matrix algebra computation
pal vectors) of angle between subspaces
Next let us examine the case of two r-dimensional The conventional method of computing the angle
(r ≤ n) subspaces A, B of an n-dimensional Eu- θA,B between two r-dimensional subspaces A, B ⊂
clidean vector space Rn . The situation is depicted Rn spanned by two sets of vectors {a 01 , a 02 , . . . a 0r }
in Fig. 2. Each subspace A, B is spanned by a set and {b 01 , b 02 , . . . b 0r } is to first arrange these vectors
of r linearly independent vectors as column vectors into two n × r matrices
A = span{a 1 , . . . , a r } ⊂ Rn , MA = [a 01 , . . . , a 0r ], MB = [b 01 , . . . , b 0r ]. (6)
n
B = span{b 1 , . . . , b r } ⊂ R . (3)
Then standard matrix algebra methods of QR de-
Using Fig. 2 we introduce the following notation composition and singular value decomposition are
for principal vectors. The angular relationship be- applied to obtain
Vectors and Bivectors
Vectors and Bivectors
• r pairs of singular unit vectors a k , b k and b a
oriented
• r singular values σk = cos θk = a k · b k . unit area
1+a2 e2, b =
This approach is very computation b1e1+b2e2
intensive. a b
(a1b2 a2b1) e1e2 = |a||b| sin e1e2
5 Even more subtle b∧a …oriented
ways area spanned by a,b
0 ⇔ a || b ⇔ b = a , ∊R R
Per Ake Wedin in his 1983th direction a : {x ∊R | x∧a = 0}
contribution [2] to
n a
conference on Matrix Pencils entitled On Angles
between Subspaces of a rs can be
Finite freely reshaped
Dimensional Inner
a∧b
Product Space first carefully treats the= above
a∧(bmen- + a) a
tioned matrix algebra approach to computing
a product of orthogonal thevectors
angle θA,B in great detail and clarity. Towards the
∧bless
a∧b = ab’ a page
b’ , one b’ (a
( to· b’ = 0 )
b’
end of his paper he dedicates than
mentioning an alternative method starting out with a
the words: But there are even more subtle ways to
define angle functions. Figure 3: Bivectors a ∧ b as oriented area elements
There he essentially reviews how r-dimensional can be reshaped (e.g. by b → b + µa, µ ∈ R) with-
subspaces A, B ⊂ Rn can be represented by r-vectors out changing their value (area and orientation). The
(blades) in Grassmann algebra A, B ∈ Λ(Rn ): bottom figure shows orthogonal reshaping into the
form of an oriented rectangle.
A = {x ∈ Rn |x ∧ A = 0},
B = {x ∈ Rn |x ∧ B = 0}. (7)
The angle θA,B between the two subspaces A, B ∈ Rn 6 Clifford (geometric) algebra
can then be computed in a single step
Clifford (geometric) algebra is based on the geomet-
ric product of vectors a, b ∈ Rp,q , p + q = n
A·Be
cos θA,B = = cos θ1 cos θ2 . . . cos θr , (8)
|A||B| ab = a · b + a ∧ b, (9)
where the inner product is canonically defined on
and the associative algebra Clp,q thus generated
the Grassmann algebra Λ(Rn ) corresponding to the
with R and Rp,q as subspaces of Clp,q . a · b is
geometry of Rn . The tilde operation is the reverse
the symmetric inner product of vectors and a ∧ b
operation representing a dimension dependent sign
is Grassmann’s outer product of vectors represent-
change B e = (−1) r(r−1)
2 B, and |A| represents the ing the oriented parallelogram area spanned by a, b,
norm of blade A, i.e. |A|2 = A · A, e and similarly
compare Fig. 3.
2
|B| = B · B. Wedin refers to earlier works of L.
e As an example we take the Clifford geometric al-
Andersson [1] in 1980, and a 1963 paper of Q.K. Lu gebra Cl3 = Cl3,0 of three-dimensional (3D) Eu-
[5]. clidean space R3 = R3,0 . R3 has an orthonormal
Yet equipping a Grassmann algebra Λ(Rn ) with a basis {e1 , e2 , e3 }. Cl3 then has an eight-dimensional
canonical inner product comes close to introducing basis of
Clifford’s geometric algebra Cln = Cl(Rn ). And
there is another good reason to do that, as e.g. H. {1, e1 , e2 , e3 , e2 e3 , e3 e1 , e1 e2 , i = e1 e2 e3 }. (10)
Li explains in his excellent 2008 textbook Invariant
| {z } | {z } | {z }
vectors area bivectors volume trivector
Algebras and Geometric Reasoning [4]: ... to allow
sums of angles to be advanced invariants, the inner- Here i denotes the unit trivector, i.e. the oriented
product Grassmann algebra must be extended to the volume of a unit cube, with i2 = −1. The even
Clifford algebra ... This is why I have decided to grade subalgebra Cl3+ is isomorphic to Hamilton’s
immediately begin in the next section with Clifford’s quaternions H. Therefore elements of Cl3+ are also
geometric algebra instead of first reviewing inner- called rotors (rotation operators), rotating vectors
product Grassmann algebra. and multivectors of Cl3 .
In general Clp,q , p+q = n is composed of so-called Whereas the antisymmetric outer product part gives
r-vector subspaces spanned by the induced bases the bivector, which represents the oriented area of
the parallelogram spanned by a and b
{e k1 e k2 . . . e kr | 1 ≤ k1 < k2 < . . . < kr ≤ n},
(11) a ∧ b = (a1 b2 − a2 b1 )e 1 e 2 = |a||b| sin θa ,b e 1 e 2 .
each with dimension nr . The total dimension of (18)
Pn
the Clp,q therefore becomes r=0 nr = 2n . The parallelogram has the (signed) scalar area
General elements called multivectors M ∈ |a||b| sin θa ,b and its orientation in the space Rn
Clp,q , p + q = n, have k-vector parts (0 ≤ k ≤ n): is given by the oriented unit area bivector e 1 e 2 .
scalar part Sc(M ) = hM i = hM i0 = M0 ∈ R, vec- Two non-zero vectors a and b are parallel, if and
tor part hM i1 ∈ Rp,q , bi-vector hM i2 , . . . , and only if a ∧ b = 0, i.e. if and only if sin θa ,b = 0
Vn part
pseudoscalar part hM in ∈ Rp,q
a ∧ b = 0 ⇔ a k b ⇔ b = αa, α ∈ R. (19)
n
2
X We can therefore use the outer product to represent
M= MA e A = hM i+hM i1 +hM i2 +. . .+hM in .
a line A = span{a} with direction vector a ∈ Rn as
A=1
(12)
A = {x ∈ Rn | x ∧ a = 0}. (20)
The reverse of M ∈ Clp,q defined as
Moreover, bivectors can be freely reshaped (see Fig.
n
X k(k−1) 3), e.g.
M
f= (−1) 2 hM ik , (13)
k=0
a ∧ b = a ∧ (b + µa), µ ∈ R, (21)
often replaces complex conjugation and quaternion because due to the antisymmetry a ∧a = 0. This re-
conjugation. Taking the reverse is equivalent to re- shaping allows to (orthogonally) reshape a bivector
versing the order of products ob basis vectors in the to rectangular shape
basis blades of (11). For example the reverse of the
bivector e 1 e 2 is a ∧ b = ab 0 , a ⊥ b(i.e. a · b 0 = 0) (22)
1 e 2 = e 2 e 1 = −e 1 e 2 ,
e] (14) as indicated in Fig. 3. The shape may even chosen
as square or circular, depending on the application
because only the antisymmetric outer product part in mind.
e 2 e 1 = e 2 ∧ e 1 = −e 1 ∧ e 2 is relevant. The total antisymmetry of the trivector x ∧ a ∧ b
The scalar product of two multivectors M, Ne ∈ means that
Clp,q is defined as
x ∧ a ∧ b = 0 ⇔ x = αa + βb, α, β ∈ R. (23)
M ∗N
e = hM N
e i = hM N
e i0 . (15)
Therefore a plane B is given by a simple bivector
e ∈ Cln = Cln,0 we get M ∗ N (also called 2-blade) B = a ∧ b as
For M, N
P
e =
A MA NA . The modulus |M | of a multivector M ∈ B = {x ∈ Rn | x ∧ B = 0}. (24)
Cln is defined as
X A three-dimensional volume subspace C is simi-
|M |2 = M ∗ M
f= MA2 . (16) larly given by a 3-blade C = a ∧ b ∧ c as
A
C = {x ∈ Rn | x ∧ C = 0}. (25)
6.1 Subspaces described in geometric Finally a blade Dr = b 1 ∧ b 2 ∧ . . . ∧ b r , b l ∈
algebra Rn , 1 ≤ l ≤ r ≤ n describes an r-dimensional vector
subspace
In Cln symmetric inner product part of two vectors
a = a1 e 1 +a2 e 2 , b = b1 e 1 +b2 e 2 yields the expected D = {x ∈ Rp,q |x ∧ D = 0}. (26)
result
Its dual blade
a · b = a1 b1 + a2 b2 = |a||b| cos θa ,b . (17) D∗ = Di−1
n (27)
describes the complimentary (n − r)-dimensional therefore
vector subspace D⊥ . The magnitude of the blade |Ar | = |a 1 | . . . |a r |. (36)
Dr ∈ Cln is nothing but the volume of the r-
Every r-blade Ar can therefore be written as a
dimensional parallelepiped spanned by the vectors
product of the scalar magnitude |Ar | times the ge-
{b 1 , b 2 , . . . , b r }.
ometric product of exactly r mutually orthogonal
Just as we were able to orthogonally reshaped a
unit vectors {b br}
a 1, . . . , a
bivector to rectangular or square shape we can re-
shape every r-blade Ar to a geometric product of Ar = |Ar |b
a 1a
b2 . . . a
br. (37)
mutually orthogonal vectors
Please note well, that this rewriting of an r-blade
Ar = a 01 ∧ a 02 ∧ . . . a 0r = a 1 a 2 . . . a r , (28) in geometric algebra does not influence the overall
with pairwise orthogonal and anticommuting vec- result on the left side, the r-blade Ar is before and
tors a 1 ⊥ a 2 ⊥ . . . ⊥ a r . The reverse Aer of after the rewriting the very same element of the geo-
the geometric product of orthogonal vectors Ar = metric algebra Cln . But for the geometric interpre-
a 1 a 2 . . . a r is therefore clearly tation of the geometric product AB of two r-blades
A, B ∈ Cln the orthogonal reshaping is indeed a key
r(r−1)
(a 1 a 2 . . . a r )∼ = a r . . . a 2 a 1 = (−1) 2 a 1a 2 . . . a r , step.
(29) After a short discussion of reflections and rota-
tions implemented in geometric algebra, we return
by simply counting the number r(r−1)
2 of permuta-
to the geometric product of two r-blades A, B ∈ Cln
tions necessary.
and present our key insight.
Paying attention to the dimensions we find that
the outer product of an r-blade B with a vector a
increases the dimension (grade) by +1 6.2 Reflections and rotations
a ∧ B = haBir+1 . (30) A simple application of the geometric product is
shown in Fig. 4 (left) to the reflection of a point
Opposite to that, the inner product (or left contrac- vector x at a plane with normal vector a, which
tion) with a vector lowers the dimension (grade) by means to reverse the component of x parallel to a
−1 (perpendicular to the plane)
a · B = haBir−1 . (31)
a
The geometric product of two r-blades A, B contains x −→ x 0 = −a −1 x a, a −1 = . (38)
a2
therefore at most the following grades
Two reflections lead to a rotation by twice the angle
AB = hABi0 +hABi2 +. . .+hABi2min(r,[n/2]) , (32) between the reflection planes as shown in Fig. 4
(right)
where the limit [n/2] (entire part of n/2) is due to
the dimension limit of Rn . x −→ x 00 = a −1 b −1 x ab = (ab)x ab = R−1 x R,
The inner product of vectors is properly general- (39)
ized in geometric algebra by introducing the (left) with rotation operator (rotor) R = ab ∝ cos θa ,b +
contraction of the r-blade A = Ar onto the s-blade ia ,b sin θa ,b , where the unit bivector ia ,b represents
B = Bs as the plane of rotation.
Ar cBs = hABis−r . (33)
For blades of equal grade (r = s) we thus get the 7 Geometric information in
symmetric scalar
the geometric product of two
Ar cBr = hABi0 = hBAi0 = A ∗ B. (34)
subspace r-blades
Finally the product of a blade with its own re-
verse is necessarily scalar. Introducing orthogonal From the foregoing discussion of the representation
reshaping this scalar is seen to be of r-dimensional subspaces A, B ⊂ Rn by the blades
A = a 01 ∧ . . . a 0r and B = b 01 ∧ . . . b 0r , from the free-
er = a 1 . . . a r a r . . . a 1 = a 21 . . . a 2r = |Ar |2 ,
Ar A dom of orthogonally reshaping these blades and fac-
(35) toring out the blade magnitudes |A| and |B|, and
−x|| x|| x' |A||B|(s1 c2 . . . cr i1 +c1 s2 . . . cr i2 +. . .+c1 c2 . . . sr ir ),
x x'' α x etc., up to the 2r-vector (or 2[n/2]-vector) part
x' x⊥ hABi
e 2r = |A||B|s1 s2 . . . sr i1 i2 . . . ir of the geometric
product AB e arise and what information they carry.
a b
α/2
Obviously the scalar part yields the cosine of the
angle between the subspaces represented by the two
a r-vectors A, B ∈ Cln
hABi
e 0 A∗B e
cos θAB = = , (43)
|A||B| |A||B|
Figure 4: Reflection at a plane with normal a (left)
and rotation as double reflection at planes normal which exactly corresponds to P.A. Wedin’s formula
to a, b (right). from inner-product Grassmann algebra.
The bivector part consists of a sum of (principal)
plane bivectors, which can in general be uniquely
from the classical results of matrix algebra, we now decomposed into its constituent sum of 2-blades by
know that we can rewrite the geometric product AB the method of Riesz, described also in [6], chapter
in mutually orthogonal products of pairs of principal 3-4, equation (4.11a) and following.
vectors a k , b k , 1 ≤ k ≤ r The magnitude of the 2r-vector part allows to
compute the product of all sines of principal angles
AB
e = a 1a 2 . . . a r b r . . . b 2b 1 = a 1b 1a 2b 2 . . . a r b r .
(40)
|hABi
e 2r |
s1 s2 . . . sr = ± . (44)
The geometric product |A||B|
a r b r = |a r ||b r |(cos θa r ,b r + ia r ,b r sin θa r ,b r ) Let us finally refine our considerations to two gen-
= |a r ||b r |(cr + ir sr ), (41) eral r-dimensional subspaces A, B, which we take to
partly intersect and to be partly perpendicular. We
with cr = cos θa r ,b r and sr = sin θa r ,b r in the mean by that, that the dimension of the intersect-
above expression for AB e is composed of a scalar ing subspace be s ≤ r (s is therefore the number
and a bivector part. The latter is proportional to of principal angles equal zero), and the number of
the unit bivector ir representing a (principal) plane principle angles with value π/2 be t ≤ r − s. For
orthogonal to all other principal vectors. ir there- simplicity we work with normed blades (i.e. after
fore commutes with all other principal vectors, and dividing with |A||B|. The geometric product of the
hence the whole product a r b r (a rotor) commutes the r-blades A, B ∈ Cln then takes the form
with all other principal vectors. A completely analo-
gous consideration applies to all products of pairs of AB
e
principal vectors, which proofs the second equality = (cs+1 cs+2 . . . cr−t +
in (40). + ss+1 cs+2 . . . cr−t is+1 + cs+1 ss+2 . . . cr−t is+2 +
We thus find that we can always rewrite the prod-
. . . + cs+1 cs+2 . . . sr−t ir−t +
uct AB e as a product of rotors
..
AB
e .
= |A||B|(c1 + i1 s1 )(c2 + i2 s2 ) . . . (cr + ir sr ) + ss+1 ss+2 . . . sr−t is+1 is+2 . . . ir−t )ir−t+1 . . . ir .
(45)
= |A||B|(c1 c2 . . . cr +
+ s1 c2 . . . cr i1 + c1 s2 . . . cr i2 + . . . + c1 c2 . . . sr ir + We thus see, that apart from the integer dimensions
.. s for parallelity (identical to the dimension of the
. meet of blade A with blade B) and t for perpendic-
+ s1 s2 . . . sr i1 i2 . . . ir ) (42) ularity, the lowest non-zero grade of dimension 2t
gives the relevant angular measure
We realize how the scalar part hABi e 0 =
|A||B|c1 c2 . . . cr , the bivector part hABi2
e = cos θAB = cos θs+1 cos θs+2 . . . cos θr−t . (46)
While the maximum grade part gives again the [3] L. Dorst et al, Geometric Algebra for Comp.
product of the corresponding sinuses Sc., Morgan Kaufmann, 2007.
sin θs+1 sin θs+2 . . . sin θr−t . (47) [4] H. Li, Invariant Algebras and Geometric Rea-
soning, World Scientific, Singapore, 2008.
Dividing the product AB e by its lowest grade part
cs+1 cs+2 . . . cr−t ir−t+1 . . . ir gives a multivector with [5] Q.K. Lu, The elliptic geometry of extended
maximum grade 2(r − t − s), scalar part one, and spaces, Acta Math. Sinica, 13 (1963), pp. 49–
bivector part 62; translated as Chinese Math. 4 (1963), pp.
54–69.
ts+1 is+1 + ts+2 is+2 + . . . + tr−t ir−t , (48)
[6] D. Hestenes, G. Sobczyk, Clifford Algebra to
where tk = tan θk . Splitting this bivector into its Geometric Calculus, Kluwer, 1984.
constituent bivector parts further yields the (prin-
cipal) plane bivectors and the tangens values of the [7] E. Hitzer, K. Tachibana, S. Buchholz, I. Yu
principle angles θk , s < k ≤ r − t. This is the only Carrier method for the general evaluation and
somewhat time intensive step. control of pose, molecular conformation, track-
ing, and the like Advances in Applied Clifford
Algebras, Vol. 19(2), pp. 339-364 (2009).
8 Conclusion
Let us conclude by discussing possible future appli-
cations of these results. The complete relative orien-
tation information in AB e should be ideal for a sub-
space structure self organizing map (SOM) type of
neural network. Not only data points, but the topol-
ogy of whole data subspace structures can then be
faithfully mapped to lower dimensions. Our discus-
sion gives meaningful results for partly intersecting
and partly perpendicular subspaces. Apart from ex-
tracting the bivector components, all computations
are done by multiplication. Projects like fast Clif-
ford algebra hardware developed at the TU Darm-
stadt (D. Hildenbrand et al) should be of interest
for applying the results of the paper to high dimen-
sional data sets. An extension to offset subspaces
(of projective geometry) and r-spheres (of confor-
mal geometric algebra) may be possible.
Acknowledgments
Soli deo gloria. I do thank my dear family, H. Ishi,
D. Hildenbrand and V. Skala.
References
[1] L. Andersson, The concepts of angle between
subspaces ... unpublished notes, Umea (1980).
[2] P. A. Wedin, On angles between subspaces of a
finite-dimensional inner product space, in Ma-
trix Pencils, Bo Kagstram and Axel Ruhe, eds.,
Springer-Verlag, Berlin, 1983, pp. 263–285.