Vectors and Matrices Notes.

Jonathan Coulthard
[email protected]

1 Index Notation
Index notation may seem quite intimidating at first, but once you get used to it, it will allow
us to prove some very tricky vector and matrix identities with very little effort. As with most
things, it will only become clearer with practice, and so it is a good idea to work through the
examples for yourself, and try out some of the exercises.

Example: Scalar Product

Let’s start off with the simplest possible example: the dot product. For real column vectors a
and b,
 
b1

b2 

a · b = aT b = a1 a2 a3 · · · b  = a1 b1 + a2 b2 + a3 b3 + · · · (1)

 3
..
.
or, written in a more compact notation
X
a·b= ai bi , (2)
i

where the σ means that we sum over all values of i.

Example: Matrix-Column Vector Product

Now let’s take matrix-column vector multiplication, Ax = b.
    
A11 A12 A23 · · · x1 b1
A21 A22 A23 · · ·  x2  b2 
A31 A32 A33 · · ·  x3  = b3  (3)
    
    
.. .. .. .. .. ..
. . . . . .

You are probably used to multiplying matrices by visualising multiplying the elements high-
lighted in the red boxes. Written out explicitly, this is

b2 = A21 x1 + A22 x2 + A23 x3 + · · · (4)

If we were to shift the A box and the b box down one place, we would instead get

b3 = A31 x1 + A32 x2 + A33 x3 + · · · (5)

1
It should be clear then, that in general, for the ith element of b, we can write

bi = Ai1 x1 + Ai2 x2 + Ai3 x3 + · · · (6)

Or, in our more compact notation, X

bi = Aij xj . (7)
j

Note that if the matrix A had only one column, then i would take only one value (i = 1).
b would then also only have one element (b1 ) making it a scalar. Our matrix-column vector
product would therefore reduce exactly to a dot product. In fact, we can interpret the elements
bi as the dot product between the row vector which is the ith row of A, and the column vector
x.

Example: Matrix-Matrix multiplication

One last simple example before we start proving some more nontrivial stuff. Consider the matrix
product AB = C.
    
A11 A12 A23 · · · B11 B12 B23 · · · C11 C12 C23 · · ·
A21 A22 A23 · · ·  B21 B22 B23 · · · C21 C22 C23 · · ·
A31 A32 A33 · · ·  B31 B32 B33 · · · = C31 C32 C33 · · · (8)
    
    
.. .. .. .. .. .. .. .. .. .. .. ..
. . . . . . . . . . . .

I have once again marked with a box the way that you are probably used to seeing these
multiplications done. Explicitly,

C32 = A31 B12 + A32 B22 + A33 B32 + · · · (9)

As with the previous example, you might interpret C23 as the dot product between the row
vector A3i , and the column vector Bi2 , i.e.
X
C23 = A2k Bk3 (10)
k

it is clear how this generalises to any element Cij ,

X
Cij = Aik Bkj (11)
k

The rule for matrix multiplication is: “make the inner index (k, in this case) the same, and sum
over it.”

Example: Trace of a product of matrices.

The trace of a matrix is defined to be the sum of its diagonal elements
X
Tr(C) = Cii (12)
i

We would like to prove that

Tr(AB) = Tr(BA) (13)

2
First, let’s take the definition of the matrix product (in index form) and plug it into the definiton
of the trace, i.e. plug X
Cij = Aik Bkj (14)
k
into X
Tr(C) = Cii . (15)
i
We obtain !
X X
Tr(AB) = Aik Bki (16)
i k
Now Aik , and Bki are just scalars (i.e. single elements of the matrices A and B respectively),
and so we can commute them.
!
X X
Tr(AB) = Bki Aik (17)
i k

Now, the thing inside the brackets is almost a matrix product, but we are summing over the
wrong index (the outer index rather than the inner one). The question is, can we swap the
order of the two summations? Because addition is commutative (a + b = b + a), we can 1 .
Exercise: By writingPout thePsums with a small number of terms, convince yourself that you
can indeed commute i and k .
P P
Finally, by swapping i and k , we obtain
!
X X
Tr(AB) = Bki Aik ,
k i
X
= (BA)kk ,
k
= Tr(BA). (18)
Exercise: Now try a slightly more complicated example for yourself. Using index notation,
prove that Tr(AB · · · YZ) is invariant under cyclic permutations of the matrices.

1.1 Einstein Summation convention

Our notation is much more compact than writing out huge matrices and trying to figure out
how the multiplications, etc. work in general. However, writing out Σs can become very
cumbersome. Since we have
XX XX
(stuff) = (stuff) (19)
i j i j
X XX
C (stuff) = C(stuff), (20)
j i j
(21)
we can just drop the Σs entirely, and adopt what is called Einstein summation convention. It
is simply summarised as follows:
1
Small health warning: Swapping the order of the summations amounts to reordering the terms in the sum.
This is fine if we are just summing over a finite number of terms. However, if the sums are infinite, we can only
reorder the terms if the sum converges absolutely. If it converges only conditionally, then we cannot reorder the
terms (see wikipedia for the definitions of these terms). You are unlikely to encounter a situation where this is
an issue, but be aware that such situations exist.

3
 • If an indexPappears twice in a term, then it is implied that we sum over it. For example,
Aij Bjk ≡ j Aij Bjk .
This is a very powerful convention that you will use extensively when you learn special and
general relativity in your third year. However, in the immediate future, it can also make proofs
of various matrix and vector identities very easy! For instance our proof of Tr(AB) = Tr(BA).
can be written as
Tr(AB) = Aik Bki
= Bki Aik (22)
= Tr(BA). (23)
(Note that in a problem sheet, or in an exam, you should explain each line in this proof—the
notation makes it look much more trivial than it really is!)

1.1.1 Tips
As a wise man once said: “With great power comes great responsibility.” While Einstein sum-
mation convention can indeed make our lives much easier, it can also produce a great deal of
nonsense if you are not very careful when keeping track of your indices. Here are some tips for
doing so:
• Free indices appear only once in an expression and thus are not summed over. Dummy
indices appear twice, and are implicitly summed over.
• To help avoid confusion, it is a good idea to use roman letters (i, j, k) for free indices, and
greek letters (λ, µ, ν) for dummy indicies.
• Dummy indices should never appear in the “final answer”.
• The free indices should always be the same in every term in an expression.
• An index should never appear more than twice in a single term.

1.1.2 The Kronecker Delta

The Kronecker delta is a useful symbol which crops up all the time. It is defined as
(
1, if i = j
δij = (24)
0, otherwise
It should be clear that this is basically a representation of the identity matrix. It also has the
useful property that if you sum over one of the indices, then it kills the sum, and replaces the
dummy index with the other (free) index. For example,
X
aij δjn = ain . (25)
j

(since the only nonzero term in the sum is j = n). If we were to use Einstein summation
convention, then we would write the above as
aiµ δµn = ain . (26)
As another example, the scalar product between two vectors a · b can be written as:
a · b = aµ bν δµν = aµ bµ (27)
Where the δµν forces the indices of a and b to be equal.

4
1.1.3 The Levi-Civita symbol
The Levi-Civita symbol, εijk is another handy object. It is defined as

1,
 if ijk is a cyclic permutation of 123
εijk = −1, if ijk is an anticyclic permutation of 123 (28)

0, if any two of i, j, or k are equal.


Why is such a thing useful? Well, let’s consider the object εiµν aµ bν . It has one free index, i, so
it is a vector. What are its components? If we set i = 1, from the definition of ε we see that
the only nonzero terms in the sum will be µ, ν 6= 0. This leaves us with µ, ν = 2 or 3. Writing
out these components explicitly, we find

ε1µν aµ bν = ε123 a2 b3 + ε132 a3 b2 (29)

= a2 b3 − a3 b2 . (30)

Which you probably recognise as the first component of the vector a × b. Indeed, if we go
through the components, we will indeed find that

εiµν aµ bν = (a × b)i . (31)

An identity you will find useful is

εµjk εµlm = δjl δkm − δjm δkl . (32)

The proof of this identity is just the evaluation of all of the cases (thankfully most of them are
zero!).
Exercise: Prove the identity (32).

Example: The scalar triple-product

In the problem sheet on vectors, we (without proof) made use of the fact that the scalar triple-
product, a · (b × c), is unchanged under cyclic permutations of the vectors a, b, and c. Armed
with our new notation, proving this becomes trivial!

a · (b × c) = ελµν aλ bµ cν (33)
= ενλµ aλ bµ cν (34)
= ενλµ cν aλ bµ (35)
= c · (a × b) (36)

We immediately see that (since everything on the right commutes), cyclicly permuting the
vectors corresponds to cyclicly permuting the indices λµν in the ε. Since λµν is unchanged
under cyclic permutations, a · (b × c) is unchanged under cyclic permutations of the vectors!
Exercise: Which of the following expressions are valid? If possible, write them in matrix-
column vector notation. If they are not valid, why not?
1. ∇µ xµ

2. Aiσ xσ

3. ελµν xλµ

5
 4. ai bj + ck dj

5. εiµν ενσρ aµ bσ cρ

You now might want to go back and have a look at the problem sheet from week 2 “Vectors
and matrices I” (specifically, Q. 24 of the class problems) for some more practice. Try proving
them all with index notation and Einstein summation convention!