Vectors for Beginners

Leo Dorst
September 6, 2007

1 Three ways of looking at linear algebra

We will always try to look at what we do in linear algebra at three levels:

• geometric: drawing a picture. This is often related to the application.

• computational : giving a formula. This is related to the implementation
(in Matlab or other code).
• symbolic: specifying the essence of the computation, using the language el-
ements of linear algebra. Vectors and hyperplanes are the words, products
and matrices (we’ll meet those later) are the verbs.
To use linear algebra effectively in computer science, you have to try and encode
the geometric pictures of your application, into the symbolic level; the compu-
tational level then follows automatically. Thinking at the computational level is
tiresome, and will make linear algebra appear like a bag of tricks to memorize,
rather than as a language to master fluently.
So the order in the practice of computer science is typically: geometry,
symbolic, computation. Often the order of definition in the Bretscher book will
be: geometry, computation, symbolic; or even computation, symbolic, geometry.

2 Vector properties
What are vectors?
• geometric: For now, vectors are like arrows starting from a common point
(the origin) in an n-dimensional space. In our examples we use 2-D and
3-D. A vector has a direction, and a length. That length is called its norm.
• symbolic: We will denote a vector as a variable with an arrow overhead:
the vector ~x. The norm is written as k~xk. We can only specify the direction
relative to standard vectors. In n dimensions, you need n standard vectors,
called a basis. We will denote those standard vectors as ~e1 , · · · ~en . We
often choose them perpendicular and of unit length; then they point in
the direction of the coordinate axes that in high school you used to call

1
 the x-axis, y-axis, et cetera. Since we run out of the alphabet soon, and
because programs typically run over the indices of the dimensions, we will
use ~ei instead, with i running from 1 to n.
• computational : You can think of an n-D vector as n numbers. These
numbers give the components relative to the standard vectors. Adding
these components (see below) gives the vector. It is traditional to write
the n numbers in a column, with square brackets, and label then with the
basis vector they refer to. So
 
x1
~x 7→  x2  (1)
x3

is the computational representation of the 3-dimensional vector ~x in the

basis {~e1 , ~e2 , ~e3 }.
You see that the geometry and the computational representation have to be
specific on the dimensionality, whereas the symbolic representation need not.
That is an advantage, for we are going to need spaces of many different dimen-
sionalities in computer science.

3 Basic operations on vectors: scaling and adding

3.1 Scaling
You can change the length of a vector, making it λ times longer. Here λ is a
real number, allowed to be zero or negative.

• geometric: Multiplying the vector ~x by λ makes the vector λ times as

long. The resulting vector still points in the same direction is λ > 0, in
the opposite direction if λ < 0, and has lost its direction when λ = 0 (it
then gives the null vector: no direction, no length, so hardly a vector at
all).

• symbolic: We denote the product of a scalar (number) λ and a vector ~x

as λ ~x.
• computational : The computational representation of λ ~x is obtained by
multiplying each of its components by λ:
 
λx1
λ ~x 7→  λx2  . (2)
λx3

That reduces the implementation of vector scaling to the multiplication

of real numbers.

2
Figure 1: Addition of the vectors ~x and ~y to form the vector ~x + ~y . (In these
figures, we label the vectors with bold font, rather than overhead arrows.)

You should now be able to prove:

distributivity: (λ + µ) ~x = λ ~x + µ ~x (3)

3.2 Addition
You can also add two vectors ~x and ~y to make a new vector which is called their
sum.
• geometric: Vector addition is done by completing the parallelogram of
which ~x and ~y are the sides, and using the diagonal from the origin as the
resulting sum vector (see Figure 1).
• symbolic: We denote the sum as ~x + ~y .
• computational : We compute the sum as:
 
x1 + y1
~x + ~y 7→  x2 + y2  . (4)
x3 + y3

That reduces the vector addition to n additions of real numbers.

Since the computational definition defines the sum of vectors, it can be used to
prove the following symbolic properties:

commutativity: ~x + ~y = ~y + ~x
(5)
associativity: (~x + ~y ) + ~z = ~x + (~y + ~z)

Prove this for yourself, and also draw the geometry of these properties.

3
 The scaling of a vector behaves nicely relative to the addition:

distributivity: λ (~x + ~y ) = λ ~x + λ ~y (6)

Prove this for yourself.

3.3 The meaning of coordinates

Now that we have scaling and addition, we can specify the connection between
the coordinates and the symbolic notation, through the the basis vectors. It is:

~x = x1 ~e1 + x2 ~e2 + x2 ~e2 (7)

3.4 Linearity
In linear algebra, we are going to be interested in operations on vectors. For
example, rotating a vector, to point in another direction. For the moment, we
denote the rotation of a vector ~x as R(~x). We are not going to define it com-
putationally yet, but you may check geometrically that this rotation operation
has the properties:

scaling: R(λ ~x) = λ R(~x)

(8)
addition: R(~x + ~y ) = R(~x) + R(~y )

(Draw the pictures associated with these identities!)

These are called the linearity properties, and an operation with these prop-
erties is called a linear operation (or linear mapping, or linear transformation).
Linear algebra is all about these kinds of operations, and gives handy symbolic
theory and computational techniques for them.
Linear operations occur a lot in practice, and that makes linear algebra
very useful. In fact, it is so useful that if an operation is not linear, we try
to linearize it, i.e., make it linear. Sometimes we succeed exactly, by a clever
trick of choosing our vectors; sometimes we can only linearize approximately.
But linearization enormously extends the usefulness of linear algebra. Because
this is such a powerful approach to all sorts of problems (not only obviously
geometrical ones!), we teach linear algebra in computer science.
When you have a hammer, everything looks like a nail.

4 A product for vectors

There is a product for vectors, called the dot product (or inner product), which
produces a scalar number from two vectors. Since this is new, we go the opposite
direction, from computation to geometry. We define it first, and then try to
interpret the meaning.

4
 • computational : The dot product of two 3-D vectors ~x and ~y is defined as:
   
x1 y1
 x2  ·  y2  = x1 y1 + x2 y2 + x3 y3 . (9)
x3 y3

It is a number. In linear algebra, a number is called a scalar, since it has

only a scale, not a direction (in contrast to vectors). For the dot product
of vectors in n-D, just add more terms to this definition.

• symbolic: From the computational definition, you can prove for yourself
the following properties at the symbolic level:

symmetry: ~x · ~y = ~y · ~x
scaling: (λ ~x) · ~y = λ (~x · ~y ) (10)
distributivity: (~x + ~y ) · ~z = (~x · ~z) + (~y · ~z)

The dot product therefore behaves very much like multiplication of num-
bers.
However, it is not associative. We cannot even express associativity, for in
(~x · ~y ) · ~z is not defined: (~x · ~y ) is a scalar, and we do not know how to take
the dot product of a scalar and a vector ~z. Because of such difficulties,
you should be very accurate in your notation of the products. Do not use ·
for scalar multiplication. Do not use × (we will give it a different meaning
later). Just use a space for regular multiplication of scalars, and reserve ·
for a true dot product, of vectors.
• geometric: It turns out that the geometric interpretation of the dot prod-
uct is an interesting combination of angles and lengths. Let the length
(norm) of the vector ~x be k~xk, similar for ~y , and let the angle from ~x to
~y be φ. Then it can be shown that: 1

~x · ~y = k~xk k~y k cos(φ). (11)

So two things that we normally think of as different come together here:

distance and angle.
Let us play some more with the interpretations. If we take the dot product of
a vector with itself, we get computationally:

~x · ~x 7→ x21 + x22 + x23 (12)

and from (11), since the angle between ~x and itself is zero:

~x · ~x = k~xk2 . (13)
1 But we will not show this here, we just give the result. We will encounter it later in
Bretscher. You may attempt a proof now...

5
Combined, this gives us a formula to compute the norm of a vector ~x:
√ q
k~xk = ~x · ~x = x21 + x22 + x23 . (14)
This is of course Pythagoras’ theorem, applied to the coordinates of ~x, now in
3-D. Since ~x · ~x can be computed in n-D, the symbolic formula is valid in n-D,
and the computational formula can be extended simply to that case.
Another special case of (11) is when ~x and ~y are perpendicular. Then φ =
±π/2 (in high school, you called this ±90 degrees but from now on you should
work in radians), so cos(φ) = 0. Conversely, if the dot product of two non-null
vectors is zero, they must be perpendicular. So we get:
~x perpendicular to ~y ⇔ ~x · ~y = 0.
This is rather unlike what we know of multiplication of numbers: if the product
of numbers satisfies x y = 0, then x = 0 or y = 0 (or both). Vectors can have a
zero dot product without being zero.
We can also use (11) to make a formula for the angle φ between two vectors:
~x · ~y x1 y1 + x2 y2 + x3 y3
cos(φ) = =p 2 p . (15)
k~xk k~y k x1 + x22 + x23 y12 + y22 + y32
This can also be extended to n-dimensional space.
A useful special case is if ~y is a unit vector ~e, which is a vector with unit
norm k~ek = 1. Then we get:
~x · ~e = k~xk cos(φ). (16)
This is precisely the length of the perpendicular projection of ~x on ~e, see Fig-
ure 2. Therefore, if we multiple the unit vector ~e by this length, we get the
component (~x · ~e) ~e of ~x along the ~e direction. If you write all this out in coordi-
nates by the computational method (even in 2-D), you will realize the advantage
of the symbolic notation, which moreover holds for n dimensional space as well.

5 Applications of the dot product

5.1 Proof of Pythagoras
You can now prove Pythagoras’ theorem for vectors:
if ~x is perpendicular to ~y ⇔ k~x + ~y k2 = k~xk2 + k~y k2 . (17)
Exercise:
1. Prove this at the symbolic level, so encode both sides in terms of things
at that level (hint: use the dot product). Avoid using the computational
approach, do not spell everything out in coordinates!
2. Draw a picture (the geometric approach). It is not quite the Pythagoras
triangle, for vectors should start at the origin!

6
Figure 2: Projection of ~x onto a unit vector ~e using the inner product, resulting
in (~x · ~e) ~e.

(a) (b)

Figure 3: Characterizing a line in 2D by a support vector d~ or a normal vector

~e and a distance δ such that d~ = δ ~e.

5.2 A line in the plane

A line in the plane can be characterized using vectors.
• geometric: Look at Figure 3a. We have an origin O, and a green line
which we want to characterize relative to this origin. A good idea is to
~ from the origin perpendicular to the line. All points of
use the vector d,
the line are obtained by “going to the end of d~ and looking left and right”.
• symbolic: What does a vector ~x need to satisfy to point exactly to a point
on the line? To encode that with previous constructions, let us make a
~ so that d~ = δ ~e. Then δ is the distance
unit vector ~e in the direction of d,
of the line to the origin, in the ~e-direction.
Now we use (16): for any ~x on the line, the projection of ~x onto ~e should
be have the length δ. You can see this clearly by comparing Figure 2 and

7
 Figure 3(b).
When we express that geometric insight symbolically, we simple get:
~x · ~e = δ (18)
as the characterization of the line. Now we can get back to the original
vector d~ by multiplying both sides by δ. The left hand side then gives ~x · d~
(why?), the right hand side δ 2 , which is the same as d~ · d~ (why?).
So ~x points to a point on the line characterized by d~ if and only if:
~x · d~ = d~ · d.
~ (19)
Actually, (18) is a bit more general, for it still works when δ = 0, whereas
(19) then has problems. How would you characterize the problem geomet-
rically (take a line through the origin; what property of the line can (19)
not encode?).
In either (18) or (19), the vectors ~e or d~ are called a normal vector for the
line. This is jargon, ‘normal’ here means ‘perpendicular’. Since ~e is a unit
vector, it is called a unit normal (vector) for the line, and since d~ ‘carries’
the line it is sometimes called the support vector of the line.
• computational : Now let us look at his computationally, taking (18) in 2-D.
Then the unit vector ~e can be specified relative to some fixed coordinate
basis {~e1 , ~e2 }. Let us have it make an angle φ with ~e1 , then its components
are:  
cos(φ)
~e 7→ (20)
sin(φ)
Verify this, using your high school knowledge of a right-angled triangle
with angle φ! Then (18) becomes, computationally:
x1 cos(φ) + x2 sin(φ) = δ. (21)
You may not recognize that yet. But in high school, you called x1 (which
is the component of ~x in the ~e1 -direction) the x-coordinate of the point ~x
points to, and x2 the y-coordinate. So we get:
x cos(φ) + y sin(φ) = δ. (22)
Also in high school, you were used to having things of the form y = f (x),
since you were dealing with functions. We are dealing with geometry, but
let us write our equation in the same form:
cos(φ) δ
y=− x+ , (23)
sin(φ) sin(φ)
though we can only do this when sin φ 6= 0. This is clearly something of
the form
y = a x + b, (24)
if we make the correct identifications by defining a and b in terms of φ and
δ. So indeed, the original (18) defines a line in its more classical form.

8
 (a) (b)

Figure 4: The representation of a plane in 3-D.

Yet (24) has problems. It is not defined when sin φ = 0, i.e. when the normal
vector is in the y-direction. The line is then vertical, and a would have to be
infinite, or we would have to switch over to a line of the form x = c. For
hand computations that is OK, but in a computer we would prefer to have a
representation that always works. So (18) is much better, for it has no problems
with lines in any direction or location. Verify that for the example of a vertical
line!

Exercises
3. Give the vector equation (18) of the line y = x + 1.
4. Give vector equation (18) of the line y = −x − 1, but using ~e =
√ the √
(1/ 2, 1/ 2).
√ √
5. Draw the line with ~e = (1/ 2, 1/ 2) and δ = 1.

6. Draw the line with d~ = (1, −1).

7. What is the distance to the origin (δ) of the line y = x + 1?
8. What is the distance to the origin (δ) of the line y = a x + b?

9. What is the distance to the origin (δ) of the line x1 − 2x2 = 1?

10. What is the distance to the origin (δ) of the line a1 x1 + a2 x2 = c?

5.3 A plane in space

We can use this also to represent planes in space.
• symbolic: In the symbolic equations we got, (18) and (19), there is nothing
stating that we are working with vectors in the plane. So we can apply
these equations in 3-D space as well.

9
 • geometric: They then are still the equations for the perpendicular pro-
jection of ~x along ~e to have the length δ. That suggests the picture of
Figure 4(b). It therefore seems to characterize a plane in space, with
support vector d~ as in Figure 4(a).
• computational : It is more difficult to define directions and angles in space,
and we will not do that here. But it is simple to verify that the symbolic
equation resulting from the dot product has the form:

a1 x1 + a2 x2 + a3 x3 = c. (25)

This represents a plane with a normal vector

 
a1
~a =  a2  , (26)
a3

at a distance
c
δ=p 2 . (27)
a1 + a22 + a23

Of course, it does not stop with 3-D. The equation (18) defines a hyperplane
an n-dimensional space, i.e. a flat element of dimensionality (n − 1). In 2-D, the
hyperplane is 1-D, and we are used to calling it a line; in 3-D, the hyperplane
is 2-D and we call that a plane.

Exercises:
11. Derive the formula (27) for the distance of a plane with equation (25) to
the origin.
 
1
12. Give the equation of a plane with normal vector  −1  at distance −1
2
from the origin.
13. Where does the plane with equation 2x1 + 3x2 − 5x3 = 6 cut the ~e1 -axis?
14. The plane with equation 2x1 + 3x2 − 5x3 = 6 cuts the plane with equation
x3 = 0 in a line. What is the equation of that line?

10