Lecture 2

Vectors and Linear Systems

∗
based on slides kindly provided by Dr. Francesca Bianchi and Dr. Martin Djukanović

Vectors and Linear Systems Linear algebra for IEM 1 / 32

Recap
Start with a linear system:

a11 x1 + · · · + a1n xn = b1
a21 x1 + · · · + a2n xn = b2
..
.
am1 x1 + · · · + amn xn = bm

Associate with it its augmented matrix:

 
a11 · · · a1n b1
 a21 · · · a2n b2 
 
 .. .. .. 
 . . . 
am1 · · · amn bm

Vectors and Linear Systems Linear algebra for IEM 2 / 32

Recap
Elementary row operations:
1 swap two rows
2 multiply a row by a non-zero constant
3 add to a row a multiple of another row

Theorem
Applying elementary row operations to the augmented matrix does not
change the solution set of the corresponding linear system.

Vectors and Linear Systems Linear algebra for IEM 3 / 32

Recap
Reduced row echelon form:
1 zero rows on the bottom
2 leading coefficients in ascending column positions
3 leading coefficients equal 1
4 leading coefficients are the only non-zero entries in their columns

Theorem
The reduced (row) echelon form of a matrix can be obtained by applying
elementary row operations and it is unique.

We can use the reduced echelon form of the augmented matrix to easily
determine the solution set of the linear system.

Vectors and Linear Systems Linear algebra for IEM 4 / 32

Using row reduction to solve a linear system
1 Write the augmented matrix of the system.
2 Use the row reduction algorithm to obtain an equivalent augmented
matrix in echelon form. Decide whether the system is consistent. If
there is no solution, stop.
3 Continue row reduction to obtain the reduced echelon form.
4 Write the system of equations corresponding to the matrix obtained
in step 3.
5 Rewrite each non-zero equation from step 4 so that its one basic
variable is expressed in terms of any free variables appearing in the
equation.

Vectors and Linear Systems Linear algebra for IEM 5 / 32

How to get the solution set from the RREF of the
augmented matrix?
1 ignore the zero rows: they correspond to the equation
0 · x1 + · · · + 0 · xn = 0, which is always satisfied.
2 if a row has a leading entry in the last column, then the system is
inconsistent (i.e. has no solutions). Indeed, a row like
 
0 ··· 0 c for some c ̸= 0

corresponds to the equation 0 = 0 · x1 + · · · + 0 · xn = c ̸= 0, which

has no solutions.
3 If there are no rows as in 2 , then the system is consistent (i.e. has at
least one solution). There are two cases:
▶ each column corresponding to a variable contains the leading entry of a
row: then there is exactly one solution.
▶ otherwise, the variables corresponding to columns not containing
leading entries are free (can take any value), and the other variables
can be expressed in terms of the free variables.
Vectors and Linear Systems Linear algebra for IEM 6 / 32
 Vectors

Vectors and Linear Systems Linear algebra for IEM 7 / 32

Vectors in Rn
An element of Rn is an ordered tuple (x1 , x2 , . . . , xn ) that we identify with:
a (row or column) vector with n entries
a point in space of dimension n

The vector all of whose entries are zero is called the zero vector and
denoted by 0.
Vectors in Rn can be thought of as two pieces of information:
1) a direction
2) a magnitude or length
This concept can be depicted with the standard arrows when n ≤ 3.

Vectors and Linear Systems Linear algebra for IEM 8 / 32

Question
In what sense is the arrow “the same thing” as the point?

The point is the end-point of the arrow.

The origin is always a fixed reference point, so to speak.

Vectors and Linear Systems Linear algebra for IEM 9 / 32

Example: n = 1
An element of R1 is a column vector with 1 entry. We may just think of it
as an element in R, which can be represented on the real line.

√ √
v = 3, − 3v v
ˆ x
√ 0
− 3v = −3

Vectors and Linear Systems Linear algebra for IEM 10 / 32

Vectors in R2
An element of R2is an ordered pair (x, y ) that we identify with the
x
column vector and represent by a point/arrow in the Cartesian plane.
y

v = (−2, 1) x

Vectors and Linear Systems Linear algebra for IEM 11 / 32

Vectors in R2

An element of R2is an ordered pair (x, y ) that we identify with the

x
column vector and represent by a point/arrow in the Cartesian plane.
y

Vectors and Linear Systems Linear algebra for IEM 12 / 32

Addition and scalar multiplication
Vectors in Rn can be added and scaled (multiplied by numbers in R).
Both operations are performed coordinate-wise, that is:

(x1 , x2 , . . . , xn ) + (y1 , y2 , . . . , yn ) = (x1 + y1 , x2 + y2 , . . . , xn + yn )

α · (x1 , ×2 , . . . , xn ) = (αx1 , αx2 , . . . , αxn )

Vectors and Linear Systems Linear algebra for IEM 13 / 32

Scalar multiplication
y

v
1
v
v = (3, 1) 2
  − 21 v x
1 3 1
v= ,
2 2 2
 
1 3 1
− v = − ,−
2 2 2

Vectors and Linear Systems Linear algebra for IEM 14 / 32

Addition
y

v1

v1 = (3, 1), v2 = (−2, −3) v1 + v2

v2
v1 +v2 = (3−2, 1−3) = (1, −2)

Figure: The parallelogram rule

Vectors and Linear Systems Linear algebra for IEM 15 / 32

Properties of vectors in Rn
For all vectors u, v, w ∈ Rn and for all scalars α, β ∈ R
1 (u + v) + w = u + (v + w) associativity of +
2 u+v =v+u commutativity of +
3 u+0=u identity element for +
4 u + (−u) = 0 inverse element for +
5 α · (β · u) = (α · β) · u associativity of ·
6 (α + β) · u = α · u + β · u distributivity of · over +
α · (u + v) = α · u + α · v
7 1 · u = u.

Here −u = (−1) · u.

Vectors and Linear Systems Linear algebra for IEM 16 / 32

Directions
Definition
Two vectors in Rn are said to have the same direction if one of them is a
positive scalar multiple of the other; they are said to have opposite
directions if one of them is a negative scalar multiple of the other.

Example
On slide 14, v and 12 v have the same direction and v and − 21 v have
opposite directions.

Vectors and Linear Systems Linear algebra for IEM 17 / 32

Linear combinations
Let v1 , . . . , vn ∈ Rm . Then a linear combination of v1 , . . . , vn is a vector
of the form
α1 v1 + · · · + αn vn ,
where α1 , . . . , αn ∈ R.
Example
(5, 2) ∈ R2 is a linear combination of (1, 1) and (0, 1). Indeed,

(5, 2) = 5(1, 1) − 3(0, 1).

How do we check in general whether a vector in Rn is a linear combination

of some given vectors?

Vectors and Linear Systems Linear algebra for IEM 18 / 32

How to check if a vector is a linear combination of given
vectors?
Example
Determine whether (9, 2, 7) is a linear combination of (1, 2, −1) and
(6, 4, 2). The question is whether there exist α1 , α2 ∈ R such that

α1 (1, 2, −1) + α2 (6, 4, 2) = (9, 2, 7)

i.e.
(α1 + 6α2 , 2α1 + 4α2 , −α1 + 2α2 ) = (9, 2, 7).
In other words, we need to solve the following linear system in α1 , α2 :

α1 + 6α2 = 9

2α1 + 4α2 = 2

−α1 + 2α2 = 7


Vectors and Linear Systems Linear algebra for IEM 19 / 32

How to check if a vector is a linear combination of given
vectors?
Example (continued)
The augmented matrix of this system is
     
1 6 9 E21 (−2) 1 6 9 1 6 9
E31 (1) E32 (1)
 2 4 2  −− −−−→  0 -8 -16  −−−−→  0 -8 -16 
-1 2 7 0 8 16 0 0 0
   
1 6 9 1 0 -3
E2 (−1/8) E12 (−6)
−
−−−−− →  0 1 2  −−−−−→  0 1 2 
0 0 0 0 0 0

We conclude that

(9, 2, 7) = −3 · (1, 2, −1) + 2 · (6, 4, 2)

so our vector is a linear combination of the two given ones.

Vectors and Linear Systems Linear algebra for IEM 20 / 32
How to check if a vector is a linear combination of given
vectors?
Exercise
Show that (4, −1, 8) is not a linear combination of (1, 2, −1) and (6, 4, 2).
That is, show that the corresponding linear system is inconsistent.

Vectors and Linear Systems Linear algebra for IEM 21 / 32

The linear span
Definition
If v1 , . . . , vn ∈ Rm , then the span of v1 , . . . , vn , denoted Span{v1 , . . . , vn },
is the subset of Rm consisting of all possible linear combinations of
v1 , . . . , vn :

Span{v1 , . . . , vn } = {α1 v1 + · · · + αn vn : α1 , . . . , αn ∈ R}.

Question: Is (9, 2, 7) contained in the set Span{(1, 2, −1), (6, 4, 2)}?

Answer: Yes. We showed (9, 2, 7) = −3(1, 2, −1) + 2(6, 4, 2).
Question: Is (4, −1, 8) a vector in Span{(1, 2, −1), (6, 4, 2)}?
Answer: No. By the previous exercise, (4, −1, 8) is not a linear
combination of (1, 2, −1) and (6, 4, 2).

Vectors and Linear Systems Linear algebra for IEM 22 / 32

Example
For v ∈ R3 , the set L = Span{v} ⊂ R3 is a line passing through 0.
For v1 , v2 ∈ R3 , the set P = Span{v1 , v2 } ⊂ R3 is a plane passing
through 0.

...with a few caveats:

If v = 0 then L = {0} (a point).
Otherwise, L is a line.
If v1 = v2 = 0 then P = {0} (a point).
Otherwise, if v1 = cv2 for some c ̸= 0 then
L = {αv1 : α ∈ R} = {αv2 : α ∈ R} (a line).
Otherwise, P is a plane.

Vectors and Linear Systems Linear algebra for IEM 23 / 32

§1.4 The matrix equation Ax = b

Definition
If A is an m × n matrix, with columns a1 , · · · , an , and if x ∈ Rn , then the
product of A and x, denoted by Ax, is the linear combination of the
columns of A using the corresponding entries in x as coefficients; that is,
 
x1
 .. 
Ax = [a1 a2 . . . an ]  .  = x1 a1 + x2 a2 + · · · + xn an .
xn

Example
 
  −2          
2 5 0 1   1  = −2 2 + 1 5 + 0 0 + −1 1 = 0

3 −1 1 −3  0  3 −1 1 −3 −4
−1
Vectors and Linear Systems Linear algebra for IEM 24 / 32
For a given m × n matrix A and a given b vector in Rm find x ∈ Rn such
that Ax = b. The equation Ax = b is called a matrix equation.
Theorem
If A is an m × n matrix, with columns a1 , · · · , an , and if b ∈ Rm , the
matrix equation
Ax = b,
that is
x1 a1 + x2 a2 + · · · + xn an = b
has the same solution set as the system of linear equations whose
augmented matrix is [a1 a2 . . . an b].

Note: The theorem says that the matrix equation Ax = b has at least one
solution if and only if b is a linear combination of the columns of A; that
is, b ∈ Span{a1 , . . . , an }.

Vectors and Linear Systems Linear algebra for IEM 25 / 32

Remark
Just as solutions to linear systems need not be unique, a vector is not
necessarily a linear combination of given vectors in a unique way.

Example
Let α be any scalar. Then:
     
2 1 0
=2 +α
4 2 0
       
3 1 1 0
−4 = α 0 + (3 − α) −2 + (1 − α)  2 
5 1 2 −1

Vectors and Linear Systems Linear algebra for IEM 26 / 32

Definition
A set of vectors {v1 , . . . , vp } in Rm spans Rm if every vector in Rm is a
linear combination of v1 , . . . , vp ; that is, if Span{v1 , . . . , vp } = Rm .

Example
The following vectors in Rm :

e1 = (1, 0, 0, . . . , 0), e2 = (0, 1, 0, . . . , 0), . . . , em = (0, 0, 0, . . . , 1)

span Rm . Indeed, if v = (v1 , . . . , vm ) ∈ Rm then

v = v1 e1 + v2 e2 + · · · + vn em .

Vectors and Linear Systems Linear algebra for IEM 27 / 32

Theorem
Let A be an m × n matrix. Then the following statements are logically
equivalent. That is, for a particular matrix A, either they are all true or
they are all false.
a) The equation Ax = b has at least one solution for every b in Rm .
b) Every b in Rm is a linear combination of the columns of A.
c) The columns of A span Rm .
d) The echelon form(s) of A have no zero rows - there is a pivot in every
row.
Note that item d) refers to the (coefficient) matrix A and not to the
augmented matrix [A|b].

Vectors and Linear Systems Linear algebra for IEM 28 / 32

Example
Do the vectors (1, 0, 1), (1, 2, 1) and (−3, −2, −3) span R3 ?

Vectors and Linear Systems Linear algebra for IEM 29 / 32

Example (continued)
The vectors (1, 0, 1), (1, 2, 1) and (−3, −2, −3) span R3 if and only if for
every b ∈ R3 , there exist x1 , x2 , x3 ∈ R such that

x1 (1, 0, 1) + x2 (1, 2, 1) + x3 (−3, −2, −3) = b,

or, equivalently,

(x1 + x2 − 3x3 , 2x2 − 2x3 , x1 + x2 − 3x3 ) = b,

or, again,
Ax = b,
 
1 1 −3
where x = (x1 , x2 , x3 ) and A = 0 2 −2.
1 1 −3

Vectors and Linear Systems Linear algebra for IEM 30 / 32

Example (continued)
We therefore solve the system

x1 + x2 − 3x3 = b1

2x2 − 2x3 = b2

x1 + x2 − 3x3 = b3


Its augmented matrix is

   
1 1 −3 b1 1 1 −3 b1
E31 (−1)
0 2 −2 b2  −−−−−→ 0 2 −2 b2 
1 1 −3 b3 0 0 0 b3 − b1

Thus, if b3 − b1 ̸= 0, then the system has no solutions, so e.g. (1, 0, 0) is

not in the span and the vectors do not span R3 .

Vectors and Linear Systems Linear algebra for IEM 31 / 32

Properties of the matrix–vector product Ax
Theorem
If A is an m × n matrix, v and w are vectors in Rn , and α is a scalar, then
a) A(v + w) = Av + Aw;
b) A(αv) = α(Av).

We might say that the operation v 7→ Av “respects” addition and (scalar)

multiplication.
We can do addition or scalar multiplication with v and w before or after
applying A, the result is the same.
Operations with this property are called linear.

Vectors and Linear Systems Linear algebra for IEM 32 / 32