Lab 2 - Vectors and Matrices

In this week's laboratory exercise, we will practice working with vectors and matrices, which are fundamental to the topic of linear algebra
used in many data analytics algorithms.

Once again, you can learn more about the language on python.org and Python for Beginners.

This lab exercise is adopted from Hiren Namera's GitHub on Basic Mathematics for Machine Learning. Because all the code is inside code
cells, you can just run each code cell inline rather than using a separate Python interactive window.

This Jupyter notebook is applicable for Python 3.x versions or above.

Note: This notebook is designed to have you run code cells one by one, and several code cells contain deliberate errors for demonstration
purposes. As a result, if you use the Cell > Run All command, some code cells past the error won't be run. To resume running the code in
each case, use Cell > Run All Below from the cell after the error.

Comments
Many of the examples in this lab exercise include comments. Comments in Python start with the hash character, # , and extend to the end
of the physical line. A comment may appear at the start of a line or following whitespace or code, but not within a string literal. A hash
character within a string literal is just a hash character. Since comments are to clarify code and are not interpreted by Python, they may be
omitted when typing in examples.

(A) Vectors
1. Visualize a Vector
At its simplest, a vector is a numeric element that has both magnitude and direction. The magnitude represents a distance (for example, "2
miles") and the direction indicates which way the vector is headed (for example, "East"). Vectors are defined by an n-dimensional
coordinate that describe a point in space that can be connected by a line from an origin, which can be defined arbitrarily.

For example, in a two-dimensional plane, a vector can be represented as a two dimensional coordinate consisting of an x and a y value,
and for example 2 for x and 1 for y. The origin is often set at (0,0).

Our example vector can be written as v=(2,1), but more formally we would use the following notation, in which the dimensional coordinate
values for the vector are shown as a matrix:

2
→
v = [ ] (1)
1

It can help to visualize the vector, and with a two-dimensional vector, that's fairly straightforward. We just define a two-dimensional plane,
choose a starting point, and plot the coordinate described by the vector relative to the starting point.

Below, when the code is run, it visualises the vector v on a two-dimensional plane. Note that we are using the Python library numpy in this
laboratory.

In [1]:
%matplotlib inline

import numpy as np
import matplotlib.pyplot as plt

# Here, we use a numpy array to store our vector

v = np.array([2,1])

# and we'll use a quiver plot to visualize it.

origin = [0], [0]
plt.grid()
plt.quiver(*origin, *v, scale=10, color='r')
plt.show()

Note that we can use a numpy array to define the vector in Python; so to create our (2,1) vector, we simply create a numpy array with the
elements [2,1]. We then used a quiver plot to visualize the vector, using the point (0,0) as the starting point (or origin). Our vector of (2,1) is
shown as an arrow that starts at 0,0 and moves 2 units along the x axis (to the right) and 1 unit along the y axis (up).

Try it yourself
Change the x and y value of the vector
Change the location of the origin

In [5]:
%matplotlib inline

import numpy as np
import matplotlib.pyplot as plt

# Here, we use a numpy array to store our vector

v = np.array([4,4])

# and we'll use a quiver plot to visualize it.

origin = [1], [1]
plt.grid()
plt.quiver(*origin, *v, scale=20, color='g')
plt.show()

2. Calculating Vector Magnitude and Direction

We tend to work with vectors by expressing their components as cartesian coordinates; that is, x and y (and other dimension) values that
define the number of units travelled along each dimension. So the coordinates of our (2,1) vector indicate that we must travel 2 units along
the x axis, and 1 unit along the y axis.

However, we can also work with vectors in terms of their polar coordinates; that is coordinates that describe the magnitude and direction of
the vector. The magnitude is the overall distance of the vector from tail to head, and the direction is the angle at which the vector is
oriented.

Calculating Magnitude
Calculating the magnitude of the vector from its cartesian coordinates requires measuring the distance between the arbitrary starting point
and the vector head point. For a two-dimensional vector, we're actually just calculating the length of the hypotenuse in a right-angled
triangle - so we could simply invoke the Pythagorean theorum and calculate the square root of the sum of the squares of it's components,
like this:

→
∥ v ∥ = √v 1
2
+ v2
2
(2)

The notation for a vector's magnitude is to surround the vector name with vertical bars - you can use single bars (for example, |v|) or
double bars (||v||). Double-bars are often used to avoid confusion with absolute values. Note that the components of the vector are
indicated by subscript indices (v1, v2,...v*n*),

In this case, the vector v has two components with values 2 and 1, so our magnitude calculation is:

→
∥ v ∥ = √2
2
+ 1
2
(3)

Which is:

→
∥v ∥ = √4 + 1 (4)

So:

→
∥v ∥ = √5 ≈ 2.24 (5)

You can run the following Python code to get a more precise result (note that the elements of a numpy array are zero-based)

In [6]:
import math

v = np.array([2,1])
vMag = math.sqrt(v[0]**2 + v[1]**2)
print (vMag)

2.23606797749979
In Python, numpy provides a linear algebra library named linalg that makes it easier to work with vectors - you can use the norm function
in the following code to calculate the magnitude of a vector:

In [7]:
import numpy as np

v = np.array([2,1])
vMag = np.linalg.norm(v)
print (vMag)

2.23606797749979

Calculating Direction
To calculate the direction, or amplitude, of a vector from its cartesian coordinates, you must employ a little trigonometry. We can get the
angle of the vector by calculating the inverse tangent; sometimes known as the arctan (the tangent calculates an angle as a ratio - the
inverse tangent, or tan-1, expresses this in degrees).

In any right-angled triangle, the tangent is calculated as the opposite over the adjacent. In a two dimensional vector, this is the y value over
the x value, so for our v vector (2,1):

1
tan(θ) = (6)
2

This produces the result 0.5, from which we can use a calculator to calculate the inverse tangent to get the angle in degrees:

−1 o
θ = tan (0.5) ≈ 26.57 (7)

Note that the direction angle is indicated as θ.

Run the following Python code to confirm this:

In [8]:
import math
import numpy as np

v = np.array([2,1])
vTan = v[1] / v[0]
print ('tan = ' + str(vTan))
vAtan = math.atan(vTan)
# atan returns the angle in radians, so convert to degrees
print('inverse-tan = ' + str(math.degrees(vAtan)))

tan = 0.5
inverse-tan = 26.56505117707799

3. Vector Addition (or Subtraction)

So far, we've worked with one vector at a time. What happens when you need to add/subtract two vectors.

Let's take a look at an example, we already have a vector named v, as defined here:

2
→
v = [ ] (8)
1

Now let's create a second vector, and called s like this:

−3
→
s = [ ] (9)
2

Run the cell below to create s and plot it together with v:

In [12]:
import math
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

v = np.array([2,1])
s = np.array([-3,2])
print (s)

# Plot v and s
vecs = np.array([v,s])
origin = [0], [0]
plt.grid()
plt.quiver(*origin, vecs[0,0], vecs[0,1], color=['r'], scale=10)
plt.quiver(*origin, vecs[1,0], vecs[1,1], color=['b'], scale=10)
plt.show()

[-3 2]

You can see in the plot that the two vectors have different directions and magnitudes. So what happens when we add them together?

Here's the formula:

→
z = v + s → → (10)

In terms of our vector matrices, this looks like this:

2 −3
→
z = [ ] + [ ] (11)
1 2

Which gives the following result:

2 −3 −1
→
z = [ ] + [ ] = [ ] (12)
1 2 3

Let's verify that Python gives the same result, and visualize the result in a quiver plot:

In [15]:
import math
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

v = np.array([2,1])
s = np.array([-3,2])
z = v + s
print(z)

vecs = np.array([v,s,z])
origin = [0], [0]
plt.grid()
plt.quiver(*origin, vecs[0,0], vecs[0,1], color=['r'], scale=10)
plt.quiver(*origin, vecs[1,0], vecs[1,1], color=['b'], scale=10)
plt.quiver(*origin, vecs[2,0], vecs[2,1], color=['g'], scale=10)
#plt.quiver(*origin, vecs[:,0], vecs[:,1], color=['r', 'b', 'g'], scale=10)
plt.show()

[-1 3]

Vector subtraction can be done in a similar way between two vectors v and s, by first multiplying s by the number -1, and then add to v.

4. Vector Multiplication
Vector multiplication can be performed in three ways:

Scalar Multiplication
Dot Product Multiplication
Cross Product Multiplication

In this laboratory, we will only look at the first way, that is, scalar multiplication.

Scalar Multiplication
Let's start with scalar multiplication - in other words, multiplying a vector by a single numeric value.

Suppose I want to multiply my vector by 2, which I could write like this:

→
w = 2v → (13)

Note that the result of this calculation is a new vector named w. So how would we calculate this? Recall that v is defined like this:

2
→
v = [ ] (14)
1

To calculate 2v, we simply need to apply the operation to each dimension value in the vector matrix, like this:

2 ⋅ 2
→
w = [ ] (15)
2 ⋅ 1

Which gives us the following result:

2 ⋅ 2 4
→
w = [ ] = [ ] (16)
2 ⋅ 1 2

In Python, you can apply these sort of matrix operations directly to numpy arrays, so we can simply calculate w like this:

In [16]:
import numpy as np
import matplotlib.pyplot as plt
import math
%matplotlib inline

v = np.array([2,1])

w = 2 * v
print(w)

vecs = np.array([v,w])

# Plot v and w
origin = [0], [0]
plt.grid()
plt.quiver(*origin, vecs[0,0], vecs[0,1], color=['r'], scale=10)
plt.quiver(*origin, vecs[1,0], vecs[1,1], color=['b'], scale=10)
#plt.quiver(*origin, vecs[:,0], vecs[:,1], color = ['r', 'b'], scale=10)
plt.show()

[4 2]

(B) Matrices
In general terms, a matrix is an array of numbers that are arranged into rows and columns.

1. Matrices and Matrix Notation

A matrix arranges numbers into rows and columns, like this:

1 2 3
A = [ ] (17)
4 5 6

Note that matrices are generally named as a capital letter. We refer to the elements of the matrix using the lower case equivalent with a
subscript row and column indicator, like this:

a1,1 a1,2 a1,3

A = [ ] (18)
a2,1 a2,2 a2,3

In Python, you can define a matrix as a 2-dimensional numpy.array, like this:

In [17]:
import numpy as np

A = np.array([[1,2,3],
[4,5,6]])
print(A)

[[1 2 3]
[4 5 6]]
You can also use the numpy.matrix type, which is a specialist subclass of array:

In [18]:
import numpy as np

M = np.matrix([[1,2,3],
[4,5,6]])
print (M)

[[1 2 3]
[4 5 6]]
There are some differences in behavior between array and matrix types - particularly with regards to multiplication (which we'll explore
later). You can use either, but most experienced Python programmers who need to work with both vectors and matrices tend to prefer the
array type for consistency.

2. Matrix Operations
Matrices support common arithmetic operations.

Adding Matrices
To add two matrices of the same size together, just add the corresponding elements in each matrix:

1 2 3 6 5 4 7 7 7
[ ] + [ ] = [ ] (19)
4 5 6 3 2 1 7 7 7

In this example, we're adding two matrices (let's call them A and B). Each matrix has two rows of three columns (so we describe them as
2x3 matrices). Adding these will create a new matrix of the same dimensions with the values a1,1 + b1,1, a1,2 + b1,2, a1,3 + b1,3,a2,1 + b2,1,
a2,2 + b2,2, and a2,3 + b2,3. In this instance, each pair of corresponding elements(1 and 6, 2, and 5, 3 and 4, etc.) adds up to 7.

Let's try that with Python:

In [19]:
import numpy as np

A = np.array([[1,2,3],
[4,5,6]])
B = np.array([[6,5,4],
[3,2,1]])
print(A + B)

[[7 7 7]
[7 7 7]]

Subtracting Matrices
Matrix subtraction works similarly to matrix addition:

1 2 3 6 5 4 −5 −3 −1
[ ] − [ ] = [ ] (20)
4 5 6 3 2 1 1 3 5

Here's the Python code to do this:

In [20]:
import numpy as np

A = np.array([[1,2,3],
[4,5,6]])
B = np.array([[6,5,4],
[3,2,1]])
print (A - B)

[[-5 -3 -1]
[ 1 3 5]]

Negative Matrices
The negative of a matrix, is just a matrix with the sign of each element reversed:

−5 −3 −1
C = [ ] (21)
1 3 5

5 3 1
−C = [ ] (22)
−1 −3 −5

Let's see that with Python:

In [21]:
import numpy as np

C = np.array([[-5,-3,-1],
[1,3,5]])
print (C)
print (-C)

[[-5 -3 -1]
[ 1 3 5]]
[[ 5 3 1]
[-1 -3 -5]]

Matrix Transposition
You can transpose a matrix, that is switch the orientation of its rows and columns. You indicate this with a superscript T, like this:

T 1 4
⎡ ⎤
1 2 3
[ ] = ⎢2 5⎥ (23)
4 5 6 ⎣ ⎦
3 6

In Python, both numpy.array and numpy.matrix have a T function:

In [22]:
import numpy as np

A = np.array([[1,2,3],
[4,5,6]])

print(A)
print(A.T)

[[1 2 3]
[4 5 6]]
[[1 4]
[2 5]
[3 6]]

Matrix Multiplication
Multiplying matrices is a little more complex than the operations we've seen so far. There are two cases to consider, scalar multiplication
(multiplying a matrix by a single number), and dot product matrix multiplication (multiplying a matrix by another matrix).

Scalar Multiplication
To multiply a matrix by a scalar value, you just multiply each element by the scalar to produce a new matrix:

1 2 3 2 4 6
2 × [ ] = [ ] (24)
4 5 6 8 10 12

In Python, you perform this calculation using the * operator:

In [23]:
import numpy as np

A = np.array([[1,2,3],
[4,5,6]])
print(2 * A)

[[ 2 4 6]
[ 8 10 12]]

Dot Product Matrix Multiplication

To mulitply two matrices together, you need to calculate the dot product of rows and columns. This means multiplying each of the elements
in each row of the first matrix by each of the elements in each column of the second matrix and adding the results. We perform this
operation by applying the RC rule - always multiplying Rows by Columns. For this to work, the number of columns in the first matrix must
be the same as the number of rows in the second matrix so that the matrices are conformable for the dot product operation.

Sounds confusing, right?

Let's look at an example:

9 8
⎡ ⎤
1 2 3
[ ] ⋅ ⎢7 6⎥ (25)
4 5 6
⎣ ⎦
5 4

Note that the first matrix is 2x3, and the second matrix is 3x2. The important thing here is that the first matrix has two rows, and the second
matrix has two columns. To perform the multiplication, we first take the dot product of the first row of the first matrix (1,2,3) and the first
column of the second matrix (9,7,5):

(1, 2, 3) ⋅ (9, 7, 5) = (1 × 9) + (2 × 7) + (3 × 5) = 38 (26)

In our resulting matrix (which will always have the same number of rows as the first matrix, and the same number of columns as the
second matrix), we can enter this into the first row and first column element:

38 ?
[ ] (27)
? ?

Now we can take the dot product of the first row of the first matrix and the second column of the second matrix:

(1, 2, 3) ⋅ (8, 6, 4) = (1 × 8) + (2 × 6) + (3 × 4) = 32 (28)

Let's add that to our resulting matrix in the first row and second column element:

38 32
[ ] (29)
? ?

Now we can repeat this process for the second row of the first matrix and the first column of the second matrix:

(4, 5, 6) ⋅ (9, 7, 5) = (4 × 9) + (5 × 7) + (6 × 5) = 101 (30)

Which fills in the next element in the result:

38 32
[ ] (31)
101 ?

Finally, we get the dot product for the second row of the first matrix and the second column of the second matrix:

(4, 5, 6) ⋅ (8, 6, 4) = (4 × 8) + (5 × 6) + (6 × 4) = 86 (32)

Giving us:

38 32
[ ] (33)
101 86

In Python, you can use the numpy.dot function or the @ operator to multiply matrices and two-dimensional arrays:

In [24]:
import numpy as np

A = np.array([[1,2,3],
[4,5,6]])
B = np.array([[9,8],
[7,6],
[5,4]])
print(np.dot(A,B))
print(A @ B)

[[ 38 32]
[101 86]]
[[ 38 32]
[101 86]]
This is one case where there is a difference in behavior between numpy.array and numpy.matrix, You can also use a regular multiplication
(*) operator with a matrix, but not with an array:

In [25]:
import numpy as np

A = np.matrix([[1,2,3]
,[4,5,6]])
B = np.matrix([[9,8],
[7,6],
[5,4]])
print(A * B)

[[ 38 32]
[101 86]]
Note that, unlike with multiplication of regular scalar numbers, the order of the operands in a multiplication operation is significant. For
scalar numbers, the commmutative law of multiplication applies, so for example:

2 × 4 = 4 × 2 (34)

With matrix multiplication, things are different, for example:

2 4 1 3 1 3 2 4
[ ] ⋅ [ ] ≠ [ ] ⋅ [ ] (35)
6 8 5 7 5 7 6 8

Run the following Python code to test this:

In [26]:
import numpy as np

A = np.array([[2,4],
[6,8]])
B = np.array([[1,3],
[5,7]])
print(A @ B)
print(B @ A)

[[22 34]
[46 74]]
[[20 28]
[52 76]]
There are many other operations on vector and matrices that could be covered, but due to the lack of sufficient time, I ask that you
explore how to perform other vector and matrix operations in Python with your own time.