APPLIED PRojECTS for AN

INTRODUCTORy LINEAR ALGEBRA CLASS

ANNA ZEMLyANOvA
 This work is licensed under the Creative Commons Attribution-NonCommercial 4.0
Interna- tional License. To view a copy of this license, visit
http://creativecommons.org/licenses/ by-nc/4.0/ or send a letter to Creative
Commons, PO Box 1866, Mountain View, CA 94042, USA.

2
Contents
About this book.........................................................................................................................4
A brief introduction to Matlab..................................................................................................5
Advice on good programming style.........................................................................................11
Project overview......................................................................................................................12
Project 1: Basic operations with matrices in Matlab...............................................................13
Project 2: Matrix operations and image manipulation............................................................18
Project 3: Matrix multiplication, inversion, and photo filters.................................................24
Project 4: Solving linear systems in Matlab............................................................................29
Project 5: Systems of linear equations and college football team ranking (with an example of
the Big 6:
Project 12)...............................................................................................................................34
Convolution, inner product, and image processing revisited...................................40
Project 7: Norms, angles, and your movie choices...................................................................44
Project 8: Interpolation, extrapolation, and climate change....................................................49
Project 9: Orthogonal matrices and 3D graphics.....................................................................58
Project 10: Discrete dynamical systems, linear transformations of the plane, and the Chaos
Game.......................................................................................................................................64
Project 11: Projections, eigenvectors, Principal Component Analysis, and face recognition
algo- rithms..............................................................................................................................70
Project 12: Matrix eigenvalues and the Google’s PageRank algorithm...................................74
Project 13: Social networks, clustering, and eigenvalue problems...........................................79
Project 14: Singular Value Decomposition and image compression.........................................85
Appendices...............................................................................................................................91

References..............................................................................................................................107

3
About this book

This book is based on the Applied Matrix Theory curriculum which the author taught at
Kansas State University. Applied Matrix Theory is an introductory linear algebra
undergraduate class geared primarily towards engineering students. Unlike a proof-based
class that is typically of- fered to mathematics majors, the focus here is to introduce real life
applications at the same time as teaching the standard topics of linear algebra such as
matrix operations, linear systems, inverses, vector spaces, determinants, and eigenproblems.
While teaching the class, the author of this book struggled with finding realistic applications
of the theoretical material which can be reasonably completed during a class period. The
search for real life applications has led the author to write this book which hopefully fills the
gap between linear algebra theory and appli- cations. Most of the projects in this book can
be done with minimal knowledge of programming. It is author’s hope that the projects will
encourage the exploration of data using simple concepts from linear algebra. The projects are
written in Matlab in order to harness the abundance of built in functions and computational
power of Matlab. The author attempted to incorporate real data from various open sources
into these projects. There are two labs which are dedicated to exploring operations with
matrices and to solving linear systems in Matlab (the projects 1 and 4). There are also two
labs which use more advanced material typically not covered in an introductory linear
algebra class, such as singular value decomposition and normal equations (the projects 8 and
14).The code in this book has been developed using the 2015a version of Matlab and tested on
Matlab 2019a. Depending on the version of Matlab on your computer the syntax and the
output might differ slightly. The projects in this book can be adapted to use with GNU
Octave
from theasauthor’s
well. The supplementary files necessary to complete
at http://www.math.ksu.edu/ some of the labs are available
webpage ~azem.These sources are acknowledged
A variety of printed and Internet sources inspired this book.
in the Reference section as well as the sources of data for the projects. This book has been
developed under the Open/Alternative Textbook Initiative grant from Kansas State
University, the generous support of which is gratefully acknowledged here. The author is
immensely grateful to her colleagues, Nathan Albin and Pietro Poggi-Corradini, for
proofreading the book and for the corrections. The author is grateful to Jeffrey Sheldon for
proofreading the book and suggesting style corrections.

4
A brief introduction to Matlab

MATLAB is a numerical computing software application and a high level programming lan-
guage developed by MathWorks, Inc. Initially, Matlab stood for “matrix laboratory” and was
developed specifically to perform numerical linear algebra operations. Currently Matlab has a
vast library of built in mathematical functions which allow users to perform complex
computa- tions without having to program numerical methods themselves. Because of this
Matlab is the
In this software
book we willofuse
choice for to
Matlab many scientific
illustrate and engineering
common applications.
linear algebra concepts and to perform
computations on problems with real life applications. Before we go any further, let us review
some basic commands and principles of using Matlab. For more information, please see
Matlab Help or any Matlab manual of your choice.
The Matlab desktop environment

Matlab’s desktop environment typically consists of several windows. By default, you are
likely to see the command window with a prompt. There is also usually the “Current Folder”
window which shows the directory on the computer you are working with and the
“Workspace” window with any variables which have been initialized (if you did not define
any variables yet this window will be empty). In some cases you can also see the “Command
History” window which shows the commands which have been executed recently. The default
configuration can be changed by clicking downward arrow in the upper right corner of each of
these windows or by using the switch windows button located in the upper right corner of the
main Matlab window. Matlab can be used by typing commands in a command prompt or by
executing text files with commands. These files are called Matlab scripts and have the
extension “.m”. Matlab comes with a built in editor for editing M-files. This editor can also
be one of the windows you see in your Matlab environment when you start Matlab.
The command line

The easiest way to get started with Matlab is by executing commands in the interactive
command window. The command prompt can be seen in the main window of the Matlab
environment:
>>

A Matlab command can be entered in this prompt and Matlab will produce an immediate
result. For instance, entering
>> sin(pi/3)

will generate the output

ans 0
=.8660

To suppress Matlab output you can put a semicolon at the end of your command. For
instance, the command
>> sin(pi/3);

produces no output.
to any variable However,
is stored in theobserve
Matlabthat the last
variable ans.output which has
This variable hasnot
nowbeen assigned
changed its
value to sin π/3 rounded up to the machine precision.

5
 By default
Matlab Matlab
command formatusesshort.
the short
Thisformat for output
command on the
generates screen for
an output which
realisnumbers
given bywith
the
up to 4 decimal digits. The output format can be changed by executing the command
format long
The long format produces output for real numbers with up to 15 decimal digits.

If you want to know more about a particular command, you can always type
help <name of the command>

or
doc <name of the command>
This will open Matlab Help on the command you are interested in and also produce various
examples of Matlab’s use of this command. You can also open Matlab help directly by
pressing the corresponding button in the Matlab environment. This button looks like a
question mark
Another and tip
useful is generally
to rememberlocated in the
is that upper “Up”
pressing right arrow
corner button
of the Matlab
on your“Home” menu.
keyboard while
in the command line will bring up previous commands you have executed. You can scroll
through the commands and select any of them to run again. This is especially useful if you
have been typing lengthy expressions in the command line and need to correct misprints or
simply needsometimes
Finally, to run the itcommand again.
is necessary to interrupt Matlab computations. This might happen, for
instance, if Matlab is stuck in an infinite loop or a computation simply takes too long to
finish. You can interrupt the execution of Matlab code by pressing “Ctrl-C” while in the
main Matlab window.
Matlab scripts

While simple computations can be handled by typing commands directly into the command
prompt, it is advisable to organize more complex computations in a form of Matlab scripts or
M-files. You can create a new script file by going to the menu “Home” of the main Matlab
window and clicking on the “New Script” button. This will open a new empty script file in
the It
Matlab editor.
is a good idea to keep your M-files well-documented by utilizing the comment option in
Matlab. You can create a comment in a script file by starting the code line with the
percentage symbol . While interpreting the script, Matlab will ignore everything in the line
afterYou
thecansymbol.
also create code cells within the script. A new code cell can be started by using the
double percentage symbol . You can also write a title of the cell after the sign. Matlab will
treat everything after the sign as a comment. The creation of code cells allows you to break
your code into smaller executable pieces and run those pieces individually. To do so you can
use the “Run and Advance” button at the top of the Matlab editor window. Generally
speaking, you want to split the code into code cells in some logical way. For instance, each
codeFinally,
cell canyou
be can
dedicated
publishtoyour
performing a particular
code using function
the “Publish” in your
section code.
of the menu of the Matlab
editor. By default, code is published in the HTML format. Publishing will create a document
which involves a table of contents, the code itself, and all the plots and output which have
been produced by your code.

6
Variables, vectors, matrices

Variables are used to store and manipulate the values obtained during a Matlab session.
Matlab uses the following standard types of variables: real which can be double and single
precision (double or single), several integer types (int8, int16, int32, uint8, uint16,
uint32, uint64),
By default boolean
all the (logical)
numbers and are
in Matlab character
treated(char).
as the type double which uses 64 bits to
store a real number.
Each variable needs to have an identifier or a name. There are some general considerations
for variable names in Matlab:
• The name of a variable must start with a letter. After the first character the name may
contain letters, numbers, and the underscore character.
• Variable names in Matlab are case-sensitive. This means that Price and price are
treated as different variables.
• Certain words (such as reserved words or keywords) cannot be used as variable names.
• Names of built-in functions can, but generally should not, be used as variable names.
Matlab has several different ways to define matrices (two-dimensional arrays) and vectors
(one-dimensional arrays). For instance, we can define arrays explicitly by typing in all the
values:
>> vec=[1 2 3 4]
vec =
1 2 3 4
>> mat=[1 2 3; 4 5 6]
mat =
1
4 2
5 3
6

In the vectors
define exampleand
above a1 ×
matrices 4isvector vectheand
by using a 2 operator
colon 3 matrix matForare
“:”. defined.the
instance, Another wayvec
variables to
and mat from the example above can be also defined by commands
>>
>> vec=1:4;
mat=[1:3;4:6];

We will explore other matrix operations further in this book. Matlab also has an ability to
work with multi-dimensional arrays which will be useful for us later when manipulating color
images.

Matlab has a set of operators to perform standard

Operator algebraic operations with variables:
Meaning
addition, subtraction, binary
+, − negation, unitary
multiplication, division,
the Keep in priority,
highest mind thefollowed
prioritybyinthe
which these
power operators are
binary^,power
operator executed:
unitary negationthe
−,parentheses () take
multiplication and
−
division ∗, \, and, finally, addition and subtraction
parentheses (change of+,the
−.order of operations)
∗, \
^

7
Matlab functions

Matlab has a large library of built-in functions which makes it a convenient environment for
many scientific and engineering computations. The full list of available functions can be
found in Matlab Help and on the mathworks.com webpage. You can also look into the
“Help” menu of the Matlab main window (shown as a question mark in the upper right
corner). The list of elementary functions can be viewed by executing the command
>> help elfun
from the command line.
Matlab also allows user to create their own user-defined functions which can be saved in
separate M-files and can be later called from other scripts. The syntax of such functions in its
simplest form looks like this:
function [OutputVariables] = FunctionName(InputVariables)
Description of the function
<Main body of the function>
end

Control statements

There are several standard control statements in Matlab which change the order in which the
commands of a script
The control are executed
statements or whether
can be roughly theyinto
divided are condition
executed at all.
statements and loop state-
ments. The first group of control statements we will look at is the conditional statements.
The conditional statements allow one to perform different computations or procedures
depending on whether the condition (which is given by a boolean expression) is true or false.
In the simplest form, the conditional statement looks like this:
if <condition>
<statements>
else
<statements>
end
In
be the
incorporated by using theelse
above construction, statements
elseif can
clause. A be skipped.ofAdditionally,
construction this type has more options can
the form:
if <condition1>
<statements1>
elseif <condition2>
<statements2>
elseif <conditions3>
<statements3>
...
else
<statements>
end

many switch clauses.

Theelseif statement can also in
Conditions be each
used of
in the of if...elseif
placeconditional statements
statements which
represent use
logical
(boolean) expressions which can be either true or false.
The second group of control statements consists of loop statements. A simple for loop looks

like this

8
 for LoopVariable = Range
<statements>
end
Range here consists of all the values through which the LoopVariable will iterate. For
instance, in the following code
for i=1:3
fprintf(’ d\n’,i)
end
the loop variable i iterates through the integers 1, 2, and 3, and the code itself prints the
integers fromloop
Another 1 tostatement
3 in a column format.
in Matlab is the while loop. The general form of a while statement

looks like this:

while <condition>
<statements>
end
The <statements> part of the code is executed as long as the conditional statement given by

<condition> remains true.

Logical statements
Expressions which are either true or false are called boolean or logical expressions. The logical
operators and relational operators in Matlab are given in the table below:
Operator Meaning
> (<) greater than (less than)
>= (<=) greater or equal (less or
equal) logical
logical equality
inequality
== or
and
not
∼= exclusive or
||

&&
∼
xor
The difference between the logical equality == and the assignment operator = is that the
assignment operator assigns the value on the right side of the operator to the variable on the
left side of the operator. At the same time, the logical equality operator checks whether the
twoWhen
valueswe
on compare
the righttwo
andexpressions
the left sides of the
using operator
relational are equal.
operators the result is a boolean which

can be true or false. For instance,

>> 3>2
ans =1

The variable ans here is of logical type.

9
Input and output functions

The simplest
input. way to
The input create displays
function a user defined variables
a prompt in the
string on Matlab is and
screen provided
awaitsbyforthe function
user input.
For instance,
>> side=input(’Enter the side of the square: ’)
Enter the side of the square: 3
side 3
=

The
side.code
It isabove asks to
generally inputidea
a good a number
to give(the
theside
userofan
theexplanation
square) andofstores
what it
is in the variable
expected from
them. The string ’Enter the side of the square: ’ above provides such an explanation.
Formatted output can be achieved in Matlab by using the fprintf function. For instance,
>> fprintf(’The
The area ofis
area of the square the square
equal is equal to
to 9 d\n’,side^2)

The expression %d here is 2a placeholder for the variable written after the string which, in
this case, is the value side . The symbol \n indicates that any output after this command
should
Morestart with a new
information aboutline.
the functions input and fprintf can be obtained by calling Matlab

Help with the help input or help fprintf commands.

Plotting graphs
A standard
provided by way to display
the plot information
function. in the form
In the simplest form,ofthe
a function
two-dimensional graph
plot looks likeinthis:
Matlab is
>> plot(x_vector,y_vector,’linestyle’)

Here,
the x-the
andvariables x vector
y-coordinates and
of the y vector
points on theare the vectors
graph. which contain,
The ’linestyle’ is acorrespondingly,
string variable
which contains the parameters for determining the type of line used for the graph. For
instance, ’k-o’ means that the graph will be drawn with black (’k’) solid (’-’) line and
each data point will be shown with an empty circle (’o’).
It is possible to plot multiple graphs on the same figure at the same time:
>> plot(x1_vector,y1_vector,’linestyle1’,x2_vector,y2_vector,’linestyle2’, ...)

holdAlternatively,
off commands.it is possible to add
If you don’t more
use the graphs
commandto the figure
hold on later
after by
theusing hold the
thecalling
graph, on,
plot function again will plot your graph in the previous figure window, erasing everything
which was on that figure before. If you would like a new plot to be shown in a new figure
window without erasing any of the previous plots, you can create a new figure window by
using the figure command. Finally, Matlab allows figures to be grouped together in the
form of subfigures. This can be achieved by using the subplot command.
Further reading

This introduction is designed to touch upon some of the basics of Matlab and is not meant to
be exhaustive. Some of the concepts discussed here will be applied in the sections to follow.
There are many other sources of additional information about Matlab, which include Matlab
Help, the mathworks.com website, and various Matlab manuals which can be purchased
separately.

10
Advice on good programming style

Usually the goal of a code developer is not only to write working code but also to make the
code easily understandable to other people. Here are several fairly standard tips which the
author found useful on how to write clean, easily readable, code:
1. Choose meaningful names for your variables and functions. This will make your code
significantly easier to read. For instance, the meaning of the variables Height and
Weight is much clearer than non descriptive h and w. If the variable name consists of
several words, such as carspeed and priceperitem, then capitalizing each word (for
instance, CarSpeed and PricePerItem) or using underscores (car speed and price
per item) can increase the readability of your code.
2. If your code is long, break it into smaller pieces. Give each piece only one function.
In Matlab, you can create user-defined functions and code cells for this purpose. User
defined functions can be created by using the function keyword (type doc function
in the Matlab command line to learn more about it). Code cells can be created by
entering the double percentage sign into your code. Each code cell can then be run
individually by using the “Run and Advance” button in the Matlab editor.
3. Add comments to your code by using the single percentage sign . Matlab will ignore
the rest of the line after the percentage sign. Explain the purpose of the commands
used, variables, and functions created in the comments. You will thank yourself for
doing this when you look at your code six months later. Anybody else trying to
understand your code will thank you too.
4. Don’t write dense code. Allow empty lines between the commands.
5. Use consistent indentation. This makes reading code with multiple cycles much easier.

Compare the two examples in the Fig. 1. Which one is easier to read?

Figure 1: An example of indentation usage.

6. Don’t make your lines too long. Split long commands and computations using triple dots
...

7. Clear code is usually better than clever code. Sometimes you can use a clever style of
programming to compound 5-10 lines of code into one single line. However, this code is
often much more difficult to understand.
You will discover many other tips and pointers on your own as you gain more
experience with programming.

11
Project overview

Each of the projects in this book is designed to test several concepts from linear algebra. The
goals of each project and linear algebra concepts used in it are summarized below:
Project 1: To introduce operations with matrices in Matlab such as matrix addition,
scalar multiplication, matrix multiplication, and matrix input functions in Matlab.
Project 2: To use the matrix operations introduced in the previous project to
manipulate images. Lightening/darkening, cropping, and changing the contrast of the images
are discussed.
Project 3: To use matrix multiplication to manipulate image colors similarly to photo
filters for images.
Project 4: Introduction to solving linear systems in Matlab.
Project 5: To discuss some of the strategies for the ranking of sports teams with an
example of the Big 12 conference in college football.
Project 6: To apply convolution of two matrices to blurring/sharpening of images.
Project 7: To create a simple recommender system based on the use of norms and inner
products.
Project 8: To study basic interpolation techniques and the least squares method with
application to climate data.
Project 9: To use orthogonal matrices for the rotation of 3D objects.
Project 10: To apply matrix mappings in order to generate fractals with the help of
Chaos game.
Project 11: To apply eigenvalue problems and orthogonal projections in order to create
a simple face recognition algorithm.
Project 12: To introduce Google’s PageRank algorithm and look at the eigenproblems
for the ranking of webpages.
Project 13: Application of eignevalues and eigenvectors for the data clustering with the
example of Facebook network.
Project 14: Application of singular value decomposition for image compression and
noise reduction.
Sample questions to check students’ understanding and checkpoints for introduced
variables are highlighted in the text of the projects in yellow. The templates of the labs 9-14
are given in the appendix.

12
Project 1: Basic operations with matrices in Matlab

Goals: To introduce matrix input functions and basic operations with matrices in Matlab.
The matrix operations to be studied include matrix addition and subtraction, scalar product,
matrix product and elementwise matrix product in Matlab, matrix concatenation, and
selecting submatrices.

To get started: Create a new Matlab script file and save it as lab01.m.
randi (rand), ’, *, .*, eye, zeros, ones, diag, spdiags.
Matlab commands used: length, linspace, size, max, min, mean,
sum,

What you have to submit: The file lab01.m, which you will create during the lab session.
INTRODUCTION

It is assumed that the reader is familiar with basic matrix operations and their properties,
so the definitions and facts here are given mostly for the sake of a review.
We can think of a matrix as a table of numbers:
 
a11 a12 . . . a1j . . . a1n
a21 a22 .. .. . a2j . . . a2n 
··· ··· . ··· ··· ···
ai1 ai2 . . . aij . . . ain
A   .. 
·am1
· · ·am2
· · ·. ·. ·. ·amj ··
· · . .. . a· mn

The matrix A above contains m rows and n columns. We say that A is an m × n (m-
by-n) matrix, and call m, n the dimensions of the matrix A. The matrix A above can also
be compactly
Matlab (originally as A =
writtennamed (aij), where
“MATrix aij is calledhas
LABoratory”) (i,an
j) element
extensiveoflist
theof matrix A.which
functions
work with matrices. The operations discussed in this project consist of matrix
addition/subtrac- tion, multiplication of a matrix by a constant (scalar multiplication),
transposition of a matrix, selecting submatrices, and concatenating several smaller matrices
into a larger one, matrix multi- plication and componentwise matrix multiplication, and
operations of finding maximal/minimal element in the matrix, and finding the sum/mean in
the Scalar
row/column of the matrix.
multiplication of a matrix A by a constant c consists of multiplying every element of
the matrix by this constant:
cA = (caij) .
Matrix addition and subtraction are possible if the matrices have the same dimensions
and is done componentwise:
A ± B = (aij ± bij) .
If A is an m× n matrix then the transpose AT (A’ in Matlab) of the matrix A is the n×
m matrix obtained by flipping A along its diagonal entries. Thus every column of A becomes
a row of AT , and every row of A becomes a column of AT . For instance,
T
Σ1 2 3 Σ  
14

4 56  3 
62

13
 If n is a positive integer then the n× n identity matrix, denoted by In, is a matrix which
has 1 in every diagonal entry (i, i) for 1 ≤ i ≤ n, and 0 in every other entry. There is one
identity
matrix for every dimension n. For instance,
1 0
I2 = Σ Σ  
1 00
0 1
 0 0 1
are the 2 × 2 and 3 × 3 identity matrices. 0
The identity matrices are special examples of diagonal matrices. A matrix A is diagonal if
A(i, j) is 0 whenever i ƒ= j (it is not required that aii =ƒ 0 for any i).
A matrix product C = AB of an m × n matrix A and an l × p matrix B exists if and
only if n = l. If this condition is satisfied, then C is an m × p matrix with the elements
n

Σ
cij = aikbkj.
k=1

2 3
4 0
For example if A = 1 0 and B
Observe that matrix mu
 
ltiplicatio  n is ΣthenAB
non-commutative,
Σ 2
= the product AB is given by

−1 −1 3 −1 1
   
(2 · 4 + 3 · 3) (2 · 0 + 3 · −1) (2 · 2 + 3 · 1) 17 −3 7
(1 · 4 + 0 · 3) (1 · 0 + 0 · −1) (1 · 2 + 0 · 1) = 4 0 2 .

AB = (−1 · 4 + −1 · 3) (−1 · 0 + −1 · −1) (−1 · 2 + −1 −7 1 −3

 
Σ Σ
 · 1)
1If C = 2 1 2 3
and D = 
0 1 2 then the product CD is not defined because the
number 0 0 1
1 7

of columns of C does not equal the number of rows of D.

TASKS

1. Open the lab01.m Matlab script you have created and type in the following commands:
basic operations with matrices
A=[1
B=[3 2
3 -10 4; 3 4 5 -6; 3 3 -2 5];
4 2];

This
a 3 ×will create awhile
4 matrix, new the
cellvariable
and define two newavariables
B contains 1 × 4 rowA vector.
and B. The variable
Use the A stores
“Workspace”
window of the Matlab environment to view the variables A and B.
Variables: A, B

2. Type in the following commands:

lengthA
lengthB = = length(A)
length(B)

Variables: lengthA, lengthb

14
Q1: What does the command length(A) do?

15
3. Add the vector B as the fourth row of the matrix A and save the result as a new matrix C

using the following code:

C=[A; B];

Variables: C

4. Create a new matrix D whose entries are located in the rows 2, 3, 4 and columns 3, 4 of
the matrix C:
D=C(2:4,3:4);

Variables: D

5. Create a transpose of the matrix D by using a quotation mark sign ’ in Matlab and save
the transposed matrix as the matrix E.
Variables: E
6. Check the size of the matrix E by using the Matlab size function and store the result of
this operation as two variables m, n.
Variables: m, n

7. Matlab allows for the creation of equally spaced vectors which can be useful in many
situations. For instance, run the following code:
EqualSpaced=0:pi/10:2*pi
EqualSpaced1=linspace(0,2*pi,21)

Take a look at the output produced by Matlab. Observe that both of these commands
created the vector with equally spaced entries ranging from 0 to 2π and the distances
between the elements equal to π/10.
Variables: EqualSpaced, EqualSpaced1
8. Create a new cell and find the maximal and minimal elements in each of the columns of
the matrix A. Store the result respectively in the row vectors named maxcolA and
mincolA. You can use the Matlab max and min functions:
Max, min, sum and mean
maxcolA=max(A)
mincolA=min(A)

Variables: maxcolA and mincolA

9. Now repeat the previous step but find the maximal and the minimal elements in each
row of the matrix A. Observe that there are at least two ways to approach this: you can
either use the max and min functions on the transpose of A or use these functions with
an additional argument 2 to show that you are interested in the maximum and
minimum along the second dimension (i.e. max(A,[],2)). Save the results as maxrowA
and minrowA. Finally find the maximal and minimal elements in the whole matrix A,
and save the results as maxA and minA.

Variables: maxrowA, minrowA, maxA and minA

16
 10. Repeat the last two steps using the commands mean and sum instead of max and min.
You should create six new variables, corresponding to the column/row means/sums,
and the mean/sum of the elements of the entire matrix.
11. Create a new cell and use the command randi([-4 4],5,3) to create two matrices F and
Q2: What meancolA,
do the commands mean and sumsumrowA, meanA,
meanrowA,
G Variables:
with 5 rows and 3 columns and sumcolA,
random integer entries from −4 to 4:
Matrix addition/subtraction, and multiplication
F=randi([-4 4],5,3)
G=randi([-4 4],5,3)

Variables: F, G
12. Perform the operations0.4*F, F+G, F-G, F.*G, storing the results in ScMultF, SumFG,
DiffFG, and ElProdFG respectively.

13. Check
sizeA, the sizesizeF,
Variables: of F and the size of A, storing the results in the variables sizeF and
respectively.
Variables: ScMultF,
Q3: What does SumFG,
the last DiffFG,
operation
14. Compute H=F*A.
Q4: Can we matrix-multiply F and A?
Q5: What are
Variables: H the dimensions of H?
15. Generate the identity matrix eye33 with 3 rows and 3 columns using the Matlab eye
command.
Variables: eye33

16. Run the commands:

zeros53=zeros(5,3);
ones42=ones(4,2);

Q6: What
Variables: do the functions zeros and ones
zeros53,
17. Generate the diagonal 3 × 3 matrix S with the diagonal elements {1, 2, 7} using the Matlab
diag function.
Variables: S
18. Now let us do the opposite: extract the diagonal elements from the matrix and save
them in a separate vector. The same function diag accomplishes that as well:
R=rand(6,6)
diagR=diag(R)

17
 This creates the
the diagonal 6 ×from
entries 6 matrix R with
it, and saves random
them inentries
the 1 ×from the interval
6 vector diagR.(0,Observe
1), extracts
that
the function diag has other interesting functions. To learn more about it type help
diag in the command line.
Variables: R,diagR
19. Another function which allows to create diagonal matrices is spdiags. Technically, this
function creates a sparse matrix which is a matrix with a large number of zero
elements. These matrices are stored in a special form in Matlab with only non-zero
elements are stored, and operations with them are also done in a special way. To
convert sparse form of the matrix into the regular one, the command full can be used.
Run the following code:
diag121=spdiags([-ones(10,1) 2*ones(10,1) -ones(10,1)],[-1 0
1],10,10); full(diag121)
Variables: diag121

Q7: What does this code do?

18
Project 2: Matrix operations and image manipulation

Goals: To apply elementary matrix operations such as matrix addition, subtraction, and el-
ementwise multiplication to image manipulation. We will look at the lightening/darkening of
images, selecting subimages, changing contrast, rotating and flipping images.
To get started:

• Create a new matlab script and save it as lab02.m.

• Download the image file einstein.jpg and save it in your working directory 1.
rot90, floor, ceil, round, fix.
Matlab commands used: imread, imshow, uint8, double, flip, size, ’,
zeros,

What you have to submit: The file lab02.m, which you will create during the lab session.
INTRODUCTION

An image in a computer memory can be stored as a matrix with each element of the
matrix representing a pixel of the image and containing a number (or several numbers) which
corre- sponds to the color of this pixel. If the image is a color image, then each pixel is
characterized by three numbers corresponding to the intensities of Red, Green, and Blue (the
so-called RGB color system). If the image is a grayscale image, then only one number for the
intensity of gray is needed. The intensity of each color typically ranges from 0 (black) to 255
(white). This allows for the representation of over 16 million colors in the RGB system which
is more than a human eye can distinguish. The images stored in this way are called bitmaps
(there are also images which do not use pixels, such as vector graphics, but we will not talk
about those
If you here).
have ever used an image editor before, you may know that there are several fairly
standard operations which you can perform with images. In this project we will replicate
some of them using operations on matrices. By the end of the project you will understand
how graphic editors process images to achieve different visual effects. As it turns out, many
of these operations are accomplished by manipulating the values of the pixel colors.
TASKS

1. To begin let us load the image into Matlab. This can be done by using the Matlab
command imread. This command allows Matlab to read graphic files with different
extensions. The output is an array with the dimensions equal to the dimensions of the
image, and the values corresponding to the colors of the pixels. Here we will work with
a grayscale image, so the elements in the matrix will be integers ranging from 0 to 255
in the Matlab integer format uint8. Type in the following code to load the file
“einstein.jpg” into Matlab:
Load a grayscale jpg file and represent the data as a matrix
ImJPG=imread(’einstein.jpg’);
1
The image is available at: https://commons.wikimedia.org/wiki/File:Albert_Einstein_Head.jpg and
has no known copyright restrictions. You can also use an alternative .jpg grayscale image of your choice. The
answers and the variable values will differ from what is shown here in that case.

19
 The array ImJPG is a two-dimensional array of the type uint8 which contains values
from 0 to 255 corresponding to the color of each individual pixel in the image, where 0
corresponds to black and 255 to white. You can visualize this array by going to the
“Workspace” tab of the main Matlab window and double clicking on ImJPG.
Variables: ImJPG

2. Use the size function to check the dimensions of the obtained array ImJPG:
[m,n]=size(ImJPG)
Variables: m, n
Q1: What are the dimensions of the image?

3. Check the type of the array ImJPG by using the command isinteger:
isinteger(ImJPG)

The output of the isinteger command is boolean, meaning it will produce either 1
(true) or 0 (false).
4. Find the range of colors in the image by using max and min commands and save those
elements as maxImJPG and minImJPG. Observe that execution of this command depends
on the version of Matlab on your computer.
Variables: maxImJPG, minImJPG

5. Finally, display the image on the screen by using imshow:

imshow(ImJPG);

If you did everything correctly, you should see the image on the Fig. 2 displayed on your
screen in a separate window.

Figure 2: The original image

6. Now we are ready to work with this image. Let us look at some basic image operations
we can do using matrices. It is possible to crop the image by selecting a submatrix of
the matrix ImJPG. Selecting a submatrix can be done simply by naming the rows and
columns you want to keep from the initial matrix. For instance, the following
commands
Select and display a part of the image, paste into a different image;
ImJPG_center = ImJPG(100:m-100, 100:n-70);
figure;
imshow(ImJPG_center);

20
 will create a new code cell, select the central part of the image leaving out 100 pixels
from the top and bottom, 100 pixels on the left and 70 pixels on the right, and display
the result in a new figure window.
Variables: ImJPG center
7. We can paste the selected part of the image into another image. To do this create a
zero matrix using the command
ImJPG_border=uint8(zeros(m,n));

Then paste the preselected matrix ImJPG center into matrix ImJPG border and
display the image:
ImJPG_border(100:m-100, 100:n-70)=ImJPG_center;
figure;
imshow(ImJPG_border);

Notice
default the
theconversion be of theuint8.
array will command It is necessary
type double, to use this
and imshow command
command doesbecause by
not work
correctly with this type of array. To see this, try to remove uint8 and observe what
happens when you run the code above.
Variables: ImJPG border
8. The familiar operation of flipping the image vertically can be easily realized in Matlab
by using the matrix command flip:
Rotate and flip the image horizontally/vertically
ImJPG_vertflip=flip(ImJPG);

Variables: ImJPGreverses
This command vertflip
the order of elements in each column of the matrix.
9. Transposing the matrix is achieved by adding an apostrophe ’ at the end of the matrix
name. Observe that transposing the image matrix is equivalent to rotating the image 90
degrees counterclockwise and flipping it horizontally. Try the following commands:
ImJPG_transpose=ImJPG’;
figure;
imshow(ImJPG_transpose);

Variables: ImJPG transpose

10. Flipping the image horizontally is a little more involved and requires the use of a
matrix transposition as well. For instance, this can be accomplished by using the
command:
ImJPG_horflip=flip(ImJPG_transpose)’;

Display theImJPG
Variables: horflip
resulting matrix and observe what happened to the initial image.

11. Apply the command rot90 to the image using the following code:
ImJPG90=rot90(ImJPG)
figure;
imshow(ImJPG90)

21
 Variables: ImJPG90
Q3: What does the command rot90 do?

12. Try running the following commands:

Color inversion, lightening/darkening
ImJPG_inv = 255-ImJPG;
figure;
imshow(ImJPG_inv);

Display the resulting

is subtracted image
here from the in a new figure
constant window.
255 which Observe also,does
mathematically thatnot make ImJPG
an array sense.
However, Matlab treats the constant 255 as an array of the same size as ImJPG with all
the elements equal to 255.
Variables: ImJPG inv
Q4: Explain what happened to the image.
13. It is also easy to lighten or darken images using matrix addition. For instance, the
following code will create a darker image:
ImJPG_dark=ImJPG-50;
You can darken the image even more by changing the constant to a number larger than
50. Observe
array thatnegative.
to become the command above
However, can technically
because the array make
type issome of the
uint8, elements
those of the
elements are
automatically rounded
Variables: ImJPG dark to zero.

Q5: How do you lighten the image?

14. Let us create Andy Warhol style art with the image provided. To do so we will arrange
four copies of the image into a 2 × 2 matrix. For the top left corner we will take the
unaltered image. For the top right corner we will darken the image by 50 shades of
gray. For the bottom left corner, lighten the image by 100 shades of gray, and finally, for
the bottom right corner, lighten the image by 50 shades of gray. Then we will arrange
the images together in one larger matrix using matrix concatenation. Finally, display
the resulting block matrix as a single image. If you did everything correctly, you should
see an image resembling the Fig. 3.
Variables: ImJPG Warhol
15. Matlab has several commands which allow one to round any number to the nearest
integer or a decimal fraction with a given number of digits after the decimal point.
Those functions include: floor which rounds the number towards negative infinity (to
the smaller value), ceil which rounds towards positive infinity (to the larger value),
round which rounds towards the nearest decimal or integer, and fix which rounds
towards zero.
A naive way to obtain black and white conversion of the image can be accomplished by
making all the gray shades which are darker or equal to a medium gray (described by a
value 128) to appear as a complete black, and all the shades of gray which are lighter
than this medium gray to appear as white. This can be done, for instance, by using the
code:
A naive conversion to black and white
ImJPG_bw=uint8(255*floor(ImJPG/128));
figure
imshow(ImJPG_bw);

22
 Figure 3: Andy Warhol style art with matrix concatenation
Observe that this conversion to black and white results in a loss of many details of the
image. There are possibilities to create black and white conversions without losing so
many details. Also notice the function uint8 used to convert the result back to the
integer format.
Variables: ImJPG bw
16. Now let us write code which reduces the number of shades in the image from 256 to 8
by using round function. Save your array as ImJPG8 and display it in a separate
window.
Variables: ImJPG8
17. The contrast in an image can be increased by changing the range of the possible shades
of gray in the image. Namely, we can increase the difference between the colors of the
image, for instance, by making the dark shades of gray to appear even darker. One of
the simplest ways to do this is to scalar multiply the matrix by some constant. Consider
the code:
ImJPG_HighContrast=uint8(1.25*ImJPG);

Display the image andthe

increasing/decreasing observe the result.
constant The 1.25
(we select contrast here Ifcanwebeselect
here). manipulated by
a positive
constant greater than 1, then the contrast is increased; if we select a constant less than
1, then it is decreased. Observe that this operation might lead to some of elements of
the matrix to become outside of 0 − 255 range and, hence, might lead to data loss.
Variables: ImJPG HighContrast
18. Finally, try another operation which is not linear but is still interesting. This operation
is called gamma-correction. Run the following code and observe the result:
Gamma correction
ImJPG_Gamma05=uint8((double(ImJPG)).^0.95);

23
 figure;
imshow(ImJPG_Gamma05);
ImJPG_Gamma15=uint8((double(ImJPG)).^1.05);
figure;
imshow(ImJPG_Gamma15);

Variables: ImJPG Gamma05, ImJPG Gamma15

24
Project 3: Matrix multiplication, inversion, and photo filters

Goals: We will use matrix multiplication in order to manipulate image color and achieve dif-
ferent effects.
To get started:

• Create a new Matlab script and save it as lab03.m.

• Download the file daylilies.jpg 2 and save it in your working directory. You can use
an alternative .jpg file of your choice. Your outputs will be different from shown here in
that case.

Matlab commands used: imread, imshow, size, figure, reshape, uint8,

double, for... end

What you have to submit: The file lab03.m which you will create during the lab session.
INTRODUCTION

Have you ever wondered how photo filters work? In this project we will learn how to
manip- ulate images in order to obtain different color effects. As we learned in the previous
project, an image can be represented by a matrix with the dimensions corresponding to the
dimensions of the image. Each of the elements of this matrix contains the number (or
numbers) representing the color of the corresponding pixel. If the image is a color image,
then the color of the pixel is represented by three numbers {R, G, B} (Red, Green and Blue)
with each number ranging from 0 to 255 (RGB system). We will look at several different
ways to manipulate the image colors to obtain effects similar to those of Instagram and other
graphics software.
TASKS

1. Start by loading the image into Matlab and displaying it on the screen by using the
familiar commands imshow and imread. Save the resulting array as ImJPG. You should
see something similar to the image on the Fig. 4.
Variables: ImJPG

2. Check the dimensions of the array ImJPG using the command

[m,n,l]=size(ImJPG)

Observe that Matlab will return three numbers. Since the image is colored, the
resulting array is three-dimensional. Among other things, Matlab allows operations on
multidimen- sional arrays. The first two dimensions of the array ImJPG correspond to
the horizontal and vertical dimensions (number of pixels) in the image, and the third
dimension stores three numbers corresponding to the values of Red, Green, and Blue for
each pixel. These three colors mixed together produce all other colors in the image. The
values for Red, Green and Blue can range from 0 to 255, allowing us to create 2563 =
16, 777, 216 colors. In this project, we will be manipulating these three numbers to
achieve various visual effects (similar to the way photo filters work).

2
Variables: m,n,l
The copyright to this photo belongs to the author.

25
 Figure 4: The original image showing orange and yellow daylilies
3. First of all, let us look into amount of Red, Green, and Blue color in the image. To do
this, let us extract individual layers or color channels from the image array. To obtain
the red channel of color in the image, use the following command:
redChannel = ImJPG(:, :, 1);

Similarly, extract
greenChannel green and blue
and blueChannel. color each
Display channels and
array in save them
a separate figureinusing
the figure
arrays
and imshow commands. Observe that individual color channels will be shown by
Matlab as grayscale images. That is due to the fact that we have taken “layers” off of
the matrix ImJPG and each of these layers individually looks like a two-dimensional
array with numbers ranging from 0 to 255 which is exactly the form in which grayscale
images are represented in Matlab. If you did everything correctly, you should see
something similar to the Fig. 5. The whiter the pixel appears on a particular color
channel, the larger amount of that color is contained in that pixel. On the other hand,
the darker the area is, the less corresponding color is in that part of the image. The file
daylilies.jpg appears to contain mostly green and red (surprise, surprise!).

Variables: redChannel, greenChannel, blueChannel

Figure 5: Different color channels

4. Let us convert the image into a grayscale image by using the following filter matrix:
 
1/3 1/3 1/3

 1/31/3
1/3 1/31/3
Multiplication by this matrix allows to compute an average of red, green, and blue in
each pixel. Use the following code:

26
 GrayMatrix=[1/3 1/3 1/3; 1/3 1/3 1/3; 1/3 1/3 1/3];
for i=1:m
for j=1:n
PixelColor=reshape(double(ImJPG(i,j,:)),3,1);
ImJPG_Gray(i,j,:)=uint8(GrayMatrix*PixelColor);
end;
end;
figure;
imshow(ImJPG_Gray)

This code produces an array ImJPG Gray in which the individual values of Red, Green,
and Blue have been replaced by the averages of these three colors. Observe the use of
the commands reshape, uint8, double. For every pixel of the image, first, the color
attributes (RGB) are extracted from the matrix ImJPG, then these color attributes are
treated as a vector with three components [R, G, B], and finally, the new color attributes
are obtained by multiplying the vector [R, G, B] by the filter matrix GrayMatrix on the
left. The result is saved as color attributes of the corresponding pixel in the array
ImJPG Gray. All the pixels of the array ImJPG Gray have equal number of Red, Green,
and Blue; this produces different shades of gray color. Observe that there are different
ways to create a grayscale conversion, and this is just one of them.
Q1: Explain the purpose of uint8 and double commands in the code above.
5. Modify the code above to produce a sepia conversion of the image. Instead of GrayMatrix
use the following filter matrix, and reproduce the code above with this matrix:

Variables: ImJPG Gray 0.393 0.769 0.189 

SepiaMatrix = 0.349 0.686 0.168
 
0.272 0.534 0.131

Variables: ImJPG Sepia

Save the result in the ImJPG Sepia array and display this array using imshow command.

 
6. Next, consider the filter matrix 1 0 0

 0 00 0 0
Modify the code again, using the matrix
new array ImJPG Red. Display the image. above as a filter matrix. Save the result as a
Variables: ImJPG Red
Q2: What does the transformation above do?

7. Let us permute the colors in the image. To do this repeat the steps above with the matrix:
 
0 0 1

 0 1 0P1 0 0
Save the result as a new array ImJPG Permute. If you did everything correctly, you
should see the image shown on the Fig. 6. Observe that the flowers turned purplish
blue. The

27
 matrix above is an example of a hue rotation filter. You can produce other hue rotation
effects with this more general transformation:
   
0.213 0.715 0.072 0.787 −0.715 −0.072

HueRotateMatrix =  0.213 0.2130.715

0.715 0.072

0.072 + cos(θ)
 −0.213 0.285 −0.072  +
 
−0.213 −0.715
0.213 −0.715 0.928
0.143 0.140 −0.283 .
sin(θ)  −0.787 0.715 0.072
Here θ is the angle of rotation. You can try experimenting with various values of the angle
θ to get different color effects.

Variables: ImJPG Permute

Figure 6: Colors permuted

8. Now let us delete one of the colors in the image, say, green. First, produce the filter
matrix which deletes Green in the image and keeps the values of Red and Blue intact.
Use it to filter the image. Save the result as ImJPG DeleteGreen.
Variables: ImJPG DeleteGreen

9. It is possible to invert the colors of the image by using the following code:
ImJPG_Invert=255-ImJPG;
figure;
imshow(ImJPG_Invert);
Observe that here again Matlab automatically substitutes a matrix of appropriate
dimen- sion (in this case m× n× l) with all the elements equal to 255 instead of the
constant 255. Variables: ImJPG invert
10. It is also possible to enhance/mute individual colors in the image. For instance, consider
the color transformation with the matrix:
 
1.2 0 0
0 0.75 0

SaturateMatrix = 
Save the result as ImJPG Saturate.
0 0 2
Variables: ImJPG saturate

Q3: Explain what the transformation above does to the image.

28
11. Consider the color adjusting filter generated by the matrix:
 
0.7 0.15 0.15
UserMatrix = 0.15 0.7 0.15
 
0.15 0.15 0.7
Apply this filter to the array ImJPG and save the result as ImJPG User.
Variables: ImJPG User
Q4: Explain what the transformation above does to the image.
12. Consider the matrix in the previous step. Is the transformation defined by this matrix
invertible? Which means, can you get the original image colors back once you applied
the filter? Or is some of the information lost? Use the inv command to create the
matrix for the inverse filter. Apply this matrix to the arrays ImJPG User and ImJPG.
Display the resulting images.
Q5: What does the inverse transformation do with the colors?
13. Repeat the steps above to see if the sepia transformation is invertible.

Q6: What do you see? Why do you think this happened?

14. Finally, try the non-linear color transformation such as Gamma-conversion:

ImJPG_Gamma1=uint8(double(ImJPG).^0.9)+30;
ImJPG_Gamma2=uint8(double(ImJPG).^1.1)-50;

Variables: ImJPG Gamma1, ImJPG Gamma2

Display the resulting images given by the matrices ImJPG Gamma1, ImJPG Gamma2.

29
Project 4: Solving linear systems in Matlab
Goals: To explore the built-in functions in Matlab used to solve systems of linear equations

Ax = b. To compare different options in terms of accuracy and efficiency.

To get started: Create a new script file and save it as lab04.m.

Matlab commands used: \, inv, rref, pinv, null, tic... toc

What you have to submit: The file lab04.m, which you will create during the lab session.
INTRODUCTION
All the methods of solution of systems of linear equations can be divided into two large
groups: direct and iterative methods. The direct methods aim to obtain an “exact” (up to
numerical round off error) solution of the system in a finite number of steps. The iterative
methods aim to create a convergent sequence of approximations to the exact solution of a
linear system. Iterative methods are usually used when the matrix of a system has a known
specific structure. Matlab has many built-in functions designed to solve systems of linear
equations by both direct and iterative methods. In this project we will discuss only direct
methods suchelimination
Gaussian as Gaussianinelimination.
Matlab is based on LU - or PLU -decomposition of the matrix of the
system A. We can reduce the matrix A to an upper triangular form using forward
elimination and then solve the resulting triangular system by backward substitution. In the
process of doing this, we factor the matrix A into a product of two matrices: a lower
triangular matrix L and an upper triangular matrix U. Observe that this process cannot be
always carried out without interchanges of rows, and, hence, we need an additional
permutation matrix P in our matrix decomposition. Observe, that the matrix P is necessary
even in the cases when LU -decomposition can be carried out due to the matters of numerical
stability (this is called pivoting). Gaussian elimination is the preferred method of solving the
system numerically if you do not have any additional information about the structure of the
matrix A.matrix A is square, we can define the inverse matrix of A (denoted as A−1). The
If the
matrix A−1 is also a square matrix of the same size as A. The inverse matrix exists as long as

A is not singular, and satisfies the following equality:

A · A−1 = A−1 · A = I,
where I is the identity matrix of the same size as A and A−1. If the matrix A−is invertible,
then we can multiply both sides of the system Ax = b by the inverse matrix A 1 on the left,
and obtain the following formula for the solution of the system: x = A−1b. However, as it
often happens in numerical linear algebra, this theoretical formula is inconvenient for
numerical computation of the solution. The reasons for this are two-fold. Firstly, the
computation of inverse matrix is significantly more costly than simply solving the system.
Secondly, the accuracy of the resulting solution will be worse than if LU -decomposition is
used.
Finally, yet another possibility to solve a linear system is to use the reduced row echelon
form of the augmented matrix of the system. Theoretically, this is equivalent to using Gauss-
Jordan elimination. This method of solution will be explored in this project as well. Analysis
of the corresponding algorithm shows that the resulting solution is likely to have a larger
error compared to Gaussian elimination and is likely to take more time to compute.

30
 TASKS

1. Create a new cell and type in the following commands to generate the matrix A and the
column vector b:
Matlab backslash solver
clear;
A=magic(5);
b=[10; 26; 42; 59; 38]

Variables: A, b

2. If nothing is known about the structure of the matrix of the system, the best approach
to solution is to use the Matlab backslash “\” solver:
x=A\b

In most cases, this method will produce the numerical solution of the system Ax=b with
the highest possible accuracy and in the shortest computational time.
Variables: x
3. Whenever we are talking about the accuracy of the solution, ideally we want to find the
difference between the approximate numerical solution xnum and the exact solution xex of
the system. However, this is not feasible in most situations since the exact solution xex
is generally unknown (otherwise, there is no point in using numerical methods at all).
The next best thing is to find the residual r=A*x-b. If the residual is small then we can
assume, with certain limitations, that our approximate solution is close enough to the
exact solution (this is not always true, however). Let us check how accurate the solution
is which we found in the previous step. Type the following line into your Matlab script:
r=A*x-b

The resulting vector r should be very close to zero.

Variables: r
4. The Matlab backslash “ ” operator is based on LU - or PLU decomposition of the matrix
A. The main idea−1 here \is that any square non-singular matrix A (meaning that theT
inverse matrix A exists) can be written as A = PLU (actually, in Matlab it is P
LU), where P is a permutation matrix (or matrix obtained by permuting the rows of
the identity matrix of the same size as A); L is a lower-triangular matrix (all the
elements above the main diagonal are equal to zero), and U is an upper triangular
matrix (all the elements below the main diagonal are equal zero). Matlab has a special
lu function to generate a PT LU decomposition of the matrix A. Type the following
lines into your Matlab script:
LU-factorization
[L,U,P]=lu(A);

This will create a new code cell and compute the matrices P, L, and U. Now let us use
these matrices to solve the linear system Ax = b:
x1=U\(L\(P*b))
r1=A*x1-b
err1=x-x1

31
 Observe that the difference between the solutions err1=x-x1 is exactly the zero
vector3. This is due to the fact that we found the solution using exactly the same steps
as Matlab. Observe also that at no point did we use the Matlab inverse matrix function
inv. In general, there is almost never a good reason to compute an inverse matrix.
Whenever you see an inverse matrix in a numerical linear algebra textbook it is usually
shorthand for “solve a system of linear equations here”.
Variables: L, U, P, x1, r1, err1
5. Matlab also has a separate slash operator / to solve linear systems of the type yA = b.

Try running the following command

y=b’/A

Variables: y

6. Next, let us use the inverse matrix to solve the same system of linear equations:
Inverse matrix
x2=inv(A)*b
r2=A*x2-b
err2=x-x2

Here inv(A) produces the inverse A−1 of the matrix A. This code will also produce the
solution of the system Ax = b, compute the residual, and find the difference between
the current solution and the solution obtained by the backslash operator. Observe that
the vector of difference err2 is not exactly zero unlike in the case when we used LU -
decomposition. Observe also that the residual r2 is likely to be larger than the one
obtained with the help of the backslash operator.
Variables: x2, err2, r2
7. Finally, another way of solving the system Ax = b is based on the reduced row echelon
form. The reduced row echelon form in Matlab can be produced by using the command
rref. Type in the following code:
Reduced row echelon form
C=rref([A b]);
x3=C(:,6)
err3=x-x3
r3=A*x3-b

Observe that athe

rref is also residual
much r3 is much
less efficient largerof than
method thelinear
solving residual r. Generally
systems comparedspeaking,
to the
backslash operator and its use for this purpose is not advised.
Variables: x3, err3, r3

8. To compare the computational efficiency of all three methods, run the code cell:
clear;
Computational time
Num=500;
A=rand(Num,Num)+Num*eye(Num);
3
This has been observed in both Matlab 2015a and 2019a. It has come to author’s attention that this might
not be true in other versions of Matlab.

32
 b=rand(Num,1);
tic;
x1=A\b;
toc;
tic;
x2=inv(A)*b;
toc;
tic;
C=rref([A b]);
x3=C(:,101);
toc;

This code compares all three methods in terms of the computational efficiency in the
example of a large matrix A with the dimensions Num x Num. The parameter Num in
this code is set to be 500. The matrix A is chosen in such a way that it is diagonally
row- dominant. Then by the theorem of linear algebra it is guaranteed that the matrix
A is non- singular. The Matlab function tic ... toc returns the computational time
necessary to perform the Matlab operations between tic and toc. You can try to
experiment with the parameter Num to see how the computational time changes with
the size
A. of the
Observe matrix
that solving the system with the rref function may take a very noticeable
amount of time (you probably don’t want to set Num to be more than 1000).
Variables: Num, A, b, C, x1, x2, x3
9. Finally, let us see how to use Matlab to solve rectangular systems of linear equations. Let
A be an m n matrix of the system and let b be an m 1 column vector. Let us start
with the situation when m > n. In this case, the system is overdetermined. However, it is
still possible ×
to use the Matlab backslash operator. Consider the following code:
Overdetermined system
clear;
A=[1 2 3; 4 5 6; 7 8 10; 9 11 12]
b=[1; 2; 3; 4]
x=A\b
r=A*x-b

The system Ax = b is inconsistent. The resulting “solution” x is not the solution of

the original linear system Ax = b itself (observe that the residual r is not close to
zero), but rather the solution to the so-called least squares problem. Namely, x
minimizes the length of the residual vector r = Ax b. It is known that the solution to
the least squares problem can also be obtained by solving the system of the so-called
normal equations AT Ax = AT − b. To verify that the vector x is indeed the solution to
this system, type in the following code:
y=(A’*A)\(A’*b)
err=x-y

and notice that x-y is indeed close to zero (it is rare to obtain an exact zero in
numerical computations due to roundoff errors).
Variables: A, b, x, r, y, err
10. Let us look at the last possible case when m < n. This is a case of an underdetermined
system. If this system is consistent then it has an infinite number of solutions. Consider
the following code:

33
 Underdetermined system
clear;
A=[1 2 3; 4 5 6; 7 8 9; 10 11 12]’
b=[1; 3; 5]
x=A\b
r1=A*x-b
The backslash operator in this case produces only one particular solution to the system

Ax = b. Another particular solution can be obtained by the following operations:

y=pinv(A)*b
r2=A*y-b

where the function pinv produces the so-called Moore-Penrose pseudoinverse (the
precise definition of this matrix is outside of the scope of this project). It is known that
the vector y has the smallest possible length among all the solutions of the system Ax
= b. Finally, to obtain the complete set of the solutions of the system above, consider
the function null. This function produces the complete set of the linearly independent
solutions to the homogeneous system Ax = 0. The complete solution to the
inhomogeneous system Ax = b then can be obtained by:
z=null(A)*C+x;

where the constant vector C = [C1, C2]T can be chosen arbitrarily (pick any numbers for
C1, C2 that you like!).

Variables: A, b, x, r1, y, r2, C, z

34
Project 5: Systems of linear equations and college football
team ranking (with an example of the Big 12)
Goals: To use systems of linear equations to develop a ranking for Big 12 college football
teams based on 2016-2017 season data using Colley’s and Massey’s methods.
To get started:

• Open a new Matlab script and save it as lab05.m.

• Download the files Big12th.xls,

the file Big12th.xls Scores.mat,
is provided for reference only Differentials.mat.
and and Observeinthat
won’t be used explicitly the
lab. You can also use the scores from your favorite tournament. The scores here are
based on a 10 team round-robin tournament.
Matlab commands used: load, eye, diag, sum, abs, fprintf, for ... end,
sort

What you have to submit: The file lab05.m, which you will create during the lab session.
INTRODUCTION

The present project deals with the ranking of competing sports teams. At first, this does
not appear to be a difficult problem. For instance, the easiest way to rank sports teams’
performance is to compute the percentage of games which each team has won. This method
has several drawbacks which are especially noticeable for the college football championship.
In particular, for the percentage ranking to be objective, it is necessary that every team plays
with a majority of teams participating in the competition. However, the number of teams
participating in the Bowl Championship Series (BCS) is very large and each team only plays
a small number of games. This is what makes the ranking of college football significantly
different from, say, the NFL, where a relatively small number of teams play a representative
number of games against
It is necessary to note each
that other.
there is no one true answer to a ranking problem. The rank of a
team will depend on the factors which we will take into account and on the weight we will
give to those factors. There are several questions to ask here. For instance, do point
differentials matter? Is a 60-20 win any better than a 34-33 win? Taking into account point
differentials may have both pros and cons. A 34-33 win versus a strong opponent is arguably
better than 60-20 versus an unranked team. At the same time, if all the teams have about
the same strength, than 60-20 is clearly better. At the end of the day, whether to take into
account point
In this lab differentials
we will look orat not
the is
twoa matter
methodsof of
taste and preference.
ranking currently used by BCS with an
example of the Big 12 Conference data from 2016-2017 season. As you may know, 10 teams
play in the conference: Baylor University, Iowa State University, University of Kansas,
Kansas State University, University of Oklahoma, Oklahoma State University, Texas
Christian University, University of Texas Austin, Texas Tech University, and West Virginia
University. We will look at the games only within the Big 12 conference and ignore all other
games outside of it.

35
Colley’s bias free ranking

The first method we will look at has been developed by astrophysicist Wesley Colley and is
named after him: Colley’s bias free ranking. This method does not take into account point
differentials
There areintwo
themain
gamesideas
(which again
behind thecan be both
method. a good
First and
of all, wea substitute
bad thing).the winning per-

centages by the Laplace’s rule of succession:

1 + wi
r = ,
i
2 + ti
where wi is the number of winning games for the ith team, ti is the number of total games,
and ri is the rank of the team. Laplace’s rule of succession is used for events which have only
two outcomes such as either winning or losing. The rule comes from the idea in probability
that unless an event always happens or never happens, we should avoid assigning this event
probabilities of 1 (always true) and 0 (never true). Indeed, after playing one game, a team
could either win the game or lose it. If the team won the game, than its percentage ranking is
100%; if it lost the game, its ranking is 0%. Meaning that after one game some teams are
infinitely better than others which is not very realistic. At the same time, the Laplace’s rule
of succession will give us approximately 2/3 and 1/3, meaning that the team which won one
game is about two times better than the one which lost one game - a good starting point for
further estimates. The second idea is that the rank of opposing teams should matter just as
much as the total number of wins and losses. To incorporate the opposing teams’ rankings
can write: into our system, we
w i − li w i + li w i − li ti
wi = + = + ,
2 2 2 2
where li is the number of losses for the ith team.
Now observe that with the use of Laplace’s succession rule the rankings of the teams will
hover around 1/2, so we can approximate: Σ
ti 1
=t ≈ ri .
Basically, we substitute each 1/2 with the rankingt of the j of the jth team which ith team
ri
played against. Now, let us substitute the lasti formula jback into the Laplace’s rule to obtain
the following system of linear equations 2(check it!):j

i i Σ
t j w2i − li
(2 + t )r − j ri = 1 + .
This system has N equations with N unknowns ri, where N is the number of teams playing
in the championship (in our case N = 10). Observe that the matrix of this system has special
properties - it is diagonally row dominant, meaning that the system will always have a unique
solution.
Massey’s method

Massey’s method was developed by Ken Massey in his undergraduate research project and is
now used by the BCS (Oh, the places you can go!... if you know math). Unlike Colley’s
method,

36
Massey’s takes into account point differentials as well as number of wins/losses. Again,
denote as ri the rankings of the competing teams. The data we obtain from the games can be
written in the form of linear equations:
ri − rj = dij,
meaning that ith team won (lost to) jth team by the dij point differential.
The data from the games produces a large system of linear equations of the form:

PR = B,
where P is the matrix of the system such that each row has only two non-zero elements, 1 and
−1, R is the column-vector of the rankings ri, and the vector of the right-hand-side B
contains the corresponding point differentials.
Observe that, in general, we are likely to have a lot more equations than unknowns (more
games than teams), so, in general, this system of equations is likely to be inconsistent and
not have any solutions. To see this, assume that the first team beat the second team by 20
points and the third team beat the second team by 5 points. This data implies that the first
team should beat the third team by 15 points. Of course, the point differential of the actual
game
So,(ifinstead,
it happens!) between
we will give upthe
on first
tryingteam and the
to solve thisthird team
system mightand
exactly, be different.
we will look for the
vector R which satisfies the system as close as possible. Namely, we will employ the least
squares method, meaning, that we want to find the vector R which minimizes the vector
norm:
||PR − B||2
where || · ||2 is a usual Euclidean norm: if x = (x1, x2, ..., xn) is a vector with n components,
then
.
||x||2 = x2 1+ x2 2+ ... + x2n.
It is well-known from linear algebra that this problem is equivalent to the so-called system
of normal equations written as:
PTPR = PTB,
where PT is a transpose of the matrix P. Now, PTP is an N × N matrix, where N is again the
number of the teams, so this results in a square system of linear equations.T
The last obstacle which we will need to overcome is that the matrix P P has a rank of at
most N 1 (why?), and hence, in general, the last system does not have a unique solution. To
obtain a unique solution we will change each elementTin the last row of the matrix PTP to 1
and we will change the last element in the vector P B to zero. Practically this means that
−
all the
Nowrankings obtainedthe
that youri learned in theory
this way will need
behind to sum up
the ranking, lettouszero.
create code in Matlab which will
produce the rankings of teams in the Big 12.
Scores.mat and Differentials.mat in the directory you To do so,will
first
be of all save
working in. the
Eachdata files
of these
files contains the results of the games from the Big 12 championship from the season 2016-
2017 in the form of the table (matrix). The Scores.mat contains a 10 × 10 matrix where
each row corresponds to the scores of the Big 12 teams in the following order:
1.Baylor
2.Iowa State

3. University of Kansas

37
 4. Kansas State
5.University of
Oklahoma 6.Oklahoma
State 7.Texas Christian
8.University of Texas Austin
9.Texas Tech
10.West Virginia
The element (i, j) of the matrix Scores is 1 if ith team won over jth team. Correspondingly,
if ith team lost to jth team, then the (i, j) element is −1. The diagonal elements are taken to
be 0.
The file Differentials.mat contains the matrix Differentials which is similar to Scores,

only instead of ±1 it has the point differentials of the corresponding games.

TASKS

1. Start by loading both matrices into your Matlab script using the load command:
load(’Scores.mat’);
load(’Differentials.mat’);
Variables: Scores, Differentials
2. First, consider the Colley method. Create the matrix and the right-hand side of Colley’s
system of linear equations:
Colley’s method
games=abs(Scores);
total=sum(games,2);
ColleyMatrix=2.*eye(10)+diag(total)-games;
RightSide=(1+0.5.*sum(Scores,2));
Variables: ColleyMatrix, RightSide
Q1: What do the commands eye and diag do?

3. Solve the linear system using the backslash operator:

RanksColley=ColleyMatrix\RightSide;
Variables: RanksColley

4. Display the results on the screen:

Teams={’Baylor’, ’Iowa State’, ’University of Kansas’, ’Kansas State’,...
’University of Oklahoma’, ’Oklahoma State’, ’Texas Christian’,...
’University of Texas Austin’, ’Texas Tech’, ’West Virginia’};
[RanksDescend,Order]=sort(RanksColley,’descend’);
fprintf(’\n’);
for j=1:10
fprintf(’ 8.3f -15s\n’, RanksColley(Order(j)), Teams{Order(j)})
end;

38
 Variables: Teams, Order,
Q2: Observe that the array Teams is in curly parentheses. What kind of array is
Q3: Explain what kind of formatting does the line ’%8.3f % − 15s \ n’ in the
fprintf
Q4: What does sort function
5. Now let us use Massey’s method to rank the teams. Create the matrix P and the right-
hand-side B by using the following code:
Massey’s method
l=0;
for j=1:9
for k=j+1:10
if Differentials(j,k)~=0
l=l+1;
P(l,j)=1;
P(l,k)=-1;
B(l)=Differentials(j,k);
end;
end;end;

This code scans the part of the matrix Differentials which is located above the main
diagonal. For each nonzero (i, j) (meaning that the teams i and j played a game)
element of this matrix it creates a row of the matrix P which contains only two non-
zero elements: 1 in the ith column of this row and −1 in the jth column. The vector B
of the right-hand- side is created simultaneously and has the corresponding point
differential in the game between ith and jth team.
Variables: P, B

6. Next, create the normal system of linear equations:

A=P’*P;
D=P’*B’;
Variables: A, D

7. Substitute the last row of the matrix as described above:

A(10,:)=[1
D(10)=0; 1 1 1 1 1 1 1 1 1];

8. Finally, solve the system:

RanksMassey=A\D;
Variables: RanksMassey

9. Display the results on the screen as before:

[RanksDescend,Order]=sort(RanksMassey,’descend’);
fprintf(’\n’);
for j=1:10
fprintf(’ 8.3f -15s\n’, RanksMassey(Order(j)), Teams{Order(j)})
end;

3
 Variables: RanksDescend, Order
10. Compare the results of both Colley’s and Massey’s methods. Compare both the methods
with the official results. Observe that while these two methods are used by BCS, they
do not take into account many things, such as the timing of the games, strength of
schedule, games played against opponents which are not in the Big 12, and so on. These
factors may affect the rankings and make it deviate from the official. For the data
considered in this project the comparison with the official results should be very good.
11. Create a new code cell. Switch the result of the game between the top two teams in
Colley’s ranking and run the Colley’s algorithm again. Produce the rankings of the
teams in this case.
Q5: Did the ranking of the top teams change?
12. Create a new code cell. Now, switch the result of the game between two top teams in
Massey’s ranking (those might be different teams from Colley’s algorithm!) and run the
Massey’s algorithm again. Produce the rankings of the teams in this case.

Q6: Did the ranking of the top two teams change?

3
Project 6: Convolution, inner product, and image processing
re- visited
Goal: In this project we will look at the convolution of two matrices and its application to
image processing.
To get started:

• Create a new Matlab script file and save it as lab06.m

• Download the file einstein.jpg and put it in your working directory.

Matlab commands used: load, rand, double, uint8, conv2, filter2

What you have to submit: The file lab06.m which you will create during the lab session.
INTRODUCTION

In this section we will look at the applications of matrix multiplication to the filtering of
images. The most common operations in filtering are bluring/smoothing, sharpening, and
edge detection. All of these are examples of so-called convolution filters. These filters work
by changing theatcolor
Let us look how of the pixels
filters of thisintype
the work.
image Consider
using thean
colors
imageof described
nearby pixels.
by the matrix
f = (fij) where (i, j) are integer coordinates of the pixel in an image, and the value fij
returns the color of the image. Consider h = (hkl) to be a convolution matrix (filter
matrix). These matrices might have (and usually do have) different dimensions. The
filter matrix is usually relatively small. Then a new matrix which is a convolution of f
and h (written as g = f × h) can be found by the following formula:

Σ Σ
gij = k l fklhi−k+1,j−l+1,
where the summation runs over all the indices which make sense. In particular, if an element
with the certain index does not exist, we simply assume the corresponding term to be zero.
You can think of convolution being obtained by the filter matrix h sliding over the original
image f . Notice that the formula for convolution resembles the definition of an inner product
if we treat both matrices as vectors (say, with columnwise orientation) and flip the second
both horizontally and vertically. matrix
Matlab has two functions to compute convolutions of matrices: conv2 and filter2.
These two functions are closely related. In particular, the function filter2 is equivalent to
the function conv2 with the matrix h rotated 180 degrees.
TASKS

1. To begin, let us load the file einstein.jpg and save the resulting matrix as ImJPG. Use
the size function to find the dimensions m, n of the matrix ImJPG.
Variables: ImJPG, m, n

2. Introduce some noise into the image by adding random fluctuations of color to each point:
ImJPG_Noisy=double(ImJPG)+50*(rand(m,n)-0.5);

4
 Observe that the command rand(m,n) produces a matrix of the dimension m × n filled
with pseudorandom numbers within the interval (0, 1). The amplitude of the noise is
equal to ±25 shades of gray. Also, notice the function double which converts the
variables from the type uint8 to the type double.
3. Two of the most common operations on images done with convolution filters include
smoothing and sharpening. Let us start by using two average smoothing filters given
by the matrices:
   
0 1 0 1 1 1
Kernel Average1 = 1 1 1 1  , Kernel Average2 = 1 1 1 1 
5 9
0 10 1 11
4. Using the appropriate Matlab syntax, type in the matrices Kernel Average1 and

Kernel Average2.
Variables: Kernel Average1, Kernel Average2
Q1: Why are the fractions 1/5 and 1/9 necessary?

5. Apply the filters to the noisy image by using the commands

ImJPG_Average1=conv2(Kernel_Average1,ImJPG_Noisy);
ImJPG_Average2=conv2(Kernel_Average2,ImJPG_Noisy);

Display
forget tothe resulting
convert images back
the results in separate
to the figure
integerwindows and
format by observe
using thefunction.
uint8 result. Don’t

Variables: ImJPG Average1, ImJPG Average2

6. An alternative blurring filter, Gaussian blur, is given by the matrix:

Q1: What is the size of the matrices ImJPG Average1,  Average2?
ImJPG
0 1 0
Q2: What is the size of the original 1
matrix ImJPG? Use the size function.

Gauss = 1 4 1Kernel 0 10
Q3: Which filter blurs more?
which assigns a higher weight to the8 pixel color in the center. Type in the matrix
Kernel Gauss in your Matlab file using the appropriate syntax.
Variables: Kernel Gauss
7. Perform the convolution using the function conv2 and Kernel Gauss and save the
resulting array as ImJPG Gauss. Display the result in a new window.
Variables: ImJPG Gauss
8. Observe that we can “layer” filter effects. Perform another convolution with the
Gaussian kernel on the image ImJPG Gauss, save the result as ImJPG Gauss2, and
display the result in a new window.

Q4: Devise ImJPG

Variables: a matrix for a filter which is equivalent to applying Gaussian convolution
Gauss2

Q5: What is the size of the matrix for this filter?

twice.

4
 9. It is also possible to blur the image over a larger area:
 
0 1 2 1 0
1 4 8 4 1
Kernel Large = 1  2 8 16 8 2 
80
1 4 8 4 1 

0 1 2 1 0
Apply the kernel Kernel Large to the matrix ImJPG, save the result as ImJPG Large,
and display the figure in a new figure window.
Q6: Compare
Variables: with
ImJPG applying Gaussian blur applied twice. Which one blurs more?
Large
10. An opposite action to blurring is the sharpening of an image with a convolution filter.

Type the following kernels into your Matlab file:

 
−1 −1 −1
Kernel Sharp1 =−1
 5 −1, 
0 −1 0
0 −10
 −1 9
Variables: Kernel Sharp1, Kernel Sharp2 −1 −1 −1


11. Perform the convolution of the original image ImJPG with the kernels Kernel Sharp1,
Kernel Sharp2 using the function conv2 and save the resulting arrays as ImJPG
Sharp1 and ImJPG Sharp2. Display the results in new figure windows.
12. Finally, it is possible to use convolution to detect edges in the image. Edge detection
is used for ImJPG
Variables: image Sharp1,
segmentation
ImJPG and data extraction in areas such as image processing,
Sharp2
computer vision, and machine vision. Two of the most common filters used for these are
the Sobel horizontal and vertical filters:
   
−1 0 1 −1 −2 −1
Kernel Sobel1 = −2 0 2 , Kernel Sobel2 =  0 0 0 .
−1 0 1 1 2 1
Sobel filters can be interpreted as discrete derivatives in the horizontal and vertical direc-

tions. TypeKernel
Variables: Sobel1, Kernel
in the matrices Kernel Sobel2
Sobel1, Kernel Sobel2.
13. Perform the convolution of the original image ImJPG with the Sobel kernels using the func-
tion conv2 and save the resulting arrays as ImJPG Sobel1 and ImJPG Sobel2. Display
the results in new figure windows.
14. Create a combined
Variables: image with
ImJPG Sobel1, ImJPGboth horizontal and vertical edges by summing up the
Sobel2
matrices ImJPG Sobel1, ImJPG Sobel2. Display the result in a new figure window by
using the code:
figure;
imshow(uint8(ImJPG_Sobel1+ImJPG_Sobel2));

4
15. Alternatively, Laplacian edge detection can be used with the following filter:
 
0 −1 0
Kernel Laplace = −1 4 −1 .
0 −1 0
Type in the matrixKernel Laplace. The laplace kernel is a discrete analogue of
continuous Laplacian and may be interpreted as a sum of two discrete partial
derivatives of the second order.
16. Perform the convolution of the original image ImJPG with the Laplace kernel using the
function conv2 and save the resulting array as ImJPG Laplace. Display the results in a
Variables:
new window.Kernel Laplace

Variables: ImJPG Laplace

4
Project 7: Norms, angles, and your movie choices

Goals: Use a norm and an inner product to detect similarity between users’ choices and to
create a simple recommender system.
To get started:

• Open a new Matlab script and save it as lab07.m.

4
• Download the file
movies, users userssort,
movies movies.mat whichtrial
index small, contains the and
user arrays movies,
place users
it in your working
directory.
Matlab commands used: load, size, for... end, if... end, fprintf, length,
prod, norm

What you have to submit: The file lab07.m which you will create during the lab session.
INTRODUCTION

Being able to predict users’ choices is big business. Many internet services are studying
consumer choices and preferences to be able to offer you the products you might like (and
correspondingly the products you are more likely to buy). Netflix, Pandora Internet radio,
Spotify, Facebook targeted ads, Amazon, and even online dating websites are all examples of
recommender systems. These services are sometimes willing to go to great lengths in order to
improve their algorithms. In fact, in 2006, Netflix offered 1 million dollars to the team which
would be able
Music to improve
Genome Projecttheir
whichCinematch algorithm
is a trademark predictions
of Pandora by just
Internet 10%.
radio is another example of
a recommender system. If you have ever listened to Pandora, you might have been amazed
how uncannily spot-on their next song recommendation is (at least, the author of this project
was certainly impressed by it). Every song is assigned a rating from 0 to 5 in 450 different
categories (“genes”). Thus, any song is represented by a vector with 450 components.
Mathematical comparison of these vectors allows the algorithm to suggest the next song to a
user.
Observe that data about movies, songs, consumer products, etc, can be often arranged
in a form of a vector. These vector representations can then be used in different algorithms
to compare items and predict similarity. In this project, we will use linear algebra to establish
similarities in tastes between different users. The main idea is that if we know the current
ratings from a certain user, then by comparing them with the ratings of other users in a
database, we can find users with similar tastes. Finally, we can look at the items highly rated
by those
In thisusers
lab, and suggest
we will look these
at theitems to the current
MovieLens databaseuser.
[10] which contains around 1 million
ratings of 3952 movies by 6040 users. We will create a very simple recommender system by
finding similarities between users’ tastes. Our techniques will be based on using the Euclidean
distance (norm), the standard scalar product, and finding correlation coefficients (or angles
between the vectors of data).
4
The original data is available at https://grouplens.org/datasets/movielens/.

4
 TASKS

1. Load the arrays movies, users movies, users movies sort, index small, trial user
from the file users movies.mat using the load command. The matrix users movies
should be a 6040 3952 matrix containing integer values between 0 and 5 with 1
meaning “strongly dislike” and 5 meaning “strongly like”. The 0 in the matrix means
that the user did ×not rate the movie. The array movies contains all the titles of the
movies. The matrix users movies sort contains an extract from the matrix users
movies.mat with rating for the 20 most popular movies selected. The indexes of these
popular movies are recorded in the array index small. Finally, ratings of these popular
movies by yet another user (not any of the users contained in the database) are given
by the vector trial user. It is suggested to view all the variables and their dimensions
by using the “Workspace” window of the Matlab environment:
Load the data
clear;
load(’users_movies.mat’,’movies’,’users_movies’,’users_movies_sort’,...
’index_small’,’trial_user’);
[m,n]=size(users_movies);

Variables: movies, users movies, users movies sort, index small, trial user, m, n

2. Print the titles of the 20 most popular movies by using the following code:
fprintf(’Rating is based on movies:\n’)
for j=1:length(index_small)
fprintf(’ s \n’,movies{index_small(j)})
end; fprintf(’\
n’)

Observe that by
parentheses) theusing
movieantitles are calledarray
intermediate fromindex array movies (notice the curly
the cellsmall.

3. Now let us select the users we will compare the trial user to. Here, we want to select
the people who rated all of the 20 movies under consideration. This means that there
should not be any zeros in the corresponding rows of the matrix users movies sort.
This can be accomplished by the following code:
Select the users to compare to
[m1,n1]=size(users_movies_sort);
ratings=[];
for j=1:m1
if prod(users_movies_sort(j,:))~=0
ratings=[ratings; users_movies_sort(j,:)];
end;end;

Observe thearray
rows of the use ofusers
the command prod The
movies sort. to compute the product
array ratings of the
contains elements
all the users of the
which
have rated all the popular movies from the array index small. In general, it may
happen that this array is empty or too small to create any meaningful comparison - we
won’t consider these cases in this project.
Variables: ratings, m1, n1

Q1: What does the command ratings=[] do?

4
4. Next, we can look at the similarity metric based on the Euclidean distance. The idea
here is that we treat the array trial20 user and the rows of the array ratings as
vectors in 20- dimensional real space R . Assume that all the vectors have the origin as
the beginning point. We can find the distance between the end points of the vector
trial user and each of the vectors ratings(j,:). In other words, we are looking for
the user with the closest ratings to the trial user. This can be accomplished by the
following code:
Find the Euclidean distances
[m2,n2]=size(ratings);
for i=1:m2
end;eucl(i)=norm(ratings(i,:)-trial_user);

The of the eucl

rows vector matrixcontains all the Euclidean distances between trial user and the
ratings.
Variables: eucl
5. Now let us select the user from the database with the smallest Euclidean distance from
the trial user. Instead of using the usual function min we will use a slightly more
com- plicated approach. Let us sort the elements of the vector eucl by using the
function sort. The advantage of this is that it allows us to find the second closest user,
the third closest user, etc. There may only be a small difference between the several
closest users and we might want to use their data as well.
[MinDist,DistIndex]=sort(eucl,’ascend’);
closest_user_Dist=DistIndex(1)

Variables: MinDist, DistIndex, closest user Dist

6. The similarity metric above is one of the simplest ones which can be used to compare
two objects. However, when it comes to user ratings it has certain disadvantages. For
instance, what if the users have similar tastes, but one of them consistently judges
movies harsher than the other one? The metric above would rate those two users as
dissimilar since the Euclidean distance between vectors of their opinions might be
pretty large. To rectify this problem, we can look at a different similarity metric known
in statistics as the Pearson correlation coefficient which can be defined as
N
Σ
i (xi − x¯)(
. yi − y¯ )
ΣN ,
r(x, y) = .Σ i=1 (xi − i=1 (yi −
where x = (x1, x2, . . . , xN ), y = (y1, Ny2, . . . , yN ) are two vectors with N components, and
ΣN ΣN
x¯ = 1 xi , y¯ = 1 yi are the average (mean) values of the vectors x and y. If we
N the
look at i
formula for i
N r(x, y), we can see that it accounts for the average user opinion
x¯, y¯, and also for how harshly/enthusiastically they tend to judge their movies
(defined by the magnitude of the vectors x x¯ and y y¯, where x¯ and y¯ denote
the vectors of the same size as x and y with all the components equal to x¯ and y¯
−
correspondingly). Geometrically, the correlation coefficient r(x, y) corresponds to the
cosine of the angle between the vectors x x¯ and y y¯ (the closer this angle is to
zero, the more similar are the opinions of two people). To compute the Pearson
correlation coefficient, let us centralize the columns of the matrix ratings and the
vector trial user first:
−

4
 ratings_cent=ratings-mean(ratings,2)*ones(1,n2);
trial_user_cent=trial_user-mean(trial_user);

Variables: ratings cent, trial user cent

7. Next, use the for... end loop to compute the Pearson correlation coefficients between
the rows of the matrix ratings and the vector trial user. Save the result as a vector
pearson.
Variables: pearson
8. Finally, observe that the value r(x, y) belongs to the interval (−1, 1). The closer the
coefficient is to 1, the more similar the tastes of the users are. Finally, let us sort the
vector pearson as before using the sort function. Save the results of this function as
[MaxPearson,PearsonIndex] and find the maximal correlation coefficient which will
be the first element in the matrix MaxPearson. Save this element as closest user
Pearson.

Q2: Are the

Variables: variables closest
MaxPearson, user Pearson
PearsonIndex, closestand closest
user user Dist the same?
Pearson
9. Compare the elements of the vectors DistIndex, PearsonIndex.

10. Now let us display the recommendations on the screen. Use the following list to create
the list of movies which the trial user has liked and the lists of recommendations for
him/her based on the distance criterion and the Pearson correlation coefficient
criterion:
Recommendations
recommend_dist=[];
for k=1:n
if
(users_movies(closest_user_Dist,k)=
=5) recommend_dist=[recommend_dist;
k];
end;
end;
recommend_Pearson=[];
for k=1:n
if (users_movies(closest_user_Pearson,k)==5)
recommend_Pearson=[recommend_Pearson; k];
end;
end;
liked=[];
for k=1:20
if (trial_user(k)==5)
liked=[liked; index_small(k)];
end;end;

We use the rating equal to 5 both as the criterion for liking the movie and the criterion
to recommend the movie. Of course, you can broaden this up and also include the
movies ranked as 4.
11. Finally, display the titles of the movies from the arrays liked, recommend dist,
recommend Pearson on the screen by using the procedure similar to the one used in
the step 2. liked, recommend dist, recommend Pearson
Variables:

4
12. Take a look at the list of the popular movies displayed in step 2. Chances are you might
have seen the majority of them before. Create your own vector of ratings and call it
myratings. Ratings should be integers between 1 and 5. Again, assign the rating 1 if
you really disliked the movie, and 5 if you really liked it. If you haven’t seen a
particular movie, pick its rating at random. The vector of ratings myratings should be
a row-vector with 20 elements.
Variables: myratings
13. Create a new code cell. In this cell, repeat steps 4-11 of this project and substitute the
vector trial user by the vector myratings. This should produce a personal
recommen- dation based on your own ratings.

Variables: liked, recommend dist, recommend Pearson

4
Project 8: Interpolation, extrapolation, and climate change

Goals: To explore applications of linear algebra for the interpolation of data and prediction
of future tendencies.
To get started:

• Create a new Matlab script and save it as lab08.m.

• Download 5the file temperature.mat which contains average temperatures for the past
136 years . Temperature is given in Celsius.
Matlab commands used: plot, axis, vander, interp1, orth

What you have to submit: The file lab08.m, which you will create during the lab session.
INTRODUCTION

In many practical applications it is necessary to obtain an intermediate value of a

parameter from a few known data points. If this intermediate value lies in between two
known measure- ments, we will call this process interpolation. If the value is outside of the
sampled interval, then we will call this process extrapolation. In this project we will look at
several procedures for interpolation and extrapolation and apply them to weather and
climate change prediction. The most common reason for using interpolation is not having
enough of data points. These data points can be obtained from experimental observations
which can be expensive or difficult to perform, or from past observations where the data is
simply unavailable. Interpolation is also used when the function which produces the data is
known but has a complicated struc- ture. In this case it can be computationally prohibitive
to use this function, especially if, in the course of computation, this function needs to be
called multiple times. For instance, the function can be given by a slowly convergent series
which requires computing many terms to obtain adequate accuracy. Approximating this
complicated function by a simpler function may be helpful. Interpolation of a function can be
sufficiently accurate provided that the function is smooth and there are enough data points
available. Some of the usual types of interpolation include polynomial interpolation when the
function is approximated by a polynomial, trigono- metric approximation with a truncated
More intricate interpolation methods canFourier
be usedseries, or various
if additional piece-wiseis approximations.
information available about
the approximated function.
Extrapolation, on the other hand, is a significantly more difficult procedure. The general
rule is not to extrapolate too far beyond the range of the observed parameters. There are
multiple reasons for this. For instance, some additional parameters may affect the behavior
of a function outside of the observed range of values. Imagine trying to extrapolate physical
laws applicable for regular macro-solids (such as things we use in our daily life) to nano-scale
or to astro-scales. The obtained results will be meaningless since different laws apply at those
scales. Greater uncertainty and errors while extrapolating also appear because small errors in
data measurements can potentially lead to very large errors while extrapolating the data.
These errors are somewhat bounded while interpolating between two known data points.
5
The temperature data has been taken from NASA’s website, https://climate.nasa.gov/vital-signs/
global-temperature/

4
 TASKS

1. To begin, let us look at interpolation based on the polynomial approximation. Assume

that we have n + 1 data points (measurements) (xi, yi), i = 0, . . . , n. To obtain the
values of the function in between these data points we can simply trace a polynomial of
the nth degree through the data points and use this polynomial to estimate the values
of a function in between measurements. Thus, the goal is to find a polynomial
p(x) = anxn + an−1xn−1 + . . . a1x + a0
with unknown coefficients ai, i = 0, . . . , n, which passes through the points (xi, yi). Then
the coefficients ai must satisfy the following system of linear algebraic equations

anxn i+ an−1xn−i1 + . . . a1xi + a0 = 0, i = 0, . . . , n.

We can write this system in the form

Vz = y,
where V is the so-called Vandermonde matrix:

1 x0 ... xn−1 n 
01 x0n
1 x1 ... x n−
V= 1 x1  ,

 ... 1... xn... ... ...
xnn−1 ...xn
z = [a0, a1, . . . , an]T is the unknown vector of polynomial coefficients and y = [y0, y1, . . . , yn]T
is the known vector of the function values at sample points.

We will start by approximating the data for the average temperatures in Kansas6:
Month 1 2 3 4 5 6 7 8 9 10 11 12
Temperature (High) 37 44 55 66 75 84 89 88 80 69 53 41
In our computations, we will work with high temperatures only. To begin, let us generate
a plot of the temperatures
Temperature (Low) in 17
Matlab.
23 33 43 53 63 68 65 56 44 32 22
clear;
WeatherHigh=[37 44 55 66 75 84 89 88 80 69 53 41]’;
figure;
plot(1:12,WeatherHigh,’r-x’);
axis([1,12 30 95]);
Observe that Matlab will automatically join the available data points by straight lines.
Variables: WeatherHigh
2. Assume for now, that we are only given the temperatures for four months: January, May,
August, and December. Select the corresponding data from the variable WeatherHigh
above and generate the Vandermonde matrix based on this data. Now solve the
resulting system using Matlab’s backslash linear system solver:
6
Source for the temperature data: Wikipedia https://en.wikipedia.org/wiki/Kansas#Climate,
temperature is given in Fahrenheit.

5
 x=[1,5,8,12]; Numbers for January, May, August, and December
V=vander(x);
CoefHigh=V\WeatherHigh(x);

The resulting vector CoefHigh contains coefficients of the cubic polynomial

approximating the average high annual temperature in Kansas.
Variables: V, CoefHigh
3. Matlab’s polyval function can be used to efficiently evaluate a polynomial at the given
set of points. It takes two vectors as arguments. The first vector represents the
coefficients of the polynomial and the second vector represents the points we want to
evaluate the polynomial at. Remember that you can always type help polyval or doc
polyval in the Matlab command window to see more options. Let us plot the graph of
the polynomial obtained at the previous step by using the following code:
xc=1:0.1:12;
ycHigh=polyval(CoefHigh,xc);
plot(xc,ycHigh,’b-’,1:12,WeatherHigh,’r-x’);
axis([1,12 30 95]);

This graph
Observe shows
that the difference
the command between the
axis([1,12 approximated
30 95]) sets the values and the
parameters for exact
x and values.
y axes.
Namely, the x axis will range from x = 1 to x = 12 and the y axis will range from y = 30
to y = 95.
Variables: xc, ycHigh
4. Repeat the steps above to produce a polynomial approximation based on the
temperatures from the following six months: January, March, May, August, October,
and December. You will need to use a polynomial of the fifth degree. Plot the resulting
polynomial next to the exact values as in the step above. You will see that this
approximation produces much closer results to the exact values.
5. Finally, let us use the temperatures from all twelve months. You will need to use a
polynomial of the eleventh degree this time. Modify the code above and run it in
Matlab. If you did everything correctly, you will obtain a picture which looks similar to
the one in Fig. 7. You may notice that this approximation is not necessarily better
than the

Figure 7: Approximation with the 11th degree polynomial

5
 approximation with the 5th degree polynomial due to large fluctuations near the end
points. This is typical for approximations with polynomials of a high degree on evenly
spaced grids which tend to oscillate widely near the end points. This is one of the
reasons why polynomials of a high degree are normally not used for interpolation on
evenly spaced grids 7. Additionally, if you look at the Matlab output in the command
window, you will probably see something like this
Warning: Matrix is close to singular or badly scaled. Results may be
inaccurate. RCOND = 8.296438e-17.

This is due to the fact that Vandermonde matrix V is an ill-conditioned matrix,

meaning that the system of linear equations with this matrix is difficult to solve with
high accuracy. The matrix V becomes more ill-conditioned as the dimensions of the
matrix increase. This is another reason why high degree polynomial approximations on
evenly spaced grids are rarely used in interpolation.
6. If polynomials of a high degree on evenly spaced grids are not good for interpolation,
what can be done instead? The most common method of interpolation is through the
approximation of functions with piece-wise (typically polynomial) functions. This means
that the function between data points is approximated by a small degree polynomial
(polynomials of no higher than third degree are usually used). These polynomials are
different on each of the intervals between the data points. Individual approximations
are then glued together to achieve a desired degree of smoothness at the data points.
Under smoothness here we understand the continuity of a function and its derivatives
up to the some order. Smoothness of the resulting approximation depends on the degree
of polynomials used and how the polynomial pieces are glued together. If a higher
degree of smoothness is desired, it is necessary to use polynomials of a higher degree.
For instance, if we use cubic polynomials, then it is possible to achieve continuity of the
function itself and its first two derivatives at the most.
To investigate
interp1. This this method
function of interpolation,
allows we will look
one to interpolate at the built in function.
a one-dimensional Matlab function
It has
several attributes which allow for different types of interpolation: “linear” which simply
joins the data points with straight lines, “cubic” or “pchip” which approximates the
function on each individual segment with the third degree polynomial, and “spline”
which also uses third degree polynomials, but with different junction conditions. So
what is the difference between “pchip” and “spline”? Simply put, the “pchip” method
provides a shape-preserving piece-wise cubic polynomial function which is smooth
together with its first derivative. The “spline” interpolation creates an “extra-smooth”
piece-wise cubic polynomial function which is smooth together with its first and second
derivatives. In general, “pchip” approximations have smaller oscillations near the ends
and are easier to compute.
Compare these options by using the following code:
ycHigh1=interp1(x,WeatherHigh,xc,’linear’);
ycHigh2=interp1(x,WeatherHigh,xc,’pchip’);
ycHigh3=interp1(x,WeatherHigh,xc,’spline’);
7
However, approximations with the polynomials of high degree on non-evenly spaced grids may be
extremely useful in computations and achieve very high precision. Chebyshev nodes and Chebyshev
polynomials represent a good example of this phenomena.

5
 Here the of
a vector variables x and we
points where WeatherHigh give the vectors
want to approximate of initial
the function anddata. The vector
the vectors xc is
ycHigh1,
ycHigh2, and ycHigh3 are the values at the points xc resulting from the interpolation.
Plot the vectors ycHigh1, ycHigh2, and ycHigh3 versus xc on the same plot to
observe the smoothness of the resulting data. You can see that ycHigh2 and ycHigh3
achieve better approximations than the polynomial approximation. The function
interp1 also has a possibility to extrapolate the results which we will explore shortly.
Variables: ycHigh1, ycHigh2, ycHigh3
7. Now let us look at the extrapolation of the data. Extrapolation means finding an ap-
proximation to the data outside of the initial measurement interval. For our data, we
will take the average global temperatures from the last 136 years starting from the year
18808. To begin, load the matrix temperature from the file “temperature.mat” using
the load command.
Variables: temperature
8. The matrix temperature consists of two columns. The first column represents the years
and the second the average temperatures at those years. Separate the data into two
variables:
years=temperature(:,1);
temp=temperature(:,2);

Plot the graph of temperatures using the plot function. This will produce a graph of
fluctuations of average temperatures since 1880. If you did everything correctly, you
should obtain a graph similar to the one on the Fig. 8.

Variables: years, temp

Figure 8: Average yearly temperatures.

9. Let us attempt to extrapolate the data in the future. Assume that we want to know
the temperature about 10 years into the future (again, it is advised not to attempt to
extrapolate too far beyond the initial interval). We can do this using the following
code:
futureyears=2016:2025;
8
The source of data: NASA, https://climate.nasa.gov/vital-signs/global-temperature/

5
 futuretemp1=interp1(years,temp,futureyears,’linear’,’extrap’);
futuretemp2=interp1(years,temp,futureyears,’pchip’,’extrap’);
futuretemp3=interp1(years,temp,futureyears,’spline’,’extrap’);

figure;
plot(years,temp,’b-o’,futureyears,futuretemp1,’g-o’,...
futureyears,futuretemp2,’r-x’,futureyears,futuretemp3,’k-d’);

This produces three graphs based on the different interpolation/extrapolation strategies

which were discussed above. If you did everything correctly, you should obtain three
radically different predictions for the next 10 years!
Variables: futureyears, futuretemp1, futuretemp2, futuretemp3
10. So what happened in the previous example? The local fluctuations in temperature
signifi- cantly affected the global predictions for the future. However, we are not
interested in the local fluctuations but rather in global tendencies. So let us try to
separate those two from each other and extrapolate based on global tendencies only. We
can think of temperatures in the following terms:
ti = t(yeari) + noise,
where t(year) would be the global tendency of temperature and noise represents
relatively small year-to-year fluctuations.
We will use orthogonal projections to separate the global behavior from the “noise”. As
a first approximation, let us consider how we can assess the overall average
temperature. That is, we approximate the temperature t(years) as a constant:
1Σ
t(years) ≈ t.
n
Enter the following command in the command window
>> sum(temp)/n i
n
i
This allows us to see that the average temperature for the past 136 years was 0.0244C.
11.Here is another way of thinking of the same operation using orthogonal projections. If
the
data was truly flat, then there would be a real number c so that
   
t1  1 
 =c 1 = c1.
t2
t=
.  . 

t  1 
n
That is, the average temperature vector t would be a scalar times the vector 1 of all ones.
Since the data is not flat, we can’t find such a c. However, we can do the next best
thing. We can find c to minimize the norm ǁt − c1ǁ. The best choice for c is
determined by the orthogonal projection on a vector 1:
1T t 1T t

c= and for this choice of c, c1 = 1 = proj1(t).

ǁ1ǁ2 ǁ1ǁ2

5
 Figure 9: Approximation of data by a constant function.

Moreover, a little rearranging shows that

. Σ
1 1
proj (t) = 11T t = Pt where P = 11T is the projection matrix.
2
1 ǁ1ǁ ǁ1ǁ2
Define the variable
that, define b1 toP1betothe
the variable length(temp)-dimensional
hold 1 vector.
the projection√ matrix. You In thewant
will either space
to after
use
Matlab’s norm function,
= to be the value P1*temp. or remember that ǁuǁ

Variables: P1, temp1 uT u. Also, define the variable temp1

12. Plot the resulting vector using the code:

plot(years,temp,’bo’,years,temp1,’g.’);
hold on;

You should see a similar plot as before, but now with a set of green dots representing the
projection (Fig. 9).
13. Now, examine the value of temp1 in the Matlab command window.

Q1: How does the value of temp1 relate to the average value of temperature?
14. Like vectors, matrices also have norms. The norm A of a matrix A describes how far
ǁ
A is from the zero matrix in a certain sense. Enter the following command into Matlab
norm(P1*P1-P1)
Your answer should be something like ans = 2.0134e-15.
Q2: What does the result say about the relationship between the projection matrix P
and its square P2?
15. While the previous approach of approximating the data with a constant function may be
acceptable in some cases, it is clearly not a perfect match. Let us try instead to model
the data as a linear function of yearsi:

ti = b yearsi + c + noise.

5
 If this formula was exact (with no noise), we would have that
 
  years1 1
t21  Σ Σ = byears + c1,
t= =
 years2 1
t
b
   
    c
. . .
so t would be a vector in the subspace span{years,
1 1}. However, as before, this is not
exactly true. So, we will need to use years
an orthogonal
n projection again.
There is a nice formula for projecting onto a subspace S generated by a span of several
vectors span{b1, b2, · · · ,Tbk}, but it requires that the vectors be orthonormal. That is
ǁbiǁ = 1 for each i, and b bj = 0 for all i ƒ= j.
One way to produce an orthonormal basis in Matlab is to use the Matlab orth function.
i
Please, type in the code
m=length(years);
B2=[ones(m,1) years];
Q2=orth(B2);

This creates a matrix B = [1 years], whose first column consists entirely of ones and
whose second column is the vector years, which provides the years of observation. The
new array Q2 is defined to be the result of orth(B2).
The
the subspace orth
function S. To will
checkcreate
that an
theorthonormal
columns of basis (defined
Q2 form by verify
a basis, the columns
in the of Q2) for
command
window that the values of rank(Q2) and rank([Q2,B2]) are both equal to 2.
Variables: Q2

Q3: What kind of matrix is the matrix Q2T Q2? Why?

16. Given an orthonormal basis defined by the columns of Q2 for the subspace S, the
formula for the projection matrix onto the subspace S is simple:
P2 = Q2Q2T .

Create a new cell and in this cell create a variable P2 to hold the value of the matrix
of the orthogonal projection onto the subspace . Repeat the previous steps to produce
the projection vector temp2=P2*temp and plot S this vector in the previous figure
window. This will produce the linear approximation to the given data. Run the
following code in Matlab:
norm(P2*P2-P2}

You shouldexplanation
geometric get a number
for which is close
this fact. Thetomatrices
zero again,
P1, such 5.6169e-16.
as, the
P2 are projectionThere is a
matrices,
and projecting a vector on a subspace produces a vector belonging to this subspace.
Taking a second projection, thus, will simply leave the vector as is. The projection
matrix behaves as an identity matrices on the corresponding subspace if the basis
consisting of the columns of the matrix Q2 is selected.

Variables: P2, temp2

5
 Figure 10: Approximation of data by a quadratic function.
17. Finally, let us approximate the data by using a quadratic function. Create a new cell
and reproduce the code from from the previous cell but with renamed variables (e.g., B3
instead of B2). Make sure to copy the plotting code and switch P2 to P3.
Aside
columnfrom changing
to the 2s toThis
B3 matrix. 3s, the only should
column other change you’ll need
be the vector to make
of squares is to
of the add a
elements
of the vector years, which can be produced in Matlab with the expression years.^2
(notice the dot before the power sign!).
Variables: B3, Q3, P3
18. Plot the initial data and all the approximations above on a new plot (use the command
figure). If correct, your code should produce a figure similar to the one on the Fig. 10.
Observe that the quadratic function (shown by the magenta line) appears to follow the
global tendencies of the temperature graph pretty well.

19. Now, let us make predictions for the future. Create a new cell and run the following code:
futureyears=2016:2116;
futuretemp3=interp1(years,temp3,futureyears,’spline’,’extrap’);
plot(futureyears,futuretemp3,’g-’);

This should create our prediction for an average temperature for the next 100 years. If
you did everything correctly, provided the current tendencies hold, by the year 2116 the
average temperature should be about 3.5C. Note that climate prediction is notoriously
difficult, and it is very likely that there are factors which influence the data which are
not accurately picked up by our simple model. Also, please note that in the past,
temperatures tended to rise faster than any prediction. At the same time, NASA
reports that the currently available models for climate change predict rise in
temperatures anywhere between a 2C and 6C, so our simple model predictions are right
in the middle!

5
Project 9: Orthogonal matrices and 3D graphics

Goals: To explore applications of orthogonal matrices to rotation of 3D objects.

To get started: Download the files lab09.m910, v.mat, and f.mat. The last two files contain
corresponding vertices and faces of a 3D model . Save these files in your working directory.
Matlab commands used: function, zeros, sin, cos, if... end, for... end, line,
figure, hold on, hold off, bucky, size, load
What you have to submit: The files rotation.m and lab09.m which you will create
during the lab session.
INTRODUCTION

Recall that the square matrix Q is called orthogonal if QT Q = QQT = I, where I is an

identity matrix of the same size as Q. In other words, this means that QT = Q−1. Rows
(columns) of an orthogonal matrix generate an orthonormal set of vectors. Namely, if Q =
[q1, q2, . . . , qn], where qj are the columns of the matrix Q, then qj · qk = δjk, and δjk denotes
the Another
Kronecker symbol:ofδorthogonal
property jk = 1, if j = k, and δis
matrices jk = 0, if
that x j· ƒ=
y =k.Qx · Qy for any two vectors x and y,
meaning that multiplication by an orthogonal matrix Q does not change the scalar product
of two vectors. Geometrically, this means that a linear transformation T (x) = Qx of the
space Rn is equivalent to the rotation of the space together with a possible reflection along
some hyperplane. Whether a reflection of the space is involved can be seen by looking at the
determinant of the matrix Q. If detQ = 1, then the transformation is a pure rotation. If
detQ = −1, then
Orthogonal the transformation
matrices is 3D
can be used in a rotation together
graphics. Let us with a reflection.
consider a three-dimensional object
which we want to show on a computer screen. The obvious difficulty is that the object is
three- dimensional and the screen is only two-dimensional. Thus, we need to generate a
projection to show the object on the screen. One of the easiest ways to do this is to simply
drop one of the coordinates. However, that allows us to view the object only from three
different angles which can make it difficult to understand the real shape of the object. Using
orthogonal matrices allows us to look at the object from many different angles. To accomplish
this, we need three matrices which represent rotations around each of the coordinate axes Ox,
Oy, and Oz:
     
1 0 0 cos θy 0 − sin θy cos θz − sin θz 0
Rx = 0 cos θx − sin 0 1 0 sin θz cos θz 0 .
θx  
 , Ry =  sin θy 0 cos θy  , Rz =  0 0 1
0 sin θx cos θx
Here θx, θy, and θz are the angles of rotation around the axes 0x, 0y, and 0z
correspondingly in the counterclockwise direction. Multiplying these matrices will result in a
combined rotation around all three axes:
R = RxRyRz.
In this project we will use the matrix R to create a two-dimensional projection of a three-
dimensional object.
9
The printout of the file lab09.m can be found in appendices of this book.
10
The original data for this 3D model can be found here: https://www.thingiverse.com/thing:906692. The
model is licensed under Creative Commons Attribution Share Alike license.

5
 In computer graphics, most objects are not actually smooth, but rather represented by
a polytope. In other words, they are obtained by identifying vertices on the surface of an
object and connecting those vertices with straight lines (edges). Thus, the surface of an
object actually consists of flat faces. In 3D graphics the faces are usually triangular. There is
a simple mathematical reason for this: there always exists a plane passing through any three
points in a space but not necessary through four or more points. If the number of faces is
large,
Twothen the object
things observed
are necessary to from fara away
create appears to becomputer
three-dimensional smooth. model of an object: coor-
dinates of the vertices and the edges showing the connections between the vertices. The
former can be given in a form of an n × 3 array V , where n is the number of vertices in a 3D
model. The latter can be given in several different ways, for instance, by an n × n adjacency
matrix E (hence, a 3D model can be treated as a graph!) or by an m × 3 array containing
numbers of vertices in each face of the model where m is the number of faces. We will look at
bothObserve
of thesethat
descriptions.
this project is intended as a demonstration of techniques used to create a
projection of a three dimensional object on a computer screen. Of course, Matlab has built in
procedures to work with 3D objects. We will explore these procedures at the end of the
project as well.
TASKS

1. First of all, let us create a user defined function which will return a rotation matrix
based on the user-specified angles θx, θy, and θz which correspond to the rotations
around the axes 0x, 0y, and 0z. Create a new Matlab script and save it as rotation.m
in your working directory. Define the header of this function as:
function rotmat=rotation(theta_x,theta_y,theta_z)

This function
rotation takes
matrix in three variables theta x,theta y,theta z and returns the 3 × 3
rotmat.
2. Within theMatlab
Variables:
standard function,
x, create
theta syntax
theta the
andy,the
theta z Rx, Ry, functions
matrices
trigonometric and Rz assin,
defined
cos.above by using
Variables: Rx, Ry, Rz
3. Compute the product of these matrices and assign the result to the output variable rotmat.

Variables: rotmat
4. Save the resulting function in the file rotation.m in your working directory. Now that
the user-defined function is created, we can use it to generate projections.
5. Let us start by creating a projection of a simple object such as a cube. Open the file

m551lab11.m and notice the following code in the first code cell Cube:
Vertices=[1 1 1;...
-1 1 1;...
1 -1 1;...
1 1 -1;...
-1 -1 1;...
-1 1 -1;...

5
 1 -1 -1;...
-1 -1 -1];

Edges=zeros(8,8);
Edges(1,2)=1;
Edges(1,3)=1;
Edges(1,4)=1;
Edges(2,5)=1;
Edges(2,6)=1;
Edges(3,5)=1;
Edges(3,7)=1;
Edges(4,6)=1;
Edges(4,7)=1;
Edges(5,8)=1;
Edges(6,8)=1;
Edges(7,8)=1;
Edges=Edges+Edges’;

This code will create the matrix of the vertices of the cube and the corresponding
adjacency matrix.
Variables: Vertices, Edges
6. Assign the following values to the angles of the rotations θx = π/3, θy = π/4, and θz = π/6.

Call the function rotation to generate the rotation matrix with these angles:
rotmat=rotation(theta1,theta2,theta3);
Variables: rotmat

7. Transform the coordinates of the vertices with the rotation matrix.

VertRot=Vertices*rotmat’;
Variables: VertRot
8. Create a new figure window. Moving through the elements of the matrix Edges, draw a
projection of the cube:
figure;
axis equal;
hold on;
for j=1:8
for k=j+1:8
if (Edges(j,k)==1) line([VertRot(j,1),VertRot(k,1)],
[VertRot(j,2),VertRot(k,2)]);
end;
end;
end;
hold off;

The function line draws a line through two points given by their coordinates. The
points can have two or three coordinates. Observe that to generate the projection, we
dropped the last coordinate of the vector VertRot.

Q1: Why does the second for cycle start with j+1?

6
 9. Modify the code above to obtain the projection generated by dropping the second
coordi- nate of the vector VertRot.
10. Let us consider a more complicated object. Matlab has a built in buckyball ( bucky)
procedure which produces an array containing the vertices of a buckyball and the
adjacency matrix showing how to connect the vertices:
[Edges2,Vertices2] = bucky;

Variables: Edges2, Vertices2

11. Using the same procedure as above, produce a projection of the buckyball onto the
screen. You can use the same rotation matrix rotmat as before. Find the number of
vertices in the array Vertices2 using the size function.

12. To draw a three dimensional buckyball, use the following code:

figure;
axis equal;
hold on;

for j=1:60
for k=j+1:60
if (Edges2(j,k)==1) line([Vertices2(j,1),Vertices2(k,1)],
[Vertices2(j,2),Vertices2(k,2)],... [Vertices2(j,3),Vertices2(k,3)]);
end;
end;
end;
hold off;

Observe that you can use the “3D rotation” button in the top panel of the figure
window to rotate the figure. If you use the same button on the previous figure, you will
notice that the generated projection is flat. If you did everything correctly then you will
see something similar to the Fig. 11.

Q2: Why do we use 60 as the end limit for our for cycles?

Figure 11: Buckyball in Matlab window.

6
13. Finally, this can be applied to any
11 three dimensional construction. To illustrate this
consider a model for 3D printing . Load the matrices v and f into Matlab from the
files v.mat and f.mat using load command. The array v contains the vertices of the
model, and the array f contains the triangular faces of the model. Each row of the face
array f contains the numbers which identify the vertices in the face. The corresponding
coordinates of the vertices can then be found by referring to the array v.
Variables: v, f
14. Determine the size of the matrix f by using size function. Save the dimensions of this
matrix as mFaces, nFaces. The number of rows mFaces in the array f will give you the
number of faces in the model, while the number of columns nFaces will give you the
number of vertices in each face (it should be equal to three).

Variables: mFaces, nFaces .

Figure 12: 3D model of a fawn.

15. To generate the three dimensional model, use the following code:
figure;
axis equal;
hold on;
for j=1:mFaces line([v(f(j,1),1),v(f(j,2),1)],
[v(f(j,1),2),v(f(j,2),2)],... [v(f(j,1),3),v(f(j,2),3)]);
line([v(f(j,1),1),v(f(j,3),1)],[v(f(j,1),2),v(f(j,3),2)],...
[v(f(j,1),3),v(f(j,3),3)]); line([v(f(j,2),1),v(f(j,3),1)],
[v(f(j,2),2),v(f(j,3),2)],... [v(f(j,2),3),v(f(j,3),3)]);
end;
hold off;

If you did everything correctly you will see a figure similar to Fig. 12.
16. Using a similar procedure as above, generate a two dimensional projection of the three
dimensional model. You can use the same rotation matrix rotmat as before. Store the
11
The original data for the model is available from https://www.thingiverse.com/thing:906692/#files

6
 coordinates of the projection in the matrix VertRot.
Variables: VertRot

17. Another way to generate an image like that in Fig. 17 is as follows.

create rotation matrix
rotmat2 = rotation(-pi/3,0,pi/4);
rotate vertices
vRot = v*rotmat2’;
project to xy plane
vPrj = vRot(:,1:2);
draw
figure;
axis equal;
hold on;
for j=1:mFaces line([vPrj(f(j,1),1),vPrj(f(j,2),1)],
[vPrj(f(j,1),2),vPrj(f(j,2),2)]);
line([vPrj(f(j,1),1),vPrj(f(j,3),1)],[vPrj(f(j,1),2),vPrj(f(j,3),2)]);
line([vPrj(f(j,2),1),vPrj(f(j,3),1)],[vPrj(f(j,2),2),vPrj(f(j,3),2)]);
end;
hold off;

Variables: rotmat2, vRot, vPrj

6
Project 10: Discrete dynamical systems, linear
transformations of the plane, and the Chaos Game
2
Goals: To use matrix mappings of the plane R in the form of a Chaos Game in order to
produce fractals.
To get started: Download the Matlab script lab10.m12 and save it in your working directory.
Matlab commands
hold on... used:
hold off, linspace,
cos, sin for... end, rand, randi, plot,

What you have to submit: The file lab10.m which you will modify during the lab session.
INTRODUCTION

Many processes can be described by using dynamical systems. The main idea here is
that the state of a system in the current moment may be derived by knowing the state
of the system at the previous moment(s) of time. For instance, if we know that the
velocity of a car increases with the acceleration a = 2 m/s2 and the initial velocity of the
car is v0 = 15 m/s, then we can easily compute the velocity of the car after t seconds by
using the formula v = at + v0. This is one of the simplest examples of dynamical systems.
Motion of celestial bodies and population dynamics of species are some of the examples
of dynamical systems in nature. If the time t is measured continuously then we are
talking about continuous dynamical systems. If the time is measured discretely (only at
certain points), then we are talking about discrete dynamical systems. For instance, the
population of certain species at any moment of time depends on its population in the
previous years. Since offspring for many species is produced only once a year, we can
look at this population dynamics as a discrete dynamical system with time measured in
years.
In this project we will explore the connection between fractals and matrix (linear) mappings
in the form of the so-called Chaos Game. The Chaos Game is an algorithm first proposed by
M. Barnsley in 1998 in his book “Fractals everywhere”.
The basic idea of the Chaos Game is simple. Pick n points on the plane (say, vertices of a
regular n-gon), and a number r, 0 < r < 1. Pick a random point inside the polygon. Then
repeat the following steps: pick a polygon vertex at random and draw the next point at a
fraction r of the distance between the current point and the chosen vertex of the polygon.
Repeat
Thisthe process
process can with the new
be written in point.
the form of the following linear mapping. Let x = (x1, x2)
be the current point, and let v = (v1, v2) be the selected vertex of the polygon. Then the next
point can be generated by using the linear mapping T :
Σ T11 T12
T (x) = Σ
T21 T22
(x − v) + v.
Here T (x) is the next point. Repeated many times, this process sometimes (but not always)
produces a fractal. The mapping T can change from iteration to iteration or can be picked at
random with different probabilities (we will look at the examples of this type later). In the
tasks below, we will look at some limiting sets of points which can be obtained by this
procedure.
12
The printout of the file lab10.m can be found in the appendices of this book.

6
 TASKS

1. Let us start simple by creating an image of the Sierpinski Triangle using the Chaos
Game. Locate the cell titled Sierpinski Triangle. In this cell, define the vertices of
an equilateral triangle by using the following commands
clear;
t=linspace(0,2*pi,4);
t(4)=[];
v=[cos(t); sin(t)];

This code creates a set of vertices for equilateral triangle. Each vertex is a distance of 1
unit away from the origin and the first vertex is located on the Ox axis.
Variables: t,v
Q1: What does the command t(4)=[] do?

2. Next, define the starting point and the matrix of the linear transformation:
T=[1/2 0; 0 1/2]; rand-0.5];
x(:,1)=[rand-0.5;

The linear
1/2, and thetransformation
starting point with pickedT atisrandom.
the ismatrix
x(:,1) equivalent to multiplying the vector by
Variables: T,x
3. Plot the vertices of the triangle on the plane and select the number of iterations of the
process:
plot(v(1,:),v(2,:),’k*’,x(1,1),x(2,1),’b.’);
axis equal;
hold on;
Num=10000;

All
2 ×the pointsx which
N array wherewill
N isbethe
obtained
numberduring the iterative
of iterations. After process will all
computing be the
stored in the
iterations
we need, we will plot all the points from the array x using the command plot. Observe
that while it is possible to immediately plot the point we obtain at each step, multiple
calls to the plot function take a significant amount of computational time and, thus,
doing so is impractical.
Variables: Num

4. The following code creates the iterated points and plots them:
for j=1:Num
k=randi(3);
x(:,j+1)=T*(x(:,j)-v(:,k))+v(:,k);
end;
plot(x(1,:),x(2,:),’b.’);
hold off;

In this code,
command we We
randi. select an do
then integer number
a linear k from 1 with
transformation to 3 the
at matrix
randomT,bytheusing the
selected
vertex v(:,k), and the point x(:,j) obtained on the previous step. Running this code
with a large number of points may take some time. If you did everything correctly,
you should obtain

6
 Figure 13: The Sierpinski triangle
an image similar to Fig. 13. The resulting picture approximates a fractal called the
Sierpinski triangle.
There is a different procedure to generate this fractal starting with the interior of an
equilateral triangle. At each step we will connect the mid-points of the sides of the
triangle by straight lines. This subdivides the triangle into four triangles with the sides
twice smaller than the original. Now we will throw out the middle triangle. Then we
repeat this process with each of the leftover triangles, and so on. Doing this infinitely
many times will produce the Sierpinski triangle as well.
Variables: x
5. Let us try some variations of the previous procedure. First of all, let us change the ratio
in which the segment is divided. Locate the code cell titled Sierpinski triangle
with a change of ratio and run the previous code with T=[1/3 0; 0 1/3]. Don’t
forget to clear the variables first (use the command clear)!
6. We can also add rotation to each of the iterations. Locate the code cell titled Sierpinski
triangle with rotation. Use the original algorithm for Sierpinsky triangle but change
the matrix T to the following::
theta=pi/18;
T=1/2*[cos(theta) -sin(theta); sin(theta) cos(theta)];

and run the procedure again. Observe that the matrix T is equivalent to two
transforma- tions: the shrinking of the plane by 1/2 and the counterclockwise rotation of
the plane around the origin with the angle of rotation theta.
7. Next, let us look at the fractal generated starting from four vertices of a square. Locate
the code cell titled Square and run the following code:
T=[1/2 0; 0 1/2];
t=linspace(0,2*pi,5);
t(5)=[];
v=[cos(t); sin(t)];

Now repeat the iterative procedure allowing the vertex at each step to be randomly
selected from the four vertices of the square. Generate the points and plot them on a
new figure. Observe that no fractal has been created in this case.

6
 8. Locate the code cell titled Square with different ratio. Change the ratio in which
the segment is divided to 1/3 and run the code again (don’t forget to clear the
variables!) with the modified matrix T:
T=[1/3 0; 0 1/3];

Q2: Does the code generate a fractal this time?

9. Let us try to do more sophisticated things. Locate the code cell titled Square with
vertex preference (the new vertex cannot be the same as the previous)
which contains the following code:
clear;
T=[1/2 0; 0 1/2];
t=linspace(0,2*pi,5);
t(5)=[];
v=exp(1i*t);
x(:,1)=[rand-0.5; rand-0.5];
plot(real(v),imag(v),’k*’,x(1,1),x(2,1),’b.’);
axis equal;
hold on;
Num=5000;
k1=0;
for j=1:Num
k=randi(3);
if (k>=k1)
k=k+1;
end;
w=[real(v(k));imag(v(k))];
x(:,j+1)=T*(x(:,j)-w)+w;
k1=k;
end;
plot(x(1,:),x(2,:),’b.’);
hold off;

This code repeats the process for the square except that it does not allow the same
vertex to be selected twice in a row.
cannot be opposite of the Square
10. Try the next code cell titled with vertex preference (the new vertex
previous)

Square with vertex preference (new vertex cannot be opposite of the previous):
clear;
T=[1/2 0; 0 1/2];
t=linspace(0,2*pi,5);
t(5)=[];
v=exp(1i*t);
x(:,1)=[rand-0.5; rand-0.5];
plot(real(v),imag(v),’k*’,x(1,1),x(2,1),’b.’);
axis equal;
hold on;
Num=5000;
w=[real(v(1));imag(v(1))];
k1=0;

6
 for j=1:Num
k=randi(4)
if (k~=k1+2)&&(k1~=k+2)
w=[real(v(k));imag(v(k))];
x(:,j+1)=T*(x(:,j)-w)+w;
k1=k;
else
x(:,j+1)=x(:,j);
end;
end;
plot(x(1,:),x(2,:),’b.’);
hold off;

This code repeats the process above except that it does not allow for the new vertex to
lie opposite the old one.
11. Run the next code cell titled Barnsley fern. The code in this cell creates the so-called
Barnsley fern which is a fractal with striking resemblance to a fern leaf (a lot of plants
and animals in nature use fractals to their advantage!):
Barnsley fern
clear;

T1=[0.85,0.04;-0.04,0.85];
T2=[-0.15,0.28;0.26,0.24];
T3=[0.2,-0.26;0.23,0.22];
T4=[0,0;0,0.16];
Q1=[0;1.64];
Q2=[-0.028;1.05];
Q3=[0;1.6];
Q4=[0,0];
P1=0.85;
P2=0.07;
P3=0.07;
P4=0.01;

Num=15000;
x(:,1)=rand(2,1);
plot(x(1,1),x(2,:),’b.’)
axis equal;
hold on;
for j=1:Num
r=rand;
if r<=P1
x(:,j+1)=T1*x(:,j)+Q1;
elseif r<=P1+P2 x(:,j+1)=T2*x(:,j)
+Q2;
elseif r<=P1+P2+P3 x(:,j+1)=T3*x(:,j)
+Q3;
else
x(:,j+1)=T4*x(:,j)+Q4;
end
end

plot(x(1,:),x(2,:),’b.’);

6
 Figure 14: Barnsley Fern

Figure 15: Fractals

hold off;

This code alternates between four different linear mappings given by y = Tj ∗ x + Qj,
j = 1, 2, 3, 4, where the matrices Tj and the vectors Qj are given in the code. The
probabilities of each mapping being selected are not equal as well and are given by the
numbers P1, P2, P3, and P4 in the code. If you did everything correctly, you should see
the image similar to the one in Fig. 14.
12. Modify the initial code for the Sierpinski triangle to create a fractal based on a regular
hexagon. Take the matrix T to be
T=[1/4 0; 0 1/4];

If you did everything correctly, you should see the figure similar to the first image in
Fig. 15.
13. Finally, create another
Q3: Experiment Chaos Game. Useis the
the matrix T=[2/5 0;
can0find
2/5] for the linear
mapping and the with the ratio.
vertices What
of a regular largestwith
pentagon ratio yourestriction
the such
that each new
vertex cannot be the same as the previous one. Plot the result in a new figure window.
Ifthat
you did everything correctly, you should see the figure similar to the second image in
Fig. 15.
the little hexagons don’t touch (two digits after the decimal point are

6
Project 11: Projections, eigenvectors, Principal Component
Anal- ysis, and face recognition algorithms
Goals: To use eigenvalues and eigenvectors of a matrix to create a simple face recognition
algorithm.
To get started:

• Download the file lab11.m and put it in your working directory13.

• Download the archive file database.zip and unpack it into your working directory.
This will produce a folder “database”, which contains 33 image files in the format .pgm
14
.
Matlab commands used: imread, size, reshape, num2str, mean, uint8,
double, ones, eig, diag

What you have to submit: The file lab11.m which you will modify during the lab session.
INTRODUCTION

In this project we will apply principal component analysis (PCA) to create a simple face
recognition algorithm. The PCA algorithm works by identifying patterns in the data which
allows one to reduce the dimension of the dataset. We will use the following steps to apply
the PCA algorithm to face recognition:
• Download the database of N images (in our case, N = 30). All images must have the
same dimensions m × n.
• Input the images person1.pgm-person30.pgm into Matlab as arrays. Reshape each of
the arrays as a column vector Xi, i = 1, ..., N .
ΣN
• Compute the mean of all images Xmean = 1
N Xii.
• Subtract the mean image from each of the vectors Xi : X˜ i = Xi − Xmean and create a
matrix P, the ith column of which is equal to the vector X˜ i . The matrix P has 30
columns and mn rows where m, n are the dimensions of the images.
• We are interested in the eigenvalues and eigenvectors of the matrix PPT . However, the
matrix PPT is usually very large. For instance, if our database consists of the images
with dimensions m × n = 100 × 100, then the matrix PPT has the dimensions 10000 ×
10000, and computation of eigenvalues and eigenvectors becomes unfeasible. Instead, we
will use the fact that only at most N 1 eigenvalues of the matrix PPT are non-zero
(why?), Tand those non-zero eigenvalues are the same as non-zero eigenvalues of the
matrix P P (why?). The matrix PT P has theT dimensions N N which are usually much
smaller than the dimensions of the matrix PP . For instance, if there are 30 images in
then the dimensions of PT P are onlyT 30 30. Thus, it is
the database (like in our case), −
much more efficient to solve the eigenvalue problem for the matrix P P.
13
The printout of the file lab11.m can be found in appendices of this book.
14 ×
These files represent a subset of the database of faces from AT&T Laboratories Cambridge www.cl.cam.ac.
uk/research/dtg/attarchive/facedatabase.html.
×

7
• If ui is an eigenvector of the matrix PT P corresponding Tto a non-zero eigenvalue λi,
then vi = Pui is the eigenvector of the matrix PP corresponding to λi. The
eigenvectors of the matrix PPT constitute the set of components of the data with the
largest variance between them (which are called principal components).
• Compute the projections of your sample data onto the linear subspace generated by the
vectors vi. The power of the algorithm is in the dimension reduction. Our initial images
of faces “live” in the mn-dimensional real space. For instance, if the images are 100 100
pixels, then the dimension of the space is equal to 100 2 = 10000. If we would try to
compare these images to each other pixel by pixel, we would need to compare 10000 ×
pixels. However, not all of these pixels are significant. Principal Component Analysis
allows us to identify the important information about the image and compare only that
information. For instance, if we are working with a database of 30 images, then we will
need to compare only 29 numbers. That is a dimensional reduction from 10000 to only
29! The PCA algorithm is a work horse behind many modern techniques in data
mining, including face recognition algorithms.
TASKS

1. Open the file lab11.m and run the first code cell titled Input database files
into the Matlab:
Input database files into Matlab
clear;
Database_Size=30;
Reading images from the database. The image files should be located in
the subfolder ‘‘database"
for j=1:Database_Size
image_read=imread([’database\person’ num2str(j) ’.pgm’]);
[m,n]=size(image_read);
end;P(:,j)=reshape(image_read,m*n,1);

This code willarray

rect- angular input
to the database
a column images
vector, and into Matlab,
create reshape
the matrix each image
P. Observe from
the use the
of the
string function num2str in the imread command and the use of the reshape function
to change the dimensions of the arrays containing the image data. Instead of the
rectangular arrays with the elements corresponding to the pixel colors we now have a
set of long vectors where columns of each image have been “stacked” on top of each
other.
2. Compute the mean image by applying the mean function to the the matrix P row-wise.

Display the mean image by using the function imshow.

Variables: Database Size, m, n, P
Computing and displaying the mean face
mean_face=mean(P,2);
imshow(uint8(reshape(mean_face,m,n)))

Variables: mean face

3. Subtract the mean of the images from each of the columns of the matrix P:

7
 Subtract the mean face
P=double(P);
P=P-mean_face*ones(1,Database_Size);

Notice the conversion to the double type of the array P.

4. Compute the covariance matrix PT P and compute the eigenvalues and the eigenvectors
of this matrix PT P by using the built in function eig. The function eig returns two
arrays, one of which has eigenvalues on the main diagonal, and the other has
eigenvectors as columns. Observe that one of the eigenvalues is very close to zero.
Compute the covariance matrix of the set and its eigenvalues and
eigenvectors
[Vectors,Values]=eig(P’*P);
EigenVectors=P*Vectors;

Q1: WhyVectors,
Variables: Values,
does the matrix PT P have a zero
5. Display the set of eigenfaces on the same figure by running the code cell
set of eigenfaces: Display the

Display the set of eigenvaces

for j=2:Database_Size;
if j==2
EigenFaces=reshape(EigenVectors(:,j)+mean_face,m,n);
else
EigenFaces=[EigenFaces reshape(EigenVectors(:,j)
+mean_face,m,n)]; end;
end
EigenFaces=uint8(EigenFaces);
figure;
imshow(EigenFaces);

If you did everything correctly, you will see an image containing 29 eigenfaces, the first
five of which are shown in Fig. 16. Observe that the mean face has been added back to
the eigenfaces.
Variables: EigenFaces
6. The matrix PPT is symmetric. It follows from the theorems of linear algebra that the
eigenvectors corresponding to the distinct eigenvalues are orthogonal. Write code to
verify the last statement. You will need to create the matrix
Products=EigenVectors’*EigenVectors
and observe that it is a diagonal matrix. This matrix contains all possible dot products
of the eigenvectors of the matrix PPT . The fact that the eigenvectors are orthogonal
will be used in the following sections to find orthogonal projections.

Q2: Why is the matrix PPT symmetric?

Variables: Products
7. In this project we will use the obtained eigenfaces to perform several operations. Our
first task will be to “unmask” an altered face (sunglasses have been digitally added to
the image). The main idea behind this code is in computing the projection of the vector
of data (obtained from the altered image) onto the subspace spanned by the
eigenvectors. This

7
 Figure 16: The first five eigenfaces
projection is thenimage
of an altered displayed on the screen. Now run the code cell titled
(sunglasses): Recognition

Recognition of an altered image (sunglasses)

image_read=imread([’database\person30altered1.pgm’]);
U=reshape(image_read,m*n,1);
NormsEigenVectors=diag(Products);
W=(EigenVectors’*(double(U)-mean_face));
W=W./NormsEigenVectors;
U_approx=EigenVectors*W+mean_face;
image_approx=uint8(reshape(U_approx,m,n));
figure;
imshow([image_read,image_approx])

In this code, we start by uploading the image person30altered1.pgm into Matlab. The
image is then reshaped as a long vector similarly to the way it was done before. Then
we compute the projection of the new vector with the subtracted vector mean face
onto the space spanned by the set of the eigenvectors and add the vector mean face
back. Finally, the reconstructed image is displayed. Observe that we get a very good
approximation of the initial image without sunglasses.
8. Modify the code above to unmask the image person30altered2.pgm.
9. Now let us compute the approximation of the face which is not in the database. To do
this, modify the code above again to approximate the image person31.pgm. Observe,
that the results
Variables: image ofread,
the last two approximations are
U, NormsEigenVectors, W, isU somewhat
approx worse than the very first
one. This can be explained by the fact that there an insufficient number of images in
our database which do not span a wide enough variety of faces.

7
Project 12: Matrix eigenvalues and the Google’s PageRank
algo- rithm
Goals: To apply matrix eigenvalues and eigenvectors to ranking of webpages in the World
Wide Web.
To get started:

• Download the file lab12.m and put it in your working directory15.

• Download the file AdjMatrix.mat which contains the adjacency matrix of a so-called
“wiki- vote” network with 8297 nodes 16 [9].
Matlab commands used: load, size, numel, nnz, for... end, gplot

What you have to submit: The file lab12.m which you will modify during the lab session.
INTRODUCTION

According to social polls, the majority of users only look at the first few results of the
online search and very few users look past the first page of results. Hence, it is crucially
important to rank the pages in the “right” order so that the most respectable and relevant
results will come first. The simplest way to determine the rank of a webpage in a network is
to look at how many times it has been referred to by other webpages. This simple ranking
method leaves a lot to be desired. In particular, it can be easily manipulated by referring to a
certain webpage from a lot of “junk” webpages. The quality of the webpages referring to the
page we are trying to rank should matter too. This is the main idea behind the Google
PageRank
The Googlealgorithm.
PageRank algorithm is the oldest algorithm used by Google to rank the web
pages which are preranked offline. The PageRank scores of the webpages are recomputed
each time Google crawls the web. Let us look at the theory behind the algorithm. As it turns
out The
it, itmain
is based on the theorems
assumption of linearisalgebra!
of the algorithm that if you are located on any webpage then with
equal probability you can follow any of the hyperlinks from that page to another page.
This allows to represent a webpage network as a directed graph with the webpages being
the nodes, and the edges being the hyperlinks between the webpages. The adjacency
matrix of such a network is built in the following way: the (i, j)th element of this matrix
is equal to 1 if there is a hyperlink from the webpage i to the webpage j and is equal to 0
otherwise. Then the row sums of this matrix will represent numbers of hyperlinks from
each webpage and the column sums will represent numbers of times each webpage has
been referred to by other webpages. Even further, we can generate a matrix of
probabilities S such that the (i, j)th element of this matrix is equal to the probability of
traveling from ith webpage to jth webpage in the network. This probability is equal to
zero if there is no hyperlink from ith page to jth page and is equal to 1/Ni if there is a
hyperlink from ith page to jth page, where Ni is the total number of hyperlinks from ith
page. For instance, consider a sample network of only four webpages shown
on15 the Fig. 17. The matrix S for this network can be written as:
The printout of the file lab12.m is given in the appendices of this book.
16
The original network data is available here: https://snap.stanford.edu/data/.

7
 12

4 3

Figure 17: Sample network of four webpages

 0 1/3 1/3 1/3

1/2 0 0 1/2
 0 1/2 0 1/2 
S= 
0 0 0 0  (1)

There are several issues which make working with the matrix S inconvenient. First of all,
there are webpages that do not have any hyperlinks - the so-called “dangling nodes” (such as
the node 4 in Fig. 17). These nodes will correspond to zero rows of the matrix S. Moreover,
the webpages in the network may not be connected to each other and the graph of the
network may consist of several disconnected components. These possibilities lead to
undesirable properties of the matrix S which make computations with it complicated and not
evenThe
always possible.
problem of the dangling nodes can be solved by assigning all elements of the matrix S in
the rows corresponding to the dangling nodes equal probabilities 1/N , where N is the
number of the nodes in the network. This can be understood in the following way: if we
are at the dangling node we can with equal probability jump to any other page in the
network. To solve the potential disconnectedness problem, we assume that a user can
follow hyperlinks on any page with a probability 1 − α and can jump (or “teleport”) to
any other page in the network with a probability α. The number α is called a damping
factor. The value of α = 0.15 is usually taken in practical applications. The “teleport”
surfing of the network can be interpreted as a user manually typing the webpage address
in the browser or using a saved hyperlink from their bookmarks to move from one page
onto another. The introduction of the damping factor allows us to obtain the Google
matrix G in the form:
G = (1 − α)S + αE,
where E is a matrix with all the elements equal to 1/N , where N is a number of
webpages in the network.
The matrix G has nice properties. In particular, it has only positive entries and all of its
rows sum up to 1. In mathematical language, this matrix is stochastic and irreducible (you
can look up the precise definitions of these terms if you are interested). The matrix G
satisfies the following Perron-Frobenius theorem:
Theorem 1 (Perron-Frobenius) Every square matrix with positive entries has a unique
unit eigenvector with all positive entries. The eigenvalue corresponding to this eigenvector
is real and positive. Moreover, this eigenvalue is simple and is the largest in absolute value
among all the eigenvalues of this matrix.

Let us apply this theorem to the matrix G. First of all, observe that the row sums of the

7
matrix G are equal to 1. Consider the vector v1 = (1, 1, ..., 1)T /N . It is easy to see that

Gv1 = v1.
But then it follows that v1 is the unique eigenvector with all positive components, and,
therefore, by the Perron-Frobenius theorem, λ1 = 1 is the largest eigenvalue!
We are interested in the left eigenvector for the eigenvalue λ1 = 1:
uT G = uT .
1 1

Again, by the Perron-Frobenius theorem, the vector u1 is the unique unit eigenvector with all
positive components corresponding to the largest in absolute value eigenvalue λ1 = 1. We will
use Let
the us
components
look at theofjustification
this vector for the ranking
behind of webpages
this algorithm. in the
We have network.
already established that the
vector u1 exists. Consider the following iterative process. Assume that at the beginning a user
can be on any webpage in the network with equal probability:
w0 = (1/N, 1/N, ..., 1/N ).
After 1 step (one move from one webpage to another using hyperlinks or teleporting), the
probability vector of being on the ith webpage is determined by the ith component of the
vector w1 = w0G.
After two moves the vector of probabilities becomes

w2 = w1G = w0G2,
and so on.
We hope that after a large number of steps n, the vector w = w0Gn starts approaching
some kind of limit vector w∗, wn → w∗. It turns out that due ton the properties of the matrix
G this limit vector w∗ indeed exists and it is exactly the eigenvector corresponding to the
largest eigenvalue, namely, λ1 = 1. Moreover, numerical computation of matrix eigenvalues is
actually based on taking the powers of the matrix (it is called the Power method) and not on
solving theassume
Let us characteristic
that theequation!
vector w∗ is a non-negative vector whose entries sum to 1. Then the
components of this vector represent the probabilities of being on each webpage in the
network after a very large number of moves along the hyperlinks. Thus, it is perfectly
reasonable to take these probabilities as ranking of the webpages in the network.
TASKS

1. Open the file lab12.m. In the code cell titled Load the network data load the data
from the file AdjMatrix.mat into Matlab by using the load command. Save the
resulting matrix as AdjMatrix. Observe that the adjacency matrices of real networks
are likely to be very large (may contain millions of nodes or more) and sparse. Check
the sparsity of the matrix AdjMatrix using the functions numel and nnz. Denote the
ratio of non-zero elements as nnzAdjMatrix. If you did everything correctly you should
obtain that only 0.15% of the elements of the matrix AdjMatrix are non-zero.

Variables: AdjMatrix, nnzAdjMatrix

7
2. Check the dimensions of the matrix AdjMatrix using the size function. Save the
dimen- sions as new variables m and n.
Variables: m, n
3. Observe that while the network described by the matrix AdjMatrix is not large at all
from the viewpoint of practical applications, computations with this matrix may still
take a noticeable amount of time. To save time, we will cut a subset out of this
network and use it to illustrate the Google PageRank algorithm. Introduce a new
variable NumNetwork and set its value to 500. Then cut a submatrix AdjMatrixSmall
out of the matrix AdjMatrix and plot the graph represented by the matrix
AdjMatrixSmall by running the following code cell:
Display a small amount of network
NumNetwork=500;
AdjMatrixSmall=AdjMatrix(1:NumNetwork,1:NumNetwork);
for j=1:NumNetwork
coordinates(j,1)=NumNetwork*rand;
coordinates(j,2)=NumNetwork*rand;
end;
gplot(AdjMatrixSmall,coordinates,’k-*’);

This
of thewill plot the
nodes. subgraph
Notice of the
the use firstfunction
of the 500 nodes in thetonetwork
gplot producewith
this random
graph. locations
Observe
that Matlab has special functions graph and digraph for working with graphs, but
those functions are a part of the special package “Graph and Network Algorithms”
which may not be immediately available. Simpler methods, as shown above, will be
sufficient for our purposes.
Variables: AdjMatrixSmall, coordinates, NumNetwork
4. Set the parameter α = 0.15. Introduce the vector w0=(1,1,...,1)/NumNetwork,
5 and
compute the consequent vectors w1 = w0G, w2 = w1G, w3 = w2G, w5 = w0G , w10 =
w0G10. Compute the difference δw = w10 − w5. Observe that the sequence wn converges
to a certain limit vector w∗ very fast.
Variables: w0, w1, w2, w3, w5, w10, deltaw
5. Compute the eigenvalues and the left and the right eigenvectors of the matrix G using
the function eig. Observe that the right eigenvector corresponding to the eigenvalue λ1
= 1 is proportional to the vector v1 = (1, 1, ..., 1). To compute the left eigenvectors, use
the function eig on the matrix G’. Select the left eigenvector corresponding to the
eigenvalue λ1 = 1 and denote it as u1.
Variables: u1
6. Observe that by default the vector u1 is not scaled to have all positive components
(even though all the components of the vector u1 will have the same sign). Normalize
this vector by using the code:
u1=abs(u1)/norm(u1,1);

This will create a vector with all positive components whose entries sum to 1 (called a
probability vector).
7. Use the function max to select the maximal element and its index in the array.

Variables: MaxRank, PageMaxRank

7
8. Find out whether the highest ranking webpage is the same as the page with the most
hyperlinks pointed to it. To do so, create the vector of column sums of the matrix G
and save it as MostLinks. Use the function max again to select the element with the
maximal number of links.
Variables: MostLinks, MaxLinks, PageMaxLinks
9. Compare if MaxRank and MaxLinks are the same.
Q1: What is the number of hyperlinks pointing to the webpage MaxRank?

(look at the vector MostLinks to find out).

7
Project 13: Social networks, clustering, and eigenvalue problems

Goals: To apply linear algebra techniques to data clustering.

To get started:

• Download the file lab13.m and put it in your working directory17.

• Download the 18file social.mat which contains the adjacency matrix of a simulated
social network .
Matlab commands used: sum, diag, eig, for...end, if... then... end, load,
clear, spy, eye

What you have to submit: The file lab13.m which you will modify during the lab session.
INTRODUCTION

Many practical applications deal with large amounts of data. One of the standard
problems in data science is to group this data in clusters containing similar items. In many
cases, clustering can be easily accomplished by the human brain (take a look at the attached
pictures).

In general, creating an algorithm which allows us to separate the data into clusters is a
difficult task. The data in many cases is multidimensional and cannot be visualized easily.
Even deciding how many clusters to select is not necessary an easy task. There are currently
several widely used algorithms which allow for the separation of the data points by similarity.
All In
these
thisalgorithms
project, wehave
willtheir ownspectral
use the advantages and disadvantages.
clustering algorithm based on the Fiedler’s vector.
We will look at the graph representing an individual’s profile on a social network. The profile
has a total of 351 “friends”. Some of these friends know each other and are friends with each
other as well. This friendship network can be represented as a graph with nodes being friends
and edges showing the friendship connection between the individuals. The goal of clustering
is to separate this graph into components which are tightly connected within themselves and
have fewer connections to the outside (the so-called communities). The method explored here
is based on the theorem by M. Fiedler, who has shown that for a connected graph (meaning
that there is a path between any two vertices) the eigenvector corresponding to the second
smallest eigenvalue of the Laplacian of the graph allows the graph to be split into two
maximally intraconnected components.
17
The printout of the file lab13.m can be found in the appendices of this book.
18
This data has been artificially created.

7
 Figure 18: An example of a simple graph
TASKS

1. We will start by considering an example of a simple graph: Create a new code cell with
the title Small graph. Type in the adjacency matrix of this graph into Matlab and
save it in a variable AdjMatrix.
Variables: AdjMatrix
2. Find the row sums of the matrix AdjMatrix (use the sum function) and save it as RowSums.
Variables: RowSums

3. Compute the Laplacian of the graph using the following command:

LaplaceGraph=diag(RowSums)-AdjMatrix;

Observe
RowSums that
on the diag(RowSums)
themain diagonal andcommand creates a else.
zeros everywhere diagonal matrix
Observe with
also thethe
that vector
ma-
trix LaplaceGraph is a symmetric positive Tsemi-definite singular matrix. Recall that
a matrix A is positive semi-definite if xAx ≥ 0 for all column vectors x. To check
that it is singular, multiplyT the matrix LaplaceGraph on the right by the column
vector [1, 1, 1, 1, 1, 1, 1, 1, 1] .
Variables: LaplaceGraph

4. Find the eigenvalues and eigenvectors of the matrix LaplaceGraph using the eig function:
[V,D]=eig(LaplaceGraph);

Here
the matrix D is aVdiagonal
the matrix containsmatrix
all thewith
righteigenvalues
eigenvectors
of of
thethe matrix
matrix LaplaceGraph
LaplaceGraph on and
the
main diagonal. Since the Laplacian is a symmetric graph, its eigenvalues are real. To
simplify our task, let us sort the eigenvalues from smallest to largest. We will need to
sort the eigenvectors along with them. Here is how to do that.
sort eigenvalues and eigenvectors
[d,ind] = sort(diag(D));
D = D(ind,ind);
V = V(:,ind);

8
 Variables: V, D
5. View the elements of the matrices V and D using the “Workspace” window of the
Matlab environment. Observe that one of the eigenvalues of the matrix LaplaceGraph
is zero (this is due to the fact that the matrix LaplaceGraph is singular). The
eigenvector for this eigenvalue is proportional to the vector [1, 1, 1, 1, 1, 1, 1, 1, 1]T . All
other eigenvalues of the matrix LaplaceGraph are positive. This follows from the fact
that the matrix LaplaceGraph is positive semi-definite and the graph corresponding to
this Laplacian is connected. Find the smallest positive eigenvalue of the matrix
LaplaceGraph (since we sorted the eigenvalues, it is the second diagonal element of the
matrix D).

6. Identify the eigenvector corresponding to the second smallest eigenvalue:

V2=V(:,2);

In
do the
thisnext step, we will
consistently, use
let us the entries
make sure that V2 is
of V2 tothe
group
unitthe elements. with
eigenvector To make surefirst
positive we
entry:
make sure the first entry is positive
if V2(1) < 0
V2 = -V2;
end

Variables: V2

7. Separate the elements of the eigenvector V2 onto positive and negative:

pos=[];
neg=[];
for j=1:9
if V2(j)>0
pos=[pos,j];
else
neg=[neg,j];
end;
end;

If everything is done correctly, the arrays pos and neg will contain correspondingly 1,
2, 3, 4 and 5, 6, 7, 8, 9 which would be an intuitive clustering choice.
Variables: pos, neg
8. Now let us apply this technique to a larger graph when the results cannot be visualized
so easily. To do this, load the matrix Social which is an adjacency matrix of a network
of 351 Facebook friends. The (i, j)th element is equal to 1 if the ith and jth friends are
also friends with each other and is equal to 0 otherwise:
Load the data
clear;
load(’social.mat’,’Social’);
spy(Social);

The
Socialcommand spy(Social)
on a figure, willdots
putting blue plotfor
thenon-zero
sparsityelements
pattern ofofthe
thematrix
adjacency
(you matrix
should
see something similar to the first image in Fig. 19). Observe that it is difficult to
distinguish any kind of underlying pattern in this matrix. Our goal will be to cluster
this set of data into two

8
 Figure 19: Adjacency matrices of the unsorted and sorted data.

maximally intraconnected groups.

Variables: Social
9. To do this, repeat the procedure we performed on a smaller matrix. Define the matrices
DiagSocial and LaplaceSocial which are correspondingly the vector of row sums of
the matrix Social and the Laplacian of this matrix.
Variables: DiagSocial, LaplaceSocial

10. Compute the eigenvalues and the eigenvectors:

[V,D]=eig(LaplaceSocial);

Variables: V,D
11. As before, identify the second smallest eigenvalue and the corresponding eigenvector V2
and sort the vertices of the graph by the positive and negative components of the
vector V2. Make the necessary adjustment to ensure that V2(1)>0.
Variables: V2
12. Permute the adjacency matrix Social using the permutation generated by the lists pos,
neg:
order=[pos,neg];
[m,n]=size(Social);
iden=eye(m,m);
for j=1:351;
for k=1:351;
P(j,k)=iden(order(j),k);
end;
end;
SocialOrdered=P*Social*P’

Q1: Whatorder,
Variables: P, command P*Social*P’ do to the rows and columns of the
does the

Social?

8
13. Plot the matrix SocialOrdered using the function spy again. If you did everything cor-
rectly, you will see the matrix separating into two denser connected subsets similarly to
what is pictured in the second image in Fig. 19.
14. What if we want more clusters? There are two ways to proceed. First, we can explore the
eigenvector V3 corresponding to the third smallest eigenvalue. We will group the nodes
in the four clusters: the ones which have positive components in both eigenvectors V2,
V3 (++ group), the ones which have a positive component in V2 and a negative
component in V3 (+ ), the ones which have a negative component in V2 and a positive
component in V3 ( +), and, finally, the ones with negative components in both vectors
( ). This generates 4 distinct groups and can be accomplished by the code in the
following −code cell:
−
Cluster in 4 groups
V3=V(:,3);
if V3(1) < 0
V3 = -V3;
end
pp=[];
pn=[];
np=[];
nn=[];
for j=1:m
if (V2(j)>0)
if (V3(j)>0)
pp=[pp,j];
else
pn=[pn,j];
end;
else
if (V3(j)>0)
np=[np,j];
else
nn=[nn,j];
end;
end;
end;
order=[pp,pn,np,nn];
iden=eye(m,m);
for j=1:351;
for k=1:351;
P(j,k)=iden(order(j),k);
end;
end;
SocialOrdered=P*Social*P’
figure;
spy(SocialOrdered)
15. An alternative way to obtain more clusters is to use the Fiedler vector procedure
iteratively, meaning that we will apply it again to the clusters obtained in the previous
step. This can be accomplished by running the following code:
Second order of Fiedler
SocialPos=Social(pos,pos);
SocialNeg=Social(neg,neg);
rowsumpos=sum(SocialPos,2);

8
 DiagSocialPos=diag(rowsumpos);
LaplaceSocialPos=DiagSocialPos-SocialPos;
[VPos,DPos]=eig(LaplaceSocialPos);
V2Pos=VPos(:,2);
[mpos,npos]=size(V2Pos);
posp=[];
posn=[];
for j=1:mpos
if (V2Pos(j)>0)
posp=[posp,pos(j)];
else
posn=[posn,pos(j)];
end;
end;
rowsumneg=sum(SocialNeg,2);
DiagSocialNeg=diag(rowsumneg);
LaplaceSocialNeg=DiagSocialNeg-SocialNeg;
[VNeg,DNeg]=eig(LaplaceSocialNeg);
V2Neg=VNeg(:,2);
[mneg,nneg]=size(V2Neg);
negp=[];
negn=[];
for j=1:mneg
if (V2Neg(j)>0)
negp=[negp,neg(j)];
else
negn=[negn,neg(j)];
end;
end;
ordergen=[posp,posn,negp,negn];
iden=eye(m,m);
for j=1:351;
for k=1:351;
P(j,k)=iden(ordergen(j),k);
end;
end;
SocialOrderedGen=P*Social*P’
figure;
spy(SocialOrderedGen)

Observe that the densely connected components obtained by the last two methods
might be different. In the context of a social network, the dense clusters may represent
groups of people who have common connections in real life. For instance, your family,
your friends from high school, your colleagues, etc.

8
Project 14: Singular Value Decomposition and image
compres- sion
Goals: In this project we will discuss a singular value decomposition of a matrix and its
applications to image compression and noise filtering.
To get started:

• Download the file lab14.m and put it in your working directory19.

• Download the files einstein.jpg20 and checkers.pgm21 and put them in your working
directory.
Matlab commands used: clear, linspace, subplot, quiver, plot, hold on...hold
off, svd, axis, title, imread, imshow, size, min, for... end, cos, sin, ’,
rand, double, ones

What you have to submit: The file lab14.m which you will modify during the lab session.
INTRODUCTION

Any m × n matrix A possesses a singular value decomposition of the form:

A = UΣVT ,
where U is an orthogonal m × m matrix satisfying the condition UT U = Im, Σ is an m × n
rectangular diagonal matrix with nonnegative values σ1, σ2, . . . , σmin(m,n)T on the main
diagonal, and V is an orthogonal n × n matrix satisfying the condition V V = In. The
nonnegative numbers σ1, σ2, . . . , σmin(m,n) are called the singular values of the matrix A.
They are arranged in the decreasing order: σ1 ≥ σ2 ≥ . . . ≥ σmin(m,n) ≥ 0. Observe that the
singular value decomposition of a matrix is not unique.
Singular value decomposition is important for implementation of many numerical
algorithms in linear algebra. In this project we will look at the geometric meaning of singular
value decom- position and two applications related to image processing and noise filtering.
Observe that more efficient image compression methods exist. The examples in this project
are used mainly to show the reduction in the amount of data (so-called dimension reduction)
which can be accomplished with singular value decomposition. Dimension reduction is an
important tool in many practical applications dealing with large amounts of data, such as
statistics, data science, and machine learning.
TASKS

1. We will start with an illustration of the geometric meaning of singular value

decomposition. Let us look at a singular value decomposition of a 2 2 matrix. Open the
file lab16.m, locate the code cell 2x2 matrix, and add the following commands
19 ×
The printout of the file lab14.m can be found in the appendices of this book.
20
The image is available at: https://commons.wikimedia.org/wiki/File:Albert_Einstein_Head.jpg
21
Image from page 446 of ”The standard Hoyle; a complete guide and reliable authority upon all games of
chance or skill now played in the United States, whether of native or foreign introduction” (1909), image
appears on Flickr Commons.

8
 2x2 matrix
clear;
t=linspace(0,2*pi,100);
X=[cos(t);sin(t)];
subplot(2,2,1);
hold on;
plot(X(1,:),X(2,:),’b’);
quiver(0,0,1,0,0,’r’);
quiver(0,0,0,1,0,’g’);
axis equal
title(’Unit circle’)
hold off;

This code will create a 2 × 100 matrix X = [x1 x2 · · · x100] whose columns xi are unit
vectors pointing in various directions. The plot will show a blue circle corresponding to
the endpoints of these vectors and two vectors of the standard basis on the plane e1 =
(1, 0)T and e2 = (0, 1)T . We use the function subplot which will create a plot
containing four subplots arranged in two rows and two columns. The plot above will
occupy the first “cell” of this plot. Observe the command quiver which draws a vector
with the beginning point given by the first two arguments ((0,0) in this case) and an
ending point given by the next two arguments ((1,0) or (0,1) in the code above).
Variables: X

2. Now in the M-file, define a variable A holding the matrix

A = [ 2, 1; -1, 1 ];

and compute the singular value decomposition of this matrix using the svd function:
[U,S,V] = svd(A);
Using
U, S, V.the workspace
Perform window of the Matlab main environment, check out the matrices
the commands
U’*U
V’*V
to ascertain that the matrices U and V are orthogonal. The output should produce 2 ×
2 identity matrices.
Variables: A, U, S, V
3. Next, let us observe the geometric meaning of the individual matrices U, Σ, V (U, S, V
in our Matlab code) in the singular value decomposition. To do this, observe the
transfor- mations induced by these matrices on a unit circle and the vectors of the
standard basis e1, e2. Let us start by multiplying the coordinates of the points of the
circle and the vectors e1, e2 by the matrix V. Execute the following code:
VX=V’*X;
subplot(2,2,2)
hold on;
plot(VX(1,:),VX(2,:),’b’);
quiver(0,0,V(1,1),V(1,2),0,’r’);
quiver(0,0,V(2,1),V(2,2),0,’g’);
axis equal
title(’Multiplied by matrix V^T’)
hold off;

8
 Observe that the matrix VX contains the vectors of the matrix X transformed by the
multi- plication by the matrix V T
. Since the matrices V and VT are orthogonal,
multiplication by the matrix VT is equivalent to rotation of a plane, possibly in
combination with a reflection along some straight line. This allows us to conjecture that
the image of the unit circle under this mapping will still be a unit circle, but the vectors
of the basis will be rotated and possibly switched in orientation.

VX multiplication by the transpose of the matrix V resulted in a reflection

Q1: Did the
Variables:
the
4. Now, let us multiply the result from the previous step by the matrix Σ (S in the Matlab
code). Observe that since the matrix Σ is diagonal, then multiplication by this matrix
geometrically means stretching of the plain in two directions. To verify this, execute the
following Matlab code:
SVX = S*VX;
subplot(2,2,3);
hold on;
plot(SVX(1,:),SVX(2,:),’b’);
quiver(0,0,S(1,1)*V(1,1),S(2,2)*V(1,2),0,’r’);
quiver(0,0,S(1,1)*V(2,1),S(2,2)*V(2,2),0,’g’);
axis equal
title(’Multiplied by matrix \Sigma V^T’)
hold off;

Observe that, as expected, the unit circle is stretched and becomes an ellipsis. The
images of the standard basis vectors are stretched as well.
Variables: SVX

5. Finally, multiply the results from the last step by the matrix U to obtain:
AX = U*SVX;
subplot(2,2,4)
hold on;
plot(AX(1,:),AX(2,:),’b’);
quiver(0,0,U(1,1)*S(1,1)*V(1,1)+U(1,2)*S(2,2)*V(1,2),U(2,1)*S(1,1)*V(1,1)+...
U(2,2)*S(2,2)*V(1,2),0,’r’);
quiver(0,0,U(1,1)*S(1,1)*V(2,1)+U(1,2)*S(2,2)*V(2,2),U(2,1)*S(1,1)*V(2,1)+...
U(2,2)*S(2,2)*V(2,2),0,’g’);
axis equal
title(’Multiplied by matrix U\Sigma V^T=A’)
hold off;

Observe thatmatrix
A. Since the the result is equivalentthen
U is orthogonal, to multiplying the initial
the multiplication vector
by this X by
matrix the matrix
should result
in a rotation of the plane possibly combined with a reflection. Confirm this by
observing the images of the basis vectors.

Q2: Did multiplication by the matrix U produce a reflection of the plane?

Variables AX
6. If you answered yes to both Q1 and Q2 above, can you modify the matrices U and V in
such a way that no reflections of the plane occur? Produce the modified matrices U1
and V1 and confirm that U1*S*V1’=A. Observe that this shows that singular value
decomposition

8
 is not unique.
Variables U1, V1

7. Finally, observe that Σ Σ

σ1 0
[Av1 Av2] = AV = USVT V = US = [u1 u2]
showing that = [σ1u1 σ2u2],

Av1 = σ1u1 and Av2 = 0σ2v2.

Check this fact numerically by computing the expression A*V-U*S.
8. Now we will look at image compression using SVD. Add the following commands to
your M-file:
Image compression
clear;
ImJPG=imread(’einstein.jpg’);
figure;
imshow(ImJPG);
[m,n]=size(ImJPG);

This code loads an image as a uint8 matrix and displays it on the screen. Each entry
in the matrix corresponds to a pixel on the screen and takes a value somewhere between
0 (black) and 255 (white).
Variables: ImJPG
9. Perform a singular value decomposition of the matrix ImJPG and save the output in matri-
ces UIm, SIm, and VIm. Since ImJPG is integer-valued, you will need to use svd(double(ImJPG)).
Variables: UIm, SIm, VIm

10. Plot the singular values using the code:

figure;
plot(1:min(m,n),diag(SIm));

This shows
matrix the Notice
ImJPG. singular values
that the (the diagonal
diagonal entries
entries of SIm are SIm
of the matrix)
ordered for decreasing
in the the image
order σ1 ≥ σ2 ≥ . . . ≥ σmin(m,n) ≥ 0.
11. The idea behind compression of data with singular value decomposition is in the following.

Using the singular value decomposition, the matrix A can be written in the form:

A = σ1u1vT1+ σ2u2vT + 2 · · · + σrurvT r

where r is the rank of A and ui and vi are the ith columns of U and V respectively.
Observe that since the singular values are arranged in the decreasing order, the first
terms in this sum provide a larger contribution to the matrix A then the subsequent
terms. It may happen that for some k < r, σk+1 is small compared to σ1, and
correspondingly does not affect the matrix A too much. We should then expect

1 2 k
A ≈ σ1u1vT + σ2u2vT + · · · + σkukvT

8
 to be a good approximation to the matrix A (this is called a truncated SVD). In fact,
there is a theorem in linear algebra which states that SVD truncated to the k terms is
the best approximation to the matrix A among the matrices of the rank at most k in
the sense of Frobenius norm (which is equivalent to the regular Euclidean norm if we
consider the matrix A as an mn-dimensional vector).
This idea can be used for image compression as follows. Instead of storing the whole m× n
matrix A, we can instead store the m × k and n × k matrices
C = [u1 u2 · · · uk] and D = [σ1v1 σ2v2 · · · σkvk].
If k is much smaller than min(m, n), then storing C and D will take much less space
than storing A. Moreover, if weT wish to display the image, we can reconstruct the
approximation of A as A ≈ CD .

Add the following code to your M-file:

Approximation of the original matrix with 50, 100, and 150 singular values
for k=50:50:150
ImJPG_comp=uint8(UIm(:,1:k)*SIm(1:k,1:k)*(VIm(:,1:k))’);
figure;
imshow(ImJPG_comp)
compression percentage
pct = 1 - (numel(UIm(:,1:k))+numel(VIm(:,1:k)*SIm(1:k,1:k)))/numel(ImJPG);
fprintf(’Compression percentage for 2.0f singular values: 8.3f\n’,k,
end;pct);

This code compresses the image as described above using k = 50, k = 100, and k = 150
singular values and displays the reconstructed image. Compare the reconstructed
images with the original. The code also displays the compression percentage as pct.
12. Observe that the singular value decomposition can also be used to smooth noisy data,
especially if the data contains patterns. Data smoothing is often necessary because all
ImJPG
measurements
Variables: comp,small
contain pct errors resulting in a “noise”. This noise usually determines
the smallest singular values of the matrix. Dropping these small values, thus, not only
saves the storage space, but also allows to eliminate noise from the data.
Start a new code cell. Load the file checkers.pgm into Matlab and add some noise to
the resulting image matrix using the following code:
Noise filtering
clear;
ImJPG=imread(’checkers.pgm’)
[m,n]=size(ImJPG);
ImJPG_Noisy=double(ImJPG)+50*(rand(m,n)-0.5*ones(m,n));
figure;
imshow(ImJPG);
figure;
imshow(uint8(ImJPG_Noisy));

Variables: ImJPG, ImJPG Noisy

13. Compute the SVD of the matrix ImJPG Noisy and save the resulting decomposition ma-
trices as UIm, SIm, and VIm.

Variables: UIm, SIm, VIm

8
14. Compute the approximations of the initial image with k = 10, k = 30, and k = 50
singular values. Display the resulting approximations and compare then to the “noisy”
image. Observe that SVD significantly reduces the noise. Compare the images to the
initial image without noise. Observe also that even though SVD reduces the noise, it
also somewhat blurs the image.

9
Appendices
Printout of the file lab09.m

clear;
Vertices=[1 1 1;...
-1 1 1;...
1 -1 1;...
1 1 -1;...
-1 -1 1;...
-1 1 -1;...
1 -1 -1;...
-1 -1 -1];

Edges=zeros(8,8);
Edges(1,2)=1;
Edges(1,3)=1;
Edges(1,4)=1;
Edges(2,5)=1;
Edges(2,6)=1;
Edges(3,5)=1;
Edges(3,7)=1;
Edges(4,6)=1;
Edges(4,7)=1;
Edges(5,8)=1;
Edges(6,8)=1;
Edges(7,8)=1;

Edges=Edges+Edges’;

theta1=pi/3;
theta2=pi/4;
theta3=pi/6;

rotmat=rotation(theta1,theta2,theta3);

VertRot=Vertices*rotmat;

figure;
axis equal;
hold on;

for j=1:8
for k=j+1:8
if (Edges(j,k)==1) line([VertRot(j,1),VertRot(k,1)],
[VertRot(j,2),VertRot(k,2)]);
end;
end;
end;

hold off;

Buckyball

9
[Edges,Vertices] = bucky;
figure;
axis equal;
hold on;

theta1=pi/3;
theta2=pi/4;
theta3=pi/6;

rotmat=rotation(theta1,theta2,theta3);

VertRot=Vertices*rotmat;

for j=1:60
for k=j+1:60
if (Edges(j,k)==1) line([VertRot(j,1),VertRot(k,1)],
[VertRot(j,2),VertRot(k,2)]);
end;
end;
end;

hold off;

figure;
axis equal;
hold on;

for j=1:60
for k=j+1:60
if (Edges(j,k)==1) line([Vertices(j,1),Vertices(k,1)],
[Vertices(j,2),Vertices(k,2)],... [Vertices(j,3),Vertices(k,3)]);
end;
end;
end;

hold off;

3D fawn
clear;

load(’v.mat’,’v’);
load(’f.mat’,’f’);

[mVert,nVert]=size(v);

[mFace,nFace]=size(f);

figure;
axis equal;
hold on;
for j=1:mFace
line([v(f(j,1),1),v(f(j,2),1)],[v(f(j,1),2),v(f(j,2),2)],[v(f(j,1),3),v(f(j,2),3)]);

9
 line([v(f(j,1),1),v(f(j,3),1)],[v(f(j,1),2),v(f(j,3),2)],[v(f(j,1),3),v(f(j,3),3)]);
line([v(f(j,2),1),v(f(j,3),1)],[v(f(j,2),2),v(f(j,3),2)],[v(f(j,2),3),v(f(j,3),3)]);
end;
hold off;

figure;
axis equal;
hold on;

theta1=pi/3;
theta2=pi/4;
theta3=pi/2;

rotmat=rotation(theta1,theta2,theta3);

VertRot=v*rotmat;

for j=1:mFace
line([VertRot(f(j,1),1),VertRot(f(j,2),1)],[VertRot(f(j,1),2),VertRot(f(j,2),2)]);
line([VertRot(f(j,1),1),VertRot(f(j,3),1)],[VertRot(f(j,1),2),VertRot(f(j,3),2)]);
line([VertRot(f(j,2),1),VertRot(f(j,3),1)],[VertRot(f(j,2),2),VertRot(f(j,3),2)]);
end;

9
Printout of the file lab10.m
The Chaos Game

Sierpinski triangle

Sierpinski triangle with a change of ratio

Sierpinski triangle with rotation

Square

Square with different ratio

Square with vertex preference (new vertex cannot be the same as previous)
clear;

T=[1/2 0; 0 1/2];

t=linspace(0,2*pi,5);
t(5)=[];
v=[cos(t); sin(t)];

x(:,1)=[rand-0.5; rand-0.5];

plot(v(1,:),v(2,:),’k*’,x(1,1),x(2,1),’b.’);
axis equal;
hold on;

Num=5000;

k1=0;
for j=1:Num
k=randi(3);
if (k>=k1)
k=k+1;
end;
x(:,j+1)=T*(x(:,j)-v(:,k))+v(:,k);
k1=k;
end;

plot(x(1,:),x(2,:),’b.’);
hold off;

Square with vertex preference (new vertex cannot be opposite of the previous)
clear;

T=[1/2 0; 0 1/2];

t=linspace(0,2*pi,5);
t(5)=[];
v=[cos(t); sin(t)];

x(:,1)=[rand-0.5; rand-0.5];

9
plot(v(1,:),v(2,:),’k*’,x(1,1),x(2,1),’b.’);
axis equal;
hold on;

Num=5000;

w=[real(v(1));imag(v(1))];
k1=0;

for j=1:Num
k=randi(4)
if (k~=k1+2)&&(k1~=k+2)
x(:,j+1)=T*(x(:,j)-v(:,k))+v(:,k);
k1=k;
else
x(:,j+1)=x(:,j);
end;
end;

plot(x(1,:),x(2,:),’b.’);
hold off;

Barnsley fern
clear;

T1=[0.85,0.04;-0.04,0.85];
T2=[-0.15,0.28;0.26,0.24];
T3=[0.2,-0.26;0.23,0.22];
T4=[0,0;0,0.16];
Q1=[0;1.64];
Q2=[-0.028;1.05];
Q3=[0;1.6];
Q4=[0,0];
P1=0.85;
P2=0.07;
P3=0.07;
P4=0.01;

Num=15000;
x(:,1)=rand(2,1);
plot(x(1,:),x(2,:),’b.’)
axis equal;
hold on;
for j=1:Num
r=rand;
if r<=P1
x(:,j+1)=T1*x(:,j)+Q1;
elseif r<=P1+P2
x(:,j+1)=T2*x(:,j)+Q2;
elseif r<=P1+P2+P3
x(:,j+1)=T3*x(:,j)+Q3;
else
x(:,j+1)=T4*x(:,j);

9
 end
end

plot(x(1,:),x(2,:),’b.’);
hold off;

Hexagon
clear;

T=[1/3 0; 0 1/3];

t=linspace(0,2*pi,7);
t(7)=[];
v=[cos(t); sin(t)];

x(:,1)=[rand-0.5; rand-0.5];

plot(v(1,:),v(2,:),’k*’,x(1,1),x(2,1),’b.’);
axis equal;
hold on;

Num=10000;

for j=1:Num
k=randi(6);
x(:,j+1)=T*(x(:,j)-v(:,k))+v(:,k);
end;
plot(x(1,:),x(2,:),’b.’);
hold off;

Pentagon with a skipped vertex

clear;
T=[1/2.5 0; 0 1/2.5];

t=linspace(0,2*pi,6);
t(6)=[];
v=[cos(t); sin(t)];

x(:,1)=[rand-0.5; rand-0.5];

plot(v(1,:),v(2,:),’k*’,x(1,1),x(2,1),’b.’);
axis equal;
hold on;

Num=10000;

k1=0;

for j=1:Num
k=randi(5);
if k~=k1
x(:,j+1)=T*(x(:,j)-v(:,k))+v(:,k);

9
 k1=k;
else
x(:,j+1)=x(:,j);
end;
end;
plot(x(1,:),x(2,:),’b.’);
hold off;

9
Printout of the file lab11.m
Simple face recognition algorithm
Input database files into the Matlab
clear;
Database_Size=30;

Reading images from the database. The image files should be located in
the subfolder "database"
for j=1:Database_Size
image_read=imread([’database\person’ num2str(j) ’.pgm’]);
[m,n]=size(image_read);
P(:,j)=reshape(image_read,m*n,1);
end;

Computing and displaying the mean face

Subtract the mean face

Compute the covariance matrix of the set and its eigenvalues and eigenvectors

Display the set of eigenvaces

for j=2:Database_Size;
if j==2
EigenFaces=reshape(EigenVectors(:,j)+mean_face,m,n);
else
EigenFaces=[EigenFaces reshape(EigenVectors(:,j)
+mean_face,m,n)]; end;
end

EigenFaces=uint8(EigenFaces);

figure;
imshow(EigenFaces);

Orthogonality and symmetry

Recognition of an altered image (sunglasses)

image_read=imread([’database\person30altered1.pgm’]);
U=reshape(image_read,m*n,1);

NormsEigenVectors=diag(Products);
W=(EigenVectors’*(double(U)-mean_face));
W=W./NormsEigenVectors;

U_approx=EigenVectors*W+mean_face;

image_approx=uint8(reshape(U_approx,m,n));
figure;
imshow([image_read,image_approx])

Recognition of an altered image (lower part of the face)

Recognition of a person not in the database

9
Printout of the file lab12.m
Google PageRank algorithm on the example of random network

Load the network data

clear;

Display a small amount of network

NumNetwork=500;
AdjMatrixSmall=AdjMatrix(1:NumNetwork,1:NumNetwork);

for j=1:NumNetwork
coordinates(j,1)=NumNetwork*rand;
coordinates(j,2)=NumNetwork*rand;
end;

gplot(AdjMatrixSmall,coordinates,’k-*’);

Check the amount of links originating from each webpage

NumLinks=sum(AdjMatrixSmall,2);

Create a matrix of probabilities (Google matrix)

Element (i,j) of the matrix shows the probability of moving from i-th
page of the network to jth page. It is assumed that the user can follow
any link on the page with a total probability of 85 (all hyperlinks
are
equal), and jump (teleport) to any other page in the network with a
total probability of 15 (again, all pages are equal).

alpha=0.15;
GoogleMatrix=zeros(NumNetwork,NumNetwork);
for i=1:NumNetwork
if NumLinks(i)~=0
GoogleMatrix(i,:)=AdjMatrixSmall(i,:)./NumLinks(i);
else
GoogleMatrix(i,:)=1./NumNetwork;
end;
end;

GoogleMatrix=(1-alpha)*GoogleMatrix+alpha*ones(NumNetwork,NumNetwork)./NumNetwork;

Check that all the rows in the GoogleMatrix matrix sum to 1

SumGoogleMatrix=sum(GoogleMatrix,2);

Finding an eigenvector corresponding to 1 (why is there sucj an eigenvector)?

w0=ones(1,NumNetwork)./sqrt(NumNetwork);

w1=w0*GoogleMatrix;
w2=w1*GoogleMatrix;
w3=w2*GoogleMatrix;

w100=w0*(GoogleMatrix)^100;
w90=w0*(GoogleMatrix)^90;

9
 Check the difference between v30 and v20. Observe that it is pretty
small
deltaw=w100-w90;

Compute the eigenvalues and the right eigenvectors

[VectRight,LamRight]=eig(GoogleMatrix);
Explain the result
LamRight=diag(LamRight);

Compute the eigenvalues and the left eigenvectors

[VectLeft,LamLeft]=eig(GoogleMatrix’);
LamLeft=diag(LamLeft);

Separate the eigenvector corresponding to the eigenvalue 1 and scale it

u1=VectLeft(:,1);

u1=abs(u1)/norm(u1);

Select the maximum element and the corresponding element.

Which page is the most important in the network?

[MaxRank,PageMaxRank]=max(u1);

Check if it’s the most popular (most linked to page):

MostLinks=sum(AdjMatrixSmall);
[MaxLinks,PageMaxLinks]=max(MostLinks);

1
Printout of the file lab13.m
Facebook and graph partitioning

Simple graph

Load the data

clear;
load(’social.mat’,’Social’);
spy(Social);

Laplacian

Spectral decomposition

Clusteer in two groups

Cluster in 4 groups

Diagonal(indsmall)=large+1000;
[small,indsmall]=min(Diagonal);

V3=V(:,indsmall);

pp=[];
pn=[];
np=[];
nn=[];

for j=1:m
if (V2(j)>0)
if (V3(j)>0)
pp=[pp,j];
else
pn=[pn,j];
end;
else
if (V3(j)>0)
np=[np,j];
else
nn=[nn,j];
end;
end;
end;

order=[pp,pn,np,nn];

iden=eye(m,m);

for j=1:351;
for k=1:351;
P(j,k)=iden(order(j),k);
end;
end;

1
SocialOrdered=P*Social*P’

figure;
spy(SocialOrdered)

Second order of Fiedler

SocialPos=Social(pos,pos);
SocialNeg=Social(neg,neg);

rowsumpos=sum(SocialPos,2);

DiagSocialPos=diag(rowsumpos);

LaplaceSocialPos=DiagSocialPos-SocialPos;

[VPos,DPos]=eig(LaplaceSocialPos);

V2Pos=VPos(:,2);

[mpos,npos]=size(V2Pos);

posp=[];
posn=[];

for j=1:mpos
if (V2Pos(j)>0)
posp=[posp,pos(j)];
else
posn=[posn,pos(j)];
end;
end;

rowsumneg=sum(SocialNeg,2);

DiagSocialNeg=diag(rowsumneg);

LaplaceSocialNeg=DiagSocialNeg-SocialNeg;

[VNeg,DNeg]=eig(LaplaceSocialNeg);

V2Neg=VNeg(:,2);

[mneg,nneg]=size(V2Neg);

negp=[];
negn=[];

for j=1:mneg
if (V2Neg(j)>0)
negp=[negp,neg(j)];
else
negn=[negn,neg(j)];

1
 end;
end;

ordergen=[posp,posn,negp,negn];

iden=eye(m,m);

for j=1:351;
for k=1:351;
P(j,k)=iden(ordergen(j),k);
end;
end;

SocialOrderedGen=P*Social*P’
figure;
spy(SocialOrderedGen)

1
Printout of the file lab14.m
2x2 matrix

clear;
t=linspace(0,2*pi,100);
X=[cos(t);sin(t)];
subplot(2,2,1);
hold on;
plot(X(1,:),X(2,:),’b’);
quiver(0,0,1,0,0,’r’);
quiver(0,0,0,1,0,’g’);
axis equal
title(’Unit circle’)
hold off;

A = [ 2, 1; -1, 1 ];
[U,S,V] = svd(A);
U’*U
V’*V

VX=V’*X;
subplot(2,2,2)
hold on;
plot(VX(1,:),VX(2,:),’b’);
quiver(0,0,V(1,1),V(1,2),0,’r’);
quiver(0,0,V(2,1),V(2,2),0,’g’);
axis equal
title(’Multiplied by matrix V^T’)
hold off;

SVX = S*VX;
subplot(2,2,3);
hold on;
plot(SVX(1,:),SVX(2,:),’b’);
quiver(0,0,S(1,1)*V(1,1),S(2,2)*V(1,2),0,’r’);
quiver(0,0,S(1,1)*V(2,1),S(2,2)*V(2,2),0,’g’);
axis equal
title(’Multiplied by matrix \Sigma V^T’)
hold off;

AX = U*SVX;
subplot(2,2,4)
hold on;
plot(AX(1,:),AX(2,:),’b’);
quiver(0,0,U(1,1)*S(1,1)*V(1,1)+U(1,2)*S(2,2)*V(1,2),U(2,1)*S(1,1)*V(1,1)+...
U(2,2)*S(2,2)*V(1,2),0,’r’);
quiver(0,0,U(1,1)*S(1,1)*V(2,1)+U(1,2)*S(2,2)*V(2,2),U(2,1)*S(1,1)*V(2,1)+...
U(2,2)*S(2,2)*V(2,2),0,’g’);
axis equal
title(’Multiplied by matrix U\Sigma V^T=A’)
hold off;

Modified SVD

1
U1(:,1)=U(:,1);
U1(:,2)=-U(:,2);
V1(:,1)=V(:,1);
V1(:,2)=-V(:,2);

U1*S*V1’-A

Check
A*V-U*S

Image compression

Creates a two-dimensional array with the dimensions equal to the dimensions

of the image
clear;
ImJPG=imread(’einstein.jpg’);
figure;
imshow(ImJPG);

[m,n]=size(ImJPG);

Compute an SVD

[UIm,SIm,VIm]=svd(double(ImJPG));

plot the singular values

figure;
plot(1:min(m,n),diag(SIm));

Create approximations to the image

With 50, 100, and 150 singular values

for k=50:50:150
ImJPG_comp=uint8(UIm(:,1:k)*SIm(1:k,1:k)*(VIm(:,1:k))’);
figure;
imshow(ImJPG_comp)
compression percentage
pct = 1 - (numel(UIm(:,1:k))+numel(VIm(:,1:k)*SIm(1:k,1:k)))/numel(ImJPG);
fprintf(’Compression percentage for 2.0f singular values: 8.3f\n’,k,
pct);
end;

Noise filtering
clear;
ImJPG=imread(’checkers.pgm’)
[m,n]=size(ImJPG);

Add some noise to the image ImJPG_Noisy=double(ImJPG)

+50*(rand(m,n)-0.5*ones(m,n)); figure;
imshow(ImJPG);

figure;
imshow(uint8(ImJPG_Noisy));

1
[UIm,SIm,VIm]=svd(ImJPG_Noisy);

figure;
plot(1:min(m,n),diag(SIm),’ko’);

for k=10:20:50
ImJPG_comp=uint8(UIm(:,1:k)*SIm(1:k,1:k)*(VIm(:,1:k))’);
figure;
imshow(ImJPG_comp)
compression percentage
pct = 1 - (numel(UIm(:,1:k))+numel(VIm(:,1:k)*SIm(1:k,1:k)))/numel(ImJPG);
fprintf(’Compression percentage for 2.0f singular values: 8.3f\n’,k,
pct);
end;

1
References
[1]S. Attaway. Matlab: A practical Introduction to Programming and Problem solving.

Butterworth-Heinemann, 3rd edition, 2013.

[2]K. Bryan, T. Leise. The $25,000,000,000 Eigenvector: The Linear Algebra behind Google.
SIAM Rev., 48(3), 569 - 581, 2006.
[3]T. Chartier. When Life is Linear: From Computer Graphics to Bracketology.
Mathematical Association of America, 2015.
[4]W.N. Colley. Colley’s bias free college football ranking method: The Colley matrix ex-
plained. www.Colleyrankings.com, 2002.
[5]L. Elden. Numerical linear algebra in data mining. Acta Numerica, 15, 327-384, 2006.
[6]C. Hennig, M. Meila, F. Murthagh and R. Rocci (eds.). Handbook of Cluster
Analysis.
CRC Press, 2015.
[7]J. Keener. The Perron–Frobenius Theorem and the Ranking of Football Teams. SIAM Rev.,
35(1), 80-93, 1993.
[8]A.N. Langville, C.D. Meyer. Google’s PageRank and Beyond: The Science of Search
En- gine Rankings. Princeton University Press, 2006.
[9]J. Leskovec, R. Sosic. SNAP: A General-Purpose Network Analysis and Graph-
Mining Library,. ACM Transactions on Intelligent Systems and Technology (TIST), 8(1),
2016. [10]F.M. Harper, J.A. Konstan, The MovieLens Datasets: History and Context. ACM
Trans-

actions on Interactive Intelligent Systems (TiiS), 5(4), 2015.