Numerical Methods Project

1st Qasim Sahfique 3rd Talha Tariq Tanoli

Roll No: 200743 Roll No:200775
Bachelors of Electrical Engineering(Electronics)-6B Bachelors of Electrical Engineering(Electronics)-6B
[email protected] [email protected]

2nd Syed Rafif Raza

Roll No: 200689
Bachelors of Electrical Engineering(Electronics)-6B
[email protected]

Abstract—This project offers a comprehensive exploration of C. Display Sample Images from the CBCL Face Dataset:
the NMF algorithm, showcasing its effectiveness in factorizing
data and providing valuable insights into the structure of the Pick a subset of the dataset’s face pictures. Display the
CBCL face dataset. By exploring different aspects of NMF, selected example pictures using the image function to facilitate
including initialization and visualization of the basis images, the visualisation.
project aims to enhance understanding and utilization of this
powerful matrix factorization technique
Index Terms—NMF algorithm, CBCL face dataset D. Evaluate NMF Performance at Different Ranks:

I. I NTRODUCTION Set the NMF algorithm’s maximum number of iterations

The implementation of Lee and Seung’s Nonnegative Ma- (maxit) to the default value of 1000. To establish the algo-
trix Factorization (NMF) technique is the main goal of this rithm’s halting condition, select a convergence tolerance (es)
project. NMF is a potent method for dimensionality reduc- value (default: 0.0001). Set the specified beginning points (V0
tion, clustering, and data compression. By factorising a given and W0) as the initial values for the factor matrices V and
data matrix into two low-rank matrices, it approximates a W. Apply the NMF approach on the CBCL face dataset with
data matrix with nonnegative elements. In order to facilitate various rankings (e.g., 10, 20, 30, 40, 50). Applying the timeit
effective representation and analysis, the goal is to extract function, compute the algorithm’s runtimes for each rank.
from the data significant patterns and characteristics. The
project involves implementing the NMF algorithm and testing E. Analyze the Impact of Different Initializations:
it using the MIT CBCL face dataset. The effectiveness of the
Using separate starting matrices V0 and W0, begin the NMF
method is evaluated by measuring runtimes at various rankings
method using a rank from the previous step (for instance,
and investigating the effects of various initializations on the
30). Compare the outcomes of the factorization and objective
factorization outcomes. This study provides a thorough grasp
value calculations with various initializations. Keep an eye
of NMF and its application to actual data..
out for any changes to the objective value and the ensuing
II. TASK BREAKDOWN factorization.
A. Implement the NMF Algorithm:
Create a function to put the NMF algorithm into practise. F. Evaluate NMF Performance with Uniform Initialization:
Include parameters when defining the function, including the Start the NMF algorithm with V0 and W0 matrices where
data matrix (X), rank (r), convergence tolerance (es), and a all elements are initialized to 1. Calculate the objective value
choice of beginning points (V, W). and observe the resulting factorization. Compare the results
B. Load and Preprocess the CBCL Face Dataset: with those obtained from previous initializations.
Obtain the grayscale face images using the MIT CBCL face
dataset. Using a suitable function, for instance readmatrix, read G. Visualize Basis Images at Rank 50:
the dataset and save it as the data matrix X. Scale each image
Run the NMF algorithm on the CBCL face dataset with the
in the dataset to have a mean and standard deviation of 0.25
rank set to 50. The W factor matrix’s rows matching to the
as a preprocessing step.
base pictures should be extracted. Plot the foundational photos
Identify applicable funding agency here. If none, delete this. to see the patterns and traits you have studied.
 III. MATLAB CODE WITH EXPLANATION B. Project code

A. nmf function

– The load function is used to load the data matrix X

from the text file ”nnmf-2429-by-361-face.txt”.

– By dividing each component of X by 4, the data is

scaled to have a mean and standard deviation of 0.25.

• The function nmf accepts the following input arguments: – The imagesc function is used to construct a figure
X, a data matrix with each row representing a vectorized and display two sample pictures. Using the reshape
image; r, the rank of the factorization; maxit, the number function, the pictures are transformed from their
of iterations to perform; es, the stopping criterion; V, vectorized form to a 19x19 matrix.
an optional initial estimate for the matrix V; and W, an
optional initial estimate for the matrix W. The estimated
matrices V and W are returned.

• If no input arguments are supplied, the function verifies

the number of input arguments (nargin) and sets default
values for V, W, es, and maxit.Iterative process is
employed to run the NMF algorithm. Until the halting
requirement is satisfied, the loop iterates maxit times.
The method changes the matrices V and W every
iteration in accordance with the NMF update rules:
Revised V is V = V.*(X*W’)./(V*(W*W’))
Revised W is W = W.*(V’*X)./(V’*V*W)

• The resultant matrices V and W are guaranteed to be

nonnegative by these updating conditions.

• The goal function L is calculated as the residual X -

V*W’s squared Frobenius norm following each iteration.
The method exits the loop if the distance between
successive objective function values (abs(L0-L)) is less
than the halting condition (es*L0).
The load function is used to load the initial By calculating the Frobenius norm of the
estimations for the matrices V and W from the difference between the original data matrix X
”V0.txt” and ”W0.txt” files, respectively. and the rebuilt matrix V*W, one may determine
After that, the code executes a loop to conduct NMF the objective function value.
for each rank listed in the ranks array (10, 20, 30, Using fprintf, show the value of the goal function.
40, and 50).
An anonymous function called fun is constructed OUTPUT:
within the loop to run the nmf function with the
necessary inputs, including X, the current rank r,
a limit of 1000 iterations, a stopping criteria of
0.0001, and the starting estimates V0(:,1:r) and
W0(1:r,:).
The time it takes to execute the nmf function Fig. 1.
with the current rank is measured using the timeit
function. Explanation: The NMF algorithm’s characteris-
The rank and accompanying elapsed time are shown tics are to blame for the difference between the
using the fprintf display function. objective function value and the elapsed time.
By updating the matrices V and W, the method
OUTPUT seeks to minimise the objective function repeat-
edly. The algorithm’s convergence is influenced
by a number of variables, including the initial
values of V and W, the convergence criteria, and
the degree of data complexity. The method may
converge reasonably rapidly in terms of time, yet
nevertheless provide an objective function value
that is not ideal.
In the example provided, a relatively low
objective function value of 39.7626 indicates
that the original data and the approximation
produced by rank 50 suit each other well. But
the 6.421797-second elapsed time shows that
the method required some time to converge on
this answer. The dataset’s unique properties and
the NMF method itself can have an impact on
the pace of convergence and the value of the
objective function that is reached.

sample images ∗ Calculating objective value with v(ik) and

w(ik) initialized as 1

Matlab code

[V, W] = nmf(X, r, 1000, 0.0001, ones(m, r),

ones(r, n));
L = norm(X - V*W, ’fro’);
Time elapsed fprintf([’Objective function value with v(0)
∗ Calculating objective value and w(0) initialized as 1: ’,
num2str(L)]);
Matlab code
Restart the NMF method with matrices V and W
[V, W] = nmf(X, r, 1000, 0.0001, ones(m, r), that are initialised to one.
ones(r, n)); Determine the value of the objective function in
L = norm(X - V*W, ’fro’); this situation.
fprintf([’Objective function value: ’, Using fprintf, display the value of the goal
num2str(L)]); function.
∗ Plot the basis images (rows of W) at rank r = C ONCLUSION
50. In this project, we constructed the NMF algo-
rithm and used the MIT CBCL face dataset as
MATALB Code our testing ground. On the objective value and
factorization outcomes, we investigated the ef-
W = load(’W0.txt’); fects of various ranks, initializations, and con-
figure; vergence criteria. The NMF method performed
for i = 1:size(W, 1) well at estimating the data matrix and removing
subplot(5, 10, i); significant features. The factorization procedure
imshow(reshape(W(i,:), [19,19]),[]); produced basis pictures that provide light on the
end dataset’s underlying structural elements. We now
have a better grasp of NMF and its uses in pattern
Using the load function, this code segment loads recognition and data analysis thanks to this effort.
the matrix W from a stored file called ”W0.txt.”
The learnt basis pictures or features acquired by ACKNOWLEDGMENT
the NMF technique are represented by the matrix We would like to express our sincere gratitude
W. to our course professor, Syed Ahmed Pasha, for
The base pictures are displayed by creating a providing us with the opportunity to work on this
figure with the figure function after loading the project. His guidance and support were invalu-
matrix W. A loop that iterates across each row of able throughout the project, and he was always
W is subsequently entered by the code. The row available to address our queries and provide valu-
index is represented by the loop variable i. able insights. We appreciate his dedication to our
The subplot function is used inside the loop to learning and for encouraging us to think creatively
build a grid of subplots with 5 rows and 10 and explore new ideas. Without his guidance, this
columns. The value of i determines where the project would not have been possible. Thank you,
subplot is at in the current scene. The current Professor Syed Ahmed Pasha, for your invaluable
base picture is displayed by calling the imshow support.
function. We appreciate the efforts of Lee and Seung, who
Because each row of W represents a flattened published a paper [1] introducing the nonnegative
picture, the basic image is reshaped using the matrix factorization (NMF) technique. With uses
reshape function to create a matrix that is 19 in data reduction and clustering, their approach
by 19. The altered picture is then shown via the offers an alternative to principal components anal-
imshow function. ysis and vector quantization.
The NMF algorithm’s learnt features or patterns Based on their work, the NMF algorithm imple-
may be seen by visualising the base pictures as mentation in this project was created. We value
a grid of subplots. You may examine and com- their contribution to the subject of matrix factor-
prehend the traits and structures included in the ization as well as the fact that their approach is
data since each subplot represents one foundation available for research and use.
picture. The Database 1 of 2,429 grayscale face photos,
OUTPUT: which was provided by the MIT Centre for Bi-
ological and Computational Learning (CBCL), is
another resource we would like to recognise. In
our study, we utilised this dataset to illustrate how
the NMF method was used.
R EFERENCES
[1] D. D. Lee, and H.S. Seung, “Learning the parts
of objects by non- negative matrix factorization,”
Nature, vol. 401(6755), pp. 788–791, 1999.