Scribd
Upload
144 views6 pages

Topic: Non-Negative Matrix Factorisation: Assignment - 2

The document discusses non-negative matrix factorization (NNMF). NNMF decomposes a matrix V into two lower rank matrices W and H with non-negative elements, such that V approximates W×H. It works by iteratively updating W and H to minimize the approximation error using techniques like gradient descent. NNMF has applications in domains like text mining, image processing, and recommendation systems. The document also discusses related concepts like multiplicative update rules and how NNMF can be viewed as a probabilistic graphical model or relaxed k-means clustering.

Uploaded by

Bushra Kambo
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
144 views6 pages

Topic: Non-Negative Matrix Factorisation: Assignment - 2

The document discusses non-negative matrix factorization (NNMF). NNMF decomposes a matrix V into two lower rank matrices W and H with non-negative elements, such that V approximates W×H. It works by iteratively updating W and H to minimize the approximation error using techniques like gradient descent. NNMF has applications in domains like text mining, image processing, and recommendation systems. The document also discusses related concepts like multiplicative update rules and how NNMF can be viewed as a probabilistic graphical model or relaxed k-means clustering.

Uploaded by

Bushra Kambo
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 6

ASSIGNMENT – 2

TOPIC: NON-NEGATIVE MATRIX


FACTORISATION

MADE BY: BUSHRA KAMBO


Roll No: 6

COURSE: PGDDS 2017-2018


INSTITUTE: NISM & UNIVERSITY OF MUMBAI
Introduction
What is NNMF?

Non-negative matrix factorization (NMF or NNMF), also non-negative


matrix approximation is a group of algorithms in multivariate
analysis and linear algebra where a matrix V is factorized into (usually)
two matrices W and H, with the property that all three matrices have no
negative elements. This non-negativity makes the resulting matrices
easier to inspect.

Let matrix V be the product of the matrices W and H,

Matrix multiplication can be implemented as computing the column


vectors of V as linear combinations of the column vectors in W using
coefficients supplied by columns of H. That is, each column of V can be
computed as follows:

Where vi is the i-th column vector of the product matrix V and hi is the i-
th column vector of the matrix H.

When multiplying matrices, the dimensions of the factor matrices may


be significantly lower than those of the product matrix and it is this
property that forms the basis of NMF. NMF generates factors with
significantly reduced dimensions compared to the original matrix. For
example, if V is an m × n matrix, W is an m × p matrix, and H is
a p × n matrix then p can be significantly less than both m and n.
How does it work?
NNMF::is called Non-negative matrix factorization because it returns
features and weights with no negative values. Therefore all features
must be positive or zero values.

Clustering: NNMF is closely related to both supervised and unsupervised


methodologies (supervised because R can be seen as a training set) but
in particular NNMF is closely related to other clustering (unsupervised)
algorithms.

NMF decomposes multivariate data by creating a user-defined number of


features. Each feature is a linear combination of the original attribute
set; the coefficients of these linear combinations are non-negative.

NMF decomposes a data matrix V into the product of two lower rank
matrices W and H so that V is approximately equal to W times H. NMF
uses an iterative procedure to modify the initial values of W and H so
that the product approaches V. The procedure terminates when the
approximation error converges or the specified number of iterations is
reached.

During model apply, an NMF model maps the original data into the new
set of attributes (features) discovered by the model.

Nonnegative matrix factorization (NMF) employ two consecutive


independent steps. The first one performs data transformation
(dimensionality reduction) and the second one classifies the transformed
data using classification methods, such as nearest neighbor/centroid or
support vector machines (SVMs).

Gradient Descent: The technique used to factor which attempts to


minimize error. We can calculate error from our product using the
squared error (Actual – predicted). Once we know the error, we can
calculate the gradient in order to figure out what direction to go to
minimize the error. We keep going until we have no more error.
Types:
 Approximate non-negative matrix factorization
 Convex non-negative matrix factorization
 Nonnegative rank factorization
 Different cost functions and regularizations
 Online NMF

Applications:
 Astronomy
 Text mining
 Spectral data analysis
 Scalable Internet distance prediction
 Non-stationary speech denoising
 Bioinformatics
 Nuclear imaging

Relation to other techniques


In Learning the parts of objects by non-negative matrix factorization Lee
and Seung proposed NMF mainly for parts-based decomposition of
images. It compares NMF to vector quantization and principal component
analysis, and shows that although the three techniques may be written
as factorizations, they implement different constraints and therefore
produce different results.

NMF as a probabilistic graphical model: visible units (V) are connected to


hidden units (H) through weights W, so that V is generated from a
probability distribution with mean.
It was later shown that some types of NMF are an instance of a more
general probabilistic model called "multinomial PCA". When NMF is
obtained by minimizing the Kullback–Leibler divergence, it is in fact
equivalent to another instance of multinomial PCA, probabilistic latent
semantic analysis, trained by maximum likelihood estimation. That
method is commonly used for analyzing and clustering textual data and
is also related to the latent class model.

NMF with the least-squares objective is equivalent to a relaxed form


of K-means clustering: the matrix factor W contains cluster centroids
and H contains cluster membership indicators. This provides a
theoretical foundation for using NMF for data clustering. However, k-
means does not enforce non-negativity on its centroids, so the closest
analogy is in fact with "semi-NMF".

NMF can be seen as a two-layer directed graphical model with one layer
of observed random variables and one layer of hidden random variables.

NMF extends beyond matrices to tensors of arbitrary order. This


extension may be viewed as a non-negative counterpart to, e.g.,
the PARAFAC model.

Other extensions of NMF include joint factorisation of several data


matrices and tensors where some factors are shared. Such models are
useful for sensor fusion and relational learning.
NMF is an instance of nonnegative quadratic programming (NQP), just
like the support vector machine (SVM). However, SVM and NMF are
related at a more intimate level than that of NQP, which allows direct
application of the solution algorithms developed for either of the two
methods to problems in both domains.

What is MUR – Multiplicative update rule?


There is no inverse therefore no negative values/ entries,
We have to initialise both W & H,
Title: NETFLIX {User and Movie Relationship}

N - No of Users
M – No of Movies
Where:
Vij – Rating by User “i” to Movie “j”
{If a user has not seen the movie then value is = 0}
Vij is “0” if i th User has not rated the movie j

V = n = *

H k*m

W n*k

If K << rank (v) :: It works the best.

{K is too small to rank v}

You might also like