Lab Experiment-5

Discrete Cosine transform/Discrete Fourier transform

Aim:
To find a transformation matrix to convert the given sequence of input into Discrete cosine
transform/Discrete Fourier transform, using change of basis method.
Theory:
DFT
A discrete Fourier transform performs a transformation operation on the given function and
convert it to its frequency domain representation. To have a transformation matrix that performs
DFT operation, it is enough to convert the standard basis of given dimension using the DFT
equation which is given as

DFT has many applications and is used in spectral analysis, Data compression etc.
DCT
A discrete cosine transform (DCT) expresses a sequence of finitely many data points in terms of
sum of cosine functions oscillating at different frequencies. To have a transformation matrix that
perform DCT operation it is enough to convert the standard basis of given dimension using the
DCT equation which is given as..

DCT also has many applications in mp3 audio compression, image compression etc






Matlab code
%Program to find the transformation matrix corresponding to DFT and DCT
%computing the transformation matrix of N point DFT
clc
clear
N=input('Enter the dimension of the Signal space for finding the DFT transformation matrix :')
I=eye(N);
%implementing the DFT equation
for k=1:N
for n=1:N
x_n=I(:,k);
X_k1(n,k)=sum(x_n.*exp(-1i*2*(pi/N)*(k-1)*(n-1)));
end
end

%Implementing the DCT transformation matrix
for k=1:N
for n=1:N
x_n=I(:,k);
X_k2(n,k)=sum(x_n.*cos((pi/N)*((n-1)+0.5)*(k-1)));
end
end
disp('The DFT transformation matrix for the dimension that you entered is:')
X_k1
disp('The DCT transformation matrix for the dimension that you entered is:')
X_k2

Results
Enter the dimension of the Signal space for finding the DFT transformation matrix :
N =4
The DFT transformtion matrix for the dimension that you entered is:
X_k1 = 1.0000 1.0000 1.0000 1.0000
1.0000 0.0000 - 1.0000i -1.0000 - i -0.0000 + 1.0000i
1.0000 -1.0000 - 0.0000i 1.0000 + 0.0000i -1.0000 - 0.0000i
1.0000 -0.0000 + 1.0000i -1.0000 - 0.0000i 0.0000 - 1.0000i


The DCT transformtion matrix for the dimension that you entered is:

X_k2 =
1.0000 0.9239 0.7071 0.3827
1.0000 0.3827 -0.7071 -0.9239
1.0000 -0.3827 -0.7071 0.9239
1.0000 -0.9239 0.7071 -0.3827

Discussion
We have sucessfully implemented DCT/DFT transformation matrix which would perform
transformation operation on any vector of a given finite dimensional vector space. We have
implimented it by tranforming the standard basis for the given dimension.















Lab Experiment-6
Eigen values and Eigen vectors
Aim:
To find the Eigen values and Eigen vectors of any given m x m matrix
Theory:
An eigenvector of square matrix is a vector which, when, multiplied by the matrix, gives the
result as a scaled version of the original vector
. Ax=x
where is the scaling factor and is called the eigen value. The associated vector X is called eigen
vector of A, where A is a matrix of .
Implementation:
Finding eigen values
To find eigen values, we nd the values of which satisfy the characteristic equation of the
matrix A, namely those values of for which
det(A I) = 0
where I is the mm identity matrix.
Finding eigen vectors
Once the eigen values of a matrix (A) have been found, we can nd the eigenvectors by finding
the null space of (A I)
Eigen values and eigen vectors
Matlab code
clear
clc
eigvec_mat=[];
A=input('Enter the matrix whose Eigen values are to found:')
[m,n]=size(A);

if((m~=n))%||(det(A)==0))
disp('eigen values cannot be found for this matrix')
else
I=eye(n);
.


y=sym('lambda');
d=det(A-(y.*I));
c=fliplr(coeffs(d));

for i=1:length(c)
c1(i)=double(c(i));
end
% finding the eigen values
disp('the eigen values of the entered matrix are :')
e_values=double(roots(c1))
end

%finding the eigen vectors of the matrix A
for i=1:length(e_values)
A_lambda_I=A-e_values(i).*I;
e1=e_values(i);
eigvec=null(A_lambda_I);
eigvec_mat=[eigvec_mat eigvec];
%sprintf('the orthonormal eigen vector corresponding to eigen value=%d is %d',e1,eigvec')
End
eigvec_mat
[ ch_eigen_vec, ch_eigen_val]=eig(A)
Results
Enter the matrix whose eigen values are to found:
A = 1 -2 1
-4 2 3
1 6 3
the eigen values of the entered matrix are :
e_values = 6.8898,
-3.6178
2.7281
eigvec_mat =
-0.0446 0.4172 0.6102
0.5487 0.6410 -0.1371
0.8349 -0.6442 0.7803
Check of program
[ ch_eigen_vec, ch_eigen_val]=eig(A)
ch_eigen_vec = 0.4172 0.6102 -0.0446
0.6410 -0.1371 0.5487
-0.6442 0.7803 0.8349
ch_eigen_val = -3.6178 0 0
0 2.7281 0
0 0 6.8898
Discussion
We have sucessfully implemented the program for finding Eigen values, Eigen vectors for any
given m x m matrix and also verified our program by using built in command.















Lab Experiment 8
Applications of Eigen values and Eigen Vectors
Aim
To explore some of the applications of eigen values and eigen vectors of a matrix, i.e finding
out the solutions of a set of differential equations characterizing a continuous time system using
the eigen values and eigen vectors of the system matrix and diagonalization of a matrix.
Theory
Diagonalization: Suppose we have an n x n matrix A has n independent eigen vectors x1,x2, ...
xn. A matrix formed with x1,x2 x3... xn as columns be S. Then S
-1
AS is the diagonal eigen value
matrix D.
D =



for a 3 x 3 matrix A.
Solution of differential equations: Consider a system of linear differential equations given by

= Au
Here A is a constant matrix. Hence the solutions will be of the form

x

Thus the steps for solving the difference equations are :
1. Find the eigen values is and the n independent eigen vectors xi of A.
2. Write the initial vector u(0) as a combination cx1+c2x2... of the eigen vectors
Then u(t) = c1

x1 + c2

x2+ ....... is the desired solution.

Enter the matrix to be diagonalized:[1 3 4;3 4 5;6 7 8]
The diagonal representation of A S^(-1)AS :
14.6520 -0.0000 0
0.0000 -1.5170 -0.0000
0.0000 -0.0000 -0.1350

The diagonal eigen value matrix :
14.6520 0 0
0 -1.5170 0
0 0 -0.1350

Enter the coefficients of a system of differential equations row-wise: [-2 1;1 -2]
Enter the initial state:[1 0]
Solution of the given differential equation is :
1/2*(1/2*2^(1/2)*exp(-3*t)+1/2*2^(1/2))*2^(1/2)+1/2*(1/2*2^(1/2)+1/2*2^(1/2)*exp(-
t))*2^(1/2)
1/2*(-1/2*2^(1/2)*exp(-3*t)+1/2*2^(1/2))*2^(1/2)+1/2*(-1/2*2^(1/2)+1/2*2^(1/2)*exp(-
t))*2^(1/2)












Lab Experiment-7
Karhunen-Loeve Transform
Aim:
To apply Karhunen-Loeve transformation to given image and also show that the image can be
reconstructed with fewer number of eigen values and eigen vectors.
Theory:
Karhunen-Loeve transformation is a transformation in which the transform kernal is not fixed.In
KLT the transform kernal is dependent on the data or we can say the kernal is derived from data .
The data is represented in the form of vectors. we define the transformation as.
Y=A(Xi-
x
)
Y is the transformed matrix, A is the eigen vector matrix of the covariance matrix of data
arranged according to desending order of magnitude of its corresponding eigen values ,
x
is the
mean value (row sum) vector of the input data.
The covariance matrix is obtained from the given equation
Cx=E { (Xi-
x
) x (Xi-
x
)
T
}
We can easly reconstruct our image back by applying inverse operation
Xi=A
T
Y+
x

KLT closely related to the Principal Component Analysis (PCA) and widely used in data
analysis in many fields.








Karhunen-Loeve Transform
Matlab code
% TO implement Karhunen Loeve Transform
clc
clear
X=imread('lena_128.jpg');
X=double(X);
[m,n]=size(X);
sum_cov_mat=zeros(m,m);
for i=1:m
r(i)=mean(X(i,:));
end
mean_vec=r';
for i=1:n
diff_mean_vec(:,i)=X(:,i)-mean_vec;
end

for i=1:n
temp=diff_mean_vec(:,i);
temp_cov_mat=temp*temp';
sum_cov_mat=sum_cov_mat+temp_cov_mat;
end

avg_cov_mat=(1/n).*sum_cov_mat;
eig_vec=eig(avg_cov_mat);
[eigen_vec,eigen_values]=eig(avg_cov_mat);
eigen_vec=fliplr(eigen_vec);%arranging the eigen vectors as columns corresponding to
%decreasig order of magnitude of eigen values
eigen_vec=eigen_vec';
A=eigen_vec;
%--------------------------------------------------------------------------
%% Uncomment this part and comment the next part to see perfect reconstruction
%%start of perfect reconstruction procedure
% for i=1:length(mean_vec)
% Y(:,i)=A*(X(:,i)-mean_vec);
% end
%
% for i=1:length(mean_vec)
% Reconstructed_image(:,i)=A'*Y(:,i)+mean_vec;
% end
% Reconstructed_image=uint8(Reconstructed_image);
% imshow(Reconstructed_image)
%%End of Perfect reconstruction procedure
%--------------------------------------------------------------------------
% In the above part,we have acheived Perfect reconstruction since we have
% considered all the eigen values
%--------------------------------------------------------------------------
%Now we are going for approximate reconstruction so that we will get
%compression
n_values=input ('enter the no of largest eigen values to be considered: ')
for i=1:n_values
A_k(i,:)=A(i,:);
end

for i=1:length(A_k)
Y_k(:,i)=A_k*(X(:,i)-mean_vec);
end

for i=1:length(A_k)
Reconstructed_image(:,i)=A_k'*Y_k(:,i)+mean_vec;
end
%--------------------------------------------------------------------------
%Calculating the mean square error
%%WORK IN PROGRESS
% for i=1:n_values
% mod_eig(i)=eig_vec(i);
% end

for i=1:m
for j=1:n
error_reconstruction(i,j)=X(i,j)-Reconstructed_image(i,j);
square_error(i,j)=(error_reconstruction(i,j))^2;
end
end
disp('The Mean square error between Original image and reconstructed image is: ')
MSE=sum(sum(square_error))/(m*n)
disp('PSNR in dB')
PSNR=20*log10(max(max(X))/sqrt(MSE))
%%-------------------------------------------------------------------------
Reconstructed_image=uint8(Reconstructed_image);
figure
imshow(Reconstructed_image)
title('Reconstructed image')
figure
imshow(uint8(X))
title('Original image')



Results
Test 1
enter the no of largest eigen values to be considered: 10
n_values =10
The Mean square error between Original image and reconstructed image is:
MSE =346.6516
PSNR in dB
PSNR =22.1326


Test 2
enter the no of largest eigen values to be considered:
n_values =30
The Mean square error between Original image and reconstructed image is:
MSE =65.8073
PSNR in dB
PSNR =29.3488
Reconstructed image Original image

Test3
enter the no of largest eigen values to be considered: 50
n_values =50
The Mean square error between Original image and reconstructed image is:
MSE =17.1151
PSNR in dB
PSNR =35.1978

Discussion
We have sucessfully implemented a program for finding KLT of the given image and have also
studied the difference in the represantation of the image with different number of eigen values of
the covariance matrix. As the number of eigen values and eigen vectors considered for
reconstruction increases, we will get better reconstruction of the image. This technique can be
effectively used for compression of images. The disadvantage is the calculation of eigen values
and eigen vectores for large matrices.
Reconstructed image
Original image
Reconstructed image
Original image
Lab Experiment-8
Singular value decomposition
Aim:
To perform singular value decomposition (SVD) on any given m x n matrix
Theory:
The singular value decomposition (SVD) is a factorization of any real or complex matix of the
order m x n. Any m x n matrix can be decomposed in the form
A=U V* (here * operation indicates conjugate-transpose operation)
The left-singular vectors U are eigenvectors of AA*.
The right-singular vectors of V are eigenvectors of A*A.
is a diagoanl matrix containing the square roots of the eigenvalues of AA* or A*A, depending
upon whichever is larger
By applying SVD we can get a set of basis functions for the four important subspaces of a given
dimensional vector space.
1 Orthonormal basis for Row space A
2 Orthonormal basis for column space A
3 Orthonormal basis for null space A
4 Orthonormal basis for null space of A
T









SVD
Matlab code
% to implement singular value decomposition

clc
clear
A=input('Enter the Matrix: ');
[m n]=size(A);
A_symm_V=A*A';
A_symm_U=A'*A;
[eig_vec_V eig_val_V ]=eig(A_symm_V);
[eig_vec_U eig_val_U ]=eig(A_symm_U);

n1=max(size(A_symm_U));
n2=max(size(A_symm_V));
temp1=zeros(n1,n1);
temp2=zeros(n2,n2);
for i=1:n1
temp1(i,i)=eig_val_U(n1+1-i,n1+1-i);
end
A_symm_U=sqrt(temp1);
eig_vec_U=fliplr(eig_vec_U);

for i=1:n2
temp2(i,i)=eig_val_V(n2+1-i,n2+1-i);
end
A_symm_V=sqrt(temp2);
eig_vec_V=fliplr(eig_vec_V);

sig=zeros(m,n);
for i=1:m
for j=1:n
if (n1>=n2)
sig(i,j)=A_symm_U(i,j);
else
sig(i,j)=A_symm_V(i,j);
end
end
end
% values to be returned in the case of a function
V=eig_vec_U
sigma=sig
U=eig_vec_V
Results

V =
-0.8944 0.4472
-0.4472 -0.8944
sigma =
3.1623 0
0 2.2361
0 0
U =
0.5657 0.6000 -0.5657
-0.4243 0.8000 0.4243
0.7071 0 0.7071
A =
1 2
-2 1
2 1
Rec_A =
-1 -2
2 -1
-2 -1
Cross check
[U,sigma,V]=svd(A)
U = 0.5657 0.6000 -0.5657
- 0.4243 0.8000 0.4243
0.7071 0.0000 0.7071
sigma = 3.1623 0
0 2.2361
V = 0.8944 -0.4472
0.4472 0.8944
Discussion
We have successfully implemented a program for performing SVD for any given m x n matrix.
Lab Experiment-9-a
Generation of Random variables
Aim: To be familiar pdf/pmf and CDF of different probability distributions.
Theory:
Exponential distribution
The exponential distribution describes the arrival time of a randomly recurring independent event
sequence. is the mean waiting time for the next event recurrence. Its probability density
function is

CDF is given as


Poisson distribution
The Poisson distribution is the probability distribution of independent event occurrences in an
interval. If is the mean occurrence per interval, then the probability of having x occurrences
within a given interval is:

K=0, 1, 2, 3,
Binomial distribution
The binomial distribution is a discrete probability distribution. It describes the outcome of
n independent trials in an experiment. Each trial is assumed to have only two outcomes, either
success or failure. If the probability of a success in a trial is p, then the probability of
having k successful outcomes in an experiment of n independent trials is as follows.

k=0, 1, 2, 3,
Normal distribution
The normal distribution is defined by the following probability density function, where is
the population mean and
2
is the variance.

If a random variable X follows the normal distribution, then we write:

In particular, the normal distribution with = 0 and = 1 is called the standard normal
distribution, and is denoted as N(0,1).
Bernoulli distribution
The Bernoulli distribution is a discrete distribution having two possible outcomes labeled
by n=0 and n=1 in which n=1 ("success") occurs with probability P and n=0 ("failure") occurs
with probability q=1-p, where 0<p<1. It therefore has probability density function


which can also be written as


Rayleigh distribution

A Rayleigh distribution is often observed when the overall magnitude of a vector is related to its
directional component distribution with probability density function and distribution function
given by





Laplace distribution
In probability theory and statistics, the Laplace distribution is a continuous probability
distribution. It is also called as the double exponential distribution, because it can be thought
of as two exponential distributions .The pdf and CDF of this distribution is given by





Probability distributions
Matlab code for continuous Random variables

% Generation of continuous random variables using Inverse Transform
% Generating U(0,1)
clc;
clear all;
close all;
U = rand(1,1000);
% Uniform distribution U(a,b)
a = 1;
b = 100;
X = a+(b-a).*U;
[y,axis_x]=hist(X);
fx = y/1000;
subplot(2,1,1),bar(axis_x,y);
title('Histogram');
subplot(2,1,2),plot(axis_x,fx);
title('Uniform density function');
axis([0 100 0 1]);
% Exponential distribution exp(lambda)
lambda = 5;
X = -lambda*log(U);
[y,axis_x]=hist(X);
fx = y/1000;
figure
subplot(2,1,1),bar(axis_x,y);
title('Histogram');
subplot(2,1,2),plot(axis_x,fx);
title('Exponential density function');
axis([0 100 0 1]);
% Normal distribution N(m,sigma)
m = 5;
sigma = 0.5;
U1 = rand(1,1000);
U2 = rand(1,1000);
X1 = sqrt(-2.*log(U1)).*cos(2*pi*U2);
X = m + sigma.*X1;
[y,axis_x]=hist(X);
fx = y/1000;
figure
subplot(2,1,1),bar(axis_x,y);
title('Histogram');
subplot(2,1,2),plot(axis_x,fx);
title('Normal density function');
axis([0 10 0 0.5]);
% Laplacian distribution with mean 0 and variance 2
U1 = rand(1,1000);
U2 = rand(1,1000);
for n = 1:1000
if U1(n)<= 0.5
X(n) = log(U2(n));
else
X(n) = -1*log(U2(n));
end
end
[y,axis_x]=hist(X);
fx = y/1000;
figure
subplot(2,1,1),bar(axis_x,y);
title('Histogram');
subplot(2,1,2),plot(axis_x,fx);
title('Laplacian density function');
axis([-50 50 0 0.5]);
% Rayleigh distribution R(sigma)
sigma = 10;
X = sigma*sqrt((-1*log(U)));
[y,axis_x]=hist(X);
fx = y/1000;
figure
subplot(2,1,1),bar(axis_x,y);
title('Histogram');
subplot(2,1,2),plot(axis_x,fx);
title('Rayleigh density function');
axis([0 25 0 1]);

Matlab code for discrete Random variables
clc;
clear all;
close all;
U = rand(1,1000);
% Bernoulli distribution of parameter p
p = 0.60;
for n = 1:1000
if U(n)<=p
X(n)=1;
else
X(n)=0;
end
end
[y,axis_x]=hist(X);
fx = y/1000;
subplot(2,1,1),bar(axis_x,y);
title('Histogram');
subplot(2,1,2),stem(axis_x,fx);
title('Bernoulli probability mass function');
axis([0 1 0 1]);
% Binomial distribution B(m,p)
m = 10;
p = 0.50;
X = zeros(1,1000);
% Generating binomial random variable as a sum of m Bernoulli random variables
for i = 1:m
U(i,:)=rand(1,1000);
for n = 1:1000
if U(i,n)<=p
Y(i,n)=1;
else
Y(i,n)=0;
end
end
X = X+Y(i,:);
end
[y,axis_x]=hist(X);
fx = y/1000;
figure
subplot(2,1,1),bar(axis_x,y);
title('Histogram');
subplot(2,1,2),stem(axis_x,fx);
title('Binomial probability mass function');
axis([0 10 0 1])
Results










Discussion
We have successfully generated random variables like Uniform, Exponential, Rayleigh and
Laplace distributions from uniformly distributed random variable U in (0,1) using inverse
transform method. Normal distribution was generated using Box-Muller method. Bernoulli
random variable was generated by modeling an experiment which generates it and Binomial
random variable was generated as a sum of Bernoulli random variables.





















Lab Experiment-9-b
Verification of Central Limit Theorem

Aim:
To visualize Central Limit Theorem
Thoery:
The central limit theorem is one of the most remarkable results in probability theory.Let
X1, X2, X3.. be a sequence of independent, identically distributed random variables each
with mean and variance
2
. Then the distribution of





tends to the standard normal distribution as n . That is




n

This theorem holds for any distribution of X
i

Central Limit Theorem
Matlab code
% Verification of central limit theorem
%By Generating n iid random variables following exponential distribution
clc;
clear all;
size=10000;
U = rand(1,size);
lambda = 5;
n = 1000;
X = zeros(1,size);
X_exp=-lambda*log(rand(1,size));
[y_,axis_x_exp]=hist(X_exp);
fx_ = y_/size;
subplot(2,1,1),bar(axis_x_exp,y_);
title('Histogram');
subplot(2,1,2),plot(axis_x_exp,fx_);
title('pdf of exponential random variable')

for i = 1:n
X1= -lambda*log(rand(1,size));
n
n
n
X X X
o
+ + + ....
2 1
dx
a
2
2
x
e
2
1
a}
n
n
n
X ....
2
X
1
X
P{ }

[
s
+ + +
X = X + X1; % generating a sum 'n'of exponential random variables
end
X = (X-mean(X))./sqrt(var(X));
[y,axis_x]=hist(X);
fx = y/size;
figure
subplot(2,1,1),bar(axis_x,y);
title('Histogram');
subplot(2,1,2),plot(axis_x,fx);
title('pdf of the normalised sum random variable')

Results


0 5 10 15 20 25 30 35 40 45
0
2000
4000
6000
Histogram
0 5 10 15 20 25 30 35 40 45
0
0.2
0.4
0.6
0.8
pdf of exponential random variable


Discussion
We have verified the central limit theorem by generating a normalized sum of n iid random
variables (we have taken iid random variables as the exponential random variables in the
program) and observing that this resulting random variable has a Gaussian distribution.









-4 -3 -2 -1 0 1 2 3 4
0
1000
2000
3000
Histogram
-4 -3 -2 -1 0 1 2 3
0
0.1
0.2
0.3
0.4
pdf of the normalised sum random variable