CSL2050: Pattern Recognition & Machine Learning

End Semester Exam

IIT Jodhpur
Maximum Points: 60 Duration: 105 minutes

April 26, 2022

Instructions:
- This is a closed book exam and use of any resource is not allowed.
- Solve the questions on paper and upload a properly scanned PDF on Google classroom.
- Upload the answers of the first two questions (Q1 and Q2) separately and the remaining
questions separately.
- Any instance of cheating will be considered as academic dishonesty and appropriate penalty
will be applied.

________________________________________________________________________________________________

Q1) Answer the following in True/False. Provide proper justification for the same. (2 marks each)
A. Will the optimal lines fitted using the PCA and Linear Regression algorithms always be the
same?
B. Consider the following two statements, which of the statements is True/False
S1: A single perceptron is powerful enough to classify linearly-separable sample points.
S2: If we plot (age, weight) of class 11 and class 12 of all the students in Jodhpur, we will get
linearly separable points.
C. The Naive Bayes Classifier has a piecewise quadratic decision boundary.
D. In a two class classification problem, a point on the Bayes optimal decision boundary x ∗
always satisfies P (y = 1|x ∗ ) = P (y = 0|x ∗ )
E. The loss function that helps minimize the margin is called hinge loss.
F. Consider the following two statements, Which of the statements is True/False
S1: Hard margin — if the training data is linearly separable, we can select two parallel
hyperplanes that separate the two classes of data, so that the distance between them is as
large as possible.
S2: Soft margin — doesn’t allow some data points to stay on either the incorrect side of the
hyperplane and between margin and the correct side of the hyperplane.
G. If a MAP classifier is designed using discriminant functions gi(x), the output value from each
discriminant function must be greater than zero, i.e. gi(x) > 0 for all i.
H. With zero training samples, Bayesian estimate of a parameter is always zero.
I. Non-linear transformation with SVM always performs better than linear transformation.
J. Lower learning rate leads to underfitting of a neural network.

Q2) Write brief answers for the following questions. (2 marks each)
A. What are the conditions in which the estimations of MLE and Bayesian estimation differ and
why?
B. Provide two key differences in PCA, LDA and ICA.
C. What are the minimum and maximum values possible for covariance, correlation, cumulative
joint distribution, and F-score?
D. When Euclidean distance is equal to Mahalanobis distance.
 3×3
Q3) Let A ∈ R be a matrix such that A(i, j) = ij for all i, j = 1, 2, 3. Let v1 be the eigenvector of A
corresponding to the largest eigenvalue and let v3 be the eigenvector of A corresponding to the
T
smallest eigenvalue. Find the value of the quantity 16v1 v3. (4 marks)

Q4) Let x have an exponential density (4 marks)

Suppose that n samples x1, ..., xn are drawn independently according to p(x|θ). Compute the
maximum likelihood estimate for θ.

Q5) With reference to SVM Kernel/Hyperplanes, answer the following questions. (8 marks)
a) Suppose a dataset contains 100 positive and negative samples each. Further, suppose the
positive samples are at radius 2, and negative samples are at radius 4 from the origin. Find
out the transformation such that they become linearly separable. Also find out the
hyperplane separating positive and negative samples in the above case. Write down its
equation. How many support vectors will be there in this example.
b) Suppose two lines passing through (10,0) and (11,0) respectively are parallel to each other
and they make 45 degrees from the X-axis. Find out the width between these two lines.
c) Show that k(a,b) = (a^T * b)² is a valid kernel, where a and b are two-dimensional data points..

Q6) With respect to Perceptron answer the following question. (6 marks)

Can the complement of the following function be represented using perceptron. If yes, what will
be the decision boundary? If not, explain why. According to the last digit of your roll number,
please solve the correct problem.
[For odd roll numbers]:
y = x1’.x2’.x3 + x1.x2’.x3 + x1.x2.x3’ + x1’.x2.x3 (1)
[For even roll numbers]:
y = x1’.x2.x3 + x1.x2’.x3 + x1.x2.x3’ + x1’.x2.x3’ (2)
here ’ represents prime (compliment), “·” and “+” represents AND and OR respectively.

Q7) Consider the below example with 6 data points. (6 marks)

Suppose the number formed by the last two digits of your roll number is M, the day of your birth is
N and the month is K. (For someone with roll number B20HS031 with DOB 23rd May, M=31,
N=23, K =5). Calculate the magnitudes of x and y Coordinates of the points as follows:

Point x Coordinate y Coordinate

p1 0.5*M + 0.2*N - 0.5*K 0.5*M + 0.2*N + 0.2*K
p2 0.1*M + 0.5*N - 0.5*K 0.5*M - 0.2*N - 0.1*K
p3 0.2*M - 0.2*N - 0.5*K 0.1*M - 0.2*N + 0.5*K
p4 0.2*M + 0.1*N - 0.1*K 0.5*M - 0.1*N - 0.5*K
p5 0.2*M - 0.2*N + 0.1*K 0.5*M + 0.2*N + 0.1*K
p6 0.1*M + 0.1*N - 0.5*K 0.5*M - 0.2*N + 0.2*K
Table A

i) Show the points on a 2D plot (hand-drawn).

ii) Apply K-means clustering algorithm to find the appropriate clusters using Table A. Use K = 2.
Show all the steps and report the cluster centers and cluster id for each point.
iii) Evaluate the quality of the cluster you have obtained with SSE.

Q8) In a three class, 2-D problem, the feature vectors are normally distributed with the following
covariance matrix. The mean vectors for each class are [0.1, 0.1]^T , [2.1, 1.9]^T and [−1.5, 2.0]^T .
Assuming that the classes are equiprobable, classify the feature vector [1.6, 1.5]T according to the
Bayes minimum error probability classifier. (4 marks)