Week 5 Tutorial
Week 5 Tutorial
Question 1
equation.
for a loss function R. The term risk appears again here because decision theory
also applies to the choice of estimator.
Show that
b = Bias2 (θ)
R(θ) b + V(θ).
b
Question 2
(b) What is the possible advantage of using a nonparametric method such as KNN
over a parametric approach such as a linear regression?
Yi = µ(xi ) + εi , i = 1, . . . , n,
where each εi is a random error with mean zero and variance σ 2 that is indepen-
dent from everything else. Furthermore, assume that the training inputs are fixed
(therefore, all randomness comes from the errors).
(d) Based on the effective number of parameters, how does the model complexity
change as a function of n and k?
(e) Consider a test case x∗ . Derive the bias of fb(x∗ ) for estimating µ(x∗ ). How does
it change as a function of k?
(f) Derive the variance of the estimator fb(x∗ ). How does it change as a function of k?
Question 3
A few years ago, users found that Google Photos was automatically tagging some people
Page 2
as “gorillas”, generating negative publicity for the company. A link to the story is on
the tutorial page.
One way to prevent failures of this type is to allow for a reject option in the classifier. In
this case, the algorithm can decline to provide an answer if it’s not sufficiently confident
in the prediction.
Suppose that the possible labels are Y = 1, . . . , C and the actions are A = Y ∪ {0},
where action 0 represents the reject option. Define the loss function:
0 if y = a
L(y, a) = `r if a = 0 .
` if y 6= a
e
Page 3