NPTEL

Video Course on Machine Learning

Professor Carl Gustaf Jansson, KTH

Assignments for Week 5 2024

9 tasks with a total of 20 marks

Assignment tasks - Week 5 2024

Problem # 1 Correct Marks: 2 Theme: Explanation-based Learning

Explanation Based Learning typically takes a predefined Domain Theory and modifies it in such a way that
new more complex rules are created as specific combinations of several existing rules. The goal is to make
problem-solving more efficient and the rule formation process is guided by the problem examples considered
for training. As part of the generalization process for the newly constructed rules a specific kind of pattern
matching technique is used. Which is the established term for this technique?

A. Variable matching B. Resolution C. Recursion D. Unification

Answer: D.
 Assignment tasks - Week 5 2024
Problem # 2 Correct Marks: 3 Theme: Explanation-based Learning
Domain Theory:
Safe to travel(P,F,C) :- Healthy(P), Error free(A), No epidemics (C).
Error free(A):- Passed an inspection (A). Errorfree(A):- Worked when driven same day (A).
Healthy(P) :- Had no symptoms for three weeks (P). Healthy(P) :- Passed an health test(P)
Passed a health test (X):- On line test (X). Passed a health test (X):- Clinic test (X).
Passed an inspection (X):- Company inspection(X). Passed an inspection(X):- Authority inspection(X).
No epidemics (C):-No WHO alert (C).
Operational predicates:
Worked when driven last same day, Had no symptoms for three weeks, On line test, Clinic test, Company inspection, Authority
inspection, No WHO alert
Training Example:
Safe to travel (John, IndiaAir2020, India)
Passed a health test(John), Authority inspection(IndiaAir 2020), No WHO alert (India).
Goal concept:
Safe to travel(P, F, C)

Explain and generalize from Safe To Travel( John, IndiaAir2020, India). How does the resulting rule look like after applying an EBL
algorithm?

A. Safe to travel(John,IndiaAir2020,India)) :- Healthy(John), Error free(IndiaAir2020), No epidemics (India).

B. Safe to travel(John, IndiaAir2020, India):-Passed a health test (John), Authority inspection(IndiaAir 2020), No WHO alert
(India).
C. Safe to travel(P, A, C):-Passed a health test (P), Authority inspection(A), No WHO alert (C).
D. Safe to travel(P,A,C)) :- Healthy(P), Error free(A), No epidemics (C).
Answer: C
Assignment tasks - Week 5 2024
Problem # 3 Correct Marks: 2 Theme: Logic. Logic Programming and Inductive Logic Programming

Which of the following inference schemes correspond to Abduction?

Which of the following inference schemes correspond to Induction?
A B C

Answer: B
 Assignment tasks - Week 5 2024 Problem # 4

Problem #4 Correct Marks: 3 Theme: Logic. Logic Programming and Inductive Logic
Programming
ILP algorithms construct a lattice of clauses fabricated from the building blocks (predicates, variables, constants) specific to
each induction case and navigate that lattice in order to establish an Hypothesis set that fits the current set of examples. ILP can
work in two fashions: Top Down and Bottom Up, establishing specialization or generalization relations in the lattice.

Case
Hypotheses: insect(X).
Positive Examples:
insect(fire brat), insect(green drake), insect(cricket)
Background knowledge:
Has six legs (fire brat),
Has six legs (green drake), Has wings(green drake),
Has six legs (cricket), Has wings(cricket), Can fold wings(cricket),

How does the hypotheses set of clauses look like after a first iteration of a Top Down ILP algorithm based on the first
positive example insect(fire brat) ?

A. insect(X):-Has six legs(X).

B. insect(X):-Has six legs(X), insect(X):-Has wings(X), insect (X):-Can fold wings(X).
C. insect(fire brat):-Has six legs(fire brat).
Answer: A
Assignment tasks - Week 5 2024

Problem #5 Correct Marks: 1 Theme: Case-based Reasoning

Instance Based Learning techniques are typically used in one of the main
phases of Case Based Reasoning. Which?

A. RETRIEVE
B. REUSE
C. RETAIN
D. REVISE

Answer: A
Assignment tasks - Week 5 2024

Problem #6 Correct Marks: 2 Theme: Case-based Reasoning

Revised case
X (store)
Apply Z
New Case

W (Apply to case)

Case Base Relevant Case

Y
A XYZW = RETRIEVE REUSE RETAIN REVISE
B XYZW = RETRIEVE REVISE REUSE RETAIN
C XYZW = RETAIN REVISE REUSE RETRIEVE
D XYZW = RETAIN RETRIEVE REVISE REUSE

Answer: D
Assignment tasks - Week 5 2024

Problem #7 Correct Marks: 1 Theme: Reinforcement Learning

6
A D
A dynamic programming approach to calculating the 5 12 10 10
shortest distance from S to T gives the result= 14. 4 13 15
S B E T
Which result would a greedy forward search give?
6 18 4
C F
4

A. 26 B. 21 C. 14

Answer: A.
Assignment tasks - Week 5 2024

Problem #8 Correct Marks: 3 Theme: Reinforcement Learning

Monte Carlo Reinforcement Learning MC methods learns directly from samples of complete episodes of experience.
• First-visit MC: average returns only for first time s is visited in an episode.
• Every-Visit MC: average returns for every time s is visited in an episode.

Simplified Algorithm for first-visit Monte Carlo

Initialize state-value functions. Return list(s) ← empty list
2. For all s in all episodes E
3. return = sum of rewards r in episode from state to termination.
4. if this is the first occurrence of this state s add the calculated return to the returns list (s).
5. Calculate values of all s as average over all return lists (s).

Example: An undiscounted Markov Reward Process with two states A and B. The Transition matrix and reward function are
unknown.
Two sample episodes: E1 A->A, r=2 A->B, r=7 B->A, r= -7 A->B, r=7 B->terminate, r= -2.
E2 B->A, r= -7 A->B, r=7 B->terminate, r = -2

Which are the estimated values of A and B after considering the two episodes using the simplified first-visit algorithm?

A. V(A)= 6,V(B)= -2 B. V(A)= 7,V(B)= -2 C. V(A)= 6,V(B)= 2 D. V(A)= 7,V(B)= 2

Answer: A
 Assignment tasks - Week 5 2024
Problem #9 Correct Marks: 3 Theme: Reinforcement Learning
Q-Learning Initialize Q(s, a) arbitrarily
algorithm Repeat (for each episode)
Initialize s
Repeat (for each step of the episode)
Take action a, observe r, s’ Q(s, a) = Q(s, a) + α[r + γ max a’ Q(s’, a’) – Q(s, a)] ; s = s’
Example r=-4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
1 2 3 4 5
N -2 -2 -2 -2 -2 -2 0 0 0 2 -2 0 0 0 -2 -2 -2 -2 -2 -2

r=4 6 7 8 9 10 S -2 -2 -2 -2 -2 -2 0 0 0 2 -2 0 0 0 -2 -2 -2 -2 -2 -2

´
11 12 13 14 15 W -2 -2 -2 -2 -2 -2 0 0 0 2 -2 0 0 0 -2 -2 -2 -2 -2 -2

´
16 17 18 19 20 E -2 -2 -2 -2 -2 -2 0 0 0 2 -2 0 0 0 -2 -2 -2 -2 -2 -2

r=1

In this example, we have a 4X5 board. The start state is marked above with a blue dot. The rewards for all positions are also
marked (rewards being 1, -4 and 4). The positions on the rim of the board are terminal states. The board elements are labelled from 1 to 20
as in the mid figure. Moves are N, S, E and W. We consider two episodes: 1. 12:E->13:E->14:E->15 and 2. 12:N->7:E->8:E->9:E->10.
What are the updated versions of the Q elements for state 12, 13, 14, 7,8 and 9 after considering these two episodes? α = γ = 1.
A. 12,E=0 13,E=0, 14,E= -4 12,N=0, 7,E=0 8,E=0 9,E = 4
B. 12,E=1 13,E=1 14,E= -6 12,N=1 7,E=1 8,E=1 9,E= 6
C. 12,E=1 13,E=2 14,E= -6 12,N=1 7,E=2 8,E=3 9,E= 6
D. 12,E=1 13,E=2 14,E= -2 12,N=1, 7,E=2 8,E=3 9,E=7
Answer: B