Scribd
Upload
21 views3 pages

RL Sem 8

The document outlines the curriculum for a Reinforcement Learning course, detailing six modules covering prerequisites, key concepts, algorithms, and applications. It emphasizes important topics such as Markov Decision Processes, Dynamic Programming, and Monte Carlo Methods, along with their relevance to previous exam questions. Additionally, it provides strategic tips for effective study, highlighting the need to focus on concept-heavy modules and practice numerical examples.

Uploaded by

abcdx5795
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
21 views3 pages

RL Sem 8

The document outlines the curriculum for a Reinforcement Learning course, detailing six modules covering prerequisites, key concepts, algorithms, and applications. It emphasizes important topics such as Markov Decision Processes, Dynamic Programming, and Monte Carlo Methods, along with their relevance to previous exam questions. Additionally, it provides strategic tips for effective study, highlighting the need to focus on concept-heavy modules and practice numerical examples.

Uploaded by

abcdx5795
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 3

RL SEM 8

✅ Module 0: Prerequisites
 Probability distributions and expected values
 Basic linear algebra (e.g., inner products)

✅ Module 1: Introduction to Reinforcement Learning


Important Topics:
 Key features and elements of RL (Agent, Environment, Policy, Reward,
Value Function, Model)
 Types of Reinforcement Learning: Model-free vs Model-based, On-policy vs
Off-policy
 Reward hypothesis
 Algorithms:
o Q-Learning

o SARSA

Previous paper relevance:


Q1(a) – Types of RL
Q4(a) – Q-learning Algorithm

✅ Module 2: Bandit Problems and Online Learning


Important Topics:
 n-Armed Bandit Problem
 Action-Value Methods
 Handling Nonstationary Problems:
o Incremental implementation

o Tracking via step-size parameters

 Exploration Strategies:
o Optimistic Initial Values

o Upper-Confidence-Bound (UCB)

o Gradient Bandits

Previous paper relevance:


Q2(a) – Slot machine scenario (bandit problem)
Q4(b) – Exploration-exploitation trade-off
✅ Module 3: Markov Decision Processes (MDPs)
Important Topics:
 Agent–Environment Interface
 Goals and Rewards
 Returns and Discounting
 Markov property
 MDP Components:
o States, Actions, Transition Probabilities, Rewards

 Value Functions and Optimal Value Functions


Previous paper relevance:
Q6(a) – Components of an MDP

✅ Module 4: Dynamic Programming


Important Topics:
 Policy Evaluation (Prediction)
 Policy Improvement
 Policy Iteration
 Value Iteration
 Asynchronous Dynamic Programming
 Generalized Policy Iteration
Previous paper relevance:
Q1(b) – Methods for policy evaluation
Q3(b) – Advantages and disadvantages of asynchronous updates

✅ Module 5: Monte Carlo Methods and TD Learning


Important Topics:
 Monte Carlo Prediction
 Estimation of Action Values
 Monte Carlo Control
 First-visit vs Every-visit MC methods
 TD Prediction
 TD Control using Q-Learning
Previous paper relevance:
Q1(c) – TD vs Monte Carlo
Q2(b) – MC control scenario
Q5(a) – First-visit vs every-visit MC

✅ Module 6: Applications and Case Studies


Important Topics:
 Elevator Dispatching
 Dynamic Channel Allocation
 Job-Shop Scheduling
 Industrial/Real-world RL applications
Previous paper relevance:
Q1(d) – Elevator dispatching
Q5(b) – Scheduling algorithms

📌 Strategy Tips:
 Focus more on Modules 2 to 5 (they are both concept-heavy and
application-focused).
 Practice numerical examples like reward computation and value
iteration.
 Revise terminology and differences (e.g., TD vs MC, SARSA vs Q-
Learning).
 Prepare case study applications for Module 6 to write high-scoring
descriptive answers.

You might also like