Scribd
Upload
112 views2 pages

Syllabus

This document summarizes the EECS 558: Stochastic Control course for Fall 2023. The course will be held in person on Tuesdays and Thursdays from 10:30-12:00 in DOW 1017. It will cover various topics in stochastic control systems over 15 weeks. Evaluation will be based on homework assignments, a midterm exam, a group project, and class participation. The goal of the course is for students to develop an understanding of analyzing and optimizing controlled stochastic systems.

Uploaded by

Ấm Lee
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
112 views2 pages

Syllabus

This document summarizes the EECS 558: Stochastic Control course for Fall 2023. The course will be held in person on Tuesdays and Thursdays from 10:30-12:00 in DOW 1017. It will cover various topics in stochastic control systems over 15 weeks. Evaluation will be based on homework assignments, a midterm exam, a group project, and class participation. The goal of the course is for students to develop an understanding of analyzing and optimizing controlled stochastic systems.

Uploaded by

Ấm Lee
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

EECS 558: Stochastic Control (Fall 2023)

ˆ Lectures: TTh 10:30-12:00, DOW 1017 (IN PERSON ONLY!)

ˆ Faculty Instructor: Achilleas Anastasopoulos, 4411 EECS, [email protected], (734) 615-4024,


Office hours: Monday: 1:30-3:00pm, Wednesday 1:30-3:00pm.

ˆ Prerequisites: EECS 501 or equivalent graduate level course.


We will make heavy use of random variables and random processes, especially Markov chains.

ˆ Goal: The goal of this course is for you to develop a conceptual and practical understanding of
analysis and optimization of controlled stochastic systems.

ˆ Topics:

1. Introduction, scope, big picture, application examples (2 lectures)


2. State, observation and control random processes (2 lectures)
3. Review of random processes and Markov chains (2 lectures)
4. Markov Reward Processes (MRP) (4 lectures)
5. Finite horizon Markov Decision Processes (MDP). Dynamic programming. Applications (7
lectures)
6. Partially Observed MDP (POMDP) (2 lectures)
7. Linear Quadratic and Gaussian control (3 lectures)
8. Infinite horizon: discounted, total cost, average cost per unit time, stopping time problems (3
lectures)
9. Approximate Dynamic Programming and Learning (3 lectures)
10. Advanced topics including Dynamic Teams and Games (2 lectures)

ˆ Textbook: P. R. Kumar and P. Varaiya, “Stochastic Systems: Estimation Identification and Adap-
tive Control”.
The textbook is available for free on Canvas

ˆ References:

1. M. L. Puterman, “Markov Decision Processes: Discrete Stochastic Dynamic Progrsamming,”


Wiley, 1994.
2. D. P. Bertsekas, “Dynamic Programming and Optimal Control (Vol I and II),” Athena Scientific,
1995 (3rd Edition)
3. P. Whittle, “Optimization Over Time: Dynamic Programming and Stochastic Control (Vol I),”
Wiley, 1982
4. E. L. Porteus, “Foundations of Stochastic Inventory Theory,” Stanford Business Books, 2002

ˆ Grading:
20% Homework
35% Midterm exam
40% Project
5% Class Participation.

1
ˆ Homework: Total of 6 to 8 assignments.
All HW assignments/solutions will be distributed through Canvas. HWs will be submitted through
gradescope. Please familiarize yourselves with gradescope before the submission of the first HW.
All HWs will be due on Thursday at 11:00pm. Late homework will not be accepted, except in
extenuating circumstances such as serious illness.
I expect that the homework will be neat, well-organized, and professionally completed. Only a
randomly selected subset of each assignment will be graded.

ˆ Exams:
Midterm (in class or take home; to be decided) on Thursday, October 19.
NO FINAL EXAM
Make up exams will not be granted.

ˆ Project: The project includes a written report (10 pages, 12pt, 1inch margins, single space, single
column, plus an Appendix with more details if required, plus slides of your short presentation) and
a short (10-15min) presentation in front of the class during the last couple of lectures (possibly also
extended to some additional afternoon sessions during the last two weeks of classes, if more time is
needed).
You should form groups of exactly two people and choose either (i) two related theoretical papers
and provide a critical review (not just a summary), or (ii) an application and formulate/study the
problem using analytic/numerical techniques.
A one-page proposal for your project is due on October 26 through Gradescope. You are required
to discuss your proposal with me before submitting your final proposal.
The final report is due on December 14 through Gradescope.

ˆ Honor Code: All undergraduate and graduate students are expected to abide by the College of
Engineering Honor Code as stated in the Student Handbook and the Honor Code Pamphlet.

ˆ Students with Disabilities: Any student with a documented disability needing academic adjust-
ments or accommodations is requested to speak with the instructor during the first two weeks of
class. All discussions will remain confidential.

You might also like