University of South Florida
Industrial & Management Systems Engineering
ESI 4313/6340 Probabilistic Operations Research
Spring 2020
Time/place MW 8:00 - 9:15 am, CIS 1048; F 8:00 - 8:50 am, CIS 1048 (recitations)
[online students: the above times and rooms are for on-campus students only]
Instructor Dr. Alex Savachkin, ENC 2201, [email protected], 813-974-5577; office hours: by appointment
TA Eric Chen, [email protected], WF 11 - 1, ENC 2004
Prereqs EGN 3443, ESI 4007
Text Applied Probability and Stochastic Processes, R. Feldman & C. Valdez-Flores, 2e.
References Developing Spreadsheet-Based Decision Support Systems, S. Eksioglu et al., 2011 (VBA).
Objectives: Learn fundamental models of stochastic processes and their VBA programming applications.
Course grade: midterm 1 - 3 = 100%/3 each (tentative dates: 2/17, 3/30, 4/29).
Grade scale: A = [90, 100+), B = [80, 90), C = [70, 80), D = [60, 70), F = [0, 60) [no +/- grades].
Policies
- Any violation of academic honesty will be dispatched in accordance with the university policies.
- First day attendance: on-campus students - report to class; online students - download the syllabus from Canvas.
- Students in need of academic accommodations for a disability need to contact the SDS Office.
- If a test is missed due to a valid reason, as defined by the university, supporting documentation will be required.
- Online students must have all tests proctored. All tests must be taken on the same day/time with on-campus students
(please refer to USF Testing Services for any related questions).
- For other applicable policies, please refer to the current version of the undergraduate catalog.
Topics
I. Probability review
(sample space; events; probability axioms; conditional probability; law of total probability; Bayes’ formula; independence of
events; discrete/continuous random variables; distribution functions; important probability distributions)
II. Poisson processes
(stochastic processes; counting processes; time-homogeneous Poisson processes; merging and splitting of Poisson processes;
compound Poisson processes; VBA simulation; applications)
III. Queueing systems
(M/M/1 queues: limiting analysis, measures of effectiveness, Little’s Law; VBA simulation; applications)
IV. Discrete-time Markov processes
(one/multi-step transitions; classification of states; state communication, classes, irreducibility; recurrency & transience;
limiting probabilities; long-run average rewards; first passage/recurrence times; VBA simulation; applications)
V. Continuous-time Markov processes
(sojourn times; transition probabilities; birth & death processes; M/M/1 queues; VBA simulation; applications)
VI. Dynamic systems
(sequential decision making; dynamic programming: elements, backward recursion; myopic policies; probabilistic dynamic
programming; Markov decision processes (MDP): evaluation of stationary policies; models & applications)
Exams (topics): M1 (I, II, III, VBA), M2 (IV, V, VBA), M3 (VI, VBA).
Exams are 100% paper based (Excel not allowed), closed book/notes (cheat sheets provided). Recommended exercises and
sample exams are posted in Canvas/Modules.
ESI 4313/6340 schedule (S20)
Date Topics Video (Canvas/Modules)
1/13 Introduction Introduction
[I] random experiments, sample spaces, events;
1/15 Probability review (1)
probability axioms; conditional probability
1/20 MLK Day -
[I] conditional probability; total probability rule; Bayes'
1/22 Probability review (2)
formula; independence of events
[I] discrete/continuous rvs; exponential distribution;
1/27 Probability review (3)
binomial distribution (self-study)
[II] stochastic processes; counting processes; Poisson
1/29 Poisson processes (1)
processes: formulation, definition, examples
[II] PP: distribution of arrival times; splitting/merging;
2/3 Poisson processes (2)
compound PP; VBA simulation
[III] M/M/1 queues: introduction, limiting probabilities,
2/5 Queueing systems (1)
measures of effectiveness
2/10 [III] M/M/1 queues: Little's Law; VBA simulation Queueing systems (2)
2/12 M1 sample M1 sample
2/17 M1 -
2/19 [IV] MC: Markovian property; one-step transitions Markov chains (1)
2/24 [IV] MC: multi-step transitions Markov chains (2)
2/26 [IV] MC: state probabilities; rewards Markov chains (3)
[IV] MC: classification of states: state communication,
3/2 Markov chains (4)
classes, irreducible MC; state recurrency/transience
3/4 [IV] MC: limiting probabilities Markov chains (5)
3/9 [IV] MC: long-run average rewards; VBA simulation Markov chains (6)
3/11 [IV] MC: first passage times Markov chains (7)
3/16 Spring break -
3/18 Spring break -
[V] Markov processes: sojourn times, transition
3/23 Markov processes (1)
probabilities; birth & death processes; M/M/1 queues
3/25 M2 sample M2 sample
3/30 M2 -
[VI] dynamic programming: elements; optimal paths in
4/1 Dynamic systems (1)
directed networks; backward recursion; myopic policies
4/6 [VI] dynamic asset replacement model Dynamic systems (2)
4/8 [VI] production-inventory planning model Dynamic systems (3)
[VI] probabilistic DP (PDP); action-timing problems:
4/13 Dynamic systems (4)
game of chance (optimal stopping strategy)
4/15 [VI] PDP: production scheduling Dynamic systems (5)
4/20 [VI] MDP: evaluation of stationary policies (ETDC) Dynamic systems (6)
4/22 [VI] MDP: evaluation of stationary policies (LREAC) Dynamic systems (7)
4/27 M3 sample (self-study) -
4/29 M3 -
