Department of Statistics

Study guide

WST312 Stochastic Processes

Updated 2 March 2024

© Copyright reserved
 Studyguide/General Information

1. Introduction
1.1 Welcome!

Stochastic processes are mathematical models used for a wide variety of processes in the
physical, biological, economic sciences and actuarial fields. The work done in this course
covers part of the Institute of Actuaries course CT4/Actuarial Society of South Africa
course A202 on models. The rest of CT4/A202 is covered in a course on Survival Models
which is offered and must be done in the same year.

1.2 Educational approach

A strict mathematical approach is followed during the course and students must ensure
that they keep up to date with their work. It is strongly suggested that the student reads
through the next week’s work in advance. Students are expected to take responsibility for
their work and keep up to date in order to be successful in this module as work follows
on from the previous week. There are quite a number of theorems and proofs throughout
the course. These form a vital part of the course and the student is expected to know them
(those not required to be known will be indicated) and understand them. Although the
tests/semester tests will consist of a large percentage (60%) of proofs of these theorems
it is also required that the student be able to apply them all to a practical situation (40%).

There are quite a number of theorems and proofs throughout the course. These form a
vital part of the course and the student is expected to know them (those not required to
be known will be indicated) and understand them. Theory work will be tested at a level
that requires understanding of theorems and proofs – do not just learn the material off
by heart.

Although the test/semester tests will consist of a large percentage (60%) of proofs of
these theorems it is also required that the student be able to apply them all to a practical
situation (40%). It is vitally important a student approach the theory simultaneously to
the practical work and not as a nice-to-have extra add-on.

ClickUp will be used extensively for consultation, notes, handins, attendance, marks and
announcements. It is the student’s responsibility to ensure the email address they
regularly check is linked to their ClickUp. In addition, we will use GradeScope.com for
tests. You will be registered on GradeScope with your @tuks.co.za email address. Please
go set your password etc. there once we have registered you. Do not use a different email
address – otherwise your marks won’t sync to ClickUp.

We are not only facilitating learning in a module, we are also preparing you for the world
of work. We expect you to adhere to a code of conduct. Students are expected to email
lecturers and fellow classmates in a professional manner. When you send an email to your
lecturer, you have to use a respectful tone and include all the following aspects:
o A clear and explanatory subject line (e.g. “Submission of sick note – P
Mduli”);
 o Your full name and surname at the end of the mail;
o Your student number;
o The module involved; and
o Short and clear message.

Consultation with lecturers and assistants is via the discussion boards on ClickUp each
week. No emails consulting on the work will be replied to. No assistants may be emailed
for any matter.

Participation is compulsory. An attendance mark, counting towards your semester mark,

will be captured through engagement on the weekly discussion boards. No admin
discussion boards will count towards your attendance mark.

Memos are not provided for any practicals or tests. The student will have sufficient time
to cover the practicals during the compulsory practical sessions as well as discussion
boards and classes to complete it successfully. Test memos will be covered in class.

All discussion boards require engagement from students – answers by lecturers and
assistants will encourage thinking and not always give the answer immediately. Please
engage further in such cases to facilitate learning. Posts on discussion boards that are not
engaging e.g. “Thank you for your post”, or simply “I agree” will be deleted and not be
counted for marks.

Queries on marks allocated must be submitted by the date indicated either via
Gradescope or the discussion board allocated for this purpose. No alterations will be
discussed after that date.

Exam entrance is not negotiable.

All issues must go through the module coordinator Prof Fabris-Rotelli. Please take note of
UP’s Escalation Policy. Also take note of the general departmental Studyguide available
at www.up.ac.za/statistics

All queries via discussion boards on ClickUp. Confidential matters will be on boards
with moderated posts. Emails are reserved for extreme situations.

2. Administrative Information

Any information that students should be aware of that is not included in the Study Guide
will be communicated via ClickUP.

2.1 Contact Details

Course coordinator and
lecturer: Prof I Fabris-Rotelli
Email: [email protected]
Lecturer: Mr Andre Kleynhans
 Email: [email protected]

Assistants: Mr Arthur Antonio, Ms Annette Pienaar

See ClickUP for their consultation hours via discussion
boards.

2.2 Timetables
It is important to engage on discussion boards. Discussion boards will be responded to
regularly (see schedule on ClickUp) Monday 9am – Friday midday. The discussion boards
will be open each week 9am on Monday to midday on Friday.

Lectures and Practicals: Monday 11:30 – 12:20 (lecture), Tuesday 13;30 – 14:20
(Practical in the lab), Wednesday 10:30 – 13:20 (lecture, practical lecture, and tutorial)

2.3 Study Material

Notes for the course will be provided on ClickUp. You can use them electronically, or print
them yourself.

3. Assessments

There are two semester tests, and an additional third semester test if you miss either one
– please confirm details on your Student Portal – these are written on campus and are
closed-book.

The third semester test can only be written if you miss one of the semester tests 1 or 2. It
will cover both semester tests’ scope and will take place shortly after the second semester
test. A doctor’s note is not required to write the third semester test, however you must
sign up to write semester test 3. There will be no sick test for semester test 3.

For the examination: Sick notes must be handed in directly to faculty administration via
your portal. Valid original sick notes are accepted if issued by a medical doctor
registered at the Health Professions Council of South Africa (HPCSA). The only other type
of sick note that is accepted are those issued by an Advanced Practice Nurse (a registered
nurse with a postgraduate qualification) as determined by the South African Nursing
Council who has a BHCF practice number, provided that the diagnosis falls only within
their specific field of specialisation. See www.sanc.co.za for the specific requirements.
An affidavit will only be accepted if supported by substantiating documentation, e.g. case
report or criminal charge with case number obtained from a police station, valid medical
certificate for injuries, a death certificate for a funeral, etc. Please note that submission of
fraudulent sick notes and affidavits is a criminal offense, which will lead to disciplinary
action and may result in dismissal.

THE DISHONEST MISSING OF A TEST AS WELL AS DISHONESTY DURING THE WRITING

OF TESTS AND EXAMINATIONS WILL NOT BE TOLERATED UNDER ANY
CIRCUMSTANCES. ALL IRREGULARITIES WILL BE SEEN IN A SERIOUS LIGHT AND WILL
BE REPORTED TO THE REGISTRAR (ACADEMIC). KNOWING OF DISHONESTY WITHIN
THE CLASS AND NOT REPORTING IT IS ALSO CONSIDERED DISHONESTY AND WILL
ALSO BE REPORTED. CASES OF DISHONESTY WILL RESULT IN A 0 FOR THE RESPECTIVE
ITEM.

Practicals: Each practical will have an online practical test to complete forming part of
the semester mark. Practicals should be covered by the student each week on their own
and consultation done on the respective discussion board and in the Tuesday practical
class session. The practicals will cover R, RStudio and RMarkdown as well as a Latex
component in RMarkdown. Four practical tests will be written during the semester based
on the practicals covered during the semester. The practicals will be available from
Wednesday each week, discussed in class on Wednesdays and should be completed by
midnight on the following Tuesday (the online test will close Tuesday at midnight). You
will have two attempts for the online test but the final mark will be the average of the
attempts. This is to account for technical issues. No tests will be reset.

Online Tests:
Each week two online tests will be available on ClickUp. The first will be a preparation
test for the work to be covered in the coming week and the second a test based on what
was covered during the week (both will be available from Friday 9am to Sunday at
midnight). You may use your notes for these tests but it is advised you try without your
notes to properly test your understanding. These tests cannot be reopened once they
close. Ensure you follow the dates and times closely. You will have two attempts for these
online tests but the final mark will be the average of the attempts. This is to account for
technical issues. No tests will be reset.

For all methods of assessment, it is expected of the student to use strict notation
and mathematical reasoning whenever applicable. The notation in the notes
should be used and will be followed. If another notation is used it must be clearly
indicated, explained and defined. This module carries a weight of 18 credits, thus you
should spend a total of 180 hours during the semester on this course to master the
required skills (including lectures and practicals). There are about 16 weeks of university
lectures thus 7 hours (excluding lectures and practicals) per week is required from you
for your own study time. NOTE: This course requires you to use all knowledge gained
in the first two years of your studies. As a third-year student you should be able to,
and are expected to, show your understanding of all concepts very clearly. The ability
to present a proof or mathematical argument is very important for the degree you
are obtaining thus this ability should be shown clearly in your tests etc.

3.1 Calculation of the Semester Mark

The semester mark will be calculated as follows (all components are compulsory):

Pre- and Post-Tests 2 online tests each week 5%

Weekly online practical 1 online test for each practical 5%
Class Attendance Random attendance is taken in classes 5%
A minimum of 2 tests will count (3 or 4 will
Practical Tests count if it improves the total) 30%
 Semester Test 1 25%
Semester Test 2 25%
Engagement on discussion
boards every week Engagement on a discussion board every week 5%

The semester examination date and length will be confirmed (see your student portal). A
minimum of 40% in the examination is required to pass the course. The final mark is a
weighted average of the semester mark and the examination mark – maximum mark by
weighting either 50:50 or 40:60 for the semester mark and the examination mark
respectively.
Actuarial Science students should note that no special or sick examinations can be
taken into consideration for exemption purposes. This is in accordance with UP’s
exemption agreement with the Institute/Faculty of Actuaries and the Actuarial Society of
South Africa. Full details on exemptions can be obtained from the Department of
Insurance and Actuarial Science, Room 1-10, Mathematics Building. The University’s
regulations are as follows with regard to sick letters submitted:

3.2 Plagiarism

Plagiarism is a serious form of academic misconduct. It involves both appropriating

someone else’s work and passing it off as one’s own work afterwards. Thus, you commit
plagiarism when you present someone else's written or creative work (words, images,
ideas, opinions, discoveries, artwork, music, recordings, computer-generated work, etc.)
as your own. Only hand in your own original work. Indicate precisely and accurately
when you have used information provided by someone else. Referencing must be done in
accordance with a recognised system. Indicate whether you have downloaded
information from the Internet. For more details, visit the library’s website at:
http://www.library.up.ac.za/plagiarism/index.htm

3.3 Schedule
Fabris-Rotelli
Kleynhans
Week Dates Monday Tuesday Wednesday Thursday Friday
February
1 Lecture 2, 3
Lectures 1 - Lectures 1 -
19 to 23 Lecture 1 Practical 0 Tutorial 1
3 3
Practical 1
2 Lecture 5, 6
26 Feb to 1 Lectures 4 - Lectures 4 -
Lecture 4 Practical 1 Tutorial 2
March 6 6
Practical 2
March
3 Lecture 8, 9
Lectures 7 - Lectures 7 -
4 to 8 Lecture 7 Practical 2 Tutorial 3
9 9
Practical 3
4 Lecture 11,
Lectures 10 Lectures 10
11 to 15 Lecture 10 Practical 3 12
- 12 - 12
Tutorial 4
5 Consultation
Class Thursday Friday Public
18 to 22 Recess
Semester Timetable Timetable Holiday
Test 1 17:30
6 25 to 29 Recess Recess Recess Recess Recess
April
7 Lecture 13,
Public Practical test 14 Lectures Lectures
1 to 5
Holiday 1 Tutorial 5 13, 14 13, 14
Practical 4
8 Lecture 16,
17
Lectures 15 Lectures
8 to 12 Lecture 15 Practical 4 Tutorial 6
- 17 15 - 17
Practical 5

9 Lecture 19,
20
Practical 5 Lectures 18 Lectures 18
15 to 19 Lecture 18 Tutorial 7
-20 - 20
Practical 6

10 Lectures
Practical 6 Lectures 21 Lectures 21
22 to 26 22, 23
Lecture 21 - 23 - 23
Tutorial 8
11 Wednesday
Timetable
Practical
29 April to 3 Public Lecture 25, Lectures 24
Lecture 24 test 2
May Holiday 26 - 26
Tutorial 9
Practical 7
May
12 Lecture 28,
29 Lectures 27 Lectures 27 -
6 to 10 Lecture 27 Practical 7
Tutorial 10 - 29 29
Practical 8
13 Practical
Consultation
test 3
Lecture Lectures 30 Lectures 30
13 to 17 Practical 8 Lecture 30
Semester - 31 - 31
Tutorial 11
Test 2 17:30
Practical 9
14 Lecture 32,
33, 34 Lectures 32 Lectures 32
20 to 24 Lecture 31 Practical 9
Tutorial 12 - 33 - 33
Practical 10
15 Lectures 34 Lectures 34
27 to 31 Lecture 35 Practical 10 Voting Day
- 36 - 36
June

16 Tutorial 13
Practical test Wednesday
3 to 7 Lecture 36 Semester
4 Timetable
Test 3 17:30
4. Module Information

4.1 Learning presumed to be in place

It is presumed that the student has a good general background in mathematics and statistics,
specifically calculus, algebra, differential equations and second year mathematical statistics.

4.2 Software

For the practical component of this module, we will be utilizing the R software. This
software can be used on the student’s device through RStudio or Rstudio.cloud. More
details will be posted on ClickUP. Both R and RStudio should be installed.

4.3 Content
key: ST: proofs that need to be studied for semester tests
E: proofs that should be studied for exam
Note. All other content, including statements of theorems, need to be known for every test.
Note. Only proofs indicated with a ‘thumbs up’ with a number are examinable. Proofs not
indicated in such a manner are simply for completeness sake and will not be examined at
any point in this course.
Note. The student must be able to apply all theoretical results to practical examples.
Chapter 1: Introduction (lecture 1) Understand the concepts, be able to define and give
examples of a mathematical model, specifically a deterministic and a probabilistic/stochastic
model.

Chapter 2: Description and Definition of Stochastic Processes (lecture 2-4)

2.1 Description
● Understand the concept of a probability space and the notation used for stochastic
processes.
● Know and understand definition 2.1.1, as well as the concepts of a state space and
parameter space.

2.2 Sample Paths and Probability Descriptions

● Know and understand the concept of a sample path of a stochastic process.
● Result 2.2.1 need only be understood – the proof will not be examined. Its importance
is indicated in the note below the result.

2.3 Some General Properties of Stochastic Processes

● Know and understand definitions 2.3.1 – 2.3.6. Also be able to show that a process
(examples in section 2.4) does have or does not have the various properties.
● Theorem 2.3.1 (ST): statement and proof, be able to prove to the various
combinations or discrete and continuous state and parameter spaces (simply a
notation change)

2.4 Some Specific Stochastic Processes

● Be able to define the processes given in this section as well as prove/disprove the
various properties they satisfy and don’t satisfy.

2.5 Important Theoretical Results

● This section provides some background results needed for the syllabus.
● Theorem 2.5.1: no proof required (covered in WST322), however the statement must
be known and understood.
● Definition 2.5.1, Theorem 2.5.2, 2.5.3: revision of second year work, understand and
know the results, no proofs.
● Theorem 2.5.4 (ST): statement and proof
● Theorem 2.5.5: revision of second year work, statement only including the notes after
the theorem
● Theorem 2.5.6: revision of second year work, statement only
● Theorem 2.5.7 (ST): statement and proof

All exercises are important.

Chapter 3: Markov Chains (lecture 5 - 21)

3.1 Chapman-Kolmogorov Equations
● Be able to define a Markov chain fully.
● Understand and be able to define the concept of a transition probability in the context
of a Markov chain.
● Theorems 3.1.1, 3.1.2, 3.1.3 (ST): statement and proofs of these theorems regarding
various transitions for a sample path of a Markov chain
● Know and understand the matrix notation for the transition probabilities.
3.2 Time-Homogeneous Markov Chains
● Be able to define time-homogeneity as well as determine whether a process is time-
homogeneous. Also, understand its impact on the notation and theorems 3.1.1-3.1.3.

3.3 Classification of States

● Know and understand definition 3.3.1 and the notation.
● Theorem 3.3.1 (ST): statement and proof
● Know and understand the concepts of equivalence class and irreducible
● Theorem 3.3.2 (E): statement and proof
● Know and understand definition 3.3.2-3.3.3, theorems 3.3.3-3.3.5 (no proofs)
● Theorem 3.3.6 (E): statement and proof
● Theorem 3.3.7 (E): statement and proof
● Theorem 3.3.8 (ST): statement and proof
● Theorem 3.3.9 (E): statement and proof
● Theorem 3.3.10 (E): statement and proof
● Theorem 3.3.11 (E): statement and proof
● Know and understand definition 3.3.4.

3.4 Finite Markov Chains with Recurrent and Transient States

● Understand and be able to put a transition probability matrix in the form of equation
3.4.1.
● Theorem 3.4.1 (ST): statement and proof
(n ) (n)
● Know and understand the defined probabilities g ij , gij and matrices G , G .
● Theorem 3.4.2 (ST): statement and proof
(n )
● Example 3.4.2 derives a formula for the solution of the probabilities g ij
● The Solution of Difference Equations section provides an alternative approach to
derive the solution of example 3.4.2. The student doesn’t need to know any proofs
from this section but must be able to apply the results to any provided homogeneous
difference equation.

●
(n )
T
Know and understand hij , mij , ij and Ti .
● Theorem 3.4.3 (ST): statement and proof

3.5 Irreducible Finite Markov Chains and Limit Theorems

● Know and understand the concept of a first passage time and recurrence time for
*
irreducible Markov chains as well as the related matrix P
● Theorem 3.5.1 (ST): statement and proof
● Theorem 3.5.2 (E): statement and proof (only case 1)
● Know and understand definition 3.5.1
● Theorem 3.5.3 (E): statement and proof
● The note below theorem 3.5.3 is fundamental for solving for limiting distributions.
The examples are very important. The student should be able to solve for limiting
distributions for non-irreducible cases as well.
● Theorem 3.5.4 (ST): statement and proof
● Know and understand the concept of an occupation time
● Theorem 3.5.6 (ST): statement and proof
● Theorem 3.5.7 (ST): statement and proof (good example of a non-homogeneous
difference equation) Note that any 2 × 2 matrix P may be given and the proof should
be adjusted accordingly.
● Theorem 3.5.8 (ST): statement and proof
● Experience rating is an important application of Markov chains. The student should
be able to determine whether a given process satisfies the Markov property or not
and subsequently adjust the state space if necessary to ensure the Markov property
does hold.

3.6 Markov Chains with Countably Infinite States

● Theorem 3.6.3: statement
● Theorem 3.6.4: statement
● Theorem 3.6.6: statement
● The differences and similarities between the case of finite states (section 3.5) and
infinite states (section 3.6) should be known and understood.

3.7 Estimation of Parameters of a Markov Chain

● Be able to describe and apply the estimation problem at hand
● Know and understand result 3.7.1 (E)

3.8 Testing Whether or Not a Stochastic Process with Discrete Parameter Space and Finite
Discrete State Space is a Time-Homogeneous Markov Chain
● Be able to describe and apply the test fully.

All exercises are important.

Chapter 4: Markov Jump Processes (lectures 22 - 35)

4.1 Transition Rates and the Kolmogorov Equations
● Be able to define a Markov jump process fully.
● Understand and be able to define the concept of a transition probability in the context
of a Markov jump process, including understanding the matrix notation.
● Theorems 3.1.1-3.1.3 (ST) are also applicable for Markov jump processes and the
student should be able to prove them in this case as well.
● Know and understand definitions 4.1.2 and 4.1.3.
● Results 4.1.1 and 4.1.2 (ST): statements and proofs (these are used throughout the
chapter)
● Theorem 4.1.1 (ST): statement and proof
● Theorem 4.1.2 (ST): statement and proof (this proof should be done by the student
and handed in for marks)
● Theorem 4.1.3 (ST): statement and proof
● Know and understand the time-homogeneous case for a Markov jump process. The
notation is important.
● Know and be able to derive the solution of Kolmogorov’s forward equation in terms
of the matrix exponential.
● Theorem 4.1.4 (E): statement and proof

4.2 The Poisson Process

● The student should know the definition of a Poisson process as given in Chapter 2.
● Theorem 4.2.1 (E): statement and proof
● Theorem 4.2.2 (ST): statement and proof
● Theorem 4.2.3 (ST): statement and proof
● Theorem 4.2.4 (ST): statement and proof
● Theorem 4.2.5 (ST): statement and proof
● Theorem 4.2.6 (ST): statement and proof
● Theorem 4.2.7 (ST): statement and proof
● Theorem 4.2.8 (ST): statement and proof
● Theorem 4.2.9 (ST): statement and proof

4.3 Pure Birth Process

● Be able to explain the difference between the Poisson process and the Pure Birth
process.
● Theorem 4.3.1 (ST): statement and proof
● Know the solution of the differential equations obtained in theorem 4.3.1 and be able
to show it does satisfy the differential equations. No derivation of the solution is
required. only know P_00(t)
● Be able to explain the difference between the Yule process, Pure Birth process and
Poisson process.
● Be able to derive the differential equations for the Yule process.
● Know the solution of the differential equations for the Yule process and be able to
show it does satisfy the differential equations. No derivation of the solution is
required.

4.4 Pure Death Process

● Be able to define the Pure Death process.
● Be able to derive the differential equations for the Pure Death process.
● Know the solution of the differential equations for the Pure Death process and be able
to show it does satisfy the differential equations. No derivation of the solution is
required.

4.5 Two-State Markov Processes

● Be able to derive the limiting probabilities discussed and discuss them.
● Understand and be able to derive the limiting probabilities for the two-state example
provided.

4.6 Birth and Death Processes

● Be able to define the Birth and Death process.
● Be able to derive the differential equations for the Birth and Death process.
● Know the solution of the differential equations for the Birth and Death process. No
derivation of the solution is required.
● Be able to show it does satisfy the differential equations (this will not be tested but
may be handed in as an exercise for additional marks).
● Know and understand the alternative approach for deriving the limiting distribution
shown in this section. Be able to apply this technique to all the processes in Chapter
4.

4.7 Estimation of Transition Rates for a Markov Jump Process

● Only the estimation for the Poisson process is required.
● The student must also be able to test if the process is a Poisson process. The test of
 independence of times between events is covered in WST321.
● See the practical for more details on how to apply the theory (examinable).

4.8 The Structure of Markov Processes

● Know and understand both cases (time-homogeneous and time-inhomogeneous) and
be able to explain all similarities and differences and why they occur.
● Theorem 4.8.1 (E): statement and proof (short proof only!)
● Theorem 4.8.2 (E): statement and proof
● Theorem 4.8.3 (E): statement and proof (short proof only!)
● Theorem 4.8.4 (E): statement and proof
● Example 4.8.1 with Results 1-3 (ST)

4.9 Integrated Form of the Kolmogorov Equations

● Theorem 4.9.1 (ST): statement and proof
● Theorem 4.9.2 (ST): statement and proof
● Theorem 4.9.3 (E): statement and proof

4.10 Processes where the Transition Rates Also Depend on the Length of Time it is already
in a State
● Be able to describe how to adjust the state space of a jump process to ensure the
Markov property holds.

All exercises are important, especially E.11.