MVE220 Financial Risk: Reading Project

An introduction to Markov chains and

their applications within finance

Group Members:
David Schön Myers
Lisa Wallin
Petter Wikström
1. Introduction
Markov chains are an important mathematical tool in stochastic processes. The underlying
idea is the Markov Property, in order words, that some predictions about stochastic processes
can be simplified by viewing the future as independent of the past, given the present state of
the process. This is used to simplify predictions about the future state of a stochastic process.

This report will begin with a brief introduction, followed by the analysis, and end with tips
for further reading. The analysis will introduce the concepts of Markov chains, explain
different types of Markov Chains and present examples of its applications in finance.

1.1 Background
Andrei Markov was a Russian mathematician who lived between
1856 and 1922. He was a poorly performing student and the only
subject he didn’t have difficulties in was mathematics. He later
studied mathematics at the university of Petersburg and was
lectured by ​Pafnuty Chebyshev, known for his work in probability
theory.​ Markov’s first scientific areas were in number theory,
convergent series and approximation theory. His most famous
studies were with Markov chains, hence the name and his first
paper on the subject was published in 1906. He was also very
interested in poetry and the first application he found of Markov
chains was in fact in a linguistic analysis of Pusjkins work ​Eugene
Onegin​. [1]

1.2 Delimitations
This report will not delve too deep into the mathematical aspects
of Markov chains. Instead, it will focus on delivering a more general understanding and serve
as an introduction to the subject.

1.3 Purpose
The purpose of this report is to give a short introduction to Markov chains and to present
examples of different applications within finance.

1
2. Analysis

2.1 Introduction to Markov chains

Markov chains are a fundamental part of stochastic processes. They are used widely in many
different disciplines. A Markov chain is a stochastic process that satisfies the Markov
property, which means that the past and future are independent when the present is known.
This means that if one knows the current state of the process, then no additional information
of its past states is required to make the best possible prediction of its future. This simplicity
allows for great reduction of the number of parameters when studying such a process. [2]

In mathematical terms, the definition can be expressed as follows:

A stochastic process X = {X n , n ε N } in a countable space S is a discrete-time

Markov chain if:

F or all n ≥ 0, X n ε S
F or all n ≥ 1 and f or all i0 , ... in−1 , in ε S, we have :
P { X n = in | X n−1 = in−1 , ... , X 0 = i0 } = P { X n = in | X n−1 = in−1 } [2]

Markov chains are used to compute the probabilities of events occurring by viewing them as
states transitioning into other states, or transitioning into the same state as before. We can
take weather as an example: If we arbitrarily pick probabilities, a prediction regarding the
weather can be the following: If it is a sunny day, there is a 30% probability that the next day
will be a rainy day, and a 20% probability that if it is a rainy day, the day after will be a
sunny day. If it is a sunny day, there is therefore a 70% chance that the next day will be
another sunny day, and if today is a rainy day, there is a 80% chance that the next day will be
a rainy day as well. This can be summarized in a transition diagram, where all of the possible
transitions of states are described:

To approach this mathematically one views today as the current state, S 0 , which is a 1 × m
vector. The elements of this vector will be the current state of the process. In our weather
example, we define S = [Sunny Rainy] . Where S is called our state space, in which all the
elements are all the possible states that the process can attain. If, for example, today is a
sunny day, then the S 0 vector will be S 0 = [1 0] , because there is 100% chance of a sunny
day and zero chance of it being a rainy day. To get to the next state, the transition probability

2
matrix is required, which is just the state transition probabilities summarized in a matrix. In
this case it will be as follows:

To get to the next state, S 1 , you simply calculate the matrix product S 1 = S 0 P . Since
calculations for successive states of S is only of the type S n = S n−1 P , the general formula for
computing the probability of a process ending up in a certain state is S n = S 0 P n . This allows
for great simplicity when calculating the probabilities far into the future. For example, if
today is a sunny day then the state vector 120 days from now, S 120 , is S 120 = [0.4 0.6] . [3]

2.2 Explanation of different concepts regarding Markov chains

When approaching Markov chains there are two different types; discrete-time Markov chains
and continuous-time Markov chains. This means that we have one case where the changes
happen at specific states and one where the changes are continuous. In our report we will
mostly focus on discrete-time Markov chains.

One example to explain the discrete-time Markov chain is the price of an asset where the
value is registered only at the end of the day. The value of the Markov chain in discrete-time
is called the state and in this case the state corresponds to the closing price. A
continuous-time Markov chain changes at any time. This can be explained with any example
where the measured events happens at a continuous time and lacks “steps” in its appearance.
One well known example of continuous-time Markov chain is the poisson process, which is
often practised in queuing theory. [1]

For a finite Markov chain the state space S is usually given by S = {1, . . . , M} and the
countably infinite state Markov chain state space usually is taken to be S = {0, 1, 2, . . . }.
These different variances differ in some ways that will not be referred to in this paper. [4]

A Markov chain can be stationary and therefore be independent of the initial state in the
process. This phenomenon is also called a steady-state Markov chain and we will see this
outcome in the example of market trends later on, where the probabilities for different
outcomes converge to a certain value. However, an infinite-state Markov chain does not have
to be steady state, but a steady-state Markov chain must be time-homogenous. Which by
definition means that the transition probabilities matrix P i, j (n, n + 1) is independent of n.
[3]

2.3 Application areas of Markov chains

Since Markov chains can be designed to model many real-world processes, they are used in a
wide variety of situations. These fields range from the mapping of animal life populations to
search-engine algorithms, music composition and speech recognition. In economics and
finance, they are often used to predict macroeconomic situations like market crashes and
cycles between recession and expansion. Other areas of application include predicting asset
and option prices, and calculating credit risks. ​When considering a continuous-time financial

3
market Markov chains are used to model the randomness. The price of an asset, for example,
is set by a random factor – a stochastic discount factor – which is defined using a Markov
chain [5].

2.4 Application of Markov chains to credit risk measurement

In the application of Markov chains to credit risk measurement, the transition matrix
represents the likelihood of the future evolution of the ratings. The transition matrix will
describe the probabilities that a certain company, country, etc. will either remain in their
current state, or transition into a new state. [6] An example of this below:

[6]

The main problem in this application is determining the transition matrix. Of course, these
probabilities could be estimated by analysing historical data from credit rating agencies, such
as Standard & Poor, Moody’s and Fitch. This can, though, lead to unreliable numbers in case
the future does not develop as smoothly as in the past. It can therefore be more reliable to
base the estimations on a combination of empirical data and more subjective, qualitative data
such as opinions from experts. This is because the market view is a mixture of beliefs
determined by both historical ratings and a more extreme view of the ratings. To combine
different sources of information in this way, one may use credibility theory. Actuarial
credibility theory provides a consistent and convenient way of how to combine information,
and how to weigh the different data sources. [6]

Another problem with deciding the transition matrix is that maybe it is not appropriate to use
a homogenous Markov chain to model credit risk over time. This is because it does not
capture the time-varying behaviour of the default risk. Of course, a non-homogeneous model
could be more realistic - but on the other hand much more complicated to use. [6]

2.5 Markov chains to predict market trends

Markov chains and their respective diagrams can be used to model the probabilities of certain
financial market climates and thus predicting the likelihood of future market conditions [7].
These conditions, also known as trends, are:

● Bull markets: ​periods of time where prices generally are rising, due to the actors
having optimistic hopes of the future.
● Bear markets: ​periods of time where prices generally are declining, due to the actors
having a pessimistic view of the future.
● Stagnant markets​: periods of time where the market is characterized by neither a
decline nor rise in general prices.

4
In fair markets, it is assumed that the market information
is distributed equally among its actors and that prices
fluctuate randomly. This means that every actor has equal
access to information such that no actor has an upper hand
due to inside-information. Through technical analysis of
historical data, certain patterns can be found as well as
their estimated probabilities. [7] For example, consider a
hypothetical market with Markov properties where
historical data has given us the following patterns:
After a week characterized of a bull market trend there is
a 90% chance that another bullish week will follow.
Additionally, there is a 7.5% chance that the bull week
instead will be followed by a bearish one, or a 2.5%
chance that it will be a stagnant one. After a bearish week there’s an 80% chance that the
upcoming week also will be bearish, and so on. By compiling these probabilities into a table,
we get the following transition matrix ​M​:

We then create a 1x3 vector ​C​ which contains information about which of the three different
states any current week is in; where column 1 represents a bull week, column 2 a bear week
and column 3 a stagnant week. In this example we will choose to set the current week as
bearish, resulting in the vector C = [0 1 0] .

Given the state of the current week, we can then calculate the possibilities of a bull, bear or
stagnant week for any number of n weeks into the future. This is done by multiplying the
vector C with the matrix , giving the following:

5
From this we can conclude that as n → ∞ , the probabilities will converge to a steady state,
meaning that 63% of all weeks will be bullish, 31% bearish and 5% stagnant.
What we also see is that the steady-state probabilities of this Markov chain do not depend
upon the initial state [3]. The results can be used in various ways, some examples are
calculating the average time it takes for a bearish period to end, or the risk that a bullish
market turns bearish or stagnant.

3. Summary
Markov chains are an important concept in stochastic processes. They can be used to greatly
simplify processes that satisfy the Markov property, namely that the future state of a
stochastic variable is only dependent on its present state. This means that knowing the
previous history of the process will not improve the future predictions - which of course
significantly reduces the amount of data that needs to be taken into account.

Mathematically, Markov chains consist of a state space, which is a vector whose elements are
all the possible states of a stochastic variable, the present state of the variable, and the
transition matrix. The transition matrix contains all the probabilities that the variable will
transition from one state to another, or remain the same. To calculate the probabilities of a
variable ending up in certain states after n discrete partitions of time, one simply multiplies
the present state vector with the transition matrix raised to the power of n.

There are different types of concepts regarding Markov chains depending of the nature of the
parameters and application areas. They can be computed over discrete or continuous time.
The state space can vary to be finite or countably infinite and depending on which, behave in
different ways. A Markov chain with a countably infinite state space can be stationary which
means that the process can converge to a steady state.

Markov chains are used in a broad variety of academic fields, ranging from biology to
economics. When predicting the value of an asset, Markov chains can be used to model the
randomness. The price is set by a random factor which can be determined by a Markov chain.

Regarding the application to credit risk measurement, calculating the transition matrix is the
most important part when applying Markov chains in this subject. One way that has been
presented in this paper is to use a combination of empirical data from several years in the past
from credible credit rating institutions (Standard & Poor, Moody’s, Fitch), and other types of
more qualitative data. One should bear in mind that the homogenous Markov chain model
probably will be less realistic than a non-homogenous model, but much less complicated.

By analyzing the historical data of a market, it is possible to distinguish certain patterns in its
past movements. From these patterns, Markov diagrams can then be formed and used to
predict future market trends as well as the risks associated with them.

6
4. Further Reading
[1] A. Andersson, Introduktion till Markovkedjor: Med diskret och kontinuerlig tid.
Göteborg: HB matematiklitteratur i Göteborg, 2004.

[4] R. G. Gallager, Stochastic processes: theory for applications. United Kingdom:

Cambridge university press, 2013.

[7] Deju Zhang, Xiaomin Zhang, “Study on forecasting the stock market trend based on
stochastic analysis method”, ​International Journal of Business and Management​, vol 4, nr 6,
s. 163-164, June 2009.

5. References
[1] A. Andersson, Introduktion till Markovkedjor: Med diskret och kontinuerlig tid.
Göteborg: HB matematiklitteratur i Göteborg, 2004.

[2] B. Sericola, ​Markov Chains: Theory, Algorithms and Applications​. London: ISTE Ltd and
John Wiley & Sons Inc, 2013.

[3] T. Konstantopoulos, “Introductory lecture notes on MARKOV CHAINS AND

RANDOM WALKS”, Autumn 2009, [online],
http://www2.math.uu.se/~takis%20/L/McRw/mcrw.pdf​ [Accessed: 2017-04-03]

[4] R. G. Gallager, “Finite-state markov chains” in Stochastic processes: theory for

applications. United Kingdom: Cambridge university press, 2013, 160-213. [Online]
Avalible: ​http://www.rle.mit.edu/rgallager/documents/6.262-4vaw.pdf​.

[5] J. Hoek & R. J. Elliott, “Asset Pricing Using Finite State Markov Chain Stochastic
Discount Functions”, Stochastic Analysis and Applications, 30:5, Aug 2012, [online],
http://dx.doi.org/10.1080/07362994.2012.704852​ [Accessed: 2017-04-03]

[6] Tak-Kuen Siu , Wai-Ki Ching , S. Eric Fung & Michael K. Ng, " On a multivariate
Markov chain model for credit risk measurement," ​Quantitative Finance, ​5:6, s. 543-556, Feb
2007 [Online]
Available: : ​http://dx.doi.org/10.1080/14697680500383714​.

[7] Deju Zhang, Xiaomin Zhang, “Study on forecasting the stock market trend based on
stochastic analysis method”, ​International Journal of Business and Management​, vol 4, nr 6,
s. 163-164, June 2009 [Online] Available: CCSE Journals Online,
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.654.166&rep=rep1&type=pdf
[Accessed: 2017-04-03]