Introduction

Numerical Methods in Quantitative Finance

Li Hao
[email protected]

Numerical Methods in QF (QF5204) Introduction 1 / 40

About Me
Career
▶ Quant Researcher & Founder,
ROC Square PTE. LTD, 2021 - present
▶ Adjunct Faculty,
LKC School of Business, SMU, 2020 − present
Dept. of Math, NUS, 2016 − present
▶ Quant Analyst, Commodities and FX,
Standard Chartered Bank, 2010 − 2021

Education
▶ PhD Computer Science,
National University of Singapore, 2005 - 2010
▶ BSc Computer Science,
Fudan University, 2001 - 2005

Numerical Methods in QF (QF5204) Introduction 2 / 40

Course Objectives

Introduction of numerical methods used in quantitative finance

Understand the convention and standard models used in financial

market: a little bit more than Black-Scholes

Be able to use and implement numerical methods to solve problems

Be able to implement numerical pricers for different financial

products: a little bit more than Call and Put

Be able to prototype analytics tools to address problems in derivatives

market

Focus: WHAT are used in practice, WHY and HOW?

Numerical Methods in QF (QF5204) Introduction 3 / 40

Main Topics

Number representation and numerical errors

Binomial and trinomial tree models

Root search, interpolation, volatility smiles

Monte-Carlo simulation (MC)

Partial differential equations (PDE)

Numerical Optimization

We will discuss concepts and techniques from practitioners’ point of view.

Numerical Methods in QF (QF5204) Introduction 4 / 40

Assessment

Assignments 70%: 4 assignments, 15% each for the first 2, 20% each
for the last 2

Project 30%: group project (1 to 2 persons)

Assignments are individual. Project can be done in group of one to

two members (no extra credit for group of 1 person).

Numerical Methods in QF (QF5204) Introduction 5 / 40

Required Knowledge

There is no strict requirements or assumptions on your knowledge base,

but it would be helpful if you are familiar with
linear algebra,

probability theory,

calculas,

stochastic calculus

Most concepts that are used will be covered, briefly introduced, or referred
to recommended reading materials.

We will be using python as the programming language for implementation.

Numerical Methods in QF (QF5204) Introduction 6 / 40

Course Materials

No textbook

Slides and assignment / project handouts

Recommended further readings from different books, papers and

websites for different topics

Numerical Methods in QF (QF5204) Introduction 7 / 40

Numerical Analysis in Financial Industry
Financial industry is the industry that deals with numbers
▶ asset prices

▶ market movements (asset return and volatility)

▶ interest rates

▶ ...

Mathematical models are used to

▶ describe the relationship between the numbers

▶ abstract / extract key information from the numbers

Numerical Methods in QF (QF5204) Introduction 8 / 40

Numerical Analysis in Financial Industry

There are few cases that application of standard formulas can solve
the problem

There are increasing demand on practitioners to apply the

methodologies and adapt them appropriately to financial analyses,
pricing, risk modeling, and risk management

Numerical analysis is concerned with all aspects of the numerical

solution of a problem, from the theoretical development and
understanding of numerical methods to their practical implementation
as reliable and efficient computer programs

Numerical Methods in QF (QF5204) Introduction 9 / 40

How does it work?
If a problem cannot be solved analytically, replace it with a “nearby
problem” which can be solved more easily
It uses the language and results of linear algebra, real analysis, and
functional analysis.
There is a fundamental concern with error, its size, and its analytic
form.
Numerical analysts are interested in measuring the efficiency of
algorithms.
Numerical methods are implemented with finite precision computer
arithmetic.
Two primary concerns while using numerical methods:
computational cost
accuracy of the results

Numerical Methods in QF (QF5204) Introduction 10 / 40

Computational Cost

Same mathematical formula can have different implementations

Example − d± in Black and Scholes formula:

µt
log SeK ± 12 σ 2 t
d± = √ (1)
σ t
How to implement it so that the cost of calculation is minimum?

Numerical Methods in QF (QF5204) Introduction 11 / 40

Cost of Operations
The cost of operation depends on
Type of operands, CPU and computer architecture, programing
language
Example code measuring the cost of operations:
import timeit

def opTiming(op, opName, repeat):

elapsed_time = timeit.timeit(op, setup=’import math’, number=repeat)
print(opName, "\t", elapsed_time / repeat)

repeat = int(1e8)
opTiming("x = 5.0 + 7.0", "add", repeat)
opTiming("x = 5.0 * 7.0", "mul", repeat)
opTiming("x = 5.0 / 7.0", "div", repeat)
opTiming("x = math.log(7.0)", "log", repeat)
opTiming("x = math.exp(7.0)", "exp", repeat)
opTiming("x = math.sqrt(7.0)", "sqrt", repeat)

Numerical Methods in QF (QF5204) Introduction 12 / 40

Which implementation is faster to calculate d+ ? And why?
method 1

d1 = (log (S0 e µt /K ) + vol ∗ vol ∗ t/2)/vol/sqrt(t) (2)

method 2

stdev = vol ∗ sqrt(t) (3)

d1 = (log (S/K ) + µt)/stdev + stdev /2 (4)

m1 = """
S = 100;K = 105;vol = 0.1;t=2;mu=0.01
d1 = (math.log(S * math.exp(mu*t) / K) + vol * vol * t / 2) / vol / math.sqrt(t)
"""
m2 = """
S = 100;K = 105;vol = 0.1;t=2;mu=0.01
stdev = vol * math.sqrt(t)
d1 = (math.log(S / K) + mu*t) / stdev + stdev / 2
"""
repeat = int(1e7)
opTiming(m1, ’m1’, repeat)
opTiming(m2, ’m2’, repeat)

Numerical Methods in QF (QF5204) Introduction 13 / 40

Number Representation
Translator from our language to computer’s language
Computer uses binary number system - everything is represented as
combination of 0 and 1
The binary form of a positive integer is given by dn dn−1 . . . d1 d0 ,
which is equivalent to

dn 2n + dn−1 2n−1 + . . . + d1 21 + d0 20 (5)

The binary form of a positive number less than 1 is given by

d−1 d−2 . . . d−m−1 d−m , equivalent to

d−1 2−1 + d−2 2−2 + d−3 2−3 . . . (6)

Numerical Methods in QF (QF5204) Introduction 14 / 40

Integer Representation
For example, a binary number

110112 (7)

represents the value:

1 × 24 + 1 × 23 + 0 × 22 + 1 × 21 + 1 × 20 (8)
= 16 + 8 + 0 + 2 + 1 (9)
= 2710 (10)

If we use 32 binary bits to represent a “‘integer“‘ number, what is the

range of numbers we can represent?
unsigned: 0 to 232 − 1

signed: −231 to 231 − 1

Numerical Methods in QF (QF5204) Introduction 15 / 40

Fixed Point Representation
To represent a real number in computers, we can define a fixed point
number type simply by implicitly fixing the binary point to be at the same
position of a numeral.
For example, to represent 13.5 with binary point position at the left of the
last bit: 1101.12

1 × 23 + 1 × 22 + 0 × 21 + 1 × 20 + 1 × 2−1 (11)
= 8 + 4 + 0 + 1 + 0.5 (12)
= 13.510 (13)

To define a fixed point type, we need two parameters:

width of the number representation

binary point position

Numerical Methods in QF (QF5204) Introduction 16 / 40

 For ease of discussion, we use fixed<w, b> to denote a fixed point
type with w width and binary point position at b counting from 0 (the
least significant bit).
int type is a special case of fixed point representation:
fixed<32, 0>
For example, fixed<8, 3> denotes a 8 bit fixed point number, of
which 3 right most bits are fractional. Therefore, the number
00010110 represents:

00010.1102 = 1 × 21 + 1 × 2−1 + 1 × 2−2 = 2.75 (14)

The same bit patter 00010110 represents a different number if it is of

a different type, e.g., fixed<8, 5>:

000.101102 = 1 × 2−1 + 1 × 2−3 + 1 × 2−4 = 0.6875 (15)

Numerical Methods in QF (QF5204) Introduction 17 / 40

Negative Number Representation

Two common way of representing signed types:

Sign and magnitude representation (SMR) - use a sign bit (first bit):
0 = positive and 1 = negative, the remaining bits in the number
indicate the magnitude (absolute value)

Example with fixed<8, 0>: 0 0001100 = 1210 , 1 0001100 = −1210

Numerical Methods in QF (QF5204) Introduction 18 / 40

 Two’s-complement: the value x of a fixed<w, b> number
dw −1 dw −2 . . . d0 is given by:
w
X −2
x = −dw −1 2w −1−b + di 2i−b (16)
i=0

▶ Example with fixed<8, 0>: 00001100 = 1210 , 10001100

= −27 + 12 = −116

Numerical Methods in QF (QF5204) Introduction 19 / 40

Two’s complement representation

The advantage of using two’s complement representation is

fundamental arithmetic operations of either positive or negative
numbers are identical - including addition, subtraction, multiplication,
and even shifting.

For example, if we look at two signed fixed<4, 1> numbers: 1011 and
1101

101.1 −1 × 22 + 1 × 20 + 1 × 2−1 = −2.510 (17)

+ 110.1 −1 × 22 + 1 × 21 + 1 × 2−1 = −1.510 (18)
|{z} 100.0
1 ← −410 (19)
overflow bit

Note that the left most overflow bit is discarded.

Numerical Methods in QF (QF5204) Introduction 20 / 40

 Converting a real number to signed fixed<w, b>
input: a real number x, width w, binary position b
output: array a of size w, a[i] is the i-th bit of the fixed point
representation
w
X −2
w −1−b
x = −dw −1 2 + di 2i−b (20)
i=0

1 def toFixedPoint(x : float, w : int, b : int) -> [int]:

2 # set a[w-1] to 1 if x < 0, otherwise set a[w-1] to 0
3 a = [0 for i in range(w)]
4 if x < 0:
5 a[0] = 1
6 x += 2**(w-1-b)
7 for i in range(1, w):
8 y = x / (2**(w-1-i-b))
9 a[i] = int(y) # round y down to integer
10 x -= a[i] * (2**(w-1-i-b))
11 return a
12
13 print(toFixedPoint(-10, 8, 1))
14 print(toFixedPoint(-9.5, 8, 1))
15 print(toFixedPoint(9.25, 8, 2))

Numerical Methods in QF (QF5204) Introduction 21 / 40

Range Matters

It is not difficult to notice that the range represented by fixed point

representation is limited.

If we try to represent a number beyond the range of the type we do

not get sensible result:

1 print(toFixedPoint(20, 8, 3))
2 print(toFixedPoint(20, 9, 3))

Numerical Methods in QF (QF5204) Introduction 22 / 40

 We need to deal with those cases:
1 def toFixedPoint2(x : float, w : int, b : int) -> [int]:
2 # set a[w-1] to 1 if x < 0, otherwise set a[w-1] to 0
3 a = [0 for i in range(w)]
4 if x < 0:
5 a[0] = 1
6 x += 2**(w-1-b)
7 for i in range(1, w):
8 y = x / (2**(w-1-i-b))
9 if int(y) > 1:
10 raise OverflowError(’fixed<’ + str(w) + "," + str(b) + "> is not
sufficient to represent " + str(x))
11 a[i] = int(y) # % 2 # round y down to integer
12 x -= a[i] * (2**(w-1-i-b))
13 return a
14
15 print(toFixedPoint2(20, 8, 3))

Exercise in assignment 1: implement a function converting fixed point to a real

number.

Numerical Methods in QF (QF5204) Introduction 23 / 40

 Problems caused by overflow can be serious.
On 22 Apr 2018, the price of an ERC20 token BEC plunged by 6̃0%
due to an overflow bug in smart contract

Code in smart contract:

1 function batchTransfer(address[] _receivers, uint256 _value) {
2 uint cnt = _receivers.length;
3 uint256 amount = uint256(cnt) * _value;
4 require(cnt > 0 && cnt <= 20);
5 require(_value > 0 && balances[msg.sender] >= amount);
6 ... # transfer the token
7 }

Many other ERC20 tokens’ smart contracts were found having the
same bug
Numerical Methods in QF (QF5204) Introduction 24 / 40
Microsoft Exchange Server Y2K22 Bug

On 1 Jan 2022, some Microsoft Exchange admins had to come to

work facing their first challenge of the new year: installing a patch to
fix jammed messages that started at midnight on January 1st.

The bug was due to “dates that are stored in the format
yymmddHHMM converted to a signed 32-bit integer overflowed on 1
January 2022, as 231 − 1 = 2147483647” — 2201010001 cannot be
represented as uint32.

See Microsoft Exchange team blog on the details of the bug.

Numerical Methods in QF (QF5204) Introduction 25 / 40

Floating Point Representation of Real Numbers

Real number 1234.56 can be represented in scientific notation

1.23456 × 103 (21)

Floating point representation uses scientific notation in binary format

exponent
z}|{
1.01 ×2 −1
|{z} (22)
significand

Numerical Methods in QF (QF5204) Introduction 26 / 40

 The IEEE 754 standard - the most widely used floating point
standard:
x exponent bits e1 e2 . . . ex
z }| {
0
|{z} 001 . . . 10 |1001 {z
. . . 001} (23)
1 sign bit s y significand bits m1 m2 . . . my

The number it represents is

(−1)s × (1 + significand) × 2exponent - bias (24)
y
X x
X
significand = mj × 2−j , exponent = ei × 2x−i (25)
j=1 i=1

1 import numpy as np
2 for f in (np.float32, np.float64, float):
3 finfo = np.finfo(f)
4 print(finfo.dtype, "\t exponent bits = ", finfo.nexp, "\t significand bits = ", finfo.nmant)

Floating point supports a much wider range of values

▶ 32 bit “‘float“‘: 10−38 to 1038 .
▶ 64 bit “‘float“‘: 10−308 to 10308 .
More on floating point arithmetic: What Every Computer Scientist
Should Know About Floating-Point Arithmetic
Numerical Methods in QF (QF5204) Introduction 27 / 40
Numerical Errors

A numerical error is the difference between the calculated

approximation of a number and its exact mathematical value.
▶ round off error (or rounding error)
computers have size and precision limits on their ability to represent
numbers

▶ truncation error
arises when we approximate a continuous model with a discrete one

Round off errors

Finite-precision causes round off in individual calculations
Effects of round off usually accumulate slowly

Numerical Methods in QF (QF5204) Introduction 28 / 40

Round Off Errors

1 x = 10776321
2 nsteps = 1235
3 s = x / nsteps
4 y = 0
5 for i in range(nsteps):
6 y += s
7 print(x - y)

Subtracting nearly equal numbers leads to severe loss of precision.

A similar loss of precision occurs when two numbers of very different

magnitude are added.

Round off is inevitable: good algorithms minimize the effect of round

off.

Numerical Methods in QF (QF5204) Introduction 29 / 40

Machine Epsilon
Machine epsilon is a number ϵ > 0, such that the computer considers
1+ϵ=1
1 x = 10.56
2 print(x == x + 5e-16)

It measures the effects of round off errors made when adding,

subtracting, multiplying, or dividing two numbers.
Different machine may have different machine epsilon.
Rule of thumb (but not precisely): ϵ is at the order of 10−16

1 x = 0.1234567891234567890
2 y = 0.1234567891
3 scale = 1e16
4 z1 = (x-y) * scale
5 print("z1 = ", z1)
6
7 z2 = (x*scale - y*scale)
8 print("z2 = ", z2)

Numerical Methods in QF (QF5204) Introduction 30 / 40

Truncation Error

Truncation errors are from numerical methods: introduced by

neglecting higher order terms.

Example: taylor expansion:

h2 2
f (x + h) = f (x) + hf ′ (x) + f (x) + . . . (26)
2!

Numerical Methods in QF (QF5204) Introduction 31 / 40

Taylor Theorem

Assume that f is continuous and f ′ (x), f 2 (x), . . . , f (n) (x) exists over (a, b),
let x0 ∈ [a, b], then for every x ∈ (a, b), there exists a number c that lies
between x0 and x such that
n
X f (k) (x0 ) f (n+1) (x0 )
f (x) = f (x0 ) + (x − x0 )k + (x − c)n+1 (27)
k! (n + 1)!
k=1

Numerical Methods in QF (QF5204) Introduction 32 / 40

Local truncation error

Local truncation error is the error that our increment function causes
during a single iteration.
In the case of taylor expansion,

f (x0 + h) = f (x0 ) + hf ′ (x0 ) + O(h2 ) (28)

If we approximate f (x0 + h) by f (x0 ) + hf ′ (x0 ), the local truncation error

is O(h2 ).

We say that the numerical method has order p if for any sufficiently
smooth solution of the initial value problem, the local truncation error is
O(hp+1 ).

Numerical Methods in QF (QF5204) Introduction 33 / 40

Global Truncation Error
Global truncation error is the accumulation of the local truncation error
over all of the iterations.
If we know f ′ (x) and f (x0 ), to approximate f (x0 + h) for a not so
small h, we can divide h into n steps ∆h = hn , then

y0 = f (x0 )
x1 = x0 + ∆h, y1 = y0 + f ′ (x0 )∆h
...
xi = xi−1 + ∆h, yi = yi−1 + f ′ (xi−1 )∆h
...
xn = xn−1 + ∆h, f (x0 + h) ≈ yn = yn−1 + f ′ (xn−1 )∆h

The global truncation error en = f (x0 + h) − yn .

A numerical method is convergent if global truncation error goes to
zero as the step size goes to zero.
Numerical Methods in QF (QF5204) Introduction 34 / 40
Errors of Numerical Solution

Numerical Methods in QF (QF5204) Introduction 35 / 40

Measuring Errors

Errors can be measured with either absolute error or relative error, or

both.
▶ Abseolute error: |xc − x|

|xc −x|
▶ Relative error: x where xc is the computed value and x is the exact
value

In some derivative pricing cases we take the notional of the trade to

calculate relative error.

Numerical Methods in QF (QF5204) Introduction 36 / 40

Convergence

Convergent sequence: suppose that {xn }∞ is an infinite sequence, the

sequence is said to have the limit L

lim xn = L (29)
n→∞

if for any given ϵ > 0, there exists an N such that for any n > N, we
have |xn − L| < ϵ.

Numerical Methods in QF (QF5204) Introduction 37 / 40

Order and Rate of Convergence

Let xn be a sequence that converges to a number L, if there exists a

positive constant λ, such that

|en+1 | |xn+1 − L|
lim p
= lim =λ (30)
n→∞ |en | n→∞ |xn − L|p

p is the order of convergence

Numerical Methods in QF (QF5204) Introduction 38 / 40

Order and Rate of Convergence
More intuitively |en+1 | = λ|en |p when n → ∞.
The larger p and the smaller λ, the more quickly the sequence
converges
Specifically:
▶ if p = 1 and 0 ≤ λ ≤ 1, |en+1 | = λ|en | < |en |, then
⋆ if λ = 1, the convergence is sublinear - convergence is slower than
linear
⋆ if 0 < λ < 1, the convergence is linear with rate of convergence λ
⋆ if λ = 0, the convergence is superlinear - faster than linear

▶ if p = 2, |en+1 | = λ|en |2 , λ > 0, the convergence is quadratic

▶ if p = 3, |en+1 | = λ|en |3 , λ > 0, the convergence is cubic

Numerical Methods in QF (QF5204) Introduction 39 / 40

Tolerance

A variety of criteria can be used for deciding when your approximation

solution is close enough to true solution.

Possible tests
▶ the absolute change is sufficiently small |xk+1 − xk | ≤ tol

−xk
▶ the relative change is sufficiently small | xk+1
xk+1 | ≤ tol

Which test to use depends on the application itself

Numerical Methods in QF (QF5204) Introduction 40 / 40