Back-Testing of Trading Strategies: 8.1bootstrap

8.
Back-testing of trading strategies
8.1Bootstrap
Brock et al (1992), Davidson & Hinkley (1997), Fusai & Roncoroni (2008).
Bootstrap: picking up at random an element of given sample, copying it into the new sample, and
putting it back (replacement). This random selection continues until the new sample has the same
number of elements as the original one.
rand() = 0.1; 0.22; 0.95; 0.7 => n1; n1; n4; n3
1
8. Back-testing of trading strategies
8.1Bootstrap (continued)
Block bootstrap (preserving autocorrelations)

Replacement of L sequential data
Stationary bootstrap for stationary (or weakly) samples:
Block size L is drawn from the discrete geometric distribution
Pr(L=k) = (1 p)k-1p
E[L] = 1/p => estimate of p
2
8.1Bootstrap (continued 2)
1. Model choice: random walk (with drift), ARMA, etc.
2. Bootstrapping residuals
Example: AR(1) with log returns rt = log(Pt) - log(Pt-1)
a) Estimate (with OLS) and for

rt = + rt-1 + t, t = IID(0,2); 2 = Var(t)
b) Calculate residuals
et = rt - - rt-1
c) Generate new sample with bootstrapped residuals
t = + rt-1 + t,
d) Compile new price sample
= exp(t); = log(P1)
p t p t 1 p 1
3
8.2 Markov chain Monte Carlo (MCMC)

Markov process: generic stochastic process determined with relationships
between their future, present, and past values.
Future value for the Markov process of the 1st order is determined by its
present value.
Future value for the Markov process of the 2nd order is determined by its
present value and the most recent past value, etc.
The Markov processes of the 1st order cover a very wide class of dynamic
short-memory phenomena including diffusional transfer (Brownian motion).
Discrete Markov process => Markov chain.
kth order: Pr(Xn = x | Xn-1 = xn-1, Xn-2 = xn-2,..., X1 = x1) =
Pr(Xn = x | Xn-1 = xn-1, Xn-2 = xn-2,..., Xn-k = xn-k)
Initial conditions: xn-1, ..., xn-k

Stationarity: Pr(...) does not depend on n.
4
8.2 MCMC (continued)

1st order: Pr(Xn = x | Xn-1 = xn-1, ..., X1 = x1) = Pr(Xn = x | Xn-1 = xn-1)
N2 probabilities pik = Pr(Xn = xi | Xn-1 = xk); i, k = 1, 2, ..., N => transition

matrix.
N
p
i 1
ik 1
2nd order: N3 probabilities pijk = Pr(Xn = xk | Xn-1 = xi, Xn-2 = xj) = pijk,
i, j, k = 1, 2, ..., N
N
p jik 1
i , j 1
5
8.2 MCMC (continued 2)
Implementation:
Drawings from the uniform distribution are mapped onto transition

probabilities for generating new samples.
Long memory => higher order => computational challenges
6
MCMC Example
Possible returns {-1, 0, 1}
rk = -1: Pr(rk+1 =-1 | rk = -1) + Pr(rk+1 = 0 | rk = -1) + Pr(rk+1 = 1 | rk = -1) = 1
0.5 0.3 0.2
rk = 0: Pr(rk+1 =-1 | rk = 0) + Pr(rk+1 = 0 | rk = 0) + Pr(rk+1 = 1 | rk = 0) = 1

0.3 0.4 0.3
rk = 1: Pr(rk+1 =-1 | rk = 1) + Pr(rk+1 = 0 | rk = 1) + Pr(rk+1 = 1 | rk = 1) = 1

0.2 0.3 0.5
r1 = 0; rand() = 0.5, 0.2, 0.9, 0.1 => rk = 0, 0, -1, 1, -1,

8.3 Random entry protocol
What to do about coupled samples (e.g. price & some liquidity measure
like aggregated volume at best price)?
Do we want to destroy correlations between coupled samples?
Autocorrelation bias...
Implementation:
Start trading at random point of time.
Similar to block bootstrap but the block size is determined by timing of
round-trip trading.
But: limited number of entries...
8
8.4 Comparing trading strategies

Data snooping in comparative testing on the same data sample...
One problem: many strategies that were unsuccessful in the past are
eliminated, so that only a small set of strategies is considered in the end, and
the best of them is assumed to be the best among all.
A newsletter scammer...
Bootstrap reality check (BRC) (White (2000))

T
BRC is based on Lx1 (k=1, 2, , L) statistic f k n 1 f k ,t 1 ( t )
tR
where L is the number of trading strategies, n is the number of prediction

periods indexed from R to T, T = R + n + 1, is a vector of parameters that
determine the trading strategies. t
9
8.4 Comparing trading strategies (continued)
Performance defined in terms of the excess return in respect to some

benchmark determined with 0 (e.g. risk-free interest rate)
fk,t+1() = ln[1 + yt+1Sk(t, k)] ln[1 + yt+1S0(t, 0)], k = 1, , L
yt+1 = (Xt+1 Xt)/Xt, Xt is the original price series,

Sk(t, k) and S0(t, 0) are the trading signal functions that translate the price
sequence t = {Xt -R, Xt-R+1, , XT} into the market positions.
The trading signal functions can assume values of 0 (cash), 1 (long position),
and -1 (short position).
Average return of strategy k: f=k E( fk )
10
8.4 Comparing trading strategies (continued 2)

Null hypothesis: H0: max { } f k0, k=1, ..., L; is average
fk return of strategy k
in respect to some benchmark (e.g. risk-free return)
BRC:
1) Calculate = max { }, k = 1, f..., L
V k
2) Perform stationary bootstrap for resampling fk,i , i = 1, 2, ..., B

(B is number of bootstrapped samples)
and calculate their averages
f k*,i
3) Calculate = max {( - )}, i = 1, , B
*
Vi
k = 1, ..., L f k*,i fk
4) Compare percentile of to : if it is higher than 1-p for given
significance level p, the null hypothesis is*rejected.
Vi V
11
8.4 Comparing trading strategies (continued 3)

Sullivan et al (1999): BRC on 100 years (1897 1996) of daily data for the
Dow Jones. The strategy that is the best for given sample outperforms
holding cash. However, the best in-sample strategy is not superior to the
benchmark when tested out of sample.
Hansen (2005):
BRC may have a lower power due to possible presence of poorly performing
strategies in the test. Power of the statistical test relates to the ability of
rejecting false null hypothesis. Only promising strategies should be included
in the test.
Also, Romano & Wolf (2005), and Hsu et al (2009).
12

Back-Testing of Trading Strategies: 8.1bootstrap

Uploaded by

Copyright:

Available Formats

Back-Testing of Trading Strategies: 8.1bootstrap

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Back-Testing of Trading Strategies: 8.1bootstrap

Uploaded by

Copyright:

Available Formats

8.

Back-testing of trading strategies

rand() = 0.1; 0.22; 0.95; 0.7 => n1; n1; n4; n3

Block bootstrap (preserving autocorrelations)

Stationary bootstrap for stationary (or weakly) samples:

Block size L is drawn from the discrete geometric distribution

E[L] = 1/p => estimate of p

Example: AR(1) with log returns rt = log(Pt) - log(Pt-1)

a) Estimate (with OLS) and for

8.2 Markov chain Monte Carlo (MCMC)

Initial conditions: xn-1, ..., xn-k

8.2 MCMC (continued)

N2 probabilities pik = Pr(Xn = xi | Xn-1 = xk); i, k = 1, 2, ..., N => transition

8.2 MCMC (continued 2)

Drawings from the uniform distribution are mapped onto transition

Long memory => higher order => computational challenges

rk = 0: Pr(rk+1 =-1 | rk = 0) + Pr(rk+1 = 0 | rk = 0) + Pr(rk+1 = 1 | rk = 0) = 1

rk = 1: Pr(rk+1 =-1 | rk = 1) + Pr(rk+1 = 0 | rk = 1) + Pr(rk+1 = 1 | rk = 1) = 1

r1 = 0; rand() = 0.5, 0.2, 0.9, 0.1 => rk = 0, 0, -1, 1, -1,

8.3 Random entry protocol

But: limited number of entries...

8.4 Comparing trading strategies

Bootstrap reality check (BRC) (White (2000))

where L is the number of trading strategies, n is the number of prediction

8.4 Comparing trading strategies (continued)

Performance defined in terms of the excess return in respect to some

fk,t+1() = ln[1 + yt+1Sk(t, k)] ln[1 + yt+1S0(t, 0)], k = 1, , L

yt+1 = (Xt+1 Xt)/Xt, Xt is the original price series,

Average return of strategy k: f=k E( fk )

8.4 Comparing trading strategies (continued 2)

2) Perform stationary bootstrap for resampling fk,i , i = 1, 2, ..., B

8.4 Comparing trading strategies (continued 3)

Also, Romano & Wolf (2005), and Hsu et al (2009).

You might also like