RECURSIVE BAYESIAN LOCATION OF A DISCONTINUITY IN

TIME SERIES
J.J. 0 Ruanaidh W .J . Fitzgerald K . J . Pope
Department of Engineering, University of Cambridge, U. K.

ABSTRACT number of zero right half-rows is equal to the po-

sition of the changepoint, and thus the form of the
This paper presents a Bayesian ;ipproach to the matrix G depends explicitly on the position of the
location of a discontinuity in linearly modelled changepoint m.
data. A matrix formulation is introduced which
Y1 - YO Y-1 0 0
allows the modelling of changepoints in general :YO 0 0
YZ Y1
linear models. Linear niodels investigated include
abrupt changes in the mean of a Gaussian ran- .. .. .. ..
dom variable, and pic,cewise polynomials such as Ym = Ym-1Ym-2 0 0
splines, as well as autoregressive models. The ap- Ym+l 0 0 Ym Ym-1
proach facilitates the removal of nuisance parame- .. .. .. ..
ters by integration. A general recursive techniquc YN - 0 0 YN-IYN-2
for updating Bayesian posterior densities, which
can result in large .;avings in computation. is also The likelihood of the data is given by the joint
described. probability of the noise samples.
P (d I { m )fJ b MI= HE,,
P (4 (3)
1. The Detection of Change- where o is the standard deviation of the Gaussian
noise, b 6 RP denotes the linear parameters and
points using the General M denotes prior information - or any assumptions
Piecewise Linear Model which led to this particular choice of signal model.

A matrix formulation of the signal model will be p ( d / { m } g b M ) = ( 2 r o z ) - ' e x p ( - 3eTe

) (4)
employed that enables us to treat all linear-in-the-
parameters models (such as polynomial fits to data
where { m } denotes the changepoints (non-linear
as well as autoregressive models) in exactly the
parameters) in the matrix of basis functions G.
same way. The data is modelled a!<
Substituting e = d - Gb into equation 4 gives,

where gk(i) is the value of a time dependent model ITsing a flat prior probability for the parameters
function g k ( t ) evaluated at time ti and where the a k , 6 k and a Jeffreys' prior for the standard de-
noise samples ei are drawn from a normal distribu- viation of t,he Gaussian noise, 6 Ruanaidh and
tion. This equation may be expressed in the form Fitzgerald [11 and Kheradrnandnia (21 show how
+
d := Gb e where d is an N x 1 vector of data these parameters can be integrated out to give the
points and G is an A4 x N matrix, where each following student-t distribution for the position of
column is a basis function vector used to fit the the changepoint m.
dat,a d. Each element of the vector b is the linear
coefficient for each basis vector in C:.
Fot. exaniplc, a two pole AR model may be ex-
pressed in t,he form of quation 2. Note that the

IV-513
$3.00 8 1994 IEEE
0-7803-1775-0/94
 IMI

5j
0-

Figure 1. Discrete step in Gaussian noise Figure 3. Performance of step detector

Figure 2 . Bayesian step detection Figure 4. Pole plot

where p is the number of linear coefficients. This

expression must. be evaluated for candidate values This is generally expensive to comput,e, especially
of m, and the most probable selected. when the time series is long or the number of linear
'The matrix based approach is very powerful. It parameters is large. A recursive implementation
is possible to integrate out one parameter at a resulting in significant savings in computation is
therefore highly desirable. A recursive procedure
time as shown hy Smith [3] and 6 Ruanaidh and based on the Woodbury matrix inverse update for-
Fitzgerald 111 but, this approach suffers t,he twin mula 141 for obtaining the posterior probability of
disadvantages that, one must start, from first prin- the changepoint being at position m + 1, that is
ciples for each new estimation problem and t,hat
the resultant mathematical expressions appear far
+
p ( m 1 I d M), from the posterior probability
1) ( m 1 d M) will now be defined.
more complicat,etl and difficult to work with than
expression 6 above. In contrast, equation 6 is In all, there are three cogent ternis in equation 6
directly applicable to a wide variet,y of differcat, which may be updated recursively, namely GTd,
problems as it. stantls. (G'G)-l and det (G'G). The change in pos-
terior probability can be traced back to a change
of a single row of the matrix G. Let R be the
2. Recursive Bayesian Estima- old mth row and let, Q be the new row that re-
tion places it. For example, in the case of equation 2
above we find that R = [ym-l ym-2 0 01 and
The approach described above requires K evalua- Q = [0 0 ym-l ym-2]. 'We wish to updat,e t,he key
tions of expressioii 6 for the posterior probability. matrix terms as follows,

IV-514
Figure 5. Bayesian changepoint detection in autore- Figure 7. Noisy cubic spline data with a single
gressive data changepoint

.IO"

Figure 6. Three pole autoregressive data with a sin- Figure 8. Bayesian changepoint detection in polyno-
gle changepoint mial data

(G?'G)-' + G R*R 4- Q ~ Q ) - (~7 )

(G~- The following steps are performed twice, firstly
with S = -R and secondly with S = Q.

det ( G T G ) t det (GTG- R T R + QTQ) ( 8 ) x,+1= x* + dmS (11)

w =aiST
d"G t drG - d,R + d,Q (9)
X=(1+SW)
This can be carried out' in two stages. The first
stage is to replace the mth row of G with a row
of zeroes. The second stage is t o replace this row *,+I =A
of zeros with the row matrix Q. The recursive
update scheme is as follows. Define the matricm, = +pi- W W ~ / A
@ -= (G7.G)-', x = drG, D = d'd and A =
Finally, the posterior density p ( m + 1 I d M) may
clet (GTG). b(1 computed as,
The posterior density p ( mI d M) is given by,
p ( m + l ( d M ) o c A - $[ D - x X x X T ] (16)
p (nr 1 d I ) o( A-' [ D - y axT] (10)

IV-515
 about the “true” changepoint position.
Consider the data plotted in figure 9. This geo-
physical data contains measurements of nuclear
magnetic response and conveys information about
rock structure and in particular, the boundaries
between different rock strata. 0 Ruanaidh and
Fitzgerald [l] niodelled this data as a Laplacian
random process with thirteen abrupt changes in
the mean. The results of a Bayesian analysis using
the Gibbs’ sampler and the hletropolis algorithm
are shown with the data in figure 9.

4. Conclusion
Figure 9. Well Log Data
Modelling changepoints by means of linear-in-the-
parameters model based Bayesian analysis has
been discussed. A recursive procedure, based on
3. Simulations the Woodbury formula, has been derived whose
application generally results in significant reduc-
200 data points coiisisting of Gaussian noise of tions in computation time. Simulations using
standard deviat,iori IT = 1 added to a downward polynoniials and autoregressive models have given
step from unity to zero at position TA = 140 are excellent results. Results for non Gaussian data
plotted in figurv 1. Figure 2 shows the posterior with multiple changepoints has also been pre-
proba,bility of the c,hangepoint. position. Exam- sented.
ining t,he data by eve indicates a changepoint, at Further work to be carried out by the authors at
1tt E 120. The Bayesian approach clearly deter- present includes extending the above to click de-
mines t,he correct, armwer. tection in audio signals, as part of the work being
200 data poiiits w ( w generat,ecl tiy an AR(3) sys- carried nut in the Signal Processing Laboratory on
t em whose pole positions shifted instantaneously techniques for t,he rc,storation of old gramophone
at m = 140 in figure 3. The pole positions are plot- records.
ted in figure: 4, where ‘I’ and ‘2’ signify the pole
liositions both before and after the changepoint
respectively. Figure 5 shows the posterior proba- References
bilit,y of the changepoint position for a particular
noise realisation for which the correct changepoint [‘I] J.J. 6 Ruanaidh and W.J. Fitzgerald. The
position is inferred. The question arises as 1,o how identification of discontinuities in time series
\yell the methocl works in general. Monte Ca1.10 using the bayesian piecewise linear model.
siinulations involving 1000 triitls of this dctector Technical report, Department of Engineering,
in noise redisations were ca.r- University of Cambridge, 1993.
r i d nut. The histogram of t,hv changepoint posi-
tions choseii in vach t,rial is shown in figure 6 which [2] M. Kheratlmandnia. Aspects of Bayesian
indicates that t lit! tletector performs well in most Threshold ilutoregressive Modelling. PhD the-
cases. sis, University of Kent, England, 1991.
’Two sections of it cubic spliiic fit, with added 13) A.F.M. Smith. A bayesian approach to infer-
Gaussian noise and standard deviation a == 0.01, ence about a changepoint in a sequence of ran-
comprisct the 300 (lata points in figure 7. The
dom variables. In Biometrika, pages 407-416,
changepoint which cannot be determined by eye
1975.
i s at position nt = 100. The posterior probability
of t,ht, cliangepoinl, position is shown in figure 8. [4] Press W.H. et al. Numerical Recipes i n C.
Simply applying a smoothing filter to the d;tt,a fol- Cambridge University Press, 1992.
lowed by numerical differentiat,ion does not give
i;ood rc?sult,s.Even for this extremely taxing prob-
lem, the Bayesian detect,or performs: well. ‘The
peak is I)iased, 1)iii. t lie probability mass is (.entared

IV-516