Download the full version of the ebook at

https://ebookmass.com

Machine Learning for Signal Processing: Data

Science, Algorithms, and Computational
Statistics Max A. Little

https://ebookmass.com/product/machine-learning-
for-signal-processing-data-science-algorithms-and-
computational-statistics-max-a-little/

Explore and download more ebook at https://ebookmass.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Machine Learning Algorithms for Signal and Image

Processing Suman Lata Tripathi

https://ebookmass.com/product/machine-learning-algorithms-for-signal-
and-image-processing-suman-lata-tripathi/

testbankdeal.com

Introduction to algorithms for data mining and machine

learning Yang

https://ebookmass.com/product/introduction-to-algorithms-for-data-
mining-and-machine-learning-yang/

testbankdeal.com

Fundamentals of Machine Learning for Predictive Data

Analytics: Algorithms,

https://ebookmass.com/product/fundamentals-of-machine-learning-for-
predictive-data-analytics-algorithms/

testbankdeal.com

Introduction to Algorithms for Data Mining and Machine

Learning Xin-She Yang

https://ebookmass.com/product/introduction-to-algorithms-for-data-
mining-and-machine-learning-xin-she-yang/

testbankdeal.com
Machine Learning for Biometrics: Concepts, Algorithms and
Applications (Cognitive Data Science in Sustainable
Computing) Partha Pratim Sarangi
https://ebookmass.com/product/machine-learning-for-biometrics-
concepts-algorithms-and-applications-cognitive-data-science-in-
sustainable-computing-partha-pratim-sarangi/
testbankdeal.com

Google Cloud Platform for Data Science: A Crash Course on

Big Data, Machine Learning, and Data Analytics Services
Dr. Shitalkumar R. Sukhdeve
https://ebookmass.com/product/google-cloud-platform-for-data-science-
a-crash-course-on-big-data-machine-learning-and-data-analytics-
services-dr-shitalkumar-r-sukhdeve/
testbankdeal.com

Data Science With Rust: A Comprehensive Guide - Data

Analysis, Machine Learning, Data Visualization & More Van
Der Post
https://ebookmass.com/product/data-science-with-rust-a-comprehensive-
guide-data-analysis-machine-learning-data-visualization-more-van-der-
post/
testbankdeal.com

Machine Learning for Planetary Science Joern Helbert

https://ebookmass.com/product/machine-learning-for-planetary-science-
joern-helbert/

testbankdeal.com

Signal processing for multistatic radar systems: adaptive

waveform selection, optimal geometries and pseudolinear
tracking algorithms Do■ançay
https://ebookmass.com/product/signal-processing-for-multistatic-radar-
systems-adaptive-waveform-selection-optimal-geometries-and-
pseudolinear-tracking-algorithms-dogancay/
testbankdeal.com
Machine Learning for Signal
Processing
 Machine Learning for Signal
Processing
Data Science, Algorithms, and Computational
Statistics

Max A. Little

1
 3
Great Clarendon Street, Oxford, OX2 6DP,
United Kingdom
Oxford University Press is a department of the University of Oxford.
It furthers the University’s objective of excellence in research, scholarship,
and education by publishing worldwide. Oxford is a registered trade mark of
Oxford University Press in the UK and in certain other countries
© Max A. Little 2019
The moral rights of the author have been asserted
First Edition published in 2019
Impression: 1
All rights reserved. No part of this publication may be reproduced, stored in
a retrieval system, or transmitted, in any form or by any means, without the
prior permission in writing of Oxford University Press, or as expressly permitted
by law, by licence or under terms agreed with the appropriate reprographics
rights organization. Enquiries concerning reproduction outside the scope of the
above should be sent to the Rights Department, Oxford University Press, at the
address above
You must not circulate this work in any other form
and you must impose this same condition on any acquirer
Published in the United States of America by Oxford University Press
198 Madison Avenue, New York, NY 10016, United States of America
British Library Cataloguing in Publication Data
Data available
Library of Congress Control Number: 2019944777
ISBN 978–0–19–871493–4
DOI: 10.1093/oso/9780198714934.001.0001
Printed and bound by
CPI Group (UK) Ltd, Croydon, CR0 4YY
 Preface v

Preface
Digital signal processing (DSP) is one of the ‘foundational’ but some-
what invisible, engineering topics of the modern world, without which,
many of the technologies we take for granted: the digital telephone,
digital radio, television, CD and MP3 players, WiFi, radar, to name
just a few, would not be possible. A relative newcomer by comparison,
statistical machine learning is the theoretical backbone of exciting tech-
nologies that are by now starting to reach a level of ubiquity, such as
automatic techniques for car registration plate recognition, speech recog-
nition, stock market prediction, defect detection on assembly lines, robot
guidance and autonomous car navigation. Statistical machine learning
has origins in the recent merging of classical probability and statistics
with artificial intelligence, which exploits the analogy between intelligent
information processing in biological brains and sophisticated statistical
modelling and inference.
DSP and statistical machine learning are of such wide importance to
the knowledge economy that both have undergone rapid changes and
seen radical improvements in scope and applicability. Both DSP and
statistical machine learning make use of key topics in applied mathe-
matics such as probability and statistics, algebra, calculus, graphs and
networks. Therefore, intimate formal links between the two subjects
exist and because of this, an emerging consensus view is that DSP and
statistical machine learning should not be seen as separate subjects. The
many overlaps that exist between the two subjects can be exploited to
produce new digital signal processing tools of surprising utility and effi-
ciency, and wide applicability, highly suited to the contemporary world
of pervasive digital sensors and high-powered and yet cheap, comput-
ing hardware. This book gives a solid mathematical foundation to the
topic of statistical machine learning for signal processing, including the
contemporary concepts of the probabilistic graphical model (PGM) and
nonparametric Bayes, concepts which have only more recently emerged
as important for solving DSP problems.
The book is aimed at advanced undergraduates or first-year PhD stu-
dents as well as researchers and practitioners. It addresses the founda-
tional mathematical concepts, illustrated with pertinent and practical
examples across a range of problems in engineering and science. The aim
is to enable students with an undergraduate background in mathemat-
ics, statistics or physics, from a wide range of quantitative disciplines,
to get quickly up to speed with the latest techniques and concepts in
this fast-moving field. The accompanying software will enable readers
to test out the techniques to their own signal analysis problems. The
presentation of the mathematics is much along the lines of a standard
undergraduate physics or statistics textbook, free of distracting techni-
cal complexities and jargon, while not sacrificing rigour. It would be an
excellent textbook for emerging courses in machine learning for signals.
Contents

Preface v

List of Algorithms xiii

List of Figures xv

1 Mathematical foundations 1
1.1 Abstract algebras 1
Groups 1
Rings 3
1.2 Metrics 4
1.3 Vector spaces 5
Linear operators 7
Matrix algebra 7
Square and invertible matrices 8
Eigenvalues and eigenvectors 9
Special matrices 10
1.4 Probability and stochastic processes 12
Sample spaces, events, measures and distributions 12
Joint random variables: independence, conditionals, and
marginals 14
Bayes’ rule 16
Expectation, generating functions and characteristic func-
tions 17
Empirical distribution function and sample expectations 19
Transforming random variables 20
Multivariate Gaussian and other limiting distributions 21
Stochastic processes 23
Markov chains 25
1.5 Data compression and information
theory 28
The importance of the information map 31
Mutual information and Kullback-Leibler (K-L)
divergence 32
1.6 Graphs 34
Special graphs 35
1.7 Convexity 36
1.8 Computational complexity 37
Complexity order classes and big-O notation 38
viii Contents

Tractable versus intractable problems:

NP-completeness 38

2 Optimization 41
2.1 Preliminaries 41
Continuous differentiable problems and critical
points 41
Continuous optimization under equality constraints: La-
grange multipliers 42
Inequality constraints: duality and the Karush-Kuhn-Tucker
conditions 44
Convergence and convergence rates for iterative
methods 45
Non-differentiable continuous problems 46
Discrete (combinatorial) optimization problems 47
2.2 Analytical methods for continuous convex problems 48
L2 -norm objective functions 49
Mixed L2 -L1 norm objective functions 50
2.3 Numerical methods for continuous convex problems 51
Iteratively reweighted least squares (IRLS) 51
Gradient descent 53
Adapting the step sizes: line search 54
Newton’s method 56
Other gradient descent methods 58
2.4 Non-differentiable continuous convex problems 59
Linear programming 59
Quadratic programming 60
Subgradient methods 60
Primal-dual interior-point methods 62
Path-following methods 64
2.5 Continuous non-convex problems 65
2.6 Heuristics for discrete (combinatorial) optimization 66
Greedy search 67
(Simple) tabu search 67
Simulated annealing 68
Random restarting 69

3 Random sampling 71
3.1 Generating (uniform) random numbers 71
3.2 Sampling from continuous distributions 72
Quantile function (inverse CDF) and inverse transform
sampling 72
Random variable transformation methods 74
Rejection sampling 74
Adaptive rejection sampling (ARS) for log-concave densities 75
Special methods for particular distributions 78
3.3 Sampling from discrete distributions 79
Inverse transform sampling by sequential search 79
 Visit https://ebookmass.com
now to explore a rich
collection of eBooks and enjoy
exciting offers!
 Contents ix

Rejection sampling for discrete variables 80

Binary search inversion for (large) finite sample
spaces 81
3.4 Sampling from general multivariate
distributions 81
Ancestral sampling 82
Gibbs sampling 83
Metropolis-Hastings 85
Other MCMC methods 88

4 Statistical modelling and inference 93

4.1 Statistical models 93
Parametric versus nonparametric models 93
Bayesian and non-Bayesian models 94
4.2 Optimal probability inferences 95
Maximum likelihood and minimum K-L divergence 95
Loss functions and empirical risk estimation 98
Maximum a-posteriori and regularization 99
Regularization, model complexity and data compression 101
Cross-validation and regularization 105
The bootstrap 107
4.3 Bayesian inference 108
4.4 Distributions associated with metrics and norms 110
Least squares 111
Least Lq -norms 111
Covariance, weighted norms and
Mahalanobis distance 112
4.5 The exponential family (EF) 115
Maximum entropy distributions 115
Sufficient statistics and canonical EFs 116
Conjugate priors 118
Prior and posterior predictive EFs 122
Conjugate EF prior mixtures 123
4.6 Distributions defined through quantiles 124
4.7 Densities associated with piecewise linear loss functions 126
4.8 Nonparametric density estimation 129
4.9 Inference by sampling 130
MCMC inference 130
Assessing convergence in MCMC methods 130

5 Probabilistic graphical models 133

5.1 Statistical modelling with PGMs 133
5.2 Exploring conditional
independence in PGMs 136
Hidden versus observed variables 136
Directed connection and separation 137
The Markov blanket of a node 138
5.3 Inference on PGMs 139
x Contents

Exact inference 140

Approximate inference 143

6 Statistical machine learning 149

6.1 Feature and kernel functions 149
6.2 Mixture modelling 150
Gibbs sampling for the mixture model 150
E-M for mixture models 152
6.3 Classification 154
Quadratic and linear discriminant analysis (QDA and LDA)155
Logistic regression 156
Support vector machines (SVM) 158
Classification loss functions and misclassification
count 161
Which classifier to choose? 161
6.4 Regression 162
Linear regression 162
Bayesian and regularized linear regression 163
Linear-in parameters regression 164
Generalized linear models (GLMs) 165
Nonparametric, nonlinear regression 167
Variable selection 169
6.5 Clustering 171
K-means and variants 171
Soft K-means, mean shift and variants 174
Semi-supervised clustering and classification 176
Choosing the number of clusters 177
Other clustering methods 178
6.6 Dimensionality reduction 178
Principal components analysis (PCA) 179
Probabilistic PCA (PPCA) 182
Nonlinear dimensionality reduction 184

7 Linear-Gaussian systems and signal processing 187

7.1 Preliminaries 187
Delta signals and related functions 187
Complex numbers, the unit root and complex exponentials 189
Marginals and conditionals of linear-Gaussian
models 190
7.2 Linear, time-invariant (LTI) systems 191
Convolution and impulse response 191
The discrete-time Fourier transform (DTFT) 192
Finite-length, periodic signals: the discrete Fourier trans-
form (DFT) 198
Continuous-time LTI systems 201
Heisenberg uncertainty 203
Gibb’s phenomena 205
Transfer function analysis of discrete-time LTI systems 206
 Contents xi

Fast Fourier transforms (FFT) 208

7.3 LTI signal processing 212
Rational filter design: FIR, IIR filtering 212
Digital filter recipes 220
Fourier filtering of very long signals 222
Kernel regression as discrete convolution 224
7.4 Exploiting statistical stability for linear-Gaussian DSP 226
Discrete-time Gaussian processes (GPs) and DSP 226
Nonparametric power spectral density (PSD) estimation 231
Parametric PSD estimation 236
Subspace analysis: using PCA in DSP 238
7.5 The Kalman filter (KF) 242
Junction tree algorithm (JT) for KF computations 243
Forward filtering 244
Backward smoothing 246
Incomplete data likelihood 247
Viterbi decoding 247
Baum-Welch parameter estimation 249
Kalman filtering as signal subspace analysis 251
7.6 Time-varying linear systems 252
Short-time Fourier transform (STFT) and perfect recon-
struction 253
Continuous-time wavelet transforms (CWT) 255
Discretization and the discrete wavelet transform (DWT) 257
Wavelet design 261
Applications of the DWT 262

8 Discrete signals: sampling, quantization and coding 265

8.1 Discrete-time sampling 266
Bandlimited sampling 267
Uniform bandlimited sampling: Shannon-Whittaker in-
terpolation 267
Generalized uniform sampling 270
8.2 Quantization 273
Rate-distortion theory 275
Lloyd-Max and entropy-constrained quantizer
design 278
Statistical quantization and dithering 282
Vector quantization 286
8.3 Lossy signal compression 288
Audio companding 288
Linear predictive coding (LPC) 289
Transform coding 291
8.4 Compressive sensing (CS) 293
Sparsity and incoherence 294
Exact reconstruction by convex optimization 295
Compressive sensing in practice 296
xii Contents

9 Nonlinear and non-Gaussian signal processing 299

9.1 Running window filters 299
Maximum likelihood filters 300
Change point detection 301
9.2 Recursive filtering 302
9.3 Global nonlinear filtering 302
9.4 Hidden Markov models (HMMs) 304
Junction tree (JT) for efficient HMM computations 305
Viterbi decoding 306
Baum-Welch parameter estimation 306
Model evaluation and structured data classification 309
Viterbi parameter estimation 309
Avoiding numerical underflow in message passing 310
9.5 Homomorphic signal processing 311

10 Nonparametric Bayesian machine learning and signal pro-

cessing 313
10.1 Preliminaries 313
Exchangeability and de Finetti’s theorem 314
Representations of stochastic processes 316
Partitions and equivalence classes 317
10.2 Gaussian processes (GP) 318
From basis regression to kernel regression 318
Distributions over function spaces: GPs 319
Bayesian GP kernel regression 321
GP regression and Wiener filtering 325
Other GP-related topics 326
10.3 Dirichlet processes (DP) 327
The Dirichlet distribution: canonical prior for the cate-
gorical distribution 328
Defining the Dirichlet and related processes 331
Infinite mixture models (DPMMs) 334
Can DP-based models actually infer the number of com-
ponents? 343

Bibliography 345

Index 353
List of Algorithms

2.1 Iteratively reweighted least squares (IRLS). . . . . . . . . 52

2.2 Gradient descent. . . . . . . . . . . . . . . . . . . . . . . . 53
2.3 Backtracking line search. . . . . . . . . . . . . . . . . . . . 54
2.4 Golden section search. . . . . . . . . . . . . . . . . . . . . 55
2.5 Newton’s method. . . . . . . . . . . . . . . . . . . . . . . 56
2.6 The subgradient method. . . . . . . . . . . . . . . . . . . 61
2.7 A primal-dual interior-point method for linear program-
ming (LP). . . . . . . . . . . . . . . . . . . . . . . . . . . 64
2.8 Greedy search for discrete optimization. . . . . . . . . . . 67
2.9 (Simple) tabu search. . . . . . . . . . . . . . . . . . . . . . 68
2.10 Simulated annealing. . . . . . . . . . . . . . . . . . . . . . 69
2.11 Random restarting. . . . . . . . . . . . . . . . . . . . . . . 70
3.1 Newton’s method for numerical inverse transform sampling. 73
3.2 Adaptive rejection sampling (ARS) for log-concave den-
sities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.3 Sequential search inversion (simplified version). . . . . . . 80
3.4 Binary (subdivision) search sampling. . . . . . . . . . . . 81
3.5 Gibb’s Markov-Chain Monte Carlo (MCMC) sampling. . . 83
3.6 Markov-Chain Monte Carlo (MCMC) Metropolis-Hastings
(MH) sampling. . . . . . . . . . . . . . . . . . . . . . . . . 86
5.1 The junction tree algorithm for (semi-ring) marginaliza-
tion inference of a single variable clique. . . . . . . . . . . 142
5.2 Iterative conditional modes (ICM). . . . . . . . . . . . . . 144
5.3 Expectation-maximization (E-M). . . . . . . . . . . . . . . 146
6.1 Expectation-maximization (E-M) for general i.i.d. mix-
ture models. . . . . . . . . . . . . . . . . . . . . . . . . . . 153
6.2 The K-means algorithm. . . . . . . . . . . . . . . . . . . . 172
7.1 Recursive, Cooley-Tukey, radix-2, decimation in time, fast
Fourier transform (FFT) algorithm. . . . . . . . . . . . . 210
7.2 Overlap-add FIR convolution. . . . . . . . . . . . . . . . . 224
8.1 The Lloyd-Max algorithm for fixed-rate quantizer design. 279
8.2 The K-means algorithm for fixed-rate quantizer design. . 280
8.3 Iterative variable-rate entropy-constrained quantizer design.281
9.1 Baum-Welch expectation-maximization (E-M) for hidden
Markov models (HMMs). . . . . . . . . . . . . . . . . . . 308
9.2 Viterbi training for hidden Markov models (HMMs). . . . 310
10.1 Gaussian process (GP) informative vector machine (IVM)
regression. . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
10.2 Dirichlet process mixture model (DPMM) Gibbs sampler. 337
xiv List of Algorithms

10.3 Dirichlet process means (DP-means) algorithm. . . . . . . 340

10.4 Maximum a-posteriori Dirichlet process mixture collapsed
(MAP-DP) algorithm for conjugate exponential family
distributions. . . . . . . . . . . . . . . . . . . . . . . . . . 342
List of Figures

1.1 Mapping between abstract groups 2

1.2 Rectangle symmetry group 2
1.3 Group Cayley tables 3
1.4 Metric 2D circles for various distance metrics 4
1.5 Important concepts in 2D vector spaces 5
1.6 Linear operator flow diagram 7
1.7 Invertible and non-invertible square matrices 8
1.8 Diagonalizing a matrix 9
1.9 Distributions for discrete and continuous random variables 13
1.10 Empirical cumulative distribution and density functions 20
1.11 The 2D multivariate Gausian PDF 21
1.12 Markov and non-Markov chains 25
1.13 Shannon information map 28
1.14 Undirected and directed graphs 33
1.15 Convex functions 35
1.16 Convexity, smoothness, non-differentiability 37

2.1 Lagrange multipliers in constrained optimization 42

2.2 Objective functions with differentiable and non-differentiable
points 46
2.3 Analytical shrinkage 50
2.4 Iteratively reweighted least squares (IRLS) 52
2.5 Gradient descent with constant and line search step sizes 54
2.6 Constant step size, backtracking and golden section search 55
2.7 Convergence of Newton’s method with and without line
search 57
2.8 Piecewise linear objective function (1D) 61
2.9 Convergence of the subgradient method 62
2.10 Convergence of primal-dual interior point 64
2.11 Regularization path of total variation denoising 65

3.1 Rejection sampling 75

3.2 Adaptive rejection sampling 77
3.3 Ancestral sampling 82
3.4 Gibb’s sampling for the Gaussian mixture model 83
3.5 Metropolis sampling for the Gaussian mixture model 85
3.6 Slice sampling 90

4.1 Overfitting and underfitting: polynomial models 103

xvi List of Figures

4.2 Minimum description length (MDL) 104

4.3 Cross-validation for model complexity selection 105
4.4 Quantile matching for distribution fitting 125
4.5 Convergence properties of Metropolis-Hastings (MH) sam-
pling 131

5.1 Simple 5-node probabilistic graphical model (PGM) 133

5.2 Notation for repeated connectivity in graphical models 134
5.3 Some complex graphical models 135
5.4 Three basic probabilistic graphical model patterns 137
5.5 D-connectivity in graphical models 138
5.6 The Markov blanket of a node 139
5.7 Junction trees 143
5.8 Convergence of iterative conditional modes (ICM) 145

6.1 Gaussian mixture model for Gamma ray intensity strati-

fication 151
6.2 Convergence of expectation-maximization (E-M) for the
Gaussian mixture model 152
6.3 Classifier decision boundaries 160
6.4 Loss functions for classification 161
6.5 Bayesian linear regression 163
6.6 Linear-in-parameters regression 165
6.7 Logistic regression 166
6.8 Nonparametric kernel regression 167
6.9 Overfitted linear regression on random data: justifying
variable selection 169
6.10 Lasso regression 170
6.11 Mean-shift clustering of human movement data 176
6.12 Semi-supervised versus unsupervised clustering 178
6.13 Dimensionality reduction 179
6.14 Principal components analysis (PCA): traffic count signals 181
6.15 Kernel principal components analysis (KPCA): human
movement data 184
6.16 Locally linear embedding (LLE): chirp signals 185

7.1 The sinc function 189

7.2 Heisenberg uncertainty 204
7.3 Gibb’s phenomena 206
7.4 Transfer function analysis 208
7.5 Transfer function of the simple FIR low-pass filter 214
7.6 Transfer function of the truncated ideal low-pass FIR filter216
7.7 The bilinear transform 220
7.8 Computational complexity of long-duration FIR imple-
mentations 224
7.9 Impulse response and transfer function of discretized ker-
nel regression 225
7.10 Periodogram power spectral density estimator 233
 List of Figures xvii

7.11 Various nonparametric power spectral density estimators 234

7.12 Linear predictive power spectral density estimators 236
7.13 Regularized linear prediction analysis model selection 238
7.14 Wiener filtering of human walking signals 239
7.15 Sinusoidal subspace principal components analysis (PCA)
filtering 241
7.16 Subspace MUSIC frequency estimation 242
7.17 Typical 2D Kalman filter trajectories 243
7.18 Kalman filter smoothing by Tikhonov regularization 252
7.19 Short-time Fourier transform analysis 254
7.20 Time-frequency tiling: Fourier versus wavelet analysis 256
7.21 A selection of wavelet functions 261
7.22 Vanishing moments of the Daubechies wavelet 262
7.23 Discrete wavelet transform wavelet shrinkage 263

8.1 Digital MEMS sensors 265

8.2 Discrete-time sampling hardware block diagram 267
8.3 Non-uniform and uniform and bandlimited sampling 268
8.4 Shannon-Whittaker uniform sampling in the frequency
domain 269
8.5 Digital Gamma ray intensity signal from a drill hole 271
8.6 Quadratic B-spline uniform sampling 272
8.7 Rate-distortion curves for an i.i.d. Gaussian source 277
8.8 Lloyd-Max scalar quantization 280
8.9 Entropy-constrained scalar quantization 282
8.10 Shannon-Whittaker reconstruction of density functions 283
8.11 Quantization and dithering 285
8.12 Scalar vesus vector quantization 286
8.13 Companding quantization 287
8.14 Linear predictive coding for data compression 289
8.15 Transform coding: bit count versus coefficient variance 291
8.16 Transform coding of musical audio signals 292
8.17 Compressive sensing: discrete cosine transform basis 295
8.18 Random demodulation compressive sensing 297

9.1 Quantile running filter for exoplanetary light curves 301

9.2 First-order log-normal recursive filter: daily rainfall signals302
9.3 Total variation denoising (TVD) of power spectral density
time series 303
9.4 Bilateral filtering of human walking data 304
9.5 Hidden Markov modelling (HMM) of power spectral time
series data 306
9.6 Cepstral analysis of voice signals 312

10.1 Infinite exchangeability in probabilistic graphical model

form 315
10.2 Linking kernel and regularized basis function regression 319
 Visit https://ebookmass.com
now to explore a rich
collection of eBooks and enjoy
exciting offers!
xviii List of Figures

10.3 Random function draws from the linear basis regression

model 321
10.4 Gaussian process (GP) function draws with various co-
variance functions 322
10.5 Bayesian GP regression in action 323
10.6 Gaussian process (GP) regression for classification 326
10.7 Draws from the Dirichlet distribution for K = 3 327
10.8 The Dirichlet process (DP) 331
10.9 Random draws from a Dirichlet process (DP) 332
10.10Comparing partition sizes of Dirichlet versus Pitman-Yor
processes 334
10.11Parametric versus nonparametric estimation of linear pre-
dictive coding (LPC) coefficients 339
10.12DP-means for signal noise reduction: exoplanet light curves341
10.13Probabilistic graphical model (PGM) for the Bayesian
mixture model 342
10.14Performance of MAP-DP versus DP-means nonparamet-
ric clustering 344
Mathematical foundations
1
Statistical machine learning and signal processing are topics in applied
mathematics, which are based upon many abstract mathematical con-
cepts. Defining these concepts clearly is the most important first step in
this book. The purpose of this chapter is to introduce these foundational
mathematical concepts. It also justifies the statement that much of the
art of statistical machine learning as applied to signal processing, lies in
the choice of convenient mathematical models that happen to be useful
in practice. Convenient in this context means that the algebraic conse-
quences of the choice of mathematical modeling assumptions are in some
sense manageable. The seeds of this manageability are the elementary
mathematical concepts upon which the subject is built.

1.1 Abstract algebras

We will take the simple view in this book that mathematics is based on
logic applied to sets: a set is an unordered collection of objects, often
real numbers such as the set {π, 1, e} (which has three elements), or the
set of all real numbers R (with an infinite number of elements). From
this modest origin it is a remarkable fact that we can build the entirety
of the mathematical methods we need. We first start by reviewing some
elementary principles of (abstract) algebras.

Groups
An algebra is a structure that defines the rules of what happens when
pairs of elements of a set are acted upon by operations. A kind of
algebra known as a group (+, R) is the usual notion of addition with
pairs of real numbers. It is a group because it has an identity, the
number zero (when zero is added to any number it remains unchanged,
i.e.a+0 = 0+a = a, and every element in the set has an inverse (for any
number a, there is an inverse −a which means that a+(−a) = 0. Finally,
the operation is associative, which is to say that when operating on three
or more numbers, addition does not depend on the order in which the
numbers are added (i.e. a + (b + c) = (a + b) + c). Addition also has
the intuitive property that a + b = b + a, i.e. it does not matter if the
numbers are swapped: the operator is called commutative, and the group
is then called an Abelian group. Mirroring addition is multiplication
acting on the set of real numbers with zero removed (×, R − {0}), which
is also an Abelian group. The identity element is 1, and the inverses

Machine Learning for Signal Processing: Data Science, Algorithms, and Computational Statistics. Max A. Little.

c Max A. Little 2019. Published in 2019 by Oxford University Press. DOI: 10.1093/oso/9780198714934.001.0001
Random documents with unrelated
content Scribd suggests to you:
 'Yes; and I shall cross your path no more. Our lives are shaped out for us
to a great extent, and mine was planned out for me by others. Oh, by what
infernal fatality have you, too, the name of Laura!'

'It was, I suppose, given me by my godfathers and godmothers. You

seem to be familiar with it,' she added, with one of her merriest laughs.

Dalton knew that a lover laughed at has a lost cause; he knew too—
fatally for his own peace—that the love he had for weeks upon weeks past
been striving to stifle in his breast, was a love that he had no right to offer;
but her reception of it stung him deeply, and in reply to her laughter he said,
gravely and steadily,

'Then I am to understand that you have been amusing yourself with me

—simply flirting to keep your hand in, Mrs. Trelawney?' he asked, in a
voice that was intensely low and clear.

'Precisely so,' she said, with a nod and a saucy smile; 'playing the game
that always requires two to play it.'

'What game?'

'Love-making.'

'Cruel—cruel! God may forgive you, but I never will!' he exclaimed,

and wheeling round his horse, galloped furiously away.

How astonished Dalton would have been could he have seen the change
that came over the face and manner of the lady he had just left so abruptly.

Her eyes flashed with joyous triumph, yet they were full of welling
tears; her lip quivered; her cheeks were deeply flushed; an agitation beyond
her control made her whole form to vibrate; and as she struck her gloved
hands together she exclaimed, in a low and fervent voice, with almost a sob
in it, 'At last—at last I have completely triumphed—have ground him to the
dust! At last he loves me, and I have conquered his cold, proud heart!'
 Then leaping lightly as a girl from her horse, on reaching her own gate,
she passionately embraced and kissed little Netty again and again, greatly to
the bewilderment of the child, who had never seen her mother so agitated
before.

That night she despatched a note to the camp requesting Captain Dalton
to visit her again.

All the next day passed, and no answer came.

Her excitement became intense; she sent a messenger to the North

Camp to make inquiries, and he returned with the, to her, now most startling
tidings that the Rifles had marched that morning for embarkation, and that
her note was lying undelivered in the empty hut of Captain Dalton, who had
left the lines for Southampton.

She had boasted to him laughingly and with affected pride and
bitterness of the game she had been playing. She had held a trump card in
her hand, and now it seemed that she had played and lost it.

'I have gone too far, too far, and now may lose him altogether, and after
all—after all!' she exclaimed, with genuine dismay.

CHAPTER IX.

'THE ROUTE!'

It was so; those comfortless wooden wigwams in the lines of the North
Camp, which had known the Rifles for so many months, now, in the words
of the Book of Job, knew them no more; and nothing of the smart but
sombre battalion now remained there save a few soldiers—recruits whose
training was not complete, or men whose time of service was nearly
expired.
 The mess had been broken up, its massive and trophied service of plate
packed up and placed in the charge of Goring, who had command of the
fragment of the battalion left behind. The senior captain of a regiment was
never employed on this duty, as, for obvious reasons, his presence at
headquarters is always desirable.

On the eventful morning of their march from camp the gallant battalion
of the 'Prince Consort's Own' scarcely knew themselves in their new
'Ashanti toggery,' as they called it, which was furnished from the stores at
Pimlico, and consisted, for each man, of a grey tweed tunic, resembling a
shooting-jacket, suitable for the climate, with ample pockets; belt and
trousers of the same material, and rough canvas leggings; the head-dress, a
light grey Indian helmet, perhaps the first time such a thing had been worn
on British ground.

Soldier-like looked the Rifles in their black belts and their heavy
marching order, with knapsacks, haversacks, great-coats, canteens, and
water-bottles.

If there was little of the pomp and circumstance of war in this costume,
by repetition in numbers and by uniformity in the mass it did not seem
unimposing; and if splendour was wanting, certainly enthusiasm was not,
and loud and hearty were the cheers that rang along the Lines from one
street of huts to another, as the grey column, preceded by the bands of
several corps, began its short march to the railway which was to convey it
to Southampton just as the red sun of November, the pioneer of winter,
shone out through clouds that had a ragged and dreary look in a grey and
gloomy sky.

The moorlands around Aldershot were odorous with withered bracken,

and a stray heron might have been seen, perhaps, at Fleet Pond, motionless
amid the water as if sculptured in bronze; in the adjacent thickets the
woodsman was going forth, armed with axe and bill-hook, his dog close
behind him, heedless of war and its accompaniments, pausing, perhaps, as
he heard in the distance on the ambient air the crash of the brass bands that
led the Rifles on the first part of the long route to terrible Ashanti, or it
might be the chorus of hundreds of manly voices shouting 'Cheer, boys,
cheer,' on the wind of the early morning, but he was thinking only of the
bundles of faggots on his shoulder, the crackling fire, the clean-swept
hearth, the kettle on the hob, and the trim little wife that awaited him at
home.

Bevil Goring was accompanying the battalion to Southampton to see the

last of his friends, and to 'kill,' as he thought, 'another day of suspense,' the
long and empty days of waiting with gloomy forebodings.

It seemed to him that a few hours had wrought a curious change in both
Jerry Wilmot and Tony Dalton, but more especially in the latter, who from
being a grave, earnest, and pleasant fellow had suddenly became morose,
preoccupied, and even sullen and most impatient; one thing alone seemed to
gratify him—the sudden and speedy departure to the seat of war.

'What has come to you, my dear fellow?' asked Goring more than once;
'you look as if you were going into a fever.'

'I am in a fever of the mind, Goring,' replied Dalton, 'and I may tell you
all about it before the transport sails.'

Among the crowd that assembled to see the battalion depart were many
ladies on horseback. There was one under whose tightly-tied veil the hot
tears were falling, as she saw Jerry march past in the strange Ashanti
uniform at the head of his company; but Jerry—his sad thoughts turned
inward—saw not her, and he had no prevision that she of whom his heart
was so full at that moment—Bella Chevenix—was so near him.

'Time will test his truth,' thought the girl; 'true love does not die, but the
false only, as it depends upon outward influences. Yet time may see this
regiment return, and Jerry not with it—oh, God, if it should be—not with
it!'

And the crash of the brass bands went on, and the tramp of the steadily
marching column, the flash of accoutrements and arms, the cheers, the
chorusing, the general hubbub, all portions of a terrible phantasmagoria,
amid which he was taken away from her.
 Southampton was reached in due time, and by sound of bugle the
battalion was 'detrained,' to use the term now in use, and marched to the
steam transport which lay in those busy and stately docks, where of old the
sea had ebbed and flowed upon a silent and sandy shore, and where, it is
difficult now to believe, Canute the Dane sat in a chair, and took his part in
that well-known incident by which he rebuked the flattery of his courtiers.

By a hand gangway the grey column defiled at once on board the ship,
whose capacious womb received it. The men were speedily divided into
their watches; a guard was detailed; berths were apportioned; arms racked;
knapsacks hung on pegs or cleats; bedding inspected; duck shirts and
fatigue trousers served out; and so, for a time, the officers and sergeants had
a busy time of it; while a thousand mysterious returns, receipts, and
requisitions seemed to require the signature of the colonel and everyone
else, and these were affixed on the capstan head, the gunwale, the back of
the nearest soldier, or anything else that might be improvised as a table.

Incessant was the clatter of the donkey-engines as stores were taken on

board, baggage, shot, shell, gatling-guns, waggons, provisions, wheel-
barrows, shovels, and pickaxes in bundles. Night fell, and still the odious
hurly-burly on deck and by the gaping hatchways went on, to the sound of
many a merry chorus or song at times:

'It's no matter what you do,

If your heart be only true.
And his heart was true to his Poll.'

Though our soldiers are generally too young to have wives nowadays,
in these short-service times, a few years ago it was not so; thus several
women of the Rifle Battalion, some with babies in their arms, had followed
it to Southampton to see the last of those they might never look upon again.

'Good-bye, my poor Mary,' Goring heard a young soldier cry, looking

wistfully to his girl-wife, who stood weeping on the quay, where she held
up their baby from time to time. 'How are you to get back to camp?'
 'Never mind, Tom darling; I'm here, anyhow.'

'Have you any money?'

'No.'

'God help you, darling,' he replied, and proceeded in a mechanical but

hopeless way to investigate his pockets.

'I'll take her back, and all the women of ours who are here. Pass the
message along, lads,' cried Bevil Goring, who now gave a sergeant carte
blanche to distribute money among all for what they required, and directing
them all to meet him at the railway station next morning.

'Three cheers for Captain Goring!' was now the cry, and many men
crowded gratefully forward to salute him and shake his hand, while he felt
now that he could spend some of the rupees of Bevil Goring of Chowringee
to good purpose; and sure enough he met his strange detachment at the
station next morning; and after giving them a hearty breakfast, including
buns and cans of milk galore for the little ones, he brought them all into
camp, while the transport was steaming down the waters of the Solent, and
heading for the Channel.

But in this part of our narrative we are anticipating certain events which
occurred at Southampton, and which Dalton and Goring, but more
particularly the former, were destined to have long in their memory.

CHAPTER X.

THE SECRET OF DALTON'S LIFE.

'I cannot understand the terms on which you say you and Mrs.
Trelawney have parted,' said Goring, to whom his most valued friend
Dalton had been, as a sort of relief to his own mind, apparently making
what he called 'a clean breast of it,' and detailing his relations with the fair
widow of Chilcote Grange. 'You seem to have made love enough to her—
that I saw for myself often. You seemed to have expressed admiration
enough for her, to all of which she appears to have listened with patience
and pleasure in some instances; with impatience and petulance in others;
and yet you seem to have wound up with a kind of quarrel at last!'

'She acknowledged that she had only been amusing herself and
befooling me.'

'It would also seem by your own account that amid all the curious love-
making you never made her a direct proposal of marriage.'

'No.'

'Why?'

'I dared not,' said Dalton, sadly.

'You dared not—and why?'

'Because—because I am a married man—there now, the murder is out!'

'A married man—you, Tony Dalton!' exclaimed Goring, in utter

bewilderment.

'I, Tony Dalton—the biggest fool in Her Majesty's service,' replied that
personage, with a groan.

'Does Mrs. Trelawney know of this state of affairs?' asked Goring, after
a long pause.

'I have more than once feared as much.'

'She hinted to me once that there was a secret in your life that precluded
her reception of your addresses. Then it is so?'
 'Yes, that I am a married man,' replied Dalton, as he threw open his dark
green and silk-braided patrol jacket (which he had resumed after the march)
as if its collar choked him, tossed his half-finished cigar into the blazing
fire, and drained his glass only to replenish it again.

It was in a hotel at Southampton, not far from where the transport lay,
when they were having a 'farewell drink' after a cutlet or so, that Dalton
made this astounding revelation to his friend—one that seemed fully to
account for many peculiarities which the latter had remarked in Dalton's
intercourse with Mrs. Trelawney.

'Why, in the name of all that is wonderful, have you concealed this so
long?'

'An emotion of shame perhaps—shame at my own egregious folly tied

my tongue.'

'But when, where, how did it all come about?'

'The most miserable stories are often told in a few words, and thus told
best; and, Goring, I shall tell you mine,' replied Dalton.

'When I was being educated for the service—my parents being dead—I
was boarded by my uncle Sir John Dalton—on whose hands and generosity
I was utterly cast—with a tutor at Hastings.

'My uncle was most generous. I had quarterly as much pocket-money—

too much indeed—as a young fellow in his early teens could desire to have;
I had a horse at my command, a pleasure-boat whenever I liked it, and was
a frequent attender at the theatre; for my tutor was a careless fellow, fond of
amusement too, and did not look sufficiently after me.

'All this was some ten or twelve years ago. At the theatre there was a
young girl who figured in the bills as Miss Laura Dorillion, and who was
deemed quite a star.

'One story went that she was a lady of high family, who, in a rage for
histrionic fame, had fled from home, changed her name, and adopted the
stage as a profession; another story was that she was the only daughter of a
man of rank, whom dissipation or bad speculations on the turf had ruined;
and rumour added that, when only twelve years of age, she had played
Juliet to perfection in amateur theatricals at a fashionable West End School;
at fifteen she was a genius; at seventeen she was cast as Miss Hardcastle in
the "School for Scandal;" and more than once when I saw her as Juliet I
longed, with all my soul, to be her Romeo.

'Boylike I fell madly in love with her—in love as dreamy boys at my

then years are wont to do—and nightly I haunted the theatre, often in
defiance of my tutor, and my studies became a farce; in fact they were
utterly neglected, and I had but one thought—Laura Dorillion!

'How pretty—how sweetly pretty—the name sounded to me, and I was

never weary of repeating it to myself.

'Was she pretty, you will ask? When made-up for the stage and
surrounded by all its accessories, she looked downright lovely; but, when
watching her going from her lodgings to morning rehearsal, I was obliged
to confess to myself that my goddess had rather a large mouth, but fine eyes
with a sleepy or dreamy expression, long lashes and drooping lids of which
she could make a most seductive use; that in figure she was tall but not
ungraceful, and was neither fully grown nor developed; but there seemed a
great want of finish about her for one who was alleged to be the daughter of
a noble family. This might proceed, I thought, from the style of her toilette,
which certainly did not come from Swan & Edgar's.

'The girl was quite a favourite in Hastings; she played for, sang for, and
subscribed to many local charities, and had about her none of that fastness
of dress or demeanour peculiar to so many young girls on the stage; and so I
loved her, or thought I did. I was but a boy—it was what the French—so
happy in their phrases—call un grand caprice enflammé par des obstacles
—nothing more, perhaps; and the obstacles were my lack of independent
means to take her off the stage; my having no profession; and my uncle's
well known family pride, position, and general views regarding me, his
brother's only son, and all that sort of thing. Otherwise, I might have
continued "to sigh like a furnace," and eventually, when I went elsewhere,
forget her; but it was not to be.
 'I was not a bad-looking fellow, and always dressed scrupulously well;
thus she was not long in discovering me as I sat night after night, bouquet in
hand, in a certain pit stall; and she no doubt connected me with the beautiful
bouquets that came to the stage door nightly, in more than one instance with
little complimentary notes on pink and perfumed paper inserted therein.

'Once she appeared at the wings with one of these notes in her hand. She
blew me a kiss from the tips of her fingers, and placed the missive in her
bosom, two little actions which raised me to the seventh heaven of ecstacy.
After that Laura Dorillion sang to me, acted to me, glanced and smiled at
me in a way that completed her conquest, and, in short, I was a lost Tony
Dalton!

'As a pledge of solemn engagement, I gave her a diamond and opal ring.

'In the end I achieved an introduction in the most matter-of-fact way in

the world—just as Sir Walter Scott did to his first love—by the prosaic offer
of my umbrella on a wet day, and then my dream began to take a more
tangible form in little lunches and solid presents, in escorting her to and
from the theatre, which became an established kind of expected duty; in
walks on the Sunday mornings along the towering cliffs that overhang the
sea; along the breezy Marina; by the Lover's Seat in lonely Fairlight Glen
with its thickly wooded sides and tapestry of wild flowers; by the Dripping
Well, that an enormous beech-tree overhangs; among the ruins of the old
castle, when "the old, old tale" was told again—not of Hastin and his men,
or of Saxons or Normans—but of our love for each other, and life became
to me a species of feverish intoxication for some weeks at least.

'Some little points of manner, accent, pronunciation certainly did at

times jar upon my better taste; and she seemed, for a girl educated at a West
End seminary for young ladies, rather ignorant of the manners and customs
of that "society" which she affected in genteel comedy to pourtray upon the
stage; but the former I attributed to association with her inferiors—to wit,
the members of the company to which she belonged.

'From what you know of my disposition and general character, you may
guess the end of all this.'
 'No—I do not,' said Goring.

'I married her.'

'Whew!' whistled Goring; 'in church?'

'In church! where she was given away by the manager. The "heavy old
woman" acted as mother, two young ladies of the company were
bridesmaids, and when, tremulously, she subscribed herself in the register
Laura Dorillion, the clerk and the pew-opener gave their signatures as
witnesses. The breakfast is but a confused memory. There was no rice—no
old slippers; and we are told that no girl likes to be married without any of
the gay things which make marriage such a joyous experience—no gay
preparations—no pretty wedding in a flower-decked church—no presents—
not even a new dress!'

'Well?'

'Then came a life of misery and jealousy. I trembled when other men
went near her, and boiled with exasperation when love was openly made to
her on the stage in the mere business of the play. I had seen enough of that
done before with considerable placidity, but somehow I could not stand it
now.

'With my last quarter's allowance in my pocket, and utterly vague ideas

of the future in my mind, I left the house of my tutor and went to share her
humble lodging in a rather obscure part of Hastings, and soon the sordid
nature of our surroundings began to impress me most disagreeably, as the
bubble began to burst.

'At last there came a night which I was fated not to forget for a time.

'I had brought her home from the theatre, where she had acquitted
herself with singular skill and sweetness as blind Iolanthe in "King René's
Daughter," and she was in the act of repeating a portion of her dialogue
with Tristan as we ascended the stair—
 "Another time,
When I had pined for many tedious days,
Because my father was detained from home,
I wept for very gladness when he came!
Through tears I gave my bursting heart relief,
And at mine eyes it found a rushing vent."

'In our little sitting-room I found an elderly man, wearing a battered

grey hat girt by a black band, and clad in shabby-genteel—nay, quite
threadbare garments—standing on the hearthrug, smoking a short clay pipe,
with his coat-tails over his arms, his bleared and tipsy-looking eyes—one of
which had a white plaister over it—regarding the furniture and details of the
apartment critically, while he took a sip from a pewter mug of beer, and set
it down with a clank.

'"Hullo, my girl," he exclaimed; "here you are at last! This here is a rum
go. So this is the young gent as you have gone and made such a fool of
yourself by marrying?"

'Laura's heart was beating fast—so fast that even respiration seemed to
suffocate her; her face was blanched; her eyes had a scared expression; and
gave me a glance that seemed full of shame and agony.

'"Who is this impertinent scoundrel?" I demanded.

'"Scoundrel in your teeth again!" he exclaimed, turning up the cuffs of

his coat, threateningly, and striking his battered hat firmly on his head; "is
this your company manners, you young cub?" he added, with a frightful
imprecation.

'"Who are you, and what do you want here?" I demanded, looking about
for a stick.

'"Dabchick is my name! Jo Dabchick, clown, Banger's Circus,

Surreyside o' the river, and no mistake; and I have come here to see my own
daughter, Laura Dorillion, as she calls herself, or must it be Mrs. Antony
Dalton now—Lady Dalton perhaps that's to be, when your uncle hops his
blessed twig?"

'"Oh, father," said Laura, in a breathless voice, "why have you come,
and how did you find me out?"

'"I come because I want money; and, as for finding you out, that was
easy enough; the Hastings theatre ain't at the bottom of the sea."

"'And mother?"

'"Is there in your bed—has had a drop too much, and so I have tucked
her in there; and now what have you got for supper—tripe, sausages,
bloaters, or summat tasty, I hope? Speak—you look as lively as a couple of
glow-worms in the sunshine!"

'My soul sickened within me! And with these additions to our little
household—a slatternly, odious mother, a beery, broken-down actor, whose
line had once been genteel comedy, a clown in a circus latterly, but whose
incessant dissipation had deprived him of all employment—life became a
burden now, and my stupendous folly stood in letters of fire before me.

'Existence became unendurable, and neither Laura nor I dared to look

forward to the dark and vague future we might be doomed to drag out in the
world.

'Their arrival filled my wife with shame and anger, and I do believe with
generous sorrow for me. My quarter's pittance was soon expended; her
salary could not maintain us all. My tutor soon discovered the whole
situation, and laid it mercilessly bare before my uncle, Sir John Dalton, who
from that hour cast me off, ignored my letters and my existence, and
disinherited me by his will.

'I had no money, or means of getting any, after the best of my jewellery
and wardrobe had departed. Laura's father and mother soon proved abusive
and most obnoxious to me; they insulted me hourly, and eventually drove
me from the squalid lodgings we shared together. Laura one night took their
part; it required but that to fill up the measure of my disgust, and I found
myself wandering in the streets with all I possessed in the world—the
clothes that I wore. I rooted the love of her out of my heart; but it was long
before I could efface her image, which often a fancied resemblance in
another brought before me.

'There are some men of whom it is said that they will not acknowledge
their false steps even to their own hearts; but I am not one of them, and
must acknowledge, dear Goring, that in sackcloth and ashes I have repented
of mine.

'My haughty uncle proving obdurate to the last degree, there was no
hope for me so far as he was concerned; so I took the Queen's shilling and
sailed for India, and there I strove to forget my boyish folly, the
contemptible position I had occupied with such a father and mother-in-law,
the disgust and horror with which their advent and their surroundings
inspired me—sick, too, of the slatternly girl I had married, for slatternly she
too was in her home and when off the stage, reserving all her toilettes and
her graces for the British public.

'You know the rest. I soon got a commission through the ranks—sooner
than I could have got it through the medium of a crammer and exams. From
the hour I turned at midnight along the Marina of Hastings, and heard the
monotonous sound of the surge, as it rolled on the beach in the dark, I have
never heard of my wife or been able to trace her. Her odious parents I
discovered have been long since dead, and that she is no longer on the
stage, or, if so, bears another name, or has gone I know not where.

'I have sometimes hoped that I had been freed from her by death—
ungenerous though that hope may be, and that my uncle must have heard of
her demise, when by a codicil to his will he left me all his fortune. And now
you know why it was that I dared not make a proposal to Mrs. Trelawney—
nor did I ever think of love or marriage till I met her lately; and how I love
her, and have struggled to tear that hopeless passion from my heart, is
known only to God and to myself!'
 CHAPTER XI.

THE OLD LOVE AND THE NEW.

'Poor Dalton! you have indeed suffered deeply—paid dear for your
boyish folly,' said Goring, as the former concluded the little story of his
early life in a voice tremulous with emotion. 'Now the apparent
inconsistency of your attention to Mrs. Trelawney is quite accounted for.'

'Until I knew myself free to ask her to be my wife I had sworn in my

inner heart that I would not do so—indeed, I dared not do so; yet, for the
life of me, attracted as I was, I could not help hovering about her; but now I
am going to Ashanti, and there is an end of it! Such was the end of the fatal
passion of a foolish lad. Since those days I have never entered a theatre, and
shudder at the mere idea of a dramatic situation.'

'You are in one now,' said a sweet and tremulous voice, as Mrs.
Trelawney, who, unseen and unheard by them in their preoccupation, had
softly entered the room, stood before them.

How much or how little she had overheard they did not precisely know,
but with a smile of mingled sadness and sweetness, pride and triumph, she
threw up her veil, and the full light of the gasalier overhead fell upon her
rich, shining hair, her beautiful and animated countenance.

'Mrs. Trelawney!' exclaimed the friends together, as they started from

their chairs.

'You here—in Southampton!' added Dalton, in a voice tremulous with

bewilderment.

'Yes. Can it be that you have yet to learn that I am that Laura Dorillion
to whom you gave this opal and diamond ring, with whom you spent so
many a sweet hour, by Fairlight Glen, the East Cliffs that overhang the sea,
by the Dripping Well, and amid the old castle walls at Hastings—the Laura
whom you married, and from whom you so coldly fled?'
 Dalton tried to speak, but his voice was gone; he could but stretch his
hand towards her, without advancing, while regarding her with growing
bewilderment; so she spoke again, with tears in her voice.

'You ought to have forgiven me the humility of my origin, for that I

could not help—ay, forgiven me long ago, Anthony. Remember that "he
who cannot forgive breaks the bridge over which he must pass himself," for
"every man hath need of forgiveness," we are told.'

'My wife—you?' exclaimed Dalton.

'Laura—your own wife, whom you married in St. Clement's Church on

the tenth of August. You remember the day?'

The words were simple, but spoken with great pathos, and all her
sparkling manner seemed to have left her as she seated herself, and he hung
over her.

'Do you forgive me, Laura, and pardon me—pardon me, and love me?'

'You know that I love you.'

He was about to put an arm round her, when he paused, and said,

'But whence this name of Trelawney?'

'I assumed it from an aunt, who left me a small fortune, but for which I
—I might have been compelled to struggle in penury on the stage to support
your daughter.'

'My daughter!' exclaimed Dalton, a great love for the beautiful little girl
lie knew suddenly gushing up in his heart. 'My daughter—mine?'

'Our child,' said Laura, softly.

'Born—when?'

'Six months after you left me at Hastings.'

 'But her name—of Antoinette?'

'Is but the feminine of your own—Anthony.'

'Oh, what a blind fool I have been—Laura—Laura!'

Goring, who had been studying a picture on the wall, now thought he
might as well withdraw softly, and smoke his cigar outside.

Taller in stature, fuller in figure, more fully developed in every way, and
with a bearing, manner, and grace cultivated by those among whom she had
moved, it required a certain effort to recognise in her the girlish Laura
Dorillion of the past time. Though her whole style was different—finer and
more statuesque—and the mode of her toilette and of dressing her hair was
different, her voice and the inflections of it, her expression of eye, the droop
of the lid and flicker of the long lash, and the sweet smile of her lip were, he
now saw, all unchanged, and he pressed her to his breast in the rapture of
the moment, forgetting that the transport which was soon to bear him away
was at that supreme moment of joy having her fires banked up preparatory
to putting to sea.

'And you love me, Laura?' he never was tired of repeating, and hearing
the sweet admission that she did so. 'Oh, why have you concealed till this—
why have you concealed yourself thus, and from me?'

'I wished to try you—to test you—to compel you to love me, and I have
done so, have I not?' she asked, taking his face between her hands and
gazing tenderly into his eyes.

'You know now what fettered my tongue,' said he, with a sigh.

'I knew you were in bondage—but it was in bondage to me. Your love
for me was an insult to myself; your compliments and intentions in the
present time, were an implied insult to my past. You dared to love me,
knowing that you had a wife somewhere—where, you knew not; but you
little thought that Mrs. Trelawney, the supposed widow of a mythical
Trelawney, and Laura Dorillion were one and the same person. Now, is the
situation dramatic? Do you remember that you told me that you loved me
against your own will and conscience, and that my very name of Laura
repressed that love at times? Heavens, could you but know what I felt—
how my heart was wrung—my woman's pride alternately roused and
crushed by admissions such as these! I have suffered greatly, darling, but all
is over now,' she added, laying her cheek on his breast, while his lips were
pressed to her forehead.

After a time, she spoke again.

'I knew not that you were in the Army, or were in life. I knew not of
your existence till I met you suddenly at Aldershot, after I had lived years
of seclusion in the Channel Isles. I thanked God for the discovery; I vowed
to win you again, if I could, before I would reveal myself—and I have done
so.'

She whose love he had so longed and prayed for, and yet striven to root
out of his heart, was now his own—his own after all; and all the pent-up
love of lonely years had found reward at last.

'Often before I met you again—discovered you, and vowed to make you
mine again, I had pondered that, but for Netty's sake, whether, taking it all
in all, the good with the bad, life was worth living,' said she, her eyes full of
tears now.

'And till now, Laura, my life has seemed a gloomy and empty one. I was
often appalled by the aimlessness and isolation of it.'

'How strange it is that you never discovered me!' said Laura; 'yet I have
seen your eyes wander more than once to this engagement ring.'

'And stranger still that when I heard you sing the old, old song that was
once so familiar, making my heart thrill with troubled memories, no light
came to me. Oh, Laura, you acted well your part to this joyous ending.'

'I told you that I had found the verses in an old album, where a friend
wrote them years ago; that friend was yourself. You remember so lately
telling me that I had ruined your life?'
 'Yes, Laura, and your cruel smile.'

'God knows how at that moment I longed to cast myself on your breast,
Anthony, as I do now, and barter all my past wrongs for a single kiss!'

His Christian name again on her lips, as in the days of their boy and girl
love, ere the black change came, and how strange, yet familiar—how sweet,
how dear it sounded!

'How did you learn I was here with Goring?' he asked, tenderly.

'I learned it at the transport.'

'My darling—my darling, why have you kept all this secret so long—the
secret that you were my own?'

'As I told you, I would never be more to you than I am now, were I to
live a hundred years, and was I not right?'

'But to keep the secret so long—I might never have learned it, for to-
morrow will see us separated. Hastings was the first volume of our
romance.'

'Southampton was the second.'

'The third is Ashanti. How it will end, God alone knows, darling,' he
added, straining her in his embrace, while her tears fell fast now; 'if spared
to return to you, Laura—to you and little Netty—you will never regret your
love and trust in me at last—your confidence in my affection.'

Poor Dalton—'if spared;' he was right to say that, with the fate and
fortune of a barbarous war before him.

The old love had become the new one, and the new love was the old;
and yet it seemed that to-night both had entered on a new relationship.

And, as we have said, the two last appeals of Jerry Wilmot and Dalton
ended differently. Practically they came to the same conclusion—a
separation from those they loved.
 Laura now deplored deeply her pride and folly, as she deemed it, in
playing the game she had done so long; but the separation had to be faced
and endured; yet she watched the transport, as it steamed down
Southampton water, till it melted into the haze; and it was not until then that
she fully realised that her husband, so lately restored to her, was gone again,
and perhaps for ever.

But that her appearance on board would have excited speculation in the
battalion, she would have gone down the Channel with the steamer and
come ashore in the pilot's boat at Deal.

On the long, though rapid voyage, Dalton had ample food for reflection,
for thinking of the strangeness of his fate, that for months past he had been
associating with, meeting and seeing at intervals, and loving deeply, a
woman who was his own wife, and yet he knew it not!

Why had she played this perilous game so long?

Why, but for the temptation to win him again, and for the gratification
of a kind of affectionate vengeance. And now they were separated, each
with but a memory to the other again.

A few photos and two locks of hair—the light blond hair of his wife, a
golden curl of his little daughter—were all that poor Dalton took with him
to the burning coast of Ashanti, to remind him of the happiness he had so
lately and so briefly tasted, and might never taste more.

CHAPTER XII.

BEVIL GORING'S RESOLUTION.

In his bitter anxiety Bevil Goring condescended again to apply to Mr.

Solomon Slagg as to the movements of Lord Cadbury; but ignorant perhaps