Linear Prediction: The Problem, Its Solution and Application To Speech
Linear Prediction: The Problem, Its Solution and Application To Speech
net/publication/241802223
CITATIONS READS
12 428
3 authors, including:
David Dorran
Technological University Dublin - City Campus
33 PUBLICATIONS 147 CITATIONS
SEE PROFILE
All content following this page was uploaded by David Dorran on 24 January 2014.
2008-01-01
David Dorran
Dublin Institute of Technology, [email protected]
Mikel Gainza
Dublin Institute of Technology, [email protected]
Recommended Citation
O'Cinneide, A., Dorran, D., Gainza, M.: Linear Prediction: The Problem, its Solution and Application to Speech. DIT Internal
Technical Report.
This Working Paper is brought to you for free and open access by the Audio
Research Group at ARROW@DIT. It has been accepted for inclusion in
Conference papers by an authorized administrator of ARROW@DIT. For
more information, please contact [email protected],
[email protected].
Alan Ó Cinnéide
August 2008
4 Examples 16
4.1 Human Speech: Voiced Vowel . . . . . . . . . . . . . . . . . . . . 16
4.2 Human Speech: Unvoiced Fricative . . . . . . . . . . . . . . . . . 16
4.3 Trumpet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1
1 Overview and Introduction
Linear prediction is a signal processing technique that is used extensively in the
analysis of speech signals and, as it is so heavily referred to in speech processing
literature, a certain level of familiarity with the topic is typically required by
all speech processing engineers. This paper aims to provide a well-rounded
introduction to linear prediction, and so doing, facilitate the understanding
of the technique. Linear prediction and its mathematical derivation will be
described, with a specific focus on applying the technique to speech signals. It
is noted, however, that although progress in linear prediction has been driven
primarily by speech research, it involves concepts that prove useful to digital
signal processing in general.
First to be discussed within the paper are general linear time-invariant sys-
tems, along with its theory and mathematics, before moving into a general
description of linear prediction models. The equations that yield one variant of
linear prediction coefficients are derived and the methods involved to solve these
equations are then briefly discussed. Different interpretations of the equations
yield slightly different results, and these differences will be explained.
A section focussing specifically on the linear prediction of speech then be-
gins. The anatomical process of speech production is described, followed by
an introduction to a theoretical linear model of the process. The limitations
of applying the linear prediction model to speech are described, and comments
are also given concerning certain practicalities that are specific to the linear
prediction of speech.
The paper concludes with an implementation of linear prediction using two
different types of signal. It is hoped that the balance between theory and prac-
tical will allow the reader for easy assimilation of this technique.
2
on sun-spot analysis, but has since been applied to problems in neuro-physics,
seismology as well as speech communication.
This section will review linear systems and, elaborating upon them, derives
the mathematics of linear prediction.
A linear system is such that produces its output as a linear combination of its
current and previous inputs and its previous outputs [13]. It can be described
as time-invariant if the system parameters do not change with time. Mathemat-
ically, linear time-invariant (LTI) systems can be represented by the following
equation:
q
X p
X
y(n) = bj x(n − j) − ak y(n − k) (1)
j=0 k=1
This is the general difference equation for any linear system, with output signal
y and input signal x, and scalars bj and ak , for j = 1 . . . q and k = 1 . . . p where
the maximum of p and q is the order of the system. The system is represented
graphically in figure 1.
3
reveal the transfer function H(z) of such a system:
p
X q
X
y(n) + ak y(n − k) = bj x(n − j)
k=1 j=0
p
X q
X
ak y(n − k) = bj x(n − j) where a0 = 1
k=0 j=0
p
X q
X
ak z −k Y (z) = bj z −j X(z)
k=0 j=0
q
X
bj z −j
Y (z) j=0
⇒ H(z) = = p (2)
X(z) X
−k
ak z
k=0
The coefficients of the input and output signal samples in equation (1) reveal
the poles and zeros of the transfer function.
Linear prediction follows naturally from the general mathematics of linear
systems. As the system output is defined as a linear combination of past samples,
the system’s future output can be predicted if the scaling coefficients bj and ak
are known. These scalars are thus also known as the predictor coefficients of
the system [9].
The general linear system transfer function gives rise to three different types
of linear model, dependent on the form of the transfer function H(z) given in
equation (2) [9, 7].
• The third and most general case is the mixed pole/zero model, also called
the autoregressive moving-average (ARMA) model, where nothing is as-
sumed about the transfer function.
The all-pole model for linear prediction is the most widely studied and im-
plemented of the three approaches, for a number of reasons. Firstly, the in-
put signal, which is required for ARMA and all-zero modelling, is oftentimes
4
an unknown sequence. As such, they are unavailable for use in our deriva-
tions. Secondly, the equations derived from the all-pole model approach are
relatively straight-forward to solve, contrasting sharply with the nonlinear equa-
tions dervied from ARMA or all-zero modelling. Finally, and perhaps the most
important reason why all-pole modelling is the preferred choice of engineers,
many real world applications, including most types of speech production, can
be faithfully modeled using the approach.
Following from the linear system equation (1), one can formulate the equations
necessary to determine the parameters of an all-pole linear system, the so-called
linear prediction normal equations. First, following on from the all-pole model
(see Figure 2), a linear prediction estimate ŷ at sample number n for the output
signal y by a pth order prediction filter can be given by:
p
X
ŷ(n) = − ak y(n − k) (3)
k=1
The error or residue between the output signal and its estimate at sample n
5
The total squared error for an as-of-yet unspecified range of signal samples is
given by the following equation:
X
E= [e(n)]2
n
X
= [y(n) − ŷ(n)]2 (5)
n
X
= [y(n)]2 − 2 · y(n) · ŷ(n) + [ŷ(n)]2
n
Equation (5) gives a value indicative of the energy in the error signal. Ob-
viously, it is desirous to choose the predictor coefficients so that the value of E
is minimised over the unspecified interval. The optimal minimising values can
be determined through differential calculus, i.e. by obtaining the derivative of
equation 5 with respect to each predictor coefficient and setting that value equal
to zero.
∂E
= 0 for 1 ≤ k ≤ p
∂ak
∂ X
⇒ ( ([y(n)]2 − 2 · y(n) · ŷ(n) + [ŷ(n)]2 )) = 0
∂ak n
X ∂ X ∂
−2 y(n) · ŷ(n) + 2 ŷ(n) · ŷ(n) = 0
n
∂ak n
∂ak
X ∂ X ∂
y(n) · ŷ(n) = ŷ(n) · ŷ(n)
n
∂a k n
∂a k
∂
ŷ(n) = −y(n − k) . . . from equation (3)
∂ak
X X
⇒ y(n) · −y(n − k) = ŷ(n) · −y(n − k)
n n
X X p
X
− y(n) · y(n − k) = (− ai y(n − i)) · −y(n − k)
n n i=1
X p
X X
− y(n) · y(n − k) = ai y(n − i) · y(n − k) (6)
n i=1 n
For the sake of brevity and future utility, a correlation function φ is defined.
The expansion of this summation describes what will be called the correlation
matrix.
X
φ(i, k) = y(n − i) · y(n − k) (7)
n
6
more compactly:
p
X
−φ(0, k) = ai φ(i, k) (8)
i=1
The derived set of equations are called the normal equations of linear prediction.
The limits on the summation of the total squared energy were omitted from
equation (5) so as to give their selection special attention. The section will
show that two different but logical summation intervals lead to a two different
sets of normal equations and result in different predictor coefficients.
Given sufficient data points and appropriate limits, the normal equations
define p equations with p unknowns which can be solved by any general simul-
taneous linear equation solving algorithms, e.g. Gaussian elimination, Crout
decomposition, etc. However, certain limits lead to matrix redundancies and al-
low for efficient solutions that can significantly reduce the computational load.
The autocorrelation method of linear prediction minimises the error signal over
all time, from −∞ to +∞. When dealing with finite digital signals, the signal is
windowed such that all samples outside the interval of interest are taken to be 0
(see Figure 3). If the signal is non-zero from 0 to N − 1, then the resulting error
7
over this interval is mathematically equivalent to summing over all time.
∞
X
E= [e(n)]2
n=−∞
−1+p
NX
(9)
2
= [e(n)]
n=0
When these limits are applied to equation (7), a useful property emerges.
Because the error signal is zero outside the analysis interval, the correlation
function of the normal equations can be identically expressed in a more conve-
nient form.
−1+p
NX
φauto (i, k) = y(n − i) · y(n − k) 1 ≤ i ≤ p 1 ≤ k ≤ p
n=0
N −1+(i−k)
X
= y(n) · y(n + (i − k)) 1 ≤ i ≤ p 1 ≤ k ≤ p
n=0
8
These redundancies mean that the normal equations can be solved using
the Levinson-Durbin method, an recursive procedure that greatly reduces the
computational load.
Using these limits, an examination of the equation (7) reveals that the signal val-
ues required for the calculation extend beyond the analysis interval (see Figure
4).
Figure 4: The covariance method require −p samples (shown here in red) beyond
the analysis interval from 0 to N − 1 (shown in blue).
N
X −1
φcovar (i, k) = y(n − i) · y(n − k) 1 ≤ i ≤ p 1 ≤ k ≤ p (11)
n=0
Each of these solutions to the linear prediction normal equations has its own
strengths and weaknesses; determining which is more advantageous to use is
greatly determined by the signal being analysed. When analysis signals are long,
9
the two different solutions are virtually identical. Because of the greater redun-
dancies in the matrix defined by the autocorrelation method, it is slightly easier
to compute [9]. Experimental evidence indicates that the covariance method is
more accurate for periodic speech sounds [1], while the autocorrelation method
performs better for fricative sounds [9].
Following Figure 5, the vast majority of human speech sounds are produced in
the following manner [3]. The lungs initiate the speech process by acting as
the bellows that expels air up into the other regions of the system. The air
pressure is maintained by the intercostal and abdominal muscles, allowing for
the smooth function of the speech mechanisms. The air that leaves the lungs
then enters into the remaining regions of the speech production system via the
trachea. This organ system, consisting of the lungs, trachea and interconnecting
channels, is known as the pulmonary tract. The turbulent air stream is driven
up the trachea into the larynx. The larynx is a box-like apparatus that consists
of muscles and cartilage. Two membranes, known as the vocal folds, span the
structure, supported at the front by the thyroid cartilage and at the back by the
arytenoid cartilages. The arytenoids are attached to muscles which enable them
to approximate and separate the vocal folds. Indeed, the principal function of
the larynx, unrelated to the speech process, is to seal the trachea by maintaining
the vocal folds closed. This has the dual benefit of being able to protect the
pulmonary tract and permit the build up of pressure within the chest cavity
necessary for certain exertions and coughing [9].
The space between the vocal folds is called the glottis. A speech sound is
classified as voiced or voiceless depending on the glottal behaviour as air passes
10
Figure 5: The human speech production system. Image taken from
http://cobweb.ecn.purdue.edu/˜ee649/notes/physiology.html.
11
other chambers of the vocal tract.
The fully realised phone is radiated out of the body via either the lips or
the nose or both. This is a continuous process, during which the state and
configuration of the system’s constituents alter and change dynamically with
the thoughts of the speaker.
The acoustic theory of speech production assumes the speech production process
to be a linear system, consisting of a source and filter [11]. This model captures
the fundamentals of the speech production process described in the previous
section: a source phonation modulated in the frequency domain by a dynami-
cally changing vocal tract filter, Figure 6. According to the source-filter theory,
Figure 6: The simplified speech model proposed by the acoustic theory of speech
production.
Glottal source. The source signal is one of two states: a pulse train of a
certain fundamental frequency for voiced sounds and white noise for un-
voiced sounds. This two-state source fits reasonably well with true glottal
behaviour, though moments of mixed excitation cannot be represented
well.
12
Vocal tract filter. The vocal tract is parameterised by its resonances, which
are called formants 1 . All acoustic tubes have natural resonances, the
parameters of which are a function of its shape.
Though the vocal tract changes its shape, and thus its resonances, continuously
with running speech, it is not unreasonable to assume it static over short-time
intervals of the order of 20 milliseconds. Thus, speech production can be viewed
as a LTI system and linear prediction can be applied to it.
In truth, the speech production system is known to have some nonlinearity ef-
fects and the glottal source and vocal tract filter are not completely de-coupled.
In other words, the acoustic effects of the vocal tract has been noticed to modu-
late depending on the behaviour of the source in ways that linear systems cannot
fully describe. Additionally the vocal tract deviates from the behaviour of an
all-pole filter during the production of certain vocal sounds.
System linearity. Linear systems by definition assume that inputs to the sys-
tem have no effect on the system’s parameters [13]. In the case of the
speech production process, this means that the vibratory behaviour of the
glottis has no bearing on the formant frequencies and bandwidths - an
assumption which is sometimes violated [2]. Especially in the situation
where the pitch of the voice is high and the centre frequency first formant
low, an excitation pulse can influence the decay of the previous pulse.
All pole model. The described method of linear prediction works on the as-
sumption that the frequency response of the vocal tract consists of poles
only. This supposition is acceptable for most voiced speech sounds, but is
not appropriate for nasal and fricative sounds. During the production of
these types of utterances, zeros are produced in the spectrum due to the
trapping of certain frequencies within the tract. The use of a model lack-
ing representation of zeros in addition to poles should not cause too much
concern, as if p is of high enough order, the all pole model is sufficient for
almost all speech sounds [11].
1 The word formant comes from the Latin verb formāre meaning “to shape”.
13
Despite these limitations, all-pole linear prediction remains a highly useful
technique for speech analysis.
characterise correctly. Thus, the prediction order should be twice the number
of formants present in the signal bandwidth. For a vocal tract of 17 centimetres
14
long, there is an average of one formant per kilohertz of bandwidth.
Where p represents the prediction order and fs the signal’s sampling fre-
quency, the following formula is used as a general rule of thumb [9]:
fs
p= +γ (12)
1000
The general consensus of the speech processing community is that linear predic-
tive analysis of voiced speech should be confined to the closed glottal condition
[5]. Indeed, it has been shown that closed phase covariance method linear predic-
tion yields better formant tracking and inverse filtering results that than other
pitch synchronous and asynchronous methods [1]. During the glottal closed
phase the the signal represents a freely decaying oscillation, theoretically en-
suring the absence of source-filter interaction and thus better adhering to the
assumptions of linear prediction [15].
Some voice types are unsuited to this type of analysis [10]. In order to obtain
a unique solution to the normal equations, a critical minimum of signal samples
must exist related to the signal’s bandwidth. High-pitched voices are known
to have closed phases that are too short for analysis purposes. Other voices,
particularly breathy voices, are known to exhibit continuous glottal leakage.
There are numerous methods used to determine the closed glottal interval.
The first attempts to do so typically used special laboratory techniques, such as
electroglottography [4], recorded simultaneously with the digital audio. More
recently, efforts have focused on ascertaining the closed glottal interval through
the analysis of the recorded speech signal [14, 6]. Some of these techniques has
met significant success, particularly the DYPSA algorithm of Naylor et al. [8]
that successfully identifies the closed glottal instant in more than 90% of cases
(Figure 8).
15
Figure 8: A speech waveform with the closed glottal interval highlighted in red
and delimited by circles. These regions represent the instants of glottal closing
and opening respectively, as identified by the DYPSA algorithm.
4 Examples
Within this section, some implementations of linear prediction are given, along
with all the practical considerations taken for the analysis.
A voice sample of a male voice, recorded at a sampling rate of 44.1 kHz, was
analysed. The signal is the voiced vowel sound /a/. As it is periodic, covariance
method linear prediction during the closed glottal phase yield the most accurate
formant values.
The signal was first processed by the DYPSA algorithm to determine the
closed glottal interval, which underwent covariance analysis. The order of the
prediction filter, calculated according to formula (12), was determined to be 46,
see figure 9.
An unvoiced vocal sample was also analysed. This segment, sampled at a rate
of 9 kHz, is taken from the TIMIT speech database and is of the fricative sound
R
/ /, the sh found in both “shack” and “cash”, as pronounced by an American
female.
As autocorrelation linear prediction analysis performs better with unvoiced
sounds, that method was implemented with a filter of prediction order of 11,
16
Figure 9: Top: time-domain representation of /a/ sound. Below: The sound’s
spectrum and spectral envelope as determined by covariance method linear pre-
diction analysis.
R
Figure 10: Top: time-domain representation of / / sound. Below: The
sound’s spectrum and spectral envelope as determined by autocorrelation
method linear prediction analysis.
4.3 Trumpet
Though this report has primarily concerned itself with the linear prediction of
speech, linear prediction also has applications for musical signal processing [12].
Certain instrumental sounds, such as brass instruments, exhibit strong formant
structure that lend themselves well to modelling through linear prediction.
In this example, given in figure 11, a B[ trumpet sample was analysed, play-
ing the E[5. Trumpets are known to exhibit 3 formants, indicating a prediction
17
order of 6 is required to determine all the resonances. As the signal is periodic,
covariance method linear prediction analysis is performed.
References
[1] S. Chandra and Lin Wen. Experimental comparison between stationary
and nonstationary formulations of linear prediction applied to voiced speech
analysis. Acoustics, Speech, and Signal Processing [see also IEEE Transac-
tions on Signal Processing], IEEE Transactions on, 22(6):403–415, 1974.
[2] D. G. Childers. Speech Processing and Synthesis Toolboxes. John Wiley &
Sons, Inc., New York, 2000.
[3] Michael Dobrovolsky and Francis Katamba. Phonetics: The sounds of lan-
guage. In William O’Grady, Michael Dobrovolsky, and Francis Katamba,
editors, Contemporary Linguistics: An Introduction. Addison Wesley Long-
man Limited, Essex, 1997.
18
[6] Changxue Ma, Y. Kamp, and L. F. Willems. A frobenius norm approach
to glottal closure detection from the speech signal. Speech and Audio Pro-
cessing, IEEE Transactions on, 2(2):258–265, 1994.
[9] T. W. Parsons. Voice and speech processing. McGraw-Hill New York, 1987.
[12] Curtis Roads. The Computer Music Tutorial. MIT Press, Boston, Mas-
sachusetts, 1996.
[15] D. Wong, J. Markel, and Jr. Gray, A. Least squares glottal inverse filtering
from the acoustic speech waveform. Acoustics, Speech, and Signal Process-
ing [see also IEEE Transactions on Signal Processing], IEEE Transactions
on, 27(4):350–355, 1979.
19