Return to Main LECTURE 15: LINEAR PREDICTION

Objectives
● Objectives:
Basic Theory:
Simple Case ❍ Introduce the theory of linear prediction
General Case
Covariance ❍ Develop autocorrelation and covariance techniques for solution
Autocorrelation ❍ Understand similarities with regression
Error
❍ Explain the relationship to windowing and maximum entropy
Spectral Matching:
Spectrum ❍ Add a new technique to our signal modeling block diagram
Noise-Weighting
Preemphasis There is a classic textbook on this subject:
Typical Front End
J.D. Markel and A.H. Gray, Linear Prediction of Speech, Springer-Verlag, New
On-Line Resources: York, New York, USA, ISBN: 0-13-007444-6, 1976.
Signal Modeling
Linear Prediction This lecture also includes material from two other textbooks:
MAD Speech
J. Deller, et. al., Discrete-Time Processing of Speech Signals, MacMillan Publishing
Co., ISBN: 0-7803-5386-2, 2000.

and,

L.R. Rabiner and B.W. Juang, Fundamentals of Speech Recognition, Prentice-Hall,

Upper Saddle River, New Jersey, USA, ISBN: 0-13-015157-2, 1993.
Return to Main

Introduction:

01: Organization
(html, pdf)

Speech Signals:
ECE 8463: FUNDAMENTALS OF SPEECH
02: Production
(html, pdf) RECOGNITION
03: Digital Models Professor Joseph Picone
(html, pdf) Department of Electrical and Computer Engineering
Mississippi State University
04: Perception
(html, pdf) email: [email protected]
phone/fax: 601-325-3149; office: 413 Simrall
05: Masking URL: http://www.isip.msstate.edu/resources/courses/ece_8463
(html, pdf)
Modern speech understanding systems merge interdisciplinary technologies from Signal Processing,
06: Phonetics and Phonology Pattern Recognition, Natural Language, and Linguistics into a unified statistical framework. These
(html, pdf) systems, which have applications in a wide range of signal processing problems, represent a revolution
in Digital Signal Processing (DSP). Once a field dominated by vector-oriented processors and linear
07: Syntax and Semantics algebra-based mathematics, the current generation of DSP-based systems rely on sophisticated statistical
(html, pdf) models implemented using a complex software paradigm. Such systems are now capable of
understanding continuous speech input for vocabularies of hundreds of thousands of words in
Signal Processing: operational environments.

08: Sampling In this course, we will explore the core components of modern statistically-based speech recognition
(html, pdf) systems. We will view speech recognition problem in terms of three tasks: signal modeling, network
searching, and language understanding. We will conclude our discussion with an overview of state-of-
09: Resampling the-art systems, and a review of available resources to support further research and technology
(html, pdf) development.

10: Acoustic Transducers Tar files containing a compilation of all the notes are available. However, these files are large and will
(html, pdf) require a substantial amount of time to download. A tar file of the html version of the notes is available
here. These were generated using wget:
11: Temporal Analysis
(html, pdf)
wget -np -k -m http://www.isip.msstate.edu/publications/courses/ece_8463/lectures/current
12: Frequency Domain Analysis
(html, pdf) A pdf file containing the entire set of lecture notes is available here. These were generated using Adobe
Acrobat.
13: Cepstral Analysis
(html, pdf) Questions or comments about the material presented here can be directed to [email protected].

14: Exam No. 1

(html, pdf)

15: Linear Prediction

(html, pdf)

16: LP-Based Representations

(html, pdf)

Parameterization:

17: Differentiation
(html, pdf)

18: Principal Components

(html, pdf)
19: Linear Discriminant Analysis
(html, pdf)
 LECTURE 15: LINEAR PREDICTION

● Objectives:

❍ Introduce the theory of linear prediction

❍ Develop autocorrelation and covariance techniques for solution

❍ Understand similarities with regression

❍ Explain the relationship to windowing and maximum entropy

❍ Add a new technique to our signal modeling block diagram

There is a classic textbook on this subject:

J.D. Markel and A.H. Gray, Linear Prediction of Speech, Springer-Verlag, New York, New York, USA, ISBN: 0-
13-007444-6, 1976.

This lecture also includes material from two other textbooks:

J. Deller, et. al., Discrete-Time Processing of Speech Signals, MacMillan Publishing Co., ISBN: 0-7803-5386-2,
2000.

and,

L.R. Rabiner and B.W. Juang, Fundamentals of Speech Recognition, Prentice-Hall, Upper Saddle River, New
Jersey, USA, ISBN: 0-13-015157-2, 1993.
LINEAR PREDICTION - A SIMPLE DERIVATION
LINEAR PREDICTION - GENERAL CASE
THE COVARIANCE METHOD
THE AUTOCORRELATION METHOD
LINEAR PREDICTION ERROR
SPECTRAL MATCHING INTERPRETATION
 NOISE FLOORING
VIA THE AUTOCORRELATION FUNCTION
 SIGNAL CONDITIONING COMPENSATES FOR
MICROPHONE AND CHANNEL CHARACTERISTICS
A TYPICAL SPEECH RECOGNITION FRONT END
Index of
/publications/journals/ieee_proceedings/1993/signal_modeling
Name Last modified Size Description

Parent Directory 01-Jan-1999 12:07 -

paper_v2.pdf 22-Jun-1999 16:14 452k

Apache/1.3.9 Server at www.isip.msstate.edu Port 80

Next: Motivation from lossless tubes Up: Speech Analysis Previous: Z transforms

Linear Prediction analysis

Linear prediction analysis of speech is historically one of the most important speech analysis techniques. The basis is the
source-filter model where the filter is constrained to be an all-pole linear filter. This amounts to performing a linear
prediction of the next sample as a weighted sum of past samples:

This linear filter has the transfer function:

A good introductory article is [8], and this subject is also covered well in [1, 2, 3].

● Motivation from lossless tubes

● Parameter estimation
● The autocorrelation method
● The covariance method
● Pre-emphasis
● The LP spectrum
● Gain computation
● The lattice filter implementation
● The Itakura distance measure
● The LP cepstrum
● Log area ratios
● The roots of the predictor polynomial
● Line spectral pairs

Speech Vision Robotics group/Tony Robinson

 linear prediction of speech

introduction | demonstration | investigate | reading | credits | downloading | home

Introduction
This tool allows users to explore linear prediction of speech and other signals. It also reinforces the
concepts of windowing, preemphasis and frame size selection in speech analysis. Users can select regions
for analysis and see the results as lpc-smoothed spectra and poles. The residual signal and its spectrum is
also displayed. This tool is not a substitute for learning about linear prediction. To understand what the
various panels mean, you should read one of the texts listed below. The demo requires the MATLAB
signal processing toolbox.

The tool
Type 'lpcspect' to launch the demo. The file menu is used to load an existing signal file or to create a new
one using the createsig tool. Supported formats currently include .wav, .snd and .au sound files. The signal
will appear in panel (1), and associated linear prediction signals for the region between the cursors will be
displayed in the other panels. Use the cursors to select any segment. On cursor release, the segment
appears in panel (2) along with linear prediction residual (error signal). Panel (3) shows the DFT spectrum
of this segment, overlaid with the LPC-smoothed spectrum and the DFT of the residual (7). The poles
corresponding to the linear predictor are shown in panel (4).

The LPC order can be changed using menu (8). The signal can be preemphasised by checking the relevant
box in cluster (6). Cursors can be linked (so that a constant segment length is maintained). The waveform
segment chosen will be played on release if the associated checkbox is checked.

Clicking on any signal waveform (incuding the residual) causes it to be played. Holding the mouse button
down in the z-plane panel (4) will cause the frequency at the selected z-plane angle to be displayed in the
top left corner of the panel.

The chosen waveform segment can be Hamming-windowed (5). Unchecked, a rectangular window is
applied.
Things to investigate
1. Choose a vowel segment of speech. Examine what happens to the residual signal (and its mean
square error) as you increase the number of poles. What happens to the LPC-smoothed spectrum?
How many poles provide a comfortable fit to a vowel segment.
2. Now repeat this analysis for unvoiced sounds such as fricatives. How does the error compare?
3. Again, perform this analysis for a nasal sound.
4. Can you predict where the poles will appear for a given analysis order, and how they will change
when you modify the order?
5. Explore the effect of preemphasis on the LPC spectrum and the number of poles required to
obtain a given residual error.
6. Listen to the waveform segment and the residual signal in sequence. Can you detect a change in
timbre? How does this change manifest itself as the LPC order increases?
7. What is the effect of changing LPC order on the spectrum of the residual signal?
8. Choose a vowel sound and a relatively low LPC order (10, say). Read off the pole locations
(frequencies) by holding the mouse down in panel (4). Now use the polezero tool to recreate just
these pole locations and listen to the resulting sound.
9. Use the create option on the file menu to create a harmonic series with, say, 5 components. Do
you expect to be able to fit it well with a 10th order model? Try it.
10. Now add some noise (using create) to your harmonic series. Do the pole locations change? (Read
about the peak-hugging properties of LPC).

Further reading
● A good introduction to LPC is provided by Makhoul, J. (19??)
● This tool was written partly as an example of the interface issues in speech and hearing
demonstrations, and is described further in Cooke et al (1999) The interactive auditory
demonstrations project, Eurospeech'99, Budapest, September.

Credits etc
Produced by: Martin Cooke, April 1999.

Permissions: This demonstration may be used and modified freely by anyone. It may be distributed in
unmodified form.