Introduction to Applied Statistical Signal Analysis Guide to

Biomedical and Electrical Engineering Applications 3rd

Edition

Visit the link below to download the full version of this book:

https://medipdf.com/product/introduction-to-applied-statistical-signal-analysis-
guide-to-biomedical-and-electrical-engineering-applications-3rd-edition/

Click Download Now

Academic Press is an imprint of Elsevier
30 Corporate Drive, Suite 400, Burlington, MA 01803, USA
525 B Street, Suite 1900, San Diego, California 92101-4495, USA
84 Theobald’s Road, London WC1X 8RR, UK

This book is printed on acid-free paper.

Copyright © 2007, Elsevier Inc. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or by any

means, electronic or mechanical, including photocopy, recording, or any information
storage and retrieval system, without permission in writing from the publisher.

Permissions may be sought directly from Elsevier’s Science & Technology Rights
Department in Oxford, UK: phone: (+44) 1865 843830, fax: (+44) 1865 853333,
E-mail: [email protected]. You may also complete your request on-line
via the Elsevier homepage (http://elsevier.com), by selecting “Support & Contact”
then “Copyright and Permission” and then “Obtaining Permissions.”

Library of Congress Cataloging-in-Publication Data

Application Submitted

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library.

ISBN 13: 978-0-12-088581-7

ISBN 10: 0-12-088581-6

For information on all Academic Press publications

visit our Web site at www.books.elsevier.com

Printed in the United States of America

06 07 08 09 10 9 8 7 6 5 4 3 2 1

Working together to grow

libraries in developing countries
www.elsevier.com | www.bookaid.org | www.sabre.org
 CONTENTS

∗ = new section
PREFACE xiii
DEDICATION xv
ACKNOWLEDGMENTS xvii
LIST OF SYMBOLS xix

1
INTRODUCTION AND TERMINOLOGY
1.1 Introduction 1
1.2 Signal Terminology 3
1.2.1 Domain Types 3
1.2.2 Amplitude Types 5
1.2.3 Basic Signal Forms 6
1.2.4 The Transformed Domain—The Frequency Domain 8
1.2.5 General Amplitude Properties 9
1.3 Analog to Digital Conversion 10
1.4 Measures of Signal Properties 11
1.4.1 Time Domain 11
1.4.2 Frequency Domain 12
References 13

v
vi CONTENTS

2
EMPIRICAL MODELING AND APPROXIMATION
2.1 Introduction 15
2.2 Model Development 16
2.3 Generalized Least Squares 21
2.4 Generalities 23
2.5 Models from Linearization 24
2.6 Orthogonal Polynomials 28
2.7 Interpolation and Extrapolation 33
2.7.1 Lagrange Polynomials 34
2.7.2 Spline Interpolation 38
2.8 Overview 43
References 43
Exercises 44

3
FOURIER ANALYSIS
3.1 Introduction 51
3.2 Review of Fourier Series 53
3.2.1 Definition 53
3.2.2 Convergence 60
3.3 Overview of Fourier Transform Relationships 61
3.3.1 Continuous versus Discrete Time 61
3.3.2 Discrete Time and Frequency 63
3.4 Discrete Fourier Transform 64
3.4.1 Definition Continued 64
3.4.2 Partial Summary of DFT Properties and Theorems 65
3.5 Fourier Analysis 68
3.5.1 Frequency Range and Scaling 69
3.5.2 The Effect of Discretizing Frequency 70
3.5.3 The Effect of Truncation 73
3.5.4 Windowing 77
3.5.5 Resolution 79
3.5.6 Detrending 82
3.6 Procedural Summary 82
3.7 Selected Applications 82
References 86
Exercises 87
Appendix 3.1: DFT of Ionosphere Data 92
Appendix 3.2: Review of Properties of Orthogonal Functions 93
 CONTENTS vii

Appendix 3.3: The Fourier Transform 94

Appendix 3.4: Data and Spectral Windows 98

4
PROBABILITY CONCEPTS AND SIGNAL CHARACTERISTICS
4.1 Introduction 101
4.2 Introduction to Random Variables 102
4.2.1 Probability Descriptors 102
4.2.2 Moments of Random Variables 108
4.2.3 Gaussian Random Variable 110
4.3 Joint Probability 112
4.3.1 Bivariate Distributions 112
4.3.2 Moments of Bivariate Distributions 113
4.4 Concept of Sampling and Estimation 115
4.4.1 Sample Moments 115
4.4.2 Significance of the Estimate 119
4.5 Density Function Estimation 122
4.5.1 General Principle for  2 Approach 122
4.5.2 Detailed Procedure for  2 Approach 124
*4.5.3 Quantile-Quantile Approach 127
4.6 Correlation and Regression 130
*4.6.1 Estimate of Correlation 130
*4.6.2 Simple Regression Model 132
4.7 General Properties of Estimators 136
4.7.1 Convergence 136
4.7.2 Recursion 137
*4.7.3 Maximum Likelihood Estimation 138
4.8 Random Numbers and Signal Characteristics 139
4.8.1 Random Number Generation 140
4.8.2 Change of Mean and Variance 141
4.8.3 Density Shaping 142
References 145
Exercises 146
Appendix 4.1: Plots and Formulas for Five Probability Density Functions 154

5
INTRODUCTION TO RANDOM PROCESSES AND
SIGNAL PROPERTIES
5.1 Introduction 155
5.2 Definition of Stationarity 156
viii CONTENTS

5.3 Definition of Moment Functions 159

5.3.1 General Definitions 159
5.3.2 Moments of Stationary Processes 160
5.4 Time Averages and Ergodicity 162
5.5 Estimating Correlation Functions 166
5.5.1 Estimator Definition 166
5.5.2 Estimator Bias 168
5.5.3 Consistency and Ergodicity 168
5.5.4 Sampling Properties 170
5.5.5 Asymptotic Distributions 171
5.6 Correlation and Signal Structure 176
5.6.1 General Moving Average 176
5.6.2 First-Order MA 177
5.6.3 Second-Order MA 181
5.6.4 Overview 181
5.7 Assessing Stationarity of Signals 182
*5.7.1 Multiple Segments—Parametric 184
*5.7.2 Multiple Segments—Nonparametric 189
References 193
Exercises 194
Appendix 5.1: Variance of Autocovariance Estimate 197
Appendix 5.2: Stationarity Tests 198

6
RANDOM SIGNALS, LINEAR SYSTEMS,
AND POWER SPECTRA
6.1 Introduction 201
6.2 Power Spectra 201
*6.2.1 Empirical Approach 201
*6.2.2 Theoretical Approach 203
6.3 System Definition Review 205
6.3.1 Basic Definitions 205
6.3.2 Relationships between Input and Output 208
6.4 Systems and Signal Structure 210
6.4.1 Moving Average Process 210
6.4.2 Structure with Autoregressive Systems 211
6.4.3 Higher-Order AR Systems 215
6.5 Time Series Models for Spectral Density 219
References 225
Exercises 226
 CONTENTS ix

7
SPECTRAL ANALYSIS FOR RANDOM SIGNALS:
NONPARAMETRIC METHODS
7.1 Spectral Estimation Concepts 229
7.1.1 Developing Procedures 233
7.1.2 Sampling Moments of Estimators 234
7.2 Sampling Distribution for Spectral Estimators 239
7.2.1 Spectral Estimate for White Noise 239
7.2.2 Sampling Properties for General Random Processes 242
7.3 Consistent Estimators—Direct Methods 244
7.3.1 Spectral Averaging 224
7.3.2 Confidence Limits 248
7.3.3 Summary of Procedure for Spectral Averaging 258
7.3.4 Welch Method 259
7.3.5 Spectral Smoothing 259
7.3.6 Additional Applications 263
7.4 Consistent Estimators—Indirect Methods 264
7.4.1 Spectral and Lag Windows 264
7.4.2 Important Details for Using FFT Algorithms 266
7.4.3 Statistical Characteristics of BT Approach 267
7.5 Autocorrelation Estimation 275
References 277
Exercises 278
Appendix 7.1: Variance of Periodogram 281
Appendix 7.2: Proof of Variance of BT Spectral Smoothing 283
Appendix 7.3: Window Characteristics 284
Appendix 7.4: Lag Window Functions 285
Appendix 7.5: Spectral Estimates from Smoothing 286

8
RANDOM SIGNAL MODELING AND PARAMETRIC
SPECTRAL ESTIMATION
8.1 Introduction 287
8.2 Model Development 288
8.3 Random Data Modeling Approach 293
8.3.1 Basic Concepts 293
8.3.2 Solution of General Model 296
x CONTENTS

8.3.3 Model Order 300

8.3.4 Levinson-Durbin Algorithm 305
8.3.5 Burg Method 309
8.3.6 Summary of Signal Modeling 313
8.4 Power Spectral Density Estimation 314
8.4.1 Definition and Properties 314
8.4.2 Statistical Properties 318
8.4.3 Other Spectral Estimation Methods 320
8.4.4 Comparison of Nonparametric and Parametric Methods 322
References 323
Exercises 324
Appendix 8.1: Matrix Form of Levinson-Durbin Recursion 327

9
THEORY AND APPLICATION OF CROSS CORRELATION
AND COHERENCE
9.1 Introduction 331
9.2 Properties of Cross Correlation Functions 333
9.2.1 Theoretical Function 333
9.2.2 Estimators 334
9.3 Detection of Time-Limited Signals 339
9.3.1 Basic Concepts 340
9.3.2 Application of Pulse Detection 342
9.3.3 Random Signals 343
9.3.4 Time Difference of Arrival 345
9.3.5 Marine Seismic Signal Analysis 347
9.3.6 Procedure for Estimation 347
9.4 Cross Spectral Density Functions 349
9.4.1 Definition and Properties 349
9.4.2 Properties of Cross Spectral Estimators 351
9.5 Applications 354
9.6 Tests for Correlation between Time Series 355
9.6.1 Coherence Estimators 355
9.6.2 Statistical Properties of Estimators 358
9.6.3 Confidence Limits 359
9.6.4 Procedure for Estimation 362
9.6.5 Application 362
References 364
Exercises 365
 CONTENTS xi

*10
ENVELOPES AND KERNEL FUNCTIONS
10.1 The Hilbert Transform and Analytic Functions 367
10.1.1 Introduction 367
10.1.2 Hilbert Transform 368
10.1.3 Analytic Signal 370
10.1.4 Discrete Hilbert Transform 373
10.2 Point Processes and Continuous Signals via Kernel Functions 375
10.2.1 Concept 375
10.2.2 Nerve Activity and the Spike Density Function 378
References 382
Exercises 383

APPENDICES
Table A Values of the Standardized Normal cdf z
Table B Student’s t Distribution
Table C Chi-Square Distribution
∗
Table D Critical Points for the Q-Q Plot Correlation Coefficient Test for Normality
Table E F Distribution Significance Limit for 97.5th Percentile
Table F Percentage Points of Run Distribution

INDEX 393
This page intentionally left blank
 PREFACE

his book presents a practical introduction to signal analysis techniques that are commonly used in

T a broad range of engineering areas such as biomedical engineering, communications, geophysics,

speech, etc. In order to emphasize the analytic approaches, a certain background is necessary.
The book is designed for an individual who has a basic background in mathematics, science, and computer
programming that is required in an undergraduate engineering curriculum. In addition one needs to have
an introductory-level background in probability and statistics and discrete time systems.
The sequence of material begins with definitions of terms and symbols used for representing discrete
data measurements and time series/signals and a presentation of techniques for polynomial modeling and
data interpolation. Chapter 3 focuses on the windowing and the discrete Fourier transform. It is introduced
by presenting first the various definitions of the Fourier transform and harmonic modeling using the
Fourier series. The remainder of the book deals with signals having some random signal component
and Chapter 4 reviews the aspects of probability theory and statistics needed for subsequent topics. In
addition, histogram fitting, correlation, regression, and maximum likelihood estimation are presented. In
the next chapter these concepts are extended to define random signals and introduce the estimation of
correlation functions and tests of stationarity. Chapter 6 reviews linear systems and defines power spectra.
Chapter 7 presents classical spectral analysis and its estimators. The periodogram and Blackman-Tukey
methods are covered in detail. Chapter 8 covers autoregressive modeling of signals and parametric spectral
estimation. Chapter 9 presents the classical uses of cross correlation and coherence functions. In particular,
the practical techniques for estimating coherence function are presented in detail. Chapter 10 is a new
chapter for the third edition and covers envelope estimation and kernel functions. Presentation of these
topics is motivated by the growth in usage of these techniques. Envelope estimation is important not
only for signals such as electromyograms but also when using high frequency carrier signals such as in
ultrasound applications. The fundamentals of Hilbert transforms, analytic signals, and their estimation are

xiii
xiv PREFACE

presented. Kernel functions appear in the neuromuscular literature dealing with point processes such as
action potentials. The main purpose is to create a continuous amplitude function from a point process.
A summary of kernel functions and their implementation is presented.
The material in Chapter 10 is drawn with permission from the doctoral dissertation work of Robert
Brychta and Melanie Bernard. They are both graduate students in Biomedical Engineering at Vanderbilt
University. Robert’s research is being done in the General Clinical Research Center and Melanie’s is
being done in the Visual System’s laboratory.
The presentation style is designed for the individual who wants a theoretical introduction to the basic
principles and then the knowledge necessary to implement them practically. The mode of presentation is
to: define a theoretical concept, show areas of engineering in which these concepts are useful, define the
algorithms and assumptions needed to implement them, and then present detailed examples that have been
implemented using FORTRAN and more recently MATLAB. The exposure to engineering applications
will hopefully develop an appreciation for the utility and necessity of signal processing methodologies.
The exercises at the end of the chapters are designed with several goals. Some focus directly on the
material presented and some extend the material for applications that are less often encountered. The degree
of difficulty ranges from simple pencil and paper problems to computer implementation of simulations
and algorithms for analysis. For an introductory course, the environment and software recommended are
those that are not overly sophisticated and complex so that the student cannot comprehend the code or
script. When used as a course textbook, most of the material can be studied in one semester in a senior
undergraduate or first year graduate course. The topic selection is obviously the instructor’s choice.
Most of the examples and many of the exercises use measured signals, many from the biomedical
domain. Copies of these are available from the publisher’s Website.3 Also available, for interactive
learning, are a series of MATLAB notebooks that have been designed for interactive learning.12 These
notebooks are written in the integrated environment of Microsoft Word and MATLAB. Each notebook
presents a principle and demonstrates its implementation via script in MATLAB. The student is then asked
to exercise other aspects of the principle interactively by making simple changes in the script. The student
then receives immediate feedback concerning what is happening and can relate theoretical concepts to
real effects upon a signal. The final one or two questions in the notebooks are more comprehensive
and ask the student to make a full implementation of the technique or principle being studied. This
requires understanding all of the previous material and selecting, altering, and then integrating parts of
the MATLAB script previously used.

1
Shiavi, R., Learning Signal Processing Using Interactive Notebooks. IEEE Transactions on Education; 42:355-CD, 1999.
2
Shiavi, R. “Teaching Signal Processing Using Notebooks”. ASEE Annual Conference; Charlotte NC, June, 1999.
3
Http://books.elsevier.com/companions/9780120885817
 This book is dedicated to my wife, Gloria,
and to my parents who encouraged me and gave me
the opportunity to be where I am today.
This page intentionally left blank
 ACKNOWLEDGMENTS

The author of a textbook is usually helped significantly by the institution by which he is employed
and through surrounding circumstances. In particular I am indebted to the Department of Biomedical
Engineering and the School of Engineering at Vanderbilt University for giving me some released time
and for placing a high priority on writing this book for academic purposes. This being the third edition,
There have been three sets of reviewers. I would like to thank them because they have contributed to the
book through suggestions of new topics and constructive criticism of the initial drafts. In addition, I am
very grateful to Robert Brychta and Melanie Bernard, both graduate students in Biomedical Engineering
at Vanderbilt University. Their doctoral research provided the basis for the topics in Chapter 10.

xvii
This page intentionally left blank