Computa tion

Visua liza tion

Higher-Order Spectral
Analysis Toolbox
Users Guide
Version 2
Ananthram S wami
J erry M. Mendel
Chrysostomos L. (Max) Nikias
How to Conta ct the Authors
Please note that support and maintenance is provided by United S ignals & S ystems, Inc. (the
authors of the toolbox), and is no longer provided by The MathWorks.
[email protected] Technical suppor t and pr oduct enhancement suggest ions
[email protected] Sales, pr icing, and gener al infor mat ion
The soft war e descr ibed in t his document is fur nished under a license agr eement . The soft war e may be used
or copied only under t he t er ms of t he license agr eement . No par t of t his manual may be phot ocopied or r epr o-
duced in any for m wit hout pr ior wr it t en consent fr om Unit ed Signals & Syst ems, Inc.
FEDERAL ACQUISITION: This pr ovision applies t o all acquisit ions of t he Pr ogr am and Document at ion by
or for t he feder al gover nment of t he Unit ed St at es. By accept ing deliver y of t he Pr ogr am, t he gover nment
her eby agr ees t hat t his soft war e qualifies as "commer cial" comput er soft war e wit hin t he meaning of FAR
Par t 12.212, DFARS Par t 227.7202-1, DFARS Par t 227.7202-3, DFARS Par t 252.227-7013, and DFARS Par t
252.227-7014. The t er ms and condit ions of Unit ed Signals & Syst ems, Inc. Soft war e License Agr eement shall
per t ain t o t he gover nment s use and disclosur e of t he Pr ogr am and Document at ion, and shall super sede any
conflict ing cont r act ual t er ms or condit ions. If t his license fails t o meet t he gover nment s minimum needs or
is inconsist ent in any r espect wit h feder al pr ocur ement law, t he gover nment agr ees t o r et ur n t he Pr ogr am
and Document at ion, unused, t o Unit ed Signals & Syst ems, Inc.
Higher-Order S pectral Analysis Toolbox Users Guide
COPYRIGHT 1993 - 2001 by Unit ed Signals & Syst ems, Inc.
MATLAB, Simulink, St at eflow, Handle Gr aphics, and Real-Time Wor kshop ar e r egist er ed t r ademar ks, and
Tar get Language Compiler is a t r ademar k of The Mat hWor ks, Inc.
Ot her pr oduct or br and names ar e t r ademar ks or r egist er ed t r ademar ks of t heir r espect ive holder s.
Pr int ing Hist or y: May 1993 Fir st pr int ing (The Mat hWor ks)
Sept ember 1995 Second pr int ing (The Mat hWor ks)
J anuar y 1998 Thir d pr int ing (The Mat hWor ks)
i
Contents
About the Authors
1
Tutori al
Introduct i on . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2
Polyspectra and Li near Processes . . . . . . . . . . . . . . . . . . . . . . 1-4
Int r oduct ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-4
Definit ions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-6
Why Do We Need Higher -Or der St at ist ics? . . . . . . . . . . . . . . . 1-10
Bias and Var iance of an Est imat or . . . . . . . . . . . . . . . . . . . . . 1-11
Est imat ing Cumulant s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-12
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-14
Est imat ing Polyspect r a and Cr oss-polyspect r a . . . . . . . . . . . . 1-15
Est imat ing t he Power Spect r um . . . . . . . . . . . . . . . . . . . . . 1-15
Est imat ing Bispect r a and Cr oss-Bispect r a . . . . . . . . . . . . . 1-16
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-18
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-19
Est imat ing Bicoher ence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-20
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-20
Test ing for Linear it y and Gaussianit y . . . . . . . . . . . . . . . . . . . 1-22
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-24
i i Contents
Par amet r ic Est imat or s, ARMA Models . . . . . . . . . . . . . . . . . . 1-26
MA Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-29
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-30
AR Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-31
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-32
ARMA Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-32
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-34
AR Or der Det er minat ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-34
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-35
MA Or der Det er minat ion . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-36
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-37
Linear Pr ocesses: Impulse Response Est imat ion . . . . . . . . . . . 1-37
The Polycepst r al Met hods . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-38
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-39
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-41
The Mat suoka-Ulr ych Algor it hm . . . . . . . . . . . . . . . . . . . . . 1-41
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-42
Linear Pr ocesses: Theor et ical Cumulant s and Polyspect r a . . 1-43
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-43
Summar y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-45
Li near Pre di cti on Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-47
Levinson Recur sion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-47
Tr ench Recur sion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-49
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-50
Det er minist ic For mulat ion of FBLS . . . . . . . . . . . . . . . . . . . . . 1-53
Adapt ive Linear Pr edict ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-54
RIV Algor it hm: Tr ansver sal For m . . . . . . . . . . . . . . . . . . . . 1-56
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-57
RIV Algor it hm: Double-Lat t ice For m . . . . . . . . . . . . . . . . . . 1-58
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-60
Summar y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-61
Harmoni c Processes and DOA . . . . . . . . . . . . . . . . . . . . . . . . . 1-62
Resolut ion and Var iance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-65
AR and ARMA Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-66
Pisar enkos Met hod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-67
Mult iple Signal Classificat ion (MUSIC) . . . . . . . . . . . . . . . . . . 1-68
Minimum-Nor m Met hod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-69
ESPRIT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-70
i i i
Cr it er ion-Based Est imat or s . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-72
Cumulant -Based Est imat or s . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-74
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-75
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-77
Summar y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-79
Nonli near Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-80
Solut ion Using Cr oss-Bispect r a . . . . . . . . . . . . . . . . . . . . . . . . 1-80
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-82
Solut ion Using FTs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-82
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-83
Quadr at ic Phase Coupling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-84
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-87
Summar y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-88
Ti me-Frequency Di st ri buti ons . . . . . . . . . . . . . . . . . . . . . . . . 1-89
Wigner Spect r um . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-90
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-93
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-94
Wigner Bispect r um . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-94
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-96
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-97
Wigner Tr ispect r um . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-98
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-99
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-100
Summar y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-100
Ti me-Delay Est i mati on . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-101
A Cr oss-Cor r elat ion Based Met hod . . . . . . . . . . . . . . . . . . . . . 1-101
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-103
A Cr oss-Cumulant Based Met hod . . . . . . . . . . . . . . . . . . . . . . 1-103
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-105
A Hologr am Based Met hod . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-105
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-107
Summar y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-107
i v Contents
Case St udi e s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-108
Sunspot Dat a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-108
Canadian Lynx Dat a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-114
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-114
A Classificat ion Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-120
Laught er Dat a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-122
Pit falls and Tr icks of t he Tr ade . . . . . . . . . . . . . . . . . . . . . . . . 1-131
Data Fi le s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-134
Re fere nce s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-139
2
Reference
Function Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2
Higher -Or der Spect r um Est imat ion: Convent ional Met hods . . 2-2
Higher -Or der Spect r um Est imat ion: Par amet r ic Met hods . . . . 2-3
Quadr at ic Phase Coupling (QPC) . . . . . . . . . . . . . . . . . . . . . . . . 2-3
Second-Or der Volt er r a Syst ems . . . . . . . . . . . . . . . . . . . . . . . . . 2-4
Har monic Ret r ieval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-4
Time-Delay Est imat ion (TDE) . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-4
Ar r ay Pr ocessing: Dir ect ion of Ar r ival (DOA) . . . . . . . . . . . . . . 2-4
Adapt ive Linear Pr edict ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-5
Impulse Response (IR), Magnit ude and
Phase Ret r ieval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-5
Time-Fr equency Est imat es . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-5
Ut ilit ies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-6
Demo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-6
Mi s ce llane ous . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-7
Pr ompt ing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-7
Guided t our . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-7
Addenda . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-7

About t he Aut hor s
A b o u t th e A u th o r s
vi
About the Authors
Ananthram Sw ami
Anant hr am Swami r eceived his B.Tech, M.S. and Ph.D. degr ees in elect r ical
engineer ing fr om t he Indian Inst it ut e of Technology at Bombay, Rice
Univer sit y, and t he Univer sit y of Sout her n Califor nia, r espect ively. He has
held posit ions wit h Unocal, USC, CS-3 and Malgudi Syst ems. He is cur r ent ly
a Resear ch Scient ist at t he Ar my Resear ch Lab, Adelphi, MD.
Dr . Swami has published over fift y jour nal and confer ence paper s in t he ar eas
of modeling and par amet er est imat ion of non-Gaussian pr ocesses. He is
co-or ganizer and co-chair of t he Eight h IEEE Signal Pr ocessing Wor kshop on
St at ist ical Signal and Ar r ay Pr ocessing, Cor fu, Gr eece (J une 1996).
Jerry M. Mendel
J er r y Mendel r eceived his B.S. degr ee in Mechanical Engineer ing in 1959, his
M.S. in 1960, and his Ph.D. in 1963 in Elect r ical Engineer ing fr om t he
Polyt echnic Inst it ut e of Br ooklyn, NY. Cur r ent ly he is Pr ofessor of Elect r ical
Engineer ing at USC in Los Angeles.
Dr . Mendel is a Fellow of t he IEEE, a Dist inguished Member of t he IEEE
Cont r ol Syst ems Societ y, a member of Tau Bet a Pi, Pi Tau Sigma, and Sigma
Xi, and a r egist er ed Pr ofessional Cont r ol Syst ems Engineer in Califor nia. He
has aut hor ed mor e t han 300 t echnical paper s, t hr ee t ext books and four ot her
books r elat ed t o his r esear ch in est imat ion t heor y, deconvolut ion, higher -or der
st at ist ics, neur al net wor ks and fuzzy logic.
Chrysostomos L. (Max ) Nikias
Chr ysost omos L. (Max) Nikias r eceived his B.S. degr ee in Elect r ical and
Mechanical Engineer ing fr om t he Nat ional Technical Univer sit y of At hens,
Gr eece and his M.S. and Ph.D. degr ees in Elect r ical Engineer ing fr om t he St at e
Univer sit y of New Yor k at Buffalo in 1980 and 1982. Cur r ent ly he is a
Pr ofessor of Elect r ical Engineer ing at USC in Los Angeles, wher e he is also
Dir ect or of CRASP and Associat e Dean of Academic Resear ch.
Dr . Nikias is a Fellow of t he IEEE. He is t he aut hor of over 150 jour nal and
confer ence paper s, t wo t ext books, a monogr aph, and four pat ent s. He is
co-aut hor of t he t ext book Higher-Order S pectral Analysis: A Nonlinear S ignal
Processing Framework, Pr ent ice-Hall, Inc. 1993.

1
Tut or ial
Introduct i on . . . . . . . . . . . . . . . . . . . . . 2
Polyspectra and Li near Processes . . . . . . . . . . . 4
Li near Predi cti on Models . . . . . . . . . . . . . . . 47
Harmoni c Processe s and DOA . . . . . . . . . . . . . 62
Nonli near Processes . . . . . . . . . . . . . . . . . 80
Ti me-Frequency Di st ri butions . . . . . . . . . . . . . 89
Ti me-Delay Esti mati on . . . . . . . . . . . . . . . 101
Case St udi e s . . . . . . . . . . . . . . . . . . . . 108
Dat a Fi les . . . . . . . . . . . . . . . . . . . . . 134
Refe re nce s . . . . . . . . . . . . . . . . . . . . . 139
1 Tu to r i a l
1-2
Introduction
This sect ion of t he Users Guide descr ibes how t o begin using t he Higher -Or der
Spect r al Analysis Toolbox for your signal pr ocessing applicat ions. It assumes
familiar it y wit h basic MATLAB

,
as well as a basic under st anding of signals
and syst ems.
Ther e is much mor e infor mat ion in a st ochast ic non-Gaussian or det er minist ic
signal t han is conveyed by it s aut ocor r elat ion or power spect r um. Higher -or der
spect r a, which ar e defined in t er ms of t he higher -or der moment s or cumulant s
of a signal, cont ain t his addit ional infor mat ion. The Higher -Or der Spect r al
Analysis (HOSA) Toolbox pr ovides compr ehensive higher -or der spect r al
analysis capabilit ies for signal pr ocessing applicat ions. The t oolbox is an
excellent r esour ce for t he advanced r esear cher and t he pr act icing engineer , as
well as t he novice st udent who want s t o lear n about concept s and algor it hms
in st at ist ical signal pr ocessing.
The Higher -Or der Spect r al Analysis Toolbox is a collect ion of M-files t hat
implement a var iet y of advanced signal pr ocessing algor it hms for spect r al
est imat ion, polyspect r al est imat ion, and comput at ion of t ime-fr equency
dist r ibut ions, wit h applicat ions such as par amet r ic and nonpar amet r ic blind
syst em ident ificat ion, t ime delay est imat ion, har monic r et r ieval, dir ect ion of
ar r ival est imat ion, par amet er est imat ion of Volt er r a (nonlinear ) models, and
adapt ive linear pr edict ion. Ot her pot ent ial applicat ions include acoust ics,
biomedicine, economet r ics, explor at ion seismology, nondest r uct ive t est ing,
oceanogr aphy, plasma physics, r adar , sonar , speech et c.
For t he newcomer t o t he field of higher -or der st at ist ics (spect r a), some
excellent st ar t ing places ar e:
[T2] Mendel, J .M., Tut or ial on higher -or der st at ist ics (spect r a) in signal
pr ocessing and syst em t heor y: Theor et ical r esult s and some applicat ions,
Proc. IEEE, Vol. 79, pp. 278-305, 1991.
[T3] Nikias, C.L. and J .M. Mendel, Signal pr ocessing wit h higher -or der
spect r a, IEEE S ignal Processing Magazine, Vol. 10, No 3, pp. 10-37, J uly 1993
[T4] Nikias, C.L. and A.P. Pet r opulu, Higher-Order S pectra Analysis: A
Nonlinear S ignal Processing Framework, New J er sey: Pr ent ice-Hall, 1993.
[T1] Nikias, C.L. and M.R. Raghuveer , Bispect r um est imat ion: A digit al signal
pr ocessing fr amewor k, Proc. IEEE, Vol. 75, pp. 869-91, J uly 1987.
In tr o d u c ti o n
1-3
The field of higher -or der st at ist ics (spect r a) and it s applicat ions t o var ious
signal pr ocessing pr oblems ar e r elat ively new. As such, t her e is no guar ant ee
t hat a par t icular Higher -Or der Spect r al Analysis Toolbox r out ine will wor k
well on your dat a. Unit ed Signals Syst ems, Inc., will be updat ing and
upgr ading t he Higher -Or der Spect r al Analysis Toolbox fr om t ime t o t ime t o
incor por at e new r out ines and pr ovide user s wit h guidance on t he applicabilit y
of exist ing r out ines as mor e exper ience is obt ained t hr ough t heir use.
The Tut or ial has numer ous examples t hat r einfor ce t he t heor y and
demonst r at e how t o use t he t oolbox funct ions. All of t he dat a files used by t hese
examples ar e included in your Higher -Or der Spect r al Analysis Toolbox
dist r ibut ion disket t e, and ar e descr ibed in t he sect ion on Dat a Files. We
encour age you t o t r y out t he examples your self. Addit ional examples may be
found in t he demo, which can be invoked via hosademo. A lat er sect ion in t he
Tut or ial demonst r at es how t o deal wit h real data.
1 Tu to r i a l
1-4
Polyspectra and Linear Processes
In t his sect ion, we will define cumulant s, polyspect r a, and var ious ot her
r elat ed st at ist ics, such as bicepst r a and bicoher ence. We will discuss t est s for
linearity and Gaussianity, and we will develop cumulant -based algor it hms for
est imat ing t he par amet er s of linear (e.g., ARMA) pr ocesses.
Introduction
The not ion of decomposing a signal int o it s har monic component s dat es back t o
t he analysis of t he mot ion of planet s (t he music of t he spher es, as t he
Pyt hagor ians called it ), phases of t he moon, laws of musical har mony, Newt ons
spect r al decomposit ion of light (1677), Ber nouilli (1738) and Euler s (1755)
analysis of vibr at ing membr anes, and Pr onys appr oximat ion for vibr at ing
mechanisms (1793). Moder n Four ier Analysis, as we know it t oday, r eceived it s
foundat ions in t he wor k of Four ier (1807), alt hough t he r oot s of t he Fast
Four ier Tr ansfor m (FFT) can be t r aced back t o Gausss wor k on or bit al
mechanics (1805).
We will assume, wit hout loss of gener alit y, t hat t he pr ocesses or signals of
int er est t o us ar e zer o mean. We will also assume t hat t he pr ocesses ar e
discr et e-t ime, wit h a sampling int er val of T = 1, cor r esponding t o a nor malized
sampling fr equency of 1 Hz, so t hat t he Nyquist fr equency is 0.5 Hz.
The power spectrum is t he pr imar y t ool of signal pr ocessing, and algor it hms for
est imat ing t he power spect r um have found applicat ions in ar eas such as r adar ,
sonar , seismic, biomedical, communicat ions, and speech signal pr ocessing.
Analog equipment t o est imat e t he spect r um, namely t he spectrum analyzer,
has been ar ound for mor e t han five decades, and may be found in almost any
lab. Our t oolbox not only offer s a subst it ut e for t hat equipment ; it expands t he
analyst s t oolkit t o include algor it hms mor e sophist icat ed t han t he simple
convent ional spect r al analysis t echniques.
Po l y sp e c tr a a n d Li n e a r Pr o c e sse s
1-5
The usefulness of t he power spectrum ar ises fr om an impor t ant t heor em,
known as Wolds decomposit ion, which st at es t hat any discr et e-t ime st at ionar y
r andom pr ocess can be expr essed in t he for m,
x(n) = y(n)+ z(n)
such t hat :
1 Pr ocesses y(n) and z(n) ar e uncor r elat ed wit h one anot her ;
2 Pr ocess y(n) has a causal linear pr ocess r epr esent at ion,
wher e and u(n) is a whit e-noise pr ocess; and,
3 z(n) is singular, t hat is, it can be pr edict ed per fect ly (wit h zer o var iance)
fr om it s past .
An example of a singular pr ocess is t he har monic pr ocess,
. A pr ocess wit h z(n) 0 has a pur ely cont inuous
spect r um; addit ionally, a st r ict ly band-limit ed pr ocess is also singular .
Since r eal wor ld signals cannot be st r ict ly band limit ed, we may t hink of t he
Wold decomposit ion as decomposing a pr ocess int o a linear pr ocess (which has
a cont inuous spect r um) and a har monic pr ocess (which has a line spect r um).
It is also impor t ant t o not e t hat t he t heor em only st at es t hat u(t) is
uncor r elat ed; it does not st at e t hat u(t) is i.i.d., (higher -or der whit e). For
example, u(n) might be t he out put of an all-pass syst em whose input is an i.i.d.
pr ocess. We need higher -or der st at ist ics t o det er mine whet her or not u(t) is
i.i.d., or mer ely uncor r elat ed. Ot her mot ivat ions for using higher -or der
st at ist ics (HOS) ar e discussed t hr oughout t his Tut or ial.
y n ( ) h k ( )u n k ( )
k 0 =

=
h 0 ( ) 1 = h
2
k ( )
k 0 =

< , ,
s n ( ) j 2f n ( ) exp =
1 Tu to r i a l
1-6
Definitions
The autocorrelation funct ion or sequence of a st at ionar y pr ocess, x(n), is
defined by,
(1-1)
wher e E {} denot es t he ensemble expect at ion oper at or . The power spect r um is
for mally defined as t he Four ier Tr ansfor m (FT) of t he aut ocor r elat ion sequence
(t he Wiener -Khint chine t heor em)
(1-2)
wher e denot es t he fr equency. An equivalent definit ion is given by
(1-3)
wher e X() is t he Four ier Tr ansfor m of x(n)
(1-4)
A sufficient , but not necessar y, condit ion for t he exist ence of t he power
spect r um is t hat t he aut ocor r elat ion be absolut ely summable. The power
spect r um is r eal valued and nonnegat ive, t hat is, P
xx
() 0; if x(n) is r eal
valued, t hen t he power spect r um is also symmet r ic, t hat is, P
xx
() = P
xx
().
As we shall see next , higher -or der moment s ar e nat ur al gener alizat ions of t he
aut ocor r elat ion, and cumulant s ar e specific nonlinear combinat ions of t hese
moment s.
The fir st -or der cumulant of a st at ionar y pr ocess is t he mean, C
1x
:= E{x(t )} . The
higher -or der cumulant s ar e invar iant t o a shift of mean; hence, it is convenient
t o define t hem under t he assumpt ion of zer o mean; if t he pr ocess has nonzer o
mean, we subt r act t he mean, and t hen apply t he following definit ions t o t he
r esult ing pr ocess. The second-, t hir d- and four t h-or der cumulant s of a
zer o-mean st at ionar y pr ocess ar e defined by [4],
R
x x
m ( ) := E x

n ( )x n m + ( ) { }
P
x x
( ) R
xx
m ( ) j 2m ( ) exp
m =

=
P
xx
( ) := E X ( )X

( ) { }
X ( ) x n ( ) j 2n ( ). exp
n =

=
Po l y sp e c tr a a n d Li n e a r Pr o c e sse s
1-7
(1-5)
(1-6)
(1-7)
wher e M
2x
(m) = E{x(n) x(n + m)}, and equals C
2x
(m), for a r eal-valued pr ocess.
The fir st -or der cumulant is t he mean of t he pr ocess; and t he second-or der
cumulant is t he aut ocovar iance sequence. Not e t hat for complex pr ocesses,
t her e ar e sever al ways of defining cumulant s depending upon which t er ms ar e
conjugat ed.
The zer o-lag cumulant s have special names: C
2x
(0) is t he var iance and is
usually denot ed by ; C
3x
(0,0) and C
4x
(0,0,0) ar e usually denot ed by
3x
and

4x
. We will r efer t o t he nor malized quant it ies, as t he skewness and
as t he kurtosis. These nor malized quant it ies ar e bot h shift and scale
invar iant . If x(n) is symmet r ic dist r ibut ed, it s skewness is necessar ily zer o (but
not vice ver sa); if x(n) is Gaussian dist r ibut ed, it s kurtosis is necessar ily zer o
(but not vice ver sa). Oft en t he t er ms skewness and kur t osis ar e used t o r efer t o
t he unnor malized quant it ies,
3x
and
4x
.
If x(n) is an i.i.d. pr ocess, it s cumulant s ar e nonzer o only at t he or igin. If x(n) is
st at ist ically independent of y(n), and z(n) = x(n) + y(n), t hen
wit h similar r elat ionships holding for cumulant s of all or der s. This addit ivit y
pr oper t y simplifies cumulant -based analysis.
C
2x
k ( ) E x* n ( )x n k + ( ) { } =
C
3x
k l , ( ) E x* n ( )x n k + ( )x n l + ( ) { } =
C
2x
k ( )C
2x
l m ( ) C
2x
l ( )C
2x
k m ( )
C
4 x
k l m , , ( ) E x* n ( )x n k + ( )x n l + ( )x* n m + ( ) { } =
M
2x
*
m ( )M
2 x
k l ( )

x
2

3x

2x
3

4x

2 x
4

C
4z
k l m , , ( ) C
4x
k l m , , ( ) C
4y
k l m , , ( ), + =
1 Tu to r i a l
1-8
The cumulant s of a st at ionar y r eal-valued pr ocess ar e symmet r ic in t heir
ar gument s, t hat is,
Hence, t he fundament al r egion of suppor t is not t he ent ir e k-D plane. For
example, for k = 2, C
2x
(k), k 0, specifies C
2x
(k) ever ywher e. It is easily shown
t hat t he nonr edundant r egion for C
3x
(k,l) is t he wedge
{(k,l) : 0 l k },
and for C
4x
(k,l,m), it is t he cone,
{(k,l,m) : 0 m l k }.
The kt h-or der polyspect r um is defined as t he FTs of t he cor r esponding
cumulant sequence:
(1-8)
(1-9)
(1-10)
which ar e r espect ively t he power spect r um, t he bispect r um, and t he
t r ispect r um. Not e t hat t he bispect r um is a funct ion of t wo fr equencies, wher eas
t he t r ispect r um is a funct ion of t hr ee fr equencies. In cont r ast wit h t he power
spect r um which is r eal valued and nonnegat ive, bispect r a and t r ispect r a ar e
complex valued.
C
3x
k l , ( ) C
3x
l k , ( ) C
3x
k l k , ( ) = =
C
2x
k ( ) C
2x
k ( ) =
C
4x
k l , m , ( ) C
4x
l k m , , ( ) C
4x
k m l , , ( ) C
4x
k l k m k , , ( ) = = =
S
2x
( ) C
2x
k ( )e
j 2
k =

=
S
3x

1

2
, ( ) C
3x
k l , ( )e
j 2
1
k
e
j 2
2
l
l =

k =

=
S
4x

1

2

3
, , ( ) C
4x
k l m , , ( )e
j 2
1
( k
2
l
3
m ) + +
k l m , , =

=
Po l y sp e c tr a a n d Li n e a r Pr o c e sse s
1-9
For a r eal-valued pr ocess, symmet r y pr oper t ies of cumulant s car r y over t o
symmet r y pr oper t ies of polyspect r a. The power spect r um is symmet r ic:
S
2x
() = S
2x
(). The symmet r y pr oper t ies of t he bispect r um ar e given by [56]:
(1-11)
Hence, a nonr edundant r egion of suppor t for t he bispect r um is t he t r iangle
wit h ver t ices (0,0), (1/3,1/3) and (1/2,0); r ecall t hat we have assumed a
nor malized sampling fr equency of 1 Hz.
Symmet r y pr oper t ies of t he t r ispect r um include:
The lit er at ur e is somewhat confusing bot h in t he der ivat ion, as well as in t he
descr ipt ion of t he nonr edundant r egions. The nonr edundant r egion for a
cont inuous-t ime band-limit ed pr ocess, wit h a Nyquist fr equency of 0.5 Hz, is
t he t r iangle wit h ver t ices (0,0), (1/4,1/4), (1/2,0). A t ut or ial t r eat ment of t he
differ ences bet ween t he cont inuous-t ime and t he discr et e-t ime cases is given in
[47]; r elat ed discussions may be found in [63]. The nonr edundant r egion of t he
t r ispect r um is discussed in [6, 9, 47].
Similar t o t he cr oss-cor r elat ion, we can also define cr oss-cumulant s; for
example,
(1-12)
The cr oss-bispect r um is defined by,
(1-13)
Not e t hat t he bispect r um S
3x
(
1
,
2
) is a special case of t he cr oss-bispect r um
obt ained when x = y = z.
The cross-bicoherence is anot her useful st at ist ic which is defined as,
S
3x

1

2
, ( ) S
3x

2

1
, ( ) S
3x

1

1

2
, ( ) = =
S
3x

1

2

2
, , ( ) S
3x
*

1

2
, ( ). = =
S
4x

1

2

3
, , ( ) S
4x

1

3

2
, , ( ) S
4 x

2

1

3
, , ( ) = =
S
4 x

1

2

1

3

1
, , ( ) S
4x
*

1

2

3
, , ( ). = =
C
xy z
k l , ( ) E x

n ( )y n ( k )z n ( l ) + + { } =
S
xyz

1

2
, ( ) C
x yz
k l , ( )e
j 2
1
k
e
j 2
2
l
l =

k =

=
1 Tu to r i a l
1-10
(1-14)
The aut obicoher ence is obt ained when x = y = z. M-file bicoherx can be used t o
est imat e t he cr oss-bicoher ence, and bicoher can be used t o est imat e t he
bicoher ence.
The cross-bicepstrum of t hr ee pr ocesses is defined by
(1-15)
and is well-defined only if S
xyz
(
1
,
2
) is nonzer o ever ywher e.
Why Do We Need Higher-Order Statistics?
Mot ivat ion t o use cumulant s and polyspect r a of or der k > 2 is given by t he
following (m
k
= (m
1
. . . , m
k-1
)):
If z(n) = x(n) + y(n), and x(n) and y(n) ar e mut ually independent pr ocesses,
t hen C
kz
(m
k
) = C
kx
(m
k
) + C
ky
(m
k
).
If x(n) is Gaussian, t hen C
kz
(m
k
) = 0, k > 2.
Hence, if z(n) = x(n) + w(n), wher e w(n) is Gaussian and independent of x(n),
t hen, for k > 2, C
kz
(m
k
) = C
kx
(m
k
). Thus, we can r ecover t he higher -or der
cumulant s of a non-Gaussian signal even in t he pr esence of color ed Gaussian
noise.
Let x(n) be a linear pr ocess, t hat is, , wher e u(n) is
i.i.d. Then, it follows t hat :
(1-16)
(1-17)
(1-18)
bic
xyz

1

2
, ( )
S
xy z

1

2
, ( )
S
2x

1

2
+ ( )S
2y

1
( )S
2 z

2
( )
------------------------------------------------------------------------------- =
b
x yz
m n , ( ) S
xy z

1

2
, ( ) ( )e
j 2
1
m
e
j 2
2
n
ln
1

2
d d

=
x n ( ) h
k

k ( )u n ( k ) =
C
2x
k ( )
2u
h

n ( )h n ( k ) + =
C
3x
k l , ( )
3u
h

n ( )h n ( k )h n l + ( ) + =
C
4x
k l , m , ( )
4u
h

n ( )h n ( k )h n l + ( )h

n m + ( ) + =
Po l y sp e c tr a a n d Li n e a r Pr o c e sse s
1-11
(1-19)
(1-20)
(1-21)
wher e
ku
= C
ku
(0). Not e t hat t he power spect r um does not car r y any
infor mat ion about t he phase of H(). In cont r ast , if u(n) is non-Gaussian, t his
phase infor mat ion can be r ecover ed fr om t he higher -or der polyspect r a. Thus,
t he st andar d minimum-phase assumpt ion, which is necessar y when t he
pr ocess is Gaussian or only second-or der st at ist ics ar e used, may be dr opped.
Any pr ocess can always be consider ed t o be a linear pr ocess wit h r espect t o
it s second-or der st at ist ics; t hat is, given R
yy
, we can always find {h(k)} and
an uncor r elat ed pr ocess u(n), such t hat , R
yy
(m) = R
xx
(m), wher e
. In ot her wor ds, t he aut ocor r elat ion sequence
cannot give any evidence of nonlinear it y. In cont r ast , higher -or der
cumulant s can give evidence of nonlinear it y.
Pr ocesses of t he for m whose phase is a
polynomial in t ime t, ar e called polynomial phase pr ocesses; t he FTs of such
pr ocesses t end t o be flat , wher eas suit ably defined slices of higher -or der
spect r a r eveal st r uct ur e t hat per mit s est imat ion of p and t he a
k
s.
To summar ize, cumulant s ar e useful: (1) if t he addit ive noise is Gaussian and
t he signal is non-Gaussian, (2) t he linear syst em is non-minimum phase (t hat
is, mixed-phase), or (3) t he pr ocess is nonlinear .
Bias and Variance of an Estimator
In pr act ice, we est imat e cumulant s and polyspect r a fr om dat a. These est imat es
ar e, t hemselves, r andom, and ar e char act er ized by t heir bias and variance.
Let x(n) denot e a st at ionar y pr ocess; we assume t hat all r elevant st at ist ics
exist and have finit e values. Let s denot e some st at ist ic, defined on x(n). Let
denot e an est imat e of t he st at ist ic based on N obser vat ions, . Since
x(n) is a r andom pr ocess, t he est imat e is also r andom; clear ly, will not
S
2 x
( )
2 u
H ( )
2
=
S
3x

1

2
, ( )
3u
H
1
( )H
2
( )H

1

2
+ ( ) =
S
4x

1

2

3
, , ( )
4u
H
1
( )H
2
( )H
3
( )H

1

2

3
+ + ( ) =
x n ( ) h
k

k ( )u n ( k ) =
x t ( ) a t ( ) j ( exp a
k
t
k
)
k 0 =
p

=
s
N
x n ( ) { }
n 0 =
N 1
s
N
s
N
1 Tu to r i a l
1-12
equal s. The est imat e is a good est imat e if it is near s. This not ion is
clar ified by int r oducing t he ideas of bias and consistency.
The bias of an est imat or is defined as E{ } s; t he est imat e is said t o be
unbiased if t he bias is zer o, t hat is,
E{ } = s.
Oft en t his holds t r ue only as , in which case t he est imat e is said t o be
asymptotically unbiased.
The bias, by it self, does not complet ely char act er ize t he est imat e. If t he
est imat e is good, we expect t hat will t ake on values ar ound t he t r ue
quant it y s. The nat ur al measur e of t he spr ead is t he squar ed deviat ion ar ound
t he t r ue quant it y, s,
.
The est imat e is said t o be (asympt ot ically) consist ent if t he squar ed deviat ion
goes t o zer o, as . This condit ion is somet imes called mean-square
consistency. A consist ent est imat e is necessar ily (asympt ot ically) unbiased.
Estimating Cumulants
In pr act ice, we have a finit e amount of dat a, , and we must obt ain
consist ent est imat es of cumulant s. The sample est imat es ar e given by,
(1-22)
(1-23)
(1-24)
s
N
s
N
s
N
N
s
N
E s
N
s
2

' ;

N
x n ( ) { }
n 0 =
N 1
C

x y
k ( )
1
N
3
------- x

n N
1
=
N
2

n ( )y n k + ( ) =
M

x y
k ( )
1
N
3
------- x
n N
1
=
N
2

n ( )y n k + ( ) =
C

xy z
k l , ( )
1
N
3
------- x

n N
1
=
N
2

n ( )y n k + ( )z n l + ( ) =
Po l y sp e c tr a a n d Li n e a r Pr o c e sse s
1-13
(1-25)
wher e N
1
and N
2
ar e chosen such t hat t he summat ions involve only x(n)s wit h
; unbiased est imat es ar e obt ained if N
3
is set equal t o t he act ual
number of t er ms which ar e aver aged; for example,
Usually we set N
3
t o N and obt ain est imat es t hat ar e asympt ot ically unbiased.
Aut ocumulant s ar e obt ained when w = x = y = z. These est imat es ar e known t o
be consist ent pr ovided t he pr ocess x(n) sat isfies some weak mixing condit ions
[5]. For example, for lar ge N, t he var iance of t he sample est imat e of t he
t hir d-or der cr oss-cumulant is given by
wher e c is a finit e const ant t hat depends upon t he aut o- and cr oss-moment s
(cumulant s) of or der s 1 t hr ough 6 of t he pr ocesses x(n), y(n), and z(n).
These definit ions assume t hat t he pr ocesses ar e zer o mean; in pr act ice, t he
sample mean is r emoved fir st . Rout ines cum2x, cum3x, and cum4x may be
used t o est imat e cr oss-cumulant s of or der s 2, 3, and 4; cumest may be used t o
est imat e t he aut ocumulant s.
C

wx yz
k l m , , ( )
1
N
3
------- w

n N
1
=
N
2

n ( )x n k + ( )y n l + ( )z

n m + ( ) =
C

wx
k ( )C
yz
l m ( ) C

wy
l ( )C
x z
k m ( )
M

wz
*
m ( )M

xy
l k ( )
n 0 N 1 , [ ]
E C

x yz
N
k l , ( ) { } C
x yz
k l , ( ) =
var C

xy z
N
k l , ( ) { } c N =
1 Tu to r i a l
1-14
Ex a mples
We will simulat e a non-Gaussian ARMA pr ocess, and t hen est imat e it s
cumulant s:
rand('seed',0); randn('seed',0);
u=rpiid(1024,'exp'); n=25;
y=filter([1,2], [1,1.5,0.8], u);
for k=n:n,
cmat(:,k+n+1)=cumest(y,3,n,128,0,'biased',k);
end
subplot(121), mesh(n:n,n:n, cmat)
subplot(122), contour(n:n,n:n,cmat,8)
Time-ser ies y is segment ed int o r ecor ds of 128 samples each, wit h no over lap;
biased est imat es of t he t hir d-or der cumulant s ar e obt ained fr om each segment
and t hen aver aged; t he (i,j) element of cmat will cont ain t he est imat e of
C
3y
(i n 1,j n 1), for i,j = 1, . . ., 2

*
n + 1. You can use t he funct ion cumtrue
t o comput e and display t he t r ue cumulant s.
The cont our plot in Figur e 1-1 r eveals t he basic symmet r y of t hir d-or der
cumulant s, namely C
3y
(
1
,
2
) = C
3y
(
2
,
1
). Ot her symmet r y pr oper t ies may be
ver ified by using cumtrue t o est imat e t he t r ue cumulant s of a linear pr ocess.
Figure 1-1: Estimated Third-Order Cumulants of an ARMA(2,1) Process
(cumest)
40
20
0
20
40
50
0
50
20
15
10
5
0
5
10
15
20 10 0 10 20
25
20
15
10
5
0
5
10
15
20
25
Po l y sp e c tr a a n d Li n e a r Pr o c e sse s
1-15
Estimating Polyspectra and Cross-polyspectra
Est imat or s of polyspect r a ar e nat ur al ext ensions of est imat or s of t he power
spect r um, wit h some impor t ant differ ences in t he smoot hing r equir ement s.
Hence, it will be useful t o r eview power spect r um est imat ion t echniques fir st .
Estima ting the Pow er Spectr um
Techniques for est imat ing t he convent ional power spect r um fall int o t hr ee
br oad cat egor ies: t he nonpar amet r ic or convent ional met hods, t he par amet r ic
or model-based met hods, and t he cr it er ion-based met hods.
The fir st cat egor y includes t wo classes: t he dir ect met hods, which ar e based on
t he FT of t he obser ved dat a; and indir ect met hods, which ar e based on
comput ing t he FT of t he est imat ed aut ocor r elat ion sequence of t he dat a. The
class of par amet r ic met hods includes algor it hms such as MA, AR, and ARMA
modeling, and eigen-space based met hods such as MUSIC, Min-Nor m, et c.,
which ar e appr opr iat e for har monic models. Cr it er ion-based met hods include
Bur gs Maximum Ent r opy algor it hm and Capons Maximum-Likelihood
algor it hm.
The convent ional est imat or s ar e easy t o under st and and easy t o implement ,
but ar e limit ed by t heir r esolving power (t he abilit y t o separ at e t wo closely
spaced har monics), par t icular ly when t he number of samples is small. For
r andom signals, t hese est imat or s t ypically r equir e long obser vat ion int er vals
in or der t o achieve accept ably low values for t he var iances of t he est imat e.
The nat ur al est imat or of t he power spect r um is t he FT of ,
This est imat or , also known as t he per iodogr am, can be comput ed as t he
squar ed magnit ude of an N-point FFT of t he obser ved t ime ser ies. Since
, t he per iodogr am is an unbiased est imat or of P
xx
().
However , t he per iodogr am is not a consist ent est imat or , because
; t hat is, it s var iance does not go t o zer o as . [56]
R

xx
N
m ( )
I
x x
N
( ) R

x x
N
m ( )e
j 2m
m N 1 =
N 1

1
N
---- x k ( )e
j 2k
k 0 =
N 1

2
. = =
E R

x x
N
m ( ) { } R
x x m ( )
=
var P
xx
N
( ) ( ) P
xx
2
( ) = N
1 Tu to r i a l
1-16
If x(n) is Gaussian whit e noise wit h var iance
2
, it s power spect r um should be
flat , P
xx
() =
2
; t he var iance of t he per iodogr am est imat e, , is given by,
Not e t hat t he var iance does not go t o zer o as , t hat is, t he est imat e is not
consist ent . The covar iance bet ween est imat es at fr equencies
1
= k/ N and

2
= m/ N is zer o; t hus, on t he one hand as N incr eases, t he var iance at any
does not go t o zer o; however , t he spacing bet ween est imat es t hat ar e
uncor r elat ed decr eases as 1/N; as a consequence, t he fluct uat ions in t he
per iodogr am become mor e r apid as N incr eases. As an example, t r y
semilogy(abs(fft(rpiid(n,nor')))) for incr easing values of n. Pr oper
smoot hing smoot hs out t he fluct uat ions and yields consist ent est imat es.
It should be emphasized t hat t he var iance expr essions ar e meaningful only in
t he cont ext of r andom pr ocesses; t he FT, it self, is a ver y useful t ool for
analyzing det er minist ic signals.
The per iodogr am est imat e can be made consist ent in sever al ways, by:
Smoot hing (filt er ing) in t he fr equency domain;
Mult iplying t he aut ocor r elat ion sequence by a lag window funct ion;
Mult iplying t he t ime-domain dat a by a window funct ion; or ,
Aver aging sever al per iodogr am est imat es.
These obser vat ions car r y over t o bispect r al est imat es as we see next .
Estima ting Bispectr a a nd Cr oss- Bispectra
The nat ur al est imat e of t he cr oss-bispect r um is t he FT of t he t hir d-or der
cumulant sequence, t hat is,
(1-26)
P
xx
N
( )
var P
x x
N
( ) ( )
4
1
2 ( sin N )
N 2 ( sin )
------------------------------
,
_
2
+ . =
N
I
xy z
N
( ) C

xy z
k l , ( )e
j 2
1
k
e
j 2
2
l
l N 1 =
N 1

k N 1 =
N 1

=
1
N
2
-------X
N
*

1

2
+ ( )Y
N

1
( )Z
N

2
( ) =
Po l y sp e c tr a a n d Li n e a r Pr o c e sse s
1-17
wher e X
N
() is t he FT of . This est imat e, known as t he
cross-biperiodogram, is not a consist ent est imat e. As in t he case of t he power
spect r um, t he est imat e can be made consist ent by suit able smoot hing. The
bispect r um and t he biper iodogr ams ar e special cases obt ained when x = y = z.
Smoot hing can be accomplished by mult iplying t he t hir d-or der cumulant
est imat es by a lag window funct ion. Let w(t,s) be a 2-D window funct ion, whose
2-D FT is bounded and nonnegat ive; fur t her , assume
The window funct ion, w(t ,s), must also sat isfy t he symmet r y pr oper t ies of
t hir d-or der cumulant s. For example, 2-D lag windows may be der ived fr om 1-D
lag windows as follows,
w(t,s) = w(t)w(s)w(t s)
which sat isfies t he symmet r y condit ions of C
3x
(m,n).
Consider t he scaled-par amet er window, w
M
(t,s) = w(t/ M,s/ M), and t he
smoot hed est imat e,
(1-27)
Under t he assumpt ion t hat t he cr oss-bispect r um S
xyz
(
1
,
2
) is sufficient ly
smoot h, t he smoot hed est imat e is known t o be consist ent , wit h var iance given
by,
(1-28)
for 0 <
1
<
2
< . Not e t he implied consist ency condit ion is and
M
2
/ , as , and w
2
(t,s)dt ds < . The est imat or in (1-27), for x = y
= z, is implement ed in r out ine bispeci.
An alt er nat ive appr oach is t o per for m t he smoot hing in t he fr equency domain.
As in t he case of power spect r a, we may segment t he dat a int o K r ecor ds of
x n ( ) { }
n 0 =
N 1
w 0 0 , ( ) 1; =
w
2
t s , ( ) t d s d < ;


i
2
W
1

2
, ( )
1
d
2
d < ;


i
W
1

2
, ( )
1
d
2
d 0 = ;

S

x yz

1

2
, ( ) C

xy z
k l , ( )w
M
k l , ( )e
j 2
1
k
e
j 2
2
l
l N 1 =
N 1

k N 1 =
N 1

=
var S

xy z

1

2
, ( ) ( )
M
2
N
--------S
2 x

1

2
+ ( )S
2y

1
( )S
2z

2
( ) w
2
t s , ( ) t d s d

=
M
N N
1 Tu to r i a l
1-18
lengt h L = N/ K, comput e and aver age t he biper iodogr ams, and t hen per for m
t he fr equency smoot hing, using t he fr equency-domain filt er , W
M
(
1
,
2
), t he FT
of w
M
(t,s). In t his case,
(1-29)
for 0 <
1
<
2
< . Windowing is not r equir ed in t his case pr ovided K is lar ge;
however , t his does not ensur e t hat t he bias will go t o zer o. Rao and Gabr [56,
Sec. 2.4] have der ived a bispect r al window which is opt imum in t er ms of bias
and var iance; windows sat isfying ot her opt imalit y cr it er ia ar e discussed in [56,
40]. The Rao-Gabr window is available as an opt ion in r out ine bispeci.
Rout ine bispecdx comput es est imat es of t he cr oss-bispect r um.
Ex a mples
load qpc
bspec=bispeci(zmat,21,64,0,'unbiased',128,1);
dbspec=bispecdx(zmat,zmat,zmat,128,3,64,0);
The cont our plot s of t he t wo est imat es of t he bispect r um ar e shown in Figur e
1-2 and Figur e 1-3.
Figure 1-2 Indirect Estimate of the Bispectrum (bispeci)
var S

xyz

1

2
, ( ) ( )
M
2
L K
--------S
2x

1

2
+ ( )S
2 y

1
( )S
2z

2
( ) w
2
t s , ( ) t d s d

=
0.5 0.4 0.3 0.2 0.1 0 0.1 0.2 0.3 0.4
0.5
0.4
0.3
0.2
0.1
0
0.1
0.2
0.3
0.4
Bispectrum estimated via the indirect method
f1
f
2
Po l y sp e c tr a a n d Li n e a r Pr o c e sse s
1-19
Figure 1-3 Direct Estimate of the Bispectrum (bispecdx)
Bot h t he dir ect and t he indir ect est imat es r eveal peaks at (0.10, 0.15) and t he
11 ot her symmet r ic locat ions as indicat ed by (1.11). The dat a used in t his
example consist of quadr at ically phase-coupled har monics wit h fr equencies at
0.10, 0.15, and 0.25 Hz, and an uncoupled har monic at 0.40 Hz; quadr at ic
phase-coupling is discussed fur t her in t he sect ion on Nonlinear Pr ocesses,
wher e we will see t hat t he pr esence of pr onounced peaks in t he bispect r um is
indicat ive of nonlinear phenomena.
Let x(n) be a (zer o mean) whit e Gaussian pr ocess, wit h var iance ; let
y(n) = x
2
(n). It is easy t o show t hat t he cr oss-bispect r um of (x,x,y) should be a
const ant , equal t o . This follows by not ing t hat if a,b,c,d ar e zer o mean and
joint ly Gaussian, t hen,
E{abcd} = cum(a,b)cum(c,d) + cum(a,d)cum(b,c) + cum(a,c)cum(b,d).
Ex a mples
randn('seed',0);
x=randn(64,64); y=x.*x;
dbic=bispecdx(x,x,y,128,5);
Not ice t hat an appar ent st r uct ur e along t he axes (which is an ar t ifact due t o
t he r emoval of t he mean), consist ent est imat es along t he axes, and t he
0.5 0.4 0.3 0.2 0.1 0 0.1 0.2 0.3 0.4
0.5
0.4
0.3
0.2
0.1
0
0.1
0.2
0.3
0.4
CrossBispectrum
f1
f
2

x
2
2
x
4
1 Tu to r i a l
1-20
ant i-diagonal can be obt ained only by sufficient smoot hing; a smoot hing
window of size 5 is inadequat e. The lines
1
= 0,
2
= 0, and
1
+
2
= 0, ar e
called t he pr incipal submanifolds, [4, 6]. This example also illust r at es t hat
passing a Gaussian pr ocess t hr ough a nonlinear it y makes it non-Gaussian.
Estimating Bicoherence
Given est imat es of t he power spect r a and t he cr oss-bispect r um, we can
est imat e t he cr oss-bicoher ence as indicat ed in (1-14). It has been shown t hat
consist ent est imat es of t he power spect r um and t he bispect r um lead t o
consist ent est imat es of t he bicoher ence.
Rout ines bicoher and bicoherx may be used t o est imat e t he aut obicoher ence
and t he cr oss-bicoher ence.
Ex a mples
load qpc
dbspec=bicoher(zmat,128);
Figure 1-4 Bicoherence Estimate (bicoher)
A cont our plot of t he est imat ed bicoher ence is shown in Figur e 1-4. The dat a
consist s of quadr at ically phase-coupled har monics wit h fr equencies at 0.10,
0.5 0.4 0.3 0.2 0.1 0 0.1 0.2 0.3 0.4
0.5
0.4
0.3
0.2
0.1
0
0.1
0.2
0.3
0.4
Bicoherence estimated via the direct (FFT) method
f1
f
2
Po l y sp e c tr a a n d Li n e a r Pr o c e sse s
1-21
0.15, and 0.25 Hz, and an uncoupled har monic at 0.40 Hz. The maximum value
of t he bicoher ence is 0.6442; t he value is less t han unit y because of t he addit ive
noise, which affect s t he power spect r um est imat e.
load nl1
bicx=bicoherx(x,x,y);
You should see t he display in Figur e 1-5. The cr oss-bicoher ence is significant ly
nonzer o, and nonconst ant , indicat ing a nonlinear r elat ionship bet ween x and
y. The nonshar pness of t he peaks, as well as t he pr esence of st r uct ur e ar ound
t he or igin, indicat es t hat t he nonlinear r elat ionship is not pur ely of t he for m
y(n) = x
2
(n), and t hat x,y ar e not nar r ow-band pr ocesses. Fr om t he descr ipt ion
of nl1.mat, we see t hat y is t he out put of a second-or der Volt er r a syst em whose
input , x, is Gaussian.
Figure 1-5 Cross-Bicoherence Estimate (bicoherx)
0.5 0.4 0.3 0.2 0.1 0 0.1 0.2 0.3 0.4
0.5
0.4
0.3
0.2
0.1
0
0.1
0.2
0.3
0.4
CrossBicoherence
f1
f
2
1 Tu to r i a l
1-22
Testing for Linearity and Gaussianity
In subsequent sect ions we discuss algor it hms for t he est imat ion of par amet er s
of non-Gaussian linear pr ocesses; of cour se, t hese r out ines assume t hat t he
dat a ar e linear and non-Gaussian. We call a pr ocess, y(n), linear , if it can be
r epr esent ed by
wher e u(n) is assumed t o be i.i.d. If u(n) is Gaussian (non-Gaussian), we say
t hat y(n) is linear Gaussian (non-Gaussian). How do we know t hat t he dat a ar e
non-Gaussian, and t hat t hey ar e addit ionally linear ?
Hinich [24] has developed algor it hms t o t est for non-skewness (loosely called
Gaussianit y) and linear it y. The basic idea is t hat if t he t hir d-or der cumulant s
of a pr ocess ar e zer o, t hen it s bispect r um is zer o, and hence it s bicoher ence is
also zer o. If t he bispect r um is not zer o, t hen t he pr ocess is non-Gaussian; if t he
pr ocess is linear and non-Gaussian, t hen t he bicoher ence is a nonzer o const ant :
see (1-19), (1-20), and (1-14) wit h x = y = z. Thus, we have a hypot hesis t est ing
pr oblem for non-Gaussianit y (non-zer o bispect r um):
H1: t he bispect r um of y(n) is nonzer o;
H0: t he bispect r um of y(n) is zer o.
If hypot hesis H1 holds, we can t est for linear it y, t hat is, we have a second
hypot hesis t est ing pr oblem,
H1: t he bicoher ence of y(n) is not const ant ;
H0: t he bicoher ence of y(n) is a const ant .
If hypot hesis H0 holds, t he pr ocess is linear .
Assume t hat we have good (per fect ) est imat es of t he power spect r um, and
consider t he sample est imat e of t he squar ed bicoher ence,
(1-30)
y n ( ) h k ( )u n k ( ) ,
k

=
bic

xx x

1

2
, ( )
2
S

xx x

1

2
, ( )
2
S
2 x

1

2
+ ( )S
2x

1
( )S
2 x

2
( )
--------------------------------------------------------------------------- =
Po l y sp e c tr a a n d Li n e a r Pr o c e sse s
1-23
It has been est ablished in [6] t hat sample est imat es of t he bispect r um using
convent ional met hods ar e asympt ot ically (complex) Gaussian; addit ionally, t he
est imat es at differ ent fr equencies ar e uncor r elat ed, pr ovided t he fr equency
separ at ion is gr eat er t han t he r ecipr ocal of t he effect ive window lengt h. If
is Gaussian dist r ibut ed, we know t hat is chi-squar ed dist r ibut ed wit h
t wo degr ees of fr eedom.
If 0, t hen t he st at ist ic in(1-30) is a cent r al chi-squar ed r .v. wit h
t wo degr ees of fr eedom. The squar ed bicoher ence is summed over t he P point s
in t he nonr edundant r egion; det ails ar e given in [24]. The r esult ing st at ist ic S
is
2
dist r ibut ed, wit h 2P degr ees of fr eedom. Hence it is easy t o devise a
st at ist ical t est t o det er mine whet her t he obser ved S is consist ent wit h a cent r al
chi-squar ed dist r ibut ion; t his consist ency is r epor t ed as pr obabilit y-of-false
alar m value, t hat is, t he pr obabilit y t hat we will be wr ong in assuming t hat t he
dat a have a nonzer o bispect r um. If t his pr obabilit y is small, say 0.95, we accept
t he assumpt ion of zer o bispect r um, t hat is, we cannot r eject t he Gaussianit y
assumpt ion. This t est is implement ed in r out ine glstat.
Let us assume t hat we have est imat ed S and ar e confident t hat t he dat a is
non-Gaussian. Now if t he dat a ar e also linear , we expect t he squar ed
bicoher ence t o be const ant for all
1
and
2
. In pr act ice, t he est imat ed
bicoher ence will not be flat : we can obt ain an est imat e of t he const ant value by
comput ing t he mean value of t he bicoher ence over t he point s in t he
nonr edundant r egion; let denot e t his mean value. The squar ed bicoher ence is
chi-squar ed dist r ibut ed wit h t wo degr ees of fr eedom and noncent r alit y
par amet er . The sample int er quar t ile r ange, R, of t he squar ed bicoher ence can
be est imat ed, and compar ed wit h t he t heor et ical int er quar t ile r ange of a
chi-squar ed dist r ibut ion wit h t wo degr ees of fr eedom and noncent r alit y
par amet er . If t he est imat ed int er quar t ile r ange is much lar ger or much
smaller t han t he t heor et ical value, t hen we should r eject t he linear it y
hypot hesis. This t est is implement ed in r out ine glstat.
Not e t hat a zer o bispect r um is not pr oof of Gaussianit y, since t he higher -or der
cumulant s and polyspect r a need not be ident ically zer o. The bispect r um of a
time-reversible pr ocess is ident ically equal t o zer o. For example, consider t he
linear pr ocess , wher e u(n) is an i.i.d. pr ocess. If u(n)
is symmet r ically dist r ibut ed, t hen t he bispect r um of x(n) is zer o. If u(n) is
Gaussian, t he bispect r um as well as all higher -or der polyspect r a of x(n) ar e
ident ically zer o. If u(n) is Laplace dist r ibut ed, t he bispect r um and all
odd-or der ed polyspect r a of x(n) ar e zer o, but t he even-or der ed polyspect r a
(such as t he t r ispect r um) ar e not ident ically equal t o zer o.
S

xx x
S

x xx
2
S

xx x

1

2
, ( )
x n ( ) h k ( )u n k ( )
k

=
1 Tu to r i a l
1-24
Not e also t hat univar iat e symmet r y or zer o skewness does not mean zer o
bispect r um. In t he above example, suppose t hat , and ;
t hen, C
3x
(0,0) = , so t hat pr ocess x(n) has zer o skewness.
This does not imply t hat C
3x
(
1
,
2
) 0 or S
3x
(
1
,
2
) 0. In par t icular , let
h(0) = h(1) = 1, h(k) = 0, k > 1; t hen, x(n) = u(n) u(n 1) is easily seen t o be
symmet r ically dist r ibut ed ar ound zer o, but t he bispect r um of x(n) is not
ident ically zer o.
In gener al, we use not ions of skew and symmet r y in t he cont ext of r andom
var iables (or univar iat e pdfs); bispect r a and t r ispect r a r elat e t o r andom
pr ocesses (mult ivar iat e pdfs).
Ex a mples
load gldat
glstat(g,0.51,256);
Test st at ist ic for Gaussianit y is 22.179 wit h df = 48, Pfa = 0.9995.
Linear it y t est : R (est imat ed) = 0.88819, lambda = 0.6982, R (t heor y) = 2.9288,
N = 14.
The dat a in g ar e Gaussian dist r ibut ed. Since t he Pfa is high, we cannot r eject
t he Gaussian hypothesis; but if t he Gaussian hypot hesis holds, t he bispect r um
must be zer o, and we cannot conclude, on t he basis of t he bispect r um alone,
whet her or not t he pr ocess is linear ; hence, t he r esult s of t he linear it y t est
should be ignor ed in t his case. In t his case, t he dat a ar e Gaussian, hence, also
linear .
glstat(u,0.51,256);
Test st at ist ic for Gaussianit y is 17.4885 wit h df = 48, Pfa = 1.
Linear it y t est : R (est imat ed) = 0.72383, lambda = 0.51704, R (t heor y) = 2.7453,
N = 14.
The dat a in u ar e unifor mly dist r ibut ed. Since t he Pfa is high, we cannot r eject
t he Gaussian hypot hesis; but if t he Gaussian hypot hesis holds, t he bispect r um
must be zer o, and we cannot conclude, on t he basis of t he bispect r um alone,
whet her or not t he pr ocess is linear ; hence, t he r esult s of t he linear it y t est
should be ignor ed in t his case.
glstat(e,0.51,256);

3 u
0
h
3
k ( )
k

0 =

3u
h
3
k ( )
k

0 =
Po l y sp e c tr a a n d Li n e a r Pr o c e sse s
1-25
Test st at ist ic for Gaussianit y is 253.3529 wit h df = 48, Pfa = 0.
Linear it y t est : R (est imat ed) = 7.8894, lambda = 9.4555, R (t heor y) = 8.4655,
N = 14.
The dat a in e obey t he single-sided exponent ial dist r ibut ion. The Pfa in
r eject ing t he Gaussian hypot hesis is ver y small; hence, we ar e comfor t able in
accept ing t he hypot hesis of non-Gaussianit y. The est imat ed and t heor et ical
int er quar t ile r anges (t he Rs) ar e fair ly close t o one anot her ; hence, we accept
t he linear it y hypot hesis. The est imat e of t he int er quar t ile r ange was based on
N=14 samples.
glstat(x,0.51,256);
Test st at ist ic for Gaussianit y is 277.5194 wit h df = 48, Pfa = 0.
Linear it y t est : R (est imat ed) = 6.7513, lambda = 10.6519, R (t heor y) = 8.968,
N = 14.
x was obt ained by passing t hr ough a linear filt er ; hence, it is linear and
non-Gaussian. We r eject t he Gaussianit y assumpt ion since t he Pfa is small; we
accept t he linear it y hypot hesis since t he est imat ed and t heor et ical
int er quar t ile r anges ar e close t o one anot her .
glstat(z,0.51,256);
Test st at ist ic for Gaussianit y is 12640.0657 wit h df = 48, Pfa = 0.
Linear it y t est : R (est imat ed) = 606.9323, lambda = 492.5759, R (t heor y) =
59.9088, N = 14.
z was obt ained by passing x t hr ough a nonlinear it y, z(n) = x
3
(n): hence, z is
non-Gaussian (and nonsymmet r ic). We r eject t he Gaussianit y assumpt ion
since t he Pfa is small; we cannot accept t he linear it y hypot hesis since t he
est imat ed int er quar t ile r anges is much lar ger t han t he t heor et ical value.
glstat(l,0.51,256);
Test st at ist ic for Gaussianit y is 49.931 wit h df = 48, Pfa = 0.3965.
Linear it y t est : R (est imat ed) = 2.6047, lambda = 1.8124, R (t heor y) = 4.0038,
N = 14.
1 Tu to r i a l
1-26
The dat a in l ar e i.i.d. and Laplace dist r ibut ed (symmet r ic). Since t he Pfa is
high, we cannot r eject t he Gaussian hypot hesis; but if t he Gaussian hypot hesis
holds, t he bispect r um must be zer o, and we cannot conclude, on t he basis of t he
bispect r um alone, whet her or not t he pr ocess is linear ; hence, t he r esult s of t he
linear it y t est should be ignor ed in t his case.
Parametric Estimators, ARMA Models
So far we have looked at nonpar amet r ic est imat or s. Par amet r ic est imat or s ar e
oft en useful, eit her because t hey lead t o par simonious est imat es, or because
t he under lying physics of t he pr oblem suggest a par amet r ic model.
The basic idea is t hat if x(n) depends upon a finit e set of par amet er s, , t hen all
of it s st at ist ics can be expr essed in t er ms of . For example, we obt ain
par amet r ic est imat es of t he power spect r um by fir st est imat ing , and t hen
evaluat ing P
xx
(f| ).
The specific for m we post ulat e for t he r elat ionship bet ween and t he sequence
x(n) const it ut es a model. A popular model in t ime-ser ies analysis is t he
Aut o-Regr essive Moving-Aver age (ARMA) model,
(1-31)
wher e u(n) is assumed t o be an i.i.d. sequence, wit h var iance . The
Aut o-Regr essive (AR) polynomial is defined by,
(1-32)
wher e a(0) = 1. A(z) is assumed t o have all it s r oot s inside t he unit cir cle, t hat
is, A(z
o
) = 0 | z
o
| < 1; t his condit ion is also r efer r ed t o as t he minimum-phase
or causal and stable condit ion. In gener al, no r est r ict ions need t o be placed on
t he zer os of t he Moving-Aver age (MA) polynomial,
(1-33)
x n ( ) a k ( )x n k ( )
k 1 =
p

b k ( )u n k ( )
k 0 =
q

+ =

u
2
A z ( ) a k ( )z
k
k 0 =
p

=
B z ( ) b k ( )z
k
k 0 =
q

=
Po l y sp e c tr a a n d Li n e a r Pr o c e sse s
1-27
however , B(z) is usually assumed t o be minimum-phase. The minimum-phase
assumpt ion is usually not t r ue for discr et e-t ime pr ocesses which ar e obt ained
by sampling a cont inuous-t ime pr ocess; algor it hms based on HOS do not
r equir e t his assumpt ion.
The power -spect r um of t he ARMA pr ocess is given by,
(1-34)
Not e t hat t he power spect r um does not r et ain any phase infor mat ion about t he
t r ansfer funct ion H(z) = B(z)/A(z). Since we do not have access t o t he sequence
u(n), one eit her assumes t hat u(n) has unit var iance, or t hat b(0) = 1. Inst ead
of est imat ing P
xx
()[1/2,1/2], as in t he nonpar amet r ic appr oach, we have
t o est imat e only (p + q + 1) par amet er s, namely, , , and
.
Let h(n) denot e t he impulse r esponse of t he model in (1-31); hence, H(z) = B(z)/
A(z). The AR and MA par amet er s ar e r elat ed t o t he impulse r esponse (IR) via,
(1-35)
(1-36)
In pr act ice, t he obser ved pr ocess is noisy, t hat is,
(1-37)
wher e pr ocess w(n) is addit ive color ed Gaussian noise; t he color of t he noise is
usually not known.
Given t he noisy obser ved dat a, y(n), we want t o est imat e t he a(k)s and t he
b(k)s in (1-31). We assume t hat t he model or der s p and q ar e known.
The det er minat ion of t he model or der s p and q is an impor t ant issue, and is
discussed lat er (r out ines arorder and maorder implement AR and MA model
or der det er minat ion t echniques). In gener al, it is bet t er t o over est imat e model
P
x x
( )
u
2 B z ( )
2
A z ( )
2
------------------ z e
j 2
= =
a { k ( ) }
k 1 =
p
b { k ( ) }
k 1 =
q

u
2
a k ( )h n k ( )
k 0 =
n

b n ( ) n 0 q , [ ] , =
0 n 0 q , [ ] , =
y n ( ) x n ( ) w n ( ) + =
1 Tu to r i a l
1-28
or der s r at her t han t o under est imat e t hem (however , specific implement at ions
or algor it hms may suffer fr om zer o-divided-by-zer o t ype pr oblems).
Mot ivat ion t o use cumulant s for t he ARMA par amet er est imat ion pr oblem is
as follows:
Cor r elat ion-based met hods can be used successfully only if q = 0 (pur e AR)
and t he addit ive noise is whit e.
Even in t he noiseless case, cor r elat ion-based met hods cannot ident ify
inher ent all-pass fact or s, and cannot r esolve t he phase of t he syst em.
Non-Gaussian pr ocesses ar e not complet ely char act er ized by t heir
second-or der st at ist ics; by using higher -or der st at ist ics, we ar e exploit ing
mor e of t he infor mat ion cont ained in t he dat a.
A not e of caut ion: for odd-or der ed cumulant s,
ku
0 does not imply t hat t he
kt h or der polyspect r um can be used t o r econst r uct H(). For example, even if

3u
0, S
3x
(
1
,
2
) in (1-20) may be ident ically equal t o zer o. This can happen
when H() is a r elat ively nar r ow-band bandpass signal; explicit condit ions ar e
given in [63]. An even mor e t r ivial example is
1u
= E{u(n)} 0, but H(0) = 0, in
which case
1x
= 0. For polyspect r a of even or der , say 2k,
cannot be ident ically zer o if
2,u
0 [63].
The t r ansfer funct ion H(z) = B(z)/A(z) is said t o have an inher ent all-pass fact or ,
if a r oot of B(z) lies at 1/z
o
, wher e z
o
is a r oot of A(z). The power spect r um is
blind t o all-pass fact or s. If t he dat a ar e noise fr ee, and if t he ARMA model does
not have any inher ent all-pass fact or s, t hen t echniques based on t he
aut ocor r elat ion/power spect r um may be used t o est imat e t he model
par amet er s.
Once t he ARMA par amet er s have been est imat ed, MATLABs r out ine freqz can
be used t o est imat e t he t r ansfer funct ion and t heor et ical spect r um;
Higher -Or der Spect r al Analysis Toolbox r out ines bispect and trispect can be
used t o comput e t he t heor et ical bispect r um and slices of t he t heor et ical
t r ispect r um cor r esponding t o t he ARMA model.
Synt het ic ARMA pr ocesses can be gener at ed via r out ine armasyn; r out ine
rpiid can be used t o gener at e i.i.d. sequences wit h var ious pr obabilit y densit y
funct ions (pdfs).
S
2k
, ( ,, , ) ,
2 k u ,
S
2x
( )
k

2 u ,
k
=
Po l y sp e c tr a a n d Li n e a r Pr o c e sse s
1-29
M A M odels
Let us consider t he pur e MA case, t hat is, p = 0, in (1-31) t hr ough (1-37). It was
est ablished in [19] t hat t he pr ocess y(n) sat isfies t he set of equat ions,
(1-38)
(1-39)
wher e n = q, . . ., 2q;
3
=
3u
b(q)/
2u
, and
4
=
4u
b(q)/
2u
. Equat ions (1-38) and
(1-39) can be r eadily ver ified by using t he impulse r esponse summat ion
for mulas in (1-16) t hr ough (1-18). These equat ions ar e somet imes r efer r ed t o
as t he GM equations in t he lit er at ur e [19].
Equat ions (1-38) and (1-39) r epr esent a set of 3q + 1 linear equat ions in t he 2q
+ 1 unknowns,
m+1
b(k), k = 0, . . ., q and b
m
(k), k = 1, . . ., q, wher e m = 2 or m
= 3. In [19], MA par amet er s wer e est imat ed fr om (1-38) or (1-40) by
simult aneously solving for t he
m+1
b(k)s and b
2
(k)s or b
3
(k)s. A disadvant age
of t he met hod is t hat it is over par amet er ized, and does not t ake int o account
t he r elat ionship bet ween t he
m+1
b(k)s and t he b
2
(k)s [or b
3
(k)s].
Addit ive whit e noise may be per mit t ed, pr ovided we eliminat e t he equat ions
involving C
2y
(0); t his eliminat es q + 1 equat ions, leaving us only 2q equat ions
in t he 2q + 1 unknowns. The Min-Nor m solut ion is not useful in t his case.
A modificat ion developed by Tugnait [72] appends t he following set s of
equat ions t o t he pr eceding set of 2q equat ions:
(1-40)
(1-41)
C
2y
n ( )
3
b k ( )C
3y
n k n k , ( )
k 0 =
q

b
2
k ( )C
2 y
n k ( )
k 1 =
q

=
C
2 y
n ( )
4
b k ( )C
4 y
n k n k n k , , ( )
k 0 =
q

b
3
k ( )C
2y
n k ( )
k 1 =
q

3
C
2y
n ( ) b k ( )C
3y
k n q , ( )
k 1 =
q

C
3y
n q , ( ) =

4
C
2y
n ( ) b k ( )C
4 y
k n q 0 , , ( )
k 1 =
q

C
4y
n q 0 , , ( ) =
1 Tu to r i a l
1-30
wher e n = q, . . ., q, n 0. Now, we have 4q equat ions in 2q + 1 unknowns;
hence, we can obt ain t he least -squar es solut ion.
We solve eit her (1-38) and (1-40), or (1-39) and (1-41). Since bot h b(k) and b
2
(k)
or b
3
(k) ar e est imat ed, we need some met hod for combining t he est imat es.
Let b
1
(k) and b
2
(k) denot e t he est imat es of b(k) and b
2
(k). If all of t he est imat ed
b
2
(k)s ar e nonnegat ive, t hen t he final MA par amet er est imat e is obt ained as,
ot her wise, .
When four t h-or der cumulant s ar e used, t he met hod est imat es bot h b(k) and
b
3
(k). Let b
1
(k) and b
3
(k) denot e t he est imat es of b(k) and b
3
(k). If all t he
est imat ed b
3
(k)s have t he same sign as t he cor r esponding b
1
(k)s, t hen t he final
MA par amet er est imat e is obt ained as,
ot her wise, .
This algor it hm is implement ed in r out ine maest. If t he obser ved pr ocess is
z(n) = y(n) + g(n), wher e g(n) is addit ive noise, independent of y(n), t hen
C
2
z(n) = C
2
y(n) + C
2g
(n), n. f g(n) is whit e, C
2g
(n) = 0, n 0; hence,
C
2
z(n) = C
2
y(n), n 0. In (1-40) and (1-41), we use t he aut ocor r elat ion
C
2
y(n), n = t1, . . ., tq, hence, t he algor it hm can handle addit ive whit e noise but
not color ed noise.
Ex a mples
load ma1
bvec = maest (y,3,3,128);
The est imat ed par amet er s should be [1, 0.9714, 0.3814, 0.7759]. The t r ue
par amet er s ar e [1, 0.9, 0.385, 0.771], and t he signal is cont aminat ed wit h
whit e Gaussian noise.
bvec = maest (y,3,4,256);
The est imat ed par amet er s should be [1, 0.9608, 0.4482, 0.7343].
b

k ( ) sign b
1
k ( ) [ ]
*
0.5 b
1
2
k ( ) b
2
k ( ) + [ ]; =
b

k ( ) b
1
k ( ) =
b

k ( ) sign b
1
k ( ) [ ]
*
b
1
k ( ) b
3
k ( )
1 3
+ [ ] 2; =
b

k ( ) b
1
k ( ) =
Po l y sp e c tr a a n d Li n e a r Pr o c e sse s
1-31
AR M odels
The cumulant s of t he noisy pr ocess sat isfy t he nor mal equat ions,
(1-42)
(1-43)
(1-44)
which can be ver ified by expr essing t he cumulant s in t er ms of t he IR, h(n),
using (1-16) t hr ough (1-18), and by not ing t hat .
The r elat ionship bet ween linear prediction (LP) and AR models is discussed in
det ail in t he sect ion t it led Polyspect r a and Linear Pr ocesses. Her e, we will
just point out t hat t he least squar es solut ion t o t he LP pr oblem is given by
(1-42), wit h m = 1, . . ., p; t hese equat ions ar e called t he normal equat ions
because t he r esult ing pr edict ion-er r or sequence is or t hogonal t o t he dat a.
However , when q > 0 or when addit ive noise is pr esent in t he dat a, t he normal
equations yield inconsist ent est imat es of t he AR par amet er s. Equat ions (1-42)
t hr ough (1-44) yield consist ent est imat es, and can also be der ived by
demanding t hat t he pr edict ion er r or sequence be or t hogonal t o an
inst r ument al pr ocess der ived fr om t he dat a [59, 64]. We put normal in quot es
t o emphasize t hese differ ences.
Ident ifiabilit y of t he AR par amet er s is guar ant eed by choosing p = q p, . . ., q,
and m = q + 1, . . ., q + p; we may use mor e slices () or mor e lags (m) [67]. In
pr act ice, we use sample est imat es of t he cumulant s. This algor it hm is
implement ed in r out ine armarts. The pur e AR case cor r esponds t o q = 0 and is
implement ed in r out ine arrcest. In bot h r out ines, you can use cumulant
or der s (2, 3, or 4). It is also possible t o simult aneously solve for t he normal
equat ions based on cumulant or der s 2 and 3 or 2 and 4.
a k ( )C
2 y
m k ( )
k 0 =
p

0 m q > =
a k ( )C
3 y
m k , ( )
k 0 =
p

0 , m q > =
a k ( )C
4y
m k , , ( )
k 0 =
p

0 , m q , > =
a k ( )h n k ( )
k 0 =
p

b n ( ) =
1 Tu to r i a l
1-32
Ex a mples
Tr y t he following
load ar1
ar(:,1)=arrcest (y,2,0,2,12,128);
ar(:,2)=arrcest (y,2,0,3,12,128);
ar(:,3)=arrcest (y,2,0,4,12,128);
ar(:,4)=arrcest (y,2,0,3,12,128);
ar(:,5)=arrcest (y,2,0,4,12,128);
disp(ar)
1.0000 1.0000 1.0000 1.0000 1.0000
-1.4636 -1.5559 -1.4963 -1.4912 -1.4755
0.7664 0.8779 0.8917 0.7973 0.7927
The t r ue par amet er s ar e [1, 1.5, 0.8]. The five columns cor r espond t o AR
est imat es based on: (1) second-or der , (2) t hir d-or der , (3) four t h-or der , (4)
combined second-or der and t hir d-or der , and (5) combined second-or der and
four t h-or der cumulant s. Not e t hat combined use of aut ocor r elat ion and
cumulant s may give bet t er r esult s when t he signal-t o-noise r at io (SNR) is high.
In t he case of low or moder at e SNR, t he cor r elat ion-based est imat es will be
biased; est imat es based on t hir d-or der (four t h-or der ) cumulant s will be
unbiased if t he addit ive noise is symmet r ic (Gaussian).
ARM A M odels
As discussed in t he pr evious sect ion, we can det er mine t he AR par amet er s
easily. We will ignor e t he est imat ion er r or s, and assume t hat ,
k = 1, . . ., p; t his is just ified if t he dat a lengt hs ar e long enough t o ensur e good
est imat es of t he cumulant s. In pr act ice, er r or s in est imat ing t he a(k)s will
show up as an addit ive non-Gaussian noise t er m on t he r ight -hand side of
(1-46).
Consider t he r esidual t ime ser ies obt ained via,
(1-45)
a k ( ) a k ( ) =
z n ( ) a k ( )y n k ( )
k 0 =
p

=
Po l y sp e c tr a a n d Li n e a r Pr o c e sse s
1-33
(1-46)
Rout ine armarts uses t he r esidual t ime ser ies met hod t o est imat e t he ARMA
par amet er s it est imat es t he AR par amet er s fir st (via t he nor mal
equat ions); it t hen comput es t he AR-compensat ed t ime ser ies via (1-45), and
finally est imat es t he MA par amet er s via r out ine maest (see (1-46)). Since w(n)
was assumed t o be Gaussian, w
1
(n) is also Gaussian; if w(n) is whit e, w
1
(n) is
MA(p) noise. Rout ine maest assumes t hat t he addit ive noise is whit e, hence t he
r esult s of armarts ar e meaningful only at high SNR.
An alt er nat ive solut ion based on is implement ed in r out ine armaqs, which is a
q-slice met hod. The AR par amet er s ar e est imat ed via t he nor mal equat ions as
befor e. The impulse r esponse is t hen est imat ed via,
(1-47)
or via,
(1-48)
These equat ions can be r eadily ver ified by using (1-16) t hr ough (1-18) and
(1-35). The MA par amet er s ar e t hen obt ained via (1-35), which is r epeat ed
her e:
(1-49)
b k ( )u n k ( )
k 0 =
q

a k ( )w n k ( )
k 0 =
p

+ =
b k ( )u n k ( )
k 0 =
q

w
1
n ( )
k 0 =
p

+ =
h n ( )
a k ( )C
3y
q k n , ( )
k 0 =
p

a k ( )C
3 y
q k 0 , ( )
k 0 =
p

-----------------------------------------------------------------, n 1 q , , = =
h n ( )
a k ( )C
4y
q k n 0 , , ( )
k 0 =
p

a k ( )C
4y
q k 0 0 , , ( )
k 0 =
p

------------------------------------------------------------------------, n 1 q , , = =
b n ( ) a k ( )h n k ( )
k 0 =
p

, n 1 q. , , = =
1 Tu to r i a l
1-34
An int er est ing point is t hat we can simult aneously solve for t he AR and IR
par amet er s. This algor it hm is implement ed in r out ine armaqs.
Weight ed ver sions of (1-42) t hr ough (1-44), and nonlinear cumulant-matching
algor it hms for t he simult aneous est imat ion of AR and MA par amet er s ar e
discussed in [16] and [71].
Ex a mples
load arma1
[avec,bvec]=armaqs(y,2,1,3,10,128);
Her e we used t hir d-or der cumulant s and t he q-slice algor it hm t o est imat e t he
par amet er s of a non-Gaussian ARMA pr ocess. The est imat ed par amet er s
should be avec = [1, 0.8057, 0.6910], and bvec = [1, 1.9116]. The t r ue AR and
MA par amet er s wer e [1, 0.8, 0.65] and [1, 2], r espect ively.
[avec, bvec] = armarts(y,2,1,3,12,128);
Her e we used t hir d-or der cumulant s and t he r esidual t ime-ser ies algor it hm t o
est imat e t he par amet er s of a non-Gaussian ARMA pr ocess. The est imat ed
par amet er s should be avec = [1, 0.7540, 0.6465], and bvec = [1, 1.5665]. The
t r ue AR and MA par amet er s wer e [1, 0.8, 0.65] and [1, 2], r espect ively.
AR O rder Deter mina tion
Let and denot e t he maximum expect ed values of t he AR and MA or der s.
Let
(1-50)
(1-51)
(1-52)
Then, t he singular values of t he mat r ix,
wher e k = 2, k = 3, or k = 4 ar e comput ed.
p q
c
2y
m ( ) := C
2y
m ( )
c
3y
m ( ) := C
3 y
m ( p ) C
3y
, , m ( q , [ ]
T
c
4y
m ( ) := C
4y
m ( p 0 , ) C
4 3 ( )y
, , m ( q 0 , , [ ]
T
Po l y sp e c tr a a n d Li n e a r Pr o c e sse s
1-35
Let s(m) denot e t he singular values. If t he cumulant est imat es ar e per fect , we
expect t hat exact ly p of t he singular values will be nonzer o; wit h sample
est imat es, we expect p dominant singular values. The AR or der p is t hen given
by t he value of n, which maximizes s(n) s(n + 1), t hat is, it cor r esponds t o t he
index at which t he singular values show t he maximum dr op. This is a
nonst at ist ical t est , and is based on t he fact t hat for an ARMA(p,q) model, only
p of t he singular values should be nonzer o. This algor it hm is implement ed in
r out ine arorder.
Ex a mples
load arma1
p=arorder(y,3);
You should see t he display in Figur e 1-6. The est imat ed AR or der is 2.
The t ime-ser ies y cor r esponds t o a non-Gaussian ARMA(2,1) pr ocess,
cont aminat ed by AWGN, wit h SNR of 20 dB. The or der det er minat ion is based
on t hir d-or der cumulant s.
Figure 1-6 Estimate of AR Order Using SVD Method (arorder)
C
k
c
k y
q 1 + ( ) ... c
k y
q p + ( )
c
k y
q p 1 + + ( ) ... c
k y
q 2p 1 + ( )
=

.
.
.
1 2 3 4 5 6 7 8 9
0
0.1
0.2
0.3
0.4
0.5
0.6
difference in singular values, cumulant order=3
1 Tu to r i a l
1-36
M A O r der Deter mina tion
The basic idea is t hat for an MA(q) pr ocess, t he t r ue values of t he cumulant
C
3y
(m,0) will be ident ically zer o, if m > q. When t he t r ue cumulant s ar e
r eplaced by sample est imat es, t he est imat ed values of C
3y
(q + i,0), i > 0 will not
be ident ically zer o; a st at ist ical t est is used t o det er mine whet her t he est imat ed
values ar e close t o zer o. This t est is based on est imat ing t he t heor et ical
var iance of t he sample est imat es of C
3y
(m,0).
Sample est imat es, , and t heir var iances ar e est imat ed for m r anging
fr om qmin t o qmax, which r eflect our a priori knowledge of t he bounds on t he
t r ue or der q.
For an MA(q) pr ocess, t he asympt ot ic var iance of t he sample est imat e of
C
3y
(q + 1,0) can be est imat ed via [20]
wher e N is t he lengt h of t he t ime ser ies.
The sample est imat es ar e asympt ot ically Gaussian and unbiased; hence, t he
t hr eshold t
c
in
is given by
wher e erfinv is t he MATLAB inver se er r or funct ion; in pr act ice, we use t he
sample est imat e . Let m
o
denot e t he lar gest value of m in t he r ange qmin t o
qmax for which | c
3y
(m + 1,0)| > t
c
(m + 1) (so t hat t he hypot hesis of MA(m
o
)
model fails); t hen, t he est imat ed or der is q = m
o
+ 1; if such an m does not exist ,
t he MA or der is declar ed t o be qmax + 1. This algor it hm is implement ed in
r out ine maorder.
This is a st at ist ical t est , and pa specifies t he fr act ion of t he t ime t hat t he t est
r esult s will be wr ong. In ot her wor ds, in a Mont e Car lo simulat ion of 1000
t r ials, one should expect t he t est r esult s t o be wr ong 1000
*
pa t imes.
C

3 y
m 0 , ( )

q 1 + ( )
1
N
2
------- 1
j
N
----
,
_
j q =
2q 1 +

i 1 =
N

y
2
i ( )y i q 1 + + ( ) c
3y
q 1 0 , + ( ) [ ] =
y
2
i j + ( )y i ( j q 1 ) c
3y
q ( 1 0 , ) + + + + [ ]
Pr c
3y
m 1 0 , + ( ) t
c
m 1 + ( ) { } 1 pa =
t
c
m 1 + ( ) eri n v 1 pa ( ) 2
2
m 1 + ( ) [ ] = ,

Po l y sp e c tr a a n d Li n e a r Pr o c e sse s
1-37
Ex a mples
load ma1;
q=maorder(y,0,6);
The following t able will be displayed:
The columns in t he t able cor r espond t o t he est imat ed var iance of c
3y
(q,0), t he
est imat ed value of c
3y
(q,0), t he cor r esponding t hr eshold, and whet her or not
t he absolut e value of t he est imat ed c
3y
(q,0) exceeded t he t hr eshold.
The t ime-ser ies y cor r esponds t o an MA(3) pr ocess, cont aminat ed by AWGN,
wit h SNR of 20 dB.
Linear Processes: Impulse Response Estimation
The basic model her e is,
(1-53)
(1-54)
wher e addit ive noise w(n) is assumed t o be symmet r ic dist r ibut ed (not
necessar ily Gaussian). Pr ocess u(n) is i.i.d., non-Gaussian, independent of t he
noise, and sat isfies 0 < | C
3u
(0,0)| < .
q var(cqk) cqk thres result
0 7.17383e003 7.31541e001 5.24957e001 0
1 6.60864e002 1.37221e001 1.59332e001 1
2 3.76124e002 4.11813e001 3.80114e001 0
3 7.04832e003 5.06864e003 1.64547e001 1
4 3.77757e003 3.31784e002 1.20463e001 1
5 1.31938e003 6.87545e002 7.11923e002 1
6 4.47359e003 1.43717e003 1.31092e001 1
Estimated MA order is 3
x n ( ) h k ( )u n k ( )
k =

=
y n ( ) x n ( ) w n ( ) + =
1 Tu to r i a l
1-38
We will discuss t wo algor it hms, bot h based on t he not ion of t he logar it hm of t he
bispect r um.
The Polycepstr a l M ethods
It is assumed t hat H(z) has no zer os on t he unit cir cle. In t his case, t he
cepst r um of H(z) is well defined and is given by [42]
(1-55)
The bicepst r um is also well defined [43] and is given by,
(1-56)
(1-57)
The complex cepst r um, , is t he inver se FT (IFT) of t he log of t he FT of h(n);
hence, h(n) is t he IFT of t he exponent ial of t he FT of t he complex cepst r um;
ot her t echniques t o obt ain h(n) fr om ar e descr ibed in [42]. Not ice t hat for
a linear pr ocess, t he bicepst r um is nonzer o only along t he axes, m = 0, n = 0,
and t he diagonal line, m = n.
Dir ect implement at ion of (1-56) demands 2-D phase unwr apping. Pan and
Nikias [43] have developed an alt er nat ive met hod, based on t he r elat ionship,
(1-58)
which, t aking int o account (1-57), r educes t o
The complex cepst r um is known t o be exponent ially bounded; hence, one may
r eplace t he infinit e summat ion over m by a finit e summat ion, m = q t o m = +p,
h

k ( ) d

j 2k ( ) exp H ln ( ) =
b

m n , ( ) d
1
d
2
e
j 2
1
m
e
j 2
2
n
S
3x
ln
1

2
, ( )

=
h

m ( ) m ( ) h

n ( ) n ( ) h

m ( ) m n ( ) + + =

3u
ln m ( ) n ( ) +
h

k ( )
h

k ( )
m b

m ( n )C
3y
k m l n , ( ) ,
n =

m =

k C
3y
k l , ( ) =
m h

m ( ) C
3y
k m n , ( ) C
3y
k m + l m + , ( ) + [ ]
m =

k C
3y
k l , ( ) =
Po l y sp e c tr a a n d Li n e a r Pr o c e sse s
1-39
wher e p and q may be based on some a priori knowledge; not e t hat t he p and q
have not hing t o do wit h t he or der s of an ARMA(p,q) pr ocess. Now we have a set
of linear equat ions for est imat ing a finit e set of p + q par amet er s.
(1-59)
Not e t hat t he above equat ion does not involve ; t he cepst r um at t he or igin
is r elat ed t o t he over all gain of t he syst em. Because of t he inher ent scalar
ambiguit y in est imat ing H(z) fr om power spect r a or cumulant spect r a, we let
= 1. In pr act ice sample est imat es of t he t hir d-or der cumulant s ar e used;
t his algor it hm is implement ed in r out ine biceps, wher e | k| max(p,q) and
| l| max(p,q)/2, which ar e t he r ecommended r anges in [43].
Ex a mples
load ma1
[hest, ceps]=biceps(y,8,8,128);
You should see t he display in Figur e 1-7. Not e t hat t he complex cepst r um
decays r apidly t o zer o. The t r ue MA par amet er s ar e [1, 0.9, 0.385, 0.771].
Figure 1-7 Cepstrum and IR Estimated by Biceps
m h

m ( ) C
3y
k m n , ( ) C
3y
k m + l m + , ( ) + [ ]
m q =
p

k C
3 y
k l , ( ) =
h

0 ( )
h

0 ( )
8 6 4 2 0 2 4 6 8
1
0.5
0
0.5
1
1.5
complex cepstrum
sample number
40 30 20 10 0 10 20 30 40
5
0
5
10
impulse response
sample number
1 Tu to r i a l
1-40
An alt er nat ive appr oach is based on t he FFT. We may r ewr it e (1-58) as [43]
(1-60)
wher e IFT denot es 2-D inver se Four ier t r ansfor m. Then we make use of (1-57)
t o obt ain t he s fr om . The FFT appr oach is useful if t he
cepst r um has long suppor t . This algor it hm is implement ed in r out ine
bicepsf.
Mor e gener ally, t he polycepst r um is defined as t he inver se FT of t he log of t he
cor r esponding polyspect r um (assuming t hat it exist s),
For a linear pr ocess, t he polycepst r um is nonzer o only along t he k 1 axes, and
t he main diagonal,
Tekalp and Er dem [69] showed a pr ocess, whose polycepst r um exist s, is linear
if and only if it s polycepst r um has t he above k-line r egion of suppor t . Based on
t his, t hey have also pr oposed measur es of linear it y.
m b

m n , ( ) I FT
FT k C
3 y
k l , ( ) ( )
FT C
3y
k l , ( ) ( )
----------------------------------------
,
_
=
h

k ( ) m b

m n , ( )
h

k ( )
b

k x
m
1
m
k 1
, , ( ) d
1
d

k 1
e
j 2
1
m
1
ej 2

k 1
m
k 1 =

=
S
k x
ln
1

k 1
, , ( )
b

k x
m
1
m
k 1
, , ( ) h

m
l
( ) m
n
( )
n l n 1 = ;
k 1

l 1 =
k 1

=
h

+ m
l
( ) m
1
m
l
( )
l 2 =
k 1

k u
ln + m
l
( )
l 2 =
k 1

Po l y sp e c tr a a n d Li n e a r Pr o c e sse s
1-41
Ex a mples
load ma1
[hest, ceps]=bicepsf(y, 8);
You should see t he display in Figur e 1-8. Not e t hat t he complex cepst r um
decays r apidly t o zer o. The t r ue MA par amet er s ar e [1, 0.9, 0.385, 0.771]; not e
t he scale ambiguit y (including sign) in t he est imat ed impulse r esponse.
Figure 1-8 Cepstrum and IR Estimated by bicepsf
The M a tsuoka - Ulr ych Algor ithm
We can wr it e t he bispect r um in t er ms of it s magnit ude and phase as,
S
3x
(
1
,
2
) = M(
1
,
2
)exp(j(
1
,
2
)).
Let H() = | H()| exp(j()) so t hat
(1-61)
(1-62)
80 60 40 20 0 20 40 60 80
0.5
0
0.5
1
1.5
complex cepstrum
80 60 40 20 0 20 40 60 80
2
1
0
1
2
3
impulse response
M
1

2
, ( ) ln
3u
ln H
1
( ) ln H
2
( ) ln H
1

2
+ ( ) ln + + + =

1

2
, ( )
1
( )
2
( )
1

2
+ ( ) mod 2 + =
1 Tu to r i a l
1-42
wher e
1
,
2
t ake on discr et ized values on a gr id. The assumpt ion t hat |
3u
| =
1 will only int r oduce an over all scalar ambiguit y; hence, (1-61) r epr esent s a set
of linear equat ions in t he log-magnit ude ln| H()| . It is shown in [36] t hat a full
r ank set of linear equat ions can be obt ained by appr opr iat e choice of
1
and
2
.
The phase r elat ionship in (1-62) holds only modulo 2. If t he unwr apped 2-D
phase wer e available, t hen (1-62) yields a linear set of equat ions in t he phase
of t he t r ansfer funct ion, (). This algor it hm is implement ed in r out ine matul,
wher e we also incor por at e t he phase-unwr apping algor it hms of [54]. Because
of t he phase ambiguit y, t his r out ine is not r ecommended for r out ine use.
Ex a mples
cmat=cumtrue([1 3.5 1.5],[1],3,5);
bsp=fft2(flipud(cmat),64,64);
hest=matul(bsp);
You should see t he display in Figur e 1-9. Her e we comput ed t he t r ue
bispect r um of a MA(2) pr ocess, by evaluat ing t he FT of it s t r ue cumulant
sequence. We t hen used t he Mat suoka-Ulr ych algor it hm t o est imat e t he
impulse r esponse. Not e t he scale ambiguit y (including sign).
Figure 1-9 IR Estimated via the Matsuoka-Ulrych Algorithm (matul)
40 30 20 10 0 10 20 30 40
1
0.8
0.6
0.4
0.2
0
0.2
0.4
0.6
Estimated impulse response
sample number
Po l y sp e c tr a a n d Li n e a r Pr o c e sse s
1-43
Linear Processes: Theoretical Cumulants and
Polyspectra
If x(n) is an ARMA pr ocess, t hat is,
t hen, it s impulse r esponse can be calculat ed via
, wher e h(n) = 0 for n < 0, h(0) = 1, and b(n) = 0,
if .
For a linear pr ocess x(n) (x(n) = , wher e u(n) is i.i.d.)
cumulant s and polyspect r a of or der s 2,3,4 ar e given by (1-16) t hr ough (1-18)
and (1-19) t hr ough (1-21).
We can est imat e t he t r ue cumulant s using r out ine cumest, t he t heor et ical
bispect r um via r out ine bispect, and slices of t he t heor et ical t r ispect r um via
r out ine trispect.
Ex a mples
Fir st , we will comput e t he t heor et ical t hir d-or der cumulant s of an ARMA(2,1)
pr ocess, wit h AR par amet er s, [1, 1.5, 0.8], and MA par amet er s, [1, 2]; we will
t hen display t he est imat es using MATLABs funct ions mesh or contour:
cmat=cumtrue([1,-2],[1,1.5,0.8],3,25);
clf, subplot(121)
mesh(25:25,25:25,cmat), grid on
subplot(122), contour(25:25,25:25,cmat,8,),grid on
You should see t he display in Figur e 1-10.
x n ( ) x n k ( )
k 1 =
p

b k ( )u n k ( )
k 0 =
q

+ = ,
h n ( ) h n k ( )
k 1 =
p

b n ( ) + =
n 0 q , [ ]
h k ( )u n k ( )
k

1 Tu to r i a l
1-44
Figure 1-10 True Third-Order Cumulants of an ARMA(2,1) Process (cumtrue)
Let us now comput e t he t heor et ical bispect r um of anot her ARMA(2,1) pr ocess.
ma=[1 2]; ar=[1 0.8 0.65];
bisp=bispect (ma,ar,128);
You should see t he display on Figur e 1-11. The 12 dot t ed lines emanat ing fr om
t he or igin divide t he bifr equency domain of t he bispect r um int o 12 r egions of
symmet r y.
Figure 1-11 Bispectrum of an ARMA(2,1) Process (bispect)
40
20
0
20
40
50
0
50
15
10
5
0
5
10
20 10 0 10 20
25
20
15
10
5
0
5
10
15
20
25
0.5 0.4 0.3 0.2 0.1 0 0.1 0.2 0.3 0.4
0.5
0.4
0.3
0.2
0.1
0
0.1
0.2
0.3
0.4
f1
f
2
Po l y sp e c tr a a n d Li n e a r Pr o c e sse s
1-45
We will comput e sever al slices of t he t r ispect r um and t hen display t hem as a
movie:
clf,clear
ma=[1 2]; ar=[1 0.8 0.65]; nfft=64;
n=10; M=moviein(2*n+1);
for k=n:n
trispect(ma,ar,nfft,k/(2*n));
M(:,k+n+1)=getframe;
end
movie (M)
clear M
We comput ed sever al slices of t he t heor et ical t r ispect r um of an ARMA pr ocess;
t he slices cor r espond t o
3
= 0.5, 0.45, . . ., 0.45, 0.5. Then we used MATLABs
movie command t o display t hem; moviein and getframe ar e also MATLAB
commands. It is inst r uct ive t o change t he MA par amet er s t o [1 1], and t o
compar e t he

3
= 0 slice wit h t he t heor et ical bispect r um cr eat ed by bispect.
NOTE: if you encount er OUT OF MEMORY pr oblems in r unning t he example,
clear t he wor kspace and t hen t r y again. If t he pr oblem per sist s, decr ease n t o,
say, 5; or inst all mor e memor y on your machine.
Summary
We have discussed var ious algor it hms for est imat ing bispect r a and t r ispect r a,
as well as algor it hms for t he blind syst em ident ificat ion pr oblem.
Sample est imat es of cr oss-cumulant s of or der s 2, 3 and 4 may be obt ained via
cum2x, cum3x, and cum4x; aut ocumulant s can be est imat ed via cumest.
Nonpar amet r ic est imat es of t he bispect r um can be obt ained via r out ines
bispeci and bispecd, which implement t he indir ect and dir ect est imat or s. The
dir ect est imat e of t he cr oss-bispect r um can be obt ained via r out ine bispecdx.
Aut o- and cr oss-bicoher ences can be est imat ed via bicoher and bicoherx. The
t heor et ical bispect r um of a linear pr ocess can be comput ed via bispect;
trispect comput es slices of t he t heor et ical t r ispect r um of a linear pr ocess.
M-file glstat implement s t est s for non-Gaussianit y (act ually nonzer o
bispect r um) and for linear it y.
1 Tu to r i a l
1-46
Thr ee r out ines ar e available for nonpar amet r ic est imat ion of t he impulse
r esponse of a linear non-Gaussian pr ocess: matul implement s t he
Mat suoka-Ulr ych met hod; given t he bispect r um of t he dat a, it est imat es t he
amplit ude and phase of t he t r ansfer funct ion separ at ely; biceps uses t he
bicepst r al met hod; it uses sample est imat es of t he t hir d-or der cumulant ;
bicepsf is a fr equency-domain implement at ion of biceps and is useful if t he
cepst r a do not decay fast enough.
Sever al algor it hms ar e available for est imat ing t he (ARMA) par amet er s of
non-Gaussian linear pr ocesses. The ARMA or der s can be est imat ed via
arorder and maorder. Rout ine arrcest can be used t o est imat e t he AR
par amet er s of AR or ARMA pr ocesses, based on second-, t hir d-, and
four t h-or der cumulant s. The est imat es, based on cumulant s of differ ent or der s,
may not be t he same, if t he pr ocess is nonlinear or has inher ent all-pass fact or s,
or is noisy. The par amet er s of an MA pr ocess, cont aminat ed by whit e noise,
may be est imat ed via r out ine maest.
ARMA par amet er s can be est imat ed via t he r esidual t ime-ser ies algor it hm in
armarts; t he MA est imat ion par t of t his t hr ee-st ep algor it hm wor ks well only
under good SNR condit ions. Rout ine armaqs implement s t he q-slice algor it hm;
it est imat es bot h AR and MA par amet er s.
In t his sect ion, we have given a quick over view of t he ar ea of higher -or der
st at ist ics; for mor e ext ended t ut or ial exposit ions, see [41, 37, 38].
Li n e a r Pr e d i c ti o n M o d e l s
1-47
Linear Prediction Models
The AR model is also obt ained if one consider s t he pr oblem of linear pr edict ion
(LP). Depending upon whet her one chooses for war d-, backwar d- or
for war d-backwar d pr edict ion cr it er ia, differ ent solut ions ar e obt ained.
The Levinson algor it hm enables efficient est imat ion of t he par amet er s of an
AR pr ocess fr om it s aut ocor r elat ion (AC) sequence. In t he det er minist ic
cont ext , AR modeling leads t o t he for war d backwar ds least squar es (FBLS)
solut ion. The RLS algor it hm is a t ime-r ecur sive solut ion t o t he LP pr oblem,
wher eas t he lat t ice algor it hm pr ovides a t ime- and or der -r ecur sive solut ion.
Levinson Recursion
Consider t he linear pr edict ion pr oblem for a st at ionar y pr ocess, x(n). In t he
for war d-pr edict ion pr oblem, we want t o choose t o minimize t he
for war d pr edict ion er r or var iance, ,
(1-63)
(1-64)
wher e t he subscr ipt p denot es t he or der of t he pr edict or . Similar ly, in t he
backwar d-pr edict ion pr oblem, we want t o choose t o minimize t he
backwar d pr edict ion er r or var iance, ,
(1-65)
(1-66)
a
p
k ( ) { }
k 1 =
p
P
p

P
p

E e
p

n ( )
2

' ;

=
e
p

n ( ) a
p
*
k ( )x n k ( )
k 0 =
p

a
p
0 ( ) 1 = , =
c
p
k ( ) { }
k 1 =
p
P
p
b
P
p
b
E e
p
b
n ( )
2

' ;

=
e
p
b
n ( ) c
p
*
k ( )x n k ( )
k 0 =
p

c
p
p ( ) 1 = , =
1 Tu to r i a l
1-48
This leads t o t he nor mal equat ions
(1-67)
(1-68)
wher e is t he (p + 1) (p + 1) aut ocor r elat ion mat r ix, wit h (m,n) ent r y,
R
xx
(n m). It follows immediat ely t hat
(1-69)
The Levinson-Dur bin r ecur sion implement ed in r out ine trench pr ovides an
efficient or der -r ecur sive solut ion of t he nor mal equat ions: it r ecur sively solves
(1-67), for p = 1,2,. . . .
The r ecur sion is given by: for m = 1, . . ., p, [32]
(1-70)
(1-71)
(1-72)
(1-73)
wit h init ial condit ions P
0
= R
xx
(0),
0
R
xx
(1), and a
0
(0) = 1. In pr act ice, we
r eplace R
xx
(k) by it s (biased) sample est imat e. The r esult ing solut ion is
a
p
P
p

0
=
c
p
0
P
p
b
= ,
c k ( ) a
*
p k ( ) k , 0 p P
p

; , , P
p
b
. = = =

m 1
a
m 1
k ( )R
x x
k m ( )
k 0 =
m 1

m 1
P
m 1
--------------- =
P
m
P
m 1
1
m

2
( ) =
a
m
k ( ) a
m 1
k ( )
m
+ a
*
m 1
m k ( ) k 0 m , , = , , =
Li n e a r Pr e d i c ti o n M o d e l s
1-49
guar ant eed t o be st able if biased est imat es ar e used. Necessar y and sufficient
condit ions for t he st abilit y of t he est imat e ar e |
m
| < 1
m
.
The r ecur sion involves comput at ion of t he
m
s, which ar e also called t he
r eflect ion coefficient s; a necessar y and sufficient condit ion for st abilit y
is |
m
| < 1.
Trench Recursion
If t he mat r ix is Toeplit z but not Her mit ian symmet r ic, t hen (1-69) does not
hold; t his, for example, is t he case when t he so-called higher -or der Yule Walker
equat ions,
(1-74)
ar e used, or when t he cumulant-based normal equations ar e used, for example,
(1-75)
In bot h cases t he r esult ing mat r ix is Toeplit z but not symmet r ic. An efficient
or der -r ecur sive solut ion t o t he syst em of equat ions is given by t he Tr ench
r ecur sion [74].
Let denot e t he nonsymmet r ic Toeplit z mat r ix whose (i, j) ent r y is (i j). The
r ecur sion is given by
(1-76)
(1-77)
a
p
k ( )R
xx
m k ( )
k 0 =
p

0 m q 1 + q p + , , = , =
a
p
k ( )C
3x
m k , ( )
k 0 =
p

0 m q 1 + q p. + , , = , =

m 1

a
m 1
k ( ) m k ( )
k 0 =
m 1

m 1
b
c
m 1
k ( ) 1 k ( )
k 0 =
m 1

=
1 Tu to r i a l
1-50
(1-78)
(1-79)
(1-80)
(1-81)
(1-82)
(1-83)
(1-84)
wit h init ial condit ions P
0
= (0), a
0
(0) = 1, and c
0
(0) = 1.
The inver sion algor it hm r equir es t hat t he mat r ix be st r ongly nonsingular ,
t hat is, R, as well as all of it s pr incipal minor s, ar e nonsingular . The condit ion
of nonsingular it y is weaker t han t he usual assumpt ion of posit ive-definit eness.
In t he symmet r ic posit ive-definit e case, t he s ar e bounded by unit y, and t he
P
m
s ar e nonincr easing, and c
m
(k) = a
m
(m k), k = 0, . . ., m. In t he
non-Her mit ian case, t he s ar e not bounded, and t he P
m
s may incr ease or
decr ease.
This algor it hm is implement ed in trench.
Ex a mples
Let us comput e t he aut ocor r elat ion sequence for an AR(4) model, wit h AR
par amet er s [1,0.3,0.1,0.39,0.72] , and t hen apply t he Tr ench r ecur sion.
ar=[1,0.3,0.1,0.39,0.72]; ma=1; nlags=6;
rvec=cumtrue (ma,ar,2,nlags);
[a2,c2,p2,gf2,gb2]=trench( rvec(7:13), rvec(7:1:1));

m 1

P
m 1
--------------- =

m
b

m 1
b
P
m 1
--------------- =
P
m
P
m 1
1
m

m
b
( ) =
a
m
m ( )
m

=
c
0
m ( )
m
b
=
a
m
k ( ) a
m 1
k ( )
m

c
m 1
k 1 ( ) + k 1 m 1 , , = , =
c
m
k ( ) c
m 1
k 1 ( )
m
b
a
m 1
k ( ) + k 1 m 1 , , = , =
Li n e a r Pr e d i c ti o n M o d e l s
1-51
The AR mat r ix, a2, should be
Not e t hat t he AR vect or s for or der s gr eat er t han t hr ee ar e all essent ially t he
same. The pr edict ion er r or var iance, p2, should be
[2.4049, 2.3881, 2.0764, 1.0000, 1.0000, 1.0000]
The for war d r eflect ion coefficient s, gf2, should be
[0.0764, 0.0834, 0.3613, 0.7200, 0.0000, 0.0000]
Since t he Toeplit z mat r ix in t his example is symmet r ic, gb2=gb1, and
c2=flipud(a2). Since t he aut ocor r elat ion sequence cor r esponds t o an AR(4)
model, t he r eflect ion coefficient s ar e zer o for m > 4, and t he pfes r emain
const ant for m 4.
Now let us r epeat t his example using t hir d-or der cumulant s:
1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
0.0764 0.0700 0.0399 0.0300 0.3000 0.3000
0 0.0834 0.0581 0.1000 0.1000 0.1000
0 0 0.3613 0.3900 0.3900 0.3900
0 0 0 0.7200 0.7200 0.7200
0 0 0 0 0.0000 0.0000
0 0 0 0 0 0.0000
rvec=cumtrue (ma, ar, 3,
nlags);
% third-order cumulants
rvec=rvec(:,nlags+1); % the C(m,0) slice
c=rvec(nlags+1:2*nlags); % positive lags
r=rvec(nlags+1:1:1); % negative lags
[a4, c4, p4, gf4, gb4]=trench(c,r);
1 Tu to r i a l
1-52
The for war d AR mat r ix, a4, should be
The backwar d AR coefficient mat r ix, c4, should be
and t he prediction-error variances, p4, should be
[0.8892, 0.8810, 0.8417, 1.0000, 1.0000]
The for war d r eflect ion coefficient s, gf4, ar e,
[0.2278, 0.1163, 0.2757, 0.7200, 0.0000]
and t he backwar d r eflect ion coefficient s, gb4, ar e
[0.1956, 0.0786, 0.1620, 0.2612, 0.0045]
Since t he Toeplit z mat r ix is not symmet r ic, gf4gb4, and a4flipud(c4). Not e
also t hat t he pr edict ion er r or var iances may incr ease wit h t he pr edict ion or der .
(One can find t he AR par amet er s by minimizing t he var iance of t he
AR-compensat ed pr ocess, but not by minimizing t he skewness.) Alt hough t he
cor r elat ion and cumulant sequences ar e der ived fr om t he same AR model, t he
lower -or der AR model fit s ar e quit e differ ent .
1.0000 1.0000 1.0000 1.0000 1.0000
0.2278 0.2050 0.1834 0.3000 0.3000
0 0.1163 0.0673 0.1000 0.1000
0 0 0.2757 0.3900 0.3900
0 0 0 0.7200 0.7200
0 0 0 0 0.0000
0 0 0 0 0.0045
0 0 0 0.2612 0.2599
0 0 0.1620 0.1141 0.1137
0 0.0786 0.0454 0.0278 0.0261
0.1956 0.1776 0.1588 0.0868 0.0900
1.0000 1.0000 1.0000 1.0000 1.0000
Li n e a r Pr e d i c ti o n M o d e l s
1-53
Deterministic Formulation of FBLS
In t he det er minist ic for mulat ion of t he linear pr edict ion pr oblem, r at her t han
minimize t he var iance of t he pr edict ion er r or s, we minimize t he sum of squar ed
er r or s, t hat is,
(1-85)
(1-86)
This leads t o t he deterministic normal equation,
(1-87)
(1-88)
wher e t he det er minist ic cor r elat ion mat r ix is given by
(1-89)
(1-90)
Mat r ix is Her mit ian, but not Toeplit z, and t he r elat ionship in (1-69) is not
sat isfied (for finit e N).
P
p
b
e
p
b
n ( )
2
n p 1 + =
N

=
P

b
e
p
b
n ( )
2
n p 1 + =
N

=
a
p
P
p

0
=
c
p
0
P
p
b
=
x n ( )x
H
n ( )
n p 1 + =
N

=
x n ( ) x n ( ) x n 1 ( ) x n p ( ) , , , [ ] =
T
1 Tu to r i a l
1-54
In t he for war d-backwar d least squar es (FBLS) pr oblem, we
minimize assuming . This leads t o t he
det er minist ic nor mal equat ions,
(1-91)
wher e is given by
(1-92)
wher e
(1-93)
If is nonsingular , we can solve (1-91) dir ect ly; if it is singular , t hen, t he
Min-Nor m solut ion is usually used.
The solut ion a
p
may be int er pr et ed as pr oviding an AR(p) model for t he dat a;
hence, we can comput e t he spect r um of t he pr ocess x(n) in t er ms of t he
est imat ed AR par amet er s. Addit ional det ails may be found in [34, 35].
Adaptive Linear Prediction
The Bur g algor it hm pr ovides a r ecur sive solut ion t o minimizing t he
mean-squar e er r or over a t ime int er val of lengt h n, and is appr opr iat e when t he
dat a may be consider ed t o be st at ionar y over t hat t ime int er val. Oft en, t he dat a
ar e nonst at ionar y; in t his case, we can est imat e power spect r a over a sliding
window of fixed lengt h; however , as we have seen befor e, t he r esult ing est imat e
will have poor r esolut ion and high var iance. An alt er nat ive is t o fit a
par amet r ic model, and t o allow t he model par amet er s t o var y wit h t ime. For
example, we may fit a t ime-var ying AR model, t hat is, we per for m t ime-var ying
linear pr edict ion.
The det er minist ic nor mal equat ions at t ime n ar e given by (see (1-87) and
(1-88))
P
p
P
p

b
+ = c
p
k ( ) a
p
*
p k ( ) =

a
p
P
p
0
=

x n ( )x
H
n ( )
n p 1 + =
N

x n ( )x
H
n ( ) + =
x n ( ) x n p ( ) x n ( ) , , [ ] =
H

Li n e a r Pr e d i c ti o n M o d e l s
1-55
(1-94)
(1-95)
wher e t he det er minist ic cor r elat ion mat r ix (n) is given by,
(1-96)
wher e , (0 < < 1), is t he for get t ing fact or , which is int r oduced so t hat t he
algor it hm will be sensit ive t o t he nonst at ionar y envir onment .
We could solve (1-94) or (1-95) at each n, but t he comput at ional load is ver y
high.
Let t he or der p be fixed, and let a
p
(n) = [1, w(n)]
T
. Then, a t ime-r ecur sive
solut ion is given by,
(1-97)
(1-98)
(1-99)
(1-100)
wit h init ial values, w(0) = 0 and P(0) =
-1
I, wher e is a small posit ive const ant .
n ( )a
p
n ( )
P
p

n ( )
0
=
n ( )c
p
n ( )
0
P
p
b
n ( )
=
n ( ) x k ( )x
H
k ( )
k p 1 + =
n

n k
=
k n ( )

1
P n 1 ( )x n ( )
1
1
x
H
n ( )P n 1 ( )x n ( ) +
----------------------------------------------------------------------- =
n ( ) x n 1 + ( ) w
H
n 1 ( )x n ( ) =
w n ( ) w n 1 ( ) k n ( )
*
n ( ) + =
P n ( )
1
P n 1 ( )
1
k n ( )x
H
n ( )P n 1 ( ) =
1 Tu to r i a l
1-56
k(n) is t he gain vector, (n) is t he a priori prediction error, and P(n) =
-1
(n).
This algor it hm is known as t he Recur sive Least Squar es Algor it hm (RLS), and
is a special case of t he Recur sive Inst r ument al Var iable Algor it hm (RIV),
which is discussed next .
RIV Algorithm: Tr a nsver sa l For m
Consider t he noisy AR pr ocess,
wher e u(n) is assumed t o be zer o mean i.i.d., and independent of addit ive noise
w(n), which may be color ed. The nor mal equat ions, wit h m = 1, . . ., p,
(1-101)
wher e z(n) = y(n), lead t o inconsist ent est imat es because of t he noise w(n). If
w(n) can be modeled as MA noise, t hat is,
wher e g(n) is i.i.d., t hen, consist ent est imat es can be obt ained by using z(n) =
y(n d), wher e d > p + q
w
. Pr ocess z(n) is called an instrumental variable (IV)
if, (1) it is uncor r elat ed wit h t he addit ive noise w(n), t hat is, E{w(n)z(n + m)} =
0, m > 0; and (2) wit h m = 1, . . ., M p, we get a full r ank set of linear equat ions
fr om (1-01). The delayed pr ocess z(n) = y(n d) is an IV when w(n) is MA(q
w
)
noise and d > p + q
w
. Appr opr iat e choices of z(n) lead t o est imat es based on
higher -or der cumulant s; for example, z(n) = y(n)y(n ), leads t o est imat es
based on t he 1-D slice C
3y
(m,) of t he t hir d-or der cumulant ; ot her examples ar e
discussed in [64]. For a discussion of optimal IVs, see [58]. When z(n) y(n),
t he r esult ing mat r ix is no longer Her mit ian symmet r ic. A var iat ion of RLS t hat
is appr opr iat e for t his case is called t he Recursive Instrumental Variable (RIV)
algor it hm, and is implement ed in rivtr. The algor it hm is given by:
y n ( ) x n ( ) w n ( ); + = x n ( ) a k ( )u n k ( )
k 1 =
p

=
a k ( )E y n ( )z n m k + ( ) { }
k 0 =
p

0 m 0 > , =
w n ( ) h
w
n k ( )g n ( )
k 0 =
q
w

= ,
Li n e a r Pr e d i c ti o n M o d e l s
1-57
Comput e t he weight vect or w(i) r ecur sively, as
(1-102)
(1-103)
(1-104)
(1-105)
wher e t he desir ed signal d(n) = y(n + 1). The init ial values ar e w(0) = 0 and
P(0) =
-1
I, wher e is a small posit ive const ant . Her e y(n) is t he obser ved
pr ocess, and z(n) is t he IV; when z(n) = y(n), t he RIV r educes t o RLS.
Ex a mples
load riv
ar(:,1)=rivtr(y,2);
ar(:,2)=rivtr(y,3);
ar(:,3)=rivtr(y,4);
ar(:,4)=rivtr(zw,4);
ar(:,5)=rivtr(zc,4);
Now, let us look at t he five st eady-st at e AR par amet er est imat es.
disp(ar)
The dat a used in t his example cor r espond t o an AR(2) model, wit h AR
par amet er vect or [1, 1.5, 0.8]. The vect or y cont ains t he noiseless signal.
Addit ive whit e Gaussian noise was added t o y t o obt ain t he noisy signal zw
wit h a SNR of 10 dB. Color ed Gaussian noise, gener at ed by passing a whit e
Gaussian sequence t hr ough t he AR filt er [1,0,0.49], was added t o y t o obt ain zc,
also at a SNR of 10 dB.
1.000 1.000 1.000 1.000 1.000
1.4789 1.5075 1.4931 1.5354 1.4686
0.7600 0.7910 0.7867 0.8248 0.7684
k n ( )

1
P n 1 ( )z n ( )
1
1
y
H
n ( )P n 1 ( )z n ( ) +
----------------------------------------------------------------------- =
n ( ) d n ( ) w
H
n 1 ( )y n ( ) =
w n ( ) w n 1 ( ) k n ( )
*
n ( ) + =
P n ( )
1
P n 1 ( )
1
k n ( )y
H
n ( )P n 1 ( ) =
1 Tu to r i a l
1-58
RIV Algorithm: Double- La ttice For m
An or der - and t ime-r ecur sive solut ion of (1-94) and (1-95)leads t o t he double
lat t ice algor it hm, and is given below.
St ar t ing wit h n = 1, evaluat e (1-106) t hr ough (1-117) for m = 1, 2, . . ., M, wher e
M is t he final desir ed or der ; t hen r epeat wit h n = 2 and so on:
(1-106)
(1-107)
(1-108)
(1-109)
(1-110)
(1-111)
(1-112)
(1-113)
(1-114)

m 1

n ( )
m 1

n 1 ( )

m 1
*
n ( )b

m 1
n 1 ( )

m 1
*
n 1 ( )
--------------------------------------------------------- + =

m 1
b
n ( )
m 1
b
n 1 ( )
b
m 1
*
n 1 ( )

m 1
n ( )

m 1
*
n 1 ( )
--------------------------------------------------------- + =

m ,
n ( )

m 1

n ( )
B
m 1
n 1 ( )
---------------------------------- =

b m ,
n ( )

m 1
b
n ( )
F
m 1
n ( )
------------------------ =
n ( )
F
m 1
*
n ( )
B
m 1
*
n 1 ( )
---------------------------------- =

m
n ( )
m 1
n ( )
m ,
*
n ( )b
m 1
n 1 ( ) + =
b
m
n ( ) b
m 1
n 1 ( )
b m ,
*
n ( )
m 1
n ( ) + =

m
n ( )

m 1
n ( ) n ( )
b m ,
n ( )b

m 1
n 1 ( ) + =
b

m
n ( ) b

m 1
n 1 ( )
1
n ( )
m ,
n ( )

m 1
n ( ) + =
Li n e a r Pr e d i c ti o n M o d e l s
1-59
(1-115)
(1-116)
(1-117)
Not e t hat (1-106) t hr ough (1-117) ar e in t he pr oper or der r equir ed for
successful evaluat ion.
The init ial values for t he algor it hm ar e given by
(1-118)
(1-119)
(1-120)
(1-121)
(1-122)
(1-123)
The following discussion is adapt ed fr om [64], which also gives a der ivat ion of
t he algor it hm.
Equat ions (1-111)and(1-112) wit h t he init ial condit ion in (1-120)define a lat t ice
st r uct ur e excit ed by t he obser ved pr ocess y(n). Equat ions (1-113) and (1-114)
wit h t he init ial condit ion in (1-121) define a lat t ice st r uct ur e excit ed by t he
F
m
n ( ) F
m 1
n ( )

m 1

n ( )
m 1
b
n ( )
B
m 1
n 1 ( )
------------------------------------------------- =
B
m
n ( ) B
m 1
n 1 ( )

m 1

n ( )
m 1
b
n ( )
F
m 1
n ( )
------------------------------------------------- =

m
n 1 ( )
m 1
n 1 ( )
b

m 1
*
n 1 ( )b
m 1
n 1 ( )
B
m 1
*
n 1 ( )
------------------------------------------------------------------- =

m 1

0 ( )
m 1
b
0 ( ) 0 = =
F
m 1
0 ( ) B
m 1
0 ( ) 1 = =

0
n ( ) b
0
n ( ) y n ( ) = =

0
n ( ) b

0
n ( ) z n ( ) = =
F
0
n ( ) B
0
n ( ) F
0
n 1 ( ) z n ( )y
*
n ( ) + = =

0
n ( ) 1 =
1 Tu to r i a l
1-60
associat ed pr ocess z(n). The t wo lat t ices ar e coupled t hr ough t he t ime-updat e
equat ions for t he s, (1-106) and (1-107), and t he or der -updat e equat ion for t he
conver sion fact or , (1-117). When y(n) = z(n), t he t wo lat t ices collapse int o one.
In t he above,
m
(n) and b
m
(n) ar e t he mt h or der pr edict ion er r or s for t he
obser ved pr ocess y(n); and ar e t he mt h or der pr edict ion er r or s for
t he associat ed pr ocess z(n);
,m
(n) and
b,m
(n) ar e t he r eflect ion coefficient s for
t he for war d/backwar d pr edict ion of y(n), and may be int er pr et ed as nor malized
cr oss-cor r elat ions bet ween t he for war d pr edict ion er r or in one lat t ice and t he
backwar d pr edict ion er r or in t he ot her ; (n) is t he scale fact or for conver t ing
t he r eflect ion coefficient s associat ed wit h y(n) t o t hose associat ed wit h z(n);
F
m
(n) is t he cor r elat ion bet ween t he mt h or der for war d pr edict ion er r or s,
m
(n)
and ; B
m
(n) is t he cor r elat ion bet ween t he mt h or der backwar d
pr edict ion er r or s, b
m
(n) and ;
m
(n) is t he scale fact or for conver t ing t he
a priori pr edict ion er r or s t o t heir a posteriori values; it s absolut e value is
bounded by unit y; is t he cr oss-cor r elat ion bet ween t he a posteriori
backwar d pr edict ion er r or , b
m-1
(n 1), and t he a priori t ilded for war d
pr edict ion er r or , ; and is t he cr oss-cor r elat ion bet ween

m-1
(n) and . Time-updat e equat ions for F
m
(n) and B
m
(n) ar e
given by (1-115) and (1-116).
This algor it hm is implement ed in r out ine rivdl.
Ex a mples
load riv
ar(:,1)=rivdl(y,2);
ar(:,2)=rivdl(y,3);
ar(:,3)=rivdl(y,4);
ar(:,4)=rivdl(zw,4);
ar(:,5)=rivdl(zc,4);
Now let s see t he five AR par amet er est imat es:
disp(ar)
1.000 1.000 1.000 1.000 1.000
1.4814 1.5106 1.4941 1.5358 1.4692
0.7623 0.7941 0.7876 0.8252 0.7690

m
n ( ) b

m
n ( )

m
n ( )
b

m
n ( )

m 1
b
n ( )

m 1
n ( )
m 1

n ( )

m 1
n 1 ( )
Li n e a r Pr e d i c ti o n M o d e l s
1-61
The dat a used in t his example cor r espond t o an AR(2) model, wit h AR
par amet er vect or [1, 1.5, 0.8]. The vect or y cont ains t he noiseless signal.
Addit ive whit e Gaussian noise was added t o y t o obt ain t he noisy signal zw wit h
a SNR of 10 dB. Color ed Gaussian noise, gener at ed by passing a whit e
Gaussian sequence t hr ough t he AR filt er [1, 0, 0.49], was added t o y t o obt ain
zc, also at a SNR of 10 dB.
Summary
We have discussed sever al linear pr edict ion pr oblems. The Tr ench algor it hm is
implement ed in trench: it comput es t he backwar d and for war d AR pr edict ion
filt er s associat ed wit h a given cumulant slice. Rout ines rivtr and rivdl
implement t he t r ansver sal and double lat t ice ver sion of t he Recur sive
Inst r ument al Var iable (RIV) met hod: t hese algor it hms may be based upon t he
cor r elat ion or a slice of t he t hir d- or four t h-or der cumulant , depending upon t he
Instrumental Variable (IV) used; ivcal can be used t o comput e t he IV.
1 Tu to r i a l
1-62
Harmonic Processes and DOA
In t he har monic r et r ieval pr oblem, t he obser ved dat a ar e of t he for m
(1-124)
wher e w(n) is addit ive noise;
k
s ar e t he amplit udes,
k
s ar e t he fr equencies,
and
k
s ar e t he phases. Addit ional assumpt ions ar e oft en made r egar ding t he
phases; t he t ypical assumpt ion is t hat t he phases ar e independent r andom
var iables unifor mly dist r ibut ed over [0, 2]. Such an assumpt ion implies t hat
mult iple r ealizat ions ar e available, t hat is,
(1-125)
wher e m denot es t he r ealizat ion number . If only a single r ealizat ion is
available, t hen t he phases ar e nonr andom const ant s. Since 2cos() = e
j
+e
-j
,
we not e t hat a signal consist ing of p r eal har monics can be handled by t he
model in (1-125) wit h 2p fr equencies, t
k
, k = 1, . . ., p.
The pr oblem is t o det er mine p and t hen t o est imat e t he fr equencies, amplit udes
and phases of each har monic. Int uit ively, we expect t he power spect r um of x(n)
t o consist of peaks at =
k
, per haps masked by t he spect r um of t he noise. The
classical power spect r um est imat or s (Blackman-Tukey and Welch) can be used
only if t he fr equencies ar e well-separ at ed, t hat is, min|
m

n
| > 1/L, wher e
L is t he lengt h of t he est imat ed aut ocor r elat ion sequence in t he
Blackman-Tukey met hod, or t he block size in t he Welch met hod; hence,
par amet r ic models ar e useful for shor t dat a lengt hs.
The model in (1-125) is similar t o t he model for t he obser ved dat a in t he ar r ay
pr ocessing/dir ect ion of ar r ival (DOA) pr oblem. This pr oblem ar ises in var ious
applicat ions, such as r adar , sonar , geophysics, and analysis of EEG and ECG
signals. An ar r ay of N independent sensor s pr ovides samples of t he dat a at
differ ent point s in space. Dat a ar e collect ed simult aneously fr om t he set of
sensor s; t he set collect ed at a single inst ant of t ime is called a snapshot, and is
a vect or of N obser vat ions. The signal r ecor ded at each sensor consist s of sensor
y n ( )
k
j 2n
k
j
k
+ ( ) exp
k 1 =
p

w n ( ) + x n ( ) = w n ( ) + =
y
m
n ( )
k
j 2n
k
j
k m ,
+ ( ) exp
k 1 =
p

w
m
n ( ) + x
m
n ( ) = w
m
n ( ) + =
H a r m o n i c Pr o c e sse s a n d D O A
1-63
noise as well as signals fr om p sour ces, assumed t o be uncor r elat ed wit h one
anot her . The signal r ecor ded at t he nt h sensor and mt h snapshot is
wher e s
k
(m) is t he signal emit t ed by t he kt h sour ce, and
k,n
is t he signal delay
(wit h r espect t o some ar bit r ar y r efer ence point ) at t he nt h sensor . Typically,
t he sensor s ar e assumed t o be isotropic, and t he sour ce signals ar e assumed t o
be plane waves or iginat ing fr om t he far -field of t he ar r ay. The signals ar e
assumed t o be nar r ow-band, and as such t hey ar e char act er ized by a single
fr equency,
o
or a single wavelengt h = c/
o
, wher e c is t he speed of signal
pr opagat ion. The r eceived signal can now be wr it t en as,
(1-126)
wher e
k
(m) is t he amplit ude of t he kt h sour ce signal at snapshot m; and (k,n)
is t he phase delay of t he kt h sour ce signal at t he nt h sensor , and depends upon
t he ar r ay geomet r y.
In a cir cular ar r ay, t he sensor s ar e assumed t o be equally dist r ibut ed in angle
on t he cir cumfer ence of a cir cle wit h r adius R. Let t he bear ings,
k
, be
measur ed wit h r espect t o t he r adius vect or passing t hr ough a sensor . The
phase delay is t hen given by [23]
(1-127)
For a linear ar r ay, t he bear ing of t he plane wave is defined as t he angle
bet ween t he plane wave fr ont and t he nor mal t o t he ar r ay. The phase delay is
given by
(1-128)
y
m
n ( ) A
k
s
k
m
k n ,
( )
k 1 =
p

w
m
n ( ), + =
y
m
n ( )
k
m ( ) j k n , ( ) ( ) exp
k 1 =
p

w
m
n ( ) + = =
m 1 M , , n 1 N , , , = , =
k n , ( )
2R

-----------
2n
N
-----------
k

,
_
cos
2n
N
---------- -
,
_
cos =
k n , ( )
2d
n

--------------
k
( ) sin =
1 Tu to r i a l
1-64
wher e
k
is t he bear ing of t he kt h sour ce, and d
n
is t he locat ion of t he nt h sour ce
wit h r efer ence t o some ar bit r ar y or igin. For a unifor mly spaced linear ar r ay,
wit h spacing d, we can set d
n
= nd. The obser ved signals ar e t hus given by,
(1-129)
Compar ing (1-129) wit h (1-125), we see t hat t he t er m dsin(
k
)/ plays t he r ole
of fr equency, and
k
(m) plays t he r ole of t he complex amplit ude
k
exp(j
k,m
).
Time samples, indexed by n in (1-125), cor r espond t o spat ial samples, or sensor
number s in (1-129). The r ealizat ion number m in (1-125) is int er pr et ed as t he
snapshot number in (1-129). In (1-124), we assume 0 <
k
< 1 in or der t o ensur e
t hat t her e is no aliasing. Similar ly, in (1-129), we assume t hat 2d t o ensur e
t hat t her e is no aliasing; a common assumpt ion is = 2d (half-wavelengt h
spacing).
Thus, a solut ion t o t he har monic r et r ieval pr oblem in (1-125) also applies t o t he
DOA pr oblem in (1-129). In t he for mer , we est imat e power spect r a, as funct ions
of fr equency; in t he lat t er , we est imat e angular spect r a, as funct ions of angle, .
The following t able st at es t he dualit y bet ween t he har monic r et r ieval and t he
DOA pr oblems:
The aut ocor r elat ion mat r ix
xx
wit h (k,l) ent r y R
xx
(l k) = E{x*(n)x(n + k l)},
which appear s in t he har monic r et r ieval pr oblem, will be r eplaced by t he
Harmonic Retrieval DOA
Realizat ion Snapshot
Time sample Sensor sample
Fr equency Angular fr equency d sin()/
Random phases Random complex amplit udes
0 < < 1 d /2
Tempor al cor r elat ion mat r ix Spat ial cor r elat ion mat r ix
Per iodogr am Beamfor mer
y
m
n ( )
k
m ( ) j 2n
d
k
( ) sin

------------------------
,
_
exp
k 1 =
p

w
m
n ( ) + = m 1 M , , =
H a r m o n i c Pr o c e sse s a n d D O A
1-65
spat ial cor r elat ion mat r ix, = E{x(n)x(n)
H
}, wher e x(n) is t he nt h snapshot , in
t he DOA pr oblem. The FT of t he spat ial (t empor al) cor r elat ion sequence is
called t he beamformer (per iodogr am).
The aut ocor r elat ion sequence of t he signal in (1-125) is given by,
(1-130)
which follows because t he har monics ar e uncor r elat ed (phases ar e mut ually
independent ). Not e t hat t he aut ocor r elat ion sequence is not absolut ely
summable; however , t he FT of R
yy
() exist s, pr ovided we ar e willing t o use t he
Dir ac delt a funct ions defined by g()
D
() := g(0) [5, p.17].
In pr inciple, one can est imat e t he fr equencies by picking peaks in t he
est imat ed power spect r um; t he var iance of t his fr equency est imat or is of or der
1/N
3
[22], wher e N is t he lengt h of t he t ime ser ies. The power spect r um is
limit ed by it s poor fr equency r esolut ion, t hat is, fr equency separ at ion must be
lar ger t han 1/N. In pr act ice, t his sever ely limit s t he applicabilit y of t he power
spect r um as a dir ect est imat or of t he fr equency.
Synt het ics for t he t wo pr oblems can be gener at ed via harmgen and doagen.
Resolution and Variance
If x(n) consist s of t wo complex har monics, at fr equencies
1
and
2
, we expect
t o see t wo peaks in t he power spect r um. The r esolut ion of a power spect r al
densit y est imat or is t he smallest value of |
1

2
| t hat leads t o t wo discer nible
peaks. The r esolut ion of t he classical power spect r um est imat or s (e.g., Welch
and Blackman-Tukey) is of or der 1/M, wher e M is t he effect ive window lengt h.
In cont r ast , t he r esolut ion of t he per iodogr am is 1/N, and since M N, t he
per iodogr am exhibit s higher r esolut ion t han t he Blackman-Tukey and Welch
est imat or s. The choice of t he window funct ion dict at es t he r esolut ion-var iance
t r adeoff: if W() has a br oad main lobe, t he power spect r um est imat e will be
smoot her , t he var iance of t he est imat e will be smaller , and t he r esolut ion will
be lower .
It should be not ed t hat zer o-padding t he dat a (or t he cor r elat ion sequence) does
not impr ove t he r esolut ion, it only makes t he spect r al est imat e denser (t his
may lead t o impr oved accur acy in est imat ing t he locat ions of well-separ at ed
peaks).
R
y y
( )
k
2
k 1 =
p

j 2
k
exp R
w
( ) + =
1 Tu to r i a l
1-66
AR and ARMA Models
The basic idea, in t he par amet r ic est imat or s, is t hat t he noiseless har monic
pr ocess in (1-124) obeys a self-dr iving AR(p) model, t hat is, [28],
(1-131)
wher e a(0) = 1, and t he r oot s of t he A(z) polynomial ar e given by ,
k = 1, . . ., p. Hence, we can use any st andar d algor it hm t o est imat e t he AR
par amet er s; t he fr equencies can be det er mined fr om t he r oot s of t he est imat ed
AR polynomial. The det er minat ion of p is now equivalent t o t he AR model or der
det er minat ion pr oblem.
In t he pr esence of addit ive noise, pr ocess x(n) in (1-124) sat isfies t he special
ARMA(p,p) model,
(1-132)
wher e w(n) is addit ive noise, not t he dr iving noise of an ARMA model. Now, we
can use any of t he standard ARMA par amet er est imat ion t echniques t o
est imat e t he a(k)s.
Mult iplying (1-132) by y*(n k), and t aking expect at ions, we obt ain,
(1-133)
wher e we have assumed t hat t he addit ive noise is whit e; hence, wit h k > p, we
obt ain a set of equat ions fr om which we can est imat e t he a(k)s; t hese equat ions
ar e somet imes called t he ext ended nor mal equat ions.
The analysis ext ends t o t he r eal har monic case wher e we obt ain an AR(2p) or
ARMA(2p,2p) model, wit h r oot s at , k = 1, . . ., p. Since t he r oot s ar e
all on | z| = 1, it t ur ns out t hat a(k) = a(2p k), k = 0, . . ., 2p.
a k ( )x n k ( )
k 0 =
p

0, =
z e
j 2
k
=
a m ( )y n m ( )
m 0 =
p

a m ( )w n m ( )
m 0 =
p

=
a m ( )R
y y
k m ( )
m 0 =
p

w
2
a k ( ) =
z e
j t 2
k
=
H a r m o n i c Pr o c e sse s a n d D O A
1-67
Pisarenkos Method
The M M aut ocor r elat ion mat r ix of t he pr ocess y(n) in (1-124) may be wr it t en
as,
(1-134)
wher e S is a Vander monde mat r ix wit h (m,n) element exp(j2(m 1)
n
),
m = 1, . . ., p; and D is t he diagonal mat r ix of t he sinusoidal power s,
D = diag(P
1
, . . ., P
p
). Since t he fr equencies ar e assumed t o be dist inct , mat r ix
S has full r ank p (M p). It follows immediat ely t hat t he p lar gest eigenvalues
of t he mat r ix R
yy
ar e given by, , k = 1, . . ., p; t he r emaining M p
eigenvalues ar e all equal t o . Wit h M = p + 1, it follows fr om (1-133) t hat [48]
(1-135)
The vect or a is t he eigenvect or cor r esponding t o t he minimum eigenvalue of t he
(p + 1) (p + 1) cor r elat ion mat r ix
yy
. Once a has been est imat ed, t he
fr equencies can be est imat ed as t he r oot s of t he A(z) polynomial. The minimum
eigenvalue solut ion t o (1-135) is known as Pisar enkos met hod, and is
implement ed in r out ines harmest and doa. In or der t o ensur e
posit ive-definit eness of mat r ix
yy
, it is essent ial t o use biased sample
est imat es of t he aut ocor r elat ion. For a signal consist ing of p r eal har monics,
t he or der should be t aken as 2p.
The Pisar enko algor it hm separ at es t he eigenvect or s and eigenvalues of t he
mat r ix
yy
int o t wo classes: t hose associat ed wit h t he signal and t hose
associat ed wit h t he noise. Ext ensions of t he idea have r esult ed in t he
high-r esolut ion eigen-subspace met hods, such as t he Min-Norm and MUS IC
algor it hms.
R
y y
R
x x
=
w
2
I + S DS
H

w
2
I + =
P
k

w
2
+

w
2

y y
a
w
2
a . =
1 Tu to r i a l
1-68
Multiple Signal Classification (MUSIC)
Let M > p. The aut ocor r elat ion mat r ix
yy
has a linear ly independent
(or t hogonal) set of eigenvect or s. Denot e t he eigenvalues and eigenvect or s by
k

and v
k
. Define t he M element signal vect or ,
(1-136)
The eigenvect or s cor r esponding t o t he p lar gest eigenvalues ar e said t o
const it ut e t he signal subspace; t he r emaining M p eigenvect or s const it ut e t he
noise subspace. The signals, t hat is, t he har monics in (1-124), lie in t he signal
subspace and ar e or t hogonal t o t he noise subspace, t hat is,
hence, wit h
(1-137)
one can sear ch for t he set of fr equencies such t hat e
H
()v
k
= 0, k = p + 1, . . ., M.
When sample est imat es of t he aut ocor r elat ion ar e used, t hese r elat ionships
will hold only appr oximat ely. In MUSIC (Mult iple Signal Classificat ion) one
det er mines t he fr equencies of t he har monics by looking for peaks in t he
spectrum defined by
(1-138)
wher e w(k) = 1. Usually, t he cor r elat ion mat r ix used in t he FBLS met hod is
used. When w(k) = 1/
k
, we get t he so-called Eigenvect or solut ion, and when
w(k) = (k M), we get t he Pisar enko solut ion. These algor it hms ar e
implement ed in r out ines harmest and doa.
e
k
e
j 2
k
e
j 2
k
e
j 2M
k
, , [ ]
T
=
e
k
v
H
i
0, = i p 1 + M; , , = k 1 p; , , =
e ( ) e
j 2
e
j 2
e
j 2M
, , [ ]
T
=
P
yy
( ) w k ( ) e
H
( )v
k
2
k p 1 + =
M

=
1
H a r m o n i c Pr o c e sse s a n d D O A
1-69
Minimum-Norm Method
The minimum-norm (Min-Nor m) met hod is also an eigen-space met hod.
Mot ivat ed by t he nor mal equat ions for t he har monic signal model, one seeks
t he AR vect or , a, which is or t hogonal t o t he signal subspace (r ecall t he
discussion of t he Pisar enko met hod). Let us decompose t he signal subspace, as
(1-139)
The condit ion , can be r ewr it t en as,
(1-140)
wher e w = [a(1), . . ., a(p)]
T
. Equat ion (1-140) r epr esent s p equat ions in M 1
unknowns. Hence, if p < M 1, t he set of equat ions is under det er mined, and
t he solut ion is not unique. A unique solut ion is obt ained by choosing t he
Min-Nor m solut ion, t hat is,
(1-141)
This solut ion, called t he minimum-norm solut ion, was developed by
Kumar esan and Tuft s and is implement ed in r out ines harmest and doa.
Pisar enkos met hod uses a single noise subspace eigenvect or and, hence, is not
as r obust as t he gener al MUSIC est imat or . The Min-Nor m met hod has been
r epor t ed t o have a smaller bias t han t he MUSIC algor it hm for est imat ing t he
fr equencies. An over view of eigen-met hods is given in [26].
A modificat ion of t he FBLS algor it hm has been r epor t ed by Kumar esan and
Tuft s [23], wher e t he cor r elat ion mat r ix is r eplaced by t hr owing out t he
noise subspace eigenvect or s, t hat is, if
k
and v
k
denot e t he eigenvalues and
eigenvect or s of , t hen,
(1-142)
Q
s
v
1
v
p
, , [ ]
x
s
T
X
s
= =
Q
s
T
a 0 =
X
s
T
w x
s
=
w 1 x
s
H
x
s
[ ]
1
X
s
*
x
s
=

k
v
k
v
k
H
k 1 =
p

=
1 Tu to r i a l
1-70
This pr ocedur e of r eplacing by is somet imes called t he low-r ank SVD
appr oximat ion. The algor it hm t hen pr oceeds as in FBLS. Kumar esan and
Tuft s [29] have est ablished t hat in t he noiseless case, under t he assumpt ion,
p M N p/2, p of t he r oot s of A(z) lie on t he unit cir cle, and cor r espond t o
t he t r ue fr equencies; t he r emaining (M p) r oot s ar e unifor mly dist r ibut ed in
angle inside t he unit cir cle; t hese r oot s ar e called t he extraneous zer os.
High-r esolut ion est imat es ar e obt ained by choosing lar ge values for t he
pr edict or or der , t hat is, M 3N/4.
ESPRIT
Est imat ion of Signal Par amet er s via Rot at ional Invar iance Techniques
(ESPRIT) is an eigen-space met hod developed by Kung, Rao et al [30], and by
Roy and Kailat h [57]; however , t he basic idea behind t his algor it hm is
consider ably differ ent fr om t hat in MUSIC and Min-Nor m met hods.
In ESPRIT, t he basic idea is t o decompose t he sensor ar r ay int o t wo (possibly
over lapping) subar r ays, and t hen t o use t he cr oss-cor r elat ion bet ween t he t wo
subar r ays t o est imat e t he sour ce bear ings. The discussion her e is in t er ms of
t he DOA pr oblem, but as we have seen ear lier , t he t r eat ment is applicable t o
t he har monic r et r ieval pr oblem as well.
Assume t hat we have t wo ar r ays, such t hat element s of one ar r ay ar e at a
const ant displacement wit h r espect t o cor r esponding element s of t he ot her
ar r ay. For example, if we have an unifor mly spaced linear ar r ay of N element s,
we can par t it ion t his ar r ay int o t wo subar r ays, each consist ing of L element s,
wit h a displacement of = N L bet ween t he t wo subar r ays, t hat is, wit h
sensor s 1, . . ., N in t he fir st subar r ay, and sensor s + 1, . . ., N in t he second
ar r ay.
Let y
m
(n) and z
m
(n), n = 1, . . ., L, denot e t he signals at t he element s of t he t wo
subar r ays, t hat is,
(1-143)
(1-144)

y
m
n ( )
k
m ( ) j k n , ( ) ( ) exp
k 1 =
p

w
m
n ( ) + =
z
m
n ( )
k
m ( ) j k n , ( ) ( )e
j 2d
k
( )/ sin
exp
k 1 =
p

v
m
n ( ) + =
H a r m o n i c Pr o c e sse s a n d D O A
1-71
The mt h snapshot can be wr it t en in vect or for m as,
(1-145)
(1-146)
wher e is a diagonal mat r ix wit h (k,k) ent r y, exp(j2dsin(
k
)/); mat r ix A has
(n,k) ent r y, exp(j(k,n)); a
m
= [
1
(m), . . .,
M
(m)]
T
is t he sour ce signal vect or at
t he mt h snapshot ; and, w
m
and v
m
ar e t he noise vect or s. The bear ing
infor mat ion can be r ecover ed fr om mat r ix .
The aut o- and cr oss-cor r elat ion mat r ices of y(n) and z(n) ar e given by,
(1-147)
(1-148)
wher e we have assumed t hat t he addit ive noise is whit e (spat ially uncor r elat ed
fr om sensor t o sensor ), and has var iance
2
.
Mat r ix is t he signal cor r elat ion mat r ix and is nonsingular if
t he sour ces ar e incoher ent .
The st r uct ur e of mat r ix J
L
depends upon t he common element s of t he t wo sub-
ar r ays. If t he t wo subar r ays have no common element s, t hen J
L
is t he zer o
mat r ix. In t he unifor m linear ar r ay example consider ed ear lier ,
z
m
(n) = y
m +
(n), m = 1, . . ., L , and
In any case, given t he knowledge of t he geomet r ies of t he t wo ar r ays, we can
comput e mat r ix J
L
.
y
m
Aa
m
w
m
+ =
z
m
Aa
m
v
m
+ =

yy
E y
m
y
m
H

' ;

AS A
H

2
I
L
+ = =

y z
E y
m
z
m
H

' ;

AS
H
A
H

2
J
L
+ = =
S E a
m
a
m
H

' ;

=
J
L
0
L
0

I
L
0
L
=
1 Tu to r i a l
1-72
We can est imat e
2
as t he smallest eigenvalue of mat r ix
yy
. We can now
eliminat e t he noise cont r ibut ion, and cr eat e t he signal cor r elat ion mat r ices,
(1-149)
(1-150)
Given t wo n n mat r ices, A and B, t he gener alized eigenvalues of (A,B), ar e
defined by [21]
If B is nonsingular , t her e ar e exact ly n gener alized eigenvalues, and t hese ar e
ident ical wit h t he eigenvalues of B
-1
A. The singular B case is discussed in [21].
It is shown in [30, 57] t hat t he gener alized eigenvalues of (C
yy
,C
yz
) ar e given by

k
= exp(j2dsin(
k
)/), k = 1, . . ., p. Once
k
has been est imat ed, we can
r eadily obt ain t he
k
s. This algor it hm for a unifor mly spaced linear ar r ay
is implement ed in r out ine doa; t he gener alized eigenvalues ar e comput ed via
t he MATLAB built -in funct ion eig. Algor it hms based on four t h-or der
cumulant s ar e discussed next .
Criterion-Based Estimators
The FT, at fr equency
o
, can be viewed as a filt er t hat passes only t he
component s wit h fr equency
o
, while suppr essing t he r est . Wit h finit e dat a
lengt hs, however , t he spect r al est imat e suffer s fr om t he pr oblems of sidelobe
leakage (wit h det ails dict at ed by t he choice of window funct ion).
Consider t he lengt h p finit e impulse r esponse (FIR) filt er ,
(1-151)
wher e t he input x(n) is assumed t o be zer o mean. The var iance of t he filt er
out put is given by,
C
y y

yy

2
I
L
AS A
H
= =
C
y z

y z

2
I
L
AS
H
A
H
= =
A B , ( ): z :d et A z B ( ) { 0 }. = =
y n ( ) a k ( )x n k ( )
k 0 =
p

=
H a r m o n i c Pr o c e sse s a n d D O A
1-73
(1-152)
wher e a = [a(0), . . ., a(p)]
T
, and R
xx
is t he (p + 1) (p + 1) aut ocor r elat ion
mat r ix.
We would like t he out put var iance t o be a measur e of t he power spect r al
densit y of x(n) at fr equency
o
. Alt er nat ively, t he FIR filt er should pass a
sinusoid at fr equency
o
wit h unit y gain; hence, we have t he const r aint ,
(1-153)
We also want t o minimize t he cont r ibut ion due t o sinusoids at fr equencies
ot her t han
o
; t his can be achieved by minimizing t he out put var iance in
(1-152) subject t o t he const r aint in (1-153). The r esult ing est imat or is given by
(1-154)
wher e e = [1,e-
-j2
, . . ., e
-j2p
]
H
. This est imat or , which is var iously called
Capons maximum-likelihood est imat or or t he Minimum Var iance
Dist or t ionless est imat or [8, 27], is not a t r ue power spect r al densit y est imat or .
For example, if x(n) consist s of a har monic at fr equency
o
and power P t hat is
obser ved in whit e noise wit h var iance , t hen,
wher e p is t he number of aut ocor r elat ion lags used; not e t hat t he noise power
cont r ibut ion has been r educed by a fact or of p. The r esolving power of t his
est imat or is r epor t ed t o be [27]
wher e S NR is t he signal-t o-noise r at io (r at io of signal var iance t o noise
var iance), and p is t he lengt h of t he aut ocor r elat ion sequence. This est imat or
is implement ed in r out ines harmest and doa.

y
2
E y n ( )
2
{ } a
H
R
xx
a = =

y
2
a k ( ) j 2k
o
( ) exp
k 0 =
p

1. =
P
m l
( ) e
H
( )R
x x
1
e ( ) [ ] =
2

w
2
P
m l

o
( ) P
w
2
/p, + =

1
p S N R
----------------------
1 Tu to r i a l
1-74
The maximum-ent r opy spect r al est imat or of Bur g is based on t he availabilit y
of exactly known aut ocor r elat ion lags, , (in which case a
Gaussian pdf and an AR(p) model ar e obt ained for pr ocess x(n)). Rout ine
trench may be used t o est imat e t he AR model par amet er s. The r esolving power
is r epor t ed t o be
For SNR 1, t he maximum ent r opy est imat or has bet t er r esolving power t han
does Capons maximum-likelihood est imat or , which in t ur n is bet t er t han t he
Blackman-Tukey est imat or .
Cumulant-Based Estimators
In [66] it was est ablished t hat t he four t h-or der cumulant s of t he pr ocess in
(1-124) ar e given by
(1-155)
In par t icular , t he diagonal slice is given by,
(1-156)
If w(n) is Gaussian, t hen C
4w
(m,m,m) 0. If w(n) is i.i.d. non-Gaussian, t hen
C
4w
(m,l,n) is a delt a funct ion at t he or igin. Not e t hat when w(t) is Gaussian or
i.i.d. non-Gaussian, (1-156) is similar t o t he aut ocor r elat ion sequence of t he
noiseless signal. Consequent ly, all of t he analyses based on t he aut ocor r elat ion
car r y over t o t he four t h-or der cumulant . In par t icular , t he development of t he
Eigenvect or , Pisar enko, ML-Capon, AR, MUSIC, Min-Nor m, and beamfor mer
spectral est imat es based on second-or der st at ist ics, can also be based on
four t h-or der st at ist ics [44]. Similar ly we can base ESPRIT on t he four t h-or der
cr oss-cumulant mat r ix as well. The four t h-or der st at ist ics ar e most useful
when t he addit ive noise is nar r ow-band Gaussian.
Rout ine harmest implement s algor it hms for t he har monic r et r ieval pr oblem: it
est imat es t he spect r um using t he Eigenvect or , MUSIC, Pisar enko, ML, and AR
algor it hms based on t he diagonal slice of t he four t h-or der cumulant .
R
xx
m ( ) { }
m 0 =
p

1
pS N R
-----------------.
C
4 y
l m n , , ( ) e
j 2
k
l m n + ( )

k
4
C
4w
l m n , , ( ) +
k 1 =
p

=
C
4y
m m m , , ( ) e
j 2
k
m

k
4
C
4w
m m m , , ( ) +
k 1 =
p

=
H a r m o n i c Pr o c e sse s a n d D O A
1-75
Rout ine doa implement s algor it hms for t he dir ect ion of ar r ival pr oblem: it
est imat es t he angular spectrum using t he Eigenvect or , MUSIC, Pisar enko,
ML-Capon, ML, and AR algor it hms based on t he diagonal slice of t he
four t h-or der cumulant . Rout ine doa implement s t he cor r esponding algor it hms
based on second-or der st at ist ics; it also implement s ESPRIT.
Ex a mples
load harm
Pxx=harmest(zmat,12,4,unbiased,256,4);
You should see t he display in Figur e 1-12.
Figure 1-12 Singular Values of Cumulant Matrix and Estimated Spectra
(harmest)
The dat a consist of t wo unit amplit ude r eal har monics at fr equencies 0.1 Hz
and 0.2 Hz, and ar e cont aminat ed by color ed Gaussian noise, wit h a var iance
of 0.5; t he noise spect r um has a pole at 0.15 Hz, wit h a damping fact or of 0.9.
The SVD plot is indicat ive of t wo r eal har monics, and t he spect r um appear s t o
peak at t he cor r ect fr equencies, 0.1 and 0.2 Hz. The four t h-or der cumulant s do
not see t he Gaussian pr ocess at 0.15 Hz. Not e t hat t he power spect r a ar e
displayed in dB scale, t hat is, 10log
10
(Pxx)
Pxx = harmest(zmat,12,6,'unbiased',256,2);
0 5 10 15
0
2
4
sval: cum4
0.5 0 0.5
40
20
0
pxx
0.5 0 0.5
50
0
e
i
g
0.5 0 0.5
40
20
0
m
u
s
i
c
0.5 0 0.5
100
50
0
p
i
s
a
r
0.5 0 0.5
40
20
0
m
l
0.5 0 0.5
100
50
0
a
r
0.5 0 0.5
40
20
0
m
i
n

n
o
r
m
1 Tu to r i a l
1-76
You should see t he display in Figur e 1-13. The SVD plot is indicat ive of t hr ee
r eal har monics, and t he spect r um appear s t o peak at t he cor r ect fr equencies,
0.1 and 0.2 Hz, as well as at 0.15 Hz, which is due t o t he Gaussian noise.
Figure 1-13 Singular Values of Correlation Matrix, and Estimated Spectra
(harmest)
0 5 10 15
0
5
10
sval: cum2
0.5 0 0.5
40
20
0
pxx
0.5 0 0.5
50
0
e
i
g
0.5 0 0.5
40
20
0
m
u
s
i
c
0.5 0 0.5
100
50
0
p
i
s
a
r
0.5 0 0.5
40
20
0
m
l
0.5 0 0.5
100
50
0
a
r
0.5 0 0.5
50
0
m
i
n

n
o
r
m
H a r m o n i c Pr o c e sse s a n d D O A
1-77
Ex a mples
load doa1
[spec,theta]=doa(ymat,0.5,1,3,2,1);
You should see t he display in Figur e 1-14; not e t hat t he angular spect r a ar e
plot t ed in dB scale, t hat is, 10log
10
(s pe c).
The singular value plot indicat es t he possible pr esence of t hr ee sour ces. The
obser ved signal consist s of signals fr om t wo sour ces at 15 and 25 degr ees,
and is cont aminat ed wit h spat ially cor r elat ed noise t hat act s as a vir t ual
sour ce at a bear ing of 30 degr ees. The bear ings est imat ed by ESPRIT should
be 15.4634, 25.8851 and 29.9855. As expect ed, t he Eigenvect or , ML, AR,
MUSIC and Min-Nor m met hods r esolve t he t wo sour ces and t he vir t ual sour ce
due t o t he noise, wher eas t he beamfor mer does not . The MUSIC and Min-Nor m
est imat es ar e vir t ually ident ical in t his example. The ver t ical lines and cir cles
denot e t he t r ue bear ings; t he cr oss denot es t he vir t ual bear ing of t he spat ially
cor r elat ed noise.
Figure 1-14 Singular Values of Spatial Correlation Matrix, and Estimated
Angular Spectra (doa)
0 5 10
0
20
40
svals of c2
100 50 0 50 100
100
50
0
p
i
s
a
r
100 50 0 50 100
40
20
0
e
i
g
100 50 0 50 100
40
20
0
m
u
s
i
c
100 50 0 50 100
40
20
0
m
l
100 50 0 50 100
40
20
0
a
r
100 50 0 50 100
40
20
0
m
i
n
100 50 0 50 100
20
10
0
b
e
a
m
1 Tu to r i a l
1-78
load doa1
[spec,theta]=doa(ymat,0.5,1.2);
The singular value plot indicat es t he possible pr esence of t wo sour ces; t he
est imat ed angular spect r a show peaks at ar ound t he t r ue bear ings of 15 and
25 degr ees; not ice t hat t he spat ially cor r elat ed Gaussian noise sour ce has
been vir t ually suppr essed by using four t h-or der cumulant s. The ver t ical lines
and cir cles denot e t he t r ue bear ings; t he cr oss denot es t he vir t ual bear ing of
t he spat ially cor r elat ed noise. The bear ings est imat ed by ESPRIT should be
15.0066 and 25.1361.
Figure 1-15 Singular Values of Spatial Cumulant Matrix, and Estimated
Angular Spectra (doa)
0 5 10
0
200
400
svals of c4
100 50 0 50 100
100
50
0
p
i
s
a
r
100 50 0 50 100
50
0
e
i
g
100 50 0 50 100
40
20
0
m
u
s
i
c
100 50 0 50 100
40
20
0
m
l
100 50 0 50 100
50
0
a
r
100 50 0 50 100
40
20
0
m
i
n
100 50 0 50 100
40
20
0
b
e
a
m
H a r m o n i c Pr o c e sse s a n d D O A
1-79
Summary
The est imat ion of t he fr equencies of har monics obser ved in noise, or t hat of
det er mining t he bear ings of sour ces in t he DOA pr oblem can be t r eat ed fr om
eit her a par amet r ic or a nonpar amet r ic view point . In t he lat t er case, we obt ain
nonpar amet r ic est imat es of spect r a and angular spect r a. In t he for mer case, we
have sever al high-r esolut ion algor it hms.
Rout ine harmest can be used for t he har monic r et r ieval pr oblem; t he user has
t he choice of using second- or four t h-or der cumulant s. This r out ine est imat es
power spect r a using t he MUSIC, Eigenvect or , Pisar enko, ML (Capon), AR and
Min-Nor m met hods, based on eit her t he diagonal slice of four t h-or der
cumulant s, or on t he covar iance; it also est imat es t he convent ional
per iodogr am. The number of har monics can usually be det er mined by
examining t he singular value plot gener at ed by harmest.
Rout ine doa can be used t o solve t he dir ect ion-of-ar r ival pr oblem (DOA) for a
unifor mly spaced linear ar r ay. It est imat es angular spect r a using t he
Beamfor mer , ML-Capon, AR, MUSIC, Pisar enkos met hod, eigenvect or ,
Min-Nor m and ESPRIT algor it hms, based on t he spat ial cr oss-cor r elat ion or
cr oss-cumulant . Rout ine pickpeak may be used t o pick peaks in t he est imat ed
(angular ) spect r a.
1 Tu to r i a l
1-80
Nonlinear Processes
The simplest nonlinear syst em is t he second-or der Volt er r a syst em whose
input -out put r elat ionship is defined by
(1-157)
The cor r esponding fr equency domain r epr esent at ion is
(1-158)
and is obt ained by Four ier Tr ansfor ming bot h sides of (1-157). The
t ime-domain pr oduct t er m x(n k)x(n l) leads t o convolut ion in t he fr equency
domain, which is r epr esent ed by t he condit ion =
1
+
2
. It is usually assumed
t hat t he quadr at ic ker nel is r eal and symmet r ic, t hat is, q(k,l) = q(l,k), or
equivalent ly, Q(
1
,
2
) = Q(
2
,
1
) = Q*(
1
,
2
). It is r eadily ver ified t hat
Q(
1
,
2
) in t he r egion |
2
|
1
,0
1
1/4, specifies Q(
1
,
2
) ever ywher e.
The basic pr oblem is given x(n) and y(n), n = 1, . . ., N, we want t o est imat e t he
linear par t , h(k), and t he quadr at ic par t , q(k,l).
Solution Using Cross-Bispectra
In [70], an algor it hm t o est imat e t he par amet er s of t he model in (1-157) was
developed, under t he assumpt ion t hat x(n) is st at ionar y Gaussian. Fr om
(1-157), we can see t hat t he cr oss-spect r um is given by,
(1-159)
The t er m involving t he t r iple pr oduct of t he X()s disappear s since x(n) is
symmet r ically dist r ibut ed. Since S
xx
() and S
xy
() can be est imat ed, we can
est imat e H(), t he linear par t .
y n ( ) h k ( )x n ( k )
k 0 =

q k l , ( )x n ( k )x n l ( )
l 0 =

k 0 =

+ =
Y ( ) H ( )X ( ) = Q
1

2
, ( )X
1
( )X
2
( )

1

2
+ =

+
S
y x
( ) E Y ( )X* ( ) { } H ( )E X ( )X* ( ) { } H ( )S
xx
( ) = = =
N o n l i n e a r Pr o c e sse s
1-81
Consider t he cr oss-cumulant
(1-160)
(1-161)
which follows since x(t) is symmet r ically dist r ibut ed. Assuming Gaussianit y,
we can r ewr it e t he last equat ion in t he fr equency domain, as
(1-162)
which is obt ained under t he assumpt ion t hat q(k,l) = q(l,k). The
cr oss-bispect r um, S
yxx
(
1
,
2
) can be est imat ed via r out ine bispecdx and t he
spect r a can be est imat ed via r out ine spectrum; we can t hen est imat e t he
quadr at ic t r ansfer funct ion, Q(
1
,
2
) fr om (1-162). This algor it hm is
implement ed in r out ine nltick [70]. Not e t hat t he input pr ocess is assumed t o
be Gaussian.
C
xx y
, ( ) E x n ( )x n + ( )y n + ( ) { } =
q k l , ( )E x n ( )x n + ( )x n k + ( )x n l + ( ) { }
l 0 =

k 0 =

=
E{x n ( )x n + ( ) h k ( )x n k + ( )
k 0 =

=
q k l , ( )x n k + ( )x n l + ( ) }
l 0 =

k 0 =

+
S
yxx

1

2
, ( ) 2Q
1

2
, ( )S
xx

1
( )S
x x

2
( ) S
xx

2
( )
1

2
+ ( )E y n ( ) { } + =
1 Tu to r i a l
1-82
Ex a mples
load nl1
[h,q] = nltick(x,y,128,1);
You should see t he display on Figur e 1-16. A descr ipt ion of t he signals x and y
in nl1.mat is given in t he sect ion Dat a Files. The t r ue linear and quadr at ic
impulse r esponses ar e shown in Figur e 1-51.
Figure 1-16 Transfer Functions Estimated by nltick
Solution Using FTs
In [50], anot her algor it hm t o est imat e t he par amet er s of t he model in (1-157)
was developed, wit hout assuming t hat x(n) is Gaussian; indeed, x(n) may be
det er minist ic. Given t he input and out put sequences, x(n) and y(n), we can
comput e t heir FTs, X() and Y() on a suit able gr id,
m
= m/ M, wher e M is t he
FFT lengt h. We can collect (1-158) at t he var ious fr equencies int o a set of
equat ions t hat ar e linear in t he unknowns, namely H() and Q(
1
,
2
) [50].
0 0.2 0.4 0.6
10
4
10
3
10
2
10
1
10
0
linear TF
f
0.4 0.2 0 0.2 0.4
0.4
0.2
0
0.2
0.4
quadratic TF
f1
f
2
0 50 100 150
0.1
0
0.1
0.2
0.3
0.4
linear part: IR
t
10 20 30
5
10
15
20
25
30
quadratic part: IR
t1
t
2
N o n l i n e a r Pr o c e sse s
1-83
Let r = m mod 2, and let m
+
:= (m + r)/2, m
-
:= (m r)/2, . For m = 0, . . ., M/2,
let t he unknown par amet er vect or be
and,
Then, we have t he syst em of linear equat ions,
Y(m) = A
T
b(m),
which is over det er mined; we can obt ain t he least -squar es solut ion for each
fr equency gr id point , m.
This algor it hm is implement ed in r out ine nlpow.
Ex a mples
load nl1
[h1,q1]=nlpow (x,y,128);
You should see t he display on Figur e 1-17.
Figure 1-17 Transfer Functions Estimated by nlpow
b m ( ) H m ( ) Q m
+
m
-
, ( ) Q m
+
1 + m
-
1 , ( ) Q
M
4
----- m
M
4
----- ,
,
_
, , , , =
T
A X m ( ) X , m
+
( )X m
-
( ) X m
+
1 + ( )X m
-
1 ( ) X
M
4
-----
,
_
X m
M
4
-----
,
_
, , , =
T
0 0.2 0.4 0.6
10
4
10
3
10
2
10
1
10
0
linear TF
f
0.2 0.1 0 0.1 0.2
0.2
0.1
0
0.1
0.2
quadratic TF
f1
f
2
0 50 100 150
0.1
0
0.1
0.2
0.3
0.4
linear part: IR
t1
10 20 30
5
10
15
20
25
30
quadratic part: IR
t1
t
2
1 Tu to r i a l
1-84
load nl2
[h2,q2]=nlpow (x,y,128);
You should see t he display on Figur e 1-18.
Figure 1-18 Transfer Functions Estimated by nlpow
As expect ed, t he plot s in t he t wo figur es ar e quit e similar . A descr ipt ion of t he
signals x and y in t he files nl1.mat and nl1.mat may be found in t he sect ion
Dat a Files. The t r ue linear and quadr at ic impulse r esponses ar e shown in
Figur e 1-51.
Quadratic Phase Coupling
Phase coupling occur s due t o nonlinear int er act ions bet ween har monic
component s. Thr ee har monics wit h fr equencies
k
and phases
k
, k = 1,2,3, ar e
said t o be quadr at ically phase coupled if
3
=
1
+
2
and
3
=
1
+
2
. Quadr at ic
phase coupling (coupling at sum and differ ence fr equencies) occur s when a
signal is passed t hr ough a squar e-law device, for example, and may be det ect ed
fr om t he bispect r um [53].
Consider t he signal x(n), which is a mixt ur e of har monics wit h independent
phases, and quadr at ically phase coupled sinusoids, t hat is,
0 0.2 0.4 0.6
10
4
10
3
10
2
10
1
10
0
linear TF
f
0.2 0.1 0 0.1 0.2
0.2
0.1
0
0.1
0.2
quadratic TF
f1
f
2
0 50 100 150
0.1
0
0.1
0.2
0.3
0.4
linear part: IR
t1
10 20 30
5
10
15
20
25
30
quadratic part: IR
t1
t
2
N o n l i n e a r Pr o c e sse s
1-85
(1-163)
wher e , t he s ar e all dist inct , and , and
i

ar e all i.i.d. and unifor mly dist r ibut ed over [,].
Then, t he t hir d-or der cumulant of x(n) is given by [66]
(1-164)
The diagonal slice, C
3x
(,), is given by
(1-165)
x n ( )
i
2
i
n
i
+ ( ) cos
i 1 =
N

i k
q
2
i k
q
n
i k
q
+ ( ) cos
k 1 =
N

i 1 =
N
q

+ =

i 3
q

i 1
q
=
i 2
q
+
i 3
q

i 1
q
=
i 2
q
+ ,
i 1
q

i 2
q
,
C
3x

1

1
, ( )
1
4
---
i 1
q

i 2
q

i 3
q
( )
i 1 =
N
q

=
2
i 1
q

1
2
i 2
q

2
+ ( ) cos [ 2
i 1
q

1
2
i 3
q

2
( ) cos +
2
i 2
q

1
2
i 3
q

2
( ) cos 2
i 2
q

1
2
i 1
q

2
+ ( ) cos + +
2
i 3
q

1
2
i 1
q

2
( ) cos 2
i 3
q

1
2
i 2
q

2
+ ( ) cos + + ]
C
3x
, ( )
1
2
---
i j
q
j 1 =
3

2
i k
q
( ) cos
k 1 =
3

i 1 =
N
q

=
1 Tu to r i a l
1-86
The 2-D FT of (1-164) yields t he bispect r um,
(1-166)
It is evident t hat t he nonr edundant r egion of t he bispect r um shows peaks only
at t he phase and fr equency coupled bifr equencies, . The FT of t he
diagonal slice in (1-165), on t he ot her hand, displays peaks at each of t he t hr ee
fr equencies involved in t he phase-coupling.
Consider () wit h N
q
= 1. We have, omit t ing unnecessar y super scr ipt s,
(1-167)
wher e
3
=
2
+
1
. Not e t hat t he diagonal slice of t he t hir d-or der cumulant is
expr essed as t he sum of t hr ee har monics. Fr om our discussions on AR models
for har monics, we not e t hat C
3x
(,) sat isfies a self-dr iven AR(6) model, whose
r oot s ar e at exp(tj2
k
), k = 1, 2, 3. If we est imat e t he AR polynomial, we can
comput e t he par amet r ic bispect r um
(1-168)
The par amet r ic bispect r um est imat or is implement ed in r out ine qpctor.
S
3

1

2
, ( )
1
8
---
i 1
q

i 2
q

i 3
q
( )
i 1 =
N
q

=

1

i 1
q
+ ( )
2

i 2
q
+ ( )
1

i 2
q
+ ( )
2

i 1
q
+ ( ) + [
+
1

i 1
q
( )
2

i 2
q
( )
1

i 2
q
( )
2

i 1
q
( ) +
+
1

i 1
q
+ ( )
2

i 3
q
( )
1

i 3
q
( )
2

i 1
q
+ ( ) +
+
1

i 2
q
+ ( )
2

i 3
q
( )
1

i 3
q
( )
2

i 2
q
+ ( ) +
+
1

i 1
q
( )
2

i 3
q
+ ( )
1

i 3
q
+ ( )
2

i 1
q
( ) +
+
1

i 2
q
( )
2

i 3
q
+ ( )
1

i 3
q
+ ( )
2

i 2
q
( ) + ]

i 1
q

i 2
q
, ( )
C
3x
, ( )
1
2
---
1

3
2
k
( ) cos
k 1 =
3

=
S
3x

1

2
, ( ) A
1
( )A
2
( )A*
1

2
+ ( ) =
N o n l i n e a r Pr o c e sse s
1-87
Ex a mples
load qpc
[arvec,bspec] = qpctor(zmat,18,12);
Maximum of bispectrum: B(0.1484,0.1016) = 4239
You should see t he display on Figur e 1-19. Not e t hat
1
+
2
0.25, which
cor r esponds t o t he t hir d peak in t he amplit ude spect r um, so t hat we may
conclude t hat t hr ee of t he four har monics ar e quadr at ically phase coupled. The
singular value plot shows six significant singular values cor r esponding t o one
quadr at ically coupled t r iplet ; as in t he case of t he power spect r um (har monics
in noise), over est imat ing t he number of har monics usually leads t o bet t er
r esult s; in t his case, we used an AR or der of 12.
Figure 1-19 Parametric Estimate of bispectrum (qpctor)
Ideally, A() cont ains impulses at t
k
, k = 1,2,3, and S
3x
(
1
,
2
) is given by
(1-169)
0 0.2 0.4 0.6
10
2
10
1
10
0
Power Spectrum
frequency
0 5 10 15 20
0
1
2
3
4
5
Singular values
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45
0
0.05
0.1
0.15
0.2
f1
f
2
Estimated Parametric Bispectrum
S
3x

1

2
, ( )
k 1 =
3

1

k
t
,


_

k 1 =
3

2

l
t
,


_

m 1 =
3

1

2

m
t +
,


_
=
1 Tu to r i a l
1-88
Fr om (1-166), we not e t hat it suffices t o examine t he bispect r um in t he r egion
0
2

1
1/2. If
2
21, t he par amet r ic bispect r um will show a single
impulse at (
1
,
2
), cor r esponding t o k = 1, l = 2, m = 3 in (1-169), indicat ing t he
exist ence of t he quadr at ically phase-coupled t r iplet (
1
,
2
,
1
+
2
). If
2
= 2
1
,
t he bispect r um will have an addit ional peak at (
1
,
2
) = (
1
,
1
), cor r esponding
t o k = 1, l = 1, m = 2 in (1-169), which can also r esult fr om t he phase-coupled
t r iplet (
1
,
1
,2
1
).
It is impor t ant t o not e t hat t he pr ocess in (1-163) is not er godic, t hat is,
consist ent est imat es of t he t hir d-or der cumulant , C
3x
(
1
,
2
), cannot be obt ained
fr om a single r ealizat ion, unless fr equency-coupling is always accompanied by
phase coupling [68]. Indeed, t he r andom phase assumpt ion it self suggest s t hat
mult iple r ealizat ions ar e r equir ed. Fur t her , cr eat ing mult iple r ealizat ions by
r ecor d segment at ion does not lead t o consist ent est imat es. Given a single
r ealizat ion, C
3x
(
1
,
2
) will show impulses if fr equency coupling exist s, t hat is,

3
=
2
+
1
. Given mult iple r ealizat ions, C
3x
(
1
,2) (and C
3x
(,)) will be nonzer o
only if bot h fr equency and phase coupling exist , t hat is,
3
=
2
+
1
, and
3
=
2

+
1
.
The par amet r ic bispect r um in (1-168) shows a peak whenever A(
1
), A(
2
), and
A(
1
+ 2) have peaks. Hence, when N
q
> 1, spur ious peaks might r esult fr om
int er act ions bet ween differ ent t r iplet s of t he for m,
In t he case of N
q
>1, int er act ions may occur bet ween differ ent QPC t r iplet s
leading t o addit ional peaks in t he par amet r ic bispect r um given by (1-168).
Summary
The Higher -Or der Spect r al Analysis Toolbox offer s t wo algor it hms t o est imat e
t he par amet er s of a second-or der Volt er r a model: nltick, which assumes t hat
t he input is Gaussian, and nlpow, which is applicable t o ar bit r ar y input s. Not e
t hat bot h algor it hms r equir e access t o input s and out put s. Quadr at ic-phase
coupling (QPC) can be det ect ed using r out ine qpctor. Rout ine qpcgen can be
used t o gener at e synt het ics for t he QPC pr oblem, and nlgen can be used t o
comput e t he out put of a second-or der Volt er r a syst em.

i l
q

j m
q
+
k n
q
. =
Ti m e -Fr e q u e n c y D i str i b u ti o n s
1-89
Time-Frequency Distributions
The popular and nat ur al t ool in linear syst em analysis is t he Four ier
t r ansfor m, which decomposes a signal int o it s fr equency component s. The
power spect r um (or ener gy spect r um) gives us infor mat ion about t he fr equency
component s in t he dat a, but not about t he t empor al localizat ion of t hese
component s. Alt er nat ively, t he or iginal t ime signal it self gives us good t ime
r esolut ion (obviously!) but does not give us any idea about fr equency
localizat ion, except in t r ivial cases. So far , we have defined spect r a, bispect r a,
and t r ispect r a, only for st at ionar y pr ocesses; as such, t hey ar e inappr opr iat e
for t he analysis of nonst at ionar y pr ocesses or t r ansient signals. For
nonst at ionar y pr ocesses, we can define t ime-var ying cumulant s and
polyspect r a (in t he same way t hat one t alks about t ime-var ying covar iances
and power spect r a). Consider t he signal, y(t) = s(t) + g(t), wher e s(t) is a
det er minist ic signal and g(t ) is st at ionar y zer o-mean noise (per haps Gaussian).
The pr ocess y(t ) is nonst at ionar y: it s mean is s(t ), and for k > 1, c
ky
= c
kg
; t hus,
only t he fir st -or der cumulant car r ies infor mat ion about t he signal s(t ). In
cont r ast , t he higher -or der moment s of y(t) depend bot h upon s(t) and g(t ); t hus,
in t his case, it is pr efer able t o use higher -or der moment s [63]. For such signals,
t ime-fr equency dist r ibut ions t hat descr ibe t he t empor al evolut ion of t he
spect r um or t he polyspect r um (moment spect r um) ar e useful t ools.
A t ime-fr equency dist r ibut ion (TFD) is a t r ansfor m t hat maps a 1-D signal int o
a 2-D t ime-fr equency map, which descr ibes how t he spect r al cont ent of t he dat a
evolves wit h t ime. As such, TFDs ar e t he nat ur al t ools for t he analysis,
synt hesis, int er pr et at ion, and pr ocessing of nonst at ionar y signals. Among t he
mor e well-known TFDs ar e t he shor t -t ime Four ier t r ansfor m (STFT) [1], t he
Gabor r epr esent at ion [17], and t he Wavelet t r ansfor m [33, 55].
Linear it y is a desir able pr oper t y of algor it hms used in analyzing linear
syst ems; however , quadr at ic t ime-fr equency dist r ibut ions have been pr oposed,
analyzed, and int er pr et ed as t ime-var ying power spect r a [13, 25]. The so-called
Cohen class of shift -invar iant dist r ibut ions includes special cases such as t he
Spect r ogr am, Rihaczek, Page, Wigner -Ville dist r ibut ion (WD) and
Choi-Williams dist r ibut ions [13, 25].
The WD has become quit e popular because it possesses a number of useful
pr oper t ies, and has been used in t he analysis of phase modulat ed signals,
which ar e common in r adar and sonar [3]. An impor t ant pr oper t y of t he WD is
t hat ever y member of Cohens class can be int er pr et ed as a 2-D linear ly filt er ed
ver sion of t he WD [25]. A t hir d-or der WD was int r oduced in [18]; it was
1 Tu to r i a l
1-90
gener alized and it s pr oper t ies wer e st udied in [14, 15, 60, 61, 62]. Higher -or der
WDs descr ibe t he evolut ion in t ime of t he higher -or der moment spectra of t he
signal.
Wigner Spectrum
The Wigner dist r ibut ion (WD) was int r oduced in 1932 by Wigner in t he cont ext
of quant um mechanics; it s usefulness t o pr oblems in communicat ion t heor y
was discover ed by Ville in 1948; consequent ly, it is oft en called t he
Wigner -Ville dist r ibut ion. A ser ies of paper s by Classen and Mecklenbr aker
[11] discussed t he usefulness of t he WD for t ime-fr equency analysis of
cont inuous and discr et e-t ime signals, and was devot ed t o applicat ions in digit al
signal pr ocessing; sampling issues ar e also discussed in [12]. Tut or ial
over views of t he WD and it s r elat ionships wit h ot her t ime-fr equency
dist r ibut ions (TFD) may be found in [3, 13, 25].
In or der t o differ ent iat e t he second-or der WD fr om t he higher -or der WDs (t o
be int r oduced lat er ), we will r efer t o t he convent ional WD as t he Wigner
spect r um (WS).
In cont r ast t o t he STFT, which is r esolut ion limit ed eit her in t ime or in
fr equency (dict at ed by t he window funct ion), and suffer s fr om smear ing and
sidelobe leakage, t he WS offer s excellent r esolut ion in bot h t he fr equency and
t ime domains.
The Wigner cr oss spect r um (WCS) of t wo signals, x(t ), y(t), is defined via,
(1-170)
(1-171)
wher e X() and x(t) const it ut e an FT pair . The aut o WS is obt ained when
x(t) = y(t ), and is a bilinear funct ion of t he signal x(t ).
The ambiguity function, which is widely used in r adar , is defined as t he 2-D FT
of t he WS, and can also be expr essed as
(1-172)
W
x y
t , ( ) := x

t 2 + ( )y* t 2 ( )e
j w
d
:=
1
2
------ X

2 + ( )Y * 2 ( )e
j t
d
AF , ( ) x* t

2
---
,
_
x t

2
--- +
,
_
e
j 2t
t d

=
Ti m e -Fr e q u e n c y D i str i b u ti o n s
1-91
Cohens gener al class of TFDs is defined by
(1-173)
wher e t he ker nel (,) specifies t he par t icular dist r ibut ion t hat is obt ained.
Let x(n) = y(n) + z(n); it follows t hat
(1-174)
Since t he WS is a quadr at ic t r ansfor m, t he WS of t he sum of t wo signals is not
t he sum of t he individual WSs, but also has cr oss-t er ms. These cr oss-t er ms
make it difficult t o int er pr et t he WS. These cr oss-t er ms may be suppr essed by
appr opr iat e filt er ing, in t he ambiguit y funct ion domain, t hat is, by appr opr iat e
choice of t he ker nel (,).
In or der t o r educe t he effect s of t he cr oss-t er ms, Choi and Williams [10]
pr oposed using t he ker nel
(1-175)
wher e t he par amet er cont r ols t he amount of at t enuat ion (t he amplit ude of
t he cr oss-t er ms is pr opor t ional t o ). Unfor t unat ely, incr eased suppr ession of
cr oss-t er ms invar iably leads t o smear ing or loss of r esolut ion of t he aut o t er ms
in t he t ime-fr equency plane.
In pr act ice, signals ar e sampled in t ime, and we comput e FTs also on a sampled
fr equency gr id. The discr et izat ion in t ime and fr equency of t he cont inuous-t ime
WS leads t o t he nonaliasing r equir ement t hat t he or iginal signal be sampled at
t wice t he Nyquist r at e [12].
The discr et e-t ime algor it hm is given next [13, 25]. Let t he inst ant aneous
cr oss-cor r elat ion be defined by
(1-176)
wher e n is ident ified wit h t ime and m wit h lag. The WCS is t hen defined by,
W
C
t , ( ) , ( )AF , ( )e
j 2t j 2
d d

=
W
xx
n , ( ) W
yy
n , ( ) W
z z
n , ( ) W
y z ,
n , ( ) W
z y ,
n , ( ) + + + =
, ( )
2

2
( ) exp =
r
x y
m n , ( ) x* n m ( )y n m + ( ) =
1 Tu to r i a l
1-92
(1-177)
The WS is obt ained when x(n) = y(n). The or iginal signal must be sampled at
t wice t he Nyquist r at e or fast er , in or der t o avoid aliasing. In pr act ice, t he
fr equency var iable is also discr et ized, = k/ K, wher e K cont r ols t he fr equency
r esolut ion. The WCS in (1-177) can also be implement ed via t wo FFT
algor it hms, following t he appr oach in [46]. Bot h appr oaches demand t he same
comput at ional and st or age complexit y. This algor it hm in (1-177) is
implement ed in r out ine wig2.
The ambiguit y funct ion, AF, can be comput ed as
(1-178)
The AF is mult iplied by t he Choi-Williams filt er ,
(1-179)
A 2-D FT ( t o n and m t o ) yields t he filt er ed WS. Cr oss-t er ms can usually be
suppr essed via t his appr oach, but wit h a concomit ant loss of r esolut ion. In
pr act ice, is a discr et ized fr equency gr id. This algor it hm is implement ed in
r out ine wig2c.
W
xy
n , ( ) r
xy
m n , ( ) j m ( ) exp
m

=
AF m , ( ) r
x y
m n , ( ) j 2n ( ) exp
n

=
w m , ( ) m ( )
2
( ) exp =
Ti m e -Fr e q u e n c y D i str i b u ti o n s
1-93
Ex a mples
clf
load wigdat
subplot(221), wig2(s2,[],0);
subplot(222), wig2(s2);
subplot(223), wig2(s3);
subplot(224), wig2(s4);
You should see t he display in Figur e 1-20.
Figure 1-20 Wigner Spectra (wig2)
Signal s2 is a har monic wit h a nominal fr equency of 0.05 Hz, which has been
mult iplied by a Gaussian window, and is cent er ed at n = 20. Not e t hat t he WS
of t he signal is concent r at ed ar ound it s nominal cent er fr equency; not e also t he
int er act ion bet ween t he ener gies at t he posit ive and negat ive fr equencies
giving r ise t o t he int er fer ence t er ms ar ound D.C. By using t he analyt ic ver sion
of t he signal, t he negat ive fr equency t er ms ar e suppr essed; t his also suppr esses
t he dist or t ion t er ms ar ound D.C.
0.2 0.1 0 0.1 0.2
10
20
30
40
50
60
t
i
m
e

i
n

s
a
m
p
l
e
s
frequency
WS
0.2 0.1 0 0.1 0.2
10
20
30
40
50
60
t
i
m
e

i
n

s
a
m
p
l
e
s
frequency
WS
0.2 0.1 0 0.1 0.2
10
20
30
40
50
60
t
i
m
e

i
n

s
a
m
p
l
e
s
frequency
WS
0.2 0.1 0 0.1 0.2
10
20
30
40
50
60
t
i
m
e

i
n

s
a
m
p
l
e
s
frequency
WS
1 Tu to r i a l
1-94
Signal s3 is a har monic wit h a nominal fr equency of 0.15 Hz, which has been
mult iplied by a Gaussian window, and is cent er ed at n = 50. Not e t hat t he WS
of t he signal is concent r at ed ar ound it s nominal cent er fr equency.
In t he last panel, we comput e t he WS of t he sum signal s4=s2+s3; not e t he
cr oss-t er ms in t he WS; some of t hese can be eliminat ed by using r out ine wig2c.
Ex a mples
load wigdat
wig2c(s4);
Compar e t he display in Figur e 1-21 wit h t he (2,2) panel of Figur e 1-20; not e
t hat t he cr oss-t er ms have been suppr essed.
Figure 1-21 Wigner Spectrum With Choi-Williams Smoothing (wig2c)
Wigner Bispectrum
The not ion of Wigner spect r a can be gener alized t o Wigner bispect r a (WB) and
Wigner t r ispect r a (WT) by consider ing t he FTs of appr opr iat ely defined
inst ant aneous t r iple and quadr uple pr oduct s.
0.25 0.2 0.15 0.1 0.05 0 0.05 0.1 0.15 0.2
10
20
30
40
50
60
t
i
m
e

i
n

s
a
m
p
l
e
s
frequency
CWS s=0.05
Ti m e -Fr e q u e n c y D i str i b u ti o n s
1-95
Define t he inst ant aneous t r iple-pr oduct
(1-180)
wher e = 1/3 and = 2/3. The WB is t hen given by
(1-181)
This algor it hm is implement ed in r out ine wig3, wher e t he slice
1
=
2
is
comput ed. Not e t hat t he or iginal signal should be sampled at t wice t he Nyquist
r at e, in or der t o avoid aliasing. It is shown in t hat t he fr equency axes ar e scaled
by t he fact or of 2/3, which can be easily fixed.
As in t he case of t he WS, t he WB also suffer s fr om t he pr oblems of cr oss-t er ms;
one can at t empt t o suppr ess t hese t er ms by appr opr iat e filt er ing. Define t he
smoot hing ker nel,
(1-182)
The filt er ed WB is t hen given by
This algor it hm is implement ed in r out ine wig3c, wher e t he slice
1
=
2
is
comput ed. It should also be not ed t hat t he applicat ion of t he Choi-Williams
filt er t o t he WB does not guar ant ee pr eser vat ion of t he aut o-t er ms;
consequent ly, r out ine wig3c should be used wit h gr eat caut ion.
r
3
t
1

2
, , ( ) x* t
1

2
( )x t
1

2
+ ( )x t
1

2
+ ( ) =
W n
1

2
, , ( ) d
1
d
2
e
j 2
1

1

2

2
+ ( )
r
3
t
1

2
, , ( )

=

1

2
, , ( )
2

1
2

2
2
+ ( ) ( ) exp =
W n
1

2
, , ( )
1

2
u e
j 2
1

1

2

2
+ ( )
e
j 2 t
e
j 2u
d d d

=
r
3
t
1

2
, , ( )
1

2
, , ( )
1 Tu to r i a l
1-96
Ex a mples
clf
load wigdat
subplot(221), wig3(s2,[],0);
subplot(222), wig3(s2);
subplot(223), wig3(s3);
subplot(224), wig3(s4);
You should see t he display in Figur e 1-22.
Figure 1-22 Wigner bispectra (wig3)
Signal s2 is a har monic wit h a nominal fr equency of 0.05 Hz, which has been
mult iplied by a Gaussian window, and is cent er ed at n = 20. Not e t hat t he WB
of t he signal is concent r at ed ar ound it s nominal cent er fr equency; not e also t he
int er act ion bet ween t he ener gies at t he posit ive and negat ive fr equencies
giving r ise t o t he int er fer ence t er ms ar ound D.C. By using t he analyt ic ver sion
of t he signal, t he negat ive fr equency t er ms ar e suppr essed; t his also suppr esses
t he dist or t ion t er ms ar ound D.C.
0.2 0.1 0 0.1 0.2
10
20
30
40
50
60
t
i
m
e

i
n

s
a
m
p
l
e
s
frequency
WB f1=f2
0.2 0.1 0 0.1 0.2
10
20
30
40
50
60
t
i
m
e

i
n

s
a
m
p
l
e
s
frequency
WB f1=f2
0.2 0.1 0 0.1 0.2
10
20
30
40
50
60
t
i
m
e

i
n

s
a
m
p
l
e
s
frequency
WB f1=f2
0.2 0.1 0 0.1 0.2
10
20
30
40
50
60
t
i
m
e

i
n

s
a
m
p
l
e
s
frequency
WB f1=f2
Ti m e -Fr e q u e n c y D i str i b u ti o n s
1-97
Signal s3 is a har monic wit h a nominal fr equency of 0.15 Hz, which has been
mult iplied by a Gaussian window, and is cent er ed at n = 50. Not e t hat t he WB
of t he signal is concent r at ed ar ound it s nominal cent er fr equency.
In t he last panel, we comput e t he WB of t he sum signal s4=s2+s3; not e t he
cr oss-t er ms in t he WS; some of t hese can be eliminat ed by using r out ine wig3c.
A dir ect consequence of t he definit ion of t he WB is t hat t he fr equency axes ar e
scaled by t he fact or 2/3 [4][5]. In t his r out ine, t his scaling has been undone so
t hat t he peaks appear at t he expected fr equencies.
Ex a mples
load wigdat
wig3c(s4,256,0.2,1)
Compar e t he display in Figur e 1-23 wit h t he (2,2) panel in Figur e 1-20 and wit h
Figur e 1-21; not ice t he suppr ession of cr oss-t er ms.
Figure 1-23 Wigner bispectrum With Choi-Williams Smoothing (wig3c)
0.25 0.2 0.15 0.1 0.05 0 0.05 0.1 0.15 0.2
10
20
30
40
50
60
t
i
m
e

i
n

s
a
m
p
l
e
s
frequency
CWB f1=f2 s=0.2
1 Tu to r i a l
1-98
Wigner Trispectrum
Define t he inst ant aneous four t h-or der pr oduct ,
(1-183)
wher e
The WT is t hen given by
(1-184)
Not ice t hat t wo t er ms ar e conjugat ed; also not e t he sign of t he
3

3
t er m. This
algor it hm is implement ed in r out ine wig4 wher e t he slice
1
=
2
=
3
= is
comput ed, t hat is, we comput e
(1-185)
Not e t hat t he or iginal signal should be sampled at t wice t he Nyquist r at e, in
or der t o avoid aliasing. Also not e t hat t he fr equency axes ar e scaled by t he
fact or of 1/2, which can be easily fixed.
The WT also suffer s fr om t he pr esence cr oss-t er ms. As in t he case of WS, we
can at t empt t o suppr ess t he cr oss t er ms by appr opr iat e filt er ing.
Define t he smoot hing ker nel,
(1-186)
The filt er ed WT is t hen given by
This algor it hm is implement ed in r out ine wig4c, wher e t he slice
1
=
2
=
3

is comput ed; t he r esult ing sliced WT is r eal valued. As in t he case of t he WS,
cr oss-t er ms can usually be suppr essed by t his appr oach, but wit h a
concomit ant loss of r esolut ion. In cont r ast t o t he WB, t he filt er ing does not
unduly dist or t t he aut o t er ms [14, 15].
r
4
t
1

2

3
, , , ( ) x* t ( )x t
1
+ ( )x t
2
+ ( )x* t
3
+ ( ) =
:
1

2

3
+ + ( ) 4. =
W n
1

2

3
, , , ( )
1

2

3
e
j 2
1

1

2

2

3

3
+ ( )
r
4
t
1

2

3
, , , ( ) d d d

=
W
s
n , ( )
1

2

3
e
j 2
1

2

3
+ + ( )
r
4
t
1

2

3
, , , ( ) d d d

=

1

2

3
, , , ( )
2

1
2

2
2

3
2
+ + ( ) ( ) exp =
W n
1

2

3
, , , ( ) d
1

2
d
3
u e
j 2
1

1

2

2

3

3
+ + ( )
d d d

=
Ti m e -Fr e q u e n c y D i str i b u ti o n s
1-99
Ex a mples
clf, load wigdat
subplot(221), wig4(s2,[],0);
subplot(222), wig4(s2);
subplot(223), wig4(s3);
subplot(224), wig4(s4);
You should see t he display in Figur e 1-24. Signal s2 is a har monic wit h a
nominal fr equency of 0.05 Hz, which has been mult iplied by a Gaussian
window, and is cent er ed at n = 20. Not e t hat t he WT of t he signal is
concent r at ed ar ound it s nominal cent er fr equency; not e also t he int er act ion
bet ween t he ener gies at t he posit ive and negat ive fr equencies giving r ise t o t he
int er fer ence t er ms ar ound D.C. By using t he analyt ic ver sion of t he signal, t he
negat ive fr equency t er ms ar e suppr essed; t his also suppr esses t he dist or t ion
t er ms ar ound D.C.
Signal s3 is a har monic wit h a nominal fr equency of 0.15 Hz, which has been
mult iplied by a Gaussian window, and is cent er ed at n = 50. Not e t hat t he WT
of t he signal is concent r at ed ar ound it s nominal cent er fr equency.
In t he last panel, we comput e t he WS of t he sum signal s4=s2+s3; not e t he
cr oss-t er ms in t he WT; some of t hese can be eliminat ed by using r out ine wig4c.
Figure 1-24 Wigner trispectra (wig4)
0.2 0.1 0 0.1 0.2
10
20
30
40
50
60
t
i
m
e

i
n

s
a
m
p
l
e
s
frequency
WT
0.2 0.1 0 0.1 0.2
10
20
30
40
50
60
t
i
m
e

i
n

s
a
m
p
l
e
s
frequency
WT
0.2 0.1 0 0.1 0.2
10
20
30
40
50
60
t
i
m
e

i
n

s
a
m
p
l
e
s
frequency
WT
0.2 0.1 0 0.1 0.2
10
20
30
40
50
60
t
i
m
e

i
n

s
a
m
p
l
e
s
frequency
WT
1 Tu to r i a l
1-100
Ex a mples
load wigdat
wig4c(s4);
Compar e t he display in Figur e 1-25 wit h t he (2,2) panel of Figur e 1-24, and
Figur e 1-21 and Figur e 1-23; not e t hat t he cr oss-t er ms in Figur e 1-24 have
been suppr essed.
Figure 1-25 Wigner Trispectrum With Choi-Williams Filtering (wig4c)
Summary
The Higher -Or der Spect r al Analysis Toolbox offer s r out ines t o comput e t he
WS, and slices of t he WB and t he WT, and ar e implement ed in wig2, wig3, and
wig4. Rout ines wig2c, wig3c, and wig4c implement t he so-called
Choi-Williams filt er , which can help suppr ess t he cr oss-t er ms in t he WS, WB,
and WT. It should be not ed t hat t his filt er ing may, in fact , dest r oy signal t er ms
in t he WB (but not in WS or WT). Wigner spect r a of differ ent or der s offer
differ ent per spect ives on t he dat a, and ar e useful for explor at or y dat a analysis.
0.25 0.2 0.15 0.1 0.05 0 0.05 0.1 0.15 0.2
10
20
30
40
50
60
t
i
m
e

i
n

s
a
m
p
l
e
s
frequency
CWT s=0.05
Ti m e -D e l a y Esti m a ti o n
1-101
Time-Delay Estimation
The t ime-delay est imat ion pr oblem occur s in var ious applicat ions, for example,
in t he det er minat ion of r ange and bear ing in r adar and sonar . It also has
esot er ic applicat ions, such as t he measur ement of t he t emper at ur e of a molt en
alloy by measur ing t he passage t ime of a signal. Ot her applicat ions include
analysis of EEG dat a.
The basic model is as follows: t wo sensor s r ecor d delayed r eplicas of a signal,
in t he pr esence of noise:
(1-187)
(1-188)
wher e D is t he delay of t he signal at t he y-sensor r elat ive t o t he signal at t he
x-sensor , A is t he r elat ive amplit ude gain, and w
x
(t) and w
y
(t) ar e sensor noises.
Given x(t), y(t), t =0 . . ., N 1, we want t o est imat e t he delay D.
The basic idea is t o shift t he signal y(t) and compar e t he shift ed wavefor m wit h
x(t); t he best mat ch occur s when t he shift equals t he delay D. This not ion is
made mor e pr ecise by using t he cr oss-cor r elat ion bet ween t he t wo signals. We
assume t hat s(t ) is a st at ionar y pr ocess, and t hat t he noises ar e zer o mean.
A Cross-Correlation Based Method
The cr oss-cor r elat ion bet ween t he t wo signals x(t) and y(t ) is given by,
(1-189)
wher e is t he cr oss cor r elat ion bet ween t he t wo noise pr ocesses, and
R
ss
() is t he aut ocor r elat ion of t he signal. If t he noises ar e uncor r elat ed, R
xy
()
will have a peak at = D, t he unknown delay. In pr act ice, due t o effect s of finit e
lengt h est imat es, and due t o t he pr esence of noise, t he cr oss-cor r elat ion
est imat e may not have a shar p peak.
The dat a may be pr efilt er ed in or der t o shar pen t he peak; equivalent ly, we can
mult iply t he est imat ed cr oss-cor r elat ion by a window funct ion. Differ ent
choices of t he window funct ion lead t o differ ent est imat es. The most popular
x t ( ) s t ( ) w
x
t ( ) + =
y t ( ) As t D ( ) w
x
t ( ) + =
R
x y
( ) AR
s s
D ( ) R
w
x
w
y
,
( ) + =
R
w
x
w
y
,
( )
1 Tu to r i a l
1-102
window funct ion is t he maximum-likelihood window of Hannan and
Thompson, which is descr ibed below.
Let S
xy
() denot e t he cr oss-spect r um bet ween t he t wo signals, x and y; and let
S
xx
() and S
yy
() denot e t he aut ospect r a of x and y. The squar ed coher ence
funct ion is defined by
The opt imal-maximum-likelihood window is t hen
and t he windowed cr oss-cor r elat ion, R
xy
(m), is t he IFT of W()S
xy
().
Est imat es of t he aut o- and cr oss- spect r a and t he coher ence can be obt ained via
t he MATLAB r out ine spectrum; t he segment lengt h must be at least t wice t he
expect ed maximum delay. Since good est imat es of t he spect r a demand a lar ge
number of segment s, it is cr it ical t hat t he lengt hs of t he t ime-ser ies, x and y, be
much lar ger t han t he expect ed maximum delay.
An init ial est imat e, d, of t he delay D is given by t he locat ion of t he peak of R(m).
A t hr ee-point int er polat ion may be used t o impr ove t he delay est imat e [39]
The opt imal-maximum-likelihood window est imat or is implement ed in r out ine
tder.
C
xy
( )
S
xy
( )
2
S
xx
( )S
y y
( )
-----------------------------------. =
W ( )
1
S
xy
( )
--------------------
C
xy
( )
1 C
xy
( ) ( )
--------------------------------, =
D

2D 1
2
-----------------
R d ( ) R d 1 ( )
R d 1 + ( ) 2 *R d ( ) R d 1 ( ) +
------------------------------------------------------------------------------. =
Ti m e -D e l a y Esti m a ti o n
1-103
Ex a mples
load tde1
delay=tder(s1,s2,30,64,0,64);
Estimated delay= 15.9987
You should see t he display on Figur e 1-26 and t he Est imat ed delay
(Est imat ed=15.997). The t wo signals ar e cor r upt ed by spat ially cor r elat ed
noise; t he noise cor r elat ion shows a st r ong peak at a delay of 5 samples; t he
signal delay is 16 samples.
Figure 1-26 Time-Delay Estimated Using Cross-Correlation (tder)
A Cross-Cumulant Based Method
The cr oss-cor r elat ion based met hod assumes t hat t he sensor noises ar e
uncor r elat ed. If t he noise pr ocesses ar e cor r elat ed, it may not be possible t o
det ect t he peak of R
ss
() since it may be masked by (see (1-189)). If
t he signals ar e non-Gaussian, and t he noise pr ocesses ar e Gaussian, we can
use t hir d-or der cumulant s, even if t he noise pr ocesses ar e cor r elat ed.
80 60 40 20 0 20 40 60 80
0.4
0.2
0
0.2
0.4
0.6
0.8
1
TDER: Windowed Rxy: delay = 16
R
w
x
w
y
,
( )
1 Tu to r i a l
1-104
Let P be t he maximum expect ed delay, and assume t hat t he delay D is an
int eger . Then, [39, 40]
(1-190)
wher e a(n) = 0, n D, and a(D) = 1. Consider t he t hir d-or der cumulant s,
(1-191)
(1-192)
Subst it ut ing (1-190) int o (1-191), we obt ain
(1-193)
Using t his equat ion for var ious values of and , we get a syst em of linear
equat ions in t he a(i)s, namely,
C
xxx
a = c
yxx
.
The est imat ed delay is t he index n which maximizes | a(n)| . A low r ank
appr oximat ion of t he cumulant mat r ix C
xxx
may be used t o impr ove t he
r obust ness t o noise. This algor it hm is implement ed in r out ine tde.
y n ( ) a i ( )x n i ( )
i P =
P

w n ( ) + =
C
y xx
, ( ) : E y* n ( )x n + ( )x n + ( ) { } =
C
x xx
, ( ) : E x* n ( )x n + ( )x n + ( ) { } =
C
yx x
, ( ) a i ( )C
x xx
i + i + , ( )
i P =
P

=
Ti m e -D e l a y Esti m a ti o n
1-105
Ex a mples
load tde1
[delay,avec]=tde(s1,s2,30,128);
Estimated delay= 16
You should see t he display in Figur e 1-27, and t he Est imat ed delay (Est imat ed
delay=16). The t wo signals ar e cor r upt ed by spat ially cor r elat ed noise; t he
cr oss-cor r elat ion bet ween t he noises at t he t wo sensor s has a st r ong peak at a
delay of 5 samples; t he signal delay is 16 samples.
Figure 1-27 Time-Delay Estimated Using Cross-Cumulants (tde)
A Hologram Based Method
The algor it hm descr ibed in t he pr evious sect ion can also be implement ed in t he
fr equency domain. Define aut o- and cr oss-bispect r a,
30 20 10 0 10 20 30
0.2
0
0.2
0.4
0.6
0.8
1
TDE: parameter vector, delay = 16
Delay in samples
B
xx x

1

2
, ( ) E X
1
( )X
2
( )X*
1

2
+ ( ) { } S
xx x

1

2
, ( ) = =
B
x yz

1

2
, ( ) E X
1
( )X
2
( )X*
1

2
+ ( ) { } S
xx x

1

2
, ( )e
j 2D
2

= =
1 Tu to r i a l
1-106
The t hir d-or der hologram, h(), is t hen defined by [39, 40]
(1-194)
(1-195)
(1-196)
The hologr am should display a st r ong peak at t he locat ion of t he t r ue delay.
Since t hir d-or der st at ist ics ar e used, t he met hod is insensit ive (in t heor y) t o
bot h spat ially and t empor ally color ed noise which is symmet r ically dist r ibut ed
(e.g., Gaussian). In pr act ice, t he est imat ed hologr am, h(), will not be an
impulse; hence, we est imat e D as t he index , which maximizes | h()| . This
algor it hm is implement ed in r out ine tdeb.
h ( ) d
1
d
2
j 2
2
( ) exp
B
xy x

1

2
, ( )
B
xx x

1

2
, ( )
--------------------------------

=
d
1
d
2
j 2
2
( ) exp e
j 2D
2

=
D ( ) =
Ti m e -D e l a y Esti m a ti o n
1-107
Ex a mples
load tde1
delay=tdeb(s1,s2,30);
Delay estimated by TDEB is 16
You should see t he display on Figur e 1-28, and t he Est imat ed delay (Est imat ed
delay=16). The t wo signals ar e cor r upt ed by spat ially cor r elat ed noise; t he
cr oss-cor r elat ion bet ween t he noises at t he t wo sensor s has a st r ong peak at a
delay of 5 samples; t he signal delay is 16 samples.
Figure 1-28 Time-Delay Estimated Using Cross-Bispectrum (tdeb)
Summary
The basic idea in t ime-delay est imat ion is t o locat e t he peak in t he
cr oss-cor r elat ion bet ween t wo signals; r out ine tder implement s t he
Maximum-likelihood window of Hannan and Thompson, and is useful if t he
noises at t he t wo sensor s ar e not spat ially cor r elat ed.
Rout ines tde and tder use t hir d-or der cr oss-cumulant s and cr oss-bispect r a
r espect ively; hence, t he sensor noises may be spat ially cor r elat ed, pr ovided
t hey ar e symmet r ic (cr oss-bispect r um of t he noises is zer o). Rout ine tdegen can
be used t o gener at e synt het ics for t he TDE pr oblem.
80 60 40 20 0 20 40 60 80
10
0
10
20
30
40
50
60
70
TDEB: Hologram delay = 16
1 Tu to r i a l
1-108
Case Studies
In t his sect ion, we will use t he Higher -Or der Spect r al Analysis Toolbox t o
pr ocess some r eal dat a; in doing so, we will lear n some of t he pit falls and t r icks
of t he t r ade.
Sunspot Data
Schust er was, per haps, t he fir st t o use t he per iodogr am t o analyze sunspot
dat a in t he ninet eent h cent ur y; subsequent ly, t hese dat a have been vir t ually
adopt ed as a st andar d by st at ist icians and signal analyst s. These dat a ar e
known t o have an appr oximat e 11-year cycle. Sunspot dat a for t he year s
1700-1987 ar e available in t he file sunspot.dat, which is included in t he
st andar d dist r ibut ion package.
Figur e 1-29 shows t he dat a and t he hist ogr am; t he lat t er is indicat ive of an
exponent ial dist r ibut ion. We used cumest t o obt ain t he est imat es of some
summar y st at ist ics, which ar e shown in t he t hir d column of Table 1-1; t he last
column is discussed lat er .
Figure 1-29 Sunspot Data
0 50 100 150 200 250 300
0
50
100
150
200
0 20 40 60 80 100 120 140 160 180 200
0
20
40
60
80
100
C a se Stu d i e s
1-109
Table 1-1: Summary Statistics for Sunspot Data
The dat a in Figur e 1-29 display an appar ent per iodicit y; we can use harmest t o
ver ify t his. The (1,1) panel of Figur e 1-30 displays t he singular values of t he
covar iance mat r ix; t he plot indicat es t hat only t hr ee singular values ar e
significant (p = 3); hence, we used an or der of 3. The cor r esponding power
spect r al est imat es ar e shown in t he r emaining panels of Figur e 1-30: All of t he
est imat es have a st r ong peak at 0 Hz and at appr oximat ely t0.1 Hz. We can use
pickpeak t o accur at ely locat e t he peaks in t he power spect r a; we can also use
roots t o est imat e t he locat ions of t he r oot s of t he AR polynomial est imat ed by
t he AR met hod; doing so, we obt ain an est imat e of 10.64 year s for t he per iod.
Figure 1-30 Sunspot Data: Power Spectra
Original Differenced
Mean = E{x(t )} 48.434 0.085
Var iance
2
= E{(x(t ) )
2
} 1548.781 547.773
Skewness E{(x(t ) )
3
}/
3
1.038 0.904
Kur t osis E{(x(t ) )
4
}/
4
3 0.657 1.502
0 10 20 30 40
0
1
2
x 10
5
sval: cum2
0.5 0 0.5
100
50
0
pxx
0.5 0 0.5
100
50
0
e
i
g
0.5 0 0.5
40
20
0
m
u
s
i
c
0.5 0 0.5
100
50
0
p
i
s
a
r
0.5 0 0.5
40
20
0
m
l
0.5 0 0.5
200
100
0
a
r
0.5 0 0.5
40
20
0
m
i
n