Signal Approximation using the Bilinear Transform

by

Archana Venkataraman
Submitted to the Department of Electrical Engineering and Computer Science
in partial fulfillment of the requirements for the degree of

Masters of Engineering in Electrical Engineering and Computer Science

at the MASSACHiUSETTS INSTITUTE

OF FECHNOLOGY
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
NOV 1 3 2008
August 2007

@ Archana Venkataraman, MMVII. All rights reserved. LIE3RARIES

The author hereby grants to MIT permission to reproduce and distribute publicly
paper and electronic copies of this thesis document in whole or in part.

Author ... ,
Department of Electrical Engineering and Computer Science
August 10. 2007

Certified by.........
Alan V. Oppenheim
MacVicar Faculty F o /Ford Professor of Engineering
Department of Electrical Engineering and Computer Science
Thesis Supervisor

Accepted by .......................
Arthur C. Smith
Professor of Electrical Engineering
Chairman, Department Committee on Graduate Theses

ARCHIVES
 Signal Approximation using the Bilinear Transform
by
Archana Venkataraman

Submitted to the Department of Electrical Engineering and Computer Science

on August 10, 2007, in partial fulfillment of the
requirements for the degree of
Masters of Engineering in Electrical Engineering and Computer Science

Abstract
This thesis explores the approximation properties of a unique basis expansion. The expan-
sion implements a nonlinear frequency warping between a continuous-time signal and its
discrete-time representation according to the bilinear transform. Since there is a one-to-
one mapping between the continuous-time and discrete-time frequency axes, the bilinear
representation avoids any frequency aliasing distortions.
We devote the first portion of this thesis to some theoretical properties of the bilinear
representation, including the analysis and synthesis networks as well as bounds on the basis
functions. These properties are crucial when we further analyze the bilinear approximation
performance. We also consider a modified version of the bilinear representation in which
the continuous-time signal is segmented using a short-duration window. This segmenta-
tion procedure affords greater time resolution and, in certain cases, improves the overall
approximation quality.
In the second portion of this thesis, we evaluate the approximation performance of the
bilinear representation in two different applications. The first is approximating instrumen-
tal music. We compare the bilinear representation to a discrete cosine transform based
approximation technique. The second application is computing the inner product of two
continuous-time signals for a binary detection problem. In this case, we compare the bilinear
representation with Nyquist sampling.

Thesis Supervisor: Alan V. Oppenheim

Title: MacVicar Faculty Fellow, Ford Professor of Engineering
Department of Electrical Engineering and Computer Science
Acknowledgments

I am greatly indebted to my research advisor, Professor Alan Oppenheim. His devotion

to and concern for students is unparalleled by anyone else I have come across. He has
continually pushed me to achieve my full potential, but at the same time, he has remained

a friendly and supportive mentor. His comments and ideas have helped to expand and

enrich my thesis work, and his patience and encouragement have made my first year in

graduate school an extremely positive one.

I would also like to thank Professor Vivek Goyal, head of the Signal Transformation and

Information Representation group at MIT. Not only was his class (Wavelets, Approximation

and Compression) instrumental in shaping my final MEng thesis, but his friendly one-on-one

discussions have been invaluable to my research work and experience.

I would like to acknowledge past and present members of the Digital Signal Process-

ing Group (DSPG): Tom Baran, Petros Boufounos, Sourav Dey, Zahi Karam, Al Khar-

bouch, Jon Paul Kitchens, Joonsung Lee, Charlie Rohrs, Melanie Rudoy, Joe Sikora, Eric

Strattman, Dennis Wei and Matthew Willsey. Over the past two years, these individuals

have sustained an exciting, collaborative and friendly research environment in which I could

continually challenge myself and broaden my horizons. I would especially like to thank Tom

and Sourav for their insightful conversations, Melanie for always being there when I needed

someone to chat with, Zahi for keeping my temper well-honed while somehow managing to

make me smile, and lastly, Eric for running the DSPG office smoothly and efficiently.

A special thanks to Austin Che, who has been my personal LaTex guru for the the past

three months. Without his help this document would have been infinitely more daunting

and tedious to compile. Thank you, also, for being my shoulder to lean on through all the

hard times here at MIT.

Finally, to my family: thank you for your love and support all these years. You were the

ones who first nurtured my curiosity and desire to learn. Your encouragement has played a

major role in all of my successes. At the same time, you have stood besides me through all
the turmoil and trials of my life thus far. Words cannot adequately express all that I owe

you, and I hope that one day, I can repay at least a fraction of all you have done for me.
To my family
Contents

1 Introduction 15
1.1 Signal Representation Using Basis Expansions . .......... ... . 15

1.1.1 Properties of an Orthonormal Signal Representation . ........ 16

1.1.2 Properties of a Biorthogonal Signal Representation .......... 17

1.2 The Necessity for Signal Approximation . .................. . 18

1.3 Signal Representation using the Bilinear Transform . ............. 18

1.4 Thesis Organization ................... ........... 19

2 The Bilinear Signal Representations 21

2.1 The Orthonormal Bilinear Signal Representation . .............. 21

2.1.1 The Orthonormal Basis Functions . .................. 22

2.1.2 Analysis and Synthesis Networks for the Orthonormal Expansion . . 22

2.2 The Biorthogonal Bilinear Signal Representation . .............. 24

2.2.1 The Primal and Dual Basis Functions . ................ 25

2.2.2 Analysis and Synthesis Networks for the Biorthogonal Expansion . 26

3 Properties of the Bilinear Signal Representations 31

3.1 The Orthonormal Representation . .................. ..... 31

3.1.1 Orthonormal Signal Space ................... ..... 32

3.1.2 Bounding the Basis Functions ................... ... 32

3.2 The Biorthogonal Representation ................... ..... 33
3.2.1 Weighted Energy Relationship ................... .. 33

3.2.2 Biorthogonal Signal Space . .................. ..... 34

3.2.3 Computing Inner Products using the Biorthogonal Expansion . . .. 36
3.2.4 Bounds on the Primal and Dual Basis Functions . .......... 37
4 MATLAB Implementation of Bilinear Analysis and Synthesis

4.1 The Analysis Network ...................... . .. ...

.. 39
4.2 The Synthesis Network . . ..................... . . . .. .
.. 40
4.3 Limitations of the Continuous Time Approximations . . . . . . . . . . . . . 40
4.3.1 The Analysis Network . ................. . ... ..
.. 41

4.3.2 The Synthesis Network ................. ....

. ... 41

5 Approximation Properties of the Bilinear Representations 43

5.1 Approximation of Signals . . ................... . . . . . 44
5.1.1 Linear vs. Nonlinear Approximation . . . . . . . ... . . . . . 44
5.1.2 Error Metrics......................... . . . .. . 45
5.1.3 Qualitative Measure of Approximation Performance . . . . . . . . . 46
5.2 Effect of the Parameter, a . . ................... . . . . 46

5.3 Exact Representation of a Signal using a Finite Number of I)T Expansion

Coefficients ............................

5.4 Deviating from an Exact Representation ............

5.5 Additional Signal Characteristics which Affect the Bilinear A pproximation

Performance ....... ..................... .. . . . . . . 54

5.5.1 Distribution of Energy over Time . ........... .. . . . . . . 54

5.5.2 Isolated Discontinuities ................. . ... . . . 57

5.5.3 Signal Decay Rate . ................... ... . . . .

. 60

5.6 Summary .... ............................ ..

... . 64

6 The Windowed Bilinear Representation

6.1 The W indow Function w(t) ............................

6.2 Relationship to Original Representation ....................

6.3 Approximation Using the Windowed Representation .............

6.3.1 Segmenting with a Rectangular Window ...............

6.3.2 Comparison of a Rectangular, Bartlett and Raised Cosine Window

7 Approximation of Audio Signals

7.1 Details of the Simulation Setup .........................

7.2 Simulation Results ................................
 7.3 Discussion .....................................

8 The Binary Detection Problem

8.1 The Nyquist Signal Representation .......................

8.1.1 Ideal (Unconstrained) Nyquist Detection . ...............

8.1.2 Constrained Nyquist Detection . ....................

8.2 Bilinear Matched Filtering - A Theoretical Analysis .............

8.2.1 The Orthonormal Representation . ...................

8.2.2 The Biorthogonal Representation . ...................

8.2.3 Variations of the Bilinear Representations . ..............

9 Matched Filtering Simulations 95

9.1 Details of the Simulation Setup . ....... . . . . . . . . . . . ... 95

9.2 Simulation Results ................................ 97

9.3 Discussion . ............................. . ...... 102

A The Family of Generalized Laguerre Polynomials 105

A.1 Definition ......................................... 105

A.2 Properties ..................... . ........ 106

A.2.1 Orthogonality .............. .. .. ........... 106

A.2.2 Recurrence Relation . ........ . . . . . . . . . . . . ... 107

A.2.3 Signal Space . ................... . . . . . . . . . .. 107

A.2.4 Bounds on the Generalized Laguerre Polynomials . . . . . . . . . . . 108

B Additional Properties of the Bilinear Representations 109

B.1 Representing Anti-causal Signals . . . . . . . . . . . . . . . 109

B.1.1 The Orthonormal Representation . . . . . . . . . . . . . . . 109

B.1.2 The Biorthogonal Representation . . . . . . . . . . . . . . . . 110

B.2 Conditions to Preserve Correlation . . . . . . . . .... . . . . . . 111

B.3 Noise Analysis for the Analysis and Synthesis Networks . . . . . . . . . . 112
B.3.1 The Low-pass Filter Stage . . . . . . . . . . . . . . . . . . . . 112

B.3.2 The All-pass Filter Stage . ............ . . . . . . . 114

B.3.3 Combined Effect of Input and Component Noise . . . . . . . . . . 115
List of Figures

2-1 A, (t) for different index values using a = 1

2-2 ,, (t) for different values of a using n = 5

2-3 Orthonormal analysis network to compute the expansion coefficients.....
2-4 Orthonormal synthesis network to reconstruct a continuous-time signal from

its bilinear expansion coefficients........

2-5 0, (t) for different index values using a = 1

2-6 ,, (t) for different values of a using n = 5

2-7 h,,(t) for different index values using a = 1

2-8 h,,(t) for different values of a using n = 5

2-9 Biorthogonal analysis network to compute the bilinear expansion coefficients.

2-10 Biorthogonal synthesis network to reconstruct a continuous-time signal from

its bilinear expansion coefficients. .........................

3-1 First-order cascade used to compute the coefficients bg[n]

9 for the biorthogonal
inner product ....................................

4-1 Orthonormal basis functions using a = 10 ...................

5-1 Orthonormal Bilinear Expansion Coefficients for f(t) - sin(10t) .......

5-2 Metrics of Linear and Nonlinear Approximation Performance when using the

Orthonormal Expansion for f(t) - sin(10t) ..................

5-3 Sorted Bilinear Coefficients for fk(t) tke - at ,
a = 20 . ............
5-4 Sorted Bilinear Coefficients for fk(t) , t 3e - kt ,
a = 20 . ............

5-5 Partial Sum Sequences for fk(t) t 3e - kt , a = 20 ...............

5-6 Sorted Bilinear Coefficients for fk(t) - e - at sin(kt), a = 20 . .........
5-7 Partial Sum Sequences for fk(t) _ e - at sin(kt), a = 20 ............
5-8 Sorted Bilinear Coefficients for Shifted sinc Functions, a=10 . . . . . . ... 58
5-9 Partial Sum Sequences for Shifted sinc Functions, a=10 . .......... 59
5-10 Original Signal (black) and its Bilinear Reconstructions (color) ....... 60
5-11 Sorted Coefficients and Partial Sum Sequences for Rectangular Pulse, a = 62.8 61

5-12 Sorted Bilinear Coefficients for fl(t) and f 2 (t) . ................ 62

5-13 Partial Sum Sequences for fl(t) and f 2 (t) . .................. 63

6-1 Segmenting the Original CT Signal using a Non-Overlapping Window ... 65

6-2 Rectangular, Bartlett and Hanning Windows to Segment a CT Signal ... 67
6-3 Original Continuous-Time Signal and its Shifted Version. . .......... 68

7-1 Normalized Error f(t) -- fir(t) for the Piano Sound Clip ..... . ...... 82
7-2 Normalized Error f(t) - fA (t) for the Violin Sound Clip . .......... 83
7-3 Reconstructed Audio Segment of a Piano Sound Clip . ............ 84
7-4 Reconstructed Audio Segment of a Violin Sound Clip ............. 85

8-1 Binary detection scenario. The transmitted signal r(t) is corrupted by AWGN.

The received signal x(t) consists either of noise or the signal s(t) plus noise. 87

9-1 Magnitude of S(jw) and Noise Power Spectrum with Sample Spacing T. .. 95
9-2 ROC Curves for sl(t); 5 DT Multiplies, a = 100 . ............... 98
9-3 ROC Curves for s2(t); 25 DT Multiplies, a = 100 . .......... . . 99
9-4 ROC Curves for s:3(t); 25 DT Multiplies, a = 100 . ......... ..... 100
9-5 ROC Curves for s 4 (t): 5 DT Multiplies, a = 62.8 . .............. 101

A-I L( (x) for different index values using a = 0 . ................ 106

)
A-2 L( (x) for different values of a using n = 5 . ................. 107

B-1 Orthonormal analysis network for anti-causal CT signals. . ........ . 110

B-2 Orthonormal synthesis network for anti-causal CT signals ........... 110
B-3 Biorthogonal analysis network for anti-causal CT signals. . .......... 111
B-4 Biorthogonal synthesis network for anti-causal CT signals ........... 111
B-5 Block and Implementation Diagram for the Low-pass Filter Stage. ...... 113
B-6 Block and Implementation Diagram for the All-pass Filter Stage. ....... 114
List of Tables

6
6.1 ON for f(t) ~ sinc(100(t - 0.5)) ................... ..... 70

6.2 6BiO for f(t) ~ sinc(100(t - 0.5)) ................... ..... 71

6.3 E when using a Rectangular window for f(t) - sinc(50(t - 0.5)) ....... 74

6.4 e when using a Bartlett triangular window for f(t) - sinc(50(t - 0.5)) . . . 75

6.5 E when using a Raised Cosine window for f(t) - sinc(50(t - 0.5)) ...... 76

7.1 Normalized Reconstruction Error for the Piano Sound Clip, a = 8820 . . .. 79

7.2 Normalized Reconstruction Error for the Violin Sound Clip, a = 14700 . . . 80
Chapter 1

Introduction

Recent technological advancements have allowed for a rapid growth in the speed and capabil-
ities of digital processors. These improvements have made it easier, and often more efficient,

to perform many signal processing operations in the digital domain. However, much of the
valuable real-world information encountered remains analog or continuous-time in nature.

1.1 Signal Representation Using Basis Expansions

We can circumvent the above dilemma by representing a continuous-time (CT) signal f(t)

using a basis expansion as follows:

f (t)= f [n]'n(t) (1.1)

The set of functions {'n(t)}n=_-o is a countable set of basis functions and the coefficients

f[n] are the expansion coefficients with respect to a chosen basis.

Since the basis functions are known a priori, the CT signal f(t) is completely specified

by the discrete-time (DT) sequence of expansion coefficients f[n]. Therefore, it can now be

manipulated using DT operations.

We assume that the reader is familiar with the following concepts related to basis expansions,
as they will be referenced throughout the remainder of this thesis:

We refer to analysis as the process of computing the expansion coefficients from the

original continuous-time signal and the given set of basis functions. Similarly, we refer
to synthesis as the process of reconstructing a continuous-time signal from its expansion
coefficients according to Equation (1.1).
Additionally, we define the standard inner product, in continuous and discrete time, as

(f,g)c = f(t)g(t)dt (1.2)

(f g)d = f[ng[in] (1.3)

We consider two signals to be orthogonal if their standard inner product is zero, and we
shall denote the signal energy as the standard inner product of a signal with itself.

In the following subsections we introduce two commonly-used types of basis expansions:

specifically the orthonormal and the biorthogonal signal representations. We assume that
all signals and all sequences have finite energy and are real-valued.

1.1.1 Properties of an Orthonormal Signal Representation

In an orthonormal expansion, the set of basis functions {An(t)},3=_. are chosen to satisfy
the condition

(An, Am)c = (1.4)

0, otherwise

In particular, Equation (1.4) indicates that the basis functions are mutually orthogonal to
each other and have unit energy.
One advantage of an orthonormal representation is that the expansion coefficients can
be obtained via an inner product with the respective basis function. Mathematically, if
f(t) = Em f[m]Am(t), then it follows from Equation (1.4) that

(f, An)c = f [m](Am, An)c = f [n] (1.5)

An orthonormial representation will also preserve the standard inner product between

continuous time and discrete time. Namely, if f(t) = E f[[n]An(t) and g(t) = 'n g[n]An(t),
then we have

(f, g)c = (f,9)d (1.6)

The above property is extremely useful in our consideration of matched filtering applications
since we can compute the inner product of two CT signals as the inner product of their

expansion coefficient sequences.

1.1.2 Properties of a Biorthogonal Signal Representation

In a biorthogonal expansion, the continuous-time signal is represented as

f(t)= f [n]n(t) (1.7)

n=-o

For convenience, we refer to the set of functions {On(t)}n=_ as the primal basis. Once

again fin] is the associated sequence of expansion coefficients.

The difference here is that we no longer constrain the primal basis functions On(t) to be

orthogonal. Instead, we define a dual basis {ha(t)}n__-c which satisfies the condition:

(hn(t), m (t)) = (1.8)

0, otherwise

The combined set of functions {((t), hn(t)}_. now constitutes a biorthogonal signal

representation.

There are some key differences between an orthonormal and a biorthogonal expansion,

which will have direct implications on the applications we consider. First, since the basis

functions n (t) are not necessarily orthogonal, the expansion coefficients f[n] cannot be

computed according to Equation (1.5). Rather, we use an inner product with the dual basis

functions hn(t). Mathematically, if f(t) = Em f [m]m(t), then

(f, hn)c = f[](¢m, hn)c = f[n] (1.9)

A potential problem with a biorthogonal representation is that it may be extremely difficult

to find and/or manipulate the dual basis.

A second consequence of the basis functions not being orthonormal is that the standard

inner product of two CT signals is no longer equal to the standard inner product of their

DT expansion coefficient sequences. Depending on the specific representation, it may be

fairly complicated to compute the expression in Equation (1.2).

Although these differences add a degree of complexity to implementing a biorthogonal

expansion, relaxing the orthonormality constraint allows for a much broader class of signal

representations. This opens the door to other desirable properties, which cannot be achieved

using an orthonormal expansion.

1.2 The Necessity for Signal Approximation

One major drawback to using a basis expansion given in Equation (1.1) is that most typical
CT signals will have an infinite number of non-zero expansion coefficients. However, due to

practical limitations such as processing time and hardware capability, we are restricted to
only a finite number of DT values.

If the basis functions remain unchanged, it is necessary to find a suitable approximation

to the original signal of the form

f(t) = S[r] n(t) (1.10)

nEIM

In Equation (1.10) IM represents a finite set of indices and f[n] is the modified expansion

coefficient sequence.

Unlike the case of perfect representation in Equation (1.1), the quality of the approxi-

mation f(t) is heavily dependent on the original signal, the set of basis functions, and the

particular application.

1.3 Signal Representation using the Bilinear Transform

This thesis is devoted to a specific basis expansion which maps the continuous-time S-plane
onto the discrete-time Z-plane according to the bilinear transform relationship shown in

Equation (1.11)
(1.11)
z =
a-s

By substituting z = e .i and s = jw into Equation (1.11), we see that the j-axis

(continuous time) is mapped onto the unit circle (discrete time). Furthermore, the CT and
DT frequency variables (lowercase w and uppercase Q respectively) are related through the
inverse tangent warping
S= 2 tan- () (1.12)
I1 'all
=1 (
 Effectively, the bilinear transform maps the entire range of continuous-time frequencies

onto the entire range of unique discrete-time frequencies. Since there is a one-to-one re-

lationship between the CT and DT frequency axes, this representation avoids frequency

aliasing distortions.

As we shall see, there exists both an orthonormal and a biorthogonal signal representa-

tion which achieve the desired frequency warping of Equation (1.12). In the remainder of

this thesis, we characterize the theoretical properties of these bilinear representations, focus-

ing on their relevance to signal approximation. We then evaluate the bilinear approximation

performance in an audio approximation and a binary detection application.

1.4 Thesis Organization

In Chapter 2 we introduce the orthonormal and the biorthogonal bilinear signal representa-

tions. We present the basis functions as well as the analysis and synthesis networks based

on a cascade of first-order analog systems. Chapter 3 examines important theoretical prop-

erties of each representation, including a signal-space characterization and bounds on the

various basis functions. These are useful when analyzing the approximation properties.
In Chapter 4 we summarize the approximations used to simulate CT processing in

MATLAB, including limitations to both the analysis and the synthesis methods. Chap-

ter 5 explores the bilinear approximation properties, including how they depend on the

parameter a in Equations (1.11) and (1.12) as well as characteristics of signals which are

well-approximated using the bilinear basis functions.

Chapter 6 introduces the Windowed Bilinear Representations in which the original CT

signal is segmented using a short-duration window. This representation affords greater time

resolution and, in certain cases, improves the overall approximation performance.

Chapter 7 presents simulation results when the orthonormal bilinear representation is

used to approximate instrumental music. A discrete cosine transform based representation

is used as a comparison metric.

Chapters 8 and 9 focus on using the bilinear representations to compute the standard
inner product of two CT signals for a binary detection problem. In Chapter 8 a theoretical

analysis is done for each representation and Chapter 9 presents the simulation results. The
bilinear expansions are compared with a Nyquist sampling based approach.
Chapter 2

The Bilinear Signal

Representations

This chapter introduces background material for two signal representations which imple-

ment a bilinear frequency warping between continuous and discrete time. Section 2.1 is

devoted to an orthonormal expansion while Section 2.2 focuses on the biorthogonal expan-

sion. Although much of the material is drawn from [12] and [8], the relevant expressions

and derivations are included for completeness.

2.1 The Orthonormal Bilinear Signal Representation

Let Fe(s) represent the Laplace transform of the CT signal f(t), and let Fd(Z) represent

the Z-transform of its DT expansion coefficient sequence f[n]. By extrapolating from the

ideas presented in [12] and [8], we consider the following frequency-domain relationship:

F/-(z) = a- (2.1)
z+1 z+1

Since the CT variable s has been replaced by the expression [a z , Equation (2.1) is

consisted with the bilinear transform relationship of Equation (1.11).

We can represent the right-hand sides of Equations (1.2) and (1.3) in the Laplace trans-
form and Z-transform domains, respectively. Then, by making the substitution of vari-

ables s = a (4) into the Laplace transform version of Equation (1.2), it follows that a
continuous-time to discrete-time relationship based on Equation (2.1) preserves the stan-
dard inner product between continuous and discrete time. This implies that the resulting

expansion is an orthonormal signal representation.

2.1.1 The Orthonormal Basis Functions

Similar to the technique employed in [12], the basis functions An(t) can be obtained by

substituting Equation (2.1) into the inverse Z-transform of Fd(z). Mathematically, this is

shown below:

f[n] =Fda>)1 dz
l=
273r Jt 1 "

n
1 a+s

- Fc(s)J2 a) ds (2.2)
2j j, a +s a- s

Using the relationship f f(t)g(t)dt = j f F(s)G(-s)ds, we can identify the

Laplace transform of the basis functions as

An(s) =a- (a (2.3)

As given in [12], the time-domain expression for An(t) is

An(t) =
f ,a(-1)n- -atLn-I(2at)u(t), for n > 1
(2.4)
Sv/a(-1)-neatL_,(-2at)u(-t), for n < 0

where Ln(x) is a zero-order Laguerre polynomial.

Note that Equation (2.4) is parameterized by both the index value n and the value of the

parameter a. Figures (2-1) and (2-2) illustrate the relationship between the basis function

behavior and each of the above variables.

2.1.2 Analysis and Synthesis Networks for the Orthonormal Expansion

For simplicity, in this section we present the analysis and synthesis networks appropriate

for causal continuous-time signals. Similar networks for anti-causal signals are derived in
Appendix B.

According to Equation (2.4), the basis functions are anti-causal for n < 0. This implies
 Basis Functions of the Orthonormal Representation with a = 1

1.4

1.2

0.8

L._, 0.4

0.2

-0.2

-0.4

-0.6
0 5 10 15 20 25
time, t

Figure 2-1: An (t) for different index values using a = 1

that (f, An)c = f[n] = 0 Vn < 0. Thus, for causal f(t), we will use only the positive-indexed

basis functions An(t).

We can construct an analysis and a synthesis network for this representation by exploit-

ing a recursive relationship satisfied by the Laplace transforms An(s). Specifically, An(s)

can be expressed in terms of An- 1 (s) according to

An(s) [=
a-] An-(s), for n > 2 (2.5)

where
v
AI (s) = (2.6)
a+s

Since f[n] is obtained via the inner product (f, An)c, the bilinear coefficients can be

computed using a cascade of first-order analog filters. The analysis network, similar to the

one first presented in [8], is depicted in Figure (2-3). The expansion coefficients are obtained
by sampling the output of each stage at time t = 0. Note that since a convolution is being

used to compute an inner product, the input is the time-reversed signal f(-t).

In addition, as seen from Equation (2.5), we can obtain successive basis functions by
exciting the first-order cascade shown in Figure (2-3) with a Dirac delta function, 6(t).
 Basis Functions of the Orthonormal Representation with index = 5
1

-c

- "1I Ij
5 10
time, t

Figure 2-2: A,(t) for different values of a using n = 5

=0

Figure 2-3: Orthonormal analysis network to compute the expansion coefficients.

To reconstruct a CT signal, the basis functions are scaled by the corresponding expansion

coefficient, and the result is summed. This synthesis network, similar to the one first

presented in [8], is shown in Figure (2-4).

2.2 The Biorthogonal Bilinear Signal Representation

Linear time-invariant (LTI) filtering in continuous time and linear shift-invariant (LSI)

filtering in discrete time are cornerstones of many signal processing applications. By relaxing
the orthonormality constraint, a biorthogonal bilinear expansion which maps a continuous-
time LTI system onto a discrete-time LSI system was developed in [8]. This section provides
an overview of the representation.
 6(t)

Figure 2-4: Orthonormal synthesis network to reconstruct a continuous-time signal from its
bilinear expansion coefficients.

2.2.1 The Primal and Dual Basis Functions

It is shown in [8] that a necessary and sufficient condition to achieve the LTI-+LSI mapping

is that the Laplace transforms of the primal basis functions have the following form:

Sn(s) = [~i(s)]n (2.7)

Moreover, the CT and DT complex frequency variables (s and z respectively) are related
by the function b1 (s) as follows:

z1-= ~(s) (2.8)

Given the bilinear transform relationship in Equation (1.11), we identify the primal basis

n(s)= () . As given in [8], the time-domain expression for 'n(t) is

2a(-1)n-1e-atL(~1(2at)u(t)+ (-1)nd(t), for n > 0

= 6(t),
6n(t) for n = 0 (2.9)

2a(-1)n-1eatL(') (-2at)u(-t)+ (-1)nb(t), for n < 0

where L~- 1 (x) = - Ln(X) is a first-order Laguerre polynomial.

We obtain the dual basis functions by noting that {¢n(t)} =- satisfies the following

weighted orthogonality condition:

o n. =m

-0
Stn(t)Om(t)dt = '
0, otherwise
(2.10)
Thus for n 0 0, the dual basis functions are hn (t) = 1t,(t)
The impulse response ho(t) is derived in [8] by constraining the CT signal f(t) to be

causal and bounded. The dual basis functions are then given by the following rational
Laplace transform expressions:

for n = 0
Ha(s) = a+s n-i (2.11)
2 for n> 0
1(a+s) as)

Figures (2-5) and (2-6) show plots of the primal basis functions (minus the impulse at

the origin) as the index and the value of a change, respectively. Figures (2-7) and (2-8)
depict the same relationships for the dual basis functions.

Basis Functions of the Orthonormal Representation with a = 1

,o
0
LL

cn

time, t

Figure 2-5: 0n(t) for different index values using a = 1

2.2.2 Analysis and Synthesis Networks for the Biorthogonal Expansion

Once again we present the analysis and synthesis networks for causal continuous-time sig-
nals. The networks for anti-causal signals are derived in Appendix B.
From Equation (2.11), the dual basis functions satisfy a recursive relationship similar to

Equation (2.5). Therefore, we can compute the sequence f[n] using a cascade of first-order
systems as shown in Figure (2-9). Again, the network input is the time-reversed signal
 Basis Functions of the Orthonormal Representation with index = 5

c
.2
0
C
U-

u,
cn
Cu

0 5 10 15
time, t

Figure 2-6: n(t) for different values of a using n = 5

f(-t), and the outputs along the chain are sampled at time t = 0 to obtain successive
expansion coefficients.
We reconstruct the CT signal using the primal basis functions In(s). Since these func-

tions also have a recursive structure, the synthesis network is similar to that of the or-

thonormal representation, as depicted in Figure (2-10).

 Dual Basis Functions of the Orthonormal Representation with a = 1

0.4

0.2

-0.2

-0.4

-0.6

-0.8

-1I
0 5 10 15
time, t

Figure 2-7: hn (t) for different index values using a = 1

Dual Basis Functions of the Orthonormal Representation with index = 5

0 5 10
time, t

Figure 2-8: hn(t) for different values of a using n = 5

 f(-t).
=0

f[0j f[1] f[2] f[3] f[4]

Figure 2-9: Biorthogonal analysis network to compute the bilinear expansion coefficients.

6(t) ..

f(t)
Figure 2-10: Biorthogonal synthesis network to reconstruct a continuous-time signal from
its bilinear expansion coefficients.
Chapter 3

Properties of the Bilinear Signal

Representations

In this chapter we derive some key properties of the two bilinear expansions. These will

be useful in understanding the approximation behavior of these representations. Since this

thesis concentrates on real-valued, causal and bounded CT signals, we specialize our analysis

to the subset of orthonormal basis functions {An(t)}I 1 and the subset of primal/dual

biorthogonal basis functions {(n (t), hn,(t) ~.

Section 3.1 focuses on properties of the orthonormal expansion, and Section 3.2 covers

the biorthogonal expansion. Much of the information in this chapter is drawn from proper-

ties of the generalized Laguerre polynomials. These can be found in Appendix A. Finally,

additional properties of the bilinear representations, including a preliminary noise analysis

of the first-order cascades, can be found in Appendix B.

3.1 The Orthonormal Representation

In this section, we present two important properties of the orthonormal bilinear signal

representation, specifically a characterization of the orthonormal signal space and bounds

on the orthonormal basis functions, A,(t).
3.1.1 Orthonormal Signal Space

As stated in Chapter 2, the orthonormal representation preserves the standard inner product

between continuous and discrete time. This implies the following energy relationship:

f 2 (t)dt = f 2 [n] (3.1)

n=l

By considering the M-term approximation fMl(t) = f[n]An(t), it follows from

Equation (3.1) that the energy of the error el (t) = f(t) - fsf(t) is

Jf)0 e2(t)dt = f 2[n]

- f 2 (t)dt- Ef2[T] (3.2)

ri=l

Since the first term in the right-hand side of Equation (3.2) is finite and f 2 [n] is always

non-negative, as AlM - the energy of the error goes to zero. Thus the set of orthonorinal

basis functions {An(t) }, o spans the space of causal, finite energy signals. We shall denote

this signal space as L2 (O,0o0).

3.1.2 Bounding the Basis Functions

From Equation (2.4) the expression for An(t), n > 0 is

An(t) = V/ (-1)-le-atL,_l(2at)u(t) (3.3)

As developed in Appendix A, we can rewrite Equation (3.3) in terms of the normalized

zero-order Laguerre polynomial with argument x = 2at.

An(t) = i2a(-1)n-1(2at) (3.4)

By applying Equation (A.7) and using v = 4(n - 1) - 2 = 4n - 6, we can bound the

normalized Laguerre polynomial (n-1(2at) according to:

0<t< 1
-- 2a(4n-6)
1 t < 2n-3
(2at(4n - 6))-1/4 2a(4n-6) - - 2a
|Kn-1(2at) C 2n-3 <t<3
(4n- 6) - 1 /4 [(4n - 6)1/3 + 4n - 6 - 2atl] -1/4 n-

e-3at

(3.5)

Combining Equations (3.4) with Equation (3.5), the orthonormal basis functions are

bounded by a similar expression with a different constant factor in front. Specifically

0<t< 1
1, 0< 2a(4n-6)
< t < 2n-3
(2at(4n - 6))-1/4 2a(4n-6) - - 2a
IAn(t)l < D 2n-3 < t <3 2n-3
(4n - 6)-/4 [(4n - 6)1/3 + 14n - 6 - 2atl -1/4 2a 32a

e-t. t3(2 7j)

(3.6)

Although cumbersome, Equation (3.6) will play a role in Chapter 5 when we analyze the

approximation behavior of the bilinear representations.

3.2 The Biorthogonal Representation

In a parallel fashion to Section 3.1, we now explore important properties of the biorthogonal

bilinear representation.

3.2.1 Weighted Energy Relationship

As stated previously, the biorthogonal signal representation does not preserve the standard

inner product between continuous and discrete time. Consequently, instead of satisfying

Equation (3.1), we have a weighted energy relationship between the CT signal and its DT

expansion coefficients. By applying Equation (2.10), this can be written

] tf2(t)dt
n=O
nf 2 [n] (3.7)

The weighting functions are given by t, in continuous time, and n, in discrete time. Equa-
tion (3.7) will be useful when characterizing the signal space of the primal basis functions.
3.2.2 Biorthogonal Signal Space

As discussed in Section 2.2, the primal basis functions are given in terms of the first-order

Laguerre polynomials. From Appendix A, the polynomial set {L (1) (x)}: o forms a basis
for causal functions f(t) satisfying

If (I)12 xe-:dx < oc (3.8)

.0

Let us now define L 2 (0, -c; t) to be the space of functions f(t) which satisfy

2
0 tlf(t)| dt < oc
It follows from Equations (2.9) and (3.8) that the set of primal basis functions {f,,(t)}~
) 0

forms a basis for L 2 (0, C; t).

To see this, assume there exists f(t) E L 2 (0, c: t) orthogonal to <n(t) Vn > 0. This implies

that all its expansion coefficients f[n] are zero. Equivalently

e-at f(t)dt = 0 (3.9)

JX
0

= Ofor n > 0 (3.10)

Stn(t) f(t)dt
on

Notice that since t - 6(t) = 0, we do not need to consider the impulses in Equation (2.9).

Because {hn(t) }no consists of polynomials weighted by a decaying exponential, there

must be a linear combination which yields

Sanhn(t) = e-attn Vn > 0 (3.11)

k=O

From Equations (3.9) and (3.10), it follows that fo e-attn f(t)dt = 0 Vn > 0.

Now consider the Laplace transform G(s) = fo e-te-attf(t)dt. The magnitude of G(s)
can be bounded as follows:

IG(s)| =
0 e-ste-attf(t)dt

S }te-att f(t) dt
0oeR-R

* [j0 e-2(Rs}+a)ttdt] 1L/ tft)Ifdt 2 (3.12)

where the final expression is obtained using the Cauchy-Schwartz inequality.

Since the second term in Equation (3.12) is finite, G(s) is well-defined and analytic in
the half-plane f{s} > -a. Thus, G(s) can be represented in this region using a Taylor
expansion about the point s = 0

G(s)= : G(s)|s=o n! (3.13)

n=0

However, notice that

d G(s)1s=o = (-1)n e- attn+lf(t) = 0 Vn > 0

Therefore, G(s) = 0 --+ f(t) = 0

One way to verify the above signal space characterization is by considering the weighted
energy of the error between the signal f(t) and its M-term approximation as shown

t e2 (t)dt = n, 2[n]
0n=M+1
Oco M
= t. f2(t)dt nf2[n] (3.14)
n=1

Once again, the first term in the right-hand side of Equation (3.14) is finite and nf 2[n] is
non-negative for n > 0. Thus, the bilinear representation should converge to the true CT
signal in the sense that the weighted energy of the error goes to zero as M - oc.
Finally, we remark that although the biorthogonal bilinear representation does not span
the entire space L 2 (0, 00), most real-world analog signals are bounded with finite duration.
These signals lie within L2(O, oc; t) and can therefore be represented using the biorthogonal

bilinear expansion.

3.2.3 Computing Inner Products using the Biorthogonal Expansion

Although the standard inner product of two signals is not preserved, we can use the biorthog-

onal representation f (t) = 'o f [n]On(t) to manipulate the CT inner product expression
according to

f(t)g(t)dt = f[n](t) g(t)dt

= [n] g(t) (t)) (3.15)

71=0 "

The expression o' g(t)On(t) in Equation (3.15) is a scalar coefficient. Therefore, the

CT inner product of two signals can be expressed using the expansion coefficients f [n] and a
new set of coefficients which we refer to as bg [n] (the subscript denotes the corresponding CT

signal). Essentially, the signal g(t) is expanded with respect to the dual basis, {hn(t)} .o,
and the coefficients bg [n] are the expansion coefficients of this new representation.

Since the Laplace transforms of successive primal basis functions 4,(s) = (a) differ
by an all-pass filter, the coefficient sequence b9 [n] can be computed by sampling the outputs

of a first-order cascade similar to the ones in Figures (2-3) and (2-9). This network is shown

in Figure (3-1).

g(-t)-

Figure 3-1: First-order cascade used to compute the coefficients bg [n] for the biorthogonal
inner product.
3.2.4 Bounds on the Primal and Dual Basis Functions

Let Ob,n(t) denote the bounded portion of the primal basis functions. For n > 0, we can
express this quantity as follows:

Ob.n(t) = 2a(-1)n- -atL(l (2at)u(t) (3.16)

From Chapter 2, the corresponding dual basis functions are hn(t) = t bn(t)

Using Equation (A.6) we can rewrite Ob,n(t) and h,(t) in terms of the normalized first-
order Laguerre polynomial with argument x = 2at.

t n (- 1)n-1(l) 1 (2at) (3.17)

Cb,n (t)

hn(t) = (_1)n-1((2at) (3.18)

By applying Equation (A.7) and using v = 4(n-1), the normalized Laguerre polynomial

(1 (2at) is bounded according to

(8at(n - 1))1/2. O<t< 1

-- 8a(n-1)
/ ,
(8at(n -
4l'-
1 (2at)l < c - 1
1 <t< a(-1

(4n - 4) /4 [(4n - 4)1/3 + 14n - 4 - 2atl] -1/4 a a

- 3
e t
(3.19)

Combining Equations (3.19) with Equations (3.17) and (3.18), we have

4a n-i 1
SO<t< 8-a(n-1)
San2
1 <t< n-1
1/4 8a(n-1) - - a
Ib,n(t)l _ c 3 1/4 n-I < t < 3
2t (n-1)
[(4n - 4)1/3 + 14n - 4 - 2atl] -1/4
-
e- sV2
t
(3.20)
 4at
4at - 0 <t< 1
- 8a(n-1)

at ) 1/4 1 <t< n-I

(2n2(n-1) ) 8a(n-1) - - a
Ihn(t)l < C / _zr> \
n2(n- ,
[(4n - 4)3 + 14n - 4 - 2at ] -1/4 n-1 <t<3 n-

e-3t 2at

(3.21)
Once again these bounds will be useful when analyzing the approximation properties of the

bilinear representations.
Chapter 4

MATLAB Implementation of
Bilinear Analysis and Synthesis

The bilinear representations are simulated numerically using MATLAB. Since MATLAB
operates on discrete-time vectors rather than continuous-time signals, it is important to
consider the analysis and synthesis network implementations. This chapter provides an
overview of the techniques used to approximately implement these systems as well as some
of the limitations.

4.1 The Analysis Network

In order to compute the bilinear expansion coefficients according to Figures (2-3) and (2-9),
we use the trapezoidal rule for integration. This approximation allows us to represent a CT
derivative using a DT first-difference operation.

For the two bilinear expansions, we implement this technique on a single-pole system
with transfer function and input/output differential equation shown below:

H(s) = A (4.1)
a+s

-y(t) = :(t) = Ax(t) - ay(t) (4.2)

By integrating Equation (4.2) over an interval of length T and using the trapezoidal
rule for integration, specifically,

f(r)d - [f(nT) + f(nT - T)] (4.3)

nT-T 2

we can write the samples of the output signal, y(nrT) in terms of the input samples x(rnT)
according to the expression

y (nnT) + x(nT - T)] + [1 - aT/2] y(nT - T) (4.4)

1+ aT/2 21
Thus, by transforming the differential equation of Equation (4.2) into an integral equa-
tion, we can approximate the sampled output of a first-order CT filter using only discrete-
time operations. In addition, by decomposing the all-pass filters as

-. - 2a- 1 (4.5)
a+s a+s

Equation (4.4) can be used for all of the first-order systems in Figures (2-3) and (2-9).

4.2 The Synthesis Network

Ideally, we would like to reconstruct a CT signal from its bilinear expansion coefficients. This
is necessary when analyzing both the bilinear approximation properties and the associated
approximation errors. However, since MATLAB operates on DT vectors, all "continuous-
time" signals must really be vectors of very finely-spaced time samples. Therefore, in
evaluating the re-synthesis, we reconstruct samples of the bilinear expansion and compare
it to samples of the original vector.
We obtain samples of the reconstructed signal by relying on the MATLAB function
impulse to generate samples of A (t) and On(t). However, since impulse cannot represent
the Dirac delta functions in n,(t), the biorthogonal reconstruction is inaccurate at one
point, namely t = 0.

4.3 Limitations of the Continuous Time Approximations

While convenient for numerical simulation, there are limitations to the approximations
discussed above. These issues are somewhat different for the analysis and the synthesis
networks due to the techniques used in each case.

4.3.1 The Analysis Network

For the analysis network, the trapezoidal approximation seems empirically reasonable only
when the CT signal is sampled above 10 times its Nyquist rate. This means that the vectors
of sampled CT signals grow very rapidly in size as the signal duration increases, and large
amounts of computation may be required to calculate the expansion coefficients.
Additionally, the trapezoidal rule for integration introduces errors into the bilinear co-
efficients. For comparison, the coefficients obtained when using symbolic integration to
evaluate the inner products differ slightly from those obtained numerically using MATLAB.
Although the accuracy may be improved by using a higher-order derivative approxima-
tion, it is unclear whether the additional improvement will justify the inevitable increase in
computational complexity.

4.3.2 The Synthesis Network

The main limitation of the synthesis network is that MATLAB does not have enough dy-
namic range to represent the time samples of high-index basis functions. This can be seen
by considering the form of the bilinear basis functions (minus the Dirac impulse in the
biorthogonal case)

n-1
A (t) = e-at E ak(2at)k (4.6)
k=O
n-I
n(t) = e- at
3k(2at)k (4.7)
k=O

As the index increases, the value of the leading monomial terms in Equations (4.6) and
(4.7) increase exponentially. At some point, these values cannot be represented in the given
dynamic range. This means that the reconstructions of the basis functions and the CT
signal will not be accurate. This effect is illustrated by plotting the basis function An(t) for
two different indices.
As seen in Figure (4-1), the lower-index basis function (blue) is smooth and well-behaved.
This is very similar to the behavior shown in Figures (2-1) and (2-2). Conversely, the
high-index basis function (green) exhibits violent oscillations, which are uncharacteristic of
 Basis Functions of the Orthonormal Representation with a = 10
1

-0

-1
Sz 0

time,t

Figure 4-1: Orthonormal basis functions using a = 10

either a polynomial or a decaying exponential. The behavior suggests that the green curve

is exceeding the allotted dynamic range, and is therefore being clipped.

While it may be possible to mitigate this effect by changing the order in which polynomi-
als are evaluated and summed, the dynamic range will always be a fundamental limitation

of using digital processing for polynomial sums.

Although comparatively less important, another problem with the synthesis network is

that obtaining samples of the inverse transform is very computationally expensive. This

becomes especially noticeable for high-order transfer functions.

Chapter 5

Approximation Properties of the

Bilinear Representations

We introduced the technique of signal approximation in Chapter 1 and presented a brief

argument as to why it is necessary in several real-world applications. Chapters 2 and 3 have

focused on a specific expansion which implements a bilinear frequency warping relationship.

For the remainder of this thesis, we turn our attention to the approximation properties of
these bilinear basis functions.

In this chapter we study the approximation problem from a more theoretical perspective.

We observe the approximation behavior for specific analytical signals and draw conclusions

about signal characteristics which can be well-approximated with the bilinear functions.

Chapter 6 examines an alternative representation in which the original CT signal is seg-

mented using a short-duration window. The final three chapters are devoted to signal

approximation applications.

Section 5.1 of this chapter introduces some basic concepts of signal approximation. In

Section 5.2 we investigate the effect of the the parameter a on the approximation perfor-
mance. Sections 5.3 and 5.4 focus on classes of signals which are well-approximated using
the bilinear basis functions, and Section 5.5 considers some general signal characteristics

that impact the approximation performance. In Section 5.6 we summarize our findings as
well as compare the orthonormal and biorthogonal representations.
5.1 Approximation of Signals

Equation (1.10) gives a general expression for the approximation f(t) in which the expansion
coefficients may be modified based on the index set I[. In this thesis, however, we con-
sider only approximations for which the expansion coefficient sequence remains unchanged.

Namely, the Al-term approximation of f(t) = r-o,_ f[nn] n(t) is written

fii(t) = f[nl] 'n(t) (5.1)

•n E IM

where IM denotes a set of ill index values, and f[n] is the original n th expansion coefficient.

The advantage of using Equation (5.1) is that we do not need to perform additional com-

putations to obtain modified coefficients for the signal approximation. This method will be
used for both the orthonorimal and the biorthogonal representations.

The material in this section is drawn from Chapter 9 of Mallat [7]. Although much

of the information has been developed for orthonormal signal representations, we make

generalizations to the biorthogonal case.

5.1.1 Linear vs. Nonlinear Approximation

In this discussion, we consider both linear and nonlinear approximations, as defined below.

Linear Approximation refers to having a fixed index set Il, irrespective of the CT

signal being approximated. Given the bilinear analysis and synthesis networks shown in

Chapter 2, a reasonable linear approximation scheme is to retain the first AI terms of the
expansion. This is because obtaining successive basis functions and/or expansion coefficients

corresponds to adding first-order analog filters to the cascade. Therefore, selecting the first

MA terms minimizes the hardware requirements of the system.

Nonlinear Approximation refers to selecting the index set I/j based on the characteristics

of f(t). Because the indices vary from signal to signal, the approximation of f (t)+ f(t) does
not necessarily equal the sum of individual approximations, fi (t)+ f 2 (t). However, nonlinear
approximation always has the potential to perform better than linear approximation because
the latter is a subset of the former.
5.1.2 Error Metrics

Given the original CT signal f(t) and its M-term approximation fAf(t), we will consider

the following error metrics:

2
E[M] (f(t) 1 1(t)) dt (5.2)

6[M] = f2(t)dt - f 2(t)dt (5.3)

e[Ml] is the energy of the difference, also termed the reconstruction error, between the origi-

nal and the approximated signals. In contrast, 6[MJ is the difference in energy between the

two signals. As discussed in Chapters 7-9, E[M] is our chosen error metric when approxi-

mating audio signals whereas 6[M] is more significant in the binary detection problem.

For the orthonormal bilinear representation f(t) = " I f[n]An(t), it is straightforward to

show that

F[M] = 6[M] = 1 f 2[n] (5.4)

nl IM

Therefore, to minimize E[M] and 6[M] in this case, we should select the indices TIw corre-

sponding to the largest coefficient magnitudes If[ n] in the expansion.

For the biorthogonal bilinear representation f(t) = f[n]¢n(t), there is no straight-
fo
forward way of minimizing the quantity E[M] short of computing the reconstruction error

for every M-term subset of expansion coefficients. However, we suggest below a simple
procedure to minimize the quantity 6[M].

With bf [n] denoting the sequence of secondary expansion coefficients from Figure (3-1),

we define the inner product sequence as follows:

IP[n] = f [nlbf[n], for 0 < n < oc (5.5)

Note that the signal energy is fo f 2 (t)dt = C o IP[n].

By drawing from the orthonormal case, we obtain the nonlinear approximation set IM by
selecting the (same) indices of both f [n] and bf [n] which correspond to the maximum values
in IP[n]. While this may not be the theoretically optimal way to minimize Equation (5.3),

it is a practical solution because we do not need to modify our coefficient sequences, and

we do not need to combinatorially search over all possible M-term approximations.

 One repercussion of the proposed method is that minimizing 6[AI] is not equivalent

to minimizing [MI] in the biorthogonal case. Therefore, we may end up with a large
reconstruction error using the set I1 1 from above. To avoid such a situation, we will consider

only the orthonormal representation for audio approximation.

5.1.3 Qualitative Measure of Approximation Performance

We can combine the material presented in Chapter 9 of [7] with that of the previous section
to determine a metric for evaluating the bilinear approximation perf6rmance. Throughout

this section, we use fr [n] to denote the sequence of orthonormal expansion coefficients

when sorted (in descending order) by absolute value, and IP,[n] to denote the sorted inner
product coefficients from Equation (5.5).
Theorem 9.4 in [7] relates the decay of Ifr[n]l with the decay of the Al-term approxima-
- 2s
tion error e[M]. It states that if fr,[nl < Cn-", then e[Al] _ 21
9 . The author's
main conclusion is that the orthonormal nonlinear approximation performance depends

heavily on the decay of sorted expansion coefficients, with a faster decay corresponding to
a smaller Ml-term error.

We can extend this reasoning to the biorthogonal case by considering the decay of IP[n].
Thus, we qualitatively compare the orthonormal and biorthogonal nonlinear approximation
performances by observing how quickly the sequences f~ [n] and IP,[n] decayv.
Additionally, we can evaluate the M-term linear approximations by plotting the partial

sum sequences

M
Sov[AI] = f [n] (5.6)
n=l
Al-1
SBio[AI] = 1 IP[n] (5.7)
n=O

and by observing how quickly they approach the original signal energy.

5.2 Effect of the Parameter, a

Figures (2-2), (2-6) and (2-8) indicate that the basis function behavior depends on the
parameter a. Predictably, this has a direct impact on the approximation performance of
both the orthonormal and the biorthogonal expansions. In this section we evaluate how the

choice of a affects, in particular, the growth of partial sums SoN[M] and SBio[M], as well

as the decay of sorted coefficient f,[n] and IPr[n].

For simplicity, we analyze the bilinear approximations for the windowed sinusoid f(t)

sin(10t), for 0 < t < 1, when using different values of a, and generalize from this informa-
tion. The signal f(t) is normalized to have unit energy.

Figure (5-1) shows the orthonormal bilinear coefficients for a = 1, 10, 100, and 1000 respec-

tively. Figure (5-2) shows the corresponding sorted coefficients and partial sum sequence.

The biorthogonal expansion exhibits a similar behavior.

0.4

S0.3
m
0.2

-3
0
1 0.1
0
8e
.F
03

0 F -0.1

-0.2

S-0.3

-0.4
_n00 0 0 0 4

300 400 500 0 100 200 300 400 500

0 100 200
Coefficient Index, n Coefficient Index, n

(a) a = 1 (b) a = 10

.2 .2
0Y
0

2
0 2
0

Coefficient Index, n

(c) a = 100 (d)a = 1000

Figure 5-1: Orthonormal Bilinear Expansion Coefficients for f(t) - sin(10t)

 Sorted Orthonormal Bilinear Coefficients of f(t) - sin(10t)

• 0.18

> 0.16

& 0.14
0
0o 0.12
ca
0.1

E 0.08
o

- 0.06
0
P 0.04
o
0.02

0
10 20 30 40 50 60 70 80 90 100
Coefficient Index, n

(a) Sorted Orthonormal Expansion Coefficients

Partial Sum of Orthonormal Expansion Coefficients for f(t) - sin(l Ot)

50 100 150 200 250 300 350 400 450 500

Coefficient Index, n

(b) Orthonormal Partial Sum Sequence

Figure 5-2: Metrics of Linear and Nonlinear Approximation Performance when using the
Orthonormal Expansion for f(t) - sin(10t)
 As seen, both the fastest decay and the fastest growth in Figure (5-2) occur when a = 10,

or the carrier frequency of the original signal f(t). We can argue this result by looking at

the group delay of the all-pass filters in the bilinear first-order cascades.

d (a - jw1 2a (5.8)
dw a +jw) a2 w 2

Consider a finite-duration signal whose energy is tightly concentrated around a center

frequency of wo. For a narrow-band signal, the effect of an all-pass filter can be roughly

approximated by a time delay of 'g(Wo). However, since the signal has finite duration, suc-
cessive delays of Tg(wo) will eventually shift the majority of its energy beyond the sampling

time, t = 0. After this point, the remaining expansion coefficients are approximately zero.

Therefore, in order to minimize the number of significant DT coefficients, we should

maximize the group delay, since this is equivalent to minimizing the number of stages

before the signal has been shifted beyond the time t = 0. Equation (5.8) is maximized
when a = w , , meaning this value should yield the fastest coefficient decay.

Although many of the signals we would like to approximate may not be narrow-band,

the above analysis suggests a reasonable way of initializing the value of a. Namely, we can

employ the following 'maximin' strategy:

For a signal with frequency content effectively band-limited to WM, choose a = WJI.

Because the group delay in Equation (5.8) is monotonically decreasing in w, we are

guaranteed a group delay greater than or equal to -rg(WM) for all frequencies in the range

of interest. Once the parameter a has been initialized, we may be able to adjust its value,

based on the CT signal f(t), to achieve better approximation performance.

5.3 Exact Representation of a Signal using a Finite Number

of DT Expansion Coefficients

As developed in Chapter 2 the bilinear basis functions have Laplace transforms An (s) =

a+s - and n"(s) = (s. From these expressions, it is easy to verify that

signals which can be exactly represented using a finite number of expansion coefficients
have rational Laplace transforms with all poles located at s = -a.
 Consider Al-term linear approximation discussed in Section 5.1. A CT signal within the

span of the first Al basis functions has a rational Laplace transform with at most MA poles

at s = -a and at most (MA- 1) zeros elsewhere in the S-plane. In the time domain, this
corresponds to functions of the form

f(t) = ( -C, tn e-atu(t) (5.9)

n=0

To illustrate the above property, Figure (5-3) shows the sorted orthonormal expansion

coefficients and the sorted biorthogonal inner product coefficients for the functions sk(t) -

tke - at for k = 1....5. Each Sk(t) is normalized to have unit energy. As seen, a function
with a kth order monomial term has at most k +1 non-zero (i.e. significant) coefficients.

If we are now allowed to select any Al expansion coefficients, the set of exactly-represented

signals is still restricted to rational Laplace transforms with all poles located at s = -a.

However, there is now a constraint on the zero locations. Specifically, if I1 represents the

set of retained indices, then we can only represent signals which are linear combinations of

the lM Laguerre polynomials weighted by a decaying exponential. Thus, we do not gain

much more intuition by looking at a nonlinear approximation.

5.4 Deviating from an Exact Representation

We conjecture that a signal which has a rational Laplace transform with pole location(s)

near the value s = -- a should be well-represented using a finite number of bilinear basis

functions. However, the approximation performance should worsen as the poles move farther

away from s = -a.

In this section, we use two methods of gradually altering the pole locations. First, we

change the exponential decay of the signal. This is equivalent to sliding the poles along the

real axis. Second, we modulate the signal with sinusoids of varying frequencies. This moves

the (conjugate) poles perpendicular to the real axis.

3 -
Consider signals of the form fk(t) _ t e kt, normalized to have unit energy. Figures (5-4)

and (5-5) show the sorted bilinear coefficients and the partial sum sequences, respectively,

as the exponential decay k is varied. A parameter value of a = 50 is used to compute all

bilinear expansions.
 Sorted Orthonormal Expansion Coefficients of f(t) ~ tk*exp(-at)

(DJ

0
-a
CL

X
0
0
C
0
(=
0
0
a)
LU
0
C
0O
0
0

t
0,

7
Coefficient Index, n

(a) Orthonormal Expansion Coefficients

Sorted Biorthogonal IP Coefficients of f(t) - tk*exp(-at)

0.

0.

- 0.
00.
0

0
0

0.

= 0.

0.

1 2 3 4 5 6 (
Coefficient Index, n

(b) Biorthogonal Inner Product Coefficients

Figure 5-3: Sorted Bilinear Coefficients for fk(t) tke - at, a = 20

 Sorted Orthonormal Bilinear Coefficients of f(t) - t3 exp(-kt)
Ie
U.

o 0.4

> 0.
a
. 0.3
o0

0.

= 0.

0
0.1
0
" 0.
0
o
a 0.0

2 4 6 8 10 12 14 16 18 20
Coefficient Index, n

(a) Orthonormal Expansion Coefficients

Sorted Biorthogonal Bilinear IP Coefficients of f(t) - t3 exp(-kt)

C
a

o
o
C-)

CEu

a
(1 0
rO
o
._
.GC

0
iz

o
0)
.m
0

0
"o
cO

2 4 6 8 10 12 14 16 18 20
Coefficient Index, n

(b) Biorthogonal Inner Product Coefficients

Figure 5-4: Sorted Bilinear Coefficients for fk(t) t3 e - kt, a = 20

 3
Partial Sum of Orthonormal Expansion Coefficients for f(t) - t exp(-kt)

Coefficient Index, n

(a) Orthonormal Expansion Coefficients

3
Partial Sum of Biorthogonal IP Coefficients for f(t) - t exp(-kt)

2 4 6 8 10 12 14 16 18 20
Coefficient Index, n

(b) Biorthogonal Inner Product Coefficients

3 - kt
Figure 5-5: Partial Sum Sequences for fk(t) , t e , a = 20
 As seen, the best approximation performance occurs when k = a = 50, corresponding

to the exact representation case. Notice, however, that the approximation performance is

worse for k < a than for k > a. This effect will be discussed further in Section 5.5.

Now consider signals of the form fk (t) - e- at sin(kt). once again, normalized to have unit

energy. Figures (5-6) and (5-7) show the sorted bilinear coefficients and the partial sum

sequences, respectively, as the carrier frequency k is varied. A parameter value of a = 20 is

used to compute all bilinear expansions.

As seen, the approximation performance is best for k = 5 and worsens fairly rapidly for

k > 20. This behavior ties back to the group delay argument from the previous section. As

the modulating frequency increases, the corresponding group delay decreases - -. This

implies that there should be more and more significant terms in the expansion as k increases.

5.5 Additional Signal Characteristics which Affect the Bilin-

ear Approximation Performance

In the preceding section we investigated signals with rational Laplace transforms, focusing

on how the pole location affects the approximation. In this section we look at more general

signal properties which play a large role in the bilinear approximation performance. These

include the energy distribution over time, isolated discontinuities, and the signal decay rate.

5.5.1 Distribution of Energy over Time

In Section 5.4 we observed that, relative to the value of a, the approximation performance

is worse for slower exponential decays, than for faster ones. We can extend this observation

to a more general class of signals by examining the energy distribution over time. To this

end, consider signals of the following form:

(5.10)
fn(t)fnM~ sinc(10(t - n)), -t < 1 (5.10)
0. otherwise

Again, the signal fn(t) has been normalized for unit energy. A windowed sinc pulse is
chosen for this experiment because it does not have a rational Laplace transform, but it
does have a large energy concentration around its main lobe.
 Sorted Orthonormal Bilinear Coefficients of f(t) - exp(-20t)*sin(kt)

(d

0D

0
C

0
0
"c
t
0

~0
t0
C'

2 4 6 8 10 12 14
Coefficient Index, n

(a) Orthonormal Expansion Coefficients

Sorted Biorthogonal Bilinear IP Coefficients of f(t) - exp(-20t)*sin(kt)

L0
0
._2
o
0

O
a.
.

0
0)
-C

.2
m
0

2 4 6 8 10 12 14
Coefficient Index, n

(b) Biorthogonal Inner Product Coefficients

-a
Figure 5-6: Sorted Bilinear Coefficients for fk(t) - e tsin(kt), a = 20
 Partial Sum of Orthonormal Expansion Coefficients for f(t) - exp(-20t)*sin(kt)

0.

0.

0.

0.

0.

0.

0.

0.

0.

2 4 6 8 10 12 14
Coefficient Index, n

(a) Orthonormal Expansion Coefficients

Partial Sum of Biorthogonal IP Coefficients for f(t) - exp(-20t)*sin(kt)

2 4 6 8 10 12
Coefficient Index, n

(b) Biorthogonal Inner Product Coefficients

Figure 5-7: Partial Sum Sequences for fk(t) - at

sin(kt), a = 20
 Figures (5-8) and (5-9) depict the sorted bilinear coefficients and the partial sum se-

quences, respectively. A value of a = 10 is used for all bilinear expansions.

Clearly, as the energy concentration moves farther from the time origin, the approxima-

tion performance worsens. We can justify this observation using the basis function bounds

from Equations (3.6) and (3.20). If we assume that the exponentially decaying portion of
the bounds is negligible and approximately zero, the time support of An(t) and ,(t) is a

linearly-increasing function of the index n. The increasing time support is shown pictorially
in Figures (2-1) and (2-5). Consequently, we require contributions from a greater number

of basis functions in order to represent signal content at later times. This is also the reason

why slowly-decaying exponentials require more expansion terms relative to quickly decaying

exponentials in Section 5.4.

5.5.2 Isolated Discontinuities

A rectangular pulse is used to examine the effect of isolated discontinuities on the bilinear

approximation and reconstruction. Mathematically,

1, 0 < t < 0.1

(t) (5.11)
0, otherwise

A value of a = 68.2 is used to compute the bilinear expansions, since this corresponds to

the first zero-crossing in the frequency response.

Figure (5-11) shows the sorted bilinear coefficients and partial sum sequences for both

representations. As seen, the partial sum sequences converge very slowly towards the origi-

nal signal energy. This results in large reconstruction errors around the signal discontinuities

at t = 0, 0.1 as shown in Figure (5-10).

The oscillatory behavior in Figure (5-10) can be attributed to the smooth nature of the

bilinear functions, which cannot represent discontinuities very well. Furthermore, because

the basis functions have increasing time support, the effect of a discontinuity is not localized
to a small number of expansion coefficients.
 Sorted Orthonormal Bilinear Coefficients of f(t) - sinc(10(t-n))

CU
(p

E
0m
.m

r-
e0
r0
o
U)
on
0

5 10 15 20 25 30 35 40 45 u00
Coefficient Index, n

(a) Orthonormal Expansion Coefficients

Sorted Biorthogonal Bilinear IP Coefficients of f(t) - sinc(l1O(t-n))

cu
(
0

0
C)

c-
,
(D
Cu

0
0)
0
-c
t

0
C

UI)

5 10 15 20 25 30 35 40 45 50
Coefficient Index, n

(b) Biorthogonal Inner Product Coefficients

Figure 5-8: Sorted Bilinear Coefficients for Shifted sinc Functions, a=10
 Partial Sum of Orthonormal Expansion Coefficients for f(t) - sinc(10(t-n))

.(
,
.a
o

U0.0C

E
0
C
0.
t_
0C

E
=1
c)

(U
0t
CL

5 10 15 20 25 30 35 40 45 50
Coefficient Index, n

(a) Orthonormal Expansion Coefficients

Partial Sum of Biorthogonal IP Coefficients for f(t) - sinc(10(t-n))

0
(n
o

ao0
.o
m
0)
a00)
0
E

U
cc
0-_

5 10 15 20 25 30 35 40 45 50
Coefficient Index, n

(b) Biorthogonal Inner Product Coefficients

Figure 5-9: Partial Sum Sequences for Shifted sinc Functions, a=10
 Plotof f(t) - rect(0.lt) and Its BilinearReconstructions
Plot of f(t) - rect(0.lt) and Its Bilinear Reconstructions

Original
3 - BiorthogonalReconstruction

2.5

2
a_

Ca

cc

.
0.5

0 0.2 0.4 0.6 0.8

time, t
time, t

(a) Orthonormal Representation, a = 62.8 (b) Biorthogonal Representation, a = 62.8

Figure 5-10: Original Signal (black) and its Bilinear Reconstructions (color)

5.5.3 Signal Decay Rate

We use following sinc and sinusoidal pulses to study the effect of decay rate on the bilinear
approximation properties:

(t) - /sin(20t),0< < (5.12)

f/(t) 1 (5.12)
0, otherwise
sinc(20t), o<t<
f 2 (t) 0 1 (5.13)
0, otherwise

A value of a = 20 is used to compute all bilinear expansions.

Figures (5-12) and (5-13) show the sorted bilinear coefficients and partial sum sequences,
respectively. As seen, the approximation performance is much better for the sinc pulse,

which decays as 1, than for the sinusoidal pulse, which does not decay.

However, when using a = 20 for the bilinear expansions, the approximation performance

shown in Figures (5-12) and (5-13) is worse than that for any of the rational Laplace

transforms considered in Section 5.4. This indicates that an exponential decay rate is
preferable over a polynomial one.
 Sorted Orthonormal and Biorthogonal Bilinear Coefficients for f(t) - rect(O.lt)
U.

0.3

(D
0.
ca
E
. 0.2
(D
o0 O.

0.

(D

0 0.

0.0

2 4 6 8 10 12 14 16 18
Coefficient Index, n

(a) Sorted Bilinear Coefficients

Orthonormal and Biorthogonal Partial Sum Sequences for f(t) - rect(0.1t)

1
Orthonormal
0.9 - Biorthogonal -

0.8

( 0.7

CT

0.5

0.4
.O
a

Il I | i I I I I | I

10 20 30 40 50 60 70 80 90 100
Coefficient Index, n

(b) Partial Sum Sequences

Figure 5-11: Sorted Coefficients and Partial Sum Sequences for Rectangular Pulse, a = 62.8
 Sorted Orthonormal Bilinear Coefficients for Standard Signals

5 10 15 20 25 30 35 40 45 50
Coefficient Index, n

(a) Orthonormal Expansion Coefficients

Sorted Biorthogonal Bilinear IP Coefficients for Standard Signals

-- Sinusoidal Pulse
-- - Sinc Pulse

5 10 15 20 25 30 35 40 45 50
Coefficient Index, n

(b) Biorthogonal Inner Product Coefficients

Figure 5-12: Sorted Bilinear Coefficients for fi (t) and f2 (t)

 Partial Sum of Orthonormal Expansion Coefficients for Standard Signals

10 20 30 40 50 60 70 80 90 100
Coefficient Index, n

(a) Orthonormal Expansion Coefficients

Partial Sum of Biorthogonal IP Coefficients for Standard Signals

10 20 30 40 50 60 70 80 90
Coefficient Index, n

(b) Biorthogonal Inner Product Coefficients

Figure 5-13: Partial Sum Sequences for fi (t) and f 2(t)

5.6 Summary

The results presented in this chapter provide a number of key insights into the bilinear

approximation properties.

As described in Section 5.2, the parameter a plays an important role in determining the

group delay 7g(w) of the all-pass filters in Figures (2-3) and (2-9). We suggest initializing
the value of a to the maximum frequency of interest in order to reduce the number of

significant expansion terms.

From Section 5.3, signals which can be exactly represented using a finite number of

bilinear basis functions must have rational Laplace transforms with all poles located at

s = -a. Section 5.3 shows that rational transforms with poles close to s = -a are well-

approximated using the bilinear expansions. However, the performance worsens as the poles

move away from this location, both along the real axis as well as perpendicular to it.

Section 5.5 investigates general signal characteristics which impact the approximation

performance. Due to the behavior of An(t) and On(t), the bilinear representat ions are best-
suited to signals with their energy concentrated near the time origin. Furthermore, the

presence of isolated discontinuities results in a slow growth of the sequences Sov [MA] and

Sio[M]. This translates into large reconstruction errors around the discontinuities. The
approximation is also sensitive to the decay rate. In general. signals with an exponential

decay are preferred to those which do not decay or which have a polynomial decay.

We conclude this chapter by noting that. from Figures (5-3) through (5-13), the approxima-

tion performance of the orthonormal and the biorthogonal representations are very similar.

In fact, the main difference between the two expansions seems to be the overall imnple-

mentation. Namely, for the orthonormal expansion, we need to consider only the relative

magnitudes of the expansion coefficients to determine the index set I,!. However, for the

biorthogonal representation, we need both the expansion coefficients f[n] and a sequence
of secondary coefficients bf [n] to select the significant indices. Not only does the nonlinear

approximation become more complicated, but additional hardware is required to compute

the second set of bilinear coefficients.
Chapter 6

The Windowed Bilinear

Representation

This chapter explores a modified bilinear representation in which we segment the original
CT signal into finite-duration blocks and compute the expansion of each block separately.
Mathematically, we treat the original CT signal as a sum of segments according to

f(t) = fk(t - kT) (6.1)

k=O

fk(t) = f(t + kT)w(t) (6.2)

such that the finite-duration window function, w(t), satisfies

Ew(t - kT) = 1, Vt (6.3)

An example of this representation is shown pictorially in Figure (6-1).

f 2 (t) f 4 (t)

T 2T 3T 4T

Figure 6-1: Segmenting the Original CT Signal using a Non-Overlapping Window.

 There are many reasons for investigating the windowed bilinear representation. From an

approximation perspective, we are most interested in the potential time resolution that it

provides. Namely, since the amount of energy present in a signal often varies considerably

over time, it may be possible to achieve a lower approximation error by allocating more

expansion terms to segments with a lot of signal energy at the expense of those segments

with less signal content.

Also, from an implementation perspective, this representation allows us to control the

latency when computing the bilinear expansion. This is important considering that the

input to the first-order cascades in Figures (2-3) and (2-9) is the time-reversed signal .f(-t).

Therefore, we must obtain and store the entire CT signal before computing its expansion

coefficients. Depending on the signal duration, the associated latency may become much

longer than desired.

Section 6.1 summarizes the importance of the function w(t) in Equation (6.2) and presents

the window choices used for each of the applications in Chapters 7-9. Section 6.2 outlines the

relationship between the original and the windowed bilinear representations. In Section 6.3

we examine the approximation properties of this representation as it applies to both the

binary detection and the audio approximation applications. We will see that the windowed

representation affords certain advantages for each application.

6.1 The Window Function w(t)

The choice of window depends on both the particular application as well as the bilinear

approximation properties. As seen in Section 5.5, the bilinear representation is sensitive

to discontinuities, the energy distribution over time and the signal decay rate. In contrast,

the window function is constrained by Equation (6.3). Therefore, we must balance several

factors in selecting an appropriate w(t).

When computing inner products for the binary detection application, we consider only the

rectangular window

R(t) = 1. O<t<T (6.4)

0, otherwise
 The primary reason for this choice is to maintain simplicity. Namely, since the resulting

segments do not overlap in time, we can just sum the inner products of individual segments
to compute the desired quantity. Furthermore, under our assumption of additive white

Gaussian noise (see Chapter 8), the noise remains uncorrelated between segments. This

means that we do not complicate our overall detection problem.

The rectangular window also has an advantage in terms of its approximation perfor-

mance. For a given value of T in Equation (6.1), the rectangular window has the shortest

segment duration. As explained in Chapter 5, short-duration signals often have fewer sig-

nificant expansion coefficients which translates to a better approximation.

For the audio approximation application in Chapter 7, we will compare the performance of

a rectangular window with that of a Bartlett triangular window and a raised cosine window

with 50 percent overlap. Mathematically, these windows can be expressed

S O<t<T
WB(t) = 2- t T < t < 2T (6.5)

0, otherwise

1, O < It - TI < T

wc(t) = 0.5 (1 + cos ( (It - TI - ))) It - TI < 3T (6.6)

0, otherwise

The rectangular, Bartlett and raised cosine windows are depicted in Figure (6-2).

WR(t) WB(t) wc(t)

T T 2T

Figure 6-2: Rectangular, Bartlett and Hanning Windows to Segment a CT Signal.

One advantage that the Bartlett and raised cosine windows have over the rectangular one
is a smoother transition between consecutive segments. This reduces the effect of boundary

discontinuities (see Figure 5-10), which can be perceptually disturbing in the reconstructed
signal. A second advantage is that, due to the shapes of wB(t) and wc(t), the resulting
segments decay to zero. This is unlike segments generated by the rectangular window,
which has unity gain for 0 < t < T. According to Section 5.5 this gradual decay may have

a positive impact on the bilinear approximation performance.

The major disadvantages of the Bartlett and raised cosine windows are that the segment

durations are longer, and that the signal energy will generally be found near the center of

each segment, rather than at the beginning. In contrast, bilinear approximation seems to

perform best when the signal energy is concentrated near the time origin. Furthermore, the

number of significant coefficients usually increases with the signal duration.

6.2 Relationship to Original Representation

Equation (6.1) leads to a theoretical relationship between the original and the windowed bi-

linear representations. First, consider the two CT signals shown in Figure (6-3). Specifically,

ft(t) and f~2(t) consist of the same pulse shape located at the origin and at to respectively. In
the windowed representation f 2 (t) corresponds to the actual time segment, but the bilinear
coefficients are computed as if it were fl(t).

fh (t) f2 (t)

4 T to to + T

Figure 6-3: Original Continuous-Time Signal and its Shifted Version.

Next, from Chapter 2 that the orthonormal and biorthogonal bilinear frequency-warping

relationships are

FON (ej) e2 / F (jatan(iQ/2))

+------+ (6.7)
=
FBiO (ej ) = F (jatan(Q/2)) (6.8)

where FON (eij ) and FBiO (eji) are the Fourier transforms of the expansion coefficient
sequences and Q is the discrete-time frequency variable.
 Using Equations (6.7) and (6.8), the coefficient sequences fi[n] and f2[n] are related by

F2 (eF) F 1 (ej '

) ejatan(Q/ 2 )to (6.9)

The above expression is true for both the orthonormal and the biorthogonal representations.

As seen, shifting a CT signal corresponds to a nonlinear phase factor in the DT expansion

coefficients. We can extrapolate from Equation (6.9) to express the coefficients of the

original signal f(t) in terms of the expansion coefficients of its windowed segments fk(t),

specifically
F (e j - 1) Fk
(en) ejatan(Q/2 )kT (6.10)
k=O

Equation (6.10) can also be used to compute the true bilinear expansion coefficients of a

CT signal from its windowed representation.

6.3 Approximation Using the Windowed Representation

The following simulations are used to investigate the effect that segmentation has on the

bilinear approximation properties. We first examine the performance when using a rect-

angular window to segment the CT signal, in the context of binary detection. Then we

compare the performances of segmenting with a rectangular, a Bartlett triangular and a

raised cosine window, as they would apply to the audio approximation problem.

The following terminology denotes the three linear and nonlinear approximation meth-
ods employed in this section:

Linear The first M coefficients are retained from each segment.

Nonlinear 1 The M largest coefficients are retained from each segment.

Nonlinear 2 The largest coefficient is selected from each segment. Then, for the remaining,

the largest coefficients overall are kept.

All results presented in this section are based on the class of signals

f(t sinc(p(t - 0.5)), 1 (6.11)

S0, otherwise
Since the energy of a sine function is largely concentrated around its main lobe, it is

representative of a signal whose energy content varies considerably over time. All signals
are normalized to have unit energy.

6.3.1 Segmenting with a Rectangular Window

As discussed in Chapter 8, our objective in the binary detection application is to capture

the maximum amount of energy from the original CT signal for a fixed number of DT

coefficients. Therefore, if In represents the set of retained indices in segment n, then our

error metrics become

..,.
dON = - ,(t)dt
f[k] (6.12)
n= k 6 In

6
BiO = J f )(t)dt -> 3 IP,[k] (6.13)
0 n=1
kEA

The data presented in this section is based on the signal f(t) from Equation (6.11) with

p = 100. The bilinear representations are calculated using a = p = 100.

No. of DT Window Approximation Type

Coefficients Duration, T Linear Nonlinear 1 Nonlinear 2
Original 0.6840 0.4567
0.2 sec 0.6832 0.4532 0.0323
50 0.25 sec 0.5088 0.4156 0.3044
0.34 sec 0.6533 0.4260 0.0930
0.5 sec 0.5056 0.4062 0.3819
Original 0.4487 0.2166 -
0.2 sec 0.4487 0.2319 0.0078
100 0.25 sec 0.427.4 0.3474 0.1822
0.34 sec 0.4234 0.1871 0.0102
0.5 sec 0.4243 0.3412 0.2812
Original 0.1588 0.0231
0.2 sec 0.1089 0.0422 0.0031
200 0.25 sec 0.3559 0.2478 0.1014
0.34 sec 0.1032 0.0322 0.0028
0.5 sec 0.3568 0.2455 0.1663

6
Table 6.1: 0N for f(t) - sinc(100(t - 0.5))

Table (6.1) shows the orthonormal bilinear approximation performance for different rectan-

gular window sizes and each of the three approximation techniques.

 There are two main points to note from this data. First, in many cases the segmented

representation achieves a lower error than the original bilinear representation when using

the Nonlinear 2 approximation technique. This can be attributed to the increased time

resolution, since more coefficients are allotted to the segment(s) containing the main lobe

of the sinc function. The improvement in performance becomes especially pronounced as

the total number of DT coefficients decreases.

The second point is that the approximation performance for window durations T =

0.25, 0.5 is notably poorer than for the other two window lengths. This is because these

values of T divide the main lobe in half, meaning that one segment will contain a rapidly-

increasing signal with a large amount of energy. From Section 5.5, as the signal energy moves
farther from the time origin, the number of coefficients needed to represent it increases. In

contrast, window durations of T = 0.2, 0.34 capture the entire main lobe in a single segment.

This has the advantages of isolating most of the energy in one segment and of requiring

fewer coefficients to represent. Notice, however, that any segment duration outperforms the

original representation as the number of coefficients decreases to 50.

Table (6.2) shows the biorthogonal approximation performance for different rectangular
window sizes and each of the three approximation techniques.

No. of DT Window Approximation Type

Coefficients Duration, T Linear Nonlinear 1 Nonlinear 2
Original 0.9933 0.9913
0.2 sec 0.6795 0.4651 0.0356
50 0.25 sec 0.5163 0.3346 0.2242
0.34 sec 0.6730 0.4517 0.0979
0.5 sec 0.5071 0.3300 0.3034
Original 0.9916 0.9902 -
0.2 sec 0.4449 0.2149 0.0063
100 0.25 sec 0.4291 0.2657 0.1039
0.34 sec 0.4163 0.2102 0.0103
0.5 sec 0.4252 0.2637 0.2043
Original 0.9902 0.9897 -
0.2 sec 0.1287 0.0492 0.0015
200 0.25 sec 0.3563 0.1685 0.0278
0.34 sec 0.1073 0.0307 0.0028
0.5 sec 0.3578 0.1690 0.0909

6
Table 6.2: BiO for f(t) - sinc(100(t - 0.5))
 The most striking result is the vast performance difference between the original (un-

segmented) representation and the windowed representation. Once again, the greatest gain
from segmentation is realized when using the Nonlinear 2 approximation technique. How-

ever, in this case, the windowed representation achieves a lower error for any of the three

methods. A possible reason for this behavior is that the infinite-duration sinc function
does not fall in the class of signals which can be represented using the biorthogonal ex-
pansion since ft _ sinc2 (t - 0.5)dt - oc. In contrast, when the sin is windowed into
finite-duration blocks, each segment can individually be represented using the primal basis

functions ci(t).

Finally, notice that the performance when using window durations of T = 0.25, 0.5 is

notably poorer than for the other two window lengths. Once again, this is because the main

lobe is divided into two parts, so the segment with a rapidly-increasing signal requires a large

number of DT coefficients to represent. Nevertheless, even this performance degradation is

preferable to using the original representation.

From Tables (6.1) and (6.2) we conclude that a much better approximation performance

can be realized by segmenting the CT signal using a rectangular window and employing the

Nonlinear 2 technique.

Another noteworthy observation is that the performance, as measured by doN and 6

so,
of the windowed orthonormal and the windowed biorthogonal representations are fairly

similar. This is in accordance with the results seen in Chapter 5. However, the original

orthonormal and the original biorthogonal representations differ drastically in performance.

This indicates that the representations behave similarly for only a specific class of signals.

A precise characterization of this class is left to future study.

6.3.2 Comparison of a Rectangular, Bartlett and Raised Cosine Window

In the audio approximation problem (see Chapter 7), our goal is to achieve a representation

which is as close as possible to the original CT signal, as measured by the squared recon-

struction error, for a fixed number of DT coefficients. Therefore, if In represents the set of
retained indices in segment n, then our error metric is

= (t) - f[kAk(t - T) dt (6.14)

10 n=1 ke [)
We limit ourselves to the orthonormal representation for this application.

The data presented in this section is based on the signal f(t) from Equation (6.11)
with p = 50. The bilinear expansion coefficients and reconstructions are computed using

a = p = 50. We consider a lower frequency sinc in this section to avoid the dynamic range

issue, discussed in Section 4.3. As an added precaution, only the first 100 basis functions

are generated during synthesis. However, this does not impact the approximation results.

Tables (6.3), (6.4) and (6.5) show the reconstruction errors when using different rectangular,
Bartlett and raised cosine window sizes for each of the three approximation techniques.

From Figure (6-2) the rectangular window duration is equal to the shift T, the Bartlett

window duration is 2T, and the effective raised cosine window duration is 1.5T.

On average, the rectangular window achieves the lowest reconstruction error, followed

by the raised cosine window, which has the second-lowest error. This is true regardless of

the approximation technique used. There are two factors which contribute to this trend.

First, for a given window shift T, the rectangular window has the shortest duration, and

the raised cosine window has the next-shortest duration. As previously discussed, higher

order basis functions are needed to represent longer duration signals. Since this often

translates to a slower decay of sorted coefficients, the rectangular window has an advantage

in terms of its approximation properties.

Second, for a given value of T, the Bartlett and raised cosine windows require an addi-

tional segment relative to the rectangular window. The extra segment confers a disadvan-
tage because the same number of DT coefficients is being used to represent, not only longer

duration blocks, but a greater number of blocks as well. One point to note is that, as the

number of DT coefficients increases, the effect of this additional segment should become

less and less significant.

Although the overlapping windows have some drawbacks when minimizing the squared

reconstruction error, the resulting approximations have better perceptual quality. This is

due to the smoother transition between subsequent blocks which eliminates artificial discon-

tinuities between segments. The effect of such discontinuities is illustrated in Figure (5-10).
This issue is discussed further in Chapter 7.

We can use Tables (6.3), (6.4) and (6.5) to compare the windowed bilinear representations
with the original. As seen, the Nonlinear 2 approximation technique often achieves a lower
reconstruction error for the same number of DT coefficients. This is true for any of the three

window types with appropriate choice of the parameter T. The enhanced performance is

attributed to the increased time resolution which comes from windowing the CT signal. In

this particular case, more coefficients are used for the segment(s) containing the main lobe

of the sinc function, thus reducing the total squared error.

As a final comment, the windowed representations offer an additional benefit, specific

to the MATLAB implementation discussed in Chapter 4. Namely, having shorter duration

signals increases the number of basis functions that can be generated before encountering the

dynamic range issue of Section 4.3. Therefore, segmentation makes it possible to accurately

reconstruct the audio signals in the following chapter.

No. of DT Window Approximation Type

Coefficients Shift, T Linear Nonlinear 1 Nonlinear 2
Original 0.9598 0.6033 -
0.167 sec 0.6276 0.5259 0.4012
0.20 sec 0.9931 0.6446 0.2879
13
0.25 sec 0.5706 0.4786 0.4042
0.34 sec 0.9730 0.5994 0.2854
0.50 sec 0.5389 0.4570 0.4570
Original 0.6744 0.3768
0.167 sec 0.6071 0.4210 0.1942
0.20 sec 0.6907 0.3687 0.0582
25
0.25 sec 0.5326 0.3866 0.2508
0.34 sec 0.6614 0.3399 0.0833
0.50 sec 0.5161 0.4151 0.3805
Original 0.4355 0.2000 -
0.167 sec 0.4427 0.2826 0.0861
0.20 sec 0.4350 0.1484 0.0143
50
0.25 sec 0.4117 0.2748 0.1233
0.34 sec 0.4553 0.1278 0.0173
0.50 sec 0.3892 0.3255 0.2856
Original 0.2834 0.1533
0.167 sec 0.3556 0.2083 0.0612
0.20 sec 0.1951 0.062 0.0073
75
0.25 sec 0.3530 0.2060 0.0841
0.34 sec 0.2105 0.0655 0.0075
0.50 sec 0.3408 0.2856 0.2335

Table 6.3: e when using a Rectangular window for f(t) - sinc(50(t - 0.5))
 No. of DT Window Approximation Type
Coefficients Shift, T Linear Nonlinear 1 Nonlinear 2
Original 0.9598 0.6033
0.167 sec 0.9864 0.8731 0.8731
0.20 sec 0.9957 0.7063 0.4769
13
0.25 sec 0.9842 0.8248 0.6130
0.34 sec 0.9984 0.7229 0.5708
0.50 sec 0.9917 0.8070 0.6724
Original 0.6744 0.3768
0.167 sec 0.9775 0.6957 0.2086
0.20 sec 0.7796 0.5700 0.2645
25
0.25 sec 0.9793 0.6622 0.2293
0.34 sec 0.8181 0.5709 0.3802
0.50 sec 0.9885 0.7029 0.4407
Original 0.4355 0.2000
0.167 sec 0.7272 0.4314 0.0285
0.20 sec 0.6219 0.3211 0.0960
50
0.25 sec 0.7461 0.4634 0.0422
0.34 sec 0.6662 0.3747 0.2022
0.50 sec 0.8150 0.5263 0.2085
Original 0.2834 0.1533
0.167 sec 0.5748 0.2781 0.0160
0.20 sec 0.4441 0.2194 0.0334
75
0.25 sec 0.5855 0.3015 0.0175
0.34 sec 0.5028 0.2402 0.1192
0.50 sec 0.6616 0.3717 0.1606

Table 6.4: e when using a Bartlett triangular window for f(t) ~ sinc(50(t - 0.5))
No. of DT Window Approximation Type
Coefficients Shift, T Linear Nonlinear 1 Nonlinear 2
Original 0.9598 0.6033
0.167 sec 1.000 0.8518 0.8518
0.20 sec 0.8450 0.6599 0.4011
13
0.25 sec 0.9999 0.7980 0.5803
0.34 sec 0.8315 0.6260 0.5239
0.50 sec 0.9999 0.7863 0.6031
Original 0.6744 0.3768 -
0.167 sec 0.9719 0.6443 0.1225
0.20 sec 0.6033 0.4307 0.1775
25
0.25 sec 0.9533 0.6259 0.1418
0.34 sec 0.7142 0.4686 0.3216
0.50 sec 0.9976 0.6567 0.3031
Original 0.4355 0.2000
0.167 sec 0.6109 0.3016 0.0196
0.20 sec 0.3952 0.2273 0.0549
50
0.25 sec 0.6790 0.3460 0.0218
0.34 sec 0.4647 0.2784 0.1216
0.50 sec 0.7371 0.4439 0.0969
Original 0.2834 0.1533 -
0.167 sec 0.5326 0.2011 0.0118
0.20 sec 0.2861 0.1494 0.0204
0.25 sec 0.5225 0.1933 0.0104
0.34 sec 0.3248 0.1793 0.0520
0.50 sec 0.5866 0.2483 0.0337

Table 6.5: e when using a Raised Cosine window for f(t) - sinc(50(t - 0.5))
Chapter 7

Approximation of Audio Signals

In Chapters 5 and 6 we discussed the bilinear approximation properties in the context of

various synthetic waveforms. In this chapter we use the orthonormal bilinear representation

to approximate audio signals. This choice is motivated by the fact that instrumental music
can often be represented using decaying harmonic functions, thereby falling into the class
of potentially well-approximated signals discussed in Section 5.5.

Section 7.1 of this chapter outlines the experimental details. In Section 7.2 we present

the simulation results for two different audio clips. A short discussion of the results is found

in Section 7.3.

We consider only the orthonormal representation for this application. This is due to

the difficulty in selecting the appropriate subset of biorthogonal expansion coefficients to

minimize the approximation error [M] in Equation (5.2). Since we desire an approximation

which is perceptually similar to the original signal, achieving a close reconstruction seems

like a reasonable goal.

7.1 Details of the Simulation Setup

The audio signals used in this experiment are read from .wav files, which have been sampled

at a rate of 44.1kHz. Since audio content can vary considerably over time, we segment the

signal, using a discrete-time window function, and calculate the expansion for each segment
individually. The window functions are DT counterparts of the Rectangular, Bartlett and

raised cosine windows presented in Chapter 6.

In order to use the MATLAB analysis approximations, the segments are interpolated
to achieve finer sampling. However, they are reconstructed at 44.1kHz,. so that we can

compare them with the original audio signal.

We compare the approximation performance of the bilinear representation with that of a

Discrete Cosine Transform (DCT) based representation. Since variants of the DCT are
often used in audio compression standards. it provides a reasonable benchmark against
which we can evaluate the bilinear representation. In particular, we employ the following

orthonormal DCT-IV expansion:

r N-1
[k] x[n cos (n + 1/2)(k + 1/2). k = 0 .N-1 (7.1)

x[n] 2 X[k] cos + 1/2)(k + 1/2)), n = 0...,N - 1 (7.2)

r=O

In the above expressions, xf[n] is the original DT signal, X[k] is its DCT-IV coefficients, and
N denotes the total number of (uninterpolated) samples in each windowed segment.

We implement the 'Nonlinear 1' and 'Nonlinear 2' approximation techniques from Sec-

tion 6.3 in order to reduce the (total) number of bilinear or DCT coefficients.

In the Nonlinear 1 method we retain an equal number of coefficients from each segment.

In the Nonlinear 2 method we retain the largest coefficient in each segment and, for the

remaining ones, select the largest coefficients over all blocks. The goal of this method is to

achieve greater accuracy in regions with more signal content at the expense of regions that

have less signal energy.

7.2 Simulation Results

The bilinear and DCT representations are used to approximate two distinct .way files. The
first audio clip features a piano playing smooth and flowing music, which is accentuated by

a few contrasting notes. The second clip consists of a violin playing a fast-paced country
tune with frequent changes in pitch. Based on the frequency content of each .way file, a
value of a = 8820 is chosen for the piano music, and a value of a = 14700 is used for the
violin music.
 In order to quantitatively compare the bilinear and the DCT approximation perfor-

mance, we look at the reconstruction error E[M from Equation (5.2), normalized by the

original signal energy. We refer to this quantity as the Normalized Squared Error (NSE).

Although this metric does not necessarily predict the perceptual quality of each approxi-

mation, it is useful to observe because we consciously select coefficients to minimize e[M].

The NSE for the piano and violin clips are given in Tables (7.1) and (7.2), respectively.

The 'DT Coefficient Retention' column describes the total number of (bilinear or DCT)

expansion coefficients used in the approximation relative to the number of samples in the

original .wav clip. To avoid the synthesis problems discussed in Chapter 4, the bilinear
reconstructions include only the first 100 expansion coefficients. While this does not impact
the 10% and 20% approximations, it does affect the minimum NSE that can be achieved
using the bilinear representation.

DT Coefficient Approximation Type

Window Function Representation
Retention Nonlinear 1 Nonlinear 2
100/seg 0.00298
Bilinear 20% of total 0.00497 0.00414
Rectangular 10% of total 0.03493 0.02095
100% ~0
DCT-IV 20% of total 0.00435 0.00258
10% of total 0.00888 0.00652
100/seg 0.00189
Bilinear 20% of total 0.00565 0.00424
Bartlett 10% of total 0.06259 0.03787
100% 0o
DCT-IV 20% of total 0.00080 0.00062
10% of total 0.00682 0.00424
100/seg 0.00119
Bilinear 20% of total 0.00299 0.00220
Raised Cosine 10% of total 0.03944 0.02278
100% 0
DCT-IV 20% of total 0.00089 0.00066
10% of total 0.00821 0.00513

Table 7.1: Normalized Reconstruction Error for the Piano Sound Clip, a = 8820

Figures (7-1) and (7-2) depict the normalized errors, f(t) - f, (t), for each sound clip,
while Figures (7-3) and (7-4) show plots of the original and approximated signals for a
single windowed segment. Only the Nonlinear 1 approximations are displayed since the
Nonlinear 2 performance is qualitatively very similar.
 Window Function
Window Function Representation
Representation DT Coefficient Approximation Type
Retention Nonlinear 1 Nonlinear 2
100/seg 0.05478
Bilinear 20% of total 0.08967 0.08375
Rectangular 10% of total 0.23854 0.19333
Rect angular
100% ~ 0.
0
DCT-IV 20% of total 0.01327 0.01045
10% of total 0.05375 0.04468
100/seg 0.12653
Bilinear 20% of total 0.17513 0.16905
Bartlett 10% of total 0.30930 0.26610
100% 0()
DCT-IV 20% of total 0.02656 0.02222
10% of total 0.07687 0.06724
100/seg 0.09000 -
Bilinear 20% of total 0.13164 0.12508
Raised Cosine 10% of total 0.26972 0.23044
100% 0--
DCT-IV 20% of total 0.03173 0.02671
__
_ 10% of total i 0.09148 0.07975
1
Table 7.2: Normalized Reconstruction Error for the Violin Sound Clip, a = 14700

7.3 Discussion

We use two criteria to analyze each of the approximations: the NSE and the perceptual

quality of the reconstructed signal.

We see from Tables (7.1) and (7.2) that the DCT-IV representation. in general, achieves

5-10X lower NSE than the bilinear representation for the same window function, number of

coefficients and approximation type. This observation is further supported by comparing

the dynamic range of the normalized errors for the bilinear and the DCT-IV approximations

in Figures (7-1) and (7-2).

In addition, Figures (7-3) and (7-4) indicate that the bilinear representation is not able
to capture rapid signal fluctuations. This is evidenced most strikingly from the fact that

even the 100-coefficient reconstructions cannot represent the violin music as well as the 20%
DCT-IV approximations.

The data from Section 7.2 also suggests that the rectangular window typically has the
lowest NSE followed by the raised cosine window. This pattern is very similar to the results
presented in Section 6.3.2.
There are some notable differences when listening to each signal approximation, as com-

pared with the NSE. For example, the bilinear approximations sound worst when using a

rectangular window. Due to the discontinuities between segments, there is a loud hissing

noise present throughout the entire clip. Furthermore, the raised cosine approximations

sound the same as the Bartlett ones, and the Nonlinear 1 and Nonlinear 2 approximations

are virtually indistinguishable when heard.

The DCT-IV approximations generally sound better than the bilinear ones for a 20%

coefficient retention because they are much cleaner and closer to the original. This is

especially true for the violin clip, since even the 100-coefficient bilinear reconstructions sound

fuzzy. Additionally, the DCT-IV expansion seems more robust because, when gradually

reducing the total number of coefficients, distortions become noticeable first in the bilinear

representation. This is consistent across window functions and approximation types.

However, as the coefficient retention decreases, the bilinear and DCT-IV representa-

tions eventually suffer from different distortions. For the bilinear representation, the fuzzy

background noise becomes increasingly louder until it overwhelms any music present. In

contrast, the DCT-IV representation starts to lose important high-frequency information,

resulting in unpleasant low-frequency harmonics. In these cases, deciding which is the

preferred approximation becomes very subjective.

Our conclusion from the above results would suggest that the DCT-IV representation is

better at approximating audio signals. Not only do the DCT-IV approximations achieve

much lower NSE, but the bilinear basis functions are not well-suited for capturing rapid

signal fluctuations. Therefore, using the DCT-IV representation over the bilinear one can

lead to a better-sounding approximation.

We may also conclude that, although it achieves a lower NSE in the bilinear case, using

a non-overlapping rectangular window results in a poorer-sounding approximation than for

either of the two overlapping window choices. This is due to the discontinuities between

segments, which produces a fuzzy background noise throughout the approximated clip.
 -
x 10 ,

-- 10%, NL1
0.015 -- 20%, NL1
-- 100 Coeffs/seg

0.01 I, ,11
I Il

Sample Index, n x 10

(a) Rectangular Window, Bilinear (b) Rectangular Window, DCT

x 10

0.015 , - 10%, NL1 -- 10%, NL1

-- 120%,NL1Coeffs/seg - 20%, NL1
: 100 Coeffs/seg

2
S

-0.00
g 00

'300

rr

0015.I .11 1
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
Sample Index, n Sample Index, n x 104
x 104

(c) Bartlett Window, Bilinear (d) Bartlett Window, DCT

3
x 10
- 10%, NL1
20%, NL1 -
2015 - 100 Coeffs/seg

15k
I

''''''''"'
5
U
11
1 2 3 4 5 6 7 8 1 2 3 4 6 7 8
Sample Index, n Sample Index, n x 104

(e) Raised Cosine Window, Bilinear (f) Raised Cosine Window, DCT

Figure 7-1: Normalized Error f(t) - fNi(t) for the Piano Sound Clip
 -3
x 10
10

f8
6
7 4

2
0
-2

au -6

8[I

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
x 104 Sample Index, n 4
Sample Index, n x 10

(a) Rectangular Window, Bilinear (b) Rectangular Window, DCT

-
x10

10%, NL1
i[ __--20%, NL1

-2

-4

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
Sample Index. n oapi inox,
x 104 Sample Index, n

(c) Bartlett Window, Bilinear (d) Bartlett Window, DCT

3
x 10
S 10%, NL1
I6 - 20%, NL1

2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
. einex
...
Sample Index, n Sample Index, n

(e) Raised Cosine Window, Bilinear (f) Raised Cosine Window, DCT

Figure 7-2: Normalized Error f(t) - fM (t) for the Violin Sound Clip
 Sample Index, n

(a) Rectangular Window, Bilinear (b) Rectangular Window, DCT

0 50 100 150 200

Sample Index, n Sample Index, n

(c) Bartlett Window, Bilinear (d) Bartlett Window, DCT

Sample Index, n Sample Index, n

(e) Raised Cosine Window, Bilinear (f) Raised Cosine Window, DCT

Figure 7-3: Reconstructed Audio Segment of a Piano Sound Clip

 Sample Index, n

(a) Rectangular Window, Bilinear (b) Rectangular Window, DCT

(c) Bartlett Window, Bilinear (d) Bartlett Window, DCT

50 100 150 200

Sample Index, n

(e) Raised Cosine Window, Bilinear (f) Raised Cosine Window, DCT

Figure 7-4: Reconstructed Audio Segment of a Violin Sound Clip

Chapter 8

The Binary Detection Problem

Chapter 7 focused on an audio approximation problem. We now consider a different type

of application, corresponding to binary detection. In this context our goal is to determine

whether a desired signal s(t) is present (Hypothesis 1, HI) or absent (Hypothesis 0, Ho) by

analyzing a noisy received signal x(t). This is depicted pictorially in Figure (8-1).

1-----I

| (t) Ho : r(t) = 0, x(t) = q(t)

SHl: r(t) = s(t), x(t) = s(t) + qr(t)

r(t) -, + :x(t) q(t) - zero-mean AWGN, o2

Figure 8-1: Binary detection scenario. The transmitted signal r(t) is corrupted by AWGN.
The received signal x(t) consists either of noise or the signal s(t) plus noise.

We assume that 7(t) is additive white Gaussian noise (AWGN) with a constant power
spectrum, P,(jw) = a. In addition, the value of a2 does not depend on whether or not s(t)

is transmitted. In this case, the well-known decision rule involves comparing the integral of

the desired and received signals with a threshold y. Mathematically,

j x(t)s(t)dt y (8.1)

where we assume that the desired signal s(t) is real-valued and causal
In theory the left-hand side of Equation (8.1) can be computed using an analog matched

filter with impulse response h(t) = s(-t). However, due to the complexity of real-world
signals, it is usually difficult and/or impossible to design such a filter. We can circumvent
this problem by expanding the CT signals in a specific basis according to Equation (1.1),
and then manipulating the DT expansion coefficients to obtain the integral.

We highlight the relationship between binary detection and signal approximation by further
consideration of the random variable V = f x(t)s(t)dt.
It follows from the assumptions on rl(t) that the integral f rj(t)s(t)dt is a zero-mean
Gaussian random variable. Therefore, V can be characterized under each hypothesis

Ho V= frl(t)s(t)dt N(O, (")

H1 : V= s (t)dt + q(t)s(t)dt - N(Ev,u )

where N((p,a 2) denotes a Gaussian random variable with mean M and variance a2. The

bias and variance from the above expressions are

Ev = s(t)dt (8.2)

( 2= E j.rj(t)s(t)dt (8.3)

We note that the value of c 2 depends heavily on the expansions used for x(t) and s(t).

The associated detection and error probabilities can be calculated in terms of Ev.,ao2
and using Gaussian Q-function, Q(x) = 1 e-I/ 2dt, as follows:

Pm = Pr [error H] = Pr [V < 7H
1 ]= Q (8.4)

P = Pr [correctlHi] = Pr [V > ,IjHj = 1 Q (8.5)

Pfa - Pr [error Ho] = Pr [V > - Ho] = Q (8.6)

PC = Pr [correctlHo] = Pr [V < o = I- Q (8.7)

From the preceding analysis, the Gaussian distributions under Ho and H 1 are separated
by the bias Ev. Therefore, the larger this quantity, the better our ability to distinguish
between the two hypotheses. According to Equation (8.2), for a perfect signal represen-
tation, Ev should equal the desired signal energy. However, if we constrain the number
of DT expansion coefficients used to compute V, then the application becomes similar to
an approximation problem. The difference is that we would like to capture the maximum

amount of energy from s(t), to differentiate between Ho and H 1, rather than represent the
entire signal. This is the rationale behind the metric 6[M] from Chapters 5 and 6.

In this chapter we analyze the theoretical performance when using different signal repre-

sentations for binary detection. Section 8.1 is devoted to Nyquist sampling and Section 8.2
explores the orthonormal and the biorthogonal bilinear signal representations. From Equa-

tions (8.4) through (8.7), the binary detection performance is largely determined by Ev and

OrT. Therefore, our focus is to compute these quantities for each representation. Additional

expressions and derivations relevant to this topic can be found in [4], [14] and [10].

8.1 The Nyquist Signal Representation

A commonly-used representation is Nyquist sampling. The CT signal is expressed as

f(t) = f(nT)sinc - n (8.8)

According to Equation (8.8), the Nyquist basis functions are scaled and shifted sinc func-

tions, and the expansion coefficients are equally-spaced time samples of the CT signal.

Using the orthogonality of sinc functions, we have the following inner product relation-

ship between two CT signals and their time samples:

Sf(t)g(t)dt = T f(nT)g(nT) (8.9)

-x n=-o

This relationship will be useful in determining Ev and ar for the Nyquist representation.

8.1.1 Ideal (Unconstrained) Nyquist Detection

The Nyquist basis functions span the subspace of finite-energy signals that are bandlimited
to IwI j. Consequently, in order to compute the integral of Equation (8.1) using this

expansion, we assume that the noise power spectrum is flat up to the frequency WM = T
and zero afterwards. Moreover, wMA must be greater than the effective bandwidth of s(t).
Mathematically,

P,(jw) = T (8.10)

Equation (8.10) implies that the noise samples rl(kT) are independent and identically
distributed (i.i.d.) according to N(O. o-/T).
We can now represent the desired signal and noise according to Equation (8.8). From
Equation (8.9), it follows that

E, = T
7 (kT) (8.11)
k=O

0 = a [T s2(kT) (8.12)
k=0

The detection and error probabilities can be computed using Equations (8.4) through (8.7).

8.1.2 Constrained Nyquist Detection

If we constrain the maximum number of digital multiplies allowed to compute f z(t) s(t)dt,
it may not be possible to use all the Nyquist samples. In this section we consider two
methods of reducing the number of samples. The first is to select samples with the largest
magnitude. The second is to time alias the Nyquist samples. Again, our focus in each case
is to determine Ev and a .

Selecting a Subset of Nyquist Samples

According to Chapter 5, we should retain the subset of samples with the largest magnitudes
in order to preserve the most signal energy. Since the noise samples are i.i.d., if I denotes
the set of selected indices, we can calculate EV and a2- by summing over I instead of over
all indices. Specifically,

2
Ev = T 1S (kT) (8.13)
kel

= o 2T s2(kT) (8.14)
[ kl I

The performance depends on how well the samples in I approximate the energy of s(t).
Time-Aliasing the Nyquist Samples

Another way to restrict the number of multiplications is to time-alias the sample magni-

tudes. In particular, we replace s(kT) and rl(kT) with the coefficients .[k] and -[k] as

defined below:

,0

.[k] = |[s(kT+nMT)|, for0<k MAI-1 (8.15)

n=O

[k] = +r(kT + nMT), for 0 < k < M - 1 (8.16)

n=O

The ±1 factor is set based on whether the sample s(kT + nMT) is positive or negative. It

does not alter the variance of 77(kT + nMT).

Since the samples q(kT) are i.i.d, the sequence [k] contains independent values. Fur-

thermore, because the Gaussian distribution is symmetric, the ±1 factor should not intro-
duce correlation between 9[k] and '[k]. Therefore, the bias and variance are

l-1
2
Ev = T '[k] (8.17)
k=0
M-1
S= T2 > 2
[k]E [k2 [k]]
k=0
Al- 1
= T 20 2 E g2 [k]rmk (8.18)
k=O

The value mk represents the number of noise samples summed in each [k]. If s(t) is a
finite-duration signal, only a finite number of samples x(kT) will be used for the integral in

Equation (8.1). In this case Equation (8.18) remains bounded.

The primary advantage of time aliasing is that we can exploit signal energy from all

Nyquist samples. The primary disadvantage is that time aliasing increases the noise vari-

ance, which may worsen the overall performance

8.2 Bilinear Matched Filtering - A Theoretical Analysis

We now derive expressions for Ev and a when using the bilinear representations. The
analysis in this section is based on CT signals and basis functions. The modifications when
simulating the matched filter solution in MATLAB will be discussed in Chapter 9.
8.2.1 The Orthonormal Representation

Given the orthonormal expansion coefficients s[nr] and x[n], it follows that .fo x(t)s(t)dt =

-ri s[n]x[n]. In this application we assume that the sequence .s[n] has already been
computed and stored and that the coefficients x[n] are calculated once the received signal
has been obtained.

The first step to computing E- and ar- is to express x[n] using the orthonormal basis

functions, in the case when s(t) is present

X[11] = x(t)A(t)dt = [s(t) +q(t)] An(t)dt

= s[n] + qt)An(t)dt (8.19)

By substituting Equation (8.19) for x[n] and by using the orthonormality relationship
in Equation (1.4), we have

M-1
E- = .s2 [k] (8.20)
k=0

n=.

,2 sY[n]
2 (8.21)

Since E ' s 2 [n] = TE" s802 (kT), in the limit of a perfect signal representation, the

bias and variance for the orthonormal and Nyquist representations are equal. This implies
that both expansions would yield the same binary detection performance.

8.2.2 The Biorthogonal Representation

Given the biorthogonal expansion coefficients x [n] and the secondary coefficients b,[n], it
follows that f O" x(t)s(t)dt = E o x[n]bs[n]. Similar to the orthonormal case, we assume
that the sequence b [n] has already been computed and stored. The coefficients x[n] are
calculated once x(t) has been obtained.
Again, we first express xl[n] using the biorthogonal basis functions in the case when the

desired signal is present.

n[] = x(t)h,,,(t)dt = [s(t) + rj(t)] h,(t)dt

= .s[n] + f q(t)hn(t)dt (8.22)

The sequence s[n] represents the biorthogonal expansion coefficients of s(t).

From Equation (8.22), the quantities Ev and o-, will depend on the dual basis inner prod-

ucts. Since the set {h(t)}__ is not orthogonal. we employ the recurrence relationship in

Equation (A.4) with a = 0:

L(x) - [L I(x) - L (x)] (8.23)

The dual basis functions (for n > 0) can now be expressed

S=- 0
hn(t) = e-atLo(2at), (8.24)
- [L,_1(2at) - Ln(2at)] u(t), n> 0
( n-1)-eat

By substituting x = 2at and applying the orthogonality relationship of Laguerre poly-

nomials from Equation (A.3), the inner product of two dual basis functions is given by

for M= 0
-n=

J h (t)h(t)dt for m = n # 0
(8.25)
An (t) hm (t) dt
for n= m+ 1 or m = n+ 1 n, m f 0

otherwise

If we replace z[n] using Equation (8.22) and apply the inner products from Equation (8.25),
the bias and variance are

Ev i s [n]b[n] (8.26)
n=O
 S[ (t)( bn[ ]h(t) dt
a
= b[O] + I [n] + b, [n]b, [n + 1] (8.27)
n=1 n=0

Note that Z--i s[n]bs[n] is the energy of the signal s(t). Furthermore. since s(t) can

be expressed using the dual basis as s(t) = E'L obs[n]h,,(t), the bracketed term in Equa-

tion (8.27) is also equal to the desired signal energy. Therefore, in the limit of a perfect
signal representation, the detection performance of the Nyquist, orthonormal and biorthog-

onal representations are equivalent.

8.2.3 Variations of the Bilinear Representations

If we constrain the number of digital multiplies, it will not in general be possible to use com-
plete expansions for the signals x(t) and s(t). In this application we rely on the windowed

bilinear representations and employ the Linear, Nonlinear 1 and Nonlinear 2 techniques

(discussed in Section 6.3) to reduce the total number of DT coefficients.

The expressions for Ev and ao- are very similar to the ones already derived. Partial

biases and variances are calculated by summing over the retained indices in each segment.
Furthermore, since we use a non-overlapping rectangular window, the final Ev and a. are
obtained by summing these partial values over all segments.
Chapter 9

Matched Filtering Simulations

This chapter presents MATLAB simulation results of binary detection as described in Chap-

ter 8. Specifically, we compare the detection performances when using the Nyquist, the
orthonormal and the biorthogonal representations to compute the integral f x(t)s(t)dt.

Section 9.1 of this chapter outlines the experimental details. In Section 9.2 we display a

series of Receiver Operating Characteristic curves for different desired signals s(t). A short
discussion of the results is found in Section 9.3.

9.1 Details of the Simulation Setup

We assume that all signals have been sampled at a rate approximately 100 times greater than

the Nyquist rate of s(t). To simulate discrete-time AWGN, i.i.d. noise with a distribution

N(0, o-/T) is added to each sample of the transmitted signal to form the received sequence

x(nT). This ensures that the additive channel noise is white over the frequency range of

interest. Representative signal and noise spectra are shown in Figure (9-1).

IS(JU)| PI,(jw)

Figure 9-1: Magnitude of S(jw) and Noise Power Spectrum with Sample Spacing T.
 Since the bilinear expansions do not have an associated 'sampling rate', we look at the

detection performance of each representation for a fixed number of discrete-time multiplies.

This is equivalent to constraining the number of coefficients ulsed to calculate j' (t)s(t)dt.

For the Nyquist representation, the reduced set of time samples are obtained directly

from the oversampled sequences s(nT) and X(nT) via two different methods. First, we

select the subset of samples corresponding to the largest magnitudes in s(n.T). Second. we

time-alias the sequences. In this case, each sample is multiplied by ±1 according to the sign

of the corresponding sample in s(nT).

For the bilinear representations, the sequences are first segmented using a non-overlapping
rectanglular window. The necessary coefficients are then computed according to Section 4.1.
In this application we use the Linear, Nonlinear 1 and Nonlinear 2 techniques from Sec-

tion 6.3 to reduce the total number of DT coefficients. The indices are retained based on

the (largest) orthonormal expansion coefficients and the (largest) biorthogonal inner prod-
uct coefficients of the windowed s(t). The same index set is used for both the desired and

received signals.

In this chapter we compare the detection performances for the four desired signals:

2
sl(t) tf-T5t a(t)

sin(100t), 0 < t< 1

)
0, otherwise

inc(100(t - 0.5)), 0 < t< 1

(t) '
0, otherwise

S t) N 1. 0 < t < 0.1

0. otherwise

Once again, all signals are normalized to have unit energy.

The signal sl(t) has a rational Laplace transform and should be well--approximated
using the bilinear basis functions. s 2 (t) is narrow-band in frequency with a constant energy
distribution over time, while s 3 (t) has a wider frequency range and its energy concentrated
around the main lobe. Finally, s 4 (t) is a short-duration signal with isolated discontinuities
at t = 0, 0.1.
9.2 Simulation Results

We assess the binary detection performance by examining Receiver Operating Characteristic

(ROC) curves. In Figures (9-2)-(9-5) the probability of detection, Pd., is plotted on the

vertical axis, and the probability of false alarm, Pf, is plotted on the horizontal axis. Since

we desire a large Pd for a given Pf , we would like the curves to lie near the upper-left

corner of the graph. Consequently, an ROC curve which lies entirely above another has

better performance.

The results presented in this section are based on the theoretical expressions derived in

Chapter 8. A separate set of Monte Carlo experiments were done for each representation
and approximation technique. In all cases. the Monte Carlo simulations converged to the
theoretical performance as the number of trials increased.

Figures (9-2)-(9-5) depict ROC curves for a fixed number of DT multiplies. Throughout

this section we assume that Ho and H 1 are equally likely, and that the channel noise power

is or = 1. The acronyms SLS and PA stand for 'Select Largest Samples' and 'Positive
Aliased'. They denote the two Nyquist coefficient-reduction techniques. Each individual

plot in Figures (9-2)-(9-5) compares the two Nyquist methods with a different bilinear

approximation (Linear, Nonlinear 1 or Nonlinear 2).

Figures (9-2) and (9-5) present the ROC curves for the desired signals s 1 (t) and s 4 (t),

respectively. In both cases the signal duration is artificially truncated to tm,, = 1, and the

integrals f x(t)s ,(t)dt are approximated using 5 DT multiplies. In this case the signals
are not segmented, meaning that the Nonlinear 1 and Nonlinear 2 methods yield identical

results. We use values of a = 100 and a = 62.8 to compute the bilinear coefficients for the

two detection systems.

Figures (9-3) and (9-4) illustrate the ROC curves for the desired signals s 2 (t) and s3(t),

respectively. In each case, the signal is segmented with a rectangular window of length

0.34sec, and the inner products are computed using a total of 25 DT multiplies. Addition-

ally, the bilinear expansions are obtained using a value of a = 100.

 Probability of False Alarm

(a) Linear Approximation

I
Probability of False Alarm

(b) Nonlinear 1 Approximation

Figure 9-2: ROC Curves for si(t); 5 DT Multiplies, a = 100

 of FalseAlarm
Probability

(a) Linear Approximation

(b) Nonlinear 1 Approximation

0 0.2 0.4 0.6 0.8

Probability of FalseAlarm

(c) Nonlinear 2 Approximation

Figure 9-3: ROC Curves for s2 (t); 25 DT Multiplies, a = 100

 (a) Linear Approximation

c
-0

ca

2f
a-

Probability of False Alarm

(b) Nonlinear 1 Approximation

gr-
.o

t_
0
B
0
a.

Probability of False Alarm

(c) Nonlinear 2 Approximation

Figure 9-4: ROC Curves for sa(t); 25 DT Multiplies, a = 100

100
.-
C
.Q)
0
0

0
ca
0
Q-.

0.4 U.b U.0 I

Probability of False Alarm

(a) Linear Approximation

c
.2
5

.0
o
0P

0 0.2 0.4 0.6 0.8 1

Probability of False Alarm

(b) Nonlinear 1 Approximation

Figure 9-5: ROC Curves for s 4 (t); 5 DT Multiplies, a = 62.8

9.3 Discussion

The bilinear ROC curves in Figures (9-2)-(9-5) conform to the intuition gained from Chap-

ters 5 and 6. Specifically, the bilinear representations can well-approximate the integral

f x(t)s(t)dt for signals with rational Laplace transforms, such as si(t), and for short-

duration signals with energy concentrated near the time origin. such as s4(t). In both
examples, the performance is close to ideal when using only 5 DT multiplies.

In contrast, the bilinear representations do not perform as well when the desired signal

is 8 2 (t) in part because the sinusoidal pulse has its energy distributed evenly throughout
the signal duration. The bilinear detection is even worse for s3(t) quite likely because the

sinc pulse has its energy concentrated around t = 0.5 rather than at the origin. In these

cases, the bilinear performance is not ideal, even when using 25 DT multiplies to compute

the inner products.

Aside from distinguishing the well-approximated signals from the poorly-approximated

ones, the plots in Section 9.2 reveal other trends. For example, there is a dramatic difference

between the linear and nonlinear ROC curves in Figures (9-3) and (9-4). This indicates

that being able to select the largest coefficients is very important when approximating the

inner product. Also, the orthonormal and biorthogonal performance is very similar in all

cases. This is consistent with the results of previous chapters. One more important point

is that there is very little difference between the Nonlinear 1 and Nonlinear 2 curves for the

sinusoidal pulse. This is because each segment of the windowed signal is almost identical.

The Nyquist SLS and PA methods behave very differently than the bilinear approximations.

SLS does well when the signal energy is concentrated in a small region and can be

captured using a few samples. This is true of the pulse sa(t). Although the nonlinear

approximations perform better than SLS in Figure (9-4), as the number of DT multiplies

increases, the SLS performance improves very rapidly and approaches the ideal case ahead
of the bilinear representations.

PA performs well when the signal energy is spread evenly over time, like the sinusoidal

pulse s2 (t). In this case, both the magnitude and the noise variance of the aliased samples
increase linearly. This can be seen from Equations (8.15) and (8.16). The PA method does
not perform well for rapidly-decaying signals because the magnitude of the aliased samples
grows more slowly than the noise variance.

102
The ROC curves based on synthetic signals sI(t)-s 4 (t) indicate that using the bilinear

representations may be appropriate in certain binary detection scenarios. In three out of

the four cases, the Nonlinear 1 and Nonlinear 2 performances exceed those of the two Nyquist

methods. Furthermore, in applications where the desired signal s(t) is not appropriately

band-limited, the bilinear representations may provide a favorable alternative to eliminating

signal content through an anti-aliasing filter.

On the other hand, if there is a constraint on the number of filter stages in the first-order
analysis cascades (i.e. a constraint on the analog hardware), the bilinear representations

may not be a good choice. This is because linear approximation performs much worse than

the Nyquist methods for both the sinusoidal and the sinc pulses.

103
104
Appendix A

The Family of Generalized

Laguerre Polynomials

Many properties of the two bilinear representations are derived from those of the generalized

Laguerre polynomials, L(" ) (x). Therefore, this appendix provides a brief overview of these

functions and their properties. For a more detailed treatment of Laguerre polynomials, refer

to [5], [13] and [3].

A.1 Definition

The generalized Laguerre polynomials, L,( )

(x), are characterized by two parameters, the

index, n, and the order value, a. They are defined according to the expression

L(a)( x dn
L(-)( d= (e- xn+) (A.1)
( n! dxn

Observe that the multiplicative term in Equation (A.1) cancels the factors of ex and za in

the derivative term. Consequently, the resulting function is simply a polynomial in x.

By using the product rule of differentiation, Equation (A.1) can be expanded into an

(n + 1)-term polynomial as shown below

n F(n + a + I) (-x)J (A2)

L ((A.2)
La)
oa Er F(n-j+ 1)F(j+a+1) j!
j=0

Plots of the generalized Laguerre polynomials are shown in Figures (A-l) and (A-2).

105
 Generalized Laguerre Polynomial with alpha = 0

E
o
C
01.

-a
-J

0 1 2 3 4 5
time, t

Figure A-i: L(C) (x) for different index values using a = 0

A.2 Properties

Below we describe some well-known properties of Laguerre polynomials. These are used

extensively when deriving the bilinear representation properties in Chapter 3.

A.2.1 Orthogonality

For a given order value, a, the generalized Laguerre polynomials are orthogonal with respect
-
to the weighting function xe x. Mathematically

~r F(n+a+l) n = m
= r(n+l) (A.3)
o L) (x)L()(x)xe-dz
o 0, otherwise

Equation (A.3) can be used to verify that the basis functions An(t) are, indeed, orthonormal,
and that the primal basis functions, n,(t), satisfy Equation (2.10).

106
 Generalized Laguerre Polynomial with index = 5

-5

_1
0 2 4 6 8
time, t

Figure A-2: L ( )
(x) for different values of a using n = 5

A.2.2 Recurrence Relation

Differentiating Equation (A.2) with respect to x yields the following recurrence relation
between L( a ) (x) and its first derivative:

d L(a)(x) = -Ln 1 -
(x)= nL(a)(x)- (n +a)L(a)l(x (A.4)
dx n n1 x nn )

Equation (A.4) is important for analyzing the biorthogonal matched filtering network.

A.2.3 Signal Space

In [5] it is shown that the set of generalized Laguerre polynomials {Lna)(x)}jo 0 forms a
basis for causal functions, f(x) = 0 for x < 0, which satisfy the following condition:

00
If (x)12 xe-xdx < 00 (A.5)

Equation (A.5) is used when characterizing the span of the primal biorthogonal basis func-
tions, 0n(t) in Chapter 3.

107
A.2.4 Bounds on the Generalized Laguerre Polynomials

Define the normalized Laguerre polynomial as

c",(x) r( ne + ) 2(x)u(x) (A.6)

If C, ~ > 0 are constants and we let v = 4n + 2a + 2, then according to [13], the function

(a) (xr) is bounded as follows:

(xv)a ./ 2 , O< < 1/v

(rv)-, 1/v < x < v/2

IC'(X) c v-/' (V1/- + IV - xl)-/ v/2 < x < 3v/2
(A.7)

e-, x > 3v/2

Equation (A.7) is used to derive bounds on the functions An(t). 6n(t) and hn(t). These
bounds are, in turn, useful when analyzing the approximation properties of the two bilinear

representations.

108
Appendix B

Additional Properties of the

Bilinear Representations

B.1 Representing Anti-causal Signals

In this thesis, we have represented only causal continuous-time signals which satisfy f(t) =

0, Vt < 0. While this does not impede our investigation of the bilinear approximation

properties, from a theoretical perspective, it is worthwhile to consider how we may represent

anti-causal CT signals as well.

From Equations (2.4) and (2.9) we note that A,(t), 0,(t) and h,(t) are zero for t > 0 or

t < 0 depending on the index value, n. Therefore, we can divide a two-sided signal into its

causal and its anti-causal parts and compute the bilinear expansions of each one separately.

In this section we derive both the orthonormal and the biorthogonal analysis and syn-

thesis networks for bounded anti-causal CT signals, f(t).

B.1.1 The Orthonormal Representation

From Equation (2.4) if f(t) = 0 Vt > 0 the only basis functions with indices n < 0 will

contribute to the expansion. By inspection, the analysis and synthesis networks for anti-
causal inputs are described by the first-order cascades in Figures (B-1) and (B-2).

109
 Figure B-1: Orthonormal analysis network for anti-causal CT signals.

6(t)

f(t)
Figure B-2: Orthonormal synthesis network for anti-causal CT signals.

B.1.2 The Biorthogonal Representation

Recall from Equation (2.9) that for anti-causal signals, only primal basis functions with
index n < 0 contribute to the overall expansion.
As seen in Chapter 2, the dual basis functions for 'n - 0 are given by h,,(t) = Et n(t).
In the Laplace transform domain, this corresponds to

H,(s) -- d= n(s) = (a s)2 s-n- (B.1)

ds (a -8 a- s

The final step is to find Ho(s) by constraining f(t) to remain bounded for all time. Since

the primal basis functions ,n(t) have an impulse at the origin, the expansion coefficients

must satisfy
-1

f[0] = - f [n] (B.2)

By expressing the expansion coefficients f[n] as an inner product in the Laplace trans-
form domain and by substituting Equation (B.1) into Equation (B.2), it follows that

1
Ho(s) = (B.3)
a-s

110
 Using the above expressions for n (s) and H,(s), the analysis and the synthesis networks
first-order cascades for anti-causal inputs are depicted in Figures (B-3) and (B-4).

Figure B-3: Biorthogonal analysis network for anti-causal CT signals.

6(t)

f(t)
Figure B-4: Biorthogonal synthesis network for anti-causal CT signals.

B.2 Conditions to Preserve Correlation

In [8] the authors derive necessary and sufficient conditions to map a continuous-time LTI

system onto a discrete-time LSI system. This is equivalent to preserving convolution be-

tween the CT and DT domains. Below, we follow a similar approach and derive the neces-

sary conditions to preserve correlationbetween continuous and discrete time. This property

may be useful in certain applications.

Let f(t) = -nEZ f[n]'n,(t) and let g(t) and h(t) be defined in a similar fashion. We

seek conditions on the basis functions {n(t)}nEz so that

g(t) = / f(r)h(T - t)dt --- g[n] = f [k]h[k - n] (B.4)

keZ

111
 Since F(s) = E,,Z f[rn]an(S) and H(-s) = EnEZ h[n]xFn(-s), we express the Laplace
transform of g(t) in Equation (B.4) as

nEZ \kEZ rEZ

= f[k]h[r]4k (s8) r (-S)

kEZ rEZ

>
:= f[k]h[k - n]Q k(s)k-n(-s) (B.5)
kEZ nEZ

It follows that I,(s) must satisfy Equation (B.6) below in order for the expansion to

preserve correlation.

Tn(s) = W'k(s)'Pk-n,(-S) (B.6)

Notice that if T,,(s) = I-,,(-s), then the expansion also preserves convolution.

Since the primal basis functions ,,(s) = ( satisfies the above criterion, the
biorthogonal representation preserves correlation as well as convolution between continuous
and discrete time.

B.3 Noise Analysis for the Analysis and Synthesis Networks

The presence of noise is inevitable in any real-world system implementation. In this section,

we present simple noise models for the bilinear analysis and synthesis networks.

We consider two types of noise. The first is additive noise at the input to the first-order
cascade, q(t). The second is additive noise introduced by the analog components, t(t). We

assume that component noise is added after each non-unity gain factor in each stage, and

that it is uncorrelated with all other noise sources.

Since, the first-order cascades consist only of low-pass and all-pass filters, we examine

separately the effect of input and component noise on each type of filter.

B.3.1 The Low-pass Filter Stage

An implementation for a low-pass filter stage with system function H(s) = is depicted

in Figure (B-5). The block with transfer function s corresponds to an integrator.

112
 X(s) Y() rq(t) .Y(s)

Figure B-5: Block and Implementation Diagram for the Low-pass Filter Stage.

The transfer function for the additive input noise, r(t), is H,,(s) = H(s) = . Assum-

ing that the noise has a power spectrum P (jw), the output power spectrum is

P,(jW) = Pr(jw)lHq(jw)l2 P'(jw) a2 2+2 (B.7)

As seen in Figure (B-5) there are two component noise sources in the low-pass filter

stage. The noise l1 (t) arises from the gain A in the numerator of H(s) and the noise

E2(t) comes from the constant a in the denominator of H(s). Using the low-pass filter
implementation diagram, we can derive the following noise-to-output transfer functions:

1
H (s) = (B.8)
a+s
-1
H, (s ) (B.9)
a+s

Notice that both transfer functions are low-pass filters.

Assuming that the noise sources have power spectra Pt (jw) and P2(jw), the output

power spectra can be expressed

P,'SI (jW) = Psi (Jw) 2 (B.10)

( 1

Pye,(jw)
2 = P( a2 + W2 (B.11)

If either constant in Figure (B-5) is +1, then the corresponding noise source is zero.

This is because unity gain is equivalent to a wire, which we assume to be noiseless.

113
B.3.2 The All-pass Filter Stage

An implementation for an all-pass filter stage with system function H(s) = - 5 is depicted
in Figure (B-6).

X(s) Y(s) (t) -Y(s)

Figure B-6: Block and Implementation Diagram for the All-pass Filter Stage.

Once again, the transfer function for the additive input noise is given by H(s) = H(s) =
-. In this case, the output power spectrum is

Pj.,j(jW) = P,(jW)|Hrj(jw) 2 = P(jw) (B.12)

Thus, the noise spectrum remains unchanged by the all-pass filter stage.

From Figure (B-6), the constants a in the numerator and denominator of H(s) can be

combined into a single gain element. Therefore, the all-pass filter only has one compo-
nent noise source. Using the implementation diagram, we can derive the following transfer

function for the component noise source E(t):

2s
H,(s) = (B.13)
s+a

with output power spectrum

-,( =U (w) 24 2(B.14)

( a2 + w2

Equation (B.14) corresponds to a high-pass filter. Since the transfer function H (s)

eliminates low-frequency noise, this may be beneficial should the CT input signal be low-
pass in nature.

114
B.3.3 Combined Effect of Input and Component Noise

When analyzing the combined effects of input and component noise, we assume that the
additive input noise is white with variance a 2 and that each component noise source is white
with variance a 2. The output power spectrum after the nth stage is denoted by Py,n(jw).

The Orthonormal Networks

The orthonormal cascade consists of one low-pass filter followed by a succession of all-pass
filters. The output power spectrum after the low-pass filter is

Py,1(jw) = aB 2+2
a '17+2 2 2
+ W2
2) 2o (B.15)

At each subsequent stage, additional coefficient noise is added according to Equa-

tion (B.14), however noise present at the input to a stage is not modified by an all-pass
filter. Therefore, the output power spectra for n > 1 is

( 4w 2
02
Py,n (jw) = (2 V)
2a ,
2
1
2
),
+'
-(n- 1) (2 )o
(B.16)
(B.16)

The Biorthogonal Analysis Network

The biorthogonal analysis network consists of two low-pass filter stages followed by a cascade
of all-pass filters. The output power spectrum after the first low-pass filter is

Py, = 2 1(w) + 2 (B.17)

Notice that since there is only one non-unity coefficient, there is only one component
noise source. After the second low-pass filter, two coefficient noise sources are added and
the noise from the previous stage is modified. The output power spectrum becomes

( 4a2 4a 2 +2 ( (B.18)
Py,2 (jw) = ( 2
(a2 + w2)2 )
+ ±
(a2 + W2)2 + 2 2±+ 2 (B.18)

Finally, the all-pass filters add coefficient noise sources but do not modify noise present
at the input. The output power spectra for n > 2 is

4a2 2+ 2 ) 2 + 4(n -W22 2 (B.19)

S (a ,,2) 2 (2 2)2 a2 2
2

115
The Biorthogonal Synthesis Network

The biorthogonal synthesis network consists only of all-pass filters. This means that each
stage adds a coefficient noise source but does not modify existing noise at the input. The

output power spectra (for all) n is

P.(Jw) = + n((aa 2 (B.20)

From the above analysis we recognize that although increasing the number of stages in the

cascade allows for a more accurate signal representation, we suffer performance degradation

with the addition of a linearly-increasing number of noise sources.

116
Bibliography

[1] E. W. Cheney. Introduction to Approximation Theory. McGraw-Hill, New York, 1966.

[2] Ole Christensen and Khadija L. Christensen. Approximation Theory: From Taylor

Polynomials to Wavelets. Birkhauser, Boston, 2004.

[3] E. H. Doha. On the connection coefficients and recurrence relations arising from ex-

pansions in series of laguerre polynomials. Journal of Physics A: Mathematical and

General, pages 5449-5462, 2003.

[4] Robert G. Gallager. 6.450: Principles of digital communication, August 23, 2006.

[5] Allan M. Krall. Hilbert Space Boundary Value Problems and Orthogonal Polynomials.

Birkhauser Verlag, Basel, 2002.

[6] Y. W. Lee. Statistical Theory of Communication. John Wiley and Sons, New York,
1960.

(7] S. G. Mallat. A Wavelet Tour of Signal Processing. Academic Press, 1999.

[8] Alan V. Oppenheim and Donald H. Johnson. Discrete representation of signals. Proc.

of the IEEE, 60(3):681-691, 1972.

[9] Alan V. Oppenheimrn, Ronald W. Schafer, and John R. Buck. Discrete-Time Signal

Processing. Prentice Hall, Upper Saddle River, 1999.

[10] Alan V. Oppenheim and George C. Verghese. Signals, systems and inference, 2006.

[11] Alan V. Oppenheim and Alan S. Willsky with S. Hamid Nawab. Signals and Systems.

Prentice Hall Signal Processing Series, Upper Saddle River, 1997.

117
[12] Kenneth Steiglitz. The General Theory of Digital Filters with Applications to Spectral

Analysis. PhD thesis, New York University, 1963.

[13] Sundaram Thangavelu. Lectures on Hermite and Laguerre Expansions. Princeton

University Press, 1993.

[14] George L. Turin. An introduction to matched filters. IRE Transactions on Information

Theory, pages 311--329, 1960.

[15] Martin Vetterli. Wavelets, approximation and compression. IEEE Signal Processing

iiMagazine, pages 53-73, 2001.

118