Wavelet for filter design

Hsin-Hui Chen Graduate Institute of Communication Engineering National Taiwan University, Taipei, Taiwan, ROC

Abstract
Wavelets, filter banks, and multiresolution signal analysis, which have been used independently in the fields of applied mathematics, signal processing, and computer vision, respectively, have recently converged to form a single theory. In this toturial we first compare the wavelet transform with the more classical short-time Fourier transform approach to signal analysis. Then we make the exploration of the relations between wavelets, filter banks, and multiresolution signal processing. We briefly review perfect reconstruction filter banks, which can be used both for computing the discrete wavelet bases, provided that the filters meet a constraint known as regularity. Given a lowpass filter, we derive necessary and sufficient conditions for the existence of a complementary highpass filter that will permit perfect reconstruction..

Chapter 1 Introduction
The analysis of nonstationary signals often involves a compromise between how well transitions of discontinuities can be located, and how finely long-term behavior can be identified. In wavelet analysis one looks at different scales or resolutions: a rough approximation of the signal might look stationary, while at a detailed level (using a small window) discontinuities become apparent. The multiresolution, or the essence of the wavelet transform, which has recently become quite popular. The wavelet analysis is performed using a single prototype function called a wavelet, which can be thought of as a bandpass filter. Fine temporal analysis is done with contracted (high-frequency) versions of the wavelet, while fine frequency analysis uses dilated (low-frequency) versions. Multiresolution approaches have been popular for computer vision problems from range detection to motion estimation. An important application to image coding called a pyramid is closely related both to subband coding and to wavelets. Mallat used this concept of multiresolution analysis to define wavelets, and Daubechies constructed compactly supported orthonormal wavelets based on iterations of discrete filters. Notation: The set of real numbers will be represented by R ( R + being the set of positives reals), the set of integers is Z . The inner product over the space of

square-summable sequences l 2 ( Z ) is

< a[n], b[n] >=

n =

a [n]b[n]
*

Where a[n] , b[n] l 2 ( Z ) , and *denotes complex conjugation. We define a[n] 2

=< a[n], a[n] >= a[n] = max n a[n] . Similarly, over the space of square-integrable

functions L2 ( R ) we have the inner product:

< f ( x), g ( x) >=

f * ( x) g ( x)dx
2

where f ( x) , g ( x) L2 ( R) . The norm is given by f ( x) 2 =< f ( x), f ( x) > .The z transform of a sequence is defined by H ( z ) =

n =

h(n) z

. The reversed version of a

% (n) = h( L 1 n) . We shall use the sequence which is nonzero for n = 0,1,..., L 1 is h notation{h(n)} = [h(0), h(1),..., h( L 1)] when we want to indicate the coefficients of an FIR filter. Note that we will consider only filters with real coefficients, unless otherwise specified.

Chapter 2 Wavelets, multiresolution signal processing, and filter banks

A. The Wavelet Transform
Analysis of signals using appropriate basis functions goes back at least as far as Fourier who used complex sinusoids. But a difficulty that has often been cited with this approach is that, because of the infinite extent of the basis functions, any time-local information is spread out over the whole frequency axis. Gabor addressed this problem by introducing windowed complex sinusoids as basis functions. This leads to the windowed Fourier transform :
X ( , ) = w(t ) x(t )e jwt dt

(1)

where w(.) is an appropriate window like a Gaussian. That is, X ( , ) is the Fourier transform of x(t ) windowed with w(.) shifted by . Equivalently, the basis functions are modulated versions of the window function (see Fig. 1(a)). The major advantage of the windowed or short-time Fourier transform (STFT) is that if a signal has most of its energy in a given time interval[T , T ] and frequency interval[, ] . A limitation of the STFT is that, because a single window is used for all frequencies, the resolution of the analysis is the same at all locations in the time-frequency plane (see Fig. 1(b)).

(a)

(b)

Fig. 1. Basis functions and time frequency resolution of the short-time Fourier transform (STFT). (a) Basis functions. (b) Coverage of time-frequency plane. Of course, the uncertainty principle excludes the possibility of having arbitrarily high resolution in both time and frequency, since it lower bounds the time-bandwidth product of possible basis functions by T (1/ 4 ) , where (T ) 2 and () 2 are the variances of the absolute values of the function and its Fourier transform, respectively. However, by varying he window used, one can trade resolution in time for resolution in frequency. The can be achieved with the wavelet transform, where the basis functions are obtained from a single prototype wavelet by translation and dilation/contraction: 1 ta (2) ha ,b (t ) = h( ) b a where a R + , b R . For large a , the basis function becomes a stretched version of the prototype wavelet, that is a low frequency function, while for small a , the basis function becomes a contracted wavelet, that is a short high frequency function (see Fig. 2(a)). The wavelet transform (WT) is defined as 1 ta X ( a, b) = x(t )h* ( (3) )dt b a The time-frequency resolution of the WT involves a different tradeoff to the one used by the STFT : at high frequencies the WT is sharper in time, while at low frequencies, the WT is sharper in frequency (see Fig. 2(b)).

(a)

(b)

Fig. 2. Basis functions and time-frequency resolution of the wavelet transform (WT). (a) Basis functions. (b) Coverage of time-frequency plane. Both the STFT in (1) and the WR in (3) are highly redundant when the parameters ( w, ) and (a, b) are continuous. Therefore the transforms are usually evaluated on a discrete grid on the time-frequency and time-scale plane, respectively, corresponding to a discrete set of continuous basis functions. The questions arises as to whether there is a grid such that the set of basis functions constitutes an orthonormal basis; which of course implies that there is no redundancy. In wavelet case, it is possible to design functions h(.) such that the set of translated and scaled versions of h(.) forms an orthonormal basis. The function should be at least continuous, perhaps with continuous derivatives also. Let us discretize the translation and dilation contraction parameters of the wavelet in (2):
m /2 m hmn (t ) = a0 h(a0 t nb0 )

m, n Z ,

a0 > 1, b0 0

m m which corresponds to a = a0 and b = na0 b0 . On this discrete grid, the wavelet

transform is thus
m /2 m X (m, n) = a0 x(t )h(a0 t nb0 )dt

(4)

Of particular interest is the discretization of on dyadic grid, which occurs for a0 = 2 and b0 = 1 . It is possible to construct functions h(.) so that the set
X (m, n) = 2 m / 2 x(t )h(2 m t n)dt ,

m, n Z

(5)

is orthonormal. That is
< hmn (t ), hkl (t ) >= mk kl

A classic example is the Haar basis (which is not continuous, but is of interest because of its simplicity, where:
1 h(t ) = 1 0 0 t < 1/ 2 1/ 2 t < 1 otherwise

The orthonormality is easily verified since at s given scale, translates are nonoverlapping, and because of the scale change by2, the basis functions are orthogonal across scale. The Haar basis is shown in Fig. 3(a). However, the Haar function is discontinuous, and is not generally appropriate for signal processing. A continuous set of basis functions is given in Fig. 3(b). These functions are obtained from a compactly supported wavelet constructed by Daubechies, using a length 4 FIR

filter. Longer filters lead to smoother functions. It is interesting to note that the translates and dilates of the functions in both Figs. 3(a) and (b) form orthonormal bases for L2 ( R ) functions. The purose of the present paper is to design other continuous wavelets, having additional properties like linear phase.

(b) (a) Fig. 3. Orthogonal system of scaled and translated wavelets (two scales only are shown). (a) Haar wavelet. (b) Daubechiess wavelet based on a length-4 regular filter.

B. Multiresolution Signal Processing

From a signal processing point of view, a wavelet is a bandpass filter. In the dyadic given in (5), it is actually an octave band filter. Therefore the wavelet transform can be interpreted as a constant-Q filtering with a set of octave band filters, followed by sampling at the respective Nyquist frequencies (corresponding to the bandwidth of the particular octave band). We will give a simple explanation of multiresolution and successive approximation. Call V0 the space of all band-limited functions with frequencies in the interval ( , ) . Then the set of functions ( x k ), k Z (6) forms an orthonormal basis for V0 . Call V1 the space of band-limited functions with frequencies in the interval (2 , 2 ) . Clearly the set 2 (2 x k ), k Z , is an orthonormal basis for V1 . Also,

V0 V1 (7) In particular, if x(t ) Z , then x(2t ) V1 . Now call W0 the space of bandpass functions with frequencies in the interval (2 , ) ( , 2 ) . Then V1 = V0 W0 (8) That is, W0 is the orthogonal complement in V1 of V0 . From the above, by scaling, that if Vi is the space of band-limited functions with frequencies in the interval (2 i , 2 i ) , then we get relations similar to (7) and (8): Vi Vi 1 i Z Vi 1 =Vi Wi i Z (9) (10)

where Wi is the space of bandpass functions with frequencies in the interval (2i +1 ,
2i ) (2 i , 2 i +1 ) . And by iterating (10) Vi = Wi +1 Wi + 2 Wi +3 ... Let us construct the wavelet that will span

W0 .

First

(11) the set

{ ( x k ), k Z } constitutes a basis for V0 . Thus { 2 (2 x k ), k Z } constitutes a

basis for V1 . Now, in the sampled version of V1 , ( x) is given by the perfect halfband lowpass filter with impulse response:
2cn discrete halfband filter

(12)

That is, ( x) can be written as

( x) =

n =

c (2 x n)
n

(13)

since it is the interpolation, (2 x ) ,of perfect half-band lowpass filter. ( x ) is called a scaling function because it derives an approximation in V0 of signals in V1 . In V1 , the orthogonal complement W0 to V0 is given by the halfband highpass signals. This is given by the halfband lowpass (12) modulated by (1) n , and shifted by one. Thus, ( x) is the interpolation thereof, that is,

( x) =

n =

(1) c
n

n +1

(2 x n)

(14)

Note that since the cn s are symmetric we can reverse the sign as above, which we do for later convenience. Now

(x k) (x k)

(15)

since they cover disjoint regions of the spectrum. Also < ( x k ), ( x l ) >= kl and < ( x k ), ( x l ) >= kl . It can be shown that the ( x) s span W0 , and therefore, ( x) and its integer

translates form an orthonormal basis for W0 . Thus, the wavelet for this bandpass example is given by ( x) . Now we see the picture as shown in Fig. 4. Fig. 4(a) shows the imbrications of V1 V0 V-1 , and (b) shows the added bandpass spaces Wi . While the above example may seem artificial, and leads to a wavelet (x) which is of infinite extent and has slow decay, the situation is conceptually the same for all orthonormal wavelet bases. Fig. 5 shows the corresponding division of the spectrum for a wavelet that has compact support. In particular, given an orthonormal basis for V0 made up of (x) and its integer translates, then we can find coefficients such that we can get (13) and (14).

(a)

(b) Fig. 4. Ideal division of spectrum using sinc filters. (a) Division into Vi spaces. (b) Division into Wi spaces.

(a)

(b)

(c) (d) Fig. 5. Division of spectrum with a real wavelet. (a) Dyadic stretches of the scaling function. (b) Fourier transform of the stretched scaling function, indication the nesting of the Vi spaces. (c) Dyadic stretches of the wavelet. (d) Fourier transform of the stretched wavelets, indicating the arrangement of the Wi spaces.

C. FIR Filter Banks and Compactly Supported Wavelets

We now briefly show the connection between wavelets of finite length and filter banks , as originally investigated by Daubechies. First assume we have an orthonormal basis of such functions ( x) and ( x) which obey two-scale difference equations as in (13)and (14) : < ( x + l ), ( x + k ) >= kl < ( x + l ), ( x + k ) >= kl < ( x + l ), ( x + k ) >= kl (16) (17) (18)

We will show these relations lead to a perfect reconstruction FIR filter bank. The finite support ( x) means that it can be written as a finite linear combination of the terms (2 x n) ; that is , finitely many of the cn are different from zero. From (16) we get

1 < (2 x l ), (2 x k ) >= kl 2
< ( x + l ), ( x + k ) > =< cn (2 x + 2l n), cm (2 x + 2k m) >
n m

(19)

Now, using (13)and (19),(16) can be written as (with n ' = n 2l , and m ' = m 2k ):

=< cn '+ 2l (2 x n '), cm ' (2 x m ') >

n' m'

(20)

= cn ' + 2l cn ' + 2l = kl
n'

From which it follows that cn = 2 . In other words the discrete filter, with impulse response h0 (n) = cn / 2 is orthogonal to its even translates, and with h1 (n) = (1) n h0 ( L n 1) we obtain an orthogonal perfect reconstruction FIR filter bank with orthogonal impulse responses. Thus, compactly supported wavelet bases lead to perfect reconstruction FIR filter banks. While the converse does not always hold, and is not as immediate to analyze, we discuss it here because it is the basis for the construction of compactly supported wavelets. Considering the discrete time wavelet transform in It is easily verified that subsampling by w followed by filtering with H ( z ) is equivalent to filtering with H ( z 2 ) followed by subsampling by 2 (see Fig. 6(a)). Therefore the cascade of i blocks of filtering operations followed by subsampling by 2 is equivalent to a filter H (i ) ( z ) with z transform: H ( i ) ( z ) = H ( z 2 ), i = 1, 2...
i =0 i 1
l

(21)

Followed by subsampling by 2i . We define H (0) ( z ) = 1 . Assuming that the filter H ( z ) has an impulse response of length L, the length of the filter H (i ) ( z ) is L(i ) = (2i 1)( L 1) + 1 as can be checked from (20). Of course as i we get

L(i ) . Now instead of considerings the discrete time filter, we are going to consider the function f (i ) ( x) which is piecewise constant on intervals of length 1/ 2i ,
and equal to f (i ) ( x) = 2i /2 h(i ) (n) n/2i x (n+1)/2i (22) (i ) Clearly, f ( x) is supported on the interval [0, L 1] . Note that the normalization by 2i /2 ensures that if checked that h (i )

(h

(i )

(n)) 2 = 1 then
=1.

( f

(i )

( x)) 2 dx = 1 as well. Also, it can be

= 1 when h (i 1)

(a)

(b)

(c) Fig. 6. Derivation of the equivalent iterated filter. (a) Subsampling by 2 before a filter H(z) can be written as filtering by H ( z 2 ) followed by subsampling. (b) Cascade of I filters each followed by subsampling by 2. (c) Equivalent filter, followed by subsampling by 2i . A fundamental question is to find out whether and to what the function f ( x) converges as i . Fig. 7 shows two examples of such iterations. In Fig. 7(a)
(i )

the first six iterates of the filter with impulse response [1, 3, 3, 1] show that it converges rapidly to a continuous function; while in Fig. 7(b) the iterates of the filter [-1, 3, 3, -1] tend to a discontinuous function. In other words, different filters exhibit very different behavior. Of course when constructing wavelets of compact support one would like them to be continuous functions, perhaps possessing continuous derivatives also. This can be achieved if the filter meet certain regularity constraints; so that the limit function f ( ) ( x) is continuous.

(a) (b) Fig. 7. Iteration (22) for two simple filters. (a) [1, 3, 3, 1] which converges to a continuous function. (b) [-1, 3, 3, -1] which converges to a discontinuous function.

D. Bases of Orthonormal Wavelets Constructed from Filter Banks

If filters h0 (n) and h1 (n) and their even translates form an orthonormal set in l ( Z ) , then we generate functions ( x) , ( x) , which together with their integer translates, form an orthonormal set L2 ( R ) .
2

First, using (21) and (22) f (i ) ( x) = 2i /2 h0 (m) h0(i 1) (n 2i 1 m)

m=0 L 1

n / 2 x < (n + 1) / 2i
i

(23)

To write h0(i 1) (n 2i 1 m) in terms of f (i 1) ( x) observe

f (i ) ( x) = 21/2 h0 (m) f (i 1) (2 x m)
m=0 L 1

n ' = n + 2 m) / 2i , (n + 2i 1 m + 1) / 2i ] ( x)

i 1

(24) That is, we have an expression for f (2 x m) when x is in the interval i 1 i i 1 i [(n + 2 m) / 2 , (n + 2 m + 1) / 2 ] . Making the change of variable: n ' = n + 2i 1 m
( i 1)

i n / 2i 1 2 x m < (n + 1) / 2i 1

this gives f (i ) ( x) = 21/2 h0 (m) f ( i 1) (2 x m)

m=0 L 1

(25)

Recall that when the filter is regular , f (i ) ( x) tends to a continuous limit function ( x) as i . By taking the limit in (25). ( x) itself satisfies a two-scale difference equation:

( x) = 21/ 2 h0 (n) (2 x n)
n =0

L 1

(26)

The bandpass function is defined as followed:

( x) = 21/2 h1 (n) (2 x n)
n =0

L 1

(27)

Chapter 3 General FIR perfect reconstruction filter banks and biorthogonal wavelets
Now we use perfect reconstruction filter banks( PRFBs) to construct wavelets, we briefly review the salient points here. Assume we have a filter bank as in Fig. 8, with analysis filters H 0 ( z ) and H1 ( z ) but with general synthesis filter G0 ( z ) and

% ( z ) and H % ( z ) . The output of the analysis/synthesis system is G1 ( z ) instead of H 0 1

given by
% ( z ) = 1 [G ( z ) G ( z )] H 0 ( z ) H 0 ( z ) X ( z ) X 0 1 2 H1 ( z ) H1 ( z ) X ( z )

(28)

We call the above matrix H m ( z ) ,

Fig. 8. Decomposition of x(n) using multivariate filters, and recombination to achieve perfect reconstruction. H 0 H 0 x is the projection of the signal x onto V0 , and H1 H1 x is the projection onto W0 . In order to eliminate the aliased of the signal caused by the reconstructed signal from X ( z ) , we use the synthesis filters that are related to the analysis filets as follows: [G0 ( z ) G1 ( z )] = C ( z )[ H1 ( z ) H 0 ( z )] Note that (30) = P( z ) P( z ) Where P ( z ) = H 0 ( z ) H1 ( z ) . For the filters, we introduce the polyphase notation: H i ( z ) = H i 0 ( z 2 ) + H i1 ( z 2 ) (31) det[ H m ( z )] = H 0 ( z ) H1 ( z ) H 0 ( z ) H1 ( z ) (29)
* *

where H i 0 ( z ) contains the even-indexed coefficients of the filter H i ( z ) , while H i1 ( z ) contains the odd ones. Therefore
1 1 1 0 H p ( z 2 ) = 21 H m ( z ) 1 -1 0 z

where H p ( z 2 ) is called the polyphase matrix. So we derive the following equations:

det[ H m ( z )] = 2 z 1 det[ H p ( z 2 )]

(32)

A. FIR Filter Banks

Now we make the following well-known fact.
Fact 3.1: For perfect reconstruction with FIR synthesis filters after an FIR analysis, it is necessary and sufficient that

det[ H m ( z )] = c z 2l +1

where l Z

It shows that det[ H m ( z )] is thus a pure delay as well. In order to make the results that follow less arbitrary, we shall fix c= 2. So from this and (30), we know that P ( z ) can have only a single nonzero odd-indexed coefficient: P ( z ) P( z ) = 2 p2l +1 z 2l 1 (33)

and we normalize p2l +1 = 1 . We call the polynomial P( z ) a valid polynomial if it satisfies this constraint. H 0 ( z ) and H1 ( z ) form a PR filter pair. From the above discussion, there are two possible design methods for PRFBs. First the factorization method consists of finding a valid P( z ) satisfying design criteria, and then factoring it into H 0 ( z ) and H1 ( z ) . Second, the complementary filter method starts with a filter H 0 ( z ) and then solves a system of linear equations to find complementary filtering leading to a valid P( z ) . When we designed P( z ) and factored it in terms of the analysis filters H 0 ( z ) and H1 ( z ) the synthesis filters follow with C ( z ) = c z 1 .

B. Orthogonal or Paraunitary Filter Banks

When we see how to construct unitary operators based on filters which were orthogonal to their even translates. In filtering notation this means that the even terms of the autocorrelation function are all zero., with the exception of the central one (which equals unity for normalized filters). The autocorrelation of a filter H i ( z ) is H i ( z ) H i ( z 1 ) ; thus we get the following condition: H i ( z ) H i ( z 1 ) + H i ( z ) H i ( z 1 ) = 2 i {0,1} (34)

In addition, the two filters H 0 ( z ) and H1 ( z ) were orthogonal to each other at even translates , so even terms of the cross correlation are all zero:

H 0 ( z ) H1 ( z 1 ) + h0 ( z ) H1 ( z 1 ) = 0

(35)

And if the filters satisfy (34)and (35), their impulse responses and even translates form an orthogonal basis of l 2 ( Z ) .
Fact 3.2: Consider an FIR perfect reconstruction filter bank such that H 0 ( z ) satisfies (34). Then the length of h0 (n) has to be even. In order that h0 (n 2k ) and H1 ( z ) form an orthogonal basis (that is H1 ( z ) should satisfy (34) and

(35)), it is necessary and sufficient that H1 ( z ) = z 2 k 1 H 0 ( z 1 ) (35)

From the above discussion that indicates two possible design approaches, we can make a conclusion. The first approach is to find an autocorrelation function that has only a single even-indexed coefficient different from zero. This must be the central one, since an autocorrelation function is symmetric. Such a function can be factored into its square roots H 0 ( z ) and H 0 ( z 1 ) . In particular, zeros on the unit circle have to be double, since the function must not change sign. The second method uses lattice structures to synthesize paraunitary matrices, for which complete factorizations. C. Biorthogonal or General Perfect Reconstruction
From Fact 3.1, we know that perfect reconstruction requires det[ H m ( z )] to be an odd delay, and the synthesis filters are given by (29) with C( z )=c z l . Choose l=1 for the purposes of this discussion. First note that G0 ( z ) H1 ( z ) and G1 ( z ) H 0 ( z ) have only odd coefficients, that is, % 0 (n 2k ), h1 (n 2l ) >= 0 <g (36) %1 (n 2k ), h0 (n 2l ) >= 0 <g (37) (note the time reversal in the inner product). In matrix notation H 0 G1 = 0 = H1G 0 (38)

where H i and Gi have been defined in a fashion. Now G 0 (z)H 0 ( z ) = z 1H 0 ( z )H1 ( z ) has a single nonzero odd coefficient (because it is a PR system). Similarly G1 (z)H1 ( z ) has also only a single even-indexed coefficient different from zero. That is,
% i (n 2l ), hi (n) >= l <g

(39) (40)

In operator notation H 0 ( z )G 0 (z)=H1 ( z )G1 (z)=I

Since we have a perfect reconstruction system we get G 0 (z)H 0 ( z )+G1 (z)H1 ( z )=I

(41)

Of course the last equation above indicates that no nonzero vector can lie in the colum nullspaces of both H 0 and H1 . Note that (40) implies that H 0 G 0 and H1G1 are each projections(since G i H i G i H i = G i H i ). They project onto subspaces which are not in general orthogonal . Note that (36)and (37)indicate, however, that there are orthogonality relations between the filter impulse responses. Because of (36), (37), and (39) the analysis synthesis system is termed biorthogonal. In the special case
* where we have a paraunitary solution one finds: G 0 =H* 0 and G1 =H1 , and (38)gives

that we have projections onto subspaces which are mutually orthogonal.

Chapter 4 Concluson
We introduce the relationships between wavelets, multiresolution signal analysis, and filter banks which have been developed and emphasize the equivalence between the fundamental ideas of each of these fields, The function of signals can be decomposed into different components of signals at different resolutions. In addition, strong similarities between the details of these techniques have been pointed out. In particular, we derived biorthogonal compactly supported wavelets bases with symmetries using regular FIR PRFBs. If compact support is not desired, similar techniques, using IIR filters generate orthogonal wavelet bases with symmetries.

Reference
[1] R. C. Gonzalez, R. E. Woods and S. L. Eddins, Digital image processing 2/e, Aug. 2003. [2] S. Mallat, A theory for multiresolution signal decomposition, dissertation, Univ. of Pennsylvania, Depts. Of Elect. Eng. and Comput. Sci. 1988. [3] M. Vetterli and C. Herley, Wavelets and filter banks: theory and design, IEEE Trans. Signal Process., vol. 40, no. 9, pp. 2207-2232, Sept. 1992. [4] R. C. Gonzalez, R. E. Woods and S. L. Eddins, Digital image using MATLAB Processing, 2005. [5] I. Daubechies, Orthonormal bases of compactly supported wavelets, Comm. Pure and Appl. Math. 41, 909-996 (1988)