FIR Filter Design via Semidenite Programming and

Spectral Factorization
Shao-Po Wu, Stephen Boyd, Lieven Vandenberghe
Information Systems Laboratory
Stanford University, Stanford, CA 94305
[email protected], [email protected], [email protected]

Abstract phase constraints (e.g., xn symmetric around the mid-

dle index) are imposed; see [19],[7] and [5]. However,
We present a new semidenite programming approach this approach leads to longer FIR lters than necessary
to FIR lter design with arbitrary upper and lower if linear phase is not required.
bounds on the frequency response magnitude. It is
shown that the constraints can be expressed as linear In this paper, we present a new way of solving the pro-
matrix inequalities (LMIs), and hence they can be eas- posed class of FIR lter design problems, based on mag-
ily handled by recent interior-point methods. Using nitude design, i.e., instead of designing the frequency
this LMI formulation, we can cast several interesting response X (!) of the lter directly, we design its power
lter design problems as convex or quasi-convex opti- spectrum, jX (!)j2 to satisfy the magnitude bounds (see
mization problems, e.g., minimizing the length of the [8] and [12, ch4]) Let r(n) denote
FIR lter and computing the Chebychev approxima- 1
X
tion of a desired power spectrum or a desired frequency r(n) = x(k)x(k + n); (2)
response magnitude on a logarithmic scale. k=,1
where we take x(k) = 0 for k < 0 or k > N , 1. The
1 Introduction sequence r(n) is symmetric around n = 0, zero for n 
,N or n  N , and r(0)  0. Note that the Fourier
We consider the problem of designing a nite impulse transform of r(n),
response (FIR) lter with upper and lower bounds 1
X
on its frequency response magnitude: given lter R(!) = r(n)e,j!n = jX (!)j2 ;
length N , nd lter tap coecients x 2 RN , x = n=,1
(x(0); : : : ; x(N , 1)), such that the frequency response is the power spectrum of x(n). If we use r as our design
,1
NX variables, we can reformulate the FIR design problem
X (!) = x(n)e,j!n in RN as
n=0
nd r = (r(0); : : : ; r(N , 1))
satises the magnitude bounds subject to L2(!)  R(!)  U 2 (!); ! 2

L(!)  jX (!)j  U (!); ! 2

 [0; ] (1) R(!)  0; ! 2 [0; ]: (3)
over the frequency range
of interest. The non-negativity constraint R(!)  0 is a neces-
sary and sucient condition for the existence of x sat-
One conventional approach to FIR lter design is isfying (2) by the Fejer-Riesz theorem (see x4). Once
Chebychev approximation of a desired lter response a solution of (3) is found, an FIR lter can be ob-
D(!), i.e., one minimizes the maximum approxima- tained via spectral factorization. An ecient method
tion error over
. Dierent approaches have been pro- of minimum-phase spectral factorization is given in Sec-
posed for FIR design via Chebychev approximation. To tion 4.
name a few, Chen and Park [6] use linear programming;
Alkhairy et al. [1], Preuss [14] and Schulist [15] use The reformulated FIR design problem (3) is a semi-
Remez algorithm and its variations; Potchinkov and innite programming problem and many methods have
Reemtsen [13] use semi-innite programming. Even been developed to solve (3) directly (see [9]). In Sec-
though these methods work well in general, they cannot tion 2 and 3, we present two relaxations of the prob-
handle constraints of the form (1), unless certain linear lem that can be solved as a linear program (LP) or a
semidenite program (SDP). In an SDP, we minimize and imposed exactly at the cost of N (N ,1)=2 auxiliary
a linear objective subject to a linear matrix inequality variables. We will use the following theorem.
(LMI) constraint:
minimize cT x Theorem 1 Given a discrete-time linear system
subject to F (x) = F0 + m
P
i=1 xi Fi  0; (A; B; C; D), A stable, (A; B; C ) minimal and D +
DT  0. The transfer function H (z ) = C (zI ,
where x 2 Rm is the optimization variable and the A),1 B + D satises
symmetric matrices F0 ; : : : ; Fm are given. The prob-
lem of nding an x that satises the LMI constraint, H (ej! ) + H  (ej! )  0 for all ! 2 [0; 2]
or proving that no such x exists, is called an SDP fea-
sibility problem. SDP feasibility problems can be cast if and only if there exists real symmetric matrix P such
as ordinary SDPs and solved [17]. that the matrix inequality
 
The LP or SDP formulation of the lter design prob- P , AT PA C T , AT PB
lems has several advantages. First, LPs and SDPs C , B T PA D + DT , B T PB  0 (5)
can be solved very eciently and conveniently using
recently-developed interior-point methods [10][17] and is satised.
related tools [16][18]. Secondly, these methods produce
a proof of infeasibility when the design specs are too Proof: By the positive-real lemma [2][3, ], the
ch2.7.2
tight. Thirdly, a wide variety of convex constraints can existence of P that satises (5) implies that H (z ) +
be expressed as LMIs, and hence easily included in the H  (z )  0, for all jz j  1. This provides the sucient
SDP problem. condition.
Let C denote the region fz j jz j > 1g on the
2 LP formulation complex plane. Since exp(,H (z )) is analytic in C ,
j exp(,H (z ))j assumes its maximum on the boundary
A common practice of relaxing the semi-innite pro- of C by the maximum modulus principle. From the fact
gram (3) is to solve a discretized version of it, i.e., that
impose the constraints only on a nite subset of the j exp(,H (z ))j = exp(,<H (z ));
[0; ] interval and the problem becomes <H (z )  0 for all jz j = 1 and limjzj!1 <H (z ) = D +
nd r = (r(0); : : : ; r(N , 1)) DT  0, we conclude that <H (z )  0 everywhere in C .
Thus, by the positive-real lemma, there exists P that
subject to L2(!i )  R(!i )  U 2 (!i ); !i 2
satises (5) and the necessary condition is proved. 
R(!i )  0; i = 1; : : : ; M; (4)
Observe that R(!) has the form
where 0  !1 < !2 <


< !M  . Since R(!i )
is a linear function in r for each i, (4) is in fact a lin- R(!) = H (ej! ) + H  (ej! );
ear program and can be eciently solved. When M is
suciently large, the LP formulation gives very good where
approximations of (3) in practice. A rule of thumb of
choosing M , M  15N , is recommended in [1]. H (ej! ) = 12 r(0) + r(1)e,j! +


+ r(N , 1)e,j!(N,1):
However, no matter how large M is, if R(!)  0 does In order to apply Theorem 1, we would like to dene
not hold for all ! 2 [0; ], no x 2 RN satises (2) (A; B; C; D) in terms of r such that
and the spectral factorization fails. One way to resolve
this problem is to solve (4) with the non-negativity C (zI , A),1 B + D
constraint tightened to = 12 r(0) + r(1)z ,1 +


+ r(N , 1)z ,(N,1): (6)
R(!i )  ; i = 1; : : : ; M
An obvious choice is the controllability canonical form:
for an appropriate  > 0, so that even between fre- 2 3
quency samples, R(!)  0. 0 0


0 1
2 3
61 0
6


077 07
6
A = 66 .
6 1 7
B = .. 775
6

3 SDP formulation 4 ..
... ... .. 775 6
.
4
. 0
0 1 0 (7)
In this section, we will show that the non-negativity of
r(N , 1) D = 12 r(0):
 
R(!) for all ! 2 [0; ] can be cast as an LMI constraint C = r(1) r(2)



Of course the realization is not unique, e.g., Thus, for z = ej! we have
(T ,1AT; T ,1B; CT; D) realizes the same transfer func- 1
X 1
X
tion. (!) = an cos !n and '(!) = , an sin !n;
n=0 n=0
It can be easily checked that (A; B; C; D) given by which implies (!) and '(!) are Hilbert transform
(7) satises (6) and all the hypotheses of Theorem 1. pairs. To determine Xmp(z ), we rst nd '(!) from
Therefore the existence of r and symmetric P that sat- (!) via the Hilbert transform. In practice, this step
isfy the matrix inequality (5) is the necessary and suf- can be replaced by two Fourier transforms of order
cient condition for R(!)  0, for all ! 2 [0; ], by N~ , with N~  N . Then we construct Xmp from
Theorem 1. (!) and '(!). A third Fourier transform of order
Note that (5) depends anely on r and P . Thus we N yields the coecients of Xmp(z ), which gives the
can formulate the SDP feasibility problem: desired minimum-phase FIR lter coecients.
nd r 2 RN and P = P T 2 RN,1N,1
subject to L2 (!i )  R(!i )  U 2(!i ); !i 2
5 Extensions
 
P , AT PA C T , AT PB  0 (8)
C , B T PA D + DT , B T PB We have shown that FIR design with magnitude
bounds can be cast as SDP feasibility problems. In
with (A; B; C; D) given by (7). The SDP feasibility fact, many extensions of the problem can be handled by
problem (8) can be cast as an ordinary SDP and solved simply adding a cost function and/or LMI constraints
eciently. to our SDP formulation. We will give a few examples
in this section.
4 Spectral factorization 5.1 Minimum-length FIR design
The length of an FIR-lter is a quasi-convex function
Given r 2 RN be the solution of (3), (4) or (8), the of its coecients [4]. Hence, the problem of nding the
desired N -tap FIR lter can be obtained via spectral minimum-length FIR lter given magnitude upper and
factorization by the Fejer-Riesz theorem: lower bounds
minimize N
Theorem 2 (Fejer-Riesz) If a complex function subject to L(!i )  jXN (!i )j  U (!i ); i = 1; : : : ; M
W (z ) : C ! C satises is quasi-convex and can be solved using bisection on N .
m
X Each iteration of the bisection involves solving an SDP
W (z ) = w(n)z ,n and W (z )  0 8 jz j = 1; feasibility problem (8).
n=,m
then there exists Y (z ) : C ! C and y(0); : : : ; y(m) 2 C 5.2 Chebychev approximation on power spec-
such that trum
m Another interesting extension is the Chebychev approx-
Y (z ) =
X
y(n)z ,n and W (z ) = jY (z )j2 8 jz j = 1: imation of a desired power spectrum
 
n=0 minimize i=1max jX (!i )j2 , jD(!i )j2  ; (9)
;:::;M
Y(z) is unique if we further impose the condition that which is not convex in the lter coecients x. Using
all its roots be in the unit circle jz j  1. the technique developed in Section 3, problem (9) can
be reformulated as a convex problem in r and P :
An ecient method for minimum-phase spectral fac- minimize i=1max
 
R(!i ) , jD(!i )j2 
torization is as follows [11]. We denote the unique ;:::;M
minimum-phase factor of R(z ) by Xmp(z ). Denote 
P , AT PA C T , AT PB  0;

log Xmp(z ) by subject to C , B T PA D + DT , B T PB
log Xmp(z ) = (z ) + j'(z ); with (A; B; C; D) given in (7). This problem can be
we have (z ) = (1=2) log R(z ) and is known. Since further cast as an SDP:
Xmp(z ) is minimum-phase, log Xmp(z ) is analytic in minimize t
the region fz j jz j  1g and has the power series ex- subject to
pansion
1
jD(!i )j2 , t  R(!i )  t + jD(!i )j2 ; i = 1; : : : ; M
 
log Xmp(z ) =
X
an z ,n; jz j  1: P , AT PA C T , AT PB  0;
n=0 C , B T PA D + DT , B T PB
where t is an auxiliary variable. Upper and lower
bounds on the magnitude can also be added to the
above SDP.
1
−1

5.3 Log-Chebychev approximation on magni-

tude

dB
−20

Since the magnitude of the desired frequency response

is usually represented in decibels, it is sometimes more
natural to perform the Chebychev approximation on a
logarithmic scale:
−40

 
minimize i=1max  log jX (!i )j , log jD(!i )j : 0.06 0.12 0.5

;:::;M Normalized Frequency

Again, the problem can be cast as a convex problem in
r, P and t:
1
−1

minimize t
subject to 1=t  R(!i )=jD(!i )j2  t; i = 1; : : : ; M

dB
−20

 
P , AT PA C T , AT PB  0;
C , B T PA D + DT , B T PB
−40

which can be further reduced to an SDP using Schur

complements: 0.06 0.12 0.5

Normalized Frequency
minimize t 10

subject to R(!i )=jD(!i )j2  t; i = 1; : : : ; M 8

 
R(!i )=jD(!i )j2 1  0; i = 1; : : : ; M
6

1 t
4
Group-delay

 
P , AT PA C T , AT PB  0:
2

C , B T PA D + DT , B T PB
0

−2

−4

−6

6 Examples −8

−10

Example 1
0.06 0.12 0.5

Normalized Frequency
We design a low-pass lter of minimum length, with Figure 1: Magnitude bounds and lter response (magni-
passband [0; 0:06] and stopband [0:12; 0:5] in normal- tude and group-delay) of Example 1. The dot-
ized frequency (Nyquist rate is equal to 1), that sat- ted line indicates the magnitude bounds and
ises the magnitude bounds shown in Figure 1. The the solid line indicates the response.
minimum lter length is 20. One of the solutions is
shown in the gure. Note that the lter has roughly
linear phase in the passband.
Example 2
1
−1

Consider the same passband and stopband specica-

tions as in the previous example, we apply Chebychev
approximation to the ideal lowpass power spectrum us-
dB

−20

ing a 25-tap lter. The magnitude response of the opti-

mal lter is shown in Figure 2. Comparing to Example
1, this design has atter passband response but higher
stopband attenuation.
−40

Example 3 0.06 0.12

Normalized Frequency
0.5

We consider the same Chebychev approximation prob-

lem as in the previous example, but with the magni- Figure 2: Filter response (magnitude) of Example 2
tude bounds from Example 1. The frequency response
 1
−1

dB
−20

−40

0.06 0.12 0.5

Normalized Frequency
Figure 3: Filter response (magnitude) of Example 3
10

of the optimal 25-tap lter is shown in Figure 3. With 0

the help of magnitude bounds, this design achieves the −5

same stopband attenuation in Example 1.

dB
−10

Example 4 −15

We design the minimum-length lter that satises the −20

bandpass magnitude bounds shown in Figure 4. The re-

sult is a 24-tap lter with the frequency response mag-
−25

nitude shown in the same gure. −30

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

Normalized Frequency
10

7 Concluding remarks 5

We have presented an SDP formulation of several FIR

0

lter design problems: −5

dB

−10

 The feasibility problem: nd an FIR lter that −15

satises given upper and lower bounds on the fre- −20

quency response magnitude, or show that no such

lter exists.
−25

−30

 The problem of nding the minimum length lter

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

Normalized Frequency
that satises the upper and lower bounds.
Figure 4: Filter specication and response (magnitude)
 Chebychev-approximation of a desired power of Example 3
spectrum.
 Chebychev-approximation of a desired frequency
response magnitude on a logarithmic scale.
 Chebychev-approximation with guaranteed mag-
nitude upper/lower bounds.

Many other extensions that have not been discussed

in the paper can be handled in the same framework,
such as, maximum stopband attenuation or minimum
transition-band width FIR design given magnitude
bounds, or even linear array beam-forming. Recent
interior-point methods for semidenite programming
can solve each of these problems very eciently.
Acknowledgments programming technique. I. the method. AEU, Archiv
The authors would like to thank Babak Hassibi, Lau- fuer Elektronik und Uebertragungstechnik: Electronics
rent El Ghaoui and Herve Lebret for very helpful dis- and Communication, 48(3):135{144, 1994.
cussions and comments. [14] K. Preuss. On the design of FIR lters by com-
plex chebychev approximation. IEEE Trans. Acoust.,
Speech, Signal Processing, 37:702{712, 1989.
References [15] M. Schulist. Improvements of a complex FIR l-
[1] A. S. Alkhairy, K. F. Christian, and J. S. Lim. ter design algorithm. Signal Processing, 20:81{90, 1990.
Design of FIR lters by complex Chebyshev approx- [16] L. Vandenberghe and S. Boyd. sp: Software for
imation. IEEE Trans. Signal Processing, 41:559{572, Semidenite Programming. User's Guide, Beta Ver-
1993. sion. K.U. Leuven and Stanford University, Oct. 1994.
[2] B. Anderson and S. Vongpanitlerd. Network [17] L. Vandenberghe and S. Boyd. Semidenite pro-
analysis and synthesis: a modern systems theory ap- gramming. SIAM Review, 38(1):49{95, Mar. 1996.
proach. Prentice-Hall, 1973.
[18] S.-P. Wu and S. Boyd. sdpsol: A Parser/Solver
[3] S. Boyd, L. El Ghaoui, E. Feron, and V. Bal- for Semidenite Programming and Determinant Max-
akrishnan. Linear Matrix Inequalities in System and imization Problems with Matrix Structure. User's
Control Theory, volume 15 of Studies in Applied Math- Guide, Version Beta. Stanford University, June 1996.
ematics. SIAM, Philadelphia, PA, June 1994.
[19] W.-S. Yu, I.-K. Fong, and K.-C. Chang. An
[4] S. Boyd and L. Vandenberghe. Intro- l1 -approximation based method for synthesizing FIR
duction to convex optimization with engineer- lters. IEEE Trans. Communications, 39(8):578{581,
ing applications. Lecture Notes, Information 1992.
Systems Laboratory. Stanford University, 1995.
http://www-isl.stanford.edu/people/boyd/392/.
[5] S. Boyd, L. Vandenberghe, and M. Grant. Ef-
cient convex optimization for engineering design. In
Proceedings IFAC Symposium on Robust Control De-
sign, pages 14{23, Sept. 1994.
[6] X. Chen and T. W. Parks. Design of FIR lters
in the complex domain. IEEE Trans. Acoust., Speech,
Signal Processing, 35:144{153, 1987.
[7] J. O. Coleman. Linear-programming design of
hybrid analog/FIR receive lters to minimize worst-
case adjacent-channel interference. In Proc. Conference
on Information Sciences and Systems, Mar. 1993.
[8] O. Herrmann and H. W. Schussler. Design of
nonrecursive digital lters with minimum-phase. Elec-
tronic Letter, 6:329{330, 1970.
[9] R. Hettich. A review of numerical methods for
semi-innite programming and applications. In A. V.
Fiacco and K. O. Kortanek, editors, Semi-Innite Pro-
gramming and Applications, pages 158{178. Springer,
Berlin, 1983.
[10] Y. Nesterov and A. Nemirovsky. Interior-point
polynomial methods in convex programming, volume 13
of Studies in Applied Mathematics. SIAM, Philadel-
phia, PA, 1994.
[11] A. Papoulis. Signal Analysis. McGraw-Hill, New
York, 1977.
[12] T. W. Parks and C. S. Burrus. Digital Filter
Design. Topics in Digital Signal Processing. John Wiley
& Sons, New York, 1987.
[13] A. W. Potchinkov and R. M. Reemtsen. FIR l-
ter design in the complex domain by a semi-innite