Copyright © 1985, by the author(s).

All rights reserved.

Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. To copy otherwise, to republish, to post on servers or to redistribute to
lists, requires prior specific permission.
WAVEFORM RELAXATION: THEORY AND PRACTICE

by

J. White, F. Odeh, A. S. Vincentelli

and A. Ruehli

Memorandum No. UCB/ERL M85/65

30 July 1985
 WAVEFORM RELAXATION: THEORY AND PRACTICE

by

J. White, F. Odeh, A. S. Vincentelli, and A. Ruehli

Memorandum No. UCB/ERL M85/65

30 July 1985

ELECTRONICS RESEARCH LABORATORY

College of Engineering
University of California, Berkeley
94720
 WAVEFORM RELAXATION: THEORY AND PRACTICE

J. White, F. Odeh,. A.. S. Vincentelli,. A. Ruehli

Abstract: This paper surveys the family of Waveform Relaxation Methods for

solving large systems of ordinary nonlinear differential equations. The basic WR

algorithm will be reviewed, and many of the derivative algorithms will be

presented, along with new convergence proofs. In addition, examples will be

analyzed that illustrate several of the implementation techniques used to

improve the efficiency of the basic WR algorithm, along with theoretical results

that indicate the strengths or limitations of these techniques.

INTRODUCTION

The tremendous increase in complexity of engineering design and availability of

computing resources has made computer simulation an important and heavily used

tool for both research and engineering design. Since many simulation problems are

formulated as large systems of nonlinear ordinary differential equations (ODE's),

much research work has been devoted to solving ODF systems efficiently.

The standard approach to solving ODE systems is based on three techniques [l],
[2]:

i) Stiffly stable implicit integration methods, such as the Backward Difference formu

las, to convert the differential equations which describe the system into a sequence of

nonlinear algebraic equations.

ii) Modified Newton methods to solve the algebraic equations by solving a sequence of
linear problems.

iii) Sparse Gaussian Elimination to solve the systems of linear equations generated by

the Newton method.

 The above approach can become inefficient for large systems where different state

variables are changing at very different rates. This is because the direct application of

the integration method forces every differential equation in the system to be discretized

identically, and this discretization must be fine enough so that the fastest changing

state variable in the system is accurately represented. If it were possible to pick

different discretization points, or timesteps. for each differential equation in the system

so that each could use the largest timestep that would accurately reflect the behavior

of its associated state variable, then the efficiency of the simulation would be greatly

improved.

Several modifications of the direct method have been used that allow the indivi

dual equations of the system to use different timesteps[3][4][5][6][7][8]. The approach

that will be discussed in this paper is the family of Waveform Relaxation algorithms

[9A.9B.9C]. WR algorithms have captured considerable attention due to their favor

able numerical properties and to the success in applying the WR algorithms to the

solution of Metal-Oxide-Semiconductor (MOS) digital circuits.

In this paper we will both survey the current state of research in WR algorithms,

and present new theoretical and practical results The paper is organized as two parts.

In the first part we will present the theoretical background for the basic WR algorithm

and it derivatives. We will start by introducing waveform relaxation with a simple

example, and follow with the basic algorithm. Then a new proof of the convergence,

one that demonstrates that the WR algorithm is a contraction mapping in a particular

norm, will be presented. Extensions to the basic algorithm that allow for modified

iteration equations (including discrete approximations) will be presented and it will be

shown that the convergence of such extensions follows directly from the proof that the

WR algorithm is a contraction mapping. The extension of the Newton Method to func

tion spaces will then be presented, and its convergence proved using lemmas from the
basic theorem. Finally, discretization approximations will be considered in more detail,

by comparing relaxation and explicit integration methods for a sample stiff problem.

In the second part we will analyze examples that illustrate several of the imple

mentation techniques used to improve the efficency of the basic WR algorithm, and

prove theorems that indicate the strengths or limitations of these techniques. We will

start by considering approaches for partitioning large systems into loosely coupled

subsystems. We will then examine how breaking the simulation interval into pieces,

called windows, can be used to reduce the number of relaxation iterations required to

achieve convergence. Two techniques for reducing the iteration computation will then

be presented. The first is based on performing one iteration of a Newton method with

each relaxation iteration, and the second is based on exploiting piecewise linearity.

Because the WR algorithm has proved to be an efficient technique for simulating MOS

digital circuits, the examples used throughout this paper are drawn from this area. In

order to more clearly demonstrate both the practicality of the WR algorithm, and the

specific nature of its efficiencies, we will end the second section, and the paper, by exa

mining in detail application of the WR algorithm to the simulation of MOS digital cir

cuits.

SECTION 1 - THE WR ALGORITHM AND ITS DERIVATIVES

SECTION 1.1 - THE BASIC WR ALGORITHM

We will start this section with a simple illustrative example, and then present

the general WR algorithm. Consider the first-order two-dimensional differential equa

tion in: x U ) 6 IR2 on t € [0.7] .

xl = fl(xl,x2.t) x1(0) = x10 (1.1.1a)

x2 - f2(xx,x2,t) x2{0') = x2o (1.1.1b)

The basic idea of the waveform-relaxation algorithm is to fix the waveform

x2:[0.T]-*JR and solve Eqn. (1.1.la) as a one dimensional differential equation in

xj(r). The solution thus obtained for x\(t) can be substituted into Eqn. (1.1.lb)
which will then reduce to another first-order differential equation in one variable,

x2(t) . Eqn. (1.1.1a) is then re-solved using the new solution for x2(t) and the pro
cedure is repeated.

Alternately, fix the waveform x2(t) in Eqn. (1.1.la) and fix x}(t) in Eqn.

(1.1.lb) and solve both one dimensional differential equations simultaneously. Use the

solution obtained for x2 in Eqn. (1.1.1b) and the solution obtained for Xi in Eqn.

(1.1.la) and re-solve both equations.

In this fashion, iterative algorithms have been constructed. Either replaces the

problem of solving a differential equation in two variables by one of solving a sequence

of differential equations in one variable. As described above, these two waveform

relaxation algorithms can been seen as the analogues of the Gauss-Seidel and the

Gauss-Jacobi techniques for solving nonlinear algebraic equations. Here, however, the

unknowns are waveforms (elements of a function space), rather than real variables.

In this sense, the algorithms are techniques for time-domain decoupling of differential

equations.

The most general formulation of a system of nonlinear differential equations is

the following implicit formulation:

Fix (r). xU), u(t) ) = 0 x(0) = x{) (1.1.2)

where x(t ) € IR" on t € [O.T] : u(t) € IRr on t € [0.7] is piecewise continuous: and

F: JR" x JRn x JRr -* IR" is continuous.

In order to guarantee that WR applied to Eqn. (1.1.2) will converge to the

system's solution, we first must guarantee that Eqn. (1.1.2) has a solution. If we
require that there exists a transformation of Eqn. (1.1.2) to the form y = / (y ,u)
where / is Lipshitz continuous with respect to y for all u , then a unique solution for

the system exists[22]. Although there are many sets of broad constraints on F that
guarantee the existence of such a transformation, the conditions can be difficult to ver

ify in practice. In addition, for the above system, it is difficult to determine how to

assign variables to equations when applying the WR algorithm. That is, when solving

the Ft equation of the system in the iteration process, what Xj variable should be

solved for implicitly. If a poor choice is made, the relaxation may not converge[9B].

Rather than carefully considering the existence and assignment questions, which

will complicate the analysis that follows without lending much insight, we will con

sider the following less general form, in which many practical problems, particularly

circuit simulation, can be described.

C(x(t), uU))x (t) = f (x(t).u(t)) x(0) = x0 (1.1.3)

where x(t) € IRn on t € [0.7] : u(t) € IRr on t € [0.7] is piecewise continuous;

C: IRn x IRr —» IR"*" is such that C Or , u )_1 exists and is uniformly bounded with
respect to x ,u ; and / : IR" x W -»IR" is globally Lipschitz continuous with respect to

x for all u (t ) 6 lRr.

The fact that C (x , u ) has a well-behaved inverse guarantees the existence of a

normal form for Eqn (1.1.3). and that x (t) € IR" is the vector of state variables for

the system. Then as / is globally Lipschitz with respect to x for all u , C (x . u )-1 is
uniformly bounded, and u (t ) is piecewise continuous, there exists a unique solution to

Eqn. (1.1.3).

The WR algorithm for solving the above system is as follows:

Algorithm 1.1.1 (WR Gauss-Seidel Algorithm for solving Eqn. (1.1.3))
Comment:
The superscript k denotes the iteration count,
the subscript i denotes the component index
of a vector and € is a small positive number.

guess waveform x°(t) ;t € [0,7]

such that x°(0) = x0
(for example, set x°(t) = x0,t € [0,7] );
repeat {
 k<-k+l

foreach (i € { l,..,n } {
solve

tX\ »•*!'+1 » • .x* 1.u)xj +

t Ci}{x\. ' ' * 'Xi 'Xi+1 • • ' ' .X*-1,!/)^*"1—

j =i +1

fi(x\ . • • ^/.xZ+l1 . • • • .X*"1.!/) =0

for ( x*(? ) : £ € [0,7] ). with the initial condition x*(0) = x, .

}
}until ( maxj<,<„ max, € [oj]'XK* ) ~ ** KOI ^ € )
that is. until the iteration converges. •

Note that the differential equation in Algorithm (1.1.1) has only one unknown vari

able x* . The variables x/^1. • • • ,x*_1 are known from the previous iteration and the
variables x\ . • • • .x^-i have already been computed. Also, the Gauss-Jacobi version of
the WR Algorithm for Eqn. (1.1.3) can be obtained from Algorithm (1.1.1) by replac

ing the foreach statement with the forall statement and adjusting the iteration

indices.

SECTION 1.2 - CONVERGENCE PROOF FOR THE BASIC WR ALGORITHM

If the matrix C(x,w) is diagonally dominant and Lipschitz continous with

respect to x for all u then both the Gauss-Seidel and the Gauss-Jacobi versions of

Algorithm (l.l.l) are guaranteed to converge. In [9A], it was shown that the WR

algorithm converges when applied to Eqn. (1.1.3) if C(x,w) is diagonally dominant

and independent of x. As many systems that are modeled in the form of Eqn. (1.1.3)

include a dependence of C on x , we will present a more general convergence proof that

extends the original theorem to include these systems. In addition, we will prove the

WR method is a contraction in a simpler norm than the one used in the original

theorem.
 We will prove the theorem by first showing that if C (x ,u ) is diagonally dom
inant, then there exists a bound on the xk 's generated by the WR algorithm that is
independent of k . Using this bound, we will show that the assumption that C (x ,u)
is Lipschitz continuous implies there exists a norm on 1R" such that for arbitrary posi

tive integers j and k :

Hk*Kt>-x>+Kty\ ^y\\xk(t)-xHtl +l1\\xk+Kt)-x->+1U)\\ +Z2l|x*(r)-x;(OII

for some y< 1 and l\l2 < °° for all t € [0.7] . In the properly chosen norm || «i|6 on
C ([0,7],IRn ), the above equation implies that

ll**+1-*'+1H* <\\xk-xi\\b
and therefore the sequence { x* } converges by the contraction mapping theorem. As
x* (0) = x0 for all k, { x* } converges as well.

Before formally proving this basic WR convergence theorem we will state the

well-known contraction mapping theorem[l6], and a few lemmas which will be used

in the course of the proof.

The Contraction Mapping Theorem; Let Y be a Banach space and F:Y-*Y. If F is

such that \\F(y ) - F(x)|| < y||y-x|| for all x ,y € Y , for some y 6 [0.1) , then F

has a unique fixed point y such that F (y ) —y. Furthermore, for any initial guess

y° € Y the sequence { yk € Y } generated by the fixed point algorithm

yk = F(y*-1) converges uniformly to y .

Lemma 1.2.1; If C(x,u) € IRnxn is diagonally dominant uniformly over all x G IR" .

u 6 IRr then given any collection of vectors { x1, • • ,xn } . x' € IR" . and any
u e W . the matrix C(x1. • • • ,x" ,u ) € IR"™ defined by

Cfj (x *, • • • pcn ,u ) = Cij (x' ,u ) is also diagonally dominant. In other words, let Cp be
the matrix constructed by setting the ith row of Cp equal to the ith row of the given

matrix C (x' ,u) . Then this new matrix is also diagonally dominant.

Lemma 1.2.1 follows directly from the definition of diagonal dominance.

 8

Lemma 1.2.2; Let C € lRnxn be any strictly diagonally dominant matrix. Let L,

strictly lower triangular, U, strictly upper triangular, and D, diagonal, be such that

C = L + D + U . Then \\D~KL +U )||« < 1 and || (D +L TWl«, < 1.

Lemma 1.2.2 is a standard result in matrix theory[24].

Lemma 1.2.3; Let x ,y 6 C ([0,7],IRn ) . If there exists some norm on IR" such that

II* (rS < *(OII +/il|xU)|| + Z2||y(OII (1.2.1)

for some positive numbers l\X2 < ©o and y < 1 then there exists a norm || • \\b on
C([0,7],IR'1) and a positive number a < 1 such that

11*11* ^ *IU + Zjlx^)!! + Zj|y(0)|| (1.2.2)

Proof of Lemma 1.2.3;

Substituting Jx(r)rfT + x(0) for x(t) in Eqn. (1.2.1), performing an analogous

o

substitution for y (t), multiplying the entire equation by e~bl, and moving the norms

inside the integral yields:

e-*'||x(f)|| < ye-*'lbKOII +Z]e-fe'/||x(r)|kT + Z1e-^||x(0)|| + (1.2.3)

o

l2e-b'f\\y(T)\\dT + l2e-b'\\y(0)l
o

Let || • ||6 be defined by ||/ \\b =max[0,r?~bl 11/ U )|| . This is a norm on C ([0.7IlR" ) for
any finite positive number b > 0 and is equivalent to the uniform norm on

C([0.7].IR" ) . Then Eqn. (1.2.3) implies

II* IU < y\\y\\b + maxioj}[Zie-6'/e6VT ||x||, +I,e"*||x(0)|| + (1.2.4)

o

l2e-blJebrdr\\y\\b +he^'h(0)|| ]
o

t ..

And since e-6' febrd r ^ -=-, then for b >l 1we can write
 \\x\\b <£J4=rltf
1—l ib
I'* +*(0)H +z2bCo)||. (1.2.5)
(y+Z25_1)
In this case v is less than 1, so there exists a finite B for which ,— = a < 1 .
' 1-Z^B"1
Let the b in Eqn. (1.2.5) be set equal to this B to get

Bills < «l|y||B + Z,||*(0)|| + Z2||y(0)|| (1.2.6)

which completes the proof. •

Now we prove the following WR convergence theorem for systems of equations

of the form of Eqn (1.1.3).

Theorem 1.2.1; If, in addition to the assumptions of Eqn. (1.1.3),

C(x (t ).u (r )) 6 ]Rnxn is strictly diagonally dominant uniformly over all x (t ) € IR"

and u {t) € IRr and Lipschitz continuous with respect to x (t) for all u (t). then the

sequence of waveforms { x* } generated by the Gauss-Seidel or Gauss-Jacobi WR algo

rithm will converge uniformly to the solution of Eqn. (1.1.3) for all bounded intervals

[0.7] .

Proof of Theorem 1.2.1;

We will present the proof only for the Gauss-Seidel WR algorithm, as the proof

for the Gauss-Jacobi case is almost identical. The equations for one iteration of the

Gauss-Seidel WR algorithm applied to Eqn. (1.1.3) can be written in matrix form as

C(xk+1.xk .u)xk+1 = f (xi+1.xk ,u)

where C,-; (x*+1, x* ,u ) = C^ (x* +1 , • • • , x/ +1, x/+1. • • • . xk,u ) and
7i(**+1.jc*^) = /,(jei+1. • •.xf+1.x/+1. •••,x*.w). Let
C(xl +1, x* ,u ) = Lk+1 + Dk +1 —Uk +1 where Lk +1 is strictly lower triangular, Uk+i is
upper triangular, and Dk+i is diagonal (Note that by Lemma 1.2.1. the matrix C is

diagonally dominant because C is diagonally dominant). Rearranging the iteration

equation yields:
 10

x*+1 = Ui+i+A+ir1 [Ul+1xk + / (x*+1.x* .u) ]. (1.2.7)

Taking the difference between Eqn. (1.2.7) at iteration k +1 and at iteration j +1 yields

i*+i_jc'+i = a^+z^+jrty.H** -aJ+1+Dj+1riUj+1xj + (1.2.8)

at+i+Z)^,)-1/ (x*+1.x* ,u)~ Uj+i+Dj+O-1} (x^.x^.a)
Using the Lipschitz continuity of / and that II (Z,jt+i+At+i)_1H <J5T for some K <oo
independent of x and /:, (because C(x ,u) is uniformly diagonally dominant with

respect to x ) in Eqn. 1.2.8 leads to

iix*+l(o-x;+1(on ^i&w+Kty-x'+Ktyi + i2K\\xku)-xHt)\\ + (1.2.9)

iia,+1+z),+1ri-(Lj+1+/>j+1)-iii ii/(x^+i.x^.w)ii +
II Uk+i+At +xTlUk +1x* {t )-(Lj +i+Dj +lTlUj +lxJ it )||
where l\ is the Lipschitz constant of / with respect to its first argument, and Z2 is the

Lipschitz constant of / with respect to its second argument. That C (x ,u) is uni
formly diagonally dominant and Lipschitz continuous with respect to x for all u

implies (Lk +Dk )-1 and (Lk +Dk )-1f/x. are also Lipschitz continuous in the same
manner. It then follows that there exist some positive finite numbers &j , k2 . k3 , k4

such that

\\xk+1(t)-xi+\t)\\ =Z1#Hx*+1(0-*-/+1(OII +Z2#||x*(r)-*-'(r)ll + (1.2.10)

[*3ll**+1(0-*;+1(r)|| + *J**a)-*'(OI|]||/(*-'+,.*',i/)li +
[*,||**+1a )-x->+1U )|| + *2||x* it }-xHt )|| ] II*' (r )|| +y||xA U )-x' (t )||
where kl is the Lipschitz constant of {Lk +Dk )~1Uk with respect to its first x argu
ment (see definition of Lk . Uk and Dk above). k2 is the Lipschitz constant with respect

to the second x argument. k3 and k4 are the Lipschitz constants for (Lk +Dk )_1 with
respect to its first and second x arguments, and y is such that || (Lk +Dk }~lUk || < y < 1
independent of k (by Lemma 1.2.2).

To establish a bound on the terms in Eqn. (1.2.10) involving ||x* (t )|| and
11/ (xi+1, xj ,u)\\ it is necessary to show that the x* *s and therefore the x* *s and
/ (•)*s are bounded a priori. We prove such a bound exists in the following lemma.
 11

Lemma 1.2.4; If C(x.w) in Eqn. (1.1.3) is strictly diagonally dominant and Lipschitz

continuous then the x*(0's produced by Algorithm (1.1.1) are bounded independent
of k .

Proof of Lemma 1.2.4

If ||*|| is the Z^norm on IRn . by Lemma 1.2.1 || (Z,^ +1+Z)it +1)~1^jt +1H < 1. From

Eqn. (1.2.7).

II**+1U)II <y{\xk(t)\\ +||Ufc+1+At+1r1|| ||/(x*+1(0.x*U).iOII (1.2.11)

for some positive number y < 1 . As / (x ,u ) is globally Lipschitz continuous with

respect to x , there exist finite positive constants l\,l2 such that

\\f(x.y.u)-f{w.z.u)\\ <Zil|x-w|| +Z2||y-z|| (1.2.12)

for all u ,x ,y ,w ,z € JR" . From Eqn. (1.2.11) and Eqn. (1.2.12) and using the fact

that || (Lk +i+A +i)-1ll is bounded by some K < c© for all k :

\\xk+1(t )|| < yllx* U )|| +l1K\\xk+Kt )ll +l2K\\xk (t )|| +*||/ (0,0.u )|| (1.2.13)
Eqn. (1.2.13) is in the form to apply a slightly modified Lemma 1.2.3. Therefore there
exists some || •||fc such that

||x*+1||6 < a\\xk\\b + (hK+l.lOWxm + *ll/ (0,0. u)\\ (1.2.14)

where oc < 1. This implies that

\\xk+\ ^ -J-KZ^+Z^llxtOH +K\\f (0.0. u)\\ ]+(<*)* \\x\ (1.2.15)

l—oc

for all k. Then, since ||x°||fe must be bounded given a finite x(0), and
\\xk+% =max,0J)e-*'||x*+1(r)|| .

||**+1(r )ll < e^-i-KZ^+Z^HxU))!! +K\\f (0.0.u)|| ]+||x%, ] = M

1—Of
(1.2.16)
which proves the lemma. •

In Lemma 1.2.4 it was proved that ||x*(*)|| is bounded a priori by some M.

This implies xk (t ) is bounded on [0.7]. Using the Lipschitz continuity property of / ,
a bound. N, can be derived for ||/ (xk+1(t ), x*(0." )ll - Applying these bounds to
Eqn. 1.2.10 we get
 12

||x*+1(0-*;+1(OII < y\\xk(t)-xnt)\\ + (1.2.17)

(llK^k1M+k^N)\\xk+Kt)-xi+Kt1i\ +(Z2i:+Afik2+A:4iv)||xA(f)-x^(r)||
where y < 1. Eqn. (1.2.17) is of the form to apply Lemma 1.2.3. As
x*+1(0)-xj+1(0) = o for all k ./.Lemma 1.2.3 implies

ll**+1-*j+1ll* ^ aHx*-Jc'||A (1-2.18)

for some norm on C([0.7],IRn ) and for some a < 1 . As C([0,7],IRn ) is complete in

any one of the B norms, by the contraction mapping theorem xk converges to some
x € C([0,7],IRn) which is a fixed point of Eqn. (1.2.5). Any fixed point x of Eqn.

(1.2.5) is a solution to Eqn. (1.1.3) if x (0) = x0 . xk (0) = x0 for all k , therefore xk

converges to the unique solution of Eqn. (1.1.3). The sequence { xk } converges
because integration from 0 to 7, which maps x (t) to x (t), is a bounded continuous
function •.

SECTION 1.3 - NONSTATIONARY WR ALGORITHMS

Algorithm 1.1.1 is stationary in the sense that the equations that define the

iteration process do not change with the iterations. A straight-forward generalization

is to allow these iteration equations to change, and to consider under what conditions

the relaxation still converges [9]. There are two major reasons for studying nonsta-
tionary algorithms. The solution of the ordinary differential equations in the inner

loop of Algorithm 1.1.1 cannot be obtained exactly. Instead numerical methods com

pute the solution with some error which is in general controlled, but which cannot be

eliminated. However, the discrete approximation can be interpreted as the exact solu

tion to a perturbed system. Since the approximation changes with the solutions, the

perturbed system changes with each iteration. Hence, practical implementations of WR

that must compute the solution to the iteration equations approximately can be inter

preted as nonstationary methods.

The second reason for studying nonstationary methods is that they can be used to

improve the computational efficiency of the basic WR algorithm. An approach would

 13

be to improve the accuracy of the computation of the iteration equations as the relaxa

tion approaches convergence. In this way, accurate solutions to the original system

would still be obtained, but unnecessarily accurate computation of the early iteration

waveforms, which are usually far from the final solution, is avoided.

In this section we show that nonstationary WR algorithms converge as a direct

consequence of the contraction mapping property of the original WR algorithm. That

is, given mild assumptions about the relationship between a general stationary contrac

tion map and a nonstationary map, the nonstationary map will produce a sequence

that will converge to within some tolerance. And if in the limit as k —»oo the nonsta

tionary map approaches the stationary map, then the sequence generated by the nons

tationary map will converge to the fixed point of the original map. In later sections we

will lean on these results to guarantee the convergence of implementations of WR-

based algorithms.

Theorem 1.3.1: Let Y be a Banach space and F, Fk :K -»}' . Define y *+1 = F(y * ) and
~*+i _ pk Qk ) If .F is a contraction mapping with contraction factor y (See section
1.2). \\F(y)- Fk(y)\\ < 8* for all y € Y, and z € Y, is such that z = F{z), then

for any e > 0 there exists a 8 < 1 such that if 8; < 8 for all k then

limlly*—y*-1|| < € and klim|U—y*ll < -^1—y . Futhermore. if klim8* -♦ 0 then

k -•oo -«oo -*x>

limlly'-y*-1!! - 0 and lim||z-y*|| - 0.

k -*oo k —«o

Proof of Theorem 1.3.1

Taking the norm of the difference between the k'h and k+lsl iteration of the
nonstationary algorithm we get:

II5*+I-5*II <\\Fk+1Qk)-Fk(yk-1)W (1-3.1)

Given that \\Fk (y )-F(y )|| < 8* for all y €Y

H5*+1-y*ll < WFCy'l-FCy1-1! + 8k + 8*+1. (1.3.2)

Using the contraction property of F ,
 14

lly*+1-5*ll < -Hly*-}*"1!! + 8* + 8*+1. (1.3.3)

Unfolding the iteration equation into direct sum form.

Ily*+1-y*ll6 < 8k+1+8k +iy-^+S'"-1). (1.3.4)

i=l

If 8k < 8 for all k then from Eqn. (1.3.4)

lim||5*+1-5*|| ^2S(1 +—L-).

k->oo 1—y
(1.3.5)
As y<l . lim||y*+1 —y*ll can be made as small as desired by reducing 8 , which
it -K»

proves the first part of Theorem 1.3.1.

Let y be the fixed point of F . The difference between the computed and the

exact solution at the k+Vh iteration is

\\yt+1-y\\ =llF*(y*HF(y)||. (1.3.6)

Again using the contractive property of F and that ||F(y ) —Fk (y )|| ^ 8k .

Ily*+1-yll =ylly*-yll +8*. (1.3.7)

Summing and taking the limit,

lim||yi+1-yl|, ^ 1 —y
k -*oo
* (1.3.8)
which completes the proof of the first statement of Theorem 1.3.1. The second state

ment of the theorem follows from almost identical arguments. •

In Section 1.2 we proved the WR iteration was a contraction mapping in the

appropriate norm || • \\b on C([0,7].IR" ) where B depended on the problem. To repeat

the result from that section, it was shown that:

ll**+1-*j+% <a||*'-*V||6
This WR convergence result and Theorem 1.3.1 imply that using any "reasonable"
approximation method to solve the WR iteration equations will not affect the conver

gence provided the errors in the approximation are driven to zero. In addition.

Theorem 1.3.1 indicates that it will be difficult to determine a priori how accurately

the iteration equations must be solved to guarantee convergence to within a given

 15

tolerance, because an estimate of the contraction factor of the WR algorithm is

required.

As can be seen from Eqn. (1.3.8). the WR algorithm is a contraction mapping

with respect tox(0 in a B norm. Theorem 1.3.1 then implies that the WR iteration

equations must be solved accurately with respect to x (t ) in this B norm if the itera

tions are to converge. There is a more cumbersome proof of the WR convergence

theorem in which it is shown that the WR algorithm is a contraction in x (t ). but in a

larger B norm than the one used in the proof of Theorem 1.2.1. and the size of this B

is a function of the magnitude of the off-diagonal terms of C (x ,u ) . With such a

result. Theorem 1.3.1 implies that it is only necessary to control errors in the compu

tation of x (t) to guarantee iteration convergence. However, convergence in a larger B

norm is in some sense a weaker type of convergence. So. in the case where C(x ,u) has

non-zero off-diagonal terms, it is expected that more rapid convergence would be

achieved if the x* (t) 's are computed in a way that also guarantees that the xk (t )'s
are globally accurate.

SECTION 1.4 - WAVEFORM NEWTON METHODS

The WR algorithm is an extension to function spaces of the popular relaxation

methods used to solve nonlinear algebraic problems. Another popular method for

solving nonlinear algebraic problems is the Newton-Raphson method, and its function

space extension also has practical applications. In this section we will derive the

function-space Newton method applied to systems of the form of Eqn. (1.1.3) and

prove that the method has globed convergence properties, which is not true in general

for the Newton-Raphson algorithms[22].

In order to derive a function-space extension to the Newton-Raphson algorithm,

let F(x ) ( from Eqn. (1.1.3) ) be defined as

F(x) = C(x.w)x -/(x.w) = 0 x(0) = x0 (1.4.1)

 16

where x:[OX]-»IRn ,u:[0,7]-»IRr and is piecewise continuous; C: IRnxIRr -» JRRXn is

such that Cix ,u )_1 exists and is uniformly bounded with respect to x ,u ; and
/ : IR" x IRr -» IR" is globally Lipschitz continuous with respect to x for all u .

Applying the Newton-Raphson algorithm to find an x such that F (x ) = 0 given

some initial guess x° we get

xk+i = xk _ jfi(xk)F(xk ) (1 4 2)
where JFix ) is the Frechet derivative of Fix ) with respect to x . Note that in this
case JFix) is a matrix-valued function on [0.7] . That is. JFix) is a matrix of

waveforms.

Using the definition of the Frechet derivative, we can compute JFix) ,

hh™0 (1/ \\h | )||F(x +h )- Fix ) - JF(x Xh )|| - 0. (1.4.3)

Evaluating this limit for the Fix ) given in Eqn. (1.4.1) we get

Fix +/i ) - Fix) = Cix+h ,u)ix+h) - C (x ,u )x - f (x +h ,u ) + / (x ,u )

and approximating to order ||/z|| 2

Fix +h )-Fix ) =Cix,u )h + dC[x'u\h )x - $f[x'uh +OQ\h\\ 2) (1.4.4)

6* 8*
As Eqn. (1.4.3) applies only in the limit as h -*0 , Eqn. (1.4.4) implies

JFix)h =Cix.u)h + §£^Hlhx

6x
- *f<*#\
Qx
(1.4.5)
Substituting the computed derivative into Eqn. (1.4.2) and rearranging we get

Cixk ,u)xk+l + dCi*k'u\xk+1-xk)xk = (1.4.6)

9*

9*

xi+1(0)=x0
We will refer to Eqn. (1.4.6) as the Waveform-Newton(WN) algorithm for solving

Eqn (1.1.3). It is, however, just the function-space extension of the classical Newton-

Raphson algorithm.
 17

Newton-Raphson algorithms converge quadratically when the iterated value is

close to the correct solution, but they do not in general have global convergence pro

perties. However, the WN algorithm represented by Eqn. (1.4.6) does converge for
AC ( •*• 11 ^
any initial guess, given mild assumptions on the behavior of ———-— as in the fol-
0*

lowing theorem.

c)C (x u )
Theorem 1.4.1; For any system of the form of Eqn. (1.1.3) in which ———-— is
o*

Lipschitz continuous with respect to x for all u and / is continuously differentiate,

the sequence { x* } generated by the WN algorithm converges uniformly to the solu

tion of Eqn. (1.4.1).

Proof of Theorem 4.1

For this proof of the convergence of the Waveform-Newton method we will

assume that C(x ,u) is the identity, as the proof for the general case is much more

involved, and does not provide much further insight into the nature of the conver

gence. For the case C(x ,u ) = I Eqn. (1.4.6) can be simplified to

x*+1 =/ ixk M)+ e/(;*'>l/)(x*+1-x* ).

o*
(1.4.7)
Taking the difference between Eqn. (1.4.7) at iteration k +1 and the exact solution and

substituting ixk +1—x ) + (x —xk ) for xk+1—xk yields

jc*+1-jc = / (x* ,u)-f ix,u)+ Qfixk'u)[ixk+1-x

Qx
) +ix-xk )]. (1.4.8)
ft f ix u)
As / is continuously differentiable on [0,7] and Lipschitz continuous, —-— is
0*

bounded by the Lipschitz constant Z} . With this bound,

||x*+1-x|| < lji\xk+1-x\\ +Z,||x*+1-x|| +Z1||x*-x||. (1.4.9)

Lemma 1.2.3 can be applied to Eqn. (1.4.9) (with y = 0). Therefore there exists some

b < ooand y < 1 such that

||x*+1-x||„ ^c*||x*-x||6. (1.4.10)

Therefore { x* } converges to x , the fixed point of Eqn. (1.4.1). Given x*(0) = x0
 18

for all k , { xk } converges to the solution of Eqn. (1.4.1) on any bounded interval. •
SECTION 1.5 - DISCRETIZED WR ALGORITHMS

To compute the iteration waveforms for the WR algorithm it is usually necessary

to solve systems of nonlinear ordinary differential equations. The most popular tech

niques for solving these systems are the multistep integration formulas (such as the

Backward Difference or Trapezoidal formulas[l]). These methods approximate the ori

ginal differential equation by a sequence of algebraic equations corresponding to a col

lection of discrete points in time. The error in this discretization approximation is a

function of the timesteps, which are usually chosen small enough so that the

waveforms are computed to some a priori accuracy.

The convergence theorem presented in Section 1.2 is not immediately applicable to

the convergence of this discretized WR algorithm because the differential equations

that describe the decomposed systems are not solved exactly. However, one can view

the discretized WR algorithm as a nonstationary method. The theorems presented in

Section 1.3 can then be applied to guarantee WR convergence to the solution of the

given system of ODE's when the global discretization error, a function of the

timesteps. is driven to zero with the WR iterations.

If the global discretization error is not driven to zero, one may expect that the

WR algorithm will still converge to an approximate solution of the given system of

ODE"s. In this section we show that unless the timesteps used in the numerical

method are kept below some problem-dependent bound, the WR algorithm may not

converge. We will start by analyzing a simple example that demonstrates a possible

breakdown of the WR method under discretizations. Subsequently we will prove that

the discretized WR algorithm converges if the timesteps used are "small enough".

Finally, we will end this section by comparing explicit and implicit integration

methods for WR.

 19

Consider the two node inverter circuit in Fig. 1. The current equations at each

node can be written by inspection, and are:

Cx1+g1x1 + g2(x,-x2) = 0 (1.5.1)

Cx2+g2ix2—x1) + imi(*i^c2) + *m2(*i) = 0

xj(0) =x2(0) = 0.
In order to generate a simple linear example. im\,im2 were linearized about the point

where the input and output voltages were equal to half the supply voltage. Time is

normalized to seconds to get the following 2x2 example:

*i = -*i + 0.1x2 (1.5.2)

x 2 = —A.x j H—x 2

x2(0) = x2(0) = 0.
Note that the initial conditions given for the above example identify a stable equili

brium point.

The Gauss-Seidel WR iteration equations for the linear system example are:

xk+1 =-xi'+1 + 0.1x2 (1.5.3)

•k+l _ \„*+l _ „<t+l
X 2 — AX j X 2

xi+1 (0) = x\ (0) = xk2+l (0) = xj (0) = 0.

Applying the Implicit-Euler numerical integration method with a fixed timestep

h, ( xinh ) = —[xinh )—xiin—\)h )] ) to solve the decomposed equations yields the

h

following recursion equation for x2 in ) :

x£+1 in ) = _L-
1+h
x2+1 in -1) (1.5.4)

As an example, let \=200, h =0.5 and as an initial guess use x2inh) = nh.
which is far from the exact solution x 2 inh ) = 0. The computed sequences for the ini

tial guess and first, second and third iterations of Eqn. (1.5.4) are presented in Table 1.
 20

Table 1 - Values for x2 for Several Implicit-Euler Computed WR Iterations

STEP TIME INITIAL ITER #1 _j ITER #2 ITER #3

0 0 0 0 0 0
1 0.5 0.5 -1.111 2.469 -5.487
2 1.0 1.0 -3.704 11.52 -32.92
3 1.5 1.5 -7.778 31.55 -111.6
4 2.0 2.0 -13.17 66.21 -281.3
5 2.5 2.5 -19.66 117.9 -587.5
6 3.0 3.0 -27.02 187.9 -1075
7 3.5 3.5 -35.07 276.0 -1786
8 4.0 4.0 -43.64 381.5 -2751
9 4.5 4.5 -52.60 502.9 -3992
10 5.0 5.0 -61.85 638.4 -5519

As the Table 1 indicates, the WR algorithm diverges for this example. In fact,

Eqn. (1.5.4) indicates that the WR algorithm will converge only if

(1.5.5)
il+h) V0.1\

To understand this nonconvergence phenomenon consider the Gauss-Seidel WR

algorithm applied to Eqn. (1.1.3) with Cix ,u) = C . The WR iteration equation is

(identical to Eqn. 1.2.9):

xk+1 = iL +D)~1Uxk +/(x*+1,x*.i/). (1.2.9)

Applying Implicit-Euler yields:

xk+lin +l)-xk+1in ) = U +D TlUi xk (n +l)-xk in))+ (1.5.6)

h f ixk+1in+l), x*(n+l).tt).
In the limit as h-*oo , Eqn. (1.5.6) becomes equivalent to solving

/ ixk+1in +1), x* in +l),u )=0 . Since little is assumed about / other than Lipschitz
continuity, it is unlikely that this problem can be solved, in general, with a simple

Gauss-Seidel relaxation. However, in the limit as the timestep becomes small, Eqn.

(1.5.6) becomes

xk+1in+l)-xk+1in) = iL +D)-1Uixkin+l)-xkin) ).
and from the lemma in Section 1.2, the norm of || iL + Z))_1Z/|| < 1 so the relaxation
 21

is certain to converge. The timestep h can be viewed as a parameterization of this alge

braic problem. As the timestep decreases, the problem is continuously deformed from

one that may not be solvable by relaxation to one that is guaranteed to be solvable by

relaxation. We formalize this observation in the following theorem:

Theorem 1.5.1: If, in addition to the assumptions of Theorem 1.2.1. the WR iteration

equations are solved using a stable and consistent multistep method with a fixed

timestep h , for a finite number of points, then there exists an h' > 0 such that the

sequences { xk in) } generated by the Gauss-Seidel or Gauss-Jacobi discretized WR

algorithm will converge for all 0 < h < h ' .

The proof of this theorem can be found in [26].

Now consider solving Eqn. (1.5.3) using the computationally simpler Explicit-

Euler integration formula ( xinh) = —[x((/i +l)h )—xinh )] ). The recursion equa

tion for the x2 in ) 's is:

xk2+1in+l) = il-h)xk2+lin)- (1.5.7)

0A\h2[ [(l-/i)»x}(0)] + 'f(HrHxi(/) ]

The computed sequences {x2+1 }'s for the initial guess and first, second and third itera
tions of Eqn. (1.5.7) are given in Table 2. for the case of \=200, h =0.5 and

x2inh ) = nh .

As the table indicates, the Explicit-Euler discretized WR algorithm converges for

this example. In general, if explicit integration methods are used in the WR algorithm,

the iterations will converge for any fixed timestep.

Theorem 1.5.2; If, in addition to the assumptions of Theorem 1.2.1. the WR iteration

equations are solved using an explicit multistep method with a fixed timestep h , for a

finite number of timesteps. then the sequences { xk in ) } generated by the Gauss-Seidel

or Gauss-Jacobi discretized WR algorithm will converge for all /? > 0 .
 22

Table 2 - Values for x2 for several Explicit-Euler computed WR iterations

STEP TIME INITIAL ITER #1 ITER #2 ITER #3

0 0 0 0 0 0
1 0.5 0.5 0 0 0
2 1.0 1.0 0 0 0
3 1.5 1.5 -0.625 0 0
4 2.0 2.0 -1.875 0 0
5 2.5 2.5 -3.594 0.7813 0
6 3.0 3.0 -5.625 3.125 0
7 3.5 3.5 -7.852 7.422 -0.977
8 4.0 4.0 -10.19 13.67 -4.883
9 4.5 4.5 -12.61 21.63 -13.92
10 5.0 5.0 -15.06 30.96 -29.79

Proof of Theorem 1.5.2

The proof of this theorem follows from a simple inductive argument [21]. Let
xin ) be the exact solution to the system discretized using an explicit method. Assume

xk(m) = x(m) for all m < n . Since the integration method is explicit, xk in) and
xin) are the same function of u and xk (m ) , m < n . Therefore, x* in ) = x (n ) .

x* (0) = x(0) for all k by assumption, which completes the proof. Note that this
proof guarantees that the discretized WR algorithm converges precisely to x in ) , the

exact solution to the original discretized system, in n iterations. •

It should be noted that the above proof does not show the explicit discretized WR

algorithm is a contraction, nor does it show convergence on a fixed time interval

independent of h . A more general proof can be found in [20].

Insuring relaxation convergence puts no constraints on the timesteps for the

Explicit-Euler method, or for explicit methods in general. But since these methods

have small regions of absolute stability, the timestep may be limited not by accuracy

considerations but to insure stability. For example, consider the differential equation

of Eqn. (1.5.3), but with a perturbed initial condition. x\ (0) = x2(0) = 0.1. The
exact solution will decay asymptotically to zero, but the numerical solution produced

by the Explicit-Euler algorithm will decay asymptotically to zero only if:

 23

Of course, if it is really reasonable to solve a system using an explicit integration

method with fixed timesteps. WR is a not a good algorithm to use. As the convergence

proof indicates, the WR algorithm will converge for at least one additional time step,

and probably no more, with each relaxation iteration. For that reason, it is inefficient to

compute more than one additional time step with each WR iteration. Given that, it fol

lows that there is no reason to recompute the old timepoints because the relaxation

will have converged for sure. The WR algorithm is then reduced to an explicit integra

tion algorithm applied to the entire system directly.

Still, it is interesting to examine the case when the timestep necessary to insure

convergence of the Implicit-Euler discretized WR algorithm is as small as the timestep

necessary for stability of the Explicit-Euler. Because, if implicit methods do not allow

use of much larger timesteps, the extra computations required to use them is not

worthwhile. From Eqns. (1.5.5) and (1.5.8), if A = 10 then the Implicit-Euler

timesteps are unconstrained and the Explicit-Euler timestep must be less than 1.0. If

X = 100 then the Implicit-Euler timesteps must be less than 0.37 , and the Explicit-

Euler timesteps must be less than 0.18. In addition, Implicit-Euler will continue to

allow larger timesteps than Explicit-Euler for very large X . because its timestep con

straint decreases as -7T-. where as the Explicit-Euler timestep constraint decreases as

J_
T

One can infer from the above example that the WR algorithm allows the use of

larger timesteps than a direct explicit method in most cases, but constrains the

timesteps more than a direct implicit method. The difference between the direct impli

cit method timestep constraint and that for the WR algorithm is smallest if the system

to be solved is very loosely coupled. Digital integrated circuits, for which the WR
 24

algorithm was originally developed, are not always loosely coupled. The coupling can

be quite strong, but is usually so only for short intervals. The WR algorithm is

efficient for these problems because small timesteps are required (to insure WR conver

gence) only during those intervals when the coupling is strong, and. because implicit

integration is used, the timestep can safely be made much larger for the rest of the

interval[l9].

Most of the above analysis does not extend readily to the case where different

timesteps are used for different nodes of the system. Examining the multiple timestep

case is in general a very difficult problem in numerical analysis, even for standard

methods, and nothing has been published examining this case. Since the major advan

tage of WR is that only those variables in the system that are changing rapidly use

small timesteps, this is an important missing piece of the theory about WR methods.

SECTION 2 - IMPLEMENTATION TECHNIQUES FOR WR METHODS

SECTION 2.1 - PARTITIONING METHODS

In Algorithm 1.1.1. the system equations are solved as single differential equa

tions in one unknown, and these solutions are iterated until convergence. If this kind

of node-by-node decomposition strategy is used for systems with even just a few

tightly coupled nodes, the WR algorithm will converge very slowly. As an example,

consider the three node circuit in Fig. 2a, a two inverter chain separated by a resistor-

capacitor network. In this case, the resistor-capacitor network is intended to model

wiring delays, so the resistor has a large conductance compared to the other conduc

tances in the circuit. The current equations for the system can be written down by

inspection and are:

Cxx + imlixl,vdd) + im2ix\.u) + gixx—x2) = 0

C x2 g (x2—xj) = 0

C x3 tm3(x3.x2) + im4ixz.vdd ) = 0
 25

Linearizing and normalizing time (so that the simulation interval [0,T] is converted to
[0,1]) yields a 3x3 linear equation:

l*i -10 9.5 0 xx 5

x2 9.5 -9.5 0 *2 + 0
0
*3 0 -1 -1 *3

x,(0)=x2(0)=0 x3(0)=5

Algorithm 1.1.1 was used to solve the original nonlinear system. The input uit ),
the exact solution for x2, and the first, fifth and tenth iteration waveforms generated

by the WR algorithm for x2 are plotted in Fig. 2b. As the plot indicates, the iteration

waveforms for this example are converging very slowly. The reason for this slow can

be seen by examining the linearized system. It is clear x i and x 2 are tightly coupled

by the small resistor modeling the wiring delay.

If Algorithm 1.1.1 is modified, so that x\ and x2 are lumped together and solved

directly, we get the following iteration equations:

„* +i
•k+1 -10 9.5 0 *i
xl ^k+l
•k+1
x2 9.5 -9.5 0 t
*3

•*+l _ __,.*+1 _„A +]

*3 "~ *2 *3

The modified WR algorithm now converges in one iteration, because x3 only depends

on the "block" of xx and x2, and that block is independent of x3 .

As the example above shows, lumping together lightly coupled nodes and solving

them directly can greatly improve the efficiency of the WR algorithm. For this reason,

the first step in almost every WR-based program is to partition the system, to scan all

the nodes in the system and determine which should be lumped together and solved

directly. Partitioning "well" is difficult for several reasons. If too many nodes are

lumped together, the advantages of using relaxation will be lost, but if any tightly

coupled nodes are not lumped together then the WR algorithm will converge very

slowly. And since the aim of WR is to perform the simulation rapidly, it is important
 26

that the partitioning step not be computationally burdensome.

Three approaches have been applied to solve this partitioning problem, and all

have been used successfully in programs for solving large systems. The first approach

is to require the user to partition the system [10] . This technique is reasonable for the

simulation of large digital integrated circuits because usually the large circuit has

already been broken up into small, fairly independent pieces to make the design easier

to understand and manage. However, what is a sensible partitioning from a design

point of view may not be a good partitioning for the WR algorithm. For this reason

programs that require the user to partition the system sometimes perform a "sanity
check" on the partitioning [ll] . A warning is issued if there are tightly coupled nodes
that have not been lumped together.

A second approach to partitioning, also tailored to digital integrated circuits, is

the functional extraction method[l2]. In this method the equations that describe the

system are carefully examined to try to find functional blocks (i.e. a nand gate or a

flip-flop). It is then assumed that nodes of the system that are members of the same

functional block are tightly coupled, and are therefore grouped together. This type of

partitioning is difficult to perform, since the algorithm must recognize broad classes of

functional blocks, or nonstandard blocks may not be treated properly. However, the

functional extraction method can produce very good partitions because the relative

importance of the coupling of the nodes can be accurately estimated.

The most general, and perhaps the most obvious, approach to the partitioning

problem is the "diagonal dominant loop" method [13] [12] . In this method tightly

coupled nodes are determined by examining 2x2 submatrices of the Jacobian of

/ (x ,u) and C(x ,u ). If the magnitude of the product of the diagonal terms is not

greater than the product of the off-diagonal terms by some factor a, (a good choice

will depend on the application), then the two nodes corresponding to the submatrix are
 27

lumped together. The precise algorithm is as follows:

Algorithm 1.2.1 - Diagonal Dominant Loop Partitioning

Step 1: Compute matrices L* ,LC defined by
for all ii.j in N ) {
_ max , bfiixM)
dxj
_min . dfi(x.u)
x>*< 6%
Z§(^y) = mauXICjV(x.«)l
L?i=fl \Cuixju)\
}
Step 2: Partition the system.
for all ii J in N ) {
if (Z,fzj) > aLfjLfi or L/jLJj > otL/jLfi )
x, is lumped with x-3

The diagonal dominant loop method has the advantage of simplicity and general

ity, but it is often too conservative in practice. Unnecessarily large subsystems can be

generated because only the worst-case coupling is considered when lumping nodes

together. There are also cases for which the method is not conservative enough. A

poor partitioning will be generated for systems thai include sets of nodes that are

extremely tightly coupled to each other and are also tightly coupled to other nodes in

the system. (A somewhat complicated modification to this algorithm eliminates this

difficulty[28]). The functional extraction method is much less general, and if it is to

capture a wide variety of functional blocks, can become a very complicated algorithm.

However, the functional extraction methods better estimate the effective coupling

between nodes, and therefore are likely to generate smaller subsystems.

In the case where a functional extraction method exists, but is too complicated to

apply to a large system directly, then a good mixed approach is to use the two methods

sequentially. First partition the system by applying the diagonal dominant loop

method. Then apply the functional extraction method only to any overly large
 28

subsystems. In this way a reasonable partition can be generated quickly.

SECTION 2.2 - WINDOWING

The convergence theorem presented in Section 1.2 guarantees that the WR algo

rithm is a contraction mapping in an exponentially weighted norm. In this section, we

will demonstrate by example the practical implications of this choice of norm. We

will then examine how to reduce the number of iterations required to achieve conver

gence by breaking the simulation interval into small pieces, or "windows". First we

will prove WR convergence in an unweighted norm for short intervals. As this proof

must take into account worst-case behavior, the estimate of the interval the proof pro

vides is too short to be practical. This will lead us to the conclusion that an adaptive

approach to choosing the windows will be more useful, and is a safe alternative

because the basic convergence theorem guarantees that regardless of the interval

chosen, the WR algorithm converges.

Consider the following nonlinear ordinary differential equation in

xiit).x2it) € IR with input u 6 IR that approximately describes the cross-coupled

nor logic gate in Fig. 3a (the approximate equations represent a normalization that con

verts the simulation interval [0,T] to [0,1]).

x2 = (5—Xj) —xj(x2)2 —5x lit (2.2.1)

x2 = (5—x2) —x2(x j)2

xj(0) = 5.0 x2(0) = 0.0

The Gauss-Seidel WR Algorithm given in Section 1.2 was used to solve the for

the behavior of the cross-coupled nor gate circuit approximated by the above small

system of equations. In Fig. 3b plots of the input uit), the exact solution for x lit).
and the relaxation iteration waveforms for xxit ) for the 5th, 10th and 20th iterations

are shown. The plots demonstrate a property typical of the WR algorithm when

applied to systems with strong coupling: the difference between the iteration
 29

waveforms and correct solution is not reduced at every time point in the waveform.

Instead, each iteration lengthens the interval of time, starting from zero, for which the

waveform is close to the exact solution.

As an example of "better" convergence, consider the following differential equa

tion in xltx2,x3 with input u that approximately describs the shift register in Fig. 4a

(here the simulation interval [0,7" ] has been normalized to [O.l])

Xl = (5.0 - x,) - Xliu ? - (xj - x2) (2.2.2)

x2 = (x i — x2)

x3 = (5.0 —x3) —x3(x2)2

x(0) = 0.
The Gauss-Seidel WR Algorithm given in Section 1.2 was used to solve the original

system approximated by the above system of equations. The input uit), the exact

solution for x lit). and the waveforms for x\it ) computed from the first, second, and

third iterations of the WR algorithm are plotted in Fig. 4b. As the plots for this

example show, the difference between the iteration waveforms and the correct solution

is reduced throughout the entire waveform.

Perhaps surprisingly, the behavior of the first example is consistent with the WR

convergence theorem, even though that theorem states that the iterations converge uni

formly. This is because it was proved that the WR method is a contraction map in the

following nonuniform norm on C([0,7'].lRn ):

max[0J]e-6'|l/(OII
where b >0 . / it) € IR" , and || • II is a norm on IR" . Note that ||/ it )|| can increase
as ebr without increasing the value of this function space norm. If fit) grows
slowly, or is bounded, it is possible to reduce the function space norm by reducing

Wf'it )|| only on some small interval in [O.T] , though it will be necessary to increase
this interval to decrease further the function space norm. The waveforms in the more
 30

slowly converging example above, converge in just this way: the function space norm

is decreased after every iteration of the WR algorithm because significant errors are

reduced over larger and larger intervals of time. The examples above lead to the fol

lowing definition:

Definition 2.2.1; A differential system of the form given in Eqn. (1.1.3) is said to have

the strict WR contractivity property on [0.T] . if the WR algorithm applied to the sys
tem is a contraction map in a uniform norm on [0,T] , i.e.

max[0j]||x*+1(O-x*(O|| < mzx[on\\xkit)-xk-1it)\\ (2.2.3)

where x(0 € IR" on t 6 [OX] is the solution to Eqn. (1.1.3); xk it) 6 IR" on
t € [0.T] is the k'h iterate of Algorithm 1.1.1; and || • || is any norm on IR" . If the
WR algorithm applied to the system is a contraction in a uniform norm on [0,T] for
any T > 0 then we say that the system has the strict WR contractivity property on

[O.oo) . •

For a system of equations to have the strict WR contractivity property on [O.oo)

it must be more than just loosely coupled. In addition, the decomposed equations

solved at each iteration of the waveform relaxation must be well-damped, so that

errors due to the decomposition die off in time, instead of accumulating or growing. As

the crossed nand gate example indicates, many systems of interest do not have the

strict WR contractivity property on [0.T) for all T < oo . However, we will prove

that any system that satisfies the WR convergence theorem will also have the strict

WR contractivity property on some nonzero interval.

Theorem 2.2.1; For any system of the form of Eqn. (1.1.3) which satisfies the

assumptions of the WR convergence theorem (Theorem 1.2.1) there exists af>0 such
that the system has the strict WR contractivity property on [0,r] .
Proof of Theorem 2.2.1
 31

We will prove the theorem only for the Gauss-Seidel WR algorithm but, as

before, the theorem holds for the Gauss-Jacobi case. Starting with Eqn. (1.2.8) and

substituting x k for xj,

xk+1it)-xkit)= (1.2.8)

iLk +1it )+Dk +1it ))-^. +1it )xk it ) - iLk it )+Dk it ))"1i/i. (/ )xk ~Kt ) +
iLk+l+Dk+lYlf ixk+\ xk ,u)~ (L, +Dt r1/ ixk . xk~\u )
To simplify the notation. let Ak it). Bk it) € IR"*" be defined by

Ak it ) = iLk it )+Dk it ))~xUk it ), Bk it ) = iLk it )+Dk it ))~\ It is be important to

keep in mind that iLk it )+Dk it ))~xUk it ), and iLk it )+Dk it ))_1 are functions of xk ,
and by definition, so are Ak it ) and Bk it). Expanding the above equation and

integrating,

Jixk+1ir)-xkir))d7 = JAk^ir)ixkir)-xk-lir))dr + (2.2.4)

o o

J[Ak^ir)-Akir)]xk-Kr)d7 +
o

fBk +1ir)[f ixk +1(t). xk (t), u(t))-/ (** (t). x* "Ht). uir)}d r +
o

J[Bk+lir)-Bkir)]Jixkir).xk-lir).uir))d7
o

Integrating by parts and using the fact that x*(0) —x*-1(0) = 0.

xk +i(t )_xk {t } = Ak +j(f ) [xk (t )_x*-i(r )] _ (2.2.5)

\A-Ak^ir)[xkiT)-xk-li7))d7 +f[Ak+1i7)-Aki7)]xk~1 d7 +
0 " T 0

fBk+1i7)if (Xi+1(T). Xk (T). «(T))-/ ixki7),Xk-1i7),ui7))d7 +

0

J[Bk+1i7)-Bk (t)] / (x* (t). x^-Ht), u(r))d 7

0

Taking norms, and using the Lipschitz continuities of / , Ak (t), and Bk it). and the
uniform boundedness of Bk it ) in x (see Thm 1.2.1):
 32

|x*+1(O-x*0)ll - f U1K+k1M+k3N)\\xk+1i7)-xki7)\\ ^ (2.2.6)

•yllx^O-x^'-KOII +/ il2K+k1M+2k2M+k4N)\\xki7)-xk-1iT)\\d7
o

where l\ ,l2 are the Lipschitz constants of / with respect to x*+1 and xk respec
tively; k\, k2, k3, k4 are the Lipschitz constants for Ak +J(r ), Bk +1(? ) with respect

to their xi+1 and xk arguments respectively; y = maX[x^.][(L; -¥Dk )Uk ] < 1; and M
and N are the a priori bounds on xk and / found in the proof of Theorem 1.2.1.

Note that ——Ak+1i7) = —^^Ak+1xk+1 + —^Ak+1xk <k1M+k2M. Moving

d7 dxk+1 dxk

the max ( over t ) norms outside the integrals and integrating yields

maxIO,r]||x*+1(*)-x*(z)ll ^ (2.2.7)

y+T iKh+k^+lk^ +k4N) maxrori (* it)—*

*-ir,vtfi
it ))\\.
1 -TiKlt+kiM+ksN) lJi
Since y < 1 , a r'>0 exists such that

y +T' iKl 2+k XM +2k 2M+k 4N)

= a < 1 . With this T', Eqn. (2.2.17) becomes
1 -T'UiK+kiM+ksN)

maxIO,nll**+1-**ll < amaxicnll**-*'-1!! (2.2.8)

for a < 1 , which proves the theorem. •

Theorem 2.2.1 guarantees that the WR algorithm will be a contraction mapping in

a uniform norm for any system, provided the interval of time over which the

waveforms are computed is made small enough. This suggest that the interval of

simulation [OX] should be broken up into windows, [O.rJ. [Ti.T2], • • • , [Tn-i,Tn]

where the size of each window is small enough so that the WR algorithm contracts

uniformly throughout the entire window. The smaller the window is made, the faster

the convergence. However, as the window size becomes smaller, the advantages of the

waveform relaxation are lost. Scheduling overhead increases when the windows

become smaller, since each subsystem must be processed at each iteration in every win-
 33

dow. If the windows are made very small, timesteps chosen to calculate the

waveforms will be limited by the window size rather than by the local truncation

error, and unnecessary calculations will be performed.

The lower bound for the region over which WR contracts uniformly given in

Theorem 2.2.1 is too conservative in most cases to be of direct practical use. As men

tioned above, in order for the WR algorithm to be efficient it is important to pick the

largest windows over which the iterations actually contract uniformly, but the

theorem only provides a worst-case estimate. Since it is difficult to determine a priori

a reasonable window size to use for a given nonlinear problem, window sizes are usu

ally determined dynamically, by monitoring the computed iterations!13]. Since

Theorem 1.2.1 guarantees the convergence of WR over any finite interval, a dynamic

scheme does not have to pick the window sizes very accurately. The only cost of a bad

choice of window is loss of efficiency, the relaxation will still converge.

SECTION 2.3 - RELAXATION-NEWTON METHODS

In Section 1.3 we discussed a general class of methods for improving the compu

tational efficiency of WR algorithms. The approach taken in these methods was to

approximately solve the iteration equations when the computed xk *s were far from
convergence. We proved that if the WR iteration equations are only solved approxi

mately, but the accuracy of the approximation is improved with each iteration, then

these methods have convergence properties similar to the canonical WR algorithm. The

practical question is then what approximation should be used initially, and by how

much should the accuracy be improved with each iteration. In this section we will

present a modified WR algorithm that automatically adjusts the accuracy of the com

putation to how close the iterations are to convergence. The method is an extension to

function spaces of relaxation-Newton algorithms used for solving nonlinear algebraic

systems[l4],[l5],[l6]. In these algorithms the nonlinear iteration equations are not

 34

solved exactly, but are solved approximately, by performing one step of a Newton

method. Since the accuracy of the one Newton step improves as the xk 's approach the
exact solution, the iteration equations will be solved more accurately with each itera

tion if the sequence { xk } converges.

Using the waveform-Newton method derived in Section 1.4, and performing one

step of this Newton method with each waveform relaxation iteration, yields the fol

lowing Waveform-Newton-Relaxation algorithm (WNR).

Algorithm 2.3.1 (WNR Gauss-Seidel Algorithm for solving Eqn. (1.3))

Comment:
The superscript k denotes the iteration count, the subscript
i € 1, • • • ,N denotes the component index of a vector
and € is a small positive number.
£«-0 ;
guess waveform x°(t) ; t € [OX]
such that x°(0) = x0
(for example, set x°(t) = x0,1 € [0,7*] );
repeat {
k<-k+l

for all (i in N ) {
solve

z c,j(xi.-. . . V* yit-1 . • • ,xk-\u)x{+

ec«(*i.-- • ,X,_i ,Xj ,X„ ,1/ ) k j. _i> • L .
dx,
Z Cu ix\ . • •.*/_, ,x/-\ • •• ,xk ~\u )*f-1 -
/7m .•••x?-i .*/ -1, •••** -^) -
dfi(x1.--,xi_1.xi ,••-.*„ .«)( , ,.1} =0
for ( xf(t ) ; t 6 [0,T] ). with the initial
condition X/HO) = x, .

until ( maxK, <„ max, g [oj] \xf(t ) —x* Kt ) I ^ € )

• .

Again, each equation has only one unknown variable xj- . like Algorithm 1.1.1, but
each of the nonlinear equations has been replaced by a simpler time-varying linear

problem.
 35

Given the global convergence properties of both the original WR and the WN

algorithms, it is not surprising that the WNR algorithm has global convergence proper

ties. We will state the convergence theorem, but will not present the proof because it

quite similar to the proof of the basic WR and WN convergence theorems.

Theorem 2.3.1: If, in addition to the assumptions of Eqn. 1.1.3, C(x ,u) € lRn*n is

strictly diagonally dominant uniformly over all x € IR" and ———•— is Lipschitz
Qx

continuous with respect to x for all u ; then the sequence { xk } generated by the
Gauss-Seidel or Gauss-Jacobi WNR algorithm will converge to the solution of Eqn

1.1.3 for all bounded intervals [0.7*] .

The linear time-varying systems generated by the WNR algorithm are easier to

solve numerically than the nonlinear iteration equations of the basic WR algorithm,

but the iteration equations could be futher simplified if the time-varying Jacobian is

replaced by a a time-invariant approximation. Approximating the Jacobian will of

course destroy the local quadratic convergence of the WN method. But, as an exami

nation of the convergence proof in Section 1.4 indicates, approximating the Jacobian

will not destroy the global WN convergence. In addition, loss of quadratic conver

gence may not be a significant consideration when the Newton method is used in con

junction with a relaxation method, because the relaxation converges linearly and will

dominate the rate of convergence of the combined method.

The Modified WNR method then converts the basic WR iteration equations to

much simpler linear time-invariant equations. Such systems can be solved with a

variety of efficient numerical techniques other than the standard multistep methods.

Since the problem is linear, Laplace transform techniques could be used (see Section
2.4). Also, it is possible to use methods based on replacing the solution of the

differential equation with a series of orthogonal functions (e.g. Chebyshev polynomi

als) with unknown coefficients. The problem of finding a solution to the differential
 36

equation is then reduced to determining the coefficients[l7].

SECTION 2.4 - TECHNIQUES FOR PIECEWISE-LINEAR SYSTEMS

Large systems of ODE's are usually constructed by analyzing an interconnected

network of nonlinear elements. Often, to reduce computation time, the nonlinear ele

ments are not evaluated exactly, but approximated by a linearly interpolated table of

values. Not only does this approach reduce the computation time needed to evaluate

the nonlinear elements, but it also converts the original system into a piecewise-linear

system. Inside the "bounding box" formed by the table entries, the system is linear.

In this section an approach is given for solving piecewise-linear systems using the

WR algorithm presented in Section 1.2, and Laplace transforms[l8]. This technique

not only takes advantage of a loosely coupled original system through the use of the

WR algorithm, but as the iterations are computed using Laplace transforms, the

piecewise-linearity of the problem is also exploited.

We will start this section by deriving the iteration equations for the WR algo

rithm applied to a linear differential system. Following, the steps required to compute

the WR iteration waveforms using Laplace transform techniques will be described.

We will then extend the approach to piecewise-linear systems and introduce a new

algorithm.

Consider the following autonomous linear ODE:

x = Ax x(0)=x0 (2.4.1)
where x(t) € IR" on t 6 [0,T] and A € IR";cIR" . A is linear; therefore it is

Lipschitz continuous with respect to x , and the basic WR convergence theorem guaran

tees Eqn. (2.4.1) can be solved using Algorithm 1.1.1.

Let A —L+D+U where L is strictly lower triangular, D is diagonal, and U

strictly upper triangular. The Gauss-Jacobi WR iteration equations applied to Eqn.

(2.4.1) are:
 37

xk+1 = Dxk+1 + (L+U )xk x(0) = x0 (2-4.2)

Solving Eqn. (2.4.2) by Laplace transforms.

xk+Ks) = (si -D)-^ U+U)xk(s) + x0 ] (2.4.3)

If xk(s) is a rational (vector-valued) function with real poles, so is x*+1(j) .
And by induction, if x°(s ) is a rational function with real poles, then so is xk (s ) .

Given that Jt°CO is a rational function with real poles (for example, if

x°(t )=x0 , x°(s ) = —x0 ), it is easy to compute the xk+1(s ) term from xk (s ) . If

xk has the following partial fraction expansion:

\ ttii
xk(s)= 2>-X,) "V (2.4.4)
where v' € IR" , \/ 6 IR , and m, and Mk are positive integers, then xk can be calcu
lated from Eqn. (2.4.2) as

xt+i(s)= £(s _x.) mi(sI-D )-i (Z w)v/ + (sI-D)-ix0 (2.4.5)

which can be expressed as another partial fraction expansion

;t*+i(5)= £(5-X',)""'w'". (2.4.6)

i=l

When necessary, the time domain expression for Eqn. (2.4.6) can be obtained
from

m*+1 ,. , fm'i _1
xk+Kt)= £i
LcVf,
(m , - 1)
ppw'. (2.4.7)
As indicated above, the partial fraction expansion of xk+1 is computed from the
partial fraction expansion of xk in two simple steps. First, Mk multiplications of the
matrix (L+U) by the vector v' must be performed. For large systems, the (L+U)
matrix is usually sparse, so the number of scalar multplications required to perform

the matrix-by-vector multiplication is K *n , where K is the average number of

nonzero terms in each row of the matrix. The next step is to compute the w' vectors
 38

as in Eqn. (2.4.6). This involves performing the partial fraction expansion of the

terms of the form (s—\')~mi(s—dJ)~* and s~l(s—dj )-1 where d} is the jth entry of
the diagonal matrix D. The partial fraction expansion can be computed by evaluating
Mk
Z W, residues, where Wt is the number of nonzero elements in (L +U )v,.
i=l

The only complication incurred by extending the above technique to a piecewise-

linear systems is that the solution will cross into many different linear regions. How

ever, the points in time at which the solution passes from one linear region to the next

can be thought of as defining beginning and ending points of windows in time (see Sec

tion 2.2). Inside each window the problem is linear, with initial conditions specified

by the solution's value at the time it crosses the boundary of the region. The algo

rithm can then proceed as above inside each window, with only the additional

difficulty of determining the boundary crossing times.

Before describing the algorithm, we will formally define a piecewise-linear

differential system so that we can precisely define the notion of boundary crossings.

Definition 6.1: Let RJ , j € [l, • • • ,r ] be a collection of closed sets with disjoint inte

riors, and U= {jRj. Let A} e IR"*" and bj €IR" . Then p:U-+JRn is such that
;=i

p (z ) = Aj z + bj is piecewise-linear.

Consider the following differential equation

x(t) = pU(t)) jc(0) = x0 (2.4.8)

where x(t) 6 IR" and p:W ->IR" is piecewise-linear. We assume that the Aj € IRnx"

and bj 6 IR" of p are known, and are such that />(•) is everywhere continuous. The
WR algorithm for solving systems of the form of Eqn. (2.4.8) using a region by region

application of the Laplace transform technique described above is as follows:

Algorithm 2.4.1 (Region bv region solution of Eqn. 2.4.8)

7" = 0
repeat {
 39

Find j € [1. • • ,n ] such that xk (7") e Rj

x°(s) = -x(T')
s
*<-0 .
repeat {
it«-fc+l

xk(s) = (sI-DT1[(L+U)xk-1(s)+ -bj +x0]

s J
Partial fraction expand xk (s ) and collect like terms.
*+l ro'i — 1
X (r+r')= Ze*''* w<
/=! (™ / - 1)
} untiK maxj^, <n max, g ij-^jIxk(t ) —xk Kt)\^e )
i.e. until sufficient convergence is obtained.
7 = inf [t > T' I xkU) 6 RJ].
Find the time xk (t) leaves region RJ
T' = 7" + T
} until T' > T

In order to check convergence it is necessary to compute the time domain expres

sions for the iterations. The computation of the time domain expression is a relatively

expensive operation, and is only required to check WR convergence. It is possible to

improve the efficiency of Algorithm 2.4.1 by checking the convergence only every few

iterations. Another method for improving the efficiency is somewhat more subtle.

Since the WR algorithm usually converges in a nonuniform manner (see Section 2.2),
insisting that the relaxation converge to the end of the interval of interest only to then

toss away the solution after the boundary crossing time, will usually require many

unnecessary WR iterations to be performed. Even though it is impossible to know

when the boundary crossings will occur without knowing the exact solution, finding

those times for the partially converged solutions can provide good approximations to

the boundary crossing times. Since in many cases evaluating the boundary crossing

time is very expensive, a fast approximation should be used to provide some reasonable

upper bound on the boundary crossing time. This approximation can then be used to

shorten the interval over which WR convergence must be assured. Hence, the number

of WR iterations required to achieve satisfactory convergence can be reduced at the

cost of computing these approximate boundary crossings.

 40

SECTION 2.5 - TECHNIQUES FOR MOS DIGITAL CIRCUITS

In the previous sections of this paper, we have presented several relatively gen

eral techniques for improving the efficiency of WR along with corresponding examples

related to simulating MOS circuits. We chose examples from this area because, as men

tioned in the introduction, WR has proved to be an efficient techique for solving the

large nonlinear ODE systems that describe MOS digital circuits. This is due, in part, to

characteristics of these ODE systems that are exploited by the general properties of the

WR algorithm mentioned above. Specifically, these problems are easily broken up into

loosely coupled subsystems across which relaxations converge rapidly, and different

state variables change at very different rates, so the ability of the WR algorithm to use

different timesteps for different nodes is of great practical advantage.

There are also other properties of MOS circuits that can be exploited by the WR

algorithm, and they are much more specific to the circuit simulation problem. In order

to complete our presentation of the WR algorithm we will discuss some of these tech

niques, and will end the section with experimental results that demonstrate the

strengths and weaknesses of the WR algorithm in this important area of application.

2.5.1 - THE ODE DESRDPTION OF MOS DIGITAL SUBCIRCUITS

As mentioned previously, the physical behavior of MOS digital circuits can be

represented as a system of differential equations of the following form:

C(v(*).k(0)v (r) = /(v(t ).«(*)) v(0) = v0 (2.5.1)

where v (t) € IR" is a vector of time-varying voltages, u U ) € IRr is a vector of
time-varying inputs to the circuit, C : IR" x JRr -* IR"V" is a matrix of nonlinear capaci

tances, and / : IR" v IR' -♦ IR" is a a vector function of the voltages (the currents). For

most circuits of practical interest, C(v , w) is strictly diagonally dominant and there

fore C (v , u )_1 exists and is uniformly bounded with respect to v .it; and / is globally

Lipschitz continuous with respect to x for all u (t) € IR .

 41

2.5.2 - ORDERING OF THE SUBCIRCUITS

Consider using relaxation to solve a large system of linear algebraic equations of

the form

Ax — b

where x ,b € IR" and A € IR"*" . Let L ,D ,U be strictly lower triangular, diagonal,

and strictly upper triangular matrices respectively, such that A = L+D +U. If the

classical Gauss-Seidel relaxation algorithm is used to solve the above system, the itera

tion equations can be written in matrix form as:

(L+D)xk+1-U x* =b (2.5.2)
Taking the difference between iteration k +1 and k yields the following relation:

{xk+l-xk) = (L+DTlU (xk-xk~l). (2.5.3)

assuming L +D is nonsingular (i.e. the entries in D are nonzero). By the contraction

mapping theorem, the relaxation converges if there exists some induced norm on IR"

such ihatlia+Z))-1*/!! < 1.

As an example, suppose that A is lower triangular, that is U = 0. Then the

relaxation converges in one iteration because the above norm is zero. Of course, the

order of the single equations represented by the rows of A in Eqn. 2.5.1 is not unique.

One could reverse order the equations of the system, so that b, becomes 6„+j_;:

Ax = b then becomes Ax —b where A is a row permutation of A . In this case. A is

upper triangular because A is lower triangular. If the Gauss-Seidel relaxation algo

rithm is used to solve this new problem, the above norm will no longer be zero, and

the relaxation will not converge in one iteration. If fact, it may not converge at all.

Although the above is an extreme example, it does indicate that if the Gauss-

Seidel relaxation algorithm is used, it is possible to reorder the system so that the

number of iterations required to achieve convergence can be significantly reduced. In

particular, a reordering should attempt to move as many of the large elements of the
 42

matrix into the lower triangular portion as possible.

The Gauss-Seidel WR algorithm shares this properly with the algebraic relaxation

scheme. That is, the Gauss-Seidel WR algorithm will converge more rapidly if the

ODE system can be made lower triangular. In the case of MOS digital circuits the

function / (v ,u ) that represents the currents in the circuit can be made mostly lower

triangular by a careful reordering of the equations*. This is because the MOS transis

tor is a highly directional device. The transistor currents at the drain and source ter

minals are a strong function of the voltage at the gate terminal, but the gate current is

almost unaffected by the drain and source voltages. Therefore, if the differential equa

tions of the circuit can be ordered so that the equation that is solved to determine the

voltage at the gate of a transistor can be placed before the equation that is solved to

determine the voltage at the drain and source of a the transistor, then / (v ,u ) will be
mostly lower triangular.

It is not, in general, possible to so order the equations completely. Flip-flops (See

Fig. 3a), and many other types of digital circuits, contain loops that require some

transistor's drain or source voltage equation to precede its gate voltage equation. In

these cases, convergence is still improved if the equations are ordered so that / (v ,u ) is
as lower triangular as possible.

2.5.3 - TRUNCATED RELAXATION SCHEMES

One of the practical difficulties of applying the WR algorithm to large systems is

that the entire waveform for every node in the system must be stored during the itera

tion process. For systems with many nodes and long waveforms, the required data

storage may exceed a computer's available memory. Breaking the simulation interval

into "windows" (section 2.2) reduces the storage required for each of the individual

♦When we say we wish / (v ,u ) to be mostly lower triangular, we mean that we would like the terms
Q/i
that are large for any V: to be in the lower triangular portion.
$Vj
 43

waveforms and allows larger systems to be simulated without exceeding available

memory, but very large systems will still require extra storage. One approach to solv

ing this problem without changing the WR algorithm is to store the waveforms on a

mass storage medium (e.g. a magnetic disk). Then, since only a few waveforms are

used at any one time, those waveforms can be moved into a computer's memory, and

then moved back out when no longer needed (in much the same way as virtual

memory). Another approach to reducing the memory requirement of the WR algo

rithm is to introduce a truncation approximation, which will work well for certain

classes of MOS digital circuits.

Consider the inverter chain example in Fig. 5a. The input to the first inverter and

its output for the cases of chains with two. three and four inverters are plotted in Fig.

5b. As the plots indicate, the impact on the output of the first inverter of additional

inverters diminishes with the number of inverters. This suggests that the output of

each inverter in a long chain could be computed accurately by considering only a few

inverters at a time, effectively truncating the computation. For example, compute the

output of the first inverter by solving a reduced system which ignores all the inverters

after the fifth: then compute the output of the second inverter, using the computed

output from the first inverter, by solving a reduced system ignoring all the inverters

after the sixth: and continue in this fashion until all the inverter outputs have been

computed. The advantage of this approach is that at any point in the procedure, only

the waveforms of six inverters are needed. And. as the above simple example indicates,

the error due to this truncation will be small.

Of course this algorithm will only produce accurate results if the system of equa

tions are almost unidirectional, like the inverter chain, and the truncation algorithm

follows this direction. Combinational circuits, a large class of MOS digital integrated

circuits, do share the mostly unidirectional property of the inverter chain. And since
 44

these circuits can be quite large (several thousand nodes), using the truncation

approach avoids the difficultly of storing all the waveforms.

2.5.4 - PARTIAL WAVEFORM CONVERGENCE

If the WR algorithm is used to compute the time domain behavior for very large

circuits, it is often the case that some pieces of the circuit will converge much more

rapidly than others. The overall efficiency of the WR method can be improved if the

waveforms that have already converged are not recomputed every subsequent itera

tion.

To take advantage of partial waveform convergence requires a simple modification

to Algorithm 1.1.1. Before giving the exact algorithm we present the following useful

definition.

Definition 2.5.1 Let

ZClV (v(r ). u(t )) v: (t ) = /,(v(r luU )) v'(0) = vio (2.5.4)

be the ith equation of the system in Eqn. 2.5.1. We say v} (t ) is an input to this equa
tion if there exists some a,t € IR and z ,y € IR" such that
n n

Z Ctj (z ,u (t ))yj^J^Cij (z +otej ,u (t ))j; or / f(z m(t ));*/ j(z +aej ,u (t )), where ej

is the jth unit vector. The input set for the ith equation is the set of j € [l, • • • ,n]
such that vj (t ) is an input.

The WR algorithm is then modified slightly using this notion of the set of inputs

to a given ODE.

Algorithm 2.5.1 - WR Algorithm with Partial Waveform Convergence

Comment:
The superscript k denotes the iteration count,
the subscript i denotes the component index
of a vector and € is a small positive number.
£<-0 :
guess waveform x°(t ) : t € [0,T]
such that x°(0) = x0
 45

(for example, set x°(t ) = x0,t € [0,T] ):

repeat {
fc<-fc+l
f oreach ( i in N ) {
Partial/lag = TRUE
if ( # = 1 ) Partial/lag = FALSE
For each ( j <i ,j 6 input set o/ v, )
if ( max(0j] Iv/-vf _11 > 6 ) Partial/ lag - FALSE
For each ( j^-i J € input set of v, )
if ( maxjo T)' vf "]-vj "21 > 6 ) Partial/lag = FALSE
if ( Partial/lag '= 77?t/£ ) v/ +1 = v/
else solve

Z Cy (v{ . •• • .vf.vfrj1 , • • • .v* "^ )v/+

Z Cij(vi.--.vf.v^.---.v/f-1.W)vjf-1+

//(vi . • • • .vf.Vi'+i1 . • • • .V* -1.!/) =0

for ( v/*(* ) : t € [OX] ), with the initial condition v/^O) = vio .
}
} until ( maxj^,-4„ max, 6 [0j]I v/(? ) - v/ KO I<€ )
that is, until the iteration converges. •

2.5.5 - EXPERIMENTAL RESULTS

The degree to which the WR algorithm improves circuit simulation efficiency can

be traced to two properties of a circuit. The first, mentioned before, is the differences

in the rates of change of voltages in the system, as this will determine how much

efficiency is gained by solving the subsystems with independent integration timesteps.

The second is the amount of coupling between the subsystems. If the subsystems are

tightly coupled, then many relaxation iterations will be required to achieve conver
gence, and the advantage gained by solving each subsystem with its own timestep will
be lost. To show this interaction for a practical example, we will use the Relax2.2[l3]
program to compare the computation time required to simulate a 141-node CMOS
memory circuit using standard direct methods and using the WR algorithm. In order
to demonstrate the effect of tighter coupling, the CMOS memory circuit will be simu

lated using several values of a parameter XQC. which is the percent of the gate oxide
 46

capacitance that is considered as gate-drain or gate-source overlap capacitance.

Table 3 - Direct vs WR on a Memory Circuit with Different Couplings

METHOD XQC TIMEPOINTS # WR ITERS CPU TIME

Direct 0.01 124,539 1 933s
WR 0.01 17.728 2.5 304s
Direct 0.05 122,988 1 945s
WR 0.05 19.199 3 410s
Direct 0.2 118.335 1 917s
WR 0.2 19,193 4 530s
Direct 0.33 115,233 1 895s
WR 0.33 19.315 6.5 707s

The results in Table 3 are exactly as expected. As the coupling increases, the

number of WR iterations required increases, and the difference in the simulation time

for WR and the direct method decreases.

It is possible to verify for this example our claim of the nature of the efficiencies

of using WR. Consider the number of timepoints computed by the direct method

versus the number of computed timepoints for the WR method in the final iteration.

By comparing these two numbers, a bound can be put on the maximum speed increase

that can achieved by solving different subsystems using different timesteps (Note that

we are only considering the number of timepoints computed by the WR method in the

final iteration, because we are only interested in the number of timepoints needed to

accurately compute the given waveform).

The total number of timepoints computed for each of the simulation cases of the

memory circuit example is also given in Table 3. This number is the sum of the com

puted timepoints over all the waveforms in the circuit. If most of the efficency of a

decomposition method stems from solving each of the subsystems with its own

timestep. then the maximum improvement that could be gained from a decomposition

integration method would be the ratio of the number of timepoints computed using the
 47

direct method compared to the number of timepoints computed in the final WR itera

tion. As can be seen from the Table 3, for the CMOS memory example this ratio is

approximately six. In order to compute the actual efficiency of the WR method, the

average number of WR iterations performed must be considered, because for each WR

iteration the set of timepoints is recomputed. Then, if our claims above are correct,

when the ratio of the number of timepoints for the direct method to the number of

WR timepoints is divided into the average number of relaxation iterations, the result

should be almost equal to the ratio of WR computation time to direct computation

time. And as Table 3 shows, it is.

In the above analysis we have ignored an important advantage of relaxation

methods: that they avoid large matrix solutions. This is a reasonable assumption for

the above example because the matrix operations account for only a small percentage

of the computations, even when direct methods are used. However, for much larger

problems, of the order of several thousand nodes, the time to perform the large matrix

solutions required by direct methods will dominate. In those cases WR methods

should compare even more favorably because they avoid these large matrix solutions.

Finally, in Table 4, we present several circuits that have been simulated using

RELAX2.2 with direct and WR methods.

Table 4 - CPU Time for Direct Methods vs WR for Several Industrial Circuits

Circuit Devices DIRECT WR

i uP Control 232 90s* 45s*
! Cmos Memory 621 995s* 308s*
; 4-bil Counter 259 540s* 299s*
j Inverter Chain 250 98s** 38s**
! Digital Filter 1082 1800s* 520s*
I bncode-Decode 3295 5000s* 1500s*

*On Vaxl 1/780 running Unix using Shichman-Hodges Mosfet model

*On Vaxl 1/780 running VMS using Yang-Chatterjee Mosfet model
 48

CONCLUSIONS AND AKNOWLEDGEMENTS

In this paper several of the WR algorithms that have been proposed in the litera

ture have been analyzed both from a theoretical and practical point of view. We have,

however, treated several of these aspects too lightly. In particular, research is needed

to more thoroughly understand the nature of WR convergence under discretization,

and to characterize systems for which WR algorithms contract in uniform norm. In

addition, theoretical and practical work needs to be continued on breaking large sys

tems into smaller subsystems in such a way that relaxation algorithms converge

rapidly. Finally, since the WR algorithm has a tremendous amount of inherent paral

lelism, its application to solving problems using parallel processors is also an impor

tant research question.

The authors wish to thank G. Casinovi, J. Kleckner, K. Kundert, R. Newton, and

R. Saleh, and the other members of the CAD research group at Berkeley. In addition,

we would like to thank J. Beetem, W. Donath, P. Defebve. and H. Y. Hsieh at IBM's

Yorktown Research group for many valuable discussions and suggestions. This

research has been sponsored in part by DARPA under contract NESC-N39. and in part

by the Air Force Office of Scientific Research (AFSC), United States Air Force, under

Contract No. F49620-83-C-0005. The United States Government is authorized to

reproduce and distribute reprints for government purposes notwithstanding any copy

right notation hereon.

REFERENCES

[l] C. William. Gear, "Numerical Initial Value Problems /or Ordinary Differential Equa

tions!' Prentice Hall, 1974.

[2] L.W. Nagel, "SPICE2: A computer program to simulate semiconductor circuits,"

Electronics Research Laboratory Rep. No. ERL-M520, University o/ Cali/ornia, Berkeley,

May 1975.
 49

[3] K. Sakallah and S.W. Director, "An activity-directed circuit simulation algorithm,"
Proc. IEEE Int. Con/, on Circ. and Computers, October 1980, pp.1032-1035

[4] G. De Micheli, A. Sangiovanni-Vincentelli and A.R. Newton. "New Algorithms for

the Timing Analysis of Large Circuits" Proc. 1980 Int. Symp. on Circ. and Syst., Hous
ton. 1980.

[5] B.R. Chawla. H.K. Gummel. and P. Kozah. "MOTIS - an MOS timing simulator."
IEEE Trans. Circuits and Systems, Vol. 22, pp. 901-909, 1975

[6] C. F. Chen and P. Subramaniam. "The Second Generation MOTIS Timing

Simulator— An Efficient and Accurate Approach for General MOS Circuits" Proc. 1984
Int. Symp. on Circ. and Syst., Montreal, Canada. May 1984.

[7] Keepin, W. N., "Multirate Integration of Two Time-Scale Systems," Ph.D. Disserta
tion, University o/ Arizonia, 1980

[8] C. William Gear, "Automatic Multirate Methods for Ordinary Differential Equa

tions" In/ormation Processing 80, North-Holland Pub. Co., 1980

[9A] E. Lelarasmee. A. E. Ruehli, A. L. Sangiovanni-Vincentelli. "The waveform relaxa
tion method for time domain analysis of large scale integrated circuits." IEEE Trans, on
CAD o/lC and Syst., Vol. 1. n. 3. pp.131-145. July 1982.

[9B] E. Lelarasmee. "The waveform relaxation method for the time domain analysis of

large scale nonlinear dynamical systems". Ph.D. dissertation, University o/ California,

Berkeley.

[9C] E. Lelarasmee and A. Sangiovanni-Vincentelli. "Some New Results on Waveform

Relaxation Algorithms for the Simulation of Integrated Circuits." Proc. o/ 1982 IEEE
Int. Large Scale System Symposium, Virginia Beach, Oct. 1982, pp. 371-376

[10] E. Lelarasmee and A. Sangiovanni-Vincentelli, "Relax: a new circuit simulator for

large scale MOS integrated circuits", Proc. 19th Design Automation Conference, Las

Vegas, Nevada, pp. 682-690, June 1982.

[11] P. Defebve. J. Beetem. W. Donath. H.Y. Hsieh. F. Odeh, A.E. Ruehli, P.K. Wolff,
 50

Sr.. and J. White, "A Large-Scale Mosfet Circuit Analyzer Based on Waveform Relax
ation" International Con/erence on Computer Design, Rye. New York. October 1984.
[12] C. H. Carlin and A. Vachoux, "On Partitioning for Waveform Relaxation Time-
Domain Analysis of VLSI Circuits" Proc. 1984 Int. Symp. on Circ. and Syst., Montreal,

Canada. May 1984.

[13] J. White and A. Sangiovanni-Vincentelli. "Relax2.1 - A Waveform Relaxation

Based Circuit Simulation Program" Proc. 1984 Int. Custom Integrated Circuits Con/er
ence Rochester. New York, June 1984.

[14] M. Guarini and 0. A. Palusinski, "Integration of Partitioned Dynamical Systems

using Waveform Relaxation and Modified Functional Linearization," 1983 Summer Com

puter Simulation Proceedings, Vancouver, Canada, July 1983.

[15] W.M.G. van Bokhoven, "An Activity Controlled Modified Waveform Relaxation
Method" 1983 Con/. Proc. IEEE ISCAS, Newport Beach. CA. May 1983.

[16] J. M. Ortega and W.C Rheinbolt, Iterative Solution o/ Nonlinear Equations in

Several Variables Academic Press. 1970.

[17] O. Palusinski "Continuous Expansion Methods in Computer Aided Circuit

Analysis" Proc. 1984 Int. Custom Integrated Circuits Con/erence Rochester, New York,

June 1984

[18] J. Kaye and A. Sangiovanni-Vincentelli, "Solution of piecewise linear ordinary

differential equations using waveform relaxation and Laplace transforms". Proc. 1982
Int. Con/, on Circ. and Comp., New York, Sept. 1982.

[19] W.K. Chia. T.N. Trick, and I.N. Haij. "Stability and Convergence Properties of

Relaxation Methods for Hierarchical Simulation of VLSI Circuits", Proc. 1984 Int.

Symp. on Circ. and Syst., Montreal. Canada. May 1984.

[20] F. Odeh and D Zein. "A Semidirect Method for Modular Circuits" 1983 Con/. Proc.

1EEEISCAS, May. 1983. Newport Beach, CA

[21] G. Guardabassi "Subsystemwise Simulation of Discrete-Time Interconnected

 51

Dynamical Systems." Electronics Research Laboratory Rep. No. ERL-M82/8, University

o/ Cali/ornia, Berkeley, Feb. 1982.

[22] J. K. Hale, Ordinary Differential Equations John Wiley and Sons, Inc.. 1969
[23] H. A. Antosiewicz, "Newton's method and boundary value problems". Journal 0/

Computer System Science 2(2). 177-202 (1968)

[24] Richard S. Varga Matrix Iterative Analysis Prentice-Hall. 1962

[26] F. Odeh. A. Ruehli, C. H. Carlin "Robustness Aspects of an Adaptive Waveform

Relaxation Scheme" Proc. 1983 Int. Con/erence on Computer Design, Port Chester, New

York. October. 1983

[27] J. More "Nonlinear Generalization of Matrix Diagonal Dominance with Applica

tions to Gauss-Seidel Iterations" SIAM J. Numer. Anal., Vol. 9, No. 2, June 1972

[28] J. White and A.L. Sangiovanni-Vincentelli. "Partitioning Algorithms and Parallel

Implementations of Waveform Relaxation Algorithms for Circuit Simulation" Proc.
Int. Symp. on Circ. and Syst.. Kyoto, Japan. June 1985
 52

List of Figures

Figure 1 - Inverter with Feedback

Figure 2a - Two Inverters with Delay

Figure 2b - WR Iterations for Inverter with Delay

Figure 3a - Cross-Coupled Nor Gate

Figure 3b - WR Iterations for Cross-Coupled Nor Gate

Figure 4a - Shift Register

Figure 4b - WR Iterations for Shift Register

Figure 5a - Inverter Chain

Figure 5b - Output of Inverter 1 for Several Chain Lengths

 "C 1
r^l
1
a>
+ *n
 v(t)
A

EXACT

4-

3-

ITER 10

2-

ITER 5

1 - ITER 1

u(t)
T" > t
T
 «->
=!
 v(t)
A

5-
~T
f f
4-
|
3- j
ITERS ITER 10 (ITER 20

2-

1 -
II I
1 EXACT

u(t)
0 1 J
 + «
X 1
CM 1
X
L
+
<>
51 i "—>
(
>n
+ x~ 1
1
+ 3 I
I
 _* *.
> h
+ 5>
."[
Zj + 3 I