See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/228653142

Convergence analysis on iterative methods for semidefinite systems

Article in Journal of Computational Mathematics · November 2008

CITATIONS READS

11 356

3 authors, including:

Jinbiao Wu Jinchao Xu
Peking University Pennsylvania State University
21 PUBLICATIONS 358 CITATIONS 211 PUBLICATIONS 11,089 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Jinbiao Wu on 29 September 2014.

The user has requested enhancement of the downloaded file.

Journal of Computational Mathematics, Vol.26, No.6, 2008, 797–815.

CONVERGENCE ANALYSIS ON ITERATIVE METHODS FOR

SEMIDEFINITE SYSTEMS*

Jinbiao Wu
LMAM and School of Mathematical Sciences, Peking University, Beijing 100871, China
Email: [email protected]
Young-Ju Lee
Department of Mathematics, Rutgers, The State University of New Jersey, Busch Campus,
Piscataway, NJ 08854-8019, USA
Email: [email protected]
Jinchao Xu
LMAM and School of Mathematical Sciences, Peking University, Beijing 100871, China
Department of Mathematics, The Pennsylvania State University, University Park, PA16802, USA
Email: [email protected]
Ludmil Zikatanov
Department of Mathematics, The Pennsylvania State University, University Park, PA16802, USA
Email: [email protected]

Abstract
The convergence analysis on the general iterative methods for the symmetric and pos-
itive semidefinite problems is presented in this paper. First, formulated are refined neces-
sary and sufficient conditions for the energy norm convergence for iterative methods. Some
illustrative examples for the conditions are also provided. The sharp convergence rate iden-
tity for the Gauss-Seidel method for the semidefinite system is obtained relying only on
the pure matrix manipulations which guides us to obtain the convergence rate identity
for the general successive subspace correction methods. The convergence rate identity for
the successive subspace correction methods is obtained under the new conditions that the
local correction schemes possess the local energy norm convergence. A convergence rate
estimate is then derived in terms of the exact subspace solvers and the parameters that
appear in the conditions. The uniform convergence of multigrid method for a model prob-
lem is proved by the convergence rate identity. The work can be regraded as unified and
simplified analysis on the convergence of iteration methods for semidefinite problems [8,9].
Mathematics subject classification: 65F10, 65N22, 65N55.
Key words: Semidefinite systems, Subspace correction methods, Iterative methods, Energy
norm convergence.

1. Introduction
We consider the iterative methods for the following linear problem,

Au = b, (1.1)

where A is a symmetric and positive semidefinite operator from V to V , V is a finite dimensional

Hilbert space with the inner product (·, ·) and b ∈ V is a vector in the range of A. Such
semidefinite problems arise in many areas of applied mathematics. The finite element and/or
* Received July 17, 2007 / accepted May 5, 2008 /
798 J. WU, Y.-J. LEE, J. XU AND L. ZIKATANOV

finite difference discretizations of the Poisson equation with Neumann boundary conditions [2]
and the linear elasticity equation with pure traction boundary conditions lead to such problems.
Other more sophisticated examples can be found at the linear systems obtained from generalized
finite element methods [15, 16], and the time dependent Navier-Stokes systems [3].
For such problems, in general, it is difficult to apply the direct methods in a straightforward
manner (not to mention that direct methods are very expensive for large linear systems [23]).
Iterative methods are desirable for large semidefinite systems and our focus in this paper will
be made, in particular, on the convergence analysis of the classic iterative methods and the
general subspace correction methods for semidefinte (singular) problems given in (1.1).
The studies of the classic iterative methods for singular systems and their convergence can
be traced back to Keller, [7] and there have been many investigations by many researchers since
then, see [1, 4, 8, 11] and also many references cited therein. The classical iterative methods
discussed in those works are mainly based on a matrix splitting: A = M − N and from the
setting for the iterates {uℓ }ℓ=0 :
M uℓ = N uℓ−1 + b (1.2)
or equivalently,



M uℓ − uℓ−1 = b − Auℓ−1 . (1.3)
All the convergence results require that the iterator M be an invertible matrix, except that
in [8]. Furthermore, the setting in [8] requires N (M t ) ⊂ N (A) which is necessary for the
solvability of (1.2) for xℓ .
In this paper, we study iterative methods for (1.1) in the following form:


uℓ = uℓ−1 + R b − Auℓ−1 , (1.4)
where R is a linear operator from V to V and it may be singular. We then present more refined
necessary and sufficient conditions for the energy norm convergence of the iterative method
(1.4). One advantage of such view is that no assumption on the null space is necessary.
The rest of paper will be devoted to establish a convergence rate identity for the general
successive subspace correction method. The techniques of subspace correction methods are
based on a “divide and conquer” strategy. Classic iterative methods as Gauss-Seidel method,
and many multigrid and domain decomposition methods fall into this category of methods.
Recently, authors provided a sharpest possible convergence estimate for the general subspace
correction method for singular systems in a general Hilbert space setting, [9]. The current works
are aimed to better understand the basic idea of obtaining the convergence rate estimate in a
transparent manner restricting the problem in finite dimensional spaces.
The sharp convergence rate identity for the Gauss-Seidel method for the semidefinite system,
is obtained relying only on the pure matrix manipulations. The idea will guide us to obtain
the convergence rate identity for the general successive subspace correction methods. For the
successive subspace correction methods, we assume that the local correction schemes possess
the local energy norm convergence. We then derive a new version of the convergence rate
identity [9] under minimal assumptions. We also get the convergence rate estimate in terms
of the exact subspace solvers and the parameters that appear in the conditions. In Section
4, we give an example from an electrochemical model to illustrate how to apply our identity
in designing an optimal multigrid method for such a singular system, and prove the uniform
convergence for the multigrid method. As the results in this paper, we will be able to give the
convergence criteria that are more refined and concise that that in [8, 9] and whose analysis
becomes more transparent.
Iterative methods for semidefinite systems 799

In this paper, for the convenience of exposition, we introduce some standard notation. For
any subspace W of V , i.e., W ⊂ V , W ⊥ denotes the orthogonal complement of W with respect
to the inner product, (·, ·); for two subspaces N and W of V with N ⊂ W , W/N denotes the
quotient space of W ; for a given matrix M , the range of M and the null space of M are denoted
with R(M ) and N (M ) respectively. For a given real matrix G (or an operator B : V 7→ V ),
the real transpose of G (or the adjoint operator of B) shall be denoted by Gt (or B t ).
The structure of the paper is as follows. In Section 2, we derive the necessary and sufficient
conditions for energy norm convergence of the scheme (1.4). In Section 3, we introduce the
successive subspace correction methods for the system (1.1) and present the convergence rate
identity and estimate. The proof of the technique lemma (Lemma 3.3) is given in Section 5.
An example from an electrochemical model is given in Section 4.

2. Necessary and Sufficient Conditions for the Energy Norm

Convergence of the Stationary Linear Iterative Method
A general stationary iterative method to solve the system of equation (1.1) is given by (1.4).
Note that since N (A) 6= {0}, the solution to (1.1) is not unique, but is unique in the quotient
space V /N (A).
Denote T = RA, an obvious sufficient condition for convergence of (1.4) is given by

|I − T |A < 1. (2.1)

More precisely, let {uℓ }∞

ℓ=0 be the sequence of iterates generated by (1.4) with any given initial
guess u0 . If (2.1) is satisfied, then the sequence {uℓ }∞ ℓ=0 is convergent, and the limit is a
solution to (1.1). In case, the condition (2.1) is satisfied, we shall say that (1.4) is energy norm
convergent. The goal of this section is to find necessary and sufficient conditions of the iterative
method (1.4) for the energy norm convergence (2.1).
Note that,


|u|2A − |(I − T )u|2A = (R + Rt − Rt AR)Au, u A
= 2(T u, u)A − (T u, T u)A, ∀u ∈ V, (2.2)

Denote R̄ = R + Rt − Rt AR, and Q : V 7→ R(A) be the orthogonal projection under the inner
product (·, ·). We get the following simple and important lemma.

Lemma 2.1. The scheme (1.4) is energy norm convergent if and only if

QR̄Q is symmetric and positive definite on R(A), (2.3)

or equivalently there exists a positive constant α such that

2(T u, u)A − (T u, T u)A ≥ α(u, u)A ∀u ∈ V. (2.4)

Proof. The conclusion follows directly by (2.2). 

The following theorem on the convergence of the iteration given by (1.4) is the main result
in this section.

Theorem 2.1. The assumption (2.3) or (2.4) is equivalent to the following two assumptions,
800 J. WU, Y.-J. LEE, J. XU AND L. ZIKATANOV

(A1) ∃ω ∈ (0, 2) such that (T u, T u)A ≤ ω(T u, u)A ∀u ∈ V ,

(A2) ∃β > 0 such that (T u, T u)A ≥ β(u, u)A ∀u ∈ V .

The iterative scheme (1.4) is convergent if both (A1) and (A2) (equivalently (2.3) or (2.4)) are


satisfied. Furthermore (A1) and (A2) equivalently (2.3) or (2.4) are also necessary conditions
for the energy norm convergence.

Proof. We only need to prove that (A1) and (A2) are equivalent to (2.4). Assume (A1) and
(A2) hold, we have
2
2(T u, u)A − (T u, T u) ≥ ( − 1)(T u, T u)A
ω
β(2 − ω)
≥ (u, u)A , ∀u ∈ V.
ω
Then (2.4) holds with α = β(2 − ω)/ω.
Now we assume (2.4). Since T is a linear operator, we have

(T u, T u)A ≤ kT k2 (u, u)A .

Furthermore,
α
2(T u, u)A − (T u, T u)A ≥ α(u, u)A ≥ (T u, T u)A, ∀u ∈ V,
kT k2
 
α
(T u, T u)A ≤ 2/ 1 + (T u, u)A, ∀u ∈ V,
kT k2
 
which gives (A1) with ω = 2/ 1 + kTαk2 .
By (2.4),
1/2 1/2 α
(T u, T u)A (u, u)A ≥ (T u, u)A ≥ (u, u)A , ∀u ∈ V,
2
α2
(T u, T u)A ≥ (u, u)A , ∀u ∈ V,
4
which gives (A2) with β = α2 /4. 
We would like to remark that (A1) alone is not sufficient for convergence. For example,
whenever R(T ) = N (A), (A1) is true; but there is no energy norm convergence in this case
(see Example 1).
For an operator B : V 7→ V , denote the Moore-Penrose generalized inverse [6] by B † . If
B satisfies N (B) = N (A), R(B) = R(A), then B † : V 7→ V is a simple zero extension of
B −1 : R(A) 7→ R(A), namely

B † c = 0, ∀c ∈ N (A),
† −1
B v=B v, ∀v ∈ R(A).

Now we try to estimate the convergence rate of the iteration (1.4).

Lemma 2.2. Under the assumptions (A1) and (A2), the energy norm convergence rate |I −T |A
of the iterative method (1.4) can be estimated by
r
β(2 − ω)
|I − T |A ≤ 1 − ,
ω
Iterative methods for semidefinite systems 801

and |I − T |2A = 1 − 1/K with

K= sup inf (v + c, v + c)(QR̄Q)† . (2.5)

v∈R(A),|v|A =1 c∈N (A)

Proof. Note that

|(I − T )u|2A = |u|2A − (R + Rt − Rt AR)Au, u A
,

and


(R + Rt − Rt AR)Au, u A = 2(RAu, u)A − (RAu, RAu)A
 
2 β(2 − ω) 2
≥ − 1 (RAu, RAu)A ≥ |u|A ,
ω ω
from which, we obtain
β(2 − ω)
|I − T |2A ≤ 1 − , (2.6)
ω

2
(I − RA)v, (I − RA)v A
|I − T |A = sup
v∈R(A) (v, v)A
(R̄Av, v)A
=1− inf
v∈R(A) (v, v)A

(QR̄Qv, v)
=1− inf .
v∈R(A) (A† v, v)

Noting that QR̄Q and A† are symmetric and positive definite on R(A), then we get that
|I − T |2A = 1 − 1/K with


(A† v, v) (QR̄Q)† v, v
K = sup = sup
v∈R(A) (QR̄Qv, v) v∈R(A) (v, v)A
= sup inf (v + c, v + c)(QR̄Q)† .
v∈R(A),|v|A =1 c∈N (A)


Let us consider the Gauss-Seidel method and obtain the convergence rate identity by (2.5).
We note that the Gauss-Seidel method can also be viewed as a simple successive subspace
correction method, the result shall guide us how to study the general case of successive subspace
corrections. The Gauss-Seidel method is given by


uℓ = uℓ−1 + B b − Auℓ−1 , (2.7)

where B = (D − L)−1 with A = D − L − Lt where D and −L are the diagonal matrix and strict
lower triangular matrix of A respectively. We shall now see that

Lemma 2.3. The energy norm convergence rate |I − BA|A for (2.7) is given by
1
|I − BA|2A = ρ2 = 1 − , (2.8)
K
with

S(v + c), (v + c)
K = sup inf , (2.9)
v∈R(A) c∈N (A) (v, v)A
where S = (D − L)D−1 (D − Lt ).
802 J. WU, Y.-J. LEE, J. XU AND L. ZIKATANOV

Proof. Using B̄ = B + B t − B t AB, a simple manipulation lead to

B̄ = S −1 .

By (2.5), we have


(QS −1 Q)† v, v
K= sup inf (v + c, v + c)(QS −1 Q)† = sup .
v∈R(A),|v|A =1 c∈N (A) v∈R(A) (v, v)A


We now introduce w = (QS −1 Q)† v, then w ∈ R(A) and w = S v + c(v) where c(v) =
S −1 w − v ∈ N (A) is uniquely determined by v ∈ R(A). Then


S(v + c(v)), v + c(v)
K = sup .
v∈R(A) (v, v)A

It is left to prove that

S(v + c(v)), v + c(v) = inf (S(v + c), v + c),
c∈N (A)

for any v ∈ R(A). Assume that

ξ = arg inf S(w + c), (w + c) . (2.10)
c∈N (A)

Then ξ satisfies


S(w + ξ), c = 0, ∀c ∈ N (A). (2.11)
S is symmetric and positive definite, then (2.11) admits a unique ξ ∈ N (A) for a given v ∈ R(A).
Obviously, c(v) ∈ A also satisfies (2.11) which implies that ξ = c(v). We then complete the
proof. 
We now consider the iteration method based on matrix splitting. Assume that

A = M − N,

the corresponding iteration method can be defined in the form of (1.4) with R = M † , where
M † is the Moore-Penrose generalized inverse [14] of M , namely,

uℓ = uℓ−1 + M † (b − Auℓ−1 ).

One main convergence result for (1.4) by Keller [7] is summarized as follows. The iterative
scheme (1.4) is energy norm convergent if the splitting A = M − N is P-regular, in the sense
that M is invertible, and M T + M − A is positive definite.
For some special singular systems, considered by e.g., Marek and Szyld [12], the convergence
has been studied via the theory of nonnegative matrices, for which the weak-regularity condition,
proposed in Ortega and Rheinboldt, [14] is often applied as a sufficient condition. A version of
the weak regularity condition, (see e.g., Berman and Plemmons [1]) is as follows: A splitting
A = M − N is called weakly-regular if M is invertible, and in addition, both M −1 and M −1 N
are nonnegative matrices. An example given in [8] have showed that neither P-regularity, nor
weak regularity of the matrix splitting is necessary for the convergence.
For the energy norm convergence of (1.4) with R = M † , the result in Theorem 2.1 is
new. Example 2 shows that, in the framework here, R may be singular and the assumption
N (M t ) ⊂ N (A) imposed in [8] is not necessary.
Iterative methods for semidefinite systems 803

Example 1: (A2) is necessary for the convergence

Let  
1 −3
A= . (2.12)
−3 9
Introduce an iteration scheme (1.4), where R is given by
 
1 2/3
R= .
−1/3 0
Simple algebraic computation yields

R(T ) = N (A).

It is then straightforward to see that

|(I − T )u|2A = |u|2A , ∀u ∈ V.

This means that the iteration is not convergent.

Example 2: R = M † may be singular and N (M t ) ⊂ N (A) in [8] is unnecessary

Assume  √ 
1√ − 32 2 0√
A =  − 32 2 2√ − 31 10  .
1
0 − 3 10 1
We have the following matrix splitting,
   2
√ 
1 −1 0 √0 3 2−1 √ 0
A = M − N =  −1 2 −1  −  2
3 2−1
1
3 10 − 1
.
1
√0
0 −1 1 0 3 10 − 1 0

Consider the iteration scheme (1.4), where R = M † is the Moore-Penrose generalized inverse
of M . Namely, R is given by
 
5/9 −1/9 −4/9
R =  −1/9 9/2 −1/9  .
−4/9 −1/9 5/9
Then  √ √ √ √ 
4 2 10
9 − 27 2 2 + 29 − 27
27
4
10 4
9 − 1
27 10
 √ √ √ √ 
I −T =

1
9
4
+ 27 2 59 − 272 1
2 − 27 10 1
9 + 2
27 10 
.
4 2
√ 8
√ √ √
9 − 27 2 − 27 2 + 29 + 27
5
10 4
9 − 1
27 10
By direct computation, it can be verified that

|I − T |A < 1,

which implies that the iteration is convergent. From

( )
  3 √ 1 √ t
N (M ) = span (1, 1, 1)t ,
t
N (A) = 1, 2, 5 ,
4 2

we get that N (M t ) 6⊂ N (A).

804 J. WU, Y.-J. LEE, J. XU AND L. ZIKATANOV

3. Convergence Rate Estimate for Subspace Correction Method

In this section, we first introduce the (successive) subspace correction methods for the semi-
definite system (1.1). Then, we present the convergence rate identity and estimate for general
subspace correction methods under the minimal assumptions on subspace solvers.
First of all, we assume that V is decomposed into a finite number of subspaces {Vi }Ji=1 such
that
XJ
V = Vi . (3.1)
i=1

Following [20], the general method of successive subspace corrections is given as follows.

Algorithm 3.1. If ul−1 is known, the next iterate ul is decided by the following procedure.
Define ul−1
0 = ul−1


uil−1 = ul−1 l−1
i−1 + Ri Qi b − Aui−1 , i = 1, · · · , J. (3.2)

Set ul = ul−1
J
where Ri is an operator from Vi to Vi , and the orthogonal projection with respect to (·, ·),
Qi : V 7→ Vi is defined as

(Qi v, wi ) = (v, wi ), ∀v ∈ V, w ∈ Vi .

3.1. Notation and assumptions

In what follows, we will use the following notation,

N = N (A), Ni = N ∩ Vi ,

Ni⊥ ⊂ Vi are the orthogonal complement of Ni with respect to the inner product (·, ·). Define
Ai : Vi 7→ Vi such that
(vi , wi )Ai = (vi , wi )A , ∀vi , wi ∈ Vi ,
the restriction of A on Vi . Then Ri is in some sense the approximation of A−1
i if Ai is invertible,

Ai = Qi A on Vi ,
N (Ai ) = Ni , R(Ai ) = Ni⊥ .

Denote
Ti = Ri Qi A,
which is also called the (local) subspace solver on Vi . For energy norm convergence of Algo-
rithm 3.1, some additional assumptions are necessary. These assumptions are motivated by the
simple observation that the local energy norm convergence is guaranteed, namely,

|(I − Ti )vi |Ai

|I − Ti |Ai = sup ≤ δi < 1, (3.3)
vi ∈Vi |vi |Ai

where δi ∈ [0, 1).

Iterative methods for semidefinite systems 805

The necessary and sufficient conditions for the local energy norm convergence would be the
local version assumption of (A1) and (A2). To state these assumptions, we denote Qi : Vi 7→
R(Ai ) = Ni⊥ be the orthogonal projection from Vi to Ni⊥ with respect to (·, ·), and

R̄i = Ri + Rit − Rit Qi ARi = Ri + Rit − Rit Ai Ri . (3.4)

The assumption (3.3) is equivalent to

Qi R̄i Qi is symmetric and positive definite on Ni⊥ ,

or there exits a positive constant αi such that

2(Ti vi , vi )A − (Ti vi , Ti vi )A ≥ αi (vi , vi )A .

Furthermore, (3.3) is equivalent to the following two assumptions:

(H1) ∃ωi ∈ (0, 2) such that (Ti vi , Ti vi )A ≤ ωi (Ti vi , vi )A , ∀vi ∈ Vi ,

(H2) ∃βi > 0 such that (Ti vi , Ti vi )A ≥ βi (vi , vi )A , ∀vi ∈ Vi .

Lemma 3.1. Assume that (H1) and (H2) hold. Then for all 1 ≤ i ≤ J, there exists δi ∈ [0, 1)
such that
|(I − Ti )vi |A ≤ δi |vi |A , ∀vi ∈ Vi .

where δi2 = 1 − βi (2 − ωi )/ωi .

Proof. The proof is the same as that of Lemma 2.2. 

We remark that when A is symmetric and positive definite, the assumption (H1) and (H2)
(For this case, (H2) is equivalent to Ti : Vi 7→ Vi is isomorphic) are also sufficient conditions for
the energy norm convergence of the subspace correction method [20,21]. The similar assumption
to (H2) has been added to guarantee the local parameter independent convergence in [10].

3.2. Convergence rate identity and estimate

Let u ∈ V be a solution to (1.1) and {ul : l = 0, · · · } be the iterates generated by Algo-

rithm 3.1 respectively. The following relation is standard,



u − ul = EJ u − ul−1 = · · · = EJl u − u0 ,

where
EJ = (I − TJ ) · · · (I − T1 ).

The convergence rate can be estimated by the energy norm of EJ ,

|EJ v|A
|EJ |A = sup . (3.5)
v∈R(A) |v|A

We first present a new version of the convergence rate identity when A is symmetric and positive
definite [8, 21].
806 J. WU, Y.-J. LEE, J. XU AND L. ZIKATANOV

Lemma 3.2. If A is symmetric and positive definite, under the assumptions (H1) and (H2),
the convergence rate of Algorithm 3.1 is given by kEJ kA where

1
kEJ k2A = 1 − ,
K
with
J
X

K = sup P inf vk + Tk∗ wk , vk + Tk∗ wk R̄−1
, (3.6)
kvkA =1 vk =v k
k=1
PJ
where wk = i=k+1 vi and Tk∗ = Rkt Qk A.

Now we try to extend Lemma 3.2 to semidefinite case. First we consider the case that for
all 1 ≤ i ≤ J,
R(Ri ) = Ni⊥ and N (Ri ) = Ni . (3.7)

Because Ri may be singular, we introduce the Moore-Penrose generalized inverse [6] for this
kind of singular operators. For an operator Bi : Vi 7→ Vi satisfying N (Bi ) = Ni , R(Bi ) = Ni⊥ ,
the Moore-Penrose generalized inverse, Bi† : Vi 7→ Vi is a simple zero extension of Bi−1 : Ni⊥ 7→
Ni⊥ , namely

Bi† ci = 0, ∀ci ∈ Ni ,
Bi† vi = Bi−1 vi , ∀vi ∈ Ni⊥ .

Lemma 3.3. Under the assumptions, (3.7), (H1) and (H2), the convergence rate of Algorithm
3.1 is given by |EJ |A where
1
|EJ |2A = 1 − ,
K
with K = sup|v|A =1,v∈R(A) K(v) and

J
X

K(v) = inf P inf vk + Tk∗ wk , vk + Tk∗ wk R̄†k
,
c∈N vk =v+c
k=1

PJ
where wk = i=k+1 vi and Tk∗ = Rkt Qk A.

Proof. The proof is given in Section 5. 

For the general case, we have the following theorem, which is the main result of this section.

Theorem 3.1. Under the assumptions (H1) and (H2), then the convergence rate of Algorithm
3.1 is given by |EJ |A where
1
|EJ |2A = 1 − ,
K
with K = sup|v|A =1,v∈R(A) K(v) and

J
X

K(v) = inf P inf vk + Tk∗ wk , vk + Tk∗ wk (Qk R̄k Qk )†
, (3.8)
c∈N vk =v+c
k=1

PJ
where wk = i=k+1 vi and Tk∗ = Rkt Qk A.
Iterative methods for semidefinite systems 807

Proof. In Lemma 3.3, we assume that N (Ri ) = Ni and R(Ri ) = Ni⊥ . For the general case,
we define the auxiliary operators R i as

R i = Qi Ri Qi .

By the definition of Qi , we get that

N (R i ) = Ni and R(R i ) = Ni⊥ .

Similarly, we can define the operators T i , E J , R̄ i and T̄ i corresponding to R i .

Since R(Qi A) = Ni⊥ , we have

T i v − Ti v = (Qi − I)Ri Qi Av ∈ Ni , ∀v ∈ V,

and furthermore
|EJ v|A |E J v|A
sup = sup .
v∈R(A) (v, v)A v∈R(A) (v, v)A

Applying Lemma 3.3 for R i and E J , we finally get Theorem 3.1. 

By the definition of Qk , Qk R̄k Qk : Nk⊥ 7→ Nk⊥ is symmetric and positive definite under the
assumptions (H1) and (H2), the generalized inverse of Qk R̄k Qk is relatively easy to understand.
Now we define Pi : V 7→ Ni⊥ , such that


Pi v, wi A = (v, wi )A , ∀v ∈ V, vi ∈ Ni⊥ .

The assumptions (H1) and (H2) are automatic for Ti = Pi (corresponding to Ri = A†i ) with
ωi = βi = 1. If Ti = Pi for all i , which is an important application, we have the following:

Corollary 3.1. Assume Ti = Pi for i = 1, · · · , J. We can obtain that

J J J
!
X X X
K= sup inf P inf Pk vi , vi .
|v|A =1,v∈R(A) c∈N vk =v+c
k=1 i=k i=k A

By the assumptions (H1) and (H2), we try to estimate the related terms in the convergence
rate identity (3.8).

Lemma 3.4. Assume that (H1) and (H2) hold, then

ωi
(vi , vi )(Qk R̄k Qk )† ≤ (vi , vi )A , ∀vi ∈ Vi , (3.9)
βi (2 − ωi )


Ti∗ vi , Ti∗ vi A ≤ ωi2 (vi , vi )A , ∀vi ∈ Vi . (3.10)

Proof. First we see that

βi (2 − ωi )
Ai vi , Ai vi R̄i
= 2 Ti vi , vi − Ti vi , Ti vi ≥ (vi , vi )A .
ωi
Since


(vi , vi )A = (vi , vi )Ai = Ai vi , Ai vi A†i
,
we get that

βi (2 − ωi )

Ai vi , Ai vi Qi R̄i Qi
≥ Ai vi , Ai vi A† .
ωi i
808 J. WU, Y.-J. LEE, J. XU AND L. ZIKATANOV

Qi R̄i Qi and A†i are symmetric and positive definite on R(Ai ) = Ni⊥ . By the above inequality,
we obtain
ωi ωi
(vi , vi )(Qi R̄i Qi )† ≤ (vi , vi )Ai = (vi , vi )A , ∀vi ∈ Ni⊥ .
βi (2 − ωi ) βi (2 − ωi )

Noting that (Qi R̄i Qi )† ci = 0, Aci = 0 for any ci ∈ Ni , we get the first inequality (3.9).
By the definition of Ti∗ , it is easy to see that



Ti v, w A
= v, Ti∗ w A
∀v, w ∈ V.

By (H1) we get that for any vi ∈ Ni⊥ ,

Ti vi , Ti vi A
≤ ωi2 (vi , vi )A ,

then
(Ti∗ vi , Ti∗ vi )A ≤ ωi2 (vi , vi )A , ∀vi ∈ Ni⊥ .

Since Ti∗ ci = 0 and Aci = 0 for any ci ∈ Ni , the above inequality is also valid for any vi ∈ Vi ,
which gives (3.10). 

By Lemma 3.4 and Theorem 3.1, we get the following theorem on the convergence rate
estimate.

Theorem 3.2. Assume that (H1) and (H2) hold, we have that
!
J
X 2ωk  J
X J
X 
K≤ sup inf P inf (vk , vk )A + ωk2 Pk vi , vi .
|v|A =1,v∈R(A) c∈N vk =v+c βk (2 − ωk ) A
k=1 i=k+1 i=k+1

PJ
Proof. Set wk = i=k+1 vi , we have by (3.9) and (3.10),


vk + Tk∗ wk , vk + Tk∗ wk (Q R̄ Q )†
k k k
ωk

≤ vk + Tk wk , vk + Tk∗ wk A
∗
βk (2 − ωk )
2ωk  
≤ (vk , vk )A + (Tk∗ wk , Tk∗ wk )A
βk (2 − ωk )
2ωk  
≤ (vk , vk )A + (Tk∗ Pk wk , Tk∗ Pk wk )A
βk (2 − ωk )
2ωk  
≤ (vk , vk )A + ωk2 (Pk wk , Pk wk )A
βk (2 − ωk )
2ωk  
= (vk , vk )A + ωk2 (Pk wk , wk )A .
βk (2 − ωk )

This completes the proof of the theorem. 

By Theorem 3.2, under the assumptions (H1) and (H2), the convergence rate estimate for
subspace correction methods with inexact subspace solvers is shown to be similar from the
counterpart with exact subspace solvers.
Iterative methods for semidefinite systems 809

4. A Practical Illustration
In this section, we will provide the convergence analysis of the multigrid method for a
semidefinite system from a simple linearized model of lithium-ion battery. We will construct
a multigrid method for the singular system, and show that the multigrid method converges
uniformly with respect to the mesh size h and some relevant parameters of the problem by the
theories developed in the previous section.
The example is on a linearized simple model of lithium-ion battery [18, 19]. The system of
equations can be prescribed on a bounded domain Ω ⊂ IRd (1 ≤ d ≤ 3) such that

Ω̄ = Ω̄a ∪ Ω̄s ∪ Ω̄c ⊂ IRd ,

where Ωa , Ωs and Ωc are rectangular subdomains of Ω that correspond to the negative electrode,
the separator and the positive electrode of the Lithium ion battery respectively, see Fig. 4.1.

negative electrode separator positive electrode

Γa Ωa Ωs Ωc Γc

Fig. 4.1 The domain Ω.

Set Ω′ = Ωa ∪ Ωc . The system of partial differential equations that we are interested takes
the following form:

−∇ · (∇Φe ) = S, x ∈ Ω, (4.1)
′
−∇ · (κ∇Φs ) = −S, x∈Ω, (4.2)

where Φe and Φs are electric potentials (in the electrolyte phase and solid phase respectively),
κ ≫ 1 is positive constants. The transfer current density is given by
(
S 0 + Φs − Φ e , x ∈ Ω ′ ,
S=
0, otherwise.

where S0 is piecewise constant. Proper Neumann boundary conditions are imposed for Φe , Φs ,
i.e.,
∂Φs
−κ = I, for x ∈ ∂Ω′ ,
∂n
where I is a given function and
∂Φe
= 0, for x ∈ ∂Ω.
∂n
810 J. WU, Y.-J. LEE, J. XU AND L. ZIKATANOV

We assume that Z
IdA = 0,
∂Ω′
for the solvability of the system of the partial differential equations (4.1) and (4.2).
Denote Φ = (Φe , Φs ), Ψ = (Ψe , Ψs ). Then we define a bilinear form a(Φ, Ψ):
Z Z
a(Φ, Ψ) = ∇Φe ∇Ψe dx + κ∇Φs ∇Ψs + (Φs − Φe )(Ψs − Ψe )dx (4.3)
Ω Ω′

where Φ, Ψ ∈ H 1 (Ω) × H 1 (Ω′ ). Then the weak formula of (4.1) and (4.2) is given by: Find
Φ ∈ H 1 (Ω) × H 1 (Ω′ ) such that

a(Φ, Ψ) = (F, Ψ), ∀Ψ ∈ H 1 (Ω) × H 1 (Ω′ ), (4.4)

where Z Z
(F, Ψ) = S0 (Ψe − Ψs )dx + IΦs dS.
Ω′ ∂Ω′
The solution to (4.4) is not unique. More precisely, if (Φe , Φs ) is a solution to (4.4), so is
(Φe + C, Φs + C) where C is a constant function. The finite element discretization of (4.4) will
lead to a singular system of equations.
Now we consider the finite element approximation and multigrid setting. Throughout this
section, we assume that Ω is triangulated with a nested sequence of quasi-uniform triangles
Tk = {τki } of size hk , where the quasi-uniformity constants are independent of k and hk ∼ γ k
with γ ∈ (0, 1) for k = 1, · · · , J. We also assume that for any τki ∈ Tk , τki ∩ Ω̄a ( Ω̄s , Ω̄c ) equals
to τki or ∅. Associated with each Tk , we have the finite element space of continuous piecewise
linear functions Vk ⊂ H 1 (Ω) × H 1 (Ω′ ). In this setting, it is clear that

V1 ⊂ · · · ⊂ Vk ⊂ · · · ⊂ VJ .

The finite element approximation is given as follows: Find Φh ∈ VJ = V , such that

a Φh , Ψh = F, Ψh , ∀Ψh ∈ VJ = V. (4.5)

Define a function π(x) (x ∈ Ω̄) satisfying that π(x) = 1 for x ∈ Ω̄a ∪ Ω̄c and π(x) = 0 for
x ∈ Ωs . Clearly we have the following decomposition,
nk
X
Vk = Vki ,
i=1

where Vki = span{(φik , 0), (0, πφik )} and each φik is the usual nodal basis function that is one at
the node xik and zero at the other nodes and nk the number of grid nodes, {xlk }nl=1 k
of Tk .
Now we try to construct a multigrid method which converges independent of h and the
constants κ. For this purpose, we need to define an additional space

V01 = span (1, 0), (0, 1a ), (0, 1c ) ,

where 1 denotes the function that is 1 on Ω, 1a and 1c denote the functions that are 1 on Ω̄a
and Ω̄c respectively, and 0 otherwise. Set n0 = 1, we have the following space decomposition,
nk
J X
X
V = Vki . (4.6)
k=0 i=1
Iterative methods for semidefinite systems 811

The additional subspace V01 is to guarantee the uniform convergence with respect to κ (see
(4.11)).
Under the settings outlined above, the abstract convergence theory shall start with the
following observation that the error transfer operator, E of the subspace correction method
with exact solver in each subspace Vkl can be written as follows
nk
J Y
Y

E= I − Pkl , (4.7)
k=0 l=1

where Pkl is the exact solver on Vkl , (see also [20]).

Denote |Φ|a = a(Φ, Φ)1/2 and N = span(1, π1). By a direct application of Corollary 3.1,
we obtain the following relation:
|E|2a = 1 − K −1 , (4.8)
where PJ Pnk P
2
k=0 i=1 Pki (l,j)≥(k,i) Φjl a
K = sup inf PJ inf
Pn . (4.9)
Φ∈N ⊥ c∈N k=0
k
i=1 Φik =Φ+c |Φ|a

Theorem 4.1. The norm of the error transfer operator, given by (4.9) can be bounded as
follows:
|E|2a ≤ δ < 1, (4.10)
with δ independent of the mesh size h, the number of levels J and the constants κ.

Proof. For any Φ ∈ N ⊥ , we have the decomposition Φ = Ψ + c where Ψ = (Ψe , Ψs ) ∈ (V01 )⊥

R R R
and c ∈ V01 ∩ N ⊥ . Noting that Ω Ψe dx = 0, Ωa Ψs dx = 0 and Ωc Ψs dx = 0; we get

|Φ|2a ≥ |Ψe |21,Ω + κ|Ψs |21,Ω′ & kΨe k21,Ω + κkΨs k21,Ω′ & |Ψ|2a ,

and
|Φ|2a & |Φ|2a + |Ψ|2a & |Φ − Ψ|2a = |c|2a .
Then
PJ Pnk P
2
k=0 i=1 Pki (l,j)≥(k,i) Φjl a
K ≤ sup PJ Pinf
Φ∈N ⊥ k=0
nk
i=1 Φik =Φ |Φ|a
PJ Pnk P
2
k=0 i=1 Pki (l,j)≥(k,i) Ψjl a
. sup PJ Pinf
Ψ∈(V01 )⊥ k=0
nk
i=1 Ψik =Ψ |Ψ|2a
PJ Pnk P j
2
k=0 i=1 Pki (l,j)≥(k,i) cl a
+ sup PJ inf
P
c∈V01 ∩N ⊥ k=0
nk
i=1 cik =c |c|2a
= : I + II.

The estimate on II is simply given by setting c10 = c, and other cik = 0,

II ≤ 1.

For Ψ = (Ψe , Ψs ) ∈ (V01 )⊥ , we have

|Ψ|2a & kΨe k21,Ω + κkΨs k21,Ω′ + kΨe − Ψs k20,Ω′ . (4.11)

812 J. WU, Y.-J. LEE, J. XU AND L. ZIKATANOV

Then, we obtain that by setting Ψ0k = 0,

PJ Pnk P

2
k=1 i=1 Pki (l,j)≥(k,i) Ψjl a
I. sup P J Pinf 2
Ψ∈(V01 )⊥ k=1
nk i
i=1 Ψk =Ψ
kΨ k
e 1,Ω + κkΦs k21,Ω′ + kΨe − Ψs k20,Ω′
PJ Pnk i
P j
2
k=1 i=1 Pk (l,j)≥(k,i) Ψl a
≤ sup PJ Pinf
Ψ∈V k=1
nk
i=1 Ψik =Ψ kΨe k21,Ω + κkΦs k21,Ω′
. 1.

The last inequality above is the standard estimate for multigrid method, we refer [8, 21] for the
detail of the estimate. 

5. Proof of Lemma 3.3

In this section, we will show Lemma 3.3. First we introduce the symmetrization T̄i : V →
7 Vi
of Ti as follow,
T̄i = Ti + Ti∗ − Ti∗ Ti . (5.1)
Then
T̄i = R̄i Qi A.
Because of the assumption (3.7), we have N (Rit ) = Ni and R(Rit ) = Ni⊥ ; and for T̄i : Vi 7→ Vi ,

R(T̄i ) = Ni⊥ and N (T̄i ) = Ni .

One key idea of the convergence analysis of Algorithm 3.1 is to use appropriate restrictions
of subspaces Vi ’s and the subspace solvers Ti ’s onto R(A) and then apply the theory for the
positive definite case. In order to do that, we will introduce an additional projection. The
orthogonal projection with respect to (·, ·), P : V 7→ R(A) is defined as

(Pv, w) = (v, w), ∀v ∈ V, w ∈ R(A).

In what follows, we denote the space PVi by Vei for simplicity.

Lemma 5.1. Suppose that (H1) and (H2) hold. Then the followings hold
PJ
(A0) R(A) = Vei ,
i=1

(A1) ∃ω ∈ (0, 2) such that (PTi vi , PTi vi )A ≤ ωa(PTi vi , vi )A , ∀vi ∈ Vei ,

(A2) PTi : Vei →

7 Vei is an isomorphism.

Proof. Noting that V is finite dimensional and Ti Pvi = Ti vi for any vi ∈ Vi , (A0), (A1)
and (A2) are obtained directly by (3.1), (H1) and (H2), respectively. 
Now we consider |EJ v|A which is defined by (3.5), we have

|EJ v|2A = |PEJ v|2A

= |(I − PTJ )(I − TJ−1 ) · · · (I − T1 )v|2A
= |(I − PTJ )(I − PTJ−1 ) · · · (I − T1 )v|2A
= ···
= |(I − PTJ )(I − PTJ−1 ) · · · (I − PT1 )v|2A
Iterative methods for semidefinite systems 813

Hence
|EJ |A = |(I − PTJ ) · · · (I − PT1 )|A . (5.2)

From the relation (5.2), the fact that (·, ·)A : R(A) × R(A) 7→ IR is positive definite and
Lemma 5.1, we have the following important auxiliary result by the identity in [21] and [8].

Lemma 5.2. Suppose that (H1) and (H2) hold. Then

1
|EJ |2A = (I − PTJ ) · · · (I − PT1 )|2A = 1 − ,
K
PJ
where K = sup|v|A =1,v∈R(A) K(v) and with w̃k := i=k+1 ṽi and ṽi ∈ Vei ,

J
X

K(v) = P inf (P T̄k )−1 (ṽk + PTk∗ w̃k ), (ṽk + PTk∗ w̃k ) A .
ṽk =v
k=1

Now we try to rewrite the expression of K in Lemma 5.2 in terms of real subspace operators
Ri , Ti∗ and T̄i . We first prove a lemma relating T̄i† .

Lemma 5.3. Suppose that (3.7), (H1) and (H2) hold. Then for each 1 ≤ i ≤ J, we have

P T̄i† Pi× = (P T̄i )−1 on Vei . (5.3)

where Pi× : Vei 7→ Ni⊥ , such that PPi× = I on Vei .

Proof. For any given vi ∈ Vi , assume that vi = wi + ci where wi ∈ Ni⊥ and ci ∈ Ni . Notice
that Pi× P = I on Ni⊥ . Then

P T̄i† Pi× P T̄i (Pvi ) = P T̄i† T̄i (Pvi )

= P T̄i† T̄i (wi ) = Pwi = Pvi .

Moreover

P T̄i P T̄i† Pi× (Pvi ) = P T̄i T̄i† Pi× Pwi = Pwi = Pvi .

These equalities clearly show (5.3). 

Lemma 5.4. Under the assumptions, (3.7), (H1) and (H2), the K in Lemma 5.2 is given by
the following: K = sup|v|A =1,v∈R(A) K(v) and

J
X

K(v) = inf P inf T̄k† (vk + Tk∗ wk ), vk + Tk∗ wk A
,
c∈N vk =v+c
k=1

PJ
where wk = i=k+1 vi .

Proof. First, we show that

J
X J
X



(P T̄k )−1 (ṽk + PTk∗ w̃k ), ṽk + PTk∗ w̃k A
= T̄k† (vk + Tk∗ wk ), vk + Tk∗ wk A
, (5.4)
k=1 k=1
814 J. WU, Y.-J. LEE, J. XU AND L. ZIKATANOV

PJ PJ
where ṽk = Pvk , wk = i=k+1 vi and w̃k = i=k+1 ṽi for all 1 ≤ k ≤ J. Observe that


(P T̄k )−1 (ṽk + PTk∗ w̃k ), ṽk + PTk∗ w̃k A


= P T̄k† Pi× (ṽk + PTk∗ w̃k ), ṽk + PTk∗ w̃k A (by Lemma 5.3)


= T̄k† Pi× (Pvk + PTk∗ wk ), Pvk + PTk∗ wk A


= T̄k† Pi× P(vk + Tk∗ wk ), vk + Tk∗ wk A


= T̄k† (vk + Tk∗ wk ), vk + Tk∗ wk A .

In the last equality above, we used the fact that Pi× P : Vi 7→ Ni⊥ is an orthogonal projection
and the property of T̄i† that T̄i† ci = 0 for all ci ∈ Ni . (5.4) follows by the above equalities.
Due to Lemmas 5.1 and 5.2, we may begin the proof with the following expression of K(v),
J
X

K(v) = P inf (P T̄k )−1 (ṽk + PTk∗ w̃k ), (ṽk + PTk∗ w̃k ) A . (5.5)
ṽk =v
k=1

To complete the proof, we have to show that K(v) defined in (5.5) is equal to the following
quantity
XJ


K̃(v) = inf P inf T̄k† (vk + Tk∗ wk ), vk + Tk∗ wk A . (5.6)
c∈N vk =v+c
k=1

It is straightforward to see that K̃(v) ≤ K(v) by (5.4). The reverse inequality also follows
by (5.4) from the fact that for a given v ∈ R(A), any choice c ∈ N and a decomposition of v + c
P P
such that i vi = v + c, there exist a decomposition of v such that i ṽi = v where ṽi = Pvi
(see [9] for the detail). This completes the proof. 
From the definition of Ai , we know that T̄i = R̄i Ai on Vi . Then

T̄i† = (R̄i Ai )† = A†i R̄i† on Vi .

So we get that



T̄k† (vk + Tk∗ wk ), (vk + Tk∗ wk ) A = Ak A†k R̄k† (vk + Tk∗ wk ), (vk + Tk∗ wk )


= R̄k† (vk + Tk∗ wk ), (vk + Tk∗ wk ) .

Combined Lemma 5.2, Lemma 5.4 and the above equalities, we get Lemma 3.3.

Acknowledgments. The first author was supported by NSFC 10501001, the second author
was supported in part by NSF-DMS 0753111, the third author was supported in part by NSF
DMS-0609727, NSF DMS-0749202 and NSFC-10528102, and the forth author was supported
in part by NSF DMS-0511800 and NSF DMS-0749202.

References
[1] A.B. Berma and R.J. Plemmons, Nonnegative Matrices in the Mathematical Sciences, SIAM
Classics in Applied Mathematics, Philadelphia, SIAM, 1994.
[2] P. Bochev and R.B. Lehoucq, On the finite element solution of the pure Neumann problem, SIAM
Rev., 47:1 (2005), 50-66.
 Iterative methods for semidefinite systems 815

[3] J.H. Bramble and J.E. Pasciak, Iterative techniques for time dependent Stokes problems, Comput.
Math. Appl., 33 (1997), 13-30.
[4] Z.H. Cao, A note on properties of splittings of singular symmetric positive semidefinite matrices,
Numer. Math., 88 (2001), 603-606.
[5] Q. Chang and W. Sun, Multigrid methods for singular systems, J. Comput. Math., 23:2 (2005),
177-184.
[6] H.W. Engl, M. Hanke, and A. Neubauer, Regularization of inverse problems, KluwerAcademic
Publishers Group, 1996.
[7] H.B. Keller, On the solution of singular and semidefinite linear systems by iteration, J. Soc. Indust.
Appl. Math. Ser. B Numer. Anal., 2 (1965), 281-290.
[8] Y-J. Lee, J. Wu, J. Xu and L. Zikatanov, On the convergence of iterative methods for semidefinite
linear systems, SIAM J. Matrix Anal. A., 28:3 (2006), 634-641.
[9] Y-J. Lee, J. Wu, J. Xu and L. Zikatanov, A sharp convergence estimate of the method of subspace
corrections for singular system of equations, Math. Comput., 77:262 (2008), 831-850.
[10] Y-J. Lee, J. Wu, J. Xu and L. Zikatanov, Robust subspace correction methods for nearly singular
systems, Math. Mod. Meth. Appl. S., 17:11 (2007), 1937-1963.
[11] I. Marek and D.B. Szyld, Comparison theorems for the convergence factor of iterative methods
for singular matrices, Linear Algebra Appl., 316 (2000), 67-87.
[12] I. Marek and D.B. Szyld, Algebraic schwarz methods for the numerical solution of markov chains,
Linear Algebra Appl., 386 (2004), 67-81.
[13] S.V. Nepomnyaschikh, Schwartz alternating method for solving the singular Neumann problem,
Sov. J. Numer. Anal. Math. Modelling, 5 (1990), 69-78.
[14] J.M. Ortega and Werner C. Rheinboldt, Monotone iterations for nonlinear equations with appli-
cation to Gauss-Seidel methods, SIAM J. Numer. Anal., 4 (1967), 171-190.
[15] T. Strouboulis, I. Babuška and K. Copps, The design and analysis of the generalized finite element
method, Comput. Method. Appl. M., 181:1-3 (2000), 43-69.
[16] T. Strouboulis, K. Copps and I. Babuška, The generalized finite element method: an example of
its implementation and illustration of its performance, Int. J. Numer. Meth. Eng., 47:8 (2000),
1401-1417.
[17] T. Strouboulis, K. Copps and I. Babuška, The generalized finite element method, Comput. Method.
Appl. M., 190:32-33 (2001), 4081-4193.
[18] C.Y. Wang, W.B. Gu and B.Y. Liaw, Micro-macroscopic coupled modeling of batteries and fuel
cells I. Model development, J. Electrochem. Soc., 145:10 (1998), 3407-3417.
[19] J. Wu, V. Srinivasan, J. Xu and C.Y. Wang, Newton-Krylov-Multigrid method for lithium-ion
battery simulation, J. Electrochem. Soc., 149:10 (2002), A1342-A1348.
[20] J. Xu, Iterative methods by space decomposition and subspace correction, SIAM Rev., 34 (1992),
581-613.
[21] J. Xu and L. Zikatanov, The method of subspace corrections and the method of alternating
projections in Hilbert spaces, J. Am. Math. Soc., 15:3 (2002), 573-597.
[22] J. Xu and L. Zikatanov, On multigrid methods for generalized finite element methods, In Meshfree
Methods for Partial Differential Equations (Bonn, 2001), Lect. Notes Comput. Sci. Eng., Vol. 26,
pp. 401-418, Springer, Berlin, 2003.
[23] D.M. Young, Iterative Solution of Large Linear Systems, Dover Publications Inc., Mineola, NY,
2003.

View publication stats