P-regular splitting iterative methods for non-Hermitian positive definite linear systems.
1. Introduction. Many problems in scientific computing give rise to a system of n linear equations in n unknowns,(1.1) Ax = b, A = [[a.sub.ij] [member of] [C.sup.nxn] nonsingular, and b, x [member of] [C.sup.n],
where A is a large, sparse non-Hermitian matrix. In this paper we consider the important case where A is non-Hermitian positive definite; i.e., the Hermitian part H = (A + [A.sup.*])/2 is Hermitian positive definite, where [A.sup.*] denotes the conjugate transpose of the matrix A. We note that the phrase non-Hermitian positive definite, while widely used, is a bit misleading since A could actually be Hermitian. The expression possibly non-Hermitian, positive definite matrix is more precise, but also too cumbersome. The expression strictly accretive is also used, but is not widely adopted. Large, sparse systems with non-Hermitian positive definite coefficient matrix arise in many applications, including discretizations of convection-diffusion problems [17], regularized weighted least-squares problems [13], real-valued formulations of certain complex symmetric systems [9], and so forth. In order to solve system (1.1) by iterative methods, it is useful to construct splittings of the coefficient matrix A. Such splittings are associated with stationary iterative methods, and are frequently used as preconditioners for Krylov subspace methods or as smoothers for multigrid or Schwarz-type schemes; see, e.g., [20, 31, 38]. In general, the coefficient matrix A [member of] [C.sup.nxn] is split into
(1.2) A = M - N,
where M [member of] [C.sup.nxn] is nonsingular and N [member of] [C.sup.nxn]. Then, the general form of stationary iterative methods for (1.1) can be described as follows:
(1.3) [x.sup.(i+1)] = [M.sup.-1][Nx.sup.(i)] + [M.sup.-1]b, i = 0, 1, 2, ....
The matrix T = [M.sup.-1]N is called the iteration matrix of the method (1.3). It is well known [34] that (1.3) converges for any given [x.sup.(0)] if and only if [rho](T) < 1, where [rho](T) denotes the spectral radius of the matrix T. Thus, to establish convergence results for stationary iterative methods, we need to study the spectral radius of the iteration matrix in (1.3).
Next, consider the general class of alternating iterative methods for the solution of (1.1) of the form
(1.4) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
where A = M - N = P - Q are splittings of the coefficient matrix A. Many well known iterative schemes such as the symmetric Gauss-Seidel method [1], the SSOR method [33], alternating-direction and implicit (ADI) methods [26, 34, 38], the Hermitian/skew-Hermitian splitting (HSS) methods [4, 8] and several others belong to this class of methods. To analyze the convergence of the general scheme (1.4), Benzi and Szyld [14] construct a single splitting A = B - C associated with the iteration matrix as follows. Eliminating [x.sup.(i+1/2)] from (1.4), we obtain the iterative process
(1.5)[x.sup.(i+1)] = [P.sup.-1]Q[M.sup.-1][Nx.sup.(i)] + [P.sup.1](Q[M.sup.-1] + I)b, i = 0,1,2, ...,
which is of the form (1.3), where now T = [P.sup.-1] Q[M.sup.-1] N is the iteration matrix. If A is nonsingular and 1 is not an eigenvalue of T, then there exists a unique splitting A = B - C such that T = [B.sup.-1]C = I - [B.sup.-1]A. It is not difficult to see that M + P - A is necessarily invertible and that B = M [(M + P - A).sup.-1]P. The splitting A = B - C is said to be induced by T; see [14] for details.
There have been several studies on the convergence of splitting iterative methods for non-Hermitian positive definite linear systems. In [15, pages 190-193], some convergence conditions for the splitting of non-Hermitian positive definite matrices have been established. More recently, [35] and [36] give some conditions for the convergence of splittings for this class of linear systems.
Recently, there has been considerable interest in the Hermitian and skew-Hermitian splitting (HSS) method introduced by Bai, Golub and Ng for solving non-Hermitian positive definite linear systems, see [4]; we further note the generalizations and extensions of this basic method proposed in [5, 7, 8, 3, 6] and [25]. Furthermore, these methods and their convergence theories have been shown to apply to (generalized) saddle point problems, either directly or indirectly (as a preconditioner); see [5, 2, 3, 7, 6, 35, 36, 25, 11, 12].
Continuing in this direction, in this paper we establish new results on splitting methods for solving system (1.1) iteratively, focusing on a particular class of splittings. For a given matrix A [member of] [C.sup.nxn], a splitting A = M - N with M nonsingular is called a P-regular splitting if the matrix M * + N is non-Hermitian positive definite; see [29]. It is a well-known result [37, 29] that if A is Hermitian positive definite and A = M - N is a P-regular splitting, then the splitting iterative method is convergent: [rho]([M.sup.-1]N) < 1. In this paper, we examine the spectral properties of the iteration matrix induced by P-regular splittings of a non-Hermitian positive definite matrix. Based on these properties, we construct various SOR-type methods for non-Hermitian linear systems and prove their convergence under appropriate restrictions on the choice of the relaxation parameter. While convergence results have been known for many years for Hermitian positive definite matrices, monotone matrices and H-matrices (see, e.g., [15, 20, 31, 38, 29, 21, 34]), very little appears to be known in the non-Hermitian positive definite case. Among the few studies known to us we mention [15, pages 194-195], [27], [28], and [24]. Our results are more general than the few results found in literature, and they complete the SOR theory for non-Hermitian matrices. It is our hope that these results will prove useful in the study of convergence of more sophisticated iterative schemes, including Schwarz-type and algebraic multilevel methods; see, e.g., [19] and [10].
For convenience, some of the terminology used in this paper will be given. The symbol [C.sup.nxn] will denote the set of all nxn complex matrices. Let A, B [member of] [C.sup.nxn]. We use the notation A > 0 (A [greater than or equal to] 0) if A is Hermitian positive (semi-)definite. If A and B are both Hermitian, we write A y B (A [greater than or equal to] B) if and only if A - B y 0 (A - B [greater than or equal to] 0). If A is Hermitian all the eigenvalues of A are real, and we denote by [[lambda].sub.min](A) and [[lambda].sub.max](A) the smallest (i.e., leftmost) and largest (rightmost) eigenvalues, respectively. Let A [member of] [C.sup.nxn] with H = (A + [A.sup.*])/2 and S = (A - [A.sup.*])/2 its Hermitian and skew-Hermitian parts, respectively; then A is non-Hermitian positive (semi-)definite if and only if H y 0 (H [greater than or equal to] 0). Throughout the paper, I will denote the n x n identity matrix.
The paper is organized as follows. Some convergence results for P-regular splittings of non-Hermitian positive definite linear systems are given in section 2. In section 3 we construct some SOR-type methods and use the general theory of section 2 to study their convergence. In section 4 a few numerical examples are given to demonstrate the convergence results obtained in this paper. Some conclusions are given in section 5.
2. General convergence results for P-regular splittings. In this section we establish some convergence results for P-regular splitting methods for non-Hermitian positive definite linear systems. First, some lemmas will be presented to be used in the sequel.
LEMMA 2.1. Let H, B [member of] [C.sup.nxn] be Hermitian and let S [member of] [C.sup.nxn] be skew-Hermitian. If H > 0, then [rho][[(H + S).sup.-1]B] [less than or equal to] p([H.sup.-1]B).
Proof. Since H y 0, [H.sup.-1] y 0 and [H.sup.-1]/2 y 0, it follows that [H.sup.-1]B is similar to the Hermitian matrix [H.sup.-1/2]B[H.sup.-1/2]. As a result,
(2.1) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
Similarly, (H + S)-1B is similar to the matrix
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
Hence, [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.] have the same eigenvalues and therefore
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
Let [lambda] be an eigenvalue of [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.] be a corresponding eigenvector. Then, one has
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
and consequently
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
and
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
Since S is skew-Hermitian, so is [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.] is either purely imaginary. Thus,
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
Therefore,
(2.2) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
which completes the proof.
LEMMA 2.2. (See Ortega [29, page 123].) Let A > 0, and let A = M - N be a P-regular splitting. Then [rho]([M.sup.-1]N) < 1.
THEOREM 2.3. Let A [member of] [C.sup.nxn] be non-Hermitian positive definite, and let A = M - N be a P-regular splitting with N Hermitian. Then [rho]([M.sup.-1]N) < 1.
Proof. Let H(A) = (A + [A.sup.*]) /2 and S(A) = (A - [A.sup.*])/2 be the Hermitian and skew Hermitian parts of A, respectively, and let H(M) = (M + [M.sup.*]) /2 be the Hermitian part of M. Non-Hermitian positive definiteness of A gives that H(A) > 0. Since N is Hermitian, the skew-Hermitian part of M coincides with the skew-Hermitian part of A:
S(M) = 1/2(M - [M.sup.*]*) = 1/2[(M - N) - [(M- N).sup.*]] = 1/2(A - [A.sup.*]) = S (A),
and H(A) = H(M) - N > 0. Again, A = M - N is a P-regular splitting and thus [M.sup.*] + N is positive definite, consequently H (M) + N > 0. Therefore, H (M) > 0 and H(A) = H(M) - N is a P-regular splitting. Lemma 2.2 shows [rho][[(H(M)).sup.-1]N] < 1. Since H(M) > 0, N is Hermitian and S(M) is skew-Hermitian, it follows from Lemma 2.1 that
(2.3) [rho]([M.sup.-1]N) = [rho][(H(M) + S[(M)).sup.-1]N] [less than or equal to] [rho][[(H(M)).sup.-1]N] < 1.
This completes the proof.
COROLLARY 2.4. Let A [member of] [C.sup.nxn] be non-Hermitian positive definite, and let A = M - N be a splitting with N [greater than or equal to] 0. Then [rho]([M.sup.-1]N) < 1.
REMARK 2.5. In the last two results, the condition that N be Hermitian is essential and cannot be relaxed. An obvious example is A = I - S where S = - [S.sup.*] and [parallel] S [[parallel].sub.2] [greater than or equal to] 1. Setting M= I and N = S leads to a P-regular splitting where N is non-Hermitian positive semidefinite and [rho]([M.sup.-1]N) [greater than or equal to] 1.
REMARK 2.6. In the Hermitian case, Lemma has the following converse: if A = [A.sup.*] = M - N is a P-regular splitting and [rho]([M.sup.-1]N) < 1, then Ais positive definite; see [30, page 255]. It is therefore natural to ask whether the converse of Theorem 2.3 holds. That is, given a P-regular splitting A = M - N with N = [N.sup.*] and [rho]([M.sup.-1]N) < 1, is it true that H (A) = 1/2 (A + [A.sup.*]) is positive definite? The answer is negative, as is shown by the splitting A = M - N where
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
This splitting is P-regular, N = [N.sup.*], and [rho]([M.sup.-1]N) < 1; the Hermitian part of the matrix A, however, is not positive definite.
Next, we consider the convergence of the iterative scheme (1.4) or (1.5) for non-Hermitian positive definite linear systems. In [14] the following convergence result for symmetric positive definite linear systems is proved.
THEOREM 2.7. (See [14].) Let A [member of] [R.sup.nxn] be symmetric positive definite, and let A = M - N = P - Q be both P-regular splittings. Then [rho](T) < 1, where T = [P.sup.-1]Q[M.sup.-1]N, and therefore the sequence {[x.sup.(i)]} generated by (1.4) converges to the unique solution of (1.1) for any choice of the initial guess [x.sup.(0)]. Furthermore, the unique splitting A = B - C induced by T is P-regular.
In what follows, we partially generalize this result to non-Hermitian positive definite linear systems. First, some useful lemmas are introduced.
LEMMA 2.8. (See Corollary 7.6.5 in [22].) Let A, B [member of] [C.sup.nxn] be Hermitian with A > 0. Then there exists a nonsingular matrix C [member of] [C.sup.nxn] such that A = [C.sup.*] C and B = [C.sup.*] DC, where D [member of] [R.sup.nxn] is diagonal.
LEMMA 2.9. Let B = [C.sup.*]DC [member of] [C.sup.nxn] with C [member of] [C.sup.nxn] nonsingular and [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
Then the Hermitian matrix [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.] is positive semidefinite.
Proof. Observe that 3 can be decomposed as
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
where [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.] is nonsingular since C is. Writing [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.], (2.4) shows that the Hermitian matrices B and D are congruent, and therefore they must have the same inertia. Hence, all we need to show is that D is positive semidefinite. Letting P denote the odd-even permutation matrix of order 2n, it is immediate to see that
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
Hence, [P.sup.*] D P is just a direct sum of n two-by-two Hermitian matrices, each of which is obviously positive semidefinite. This shows that D > 0, and the proof is complete.
LEMMA 2.10. Let [A.sub.i], [B.sub.j] [member of] [C.sup.nxn] be Hermitian and such that [A.sub.i] > [B.sub.i] [greater than or equal to] 0 for i = 1, 2. Then there exist positive real numbers [e.sub.1], [e.sub.2] such that 2[e.sub.1][A.sub.1] > [e.sub.1][B.sub.1] + [e.sub.2][B.sub.2] and 2[e.sub.2][A.sub.2] > [e.sub.1][B.sub.1] + [e.sub.2][B.sub.2].
Proof. Let [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]. Since [A.sub.i] > [B.sub.i] > 0 for i = 1, 2, it follows that L is a generalized M-matrix in the sense of Elsner and Mehrmann; see [18, Notation 2.3] and [23] for details. Therefore, [L.sup.*] is also a generalized M-matrix, and consequently, [18, Notation 2.3] implies that there exist positive real numbers [e.sub.1], [e.sub.2] such that [e.sub.1][A.sub.1] - [e.sub.2] [B.sub.2] > 0 and [e.sub.2][A.sub.2] - [e.sub.1][B.sub.1] > 0. Observe that [A.sub.i] > [B.sub.i] [greater than or equal to] 0 implies [e.sub.i][A.sub.i] - [e.sub.2][A.sub.2] > 0 for i = 1,2. Therefore, we have 2[e.sub.1][A.sub.1] > [e.sub.1][B.sub.1] + [e.sub.2][B.sub.2] and 2[e.sub.2][A.sub.2] > [e.sub.1][B.sub.1] + [e.sub.2][B.sub.2]. This completes the proof.
Lemma 2.11. Let A[member of] [C.sup.nxn] be non-Hermitian positive definite, and let A = M - N = P - Q be both P-regular splittings with N and Q Hermitian. Then the matrix [H.sub.[mu]] = [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.] is nonsingular for all [mu] [member of] C with [absolute value of [mu]] [less than or equal to] 1.
Proof. Let H(M) and H(P) be the Hermitian parts of Mand P, respectively. Since Ais non-Hermitian positive definite and A= M- N = P - Q are both P-regular splittings with N and Q Hermitian, one has
(2.5) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
Clearly, (2.5) implies that H(M) > 0 and H( P) > 0. Also, N and Q are both Hermitian. It follows from Lemma 2.8 that there exist two nonsingular matrices [C.sub.1], [C.sub.2] [member of] [C.sup.nxn] such that H(M) = [C.sup.*.sub.1], N = [C.sup.*.sub.1][D.sub.1][C.sub.1] and H(P) = [C.sup.*sub.2][C.sub.2], Q = [C.sup.*.sub.2][D.sub.2][C.sub.2], where [D.sub.1], [D.sub.2] [member of] [R.sup.nxn] are diagonal matrices. Following (2.5), we have
(2.6) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
Consequently,
(27) I + [D.sub.1] > 0, I - [D.sub.1] > 0; I + [D.sub.2] > 0, I - [D.sub.2] > 0,
which shows that
(2.8) I [absolute value of [D.sub.1]] > 0, I [absolute value of [D.sub.2]] > 0.
Let [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.] Furthermore,
(2.9) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]This leads to H(M) > N > 0 and H(P) > Q My 0. It then follows from Lemma 2.10 that there exist positive real numbers [e.sub.1] , [e.sub.2] such that
(210) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
Letting E = diag([e.sub.1]I, [e.sub.2]I), we have
(2.11) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
Let [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.] Then (2.10) yields [K.sub.1] > 0 and [K.sub.2] > 0. As a result, K > 0. Letting [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.] Lemma 2.9 shows that N [greater than or equal to] 0 and [[??].sub.[mu]] [greater than or equal to] 0. Therefore,
(212) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
i.e., E [H.sub.[mu]] is non-Hermitian positive definite and thus nonsingular. Since E is nonsingular, [H.sub.[mu]] is nonsingular. This completes the proof.
THEOREM 2.12. Let A [member of] [C.sup.nxn] be non-Hermitian positive definite, and let A = M - N = P - Q be both P-regular splittings with N and Q Hermitian. Then [rho](T) < 1, where T = [P.sup.-1]Q[M.sup.-1]N, and therefore the sequence {[x.sup.(i)]} generated by (1.4) converges to the unique solution of (1.1) for any choice of the initial guess [x.sup.(0)].
Proof. The proof is by contradiction. We assume that Ais an eigenvalue of T with [absolute value of [lambda] [greater than or equal to] 1. Then [lambda]I - T = [lambda]I - [P.sup.-1]Q[M.sup.-1]N is singular. As a result, P - ([[lambda].sup.-1]Q)[M.sup.-1]N is singular. Let [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.] is singular. Observe that S = P - ([mu]Q)[M.sup.-1]N = [H.sub.[mu]]/M, the Schur complement of the matrix [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.] with respect to the matrix M. It follows from the block LU decomposition [39]
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
that [H.sub.[mu]] must be singular. This contradicts Lemma 2.11, according to which matrix [H.sub.[mu]] is nonsingular for [absolute value of [mu]] [greater than or equal to] 1. Therefore T has no eigenvalue [lambda] with [absolute value of [lambda]] [greater than or equal to] 1; that is, [rho](T) < 1 and T = [P.sup.-1]Q[M.sup.1]N is convergent. This completes the proof.
REMARK 2.13. It remains an open question whether the unique splitting A= B - C induced by T in Theorem 2.12 is P-regular.
3. SOR methods for non-Hermitian positive definite systems. In this section we apply the general theory developed in the previous section to study the convergence of SOR-like methods applied to non-Hermitian positive definite systems.
Without loss of generality, we write
(3.1) A = I - L - U = (I - L + [U.sup.*]) - (U + [U.sup.*]) = (I - U + [L.sup.*]) - (L + [L.sup.*]),
where L and U are strictly lower and strictly upper triangular matrices, respectively. The successive over-relaxation method (SOR method) is defined by the iteration matrix
(3.2) [L.sub.w] = [[I - [omega](L - [U.sup.*])].sup.-1] [[omega](U + [U.sup.*]) + (1 - [omega]) I]
while the unsymmetric SOR method (USSOR method) is given by the iteration matrix
(3.3) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
where
(3.4) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
As a special case, when [omega] = [bar.omega] we have the symmetric SOR method (SSOR method), defined by the iteration matrix
(3.5) [J.sub.[omega]] = [U.sub.[omega]] [L.sub.[omega]]
THEOREM 3.1. Let A [member of] [C.sup.nxn] be non-Hermitian positive definite with H = (A + [A.sup.*])/2 its Hermitian part, and let A = I - L - U be defined by (3.1). Also, let [eta] = [[lambda].sub.min] (B) be the smallest eigenvalue of B := H + 2(U + [U.sup.*]).
(i) If [eta] [greater than or equal to] 0, then the SOR method is convergent for [omega] [member of] (0,1);
(ii) If [eta] < 0, then the SOR method is convergent for [omega] [member of] (0, 2/2-[eta]).
Proof. Let M = -1/[omega]I - (L - [U.sup.*]) and N = (1/[omega] - 1) I + (U + [U.sup.*]). Then [L.sub.w] = [M.sup.-1]N and A = M - N is a splitting of A since Mis nonsingular. Let H(M) = (M+ [M.sup.*])/2. Since N is Hermitian,
(3.6) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
(i) If [eta] [greater than or equal to] 0 and [omega] [member of] (0,1), then we have B [greater than or equal to] 0 and 2-2[omega]/[omega] > 0. Identity (3.6) shows H(M) + N = 2-2[omega]/[omega] I + B > 0; that is, M + N is positive definite. Therefore, A = M - N is a P-regular splitting of A. Hence, Theorem 2.3 yields that [rho]([L.sub.[omega]]) = [rho]([M.sup.-1]N) < 1, i.e., the SOR method is convergent.
(ii) If [eta] < 0 and [omega] [member of] (0, 2/2-[eta]), then we have with (3.6) that
(3.7) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
which shows that M + N is positive definite. As a result, A = M - N is a P-regular splitting of A. It follows again from Theorem 2.3 that [rho]([L.sub.[omega]]) = [rho]([M.sup.-1]N) < 1, i.e., the SOR method is convergent. This completes the proof.
REMARK 3.2. Theorem 3.1 becomes Theorem 1 in [27] if A = I - L + [L.sup.T] [member of] [R.sup.nxn]; hence, Theorem 3.1 generalizes the convergence result of Niethammer and Schade.
THEOREM 3.3. Let A [member of] [C.sup.nxn] be non-Hermitian positive definite with H = (A + [A.sup.*])/2 its Hermitian part, and let A = I - L - U be defined by (3.1) and > = [[lambda].sub.min] (B) and p = [[lambda].sub.min](C) be the smallest eigenvalues of B := H + 2(U + [U.sup.*]) and C := H + 2(L + [L.sup.*]), respectively.
(i) If [eta] [greater than or equal to] 0 and [mu] [greater than or equal to] 0, then the USSOR method is convergent for [omega], [omega] [member of] (0,1);
(ii) If [eta] < 0 and [mu] [greater than or equal to] 0, then the USSOR method is convergent for [omega] [member of] (0, 2/2-[eta]) and [bar.[omega]] [member of] (0,1);
(iii) If [eta] > 0 and [mu] < 0, then the USSOR method is convergent for [omega] [member of] (0,1) and [bar.[omega] [member of] (0, 2/2-[mu]);
(iv) If [eta] < 0 and p [mu] 0, then the USSOR method is convergent for [omega] [member of] (0, 2/2-[eta]) and [bar.[omega]] [member of] (0, 2/2-[mu]).
Proof. Let [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]. Then Mand P are nonsingular, N and Q are Hermitian, [L.sub.[omega]] = [M.sup.-1]N, [M.sub.[bar.[omega]]] = [P.sup.-1]Q, and A = M - N = P - Q are splittings of A. Let H(M) = (M + [M.sup.*])/2 and H(P) = (P + [P.sup.*])/2. Since N and Q are Hermitian, (3.6) holds. Furthermore,
(3.8) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
It is easy to prove that both H(M) + N > 0 and H(P) + Q > 0 when (i) [eta] > 0, [mu] [greater than or equal to] 0 and [omega], [bar.[omega]] [member of] (0,1); (ii) [eta] < 0, [mu] [greater than or equal to] 0 and [omega] [member of] (0, 2/2-[eta]), [bar.[omega]] [member of] (0,1); (iii) [eta] > 0, [mu] < 0 and [omega] [member of] (0,1), [bar.[omega]] [member of] (0, [2/2-[mu]]); and (iv) [eta] < 0, [mu] < 0 and [omega] [member of] (0, 2/2-n), [bar.[omega]] [member of] (0, 2/2-[mu]). Therefore, both M + N and P + Q are positive definite and consequently A = M - N = P - Q are P-regular splittings with N and Q Hermitian. Then Theorem 2.12 shows that
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]
i.e., the USSOR method is convergent. This completes the proof. ?
THEOREM 3.4. Let A [member of] [C.sup.nxn] be non-Hermitian positive definite with H = (A + [A.sup.*])/2 its Hermitian part, and let A= I - L - U be defined by (3.1) and > = [[lambda].sub.min](B) and [mu] = [[lambda].sub.min](C) be the smallest eigenvalues of B := H + 2(U + [U.sup.*]) and C := H + 2(L + [L.sup.*]), respectively.
(i) If [eta] [greater than or equal to] 0 and [mu], then the SSOR method is convergent for [omega] [member of] (0, 1);
(ii) If either [eta] [less than or equal to] [mu] < 0 or [eta] < 0 [less than or equal to] [mu], then the SSOR method is convergent for [omega] [member of] (0, 2/2-[eta]);
(iii) If either [mu] [less than or equal to] [eta] < 0 or [mu] < 0 [less than or equal to] [eta], then the SSOR method is convergent for [omega] [member of] (0, 2/2-[mu]);
Proof. The proof can be immediately obtained from Theorem 3.3. ?
4. Numerical experiments. In this section we describe the results of some numerical experiments with the SOR method on a set of linear systems arising from a finite element discretization of a convection-diffusion equation in two dimensions. The purpose of these experiments is not to advocate the use of SOR as a solver for this particular type of problem, but to illustrate the theory developed in this paper, in particular Theorem 3.1.
The model problem is the partial differential equation
(4.1) - [epsilon] [DELTA]u + w x [nabla] u = f,
where [epsilon] > 0, [DELTA] is the 2D Laplacian, [nabla] is the gradient, w is a prescribed vector field (the 'wind'), and / is a given scalar field (the 'source'). The solution u is sought on the unit square [OMEGA] = [0,1] x [0,1], and is subject to suitable boundary conditions. Here we consider the problem given as Example 3.1.3 in [17]: zero source (/ = 0), constant wind at a 30[degrees] angle to the left of vertical (w = (- sin [pi]/6n, cos [pi]/6)), and boundary conditions such that the solution exhibits a downstream boundary layer and an interior layer; see [17, page 118] for details.
Equation (4.1) is discretized on a uniform square grid of size 32 x 32 using Q1 Galerkin finite elements with SUPG stabilization. The resulting matrix Ais nonsymmetric and has complex eigenvalues. Its symmetric part H is positive definite, for all [epsilon] > 0. We note that A has some positive off-diagonal entries and therefore it is not an M-matrix. Prior to forming the SOR splitting, the coefficient matrix Ais diagonally scaled so that its diagonal entries are all equal to 1, hence A = I - L - U with L strictly lower and U strictly upper triangular.
We consider three problem instances, corresponding to [epsilon] = [10.sup.-1], [10.sup.-2] and [10.sup.-3], respectively. The problems becomes increasingly convection-dominated as [epsilon] decreases. In Table
4.1 we report the value of [eta] = [[lambda].sub.min](B), with B = H + 2(U + [U.sup.T]), together with the corresponding value of 2/(2 - [eta]) for the three values of e considered. Recall that according to Theorem 3.1, when [eta] < 0 (as is the case here) the SOR method is guaranteed to converge for all [omega] [member of] (0,2/(2 - [eta])). This is, however, a sufficient condition only. In practice, we found that SOR converges for [omega] [member of] (0, [bar.[omega]]) where UJ is typically somewhat larger than 2/(2 - [eta]). In all three cases, the Gauss-Seidel method ([omega] = 1) was found to diverge. Since 0 < [omega] < 1, the SOR method used here is actually an under-relaxation procedure rather than an over-relaxation one. In Table 4.1 we also report the optimal value [[omega].sub.best] of the relaxation parameter * in the SOR method, determined experimentally (to two digits of accuracy). Finally, as a baseline method we report in Table 4.1 the number of (unpreconditioned) full GMRES [32] iterations. In all our experiments, we report the number of iterations required to reduce the initial residual by five orders of magnitude, starting from a zero initial guess.
In Table 4.2 we report (under 'its') the number of SOR iterations required to solve the three linear systems with the SOR method for two distinct choices of the relaxation parameter, namely, for [omega] = 2/(2 - y) and [omega] = [[omega].sub.best]. We also include (under 'G-its') the number of iterations required by preconditioned GMRES, where the preconditioner is the SOR method with the corresponding value of u. We note that GMRES acceleration is generally not very effective, and sometimes counterproductive. For a discussion of the use of SOR as a preconditoner for Krylov subspace methods; see [16].
Finally, in Table 4.3 we show iteration counts for SOR and SOR-preconditioned GMRES for several values of [omega]. We note that for [omega] [greater than or equal to] 0.7, SOR diverges for all three problems. (For [epsilon] = [10.sup.-2] and [epsilon] = [10.sup.-3], the SOR iteration is already divergent for [omega] [greater than or equal to] 0.6.) The results show that the rate of convergence suffers some deterioration as e decreases. The results also show that GMRES acceleration with suboptimal values can be beneficial; however, the reduction in the number of iterations compared to unpreconditioned GMRES (see Table 4.1) is rather disappointing. In practice, using SOR (with the optimal [omega]) without GMRES acceleration is more effective, in terms of total costs, than using either SOR-preconditioned GMRES or unpreconditioned GMRES; the exception is the case [epsilon] = 0.1, where GMRES preconditioned with the Gauss-Seidel method converges very rapidly. This method, however, behaves poorly for smaller values of [epsilon].
We mention in passing an interesting experimental observation. In all the numerical tests reported above, the iteration matrix of the SOR method,
[L.sub.L] = [[I - [omega](L - [U.sup.*])].sup.-1][[omega](U + [U.sup.*) + (1 - [omega]) I]
was found to have purely real spectrum. This means that instead of GMRES acceleration, standard Chebyshev acceleration could be used instead. Moreover, for * small enough all the eigenvalues of [L.sub.[omega]] are positive.
Our numerical experiments provide an illustration of the convergence result in Theorem 3.1, case (ii). Similar experimental tables could be used to illustrate the other convergence results in this paper, for example for the SSOR method. In practice, of course, it is difficult to use SOR-type methods for solving this type of problem, since it is generally difficult to estimate y and therefore the SOR convergence interval (0,2/(2 - [eta])). Also, estimating [[omega].sub.best] is even more difficult. Of course, more practical methods exist for the solution of problem (4.1), such as Krylov subspace methods with more effective preconditioners or multigrid methods. In light of our results, it is possible that SOR with a small value of [omega] may prove an effective smoother for multigrid applied to problems like the ones considered here.
5. Conclusions. In this paper we have studied the convergence of P-regular splitting methods for the solution of non-Hermitian positive definite linear systems. Some of our results can be regarded as generalizations of analogous results for the Hermitian positive definite case.
As an application of our theory, we obtain new convergence conditions for SOR-like methods in the non-Hermitian case.
Acknowledgments. The first author would like to acknowledge the hospitality of the Department of Mathematics and Computer Science at Emory University, where this work was completed.
REFERENCES
[1] A. C. AITKEN, On the iterative solution of a system of linear equations, Proc. Roy. Soc. Edinburgh Sect. A, 63, (1950). pp. 52-60.
[2] Z.-Z. Bai and G. H. Golub, Accelerated Hermitian and skew-Hermitian splitting iteration methods for saddle-point problems, IMAJ. Numer. Anal., 27 (2007), pp. 1-23.
[3] Z.-Z. BAI, G. H. GOLUB, L.-Z. LU, and J.-F. YIN, Block triangular and skew-Hermitian splitting methods for positive-definite linear systems, SIAMJ. Sci. Comput., 26 (2005), pp. 844-863.
[4] Z.-Z. BAI, G. H. GOLUB, and M. K. NG, Hermitian and skew-Hermitian splitting methods for non-Hermitian positive definite linear systems, SIAMJ. Matrix Anal. Appl., 24 (2003), pp. 603-626.
[5] Z.-Z. BAI, G. H. GOLUB, and M. K. NG, On successive-overrelaxation acceleration of the Hermitian and skew-Hermitian splitting iterations, Numer. Linear Algebra Appl., 17 (2007), pp. 319-335.
[6]--On inexact Hermitian and skew-Hermitian splitting methods for non-Hermitian positive definite linear systems, Linear Algebra Appl., 428 (2008), pp. 413-440.
[7] Z.-Z. BAI, G. H. GOLUB, and J.-Y. PAN, Preconditioned Hermitian and skew-Hermitian splitting methods for non-Hermitian positive semidefinite linear systems, Numer. Math., 98 (2004), pp. 1-32.
[8] M. BENZI, A generalization of the Hermitian and skew-Hermitian splitting iteration, SIAMJ. Matrix. Anal. Appl., 31 (2009), pp. 360-374.
[9] M. BENZI and D. Bertaccini, Block preconditioning of real-valued iterative algorithms for complex linear systems, IMAJ. Numer. Anal., 28 (2008), pp. 598-618.
[10] M. BENZI, A. FROMMER, R. NABBEN and D. B. SZYLD, Algebraic theory of multiplicative Schwarz methods, Numer. Math., 89 (2001), pp. 605-639.
[11] M. BENZI, M. GANDER, and G. H. GOLUB, Optimization of the Hermitian and skew-Hermitian splitting iteration for saddle-point problems, BIT, 43 (2003), pp. 881-900.
[12] M. BENZI and G. H. GOLUB, A preconditioner for generalized saddle point problems, SIAMJ. Matrix Anal. Appl., 26 (2004), pp. 20-41.
[13] M. BENZI and M. K. NG, Preconditioned iterative methods for weighted Toeplitz least squares problems, SIAMJ. Matrix Anal. Appl., 27 (2006), pp. 1106-1124.
[14] M. BENZI and D. B. SZYLD, Existence and uniqueness of splittings for stationary iterative methods with applications to alternating methods, Numer. Math., 76 (1997), pp. 309-321.
[15] A. BERMAN and R. J. PLEMMONS, Nonnegative Matrices in the Mathematical Sciences, Academic Press, New York, NY, 1979. Reprinted by SIAM, Philadelphia, 1994.
[16] M. A. DELONG and J. M. ORTEGA, SOR as a preconditioner, Appl. Numer. Math., 18 (1995), pp. 431-440.
[17] H. ELMAN, D. SILVESTER, and A. WATHEN, Finite Elements and Fast Iterative Solvers with Applications in Incompressible Fluid Dynamics, Numerical Mathematics and Scientific Computation, Oxford University Press, Oxford, 2005.
[18] L. ELSNER and V. MEHRMANN, Convergence of block iterative methods for linear systems arising in the numerical solution of Euler equations, Numer. Math., 59 (1991), pp. 541-559.
[19] A. FROMMER and D. B. SZYLD, Weighted max norms, splittings, and overlapping Schwarz iterations, Numer. Math., 83 (1999), pp. 259-278.
[20] G. H. GOLUB and C. F. VAN LOAN, Matrix Computations, third edition, Johns Hopkins University Press, Baltimore, MD, 1996.
[21] A. HADJIDIMOS, Accelerated overrelaxation method, Math. Comp., 32 (1978), pp. 149-157.
[22] R. A. HORN and C. R. JOHNSON, Matrix Analysis, Cambridge University Press, New York, 1985.
[23] T.-Z. HUANG, S.-Q. SHEN and H.-B. LI, On generalized H-matrices, Linear Algebra Appl., 396 (2005), pp. 81-90.
[24] L. A. KRUKIERAND T. S. MARTYNOVA, Point SOR and SSOR methods for the numerical solution of the steady convection-diffusion equation with dominant convection, in Iterative Methods in Scientific Computation IV, D. R. Kincaid and A. C. Elster, Eds., IMACS Series in Computational and Applied Mathematics, 5, New Brunswick, NJ, 1999, pp. 399-404.
[25] L. LI, T.-Z. HUANG, and X.-P. LIU, Modified Hermitian and skew-Hermitian splitting methods for non-Hermitian positive-definite linear systems, Numer. Linear Algebra Appl., 14 (2007), pp. 217-235.
[26] G. I. MARCHUK, Splitting and alternating direction methods, in , Handbook of Numerical Analysis, Vol. I, P. G. Ciarlet and J. L. Lions, Eds., North Holland, New York, NY, 1990, pp. 197-462.
[27] W. NIETH AMMER and J. SCHADE, On a relaxed SOR-method applied to nonsymmetric linear systems, J. Comput. Appl. Math., 1 (1975), pp. 133-136.
[28] W. NIETHAMMER and R. S. VARGA, Relaxation methods for non-Hermitian linear systems, Results in Mathematics, 16 (1989), pp. 308-320.
[29] J. M. Ortega, Numerical Analysis. A Second Course, Academic Press, New York, NY, 1972. Reprinted by SIAM, Philadelphia, 1990.
[30]--Introduction to Parallel and Vector Solution of Linear Systems, Plenum Press, New York, NY, 1988.
[31] Y. SAAD, Iterative Methods for Sparse Linear Systems, second edition, SIAM, Philadelphia, 2003.
[32] Y. SAAD and M. H. SCHULTZ, GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems, SIAMJ. Sci. Stat. Comput., 7 (1986), pp. 856-869.
[33] J. W. SHELDON, On the numerical solution of elliptic difference equations, Mathematical Tables and Other Aids to Computation, 9 (1955), pp. 1101-112.
[34] R. S. VARGA, Matrix Iterative Analysis, second edition, Springer-Verlag, Berlin/Heidelberg, 2000.
[35] C.-L. WANG and Z.-Z. BAI, Sufficient conditions for the convergent splitting of non-Hermitian positive definite matrices, Linear Algebra Appl., 330 (2001), pp. 215-218.
[36]--Convergence conditions for splitting iteration methods for non-Hermitian linear systems, Linear Algebra Appl., 428 (2008), pp. 453-468.
[37] J. WEISSINGER, Verallgemainerungen des Seidelschen Iterationsverfahrens, Z. Angew. Math. Mech., 33 (1953), pp. 155-162.
[38] D. M. YOUNG, Iterative Solution of Large Linear Systems, Academic Press, New York, NY, 1971.
[39] F. ZHANG, The Schur Complement and Its Applications, Springer, New York, 2005.
CHENG-YI ZHANG ([dagger]) and MICHELE BENZI ([double dagger])
Dedicated to Richard S. Varga on the occasion of his 80th birthday
* Received May 18, 2009. Accepted for publication September 6, 2009. Published online January 15, 2010. Recommended by D. Szyld.
([dagger]) Department of Mathematics of School of Science, Xi'an Jiaotong University, Xi'an, Shaanxi 710049, P.R. China (zhangchengyi-2004@ 163.com).
([double dagger]) Department of Mathematics and Computer Science, Emory University, Atlanta, GA30322, USA (benzi@mathcs.emory.edu). The work of this author was supported in part by the National Science Foundation grant DMS-0511336.
Table 4.1 Values of [eta], 2/(2--[eta]) and
GMRES iterations for different values of [epsilon].
[epsilon] = [10.sup.-1] [epsilon] = [10.sup.-2]
[eta] -2.033 -2.406
2/2-[eta] 0.496 0.454
[[omega].sub.best] 0.57 0.55
GMRES 53 48
[epsilon] = [10.sup.-3]
[eta] -2.646
2/2-[eta] 0.430
[[omega].sub.best] 0.51
GMRES 52
Table 4.2
Results for [omega] =2/2-[eta] and for [omega] = [[omega].sub.best].
[epsilon] = [10.sup.-1]
[omega] [rho]([L.sub.[omega]]) its G-its
[omega] = 2/2-[eta] 0.622 32 35
[omega] = 0.581 27 33
[[omega].sub.best]
e = [10.sup.-2]
[omega] [rho]([L.sub.[omega]]) its G-its
[omega] = 2/2-[eta] 0.735 48 40
[omega] = 0.705 43 39
[[omega].sub.best]
e = [10.sup.-3]
[omega] [rho]([L.sub.[omega]]) its G-its
[omega] = 2/2-[eta] 0.776 58 44
[omega] = 0.757 52 47
[[omega].sub.best]
Table 4.3
Results for different values of [omega].
[epsilon] = [10.sup.-1]
[rho]([L.sub.[omega]]) its G-its
[omega] 0.906 176 44
0.1 0.822 83 43
0.2 0.748 53 41
0.3 0.681 39 38
0.4 0.620 31 35
0.5 0.565 32 32
0.6 > 1 [infinity] 19
1.0
[epsilon]= [10.sup.-2]
[omega] [rho]([L.sub.[omega]]) its G-its
0.1 0.913 176 44
0.2 0.848 92 43
0.3 0.796 66 42
0.4 0.754 52 40
0.5 0.720 48 39
0.6 > 1 [infinity] 39
1.0 > 1 [infinity] 131
[epsilon]= [10.sup.-3]
[omega] [rho]([L.sub.[omega]]) its G-its
0.1 0.918 182 45
0.2 0.860 100 44
0.3 0.818 74 43
0.4 0.785 61 43
0.5 0.759 53 47
0.6 >1 [infinity] 58
1.0 >1 [infinity] > 300
 Printer friendly
 Cite/link
 Email
 Feedback
| |
| Author: | Zhang, Cheng-Yi; Benzi, Michele |
|---|---|
| Publication: | Electronic Transactions on Numerical Analysis |
| Article Type: | Report |
| Date: | Apr 1, 2009 |
| Words: | 6788 |
| Previous Article: | Zeros of sections of the binomial expansion. |
| Next Article: | Alternating projected Barzilai-Borwein methods for nonnegative matrix factorization. |
| Topics: | |