Structure preserving restarts of the non-symmetric Lanczos algorithm via the implicitly shifted LR algorithm

Abstract

The implicitly shifted QR iteration is used as a restart procedure for the Arnoldi method for the calculation of a few dominant eigenvalues of a large matrix. We show that the underlying idea of implicit polynomial filtering can be utilized in much the same manner via the implicitly shifted LR iteration to create a restart procedure for the non-symmetric Lanczos algorithm for eigenvalue computations, which preserves the tri-diagonal structure of the reduced matrix.

68Q25, 68R10, 68U05

1 Introduction

The Arnoldi iteration [1] is a popular Krylov space method for calculating a few eigenvalues of a large matrix. The method relies on the generation of a sequence of Krylov vectors which determine the subspace within which approximations of the eigenvalue-eigenvector pairs are obtained. Depending on the accuracy and number of eigenpair approximations needed, the Krylov space size can become exceedingly large so that the quality of the results may be limited by the available memory. Sorensen [18] introduced an elegant procedure for restarting the Arnoldi factorization based on polynomial filters, which are applied through the implicitly shifted QR iterations on the reduced Hessenberg matrix obtained through the Arnoldi method. In particular, the use of exact shifts was shown to be successful in the convergence process of the eigenspace[18] of the specified eigenvalues. The method has subsequently found widespread application through the ARPACK library [11]. An alternative, more modern method for eigenvalue computations is the Krylov-Schur method introduced by Stewart [19], which can be found implemented in the SLEPc library [9]. Krylov-Schur restarts are known to be less sensitive than the restarting based on the implicitly shifted QR method. However, they do not preserve the Hessenberg structure of the reduced matrix which may sometimes be required. The use of QR iterations ensures that the reduced matrix preserves its Hessenberg structure through the transforms that make up the restart process. If the underlying matrix is symmetric, the Arnoldi iteration reduces to the Lanczos algorithm, and the Hessenberg matrix reduces to a symmetric tridiagonal matrix. The QR iteration preserves the symmetric tridiagonal structure as well and, as pointed out by Sorensen [18], the implicit restart process applies equally well for the Lanczos method for symmetric matrices.

The Lanczos algorithm introduced in [10] is in fact the predecessor of the Arnoldi method and can be used for non-symmetric matrices as well. In such cases it is referred to as the non-symmetric or the bi-Lanczos. It has been used in investigationss for plasma instabilities [13], linear systems solvers [8], model reduction [12], and more recently for non-linear eigenvalue problems [5]. For the non-symmetric Lanczos one builds two Krylov subspaces referred to as the right and the left Krylov subspaces. The idea behind the two methods is similar, which is to obtain a projection of the original large matrix on to an appropriate reduced subspace such that, the eigenvalues may be approximated via the eigenvalues of the reduced operator. The difference being that in the Arnoldi method one obtains an orthogonal projection on to a subspace while, in the Lanczos method one obtains an oblique projection. One would therefore like to extend the idea of implicit restarts to the non-symmetric Lanczos algorithm as well. However, the reduced matrix that one obtains in such a case is a non-symmetric tridiagonal matrix, with the tridiagonal structure being the result of the recurrence relations of the Lanczos algorithm [17]. Since the QR iterations do not preserve the banded structure of non-symmetric matrices, a straightforward application of the restart procedure put forward by Sorensen will lead to a loss of this tridiagonal structure (the Hessenberg structure will still be preserved). This loss of structure can be circumvented if one looks to the predecessor of the QR algorithm namely, the LR algorithm proposed by Rutishauser [14, 16, 15], which has the attractive property of preserving the band structure of a matrix. This property was already pointed out by Rutishauser in [14] where the banded matrices were referred to as striped matrices. As we will show in the next section, the shifted LR transform is the appropriate analogue of the restart procedure in the case of non-symmetric Lanczos iteration. The process would necessarily require refining both the right as well as the left Krylov spaces simultaneously.

The rest of the paper is organized as follows. In section 2 we start with the introduction of the non-symmetric Lanczos iteration and then develop the restart procedure. In section 3 we apply the restart process to the Grcar matrix, and make some concluding remarks in section 4.

2 Non-symmetric Lanczos

Lanczos first introduced his algorithm in [10] as a method for tridiagonalizing a matrix, but also realized that the method could be used iteratively to find eigenvalues. For an arbitry matrix $A$, the method generates a pair of Krylov subspaces $\{v_{1},\ldots,v_{m}\}$ and $\{w_{1},\ldots,w_{m}\}$, through repeated action of $A$ and $A^{H}$ respectively. We refer to these as the right and left Krylov spaces respectively and they satisfy the biorthogonality relation $w_{i}^{H}v_{j}=\delta_{ij}$. The two subspaces are generated through a three term recurrence relation which, for a Krylov space of size $m$, can be written in matrix form as

(1a) $$AV_{m}-V_{m}T_{m} = \widetilde{v}_{m+1}e_{m}^{T},$$

(1b) $$A^{H}W_{m}-W_{m}T^{H}_{m} = \widetilde{w}_{m+1}e_{m}^{T},$$

(1c) $$W_{m}^{H}V_{m} = I_{m},$$

where, $I_{m}$ represents the Identity matrix of size $m$, $T_{m}$ is a tri-diagonal matrix of size $m$ with $T_{m}^{H}$ it’s Hermitian conjugate, and $e_{m}$ is the standard unit vector. $\widetilde{v}_{m+1}$ and $\widetilde{w}_{m+1}$ represent the residual vectors at the $m^{th}$ step. If either $\widetilde{v}_{m+1}$ or $\widetilde{w}_{m+1}$ vanishes it represents the convergence of the right or the left Krylov subspaces to an invariant subspace of dimension $m$. A more serious breakdown occurs if $\widetilde{w}_{m+1}^{H}\widetilde{v}_{m+1}=0$ with both $\widetilde{v}_{m+1}\neq 0$ and $\widetilde{w}_{m+1}\neq 0$, in which case a look-ahead strategy may be employed. We do not address the issues with breakdown here since it is not specifically related to the restart procedure. We refer the reader to [13] for the look-ahead Lanczos and to [8] for a comprehensive overview on Lanczos type solvers and the related issues of breakdown.

As Sorensen points out for the Arnoldi method [18], if one is interested in an invariant subspace of dimension $m$, the starting vector of the Krylov subspace must not contain components of the generator of a cyclic subspace of dimension greater than $m$. This applies equally for the right and left Krylov subspaces generated through the Lanczos recurrence relations. Hence a non-vanishing $\widetilde{v}_{m+1}$ (respectively $\widetilde{w}_{m+1}$) implies that $v_{1}$ (respectively $w_{1}$) contains components of an invariant subspace of dimension greater than $m$. The idea behind restarts then is to discard the components of the starting vector $v_{1}$ (and $w_{1}$) along the unwanted dimensions, such that each restart process moves the Krylov space(s) closer to being invariant. For the Arnoldi method Sorensen [18] proposed to achieve this via polynomial filtering, i.e. replacing

(2a) $$v_{1}\leftarrow\psi(A)v_{1},$$

(2b) $$\psi(\lambda)=(1/\tau)\Pi_{j=1}^{p}(\lambda-\mu_{j}).$$

Obviously $\psi(\lambda)$ is the filtering polynomial, $\tau$ is a normalization constant and each $\mu_{j}$ specifies a node of the polynomial. The polynomial acts on $v_{1}$ to filter out the part of the spectrum of $A$ that is close to each $\mu_{j}$. If a particular $\mu_{j}$ corresponds to an exact eigenvalue of $A$, then components of the corresponding eigenvector are completely filtered out from $v_{1}$ (at least in exact arithmetic). The node $\mu_{j}$ is referred to as a shift since the application of the polynomial filtering relies on the shifted QR algorithm, where $\mu_{j}$ is used as the shift. As shown below for the case of a single shift, an analogous procedure can be followed using a shifted LR algorithm which achieves the same effect of applying a polynomial filter to the starting vector $v_{1}$. Starting with the Lanczos relation for the right subspace (1a), and adding and subtracting $\mu V_{m}$ we obtain

(3a) $$(A-\mu I)V_{m}-V_{m}(T_{m}-\mu I) = \widetilde{v}_{m+1}e_{m}^{T}$$

(3b) $$(A-\mu I)V_{m}-V_{m}(L_{1}R_{1}) = \widetilde{v}_{m+1}e_{m}^{T}$$

(3c) $$(A-\mu I)V_{m}L_{1}-V_{m}(L_{1}R_{1})L_{1} = \widetilde{v}_{m+1}e_{m}^{T}L_{1}$$

(3d) $$A(V_{m}L_{1})-(V_{m}L_{1})(R_{1}L_{1}+\mu I) = \widetilde{v}_{m+1}e_{m}^{T}L_{1}$$

(3e) $$AV^{\prime}_{m}-V^{\prime}_{m}T^{\prime}_{m} = \widetilde{v}_{m+1}e_{m}^{T}L_{1}.$$

Here we have set $V^{\prime}_{m}=V_{m}L_{1}$ and $T^{\prime}_{m}=(R_{1}L_{1}+\mu I)$. The matrices $L_{1},R_{1}$ are the lower and upper triangular matrices obtained from the LU decomposition of $(T_{m}-\mu I)$. The matrix $L_{1}$ can be required to be unit triangular (all entries on the main diagonal are ones), in which case the LU decomposition is unique. Furthermore, $L_{1}$ for a tridiagonal matrix only consists of one sub-diagonal (in addition to the main diagonal). One can easily recognize that the new reduced matrix $T^{\prime}_{m}$ is a result of one step of the shifted LR iteration. Hence $T^{\prime}_{m}$ retains the tridiagonal structure of the $T_{m}$ [14, 15]. The relationship between starting vectors of the two spaces $V_{m}$ and $V^{\prime}_{m}$ can be obtained by multiplying (3b) by $e_{1}$, i.e.

$$(A-\mu I)V_{m}e_{1}-(V^{\prime}_{m})R_{1}e_{1} = \widetilde{v}_{m+1}e_{m}^{T}e_{1}$$

$$\implies(A-\mu I)v_{1} = v^{\prime}_{1}\rho_{11},$$

where $\rho_{11}=e^{T}_{1}R_{1}e_{1}$. This clearly shows the filtering operation done on the original vector $v_{1}$ to generate the new vector $v^{\prime}_{1}$.

Since the Lanczos method creates a biorthogonal basis, one must simultaneously transform the left basis $W_{m}$ to maintain the biorthogonality property. It is easy to see that the necessary transform to maintain biorthogonality is $W^{\prime}_{m}=W_{m}L_{1}^{-H}$, since,

$$W^{\prime H}V^{\prime}_{m}=(W_{m}L_{1}^{-H})^{H}(V_{m}L_{1})=L_{1}^{-1}(W^{H}_{m}V_{m})L_{1}=I$$

Substituting the relation obtained from the LU decomposition $T_{m}^{H}=(R_{1}^{H}L_{1}^{H}+\bar{\mu}I)$ in to equation (1b) we can obtain the modified Lanczos relation for the left Krylov space as

(4a) $$A^{H}W_{m}-W_{m}(R^{H}_{1}L^{H}_{1}+\bar{\mu}I) = \widetilde{w}_{m+1}e_{m}^{T}$$

(4b) $$A^{H}(W_{m}L_{1}^{-H})-W_{m}(L_{1}^{-H}L_{1}^{H})(R^{H}_{1}L^{H}_{1}+\bar{\mu}I)L_{1}^{-H} = \widetilde{w}_{m+1}e_{m}^{T}L_{1}^{-H}$$

(4c) $$A^{H}(W_{m}L_{1}^{-H})-(W_{m}L_{1}^{-H})(L^{H}_{1}R^{H}_{1}+\bar{\mu}I) = \widetilde{w}_{m+1}e_{m}^{T}L_{1}^{-H}$$

(4d) $$A^{H}W^{\prime}_{m}-W^{\prime}_{m}(T^{\prime}_{m})^{H} = \widetilde{w}_{m+1}e_{m}^{T}L_{1}^{-H}.$$

Conveniently the structure of the Lanczos iteration for the left Krylov space is also preserved. Noting that $L_{1}^{-H}$ is upper triangular, one can again expose the relationship between the generating vectors of the two left Krylov spaces as

$$W^{\prime}e_{1} = (W_{m}L_{1}^{-H})e_{1}$$

$$\implies w^{\prime}_{1} = w_{1}(e_{1}^{T}L_{1}^{-H}e_{1}).$$

Clearly $w^{\prime}_{1}$ is simply a scalar multiple of the old vector $w_{1}$ and no filtering of the generating vector has occurred. In order to ensure that we filter the left Krylov space as well, we perform one step of the shifted LR iteration with the conjugated shift $\bar{\mu}$ on the reduced matrix $(T^{\prime}_{m})^{H}$ obtained in equation (4d).

(5a) $$(A-\mu I)^{H}W^{\prime}_{m}-W^{\prime}_{m}(T^{\prime}_{m}-\mu I)^{H} = \widetilde{w}_{m+1}e_{m}^{T}L_{1}^{-H},$$

(5b) $$(A-\mu I)^{H}W^{\prime}_{m}-W^{\prime}_{m}(L_{2}R_{2}) = \widetilde{w}_{m+1}e_{m}^{T}L_{1}^{-H},$$

(5c) $$(A-\mu I)^{H}W^{\prime}_{m}L_{2}-W^{\prime}_{m}(L_{2}R_{2})L_{2} = \widetilde{w}_{m+1}e_{m}^{T}L_{1}^{-H}L_{2},$$

(5d) $$A(W^{\prime}_{m}L_{2})-(W^{\prime}_{m}L_{2})(R_{2}L_{2}+\bar{\mu}I) = \widetilde{w}_{m+1}e_{m}^{T}L_{1}^{-H}L_{2},$$

(5e) $$AW^{\prime\prime}_{m}-W^{\prime\prime}_{m}(T^{\prime\prime}_{m})^{H} = \widetilde{w}_{m+1}e_{m}^{T}L_{1}^{-H}L_{2}.$$

One may again obtain the relation between the starting vectors of the two spaces as

(6) $$(A^{H}-\bar{\mu}I)w^{\prime}_{1}=w^{\prime\prime}_{1}(e_{1}^{T}R_{2}e_{1}),$$

which clearly shows the filtering operation performed on the starting vector of the left Krylov space. Obviously the appropriate transform for the right subspace to maintain orthogonality is $V^{\prime\prime}_{m}=V^{\prime}_{m}L_{2}^{-H}=V_{m}L_{1}L_{2}^{-H}$. Again, note that the upper triangular $L_{2}^{-H}$ implies that the new $v^{\prime\prime}_{1}$ is simply the scalar multiple of $v^{\prime}_{1}$ and no additional filtering occurs for $v_{1}$ in this step. One can write the corresponding modified Lanczos relation as

(7) $$AV^{\prime\prime}_{m}-V^{\prime\prime}_{m}T^{\prime\prime}_{m} = \widetilde{v}_{m+1}e_{m}^{T}L_{1}L_{2}^{-H}.$$

The above process can be repeated for $p$ unwanted shifts. We denote by $L_{1}^{p}=L_{11}L_{12}\ldots L_{1p}$ as the product of the lower triangular matrices generated due to $p$ shifted-LR steps for the right Krylov space $V_{m}$, and by $L^{p}_{2}=L_{21}L_{22}\ldots L_{2p}$ as the product of the lower triangular matrices due to the $p$ shifted-LR iterations for the left Krylov space. Then for a Krylov space size of $m=k+p$ we have two modified Lanczos relations

(8a) $$AV^{\prime\prime}_{k+p}-V^{\prime\prime}_{k+p}T^{\prime\prime}_{k+p} = \widetilde{v}_{k+p+1}e_{k+p}^{T}L_{1}^{p}(L_{2}^{p})^{-H},$$

(8b) $$A^{H}W^{\prime\prime}_{k+p}-W^{\prime\prime}_{k+p}(T^{\prime\prime}_{k+p})^{H} = \widetilde{w}_{k+p+1}e_{k+p}^{T}(L_{1}^{p})^{-H}L_{2}^{p}.$$

We may take a closer look at the structure of the residual matrices on the right hand side of equation (8a). $L_{1}^{p}$ is a product of $p$ matrices that are lower triangular with just one subdiagonal. $L_{1}^{p}$ then is lower triangular with $p$ non-zero subdiagonals. $(L_{2}^{p})^{-H}$ is upper triangular and the product $L_{1}^{p}(L_{2}^{p})^{-H}$ therefore has $p$ non-zero subdiagonals. Left multiplication by $e_{k+p}^{T}$ therefore has the form

$$e_{k+p}^{T}L_{1}^{p}(L_{2}^{p})^{-H}=(\underbrace{0,0\ldots,\theta_{k+p}}_{k},\underbrace{b_{1}^{T}}_{p})$$

where, $\theta_{k+p}=e_{k+p}^{T}(L_{1}^{p}(L_{2}^{p})^{-H})e_{k}$. Therefore the matrix on the right hand side of (8a) has zeros in the first $k-1$ columns and the $k^{th}$ column is simply $\theta_{k+p}\widetilde{v}_{k+p+1}$. The remaining columns are non-zero in general. A very similar structure is obtained for the residual matrix in the right hand side of equation (8b) with zeros in the first $k-1$ columns and the $k^{th}$ column being equal to $\phi_{k+p}\widetilde{w}_{k+p+1}$, with $\phi_{k+p}=e_{k+p}^{T}((L_{1}^{p})^{-H}L_{2}^{p})e_{k}$

Partitioning the matrices such that

$$V^{\prime\prime}_{k+p}=(V^{\prime\prime}_{k},V^{\prime\prime}_{p}), \hskip 10.0ptT^{\prime\prime}_{k+p}=\begin{pmatrix}T^{\prime\prime}_{k}&\delta_{k+1}e_{k}e_{1}^{T}\\ \beta_{k+1}e_{1}e_{k}^{T}&T^{\prime\prime}_{p}\end{pmatrix},$$

$$W^{\prime\prime}_{k+p}=(W^{\prime\prime}_{k},W^{\prime\prime}_{p}), \hskip 10.0pt(T^{\prime\prime}_{k+p})^{H}=\begin{pmatrix}(T^{\prime\prime}_{k})^{H}&\bar{\beta}_{k+1}e_{k}e_{1}^{T}\\ \bar{\delta}_{k+1}e_{1}e_{k}^{T}&(T^{\prime\prime}_{p})^{H}\end{pmatrix},$$

with the length of the $e_{i}$ vectors understood to be such that the resulting matrices are consistent. We can write the modified Lanczos relations of (8a) and (8b) as

$$A(V^{\prime\prime}_{k},V^{\prime\prime}_{p})=(V^{\prime\prime}_{k},V^{\prime\prime}_{p})\begin{pmatrix}T^{\prime\prime}_{k}&\delta_{k+1}e_{k}e_{1}^{T}\\ \beta_{k+1}e_{1}e_{k}^{T}&T^{\prime\prime}_{p}\end{pmatrix}+\begin{pmatrix}\theta_{k+p}\widetilde{v}_{k+p+1}e_{k}^{T},M_{v}\end{pmatrix},$$

$$A^{H}(W^{\prime\prime}_{k},W^{\prime\prime}_{p})=(W^{\prime\prime}_{k},W^{\prime\prime}_{p})\begin{pmatrix}(T^{\prime\prime}_{k})^{H}&\bar{\beta}_{k+1}e_{k}e_{1}^{T}\\ \bar{\delta}_{k+1}e_{1}e_{k}^{T}&(T^{\prime\prime}_{p})^{H}\end{pmatrix}+\begin{pmatrix}\phi_{k+p}\widetilde{w}_{k+p+1}e_{k}^{T},M_{w}\end{pmatrix},$$

Finally, equating the individual columns on both sides and discarding columns $k+1,\ldots,k+p$ we are left with the new Krylov spaces of order $k$ and the Lanczos relations

(10a) $$AV^{\prime\prime}_{k}-V^{\prime\prime}_{k}T^{\prime\prime}_{k} = \widetilde{v}^{\prime\prime}_{k+1}e_{k}^{T},$$

(10b) $$A^{H}W^{\prime\prime}_{k}-W^{\prime\prime}_{k}(T^{\prime\prime}_{k})^{H} = \widetilde{w}^{\prime\prime}_{k+1}e_{k}^{T},$$

(10c) $$(W^{\prime\prime}_{k})^{H}V_{k} = I.$$

The new residual vectors are defined as

(11a) $$\widetilde{v}^{\prime\prime}_{k+1} = \beta_{k+1}V^{\prime\prime}_{p}e_{1}+\theta_{k+p}\widetilde{v}_{k+p+1},$$

(11b) $$\widetilde{w}^{\prime\prime}_{k+1} = \bar{\delta}_{k+1}W^{\prime\prime}_{p}e_{1}+\phi_{k+p}\widetilde{w}_{k+p+1},$$

which may be normalized appropriately such that the inner product of the new Krylov vectors is unity. The Lanczos process may now be carried out again to generate the next $p$ vectors of the right and left Krylov spaces and the cycle may be repeated till an adequately converged subspace has been obtained. In the case that exact shifts (eigenvalues of $T_{m}$) are used for the restart procedure $\beta_{k+1}$ and $\delta_{k+1}$ are both zero in exact arithmetic.

As a final note, we mention that Della-Dora [3] introduced a class of algorithms of the GR type of which, LR and QR are special cases. Watkins then introduced generic bulge-chasing algorithms for the entire GR class of methods[20]. The shifted LR algorithm used in the restart procedure outlined above can therefore be carried out in an implicit manner through the bulge-chase sequence of Watkins [20]. For real matrices, the operations can be confined in the real space by using the double-shift strategy introduced by Francis [4]. The entire restart process is put together in algorithm 1.

Algorithm 1 Restarted nonsymmetric Lanczos

Input: $V_{k+p},\widetilde{v}_{k+p+1},W_{k+p},\widetilde{w}_{k+p+1},T_{k+p}$
Input: $\mu_{1},\mu_{2},\ldots\mu_{p}$ $\triangleright$ Unwanted shifts
Output: $V_{k},\widetilde{v}_{k+1},W_{k},\widetilde{w}_{k+1},T_{k}$

1:$AV_{k+p}-V_{k+p}T_{k+p}=\widetilde{v}_{k+p+1}e_{k+p}^{T}$ $\triangleright$ Right Lanczos relation

2:$A^{H}W_{k+p}-W_{k+p}T^{H}_{k+p}=\widetilde{w}_{k+p+1}e_{k+p}^{T}$ $\triangleright$ Left Lanczos relation

3:$W_{k+p}^{H}V_{k+p}=I$; $W_{k+p}^{H}\widetilde{v}_{k+p+1}=0$; $V_{k+p}^{H}\widetilde{w}_{k+p+1}=0$; $\triangleright$ Bi-orthogonality

4:procedure Refine right subspace

5:     $L_{r}\leftarrow I_{k+p}$

6:     for $j\leftarrow 1$ to $p$ do

7:         $T_{k+p}\leftarrow L^{-1}T_{k+p}L$ $\triangleright$ Implicitly shifted LR with shift $\mu_{j}$

8:         $L_{r}\leftarrow LL_{r}$

9:     end for

10:end procedure

11:procedure Refine left subspace

12:     $L_{l}\leftarrow I_{k+p}$

13:     for $j\leftarrow 1$ to $p$ do

14:         $T^{H}_{k+p}\leftarrow L^{-1}T^{H}_{k+p}L$ $\triangleright$ Implicitly shifted LR with shift $\bar{\mu}_{j}$

15:         $L_{l}\leftarrow LL_{l}$

16:     end for

17:end procedure

18:procedure Update

19:     $\beta\leftarrow e^{T}_{k+1}T_{k+p}e_{k}$;       $\delta\leftarrow e^{T}_{k}T_{k+p}e_{k+1}$

20:     $\theta=e_{k+p}^{T}(L_{r}(L_{l})^{-H})e_{k}$;        $\phi=e_{k+p}^{T}(L_{r}^{-H}L_{l})e_{k}$

21:     $\widetilde{v}_{k+1}\leftarrow\beta V_{k+p}e_{k+1}+\theta\widetilde{v}_{k+p+1}$ $\triangleright$ New right residual

22:     $\widetilde{w}_{k+1}\leftarrow\bar{\delta}W_{k+p}e_{k+1}+\phi\widetilde{w}_{k+p+1}$ $\triangleright$ New left residual

23:     $V_{k}\leftarrow V_{k+p}(L_{r}L_{l}^{-H})\begin{pmatrix}I_{k}\\ 0_{p}\end{pmatrix}$ $\triangleright$ New right Krylov space

24:     $W_{k}\leftarrow W_{k+p}(L_{r}^{-H}L_{l})\begin{pmatrix}I_{k}\\ 0_{p}\end{pmatrix}$ $\triangleright$ New left Krylov space

25:     $T_{k}\leftarrow\begin{pmatrix}I_{k}&0_{p}\end{pmatrix}T_{k+p}\begin{pmatrix}I_{k}\\ 0_{p}\end{pmatrix}$ $\triangleright$ New tridiagonal matrix

26:end procedure

27:return $V_{k},\widetilde{v}_{k+1},W_{k},\widetilde{w}_{k+1},T_{k}$

3 Computational Results

We present the results of computational tests performed in the Grcar matrix which is highly non-normal and has presented problems with convergence in previous studies. In his implicit restart work Sorensen [18] indeed points out that the restarted Arnoldi has trouble converging to the left-most part of the spectrum and even though the algorithm claimed convergence, what was obtained was in fact part of the pseudospectrum.

We present the results of the restarted Lanczos method applied to the $50\times 50$ Grcar matrix for $k=10$ and $p=10$. Figure 2 shows the spectrum obtained after performing eleven restarts of the Lanczos algorithm. The restarts were performed such that the $k$ eigenvalues with the largest imaginary part were retained for the reduced operator and the remaining were used as shifts to be discarded from the subspace. The same part of the spectrum was sought by Sorensen in [18]. In this particular case we obtained $\widetilde{w}_{m+1}^{T}\widetilde{v}_{m+1}\sim O(10^{-11})$ even though the individual residuals were both of order $10^{-3}-10^{-4}$. At this particular moment, one would need to employ a look-ahead step of the Lanczos however, we have shown the spectrum of the reduced matrix at this point. One may think of the spectrum to have converged to $O(10^{-4})$. As seen from the figure, the spectrum of the reduced operator matches quite well with the original spectrum of the Grcar matrix. In figure 2, we show the convergence history of the individual eigenvalues with each restart step as the Lanczos algorithm progresses. Clearly the restart process is working well to shift the Krylov spaces towards the wanted region of the spectrum. The error in the eigenvalues is of $O(10^{-7})$ even though the perturbation to the reduced matrix is of order $10^{-4}$. This is in contrast to the results obtained in [18] which reported large perturbations to the eigenvalues even when the Arnoldi method reported convergence. We suspect this is due to the fact that the non-symmetric Lanczos approximates both the right and left eigenspaces simultaneously and leads to a lower error in the truncated matrix. We expect this to be particularly useful in hydrodynamic problems where highly non-normal matrices are a routine occurrence and where problems with convergence of the spectrum have often been reported (see [2] for example).

Figure 1: Spectrum obtained from the restarted Lanczos after $11$ restart steps.

Figure 2: Error in the eigenvalues after each restart $i$.

At this point we make a note of caution that for large Grcar matrices our attempts have been somewhat less successful and the convergence sometimes fails. We expect this is due to the naive implementations of the implicit LR method that we have done in Julia where, the standard checks for small sub-diagonal elements have not been performed. In our investigations of such cases we indeed do find small sub diagonal elements which lead to loss of precision. Very small sub diagonal elements also lead to rapid loss of biorthogonality of the two subspaces. Similarly we have not paid attention to the issue arising out of the Lanczos algorithm itself, except for employing a double (two-sided) Gram-Schmidt to ensure biorthogonality. We also note that for large number of restarts, the biorthogonality property of the two subspaces is progressively lost, probably due to accruing floating point errors. Hence we expect some method of reorthogonalization would be required to ensure stability for very long calculations, which has not been done in the current work. The (implicit) LR decomposition is not unique. Uniqueness is ensured for unit main diagonal of the lower triangular matrix however, this does not pay any heed to conditioning of the transforming matrices. A better strategy of building the matrices could be pursued which has better conditioning while at the same time preserves the tridiagonal structure. These issues would require careful implementation of all the individual components and we do not address those in the current work.

Finally we note that very similar work has been reported in [7] where hyperbolic transforms (HR) are used for restarts of the bi-Lanczos method, and in [6] where the LR transformations are used, albeit in the context of model reduction.

4 Conclusion

We present an algorithm to restart the non-symmetric Lanczos method which is based on the idea of polynomial filtering via the implicit QR method proposed by Sorensen [18] for restarting the Arnoldi iteration. The (implicitly) shifted LR method is shown to be the appropriate analogue for restarting the non-symmetric Lanczos algorithm for structure preserving restarts. It is shown that the polynomial filtering process needs to be carried out for both the right and the left Krylov spaces and the appropriate transforms for maintaining biorthogonality of the two spaces are highlighted. Computational results are shown for a Grcar matrix which is known to be highly non-normal and the spectrum is found to converge adequately even when the residual error is relatively large.