This page intentionally left blank

Practical Extrapolation Methods

An important problem that arises in many scientic and engineering applications is
that of approximating limits of innite sequences {Am }. In most cases, these sequences
converge very slowly. Thus, to approximate their limits with reasonable accuracy, one
must compute a large number of the terms of {Am }, and this is generally costly. These
limits can be approximated economically and with high accuracy by applying suitable
extrapolation (or convergence acceleration) methods to a small number of terms of {Am }.
This book is concerned with the coherent treatment, including derivation, analysis, and
applications, of the most useful scalar extrapolation methods. The methods it discusses
are geared toward problems that commonly arise in scientic and engineering disciplines. It differs from existing books on the subject in that it concentrates on the most
powerful nonlinear methods, presents in-depth treatments of them, and shows which
methods are most effective for different classes of practical nontrivial problems; it also
shows how to ne-tune these methods to obtain best numerical results.
This book is intended to serve as a state-of-the-art reference on the theory and practice of
extrapolation methods. It should be of interest to mathematicians interested in the theory
of the relevant methods and serve applied scientists and engineers as a practical guide
for applying speed-up methods in the solution of difcult computational problems.

Avram Sidi is Professor of Numerical Analysis in the Computer Science Department

at the TechnionIsrael Institute of Technology and holds the Technion Administration
Chair in Computer Science. He has published extensively in various areas of numerical
analysis and approximation theory and in journals such as Mathematics of Computation,
SIAM Review, SIAM Journal on Numerical Analysis, Journal of Approximation Theory, Journal of Computational and Applied Mathematics, Numerische Mathematik, and
Journal of Scientic Computing. Professor Sidis work has involved the development of
novel methods, their detailed mathematical analysis, design of efcient algorithms for
their implementation, and their application to difcult practical problems. His methods
and algorithms are successfully used in various scientic and engineering disciplines.

CAMBRIDGE MONOGRAPHS ON
APPLIED AND COMPUTATIONAL
MATHEMATICS
Series Editors
P. G. CIARLET, A. ISERLES, R. V. KOHN, M. H. WRIGHT

10

Practical Extrapolation Methods

The Cambridge Monographs on Applied and Computational Mathematics reect the

crucial role of mathematical and computational techniques in contemporary science.
The series presents expositions on all aspects of applicable and numerical mathematics,
with an emphasis on new developments in this fast-moving area of research.
State-of-the-art methods and algorithms as well as modern mathematical descriptions
of physical and mechanical ideas are presented in a manner suited to graduate research
students and professionals alike. Sound pedagogical presentation is a prerequisite. It is
intended that books in the series will serve to inform a new generation of researchers.

Also in this series:

A Practical Guide to Pseudospectral Methods, Bengt Fornberg
Dynamical Systems and Numerical Analysis, A. M. Stuart and A. R. Humphries
Level Set Methods and Fast Marching Methods, J. A. Sethian
The Numerical Solution of Integral Equations of the Second Kind,
Kendall E. Atkinson
Orthogonal Rational Functions, Adhemar Bultheel, Pablo Gonzalez-Vera,
Erik Hendriksen, and Olav Njastad
Theory of Composites, Graeme W. Milton
Geometry and Topology for Mesh Generation, Herbert Edelsbrunner
SchwarzChristoffel Mapping, Tobin A. Driscoll and Lloyd N. Trefethen

Practical Extrapolation Methods

Theory and Applications

AVRAM SIDI
TechnionIsrael Institute of Technology


Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, So Paulo
Cambridge University Press
The Edinburgh Building, Cambridge , United Kingdom
Published in the United States by Cambridge University Press, New York
www.cambridge.org
Information on this title: www.cambridge.org/9780521661591
Cambridge University Press 2003
This book is in copyright. Subject to statutory exception and to the provision of
relevant collective licensing agreements, no reproduction of any part may take place
without the written permission of Cambridge University Press.
First published in print format 2003
ISBN-13 978-0-511-06862-1 eBook (EBL)
ISBN-10 0-511-06862-X eBook (EBL)
ISBN-13 978-0-521-66159-1 hardback
ISBN-10 0-521-66159-5 hardback

Cambridge University Press has no responsibility for the persistence or accuracy of

s for external or third-party internet websites referred to in this book, and does not
guarantee that any content on such websites is, or will remain, accurate or appropriate.

Contents

Preface

page xix

Introduction
0.1 Why ExtrapolationConvergence Acceleration?
0.2 Antilimits Versus Limits
0.3 General Algebraic Properties of Extrapolation Methods
0.3.1 Linear Summability Methods and the
SilvermanToeplitz Theorem
0.4 Remarks on Algorithms for Extrapolation Methods
0.5 Remarks on Convergence and Stability of Extrapolation Methods
0.5.1 Remarks on Study of Convergence
0.5.2 Remarks on Study of Stability
0.5.3 Further Remarks
0.6 Remark on Iterated Forms of Extrapolation Methods
0.7 Relevant Issues in Extrapolation

1
1
4
5
7
8
10
10
10
13
14
15

I The Richardson Extrapolation Process and Its Generalizations

19

1 The Richardson Extrapolation Process

1.1 Introduction and Background
1.2 The Idea of Richardson Extrapolation
1.3 A Recursive Algorithm for the Richardson Extrapolation Process
1.4 Algebraic Properties of the Richardson Extrapolation Process
1.4.1 A Related Set of Polynomials
1.4.2 An Equivalent Alternative Denition of Richardson Extrapolation
1.5 Convergence Analysis of the Richardson Extrapolation Process
1.5.1 Convergence of Columns
1.5.2 Convergence of Diagonals
1.6 Stability Analysis of the Richardson Extrapolation Process
1.7 A Numerical Example: Richardson Extrapolation on
the Zeta Function Series
1.8 The Richardson Extrapolation as a Summability Method
1.8.1 Regularity of Column Sequences

21
21
27
28
29
29
31
33
33
34
37

vii

38
39
39

viii

Contents

1.8.2 Regularity of Diagonal Sequences

1.9 The Richardson Extrapolation Process for Innite Sequences

40
41

2 Additional Topics in Richardson Extrapolation

2.1 Richardson Extrapolation with Near Geometric and Harmonic {yl }
2.2 Polynomial Richardson Extrapolation
2.3 Application to Numerical Differentiation
2.4 Application to Numerical Quadrature: Romberg Integration
2.5 Rational Extrapolation

42
42
43
48
52
55

3 First Generalization of the Richardson Extrapolation Process

3.1 Introduction
3.2 Algebraic Properties
( j)
3.3 Recursive Algorithms for An
3.3.1 The FS-Algorithm
3.3.2 The E-Algorithm
3.4 Numerical Assessment of Stability
3.5 Analysis of Column Sequences
( j)
3.5.1 Convergence of the ni
( j)
3.5.2 Convergence and Stability of the An
3.5.3 Convergence of the k
3.5.4 Conditioning of the System (3.1.4)
3.5.5 Conditioning of (3.1.4) for the Richardson Extrapolation Process
on Diagonal Sequences
3.6 Further Results for Column Sequences
3.7 Further Remarks on (3.1.4): Related Convergence
Acceleration Methods
3.8 Epilogue: What Is the E-Algorithm? What Is It Not?

57
57
59
61
62
64
66
67
68
69
72
73

4 GREP: Further Generalization of the Richardson Extrapolation Process

4.1 The Set F(m)
4.2 Denition of the Extrapolation Method GREP
4.3 General Remarks on F(m) and GREP
4.4 A Convergence Theory for GREP
4.4.1 Study of Process I
4.4.2 Study of Process II
4.4.3 Further Remarks on Convergence Theory
4.4.4 Remarks on Convergence of the ki
4.4.5 Knowing the Minimal m Pays
4.5 Remarks on Stability of GREP
4.6 Extensions of GREP

81
81
85
87
88
89
90
91
92
92
92
93

95
95
96

The D-Transformation: A GREP for Innite-Range Integrals

5.1 The Class B(m) and Related Asymptotic Expansions
5.1.1 Description of the Class A( )

74
76
77
79

Contents

5.2
5.3
5.4

5.5
5.6
5.7

5.1.2 Description of the Class B(m)

5.1.3 Asymptotic Expansion of F(x) When f (x) B(m)
5.1.4 Remarks on the Asymptotic Expansion of F(x) and
a Simplication
Denition of the D (m) -Transformation
5.2.1 Kernel of the D (m) -Transformation
A Simplication of the D (m) -Transformation:
The s D (m) -Transformation
How to Determine m
5.4.1 By Trial and Error
5.4.2 Upper Bounds on m
Numerical Examples
Proof of Theorem 5.1.12
Characterization and Integral Properties of Functions in B(1)
5.7.1 Integral Properties of Functions in A( )
5.7.2 A Characterization Theorem for Functions in B(1)
5.7.3 Asymptotic Expansion of F(x) When f (x) B(1)

The d-Transformation: A GREP for Innite Series and Sequences

6.1 The Class b(m) and Related Asymptotic Expansions
( )
6.1.1 Description of the Class A0
6.1.2 Description of the Class b(m)
6.1.3 Asymptotic Expansion of An When {an } b(m)
6.1.4 Remarks on the Asymptotic Expansion of An and
a Simplication
6.2 Denition of the d (m) -Transformation
6.2.1 Kernel of the d (m) -Transformation
6.2.2 The d (m) -Transformation for Innite Sequences
6.2.3 The Factorial d (m) -Transformation
6.3 Special Cases with m = 1
6.3.1 The d (1) -Transformation
6.3.2 The Levin L-Transformation
6.3.3 The Sidi S-Transformation
6.4 How to Determine m
6.4.1 By Trial and Error
6.4.2 Upper Bounds on m
6.5 Numerical Examples
6.6 A Further Class of Sequences in b(m) : The Class b (m)
( ,m) and Its Summation Properties
6.6.1 The Function Class A
0
6.6.2 The Sequence Class b (m) and a Characterization Theorem
6.6.3 Asymptotic Expansion of An When {an } b (m)
6.6.4 The d (m) -Transformation
6.6.5 Does {an } b (m) Imply {an } b(m) ? A Heuristic Approach
6.7

Summary of Properties of Sequences in b(1)

ix
98
100
103
103
105
105
106
106
107
111
112
117
117
118
118
121
121
122
123
126
129
130
131
131
132
132
132
133
133
133
134
134
137
140
140
142
145
147
148
150

Contents
6.8

A General Family of Sequences in b(m)

6.8.1 Sums of Logarithmic Sequences
6.8.2 Sums of Linear Sequences
6.8.3 Mixed Sequences

152
153
155
157

Recursive Algorithms for GREP

7.1 Introduction
7.2 The W-Algorithm for GREP(1)
7.2.1 A Special Case: (y) = y r
7.3 The W(m) -Algorithm for GREP(m)
7.3.1 Ordering of the k (t)t i
7.3.2 Technical Preliminaries
7.3.3 Putting It All Together
7.3.4 Simplications for the Cases m = 1 and m = 2
7.4 Implementation of the d (m) -Transformation by the W(m) -Algorithm
7.5 The EW-Algorithm for an Extended GREP(1)

158
158
159
163
164
164
166
167
171
172
173

Analytic Study of GREP(1) : Slowly Varying A(y) F(1)

( j)
8.1 Introduction and Error Formula for An
8.2 Examples of Slowly Varying a(t)
8.3 Slowly Varying (t) with Arbitrary tl
8.4 Slowly Varying (t) with liml (tl+1 /tl ) = 1
8.4.1 Process I with (t) = t H (t) and Complex
8.4.2 Process II with (t) = t
8.4.3 Process II with (t) = t H (t) and Real
8.5 Slowly Varying (t) with liml (tl+1 /tl ) = (0, 1)
8.6 Slowly Varying (t) with Real and tl+1 /tl (0, 1)
8.6.1 The Case (t) = t
8.6.2 The Case (t) = t H (t) with Real and tl+1 /tl (0, 1)

176
176
180
181
185
185
189
190
193
196
196
199

Analytic Study of GREP(1) : Quickly Varying A(y) F(1)

9.1 Introduction
9.2 Examples of Quickly Varying a(t)
9.3 Analysis of Process I
9.4 Analysis of Process II
9.5 Can {tl } Satisfy (9.1.4) and (9.4.1) Simultaneously?

203
203
204
205
207
210

10 Efcient Use of GREP(1) : Applications to the D (1) -, d (1) -,

and d (m) -Transformations
10.1 Introduction
10.2 Slowly Varying a(t)
10.2.1 Treatment of the Choice liml (tl+1 /tl ) = 1
10.2.2 Treatment of the Choice 0 < tl+1 /tl < 1
10.3 Quickly Varying a(t)

212
212
212
212
215
216

Contents
11 Reduction of the D-Transformation for Oscillatory Innite-Range
D-,
W -, and mW -Transformations
Integrals: The D-,
11.1 Reduction of GREP for Oscillatory A(y)
11.1.1 Review of the W-Algorithm for Innite-Range Integrals

11.2 The D-Transformation

11.2.1 Direct Reduction of the D-Transformation
11.2.2 Reduction of the s D-Transformation

11.3 Application of the D-Transformation

to Fourier Transforms

11.4 Application of the D-Transformation to Hankel Transforms

11.5 The D-Transformation

11.6 Application of the D-Transformation

to Hankel Transforms

11.7 Application of the D-Transformation

to Integrals of Products of
Bessel Functions
11.8 The W - and mW -Transformations for Very Oscillatory Integrals

11.8.1 Description of the Class B

11.8.2 The W - and mW -Transformations
11.8.3 Application of the mW -Transformation to Fourier,
Inverse Laplace, and Hankel Transforms
11.8.4 Further Variants of the mW -Transformation
11.8.5 Further Applications of the mW -Transformation
11.9 Convergence and Stability
11.10 Extension to Products of Oscillatory Functions

xi

218
218
219
220
220
221
222
223
224
225
226
227
227
228
231
234
235
235
236

12 Acceleration of Convergence of Power Series by the d-Transformation:

Rational d-Approximants
12.1 Introduction
12.2 The d-Transformation on Power Series
12.3 Rational Approximations from the d-Transformation
12.3.1 Rational d-Approximants
12.3.2 Closed-Form Expressions for m = 1
12.4 Algebraic Properties of Rational d-Approximants
12.4.1 Pade-like Properties
12.4.2 Recursive Computation by the W(m) -Algorithm
12.5 Prediction Properties of Rational d-Approximants
12.6 Approximation of Singular Points
12.7 Efcient Application of the d-Transformation to Power Series with APS
12.8 The d-Transformation on Factorial Series
12.9 Numerical Examples

238
238
238
240
240
241
242
242
244
245
246
247
249
250

13 Acceleration of Convergence of Fourier and Generalized Fourier Series

by the d-Transformation: The Complex Series Approach with APS
13.1 Introduction
13.2 Introducing Functions of Second Kind
13.2.1 General Background
13.2.2 Complex Series Approach

253
253
254
254
254

xii

Contents
13.2.3 Justication of the Complex Series Approach and APS
13.3 Examples of Generalized Fourier Series
13.3.1 Chebyshev Series
13.3.2 Nonclassical Fourier Series
13.3.3 FourierLegendre Series
13.3.4 FourierBessel Series
13.4 Convergence and Stability when {bn } b(1)
13.5 Direct Approach
13.6 Extension of the Complex Series Approach
13.7 The H-Transformation

255
256
256
257
257
258
259
259
260
261

14 Special Topics in Richardson Extrapolation

14.1 Conuence in Richardson Extrapolation
14.1.1 The Extrapolation Process and the SGRom-Algorithm
14.1.2 Treatment of Column Sequences
14.1.3 Treatment of Diagonal Sequences
14.1.4 A Further Problem
14.2 Computation of Derivatives of Limits and Antilimits
14.2.1 Derivative of the Richardson Extrapolation Process
d
14.2.2 d
GREP(1) : Derivative of GREP(1)

263
263
263
265
266
267
268
269
274

II Sequence Transformations

277

15 The Euler Transformation, Aitken 2 -Process, and

Lubkin W -Transformation
15.1 Introduction
15.2 The EulerKnopp (E, q) Method
15.2.1 Derivation of the Method
15.2.2 Analytic Properties
15.2.3 A Recursive Algorithm
15.3 The Aitken 2 -Process
15.3.1 General Discussion of the 2 -Process
15.3.2 Iterated 2 -Process
15.3.3 Two Applications of the Iterated 2 -Process
15.3.4 A Modied 2 -Process for Logarithmic Sequences
15.4 The Lubkin W -Transformation
15.5 Stability of the Iterated 2 -Process and Lubkin Transformation
15.5.1 Stability of the Iterated 2 -Process
15.5.2 Stability of the Iterated Lubkin Transformation
15.6 Practical Remarks
15.7 Further Convergence Results

279
279
279
279
281
283
283
283
286
287
289
290
293
293
294
295
295

16 The Shanks Transformation

16.1 Derivation of the Shanks Transformation
16.2 Algorithms for the Shanks Transformation

297
297
301

Contents
16.3 Error Formulas


m
16.4 Analysis of Column Sequences When Am A +
k=1 k k
16.4.1 Extensions
16.4.2 Application to Numerical Quadrature
16.5 Analysis of Column Sequences When {Am } b(1)
16.5.1 Linear Sequences
16.5.2 Logarithmic Sequences
16.5.3 Factorial Sequences
16.6 The Shanks Transformation on Totally Monotonic and
Totally Oscillating Sequences
16.6.1 Totally Monotonic Sequences
16.6.2 The Shanks Transformation on Totally Monotonic Sequences
16.6.3 Totally Oscillating Sequences
16.6.4 The Shanks Transformation on Totally Oscillating Sequences
16.7 Modications of the -Algorithm

xiii
302
303
309
312
313
313
316
316
317
317
318
320
321
321

17 The Pade Table

17.1 Introduction
17.2 Algebraic Structure of the Pade Table
17.3 Pade Approximants for Some Hypergeometric Functions
17.4 Identities in the Pade Table
17.5 Computation of the Pade Table
17.6 Connection with Continued Fractions
17.6.1 Denition and Algebraic Properties of Continued Fractions
17.6.2 Regular C-Fractions and the Pade Table
17.7 Pade Approximants and Exponential Interpolation
17.8 Convergence of Pade Approximants from Meromorphic Functions
17.8.1 de Montessuss Theorem and Extensions
17.8.2 Generalized Koenigs Theorem and Extensions
17.9 Convergence of Pade Approximants from Moment Series
17.10 Convergence of Pade Approximants from Polya Frequency Series
17.11 Convergence of Pade Approximants from Entire Functions

323
323
325
327
329
330
332
332
333
336
338
338
341
342
345
346

18 Generalizations of Pade Approximants

18.1 Introduction
18.2 Pade-Type Approximants
18.3 Vanden BroeckSchwartz Approximations
18.4 Multipoint Pade Approximants
18.4.1 Two-Point Pade Approximants
18.5 HermitePade Approximants
18.5.1 Algebraic HermitePade Approximants
18.5.2 Differential HermitePade Approximants
18.6 Pade Approximants from Orthogonal Polynomial Expansions
18.6.1 Linear (Cross-Multiplied) Approximations
18.6.2 Nonlinear (Properly Expanded) Approximations

348
348
348
350
351
353
354
355
355
356
356
358

xiv

Contents

18.7 BakerGammel Approximants

18.8 PadeBorel Approximants

360
362

19 The Levin L- and Sidi S-Transformations

19.1 Introduction
19.2 The Levin L-Transformation
19.2.1 Derivation of the L-Transformation
19.2.2 Algebraic Properties
19.2.3 Convergence and Stability
19.3 The Sidi S-Transformation
19.4 A Note on Factorially Divergent Sequences

363
363
363
363
365
367
369
371

20 The Wynn - and Brezinski -Algorithms

20.1 The Wynn -Algorithm and Generalizations
20.1.1 The Wynn -Algorithm
20.1.2 Modications of the -Algorithm
20.2 The Brezinski -Algorithm
20.2.1 Convergence and Stability of the -Algorithm
20.2.2 A Further Convergence Result

375
375
375
376
379
380
383

21 The G-Transformation and Its Generalizations

21.1 The G-Transformation
21.2 The Higher-Order G-Transformation
21.3 Algorithms for the Higher-Order G-Transformation
21.3.1 The rs-Algorithm
21.3.2 The FS/qd-Algorithm
21.3.3 Operation Counts of the rs- and FS/qd-Algorithms

384
384
385
386
386
387
389

22 The Transformations of Overholt and Wimp

22.1 The Transformation of Overholt
22.2 The Transformation of Wimp
22.3 Convergence and Stability
22.3.1 Analysis of Overholts Method
22.3.2 Analysis of Wimps Method

390
390
391
392
392
394

23 Conuent Transformations
23.1 Conuent Forms of Extrapolation Processes
23.1.1 Derivation of Conuent Forms
23.1.2 Convergence Analysis of a Special Case
23.2 Conuent Forms of Sequence Transformations
23.2.1 Conuent -Algorithm
23.2.2 Conuent Form of the Higher-Order G-Transformation
23.2.3 Conuent -Algorithm
23.2.4 Conuent Overholt Method

396
396
396
399
401
401
402
403
404

Contents
23.3 Conuent D (m) -Transformation
23.3.1 Application to the D (1) -Transformation and Fourier Integrals

xv
405
405

24 Formal Theory of Sequence Transformations

24.1 Introduction
24.2 Regularity and Acceleration
24.2.1 Linearly Convergent Sequences
24.2.2 Logarithmically Convergent Sequences
24.3 Concluding Remarks

407
407
408
409
410
411

III Further Applications

413

25 Further Applications of Extrapolation Methods and

Sequence Transformations
25.1 Extrapolation Methods in Multidimensional Numerical Quadrature
25.1.1 By GREP and d-Transformation
25.1.2 Use of Variable Transformations
25.2 Extrapolation of Numerical Solutions of Ordinary Differential Equations
25.3 Romberg-Type Quadrature Formulas for Periodic Singular and
Weakly Singular Fredholm Integral Equations
25.3.1 Description of Periodic Integral Equations
25.3.2 Corrected Quadrature Formulas
25.3.3 Extrapolated Quadrature Formulas
25.3.4 Further Developments
25.4 Derivation of Numerical Schemes for Time-Dependent Problems
from Rational Approximations
25.5 Derivation of Numerical Quadrature Formulas from
Sequence Transformations
25.6 Computation of Integral Transforms with Oscillatory Kernels
25.6.1 Via Numerical Quadrature Followed by Extrapolation
25.6.2 Via Extrapolation Followed by Numerical Quadrature
25.6.3 Via Extrapolation in a Parameter
25.7 Computation of Inverse Laplace Transforms
25.7.1 Inversion by Extrapolation of the Bromwich Integral
25.7.2 Gaussian-Type Quadrature Formulas for the Bromwich Integral
25.7.3 Inversion via Rational Approximations
25.7.4 Inversion via the Discrete Fourier Transform
25.8 Simple Problems Associated with {an } b(1)
25.9 Acceleration of Convergence of Innite Series with Special Sign Patterns
25.9.1 Extensions
25.10 Acceleration of Convergence of Rearrangements of Innite Series
25.10.1 Extensions
25.11 Acceleration of Convergence of Innite Products
25.11.1 Extensions

415
415
415
418
421
422
422
423
425
427
428
430
433
433
434
435
437
437
437
439
440
441
442
443
443
444
444
445

xvi

Contents

25.12 Computation of Innite Multiple Integrals and Series

25.12.1 Sequential D-Transformation for s-D Integrals
25.12.2 Sequential d-Transformation for s-D Series
25.13 A Hybrid Method: The RichardsonShanks Transformation
25.13.1 An Application
25.14 Application of Extrapolation Methods to Ill-Posed Problems
25.15 Logarithmically Convergent Fixed-Point Iteration Sequences

446
447
447
449
450
453
454

IV Appendices

457

Review of Basic Asymptotics

A.1 The O, o, and Symbols
A.2 Asymptotic Expansions
A.3 Operations with Asymptotic Expansions

459
459
460
461

The Laplace Transform and Watsons Lemma

B.1 The Laplace Transform
B.2 Watsons Lemma

463
463
463

C The Gamma Function

465

D Bernoulli Numbers and Polynomials and the EulerMaclaurin Formula

D.1 Bernoulli Numbers and Polynomials
D.2 The EulerMaclaurin Formula
D.2.1 The EulerMaclaurin Formula for Sums
D.2.2 The EulerMaclaurin Formula for Integrals
D.3 Applications of EulerMaclaurin Expansions
D.3.1 Application to Harmonic Numbers
D.3.2 Application to Cauchy Principal Value Integrals
D.4 A Further Development
D.5 The EulerMaclaurin Formula for Integrals with Endpoint Singularities
D.5.1 Algebraic Singularity at One Endpoint
D.5.2 Algebraic-Logarithmic Singularity at One Endpoint
D.5.3 Algebraic-Logarithmic Singularities at Both Endpoints
D.6 Application to Singular Periodic Integrands

467
467
468
468
469
470
470
471
471
472
472
473
474
475

E The Riemann Zeta Function and the Generalized Zeta Function

E.1 Some Properties of (z)


z
E.2 Asymptotic Expansion of n1
k=0 (k + )

477
477
477

F Some Highlights of Polynomial Approximation Theory

F.1 Best Polynomial Approximations
F.2 Chebyshev Polynomials and Expansions

480
480
480

G A Compendium of Sequence Transformations

483

Contents

xvii

H Efcient Application of Sequence Transformations: Summary

488

I FORTRAN 77 Program for the d (m) -Transformation

I.1 General Description
I.2 The Code

493
493
494

Bibliography

501

Index

515

Preface

An important problem that arises in many scientic and engineering applications is that
of nding or approximating limits of innite sequences {Am }. The elements Am of such
sequences can show up in the form of partial sums of innite series, approximations from
xed-point iterations of linear and nonlinear systems of equations, numerical quadrature
approximations to nite- or innite-range integrals, whether simple or multiple, etc. In
most applications, these sequences converge very slowly, and this makes their direct
use to approximate limits an expensive proposition. There are important applications in
which they may even diverge. In such cases, the direct use of the Am to approximate
their so-called antilimits would be impossible. (Antilimits can be interpreted in appropriate ways depending on the nature of {Am }. In some cases they correspond to analytic
continuation in some parameter, for example.)
An effective remedy for these problems is via application of extrapolation methods
(or convergence acceleration methods) to the given sequences. (In the context of innite
sequences, extrapolation methods are also referred to as sequence transformations.)
Loosely speaking, an extrapolation method takes a nite and hopefully small number of
the Am and processes them in some way. A good method is generally nonlinear in the
Am and takes into account, either explicitly or implicitly, their asymptotic behavior as
m in a clever fashion.
The importance of extrapolation methods as effective computational tools has long
been recognized. Indeed, the Richardson extrapolation and the Aitken 2 -process, two
popular representatives, are discussed in some detail in almost all modern textbooks on
numerical analysis, and Pade approximants have become an integral part of approximation theory. During the last thirty years a few books were written on the subject and
various comparative studies were done in relation to some important subclasses of convergence acceleration problems, pointing to the most effective methods. Finally, since
the 1970s, international conferences partly dedicated to extrapolation methods have been
held on a regular basis.
The main purpose of this book is to present a unied account of the existing literature
on nonlinear extrapolation methods for scalar sequences that is as comprehensive and
up-to-date as possible. In this account, I include much of the literature that deals with
methods of practical importance whose effectiveness has been amply veried in various
surveys and comparative studies. Inevitably, the contents reect my personal interests
and taste. Therefore, I apologize to those colleagues whose work has not been covered.
xix

xx

Preface

I have left out completely the important subject of extrapolation methods for vector
sequences, even though I have been actively involved in this subject for the last twenty
years. I regret this, especially in view of the fact that vector extrapolation methods have
had numerous successful applications in the solution of large-scale nonlinear as well as
linear problems. I believe only a fully dedicated book would do justice to them.
The Introduction gives an overview of convergence acceleration within the framework of innite sequences. It includes a discussion of the concept of antilimit through
examples. Following that, it discusses, both in general terms and by example, the development of methods, their analysis, and accompanying algorithms. It also gives a detailed
discussion of stability in extrapolation. A proper understanding of this subject is very
helpful in devising effective strategies for extrapolation methods in situations of inherent
instability. The reader is advised to study this part of the Introduction carefully.
Following the Introduction, the book is divided into three main parts:
(i) Part I deals with the Richardson extrapolation and its generalizations. It also prepares
some of the background material and techniques relevant to Part II. Chapters 1 and
2 give a complete treatment of the Richardson extrapolation process (REP) that
is only partly described in previous works. Following that, Chapter 3 gives a rst
generalization of REP with some amount of theory. The rest of Part I is devoted
to the generalized Richardson extrapolation process (GREP) of Sidi, the Levin
Sidi D-transformation for innite-range integrals, the LevinSidi d-transformation
for innite series, Sidis variants of the D-transformation for oscillatory inniterange integrals, the SidiLevin rational d-approximants, and efcient summation
of power series and (generalized) Fourier series by the d-transformation. (Two
important topics covered in connection with these transformations are the class
of functions denoted B(m) and the class of sequences denoted b(m) . Both of these
classes are more comprehensive than those considered in other works.) Efcient
implementation of GREP is the subject of a chapter that includes the W-algorithm
of Sidi and the W(m) -algorithm of Ford and Sidi. Also, there are two chapters that
provide a detailed convergence and stability analysis of GREP(1) , a prototype of
GREP, whose results point to effective strategies for applying these methods in
different situations. The strategies denoted arithmetic progression sampling (APS)
and geometric progression sampling (GPS) are especially useful in this connection.
(ii) Part II is devoted to development and analysis of a number of effective sequence
transformations. I begin with three classical methods that are important historically
and that have been quite successful in many problems: the Euler transformation (the
only linear method included in this book), the Aitken 2 -process, and the Lubkin
transformation. I next give an extended treatment of the Shanks transformation along
with Pade approximants and their generalizations, continued fractions, and the qdalgorithm. Finally, I treat other important methods such as the G-transformation, the
Wynn -algorithm and its modications, the Brezinski -algorithm, the Levin Land Sidi S-transformations, and the methods of Overholt and of Wimp. I also give
the conuent forms of some of these methods. In this treatment, I include known
results on the application of these transformations to so-called linear and logarithmic
sequences and quite a few new results, including some pertaining to stability. I use

Preface

xxi

the latter to draw conclusions on how to apply sequence transformations more

effectively in different situations. In this respect APS of Part I turns out to be an
effective strategy when applying the methods of Part II to linear sequences.
(iii) Part III comprises a single chapter that provides a number of applications to problems
in numerical analysis that are not considered in Parts I and II.
Following these, I have also included as Part IV a sequence of appendices that provide
a lot of useful information recalled throughout the book. These appendices cover briey
several important subjects, such as asymptotic expansions, EulerMaclaurin expansions,
the Riemann Zeta function, and polynomial approximation theory, to name a few. In
particular, Appendix D puts together, for the rst time, quite a few EulerMaclaurin
expansions of importance in applications. Appendices G and H include a summary of the
extrapolation methods covered in the book and important tips on when and how to apply
them. Appendix I contains a FORTRAN 77 code that implements the d-transformation
and that is applied to a number of nontrivial examples. The user can run this code as is.
The reader may be wondering why I separated the topic of REP and its generalizations,
especially GREP, from that of sequence transformations and treated it rst. There are
two major reasons for this decision. First, the former is more general, covers more cases
of interest, and allows for more exibility in a natural way. For most practical purposes,
the methods of Part I have a larger scope and achieve higher accuracy than the sequence
transformations studied in Part II. Next, some of the conclusions from the theory of
convergence and stability of extrapolation methods of Part I turn out to be valid for the
sequence transformations of Part II and help to improve their performance substantially.
Now, the treatment of REP and its generalizations concerns the problem of nding
the limit (or antilimit) as y 0+ of a function A(y) known for y (0, b], b > 0. Here
y may be a continuous or discrete variable. Such functions arise in many applications in
a natural way. The analysis provided for them suggests to which problems they should
be applied, how they should be applied for best possible outcome in nite-precision
arithmetic, and what type of convergence and accuracy one should expect. All this also
has an impact on how sequence transformations should be applied to innite sequences
{Am } that are not directly related to a function A(y). In such a case, we can always
draw the analogy A(y) Am for some function A(y) with y m 1 , and view the
relevant sequence transformations as extrapolation methods. When treated this way, it
becomes easier to see that APS improves the performance of these transformations on
linear sequences.
The present book differs from already existing works in various ways:
1. The classes of (scalar) sequences treated in it include many of those that arise in
scientic and engineering applications. They are more comprehensive than those
considered in previous books and include the latter as well.
2. Divergent sequences (such as those that arise from divergent innite series or integrals, for example) are treated on an equal footing with convergent ones. Suitable
interpretations of their antilimits are provided. It is shown rigorously, at least in some
cases, that extrapolation methods can be applied to such sequences, with no changes,
to produce excellent approximations to their antilimits.

xxii

Preface

3. A thorough asymptotic analysis of tails of innite series and innite-range integrals

is given. Based on this analysis, detailed convergence studies for the different methods
are provided. Considerable effort is made to obtain the best possible results under the
smallest number of realistic conditions. These results either come in the form of
bona de asymptotic expansions of the errors or they provide tight upper bounds on
errors. They are accompanied by complete proofs in most cases. The proofs chosen
for presentation are generally those that can be applied in more than one situation and
hence deserve special attention. When proofs are not provided, the reader is referred
to the relevant papers and books.
4. The stability issue is formalized and given full attention for the rst time. Conclusions
are drawn from stability analyses to devise effective strategies that enable one to
obtain the best possible accuracy in nite-precision arithmetic, also in situations
where instabilities are built in. Immediate applications of this are to the summation
of so-called very oscillatory innite-range integrals, logarithmically convergent (or
divergent) series, power series, Fourier series, etc., where most methods have not
done well in the past.
I hope the book will serve as a reference for researchers in the area of extrapolation
methods and for scientists and engineers in different computational disciplines and as a
textbook for students interested in the subject. I have kept the mathematical background
needed to cope with the material to a minimum. Most of what the reader needs and
is not covered by the standard academic curricula is summarized in the appendices. I
would like to emphasize here that the subject of asymptotics is of utmost importance to
any treatment of extrapolation methods. It would be impossible to appreciate the beauty
of the subject of extrapolation the development of the methods and their mathematical analyses without it. It is certainly impossible to produce any honest theory of
extrapolation methods without it. I urge the reader to acquire a good understanding of
asymptotics before everything else. The essentials of this subject are briey discussed
in Appendix A.
I wish to express my gratitude to my friends and colleagues David Levin, William F.
Ford, Doron S. Lubinsky, Moshe Israeli, and Marius Ungarish for the interest they took
in this book and for their criticisms and comments in different stages of the writing. I
am also indebted to Yvonne Sagie and Hadas Heier for their expert typing of part of the
book.
Lastly, I owe a debt of gratitude to my wife Carmella for her constant encouragement
and support during the last six years while this book was being written. Without her
understanding of, and endless patience for, my long absences from home this book
would not have seen daylight. I dedicate this book to her with love.
Avram Sidi
Technion, Haifa
December 2001

Introduction

0.1 Why ExtrapolationConvergence Acceleration?

In many problems of scientic computing, one is faced with the task of nding or approximating limits of innite sequences. Such sequences may arise in different disciplines
and contexts and in various ways. In most cases of practical interest, the sequences
in question converge to their limits very slowly. This may cause their direct use for
approximating their limits to become computationally expensive or impossible.
There are other cases in which these sequences may even diverge. In such a case, we
are left with the question of whether the divergent sequence represents anything, and if
so, what it represents. Although in some cases the elements of a divergent sequence can
be used as approximations to the quantity it represents subject to certain conditions, in
most other cases it is meaningless to make direct use of the sequence elements for this
purpose.
Let us consider two very common examples:
(i) Summation of innite series: This is a problem that arises in many scientic disciplines, such as applied mathematics, theoretical physics, and theoretical chemistry. In
this problem, the sequences in question are those of partial sums. In some cases, the terms


ak of a series
k=0 ak may be known analytically. In other cases, these terms may be
generated numerically, but the process of generating more and more terms may become
very costly. In both situations, if the series converges very slowly, the task of obtaining


good approximations to its sum only from its partial sums An = nk=0 ak , n = 0, 1, . . . ,
may thus become very expensive as it necessitates a very large number of the terms ak .
In yet other cases, only a nite number of the terms ak , say a0 , a1 , . . . , a N , may be
known. In such a situation, the accuracy of the best available approximation to the sum


of
k=0 ak is normally that of the partial sum A N and thus cannot be improved further.
If the series diverges, then its partial sums have only limited direct use. Divergent series
arise naturally in different elds, perturbation analysis in theoretical physics being one of
them. Divergent power series arise in the solution of homogeneous ordinary differential
equations around irregular singular points.
(ii) Iterative solution of linear and nonlinear systems of equations: This problem
occurs very commonly in applied mathematics and different branches of engineering.
When continuum problems are solved by methods such as nite differences and nite
elements, large and sparse systems of linear and/or nonlinear equations are obtained. A

Introduction

very attractive way of solving these systems is by iterative methods. The sequences in
question for this case are those of the iteration vectors that have a large dimension in
general. In most cases, these sequences converge very slowly. If the cost of computing
one iteration vector is very high, then obtaining a good approximation to the solution of
a given system of equations may also become very high.
The problems of slow convergence or even divergence of sequences can be overcome
under suitable conditions by applying extrapolation methods (equivalently, convergence
acceleration methods or sequence transformations) to the given sequences. When appropriate, an extrapolation method produces from a given sequence {An } a new sequence
{ A n } that converges to the formers limit more quickly when this limit exists. In case
the limit of {An } does not exist, the new sequence { A n } produced by the extrapolation
method either diverges more slowly than {An } or converges to some quantity called the
antilimit of {An } that has a useful meaning and interpretation in most applications. We
note at this point that the precise meaning of the antilimit may vary depending on the
type of the divergent sequence, and that several possibilities exist. In the next section,
we shall demonstrate through examples how antilimits may arise and what exactly they
may be.
Concerning divergent sequences, there are three important messages that we would
like to get across in this book: (i) Divergent sequences can be interpreted appropriately in
many cases of interest, and useful antilimits for them can be dened. (ii) Extrapolation
methods can be used to produce good approximations to the relevant antilimits in an
efcient manner. (iii) Divergent sequences can be treated on an equal footing with convergent ones, both computationally and theoretically, and this is what we do throughout
this book. (However, everywhere-divergent innite power series, that is, those with zero
radius of convergence, are not included in the theoretical treatment generally.)
It must be emphasized that each A n is determined from only a nite number of the
Am . This is a basic requirement that extrapolation methods must satisfy. Obviously, an
extrapolation method that requires knowledge of all the Am for determining a given A n
is of no practical value.
We now pause to illustrate the somewhat abstract discussion presented above with
the Aitken 2 -process that is one of the classic examples of extrapolation methods.
This method was rst described in Aitken [2], and it can be found in almost every book
on numerical analysis. See, for example, Henrici [130], Ralston and Rabinowitz [235],
Stoer and Bulirsch [326], and Atkinson [13].
Example 0.1.1 Let the sequence {An } be such that
An = A + an + rn with rn = bn + o(min{1, ||n }) as n ,

(0.1.1)

where A, a, b, , and are in general complex scalars, and

a, b = 0, , = 0, 1, and || > ||.

(0.1.2)

As a result, rn bn = o(n ) as n . If || < 1, then limn An = A. If || 1,

then limn An does not exist, A being the antilimit of {An } in this case. Consider now
the Aitken 2 -process, which is an extrapolation method that, when applied to {An },

0.1 Why ExtrapolationConvergence Acceleration?

produces a sequence { A n } with

A n =

An An+2 A2n+1
An 2An+1 + An+2



 An An+1 


 An An+1 
=
,
 1
1 

 An An+1 

(0.1.3)

where Am = Am+1 Am , m 0. To see how A n behaves for n , we substitute

(0.1.1) in (0.1.3). Taking into account the fact that rn+1 rn as n , after some
simple algebra it can be shown that
| A n A| |rn | = O(n ) = o(n ) as n ,

(0.1.4)

for some positive constant that is independent of n. Obviously, when limn rn = 0,

the sequence { A n } converges to A whether {An } converges or not. [If rn = 0 for n N ,
then A n = A for n N as well, as implied by (0.1.4). In fact, the formula for A n in (0.1.3)
is obtained by requiring that A n = A when rn = 0 for all large n, and it is the solution for
A of the equations Am = A + am , m = n, n + 1, n + 2.] Also, in case {An } converges,
{ A n } converges more quickly and to limn An = A, because An A an as n
from (0.1.1) and (0.1.2). Thus, the rate of convergence of {An } is enhanced by the factor
| A n A|
= O(|/|n ) = o(1) as n .
|An A|

(0.1.5)

A more detailed analysis of A n A yields the result

( ) n
A n A b
as n ,
( 1)2
2

(0.1.6)

that is more rened than (0.1.4) and asymptotically best possible as well. It is clear
from (0.1.6) that, when the sequence {rn } does not converge to 0, which happens when
|| 1, both {An } and { A n } diverge, but { A n } diverges more slowly than {An }.
In view of this example and the discussion that preceded it, we now introduce the
concepts of convergence acceleration and acceleration factor.
Denition 0.1.2 Let {An } be a sequence of in general complex scalars, and let { A n } be
the sequence generated by applying the extrapolation method ExtM to {An }, A n being
determined from Am , 0 m L n , for some integer L n , n = 0, 1, . . . . Assume that
limn A n = A for some A and that, if limn An exists, it is equal to this A. We shall
say that { A n } converges more quickly than {An } if
| A n A|
= 0,
n |A L A|
n
lim

(0.1.7)

whether limn An exists or not. When (0.1.7) holds we shall also say that the
extrapolation method ExtM accelerates the convergence of {An }. The ratio Rn =
| A n A|/|A L n A| is called the acceleration factor of A n .

Introduction

The ratios Rn measure the extent of the acceleration induced by the extrapolation
method ExtM on {An }. Indeed, from | A n A| = Rn |A L n A|, it is obvious that Rn
is the factor by which the acceleration process reduces |A L n A| in generating A n .
Obviously, a good extrapolation method is one whose acceleration factors tend to zero
quickly as n .
In case {An } is a sequence of vectors in some general vector space, the preceding
denition is still valid, provided we replace |A L n A| and | A n A| everywhere with
 A L n A  and  A n A , respectively, where   is the norm in the vector space
under consideration.

0.2 Antilimits Versus Limits

Before going on, we would like to dwell on the concept of antilimit that we mentioned
briey above. This concept can best be explained by examples to which we now turn.
These examples do not exhaust all the possibilities for antilimits by any means. We shall
encounter more later in this book.
Example 0.2.1 Let An , n = 0, 1, 2, . . . , be the partial sums of the power series

n


k
k
k=0 ak z , that is, An =
k=0 ak z , n = 0, 1, . . . . If the radius of convergence
of this series is nite and positive, then limn An exists for |z| < and is a function


k
f (z) that is analytic for |z| < . Of course,
k=0 ak z diverges for |z| > . If f (z) can
be continued analytically to |z| = and |z| > , then the analytic continuation of f (z)
is the antilimit of {An } for |z| .
As an illustration, let us pick a0 = 0 and ak = 1/k, k = 1, 2, . . . , so that = 1
and limn An = log(1 z) for |z| 1, z = 1. The principal branch of log(1 z) that
is analytic for all complex z  [1, +) serves as the antilimit of {An } in case |z| > 1
but z  [1, +).
Example 0.2.2 Let An , n = 0, 1, 2, . . . , be the partial sums of the Fourier series

n


ikx
ikx

k= ak e ; that is, An =
k=n ak e , n = 0, 1, 2, . . . , and assume that C 1 |k|

|ak | C2 |k| for all large |k| and some positive constants C1 and C2 and for some 0,
so that limn An does not exist. This Fourier series represents a 2 -periodic generalized function; see Lighthill [167]. If, for x in some interval I of [0, 2 ], this generalized
function coincides with an ordinary function f (x), then f (x) is the antilimit of {An } for
x I . (Recall that limn An , in general, exists when < 0 and an is monotonic in n.
It exists unconditionally when < 1.)
As an illustration, let us pick a0 = 0 and ak = 1, k = 1, 2, . . . . Then the se



ikx
ries
represents the generalized function 1 + 2
k= ak e
m= (x 2m ),
where (z) is the Dirac delta function. This generalized function coincides with the ordinary function f (x) = 1 in the interval (0, 2), and f (x) serves as the antilimit of
{An } for n when x (0, 2).
Example 0.2.3 Let 0 < x0 < x1 < x2 < , limn xn = , s = 0 and real, and let
x
An be dened as An = 0 n g(t)eist dt, n = 0, 1, 2, . . . , where C1 t |g(t)| C2 t
for all large t and some positive constants C1 and C2 and for some 0, so that

0.3 General Algebraic Properties of Extrapolation Methods

limn An does not exist.

such cases, the antilimit of {An } is the Abel sum of
 In many
ist
g(t)e
dt
(see, e.g., Hardy [123]) that is dened by
the divergent
integral
0
 t

ist
ist
lim
e
g(t)e
dt.
[Recall
that
0 g(t)e dt exists and limn An =
 0+ ist0
0 g(t)e dt, in general, when < 0 and g(t) is monotonic in t for large t. This is
true unconditionally when < 1.]
an illustration, let us pick g(t) = t 1/2 . Then the Abel sum of the divergent integral
 As1/2

eist dt is ei3/4 /(2s 3/2 ), and it serves as the antilimit of {An }.

0 t
in (0, 1) satisfying h 0 > h 1 > h 2 > , and
Example 0.2.4 Let {h n } be a sequence
1
limn h n = 0, and dene An = h n x g(x) d x, n = 0, 1, 2, . . . , where g(x) is continuously differentiable on [0, 1] a sufcient number of times with g(0) = 0 and
is in general complex and  1 but = 1, 2, . . . . Under these conditions
limn An does not exist.
 1The antilimit of {An } in this case is the Hadamard nite part
of the divergent integral 0 x g(x) d x (see Davis and Rabinowitz [63]) that is given by
the expression
 1 
m1
m1

 g (i) (0) 
g (i) (0)
1
+
xi dx
x g(x)

+
i
+
1
i!
i!
0
i=0
i=0
with m > 
 1 exists as an ordinary integral.
 1 1 so that the integral in this expression
[Recall that 0 x g(x) d x exists and limn An = 0 x g(x) d x for  > 1.]
As an illustration,
g(x) = (1 + x)1 and = 3/2. Then the Hadamard
 1 3/2let us pick
1
(1 + x) d x is 2 /2, and it serves as the antilimit of {An }.
nite part of 0 x
Note that limn An = + but the associated antilimit is negative.
Example 0.2.5 Let s be the solution to the nonsingular linear system of equations
(I T )x = c, and let {xn } be dened by the iterative scheme xn+1 = T xn + c, n =
0, 1, 2, . . . , with x0 given. Let (T ) denote the spectral radius of T . If (T ) > 1, then
{xn } diverges in general. The antilimit of {xn } in this case is the solution s itself. [Recall
that limn xn exists and is equal to s when (T ) < 1.]
As should become clear from these examples, the antilimit may have different meanings depending on the nature of the sequence {An }. Thus, it does not seem to be possible
to dene antilimits in a unique way, and we do not attempt to do this. It appears, though,
that studying the asymptotic behavior of An for n is very helpful in determining
the meaning of the relevant antilimit. We hope that what the antilimit of a given divergent sequence is will become more apparent as we proceed to the study of extrapolation
methods.

0.3 General Algebraic Properties of Extrapolation Methods

We saw in Section 0.1 that an extrapolation method operates on a given sequence {An }
to produce a new sequence { A n }. That is, it acts as a mapping from {An } to { A n }. In all
cases of interest, this mapping has the general form
A n = n (A0 , A1 , . . . , A L n ),

(0.3.1)

Introduction

where L n is some nite positive integer. (As mentioned earlier, methods for which
L n = are of no use, because they require knowledge of all the Am to obtain A n with
nite n.) In addition, for most extrapolation methods there holds
A n =

Kn


ni Ai ,

(0.3.2)

i=0

where K n are some nonnegative integers and the ni are some scalars that satisfy
Kn


ni = 1

(0.3.3)

i=0

for each n. (This is the case for all of the extrapolation methods we consider in this
work.) A consequence of (0.3.2) and (0.3.3) is that such extrapolation methods act as
summability methods for the sequence {An }.
When the ni are independent of the Am , the approximation A n is linear in the Am , thus
the extrapolation method that generates { A n } becomes a linear summability method. That
is to say, this extrapolation method can be applied to every sequence {An } with the same
ni . Both numerical experience and the different known convergence analyses suggest
that linear methods are of limited scope and not as effective as nonlinear methods.
As the subject of linear summability methods is very well-developed and is treated in
different books, we are not going to dwell on it in this book; see, for example, the books
by Knopp [152], Hardy [123], and Powell and Shah [231]. We only give the denition of
linear summability methods at the end of this section and recall the SilvermanToeplitz
theorem, which is one of the fundamental results on linear summability methods. Later
in this work, we also discuss the Euler transformation that has been used in different
practical situations and that is probably the most successful linear summability method.
When the ni depend on the Am , the approximation A n is nonlinear in the Am . This
implies that if Cm = Am + Bm , m = 0, 1, 2, . . . , for some constants and , and
{ A n }, { B n }, and {C n } are obtained by applying a given nonlinear extrapolation method to
{An }, {Bn }, and {Cn }, respectively, then C n = A n + B n , n = 0, 1, 2, . . . , in general.
(Equality prevails for all n when the extrapolation method is linear.) Despite this fact,
most nonlinear extrapolation methods enjoy a sort of linearity property that can be described as follows: Let = 0 and be arbitrary constants and consider Cm = Am + ,
m = 0, 1, 2, . . . . Then
C n = A n + , n = 0, 1, 2, . . . .

(0.3.4)

In other words, {Cn } = {An } + implies {C n } = { A n } + . This is called the quasilinearity property and is a useful property that we want every extrapolation method
to have. (All extrapolation methods treated in this book are quasi-linear.) A sufcient
condition for this to hold is given in Proposition 0.3.1.
Proposition 0.3.1 Let a nonlinear extrapolation method be such that the sequence { A n }
that it produces from {An } satises (0.3.2) with (0.3.3). Then the sequence {C n } that
it produces from {Cn = An + } for arbitrary constants = 0 and satises the

0.3 General Algebraic Properties of Extrapolation Methods

quasi-linearity property in (0.3.4) if the ni in (0.3.2) depend on the Am through the

Am = Am+1 Am only and are homogeneous in the Am of degree 0.
Remark. We recall that a function f (x1 , . . . , x p ) is homogeneous of degree r if, for
every = 0, f (x1 , . . . , x p ) = r f (x1 , . . . , x p ).

Kn
Proof. We begin by rewriting (0.3.2) in the form A n = i=0
ni ({Am })Ai . Similarly, we

Kn
have C n = i=0 ni ({Cm })Ci . From (0.3.1) and the conditions imposed on the ni , there
exist functions Dni ({u m }) for which
ni ({Am }) = Dni ({Am }) and ni ({Cm }) = Dni ({Cm }),

(0.3.5)

where the functions Dni satisfy for all = 0

Dni ({u m }) = Dni ({u m }).

(0.3.6)

This and the fact that {Cm } = {Am } imply that

ni ({Cm }) = Dni ({Cm }) = Dni ({Am }) = ni ({Am }).

(0.3.7)

From (0.3.2) and (0.3.7) we have, therefore,

C n =

Kn


ni ({Am })( Ai + ) = A n +

i=0

Kn


ni ({Am }).

(0.3.8)

i=0

The result now follows by invoking (0.3.3).

Example 0.3.2 Consider the Aitken 2 -process that was given by (0.1.3) in Example 0.1.1. We can reexpress A n in the form
A n = n,n An + n,n+1 An+1 ,

(0.3.9)

with
n,n =

An+1
An
, n,n+1 =
.
An+1 An
An+1 An

(0.3.10)

Thus, ni = 0 for 0 i n 1. It is easy to see that the ni satisfy the conditions of

Proposition 0.3.1 so that the 2 -process has the quasi-linearity property described in
(0.3.4). Note also that for this method L n = n + 2 in (0.3.1) and K n = n + 1 in (0.3.2).

0.3.1 Linear Summability Methods and the SilvermanToeplitz Theorem

We now go back briey to linear summability methods. Consider the innite matrix

00 01 02
10 11 12

M = ,
(0.3.11)
20 21 22

.. .. ..
. . .

Introduction

where ni are some xed scalars. The linear summability method associated with M is
the linear mapping that transforms an arbitrary sequence {An } to another sequence {An }
through
An =

ni Ai , n = 0, 1, 2, . . . .

(0.3.12)

i=0

This method is regular if limn An = A implies limn An = A. The Silverman

Toeplitz theorem that we state next gives necessary and sufcient conditions for a linear
summability method to be regular. For proofs of this fundamental result see, for example,
the books by Hardy [123] and Powell and Shah [231].
Theorem 0.3.3 (SilvermanToeplitz theorem). The summability method associated with
the matrix M in (0.3.11) is regular if and only if the following three conditions are
fullled simultaneously:


(i) limn i=0
ni = 1.
(ii) limn ni = 0, i = 0, 1, 2, . . . .


|ni | < .
(iii) supn i=0
Going back to the beginning of this section, we see that (0.3.3) is analogous to condition
(i) of Theorem 0.3.3. The issue of numerical stability discussed in Section 0.5 is very
closely related to condition (iii), as will become clear shortly.
The linear summability methods that have been of practical use are those whose

n
ni Ai .
associated matrices M are lower triangular, that is, those for which An = i=0
Excellent treatments of these methods from the point of view of convergence acceleration,
including an extensive bibliography, have been presented by Wimp [363], [364], [365],
[366, Chapters 24].

0.4 Remarks on Algorithms for Extrapolation Methods

A relatively important issue in the subject of extrapolation methods is the development
of efcient algorithms (computational procedures) for implementing existing extrapolation methods. An efcient algorithm is one that involves a small number of arithmetic
operations and little storage when storage becomes a problem.
Some extrapolation methods already have known closed-form expressions for the
sequences { A n } they generate. This is the case, for example, for the Aitken 2 -process.
One possible algorithm for such methods may be the direct computation of the closedform expressions. This is also the most obvious, but not necessarily the most economical,
approach in all cases.
Many extrapolation methods are dened through systems of linear or nonlinear equations, that is, they are dened implicitly by systems of the form
n,i ( A n , 1 , 2 , . . . , qn ; {Am }) = 0, i = 0, 1, . . . , qn ,

(0.4.1)

in which A n is the main quantity we are after, and 1 , 2 , . . . , qn are additional auxiliary
unknowns. As we will see in the next chapters, the better sequences { A n } are generated

0.4 Remarks on Algorithms for Extrapolation Methods

by those extrapolation methods with large qn , in general. This means that we actually
want to solve large systems of equations, which may be a computationally expensive
proposition. In such cases, the development of good algorithms becomes especially
important. The next example helps make this point clear.
Example 0.4.1 The Shanks [264] transformation of order k is an extrapolation method,
which, when applied to a sequence {An }, produces the sequence { A n = ek (An )}, where
ek (An ) satises the nonlinear system of equations
Ar = ek (An ) +

k


i ri , n r n + 2k,

(0.4.2)

i=1

where i and i are additional (auxiliary) 2k unknowns. Provided this system has a
solution with i = 0 and i = 0, 1 and i = j if i = j, then ek (An ) can be shown to
satisfy the linear system
Ar = ek (An ) +

k


i Ar +i1 , n r n + k,

(0.4.3)

i=1

where i are additional (auxiliary) k unknowns. Here Am = Am+1 Am , m =

0, 1, . . . , as before. [In any case, we can start with (0.4.3) as the denition of ek (An ).]
Now, this linear system can be solved using Cramers rule, giving us ek (An ) as the ratio
of two (k + 1) (k + 1) determinants in the form



An
An+1
An+k 

 An An+1 An+k 




..
..
..


.
.
.



 A
n+k1 An+k An+2k1
(0.4.4)
ek (An ) = 
.


1
1
1


 An An+1 An+k 




..
..
..


.
.
.


 A

A
A
n+k1

n+k

n+2k1

We can use this determinantal representation to compute ek (An ), but this would be very
expensive for large k and thus would constitute a bad algorithm. A better algorithm is
one that solves the linear system in (0.4.3) by Gaussian elimination. But this algorithm
too becomes costly for large k. The -algorithm of Wynn [368], on the other hand, is
very efcient as it produces all of the ek (An ), 0 n + 2k N , that are dened by
A0 , A1 , . . . , A N in only O(N 2 ) operations. It reads
(n)
1
= 0, 0(n) = An , n = 0, 1, . . . ,
(n)
(n+1)
= k1
+
k+1

1
k(n+1)

k(n)

, n, k = 0, 1, . . . ,

(0.4.5)

and we have
(n)
ek (An ) = 2k
, n, k = 0, 1, . . . .

(0.4.6)

10

Introduction

Incidentally, A n in (0.1.3) produced by the Aitken 2 -process is nothing but e1 (An ).

[Note that the Shanks transformations are quasi-linear extrapolation methods. This can
be seen either from the equations in (0.4.2), or from those in (0.4.3), or from the determinantal representation of ek (An ) in (0.4.4), or even from the -algorithm itself.]
Finally, there are extrapolation methods in the literature that are dened exclusively
by recursive algorithms from the start. The -algorithm of Brezinski [32] is such an
extrapolation method, and it is dened by recursion relations very similar to those of the
-algorithm.

0.5 Remarks on Convergence and Stability of Extrapolation Methods

The analysis of convergence and stability is the most important subject in the theory of
extrapolation methods. It is also the richest in terms of the variety of results that exist
and still can be obtained for different extrapolation methods and sequences. Thus, it is
impossible to make any specic remarks about convergence and stability at this stage.
We can, however, make several remarks on the approach to these topics that we take in
this book. We start with the topic of convergence analysis.

0.5.1 Remarks on Study of Convergence

The rst stage in the convergence analysis of extrapolation methods is formulation of
conditions that we impose on the {An }. In this book, we deal with sequences that arise
in common applications. Therefore, we emphasize mainly conditions that are relevant
to these applications. Also, we keep the number of the conditions imposed on the {An }
to a minimum as this leads to mathematically more valuable and elegant results. The
next stage is analysis of the errors A n A under these conditions. This analysis may
lead to different types of results depending on the complexity of the situation. In some
cases, we are able to give a full asymptotic expansion of A n A for n ; in other
cases, we obtain only the most dominant term of this expansion. In yet other cases, we
obtain a realistic upper bound on | A n A| from which powerful convergence results
can be obtained. An important feature of our approach is that we are not content only
with showing that the sequence { A n } converges more quickly than {An }, that is, that
convergence acceleration takes place in accordance with Denition 0.1.2, but instead
we aim at obtaining the precise asymptotic behavior of the corresponding acceleration
factor or a good upper bound for it.

0.5.2 Remarks on Study of Stability

We now turn to the topic of stability in extrapolation. Unlike convergence, this topic may
not be common knowledge, so we start with some rather general remarks on what we
mean by stability and how we analyze it. Our discussion here is based on those of Sidi
[272], [300], [305], and is recalled in relevant places throughout the book.
When we compute the sequence { A n } in nite-precision arithmetic, we obtain a sequence { A n } that is different from { A n }, the exact transformed sequence. This, of course,

0.5 Remarks on Convergence and Stability of Extrapolation Methods

11

is caused mainly by errors (roundoff errors and errors of other kinds as well) in the An .
Naturally, we would like to know by how much A n differs from A n , that is, we want
to be able to estimate | A n A n |. This is important also since knowledge of | A n A n |
assists in assessing the cumulative error | A n A| in A n . To see this, we start with


| A n A n | | A n A| | A n A| | A n A n | + | A n A|.
(0.5.1)
Next, let us assume that limn | A n A| = 0. Then (0.5.1) implies that | A n A|
| A n A n | for all sufciently large n, because | A n A n | remains nonzero.
We have observed numerically that, for many extrapolation methods that satisfy (0.3.2)
with (0.3.3), | A n A n | can be estimated by the product n  (n) , where
n =

Kn


|ni | 1 and  (n) = max{|i | : ni = 0},

(0.5.2)

i=0

and, for each i, i is the error in Ai . The idea behind this is that the ni and hence n do
not change appreciably with small errors in the Ai . Thus, if Ai + i are the computed Ai ,

Kn

Kn
ni (Ai + i ) = A n + i=0
ni i .
then A n , the computed A n , is very nearly given by i=0
As a result,

 Kn




ni i  n  (n) .
(0.5.3)
| An An | 
i=0

The meaning of this is that the quantity n [that always satises n 1 by (0.3.3)]
controls the propagation of errors in {An } into { A n }, in the sense that the absolute
computational error | A n A n | is practically the maximum of the absolute errors in the
Ai , 0 i K n , magnied by the factor n . Thus, combining (0.5.1) and (0.5.3), we
obtain
| A n A|  n  (n) + | A n A|

(0.5.4)

| A n A|
| A n A|
 (n)
 n
+
, provided A = 0,
|A|
|A|
|A|

(0.5.5)

for the absolute errors, and

for the relative errors.

The implication of (0.5.4) is that, practically speaking, the cumulative error | A n A| is
at least of the order of the corresponding theoretical error | A n A| but it may be as large
as n  (n) if this quantity dominates. [Note that because limn | A n A| = 0, n  (n)
will dominate | A n A| for sufciently large n.] The approximate inequality in (0.5.5)
reveals even more in case {An } is convergent and hence A n , A, and the Ai , 0 i K n ,
are all of the same order of magnitude and the latter are known correctly to r signicant decimal digits and n is of order 10 s , s < r . Then  (n) /|A| will be of order 10r ,
and, therefore, | A n A|/|A| will be of order 10 sr for sufciently large n. This implies
that A n will have approximately r s correct signicant decimal digits when n is sufciently large. When s r , however, A n may be totally inaccurate in the sense that it

12

Introduction

may be completely different from A n . In other words, in such cases n is also a measure
of the loss of relative accuracy in the computed A n .
One conclusion that can be drawn from this discussion is that it is possible to achieve
sufcient accuracy in A n by increasing r , that is, by computing the An with high accuracy.
This can be accomplished on a computer by doubling the precision of the oating-point
arithmetic used for computing the An .
When applying an extrapolation method to a convergent sequence {An } numerically,
we would like to be able to compute the sequence { A n } without | A n A n | becoming
unbounded for increasing n. In view of this and the discussion of the previous paragraphs,
we now give a formal denition of stability.
Denition 0.5.1 If an extrapolation method that generates from {An } the sequence { A n }
satises (0.3.2) with (0.3.3), then we say that it is stable provided supn n < , where

Kn
|ni |. Otherwise, it is unstable.
n = i=0
The mathematical treatment of stability then evolves around the analysis of {n }. Note
also the analogy between the denition of a stable extrapolation method and condition
(iii) in the SilvermanToeplitz theorem (Theorem 0.3.3).
It is clear from our discussion above that we need n to be of reasonable size relative to
the errors in the Ai for A n to be an acceptable representation of A n . Obviously, the ideal
case is one in which n = 1, which occurs when all the ni are nonnegative. This case
does arise, for example, in the application of some extrapolation methods to oscillatory
sequences. In most other situations, however, direct application of extrapolation methods
without taking into account the asymptotic nature of {An } results in large n and even
unbounded {n }. It then follows that, even though { A n } may be converging, A n may
be entirely different from A n for all large n. This, of course, is a serious drawback that
considerably reduces the effectiveness of extrapolation methods that are being used. This
problem is inherent in some methods, and it can be remedied in others by proper tuning.
We show later in this book how to tune extrapolation methods to reduce the size of n
and even to stabilize the methods completely.
Numerical experience and some theoretical results suggest that reducing n not only
stabilizes the extrapolation process but also improves the theoretical quality of the sequence { A n }.
Example 0.5.2 Let us see how the preceding discussion applies to the 2 -process on
the sequences {An } discussed in Example 0.1.1. First, from Example 0.3.2 it is clear that
n =

1 + |gn |
An+1
, gn =
.
|1 gn |
An

(0.5.6)

Next, from (0.1.1) and (0.1.2) we have that limn gn = = 1. Therefore,

lim n =

1 + ||
< .
|1 |

(0.5.7)

This shows that the 2 -process on such sequences is stable. Note that, for all large n,
| A n A| and n are proportional to |1 |2 and |1 |1 , respectively, and hence

0.5 Remarks on Convergence and Stability of Extrapolation Methods

13

are large when is too close to 1 in the complex plane. It is not difcult to see that
they can be reduced simultaneously in such a situation if the 2 -process is applied to
a subsequence {An }, where {2, 3, . . . }, since, for even small , is farther away
from 1 than is.
We continue our discussion of | A n A| assuming now that the Ai have been computed
with relative errors not exceeding . In other words, i = i Ai and |i | for all i.
(This is the case when the Ai have been computed to maximum accuracy that is possible
in nite-precision arithmetic with rounding unit u; we have = u in this situation.) Then
(0.5.4) becomes
| A n A|  n In ({As }) + | A n A|, In ({As }) max{|Ai | : ni = 0}. (0.5.8)
Obviously, when {An } converges, or diverges but is bounded, the term In ({As }) remains bounded as n . In this case, it follows from (0.5.8) that, provided n
remains bounded, | A n A| remains bounded as well. It should be noted, however,
that when {An } diverges and is unbounded, In ({As }) is unbounded as n , which
causes the right-hand side of (0.5.8) to become unbounded as n , even when
n is bounded. In such cases, | A n A| becomes unbounded, as we have observed in
all our numerical experiments. The hope in such cases is that the convergence rate
of the exact transformed sequence { A n } is much greater than the divergence rate of
{An } so that sufcient accuracy is achieved by A n before In ({As }) has grown too
much.
We also note that, in case {An } is divergent and the Ai have been computed with
relative errors not exceeding , numerical stability can be assessed more accurately by
replacing (0.5.3) and (0.5.4) by
| A n A n | 

Kn


|ni | |Ai |

(0.5.9)

|ni | |Ai | + | A n A|,

(0.5.10)

i=0

and
| A n A| 

Kn

i=0

respectively. Again, when the Ai have been computed to maximum accuracy that is
possible in nite-precision arithmetic with rounding unit u, we have = u in (0.5.9) and
(0.5.10). [Of course, (0.5.9) and (0.5.10) are valid when {An } converges too.]

0.5.3 Further Remarks

Finally, in connection with the studies of convergence and stability, we have found
it very useful to relate the given innite sequences {Am } to some suitable functions
A(y), where y may be a discrete or continuous variable. These relations take the form
Am = A(ym ), m = 0, 1, . . . , for some positive sequences {ym } that tend to 0, such
that limm Am = lim y0+ A(y) when limm Am exists. In some cases, a sequence
{Am } is derived directly from a known function A(y) exactly as described. The sequences

14

Introduction

discussed in Examples 0.2.3 and 0.2.4 are of this type. In certain other cases, we can show
the existence of a suitable function A(y) that is associated with a given sequence {Am }
even though {Am } is not provided by a relation of the form Am = A(ym ), m = 0, 1, . . . ,
a priori.
This kind of an approach is obviously of greater generality than that dealing with
innite sequences alone. First, for a given sequence {Am }, the related function A(y) may
have certain asymptotic properties for y 0+ that can be very helpful in deciding what
kind of an extrapolation method to use for accelerating the convergence of {Am }. Next,
in case A(y) is known a priori, we can choose {ym } such that (i) the convergence of
the derived sequence {Am = A(ym )} will be easier to accelerate by some extrapolation
method, and (ii) this extrapolation method will also enjoy good stability properties. Finally, the function A(y), in contrast to the sequence {Am }, may possess certain analytic
properties in addition to its asymptotic properties for y 0+. The analytic properties
may pertain, for example, to smoothness and differentiability in some interval (0, b]
that contains {ym }. By taking these properties into account, we are able to enlarge considerably the scope of the theoretical convergence and stability studies of extrapolation
methods. We are also able to obtain powerful and realistic results on the behavior of the
sequences { A n }.
We shall use this approach to extrapolation methods in many places throughout this
book, starting as early as Chapter 1.
Historically, those convergence acceleration methods associated with functions A(y)
and derived from them have been called extrapolation methods, whereas those that
apply to innite sequences and that are derived directly from them have been called
sequence transformations. In this book, we also make this distinction, at least as far
as the order of presentation is concerned. Thus, we devote Part I of the book to the
Richardson extrapolation process and its various generalizations and Part II to sequence
transformations.

0.6 Remark on Iterated Forms of Extrapolation Methods

One of the ways to apply extrapolation methods is by simply iterating them. Let us again
consider an arbitrary extrapolation method ExtM. The iteration of ExtM is performed
as follows: We rst apply ExtM to {C0(n) = An } to obtain the sequence {C1(n) }. We next
(n)
(n)
apply it to {Cs(n) }
n=0 to obtain {C s+1 }n=0 , s = 1, 2, . . . . Let us organize the C s in a
two-dimensional array as in Table 0.6.1. In general, columns of this table converge, each
column converging at least as quickly as the one preceding it. Diagonals converge as
well, and they converge much quicker than the columns.
It is thus obvious that every extrapolation method can be iterated. For example, we
can iterate ek , the Shanks transformation of order k discussed in Example 0.4.1, exactly
as explained here. In this book, we consider in detail the iteration of only two classic
methods: The 2 -process, which we have discussed briey in Example 0.1.1, and the
Lubkin transformation. Both of these methods and their iterations are considered in detail
in Chapter 15. We do not consider the iterated forms of the other methods discussed in
this book, mainly because, generally speaking, their performance is not better than that

0.7 Relevant Issues in Extrapolation

15

Table 0.6.1: Iteration of an extrapolation method ExtM.

Each column is obtained by applying ExtM to the
preceding column
C0(0)
C0(1)
C0(2)
C0(3)
..
.

C1(0)
C1(1)
C1(2)
..
.

C2(0)
C2(1)
..
.

C3(0)
..
.

..

provided by the straightforward application of the corresponding methods; in some cases

they even behave in undesired peculiar ways. In addition, their analysis can be done very
easily once the techniques of Chapter 15 (on the convergence and stability of their column
sequences) are understood.

0.7 Relevant Issues in Extrapolation

We close this chapter by giving a list of issues we believe are relevant to both the theory
and the practice of extrapolation methods. Inevitably, some of these issues are more
important than others and deserve more attention.
The rst issue we need to discuss is that of development and design of extrapolation
methods. Being the start of everything, this is a very important phase. It is best to embark
on the project of development with certain classes of sequences {An } in mind. Once we
have developed a method that works well on those sequences for which it was designed,
we can always apply it to other sequences as well. In some cases, this may even result
in success. The Aitken 2 -process may serve as a good example to illustrate this point
in a simple way.
Example 0.7.1 Let A, , a, and b be in general complex scalars and let the sequence
{An } be such that


 0.
An = A + n an s + bn s1 + O(n s2 ) as n ; = 0, 1, a = 0, and s =
(0.7.1)
Even though the 2 -process was designed to accelerate the convergence of sequences
{An } whose members satisfy (0.1.1), it turns out that it accelerates the convergence of
sequences {An } that satisfy (0.7.1) as well. Substituting (0.7.1) in (0.1.3), it can be shown
after tedious algebraic manipulations that

2

n s2

An A K n
as n ; K = as
,
(0.7.2)
1
as opposed to An A an n s as n . Thus, for || < 1 both {An } and { A n } converge, and { A n } converges more quickly.

16

Introduction

Naturally, if we have to make a choice about which extrapolation methods to employ

regularly as users, we would probably prefer those methods that are effective accelerators
for more than one class of sequences. Finally, we would like to remark that, as the design
of extrapolation methods is based on the asymptotic properties of An for n in
many important cases, there is great value to analyzing An asymptotically for n .
In addition, this analysis also produces the conditions on {An } that we need for the
convergence study, as discussed in the second paragraph of Section 0.5. In many cases
of interest, this study may turn out to be very nontrivial and challenging.
The next issue is the design of good algorithms for implementing extrapolation methods. Recall that there may be more than one algorithm for implementing a given method.
As mentioned before, a good algorithm is one that requires a small number of operations
and little storage. In addition, it should be as stable as possible numerically. Needless to
say, from the point of view of a user, we should always prefer the most efcient algorithm for a given method. Note that the development of algorithms, because it relies on
algebraic manipulations only, is not in general as important and illuminating as either
the development of extrapolation methods or the analysis of these methods, an issue we
discuss below, or the asymptotic study of {An }, which precedes all this. After all, the
quality of the sequence { A n } is determined exclusively by the extrapolation method that
generates it and not by whichever algorithm is used for implementing this extrapolation
method. Algorithms are only numerical means by which we obtain the { A n } already
uniquely determined by the extrapolation method. For these reasons, we reduce our
treatment of algorithms to a minimum.
Some of the literature deals with so-called kernels of extrapolation methods. The
kernel of an extrapolation method is the class of sequences {An } for which A n = A for
all n, where A is the limit or antilimit of {An }. Unfortunately, there is very little one can
learn about the convergence behavior and stability of the method in question by looking
at its kernel. For this reason, we have left out the treatment of kernels almost completely.
We have considered in passing only those kernels that are obtained in a trivial way.
Given an extrapolation method and a class of sequences to which it is applied, as
we mentioned in the preceding section, the most important and challenging issues are
those of convergence and stability. Proposing realistic and useful analyses for convergence and stability has always been a very difcult task because most of the practical
extrapolation methods are highly nonlinear. As a result, very few papers have dealt with
convergence and stability, where we believe more efforts should be spent. A variety of
mathematical tools are needed for this task, asymptotic analysis being the most crucial
of them.
As for the user, we believe that he should be at least familiar with the existing convergence and stability results, as these may be very helpful in deciding which extrapolation
method should be applied, to which problem, how it should be applied, and how it
should be tuned for good stability properties. In addition, having at least some idea
about the asymptotic behavior of the elements of the sequence whose convergence is
being accelerated is of great assistance in efcient application of extrapolation methods.
We would like to make one last remark about extrapolation methods as opposed
to algorithms by which methods are implemented. In recent years, there has been an

0.7 Relevant Issues in Extrapolation

17

unfortunate confusion of terminology, in that many papers use the concepts of method and
algorithm interchangeably. The result of this confusion has been that effectiveness due
to extrapolation methods has been incorrectly assigned to extrapolation algorithms. As
noted earlier, the sequences { A n } are uniquely dened and their properties are determined
only by extrapolation methods and not by algorithms that implement the methods. In
this book, we are very careful to avoid this confusion by distinguishing between methods
and algorithms.

Part I
The Richardson Extrapolation Process
and Its Generalizations

1
The Richardson Extrapolation Process

1.1 Introduction and Background

In many problems of practical interest, a given innite sequence {An } can be related
to a function A(y) that is known, and hence is computable, for 0 < y b with some
b > 0, the variable y being continuous or discrete. This relation takes the form An =
A(yn ), n = 0, 1, . . . , for some monotonically decreasing sequence {yn } (0, b] that
satises limn yn = 0. Thus, in case lim y0+ A(y) = A, limn An = A as well.
Consequently, computing limn An amounts to computing lim y0+ A(y) in such a
case, and this is precisely what we want to do.
Again, in many cases of interest, the function A(y) may have a well-dened expansion
for y 0+ whose form is known. For example and this is the case we treat in this
chapter A(y) may satisfy for some positive integer s
A(y) = A +

s


k y k + O(y s+1 ) as y 0+,

(1.1.1)

k=1

where k = 0, k = 1, 2, . . . , s + 1, and 1 < 2 < < s+1 , and where k are

constants independent of y. Obviously, 1 > 0 guarantees that lim y0+ A(y) = A.
When lim y0+ A(y) does not exist, A is the antilimit of A(y) for y 0+, and in this
case i 0 at least for i = 1. If (1.1.1) is valid for all s = 1, 2, 3, . . . , and 1 <
2 < , with limk k = +, then A(y) has the true asymptotic expansion
A(y) A +

k y k as y 0+,

(1.1.2)

k=1

k
whether the innite series
k=1 k y converges or not. (In most cases of interest, this
series diverges strongly.) The k are assumed to be known, but the coefcients k need
not be known; generally, the k are not of interest to us. We are interested in nding A
whether it is the limit or the antilimit of A(y) for y 0+.
Suppose now that 1 > 0 so that lim y0+ A(y) = A. Then A can be approximated by
A(y) with sufciently small values of y, the error in this approximation being A(y) A =
O(y 1 ) as y 0+ by (1.1.1). If 1 is sufciently large, A(y) can approximate A well
even for values of y that are not too small. If this is not the case, however, then we may have
to compute A(y) for very small values of y to obtain reasonably good approximations

21

22

1 The Richardson Extrapolation Process

to A. Unfortunately, this straightforward idea of reducing y to very small values is not

always applicable. In most cases of interest, computing A(y) for very small values of
y either is very costly or suffers from loss of signicance in nite-precision arithmetic.
The deeper idea of the Richardson extrapolation, on the other hand, is to somehow
eliminate the y 1 term from the expansion in (1.1.1) and to obtain a new approximation
A1 (y) to A whose error is A1 (y) A = O(y 2 ) as y 0+. Obviously, A1 (y) will be a
better approximation to A than A(y) for small y since 2 > 1 . In addition, if 2 is
sufciently large, then we expect A1 (y) to approximate A well also for values of y that
are not too small, independently of the size of 1 . At this point, we mention only that
the Richardson extrapolation is achieved by taking an appropriate weighted average
of A(y) and A(y) for some (0, 1). We give the precise details of this procedure in
the next section.
From (1.1.1), it is clear that A(y) A = O(y 1 ) as y 0+, whether 1 > 0 or not.
Thus, the function A1 (y) that results from the Richardson extrapolation can be a useful
approximation to A for small values of y also when 1 0, provided 2 > 0. That is
to say, lim y0+ A1 (y) = A provided 2 > 0 whether lim y0+ A(y) exists or not. This
is an additional fundamental and useful feature of the Richardson extrapolation.
In the following examples, we show how functions A(y) exactly of the form we have
described here come about naturally. In these examples, we treat the classic problems
of computing by the method of Archimedes, numerical differentiation by differences,
numerical integration by the trapezoidal rule, summation of an innite series that is
used in dening the Riemann Zeta function, and the Hadamard nite parts of divergent
integrals.
Example 1.1.1 The Method of Archimedes for Computing The method of Archimedes for computing consists of approximating the area of the unit disk (that is nothing
but ) by the area of an inscribed or circumscribing regular polygon. If this polygon is
inscribed in the unit disk and has n sides, then its area is simply Sn = (n/2) sin(2/n).
Obviously, Sn has the (convergent) series expansion
Sn = +

(1)i (2)2i+1 2i
1
n ,
2 i=1 (2i + 1)!

(1.1.3)

and the sequence {Sn } is monotonically increasing and has as its limit.
If the polygon circumscribes the unit disk and has n sides, then its area is Sn =
n tan(/n), and Sn has the (convergent) series expansion
Sn = +


(1)i 4i+1 (4i+1 1) 2i+1 B2i+2
i=1

(2i + 2)!

n 2i ,

(1.1.4)

where Bk are the Bernoulli numbers (see Appendix D), and the sequence {Sn } this time
is monotonically decreasing and has as its limit.
As the expansions given in (1.1.3) and (1.1.4) are also asymptotic as n , Sn in
both cases is analogous to the function A(y). This analogy is as follows: Sn A(y),
n 1 y, k = 2k, k = 1, 2, . . . , and A. The variable y is discrete and assumes
the values 1/3, 1/4, . . . .

1.1 Introduction and Background

23

Finally, the subsequences {S2m } and {S32m } can be computed recursively without
having to know , their computation involving only square roots. (See Example 2.2.2 in
Chapter 2.)
Example 1.1.2 Numerical Differentiation by Differences Let f (x) be continuously
differentiable at x = x0 , and assume that f  (x0 ), the rst derivative of f (x) at x0 , is
needed. Assume further that the only thing available to us is f (x) , or a procedure that
computes f (x), for all values of x in a neighborhood of x0 .
If f (x) is known in the neighborhood [x0 a, x0 + a] for some a > 0, then f  (x0 )
can be approximated by the centered difference 0 (h) that is given by
0 (h) =

f (x0 + h) f (x0 h)
, 0 < h a.
2h

(1.1.5)

Note that h here is a continuous variable. Obviously, limh0 0 (h) = f  (x0 ). The accuracy of 0 (h) is quite low, however. When f C 3 [x0 a, x0 + a], there exists
(h) [x0 h, x0 + h], for which the error in 0 (h) satises
0 (h) f  (x0 ) =

f  ( (h)) 2
h = O(h 2 ) as h 0.
3!

(1.1.6)

When the function f (x) is continuously differentiable a number of times, the error
0 (h) f  (x0 ) can be expanded in powers of h 2 . For f C 2s+3 [x0 a, x0 + a], there
exists (h) [x0 h, x0 + h], for which we have
0 (h) = f  (x0 ) +

s

f (2k+1) (x0 ) 2k
h + Rs (h),
(2k + 1)!
k=1

(1.1.7)

where
Rs (h) =

f (2s+3) ( (h)) 2s+2

h
= O(h 2s+2 ) as h 0.
(2s + 3)!

(1.1.8)

The proof of (1.1.7) and (1.1.8) can be achieved by expanding f (x0 h) in a Taylor
series about x0 with remainder.
The difference 0 (h) is thus seen to be analogous to the function A(y). This analogy
is as follows: 0 (h) A(y), h y, k = 2k, k = 1, 2, . . . , and f  (x0 ) A.
When f C [x0 a, x0 + a], the expansion in (1.1.7) holds for all s = 0, 1, . . . .
As a result, we can replace it by the genuine asymptotic expansion
0 (h) f  (x0 ) +


f (2k+1) (x0 ) 2k
h as h 0,
(2k + 1)!
k=1

(1.1.9)

whether the innite series on the right-hand side of (1.1.9) converges or not.
As is known, in nite-precision arithmetic, the computation of 0 (h) for very small
values of h is dominated by roundoff. The reason for this is that as h 0 both f (x0 + h)
and f (x0 h) tend to f (x0 ), which causes the difference f (x0 + h) f (x0 h) to
have fewer and fewer correct signicant digits. Thus, it is meaningless to carry out the
computation of 0 (h) beyond a certain threshold value of h.

24

1 The Richardson Extrapolation Process

Example 1.1.3 Numerical Quadrature

by Trapezoidal Rule Let f (x) be dened on
1
[0, 1], and assume that I [ f ] = 0 f (x) d x is to be computed by numerical quadrature.
One of the simplest numerical quadrature formulas is the trapezoidal rule. Let T (h) be
the trapezoidal rule approximation to I [ f ], with h = 1/n, n being a positive integer.
Then, T (h) is given by


n1

1
1
f (0) +
(1.1.10)
f ( j h) + f (1) .
T (h) = h
2
2
j=1
Note that h for this problem is a discrete variable that takes on the values 1, 1/2, 1/3, . . . .
It is well known that T (h) tends to I [ f ] as h 0 (or n ), whenever f (x) is Riemann
integrable on [0, 1]. When f C 2 [0, 1], there exists (h) [0, 1], for which the error
in T (h) satises
T (h) I [ f ] =

f  ( (h)) 2
h = O(h 2 ) as h 0.
12

(1.1.11)

When the integrand f (x) is continuously differentiable a number of times, the error
T (h) I [ f ] can be expanded in powers of h 2 . For f C 2s+2 [0, 1], there exists (h)
[0, 1], for which
T (h) = I [ f ] +

s


B2k  (2k1)
f
(1) f (2k1) (0) h 2k + Rs (h),
(2k)!
k=1

(1.1.12)

where
Rs (h) =

B2s+2
f (2s+2) ( (h))h 2s+2 = O(h 2s+2 ) as h 0.
(2s + 2)!

(1.1.13)

Here B p are the Bernoulli numbers as before. The expansion in (1.1.12) with (1.1.13) is
known as the EulerMaclaurin expansion (see Appendix D) and its proof can be found
in many books on numerical analysis.
The approximation T (h) is analogous to the function A(y) in the following sense:
T (h) A(y), h y, k = 2k, k = 1, 2, . . . , and I [ f ] A.
Again, for f C 2s+2 [0, 1], an expansion that is identical in form to (1.1.12) with
(1.1.13) exists for the midpoint rule approximation M(h), where
M(h) = h

n


f ( j h 12 h).

(1.1.14)

j=1

This expansion is
M(h) = I [ f ] +

s


B2k ( 12 )  (2k1)
f
(1) f (2k1) (0) h 2k + Rs (h), (1.1.15)
(2k)!
k=1

where, again for some (h) [0, 1],

Rs (h) =

B2s+2 ( 12 ) (2s+2)
( (h))h 2s+2 = O(h 2s+2 ) as h 0.
f
(2s + 2)!

(1.1.16)

Here B p (x) is the Bernoulli polynomial of degree p and B2k ( 12 ) = (1 212k )B2k ,
k = 1, 2, . . . .

1.1 Introduction and Background

25

When f C [0, 1], both expansions in (1.1.12) and (1.1.15) hold for all s =
0, 1, . . . . As a result, we can replace both by genuine asymptotic expansions of the
form
Q(h) I [ f ] +

ck h 2k as h 0,

(1.1.17)

k=1

where Q(h) stands for T (h) or M(h), and ck is the coefcient of h 2k in (1.1.12) or
(1.1.15). Generally, when f (x) is not analytic in [0, 1], or even when it is analytic there


2k
but is not entire, the innite series
in (1.1.17) diverges very strongly.
k=1 ck h
Finally, by h = 1/n, the computation of Q(h) for very small values of h involves a
large number of integrand evaluations and hence is very costly.
Example 1.1.4 Summation of the Riemann Zeta Function Series Let An =

n
z
m=1 m , n = 1, 2, . . . . When z > 1, limn An = (z), where (z) is the
Riemann Zeta function. For z 1, on the other hand, limn An does not exist.


z
is taken as the denition of (z) for z > 1.
Actually, the innite series
m=1 m
With this denition, (z) is an analytic function of z for z > 1. Furthermore, it can be
continued analytically to the whole z-plane with the exception of the point z = 1, where
it has a simple pole with residue 1.
For all z = 1, i.e., whether limn An exists or not, we have the well-known asymptotic expansion (see Appendix E)



1z
1 
(1)i
(1.1.18)
An (z) +
Bi n zi+1 as n ,
i
1 z i=0

where Bi are the Bernoulli numbers as before and ai are the binomial coefcients. We
also recall that B3 = B5 = B7 = = 0, and that the rest of the Bi are nonzero.
The partial sum An is thus analogous to the function A(y) in the following
sense: An A(y), n 1 y, 1 = z 1, 2 = z, k = z + 2k 5, k = 3, 4, . . . , and
(z) A provided z = m + 1, m = 0, 1, 2, . . . . Thus, (z) is the limit of {An } when
z > 1, and its antilimit otherwise, provided z = m + 1, m = 0, 1, 2, . . . . Obviously,
the variable y is now discrete and takes on the values 1, 1/2, 1/3, . . . .
Note also that the innite series on the right-hand side of (1.1.18) is strongly divergent.
Example 1.1.5 Numerical Integration
of Periodic Singular Functions Let us now
1
consider the integral I [ f ] = 0 f (x) d x, where f (x) is a 1-periodic function that
is innitely differentiable on (, ) except at the points t + k, k = 0, 1,
2, . . . , where it has logarithmic singularities, and can be written in the form f (x) =

g(x) log |x t| + g(x)

when x, t [0, 1]. For example, with u C (, ) and
periodic with period 1, and with c some constant, f (x) = u(x) log (c| sin (x t)|)
= u(t) log( c). Sidi and
is such a function. For this f (x), we have g(t) = u(t) and g(t)
Israeli [310] derived the corrected trapezoidal rule approximation
 
n1

h

f (t + i h) + g(t)h
+ g(t)h log
, h = 1/n, (1.1.19)
T (h; t) = h
2
i=1

26

1 The Richardson Extrapolation Process

for I [ f ], and showed that T (h; t) has the asymptotic expansion

T (h; t) I [ f ] 2


 (2k)
k=1

(2k)!

g (2k) (t)h 2k+1 as h 0.

(1.1.20)

d
Here  (z) = dz
(z). (See Appendix D.)
The approximation T (h; t) is analogous to the function A(y) in the following sense:
T (h; t) A(y), h y, k = 2k + 1, k = 1, 2, . . . , and I [ f ] A. In addition, y
takes on the discrete values 1, 1/2, 1/3, . . . .

Example
 1 1.1.6 Hadamard Finite Parts of Divergent Integrals Consider the integral 0 x g(x) d x, where g C [0, 1] and is generally complex such that =
1, 2, . . . . When  > 1, the integral exists in the ordinary sense. In case g(0) = 0
and  1, the integral does not exist in the ordinary sense since x g(x) is not integrable at x = 0, but itsHadamard nite part exists, as we mentioned in Example 0.2.4.
1
Let us dene Q(h) = h x g(x) d x. Obviously, Q(h) is well-dened and computable
for h (0, 1]. Let m be any nonnegative integer. Then, there holds
 1 
m1
m1
 g (i) (0) 
 g (i) (0) 1 h +i+1
xi dx +
. (1.1.21)
x g(x)
Q(h) =
i!
i! + i + 1
h
i=0
i=0
Now let m >  1. Expressing the integral term in (1.1.21) in the form
1 h
0 0 , using the fact that
g(x)

m1

i=0

1
h

g (i) (0) i
g (m) ( (x)) m
x =
x , for some (x) (0, x),
i!
m!

and dening


I () =

x
0


g(x)

m1

i=0


m1

g (i) (0)
g (i) (0) i
1
x dx +
,
i!
+ i + 1 i!
i=0

(1.1.22)

and g (m)  = max0x1 |g (m) (x)|, we obtain from (1.1.21)

Q(h) = I ()

m1

i=0

g (i) (0) h +i+1

g (m)  h +m+1
+ Rm (h); |Rm (h)|
,
i! + i + 1
m!  + m + 1
(1.1.23)

[Note that, with m >  1, the integral term in (1.1.22) exists in the ordinary sense
and I () is independent of m.] Since m is also arbitrary in (1.1.23), we conclude that
Q(h) has the asymptotic expansion
Q(h) I ()


g (i) (0) h +i+1
as h 0.
i! + i + 1
i=0

(1.1.24)

Thus, Q(h) is analogous to the function A(y) in the following sense: Q(h) A(y),
h y, k = + k, k = 1, 2, . . . , and I () A. Of course, y is a continuous variable
in this case. When the integral exists in the
 1 ordinary sense, I () = limh0 Q(h); otherwise, I () is the Hadamard nite part of 0 x g(x) d x and serves as the antilimit of Q(h)

1.2 The Idea of Richardson Extrapolation

27

1
as h 0. Finally, I () = 0 x g(x) d x is analytic in for  > 1 and, by (1.1.22),
can be continued analytically to a meromorphic function with simple poles possibly at
= 1, 2, . . . . Thus, the Hadamard nite part is nothing butthe analytic continuation
1
of the function I () that is dened via the convergent integral 0 x g(x) d x,  > 1,
to values of for which  1, = 1, 2, . . . .
Before going on, we mention that many of the developments of this chapter are due
to Bulirsch and Stoer [43], [45], [46]. The treatment in these papers assumes that the k
are real and positive. The case of generally complex k was considered recently in Sidi
[298], where the function A(y) is allowed to have a more general asymptotic behavior
than in (1.1.2). See also Sidi [301].

1.2 The Idea of Richardson Extrapolation

We now go back to the function A(y) discussed in the second paragraph of the preceding
section. We do not assume that lim y0+ A(y) necessarily exists. We recall that, when it
exists, this limit is equal to A in (1.1.1) ; otherwise, A there is the antilimit of A(y) as
y 0+. Also, the nonexistence of lim y0+ A(y) immediately implies that i 0 at
least for i = 1.
As mentioned in the third paragraph of the preceding section, A(y) A = O(y 1 )
as y 0+, and we would like to eliminate the y 1 term from (1.1.1) and thus obtain
a new approximation to A that is better than A(y) for y 0+. Let us pick a constant
(0, 1), and set y  = y. Then, from (1.1.1) we have
A(y  ) = A +

s


k k y k + O(y s+1 ) as y 0 + .

(1.2.1)

k=1

Multiplying (1.1.1) by 1 and subtracting from (1.2.1), we obtain

A(y  ) 1 A(y) = (1 1 )A +

s

(k 1 )k y k + O(y s+1 ) as y 0 + .
k=2

(1.2.2)
Obviously, the term y 1 is missing from the summation in (1.2.2). Dividing both sides
of (1.2.2) by (1 1 ), and identifying
A(y, y  ) =

A(y  ) 1 A(y)
1 1

(1.2.3)

as the new approximation to A, we have

A(y, y  ) = A +

s

k 1
k y k + O(y s+1 ) as y 0+,
1
1

k=2

(1.2.4)

so that A(y, y  ) A = O(y 2 ) as y 0+, as was required. It is important to note that

(1.2.4) is exactly of the form (1.1.1) with A(y) and the k replaced by A(y, y  ) and the
k 1
k , respectively.
11

28

1 The Richardson Extrapolation Process

We can now continue along the same lines and eliminate the y 2 term from (1.2.4).
This can be achieved by combining A(y, y  ) and A(y  , y  ) with y  = y  = 2 y. The
resulting new approximation is
A(y, y  , y  ) =

A(y  , y  ) 2 A(y, y  )
,
1 2

(1.2.5)

and we have
A(y, y  , y  ) = A +

s

k 1 k 2
k y k + O(y s+1 ) as y 0+, (1.2.6)
1
2
1

k=3

so that A(y, y  , y  ) A = O(y 3 ) as y 0+.

This process, which is called the Richardson extrapolation process, can be repeated to
eliminate the terms y 3 , y 4 , . . . , from the summation in (1.2.6). We do this in the next
section by formalizing the preceding procedure, where we give a very efcient recursive
algorithm for the Richardson extrapolation process as well.
Before we end this section, we would like to mention that the preceding procedure
was rst described [for the case k = 2k, k = 1, 2, . . . , in (1.1.1)] by Richardson [236],
who called it deferred approach to the limit. Richardson applied this approach to improve
the nite difference solutions of some partial differential equations, such as Laplaces
equation in a square. Later, Richardson [237] used it to improve the numerical solutions of
an integral equation. Finally, Richardson [238] used the idea of extrapolation to accelerate
the convergence of sequences, to compute Fourier coefcients, and to solve differential
eigenvalue problems. For all these details and many more of the early references on this
subject, we refer the reader to the excellent survey by Joyce [145]. In the remainder of


kr
this work, we refer to the extrapolation procedure concerning A(y) A +
k=1 k y
as y 0+, where r > 0, as the polynomial Richardson extrapolation process, and we
discuss it in some detail in the next chapter.

1.3 A Recursive Algorithm for the Richardson Extrapolation Process

Let us pick a constant (0, 1) and y0 (0, b] and let ym = y0 m , m = 1, 2, . . . .
Obviously, {ym } is a decreasing sequence in (0, b] and limm ym = 0.
Algorithm 1.3.1
( j)

1. Set A0 = A(y j ), j = 0, 1, 2, . . . .
( j)
2. Set cn = n and compute An by the recursion
( j+1)

A(nj) =
( j)

( j)

An1 cn An1
, j = 0, 1, . . . , n = 1, 2, . . . .
1 cn

The An are approximations to A produced by the Richardson extrapolation process.

( j)
We have the following result concerning the An .

1.4 Algebraic Properties of the Richardson Extrapolation Process

29

Table 1.3.1: The Romberg table

A(0)
0
A(1)
0
A(2)
0
A(3)
0
..
.

A(0)
1
A(1)
1
A(2)
1
..
.

A(0)
2
A(1)
2
..
.

A(0)
3
..
.

..

( j)

Theorem 1.3.2 In the notation of the previous section, the An satisfy

A(nj) = A(y j , y j+1 , . . . , y j+n ).

(1.3.1)

The proof of (1.3.1) can be done by induction and is left to the reader.
( j)
The An can be arranged in a two-dimensional array, called the Romberg table, as in
Table 1.3.1. The arrows in the table show the ow of computation.
Given the values A(ym ), m = 0, 1, . . . , N , and cm = m , m = 1, 2, . . . , N , this
( j)
algorithm produces the 12 N (N + 1) approximations An , 1 j + n N , n 1.
( j)
1
The computation of these An can be achieved in 2 N (N + 1) multiplications, 12 N (N +
1) divisions, and 12 N (N + 3) additions. This computation also requires 12 N 2 + O(N )
storage locations. When only the diagonal approximations A(0)
n , n = 1, 2, . . . , N , are
required, the algorithm can be implemented with N + O(1) storage locations. This
( j) n
can be achieved by computing Table 1.3.1 columnwise and letting {An } Nj=0
overwrite
( j) N n+1
{An1 } j=0 . It can also be achieved by computing the table row-wise and letting the
row {A(ln)
}ln=0 overwrite the row {A(l1n)
}l1
n
n
n=0 . The latter approach enables us to introduce A(ym ), m = 0, 1, . . . , one by one. As we shall see in Section 1.5, the diagonal
( j)
sequences {An }
n=0 have excellent convergence properties.

1.4 Algebraic Properties of the Richardson Extrapolation Process

1.4.1 A Related Set of Polynomials
( j)

As part of the input to Algorithm 1.3.1 is the sequence {A(m)

0 = A(ym )}, the An are, of
( j)
course, functions of the A(ym ). The relationship between {A(ym )} and the An , however,
is hidden in the algorithm. We now investigate the precise nature of this relationship.
We start with the following simple lemma.
( j)

( j)

Lemma 1.4.1 Given the sequence {B0 }, dene the quantities {Bn } by the recursion
( j+1)

( j)

Bn( j) = (nj) Bn1 + (nj) Bn1 , j = 0, 1, . . . , n = 1, 2, . . . ,

( j)

(1.4.1)

( j)

where the scalars n and n satisfy

(nj) + (nj) = 1, j = 0, 1, . . . , n = 1, 2, . . . .

(1.4.2)

30

1 The Richardson Extrapolation Process

( j)

(m)
Then there exist scalars ni that depend on the (m)
k and k , such that

Bn( j) =

n


( j)

( j+i)

ni B0

and

i=0

n


( j)

ni = 1.

(1.4.3)

i=0

Proof. The proof of (1.4.3) is by induction on n.

( j)

( j)

Lemma 1.4.1 and Algorithm 1.3.1 together imply that An is of the form An =

n

n
( j) ( j+i)
( j)
( j)
for some ni that satisfy i=0
ni = 1. Obviously, this does not reveal
i=0 ni A0
( j)
anything fundamental about the relationship between An and {A(ym )}, aside from the
( j)
assertion that An is a weighted average of some sort of A(yl ), j l j + n. The
following theorem, on the other hand, gives a complete description of this relationship.
Theorem 1.4.2 Let ci = i , i = 1, 2, . . . , and dene the polynomials Un (z) by
Un (z) =

n
n


z ci

ni z i .
1 ci
i=1
i=0

(1.4.4)

( j)

Then An is related to the A(ym ) through

A(nj) =

n


ni A(y j+i ).

(1.4.5)

i=0

Obviously,
n


ni = 1.

(1.4.6)

i=0

Proof. From (1.4.4), we have

Un (z) =

z cn
Un1 (z),
1 cn

(1.4.7)

from which we also have, with ki = 0 for i < 0 or i > k for all k,
ni =

n1,i1 cn n1,i
, 0 i n.
1 cn

(1.4.8)

( j)

Now we can use (1.4.8) to show that An , as given in (1.4.5), satises the recursion
relation in Algorithm 1.3.1. This completes the proof.


From this theorem we see that, for the Richardson extrapolation process we are dis( j)
cussing now, the ni alluded to above are simply ni ; therefore, they are independent of
j as well.
The following result concerning the ni will be of use in the convergence and stability
analyses that we provide in the next sections of this chapter.

1.4 Algebraic Properties of the Richardson Extrapolation Process

31

Theorem 1.4.3 The coefcients ni of the polynomial Un (z) dened in Theorem 1.4.2
are such that
n


|ni | |z|i

i=0

n

|z| + |ci |
i=1

|1 ci |

(1.4.9)

In particular,
n


|ni |

i=0

n

1 + |ci |
i=1

|1 ci |

(1.4.10)

If ci , 1 i n, all have the same phase, which occurs when i , 1 i n, all have the
same imaginary part, then equality holds in both (1.4.9) and (1.4.10). This takes place,
in particular, when ci , 1 i n, are all real positive or all real negative. Furthermore,

n
|ni | = 1 for the case in which ci , 1 i n, are all real negative.
we have i=0
Theorem 1.4.3 is stated in Sidi [298] and its proof can be achieved by using the
following general result that is also given there.

n

Lemma 1.4.4 Let Q(z) =

z 1 , z 2 , . . . , z n . Then

i=0

n


ai z i , an = 1. Denote the zeros of Q(z) by

|ai | |z|i

i=0

n


(|z| + |z i |),

(1.4.11)

i=1

whether the an and/or z i are real or complex. Equality holds in (1.4.11) when
z 1 , z 2 , . . . , z n all have the same phase. It holds, in particular, when z 1 , z 2 , . . . , z n are
all real positive or all real negative.
Proof. Let


k1 <k2 <<ki

n

n

a i z i ,
Q(z)
= i=1
(z + |z i |) = i=0
i
s=1 z ks , i = 1, 2, . . . , n, we have


|ani |

i


a n = 1.

From

|z ks | = a ni , i = 1, . . . , n.

(1)i ani =

(1.4.12)

k1 <k2 <<ki s=1

Thus,
n

i=0

|ai | |z|i

n


a i |z|i = Q(|z|),

(1.4.13)

i=0

from which (1.4.11) follows. When the z i all have the same phase, equality holds in
(1.4.12) and hence in (1.4.13). This completes the proof.

1.4.2 An Equivalent Alternative Denition of Richardson Extrapolation

Finally, we give a different but equivalent formulation of the Richardson extrapolation
( j)
process in which the An are dened by linear systems of equations. In Section 1.2, we
showed how the approximation A(y, y  ) that is given by (1.2.3) and satises (1.2.4) can

32

1 The Richardson Extrapolation Process

be obtained by eliminating the y 1 term from (1.1.1). The procedure used to this end is
equivalent to solving the linear system
A(y) = A(y, y  ) + 1 y 1

A(y  ) = A(y, y  ) + 1 y  1 .
( j)

Generalizing this, we have the following interesting result for the An .

( j)

Theorem 1.4.5 For each j and n, An with the additional parameters 1 , . . . , n satisfy
the linear system
A(yl ) = A(nj) +

n


k ylk , j l j + n.

(1.4.14)

k=1

Proof. Letting l = j + i and multiplying both sides of (1.4.14) by ni and summing

from i = 0 to i = n and invoking (1.4.6), we obtain
n


ni A(y j+i ) = A(nj) +

i=0

n

k=1

n


k
ni y j+i
.

(1.4.15)

i=0

By y j+i = y j i and ck = k , we have

n

i=0

k
ni y j+i
= y j k

n


ni cki = y j k Un (ck ).

(1.4.16)

i=0

The result now follows from this and from the fact that
Un (ck ) = 0, k = 1, . . . , n.

(1.4.17)

Note that the new formulation of (1.4.14) is expressed only in terms of the ym and
without any reference to . So it can be used to dene an extrapolation procedure not only
for ym = y0 m , m = 0, 1, . . . , but also for any sequence {ym } (0, b]. This makes the
Richardson extrapolation more practical and useful in applications, including numerical
integration. We come back to this point in Chapter 2.
( j)
Comparing the equations in (1.4.14) that dene An with the asymptotic expansion
of A(y) for y 0+ that is given in (1.1.2), we realize that the former are obtained from
the latter by truncating the asymptotic expansion at the term n y n , replacing by =,
( j)
A by An , and k by k , k = 1, . . . , n, and nally collocating at y = yl , l = j, j +
1, . . . , j + n. This forms the basis for the different generalizations of the Richardson
extrapolation process in Chapters 3 and 4 of this work.
Finally, the parameters 1 , 2 , . . . , n in (1.4.14) turn out to be approximations to
1 , 2 , . . . , n in (1.1.1) and (1.1.2). In fact, that k tends to k , k = 1, . . . , n, as
j with n xed can be proved rigorously. In Chapter 3, we prove a theorem on
the convergence of the k to the respective k within the framework of a generalized
Richardson extrapolation process, and this theorem covers the present case. Despite this

1.5 Convergence Analysis of the Richardson Extrapolation Process

33

positive result, the use of k as an approximation to k , k = 1, . . . , n, is not recommended in nite-precision arithmetic. When computed in nite-precision arithmetic, the
k turn out to be of very poor quality. This appears to be the case in all generalizations
of the Richardson extrapolation process as well. Therefore, if the k are required, an
altogether different approach needs to be adopted.

1.5 Convergence Analysis of the Richardson Extrapolation Process

Because the Richardson extrapolation process, as described in the preceding section,
produces a two-dimensional array of approximations to A, we may have an innite number of sequences with elements in this array that we may analyze. In particular, each
column or each diagonal in Table 1.3.1 is a bona de sequence. Actually, columns and
diagonals are the most widely studied sequences in Richardson extrapolation and its
various generalizations. In addition, diagonal sequences appear to have the best convergence properties. In this section, we give a thorough analysis of convergence of columns
and diagonals.
1.5.1 Convergence of Columns
( j)

By Theorem 1.3.2 and by (1.1.1), (1.2.4), and (1.2.6), we already know that An A =

O(y j n+1 ) as j , for n = 0, 1, 2. The following theorem gives an asymptotically

( j)
optimal result on the behavior of the sequence {An }
j=0 for arbitrary xed n.
Theorem 1.5.1 Let the function A(y) be as described in the second paragraph of
Section 1.1.
( j)

(i) In case the integer s in (1.1.1) is nite and largest possible, An A has the complete
expansion
A(nj) A =

s


Un (ck )k y j k + O(y j s+1 ) as j ,

k=n+1

= O((n+1 ) j ) as j ,

(1.5.1)
( j)

for 1 n s, where Un (z) is as dened in Theorem 1.4.2 For n s + 1, An A

satises

A(nj) A = O(y j s+1 ) = O((s+1 ) j ) as j .

(1.5.2)
( j)

(ii) In case (1.1.2) holds, that is, (1.1.1) holds for all s = 1, 2, 3, . . . , An A has the
complete asymptotic expansion
A(nj) A

Un (ck )k y j k as j ,

k=n+1

= O((n+1 ) j ) as j .

(1.5.3)
( j)

All these results are valid whether lim y0+ A(y) and lim y An exist or not.

34

1 The Richardson Extrapolation Process

Proof. The result in (1.5.1) can be seen to hold true by induction starting with (1.1.1),
(1.2.4), and (1.2.6). However, we use a different technique to prove (1.5.1). Invoking
Theorem 1.4.2 and (1.1.1), we have
A(nj)

n



ni A +

i=0

= A+

s


k
k y j+i

s+1
O(y j+i
)

as j ,

k=1
s


k=1

n


k
ni y j+i
+ O(y j s+1 ) as j .

(1.5.4)

i=0

The result follows by invoking (1.4.16) and (1.4.17) in (1.5.4). The proof of the rest of
the theorem is easy and is left to the reader.


Corollary 1.5.2 If n+ is the rst nonzero n+i with i 1 in (1.5.1) or (1.5.3), then we
have the asymptotic equality

A(nj) A Un (cn+ )n+ y j n+ as j .

(1.5.5)

The meaning of Theorem 1.5.1 and its corollary is that every column is at least as
good as the one preceding it. In particular, if column n converges, then column n + 1
converges at least as quickly as column n. If column n diverges, then either column n + 1
( j)
converges or it diverges at worst as quickly as column n. In any case, lim j An = A
if n+1 > 0. Finally, if k = 0 for each k = 1, 2, 3, . . . , and lim y0+ A(y) = A, then
each column converges more quickly than all the preceding columns.

1.5.2 Convergence of Diagonals

The convergence theory for the diagonals of Table 1.3.1 turns out to be much more involved than that for the columns. The results pertaining to diagonals show, however, that
diagonals enjoy much better convergence than columns when A(y) satises (1.1.2) or,
equivalently, when A(y) satises (1.1.1) for all s.
( j)
We start by deriving an upper bound on |An A| that is valid for arbitrary j and n.
Theorem 1.5.3 Let the function A(y) be as described in the second paragraph of Section 1.1. Let



s



(1.5.6)
k y k /y s+1 .
s+1 = max  A(y) A
0yy0

k=1

Then, for each j and each n s, we have



n

 ( j)
|cs+1 | + |ci |
 A A s+1 |y s+1 |
,
n
j
|1 ci |
i=1
with ci = i , i = 1, 2, . . . .

(1.5.7)

1.5 Convergence Analysis of the Richardson Extrapolation Process

35

Proof. From (1.5.6), we see that if we dene

Rs (y) = A(y) A

s


k y k ,

(1.5.8)

k=1

then |Rs (y)| s+1 |y s+1 | for all y (0, y0 ]. Now, substituting (1.1.1) in (1.4.5), and
using (1.4.6), and proceeding exactly as in the proof of Theorem 1.5.1, we have on
account of s n
A(nj) = A +

n


ni Rs (y j+i ).

(1.5.9)

i=0

Therefore,
|A(nj) A|

n


|ni | |Rs (y j+i )| s+1

i=0

n


s+1
|ni | |y j+i
|,

(1.5.10)

i=0

which, by y j+i = y j i and cs+1 = s+1 , becomes

|A(nj) A| s+1 |y j s+1 |

n


|ni | |cs+1 |i .

(1.5.11)

i=0

Invoking now Theorem 1.4.3 in (1.5.11), we obtain (1.5.7).

Interestingly, the upper bound in (1.5.7) can be computed numerically since the ym and
the ck are available, provided that s+1 can be obtained. If a bound for s+1 is available,
then this bound can be used in (1.5.7) instead of the exact value.
The upper bound of Theorem 1.5.3 can be turned into a powerful convergence theorem
for diagonals, as we show next.
Theorem 1.5.4 In Theorem 1.5.3, assume that
i+1 i d > 0 for all i, with d xed.

(1.5.12)

(i) If the integer s in (1.1.1) is nite and largest possible, then, whether lim y0+ A(y)
exist or not,
A(nj) A = O((s+1 )n ) as n .

(1.5.13)

(ii) In case (1.1.2) holds, that is, (1.1.1) holds for all s = 0, 1, 2, . . . , for each xed
( j)
j, the sequence {An }
n=0 converges to A whether lim y0+ A(y) exists or not. We
have at worst
A(nj) A = O(n ) as n , for every > 0.

(1.5.14)

(iii) Again in case (1.1.2) holds, if also k y0k = O(ek ) as k for some < 2
and 0, then the result in (1.5.14) can be improved as follows: For any  > 0
such that +  < 1, there exists a positive integer n 0 that depends on , such that

|A(nj) A| ( + )dn

/2

for all n n 0 .

(1.5.15)

36

1 The Richardson Extrapolation Process

Proof. For the proof of part (i), we start by rewriting (1.5.7) in the form


n

 ( j)
1 + |ci /cs+1 |
 A A s+1 |y s+1 | |cs+1 |n
, n s.
n
j
|1 ci |
i=1

(1.5.16)

By (1.5.12), we have |ci+1 /ci | d < 1, i = 1, 2, . . . . This implies that the innite



ci and hence the innite products i=1
|1 ci | converge absolutely, which
series i=1
k
guarantees that infk i=1 |1 ci | is bounded away from zero. Again, by (1.5.12), we
have |ci /cs+1 | (is1)d for all i s + 1 so that
n


(1 + |ci /cs+1 |)

i=1

s+1


(1 + |ci /cs+1 |)

i=1

<

s+1


ns1


(1 + id )

i=1

(1 + |ci /cs+1 |)

i=1

(1 + id ) K s < .

(1.5.17)

i=1

Consequently, (1.5.16) gives

A(nj) A = O(|cs+1 |n ) as n ,

(1.5.18)

which is the same as (1.5.13). This proves part (i).

To prove part (ii), we observe that when s takes on arbitrary values in (1.1.1), (1.5.13)
is still valid because n there. The result in (1.5.14) now follows from the fact that
s+1 + as s .
For the proof of part (iii), we start by rewriting (1.5.7) with s = n in the form



n
n
1 + |cn+1 /ci |
n+1
( j)
.
(1.5.19)
|ci |
|An A| n+1 |y j |
|1 ci |
i=1
i=1
Again, by (1.5.12), we have |cn+1 /ci | (ni+1)d , so that
n
n




(1 + |cn+1 /ci |)
(1 + id ) <
(1 + id ) K  < .
i=1

i=1

(1.5.20)

i=1

Thus, the product inside the second pair of parentheses in (1.5.19) is bounded in n. Also,
from the fact that i 1 + (i 1)d, there follows
n


|ci | =

n
i=1

i

n1 +dn(n1)/2 .

(1.5.21)

i=1

Invoking now the condition on the growth rate of the k , the result in (1.5.15) follows.


Parts (i) and (iii) of this theorem are essentially due to Bulirsch and Stoer [43], while
part (ii) is from Sidi [298] and [301].
( j)
The proof of Theorem 1.5.4 and the inequality in (1.5.19) suggest that An A is
n
O( i=1 |ci |) as n for all practical purposes. More realistic information on the
n
( j)
convergence of An as n can be obtained by analyzing the product i=1
|ci |
carefully.

1.6 Stability Analysis of the Richardson Extrapolation Process

37

Let us return to the case in which (1.1.2) is satised. Clearly, part (ii) of Theorem 1.5.4
says that all diagonal sequences converge to A superlinearly in the sense that, for xed
( j)
j, An A tends to 0 as n like en for every > 0. Part (iii) of Theorem 1.5.4
( j)
says that, under suitable growth conditions on the k , An A tends to 0 as n
2
like en for some > 0. These should be compared with Theorem 1.5.1 that says that
column sequences, when they converge, do so only linearly, in the sense that, for xed
( j)
n, An A tends to 0 as j precisely like (n+ ) j for some integer 1. Thus,
the diagonals have much better convergence than the columns.

1.6 Stability Analysis of the Richardson Extrapolation Process

From the discussion on stability in Section 0.5, it is clear that the propagation of errors
( j)
( j)
in the A(ym ) into An is controlled by the quantity n , where
n( j) =

n


( j)

|ni | =

i=0

n


|ni |

i=0

n

1 + |ci |
i=1

|1 ci |

(1.6.1)

that turns out to be independent of j in the present case. In view of Denition 0.5.1, we
have the following positive result.
Theorem 1.6.1
(i) The process that generates {An }
j=0 is stable in the sense that
( j)

sup n( j) =
j

n


|ni | < .

(1.6.2)

i=0

(ii) Under the condition (1.5.12) and with ci = i , i = 1, 2, . . . , we have

lim sup
n

n


|ni |

i=0


1 + |ci |
i=1

|1 ci |

< .

(1.6.3)

Consequently, the process that generates {An }

n=0 is also stable in the sense that
( j)

sup n( j) < .

(1.6.4)

Proof. The validity of (1.6.2) is obvious. That (1.6.3) is valid follows from (1.4.10)

in Theorem 1.4.3 and the absolute convergence of the innite products i=1
|1 ci |
that was demonstrated in the proof of Theorem 1.5.4. The validity of (1.6.4) is a direct
consequence of (1.6.3).


Remark. In case i all have the same imaginary part, (1.6.3) is replaced by
lim

n

i=0

|ni | =


1 + |ci |
i=1

|1 ci |

(1.6.5)

38

1 The Richardson Extrapolation Process

Table 1.7.1: Richardson extrapolation on the Zeta function series with z = 1 + 10i.
( j)
( j)
Here E n = |An A|/|A|
( j)

( j)

( j)

( j)

( j)

( j)

( j)

E0

E1

E2

E3

E4

E5

E6

0
1
2
3
4
5
6
7
8
9
10
11
12

2.91D 01
1.38D 01
8.84D 02
7.60D 02
7.28D 02
7.20D 02
7.18D 02
7.17D 02
7.17D 02
7.17D 02
7.17D 02
7.17D 02
7.17D 02

6.46D 01
2.16D 01
7.85D 02
3.57D 02
1.75D 02
8.75D 03
4.38D 03
2.19D 03
1.10D 03
5.49D 04
2.75D 04
1.37D 04

6.94D 01
1.22D 01
2.01D 02
4.33D 03
1.04D 03
2.57D 04
6.42D 05
1.60D 05
4.01D 06
1.00D 06
2.50D 07

2.73D 01
2.05D 02
1.06D 03
5.97D 05
3.62D 06
2.24D 07
1.40D 08
8.74D 10
5.46D 11
3.41D 12

2.84D 02
8.03D 04
1.28D 05
1.89D 07
2.89D 09
4.50D 11
7.01D 13
1.10D 14
1.71D 16

8.93D 04
8.79D 06
4.25D 08
1.67D 10
6.47D 13
2.52D 15
9.84D 18
3.84D 20

8.82D 06
2.87D 08
4.18D 11
4.37D 14
4.31D 17
4.21D 20
4.11D 23

This follows from the fact that now

n

i=0

|ni | =

n

1 + |ci |
i=1

|1 ci |

(1.6.6)

n
as stated in Theorem 1.4.3. Also, i=0
|ni | is an increasing function of for i > 0,
i = 1, 2, . . . . We leave the verication of this fact to the reader.
( j)
As can be seen from (1.6.1), the upper bound on n is inversely proportional to
n
the product i=1 |1 ci |. Therefore, the processes that generate the row and column
sequences will be increasingly stable from the numerical viewpoint when the ck are as
far away from unity as possible in the complex plane. The existence of even a few of
( j)
the ck that are too close to unity may cause n to be very large and the extrapolation
processes to be prone to roundoff even though they are stable mathematically. Note that
we can force the ck to stay away from unity by simply picking small enough. Let
( j)
us also observe that the upper bound on |An A| given in Theorem 1.5.3 is inversely
n
( j)
proportional to i=1 |1 ci | as well. It is thus very interesting that, by forcing n to
be small, we are able to improve not only the numerical stability of the approximations
( j)
An , but their mathematical quality too.

1.7 A Numerical Example: Richardson Extrapolation

on the Zeta Function Series
Let us consider the Riemann Zeta function series considered in Example 1.1.4. We
apply the Richardson extrapolation process to A(y), with y, A(y), and the k exactly
as in Example 1.1.4, and with y0 = 1, yl = y0 l , l = 1, 2, . . . , and = 1/2. Thus,
A(yl ) = A2l , l = 0, 1, . . . .
( j)
Table 1.7.1 contains the relative errors |An A|/|A|, 0 n 6, for (z) with z =
1 + 10i, and allows us to verify the result of Theorem 1.5.1 concerning column sequences
( j+1)
( j)
numerically. For example, it is possible to verify that E n /E n tends to |cn+1 | = n+1

1.8 The Richardson Extrapolation as a Summability Method

39

Table 1.7.2: Richardson extrapolation on the Zeta function series with

z = 2 (convergent), z = 1 + 10i (divergent but bounded), and z = 0.5
( j)
( j)
(divergent and unbounded). Here E n (z) = |An A|/|A|
n

E n(0) (2)

E n(0) (1 + 10i)

E n(0) (0.5)

0
1
2
3
4
5
6
7
8
9
10
11
12

3.92D 01
8.81D 02
9.30D 03
3.13D 04
5.33D 06
3.91D 08
1.12D 10
1.17D 13
4.21D 17
5.02D 21
1.93D 25
2.35D 30
9.95D 33

2.91D 01
6.46D 01
6.94D 01
2.73D 01
2.84D 02
8.93D 04
8.82D 06
2.74D 08
2.67D 11
8.03D 15
7.38D 19
2.05D 23
1.70D 28

1.68D + 00
5.16D 01
7.92D 02
3.41D 03
9.51D 05
1.31D 06
7.60D 09
1.70D 11
1.36D 14
3.73D 18
3.36D 22
9.68D 27
8.13D 30

as j . For z = 1 + 10i we have |c1 | = 1, |c2 | = 1/2, |c3 | = 1/22 , |c4 | = 1/24 ,
|c5 | = 1/26 , |c6 | = 1/28 , |c7 | = 1/210 . In particular, the sequence of the partial sums
{An }
n=0 that forms the rst column in the Romberg table diverges but is bounded.

Table 1.7.2 contains the relative errors in the diagonal sequence {A(0)
n }n=0 for (z)
with z = 2 (convergent series), z = 1 + 10i (divergent but bounded series), and z = 0.5
(divergent and unbounded series). The rate of convergence of this sequence in every case
is remarkable.
Note that the results of Tables 1.7.1 and 1.7.2 have all been obtained in quadrupleprecision arithmetic.

1.8 The Richardson Extrapolation as a Summability Method

( j)

In view of the fact that An is of the form described in Theorem 1.4.2 and in view of the
brief description of linear summability methods given in Section 0.3, we realize that the
Richardson extrapolation process is a summability method for both its row and column
sequences. Our purpose now is to establish the regularity of the related summability
methods as these are applied to arbitrary sequences {Bm } and not only to {A(ym )}.

1.8.1 Regularity of Column Sequences

Let us consider the polynomials Un (z) dened in (1.4.4). From (1.4.5), the column
( j)
sequence {An }
j=0 with xed n is that generated by the linear summability method
whose associated matrix M = [ jk ]
j,k=0 [cf. (0.3.11)], has elements given by
jk = 0 for 0 k j 1 and k j + n + 1, and
j, j+i = ni for 0 i n, j = 0, 1, 2, . . . .

(1.8.1)

40

1 The Richardson Extrapolation Process

Thus, the matrix M is the following upper triangular band matrix with band width n + 1:

n0 n1 nn 0 0 0
0 n0 n1 nn 0 0

M =
0 0 n0 n1 nn 0 .

Let us now imagine that this summability method is being applied to an ar

bitrary sequence {Bm } to produce the sequence {Bm } with B j =
k=0 jk Bk =

n
i=0 ni B j+i , j = 0, 1, . . . . From (1.4.6), (1.6.2), and (1.8.1), we see that all the
conditions of Theorem 0.3.3 (SilvermanToeplitz theorem) are satised. Thus, we have
the following result.
Theorem 1.8.1 The summability method whose matrix M is as in (1.8.1) and generates
( j)
also the column sequence {An }
every convergent sequence
j=0 is regular. Thus, for



{Bm }, the sequence {Bm } generated from it through {Bm } =
k=0 mk Bk , m = 0, 1, . . . ,
converges as well and limm Bm = limm Bm .

1.8.2 Regularity of Diagonal Sequences

Let us consider again the polynomials Un (z) dened in (1.4.4). From (1.4.5), the diagonal
( j)
sequence {An }
n=0 with xed j is that generated by the linear summability method whose
associated matrix M = [nk ]
n,k=0 has elements given by
nk = 0 for 0 k j 1 and k j + n + 1, and
n, j+i = ni for 0 i n, n = 0, 1, 2, . . . .

(1.8.2)

Thus, the matrix M is the following shifted lower triangular matrix with zeros in its rst
j columns:

0 0 00 0 0 0
0 0 10 11 0 0

M =
0 0 20 21 22 0 .

Let us imagine that this summability method is being applied to an arbitrary se


n
quence {Bm } to produce the sequence {Bm } with Bn =
k=0 nk Bk =
i=0 ni B j+i ,
n = 0, 1, . . . . We then have the following result.
Theorem 1.8.2 The summability method whose matrix M is as in (1.8.2) and generates
( j)
also the diagonal sequence {An }
n=0 is regular provided (1.5.12) is satised. Thus, for
every convergent sequence {Bm }, the sequence {Bm } generated from it through Bm =



k=0 mk Bk , m = 0, 1, . . . , converges as well and limm Bm = limm Bm .
Proof. Obviously, conditions (i) and (iii) of Theorem 0.3.3 (SilvermanToeplitz theorem)
are satised by (1.4.6) and (1.6.4), respectively. To establish that condition (ii) is satised,

1.9 The Richardson Extrapolation Process for Innite Sequences

41

it is sufcient to show that

lim ni = 0 for nite i = 0, 1, . . . .

(1.8.3)

We do
induction on i. From (1.4.4), we have that n0 =
n this  by
n
n
/
c
c ). Under (1.5.12),
(1)n
i=1 i
i=1 (1
i=1 (1 ci ) has a nonzero limit
ni
for n and limn i=1 ci = 0 as shown previously. Thus, (1.8.3) holds for
i = 0. Let us assume that (1.8.3) holds for i 1. Now, from (1.6.4), |ni | is bounded in
n for each i. Also limn cn = 0 from (1.5.12). Invoking in (1.4.8) these facts and the
induction hypothesis, (1.8.3) follows. This completes the proof.

1.9 The Richardson Extrapolation Process for Innite Sequences

Let the innite sequence {Am } be such that
Am = A +

s


m
k ckm + O(cs+1
) as m ,

(1.9.1)

k=1

where ck = 0, k = 1, 2, . . . , and |c1 | > |c2 | > > |cs+1 |. If (1.9.1) holds for all s =
1, 2, . . . , with limk ck = 0, then we have the true asymptotic expansion
Am A +

k ckm as m .

(1.9.2)

k=0

We assume that the ck are known. We do not assume any knowledge of the k , however.
It should be clear by now that the Richardson extrapolation process of this chapter can
be applied to obtain approximations to A, the limit or antilimit of {Am }. It is not difcult
( j)
to see that all of the results of Sections 1.31.8 pertaining to the An apply to {Am } with
no changes, provided the following substitutions are made everywhere: A(ym ) = Am ,
k = ck , y0 = 1, and ymk = ckm . In addition, s+1 should now be dened by



s



m 
s+1 = max Am A
k ckm /cs+1
, s = 1, 2, . . . .
m

k=1

We leave the verication of these claims to the reader.

2
Additional Topics in Richardson Extrapolation

2.1 Richardson Extrapolation with Near Geometric and Harmonic {yl }

In Theorem 1.4.5, we showed that the Richardson extrapolation process can be dened
via the linear systems of equations in (1.4.14) and that this denition allows us to use
( j)
arbitrary {yl }. Of course, with arbitrary {yl }, the An will have different convergence
l
and stability properties than those with yl = y0 (geometric {yl }) that we discussed at
length in Chapter 1. In this section, we state without proof the convergence and stability
( j)
properties of the column sequences {An }
j=0 for two essentially different types of {yl }.
In both cases, the stated results are best possible asymptotically.
The results in the following theorem concerning the column sequences in the extrapolation table with near geometric {yl } follow from those given in Sidi [290], which are
the subject of Chapter 3.
Theorem 2.1.1 Let A(y) be exactly as in Section 1.1, and choose {yl } such that
liml (yl+1 /yl ) = (0, 1). Set ck = k for all k. If n+ is the rst nonzero n+i
with i 1, then


n
cn+ ci

A(nj) A
n+ y j n+ as j ,
1

c
i
i=1
( j)

and lim j n exists and

lim n( j) =

n

i=0

|ni |

n

1 + |ci |
i=1

|1 ci |

n


ni z i

i=0

n

z ci
,
1
ci
i=1

with equality when all the k have the same imaginary part.
The results of the next theorem that concern harmonic {yl } have been given recently in
Sidi [305]. Their proof is achieved by using Lemma 16.4.1 and the technique developed
following it in Section 16.4.
Theorem 2.1.2 Let A(y) be exactly as in Section 1.1, and choose
yl =

c
, l = 0, 1, . . . , for some c, , q > 0.
(l + )q
42

2.2 Polynomial Richardson Extrapolation

43

If n+ is the rst nonzero n+i with i 1, then





n
i n+ n+ qn+
( j)
as j ,
An A n+
c
j
i
i=1
and
n( j)


n

1 

|i |

i=1

2j
q

n
as j .
( j)

Thus, provided n+1 > 0, there holds lim j An = A in both theorems, whether
lim y0+ A(y) exists or not. Also, each column is at least as good as the one preceding
it. Finally, the column sequences are all stable in Theorem 2.1.1. They are unstable in
( j)
Theorem 2.1.2 as lim j n = . For proofs and more details on these results, see
Sidi [290], [305].
Note that the Richardson extrapolation process with yl as in Theorem 2.1.2 has been
used successfully in multidimensional integration of singular integrands. When used
with high-precision oating-point arithmetic, this strategy turns out to be very effective
despite its being unstable. For these applications, see Davis and Rabinowitz [63] and
Sidi [287].

2.2 Polynomial Richardson Extrapolation

In this section, we give a separate treatment of polynomial Richardson extrapolation that
we mentioned in passing in the preceding chapter. This method deserves independent
treatment as it has numerous applications and a special theory.
The problem we want to solve is that of determining lim y0 A(y) = A, where A(y),
for some positive integer s, satises
A(y) A +

s


k y r k + O(y r (s+1) ) as y 0.

(2.2.1)

k=1

Here k are constants independent of y, and r > 0 is a known constant. The k are not
necessarily known and are not of interest. We assume that A(y) is dened (i) either for
y 0 only, in which case r may be arbitrary and y 0 in (2.2.1) means y 0+,
(ii) or for both y 0 and y 0, in which case r may be only a positive integer and
y 0 from both sides in (2.2.1).
In case (2.2.1) holds for every s, A(y) will have the genuine asymptotic expansion
A(y) A +

k y r k as y 0.

(2.2.2)

k=1

We now use the alternative denition of the Richardson extrapolation process that was
given in Section 1.4. For this, we choose {yl } such that yl are distinct and satisfy
y0 > y1 > > 0; lim yl = 0, if A(y) dened for y 0 only,
l

|y0 | |y1 | ; lim yl = 0, if A(y) dened for y 0 and y 0. (2.2.3)

l

44

2 Additional Topics in Richardson Extrapolation

( j)

Following that, we dene the approximation An to A via the linear equations

A(yl ) = A(nj) +

n


k ylr k , j l j + n.

(2.2.4)

k=1

This system has a very elegant solution that goes through polynomial interpolation. For
convenience, we set t = y r , a(t) = A(y), and tl = ylr everywhere. Then the equations
in (2.2.4) assume the form
a(tl ) = A(nj) +

n


k tlk , j l j + n.

(2.2.5)

k=1
( j)

It is easy to see that An = pn, j (0), where pn, j (t) is the polynomial in t of degree at
most n that interpolates a(t) at the points tl , j l j + n.
Now the polynomials pn, j (t) can be computed recursively by the NevilleAitken
interpolation algorithm (see, for example, Stoer and Bulirsch [326]) as follows:
pn, j (t) =

(t t j+n ) pn1, j (t) (t t j ) pn1, j+1 (t)

, p0, j (t) = a(t j ). (2.2.6)
t j t j+n

Letting t = 0 in this formula, we obtain the following elegant algorithm, one of the most
useful algorithms in extrapolation, due to Bulirsch and Stoer [43]:
Algorithm 2.2.1
( j)

1. Set A0 = a(t j ), j = 0, 1, . . . .
( j)
2. Compute An by the recursion
( j+1)

A(nj) =

( j)

t j An1 t j+n An1

, j = 0, 1, . . . , n = 1, 2, . . . .
t j t j+n

From the theory of polynomial interpolation, we have the error formula

a(t) pn, j (t) = a[t, t j , t j+1 , . . . , t j+n ]

n


(t t j+i ),

i=0

where f [x0 , x1 , . . . , xs ] denotes the divided difference of order s of f (x) over the set
of points {x0 , x1 , . . . , xs }. Letting t = 0 in this formula, we obtain the following error
( j)
formula for An :
A(nj) A = (1)n a[0, t j , t j+1 , . . . , t j+n ]

n


t j+i .

(2.2.7)

i=0

We also know that in case f (x) is real and f C s (I ), where I is some interval containing {x0 , x1 , . . . , xs }, then f [x0 , x1 , . . . , xs ] = f (s) ( )/s! for some I . Thus, when
a(t) is real and in C n+1 (I ), where I is an interval that contains all the points tl , l = j,
j + 1, . . . , j + n, and t = 0, (2.2.7) can be expressed as
A(nj) A = (1)n

n
a (n+1) (t j,n ) 
t j+i , for some t j,n I.
(n + 1)! i=0

(2.2.8)

2.2 Polynomial Richardson Extrapolation

45

By approximating a[0, t j , t j+1 , . . . , t j+n ] by a[t j , t j+1 , . . . , t j+n+1 ], which is acceptable, we obtain the practical error estimate
 n


 
( j)


|t j+i | .
|An A| a[t j , t j+1 , . . . , t j+n+1 ]
i=0

Note that if a(t) is real and a (n+1) (t) is known to be of one sign on I , and tl > 0 for all
( j)
l, then the right-hand side of (2.2.8) gives important information about An . Therefore,
(n)
let us assume that, for each n, a (t) does not change sign on I , and that tl > 0 for all l.
( j)
It then follows that (i) if a (n) (t) has the same sign for all n, then An A alternates in
( j)
( j)
sign as a function of n, which means that A is between An and An+1 for all n, whereas
( j)
(ii) if (1)n a (n) (t) has the same sign for all n, then An A is of one sign for all n,
( j)
which implies that An are all on the same side of A.
Without loss of generality, in the remainder of this chapter we assume that A(y) [a(t)]
is real.
Obviously, the error formula in (2.2.8) can be used to make statements on the conver( j)
gence rates of An both as j and as n . For example, it is easy to see that,
for arbitrary tl ,
A(nj) A = O(t j t j+1 t j+n ) as j .

(2.2.9)

Of course, when yl = y0 l for all l, the theory of Chapter 1 applies with k = r k for
all k. Similarly, Theorems 2.1.1 and 2.1.2 apply when liml (yl+1 /yl ) = and yl =
c/(l + )q , respectively, again with k = r k for all k. In all three cases, it is not necessary
( j)
to assume that a(t) is differentiable. For other choices of {tl }, the analysis of the An
turns out to be much more involved. This analysis is the subject of Chapter 8.
( j)
We end this section by presenting a recursive method for computing the n for the
case in which tl > 0 for all l. As before,
A(nj) =

n


( j)

ni a(t j+i ) and n( j) =

i=0

( j+1)

( j)

( j)

( j)

|ni |,

i=0

and from the recursion relation among the

ni =

n


( j)
An

it is clear that

( j)

t j n1,i1 t j+n n1,i

t j t j+n

, i = 0, 1, . . . , n,

( j)

( j)

with 0,0 = 1 and ni = 0 for i < 0 and i > n. From this, we can see that (1)n+i ni >
0 for all n and i when t0 > t1 > > 0. Thus,
( j+1)

( j)

|ni | =

( j)

t j |n1,i1 | + t j+n |n1,i |

t j t j+n

, i = 0, 1, . . . , n.

Summing both sides over i, we nally obtain, for t0 > t1 > > 0,
( j+1)

n( j) =

( j)

t j n1 + t j+n n1

( j)
, j 0, n 1; 0 = 1, j 0.
t j t j+n
( j)

(2.2.10)

Obviously, this recursion for the n can be incorporated in Algorithm 2.2.1 in a straightforward manner.

46

2 Additional Topics in Richardson Extrapolation

Table 2.2.1: Polynomial Richardson extrapolation on the Archimedes method for
( j)
( j)
approximating by inscribed regular polygons. Here E n,i = ( An )/
( j)

( j)

( j)

( j)

( j)

( j)

( j)

E 0,i

E 1,i

E 2,i

E 3,i

E 4,i

E 5,i

E 6,i

0
1
2
3
4
5
6
7
8
9
10

3.63D 01
9.97D 02
2.55D 02
6.41D 03
1.61D 03
4.02D 04
1.00D 04
2.51D 05
6.27D 06
1.57D 06
3.92D 07

1.18D 02
7.78D 04
4.93D 05
3.09D 06
1.93D 07
1.21D 08
7.56D 10
4.72D 11
2.95D 12
1.85D 13

4.45D 05
7.20D 07
1.13D 08
1.78D 10
2.78D 12
4.34D 14
6.78D 16
1.06D 17
1.65D 19

2.42D 08
9.67D 11
3.80D 13
1.49D 15
5.81D 18
2.27D 20
8.86D 23
3.46D 25

2.14D 12
2.12D 15
2.08D 18
2.03D 21
1.99D 24
1.94D 27
1.90D 30

3.32D 17
8.21D 21
2.01D 24
4.91D 28
1.20D 31
6.92D 34

9.56D 23
5.89D 27
3.61D 31
5.35D 34
6.62D 34

Remark. Note that all the above applies to sequences {Am } for which
Am A +

k tmk as m ,

k=1

where tm are distinct and satisfy

|t0 | |t1 | ;

lim tm = 0.

We have only to make the substitution A(ym ) = Am throughout.

Example 2.2.2 We now consider Example 1.1.1 on the method of Archimedes for
approximating . We use the notation of Example 1.1.1. Thus, y = n 1 , hence t =
y 2 = n 2 , and A(y) = a(t) = Sn .
We start with the case of the inscribed regular polygon, for which a(t) is innitely
differentiable everywhere and has the convergent Maclaurin expansion
a(t) = +

k t k ; k = (1)k |k | = (1)k

k=1

(2 )2k+1
, k = 1, 2, . . . .
2[(2k + 1)!]

It can be shown that, for t 1/32 , the Maclaurin series of a(t) and of its derivatives are
alternating Leibnitz series so that (1)r a (r ) (t) > 0 for all t 1/32 .
Choosing yl = y0 l , with y0 = 1/4 and = 1/2, we have tl = 4l2 and A(yl ) =
a(tl ) = S42l Al , l = 0, 1, . . . . By using some trigonometric identities, it can be shown
that, for the case of the inscribed polygon,

2An
, n = 0, 1, . . . .
A0 = 2, An+1 = 

1 + 1 (An /2n+1 )2
( j)

( j)

Table 2.2.1 shows the relative errors E n,i = ( An )/ , 0 n 6, that result from
( j)
applying the polynomial Richardson extrapolation to a(t). Note the sign pattern in E n,i
r (r )
that is consistent with (1) a (t) > 0 for all r 0 and t t0 .

E 0,c

2.73D 01
5.48D 02
1.31D 02
3.23D 03
8.04D 04
2.01D 04
5.02D 05
1.26D 05
3.14D 06
7.84D 07
1.96D 07

0
1
2
3
4
5
6
7
8
9
10

( j)

1.80D 02
8.59D 04
5.05D 05
3.11D 06
1.94D 07
1.21D 08
7.56D 10
4.73D 11
2.95D 12
1.85D 13

( j)

E 1,c

2.86D 04
3.36D 06
4.93D 08
7.59D 10
1.18D 11
1.84D 13
2.88D 15
4.50D 17
7.03D 19

( j)

E 2,c

1.12D 06
3.29D 09
1.20D 11
4.63D 14
1.80D 16
7.03D 19
2.75D 21
1.07D 23

( j)

E 3,c

1.10D 09
8.03D 13
7.35D 16
7.07D 19
6.87D 22
6.71D 25
6.55D 28

( j)

E 4,c

2.68D 13
4.90D 17
1.12D 20
2.70D 24
6.56D 28
1.60D 31

( j)

E 5,c

1.63D 17
7.48D 22
4.28D 26
2.57D 30
1.53D 34

( j)

E 6,c

Table 2.2.2: Polynomial Richardson extrapolation on the Archimedes method for approximating by circumscribing regular
( j)
( j)
polygons. Here E n,c = ( An )/

48

2 Additional Topics in Richardson Extrapolation

Table 2.2.3: Polynomial Richardson extrapolation on the
(0)
(0)
and E n,c
Archimedes method for approximating . Here E n,i
are exactly as in Table 2.2.1 and Table 2.2.2 respectively
n

(0)
E n,i

(0)
E n,c

0
1
2
3
4
5
6
7
8
9

3.63D 01
1.18D 02
4.45D 05
2.42D 08
2.14D 12
3.32D 17
9.56D 23
5.31D 29
4.19D 34
5.13D 34

2.73D 01
+1.80D 02
2.86D 04
+1.12D 06
1.10D 09
+2.68D 13
1.63D 17
+2.49D 22
9.51D 28
+8.92D 34

We have an analogous situation for the case of the circumscribing regular polygon. In
this case too, a(t) is innitely differentiable for all small t and has a convergent Maclaurin
expansion for t 1/32 :
a(t) = +


k=1

k t k ; k =

(1)k 4k+1 (4k+1 1)B2k+2 2k+1

> 0, k = 1, 2, . . . .
(2k + 2)!

From this expansion it is obvious that a(t) and all its derivatives are positive for t 1/32 .
Choosing the tl exactly as in the previous case, we now have
A0 = 4, An+1 =

1+

2An
1 + (An /2n+2 )2
( j)

, n = 0, 1, . . . .

( j)

Table 2.2.2 shows the relative errors E n,c = ( An )/ that result from applying
( j)
the polynomial Richardson extrapolation to a(t). Note the sign pattern in E n,c that is
(r )
consistent with a (t) > 0 for all r 0 and t t0 .

Finally, in Table 2.2.3 we give the relative errors in the diagonal sequences {A(0)
n }n=0
for both the inscribed and circumscribing polygons. Note the remarkable rates of convergence.
In both cases, we are able to work with sequences {Am }
m=0 whose computation involves only simple arithmetic operations and square roots.
Note that the results of Tables 2.2.12.2.3 have all been obtained in quadrupleprecision arithmetic.
2.3 Application to Numerical Differentiation
The most immediate application of the polynomial Richardson extrapolation is to numerical differentiation. It was suggested by Rutishauser [245] about four decades ago.
This topic is treated in almost all books on numerical analysis. See, for example, Ralston
and Rabinowitz [235], Henrici [130], and Stoer and Bulirsch [326].
Two approaches to numerical differentiation are discussed in the literature: (i) polynomial interpolation followed by differentiation, and (ii) application of the polynomial Richardson extrapolation to a sequence of rst-order divided differences.

2.3 Application to Numerical Differentiation

49

Although the rst approach is the most obvious for differentiation of numerical data, the
second is recommended, and even preferred, for differentiation of functions that can be
evaluated everywhere in a given interval.
We give a slightly generalized version of the extrapolation approach and show that the
approximations produced by this approach can also be obtained by differentiating some
suitable polynomials of interpolation to f (x). Although this was known to be true for
some simple special cases, the two approaches were not known to give identical results
in general. In view of this fact, we reach the interesting conclusion that the extrapolation
approach has no advantage over differentiation of interpolating polynomials, except for
the simple and elegant algorithms that implement it. We show that these algorithms can
also be obtained by differentiating the NevilleAitken interpolation formula.
The material here is taken from Sidi [304], where additional problems are also
discussed.
Let f (x) be a given function that we assume to be in C (I ) for simplicity. Here, I is
some interval. Assume that we wish to approximate f  (a), where a I .
Let us rst approximate f  (a) by the rst-order divided difference (h) =
[ f (a + h) f (a)]/ h. By the fact that the Taylor series of f (x) at a, whether convergent
or not, is also its asymptotic expansion as x a, we have


f (k+1) (a) k
h as h 0.
(h) f  (a) +
(k + 1)!
k=1
Therefore, we can apply the polynomial Richardson extrapolation to (h) with an arbitrary sequence of distinct h m satisfying
|h 0 | |h 1 | ; a + h m I, m = 0, 1, . . . ;

lim h m = 0.

(Note that we do not require the h m to be of the same sign.) We obtain

( j)

A0 = (h j ), j = 0, 1, . . . ,
( j+1)

A(nj) =

( j)

h j An1 h j+n An1

, j = 0, 1, . . . , n = 1, 2, . . . .
h j h j+n

(2.3.1)

The following is the rst main result of this section.

Theorem 2.3.1 Let xm = a + h m , m = 0, 1, . . . , and denote by Q n, j (x) the polynomial
of degree n + 1 that interpolates f (x) at a and x j , x j+1 , . . . , x j+n . Then
Q n, j (a) = A(nj) for all j, n.
Hence
A(nj) f  (a) = (1)n

f (n+2) (n, j )
h j h j+1 h j+n , for some n, j I. (2.3.2)
(n + 2)!

Proof. First, we have A0 = (h j ) = Q 0, j (a) for all j, as can easily be shown. Next,
from the NevilleAitken interpolation algorithm in (2.2.6), we have
( j)

Q n, j (x) =

(x x j+n )Q n1, j (x) (x x j )Q n1, j+1 (x)

,
x j x j+n

50

2 Additional Topics in Richardson Extrapolation

which, upon differentiating at x = a, gives

Q n, j (a) =

h j Q n1, j+1 (a) h j+n Q n1, j (a)

h j h j+n

Comparing this with (2.3.1), and noting that Q n, j (a) and An satisfy the same recursion
relation with the same initial values, we obtain the rst result. The second is merely
the error formula that results from differentiating the interpolation polynomial Q n, j (x)
at a.


( j)

Special cases of the result of Theorem 2.3.1 have been known for equidistant xi . See,
for example, Henrici [130].
The second main result concerns the rst-order centered difference 0 (h) =
[ f (a + h) f (a h)]/(2h), for which we have
0 (h) f  (a) +


f (2k+1) (a) 2k
h as h 0.
(2k + 1)!
k=1

We can now apply the polynomial Richardson extrapolation to 0 (h) with an arbitrary
sequence of distinct positive h m that satisfy
h 0 > h 1 > ; a h m I, m = 0, 1, . . . ;

lim h m = 0.

We obtain
( j)

B0 = 0 (h j ), j = 0, 1, . . . ,
( j+1)

Bn( j)

( j)

h 2j Bn1 h 2j+n Bn1

h 2j h 2j+n

, j = 0, 1, . . . , n = 1, 2, . . . .

(2.3.3)

Theorem 2.3.2 Let xm = a h m , m = 0, 1, . . . , and denote by Q n, j (x) the

polynomial of degree 2n + 2 that interpolates f (x) at the points a and x j ,
x( j+1) , . . . , x( j+n) . Then
Q n, j (a) = Bn( j) for all j, n.
Hence
Bn( j) f  (a) = (1)n+1

f (2n+3) (n, j )
(h j h j+1 h j+n )2 , for some n, j I. (2.3.4)
(2n + 3)!

Proof. First, we have B0 = 0 (h j ) = Q 0, j (a) for all j, as can easily be shown. Next,
the Q n, j (x) satisfy the following extension of the NevilleAitken interpolation algorithm
(which seems to be new):
( j)

Q n, j (x) =

(x x j+n )(x x( j+n) )Q n1, j (x) (x x j )(x x j )Q n1, j+1 (x)

.
h 2j h 2j+n

2.3 Application to Numerical Differentiation

51

Upon differentiating this equality at x = a, we obtain

Q n, j (a) =

h 2j Q n1, j+1 (a) h 2j+n Q n1, j (a)

h 2j h 2j+n

Comparing this with (2.3.3), and noting that Q n, j (a) and An satisfy the same recursion
relation with the same initial values, we obtain the rst result. The second is merely
the error formula that results from differentiating the interpolation polynomial Q n, j (x)
at a.


( j)

We can extend the preceding procedure to the approximation of the second derivative
f  (a). Let us use (h) = [ f (a + h) 2 f (a) + f (a h)]/ h 2 , which satises
(h) f  (a) + 2


f (2k+2) (a) 2k
h as h 0.
(2k + 2)!
k=1

We can now apply the polynomial Richardson extrapolation to (h) with an arbitrary
sequence of distinct positive h m that satisfy
h 0 > h 1 > ; a h m I, m = 0, 1, . . . ;

lim h m = 0.

We obtain
( j)

C0 = (h j ), j = 0, 1, . . . ,
( j+1)

Cn( j) =

( j)

h 2j Cn1 h 2j+n Cn1

h 2j h 2j+n

, j = 0, 1, . . . , n = 1, 2, . . . .

(2.3.5)

Theorem 2.3.3 Let xi and Q n, j (x) be exactly as in Theorem 2.3.2. Then

Q n, j (a) = Cn( j) for all j, n.
Hence
Cn( j) f  (a) = (1)n+1 2

f (2n+4) (n, j )
(h j h j+1 h j+n )2 , for some n, j I. (2.3.6)
(2n + 4)!

The proof can be carried out exactly as that of Theorem 2.3.2, and we leave it to the
reader.
In practice, we implement the preceding extrapolation procedures by picking h m =
h 0 m for some h 0 and some (0, 1), mostly = 1/2. In this case, Theorem 1.5.4
( j)
( j)
( j)
guarantees that all three of An f  (a), Bn f  (a), and Cn f  (a) tend to zero
faster than en as n , for every > 0. Under the liberal growth condition that

maxxI | f (k) (x)| = O(ek ) as k , for some < 2 and , they tend to zero as
2
2
n , like n /2 for (h), and like n for 0 (h) and (h). This can also be seen from
(2.3.2), (2.3.4), and (2.3.6). [Note that the growth condition mentioned here covers the
cases in which maxxI | f (k) (x)| = O((k)!) as k , for arbitrary .]
The extrapolation processes, with h l as in the preceding paragraph, are stable, as
follows from Theorem 1.6.1 in the sense that initial errors in the (h m ), 0 (h m ), and
(h m ) are not magnied in the course of the process. Nevertheless, we should be aware

52

2 Additional Topics in Richardson Extrapolation

Table 2.3.1: Errors in numerical differentiation via Richardson extrapolation of

f (x) = 2 1 + x at x = 0. Here h 0 = 1/4 and = 1/2, and centered

differences are being used

8.03D 03
1.97D 03
4.89D 04
1.22D 04
3.05D 05
7.63D 06

5.60D 05
3.38D 06
2.09D 07
1.30D 08
8.15D 10

1.30D 07
1.95D 09
3.01D 11
4.62D 13

8.65D 11
3.33D 13
8.44D 15

4.66D 15
7.11D 15

7.11D 15

of the danger of roundoff dominating the computation eventually. We mentioned in

Example 1.1.2 that, as h tends to zero, both f (a + h) and f (a h) become very close
to f (a) and hence to each other, so that, in nite-precision arithmetic, 0 (h) has fewer
correct signicant gures than f (a h). Therefore, carrying out the computation of
0 (h) beyond a certain threshold value of h is useless. Obviously, this threshold depends
on the accuracy with which f (a h m ) can be computed. Our hope is to obtain a good
approximation Bn(0) to f  (a) while h n is still sufciently larger than the threshold value
of h. This goal seems to be achieved in practice. All this applies to (h) and (h) as
well. The following theorem concerns this subject in the context of the extrapolation of
the rst-order centered difference 0 (h) for computing f  (a). Its proof can be achieved
by induction on n and is left to the reader.
Theorem 2.3.4 Let h m = h 0 m , m = 0, 1, . . . , for some (0, 1), in (2.3.3). Suppose
that f (x), the computed value of f (x) in 0 (h), has relative error bounded by , and
( j)
( j)
denote by B n the Bn obtained from (2.3.3) with the initial conditions
( j)
j ) = [ f (a + h j ) f (a h j )]/(2h j ), j = 0, 1, . . . .
B 0 = (h

Then
| B (nj) Bn( j) | K n  f  h 1
j+n ,
where
Kn =

n

1 + 2i+1
i=1

1 2i

< K =


1 + 2i+1
i=1

1 2i

< ,  f  = max | f (x)|.

|xa|h 0

Let us apply the method just described to the function f (x) = 2 1 + x with 0 (h) and
a = 0. We have f  (0) = 1. We pick h 0 = 1/4 and = 1/2. We use double-precision
( j)
arithmetic in our computations. The errors |Bn f  (0)|, ordered as in Table 1.3.1, are
given in Table 2.3.1. As this function is analytic in (1, +), the convergence results
mentioned above hold.

2.4 Application to Numerical Quadrature: Romberg Integration

1
In Example 1.1.3, we discussed the approximation of the integral I [ f ] = 0 f (x) d x
by the trapezoidal rule T (h) and the midpoint rule M(h). We mentioned there that if

2.4 Application to Numerical Quadrature: Romberg Integration

53

f C [0, 1], then the errors in these approximations have asymptotic expansions in
powers of h 2 , known as EulerMaclaurin expansions. Thus, the polynomial Richardson
extrapolation process can be applied to the numerical quadrature formulas T (h) and
M(h) to obtain good approximations to I [ f ]. Letting Q(h) stand for either T (h) or
M(h), and picking a decreasing sequence {h m }
m=0 from {1, 1/2, 1/3, . . . }, we compute
( j)
the approximations An to I [ f ] as follows:
( j)

A0 = Q(h j ), j = 0, 1, . . . ,
( j+1)

A(nj)

( j)

h 2j An1 h 2j+n An1

h 2j h 2j+n

, j = 0, 1, . . . , n = 1, 2, . . . .

(2.4.1)

This scheme is known as Romberg integration. In the next theorem, we give an error
( j)
expression for An that is valid for arbitrary h m and state convergence results for column
sequences as well.
( j)

Theorem 2.4.1 Let f (x), I [ f ], Q(h), and An be as in the preceding paragraph. Then
the following hold:
(i) There exists a function w(t) C [0, 1] such that w(m 2 ) = Q(m 1 ), m =
1, 2, . . . , for which
A(nj) I [ f ] = (1)n

2
 n
w(n+1) (t j,n ) 
h j+i , for some t j,n (t j+n , t j ),
(n + 1)!
i=0

(2.4.2)

where tm = h 2m for each m. Thus, for arbitrary h m , each column sequence {An }
j=0
converges, and there holds
( j)

A(nj) I [ f ] = O((h j h j+1 h j+n )2 ) as j .

(2.4.3)

(ii) In case h m+1 / h m 1 as m , there holds

A(nj) I [ f ] (1)n

( + 1)n+1
2(n+1+)
wn+1+ h j
as j ,
(n + 1)!

(2.4.4)

where wk = ek [ f (2k1) (1) f (2k1) (0)], ek = B2k /(2k)! for Q(h) = T (h) and ek =
B2k ( 12 )/(2k)! for Q(h) = M(h), and wn+1+ , with 0, is the rst nonzero wk with
k n + 1. Thus, each column converges at least as quickly as the one preceding it.
This holds when h m = 1/(m + 1), m = 0, 1, . . . , in particular.
Proof. From Theorem D.4.1 in Appendix D, Q(h) can be continued to a function w(t)
C [0, 1], such that t = 1/m 2 when h = 1/m and w(m 2 ) = Q(m 1 ), m = 1, 2, . . . .
From this and from (2.2.8), we obtain (2.4.2), and (2.4.3) follows from (2.4.2). Now, from


k
the proof of Theorem D.4.1, it follows that w(t) I [ f ] +
k t as t 0+.
k=1 w

From the fact that w(t) C [0, 1], we also have that w(n+1) (t) k=n+1+ k(k
1) (k n)wk t kn1 as t 0+. This implies that w (n+1) (t) ( + 1)n+1 wn+1+ t
as t 0+. Invoking this in (2.4.2), and realizing that t j,n t j as j , we nally
obtain (2.4.4).

54

2 Additional Topics in Richardson Extrapolation

( j)

If we compute the An as in (2.4.1) by picking h m = h 0 m , where h 0 = 1 and = 1/q,

where q is an integer greater than 1, and if f C 2s+2 [0, 1], then from Theorem 1.5.1
we have
A(nj) I [ f ] = O(2( p+1) j ) as j , p = min{n, s},
and from part (i) of Theorem 1.5.4 (with d = 2 there)
A(nj) I [ f ] = O(2(s+1)n ) as n .
If f C [0, 1], then by part (ii) of Theorem 1.5.4
A(nj) I [ f ] = O(en ) as n , for every > 0.
The upper bound given in (1.5.19) now reads



n
1 + 2i
2
( j)
(2n+2)
|An I [ f ]| |en+1 | max | f
(x)|
2 j(n+1) n +n ,
2i
x[0,1]
1
i=1
where ek = B2k /(2k)! for Q(h) = T (h) and ek = B2k ( 12 )/(2k)! for Q(h) = M(h), and
( j)
this is a very tight bound on |An I [ f ]|. Part (iii) of Theorem 1.5.4 applies (with
2
( j)
d = 2), and we practically have |An I [ f ]| = O(n ) as n , again provided

f C [0, 1] and provided maxx[0,1] | f (k) (x)| = O(ek ) as k for some < 2
and . [Here, we have also used the fact that ek = O((2 )2k ) as k .] As mentioned
before, this is a very liberal growth condition for f (k) (x). For analytic f (x), we have a
growth rate of f (k) (x) = O(k!ek ) as k , which is much milder than the preceding
growth condition. Even a large growth rate such as f (k) (x) = O((k)!) as k for
some > 0 is accommodated by this growth condition. The extrapolation process is
stable, as follows from Theorem 1.6.1, in the sense that initial errors in the Q(h m ) are
not magnied in the course of the process.
Romberg [240] was the rst to propose the scheme in (2.4.1), with = 1/2. A
thorough analysis of this case was given in Bauer, Rutishauser, and Stiefel [20], where
the following elegant expression for the error in case f C 2n+2 [0, 1] and Q(h) = T (h)
is also provided:
A(nj) I [ f ] =

4 j(n+1) B2n+2 (2n+2)

f
( ) for some (0, 1).
2n(n+1) (2n + 2)!

 1Let us apply the Romberg integration with Q(h) = T (h) and = 1/2 to the integral
0 f (x) d x when f (x) = 1/(x + 1) for which I [ f ] = log 2. We use double-precision
( j)
arithmetic in our computations. The relative errors |An I [ f ]|/|I [ f ]|, ordered as in
Table 1.3.1, are presented in Table 2.4.1. As this function is analytic in (1, +), the
convergence results mentioned in the previous paragraph hold.
As we can easily see, when computing Q(h k+1 ) with h m = m , we are using all the
integrand values of Q(h k ), and this is a useful feature of the Romberg integration. On the
other hand, the number of integrand values increases exponentially like 1/k . Thus, when
n
= 1/2, A(0)
n that is computed from Q(h i ), 0 i n, requires 2 integrand evaluations,
so that increasing n by 1 results in doubling the number of integrand evaluations. To keep
this number to a reasonable size, we should work with a sequence {h m } that tends to 0 at

2.5 Rational Extrapolation

55

1
Table 2.4.1: Errors in Romberg integration for 0 (x + 1)1 d x. Here h 0 = 1 and
= 1/2, and the trapezoidal rule is being used
8.20D 02
2.19D 02
5.59D 03
1.41D 03
3.52D 04
8.80D 05

1.87D 03
1.54D 04
1.06D 05
6.81D 07
4.29D 08

3.96D 05
1.04D 06
1.98D 08
3.29D 10

4.29D 07
3.62D 09
1.94D 11

1.96D 09
5.30D 12

3.39D 12

a rate more moderate than 2m . But for Romberg integration there is no sequence {h m }
with h m = m and (1/2, 1). We can avoid this problem by using other types of {h m }.
Thus, to reduce the number of integrand values required to obtain A(0)
n with large n, two
types of sequences {h m } have been used extensively in the literature: (i) h m+1 / h m
for some (0, 1), and (ii) h m = 1/(m + 1). The extrapolation process is stable in the
rst case and unstable in the second. We do not pursue the subject further here, but we
come back to it and analyze it in some detail later in Chapter 8. For more details and
references, we refer the reader to Davis and Rabinowitz [63].

2.5 Rational Extrapolation

Bulirsch and Stoer [43] generalized the polynomial Richardson extrapolation by replacing the interpolating polynomial pn, j (t) of Section 2.2 by an interpolating rational function qn, j (t). The degree of the numerator of qn, j (t) is n/2, the degree of its denominator
is (n + 1)/2, and qn, j (tl ) = a(tl ), j l j + n. The approximations to A, which we
( j)
( j)
now denote Tn , are obtained by setting t = 0 in qn, j (t), that is, Tn = qn, j (0). The
resulting method is called rational extrapolation. Bulirsch and Stoer give the following
( j)
elegant algorithm for computing the resulting Tn :
Algorithm 2.5.2
( j)

( j)

1. Set T1 = 0, T0 = a(t j ), j = 0, 1, . . . .
( j)
2. For j = 0, 1, . . . , and n = 1, 2, . . . , compute Tn recursively from
( j+1)

( j+1)

Tn( j) = Tn1 +

tj
t j+n

( j)

Tn1 Tn1
( j+1)

( j)

Tn1 Tn1
( j+1)

( j+1)

Tn1 Tn2


1

For a detailed derivation of this algorithm, see also Stoer and Bulirsch [326, pp. 67
71]. For more information on this method and its application to numerical integration
and numerical solution of ordinary differential equations, we refer the reader to Bulirsch
and Stoer [44], [45], [46].
( j)
Another approach that is essentially due to Wynn [369] and that produces the T2s can
be derived through the Thiele continued fraction for rational interpolation; see Stoer and
Bulirsch [326, pp. 6367]. Let R2s, j (x) be the rational function in x with degree of

56

2 Additional Topics in Richardson Extrapolation

numerator and denominator equal to s that interpolates f (x) at the points

( j)
x j , x j+1 , . . . , x j+2s . It turns out that 2s limx R2s, j (x) can be computed by the
following recursive algorithm:
( j)

( j)

1 = 0, 0 = f (x j ), j = 0, 1, . . . ,
( j)

( j+1)

k+1 = k1 +

x j+k+1 x j
( j+1)

( j)

, j, k = 0, 1, . . . .

(2.5.5)

( j)

The k are called the reciprocal differences of f (x). A determinantal expression for
( j)
2s is given in Norlund [222, p. 419].
By making the substitution t = x 1 in our problem, we see that q2s, j (x 1 ) is a rational
function of x with degree of numerator and denominator equal to s, and it interpolates
( j)
1
1
1
a(x 1 ) at the points t 1
j , t j+1 , . . . , t j+2s . In addition, limx q2s, j (x ) = T2s . Thus,
( j)
the T2s can be computed via the following algorithm.
Algorithm 2.5.3
( j)

( j)

1. Set r1 = 0, r0 = a(t j ), j = 0, 1, . . . .
( j)
2. Compute rn recursively from
( j)

( j+1)

rk+1 = rk1 +
( j)

1
t 1
j+k+1 t j
( j+1)

rk

( j)

rk

, j, k = 0, 1, . . . .

( j)

Of course, here r2s = T2s for all j and s.

The following convergence theorem is stated in Gragg [105].


( j)
i t i as t 0+, and let Tn be as in Algorithm
Theorem 2.5.4 Let a(t) A + i=1
2.5.2 with positive tl that are chosen to satisfy tl+1 /tl for some (0, 1). Dene


 m m+1 . . . m+r 1 


 m+1 m+2 . . . m+r 


(m)
Hr =  .
.
..
..

 ..
.
.





...
m+r 1

m+r

m+2r 2

Assume that Hr(m) = 0 for m = 0, 1 and all r 0, and dene

(0)
(1)
e2s = Hs+1
/Hs(0) and e2s+1 = Hs+1
/Hs(1) .

Then
Tn( j) A = [en+1 + O(t j )](t j t j+1 t j+n ) as j .
Letting t = h 2 and t j = h 2j , j = 0, 1, . . . , Algorithm 2.5.3 can be applied to the
trapezoidal or midpoint rule approximation Q(h) of Section 2.4. For this application,
see Brezinski [31]. See also Wuytack [367], where a different implementation of rational
extrapolation is presented.

3
First Generalization of the Richardson
Extrapolation Process

3.1 Introduction
In Chapter 1, we considered the Richardson extrapolation process for a sequence {Am }
derived from a function A(y) that satises (1.1.1) or (1.1.2), through Am = A(ym ) with
ym = y0 m , m = 0, 1, . . . . In this chapter, we generalize somewhat certain aspects of
the treatment of Chapter 1 to the case in which the function A(y) has a rather general
asymptotic behavior that also may be quite different from the ones in (1.1.1) or (1.1.2).
In addition, the ym are now arbitrary. Due to the generality of the asymptotic behavior
of A(y) and the arbitrariness of the ym , and under suitable conditions, the approach of
this chapter may serve as a unifying framework within which one can treat the various
extrapolation methods that have appeared over the years. In particular, the convergence
and stability results from this approach may be directly applicable to specic convergence
acceleration methods in some cases. Unfortunately, we pay a price for the generality of
the approach of this chapter: The problems of convergence and stability presented by it
turn out to be very difcult mathematically, especially because of this generality. As a
result, the number of the meaningful theorems that have been obtained and that pertain
to convergence and stability has remained small.
Our treatment here closely follows that of Ford and Sidi [87] and of Sidi [290].
Let A(y) be a function of the discrete or continuous variable y, dened for y (0, b]
for some b > 0. Assume that A(y) has an expansion of the form
A(y) = A +

s


k k (y) + O(s+1 (y)) as y 0+,

(3.1.1)

k=1

where A and the k are some scalars independent of y and {k (y)} is an asymptotic
sequence as y 0+, that is, it satises
k+1 (y) = o(k (y)) as y 0+, k = 1, 2, . . . .

(3.1.2)

Here, A(y) and k (y), k = 1, 2, . . . , are assumed to be known for y (0, b], but the
k are not required to be known. The constant A that is in many cases lim y0+ A(y)
is what we are after. When lim y0+ A(y) does not exist, A is the antilimit of A(y) as
y 0+, and in this case lim y0+ i (y) does not exist at least for i = 1. If (3.1.1) is

57

58

3 First Generalization of the Richardson Extrapolation Process

valid for every s = 1, 2, . . . , then A(y) has the bona de asymptotic expansion
A(y) A +

k k (y) as y 0 + .

(3.1.3)

k=1

Note that the functions A(y) treated in Chapter 1 are particular cases of the A(y)
treated in this chapter with k (y) = y k , k = 1, 2, . . . . In the present case, it is not
assumed that the k (y) have any particular structure.
Denition 3.1.1 Let A(y) be as described above. Pick a decreasing positive sequence
( j)
{ym } (0, b] such that limm ym = 0. Then the approximation An to A, whether A
is the limit or antilimit of A(y) for y 0+, is dened through the linear system
A(yl ) = A(nj) +

n


k k (yl ), j l j + n,

(3.1.4)

k=1

1 , . . . , n being the additional (auxiliary) unknowns. We call this process that generates
( j)
the An as in (3.1.4) the rst generalization of the Richardson extrapolation process.
( j)

Comparing the equations (3.1.4) that dene An with the expansion of A(y) for
y 0+ given in (3.1.3), we realize that the former are obtained from the latter by
( j)
truncating the asymptotic expansion at the term n n (y), replacing by =, A by An ,
and k by k , k = 1, . . . , n, and nally collocating at y = yl , l = j, j + 1, . . . , j + n.
Note the analogy of Denition 3.1.1 to Theorem 1.4.5. In the next section, we show
that, at least formally, this generalization of the Richardson extrapolation process does
perform what is required of it, namely, that it eliminates k (y), k = 1, . . . , n, from the
expansions in (3.1.1) or (3.1.3).
Before we go on, we would like to mention that the formal setting of the rst generalization of the Richardson extrapolation process as given in (3.1.1)(3.1.4) is not new. As
far as is known to us, it rst appeared in Hart et al. [125, p. 39]. It was considered in detail
again by Schneider [259], who also gave the rst recursive algorithm for computation
( j)
of the An . We return to this in Section 3.3.
We would also like to mention that a great many convergence acceleration methods
are dened directly or can be shown to be dened indirectly through a linear system of
equations of the form (3.1.4). [The k (y) in these equations now do not generally form
asymptotic sequences, however.] Consequently, the analysis of this form can be thought
of as a unication of the various acceleration methods, in terms of which their properties
may be classied. As mentioned above, the number of meaningful mathematical results
that follow from this unication is small. More will be said on this in Section 3.7 of
this chapter.
Before we close this section, we would like to give an example of a function A(y) of
the type just discussed that arises in a nontrivial fashion from numerical integration.
Example 3.1.2 Trapezoidal
 1 Rule for Integrals with an Endpoint Singularity Consider the integral I [G] = 0 G(x) d x, where G(x) = x s log x g(x) with s > 1 and
g C [0, 1]. Let h = 1/n, where n is a positive integer. Let us approximate I [G] by

3.2 Algebraic Properties

the (modied) trapezoidal rule T (h) given by


n1
1
T (h) = h
G( j h) + G(1) .
2
j=1

59

(3.1.5)

Then, by a result due to Navot [217] (see also Appendix D), we have the EulerMaclaurin
expansion

T (h) I [G] +

ai h 2i +

i=1

bi h s+i+1 i (h) as h 0,

(3.1.6)

i=0

where
ai =

B2i (2i1)
G
(1), i = 1, 2, . . . ,
(2i)!

bi =

g (i) (0)
, i (h) = (s i) log h  (s i), i = 0, 1, . . . .
i!

(3.1.7)

Here Bk are the Bernoulli numbers, (z) is the Riemann Zeta function, and  (z) =
d
(z).
dz
Obviously, ai and bi are independent of h and depend only on g(x), and i (h) are
independent of g(x). Thus, T (h) is analogous to a function A(y) that satises (3.1.3)
along with (3.1.2) in the following sense: T (h) A(y), h y, and, in case 1 <
s < 0,
 s+i+1
i (h), i = 2k/3, k = 1, 2, 4, 5, 7, 8, . . . ,
h
(3.1.8)
k (y)
k = 3, 6, 9, . . . ,
h 2k/3 ,
Note that k (y) are all known functions.

3.2 Algebraic Properties

( j)

Being the solution to the linear system in (3.1.4), with the help of Cramers rule, An
can be expressed as the quotient of two determinants in the form


 g1 ( j)
g2 ( j) gn ( j)
a( j) 

 g1 ( j + 1) g2 ( j + 1) gn ( j + 1) a( j + 1)




..
..
..
..


.
.
.
.


g ( j + n) g ( j + n) g ( j + n) a( j + n)
1
2
n
( j)
,
(3.2.1)
An =


 g1 ( j)
g2 ( j) gn ( j) 1

 g1 ( j + 1) g2 ( j + 1) gn ( j + 1) 1



..
..
..
.. 

.
.
.
.

g ( j + n) g ( j + n) g ( j + n) 1
1

where, for convenience, we have dened

gk (m) = k (ym ), m = 0, 1, . . . , k = 1, 2, . . . , and a(m) = A(ym ), m = 0, 1, . . . .
(3.2.2)

60

3 First Generalization of the Richardson Extrapolation Process

This notation is used interchangeably throughout this chapter. It seems that Levin [161]
( j)
was the rst to point to the determinantal representation of An explicitly.
The following theorem was given by Schneider [259]. Our proof is different from that
of Schneider, however.
( j)

Theorem 3.2.1 An can be expressed in the form

A(nj) =

n


( j)

ni A(y j+i ),

(3.2.3)

i=0
( j)

with the scalars ni determined by the linear system

n


( j)

ni = 1

i=0
n


(3.2.4)

( j)

ni k (y j+i ) = 0, k = 1, 2, . . . , n.

i=0

Proof. Denoting the cofactor of a( j + i) in the numerator determinant of (3.2.1) by Ni ,

and expanding both the numerator and denominator determinants with respect to their
last columns, we have

n
Ni a( j + i)
( j)

n
,
(3.2.5)
An = i=0
i=0 Ni
from which (3.2.3) follows with
Ni
( j)
ni =
n

r =0

Nr

, i = 0, 1, . . . , n.

(3.2.6)

Thus, the rst of the equations in (3.2.4) is satised. As for the rest of the equations

n
Ni gk ( j + i) = 0 for k = 1, . . . , n, because [gk ( j), gk ( j +
in (3.2.4), we note that i=0
1), . . . , gk ( j + n)]T , k = 1, . . . , n, are the rst n columns of the numerator determinant


in (3.2.1) and Ni are the cofactors of its last column.
( j)

The ni also turn out to be associated with a polynomial that has a form very similar
to (3.2.1). This is the subject of Theorem 3.2.2 that was given originally in Sidi [290].
( j)

Theorem 3.2.2 The ni satisfy

n


( j)

( j)

ni z i =

i=0

Hn (z)
( j)

Hn (1)

(3.2.7)

( j)

where Hn (z) is a polynomial of degree at most n in z dened by



 g1 ( j)
g2 ( j) gn ( j) 1 

 g1 ( j + 1) g2 ( j + 1) gn ( j + 1) z 


Hn( j) (z) = 
..
..
..
..  .

.
.
.
.

g ( j + n) g ( j + n) g ( j + n) z n 
1

(3.2.8)

( j)

3.3 Recursive Algorithms for An

61

Proof. The proof of (3.2.7) and (3.2.8) follows from (3.2.6). We leave out the
details.



n
( j)
( j)
Note the similarity of the expression for An given in (3.2.1) and that for i=0
ni z i
given in (3.2.7) and (3.2.8).
The next result shows that, at least formally, the generalized Richardson extrapolation
( j)
process that generates An eliminates the k (y) terms with k = 1, 2, . . . , n, from the
expansion in (3.1.1) or (3.1.3). We must emphasize though that this is not a convergence
theorem by any means. It is a heuristic justication of the possible validity of (3.1.4) as
a generalized Richardson extrapolation process.
Theorem 3.2.3 Dene
Rs (y) = A(y) A

s


k k (y).

(3.2.9)

 
n
( j)
( j)
ni k (y j+i ) +
ni Rs (y j+i ),

(3.2.10)

k=1

Then, for all n, we have

A(nj) A =

s

k=n+1


n
i=0

i=0

where the summation sk=n+1 is taken to be zero for n s. Consequently, when A(y) =

s
( j)
A + k=0 k k (y) for all possible y, we have An = A for all j 0 and all n s.
Proof. The proof of (3.2.10) can be achieved by combining (3.2.9) with (3.2.3), and then
invoking (3.2.4). The rest follows from (3.2.10) and from the fact that Rs (y) 0 when




A(y) = A + sk=0 k k (y) for all possible y.
As the k (y) are not required to have any particular structure, the determinants given

n
( j)
( j)
ni z i , cannot be expressed in simple terms.
in (3.2.1) and (3.2.7), hence An and i=0
This makes their analysis rather difcult. It also does not enable us to devise algorithms
as efcient as Algorithm 1.3.1, for example.

( j)

3.3 Recursive Algorithms for An

( j)

The simplest and most direct way to compute An is by solving the linear system in
( j)
(3.1.4). It is also possible to devise recursive algorithms for computing all the An that
can be determined from a given number of the a(m) = A(ym ). In fact, there are two
such algorithms in the literature: The rst of these was presented by Schneider [259].
Schneiders algorithm was later rederived using different techniques by Havie [129],
and, after that, by Brezinski [37]. This algorithm has been known as the E-algorithm.
The second one was given by Ford and Sidi [87], and we call it the FS-algorithm. As
shown later, the FS-algorithm turns out to be much less expensive computationally than
the E-algorithm. It also forms an integral part of the W(m) -algorithm of Ford and Sidi
[87] that is used in implementing a further generalization of the Richardson extrapolation
process denoted GREP that is due to Sidi [272]. (GREP is the subject of Chapter 4, and

62

3 First Generalization of the Richardson Extrapolation Process

the W(m) -algorithm is considered in Chapter 7.) For these reasons, we start with the
FS-algorithm.

3.3.1 The FS-Algorithm

We begin with some notation and denitions. We denote an arbitrary given sequence

by b. Thus, a stands for {a(l)}l=0

. Similarly, for each k = 1, 2, . . . , gk stands
{b(l)}l=0

for {gk (l)}l=0 . Finally, we denote the sequence 1, 1, 1, . . . , by I . For arbitrary sequences
u k , k = 1, 2, . . . , and arbitrary integers j 0 and p 1, we dene




u 1 ( j)
u 2 ( j)

u p ( j)



u 2 ( j + 1) u p ( j + 1) 
u 1 ( j + 1)


u 1 ( j) u 2 ( j) u p ( j) = 
.
..
..
..


.
.
.


u ( j + p 1) u ( j + p 1) u ( j + p 1)
1

(3.3.1)
Let now
f p( j) (b) = |g1 ( j) g2 ( j) g p ( j) b( j)|.

(3.3.2)

With this notation, (3.2.1) can be rewritten as

( j)

A(nj) =

f n (a)
( j)

f n (I )

(3.3.3)

We next dene
( j)

G (pj) = |g1 ( j) g2 ( j) g p ( j)|, p 1; G 0 1,

(3.3.4)

and, for an arbitrary sequence b, we let

( j)

p( j) (b) =

f p (b)
( j)

G p+1

(3.3.5)

( j)

We can now reexpress An as

( j)

A(nj) =
( j)

n (a)
( j)

n (I )

(3.3.6)
( j)

The FS-algorithm computes the A p indirectly through the p (b) for various sequences b.
( j)
( j)
( j)
( j)
Because G p+1 = f p (g p+1 ), the determinants for f p (b) and G p+1 differ only in their
last columns. It is thus natural to seek a relation between these two quantities. This can
be accomplished by means of the Sylvester determinant identity given in Theorem 3.3.1.
For a proof of this theorem, see Gragg [106].
Theorem 3.3.1 Let C be a square matrix, and let C denote the matrix obtained by
deleting row and column of C. Also let C  ;  denote the matrix obtained by deleting

( j)

3.3 Recursive Algorithms for An

63

rows and  and columns and  of C. Provided <  and <  ,

det C det C  ;  = det C det C   det C  det C  .

(3.3.7)

If C is a 2 2 matrix, then (3.3.7) holds with C  ;  = 1.

( j)

Applying this theorem to the ( p + 1) ( p + 1) determinant f p (b) in (3.3.2) with

= 1, = p,  =  = p + 1, and using (3.3.4), we obtain
( j+1)

( j+1)

( j)

f p( j) (b)G p1 = f p1 (b)G (pj) f p1 (b)G (pj+1) .

(3.3.8)

This is the desired relation. Upon invoking (3.3.5), and letting

( j)

D (pj)

( j+1)

G p+1 G p1
( j)

( j+1)

Gp Gp

(3.3.9)

(3.3.8) becomes
( j+1)

p( j) (b)

( j)

p1 (b) p1 (b)
( j)

Dp

(3.3.10)

( j)

( j)

From (3.3.6) and (3.3.10), we see that, once the D p are known, the p (a) and
( j)
( j)
p (I ) and hence A p can be computed recursively. Therefore, we aim at developing an
( j)
efcient algorithm for determining the D p . In the absence of detailed knowledge about
the gk (l), which is the case we assume here, we can proceed by observing that, because
( j)
( j)
G p+1 = f p (g p+1 ), (3.3.5) with b = g p+1 reduces to
p( j) (g p+1 ) = 1.

(3.3.11)

Consequently, (3.3.10) becomes

( j+1)

( j)

D (pj) = p1 (g p+1 ) p1 (g p+1 ),

( j)

(3.3.12)
( j)

which permits recursive evaluation of the D p through the quantities p (gk ), k p + 1.

( j)
[Note that p (gk ) = 0 for k = 1, 2, . . . , p.] Thus, (3.3.10) and (3.3.12) provide us with
a recursive procedure for solving the general extrapolation problem of this chapter. We
call this procedure the FS-algorithm. Its details are summarized in Algorithm 3.3.2.
Algorithm 3.3.2 (FS-algorithm)
1. For j = 0, 1, 2, . . . , set
( j)

0 (a) =

a( j)
1
gk ( j)
( j)
( j)
, 0 (I ) =
, 0 (gk ) =
, k = 2, 3, . . . .
g1 ( j)
g1 ( j)
g1 ( j)
( j)

2. For j = 0, 1, . . . , and p = 1, 2, . . . , compute D p by

( j+1)

( j)

D (pj) = p1 (g p+1 ) p1 (g p+1 ),

64

3 First Generalization of the Richardson Extrapolation Process

( j)

( j)

Table 3.3.1: The p (a) and p (I ) arrays

0(0) (a)
0(1) (a)
..
.
0(L) (a)

0(0) (I )
1(0) (a)
..
.
1(L1) (a)

0(1) (I )
..
.
0(L) (I )

..

L(0) (a)

1(0) (I )
..
.
1(L1) (I )

..

L(0) (I )

( j)

and compute p (b) with b = a, I , and b = gk , k = p + 2, p + 3, . . . , by the recursion

( j+1)

p( j) (b)

( j)

p1 (b) p1 (b)
( j)

Dp

and set
( j)

A(pj) =

p (a)
( j)

p (I )

.
( j)

( j)

The reader might nd it helpful to see the relevant p (b) and the D p arranged as
( j)
( j)
in Tables 3.3.1 and 3.3.2, where we have set p,k = p (gk ) for short. The ow of
computation in these tables is exactly as that in Table 1.3.1.
Note that when a(l) = A(yl ), l = 0, 1, . . . , L, are given, then this algorithm enables
( j)
us to compute all the A p , 0 j + p L, that are dened by these a(l).
Remark. Judging from (3.3.5), one may be led to think incorrectly that the FS-algorithm
( j)
requires knowledge of g L+1 for computing all the A p , 0 j + p L, in addition to
g1 , g2 , . . . , g L that are actually needed by (3.1.4). Really, the FS-algorithm employs
g L+1 only for computing D L(0) , which is then used for determining L(0) (a) and L(0) (I ),
(0)
(0)
(0)
and A(0)
L = L (a)/ L (I ). As A L does not depend on g L+1 , in case it is not available,
we can take g L+1 to be any sequence independent of g1 , g2 , . . . , g L , and I . Thus, when
only A(yl ), l = 0, 1, . . . , L, are available, no specic knowledge of g L+1 is required
in using the FS-algorithm, contrary to what is claimed in Brezinski and Redivo Zaglia
[41, p. 62]. As suggested by Osada [226], one can avoid this altogether by computing
the A(0)
n , n = 1, 2, . . . , from
A(0)
n

(1)
(0)
(a) n1
(a)
n1
(1)
(0)
n1
(I ) n1
(I )

(3.3.13)

without having to compute D L(0) . This, of course, follows from (3.3.6) and (3.3.10).

3.3.2 The E-Algorithm

As we have seen, the FS-algorithm is a set of recursion relations that produces the
( j)
( j)
( j)
( j)
quantities p (b) and then gives A p = p (a)/ p (I ). The E-algorithm, on the other

( j)

3.3 Recursive Algorithms for An

( j)

65

( j)

Table 3.3.2: The p (gk ) and D p arrays

(0)
0,2

(0)
0,3

(0)
0,L+1

|
|

(1)
0,L+1
..
.

(0)
1,L+1
..
.

(1)
0,2

D1(0)

(1)
0,3

(0)
1,3

(2)
0,2
..
.

D1(1)
..
.

(1)
1,3
..
.

D2(0)
..
.

(2)
0,3
..
.

(L1)
0,L+1

(L2)
1,L+1

(0)
L1,L+1

(L)
0,2

D1(L1)

(L)
0,3

(L1)
1,3

D2(L2)

(L)
0,L+1

(L1)
1,L+1

(1)
L1,L+1

..

D L(0)

hand, is a different set of recursion relations that produces the quantities

( j)

p( j) (b) =
( j)

f p (b)
( j)

f p (I )

(3.3.14)

( j)

from which we have A p = p (a). Starting from

( j+1)

( j+1)

p( j) (b) = [ f p( j) (b)G p1 ]/[ f p( j) (I )G p1 ],

applying (3.3.8) to both the numerator and the denominator of this quotient, and realizing
( j)
( j)
that G p = f p1 (g p ), we obtain the recursion relation
( j+1)

p( j) (b)

( j)

( j)

( j+1)

p1 (b) p1 (g p ) p1 (b) p1 (g p )
( j)

( j+1)

p1 (g p ) p1 (g p )

(3.3.15)

where b = a and b = gk , k = p + 1, p + 2, . . . . Here the initial conditions are

( j)
0 (b) = b( j).
The E-algorithm, as given in (3.3.15), is not optimal costwise. The following form
that can be found in Ford and Sidi [87] and that is almost identical to that suggested by
Havie [129] earlier, is less costly than (3.3.15):
( j+1)

p( j) (b)
( j)

( j+1)

( j)

( j)

( j)

p1 (b) w p p1 (b)
( j)

1 wp

(3.3.16)

where w p = p1 (g p )/ p1 (g p ).
Let us now compare the operation counts of the FS- and E-algorithms when
a(l), l = 0, 1, . . . , L, are available. We note that most of the computational effort is
( j)
( j)
spent in obtaining the p (gk ) in the FS-algorithm and the p (gk ) in the E-algorithm.
( j)
3
2
The number of these quantities is L /3 + O(L ) for large L. Also, the division by D p in
( j)
the FS-algorithm is carried out as multiplication by 1/D p , the latter quantity being computed only once. An analogous statement can be made about divisions in the E-algorithm
by (3.3.15) and (3.3.16). Thus, we have the operation counts given in Table 3.3.3.
From Table 3.3.3 we see that the E-algorithm is about 50% more expensive than the
FS-algorithm, even when it is implemented through (3.3.16).
Note that the E-algorithm was originally obtained by Schneider [259] by a technique
different than that used here. The technique of this chapter was introduced by Brezinski
[37]. The technique of Havie [129] differs from both.

66

3 First Generalization of the Richardson Extrapolation Process

Table 3.3.3: Operation counts of the FS- and E-algorithms
Algorithm

No. of Multiplications

No. of Additions

No. of Divisions

L 3 /3 + O(L 2 )
2L 3 /3 + O(L 2 )
L 3 + O(L 2 )

L 3 /3 + O(L 2 )
L 3 /3 + O(L 2 )
L 3 /3 + O(L 2 )

O(L 2 )
O(L 2 )
O(L 2 )

FS
E by (3.3.16)
E by (3.3.15)

3.4 Numerical Assessment of Stability

n
( j)
( j)
( j)
ni A(y j+i ),
Let us recall that, by Theorem 3.2.1, An can be expressed as An = i=0

n
( j)
( j)
where the scalars ni satisfy i=0 ni = 1. As a result, the discussion of stability that

n
( j)
( j)
|ni |
we gave in Section 0.5 applies, and we conclude that the quantities n = i=0


( j)
( j)
( j)
n
and !n = i=0 |ni | |A(y j+i )| control the propagation into An of errors (roundoff
and other) in the A(yi ).
( j)
( j)
( j)
If we want to know n and !n , in general, we need to compute the relevant ni .
This can be done by solving the linear system in (3.2.4) numerically. It can also be accomplished by a simple extension of the FS-algorithm. Either way, the cost of computing
( j)
the ni is high and it becomes even higher when we increase n. Thus, computation of
( j)
( j)
n and !n entails a large expense, in general.
( j)
If we are not interested in the exact n but are satised with upper bounds for them,
( j)
we can accomplish this simultaneously with the computation of the An for example,
by the FS-algorithm, and at almost no additional cost.
The following theorem gives a complete description of the computation of both the
( j)
( j)
ni and the upper bounds on the n via the FS-algorithm. We use the notation of the
preceding section.
( j)

( j)

( j+1)

Theorem 3.4.1 Set wn = n1 (I )/n1 (I ), and dene

(nj) =

( j)

1
( j)

1 wn

and (nj) =

wn

( j)

1 wn

(3.4.1)

for all j = 0, 1, . . . , and n = 1, 2, . . . .

( j)

(i) The ni can be computed recursively from

( j)

( j+1)

( j)

ni = (nj) n1,i1 + (nj) n1,i , i = 0, 1, . . . , n,

(3.4.2)

(s)
= 1 for all s, and we have set ki(s) = 0 if i < 0 or i > k.
where 00
(ii) Consider now the recursion relation
( j+1)
( j)
 n( j) = |(nj) |  n1 + |(nj) |  n1 , j = 0, 1, . . . , n = 1, 2, . . . ,

(3.4.3)

( j)
( j)
with the initial conditions  0 = 0 = 1, j = 0, 1, . . . . Then

n( j)  n( j) , j = 0, 1, . . . , n = 1, 2, . . . ,
with equality for n = 1.

(3.4.4)

3.5 Analysis of Column Sequences

67

( j)

Proof. We start by reexpressing An in the form

( j+1)

( j)

A(nj) = (nj) An1 + (nj) An1 ,

(3.4.5)

which is obtained by substituting (3.3.10) with b = a and b = I in (3.3.6) and invoking

(3.3.6) again. Next, combining (3.2.3) and (3.4.5), we obtain (3.4.2). Taking the moduli
of both sides of (3.4.2) and summing over i, we obtain
( j+1)

( j)

n( j) |(nj) | n1 + |(nj) | n1 .

(3.4.6)

The result in (3.4.4) now follows by subtracting (3.4.3) from (3.4.6) and using
induction.


( j)

( j)

( j+1)

From (3.4.1)(3.4.3) and the fact that wn is given in terms of n1 (I ) and n1 (I )

( j)
that are already known, it is clear that the cost of computing the ni , 0 j + n L is
(
j)
O(L 3 ) arithmetic operations, while the cost of computing the  n , 0 j + n L, is
O(L 2 ) arithmetic operations.
( j)
Based on our numerical experience, we note that the  n tend to increase very quickly
in some cases, thus creating the wrong impression that the corresponding extrapolation
process is very unstable even when that is not the case. Therefore, it may be better to use
( j)
( j)
(3.4.2) to compute the ni and then n exactly.
( j)
A recursion relation for computing the ni within the context of the E-algorithm that
is analogous to what we have in Theorem 3.4.1 was given by Havie [129].

3.5 Analysis of Column Sequences

Because the functions k (y) in the expansions (3.1.1) and (3.1.3) have no particular
( j)
structure, the analysis of An as dened by (3.1.4) is not a very well-dened task.
Thus, we cannot expect to obtain convergence results that make sense without making
reasonable assumptions about the k (y) and the ym . This probably is a major reason for
the scarcity of meaningful results in this part of the theory of extrapolation methods.
( j)
Furthermore, all the known results concern the column sequences {An }
j=0 for xed n.
( j)
for
xed
j
seems
to
be
much
more difcult
Analysis of the diagonal sequences {An }
n=0
than that of the column sequences and most probably requires more complicated and
detailed assumptions to be made on the k (y) and the ym .
All the results of the present section are obtained under the conditions that
lim

gk (m + 1)
k (ym+1 )
= lim
= ck = 1, k = 1, 2, . . . ,
m
gk (m)
k (ym )

(3.5.1)

and
ck = cq if k = q,

(3.5.2)

in addition to that in (3.1.2). Also, the ck can be complex numbers.

In view of these conditions, all the results of this section apply, in particular, to the
case in which k (y) = y k , k = 1, 2, . . . , where k = 0 for all k and 1 < 2 < ,
when limm ym+1 /ym = with some xed (0, 1). In this case, ck = k ,

68

3 First Generalization of the Richardson Extrapolation Process

k = 1, 2, . . . . With these k (y), the function A(y) is precisely that treated in Chapter 1.
But this time the ym do not necessarily satisfy ym+1 = ym , m = 0, 1, 2, . . . , as assumed
in Chapter 1; instead ym+1 ym as m . Hence, this chapter provides additional
results for the functions A(y) of Chapter 1. These have been given as Theorem 2.1.1 in
Chapter 2.
The source of all the results given in the next three subsections is Sidi [290]; those in
the last two subsections are new. Some of these results are used in the analysis of other
methods later in the book.
Before we go on, we would like to show that the conditions in (3.5.1 and (3.5.2) are
satised in at least one realistic case, namely, that mentioned in Example 3.1.2.
Example 3.5.1 Let us consider the function A(y) of Example 3.1.2. If we choose {ym }
such that limm ym+1 /ym = (0, 1), we realize that (3.5.1) and (3.5.2) are satised.
Specically, we have
k (ym+1 )
=
ck = lim
m k (ym )

s+i+1 , i = 2k/3, k = 1, 2, 4, 5, 7, 8, . . . ,
k = 3, 6, 9, . . . .
2k/3 ,

( j)

3.5.1 Convergence of the ni

( j)

We start with a result on the polynomial Hn (z) that we will use later.
( j)

Theorem 3.5.2 For n xed, the polynomial Hn (z) satises

( j)

Hn (z)
= V (c1 , c2 , . . . , cn , z),
lim n
j
i=1 gi ( j)

(3.5.3)

where V (1 , 2 , . . . , k ) is the Vandermonde determinant dened by



 1
1 1 

 1 2 k 



( j i ).
V (1 , 2 , . . . , k ) =  .
=
.
.
..
.. 
 ..
 1i< jk

 k1 k1 k1 
k
1
2

(3.5.4)

( j)

Proof. Dividing the ith column of Hn (z) by gi ( j), i = 1, . . . , n, and letting

( j)

g i (r ) =

gi ( j + r )
, r = 0, 1, 2, . . . ,
gi ( j)

(3.5.5)

we obtain

 1
1

 ( j)
( j)
( j)

g
(1)

g
(1)

n
Hn (z)
1
n
=  .
.
..
 ..
i=1 gi ( j)
 ( j)
(
j)
g 1 (n) g n (n)


1 

z
..  .
.

zn 

(3.5.6)

3.5 Analysis of Column Sequences

69

But
gi ( j + 1)
gi ( j + r ) gi ( j + r 1)

,
gi ( j + r 1) gi ( j + r 2)
gi ( j)

( j)

g i (r ) =

(3.5.7)

so that
( j)

lim g i (r ) = cir , r = 0, 1, . . . ,

(3.5.8)

from (3.5.1). The result in (3.5.3) now follows by taking the limit of both sides of
(3.5.6).


With the help of Theorems 3.2.2 and 3.5.2 and the conditions on the ck given in (3.5.1)
( j)
and (3.5.2), we obtain the following interesting result on the ni .
Theorem 3.5.3 The polynomial
lim

n


n

( j)

i=0

ni z i is such that

( j)

ni z i = Un (z) =

i=0

n
n


z ci

ni z i .
1

c
i
i=1
i=0

(3.5.9)

( j)

Consequently, lim j ni = ni , i = 0, 1, . . . , n, as well.

( j)

This result will be of use in the following analysis of An . Before we go on, we would
like to draw attention to the similarity between (3.5.9) and (1.4.4).
( j)

3.5.2 Convergence and Stability of the An

( j)

Let us observe that from (3.3.14) and what we already know about the ni , we have
n( j) (b) =

n


( j)

ni b( j + i)

(3.5.10)

i=0

for every sequence b. With this, we can now write (3.2.10) in the form
A(nj) A =

s


k n( j) (gk ) + n( j) (rs ),

(3.5.11)

k=n+1

where we have dened

rs (i) = Rs (yi ), i = 0, 1, . . . .

(3.5.12)
( j)

( j)

The following theorem concerns the asymptotic behavior of the n (gk ) and n (rs )
as j .
Theorem 3.5.4
(i) For xed n > 0, the sequence {n (gk )}
k=n+1 is an asymptotic sequence as j ;
( j)
( j)
that is, lim j n (gk+1 )/n (gk ) = 0, for all k n + 1. Actually, for each k
n + 1, we have the asymptotic equality
( j)

n( j) (gk ) Un (ck )gk ( j) as j ,

with Un (z) as dened in the previous theorem.

(3.5.13)

70

3 First Generalization of the Richardson Extrapolation Process

( j)

(ii) Similarly, n (rs ) satises

n( j) (rs ) = O(n( j) (gs+1 )) as j .

(3.5.14)

Proof. From (3.5.10) and (3.5.5),

n( j) (gk )

n


( j)
ni gk ( j

+ i) =

i=0


n


( j) ( j)
ni g k (i)

gk ( j).

(3.5.15)

i=0

Invoking (3.5.8) and Theorem 3.5.3 in (3.5.15), and using the fact that Un (ck ) = 0 for
k n + 1, the result in (3.5.13) follows. To prove (3.5.14), we start with
 n
 
n


 ( j)  
( j)
( j) 

 (rs ) = 

(3.5.16)

r
(
j
+
i)
|ni |  Rs (y j+i ) .
n
ni s


i=0

i=0

( j)

From the fact that the ni are bounded in j by Theorem 3.5.3 and from Rs (y) =
i
s+1 (y j ) as j that follows from
O(s+1 (y)) as y 0+ and s+1 (y j+i ) cs+1
( j)
(3.5.8), we rst obtain n (rs ) = O(gs+1 ( j)) as j . The result now follows from
( j)
the fact that gs+1 ( j) = O(n (rs )) as j , which is a consequence of the asymptotic
equality in (3.5.13) with k = s + 1.


( j)

The next theorem concerns the behavior of An for j and is best possible
asymptotically.
Theorem 3.5.5
( j)

(i) Suppose that A(y) satises (3.1.3). Then An A has the bona de asymptotic
expansion
A(nj) A

k n( j) (gk ) as j ,

k=n+1

= O(gn+1 ( j)) as j .

(3.5.17)

Consequently, if n+ is the rst nonzero n+i with i 1 in (3.1.3), then we have

the asymptotic equality


n
cn+ ci
A(nj) A n+ n( j) (gn+ )
n+ n+ (y j ) as j .
1 ci
i=1
(3.5.18)
(ii) In case A(y) satises (3.1.1) with s a nite largest possible integer, and n+ is
the rst nonzero n+i with i 1 there, then (3.5.18) holds. If n+1 = n+2 = =
s = 0 or n s, we have
A(nj) A = O(s+1 (y j )) as j .
These results are valid whether lim y0+ A(y) exists or not.

(3.5.19)

3.5 Analysis of Column Sequences

71

Proof. For (3.5.17) to have a meaning as an asymptotic expansion and to be valid,

( j)
( j)
rst {n (gk )}
k=n+1 must be an asymptotic sequence as j , and then An A

s
( j)
( j)
k=n+1 k n (gk ) = O(n (gs+1 )) as j must hold for each s n + 1. Now, both
of these conditions hold by Theorem 3.5.4. Therefore, (3.5.17) is valid as an asymptotic
expansion. The asymptotic equality in (3.5.18) is a direct consequence of (3.5.17). This
completes the proof of part (i) of the theorem. The proof of part (ii) can be achieved
similarly and is left to the reader.


Remarks.
1. Comparing (3.5.17) with (3.5.11), one may be led to believe erroneously that (3.5.17)
follows from (3.5.11) in a trivial way by letting s in the latter. This is far from
being the case as is clear from the proof of Theorem 3.5.5, which, in turn, depends
entirely on Theorem 3.5.4.
2. By imposing the additional conditions that |k | < k for all k and for some > 0,


and that A(y) has the convergent expansion A(y) = A +
k=1 k k (y), and that
this expansion converges absolutely and uniformly in y, Wimp [366, pp. 188189]
has proved the result of Theorem 3.5.5. Clearly, these conditions are very restrictive.
Brezinski [37] stated an acceleration result under even more restrictive conditions that
( j)
also impose limitations on the An themselves, when we are actually investigating
the properties of the latter. Needless to say, the result of [37] follows in a trivial way
from Theorem 3.5.5 that has been obtained under the smallest number of conditions
on A(y) only.
We now consider the stability of the column sequences {An }
j=0 . From the discussion
on stability in Section 0.5, the relevant quantity to be analyzed in this connection is
( j)

n( j) =

n


( j)

|ni |.

(3.5.20)

i=0

The following theorem on stability is a direct consequence of Theorem 3.5.3.

( j)

Theorem 3.5.6 The ni satisfy

lim

n


( j)

|ni | =

i=0

n


|ni |

i=0

n

1 + |ci |
i=1

|1 ci |

(3.5.21)

with ni as dened by (3.5.9). Consequently, the extrapolation process that generates

( j)
the sequence {An }
j=0 is stable in the sense that
sup n( j) < .

(3.5.22)

In case the ck all have the same phase, the inequality in (3.5.21) becomes an equality.
( j)
In case the ck are real and negative, (3.5.21) becomes lim j n = 1.
Proof. The result in (3.5.21) follows from Theorems 3.5.3 and 1.4.3, and that in (3.5.22)
follows directly from (3.5.21).

72

3 First Generalization of the Richardson Extrapolation Process

n
( j)
( j)
|ni | is proRemark. As is seen from (3.5.21), the size of the bound on n = i=0
n
(
j)
portional to D = i=1 |1 ci |1 . This implies that n will be small provided the ci
are away from 1. We also see from (3.5.18) that the coefcient of n+ (y j ) there is proportional to D. From this, we conclude that the more stable the extrapolation process,
( j)
the better the quality of the (theoretical) error An A, which is somewhat surprising.
3.5.3 Convergence of the k
( j)

As mentioned previously, the k nk in the equations in (3.1.4) turn out to be approx( j)

imations to the corresponding k , k = 1, . . . , n. In fact, nk k for j , as we
show in the next theorem that is best possible asymptotically.
Theorem 3.5.7 Assume that A(y) is as in (3.1.3). Then, for k = 1, . . . , n, with n xed,
( j)
we have lim j nk = k . In fact,


n
cn+ 1 
cn+ ci n+ (y j )
( j)
as j , (3.5.23)
nk k n+
ck 1 i=1 ck ci
k (y j )
i=k

where n+ is the rst nonzero n+i with i 1.

Proof. By Cramers rule the solution of the linear system in (3.1.4) for k is given by
( j)

nk =

1
( j)

f n (I )

|g1 ( j) gk1 ( j) a( j) gk+1 ( j) gn ( j) I ( j)| .

Now, under the given conditions, the expansion

n

a( j) = A +
k gk ( j) + n+ g n+ ( j),

(3.5.24)

(3.5.25)

k=1

where g n+ ( j) = gn+ ( j)[1 + n+ ( j)] and n+ ( j) = o(1) as j , is valid. Substituting (3.5.25) in (3.5.24), and expanding the numerator determinant there with respect
to its kth column, we obtain
( j)

( j)

nk k =

Vnk
( j)

f n (I )

( j)

Vnk = n+ |g1 ( j) gk1 ( j) g n+ ( j) gk+1 ( j) gn ( j) I ( j)|.

(3.5.26)

( j)

The rest can be completed by dividing the ith column of Vnk by gi ( j), i = 1, . . . , n,
i = k, and the kth column by gn+ ( j), and taking the limit for j . Following the
steps of the proof of Theorem 3.5.2, we have
( j)

lim

Vnk
= n+ V (c1 , . . . , ck1 , cn+ , ck+1 , . . . , cn , 1). (3.5.27)
n

gn+ ( j)
gi ( j)
i=1
i=k

( j)

( j)

Combining this with Theorem 3.5.2 on f n (I ) = Hn (1) in (3.5.26), we obtain (3.5.23).

3.5 Analysis of Column Sequences

73

( j)

Even though we have convergence of nk to k for j , in nite-precision arith( j)

metic the computed nk is a poor approximation to k in most cases. The precise reason
for this is given in the next theorem that is best possible asymptotically.
Theorem 3.5.8 If we write
( j)

nk =

n


( j)

nki a( j + i),

(3.5.28)

i=0
( j)

which is true from (3.1.4), then the nki satisfy



n
n
n


z1 
z ci
( j)
lim k (y j )
nki z i =

nki z i ,
j
c

1
c

c
k
k
i
i=1
i=0
i=0

(3.5.29)

i=k

from which we have

n


( j)

|nki |

i=0


n
i=0


|nki |

1
as j .
|k (y j )|

(3.5.30)

( j)

Thus, if lim y0+ k (y) = 0, then the computation of nk from (3.1.4) is unstable for
large j.
( j)

Proof. Let us denote the matrix of coefcients of the system in (3.1.4) by B. Then nki are
( j)
( j)
the elements of the row of B 1 that is appropriate for nk . In fact, nki can be identied
by expanding the numerator determinant in (3.5.24) with respect to its kth column. We
have
n


z j

( j)

nki z i =

i=0

( j)

f n (I )

|g1 ( j) gk1 ( j) Z ( j) gk+1 ( j) gn ( j) I ( j)|, (3.5.31)

where Z (l) = z l , l = 0, 1, . . . . Proceeding as in the proof of Theorem 3.5.2, we obtain

n

i=0

( j)

nki z i

V (c1 , . . . , ck1 , z, ck+1 , . . . , cn , 1) 1

as j . (3.5.32)
V (c1 , . . . , cn , 1)
gk ( j)

The results in (3.5.29) and (3.5.30) now follow. We leave the details to the reader.

By the condition in (3.1.2) and Theorems 3.5.7 and 3.5.8, it is clear that both the
( j)
( j)
theoretical quality of nk and the quality of the computed nk decrease for larger k.

3.5.4 Conditioning of the System (3.1.4)

The stability results of Theorems 3.5.6 and 3.5.8 are somewhat related to the conditioning of the linear system (3.1.4) that denes the rst generalization of the Richardson
extrapolation process. In this section, we study the extent of this relation under the
conditions of the preceding section.

74

3 First Generalization of the Richardson Extrapolation Process

As in the proof of Theorem 3.5.8, let us denote the matrix of coefcients of the linear
system in (3.1.4) by B. Then the l -norm of B is given by


n

 B  = max 1 +
|gi ( j + r )| .
(3.5.33)
0r n

i=1

The row of B 1 that gives us An is nothing but (n0 , . . . , nn ), and the row that gives
( j)
( j)
( j)
nk is similarly (nk0 , . . . , nkn ), so that


n
n

( j)
( j)
 B 1  = max
|ni |,
|nki |, k = 1, . . . , n .
(3.5.34)
( j)

( j)

i=0

( j)

i=0

From Theorems 3.5.6 and 3.5.8 we, therefore, have

 B 1  max{M0 , Mn /|gn ( j)|} as j ,

(3.5.35)

with
M0 =

n


|ni | and Mn =

i=0

n


|nni |,

(3.5.36)

i=0

and these are best possible asymptotically.

From (3.5.33) and (3.5.35), we can now determine how the condition number of B
behaves for j . In particular, we have the following result.
Theorem 3.5.9 Denote by (B) the l condition number of the matrix B of the linear
system in (3.1.4); that is, (B) = B B 1 . In general, (B) as j
at least as quickly as g1 ( j)/gn ( j). In particular, when lim y0+ 1 (y) = 0, we have
(B)

Mn
as j .
|n (y j )|

(3.5.37)

Remarks. It must be noted that (B), the condition number of the matrix B, does
( j)
( j)
not affect the numerical solution for the unknowns An and nk = k , k = 1, . . . , n,

n
( j)
( j)
( j)
|ni | that
uniformly. In fact, it has no effect on the computed An . It is n = i=0
( j)
controls the inuence of errors in the a(m) = A(ym ) (including roundoff errors) on An ,
( j)
and that n M0 < as j , independently of (B). Similarly, the stability

n
( j)
( j)
|nki | and not by (B). The
properties of nk , 1 k n 1, are determined by i=0
( j)
stability properties only of nn appear to be controlled by (B).

3.5.5 Conditioning of (3.1.4) for the Richardson Extrapolation Process

on Diagonal Sequences
So far, we have analyzed the conditioning of the linear system in (3.1.4) for column
sequences under the conditions in (3.5.1) and (3.5.2). Along the way to the main result
in Theorem 3.5.9, we have also developed some of the main ingredients for treatment of
the conditioning related to diagonal sequences for the Richardson extrapolation process
of Chapter 1, to which we now turn. In this case, we have k (y) = y k , k = 1, 2, . . . ,

3.5 Analysis of Column Sequences

75

and ym+1 = ym , m = 0, 1, . . . , so that gk (n + 1)/gk (n) = k = ck , k = 1, 2, . . . .

Going over the proof of Theorem 3.5.8, we realize that (3.5.29) can now be replaced by
y j k

n


( j)

nki z i =

i=0

n
n

z ci
z1 
=
nki z i
ck 1 i=1 ck ci
i=0

(3.5.38)

i=k

that is valid for every j and n. Obviously, (3.5.33) and (3.5.34) are always valid. From
(3.5.33), (3.5.34), and (3.5.38), we can write
 B  > |y j n | and  B 1 

n


n
|nni | = |y
j |

( j)

i=0

n


|nni |,

(3.5.39)

i=0

so that
(B) = B   B 1  >

n


|nni |.

(3.5.40)

i=0

The following lemma that complements Lemma 1.4.4 will be useful in the sequel.

n
Lemma 3.5.10 Let Q(z) = i=0
ai z i , an = 1. Denote the zeros of Q(z) that are not
p
on the unit circle K = {z : |z| = 1} by z 1 , z 2 , . . . , z p , so that Q(z) = R(z) i=1 (z z i )
for some polynomial R(z) of degree n p with all its zeros on K . Then


p
n

|ai | max |R(z)|
|1 |z i || > 0.
(3.5.41)
|z|=1

i=0

i=1

Proof. We start with

n

i=0



p


 n
i

|ai | 
ai z  = |R(z)|
|z z i | for every z K .
i=0

i=1

Next, we have
n

i=0

|ai | |R(z)|

p


||z| |z i || = |R(z)|

i=1

p


|1 |z i || for every z K .

i=1

The result follows by maximizing both sides of this inequality on K .

Applying Lemma 3.5.10 to the polynomial

n


 z ci
z 1 n1
,
nni z i =
cn 1 i=1 cn ci
i=0

and using the fact that limm cm = 0, we have

n1
n

|1 |ci ||
|nni | L 1 i=r
n1
i=0
i=1 |cn ci |
for all n, where L 1 is some positive constant and r is some positive integer for which
|cr | < 1, so that |ci | < 1 for i r as well.

76

3 First Generalization of the Richardson Extrapolation Process

n1 n1
n1
|cn ci | | i=1
ci | i=1 (1 + |cn /ci |) and from
Now, from the fact that i=1
n1
|c ci |
(1.5.20), which holds under the condition in (1.5.12), we have i=1
n1
n1 n
L 2 | i=1 ci | for all n, where L 2 is some positive constant. Similarly, i=r |1 |ci || >

|1 ci |
L 3 for all n, where L 3 is some positive constant, since the innite product i=1
converges, as shown in the proof of Theorem 1.5.4. Combining all these, we have
n


|nni | L

i=0

n1


1

|ci |

for all n, some constant L > 0.

(3.5.42)

i=1

We have thus obtained the following result.

Theorem 3.5.11 When k (y) = y k , k = 1, 2, . . . , and ym+1 = ym , m = 0, 1, . . . ,
and the k satisfy the condition in (1.5.12) of Theorem 1.5.4, the condition number of
the matrix B of the linear system (3.1.4) satises
(B) > L

n1


|ci |

1

Ldn

/2+gn

for some constants L > 0 and g. (3.5.43)

i=1

Thus, (B) as n practically at the rate of dn

/2

Note again that, if Gaussian elimination is used for solving (3.1.4) in nite-precision
arithmetic, the condition number (B) has no effect whatsoever on the computed value
( j)
of An for any value of n, even though (B) for n very strongly.

3.6 Further Results for Column Sequences

The convergence and stability analysis for the sequences {An }
j=0 of the preceding
section was carried out under the assumptions in (3.1.2) on the k (y) and (3.5.1) and
(3.5.2) on the gk (m). It is seen from the proofs of the theorems of Section 3.5 that (3.1.2)
can be relaxed somewhat without changing any of the results there. Thus, we can assume
that
( j)

k+1 (y) = O(k (y)) as y 0 + for all k, and

k+1 (y) = o(k (y)) as y 0 + for innitely many k.
Of course, in this case we take the asymptotic expansion in (3.1.3) to mean that (3.1.1)
holds for each integer s. This is a special case of a more general situation treated by Sidi
[297]. The results of [297] are briey mentioned in Subsection 14.1.4.
Matos and Prevost [207] have considered a set of conditions different from those in
(3.5.1) and (3.5.2). We bring their main result (Theorem 3 in [207]) here without proof.
Theorem 3.6.1 Let the functions k (y) satisfy (3.1.2) and let 1 (y) = o(1) as y 0+
as well. Let the gk (m) = k (ym ) satisfy for all i 0 and p 0
|gi+ p ( j) gi+1 ( j) gi ( j)| 0, for j J, some J,

(3.6.1)

3.7 Further Remarks on (3.1.4): Related Convergence Acceleration Methods 77

where g0 ( j) = 1, j = 0, 1, . . . . Then, for xed n, {n (gk )}
k=n+1 is an asymptotic
( j)
( j)
sequence as j ; that is, lim j n (gk+1 )/n (gk ) = 0 for all k n + 1.
( j)

Two remarks on the conditions of this theorem are now in order. We rst realize from
(3.6.1) that gk (m) are all real. Next, the fact that 1 (y) = o(1) as y 0+ implies that
lim y0+ A(y) = A is assumed.
These authors also state that, provided n+1 = 0, the following convergence acceleration result holds in addition to Theorem 3.6.1:
( j)

lim

An+1 A

( j)

An A

= 0.

(3.6.2)

For this to be true, we must also have for each s that

A(nj) A =

s


k n( j) (gk ) + O(n( j) (gs+1 )) as j .

(3.6.3)

k=n+1

However, careful reading of the proof in [207] reveals that this has not been shown.
We have not been able to show the truth of this under the conditions given in Theorem

N
3.6.1 either. It is easy to see that (3.6.3) holds in case A(y) = A + k=1
k k (y) for
some nite N , but this, of course, restricts the scope of (3.6.3) considerably. The most
common cases that arise in practice involve expansions such as (3.1.1) or truly divergent
asymptotic expansions such as (3.1.3). By comparison, all the results of the preceding
section have been obtained for A(y) as in (3.1.1) and (3.1.3). Finally, unlike that in
(3.5.18), the result in (3.6.2) gives no information about the rates of convergence and
acceleration of the extrapolation process.
Here are some examples of the gk (m) that have been given in [207] and that are covered
by Theorem 3.6.1.
g1 (m) = g(m) and gk (m) = (1)k k g(m), k = 2, 3, . . . , where {g(m)} is a logarithmic totally monotonic sequence. [That is, limm g(m + 1)/g(m) = 1 and
(1)k k g(m) 0, k = 0, 1, 2, . . . .]
gk (m) = xmk with 1 > x1 > x2 > > 0 and 0 < 1 < 2 < .
gk (m) = ckm with 1 > c1 > c2 > > 0.


gk (m) = 1 (m + 1)k (log(m + 2))k with 0 < 1 2 , and k < k+1 if
k = k+1 .
Of these, the third example is covered in full detail in Chapter 1 (see Section 1.9), where
results for both column and diagonal sequences are presented. Furthermore, the treatment
in Chapter 1 does not assume the ck to be real, in general.

3.7 Further Remarks on (3.1.4): Related Convergence Acceleration Methods

Let us replace A(yl ) by Al and k (yl ) by gk (l) in the linear system (3.1.4). This system
then becomes
n

k gk (l), j l j + n.
(3.7.1)
Al = A(nj) +
k=0

78

3 First Generalization of the Richardson Extrapolation Process

Now let us take {gk (m)}

m=0 , k = 1, 2, . . . , to be arbitrary sequences, not necessarily
related to any functions k (y), k = 1, 2, . . . , that satisfy (3.1.2).
We mentioned previously that, with appropriate gk (m), many known convergence
acceleration methods for a given innite sequence {Am } are either directly dened or
can be shown to be dened through linear systems of the form (3.7.1). Also, as observed
by Brezinski [37], various other methods can be cast into the form of (3.7.1). Then, what
differentiates between the various methods is the gk (m) that accompany them. Again,
( j)
An are taken to be approximations to the limit or antilimit of {Am }. In this sense, the
formalism represented by (3.7.1) may serve as a unifying framework for (dening)
these convergence acceleration methods. This formalism may also be used to unify the
convergence studies for the different methods, provided their corresponding gk (m) satisfy
gk+1 (m) = o(gk (m)) as m , k = 1, 2, . . . ,

(3.7.2)

and also
Am A +

k gk (m) as m .

(3.7.3)

k=1

[Note that (3.7.3) has a meaning only when (3.7.2) is satised.] The results of Sections 3.5
and 3.6, for example, may be used for this purpose. However, this seems to be of a
limited scope because (3.7.2) and hence (3.7.3) are not satised in all cases of practical
importance. We illustrate this limitation at the end of this section with the transformation
of Shanks.
Here is a short list of convergence acceleration methods that are dened through the
linear system in (3.7.1) with their corresponding gk (m). This list is taken in part from
Brezinski [37]. The FS- and E-algorithms may serve as implementations for all the
convergence acceleration methods that fall within this formalism, but they are much less
efcient than the algorithms that have been designed specially for these methods.
The Richardson extrapolation process of Chapter 1
gk (m) = ckm , where ck = 0, 1, are known constants. See Section 1.9 and Theorem
1.4.5. It can be implemented by Algorithm 1.3.1 with n there replaced by cn .
The polynomial Richardson extrapolation process
gk (m) = xmk , where {xi } is a given sequence. It can be implemented by Algorithm 2.2.1
of Bulirsch and Stoer [43].
The G-transformation of Gray, Atchison, and McWilliams [112]
gk (m) = xm+k1 , where {xi } is a given sequence. It can be implemented by the rsalgorithm of Pye and Atchison [233]. We have also developed a new procedure called
the FS/qd-algorithm. Both the rs- and FS/qd-algorithms are presented in Chapter 21.
The Shanks [264] transformation
gk (m) = Am+k1 . It can be implemented by the -algorithm of Wynn [368]. See
Section 0.4.
Levin [161] transformations
gk (m) = Rm /(m + 1)k1 , where {Ri } is a given sequence. It can be implemented by
an algorithm due to Longman [178] and Fessler, Ford, and Smith [83] and also by the
W-algorithm of Sidi [278]. Levin gives three different sets of Rm for three methods,

3.8 Epilogue: What Is the E-Algorithm? What Is It Not?

79

namely, Rm = am for the t-transformation, Rm = (m + 1)am for the u-transformation,

and Rm = am am+1 /(am+1 am ) for the v-transformation, where a0 = A0 and am =
Am1 for m 1.
The transformation of Wimp [361]
gk (m) = (Am )k . It can be implemented by the algorithm of Bulirsch and Stoer [43].
(This transformation was rediscovered later by Germain-Bonne [96].)
The Thiele rational extrapolation
gk (m) = xmk , k = 1, 2, . . . , p, and g p+k (m) = Am xmk , k = 1, 2, . . . , p, with n = 2 p,
where {xi } is a given sequence. It can be implemented by a recursive algorithm that
involves inverted differences. By taking xi = 1/i, i = 1, 2, . . . , we obtain the extrapolation method that has been known by the name -algorithm and that is due to Wynn
[369]. -algorithm is also the name of the recursive implementation of the method.
( j)

Brezinski [37] also shows that the solution for An of the generalized rational extrapolation problem

p
( j)
An + i=1 i f i (l)

q
, l = j, j + 1, . . . , j + n, with n = p + q,
Al =
1 + i=1 i h i (l)
can be cast into the form of (3.7.1) by identifying gk (m) = f k (m), k = 1, 2, . . . , p, and
g p+k (m) = Am h k (m), k = 1, 2, . . . , q. Here { f i (m)} and {gi (m)} are known sequences.
We mentioned that (3.7.2) is not satised in all cases of practical importance. We
now want to show that the gk (m) associated with the transformation of Shanks generally do not satisfy (3.7.2). To illustrate this, let us consider applying this transformation


k
to the sequence {Am }, where Am = m
k=0 ak , m = 0, 1, . . . , with ak = (1) /(k + 1).
(The Shanks transformation is extremely effective on the sequence {Am }. In fact, it
is one of the best acceleration methods on this sequence.) It is easy to show that
limm gk+1 (m)/gk (m) = 1 = 0, so that (3.7.2) fails to be satised. This observation is not limited only to the case ak = (1)k /(k + 1), but it applies to the general case


in which ak z k i=0
i k i as k , with |z| 1 but z = 1, as well. (The Shanks
transformation is extremely effective in this general case too.) A similar statement can
be made about the G-transformation. This shows very clearly that the preceding formalism is of limited use in the analysis of the various convergence acceleration methods
mentioned above. Indeed, there is no unifying framework within which convergence
acceleration methods can be analyzed in a serious manner. In reality, we need different
approaches and techniques for analyzing the different convergence acceleration methods
and for the different classes of sequences.

3.8 Epilogue: What Is the E-Algorithm? What Is It Not?

There has been a great confusion in the literature about what the E-algorithm is and what
it is not. We believe this confusion can now be removed in view of the contents of this
chapter.
It has been written in different places that the E-algorithm is the most general extrapolation method actually known, or that it covers most of the other algorithms, or that
it includes most of the convergence acceleration algorithms actually known, or that

80

3 First Generalization of the Richardson Extrapolation Process

most algorithms for convergence acceleration turn out to be special cases of it. These
statements are false and misleading.
First, the E-algorithm is not a sequence transformation (equivalently, a convergence
acceleration method), it is only a computational procedure (one of many) for solving
( j)
linear systems of the form (3.7.1) for the unknown An . The E-algorithm has nothing to
do with the fact that most sequence transformations can be dened via linear systems of
the form given in (3.7.1).
Next, the formalism represented by (3.7.1) is not a convergence acceleration method
by itself. It remains a formalism as long as the gk (m) are not specied. It becomes a
convergence acceleration method only after the accompanying gk (m) are specied. As
soon as the gk (m) are specied, the idea of using extrapolation suggests itself, and the
source of this appears to be Hart et al. [125, p. 39].
The derivation of the appropriate gk (m) for example, through an asymptotic analysis of {Am } is much more important than the mere observation that a convergence
acceleration method falls within the formalism of (3.7.1).
Finally, as we have already noted, very little has been learned about existing extrapolation methods through this formalism. Much more can be learned by studying the
individual methods. This is precisely what we do in the remainder of this book.

4
GREP: Further Generalization of the Richardson
Extrapolation Process

4.1 The Set F(m)

The rst generalization of the Richardson extrapolation process that we dened and
studied in Chapter 3 is essentially based on the asymptotic expansion in (3.1.3), where
{k (y)} is an asymptotic sequence in the sense of (3.1.2). In general, the k (y) considered
in Chapter 3 are assumed to have no particular structure. The Richardson extrapolation
process that we studied in Chapter 1, on the other hand, is based on the asymptotic
expansion in (1.1.2) that has a simple well-dened structure. In this chapter, we describe
a further generalization of the Richardson extrapolation process due to Sidi [272], and
denoted there GREP for short, that is based on an asymptotic expansion of A(y) that
possesses both of these features. The exact form of this expansion is presented in
Denition 4.1.1. Following that, GREP is dened in the next section.
Denition 4.1.1 We shall say that a function A(y), dened for y (0, b], for some b > 0,
where y is a discrete or continuous variable, belongs to the set F(m) for some positive
integer m, if there exist functions k (y) and k (y), k = 1, 2, . . . , m, and a constant A,
such that
m

k (y)k (y),
(4.1.1)
A(y) = A +
k=1

where the functions k (y) are dened for y (0, b] and k ( ), as functions of the continuous variable , are continuous in [0, ] for some b, and for some constants rk > 0,
have Poincare-type asymptotic expansions of the form
k ( )

ki irk as 0+, k = 1, . . . , m.

(4.1.2)

i=0

If, in addition, Bk (t) k (t 1/rk ), as a function of the continuous variable t, is innitely

differentiable in [0, rk ], k = 1, . . . , m, we say that A(y) belongs to the set F(m)
. [Thus,
(m)
.]
F(m)
F
The reader may be wondering why we are satised with the requirement that the
functions k ( ) in Denition 4.1.1 be continuous on [0, ] with b, and not necessarily
on the larger interval [0, b] where A(y) and the k (y) are dened. The reason is that it
81

82

4 GREP: Further Generalization of the Richardson Extrapolation Process

is quite easy to construct practical examples in which A(y) and the k (y) are dened in
some interval [0, b] but the k ( ) are continuous only in a smaller interval [0, ], < b.
Denition 4.1.1 accommodates this general case.
We assume that A in (4.1.1) is either the limit or the antilimit of A(y) as y 0+.
We also assume that A(y) and k (y), k = 1, . . . , m, are all known (or computable)
for all possible values that y is allowed to assume in (0, b] and that rk , k = 1, . . . , m,
are known as well. We do not assume that the constants ki are known. We do not
assume the functions k (y) to have any particular structure. Nor do we assume that they
satisfy k+1 (y) = o(k (y)) as y 0+, cf. (3.1.2). Finally, we are interested in nding
(or approximating) A, whether it is the limit or the antilimit of A(y) as y 0+.
Obviously, when lim y0+ k (y) = 0, k = 1, . . . , m, we also have lim y0+ A(y) =
A. Otherwise, the existence of lim y0+ A(y) is not guaranteed. In case lim y0+ A(y)
does not exist, it is clear that lim y0+ k (y) does not exist for at least one value of k.
It is worth emphasizing that the function A(y) above is such that A(y) A is the sum
of m terms; each one of these terms is the product of a function k (y) that may have
arbitrary behavior for y 0+ and another, k (y), that has a well-dened (and smooth)
behavior for y 0+ as a function of y rk .
Obviously, when A(y) F(m) with A as its limit (or antilimit), a A(y) + b F(m) as
well, with a A + b as its limit (or antilimit).
In addition, we have some kind of a closure property among the functions in the sets
(m)
F . This is described in Proposition 4.1.2, whose verication we leave to the reader.
Proposition 4.1.2 Let A1 (y) F(m 1 ) with limit or antilimit A1 and A2 (y) F(m 2 ) with
limit or antilimit A2 . Then A1 (y) + A2 (y) F(m) with limit or antilimit A1 + A2 and
with some m m 1 + m 2 .
Another simple observation is contained in Proposition 4.1.3 that is given next.


Proposition 4.1.3 If A(y) F(m) , then A(y) F(m ) for m  > m as well.
This observation raises the following natural question: What is the smallest possible value of m that is appropriate for the particular function A(y), and how can it
be determined? As we show later, this question is relevant and has important practical
consequences.
We now pause to give a few simple examples of functions A(y) in F(1) and F(2) that
arise in natural ways. These examples are generalized in the next few chapters. They and
the examples that we provide later show that the classes F(m) and F(m)
are very rich in the
sense that many sequences that we are likely to encounter in applied work are associated
with functions A(y) that belong to the classes F(m) or F(m)
.
(2)
The functions A(y) in Examples 4.1.54.1.9 below are in F(1)
and F . Verication of
this requires special techniques and is considered later.
Example 4.1.4 Trapezoidal
 1 Rule for Integrals with an Endpoint Singularity Consider the integral I [G] = 0 G(x) d x, where G(x) = x s g(x) with s > 1 but s not an

4.1 The Set F(m)

83

integer and g C [0, 1]. Thus, the integrand G(x) has an algebraic singularity at the
left endpoint x = 0. Let h = 1/n, where n is a positive integer. Let us approximate I [G]
by the (modied) trapezoidal rule T (h) or by the midpoint rule M(h), where these are
given as


n1
n

1
G( j h) + G(1) and M(h) = h
G(( j 1/2)h). (4.1.3)
T (h) = h
2
j=1
j=1
If we now let Q(h) stand for either T (h) or M(h), we have the asymptotic expansion
Q(h) I [G] +


i=1

ai h 2i +

bi h s+i+1 as h 0,

(4.1.4)

i=0

where ai and bi are constants independent of h. For T (h), these constants are given by
ai =

B2i (2i1)
(s i) (i)
G
g (0), i = 0, 1, . . . , (4.1.5)
(1), i = 1, 2, . . . ; bi =
(2i)!
i!

where Bi are the Bernoulli numbers and (z) is the Riemann Zeta function, dened


z
for z > 1 and then continued analytically to the complex plane.
by (z) =
k=1 k
Similarly, for M(h), ai and bi are given by
ai =

D2i (2i1)
(s i, 1/2) (i)
(1), i = 1, 2, . . . ; bi =
G
g (0), i = 0, 1, . . . ,
(2i)!
i!
(4.1.6)

where D2i = (1 212i )B2i , i = 1, 2, . . . , and (z, ) is the generalized Zeta func

z
for z > 1 and then continued analytically
tion dened by (z, ) =
k=0 (k + )
to the complex plane. Obviously, Q(h) is analogous to a function A(y) F(2) in the
following sense: Q(h) A(y), h y, h 2 1 (y), r1 = 2, h s+1 2 (y), r2 = 1,
and I [G] A. The variable y is, of course, discrete and takes on the values 1, 1/2,
1/3, . . . .
The asymptotic expansions for T (h) and M(h) described above are generalizations of
the EulerMaclaurin expansions for regular integrands discussed in Example 1.1.2 and
are obtained as special cases of that derived by Navot [216] (see Appendix D) for the
offset trapezoidal rule.
Example 4.1.5 Approximation
of an Innite-Range Integral Consider the innite
range integral I [f ] = 0 (sin t/t) dt. Assume it is being approximated by the nite
x
integral
 F(x) = 0 (sin t/t) dt for sufciently large x. By repeated integration by parts
of x (sin t/t) dt, it can be shown that F(x) has the asymptotic expansion
F(x) I [ f ]

(2i)! sin x 
(2i + 1)!
cos x 
(1)i 2i 2
(1)i
as x . (4.1.7)
x i=0
x
x i=0
x 2i

We now have that F(x) is analogous to a function A(y) F(2) in the following sense:
F(x) A(y), x 1 y, cos x/x 1 (y), r1 = 2, sin x/x 2 2 (y), r2 = 2, and
F() = I [ f ] A. The variable y is continuous in this case.

84

4 GREP: Further Generalization of the Richardson Extrapolation Process

Example 4.1.6 Approximation of an Innite-Range Integral Continued Let us allow

the variable x in the previous example to assume only the discrete values xs = s ,
s = 1, 2, . . . . Then (4.1.7) reduces to
F(xs ) I [ f ]

(2i)!
cos xs 
(1)i 2i as s .
xs i=0
xs

(4.1.8)

In this case, F(x) is analogous to a function A(y) F(1) in the following sense:
F(x) A(y), x 1 y, cos x/x 1 (y), r1 = 2, and F() = I [ f ] A. The variable y assumes only the discrete values (i )1 , i = 1, 2, . . . .
Example 4.1.7 Summation of the Riemann Zeta Function Series Let us go back to


z
that
Example 1.1.4, where we considered the summation of the innite series
k=1 k
converges to and denes the Riemann Zeta function (z) for z > 1. We saw there that


the partial sum An = nk=1 k z has the asymptotic expansion given in (1.1.18). This
expansion can be rewritten in the form
An (z) + n z+1


i
i=0

ni

as n ,

(4.1.9)

where i depend only on z. From this, we see that An is analogous to a function A(y)
F(1) in the following sense: An A(y), n 1 y, n z+1 1 (y), r1 = 1, and (z)
A. The variable y is discrete and takes on the values 1, 1/2, 1/3, . . . .
Recall that (z) is the limit of {An } when z > 1 and its antilimit otherwise, provided
z = 1, 0, 1, 2, . . . .
Example 4.1.8 Summation of the Logarithmic Function Series Consider the innite


k
1
when |z| 1 but z = 1. We show
power series
k=1 z /k that converges to log(1 z)


below that, as long as z  [1, +), the partial sum An = nk=1 z k /k is analogous to a


k
converges or not. We also provide the precise
function A(y) F(1) , whether
k=1 z /k

z k /k diverges.
description of
relevant antilimit when
k=1

 thekt
Invoking 0 e dt = 1/k, k > 0, in An = nk=1 z k /k with z  [1, +), changing
the order of the integration and summation, and summing the resulting geometric series

n
t k
k=1 (ze ) , we obtain


n

zk
zet
(zet )n+1
=
dt

dt.
(4.1.10)
k
1 zet
1 zet
0
0
k=1
Now, the rst integral in (4.1.10) is log(1 z)1 with its branch
 cut along the real interval
[1, +). We rewrite the second integral in the form z n+1 0 ent (et z)1 dt and apply
Watsons lemma (see, for example, Olver [223]; see also Appendix B). Combining all
this in (4.1.10), we have the asymptotic expansion
An log(1 z)1

i
z n+1 
as n ,
n i=0 n i

(4.1.11)

where i = (d/dt)i (et z)1 |t=0 , i = 0, 1, . . . . In particular, 0 = 1/(1 z) and

1 = 1/(1 z)2 .

4.2 Denition of the Extrapolation Method GREP

85

Thus, we have that An is analogous to a function A(y) F(1) in the following sense:
An A(y), n 1 y, z n /n 1 (y), r1 = 1, and log(1 z)1 A, with log(1 z)1


having its branch cut along [1, +). When |z| 1 but z = 1,
z k /k converges and

k=1

1
k
log(1 z) is limn An . When |z| > 1 but z  [1, +),
k=1 z /k diverges, but
1
log(1 z) serves as the antilimit of {An }. (See Example 0.2.1.) The variable y is
discrete and takes on the values 1, 1/2, 1/3, . . . .
Example 4.1.9 Summation of a Fourier Cosine Series Consider the convergent


1/2
when =
Fourier cosine series
k=1 cos k/k whose sum is log(2 2 cos )

k
i
2k, k = 0, 1, 2, . . . . This series can be obtained by letting z = e in k=1 z /k
that we treated in the preceding example and then taking the real part of the latter. By


doing the same in (4.1.11), we see that the partial sum An = nk=1 cos k/k has the
asymptotic expansion
An log(2 2 cos )1/2 +

cos n 
sin n 
i
i
+
as n , (4.1.12)
i
n i=0 n
n i=0 n i

for some i and i that depend only on .

Thus, An is analogous to a function A(y) F(2) in the following sense: An
A(y), n 1 y, cos n/n 1 (y), r1 = 1, sin n/n 2 (y), r2 = 1, and log(2
2 cos )1/2 A. The variable y is discrete and takes on the values 1, 1/2, 1/3, . . . .

4.2 Denition of the Extrapolation Method GREP

Denition 4.2.1 Let A(y) belong to F(m) with the notation of Denition 4.1.1. Pick
a decreasing positive sequence {yl } (0, b] such that liml yl = 0. Let n
(n 1 , n 2 , . . . , n m ), where n 1 , . . . , n m are nonnegative integers. Then, the approximation
(m, j)
to A, whether A is the limit or the antilimit of A(y) as y 0+, is dened through
An
the linear system
j)
+
A(yl ) = A(m,
n

m

k=1

k (yl )

n
k 1
i=0

ki ylirk , j l j + N ; N =

m


n k , (4.2.1)

k=1

1
ki being the additional (auxiliary) N unknowns. In (4.2.1),
i=0 ci 0 so that
(m, j)
A(0,... ,0) = A(y j ) for all j. This generalization of the Richardson extrapolation process
(m, j)
is denoted GREP(m) . When there is no room for confusion, we
that generates the An
will write GREP instead of GREP(m) for short.
Comparing the equations in (4.2.1) with the expansion of A(y) that is given in (4.1.1)
with (4.1.2), we realize that the former are obtained from the latter by substituting in
(4.1.1) the asymptotic expansion of k (y) given in (4.1.2), truncating the latter at the term
k,n k 1 y (n k 1)rk , k = 1, . . . , m, and nally collocating at y = yl , l = j, j + 1, . . . ,


j + N , where N = m
k=1 n k .

86

4 GREP: Further Generalization of the Richardson Extrapolation Process

(m, j)

The following theorem states that An

(0.3.3).
(m, j)

Theorem 4.2.2 An

is expressible in the form (0.3.2) with

can be expressed in the form

j)
A(m,
=
n

N


(m, j)

ni

A(y j+i )

(4.2.2)

i=0
(m, j)

with the ni

N


determined by the linear system

(m, j)

ni

=1

i=0
N


(m, j)

ni

k
k (y j+i )y sr
j+i = 0, s = 0, 1, . . . , n k 1, k = 1, . . . , m.

(4.2.3)

i=0

The proof of Theorem 4.2.2 follows from a simple analysis of the linear system in
(4.2.1) and is identical to that of Theorem 3.2.1. Note that if we denote by M the matrix
(m, j)
is the rst element of the vector of unknowns,
of coefcients of (4.2.1), such that An
(m, j)
then the vector [0 , 1 , . . . , N ], where i ni for short, is the rst row of the matrix
1
M ; this, obviously, is entirely consistent with (4.2.3).
(m, j)
The next theorem shows that the extrapolation process GREP that generates An
eliminates the rst n k terms y irk , i = 0, 1, . . . , n k 1, from the asymptotic expansion
of k (y) given in (4.1.2), for k = 1, 2, . . . , m. Just as Theorem 3.2.3, the next theorem
too is not a convergence theorem but a heuristic justication of the possible validity of
(4.2.1) as an extrapolation process.
Theorem 4.2.3 Let n and N be as before, and dene
Rn (y) = A(y) A

m


k (y)

k=1

n
k 1

ks y srk .

(4.2.4)

s=0

Then
j)
A(m,
A=
n

N


(m, j)

ni

Rn (y j+i ).

(4.2.5)

i=0

n k 1


srk
for all possible y, we have
In particular, when A(y) = A + m
k=1 k (y)
s=0 ks y
(m, j)



An  = A, for all j 0 and n = (n 1 , . . . , n m ) such that n k n k , k = 1, . . . , m.
The proof of Theorem 4.2.3 can be achieved by invoking Theorem 4.2.2. We leave
the details to the reader.
Before we close this section, we would like to mention that important examples of
GREP are the Levin transformations, the LevinSidi D- and d-transformations and the
D-,
W -, and mW -transformations, which are considered in great detail in the
Sidi D-,
next chapters.

4.3 General Remarks on F(m) and GREP

87

4.3 General Remarks on F(m) and GREP

1. We rst note that the expansion in (4.1.1) with (4.1.2) is a true generalization of the


i y ir as y 0+ that forms the basis of the polynomial
expansion A(y) A + i=1
Richardson extrapolation process in the sense that the former reduces to the latter when
we let m = 1, 1 (y) = y r1 , and r1 = r in (4.1.1) and (4.1.2). Thus, GREP(1) is a true
generalization of the polynomial Richardson extrapolation process.
Next, the expansion in (4.1.1) with (4.1.2) can be viewed as a generalization also
of the expansion in (3.1.3) in the following sense: (i) The unknown constants k in
(3.1.3) are now being replaced by some unknown smooth functions k (y) that possess
asymptotic expansions of known forms for y 0+. (ii) In addition, unlike the k (y)
in (3.1.3), the k (y) in (4.1.1) are not required to satisfy the condition in (3.1.2). That
is, the k (y) in (4.1.1) may have growth rates for y 0+ that may be independent of
each other; they may even have the same growth rates. Thus, in this sense, GREP(m) is a
generalization of the Richardson extrapolation process beyond its rst generalization.
Finally, as opposed to the A(y) in Chapter 3 that are represented by one asymptotic
expansion as y 0+, the A(y) in this chapter are represented by sums of m asymp
(m)
is a very large set that further
totic expansions as y 0+. As the union m
m=1 F
we realize that GREP is a comprehensive extrapolation procedure
expands with m,
with a very large scope.
2. Because successful application of GREP to nd the limit or antilimit of a function
A(y) F(m) requires as input the integer m together with the functions k (y) and the
numbers rk , k = 1, . . . , m, one may wonder how these can be determined. As seen
from the examples in Section 4.1, this problem is far from trivial, and any general
technique for solving it is of utmost importance.
By Denition 4.1.1, the information about m, the k (y), and rk is contained in
the asymptotic expansion of A(y) for y 0+. Therefore, we need to analyze A(y)
asymptotically for y 0+ to obtain this information. As we do not want to have
to go through such an analysis in each case, it is very crucial that we have theorems
of a general nature that provide us with the appropriate m, k (y), and rk for large
classes of A(y). (Example 4.1.4 actually contains one such theorem. Being specic,
Examples 4.1.54.1.9 do not.)
Fortunately, powerful theorems concerning the asymptotic behavior of A(y) in
many cases of practical interest can be proved. From these theorems, it follows that
sets of convenient k (y) can be obtained directly from A(y) in simple ways. The fact
that the k (y) are readily available makes GREP a very useful tool for computing
limits and antilimits. (For m > 1, GREP is usually the only effective tool.)
3. The fact that there are only a nite number of structureless functions, namely, the m
functions k (y), and that the k (y) are essentially polynomial in nature, enables us
to design algorithms for implementing GREP, such as the W - and W (m) -algorithms,
that are extremely efcient, unlike those for the rst generalization. The existence of
such efcient algorithms makes GREP especially attractive to the user. (The subject
of algorithms for GREP is treated in detail in Chapter 7.)
4. Now, the true nature of A(y) as y 0+ is actually contained in the functions k (y).
Viewed in this light, the terms shape function and form factor commonly used in

88

4 GREP: Further Generalization of the Richardson Extrapolation Process

nuclear physics, become appropriate for the k (y) as well. We use this terminology
throughout.
Weniger [353] calls the k (y) remainder estimates. When m > 1, this terminology is not appropriate, because, in general, none of the k (y) by itself can be
considered an estimate of the remainder A(y) A. This should be obvious from
Examples 4.1.5 and 4.1.9.
5. We note that the k (y) are not unique in the sense that they can be replaced by some
other functions k (y), at the same time preserving the form of the expansion in (4.1.1)

m

and (4.1.2). In other words, we can have m

k=1 k (y)k (y) =
k=1 k (y) k (y), with

k ( ) having asymptotic expansions of the form similar to that given in (4.1.2).

As the k (y) are not unique, we can now aim at obtaining k (y) that are as simple
and as user-friendly as possible for use with GREP. Practical examples of this strategy,
related to the acceleration of convergence of innite-range integrals and innite series
and sequences, are considered in Chapters 5, 6, and 11.

4.4 A Convergence Theory for GREP

(m, j)

From our convention that n in An

stands for the vector of integers (n 1 , n 2 , . . . , n m ),
(m, j)
(1, j)
= An 1 in a two-dimensional
it is clear that only for m = 1 can we arrange the An
array as in Table 1.3.1, but for m = 2, 3, . . . , multidimensional arrays are needed. Note
that the dimension of the array for a given value of m is m + 1. As a result of the
multidimensionality of the arrays involved, we realize that there can be innitely many
(m, j)
sequences of the A(n 1 ,... ,n m ) that can be studied and also used in practice.
Because of the multitude of choices, we may be confused about which sequences
we should use and study. This ultimately ties in with the question of the order in
which the functions k (y)y irk should be eliminated from the expansion in (4.1.1) and
(4.1.2).
Recall that in the rst generalization of the Richardson extrapolation process we
eliminated the k (y) from (3.1.3) in the order 1 (y), 2 (y), . . . , because they satised
(3.1.2). In GREP, however, we do not assume that the k (y) satisfy a relation analogous
to (3.1.2). Therefore, it seems it is not clear a priori in what order the functions k (y)y irk
should be eliminated from (4.1.1) and (4.1.2) for an arbitrary A(y).
Fortunately, there are certain orders of elimination that are universal and give rise
(m, j)
to good sequences of An . We restrict our attention to two types of such sequences
because of their analogy to the column and diagonal sequences of Chapters 13.
1. {An }
j=0 with n = (n 1 , . . . , n m ) xed. Such sequences are analogous to the column
sequences of Chapters 1 and 3. In connection with these sequences, the limiting
process in which j and n = (n 1 , . . . , n m ) is being held xed has been denoted
Process I.
(m, j)
2. {Aq+(,,... ,) }
=0 with j and q = (q1 , . . . , qm ) xed. In particular, the sequence
(m, j)

{A(,,... ,) }
=0 with j xed is of this type. Such sequences are analogous to the diagonal sequences of Chapters 1 and 3. In connection with these sequences, the limiting
process in which n k , k = 1, . . . , m, simultaneously and j is being held xed
has been denoted Process II.
(m, j)

4.4 A Convergence Theory for GREP

89

Numerical experience and theoretical results indicate that Process II is the more ef(m,0)

fective of the two. In view of this, in practice we look at the sequences {A(,,...
,) }=0
as these seem to give the best accuracy for a given number of the A(yi ). The theory we
propose next supports this observation well. It also provides strong justication of the
practical relevance of Process I and Process II. Finally, it is directly applicable to certain
GREPs used in the summation of some oscillatory innite-range integrals and innite
series, as we will see later in the book.
(m, j)
Throughout the rest of this section, we take A(y) and An
to be exactly as in
Denition 4.1.1 and Denition 4.2.1, respectively, with the same notation. We also
dene
N 


 (m, j) 
(4.4.1)
n(m, j) =
ni  ,
i=0
(m, j)

(m, j)

where the ni are as dened in Theorem 4.2.2. Thus, n

is the quantity that controls
(m, j)
the propagation of the errors in the A(yi ) into An . Finally, we let # denote the set
of all polynomials u(t) of degree or less in t.
The following preliminary lemma is the starting point of our analysis.
(m, j)

satises

Lemma 4.4.1 The error in An

j)
A(m,
A=
n

N


(m, j)

ni

i=0

m


k
k (y j+i )[k (y j+i ) u k (y rj+i
)],

(4.4.2)

k=1

where, for each k = 1, . . . , m, u k (t) is in #n k 1 and arbitrary.

Proof. The proof can be achieved by invoking Theorem 4.2.2, just as that of Theorem 4.2.3.


A convergence study of Process I and Process II can now be made by specializing the
polynomials u k (t) in Lemma 4.4.1.

4.4.1 Study of Process I

(m, j)

Theorem 4.4.2 Provided sup j n

= $(m)
n < for xed n 1 , . . . , n m , we have

m 



j)
k (y j+i ) O(y n k rk ) as j .
|A(m,

A|

(4.4.3)
max
n
j
k=1

0iN

If, in addition, k (y j+i ) = O(k (y j )) as j , then

j)
A|
|A(m,
n

m


|k (y j )|O(y nj k rk ) as j ,

(4.4.4)

k=1

whether lim y0+ A(y) exists or not.

n k 1
Proof. Let us pick u k (t) = i=0
ki t i in Lemma 4.4.1, with ki as in (4.1.2). The result
in (4.4.3) follows from the fact that k (y) u k (y rk ) = O(y n k rk ) as y 0+, which is a

90

4 GREP: Further Generalization of the Richardson Extrapolation Process

consequence of (4.1.2), and from the fact that y j > y j+1 > , with lim j y j = 0.
(4.4.4) is a direct consequence of (4.4.3). We leave the details to the reader.




Comparing (4.4.4) with A(y j ) A = m
k=1 k (y j )O(1) as j , that follows from
(m, j)
(4.1.1) with (4.1.2), we realize that, generally speaking, {An }
j=0 converges to A more
quickly than {A(yi )} when the latter converges. Depending on the growth rates of the
(m, j)
k (yi ), {An }
j=0 may converge to A even when {A(yi )} does not. Also, more rened
results similar to those in (1.5.5) and (3.5.18) in case some or all of kn k are zero can
easily be written down. We leave this to the reader. Finally, by imposing the condition
(m, j)
< , we are actually assuming in Theorem 4.4.2 that Process I is a stable
sup j n
extrapolation method with the given k (y) and the yl .

4.4.2 Study of Process II

Theorem 4.4.3 Let the integer j be xed and assume that y j in Denition 4.1.1.
(m, j)
Then, provided supn n
= $(m, j) < , there holds

m 

( j)
j)
(m, j)

A|

$
|
(y)|
E k,n k ,
(4.4.5)
|A(m,
max
k
n
k=1

yI j,N

where I j,N = [y j+N , y j ] and

( j)

E k, = min max |k (y) u(y rk )|, k = 1, . . . , m.

u#1 y[0,y j ]

(4.4.6)

rk

If, in addition, A(y) F(m)

with Bk (t) C [0, ], k = 1, . . . , m, with the same ,
k
and max yI j,N |k (y)| = O(N ) as N for some k , k = 1, . . . , m, then
j)
A(m,
A = O(N n ) as n 1 , . . . , n m , for every > 0,
n

(4.4.7)

where = max{1 , . . . , m } and n = min{n 1 , . . . , n m }. Thus,

j)
A(m,
A = O(n ) as n 1 , . . . , n m , for every > 0,
n

(4.4.8)

holds (i) with no extra condition on the n k when 0, and (ii) provided N = O(n ) as
n for some > 0, when > 0.
Proof. For each k, let us pick u k (t) in Lemma 4.4.1 to be the best polynomial approximation of degree at most n k 1 to the function Bk (t) k (t 1/rk ) on [0, t j ], where t j = y rjk ,
in the maximum norm. Thus, (4.4.6) is equivalent to
( j)

E k, = min max |Bk (t) u(t)| = max |Bk (t) u k (t)|.

u#1 t[0,t j ]

t[0,t j ]

(4.4.9)

The result in (4.4.5) now follows by taking the modulus of both sides of (4.4.2) and manipulating its right-hand side appropriately. When A(y) F(m)
, each Bk (t) is innitely
differentiable on [0, t j ] because t j = y rjk rk . Therefore, from a standard result in polynomial approximation theory (see, for example, Cheney [47]; see also Appendix F), we

4.4 A Convergence Theory for GREP

91

have that E k, = O( ) as , for every > 0. The result in (4.4.7) now follows. The remaining part of the theorem follows from (4.4.7) and its proof is left to
the reader.


( j)

As is clear from (4.4.8), under the conditions stated following (4.4.8), Process II
converges to A, whether {A(yi )} does or not. Recalling the remark following the proof
of Theorem 4.4.2 on Process I, we realize that Process II has convergence properties
(m, j)
< , we
superior to those of Process I. Finally, by imposing the condition supn n
are actually assuming in Theorem 4.4.3 that Process II is a stable extrapolation method
with the given k (y) and the yl .
Note that the condition that N = O(n ) as n is satised in the case n k = qk +
, k = 1, . . . , m, where qk are xed nonnegative integers. In this case, n = O() as
and, consequently, (4.4.8) now reads
Aq+(,... ,) A = O( ) as , for every > 0,
(m, j)

(4.4.10)

both when 0 and when > 0.

Also, in many cases of interest, A(y), in addition to being in F(m)
, is such that its
( j)
k
associated k (y) satisfy E k, = O(exp(k )) as for some k > 0 and k > 0,
k = 1, . . . , m. (In particular, k 1 if Bk (t) k (t 1/rk ) is not only in C [0, brk ] but is
analytic on [0, brk ] as well. Otherwise, 0 < k < 1. For examples of the latter case, we
refer the reader to the papers of Nemeth [218], Miller [212], and Boyd [29], [30].) This
improves the results in Theorem 4.4.3 considerably. In particular, (4.4.10) becomes
Aq+(,... ,) A = O(exp( )) as ,
(m, j)

(4.4.11)

where = min{1 , . . . , m } and = min{k : k = }  for arbitrary  > 0.

4.4.3 Further Remarks on Convergence Theory

Our treatment of the convergence analysis was carried out under the stability conditions
(m, j)
(m, j)
< for Process I and supn n
< for Process II. Although these
that sup j n
conditions can be fullled by clever choices of the {yl } and veried rigorously as well,
they do not have to hold in general. Interestingly, however, even when they do not have
to hold, Process I and Process II may converge, as has been observed numerically in
many cases and as can be shown rigorously in some cases.
Furthermore, we treated the convergence of Process I and Process II for arbitrary m
and for arbitrary {yl }. We can obtain much stronger and interesting results for m = 1 and
for certain choices of {yl } that have been used extensively; for example, in the literature
of numerical quadrature for one- and multi-dimensional integrals. We intend to come
back to GREP(1) in Chapters 810, where we present an extensive analysis for it along
with a new set of mathematical techniques developed recently.

92

4 GREP: Further Generalization of the Richardson Extrapolation Process

4.4.4 Remarks on Convergence of the ki
(m, j)

So far we have been concerned only with the convergence of the An

and mentioned
nothing about the ki , the remaining unknowns in (4.2.1). Just as the k in the rst
generalization of the Richardson extrapolation process tend to the corresponding k under
certain conditions, the ki in GREP tend to the corresponding ki , again under certain
conditions. Even though their convergence may be quick theoretically, they themselves
are prone to severe roundoff in nite-precision arithmetic. Therefore, the ki have very
poor accuracy in nite-precision arithmetic. For this reason, we do not pursue the theoretical study of the ki any further. We refer the reader to Section 4 of Levin and Sidi
[165] for some numerical examples that also provide a few computed values of the ki .

4.4.5 Knowing the Minimal m Pays

(m, j)

given in (4.4.5) and (4.4.6) of TheoLet us go back to the error bound on An

rem 4.4.3. Of course, this bound is valid whether m is minimal or not. In
 addition,


( j)
max
|
(y)|
E k,n k vary
in many cases of interest, the factors $(m, j) and m
yI j,N
k
k=1
slowly with m. [Actually, $(m, j) is smaller for smaller m in most cases.] In other words,
(m, j)
may be nearly independent of the value of m.
the quality of the approximation An
(m, j)
The cost of computing An , on the other hand, increases linearly with m. For example, computation of the approximations A(m,0)
(,... ,) , = 0, 1, . . . , max , that are associated
with Process II entails a cost of mmax + 1 evaluations of A(y). Employing the minimal
value of m, therefore, reduces this cost considerably. This reduction is signicant, especially when the minimal m is a small integer, such as 1 or 2. Such cases do occur most
frequently in applications, as we will see later.
This discussion shows clearly that there is great importance to knowing the minimal
value of m for which A(y) F(m) . In the two examples of GREP that we discuss in the
next two chapters, we present simple heuristic ways to determine the minimal m for
various functions A(y) that are related to some innite-range integrals and innite series
of varying degrees of complexity.

4.5 Remarks on Stability of GREP

(m, j)
dened in (4.4.1) is the quantity that controls the propagation
As mentioned earlier, n
(m, j)
of errors in the A(yl ) into An , the approximation to A. By (4.2.3), we have obviously
(m, j)
(m, j)
that n
1 for all j and n. Thus, the closer n
is to unity, the better the numerical
(m, j)
(m, j)
is a function of the k (y) and the yl . As
stability of An . It is also clear that n
(m, j)
ultimately
k (y) are given and thus not under the users control, the behavior of n

depends on the yl that can be picked at will. Thus, there is great value to knowing how
to pick the yl appropriately.
(m, j)
in the most general case,
Although it is impossible to analyze the behavior of n
we can nevertheless state a few practical conclusions and rules of thumb that have been
derived from numerous applications.
(m, j)
(m, j)
is, the more accurate an approximation An
is to A theorFirst, the smaller n
(m, j)
etically as well. Next, when the sequences of the n
increase mildly or remain

4.6 Extensions of GREP

93
(m, j)

converge
bounded in Process I or Process II, the corresponding sequences of the An
to A quickly.
(m, j)
In practice, the growth of the n
both in Process I and in Process II can be reduced

very effectively by picking the yl such that, for each k, the sequences {k (yl )}l=0
are quickly varying. Quick variation can come about if, for example, k (yl ) = Ck (l)
exp(u k (l)), where Ck (l) and u k (l) are slowly varying functions of l. A function H (l)
is slowly varying in l if liml H (l + 1)/H (l) = 1. Thus, H (l) can be slowly varying if, for instance, it is monotonic and behaves like l as l , for some = 0
and = 0 that can be complex in general. The case in which u k (l) l as l ,
where = 1 and = i, produces one of the best types of quick variation in that now
k (yl ) Ck (l)(1)l as l , and hence liml k (yl+1 )/k (yl ) = 1.
To illustrate these points, let us look at the following two cases:
(i) m = 1 and 1 (y) = y for some = 0, 1, 2, . . . . Quick variation in this case
is achieved by picking yl = y0 l for some (0, 1). With this choice of the yl , we
have 1 (yl ) = y0 e(log )l for all l, and both Process I and Process II are stable. By the
 1+|ci |
(1, j)
for all j
stability theory of Chapter 1 with these yl we actually have () = i=1
|1ci |
(1, j)
and , where ci = +i1 , and hence () are bounded both in j and in . The choice
yl = y0 /(l + 1), on the other hand, leads to extremely unstable extrapolation processes,
as follows from Theorem 2.1.2.
(ii) m = 1 and 1 (y) = ei/y y for some . Best results can be achieved by picking
yl = 1/(l + 1), as this gives 1 (yl ) K l (1)l as l . As shown later, for this
(1, j)
choice of the yl , we actually have () = 1 for all j and , when is real. Hence, both
Process I and Process II are stable.
We illustrate all these points with numerical examples and also with ample theoretical
developments in the next chapters.
4.6 Extensions of GREP
Before closing this chapter, we mention that GREP can be extended to cover those
functions A(y) that are as in (4.1.1), for which the functions k (y) have asymptotic
expansions of the form
k (y)

ki y ki as y 0+,

(4.6.1)

i=0

where the ki are known constants that satisfy

ki = 0, i = 0, 1, . . . , k0 < k1 < , and lim ki = +,
i

(4.6.2)

or of the more general form

k (y)

ki u ki (y) as y 0+,

(4.6.3)

i=0

where u ki (y) are known functions that satisfy

u k,i+1 (y) = o(u ki (y)) as y 0+, i = 0, 1, . . . .

(4.6.4)

94

4 GREP: Further Generalization of the Richardson Extrapolation Process

(m, j)

The approximations An
tions
j)
+
A(yl ) = A(m,
n

m


, where n = (n 1 , . . . , n m ), are now dened by the linear equa-

k (yl )

k=1

n
k 1

ki u ki (yl ), j l j + N ; N =

i=0

m


n k . (4.6.5)

k=1

These extensions of GREP were originally suggested by Sidi [287]. It would be

reasonable to expect such extensions of GREP to be effective if and when needed.
Note that the Richardson extrapolation process of Chapter 1 is an extended GREP(1)
with 1 (y) = 1 and 1i = i+1 , i = 0, 1, . . . , in (4.6.1).
Another example of such transformations is one in which
u ki (y)

kis y srk as y 0+, i = 0, 1, . . . ,

(4.6.6)

s=i

with rk as before. An interesting special case of this is u ki (y) = 1/(y rk )i , where (z)i =


ki y irk
z(z + 1) (z + i 1) is the Pochhammer symbol. (Note that if (y) i=0


rk
as y 0+, then (y) i=0 ki /(y )i as y 0+ as well.) Such an extrapolation method with m = 1 was proposed earlier by Sidi and has been called the Stransformation. This method turns out to be very effective in the summation of strongly
divergent series. We come back to it later.

5
The D-Transformation: A GREP for
Innite-Range Integrals

5.1 The Class B(m) and Related Asymptotic Expansions

In many applications, we may need to compute numerically innite-range integrals of
the form

f (t) dt.
(5.1.1)
I[ f ] =
0

A direct way to achieve this is by truncating the innite range and taking the (numerically
computed) nite-range integral

F(x) =

f (t) dt

(5.1.2)

for some sufciently large x as an approximation to I [ f ]. In many cases, however, f (x)

decays very slowly as x , and this causes F(x) to converge to I [ f ] very slowly
as x . The decay of f (x) may be so slow that we hardly notice the convergence
of F(x) numerically. Even worse is the case in which f (x) decays slowly and oscillates
an innite number of times as x . Thus, this approach of approximating I [ f ] by
F(x) for some large x is of limited use at best.
Commonly occurring
integrals are Fourier cosine

 examples of innite-range
cos
t
g(t)
dt
and
sin
t
g(t)
dt and Hankel transforms
and
sine
transforms
0
0

J
(t)g(t)
dt,
where
J
(x)
is
the
Bessel
function
of
the
rst
kind of order . Now the

0
kernels cos x, sin x, and J (x) of these transforms satisfy linear homogeneous ordinary
differential equations (of order 2) whose coefcients have asymptotic expansions in x 1
for x . In many cases, the functions g(x) also satisfy differential equations of the
same nature, and this puts the integrands cos t g(t), sin t g(t), and J (t)g(t) in some
function classes that we denote B(m) , where m = 1, 2, . . . . The precise description of
B(m) is given in Denition 5.1.2.
As we show later, when the integrand f (x) is in B(m) for some m, F(x) is analogous
to a function A(y) in F(m) with the same m, where y = x 1 . Consequently, GREP(m) can
be applied to obtain good approximations to A I [ f ] at a small cost. The resulting
GREP(m) for this case is now called the D (m) -transformation.

Note that, in case the integral to be computed is a f (t) dt with a = 0, we apply the

D (m) -transformation to 0 f(t) dt, where f(x) = f (a + x).

95

96

5 The D-Transformation: A GREP for Innite-Range Integrals

5.1.1 Description of the Class A( )

Before we embark on the denition of the class B(m) , we need to dene another important
class of functions, which we denote A( ) , that will serve us throughout this chapter and
the rest of this work.
Denition 5.1.1 A function (x) belongs to the set A( ) if it is innitely differentiable
for all large x > 0 and has a Poincare-type asymptotic expansion of the form
(x)

i x i as x ,

(5.1.3)

i=0

and its derivatives have Poincare-type asymptotic expansions obtained by differentiating

that in (5.1.3) formally term by term. If, in addition, 0 = 0 in (5.1.3), then (x) is said
to belong to A( ) strictly. Here is complex in general.
Remarks.
1. A( ) A( 1) A( 2) , so that if A( ) , then, for any positive integer
k, A( +k) but not strictly. Conversely, if A() but not strictly, then A(k)
strictly for a unique positive integer k.
2. If A( ) strictly, then  A( 1) .
3. If A( ) strictly, and (x) = (cx + d) for some arbitrary constants c > 0 and
d, then A( ) strictly as well.
4. If , A( ) , then A( ) as well. (This implies that the zero function is
included in A( ) .) If A( ) and A( +k) strictly for some positive integer k,
then A( +k) strictly.
5. If A( ) and A() , then A( +) ; if, in addition, A() strictly, then
/ A( ) .
6. If A( ) strictly, such that (x) > 0 for all large x, and we dene (x) = [(x)] ,
then A( ) strictly.
7. If A( ) strictly and A(k) strictly for some positive integer k, such that (x) >
0 for all large x > 0, and we dene (x) = ((x)), then A(k ) strictly. Similarly,


i t +i as t 0+, 0 = 0, and if
if (x 1 ) A() strictly so that (t) i=0
(k)
strictly for some positive integer k, such that (x) > 0 for all large x > 0,
A
and we dene (x) = ((x)), then A(k) strictly.
8. If A( ) (strictly) and = 0, then  A( 1) (strictly). If A(0) , then
 A(2) .

by (y)

=
9. Let (x) be in A( ) and satisfy (5.1.3). If we dene the function (y)

is innitely differentiable for 0 y < c for some c > 0, and

y (y 1 ), then (y)


i y i , whether con (i) (0)/i! = i , i = 0, 1, . . . . In other words, the series i=0
vergent or not, is the Maclaurin series of (y).

The next two remarks are immediate

consequences of this.
10. If A(0) , then it is innitely differentiable for all large x > 0 up to and including
x = , although it is not necessarily analytic at x = .

5.1 The Class B(m) and Related Asymptotic Expansions

97

11. If x (x) is innitely differentiable for all large x > 0 up to and including x = ,
and thus has an innite Taylor series expansion in powers of x 1 , then A( ) .
This is true whether the Taylor series converges or not. Furthermore, the asymptotic
expansion of (x) as x is x times the Taylor series.
All these are simple consequences of Denition 5.1.1; we leave their verication to the
reader.
Here are a few simple examples of functions in A( ) for some values of :

The function x 3 + x is in A(3/2) since it has the (convergent) expansion





1/2 1
3/2
3
x +x =x
, x > 1,
i x 2i
i=0
that is also its asymptotic expansion as x .

The function sin(1/ x) is in A(1/2) since it has the (convergent) expansion





1
(1)i 1
sin
, all x > 0,
= x 1/2
(2i + 1)! x i
x
i=0
that is also its asymptotic expansion as x .
Any rational function R(x) whose numerator and denominator polynomials have degrees exactly m and n, respectively, is in A(mn) strictly. In addition, its asymptotic
expansion is convergent for x > a with some a > 0.
If (x) = 0 ext t 1 g(t) dt, where  < 0 and g(t) is in C [0, ) and satises


g(t) i=0
gi t i as t 0+, with g0 = g(0) = 0, and g(t) = O(ect ) as t , for
some constant c, then (x) is in A( ) strictly. This can be shown by using Watsons
lemma, which gives the asymptotic expansion
(x)

gi ( + i)x i as x ,

i=0


and the fact that  (x) = 0 ext t g(t) dt, a known property of Laplace transforms. Here (z) is the Gamma function, as usual. Let us take = 1 and g(t) =


1/(1 + t) as an example. Then (x) x 1  i=0
(1)i i! x i as x . Note that, for
1 t
x
this g(t), (x) = e E 1 (x), where E 1 (x) = x t e dt is the exponential integral,
and the asymptotic expansion of (x) is a divergent series for all x = .
The function e x K 0 (x), where K 0 (x) is the modied Bessel function of order 0 of
the second kind, is in A(1/2) strictly. This can be shown by applying the previous

result to the integral representation of K 0 (x), namely, K 0 (x) = 1 ext (t 2 1)1/2 dt,
following a suitable transformation of variable. Indeed, it has the asymptotic expansion
 


9
1
+
e x K 0 (x)
+

as x .
1
2x
8x
2! (8x)2
Before going on, we would like to note that, by the way A( ) is dened, there may
be any number of functions in A( ) having the same asymptotic expansion. In certain

98

5 The D-Transformation: A GREP for Innite-Range Integrals

places it will be convenient to work with subsets X( ) of A( ) that are dened for all
collectively as follows:
(i) A function belongs to X( ) if either 0 or A( k) strictly for some nonnegative integer k.
(ii) X( ) is closed under addition and multiplication by scalars.
(iii) If X( ) and X() , then X( +) ; if, in addition, A() strictly, then
/ X( ) .
(iv) If X( ) , then  X( 1) .
It is obvious that no two functions in X( ) have the same asymptotic expansion, since
if , X( ) , then either or A( k) strictly for some nonnegative integer
k. Thus, X( ) does not contain functions (x)  0 that satisfy (x) = O(x ) as x
for every > 0, such as exp(cx s ) with c, s > 0.


i x i that converge for all
Functions (x) that are given as sums of series i=0
large x form such a subset; obviously, such functions are of the form (x) = x R(x)
with R(x) analytic at innity. Thus, R(x) can be rational functions that are bounded at
innity, for example.
(Concerning the uniqueness of (x), see the last paragraph of Section A.2 of
Appendix A.)

5.1.2 Description of the Class B(m)

Denition 5.1.2 A function f (x) that is innitely differentiable for all large x belongs
to the set B(m) if it satises a linear homogeneous ordinary differential equation of order
m of the form
f (x) =

m


pk (x) f (k) (x),

(5.1.4)

k=1

where pk A(k) , k = 1, . . . , m, such that pk A(ik ) strictly for some integer i k k.

The following simple result is a consequence of this denition.


Proposition 5.1.3 If f B(m) , then f B(m ) for every m  > m.

Proof. It is enough to consider the case m  = m + 1. Let (5.1.4) be the ordinary differential equation satised by f (x). Applying to both sides of (5.1.4) the differential
operator [1 + (x)d/d x], where (x) is an arbitrary function in A(1) , we have f (x) =

m+1
(k)
(x) with q1 = p1 + p1 , qk = pk + pk + pk1 , k = 2, . . . , m,
k=1 qk (x) f
and qm+1 = pm . From the fact that A(1) and pk A(k) , k = 1, . . . , m, it follows


that qk A(k) , k = 1, . . . , m + 1.
We observe from the proof of Proposition 5.1.3 that if f B(m) , then, for any m  > m,


(k)
there are innitely many differential equations of the form f = m
with qk
k=1 qk f
(k)
(m)
A . An interesting question concerning the situation in which f B with minimal


(k)
m is whether the differential equation f = m
with pk A(k) is unique. By
k=1 pk f

5.1 The Class B(m) and Related Asymptotic Expansions

99

restricting the pk such that pk X(k) with X( ) as dened above, we can actually show
that it is; this is the subject of Proposition 5.1.5 below. In general, we assume that this
differential equation is unique for minimal m, and we invoke this assumption later in
Theorem 5.6.4.
Knowing the minimal m is important for computational economy when using the
D-transformation since the cost the latter increases with increasing m, as will become
clear shortly.
We start with the following auxiliary result.
Proposition 5.1.4 Let f (x) be innitely differentiable for all large x and satisfy an ordi

(k)
(x) with pk
nary differential equation of order m of the form f (x) = m
k=1 pk (x) f
(k )
(k)
A for some integers k . If m is smallest possible, then f (x), k = 0, 1, . . . , m 1,
are independent in the sense that there do not exist functions vk (x), k = 0, 1, . . . , m 1,


(k)
= 0.
not all identically zero, and vk A(k ) with k integers, such that m1
k=0 vk f
(i)
In addition, f (x), i = m, m + 1, . . . , can all be expressed in the form f (i) =

m1
(k)
, where wik A(ik ) for some integers ik . This applies, in particular,
k=0 wik f
(m)
when f B .


Proof. Suppose, to the contrary, that sk=0 vk f (k) = 0 for some s m 1 and vk A(k )



with k integers. If v0  0, then we have f = sk=1 p k f (k) , where p k = vk /v0 A(k ) ,
k an integer, contradicting the assumption that m is minimal. If v0 0, differentiat



(k)
+ vs f (m) = 0.
ing the equality sk=1 vk f (k) = 0 m s times, we have m1
k=1 wk f
(k )
for some integers k . Solving this last equaThe functions wk are obviously in A


(k)
, we obtain the differential equation
tion for f (m) , and substituting in f = m
k=1 pk f

m1
(k)
f = k=1 p k f , where p k = pk pm wk /vs A(k ) for some integers k . This too
contradicts the assumption that m is minimal. We leave the rest of the proof to the reader.


Proposition 5.1.5 If f (x) satises an ordinary differential equation of the form f (x) =

m
(k)
(x) with pk X(k ) for some integers k , and if m is smallest possible, then
k=1 pk (x) f
the functions pk (x) in this differential equation are unique. This applies, in particular,
when f B(m) .
Proof. Suppose, to the contrary, that f (x) satises also the differential equation f =

m
(k)
with qk X(k ) for some integers k , such that pk (x)  qk (x) for at least one
k=1 qk f


(k)
=
value of k. Eliminating f (m) from both differential equations, we obtain m1
k=0 vk f
0, where v0 = qm pm and vk = ( pk qm qk pm ), k = 1, . . . , m 1, vk X(k ) for
some integers k , and vk (x)  0 for at least one value of k. Since m is minimal, this is


impossible by Proposition 5.1.4. Therefore, we must have pk qk , 1 k m.
The following proposition, whose proof we leave to the reader, concerns the derivatives
of f (x) when f B(m) in particular.
Proposition 5.1.6 If f (x) satises an ordinary differential equation of the form f (x) =

m
(k)
(x) with pk A(k ) for some integers k , then f  (x) satises an ordinary
k=1 pk (x) f

100

5 The D-Transformation: A GREP for Innite-Range Integrals

(k+1)
differential equation of the same form, namely, f  (x) = m
(x) with qk
k=1 qk (x) f

(k )
( )
for some integers k , provided [1 p1 (x)] A strictly for some integer . In
A
particular, if f B(m) , then f  B(m) as well, provided limx x 1 p1 (x) = 1.
Let us now give a few examples of functions in the classes B(1) and B(2) . In these
examples, we make free use of the remarks following Denition 5.1.1.
Example 5.1.7 The Bessel function of the rst kind J (x) is in B(2) since it satisx
x2


2
2
(1)
and p2 (x) = x 2 /( 2
es y = 2 x
2 y + 2 x 2 y so that p1 (x) = x/( x ) A
x 2 ) A(0) . The same applies to the Bessel function of the second kind Y (x) and to all
linear combinations b J (x) + cY (x).
Example 5.1.8 The function cos x/x is in B(2) since it satises y = x2 y  y  so that
p1 (x) = 2/x A(1) and p2 (x) = 1 A(0) . The same applies to the function sin x/x
and to all linear combinations B cos x/x + C sin x/x.
Example 5.1.9 The function f (x) = log(1 + x)/(1 + x 2 ) is in B(2) since it satises
y = p1 (x)y  + p2 (x)y  , where p1 (x) = (5x 2 + 4x + 1)/(4x + 2) A(1) strictly and
p2 (x) = (x 2 + 1)(x + 1)/(4x + 2) A(2) strictly.
Example 5.1.10 A function f (x) A( ) strictly, for arbitrary = 0, is in B(1) since it
y  so that p1 (x) = f (x)/ f  (x) A(1) strictly. That p1 A(1) strictly
satises y = ff (x)
(x)
follows from the fact that f  A( 1) strictly.
Example 5.1.11 A function f (x) = e(x) h(x), where A(s) strictly for some positive
integer s and h A( ) for arbitrary , is in B(1) since it satises y = p1 (x)y  with
p1 (x) = 1/[  (x) + h  (x)/ h(x)] A(s+1) and s + 1 0. That p1 A(s+1) can be
seen as follows: Now  A(s1) strictly and h  / h A(1) A(s1) because s 1 is
a nonnegative integer. Therefore,  + h  / h A(s1) strictly. Consequently, p1 = (  +
h  / h)1 A(s+1) strictly.

5.1.3 Asymptotic Expansion of F(x) When f (x) B(m)

We now state a general
concerning the asymptotic
 x theorem due to Levin and Sidi [165]
(m)
behavior of F(x) = 0 f (t) dt as x when f B for some m and is integrable
at innity. The proof of this theorem requires a good understanding of asymptotic expansions and is quite involved. We refer the reader to Section 5.6 for a detailed constructive
proof.
Theorem 5.1.12 Let f (x) be a function in B(m) that is also integrable at innity. Assume,
in addition, that
( j1)

lim pk

(x) f (k j) (x) = 0, k = j, j + 1, . . . , m, j = 1, 2, . . . , m, (5.1.5)

5.1 The Class B(m) and Related Asymptotic Expansions

101

and that
m


l(l 1) (l k + 1) p k = 1, l = 1, 2, 3, . . . ,

(5.1.6)

k=1

where
p k = lim x k pk (x), k = 1, . . . , m.
x

(5.1.7)

Then
F(x) = I [ f ] +

m1


x k f (k) (x)gk (x)

(5.1.8)

k=0

for some integers k k + 1 and functions gk A(0) , k = 0, 1, . . . , m 1. Actually, if

pk A(ik ) strictly for some integer i k k, k = 1, . . . , m, then
k k max{i k+1 , i k+2 1, . . . , i m m + k + 1} k + 1, k = 0, 1, . . . , m 1.
(5.1.9)
Equality holds in (5.1.9) when the integers whose maximum is being considered are
distinct. Finally, being in A(0) , the functions gk (x) have asymptotic expansions of the
form
gk (x)

gki x i as x .

(5.1.10)

i=0

Remarks.
1. By (5.1.7), p k = 0 if and only if pk A(k) strictly. Thus, whenever pk A(ik ) with
i k < k, we have p k = 0. This implies that whenever i k < k, k = 1, . . . , m, we have
p k = 0, k = 1, . . . , m, and the condition in (5.1.6) is automatically satised as the
left-hand side of the inequality there is zero for all values of l.
2. It follows from (5.1.9) that m1 = i m always.
3. Similarly, for m = 1 we have 0 = i 1 precisely.
4. For numerous examples we have treated, equality seems to hold in (5.1.9) for all
k = 1, . . . , m.
5. The integers k and the functions gk (x) in (5.1.8) depend only on the functions pk (x)
in the ordinary differential equation in (5.1.4). This being the case, they are the same
for all solutions f (x) of (5.1.4) that are integrable at innity and that satisfy (5.1.5).
6. From (5.1.5) and (5.1.9), we also have that limx x k f (k) (x) = 0, k =
0, 1, . . . , m 1. Thus, limx x k f (k) (x) = 0, k = 0, 1, . . . , m 1, aswell.

7. Finally, Theorem 5.1.12 says that the function G(x) I [ f ] F(x) = x f (t) dt
is in B(m) if f B(m) too. This follows from the fact that G (k) (x) = f (k1) (x),
k = 1, 2, . . . .
By making the analogy F(x) A(y), x 1 y, x k1 f (k1) (x) k (y) and rk =
1, k = 1, . . . , m, and I [ f ] A, we realize that A(y) is in F(m) . Actually, A(y) is even

102

5 The D-Transformation: A GREP for Innite-Range Integrals

in F(m)
because of the differentiability conditions imposed on f (x) and the pk (x). Finally,
the variable y is continuous for this case.
All the conditions of Theorem 5.1.12 are satised by Examples 5.1.75.1.9. They
are satised by Example 5.1.10, provided  < 1 so that f (x) becomes integrable at
innity. Similarly, they are satised by Example 5.1.11 provided limx (x) =
so that f (x) becomes integrable at innity. We leave the verication of these claims to
the reader.
The numerous examples we have studied seem to indicate that the requirement that
f (x) be in B(m) for some m is the most crucial of the conditions in Theorem 5.1.12.
The rest of the conditions, namely, (5.1.5)(5.1.7), seem to be satised automatically.
Therefore, to decide whether A(y) F(x), where y = x 1 , is in F(m) for some m, it is
practically sufcient to check whether f (x) is in B(m) . Later in this chapter, we provide
some simple ways to check this point.
Finally, even though Theorem 5.1.12 is stated for functions f B(m) that are integrable
at innity, F(x) may satisfy (5.1.8)(5.1.10) also when f B(m) but is not integrable at
innity, at least in some cases. In such a case, the constant I [ f ] in (5.1.8) will be the
antilimit of F(x) as x . In Theorem 5.7.3 at the end of this chapter, we show that
(5.1.8)(5.1.10) hold (i) for all functions f (x) in B(1) that are integrable at innity and
(ii) for a large subset of functions in B(1) that are not integrable there but grow at most
like a power of x as x .
We now demonstrate the result of Theorem 5.1.12 with the two functions f (x) =
J0 (x) and f (x) = sin x/x that were shown to be in B(2) in Examples 5.1.7 and 5.1.8,
respectively.
Example 5.1.13 Let f (x) = J0 (x). From Longman [172], we have the asymptotic
expansion


[(2i + 1)!!]2 1
(1)i
2i + 1 x 2i+1
i=0

2


1
i (2i + 1)!!
+ J1 (x)
(1)
as x ,
2i
2i
+
1
x
i=0

F(x) I [ f ] J0 (x)

(5.1.11)

completely in accordance with Theorem 5.1.12, since J1 (x) = J0 (x).

Example 5.1.14 Let f (x) = sin x/x. We already have an asymptotic expansion for F(x)
that is given in (4.1.7) in Example 4.1.5. Rearranging this expansion, we also have

sin x 
(2i)!(2i + 2)
(1)i
x i=0
x 2i+1


sin x  
(2i)!

(1)i 2i as x ,
x
x
i=0

F(x) I [ f ]

completely in accordance with Theorem 5.1.12.

(5.1.12)

5.2 Denition of the D (m) -Transformation

103

5.1.4 Remarks on the Asymptotic Expansion of F(x) and a Simplication

An interesting and potentially useful feature of Theorem 5.1.12 is the simplicity of the
asymptotic expansion of F(x) given in (5.1.8)(5.1.10). This is noteworthy especially
because the function f (x) may itself have a very complicated asymptotic behavior as
x , and we would normally expect this behavior to show in that of F(x) explicitly.
For instance, f (x) may be oscillatory or monotonic or it may be a combination of
oscillatory and monotonic functions. It is clear that the asymptotic expansion of F(x),
as given in (5.1.8)(5.1.10), contains no reference to any of this; it is expressed solely
in terms of f (x) and the rst m 1 derivatives of it.
We noted earlier that F(x) is analogous to a function A(y) in F(m) , with k (y)
k1 (k1)
f
(x), k = 1, . . . , m. Thus, it seems that knowledge of the integers k may be
x
necessary to proceed with GREP(m) to nd approximations to I [ f ]. Now the k are
determined by the integers i k , and the i k are given by the differential equation that f (x)
satises. As we do not expect, nor do we intend, to know what this differential equation is
in general, determining the k exactly may not be possible in all cases. This could cause
us to conclude that the k (y) may not be known completely, and hence that GREP(m)
may not be applicable for approximating I [ f ]. Really, we do not need to know the k
exactly; we can replace each k by its known upper bound k + 1, and rewrite (5.1.8) in
the form
F(x) = I [ f ] +

m1


x k+1 f (k) (x)h k (x),

(5.1.13)

k=0

where h k (x) = x k k1 gk (x), hence h k A(k k1) A(0) for each k. Note that, if
k = k + 1, then h k (x) = gk (x), while if k < k + 1, then from (5.1.10) we have
h k (x)

h ki x i 0 x 0 + 0 x 1 + + 0 x k k

i=0

+ gk0 x k k1 + gk1 x k k2 + as x . (5.1.14)

Now that we have established the validity of (5.1.13) with h k A(0) for each k, we have
also determined a set of simple and readily known form factors (or shape functions)
k (y), namely, k (y) x k f (k1) (x), k = 1, . . . , m.
5.2 Denition of the D (m) -Transformation
We have seen that the functions h k (x) in (5.1.13) are all in A(0) . Reexpanding them in
negative powers of x + for some xed , which is legitimate, we obtain the following
asymptotic expansion for F(x):
F(x) I [ f ] +

m1

k=0

x k+1 f (k) (x)


i=0

h ki
as x .
(x + )i

(m)
Based on this asymptotic expansion of F(x), we now dene the
 LevinSidi D transformation for approximating the innite-range integral I [ f ] = 0 f (t) dt. As mentioned earlier, the D (m) -transformation is a GREP(m) .

104

5 The D-Transformation: A GREP for Innite-Range Integrals

Denition 5.2.1 Pick an increasing positive sequence {xl } such that liml xl = .
Let n (n 1 , n 2 , . . . , n m ), where n 1 , . . . , n m are nonnegative integers. Then the approx(m, j)
imation Dn
to I [ f ] is dened through the linear system
F(xl ) = Dn(m, j) +

m

k=1

xlk f (k1) (xl )

n
k 1
i=0

m

ki
,
j

j
+
N
;
N
=
nk ,
(xl + )i
k=1

(5.2.1)
> x0 being a parameter at our disposal and ki being the additional (auxiliary) N

1
(m, j)
ci 0 so that D(0,... ,0) = F(x j ) for all j. We call this GREP
unknowns. In (5.2.1), i=0
(m, j)
that generates the Dn
the D (m) -transformation. When there is no room for confusion,
we call it the D-transformation for short. [Of course, in case the k are known, the factors

xlk f (k1) (xl ) in (5.2.1) can be replaced by xl k1 f (k1) (xl ).]

We mention at this point that the P-transformation of Levin [162] for the Bromwich
integral (see Appendix B) is a D (1) -transformation with xl = l + 1, l = 0, 1, . . . .
Remarks.
1. A good choice of the parameter appears to be = 0. We adopt this choice in the
sequel. We have adopted it in all our numerical experiments as well.
2. We observe from (5.2.1) that the input needed for the D (m) -transformation is the
integer m, thefunction f (x) and its rst m 1 derivatives, and the nite intex
grals F(xl ) = 0 l f (t) dt, l = 0, 1, . . . . We dwell on how to determine m later
in this chapter. Sufce it to say that there is no harm done if m is overestimated. However, acceleration of convergence cannot be expected in every case if
m is underestimated. As for the derivatives of f (x), we may assume that they can
somehow be obtained analytically if f (x) is given analytically as well. In difcult
cases, this may be achieved by symbolic computation, for example. In case m is
small, we can compute the derivatives f (i) (x), i = 1, . . . , m 1, also numerically
with sufcient accuracy. We can use the polynomial Richardson extrapolation process of Chapter 2 very effectively for this purpose, as discussed in Section 2.3.


Finally,
F(xl ) can be evaluated in the form F(xl ) = li=0 Fi , where Fi =
 xi
xi1 f (t) dt, with x 1 = 0. The nite-range integrals Fi can be computed numerically to very high accuracy by using, for example, a low-order Gaussian quadrature
formula.
3. Because we have the freedom to choose the xl as we wish, we can make choices
that induce better convergence acceleration to I [ f ] and/or better numerical stability.
This is one of the important advantages of the D-transformation over other existing
methods that we discuss later in this book.
4. The way the D (m) -transformation is dened depends only on the integrand f (x)
and is totally independent of whether or not f (x) is in B(m) and/or satises Theorem 5.1.12. This implies that the D-transformation can be applied to any integral

5.3 A Simplication of the D (m) -Transformation: The s D (m) -Transformation 105

I [ f ], whether f (x) is in B(m) or not. Whether this application is successful depends on the asymptotic behavior of f (x). If f (x) B(m) for some m, then the D (m) transformation will produce good results. It may produce good results with some m
even when f (x) B(q) for some q > m but f (x)  B(m) , at least in some cases of
interest.
(m, j)
via the linear system in (5.2.1) may look very
5. Finally, the denition of Dn
complicated at rst and may create the impression that the D (m) -transformation is
difcult and/or expensive to apply. This impression is wrong, however. The D (m) transformation can be implemented very efciently by the W-algorithm (see Section 7.2) when m = 1 and by the W(m) -algorithm (see Section 7.3) when m 2.

5.2.1 Kernel of the D (m) -Transformation

(m)
From Denition 5.2.1, it
 xis clear that the kernel of the D -transformation (with = 0)
is all integrals F(x) = 0 f (t) dt, such that

f (t) dt =

m


f (k1) (x)

k=1

n
k 1

ki x ki , some nite n k .

(5.2.2)

i=0
(m, j)

For these integrals there holds Dn

= I [ f ] for all j, when n = (n 1 , . . . , n m ). From
the proof of Theorem 5.1.12 in Section 5.6, it can be seen that (5.2.2) holds, for example,


(k)
(x), pk (x) being a polynomial of degree at
when f (x) satises f (x) = m
k=1 pk (x) f
most k for each k. In this case, (5.2.2) holds with n k 1 = k.
Another interesting example of (5.2.2) occurs when f (x) = J2k+1 (x), k = 0, 1, . . . .
Here, J (x) is the Bessel function of the rst kind of order , as before. For instance,
when k = 1, we have






1 24
8

+ 3 + J3 (x) 1 + 2 .
J3 (t) dt = J3 (x)
x
x
x

For details, see Levin and Sidi [165].

5.3 A Simplication of the D (m) -Transformation: The s D (m) -Transformation

In Section 4.3, we mentioned that the form factors k (y) that accompany GREP are not
unique and that they can be replaced by some other functions k (y), at the same time
preserving the form of the expansion in (4.1.1) and (4.1.2). This observation enables us
to simplify the D-transformation considerably in some cases of practical interest, such
as Fourier cosine and sine transforms and Hankel transforms that were mentioned in
Section 5.1.
Let us assume that the integrand f (x) can be written as f (x) = u(x)Q(x), where Q(x)
is a simple function and u(x) is strictly in A( ) for some and may be complicated. By
Q(x) being simple we actually mean that its derivatives are easier to obtain than those

106

5 The D-Transformation: A GREP for Innite-Range Integrals

of f (x) itself. Then, from (5.1.13) we have

F(x) I [ f ] =

m1


x s f (s) (x)h s (x)

s=0

m1


x s


s  
s

s=0

k=0


u (sk) (x)Q (k) (x) h s (x)

m1
m1

k=0

s=k



s (sk)
(x)h s (x)x s Q (k) (x).
u
k

(5.3.1)

Because h s A(0) for all s and u (sk) A( s+k) , we see that u (sk) (x)h s (x)x s

A( +k+s s) so that the term inside the brackets that multiplies Q (k) (x) is in A( +k ) ,
where k , just as k , are integers satisfying k k and hence k k + 1 for each k,
as can be shown by invoking (5.1.9). By the fact that u A( ) strictly, this term can be

written as x k u(x)h k (x) for some h k A(0) . We have thus obtained
F(x) = I [ f ] +

m1


x k [u(x)Q (k) (x)]h k (x).

(5.3.2)

k=0

Consequently, the denition of the D (m) -transformation can be modied by replacing

the form factors xlk f (k1) (xl ) in (5.2.1) by xlk u(xl )Q (k1) (xl ) [or by xl k1 u(xl )Q (k1) (xl )
when the k are known]; everything else stays the same.
For this simplied D (m) -transformation, we do not need to compute derivatives of
f (x); we need only those of Q(x), which are easier to obtain. We denote this new
version of the D (m) -transformation the s D (m) -transformation.
We use this approach later to compute Fourier and Hankel transforms, for which we
derive further simplications and modications of the D-transformation.

5.4 How to Determine m


As mentioned earlier, to apply the D (m) -transformation to a given integral 0 f (t) dt,
we need to have a value for the integer m, for which f B(m) . In this section, we deal
with the question of how to determine the smallest value of m or an upper bound for it
in a simple manner.

5.4.1 By Trial and Error

The simplest approach to this problem is trial and error. We start by applying the D (1) transformation. If this is successful, then we accept m = 1 and stop. If not, we apply
the D (2) -transformation. If this is successful, then we accept m = 2 and stop. Otherwise,
we try the D (3) -transformation, and so on. Our hope is that for some (smallest) value
of m the D (m) -transformation will perform well. Of course, this will be the case when
f B(m) for this m. We also note that the D (m) -transformation may perform well for
some m even when f  B(s) for any s, at least for some cases, as was mentioned earlier.
The trial-and-error approach has almost no extra cost involved as the integrals F(xl )
that we need as input need to be computed once only, and they can be used again for

5.4 How to Determine m

107

each value of m. In addition, the computational effort due to implementing the D (m) transformation for different values of m is small when the W- and W(m) -algorithms of
Chapter 7 are used for this purpose.

5.4.2 Upper Bounds on m

We now use a heuristic approach that allows us to determine by inspection of f some
values for the integer m for which f B(m) . Only this time, we relax somewhat the
conditions in Denition 5.1.2: we assume that f (x) satises a differential equation of
the form (5.1.4) with pk A(k ) , where k is an integer not necessarily less than or equal
to k, k = 1, 2, . . . , m. [Our experience shows that, if f belongs to this new set B(m) and
is integrable at innity, then it belongs to the set B(m) of Denition 5.1.2 as well.] The
idea in the present approach is that the integrand f (x) is viewed as a product or as a sum
of simpler functions, and it is assumed that we know to which classes B(s) these simpler
functions belong.
The following results, which we denote Heuristics 5.4.15.4.3, pertain precisely to
this subject. The demonstrations of these results are based on the assumption that certain
linear systems, whose entries are in A( ) for various integer values of , are invertible.
For this reason, we call these results heuristics, and not lemmas or theorems, and refer
to their demonstrations as proofs.
Heuristic 5.4.1 Let g B(r ) and h B(s) and assume that g and h satisfy different
ordinary differential equations of the form described in Denition 5.1.2. Then
(i) gh B(m) with m r s, and
(ii) g + h B(m) with m r + s.
Proof. Let g and h satisfy
g=

r


u k g (k) and h =

k=1

s


vk h (k) .

(5.4.1)

k=1

To prove part (i), we need to show that gh satises an ordinary differential equation
of the form
rs

pk (gh)(k) ,
(5.4.2)
gh =
k=1

where pk A(k ) , k = 1, . . . , r s, for some integers k , and recall that, by Proposition 5.1.3, the actual order of the ordinary differential equation satised by gh may
possibly be less than r s too.
Let us multiply the two equations in (5.4.1). We obtain
gh =

r 
s


u k vl g (k) h (l) .

(5.4.3)

k=1 l=1

We now want to demonstrate that the r s products g (k) h (l) , 1 k r, 1 l s, can

be expressed as combinations of the r s derivatives (gh)(k) , 1 k r s, under a mild

108

5 The D-Transformation: A GREP for Innite-Range Integrals

condition. We have
(gh)

( j)

j  

j
i=0

g (i) h ( ji) , j = 1, 2, . . . , r s.

(5.4.4)

Let us now use the differential equations in (5.4.1) to express g (0) = g and h (0) = h in
(5.4.4) as combinations of g (i) , 1 i r , and h (i) , 1 i s, respectively. Next, let
us express g (i) with i > r and h (i) with i > s as combinations of g (i) , 1 i r , and
h (i) , 1 i s, respectively, as well. That this is possible can be shown by differentiating
the equations in (5.4.1) as many times as is necessary. For instance, if g (r +1) is required,
we can obtain it from
 

r
r
r


(k)
g =
uk g
=
u k g (k) +
u k g (k+1) ,
(5.4.5)
k=1

in the form

k=1

k=1



r

(u k + u k1 )g (k) /u r .
g (r +1) = (1 u 1 )g 

(5.4.6)

k=2

To obtain g (r +2) as a combination of g (i) , 1 i r , we differentiate (5.4.5) and invoke

(5.4.6) as well. We continue similarly for g (r +3) , g (r +4) , . . . . As a result, (5.4.4) can be
rewritten in the form
(gh)( j) =

s
r 


w jkl g (k) h (l) , j = 1, 2, . . . , r s,

(5.4.7)

k=1 l=1

where w jkl A( jkl ) for some integers jkl . Now (5.4.7) is a linear system of r s equations
for the r s unknowns g (k) h (l) , 1 k r, 1 l s. Assuming that the matrix of this
system is nonsingular for all large x, we can solve by Cramers rule for the products
g (k) h (l) in terms of the (gh)(i) , i = 1, 2, . . . , r s. Substituting this solution in (5.4.3), the
proof of part (i) is achieved.
To prove part (ii), we proceed similarly. What we need to show is that g + h satises
an ordinary differential equation of the form
g+h =

r +s


pk (g + h)(k) ,

(5.4.8)

k=1

where pk A(k ) , k = 1, . . . , r + s, for some integers k , and again recall that, by

Proposition 5.1.3, the order of the differential equation satised by g + h may possibly
be less than r + s too.
Adding the two equations in (5.4.1) we obtain
g+h =

r

k=1

u k g (k) +

s


vl h (l) .

(5.4.9)

l=1

Next, we have
(g + h)( j) = g ( j) + h ( j) , j = 1, 2, . . . , r + s,

(5.4.10)

5.4 How to Determine m

109

which, by the argument given above, can be reexpressed in the form

(g + h)( j) =

r

k=1

wg; jk g (k) +

s


wh; jl h (l) , j = 1, 2, . . . , r + s,

(5.4.11)

l=1

where wg; jk A(g; jk ) and wh; jl A(h; jl ) for some integers g; jk and h; jl . We observe
that (5.4.11) is a linear system of r + s equations for the r + s unknowns g (k) , 1 k r ,


and h (l) , 1 l s. The proof can now be completed as that of part (i).
Heuristic 5.4.2 Let g B(r ) and h B(r ) and assume that g and h satisfy the same
ordinary differential equation of the form described in Denition 5.1.2. Then
(i) gh B(m) with m r (r + 1)/2, and
(ii) g + h B(m) with m r .
Proof. The proof of part (i) is almost the same as that of part (i) of Heuristic 5.4.1, the
difference being that the set {g (k) h (l) : 1 k r, 1 l s} in Heuristic 5.4.1 is now
replaced by the smaller set {g (k) h (l) + g (l) h (k) : 1 k l r } that contains r (r + 1)/2
functions. As for part (ii), its proof follows immediately from the fact that g + h satises
the same ordinary differential equation that g and h satisfy separately. We leave the
details to the reader.


We now give a generalization of part (i) of the previous heuristic. Even though its
proof is similar to that of the latter, it is quite involved. Therefore, we leave its details to
the interested reader.
Heuristic 5.4.3 Let gi B(r ) , i = 1, . . . , , and assume that they all satisfy the same
ordinary differential equation of the
form described in Denition 5.1.2. Dene f =
r +1

(m)
. In particular, if g B(r ) , then (g) B(m)
with m
i=1 gi . Then f B

r +1
.
with m

The preceding results are important because most integrands occurring in practical
applications are products or sums of functions that are in the classes B(r ) for low values
of r , such as r = 1 and r = 2.
Let us now apply these results to a few examples.

m
f i (x), where f i B(1) for each i.
Example 5.4.4 Consider the function f (x) = i=1
(m  )
for some m  m. This occurs,
By part (ii) of Heuristic 5.4.1, we have that f B
in particular, in the following two cases, among many others: (i) when f i A(i ) for
some distinct i = 0, 1, . . . , since h i B(1) , by Example 5.1.10, and (ii) when f i (x) =
ei x h i (x), such that i are distinct and nonzero and h i A(i ) for arbitrary i that are not
necessarily distinct, since f i B(1) , by Example 5.1.11. In both cases, it can be shown
that f B(m) with B(m) exactly as in Denition 5.1.2. We do not prove this here but refer
the reader to the analogous proofs of Theorem 6.8.3 [for case (i)] and of Theorem 6.8.7
[for case (ii)] in the next chapter.

110

5 The D-Transformation: A GREP for Innite-Range Integrals

Example 5.4.5 Consider the function f (x) = g(x)C (x), where C (x) = b J (x) +
cY (x) is an arbitrary solution of the Bessel equation of order and g A( ) for some .
By Example 5.1.10, g B(1) ; by Example 5.1.7, C B(2) . Applying part (i) of Heuristic 5.4.1, we conclude that f B(2) . Indeed, by the fact that C (x) = f (x)/g(x) satises the Bessel equation of order , after some manipulation of the latter, we obtain the
ordinary differential equation f = p1 f  + p2 f  with
p1 (x) =

2x 2 g  (x)/g(x) x
x2
and p2 (x) =
,
w(x)
w(x)

where

w(x) = x

g  (x)
g(x)

2

g  (x)
g(x)

 

g  (x)
+ x 2 2.
g(x)

Consequently, p1 A(1) and p2 A(0) . That is, f B(2) with B(m) precisely as in
Denition 5.1.2. Finally, provided also that  < 1/2 so that f (x) is integrable at
innity, Theorem 5.1.12 applies as all of its conditions are satised, and we also have
0 max{i 1 , i 2 1} = 1 and 1 = i 2 = 0 in Theorem 5.1.12.
Example 5.4.6 Consider the function f (x) = g(x)h(x)C (x), where g(x) and C (x) are
exactly as in the preceding example and h(x) = B cos x + C sin x. From the preceding example, we have that g(x)C (x) is in B(2) . Similarly, the function h(x) is in B(2) as it
satises the ordinary differential equation y  + 2 y = 0. By part (i) of Heuristic 5.4.1,
we therefore conclude that f B(4) . It can be shown by using different techniques that,
when = 1, f B(3) , and this too is in agreement with part (i) of Heuristic 5.4.1.
q
Example 5.4.7 The function f (x) = g(x) i=1 Ci (i x), where Ci (z) are as in the preq
vious examples and g A( ) for some , is in B(2 ) . This follows by repeated application
of part (i) of Heuristic 5.4.1. When 1 , . . . , q are not all distinct, then f B(m) with
m < 2q , as shown by different techniques later.
Example 5.4.8 The function f (x) = (sin x/x)2 is in B(3) . This follows from Example 5.1.8, which says that sin x/x is in B(2) and from part (i) of Heuristic 5.4.2. Indeed,


f (x) satises the ordinary differential equation y = 3k=1 pk (x)y (k) with
p1 (x) =

2x 2 + 3
3
x
A(1) , p2 (x) = A(0) , and p3 (x) = A(1) .
4x
4
8

Finally, Theorem 5.1.12 applies, as all its conditions are satised, and we also have
0 1, 1 0, and 2 = 1. Another way to see that f B(3) is as follows: We can write
f (x) = (1 cos 2x)/(2x 2 ) = 12 x 2 12 x 2 cos 2x. Now, being in A(2) , the function
1 2
x is in B(1) . (See Example 5.1.10.) Next, since 12 x 2 B(1) and cos 2x B(2) , their
2
product is also in B(2) by part (i) of Heuristic 5.4.1. Therefore, by part (ii) of Heuristic 5.4.1, f B(3) .

q
Example 5.4.9 Let g B(r ) and h(x) = s=0 u s (x)(log x)s , with u s A( ) for every s.
Here some or all of the u s (x), 0 s q 1, can be identically zero, but u q (x)  0.

5.5 Numerical Examples

111

Then gh B(q+1)r . To show this, we start with

h (k) (x) =

q


wks (x)(log x)s , wks A( k) , for all k, s.

s=0

Treating q + 1 of these equalities, with k = 1, . . . , q + 1, as a linear system of equations

for the unknowns (log x)s , s = 0, 1, . . . , q, and invoking Cramers rule, we can show

q+1
that (log x)s = k=1 sk (x)h (k) (x), sk A(sk ) for some integers sk . Substituting

q

q+1
these in h(x) = s=0 u s (x)(log x)s , we get h(x) = k=1 ek (x)h (k) (x), ek A(k ) for
some integers k . Thus, h B(q+1) in the relaxed sense. Invoking now part (i) of Heuristic 5.4.1, the result follows.
Important Remark. When we know that f = g + h with g B(r ) and h B(s) , and

we
h separately, we should go ahead and compute 0 g(t) dt and
 can compute g and
(r )
- and D (s) -transformations, respectively, instead of computing
0 h(t) dt by the D(r +s)
-transformation. The reason for this is that less computing is
0 f (t) dt by the D
needed for the D (r ) - and D (s) -transformations than for the D (r +s) -transformation.

5.5 Numerical Examples

We now apply the D-transformation (with = 0 in Denition 5.2.1) to two inniterange integrals of different nature. The numerical results that we present have been
obtained using double-precision arithmetic (approximately 16 decimal digits). For more
examples, we refer the reader to Levin and Sidi [165] and Sidi [272].
Example 5.5.1 Consider the integral

log(1 + t)

dt = log 2 + G = 1.4603621167531195 ,
I[ f ] =
2
1
+
t
4
0
where G is Catalans constant. As we saw in Example 5.1.9, f (x) = log(1 + x)/(1 +
x 2 ) B(2) . Also, f (x) satises all the conditions of Theorem 5.1.12. We applied the
D (2) -transformation to this integral with xl = e0.4l , l = 0, 1, . . . . With this choice of
the xl , the approximations to I [ f ] produced by the D (2) -transformation enjoy a great
amount of stability. Indeed, we have that both xl f (xl ) and xl2 f  (xl ) are O(xl1 log xl ) =
O(le0.4l ) as l . [That is, k (yl ) = Ck (l) exp(u k (l)), with Ck (l) = O(l) as l ,
and u k (l) = 0.4l, and both Ck (l) and u k (l) vary slowly with l. See the discussion on
stability of GREP in Section 4.5.]
(0,2)
The relative errors |F(x2 ) I [ f ]|/|I [ f ]| and | D (0,2)
(,) I [ f ]|/|I [ f ]|, where D (,)
(0,2)
is the computed D(,) , are given in Table 5.5.1. Note that F(x2 ) is the best of all the
(0,2)
. Observe also that the D (0,2)
F(xl ) that are used in computing D(,)
(,) retain their accuracy
with increasing , which is caused by the good stability properties of the extrapolation
process in this example.
Example 5.5.2 Consider the integral

I[ f ] =
0

sin t
t

2
dt =

.
2

112

5 The D-Transformation: A GREP for Innite-Range Integrals

Table 5.5.1: Numerical results for Example 5.5.1

|F(x2 ) I [ f ]|/|I [ f ]|

| D (0,2)
(,) I [ f ]|/|I [ f ]|

0
1
2
3
4
5
6
7
8
9
10

8.14D 01
5.88D 01
3.69D 01
2.13D 01
1.18D 01
6.28D 02
3.27D 02
1.67D 02
8.42D 03
4.19D 03
2.07D 03

8.14D 01
6.13D 01
5.62D 03
5.01D 04
2.19D 05
4.81D 07
1.41D 09
1.69D 12
4.59D 14
4.90D 14
8.59D 14

As we saw in Example 5.4.8, f (x) = (sin x/x)2 B(3) and satises all the conditions of Theorem 5.1.12. We applied the D (3) -transformation to this integral with
xl = 32 (l + 1), l = 0, 1, . . . . The relative errors |F(x3 ) I [ f ]|/|I [ f ]| and | D (0,3)
(,,)
I [ f ]|/|I [ f ]|, where D (0,3) is the computed D (0,3) , are given in Table 5.5.2. Note that
(,,)

(,,)

(0,3)
F(x3 ) is the best of all the F(xl ) that are used in computing D(,,)
. Note also the loss
(0,3)
of accuracy in the D (,,) with increasing that is caused by the instability of the extrapolation process in this example. Nevertheless, we are able to obtain approximations
with as many as 12 correct decimal digits.

5.6 Proof of Theorem 5.1.12

In this section, we give a complete proof
 of Theorem 5.1.12 by actually constructing the
asymptotic expansion of the integral x f (t) dt for x . We start with the following
simple lemma whose proof can be achieved by integration by parts.
Lemma 5.6.1 Let P(x) be differentiable a sufcient number of times on (0, ). Then,
provided
lim P ( j1) (x) f (k j) (x) = 0, j = 1, . . . , k,

(5.6.1)

and provided P(x) f (k) (x) is integrable at innity, we have



k

P(t) f (k) (t) dt =
(1) j P ( j1) (x) f (k j) (x) + (1)k
x

j=1

P (k) (t) f (t) dt.

(5.6.2)
Using Lemma 5.6.1 and the ordinary differential equation (5.1.4) that is satised by
f (x), we can now state the following lemma.
Lemma 5.6.2 Let Q(x) be differentiable a sufcient number of times on (0, ). Dene
Q k (x) =

m

j=k+1

(1) j+k [Q(x) p j (x)]( jk1) , k = 0, 1, . . . , m 1,

(5.6.3)

5.6 Proof of Theorem 5.1.12

113

Table 5.5.2: Numerical results for Example 5.5.2

|F(x3 ) I [ f ]|/|I [ f ]|

| D (0,3)
(,,) I [ f ]|/|I [ f ]|

0
1
2
3
4
5
6
7
8
9
10

2.45D 01
5.02D 02
3.16D 02
2.05D 02
1.67D 02
1.31D 02
1.12D 02
9.65D 03
8.44D 03
7.65D 03
6.78D 03

2.45D 01
4.47D 02
4.30D 03
1.08D 04
2.82D 07
1.60D 07
4.07D 09
2.65D 13
6.39D 12
5.93D 11
6.81D 11

and
Q 1 (x) =

m

(1)k [Q(x) pk (x)](k) .

(5.6.4)

k=1

Then, provided
lim Q k (x) f (k) (x) = 0, k = 0, 1, . . . , m 1,

(5.6.5)

and provided Q(x) f (x) is integrable at innity, we have



Q(t) f (t) dt =

m1


Q k (x) f (k) (x) +

Q 1 (t) f (t) dt.

(5.6.6)

k=0

Proof. By (5.1.4), we rst have



Q(t) f (t) dt =

m 

k=1

[Q(t) pk (t)] f (k) (t) dt.

(5.6.7)

The result follows by applying Lemma 5.6.1 to each integral on the right-hand side of
(5.6.7). We leave the details to the reader.


By imposing additional conditions on the function Q(x) in Lemma 5.6.2, we obtain
the following key result.
Lemma 5.6.3 In Lemma 5.6.2, let Q A(l1) strictly for some integer l, l =
1, 2, 3, . . . . Let also
l =

m

k=1

l(l 1) (l k + 1) p k ,

(5.6.8)

114

5 The D-Transformation: A GREP for Innite-Range Integrals

so that l = 1 by (5.1.6). Dene

Q k (x)
Q k (x) =
, k = 0, 1, . . . , m 1,
1 l
Q 1 (x) l Q(x)
Q 1 (x) =
,
1 l

(5.6.9)

Then


Q(t) f (t) dt =

m1


Q k (x) f (k) (x) +

Q 1 (t) f (t) dt.

(5.6.10)

k=0

We also have that Q k A( k l1) , k = 0, 1, . . . , m 1, and that Q 1 A(l2) .

Thus, Q 1 (x)/Q(x) = O(x 1 ) as x . This result is improved to Q 1 A(2) when
Q(x) is constant for all x.
Proof. By the assumptions that Q A(l1) and pk A(k) , we have Qpk A(kl1)
so that (Qpk )(k) A(l1) , k = 1, . . . , m. Consequently, by (5.6.4), Q 1 A(l1) as
well. Being in A(l1) , Q(x) is of the form Q(x) = q x l1 + R(x) for some nonzero
constant q and some R A(l2) . Thus, Q 1 (x) = l q x l1 + S(x) with l as in (5.6.8)
and some S A(l2) . This can be rewritten as Q 1 (x) = l Q(x) + T (x) for some
T A(l2) whether l = 0 or not. As a result, we have


Q 1 (t) f (t) dt = l

Q(t) f (t) dt +

T (t) f (t) dt.

x


Substituting this in (5.6.6), and solving for x Q(t) f (t) dt, we obtain (5.6.10) with
(5.6.9). Also, Q 1 (x) = T (x)/(1 l ) = [S(x) l R(x)]/(1 l ).
We can similarly show that Q k A( k l1) , k = 0, 1, . . . , m 1. For this, we combine (5.6.3) and the fact that pk A(ik ) , i = 1, . . . , m, in (5.6.9), and then invoke (5.1.9).
We leave the details to the reader.


 What we have achieved through Lemma 5.6.3 is that the new integral

x Q 1 (t) f (t) dt converges to zero as x more quickly than the original integral
x Q(t) f (t) dt. This is an important step in the derivation of the result in Theorem 5.1.12.

We now apply Lemma 5.6.3 to the integral x f (t) dt, that is, we apply it with
Q(x) = 1. We can easily verify that the integrability of f (x) at innity and the conditions
in (5.1.5)(5.1.7) guarantee that Lemma 5.6.3 applies with l = 1. We obtain


f (t) dt =

m1

k=0


b1,k (x) f

(k)

(x) +

b1 (t) f (t) dt,

(5.6.11)

with b1,k A( k ) , k = 0, 1, . . . , m 1, and b1 A(1 1) strictly for some integer

1 1.

5.6 Proof of Theorem 5.1.12

Now let us apply Lemma 5.6.3 to the integral


b1 (t) f (t) dt =

m1



x

b1 (t) f (t) dt. This results in



b2,k (x) f

(k)

115

(x) +

b2 (t) f (t) dt,

(5.6.12)

k=0

with b2,k A( k 1 1) , k = 0, 1, . . . , m 1, and b2 A(2 1) strictly for some integer 2 1 + 1 2.


Next, apply Lemma 5.6.3 to the integral x b2 (t) f (t) dt. This results in


b2 (t) f (t) dt =

m1


b3,k (x) f (k) (x) +

b3 (t) f (t) dt,

(5.6.13)

k=0

with b3,k A( k 2 1) , k = 0, 1, . . . , m 1, and b3 A(3 1) strictly for some integer 3 2 + 1 3.

Continuing this way, and combining the results in (5.6.11), (5.6.12), (5.6.13), and so
on, we obtain


m1

f (t) dt =
[s;k] (x) f (k) (x) +
bs (t) f (t) dt,
(5.6.14)
x

k=0

where s is an arbitrary positive integer and

[s;k] (x) =

s


bi,k (x), k = 0, 1, . . . , m 1.

(5.6.15)

i=1

Here bi,k A( k i1 1) for 0 k m 1 and i 1, and bs A(s 1) strictly for some

integer s s1 + 1 s.
Now, from one of the remarks following the statement of Theorem 5.1.12, we have
limx x 0 f (x) = 0. Thus

bs (t) f (t) dt = o(x s 0 ) as x ,
(5.6.16)
x

Next, using the fact that [s+1;k] (x) = [s;k] (x) + bs+1,k (x) and bs+1,k A( k s 1) , we
can expand [s;k] (x) to obtain
[s;k] (x) =

s


ki x k i + O(x k s 1 ) as x .

(5.6.17)

i=0

Note that ki , 0 i s , remain unchanged in the expansionof [s  ;k] (x) for all s  > s.

Combining (5.6.16) and (5.6.17) in (5.6.14), we see that x f (t) dt has a genuine
asymptotic expansion given by


f (t) dt

m1

k=0

x k f (k) (x)

ki x i as x .

(5.6.18)

i=0

The proof of Theorem 5.1.12 can now be completed easily.

We end this section by showing that the functions x k gk (x) k (x), k =
0, 1, . . . , m 1, can also be determined from a system of linear rst-order

116

5 The D-Transformation: A GREP for Innite-Range Integrals

differential equations that are expressed solely in terms of the pk (x). This theorem
also produces the relation between the k and the k that is given in (5.1.9).
Theorem 5.6.4 Let f B(m) with minimal m and satisfy (5.1.4). Then, the functions
k (x) above satisfy the rst-order linear system of differential equations
pk (x)0 (x) + k (x) + k1 (x) + pk (x) = 0, k = 1, . . . , m; m (x) 0. (5.6.19)
In particular, 0 (x) is a solution of the mth-order differential equation
0 +

m
m


(1)k1 ( pk 0 )(k1) +
(1)k1 pk(k1) = 0.
k=1

(5.6.20)

k=1

Once 0 (x) has been determined, the rest of the k (x) can be obtained from (5.6.19) in
the order k = m 1, m 2, . . . , 1.
Proof. Differentiating the already known relation

m1

f (t) dt =
k (x) f (k) (x),
x

(5.6.21)

k=0

we obtain
f =

m1

k=1

 + k1
k 
0 + 1

(k)

m1
+ 
0 + 1


f (m) .

(5.6.22)

Since f B(m) and m is minimal, the pk (x) are unique by our assumption following
Proposition 5.1.3. We can therefore identify
pk =

k + k1
m1
, k = 1, . . . , m 1, and pm = 
,

0 + 1
0 + 1

(5.6.23)

k1

from which (5.6.19) follows. Applying the differential operator (1)k1 ddx k1 to the kth
equation in (5.6.19) and summing over k, we obtain the differential equation given in
(5.6.20). The rest is immediate.


It is important to note that the differential equation in (5.6.20) actually has a solution


i x 1i for x . It can
for 0 (x) that has an asymptotic expansion of the form i=0
be shown that, under the condition given in (5.1.6), the coefcients i of this expansion
are uniquely determined from (5.6.20). It can further be shown that, with l as dened
in (5.6.8),
1
+ O(x 1 ) as x ,
0 (x) =
1 1
so that 0 (x) + 1 = 1/(1 1 ) + O(x 1 ) and hence (0 + 1) A(0) strictly. Using this
fact in the equation m1 = (0 + 1) pm that follows from the mth of the equations in
(5.6.19), we see that m1 A(im ) strictly. Using these two facts about (0 + 1) and m1

in the equation m2 = (0 + 1) pm1 m1
that follows from the (m 1)st of the
( m2 )
. Continuing this way we can show that
equations in (5.6.19), we see that m2 A

5.7 Characterization and Integral Properties of Functions in B(1)

117


k = (0 + 1) pk+1 k+1
A( k ) , k = m 3, . . . , 1, 0. Also, once 0 (x) has been
determined, the equations from which the rest of the k (x) are determined are algebraic
(as opposed to differential) equations.

5.7 Characterization and Integral Properties of Functions in B(1)

5.7.1 Integral Properties of Functions in A( )
From Heuristic 5.4.1 and Example 5.4.4, it is clear that functions in B(1) are important
building blocks for functions in B(m) with arbitrary m. This provides ample justication
for studying them in detail. Therefore, we close this chapter by proving a characterization
theorem for functions in B(1) and also a theorem on their integral properties. For this, we
need the following result that provides the integral properties of functions in A( ) and
thus extends the list of remarks following Denition 5.1.1.


gi x i as x ,
Theorem 5.7.1 Letg A( ) strictly for some with g(x) i=0
x
and dene G(x) = a g(t) dt, where a > 0 without loss of generality. Then

G(x) = b + c log x + G(x),

(5.7.1)

where b and c are constants and G A( +1) strictly if = 1, while G A(1) if

= 1. When + 1 = 0, 1, 2, . . . , b is simply the value of 0 g(t) dt or of its

Hadamard nite part and c = 0. When + 1 = k {0, 1, 2, . . . }, c = gk . Explicit ex
pressions for b and G(x)
are given in the following proof.

Proof. Let us start by noting that a g(t) dt converges only when  + 1 < 0. Let
N be an arbitrary positive integer greater than  + 1, and dene g (x) = g(x)

N 1
i
. Thus, g A( N ) so that g (x) is O(x N ) as x  and is integrable
i=0 gi x

at innity by the fact that  N + 1 < 0. That is, U N (x) = x g (t) dt exists for all
x a. Therefore, we can write
G(x) =

 x 
N 1
a


gi t i dt + U N (a) U N (x).

Integrating the asymptotic expansion g (x)

which is justied, we obtain
U N (x)

(5.7.2)

i=0


i=N

i=N

gi x i as x term by term,

gi
x i+1 as x .
i +1

(5.7.3)

Carrying out the integration on the right-hand side of (5.7.2), the result follows with
b = U N (a)

N 1

i=0
i=1

gi
a i+1 c log a
i +1

(5.7.4)

118

5 The D-Transformation: A GREP for Innite-Range Integrals

and

G(x)
= U N (x) +

N 1

i=0
i=1

gi
x i+1 .
i +1

(5.7.5)

It can easily be veried from (5.7.4) and (5.7.5) that, despite their appearance, both b

and G(x)
are independent of N . Substituting (5.7.3) in (5.7.5), we see that G(x)
has the
asymptotic expansion

G(x)


i=0
i=1

gi
x i+1 as x .
i +1

We leave the rest of the proof to the reader.

(5.7.6)

5.7.2 A Characterization Theorem for Functions in B(1)

The following result is a characterization theorem for functions in B(1) .
Theorem 5.7.2 A function f (x) is in B(1) if and only if either (i) f A( ) for some
arbitrary = 0, or (ii) f (x) = e(x) h(x), where A(s) strictly for some positive integer
s and h A( ) for some arbitrary . If we write f (x) = p(x) f  (x), then p A() strictly,
with = 1 in case (i) and = s + 1 0 in case (ii).
Proof. The proof of sufciency is already contained in Examples 5.1.10 and 5.1.11.
Therefore, we need be concerned only with necessity. From the fact that f (x) =
f  (x), we have rst f (x) = K exp[G(x)], where K is some constant and G(x) =
p(x)
x
r g(t) dt with g(x) = 1/ p(x) and with r sufciently large so that p(x) = 0 for x r
by the fact that p A() strictly. Now because is an integer 1, we have that
g = 1/ p A(k) strictly, with k = . Thus, k can assume only one of the values
1, 0, 1, 2, . . . . From the preceding theorem, we have that G(x) is of the form

as in (5.7.5) with there replaced by k.

given in (5.7.1) with c = gk+1 and G(x)


Here, the gi are dened via g(x) i=0 gi x ki as x . Therefore, we can write

Let us recall that when k = 1, G A(k+1) strictly. The proof

f (x) = K eb x c exp[G(x)].

of case (ii) can now be completed by identifying (x) = G(x)

and h(x) = K eb x c when
k {0, 1, . . . }. In particular, s = k + 1 and = c. The proof of case (i) is completed by

A(0) strictly, as a result of

recalling that when k = 1, G A(1) so that exp[G(x)]
( )
which, f A strictly with = c. In addition, = 0 necessarily because c = g0 = 0.

5.7.3 Asymptotic Expansion of F(x) When f (x) B(1)

Theorems 5.7.1 and 5.7.2 will now be used in the study of integral properties of functions
f (x) in B(1) that satisfy f (x) = O(x ) as x for some . More specically, we are

5.7 Characterization and Integral Properties of Functions in B(1)

119

concerned with the following two cases in the notation of Theorem 5.7.2.
(i) f A( ) strictly for some = 1, 0, 1, 2, . . . . In this case, f (x) = p(x) f  (x) with
p A() strictly, = 1.
(ii) f (x) = e(x) h(x), where h A( ) strictly for some and A(s) strictly for some
positive integer s, and that either (a) limx  (x) = , or (b) limx  (x)
is nite. In this case, f (x) = p(x) f  (x) with p A() strictly, = s + 1 0.
We already know that in case (i) f (x) is integrable at innity only when  < 1. In
case (ii) f (x) is integrable at innity (a) for all when limx  (x) = and (b) for
 < s 1 when limx  (x) is nite; otherwise, f (x) is not integrable at innity.
The validity of this assertion in case (i) and in case (ii-a) is obvious; for case (ii-b), it
follows from Theorem 5.7.3 that we give next. Finally, in case (ii-a) | f (x)| C1 x  e (x)
as x , and in case (ii-b) | f (x)| C2 x  as x .
Theorem 5.7.3 Let f B(1) be as in the preceding paragraph. Then there exist a constant I [ f ] and a function g A(0) strictly such that
 x
f (t) dt = I [ f ] + x f (x)g(x),
(5.7.7)
F(x) =
0

whether f (x) is integrable at innity or not.

Remark. In case f (x) is integrable
 at innity, this theorem is simply a special case of
Theorem 5.1.12, and I [ f ] = 0 f (t) dt, as can easily be veried. The cases in which
f (x) is not integrable at innity were originally treated by Sidi [286], [300].

Proof. In case (i), Theorem 5.7.1 applies and we have F(x) = b + F(x)
for some con( +1)
( +1)

strictly. Since x f (x) is in A

strictly as well,
stant b and a function F A

g(x) F(x)/[x
f (x)] is in A(0) strictly. Theresult in (5.7.7)
 x now follows with I [ f ] = b.
x
For case (ii), we proceed by integrating 0 f (t) dt = 0 p(t) f  (t) dt by parts:
 x
 x
t=x
f (t) dt = p(t) f (t)t=0
p  (t) f (t) dt.
0

Dening next

u 0 (x) = 1; vi+1 (x) = p(x)u i (x), u i+1 (x) = vi+1
(x), i = 0, 1, . . . , (5.7.8)

we have, again by integration by parts,

 x

t=x

u i (t) f (t) dt = vi+1 (t) f (t) t=0 +
0

u i+1 (t) f (t) dt, i = 0, 1, . . . . (5.7.9)

Summing all these equalities, we obtain


0

f (t) dt =

N

i=1

t=x
vi (t) f (t)t=0 +

u N (t) f (t) dt,

0

(5.7.10)

120

5 The D-Transformation: A GREP for Innite-Range Integrals

where N is as large as we wish. Now by the fact that p A() strictly, we have vi A(i +1)
and u i A(i ) , where i = i( 1), i = 1, 2, . . . . In addition, by the fact that
v1 (x) = p(x), v1 A()
 strictly. Let us pick N > (1 +  )/(1 ) so that N +  <
1. Then the integral 0 u N (t) f (t) dt converges, and we can rewrite (5.7.10) as in



 x
N
f (t) dt = b +
vi (x) f (x)
u N (t) f (t) dt,
(5.7.11)
0

i=1

where
b = lim


N

x0



vi (x) f (x) +

u N (t) f (t) dt.

(5.7.12)

i=1

is an asymptotic sequence as x and expanding the funcRealizing that {vi (x)}i=0

tions vi (x), we see that
N


vi (x) =

i=1

r


i x i + O(vr (x)) as x , 0 = 0, r N ,

(5.7.13)

i=0

where the integers r are dened via r 1 = r + 1. Here 0 , 1 , . . . , r are

obtained by expanding only v1 (x), . . . , vr 1 (x) and are not affected by vi (x), i r .
In addition, by the fact that | f (x)| C1 x  exp[ (x)] as x in case (ii-a) and
| f (x)| C2 x  as x in case (ii-b), we also have that

u N (t) f (t) dt = O(x N +1 f (x)) as x .
(5.7.14)
x

Combining (5.7.13) and (5.7.14) in (5.7.11), and keeping in mind that N is arbitrary, we
thus obtain


 x
r
f (t) dt = I [ f ] + x f (x)
i x i + O(x r 1 ) as x , (5.7.15)
0

i=0

with I [ f ] = b. This completes the proof. (Note that I [ f ] is independent of N despite

its appearance.)


Remarks.
1. When f (x) is not integrable at innity, we can say the following
 about I [ f ]: In case (i),
I [ f ] is the Hadamard nite part
of
the
(divergent)
integral
0 f (t) dt,
 while in case

(ii), I [ f ] is the Abel sum of 0 f (t) dt that is dened as lim0+ 0 et f (t) dt.
We have already shown the former in Theorem 5.7.1. The proof of the latter can be
achieved by applying Theorem 5.7.3 to et f (t) and letting  0+. We leave the
details to the interested reader.
2. Since the asymptotic expansion of F(x) as x is of one and the same form

whether 0 f (t) dt converges or not, the D (1) -transformation can be applied to approximate I [ f ] in all cases considered above.

6
The d-Transformation: A GREP for Innite
Series and Sequences

6.1 The Class b(m) and Related Asymptotic Expansions

The summation of innite series of the form
S({ak }) =

ak

(6.1.1)

k=1

is a very common problem in many branches of science and engineering. A direct way
to achieve this is by computing the sequence {An } of its partial sums, namely,
An =

n


ak , n = 1, 2, . . . ,

(6.1.2)

k=1

hoping that An , for n not too large, approximates the sum S({ak }) sufciently well.
In many cases, however, the terms an decay very slowly as n , and this causes
An to converge to this sum very slowly. In many instances, it may even be practically
impossible to notice the convergence of {An } to S({ak }) numerically. Thus, use of the
sequence of partial sums {An } to approximate S({ak }) may be of limited benet in most
cases of practical interest.


n
Innite series that occur most commonly in applications are power series
n=0 cn z ,




Fourier cosine and sine series n=0 cn cos nx and n=1 cn sin nx, series of orthogo

nal polynomials such as FourierLegendre series
n=0 cn Pn (x), and series of other
special functions. Now the powers z n satisfy the homogeneous two-term recursion relation z n+1 = z z n . Similarly, the functions cos nx and sin nx satisfy the homogeneous
three-term recursion relation f n+1 = 2(cos x) f n f n1 . Both of these recursions involve
coefcients that are constant in n. More generally, the Legendre polynomials Pn (x), as
well as many other special functions, satisfy linear homogeneous (three-term) recursion
relations [or, equivalently, they satisfy linear homogeneous difference equations (of order
2)], whose coefcients have asymptotic expansions in n 1 for n . In many cases,
the coefcients cn in these series as well satisfy difference equations of a similar nature,
and this puts the terms cn z n , cn cos nx, cn sin nx, cn Pn (x), etc., in some sequence classes
that we shall denote b(m) , where m = 1, 2, . . . .
As will be seen shortly, the class b(m) is a discrete counterpart of the class B(m)
that was dened in Chapter 5 on the D-transformation for innite-range integrals. In

121

122

6 The d-Transformation: A GREP for Innite Series and Sequences

fact, all the developments of this chapter parallel those of Chapter 5. In particular, the
d-transformation for the innite series we develop here is a genuine discrete analogue
of the D-transformation.

( )

6.1.1 Description of the Class A0

Before going on to the denition of the class b(m) , we need to dene another class of
( )
functions we denote A0 .
( )

Denition 6.1.1 A function (x) dened for all large x > 0 is in the set A0 if it has a
Poincare-type asymptotic expansion of the form
(x)

i x i as x .

(6.1.3)

i=0
( )

If, in addition, 0 = 0 in (6.1.3), then (x) is said to belong to A0 strictly. Here is

complex in general.
( )

Comparing Denition 6.1.1 with Denition 5.1.1, we realize that A( ) A0 , and that
( )
Remarks 17 following Denition 5.1.1 that apply to the sets A( ) apply to the sets A0 as
( )
( )
well. Remarks 811 are irrelevant to the sets A0 , as functions in A0 are not required to
have any differentiability properties. There is a discrete analogue of Remark 8, however.
For the sake of completeness, we provide all these as Remarks 18 here.
Remarks.
( )

( 1)

( 2)

( )

A0
, so that if A0 , then, for any positive integer k,
1. A0 A0
( +k)
(k)
but not strictly. Conversely, if A()
A0
0 but not strictly, then A0
strictly for a unique positive integer k.
( )
( 1)
/ A0
.
2. If A0 strictly, then
( )
3. If A0 strictly, and (x) = (cx + d) for some arbitrary constants c > 0 and d,
( )
then A0 strictly as well.
( )
( )
4. If , A0 , then A0 as well. (This implies that the zero function is
( )
( )
( +k)
strictly for some positive integer k, then
included in A0 .) If A0 and A0
( +k)
strictly.
A0
( )
( +)
; if, in addition, A()
5. If A0 and A()
0 , then A0
0 strictly, then
( )
.
/ A0
( )
6. If A0 strictly, such that (x) > 0 for all large x, and we dene (x) = [(x)] ,
( )
then A0 strictly.
( )
7. If A0 strictly and A(k)
0 strictly for some positive integer k, such that (x) > 0
(k )
for all large x > 0, and we dene (x) = ((x)), then A0 strictly. Similarly,


(k)
if (t) i=0 i t +i as t 0+, 0 = 0, and if A0 strictly for some positive
integer k, such that (x) > 0 for all large x > 0, and we dene (x) = ((x)), then
A(k)
strictly.
0

6.1 The Class b(m) and Related Asymptotic Expansions

123

( )

8. If A0 (strictly), and (x) = (x + d) (x) for an arbitrary constant d = 0,

( 1)
(2)
(strictly) when = 0. If A(0)
.
then A0
0 , then A0
( )

( )

Before going on, we dene subsets X0 of A0 as follows:

( k)

( )

(i) A function belongs to X0 if either 0 or A0

strictly for some nonnegative integer k.
( )
(ii) X0 is closed under addition and multiplication by scalars.
( )
( +)
; if, in addition, A()
(iii) If X0 and X()
0 , then X0
0 strictly, then
( )
.
/ X0
( )

It is obvious that no two functions in X0 have the same asymptotic expansion,

( )
( k)
strictly for some nonnegative
since if , X0 , then either or A0
( )
integer k. Thus, X0 does not contain functions (x)  0 that satisfy (x) = O(x ) as
x for every > 0, such as exp(cx s ) with c, s > 0. Needless to say, the subsets
( )
X0 dened here are analogous to the subsets X( ) dened in Chapter 5. Furthermore,
( )
X( ) X0 .

6.1.2 Description of the Class b(m)

Denition 6.1.2 A sequence {an } belongs to the set b(m) if it satises a linear homogeneous difference equation of order m of the form
an =

m


pk (n)k an ,

(6.1.4)

k=1
(i k )
where pk A(k)
0 , k = 1, . . . , m, such that pk A0 strictly for some integer i k k.
0
1
Here  an = an ,  an = an = an+1 an , and k an = (k1 an ), k = 2, 3, . . . .

By recalling that
 an =
k

k


(1)

i=0

ki

 
k
an+i ,
i

(6.1.5)

it is easy to see that the difference equation in (6.1.4) can be expressed as an equivalent
(m + 1)-term recursion relation of the form
an+m =

m1


u i (n)an+i ,

(6.1.6)

i=0

with u i A0(i ) , where i are some integers. [Note, however, that not every sequence {an }
that satises such a recursion relation is in b(m) .] Conversely, a recursion relation of the


k
form (6.1.6) can be rewritten as a difference equation of the form m
k=0 vk (n) an = 0

124

6 The d-Transformation: A GREP for Innite Series and Sequences

(k )

with vk A0

for some integers k by using

an+i =

i  

i
k=0

k an .

(6.1.7)

This fact is used in the examples below.

The following results are consequences of Denition 6.1.2 and form discrete analogues
of Propositions 5.1.35.1.6.


Proposition 6.1.3 If {an } b(m) , then {an } b(m ) for every m  > m.
Proof. It is enough to consider m  = m + 1. Let (6.1.4) be the difference equation satised by {an }. Applying to both sides of (6.1.4) the difference operator [1 + (n)],
where (n) is an arbitrary function in A(1)
0 , and using the fact that
(u n vn ) = u n+1 vn + (u n )vn ,

(6.1.8)

k
we have an = m+1
k=1 qk (n) an with q1 (n) = p1 (n) + (n)p1 (n) (n), qk (n) =
pk (n) + (n)pk (n) + (n) pk1 (n + 1), k = 2, . . . , m, and qm+1 (n) = (n) pm (n +
(k)
1). From the fact that A(1)
0 and pk A0 , k = 1, . . . , m, it follows that qk
(k)


A0 , k = 1, . . . , m + 1.
We observe from the proof of Proposition 6.1.3 that if {an } b(m) , then, for any


k
m > m, there are innitely many difference equations of the form an = m
k=1 qk (n) an
(k)
with qk A0 . Analogously to what we did in Chapter 5, we ask concerning the situation in which {an } b(m) with minimal m whether the difference equation an =

m
(k)
k
k=1 pk (n) an with pk A0 is unique. We assume that it is in general; we can
prove that it is when pk (n) are restricted to the subsets X(k)
0 . This assumption can be
invoked to prove a result analogous to Theorem 5.6.4.
Since the cost of the d-transformation increases with increasing m, knowing the minimal m is important for computational economy.
Concerning the minimal m, we can prove the following results, whose proofs we leave
out as they are analogous to those of Propositions 5.1.4 and 5.1.5.


Proposition 6.1.4 If {an } satises a difference equation of order m of the form an =

m
(k )
k
for some integers k , and if m is smallest possible,
k=1 pk (n) an with pk A0
then k an , k = 0, 1, . . . , m 1, are independent in the sense that there do not exist
functions vk (n), k = 0, 1, . . . , m 1, not all identically zero and vk A0(k ) with k


k
integers, such that m1
0. In addition, i an , i = m, m + 1, . . . , can
k=0 vk (n) an =


(ik )
k
all be expressed in the form i an = m1
for some
k=0 wik (n) an , where wik A0
integers ik . This applies, in particular, when {an } b(m) .
Proposition 6.1.5 If {an } satises a difference equation of order m of the form an =

m
(k )
k
for some integers k , and if m is smallest possible,
k=1 pk (n) an with pk X0

6.1 The Class b(m) and Related Asymptotic Expansions

125

then pk (n) in this difference equation are unique. This applies, in particular, when
{an } b(m) .
The following proposition concerns the sequence {an } when {an } b(m) in particular.
Proposition 6.1.6 If {an } satises a difference equation of order m of the form an =

m
(k )
k
k , then {an } satises a difference
k=1 pk (n) an with pk A0 for some integers

m
( )
equation of the same form, namely, an = k=1 qk (n)k+1 an with qk A0 k for some
)
integers k , provided [1 p1 (n)] A(
0 strictly for some integer . In particular, if
(m)
(m)
{an } b , then {an } b as well, provided limn n 1 p1 (n) = 1.
Following are a few examples of sequences {an } in the classes b(1) and b(2) .
Example 6.1.7 The sequence {an } with an = u n /n, where u n = B cos n + C sin n
with arbitrary constants B and C, is in b(2) when = 2k, k = 0, 1, 2, . . . , since it
satises the difference equation an = p1 (n)an + p2 (n)2 an , where
p1 (n) =

( 1)n + 2
n+2
and p2 (n) =
, with = cos = 1.
(1 )(n + 1)
2( 1)(n + 1)

(0)
Thus, p1 A(0)
0 and p2 A0 , and both strictly. Also note that, as p1 (n) and p2 (n) are
rational functions in n, they are both in A(0) strictly. To derive this difference equation, we start with the known three-term recursion relation that is satised by the u n ,
namely, u n+2 = 2 u n+1 u n . We next substitute u k = kak in this recursion, to obtain
(n + 2)an+2 2 (n + 1)an+1 + nan = 0, and nally use (6.1.7). From the recursion relation for the u n , we can also deduce that {u n } b(2) .

Example 6.1.8 The sequence {an } with an = Pn (x)/(n + 1) is in b(2) because it satises
the difference equation an = p1 (n)an + p2 (n)2 an , where
p1 (n) =

2n 2 (1 x) + n(10 7x) + 12 6x
,
(2n 2 + 7n)(1 x) + 7 6x

p2 (n) =

n 2 + 5n + 6
.
(2n 2 + 7n)(1 x) + 7 6x

(0)
(1)
Obviously, p1 A(0)
0 and p2 A0 strictly provided x = 1. (When x = 1, p1 A0
(2)
and p2 A0 .) Also note that, as both p1 (n) and p2 (n) are rational functions of n, they
are in the corresponding sets A( ) strictly as well. This difference equation can be derived
from the known recursion relation for Legendre polynomials Pn (x), namely,

(n + 2)Pn+2 (x) (2n + 3)x Pn+1 (x) + (n + 1)Pn (x) = 0,

rst by substituting Pk (x) = (k + 1)ak to obtain
(n + 2)(n + 3)an+2 (n + 2)(2n + 3)xan+1 + (n + 1)2 an = 0,
and next by using (6.1.7). From the recursion relation for the Pn (x), we can also deduce
that {Pn (x)} b(2) .

126

6 The d-Transformation: A GREP for Innite Series and Sequences

Example 6.1.9 The sequence {an } with an = Hn /[n(n + 1)] and Hn = nk=1 1/k is in
b(2) . To see this, observe that {an } satises the difference equation an = p1 (n)an +
p2 (n)2 an , where
p1 (n) =

(n + 2)(5n + 9)
(n + 2)2 (n + 3)
and p2 (n) =
.
2(2n + 3)
2(2n + 3)

Obviously, p1 A(1) and p2 A(2) strictly, because both are rational functions of n. This
difference equation can be derived as follows: First, we have [n(n + 1)an ] = Hn =
(n + 1)1 . Next, we have {(n + 1)[n(n + 1)an ]} = 0. Finally, we apply (6.1.7). [Note


ci /n i as n , for some constants ci , we have
that, because Hn log n + i=0
an = (n) log n + (n) for some , A(2) .]
( )

Example 6.1.10 The sequence {an } with an = h(n), where h A0 for arbitrary = 0,
is in b(1) . To see this, observe that {an } satises an = p1 (n)an with
1



an
h(n + 1)
1
p1 (n) =
=
1 n +
ei n i as n ,
an
h(n)
i=0
so that p1 A(1)
0 strictly.
As a special case, consider h(n) = n 2 . Then p1 (n) = (n + 1)2 /(2n + 1) exactly.
Thus, in this case p1 is in A(1) strictly, as well as being in A(1)
0 strictly.
( )

Example 6.1.11 The sequence {an } with an = n h(n), where = 1 and h A0 for
arbitrary , is in b(1) . To see this, observe that {an } satises an = p1 (n)an with
1



an
h(n + 1)
1
=
( 1)1 +
ei n i as n ,
p1 (n) =
an
h(n)
i=1
so that p1 A(0)
0 strictly.
As a special case, consider h(n) = n 1 . Then p1 (n) = (n + 1)/[( 1)n 1] exactly.
Thus, in this case p1 is in A(0) strictly, as well as being in A(0)
0 strictly.
6.1.3 Asymptotic Expansion of An When {an } b(m)
We now state a general theorem due to Levin and Sidi [165] concerning the asymptotic behavior of the partial sum An as n when {an } b(m) for some m and


k=1 ak converges. This theorem is the discrete analogue of Theorem 5.1.12 for
innite-range integrals. Its proof is analogous to that of Theorem 5.1.12 given in Section 5.6. It can be achieved by replacing integration by parts by summation by parts
and derivatives by forward differences. See also Levin and Sidi [165] for a sketch.
A complete proof for the case m = 1 is given in Sidi [270], and this proof is now
contained in the proof of Theorem 6.6.6 in this chapter. By imposing the assump

k
tion about the uniqueness of the difference equation an = m
k=1 pk (n) an when m
is minimal, a result analogous to Theorem 5.6.4 can be proved concerning An as
well.

6.1 The Class b(m) and Related Asymptotic Expansions

127

Theorem 6.1.12 Let the sequence {an } be in b(m) and let
k=1 ak be a convergent series.
Assume, in addition, that



lim  j1 pk (n) k j an = 0, k = j, j + 1, . . . , m, j = 1, 2, . . . , m, (6.1.9)
n

and that
m


l(l 1) (l k + 1) p k = 1, l = 1, 2, 3, . . . ,

(6.1.10)

k=1

where
p k = lim n k pk (n), k = 1, . . . , m.
n

(6.1.11)

Then
An1 = S({ak }) +

m1


n k (k an )gk (n)

(6.1.12)

k=0

for some integers k k + 1, and functions gk A(0)

0 , k = 0, 1, . . . , m 1. Actually,
if pk A(i0 k ) strictly for some integer i k k, k = 1, . . . , m, then
k k max{i k+1 , i k+2 1, . . . , i m m + k + 1} k + 1, k = 0, 1, . . . , m 1.
(6.1.13)
Equality holds in (6.1.13) when the integers whose maximum is being considered are
distinct. Finally, being in A(0)
0 , the functions gk (n) have asymptotic expansions of the
form
gk (n)

gki n i as n .

(6.1.14)

i=0

Remarks.
(i k )
1. By (6.1.11), p k = 0 if and only if pk A(k)
0 strictly. Thus, whenever pk A0 with
i k < k, we have p k = 0. This implies that whenever i k < k, k = 1, . . . , m, we have
p k = 0, k = 1, . . . , m, and the condition in (6.1.10) is automatically satised.
2. It follows from (6.1.13) that m1 = i m always.
3. Similarly, for m = 1 we have 0 = i 1 precisely.
4. For numerous examples we have treated, equality seems to hold in (6.1.13) for all
k = 1, . . . , m.
5. The integers k and the functions gk (n) in (6.1.12) depend only on the functions pk (n)
in the difference equation in (6.1.4). This being the case, they are the same for all


solutions an of (6.1.4) that satisfy (6.1.9) and for which
k=1 ak converges.
6. From (6.1.9) and (6.1.13), we also have that limn n k k an = 0, k =
0, 1, . . . , m 1.
7. Finally, Theorem 6.1.12 says that the sequence {G n = S({ak }) An1 } of the re

(m)
mainders of
if {an } b(m) too. This follows from the fact that
k=1 ak is in b
k
k1
 G n =  an , k = 1, 2, . . . .

128

6 The d-Transformation: A GREP for Innite Series and Sequences

By making the analogy An1 A(y), n 1 y, n k1 (k1 an ) k (y) and rk = 1,

k = 1, . . . , m, and S({ak }) A, we realize that A(y) is in F(m) . Finally, the variable y
is discrete in this case and assumes the values 1, 1/2, 1/3, . . . .
All the conditions of Theorem 6.1.12 are satised by Examples 6.1.7 and 6.1.8. They


are satised by Examples 6.1.10 and 6.1.11 provided the corresponding series
k=1 ak
converge. The series of Example 6.1.10 converges provided  < 1. The series of
Example 6.1.11 converges provided either (i) |z| < 1 or (ii) |z| = 1, z = 1, and  < 0.
The many examples we have studied seem to indicate that the requirement that
{an } b(m) for some m is the most crucial of the conditions in Theorem 6.1.12. The
rest of the conditions, namely, (6.1.9)(6.1.11), appear to be satised automatically.
Therefore, to decide whether A(y) An1 , where y = n 1 , is in F(m) for some m, it is
practically sufcient to check whether {an } is in b(m) . Later in this chapter, we provide
some simple ways to check this point.
Finally, even though Theorem 6.1.12 is stated for sequences {an } b(m) for which


ak converges, An may satisfy (6.1.10)(6.1.12) also when {an } b(m) without

k=1

k=1 ak being convergent, at least in some cases. In such a case, the constant S({ak }) in
(6.1.12) will be the antilimit of {An }. In Theorem 6.6.6, we show that (6.1.10)(6.1.12)


hold for all {an } b(1) for which
k=1 ak converge and for a large subset of sequences


a
do
not
converge but an grow at most like a power of n
{an } b(1) for which
k=1 k
as n . As a matter of fact, Theorem 6.6.6 is valid for a class of sequences denoted
b (m) that includes b(1) .
We now demonstrate the result of Theorem 6.1.12 via some examples that were treated
earlier.

Example 6.1.13 Consider the sequence {n z }n=1 with z > 1 that was treated in
Example 4.1.7, and, prior to that, in Example 1.1.4. From Example 6.1.10, we know that
this sequence is in b(1) . The asymptotic expansion in (4.1.9) can be rewritten in the form
An1 (z) + nan

g0i n i as n ,

i=0

for some constants g0i , with g00 = (1 z)1 = 0, completely in accordance with Theorem 6.1.12. This expansion is valid also when z = 1, 0, 1, 2, . . . , as shown in
Example 4.1.7.
Example 6.1.14 Consider the sequence {z n /n}
n=1 with |z| 1 and z = 1 that was
treated in Example 4.1.8. From Example 6.1.11, we know that this sequence is in b(1) .
The asymptotic expansion in (4.1.11) is actually
An1 log (1 z)1 + an

g0i n i as n ,

i=0

for some constants g0i with g00 = (z 1)1 , completely in accordance with Theorem


6.1.12. Furthermore, this expansion is valid also when
k=1 ak diverges with z not on
the branch cut of log(1 z), that is, also when |z| 1 but z  [1, +), as shown in
Example 4.1.8.

6.1 The Class b(m) and Related Asymptotic Expansions

129

Example 6.1.15 Consider the sequence {cos n/n}

n=1 with = 2 k, k = 0, 1,
2, . . . , that was treated in Example 4.1.9. From Example 6.1.7, we know that this
sequence is in b(2) . Substituting



 cos n 

sin n
cos n
1
1
= (1 + n ) csc + cot
(1 + n ) csc 
n
n
n
in the asymptotic expansion of (4.1.12), we obtain








g0i n i + an
g1i n i as n ,
An1 log2 sin  + an
2
i=0
i=0
for some constants g0i and g1i that depend only on , completely in accordance with
Theorem 6.1.12.

6.1.4 Remarks on the Asymptotic Expansion of An and a Simplication

A very useful feature of Theorem 6.1.12 is the simplicity of the asymptotic expansion
of An1 given in (6.1.12)(6.1.14). This is worth noting especially as the term an may
itself have a very complicated behavior as n , and this behavior does not show
explicitly in (6.1.12). It is present there implicitly through k an , k = 0, 1, . . . , m 1,
or equivalently, through an+k , k = 0, 1, . . . , m 1.
As noted earlier, An1 is analogous to an A(y) in F(m) with k (y) n k1 k1 an ,
k = 1, . . . , m. This may give the impression that we have to know the integers k to
proceed with GREP(m) to nd approximations to S({ak }). As we have seen, k depend on
the difference equation in (6.1.4), which we do not expect or intend to know in general.
This lack of precise knowledge of the k could lead us to conclude that we do not know
the k (y) precisely, and hence cannot apply GREP(m) in all cases. Really, we do not need
to know the k exactly; we can replace each k by its known upper bound k + 1, and
rewrite (6.1.12) in the form
An1 = S({ak }) +

m1


n k+1 (k an )h k (n),

(6.1.15)

k=0
( k1)

A(0)
where h k (n) = n k k1 gk (n), hence h k A0 k
0 for each k. Note that, when
k = k + 1, we have h k (n) = gk (n), and when k < k + 1, we have
h k (n)

h ki n i 0 n 0 + 0 n 1 + + 0 n k k

i=0

+ gk0 n k k1 + gk1 n k k2 + as n .

(6.1.16)

Now that we have established the validity of (6.1.15) with h k A(0)

0 for each k,
we have also derived a new set of form factors or shape functions k (y), namely,
k (y) n k k1 an , k = 1, . . . , m. Furthermore, these k (y) are immediately available
and simply expressible in terms of the series elements an . [Compare them with the k (y)
of Chapter 5 that are expressed in terms of the integrand f (x) and its derivatives.]

130

6 The d-Transformation: A GREP for Innite Series and Sequences

Finally, before we turn to the denition of the d-transformation, we make one minor
change in (6.1.15) by adding the term an to both sides. This results in
An = S({ak }) + n[h 0 (n) + n 1 ]an +

m1


n k+1 (k an )h k (n).

(6.1.17)

k=1

As h 0 (n) + n 1 , just as h 0 (n), is in A(0)

0 , the asymptotic expansion of An in (6.1.17) is
of the same form as that of An1 in (6.1.15). Thus, we have
An S({ak }) +

m1


n k+1 (k an )

k=0


h ki
i=0

ni

as n .

(6.1.18)

6.2 Denition of the d (m) -Transformation

Let us reexpand the functions h k (n) in negative powers of n + for some xed , which
is legitimate. The asymptotic expansion in (6.1.18) then assumes the form
An S({ak }) +

m1


n k+1 (k an )

k=0


i=0

h ki
as n .
(n + )i

Based on this asymptotic expansion of An , we now give the denition of the LevinSidi


d-transformation for approximating the sum S({ak }) of the innite series
k=1 ak . As
(m)
(m)
mentioned earlier, the d -transformation is a GREP .

Denition 6.2.1 Pick a sequence of integers {Rl }l=0

, 1 R0 < R1 < R2 < . Let
n (n 1 , . . . , n m ), where n 1 , . . . , n m are nonnegative integers. Then the approximation
(m, j)
to S({ak }) is dened through the linear system
dn

A Rl = dn(m, j) +

m

k=1

Rlk (k1 a Rl )

n
k 1
i=0

m

ki
,
j

j
+
N
;
N
=
nk ,
(Rl + )i
k=1

(6.2.1)
> R0 being a parameter at our disposal and ki being the additional (auxiliary) N

1
(m, j)
ci 0 so that d(0,... ,0) = A j for all j. We call this GREP
unknowns. In (6.2.1), i=0
(m, j)
that generates the dn
the d (m) -transformation. When there is no room for confusion,
we call it the d-transformation for short. [Of course, if the k are known, we can replace

the factors Rlk (k1 a Rl ) in (6.2.1) by Rl k1 (k1 a Rl ).]

Remarks.
1. A good choice of the parameter appears to be = 0. We adopt this choice in the
sequel. We have adopted it in all our numerical experiments as well.
2. From (6.2.1), it is clear that the input needed for the d (m) -transformation is the integer
m, integers Rl , and the sequence elements ai , 1 i R j+N . We consider the issue
of determining m later in this chapter. We note only that no harm is done if m is
overestimated, but no acceleration of convergence should be expected in general if m
is underestimated.

6.2 Denition of the d (m) -Transformation

131

3. Because we have the freedom to pick the integers Rl as we wish, we can pick them
to induce better convergence acceleration to S({ak }) and/or better numerical stability.
This is one of the important advantages of the d-transformation over other convergence
acceleration methods for innite series and sequences that we discuss later.
4. The way the d (m) -transformation is dened depends only on the sequence {ak } and is
totally independent of whether or not this sequence is in b(m) and/or satises Theorem


6.1.12. Therefore, the d-transformation can be applied to any innite series
k=1 ak ,
whether {ak } b(m) or not. Whether this application will produce good approximations to the sum of the series depends on the asymptotic behavior of ak . If {ak } b(m)
for some m, then the d (m) -transformation will produce good results. It may produce
/ b(m) , at
good results with some m even when {ak } b(q) for some q > m but {ak }
least in some cases of interest.
5. Despite its somewhat complicated appearance in Denition 6.2.1, the d (m) transformation can be implemented very efciently by the W-algorithm (see Section 7.2) when m = 1 and by the W(m) -algorithm (see Section 7.3) when m 2. We
present the implementation of the d (1) -transformation via the W-algorithm in he next
section.
6.2.1 Kernel of the d (m) -Transformation
From Denition 6.2.1, it is clear that the kernel of the d (m) -transformation (with = 0)


is all sequences {Ar = rk=1 ak }, such that


k=r +1

ak =

n
m
k 1

(k1 ar )
ki r ki , some nite n k .
k=1

(6.2.2)

i=0
(m, j)

= S({ak }) for all j, when n = (n 1 , . . . , n m ). This

For these sequences, there holds dn


k
holds, for example, when {ak } satises ar = m
k=1 pk (r ) ar , pk (r ) being a polynomial
in r of degree at most k for each k. In this case, (6.2.2) holds with n k 1 = k.
6.2.2 The d (m) -Transformation for Innite Sequences


So far, we have dened the d (m) -transformation for innite series
k=1 ak . Noting that
an = An1 = An An1 , n = 1, 2, . . . , where A0 0, we can reformulate the d (m) transformation for arbitrary innite sequences {Ak }
k=1 , whether (6.1.15) is satised or
not.
Denition 6.2.2 Let n (n 1 , . . . , n m ) and the Rl be as in Denition 6.2.1. Let {Ak } be
(m, j)
a sequence with limit or antilimit A. Then, the approximation dn
to A along with the
additional (auxiliary) unknowns ki , 0 i n k 1, 1 k m, is dened through
the linear system
A Rl = dn(m, j) +

m


Rlk (k A Rl 1 )

k=1

In (6.2.3),

1

i=0 ci

n
k 1
i=0

m

ki
,
j

j
+
N
;
N
=
nk .
(Rl + )i
k=1

(6.2.3)
0 so that

(m, j)
d(0,... ,0)

= A j for all j. Also, A0 0.

132

6 The d-Transformation: A GREP for Innite Series and Sequences

In this form, the d (m) -transformation is a truly universal extrapolation method for
innite sequences.

6.2.3 The Factorial d (m) -Transformation

By rewriting the asymptotic expansions of the functions h k (n) in (6.1.18) in different
forms, as explained in Section 4.6 on extensions of GREP, we obtain other forms of the


d-transformation. For example, we can write an arbitrary asymptotic series i=0
i /n i


as n also in the form
i=0 i /(n)i as n , where (n)0 = 1 and (n)i =
i1
s=0 (n + s) for i 1. Here i = i , for 0 i 2, 3 = 2 + 3 , and so on. For each
i, i is uniquely determined by 0 , 1 , . . . , i .


h ki /n i as n in the form
If we now rewrite the asymptotic expansions i=0


(m)

i=0 h ki /(n)i as n , and proceed as before, we can dene the factorial d

transformation for innite series via the linear equations
A Rl = dn(m, j) +

m


Rlk (k1 a Rl )

n
k 1

k=1

i=0

m

ki
, j l j + N; N =
nk ,
(Rl + )i
k=1

(6.2.4)
and that for innite sequences via linear the equations
A Rl = dn(m, j) +

m


Rlk (k A Rl 1 )

k=1

n
k 1
i=0

m

ki
, j l j + N; N =
nk .
(Rl + )i
k=1

(6.2.5)

6.3 Special Cases with m = 1

6.3.1 The d (1) -Transformation

Let us replace the Rlk in the equations in (6.2.1) by Rl k , and take = 0 for simplicity.
When m = 1, these equations assume the form
A Rl = dn(1, j) + Rl

n1

i
, j l j + n; r = r ar ,
i
R
l
i=0

(6.3.1)

where n now is a positive integer and stands for 1 . These equations can be solved
(1, j)
for dn (with arbitrary Rl ) very simply and efciently via the W-algorithm of [278] as
follows:
( j)

M0 =
( j+1)

Mn( j) =

ARj
1
( j)
, N0 =
, j 0; r = r ar ,
R j
R j
( j)

Mn1 Mn1
1
R 1
j+n R j

( j+1)

, Nn( j) =

( j)

Nn1 Nn1
1
R 1
j+n R j

( j)

dn(1, j) =

Mn

( j)

Nn

, j, n 0.

j 0, n 1.

6.4 How to Determine m

133

6.3.2 The Levin L-Transformation

Choosing Rl = l + 1 in (6.3.1), we obtain
Ar = dn(1, j) + r

n1

i
i=0

ri

, J r J + n; r = r ar , J = j + 1,

(6.3.2)

the resulting d (1) -transformation being nothing but the famous t- and u-transformations
(1, j)
in (6.3.2) by
of Levin [161], with = 0 and = 1, respectively. Let us denote dn
( j)
( j)
Ln . Then Ln has the following known closed form that was given in [161]:




n
i n
n1
A J +i / J +i
n J n1 A J / J
i=0 (1) i (J + i)
( j)

 =
n
n 
Ln =
; J = j + 1.
n
n1
i
n1
 J / J
/ J +i
i=0 (1) i (J + i)
(6.3.3)
The comparative study of Smith and Ford [317], [318] has shown that the Levin trans

formations are extremely efcient for summing a large class of innite series
k=1 ak
(1)
with {an }

b
.
We
return
to
these
transformations
in
Chapters
12
and
19.
n=1
6.3.3 The Sidi S-Transformation

Letting m = 1 and Rl = l + 1, and replacing Rlk by Rl k , the equations in (6.2.4) assume

the form
Ar = dn(1, j) + r

n1

i
, J r J + n; r = r ar , J = j + 1. (6.3.4)
(r
)
i
i=0

The resulting factorial d (1) -transformation is the S-transformation of Sidi. Let us denote
(1, j)
( j)
( j)
dn in (6.3.4) by Sn . Then Sn has the following known closed form that was given
in [277]:


n
i n
n ((J )n1 A J / J )
i=0 (1) i (J + i)n1 A J +i / J +i
( j)

Sn =
; J = j + 1.
=
n
i n
n ((J )n1 / J )
i=0 (1) i (J + i)n1 / J +i
(6.3.5)
The S-transformation was rst used for summing innite power series in the M.Sc. thesis
of Shelef [265] that was done under the supervision of the author. The comparative study
of Grotendorst [116] has shown that it is one of the most effective methods for summing
a large class of everywhere-divergent power series. See also Weniger [353], who called
the method the S-transformation. We return to it in Chapters 12 and 19.

6.4 How to Determine m

As in the case of the D-transformation for innite-range integrals, in applying the


d-transformation to a given innite series
k=1 ak , we must rst assign a value to
the integer m, for which {an } b(m) . In this section, we deal with the question of how

134

6 The d-Transformation: A GREP for Innite Series and Sequences

to determine the smallest value of m or an upper bound for it in a simple manner. Our
approach here parallels that of Section 5.4.

6.4.1 By Trial and Error

The simplest approach to this problem is via trial and error. We start by applying the
d (1) -transformation. If this is successful, then we accept m = 1 and stop. If not, we apply
the d (2) -transformation. If this is successful, then we accept m = 2 and stop. Otherwise,
we try the d (3) -transformation, and so on. We hope that for some (smallest) value of m the
d (m) -transformation will perform well. Of course, this will be the case when {an } b(m)
for this m. We also note that the d (m) -transformation may perform well for some m even
when {an }  b(s) for any s, at least for some cases, as mentioned previously.
The trial-and-error approach has almost no extra cost involved as the partial sums A Rl
that we need as input need to be computed once only, and they can be used again for
each value of m. In addition, the computational effort due to implementing the d (m) transformation for different values of m is small when the W- and W(m) -algorithms of
Chapter 7 are used for this purpose.

6.4.2 Upper Bounds on m

We now use a heuristic approach that allows us to determine by inspection of an some
values for the integer m for which {an } b(m) . Only this time we relax somewhat the
conditions in Denition 6.1.2: we assume that an satises a difference equation of the
k)
form (6.1.4) with pk A(
0 , where k is an integer not necessarily less than or equal to
k, k = 1, 2, . . . , m. [Our experience shows that if {an } belongs to this new set b(m) and


(m)
of Denition 6.1.2 as well.] The
k=1 ak converges, then {an } belongs to the set b
idea in the present approach is that the general term an is viewed as a product or as a sum
of simpler terms, and it is assumed that we know to which classes b(s) the sequences of
these simpler terms belong.
The following results, which we denote Heuristics 6.4.16.4.3, pertain precisely to
this subject. The demonstrations of these results are based on the assumption that certain
( )
linear systems, whose entries are in A0 for various integer values of , are invertible.
For this reason, we call these results heuristics and not lemmas or theorems, and we refer
to their demonstrations as proofs.
Heuristic 6.4.1 Let {gn } b(r ) and {h n } b(s) and assume that {gn } and {h n } satisfy
different difference equations of the form described in Denition 6.1.2. Then
(i) {gn h n } b(m) with m r s, and
(ii) {gn + h n } b(m) with m r + s.
Heuristic 6.4.2 Let {gn } b(r ) and {h n } b(r ) and assume that {gn } and {h n } satisfy
the same difference equation of the form described in Denition 6.1.2. Then
(i) {gn h n } b(m) with m r (r + 1)/2, and
(ii) {gn + h n } b(m) with m r .

6.4 How to Determine m

135

(r )
Heuristic 6.4.3 Let {gn(i) }
n=1 b , i = 1, . . . , , and assume that all sequences
satisfy the same difference equation of the form described
 in Denition 6.1.2. Let
 (i)
r +1
(m)
. In particular, if {gn } b(r ) ,
an = i=1 gn for all n. Then {an } b with m

r +1

(m)
then {(gn ) } b with m
.

The proofs of these are analogous to those of Heuristics 5.4.15.4.3. They can be
achieved by using (6.1.8) and by replacing (5.4.4) with
 j (gn h n ) =

j  

j
i=0

( ji gn+i )(i h n ) =

j  
i  

j
i
i=0

s=0

( js gn )(i h n ). (6.4.1)

We leave the details to the reader.

These results are important because, in most instances of practical interest, an is a
product or a sum of terms gn , h n , . . . , such that {gn }, {h n }, . . . , are in the classes b(r )
for low values of r , such as r = 1 and r = 2.
We now apply these results to a few examples.

m (i)
an , where {an(i) } b(1) for each i. By part (ii) of
Example 6.4.4 Consider an = i=1

Heuristic 6.4.1, we conclude that {an } b(m ) for some m  m. This occurs, in particular,

m
(i )
when an = i=1 h i (n), where h i A0 for some distinct i = 0, since {h i (n)} b(1) for

m n
i h i (n), where i = 1 and are
each i, by Example 6.1.10. It also occurs when an = i=1
( )
distinct and h i A0 i for some arbitrary i not necessarily distinct, since {in h i (n)} b(1)
for each i, by Example 6.1.11. Such sequences {an } are very common and are considered
again and in greater detail in Section 6.8, where we prove that they are in b(m) with b(m)
exactly as in Denition 6.1.2.
( )

Example 6.4.5 Consider an = g(n)u n where g A0 for some and u n = B cos n +

C sin n with B and C constants. From Example 6.1.7, we already know that {u n } b(2) .
( )
From Example 6.1.10, we also know that {g(n)} b(1) since g A0 . By part (i) of
Heuristic 6.4.1, we therefore conclude that {an } b(2) . Indeed, using the technique of
Example 6.1.7, we can show that {an } satises the difference equation an = p1 (n)an +
p2 (n)2 an with
p1 (n) =

2g(n)[g(n + 1) g(n + 2)]

g(n)g(n + 1)
and p2 (n) =
,
w(n)
w(n)

where
w(n) = g(n)g(n + 1) 2 g(n)g(n + 2) + g(n + 1)g(n + 2) and = cos .
(2)
Thus, p1 , p2 A(0)
with b(m) exactly as
0 strictly when = 1, and, therefore, {an } b
in Denition 6.1.2.

Example 6.4.6 Consider an = g(n)Pn (x), where Pn (x) is the Legendre polynomial
( )
of degree n and g A0 for some . We already know that {Pn (x)} b(2) . Also,

136

6 The d-Transformation: A GREP for Innite Series and Sequences

( )

{g(n)} b(1) because g A0 , by Example 6.1.10. By part (i) of Heuristic 6.4.1, we

conclude that {an } b(2) . Indeed, using the technique of Example 6.1.8, we can show
that an = p1 (n)an + p2 (n)2 an , with
p1 (n) =

[x(2n + 3)g(n + 2) 2(n + 2)g(n + 1)]g(n)

,
w(n)
(n + 2)g(n)g(n + 1)
p2 (n) =
,
w(n)

where
w(n) = (n + 2)g(n)g(n + 1) x(2n + 3)g(n)g(n + 2) + (n + 1)g(n + 1)g(n + 2).
(2 +1)

Also, when x = 1, we have that p1 , p2 A(0)

strictly, and,
0 strictly because w A0
therefore, {an } b(2) with b(m) exactly as in Denition 6.1.2. [Recall that Example 6.1.8
is a special case with g(n) = 1/(n + 1).]
Example 6.4.7 Consider an = g(n)u n Pn (x), where g(n), u n , and Pn (x) are as in Examples 6.4.5 and 6.4.6. We already have that {g(n)} b(1) , {u n } b(2) , and {Pn (x)} b(2) .
Thus, from part (i) of Heuristic 6.4.1, we conclude that {an } b(4) . It can be shown by
different techniques that, when x = cos , {an } b(3) , again in agreement with part (i)
of Heuristic 6.4.1.
Example 6.4.8 Consider an = (sin n/n)2 . Then {an } b(3) . This follows from the fact
that {sin n/n} b(2) and from part (i) of Heuristic 6.4.2. Another way to see this is as
follows: First, write an = (1 cos 2n )/(2n 2 ) = 12 n 2 12 n 2 cos 2n. Now { 12 n 2 }
b(1) since 12 x 2 A(2) . Next, because { 12 n 2 } b(1) and {cos 2n} b(2) , their product
is also in b(2) by part (i) of Heuristic 6.4.1. Therefore, by part (ii) of Heuristic 6.4.1,
{an } b(3) .

q
( )
Example 6.4.9 Let {gn } b(r ) and h n = s=0 u s (n)(log n)s , with u s A0 for every s.
Here, some or all of the u s (n), 0 s q 1, can be identically zero, but u q (n)  0.
Then {gn h n } b(q+1)r . To show this we start with the equalities
k h n =

q


( k)

wks (n)(log n)s , wks A0

, for all k, s.

s=0

Treating q + 1 of these equalities with k = 1, . . . , q + 1 as a linear system of equations for the unknowns (log n)s , s = 0, 1, . . . , q, and invoking Cramers rule, we

q+1
( )
for some integers sk . Subcan show that (log n)s = k=1 sk (n)k h n , sk A0 sk

q

q+1
k)
s
stituting these in h n = s=0 u s (n)(log n) , we get h n = k=1 ek (n)k h n , ek A(
0
(q+1)
for some integers k . Thus, {h n } b
in the relaxed sense. Invoking now part
(i) of Heuristic 6.4.1, the result follows. We mention that such sequences (and sums
of them) arise from the trapezoidal rule approximation of simple and multidimensional integrals with corner and/or edge and/or surface singularities. Thus, the result

6.5 Numerical Examples

137

Table 6.5.1: Relative oating-point errors E (0)

= |d(1,0)
S|/|S| and (1,0) for

2
(1)
the Riemann Zeta function series k=1 k via the d -transformation, with
Rl = l + 1. Here S = 2 /6, d(1,0) are the computed d(1,0) , and R is the number
of terms used in computing d(1,0) . d(1,0)
(d) and d(1,0)
(q) are computed,

respectively, in double precision (approximately 16 decimal digits) and in

quadruple precision (approximately 35 decimal digits)

E (0)
(d)

E (0)
(q)

(1,0)

0
2
4
6
8
10
12
14
16
18
20
22
24
26
28
30

1
3
5
7
9
11
13
15
17
19
21
23
25
27
29
31

3.92D 01
1.21D 02
1.90D 05
6.80D 07
1.56D 08
1.85D 10
1.09D 11
2.11D 10
7.99D 09
6.10D 08
1.06D 07
1.24D 05
3.10D 04
3.54D 03
1.80D 02
1.15D 01

3.92D 01
1.21D 02
1.90D 05
6.80D 07
1.56D 08
1.83D 10
6.38D 13
2.38D 14
6.18D 16
7.78D 18
3.05D 20
1.03D 21
1.62D 22
4.33D 21
5.44D 20
4.74D 19

1.00D + 00
9.00D + 00
9.17D + 01
1.01D + 03
1.15D + 04
1.35D + 05
1.60D + 06
1.92D + 07
2.33D + 08
2.85D + 09
3.50D + 10
4.31D + 11
5.33D + 12
6.62D + 13
8.24D + 14
1.03D + 16

obtained here implies that the d-transformation can be used successfully to accelerate
the convergence of sequences of trapezoidal rule approximations. We come back to this in
Chapter 25.
Important Remark. When we know that an = gn + h n with {gn } b(r ) and {h n } b(s) ,


and we can compute gn and h n separately, we should go ahead and compute
n=1 gn


and n=1 h n by the d (r ) - and d (s) -transformations, respectively, instead of computing


(r +s)
-transformation. The reason for this is that, for a given required
n=1 an by the d
level of accuracy, fewer terms are needed for the d (r ) - and d (s) -transformations than for
the d (r +s) -transformation.

6.5 Numerical Examples

We now illustrate the use of the d (m) -transformation (with = 0 in Denition 6.2.1)
in the summation of a few innite series of varying complexity. For more examples,
we refer the reader to Levin and Sidi [165], Sidi and Levin [312], and Sidi [294],
[295].


z
Example 6.5.1 Consider the Riemann Zeta function series
n=1 n , z > 1. We saw
earlier that, for z > 1, this series converges and its sum is (z). Also, as we saw in
Example 6.1.13, {n z } b(1) and satises all the conditions of Theorem 6.1.12.

138

6 The d-Transformation: A GREP for Innite Series and Sequences

Table 6.5.2: Relative oating-point errors E (0)

= |d(1,0)
S|/|S| and (1,0) for

2
(1)
the Riemann Zeta function series k=1 k via the d -transformation, with Rl
as in (6.5.1) and = 1.3 there. Here S = 2 /6, d(1,0)
are the computed d(1,0) ,

(1,0) (1,0)
and R is the number of terms used in computing d . d (d) and d(1,0)
(q) are

computed, respectively, in double precision (approximately 16 decimal digits)

and in quadruple precision (approximately 35 decimal digits)

E (0)
(d)

E (0)
(q)

(1,0)

0
2
4
6
8
10
12
14
16
18
20
22
24
26
28
30

1
3
5
7
11
18
29
48
80
135
227
383
646
1090
1842
3112

3.92D 01
1.21D 02
1.90D 05
6.80D 07
1.14D 08
6.58D 11
1.58D 13
1.55D 15
7.11D 15
5.46D 14
8.22D 14
1.91D 13
1.00D 13
4.21D 14
6.07D 14
1.24D 13

3.92D 01
1.21D 02
1.90D 05
6.80D 07
1.14D 08
6.59D 11
1.20D 13
4.05D 17
2.35D 19
1.43D 22
2.80D 26
2.02D 30
4.43D 32
7.24D 32
3.27D 31
2.52D 31

1.00D + 00
9.00D + 00
9.17D + 01
1.01D + 03
3.04D + 03
3.75D + 03
3.36D + 03
3.24D + 03
2.76D + 03
2.32D + 03
2.09D + 03
1.97D + 03
1.90D + 03
1.86D + 03
1.82D + 03
1.79D + 03

In Tables 6.5.1 and 6.5.2, we present the numerical results obtained by applying the
d -transformation to this series with z = 2, for which we have (2) = 2 /6. We have
done the computations once by choosing Rl = l + 1 and once by choosing
(1)


R0 = 1, Rl =

Rl1 + 1 if  Rl1  = Rl1

, l = 1, 2, . . . ; for some > 1,
 Rl1  otherwise
(6.5.1)

with = 1.3. The former choice of the Rl gives rise to the Levin u-transformation, as
we mentioned previously. The latter choice is quite different and induces a great amount
of numerical stability. [The choice of the Rl as in (6.5.1) is called geometric progression
sampling (GPS) and is discussed in detail in Chapter 10.]
Note that, in both tables, we have given oating-point arithmetic results in quadruple
precision, as well as in double precision. These show that, with the rst choice of the
Rl , the maximum accuracy that can be attained is 11 digits in double precision and
22 digits in quadruple precision. Adding more terms to the process does not improve
the accuracy; to the contrary, the accuracy dwindles quite quickly. With the Rl as in
(6.5.1), on the other hand, we are able to improve the accuracy to almost machine
precision.


Example 6.5.2 Consider the Fourier cosine series
n=1 [cos(2n 1)]/(2n 1). This
1
series converges to the function f ( ) = 2 log | tan(/2)| for every real except at

6.5 Numerical Examples

139

(2,0)
Table 6.5.3: Relative oating-point errors E (0)
= |d (,) S|/|S| for the Fourier
cosine series of Example 6.5.2 via the d (2) -transformation, with Rl = l + 1 in the third
and fth columns and with Rl = 2(l + 1) in the seventh column. Here S = f (), d(2,0)
(,)
(2,0)
(2,0)
are the computed d(,)
, and R2 is the number of terms used in computing d(,)
.
are
computed
in
double
precision
(approximately
16
decimal
digits)
d(2,0)
(,)

R2

E (0)
( = /3)

R2

E (0)
( = /6)

R2

E (0)
( = /6)

0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

1
3
5
7
9
11
13
15
17
19
21
23
25
27
29
31

8.21D 01
1.56D + 00
6.94D 03
1.71D 05
2.84D 07
5.56D 08
3.44D 09
6.44D 11
9.51D 13
1.01D 13
3.68D 14
9.50D 15
2.63D 15
6.06D 16
8.09D 16
6.27D 15

1
3
5
7
9
11
13
15
17
19
21
23
25
27
29
31

3.15D 01
1.39D 01
7.49D 03
2.77D 02
3.34D 03
2.30D 05
9.65D 06
4.78D 07
1.33D 07
4.71D 08
5.69D 08
4.62D 10
4.50D 11
2.17D 11
5.32D 11
3.57D 11

2
6
10
14
18
22
26
30
34
38
42
46
50
54
58
62

3.15D 01
1.27D 01
2.01D 03
1.14D 04
6.76D 06
2.27D 07
4.12D 09
4.28D 11
3.97D 13
2.02D 14
1.35D 15
1.18D 15
1.18D 15
5.06D 16
9.10D 15
9.39D 14

= k , k = 0, 1, 2, . . . , where f () has singularities. It is easy to show that

{[cos(2n 1) ]/(2n 1)} b(2) . Consequently, the d (2) -transformation is very effective. It turns out that the choice Rl = (l + 1) with some positive integer is suitable.
When is away from the points of singularity, = 1 is sufcient. As approaches a
point of singularity, should be increased. [The choice of the Rl as given here is called
arithmetic progression sampling (APS) and is discussed in detail in Chapter 10.]
Table 6.5.3 presents the results obtained for = /3 and = /6 in double-precision
arithmetic. While machine precision is reached for = /3 with Rl = l + 1, only
11-digit accuracy is achieved for = /6 with the same choice of the Rl . Machine
precision is achieved for = /6 with Rl = 2(l + 1). This phenomenon is analyzed
and explained in detail in later chapters.


2n+1
Example 6.5.3 Consider the Legendre series
n=1 n(n+1) Pn (x). This series converges to
the function f (x) = log[(1 x)/2] 1 for 1 x < 1, and it diverges for all other x.
The function f (x) has a branch cut along [1, +), whereas it is well-dened for x < 1
2n+1
Pn (x)} b(2) .
even though the series diverges for such x. It is easy to show that { n(n+1)
(2)
Consequently, the d -transformation is very effective in this case too. Again, the choice
Rl = (l + 1) with some positive integer is suitable. When x < 1 and x is away
from 1, the branch point of f (x), = 1 is sufcient. As x approaches 1, should be
increased.
Table 6.5.4 presents the results obtained for x = 0.3 with Rl = l + 1, for x = 0.9
with Rl = 3(l + 1), and for x = 1.5 with Rl = l + 1, in double-precision arithmetic.

140

6 The d-Transformation: A GREP for Innite Series and Sequences

(2,0)
Table 6.5.4: Relative oating-point errors E (0)
= |d (,) S|/|S| for the Legendre
series of Example 6.5.3 via the d (2) -transformation, with Rl = l + 1 in the third
and seventh columns and with Rl = 3(l + 1) in the fth column. Here S = f (x), d(2,0)
(,)
(2,0)
(2,0)
are the computed d(,)
, and R2 is the number of terms used in computing d(,)
.
d(2,0)
are
computed
in
double
precision
(approximately
16
decimal
digits)
(,)

R2

E (0)
(x = 0.3)

R2

E (0)
(x = 0.9)

R2

E (0)
(x = 1.5)

0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

1
3
5
7
9
11
13
15
17
19
21
23
25
27
29
31

4.00D 01
2.39D 01
5.42D 02
1.45D 03
1.67D 04
8.91D 06
7.80D 07
2.72D 07
4.26D 08
3.22D 09
2.99D 10
1.58D 11
2.59D 13
4.14D 14
3.11D 15
3.76D 15

3
9
15
21
27
33
39
45
51
57
63
69
75
81
87
93

2.26D 01
1.23D 01
2.08D 02
8.43D 04
9.47D 05
2.76D 07
1.10D 06
1.45D 07
3.39D 09
7.16D 09
2.99D 10
4.72D 12
2.28D 13
1.91D 14
4.66D 15
4.89D 15

1
3
5
7
9
11
13
15
17
19
21
23
25
27
29
31

1.03D + 00
1.07D 01
5.77D 04
5.73D 04
1.35D 06
2.49D 09
2.55D 10
3.28D 13
3.52D 13
4.67D 13
6.20D 13
2.97D 13
2.27D 12
1.86D 11
3.42D 11
1.13D 11

Note that almost machine precision is reached when x = 0.3 and x = 0.9 for which the
series converges. Even though the series diverges for x = 1.5, the d (2) -transformation
produces f (x) to almost 13-digit accuracy. This suggests that the d (m) -transformation
may be a useful tool for analytic continuation. (Note that the accuracy for x = 1.5
decreases as we increase the number of terms of the series used in extrapolation. This is
because the partial sums An are unbounded as n .)

6.6 A Further Class of Sequences in b(m) : The Class b (m)

In the preceding sections, we presented examples of sequences in b(m) and showed how
we can construct others via Heuristics 6.4.1 and 6.4.2. In this section, we would like to
derive a very general class of sequences in b(m) for arbitrary m. For m = 1, this class
will turn out to be all of b(1) . For m 2, we will see that, even though it is not all of
( )
b(m) , it is quite large nevertheless. We derive this class by extending A0 and b(1) in an
appropriate fashion.
We note that the contents of this section are related to the paper by Wimp [362].
( ,m) and Its Summation Properties
6.6.1 The Function Class A
0
( )

We start by generalizing the function class A0 . In addition to being of interest in itself,

this generalization will also help us keep the notation simple in the sequel.

6.6 A Further Class of Sequences in b(m) : The Class b (m)

( ,m)

Denition 6.6.1 A function (x) dened for all large x is in the set A
0
integer, if it has a Poincare-type asymptotic expansion of the form
(x)

141
, m a positive

i x i/m as x .

(6.6.1)

i=0
( ,m)

In addition, if 0 = 0 in (6.6.1), then (x) is said to belong to A

0
complex in general.
( ,1)

strictly. Here is

( )

With this denition, we obviously have A

= A0 . Furthermore, all the remarks on
0
( )
( ,m) with suitable and obvious
the sets A0 following Denition 6.1.1 apply to the sets A
0
modications, which we leave to the reader.


( ,m) , then (x) = m1 qs (x), where qs
It is interesting to note that, if A
s=0
0


( s/m)

A0
and qs (x) i=0
s+mi x s/mi as x , s = 0, 1, . . . , m 1. In other
( )
words, (x) is the sum of m functions qs (x), each in a class A0 s . Thus, in view of Example
6.4.4 and by Theorem 6.8.3 of Section 6.8, we may conclude rigorously that if a sequence
( ,m) for some = s/m, s = 0, 1, . . . , m 1,
{an } is such that an = (n), where A
0
(m)
( ,m) may become useful in
then {an } b . Based on this, we realize that the class A
0
(m)
constructing sequences in b , hence deserves some attention.
( ,m) .
We start with the following result on the summation properties of functions in A
0
This result extends the list of remarks that succeeds Denition 6.1.1.
( ,m) strictly for some with g(x)
Theorem 6.6.2 Let g A
0

x , and dene G(n) = rn1
=1 g(r ). Then

i=0

gi x i/m as

G(n) = b + c log n + G(n),

(6.6.2)

( +1,m) strictly,
( +1,m) . If = 1, then G A
where b and c are constants and G A
0
0
(1/m,m)

if = 1. If + 1 = i/m, i = 0, 1, . . . , then b is the limit or

while G A0
antilimit of G(n) as n and c = 0. When + 1 = k/m for some integer k 0,
c = gk . Finally,

G(n)
=

m1

i=0
i/m=1

gi
n i/m+1 + O(n ) as n .
i/m + 1

(6.6.3)

Proof. Let N be an arbitrary integer greater than ( + 1)m, and dene g(x)
=



N 1
( N /m,m)
i/m

) converges since g(x)

=
. Thus, g A0
, and r =n g(r
g(x) i=0 gi x


)=
O(x N /m ) as x and  N /m < 1. In fact, U N (n) = r=n g(r
O(n N /m+1 ) as n . Consequently,

n1  
N 1

gi r i/m + U N (1) U N (n)
G(n) =
r =1

N 1

i=0

i=0

gi


n1
r =1


r i/m + U N (1) + O(n N /m+1 ) as n .

(6.6.4)

142

6 The d-Transformation: A GREP for Innite Series and Sequences

From Example 1.1.4 on the Zeta function series, we know that for i/m = 1,
n1


r i/m = ( + i/m) + Ti (n), Ti A( i/m+1) strictly,

(6.6.5)

r =1

while if k/m = 1 for some nonnegative integer k,

n1


r k/m =

r =1

n1


r 1 = log n + C + T (n), T A(1) strictly,

(6.6.6)

r =1

where C is Eulers constant. The result in (6.6.2) can now be obtained by substituting
(6.6.5) and (6.6.6) in (6.6.4), and recalling that g0 = 0 and that N is arbitrary. We also

N 1
gi ( + i/m)
realize that if i/m = 1, i = 0, 1, . . . , then b = U N (1) + i=0
and b is independent of N , and c = 0. If k/m = 1 for a nonnegative integer k, then

N 1
gi ( + i/m) and b is again independent of N , and
b = U N (1) + gk C +
i=0
i=k
1
c = gk . Finally, (6.6.3) follows from the fact that Ti (n) = i/m+1
n i/m+1 +
i/m
O(n
) as n whenever i/m = 1.

6.6.2 The Sequence Class b (m) and a Characterization Theorem

( ,m) already dened, we now go on to dene the sequence class b (m)
With the classes A
0

as a generalization of the class b(1) .

Denition 6.6.3 A sequence {an } belongs to the set b (m) if it satises a linear homo (q/m,m)
geneous difference equation of rst order of the form an = p(n)an with p A
0
for some integer q m.
Note the analogy between the classes b (m) and b(1) . Also note that b (1) is simply b(1) .
Now the difference equation in Denition 6.6.3 can also be expressed as a two-term
recursion relation of the form an+1 = c(n)an with c(n) = 1 + 1/ p(n). We make use of
this recursion relation to explore the nature of the sequences in b (m) . In this respect, the
following theorem that is closely related to the theory of Birkhoff and Trjitzinsky [25]
on general linear difference equations is of major importance.
Theorem 6.6.4
(,m)

(i) Let an+1 = c(n)an such that c A

0
is of the form

strictly with in general complex. Then an

an = [(n 1)!] exp [Q(n)] n w(n),

(6.6.7)

where
Q(n) =

m1

i=0

(0,m) strictly.
i n 1i/m and w A
0

(6.6.8)

6.6 A Further Class of Sequences in b(m) : The Class b (m)

Given that c(n)

i=0 ci n

e0 = c0 ; i =

i/m

143

as n , we have

i
, i = 1, . . . , m 1; = m ,
1 i/m

(6.6.9)

where the i are dened via

s 
 m
m
m

(1)s+1 
ci i
z
=
i z i + O(z m+1 ) as z 0.
s
c
0
s=1
i=1
i=1

(6.6.10)

(ii) The converse is also true, that is, if an is as in (6.6.7) and (6.6.8), then an+1 = c(n)an
(,m) strictly.
with c A
0
(iii) Finally, (a) 1 = = m1 = 0 if and only if c1 = = cm1 = 0, and
(b) 1 = = r 1 = 0 and r = 0 if and only if c1 = = cr 1 = 0 and
cr = 0, r {1, . . . , m 1}.
Remark. When m = 1, we have an = [(n 1)!] n n w(n), with = c0 , = c1 /c0 ,
and w(n) A(0)
0 strictly, as follows easily from (6.6.7)(6.6.10) above.


n1

Proof. We start with the fact that an = a1

r =1 c(r ) . Next, writing c(x) = c0 x u(x),


(0,m) and u(x) 1 + v(x) 1 + (ci /c0 )x i/m as x , we obtain
where u A
i=1

an = a1 c0n1 [(n 1)!]

n1



u(r ) .

(6.6.11)

r =1

Now
n1






n1
n1
u(r ) = exp
log u(r ) = exp
log[1 + v(r )] .

r =1

r =1

(6.6.12)

r =1

(1)s+1 s
By the fact that log(1 + z) =
z for |z| < 1 and v(x) = O(x 1/m ) = o(1)
s=1
s

(1)s+1
as x , it follows that log u(x) = s=1 s [v(x)]s for all large x. Since this
(convergent) series also gives the asymptotic expansion of log u(x) as x , we see that


(1/m,m) . Actually, log u(x) i x i/m as x , where 1 , . . . , m
log u(x) A
i=1
0
are dened exclusively by c1 /c0 , . . . , cm /c0 as in (6.6.10). Applying now Theorem 6.6.2


to the sum rn1
=1 log u(r ), we obtain
n1

r =1

(11/m,m) ,
log u(r ) = b + m log n + T (n), T A
0

(6.6.13)

m1
i n 1i/m + O(1) as n with 1 , . . . , m1 as in (6.6.9), and
where T (n) = i=1
b is a constant. The result now follows by combining (6.6.11)(6.6.13) and dening 0 and as in (6.6.9). This completes the proof of part (i). Also, by (6.6.9) and
(6.6.10), it is clear that c1 = = cm1 = 0 forces 1 = = m1 = 0, which in turn

144

6 The d-Transformation: A GREP for Innite Series and Sequences

forces 1 = = m1 = 0. Next, if c1 = 0, then 1 = c1 /c0 = 0 and hence 1 = 1 /

(1 1/m) = 0. Finally, if c1 = = cr 1 = 0 and cr = 0, then 1 = = r 1 = 0
and r = cr /c0 = 0 and hence r = r /(1 r/m) = 0, r {1, . . . , m 1}. This
proves half of part (iii) that assumes the conditions of part (i).
For the proof of part (ii), we begin by forming c(n) = an+1 /an . We obtain

c(n) = n (1 + n 1 )

w(n + 1)
exp [Q(n)] ; Q(n) = Q(n + 1) Q(n).
w(n)
(6.6.14)

Now X (n) (1 + n 1 ) , Y (n) w(n + 1)/w(n), and Z (n) exp [Q(n)] are each
(0,m) strictly. For X (n) and Y (n), this assertion can be veried in a straightforward
in A
0
manner. For Z (n), its truth follows from the fact that Q(n) = Q(n + 1) Q(n) is either
(0,m) . This completes the proof of part (ii). A more careful study is
a constant or is in A
0
needed to prove the second half of part (iii) that assumes the conditions of part (ii). First,
we have X (n) = 1 + O(n 1 ) as n . Next, Y (n) = 1 + O(n 11/m ) as n .

m
wi n i/m + O(n 11/m )
This is a result of the not so obvious fact that since w(n) = i=0

m
as n , then w(n + 1) = i=0 wi n i/m + O(n 11/m ) as n as well. As for
Z (n), we have two different cases: When 1 = = m1 = 0, we have Q(n) = 0 ,
from which Z (n) = e0 . When 1 = = r 1 = 0 and r = 0, r {1, . . . , m 1},
we have
Q(n) = 0 +

m1


i (1 i/m)n i/m [1 + O(n 1 )]

i=r

= 0 +

m1


i (1 i/m)n i/m + O(n 1r/m ) as n .

i=r

Exponentiating Q(n), we obtain

Z (n) = e0 [1 + r (1 r/m)n r/m + O(n (r +1)/m )] as n .
Combining everything in (6.6.14), the second half of part (iii) now follows.

The next theorem gives necessary and sufcient conditions for a sequence {an } to be
in b (m) . In this sense, it is a characterization theorem for sequences in b (m) . Theorem
6.6.4 becomes useful in the proof.
Theorem 6.6.5 A sequence {an } is in b (m) if and only if its members satisfy an+1 = c(n)an
(s/m,m) for an arbitrary integer s and c(n) = 1 + O(n 11/m ) as n .
with c A
0

Specically, if an is as in (6.6.7) with (6.6.8) and with = s/m, then an = p(n)an with
(,m) strictly, where = q/m and q is an integer m. In particular, (i) = 1 when
pA
0

m1
i n 1i/m with
= 0, Q(n) 0, and = 0, (ii) = r/m when = 0, Q(n) = i=r
r = 0, for r {0, 1, . . . , m 1}, (iii) = 0 when < 0, and (iv) = = s/m
when > 0.

6.6 A Further Class of Sequences in b(m) : The Class b (m)

145

Proof. The rst part can be proved by analyzing p(n) = [c(n) 1]1 when c(n) is given,
and c(n) = 1 + 1/ p(n) when p(n) is given. The second part can be proved by invoking
Theorem 6.6.4. We leave the details to the reader.


6.6.3 Asymptotic Expansion of An When {an } b (m)
With Theorems 6.6.26.6.5 available, we go on to the study of the partial sums An =

n
(m) with an = O(n ) as n for some . [At this point it
k=1 ak when {an } b
is worth mentioning that Theorem 6.6.4 remains valid when Q(n) there is replaced by
(1,m) , and we use this fact here.] More specically, we are concerned with the
(n), A
0
following cases in the notation of Theorem 6.6.5:
( ,m) strictly for some = 1 + i/m, i = 0, 1, . . . . In this case,
(i) an = h(n) A
0
(,m) strictly, = 1.
an = p(n)an with p A
0
(n)
( ,m) strictly for arbitrary and A
(1r/m,m) strictly
(ii) an = e h(n), where h A
0
0
for some r {0, 1, . . . , m 1}, and either (a) limn  (n) = , or (b)
(,m) strictly, =
limn (n) is nite. In this case, an = p(n)an with p A
0
r/m, thus 0 < 1.
( ,m) strictly for arbitrary , A
(1,m) and
(iii) an = [(n 1)!] e(n) h(n), where h A
0
0
is arbitrary, and = s/m for an arbitrary integer s < 0. In this case, an = p(n)an
(,m) strictly, = 0.
with p A
0


We already know that, in case (i),
k=1 ak converges only when  < 1. In case (ii),


a
converges
(a)
for
all

when
limn (n) = and (b) for  < r/m
k=1 k
when limn (n) is nite. In case (iii), convergence takes place always. In all other


cases,
k=1 ak diverges. The validity of this assertion in cases (i), (ii-a), and (iii) is
obvious, for case (ii-b) it follows from Theorem 6.6.6 below. Finally, in case (ii-a)
|an | C1 n  e(n) as n , whereas in case (ii-b) |an | C2 n  as n . A similar
relation holds for case (iii).
Theorem 6.6.6 Let {an } b (m) be as in the previous paragraph. Then there exist a
(0,m) strictly such that
constant S({ak }) and a function g A
0
An1 = S({ak }) + n an g(n),
whether

k=1

(6.6.15)

ak converges or not.

Remark. In case m = 1 and
k=1 ak converges, this theorem is a special case of
Theorem 6.1.12 as can easily be veried.
Proof. We start with the proof of case (i) as it is the simplest. In this case, Theorem 6.6.2
( +1,m) strictly. Because nan =
applies, and we have An1 = b + V (n), where V A
0
(
+1,m)

(0,m) strictly. The result in

nh(n) A
strictly as well, g(n) V (n)/(nan ) is in A
0
0
(6.6.15) now follows by identifying S({ak }) = b.

146

6 The d-Transformation: A GREP for Innite Series and Sequences

For cases (ii) and (iii), we proceed by applying summation by parts, namely,
s


xk yk = xs ys+1 xr 1 yr

k=r

to An1 =

n1
k=1

ak =

s

(xk1 )yk ,

(6.6.16)

k=r

n1
k=1

n1


p(k)ak :
n1

[p(k 1)]ak ,

ak = p(n 1)an

k=1

(6.6.17)

k=1

where we have dened p(k) = 0 for k 0. (Hence ak = 0 for k 0 too.) We can now


do the same with the series n1
k=1 [p(k 1)]ak and repeat as many times as we wish.
This procedure can be expressed in a simple way by dening
u 0 (n) = 1; vi+1 (n) = p(n 1)u i (n 1), u i+1 (n) = vi+1 (n), i = 0, 1, . . . .
(6.6.18)
With these denitions, we have by summation by parts
n1


u i (k)ak = vi+1 (n)an +

k=1

n1


u i+1 (k)ak , i = 0, 1, . . . .

(6.6.19)

k=1

Summing all these equations, we obtain

An1 =


N


n1

vi (n) an +
u N (k)ak ,

i=1

(6.6.20)

k=1

(i ,m)
(,m) strictly, we have u i A
for any positive integer N . Now, by the fact that p A
0
0
(i +1,m) , where i = i( 1), i = 1, 2, . . . . In addition, by the fact that
and vi A
0
(,m) strictly. Let us pick N > (1 +  )/(1 ) so that N +
v1 (n) = p(n 1), v1 A
0


 < 1. Then, the innite series
k=1 u N (k)ak converges, and we can write
An1 =


k=1

u N (k)ak +


N
i=1


vi (n) an
u N (k)ak .

(6.6.21)

k=n

is an asymptotic sequence as n and expanding the funcRealizing that {vi (n)}i=1

tions vi (n), we see that
N

i=1

vi (n) =

r


i n i/m + O(vr (n)) as n , 0 = 0, r N , (6.6.22)

i=0

where the integer r is dened via (r + 1)/m = r + 1. Here 0 , 1 , . . . , r are

obtained by expanding only v1 (n), . . . , vr 1 (n), and are not affected by vi (n), i r . In
addition, by the asymptotic behavior of |an | mentioned above, we also have


k=n

u N (k)ak = O(n N +1 an ) as n .

(6.6.23)

6.6 A Further Class of Sequences in b(m) : The Class b (m)

147

Combining (6.6.20)(6.6.23), we obtain



r

i/m
(r +1)/m
i n
+ O(n
) an as n , (6.6.24)
An1 = S({ak }) + n

i=0

with S({ak }) = k=1 u N (k)ak . This completes the proof. [Note that S({ak }) is independent of N despite its appearance.]


Remarks.


about S({ak }): In case
1. When
k=1 ak does not converge, we can say the following


(i), S({ak }) is the analytic continuation of the sum of
k=1 ak in at least when

an = n w(n) with w(n) independent of . In case (ii), S({ak }) is the Abelian mean


k 1/m
ak . It is also related to generalized functions as we
dened by lim0+
k=1 e
will see when we consider the problem of the summation of Fourier series and their




generalizations. [When
k=1 ak converges, we have S({ak }) =
k=1 ak .]
2. Because the asymptotic expansion of An as n is of one and the same form


(m) -transformation that we dene next can be
whether
k=1 ak converges or not, the d
applied to approximate S({ak }) in all cases considered above. We will also see soon
that the d (m) -transformation can be applied effectively as {ak } b (m) implies at least
heuristically that {ak } b(m) .

6.6.4 The d(m) -Transformation

Adding an to both sides of (6.6.15), and recalling that = q/m for q {0, 1, . . . , m}
(0,m) , we observe that
and w A
0
An S({ak }) + n an

i n i/m as n ,

(6.6.25)

i=0

with all i exactly as in (6.6.24) except q , to which we have added 1. This means that
An is analogous to a function A(y) F(1) in the following sense: An A(y), n 1 y,
n an 1 (y), r1 = 1/m, and S({ak }) A. The variable y is discrete and assumes the
values 1, 1/2, 1/3, . . . . Thus, we can apply GREP(1) to A(y) to obtain good approximations to A. In case we do not wish to bother with the exact value of , we can simply
replace it by 1, its maximum possible value, retaining the form of (6.6.25) at the same
time. That is to say, we now have nan 1 (y). As we recall, the W-algorithm can be
used to implement GREP(1) very efciently. (Needless to say, if we know the exact value
of , especially = 0, we should use it.)
For the sake of completeness, here are the equations that dene GREP(1) for the
problem at hand:
j)
A Rl = d(m,
+ Rl a Rl
n

n1

i=0

i
, j l j + n,
(Rl + )i/m

(6.6.26)

where = when is known or = 1 otherwise. Again, > R0 and a good choice

is = 0. We call this GREP(1) the d(m) -transformation.

148

6 The d-Transformation: A GREP for Innite Series and Sequences

Again, for the sake of completeness, we also give the implementation of this new
transformation (with = 0) via the W-algorithm:
( j)

M0 =

ARj
1
( j)
, N0 =
, j 0; r = r (Ar 1 ),
R j
R j

( j+1)

Mn( j) =

( j)

( j+1)

Mn1 Mn1
1/m

( j)

Nn1 Nn1

, Nn( j) =
1/m

1/m

R j+n R j

1/m

R j+n R j

, j 0, n 1,

( j)

Mn
j)
d(m,
= ( j) , j, n 0.
n
Nn
6.6.5 Does {an } b (m) Imply {an } b(m) ? A Heuristic Approach
Following Denition 6.6.1, we concluded in a rigorous manner that if an = (n),
( ,m) , then, subject to some condition on , {an } b(m) . Subject to a similar condiA
0
tion on , we know from Theorem 6.6.5 that this sequence is in b (m) as well. In view of
this, we now ask whether every sequence in b (m) is also in b(m) . In the sequel, we show
heuristically that this is so. We do this again by relaxing the conditions on the set b(m) ,
exactly as was done in Subsection 6.4.2.
(s/m,m) for an arbitrary integer
We start with the fact that an+1 = c(n)an , where c A
0
(k )
s. We now claim that we can nd functions k A0 , k integer, k = 0, 1, . . . , m 1,
such that
an+m =

m1


k (n)an+k ,

(6.6.27)

k=0

provided a certain matrix is nonsingular. Note that we can always nd functions k (n)
not necessarily in A0(k ) , k integer, for which (6.6.27) holds.
Using an+1 = c(n)an , we can express (6.6.27) in the form
gm (n)an =

m1


k (n)gk (n)an ,

(6.6.28)

k=0

where g0 (n) = 1 and gk (n) =

if we require

k1
j=0

c(n + j), k 1. We can ensure that (6.6.28) holds

gm (n) =

m1


k (n)gk (n).

(6.6.29)

k=0

(ks/m,m) , we can decompose it as in

As gk A
0
gk (n) =

m1


n i/m gki (n), gki A0 ki with ki integer,

( )

(6.6.30)

i=0

as we showed following Denition 6.6.1. Substituting (6.6.30) in (6.6.29), and equating

6.6 A Further Class of Sequences in b(m) : The Class b (m)

149

the coefcients of n i/m on both sides, we obtain

gm0 (n) = 0 (n) +

m1


gk0 (n)k (n)

k=1

gmi (n) =

m1


gki (n)k (n), i = 1, . . . , m 1.

(6.6.31)

k=1
(k )
Therefore, provided det [gki (n)]m1
k,i=1 = 0, there is a solution for k (n) A0 , k integer,
k = 0, 1, . . . , m 1, for which (6.6.27) is valid, implying that {an } b(m) , with the
conditions on b(m) relaxed as mentioned above.
For m = 2, the solution of (6.6.31) is immediate. We have 1 (n) = g21 (n)/g11 (n) and
0 (n) = g20 (n) g10 (n)1 (n), provided g11 (n)  0. With this solution, it can be shown


that {an } b(2) ,
at least in some cases that {an } b (2) and
k=1 ak convergent imply


with b(2) as described originally in Denition 6.1.2, that is, an = 2k=1 pk (n)k an ,
pk A(k)
0 , k = 1, 2. Let us now demonstrate this with two examples.

Example 6.6.7 When c(n) = n 1/2 , we have p1 (n) = 2{[n(n + 1)]1/2 1}1 and
p2 (n) = 12 p1 (n) and p1 , p2 A(0) strictly. Thus, {an } b(2) as in Denition 6.1.2. Note

that an = a1 / (n 1)! in this example.

Example 6.6.8 Assume c(n) i=0
ci n i/2 with c0 = 1 and c1 = 0. (Recall that
when = 0 and c0 = 1, it is necessary that |c1 | + |c2 | = 0 for {an } b (2) , from Theorem 6.6.5.) After tedious manipulations, it can be shown that
p1 (n) =

1
1
(1 4c2 ) + O(n 1 ) and p2 (n) = 2 n + O(1) as n ,
2
2c1
c1

(1)
(2)
so that p1 A(0)
as in Denition 6.1.2. Note
0 and p2 A0 strictly. Thus, {an } b
( ,2)
2c1 n

u(n), where u A0 for some in this case, as follows from Theothat an = e

rem 6.6.4.

We can now combine the developments above with Heuristics 6.4.1 and 6.4.2 to study
sequences {an } that look more complicated than the ones we encountered in Sections 6.1
6.4 and the new ones we encountered in this section. As an example, let us look at
( ,m) and h A
(1,m) . We rst notice that an = a + +
an = g(n) cos(h(n)), where g A
n
0
0


h i n 1i/m as n ,
an , where an = 12 g(n)eih(n) . Next, by the fact that h(n) i=0

m1
(0,m) . As a

we can write h(n) = h(n)

+ h(n),
where h(n)
= i=0
h i n 1i/m and h A
0

(0,m)
ih(n)
ih(n)
ih(n)
ih(n)

=e
e
. Now since e
A
, we have that an = u(n)eih(n)
result, e
0

(
,m)
1
ih(n)

(m)

with u(n) = 2 g(n)e

A
, satises {an } b
by Theorem 6.6.4, and hence
0

(m)
+

(2m)
by Heuristic 6.4.1.
{an } b . Finally, {an = an + an } b
( ,m) , such that =
We end this section by mentioning that, if an = h(n) A
0
(m)
1 + i/m, i = 0, 1, . . . , then {an } b in the strict sense of Denition 6.1.2. This
follows from Theorem 6.8.3 of Section 6.8.

150

6 The d-Transformation: A GREP for Innite Series and Sequences

6.7 Summary of Properties of Sequences in b(1)

From Heuristic 6.4.1 and Example 6.4.4, it is seen that sequences in b(1) are important
building blocks for sequences in b(m) with arbitrary m. They also have been a common test
ground for different convergence acceleration methods. Therefore, we summarize their
properties separately. We start with their characterization theorem that is a consequence
of Theorem 6.6.5.
Theorem 6.7.1 The following statements concerning sequences {an } are equivalent:
)
(i) {an } b(1) , that is, an = p(n)an , where p A(
0 strictly, being an integer 1.
()
(ii) an+1 = c(n)an , where c A0 strictly and is an integer, such that c(n) =
1 + O(n 2 ) as n .
( )
(iii) an = [(n 1)!] n h(n) with an integer and h A0 strictly, such that either (a)
= 0, = 1, and = 0, or (b) = 0 and = 1, or (c) = 0.

With and as in statements (i) and (ii), respectively, we have (a) = 1 when = 0,
= 1, and = 0, (b) = 0 when = 0 and = 1, and (c) = min{0, } when
= 0.
( )

Note that, in statements (iii-a) and (iii-b) of this theorem, we have that an = h(n) A0
( )
and an = n h(n) with h A0 , respectively; these are treated in Examples 6.1.10 and
6.1.11, respectively.
The next result on the summation properties of sequences {an } in b(1) is a consequence
of Theorem 6.6.6, and it combines a few theorems that were originally given by Sidi
[270], [273], [294], [295] for the different situations. Here we are using the notation of
Theorem 6.7.1.
Theorem 6.7.2 Let an be as in statement (iii) of the previous theorem with 0. That
( )
is, either (a) an = h(n) with h A0 , such that = 1, 0, 1, . . . ; or (b) an = n h(n)
( )
with = 1, | | 1, and h A0 with arbitrary; or (c) an = [(n 1)!]r n h(n) with
( )
r = 1, 2, . . . , and h A0 , and being arbitrary. Then, there exist a constant S({ak })
and a function g A(0)
0 strictly such that
(6.7.1)
An1 = S({ak }) + n an g(n),


gi n i as n , there holds = 1 and
where 1 is an integer. With g(n) i=0
1
1
g0 = in case (a); = 0 and g0 = ( 1) in case (b); while = 0 and g0 = 1,


gi = 0, 1 i r 1, and gr = in case (c). All this is true whether
k=1 ak converges or not. [In case (c), the series always converges.]
In case (a) of Theorem 6.7.2, we have
(An S({ak }))/(An1 S({ak })) an+1 /an 1 as n .
Sequences {An } with this property are called logarithmic, whether they converge or not.
In case (b) of Theorem 6.7.2, we have
(An S({ak }))/(An1 S({ak })) an+1 /an = 1 as n .

6.7 Summary of Properties of Sequences in b(1)

151

Sequences {An } with this property are called linear, whether they converge or not. When


is real and negative the series
k=1 ak is known as an alternating series, and when
is real and positive it is known as a linear monotone series.
In case (c) of Theorem 6.7.2, we have
(An S({ak }))/(An1 S({ak })) an+1 /an n r as n .
Since An S({ak }) tends to zero practically like 1/(n!)r in this case, sequences {An }
with this property are called factorial.


Again, in case (b), the (power) series
k=1 ak converges for | | < 1, the limit S({ak })
being an analytic function of inside the circle | | < 1, with a singularity at = 1. Let
us denote this function f ( ). By restricting the an further, we can prove that (6.7.1) is
valid also for all | | 1 and  [1, ), S({ak }) being the analytic continuation of f ( ),


whether
or not. (Recall that convergence takes place when | | = 1
k=1 ak converges


provided  < 0, while
k=1 ak diverges when | | > 1 with any .) This is the subject
of the next theorem that was originally given by Sidi [294]. (At this point it may be a
good idea to review Example 4.1.8.)
( )

Theorem 6.7.3 Let an = n h(n),

 where h A0 strictly for some arbitrary and
h(n) = n q w(n), where w(n) = 0 ent (t) dt with (t) = O(ect ) as t for some


i t 1+i as t 0+, such that (i) q = 0 and = if  < 0
c < 1 and (t) i=0
and (ii) q =   + 1 and = q if  0. Then, for all complex not in the
real interval [1, +), there holds

 
d q (t)
dt, g A(0)
An1 = f ( ) + an g(n); f ( ) =
0 strictly. (6.7.2)
d
et
0
Clearly, f ( ) is analytic in the complex -plane cut along the real interval [1, +).
Proof. We start with the fact that
An1


  n1
d q 
k
=
k w(k) =
w(k)
d
k=1
k=1

   n1

d q 
( et )k (t) dt.
=
d
0
k=1
n1


k q

n1

z k = (z z n )/(1 z), we obtain





n
d q
ent (t)
An1 = f ( )
dt.
(6.7.3)
d 1 et
0
 d q n

q
= n i=0 i (, t)n i with q (, t) = (1 et )1 (1 )1 = 0
As d
1 et
as t 0+, application of Watsons lemma to the integral term in (6.7.3) produces
( )
An1 f ( ) = n u(n) with u A0 strictly. Setting g(n) = u(n)/ h(n), the result in
(6.7.2) now follows.


From this and from the identity

k=1

Turning things around in Theorem 6.7.2, we next state a theorem concerning logarithmic, linear, and factorial sequences that will be of use in the analysis of other sequence
transformations later. Note that the two theorems imply each other.

152

6 The d-Transformation: A GREP for Innite Series and Sequences

Theorem 6.7.4


i n i as n , 0 = 0, = 0, 1, . . . , that is, {An } is a
(i) If An A + i=0
logarithmic sequence, then

An A + n(An )

i n i as n , 0 = 1 = 0.

i=0

(ii) If An A +
sequence, then

n
i=0

i n i as n , = 1, 0 = 0, that is, {An } is a linear

An A + (An )

i n i as n , 0 = ( 1)1 = 0.

i=0

n

i
(iii) If An A + (n!)r
as n , r = 1, 2, . . . , 0 = 0, that is, {An }
i=0 i n
is a factorial sequence, then




i
as n , r = = 0.
i n
An A + (An ) 1 +
i=r

Remark. In Theorems 6.7.26.7.4, we have left out the cases in which an =

( )
[(n 1)!] n h(n) with = r = 1, 2, . . . , and h(n) A0 . Obviously, in such a case,
An diverges wildly. As shown in Section 19.4, An actually satises




A n an 1 +
i n i as n , r = 1 .
(6.7.4)
i=r

In other words,
 {An } diverges factorially. Furthermore, when h(n) is independent of and
h(n) = n 0 ent (t) dt for some integer 0 and some (t) of exponential order,


the divergent series
k=1 ak has a (generalized) Borel sum, which, as a function of , is
analytic in the -plane cut along the real interval [0, +). This result is a special case
of the more general ones proved in Sidi [285]. The fact that An satises (6.7.4) suggests
that the d (1) -transformation, and, in particular, the L- and S-transformations, could be


to be very effective; they
effective in summing
k=1 ak . Indeed, the latter two turn out

produce approximations to the (generalized) Borel sum of
k=1 ak , as suggested by
the numerical experiments of Smith and Ford [318], Bhattacharya, Roy, and Bhowmick
[22], and Grotendorst [116].

6.8 A General Family of Sequences in b(m)

We now go back to the sequences {an } considered in Example 6.4.4 and study them
more closely. In Theorems 6.8.3 and 6.8.7, we show that two important classes of such
sequences are indeed in b(m) for some m strictly in accordance with Denition 6.1.2.
Obviously, this strengthens the conclusions of Example 6.4.4. From Theorems 6.8.4
and 6.8.8, it follows in a rigorous manner that the d (m) -transformation can be applied


effectively to the innite series
k=1 ak associated with these sequences. As a pleasant
outcome of this study, we also see in Theorems 6.8.5 and 6.8.9 that the d-transformation
can be used very effectively to accelerate the convergence of two important classes of

6.8 A General Family of Sequences in b(m)

153

sequences even when there is not enough quantitative information about the asymptotic behavior of the sequence elements. We note that all the results of this section
are new. We begin with the following useful lemma that was stated and proved as
Lemma 1.2 in Sidi [305].


Lemma 6.8.1 Let Q i (x) = ij=0 ai j x j , with aii = 0, i = 0, 1, . . . , n, and let xi , i =
0, 1, . . . , n, be arbitrary points. Then


 Q 0 (x0 ) Q 0 (x1 ) Q 0 (xn ) 



n
 Q 1 (x0 ) Q 1 (x1 ) Q 1 (xn )  


aii V (x0 , x1 , . . . , xn ),
(6.8.1)
=
 .
.
.
..
.. 
 ..
i=0


 Q (x ) Q (x ) Q (x ) 
n

where V (x0 , x1 , . . . , xn ) =

0i< jn (x j

xi ) is a Vandermonde determinant.

Proof. As can easily be seen, it is enough to consider the case aii = 1, i = 0, 1, . . . , n.

Let us now perform the following elementary row transformations in the determinant on
the left-hand side of (6.8.1):
for i = 1, 2, . . . , n do
for j = 0, 1, . . . , i 1 do
multiply ( j + 1)st row by ai j and subtract from (i + 1)st row
end for
end for
The result of these transformations is V (x0 , x1 , . . . , xn ).

The developments we present in the following two subsections parallel each other,
even though there are important differences between the sequences that are considered
in each. We would also like to note that the results of Theorems 6.8.4 and 6.8.8 below are
stronger versions of the result of Theorem 6.1.12 for the sequences {an } of this section


in the sense that they do not assume that
k=1 ak converges.

6.8.1 Sums of Logarithmic Sequences

m
( )
h i (n), where h i A0 i strictly for some distinct i =
Lemma 6.8.2 Let an = i=1

1
vik (n)n k k an for
0, 1, . . . . Then, with any integer r 0, there holds h i (n) = m+r
k=r
(0)
each i, where vik A0 for all i and k.
Proof. First, h i (n) i n i as n , for some i = 0. Consequently, k h i (n) =
1
n k u ik (n)h i (n) with u ik A(0)
0 strictly, u ik (n) = [i ]k + O(n ) as n , where we
have dened [x]k = x(x 1) (x k + 1), k = 1, 2, . . . . (We also dene u i0 (n) = 1
and [x]0 = 1 for every x.) Thus, with any nonnegative integer r , the h i (n) satisfy the
linear system
m

i=1

u ik (n)h i (n) = n k k an , k = r, r + 1, . . . , m + r 1,

154

6 The d-Transformation: A GREP for Innite Series and Sequences

that can be solved for the h i (n) by Cramers rule. Obviously, because u ik A(0)
0 for all
i and k, all the minors of the matrix of this system, namely, of

u 1r (n)
u 2r (n)
u mr (n)
u 1,r +1 (n) u 2,r +1 (n) u m,r +1 (n)

M(n) =
,
..
..
..

.
.
.
u 1,m+r 1 (n) u 2,m+r 1 (n) u m,m+r 1 (n)
(0)
are in A(0)
0 . More importantly, its determinant is in A0 strictly. To prove this last
point, we replace u ik (n) in det M(n) by its asymptotic behavior and use the fact that
[x]q+r = [x]r [x r ]q to factor out [i ]r from the ith column, i = 1, . . . , m. (Note
that [i ]r = 0 by our assumptions on the i .) This results in


 [1 r ]0

  [ r ]
m
1
 1
[i ]r  .
det M(n)
..

i=1

 [ r ]
1

m1

[2 r ]0
[2 r ]1
..
.

[m r ]0
[m r ]1
..
.

[2 r ]m1 [m r ]m1






 as n ,




which, upon invoking Lemma 6.8.1, gives



m
det M(n)
[i ]r V (1 , 2 , . . . , m ) = 0 as n .
i=1

Here, we have used the fact that [x]k is a polynomial in x of degree exactly k and the
assumption that the i are distinct. Completing the solution by Cramers rule, the result
follows.


In the following two theorems, we use the notation of the previous lemma. The rst
of these theorems is a rigorous version of Example 6.4.4, while the second is concerned
with the summation properties of the sequences {an } that we have considered so far.

m
( )
h i (n), where h i A0 i strictly for some distinct i =
Theorem 6.8.3 Let an = i=1
(m)
0, 1, . . . . Then {an } b .

m
h i (n). We obtain
Proof. Let us rst invoke Lemma 6.8.2 with r = 1 in an = i=1



m
m
vik (n), k = 1, . . . , m. By the fact that
an = k=1 pk (n)k an with pk (n) = n k i=1
(k)
vik are all in A(0)
0 , we have that pk A0 for each k. The result follows by recalling
Denition 6.1.2.



m
( )
Theorem 6.8.4 Let an = i=1
h (n), where h i A0 i strictly for some distinct i =

i
1, 0, 1, 2, . . . . Then, whether k=1 ak converges or not, there holds
An1 = S({ak }) +

m1


n k+1 (k an )gk (n),

k=0

where S({ak }) is the sum of the limits or antilimits of the series

by Theorem 6.7.2, and gk A(0)
0 for each k.

(6.8.2)

k=1

h i (k), which exist

6.8 A General Family of Sequences in b(m)

155

Proof. By Theorem 6.7.2, we rst have

An1 = S({ak }) +

m


nh i (n)wi (n),

i=1

m1
k k
where wi A(0)
Lemma 6.8.2 with
k=0 vik (n)n  an by
0 for each i. Next, h i (n) =

m
vik (n)wi (n) for
r = 0. Combining these results, we obtain (6.8.2) with gk (n) = i=1
k = 0, 1, . . . , m 1.


A Special Application
The techniques we have developed in this subsection can be used to show that the dtransformation will be effective in computing the limit or antilimit of a sequence {Sn },
for which
Sn = S +

m


Hi (n),

(6.8.3)

i=1

where Hi A0(i ) for some i that are distinct and satisfy i = 0, 1, . . . , and S is the
limit or antilimit.
Such sequences arise, for example, when one applies the trapezoidal rule to integrals
over a hypercube or a hypersimplex of functions that have algebraic singularities along
the edges or on the surfaces of the hypercube or of the hypersimplex. We discuss this
subject in some detail in Chapter 25.
If we know the i , then we can use GREP(m) in the standard way described in Chapter 4.
If the i are not readily available, then the d (m) -transformation for innite sequences
developed in Subsection 6.2.2 serves as a very effective means for computing S. The
following theorem provides the rigorous justication of this assertion.
Theorem 6.8.5 Let the sequence {Sn } be as in (6.8.3). Then there holds
Sn = S +

m


n k (k Sn )gk (n),

(6.8.4)

k=1

where gk A(0)
0 for each k.

m
Proof. Applying Lemma 6.8.2 with r = 1 to Sn S = i=1
Hi (n), and realizing that


k k
v
(n)n
 Sn for each i, where
k (Sn S) = k Sn for k 1, we have Hi (n) = m
ik
k=1
(0)


vik A0 for all i and k. The result follows by substituting this in (6.8.3).
6.8.2 Sums of Linear Sequences

m n
( )
Lemma 6.8.6 Let an = i=1 i h i (n), where i = 1 are distinct and h i A0 i for some
arbitrary i that are not necessarily distinct. Then, with any integer r 0, there holds

1
vik (n)k an for each i, where vik A(0)
in h i (n) = m+r
k=r
0 for all i and k.
Proof. Let us write an(i) = in h i (n) for convenience. By the fact that an(i) =
(i 1)in h i (n + 1) + in h i (n), we have rst an(i) = u i1 (n)an(i) , where u i1 A(0)
0

156

6 The d-Transformation: A GREP for Innite Series and Sequences

strictly. In fact, u i1 (n) = (i 1) + O(n 1 ) as n . Consequently, k an(i) =

k
1
u ik (n)an(i) , where u ik A(0)
0 strictly and u ik (n) = (i 1) + O(n ) as n . Thus,
with any nonnegative integer r , an(i) , i = 1, . . . , m, satisfy the linear system
m


u ik (n)an(i) = k an , k = r, r + 1, . . . , m + r 1,

i=1

that can be solved for the an(i) by Cramers rule. Obviously, since u ik A(0)
0 for all i and
k, all the minors of the matrix M(n) of this system that is given exactly as in the proof
of Lemma 6.8.2 with the present u ik (n) are in A(0)
0 . More importantly, its determinant is
in A(0)
0 strictly. In fact, substituting the asymptotic behavior of the u ik (n) as n in
det M(n), we obtain


m
r
(i 1) V (1 , . . . , m ) = 0 as n .
det M(n)
i=1

Completing the solution by Cramers rule, the result follows.

We make use of Lemma 6.8.6 in the proofs of the next two theorems that parallel
Theorems 6.8.3 and 6.8.4. Because the proofs are similar, we leave them to the reader.

m n
( )
i h i (n), where i = 1 are distinct and h i A0 i for
Theorem 6.8.7 Let an = i=1
some arbitrary i that are not necessarily distinct. Then {an } b(m) .

m n
Theorem 6.8.8 Let an = i=1
i h i (n), where i are distinct and satisfy i = 1 and
(i )
| | 1 and h i A0 for some arbitrary i that are not necessarily distinct. Then,


whether
k=1 ak converges or not, there holds
An1 = S({ak }) +

m1


(k an )gk (n),

(6.8.5)

k=0

where S({ak }) is the sum of the limits or antilimits of the series

exist by Theorem 6.7.2, and gk A(0)
0 for each k.

k
k=1 i h i (k),

which

A Special Application
With the help of the techniques developed in this subsection, we can now show that the
d (m) -transformation can be used for computing the limit or antilimit of a sequence {Sn },
for which
Sn = S +

m


in Hi (n),

(6.8.6)

i=1
i)
where i = 1 are distinct, Hi A(
0 for some arbitrary i that are not necessarily distinct,
and S is the limit or antilimit.
If we know the i and i , then we can use GREP(m) in the standard way described
in Chapter 4. If the i and i are not readily available, then the d (m) -transformation for

6.8 A General Family of Sequences in b(m)

157

innite sequences developed in Subsection 6.2.2 serves as a very effective means for
computing S. The following theorem provides the rigorous justication of this assertion.
As its proof is similar to that of Theorem 6.8.5, we skip it.
Theorem 6.8.9 Let the sequence {Sn } be as in (6.8.6). Then there holds
Sn = S +

m

(k Sn )gk (n),

(6.8.7)

k=1

where gk A(0)
0 for each k.
6.8.3 Mixed Sequences
We now turn to mixtures of logarithmic and linear sequences that appear to be difcult
to handle mathematically. Instead of attempting to extend the theorems proved above,
we state the following conjectures.

m 1 n

m 2
Conjecture 6.8.10 Let an = i=1
h i (n), where i = 1 are distinct and
i h i (n) + i=1
( )
( )
h i A0 i for some arbitrary i that are not necessarily distinct and h i A0 i with
i = 0, 1, . . . , and distinct. Then {an } b(m) , where m = m 1 + m 2 .
Conjecture 6.8.11 If in Conjecture 6.8.10 we have |i | 1 and i = 1, in addition to
the conditions there, then
An1 = S({ak }) +

m1


n k+1 (k an )gk (n),

k=0

where S({ak }) is the sum of the limits or antilimits of the series

(0)

k=1 h i (k), which exist by Theorem 6.7.2, and gk A0 for each k.

(6.8.8)

k
k=1 i h i (k)

and

Conjecture 6.8.12 Let the sequence {Sn } satisfy

Sn = S +

m1


in Hi (n) +

i=1

m2


H i (n)

(6.8.9)

i=1

i)
for some arbitrary i that are not necessarily
where i = 1 are distinct and Hi A(
0
distinct and H i A(0i ) with distinct i = 0, 1, . . . . Then there holds

Sn = S +

m


n k (k Sn )gk (n),

k=1

where gk A(0)
0 for each k and m = m 1 + m 2 .

(6.8.10)

7
Recursive Algorithms for GREP

7.1 Introduction
Let us recall the denition of GREP(m) as given in (4.2.1). This denition involves
the m form factors (or shape functions) k (y), k = 1, . . . , m, whose structures may be

n k 1
i y irk that behave essentially polynomially
arbitrary. It also involves the functions i=0
rk
in y for k = 1, . . . , m.
These facts enable us to design very efcient recursive algorithms for two cases of
GREP:
(i) the W-algorithm for GREP(1) with arbitrary yl in (4.2.1), and
(ii) the W (m) -algorithm for GREP(m) with m > 1 and r1 = r2 = = rm , and with arbitrary yl in (4.2.1).
In addition, we are able to derive an efcient algorithm for a special case of one of the
extensions of GREP considered in Section 4.6:
(iii) the extended W-algorithm (EW-algorithm) for the m = 1 case of the extended GREP
for which the k (y) are as in (4.6.1), with yl = y0 l , l = 0, 1, . . . , and (0, 1).
We note that GREP(1) and GREP(m) with r1 = = rm are probably the most
commonly occurring forms of GREP. The D-transformation of Chapter 5 and the dtransformation of Chapter 6, for example, are of these forms, with r1 = = rm = 1
for both. We also note that the effectiveness of the algorithms of this chapter stems
from the fact that they fully exploit the special structure of the underlying extrapolation
methods.
As we will see in more detail, GREP can be implemented via the algorithms of Chap(m, j)
that
ter 3. Thus, when A(yl ), l = 0, 1, . . . , L, are given, the computation of those An
3
2
can be derived from them requires 2L /3 + O(L ) arithmetic operations when done by
the FS-algorithm and about 50% more when done by the E-algorithm. Both algorithms
require O(L 2 ) storage locations. But the algorithms of this chapter accomplish the same
task in O(L 2 ) arithmetic operations and require O(L 2 ) storage locations. With suitable
programming, the storage requirements can be reduced to O(L) locations. In this respect,
the algorithms of this chapter are analogous to Algorithm 1.3.1 of Chapter 1.
The W-algorithm was given by Sidi [278], [295], and the W(m) -algorithm was developed by Ford and Sidi [87]. The EW-algorithm is unpublished work of the author.
158

7.2 The W-Algorithm for GREP(1)

159

7.2 The W-Algorithm for GREP(1)

We start by rewriting (4.2.1) for m = 1 in a simpler way. For this, let us rst make the
(1, j)
( j)
substitutions 1 (y) = (y), r1 = r, n 1 = n, 1i = i , and An = An . The equations
in (4.2.1) become
A(yl ) = A(nj) + (yl )

n1


i ylir , j l j + n.

(7.2.1)

i=0

A further simplication takes place by letting t = y r and tl = ylr , l = 0, 1, . . . , and

dening a(t) A(y) and (t) (y). We now have
a(tl ) = A(nj) + (tl )

n1


i tli , j l j + n.

(7.2.2)

i=0
( j)

Let us denote by Dn {g(t)} the divided difference of g(t) over the set of points
{t j , t j+1 , . . . , t j+n }, namely, g[t j , t j+1 , . . . , t j+n ]. Then, from the theory of polynomial
interpolation it is known that
Dn( j) {g(t)} = g[t j , t j+1 , . . . , t j+n ] =

n


( j)

cni g(t j+i );

i=0
( j)

cni =

n

k=0
k=i

1
, 0 i n.
t j+i t j+k

(7.2.3)
( j)

The following theorem forms the basis of the W-algorithm for computing the An . It
also shows how the i can be computed one by one in the order 0 , 1 , . . . , n1 .
Theorem 7.2.1 Provided (tl ) = 0, j l j + n, we have
( j)

A(nj) =

Dn {a(t)/(t)}
( j)

Dn {1/(t)}

(7.2.4)

( j)

With An and 0 , . . . , p1 already known, p can be determined from

p = (1)n p1

 j+n
p1
p1  ( j)

ti Dn p1 {[a(t) A(nj) ]t p1 /(t)
i t i p1 },
i= j

i=0

(7.2.5)
where the summation

1
i=0

i is taken to be zero.

Proof. We begin by reexpressing the linear equations in (7.2.2) in the form

[a(tl ) A(nj) ]/(tl ) =

n1


i tli , j l j + n.

(7.2.6)

i=0

For 1 p n 1, we multiply the rst n p of the equations in (7.2.6)

( j)
p1
by cn p1,l j tl
, l = j, j + 1, . . . , j + n p 1, respectively. Adding all the

160

7 Recursive Algorithms for GREP

resulting equations and invoking (7.2.3), we obtain

n1

( j)
( j)
Dn p1 {[a(t) A(nj) ]t p1 /(t)} = Dn p1 {
i t i p1 }.

(7.2.7)

i=0

n1

n1
i t i p1 = i=0
i t i is a
Let us put p = 1 in (7.2.7). Then, we have that i=0
polynomial of degree at most n 1. But, it is known that
Dk(s) {g(t)} = 0 if g(t) is a polynomial of degree at most k 1.

(7.2.8)

By invoking (7.2.8) in (7.2.7) with p = 1, we obtain (7.2.4).

n1
i t i p1 = 0 t 1 +
Next, let us put p = 0 in (7.2.7). Then, we have that i=0



n1
n1 i1
i1

, the summation i=1 i t

being a polynomial of degree at most n 2.
i=1 i t
Thus, with the help of (7.2.8) again, and by the fact that
Dn( j) {t 1 } = (1)n /(t j t j+1 t j+n ),

(7.2.9)

we obtain 0 exactly as given in (7.2.5). [The proof of (7.2.9) can be done by induction
with the help of the recursion relation in (7.2.21) below.]
The rest of the proof can now be done similarly.


Remark. From the proof of Theorem 7.2.1, it should become clear that both (7.2.4)
and (7.2.5) hold even when the tl in (7.2.2) are complex, although in the extrapolation
problems that we usually encounter they are real.
( j)

The expression for An given in (7.2.4), in addition to forming the basis for the Walgorithm, will prove to be very useful in the convergence and stability studies of GREP(1)
in Chapters 8 and 9.
We next treat the problem of assessing the stability of GREP(1) numerically in an
efcient manner. Here too divided differences turn out to be very useful. From (7.2.4)
( j)
and (7.2.3), it is clear that An can be expressed in the by now familiar form
A(nj) =

n


n


( j)

ni A(y j+i );

i=0

( j)

ni = 1,

(7.2.10)

i=0

with
( j)

ni =
(1, j)

Thus, n

( j)

cni
, i = 0, 1, . . . , n.
( j)
Dn {1/(t)} (t j+i )
1

(7.2.11)

( j)

= n is given by
n( j) =

n

i=0

n


|Dn {1/(t)}|

i=0

( j)

|ni | =

( j)

( j)

|cni |
.
|(t j+i )|

(7.2.12)

Similarly, we dene
!(nj) =

n

i=0

( j)

|ni | |a(t j+i )| =

( j)
n

|c | |a(t j+i )|
ni

( j)

|Dn {1/(t)}|

i=0

|(t j+i )|

(7.2.13)

7.2 The W-Algorithm for GREP(1)

( j)

161

( j)

Let us recall briey the meanings of n and !n : If the A(yi ) have been computed with
( j)
( j)
absolute errors that do not exceed , and A n is the computed An , then
( j)

( j)

| A n A(nj) |  n( j) and | A n A|  n( j) + |A(nj) A|.

( j)
If the A(yi ) have been computed with relative errors that do not exceed , and A n is
( j)
the computed An , then
( j)

( j)

| A n A(nj) |  !(nj) and | A n A|  !(nj) + |A(nj) A|.

( j)

( j)

Hence, !n is a more rened estimate of the error in the computed value of An ,

especially when A(y) is unbounded as y 0+. See Section 0.5.
Lemma 7.2.2 Let u i , i = 0, 1, . . . , be scalars, and let the ti in (7.2.3) satisfy t0 > t1 >
t2 > . Then
n


( j)

|cni | u j+i = (1) j Dn( j) {v(t)},

(7.2.14)

i=0

where v(ti ) = (1)i u i , i = 0, 1, . . . , and v(t) is arbitrary otherwise.

Proof. From (7.2.3), we observe that
( j)

( j)

cni = (1)i |cni |, i = 0, 1, . . . , n, for all j and n.

(7.2.15)

The result now follows by substituting (7.2.15) in (7.2.3).

( j)

( j)

The following theorem on n and !n can be proved by invoking Lemma 7.2.2 in

(7.2.12) and (7.2.13).
Theorem 7.2.3 Dene the functions P(t) and S(t) by
P(ti ) = (1)i /|(ti )| and S(ti ) = (1)i |a(ti )/(ti )|, i = 0, 1, 2, . . . ,

(7.2.16)

and arbitrarily for t = ti , i = 0, 1, . . . . Then

( j)

n( j) =

|Dn {P(t)}|
( j)
|Dn {1/(t)}|

( j)

and !(nj) =
( j)

|Dn {S(t)}|
( j)
|Dn {1/(t)}|
( j)

(7.2.17)

( j)

We now give the W-algorithm to compute the A p ,  p , and !n recursively. This

algorithm is directly based on Theorems 7.2.1 and 7.2.3.
Algorithm 7.2.4 (W-algorithm)
1. For j = 0, 1, . . . , set
( j)

( j)

( j)

( j)

( j)

( j)

M0 = a(t j )/(t j ), N0 = 1/(t j ), H0 = (1) j |N0 |, K 0 = (1) j |M0 |.

162

7 Recursive Algorithms for GREP

( j)

( j)

( j)

( j)

2. For j = 0, 1, . . . , and n = 1, 2, . . . , compute Mn , Nn , Hn , and K n recursively

from
( j+1)

Q (nj) =
( j)

( j)

Q n1 Q n1
,
t j+n t j

( j)

( j)

( j)

(7.2.18)
( j)

where the Q n stand for either Mn or Nn or Hn or K n .

3. For all j and n, set




( j)
 H ( j) 
 K ( j) 
Mn
 n 
 n 
( j)
( j)
( j)
An = ( j) , n =  ( j)  , and !n =  ( j)  .
 Nn 
 Nn 
Nn
( j)

( j)

(7.2.19)

( j)

Note that, when Nn is complex, |Nn | is its modulus. [Nn may be complex when
( j)
( j)
(t) is complex. Hn and K n are always real.] Also, from the rst step of the algorithm
it is clear that (t j ) = 0 must hold for all j. Obviously, this can be accomplished by
choosing the t j appropriately.
The validity of (7.2.19) is a consequence of the following result.
( j)

( j)

( j)

( j)

Theorem 7.2.5 The Mn , Nn , Hn , and K n

(7.2.18) satisfy

computed by the W-algorithm as in

Mn( j) = Dn( j) {a(t)/(t)}, Nn( j) = Dn( j) {1/(t)},

Hn( j) = Dn( j) {P(t)}, K n( j) = Dn( j) {S(t)}.

(7.2.20)

As a result, (7.2.19) is valid.

Proof. (7.2.20) is a direct consequence of the known recursion relation for divided
differences, namely,
( j+1)

Dn( j) {g(t)} =

( j)

Dn1 {g(t)} Dn1 {g(t)}

,
t j+n t j

(7.2.21)

and (7.2.4) and (7.2.17).

( j)

( j)

( j)

( j)

In view of the W-algorithm, Mn , Nn , Hn , and K n can be arranged separately in

( j)
( j)
( j)
( j)
two-dimensional arrays as in Table 7.2.1, where the Q n stand for Mn or Nn or Hn
( j)
or K n , and the computational ow is as described by the arrows.
Obviously, the W-algorithm allows the recursive computation of both the approxima( j)
( j)
( j)
tions An and their accompanying n and !n simultaneously and without having to
( j)
know the ni . This is interesting, as we are not aware of other extrapolation algorithms
( j)
( j)
shown to have this property. Normally, to obtain n and !n , we would expect to have
( j)
to determine the ni separately by using a different algorithm or approach, such as that
given by Theorem 3.4.1 in Section 3.4.

7.2 The W-Algorithm for GREP(1)

163

Table 7.2.1:
Q (0)
0

Q (1)
0
Q (2)
0
Q (3)
0
..
.

Q (0)
1
Q (1)
1
Q (2)
1
..
.

Q (0)
2
Q (1)
2
..
.

Q (0)
3
..
.

..

7.2.1 A Special Case: (y) = y r

Let us go back to Algorithm 7.2.4. By substituting (7.2.18) in (7.2.19), we obtain a
( j)
recursion relation for the An that reads
( j+1)

A(nj) =
( j)

( j)

( j)

An1 wn An1
( j)

1 wn
( j)

( j+1)

( j)

= An1 +

( j)

An1 An1
( j)

1 wn

, j 0, n 1, (7.2.22)

( j+1)

( j)

where wn = Nn1 /Nn1 . That is, the W-algorithm now computes tables for the An
( j)
and the Nn .
( j)
In case Nn are known and do not have to be computed recursively, (7.2.22) pro( j)
vides the An by computing only one table. As an example, consider (y) = y r , or,
equivalently, (t) = t. Then, from (7.2.20) and (7.2.9), we have
Nn( j) = Dn( j) {t 1 } = (1)n /(t j t j+1 t j+n ),

(7.2.23)

wn( j) = t j+n /t j ,

(7.2.24)

( j)

so that wn becomes

and (7.2.22) reduces to

( j+1)

A(nj) =

( j)

( j+1)

( j)

An1
t j An1 t j+n An1
A
( j)
= An1 + n1
, j 0, n 1. (7.2.25)
t j t j+n
1 t j+n /t j

GREP(1) in this case is, of course, the polynomial Richardson extrapolation of Chapter 2, and the recursion relation in (7.2.25) is nothing but Algorithm 2.2.1 due to Bulirsch
and Stoer [43], which we derived by a different method in Chapter 2.
In this case, we can also give the closed-form expression

n 
n

t j+k
(7.2.26)
a(t j+i ).
A(nj) =
t
t j+i
i=0 k=0 j+k
k=i

( j)

This expression is obtained by expanding Dn {a(t)/t} with the help of (7.2.3) and
( j)
then dividing by Dn {1/t} that is given in (7.2.23). Note that (7.2.26) can also be
( j)
obtained by recalling that, in this case, An = pn, j (0), where pn, j (t) is the polynomial
that interpolates a(t) at tl , l = j, j + 1, . . . , j + n, and by setting t = 0 in the resulting
Lagrange interpolation formula for pn, j (t).

164

7 Recursive Algorithms for GREP

( j)

( j)

Recall that the computation of the n can be simplied similarly: Set 0 = 1, j =

( j)
0, 1, . . . , and compute the rest of the n by the recursion relation
( j+1)

n( j)

( j)

t j n1 + t j+n n1

=
, j = 0, 1, . . . , n = 1, 2, . . . .
t j t j+n

(7.2.27)

7.3 The W(m) -Algorithm for GREP(m)

We now consider the development of an algorithm for GREP(m) when r1 = = rm = r .
We start by rewriting the equations (4.2.1) that dene GREP(m) in a more convenient way
as follows: Let t = y r and tl = ylr , l = 0, 1, . . . , and dene a(t) A(y) and k (t)
k (y), k = 1, . . . , m. Then, (4.2.1) becomes
j)
a(tl ) = A(m,
+
n

m

k=1

k (tl )

n
k 1

ki tli , j l j + N ; N =

i=0

m


nk ,

(7.3.1)

k=1

with n = (n 1 , . . . , n m ) as usual.
In Section 4.4 on the convergence theory of GREP, we mentioned that those sequences
related to Process II in which n k , k = 1, . . . , m, simultaneously, and, in particular,
(m, j)
the sequences {Aq+(,... ,) }
=0 with j and q = (q1 , . . . , qm ) xed, appear to have the
best convergence properties. Therefore, we should aim at developing an algorithm for
computing such sequences. To keep the treatment simple, we restrict our attention to the
(m, j)
sequences {A(,... ,) }
=0 that appear to provide the best accuracy for a given number of
the A(yi ). For the treatment of the more general case in which q1 , . . . , qm are not all 0,
we refer the reader to Ford and Sidi [87].
The development of the W(m) -algorithm depends heavily on the FS-algorithm discussed in Chapter 3. We freely use the results and notation of Section 3.3 throughout our
developments here. Therefore, a review of Section 3.3 is recommended at this point.
(m, j)
0
One way to compute the sequences {A(,... ,) }
=0 is to eliminate rst k (t)t , k =
1
1, . . . , m, next k (t)t , k = 1, . . . , m, etc., from the expansion of A(y) given in (4.1.1)
and (4.1.2). This can be accomplished by ordering the k (t)t i suitably. We begin by
considering this issue.
7.3.1 Ordering of the k (t)t i

Let us dene the sequences {gs (l)}l=0

, s = 1, 2, . . . , as follows:

gk+im (l) = k (tl )tli , 1 k m, i 0.

(7.3.2)

for k = 1, . . . , m, the sequence gm+k is

Thus, the sequence gk is simply {k (tl )}l=0

{k (tl )tl }l=0 for k = 1, . . . , m, etc., and this takes care of all the sequences gs , s =
1, 2, . . . , and all the k (t)t i , 1 k m, i 0. As in Ford and Sidi [87], we call this
ordering of the k (t)t i the normal ordering throughout this chapter. As a consequence
of (7.3.2), we also have

1 i m,
i (tl ),
(7.3.3)
gi (l) =
tl gim (l), i > m.

7.3 The W (m) -Algorithm for GREP(m)

165

Table 7.3.1:
A(0)
0
A(1)
0
A(2)
0
A(3)
4
..
.

A(0)
1
A(1)
1
A(2)
1
..
.

A(0)
2
A(1)
2
..
.

A(0)
3
..
.

..

Let us also denote

h k (l) =

k (tl )
gk (l)
=
, 1 k m.
tl
tl

(7.3.4)

Now every integer p can be expressed as p = m + , where =  p/m and p

mod m and thus 0 < m. With the normal ordering of the k (t)t i introduced above,
( j)
A p that is dened along with the parameters 1 , . . . , p by the equations
a(tl ) = A(pj) +

p


k gk (l), j l j + p,

(7.3.5)

k=1
(m, j)

with (i) n k = , k = 1, . . . , m, if = 0, and (ii) n k = + 1, k =

is precisely An
1, . . . , , and n k = , k = + 1, . . . , m, if > 0. Thus, in the notation of (7.3.1), the
equations in (7.3.5) are equivalent to
a(tl ) = A(pj) +

m

k=1

k (tl )

( pk)/m


ki tli , j l j + p.

(7.3.6)

i=0

Consequently, the sequence {As }

s=0 contains {A(,... ,) }=0 as a subsequence. When
( j)
( j)
( j)
( j)
(2, j)
(2, j)
(2, j)
(2, j)
m = 2, for example, A1 , A2 , A3 , A4 , . . . , are A(1,0) , A(1,1) , A(2,1) , A(2,2) , . . . ,
respectively.
( j)
In view of the above, it is convenient to order the An as in Table 7.3.1.
( j)
As A p satises (7.3.5), it is given by the determinantal formula of (3.2.1), where
( j)
a(l) stands for a(tl ) A(yl ), as in Chapter 3. Therefore, A p could be computed, for
all p, by either of the algorithms of Chapter 3. But with the normal ordering dened in
(7.3.2), this general approach can protably be altered. When p m, the FS-algorithm
( j)
is used to compute the A p because the corresponding gk (l) have no particular structure.
The normal ordering has ensured that all these unstructured gk (l) come rst. For p > m,
( j)
however, the W(m) -algorithm determines the A p through a sophisticated set of recursions
at a cost much smaller than would be incurred by direct application of the FS-algorithm.
Let us now illuminate the approach taken in the W(m) -algorithm. We observe rst that
( j)
( j)
the denition of D p in (3.3.9) involves only the G (s)
k . Thus, we look at G p for p > m.
For simplicity, we take m = 2.
When m = 2 and p 3, we have by (7.3.2) that
( j)

(m, j)

G (pj) = |1 (t j ) 2 (t j ) 1 (t j )t j 2 (t j )t j g p ( j)|.

166

7 Recursive Algorithms for GREP

If we factor out t j+i1 from the ith row, i = 1, 2, . . . , p, we obtain

G (pj) =


p


t j+i1 |h 1 ( j) h 2 ( j) g1 ( j) g2 ( j) g p2 ( j)|,

i=1
( j)

where h i (l) are as dened in (7.3.4). The determinant obtained above differs from G p
by two columns, and it is not difcult to see that there are m different columns in the
general case. Therefore, we need to develop procedures for evaluating these objects.

7.3.2 Technical Preliminaries

( j)

( j)

( j)

We start this by introducing the following generalizations of f p (b), p (b), and D p :

F p( j) (h 1 , . . . , h q ) = |g1 ( j) g p ( j) h 1 ( j) h q ( j)| F p( j) (q),

(7.3.7)

( j)

F p+1q (q)

 (pj) (h 1 , . . . , h q ) =

( j)
F p+2q (q

1)

( j)

 (pj) (q),

( j+1)

F p+1q (q)F p1q (q)

D (pj) (q) =

(7.3.8)

( j)

( j+1)

F pq (q)F pq (q)

(7.3.9)

[In these denitions and in Theorems 7.3.1 and 7.3.2 below, the h k (l) can be arbitrary.
They need not be dened by (7.3.4).]
As simple consequences of (7.3.7)(7.3.9), we obtain
( j)

( j)

( j)

( j)

F p (0) = G p , F p (1) = f p (h 1 ),
( j)

( j)

 p (1) = p (h 1 ),
( j)

(7.3.10)
(7.3.11)

( j)

D p (0) = D p ,

(7.3.12)

respectively. In addition, we dene F0 (0) = 1. From (7.3.11), it is clear that the algorithm
( j)
( j)
that will be used to compute the different p (b) can be used to compute the  p (1) as
well.
( j)
( j)
We also note that, since F p (q) is dened for p 0 and q 0,  p (q) is dened for
( j)
1 q p + 1 and D p (q) is dened for 0 q p 1, when the h k (l) are arbitrary,
The following results will be of use in the development of the W(m) -algorithm shortly.
( j)

( j)

Theorem 7.3.1 The  p (q) and D p (q) satisfy

( j+1)

 (pj) (q) =
and

( j)

 p1 (q)  p1 (q)


( j+1)
D (pj) (q) =  p2 (q)

( j)

D p (q 1)

1
( j)

 p1 (q)

, 1 q p,

1
( j+1)

 p1 (q)

(7.3.13)


, 1 q p 1.

(7.3.14)

7.3 The W (m) -Algorithm for GREP(m)

167

( j)

Proof. Consider the ( p + q) ( p + q) determinant F p (q). Applying Theorem 3.3.1

(the Sylvester determinant identity) to this determinant with = 1,  = p + q, = p,
and  = p + q, we obtain
( j+1)

( j+1)

( j)

F p( j) (q)F p1 (q 1) = F p1 (q)F p( j) (q 1) F p( j+1) (q 1)F p1 (q). (7.3.15)

Replacing p in (7.3.15) by p + 1 q, and using (7.3.8) and (7.3.9), (7.3.13) follows.
To prove (7.3.14), we start with (7.3.9). When we invoke (7.3.8), (7.3.9) becomes
( j)

D (pj) (q)

( j+1)

 p (q) p2 (q)
( j)

( j+1)

 p1 (q) p1 (q)

D (pj) (q 1).

(7.3.16)

When we substitute (7.3.13) in (7.3.16), (7.3.14) follows.

The recursion relations given in (7.3.13) and (7.3.14) will be applied, for a given p, with
( j+1)
q increasing from 1. When q = p 1, (7.3.14) requires knowledge of  p2 ( p 1),
( j)
( j+1)
and when q = p, (7.3.13) requires knowledge of  p1 ( p) and  p1 ( p). That is to say,
( j)
we need to have s (s + 1), s 0, j 0, to be able to complete the recursions in
( j)
(7.3.14) and (7.3.13). Now,  p ( p + 1) cannot be computed by the recursion relation
( j)
( j)
in (7.3.13), because neither  p1 ( p + 1) nor D p ( p) is dened. When the denition of
( j)
the h k (l) given in (7.3.4) is invoked, however,  p ( p + 1) can be expressed in simple
and familiar terms and computed easily, as we show later.
In addition to the relationships in the previous theorem, we give one more result
( j)
concerning the  p (q).
( j)

Theorem 7.3.2 The  p (q) satisfy the relation

( j)

( j)

F p+1q (q) =  (pj) (1) (pj) (2)  (pj) (q)G p+1 .

(7.3.17)

Proof. Let us reexpress (7.3.8) in the form

( j)

( j)

F p+1q (q) =  (pj) (q)F p+1(q1) (q 1).

(7.3.18)

Invoking (7.3.8), with q replaced by q 1, on the right-hand side of (7.3.18), and

continuing, we obtain
( j)

( j)

F p+1q (q) =  (pj) (q)  (pj) (1)F p+1 (0).

The result in (7.3.17) follows from (7.3.19) and (7.3.10).

(7.3.19)

7.3.3 Putting It All Together

Let us now invoke the normal ordering of the k (t)t i . With this ordering, the gk (l)
and h k (l) are as dened in (7.3.2) and (7.3.4), respectively. This has certain favorable
( j)
consequences concerning the D p .
j
Because we have dened h k (l) only for 1 k m, we see that F p (q) is dened only
( j)
( j)
for 0 q m, p 0,  p (q) for 1 q min{ p + 1, m}, and D p (q) for 0 q

168

7 Recursive Algorithms for GREP

min{ p 1, m}. Consequently, the recursion relation in (7.3.13) is valid for 1 q

min{ p, m}, and that in (7.3.14) is valid for 1 q min{ p 1, m}.
( j)
In addition, for  p ( p + 1), we have the following results.
Theorem 7.3.3 With the normal ordering and for 0 p m 1, we have
 (pj) ( p + 1) = (1) p / p( j) (gm+1 ).

(7.3.20)

( j)

Proof. First, it is clear that  p ( p + 1) is dened only for 0 p m 1 now. Next,

from (7.3.8), we have
( j)

 (pj) ( p + 1) =

F0 ( p + 1)
( j)
F1 ( p)

|h 1 ( j) h p+1 ( j)|
.
|g1 ( j) h 1 ( j) h p ( j)|

(7.3.21)

The result follows by multiplying the ith rows of the ( p + 1) ( p + 1) numerator and
denominator determinants in (7.3.21) by t j+i1 , i = 1, . . . , p + 1, and by invoking
( j)


(7.3.4) and (7.3.3) and the denition of p (b) from (3.3.5).
( j)

( j)

Theorem 7.3.3 implies that  p ( p + 1), 0 p m 1, and D p , 1 p m, can

be obtained simultaneously by the FS-algorithm. As mentioned previously, we are using
( j)
the FS-algorithm for computing D p , 1 p m, as part of the W(m) -algorithm.
Through gi (l) = tl gim (l), i > m, that is given in (7.3.3), when p m + 1, the normal
( j)
( j)
( j)
( j)
( j)
ordering enables us to relate F pm (m) to G p and hence D p (m) to D p (0) = D p ,
the desired quantity, thus reducing the amount of computation for the W(m) -algorithm
considerably.
( j)

Theorem 7.3.4 With the normal ordering and for p m + 1, D p satises

D (pj) = D (pj) (m).

(7.3.22)

( j)

Proof. First, it is clear that D p (m) is dened only for p m + 1. Next, from (7.3.7),
we have
k+m1
1

ts+i
G (s)
(7.3.23)
Fk(s) (m) = (1)mk
m+k .
i=0
( j)

Finally, (7.3.22) follows by substituting (7.3.23) in the denition of D p (m) that is given
in (7.3.9).
The proof of (7.3.23) can be achieved as follows: Multiplying the ith row of the
(m + k) (m + k) determinant that represents Fk(s) (m) by ts+i1 , i = 1, 2, . . . , m + k,
and using (7.3.3) and (7.3.4), we obtain
k+m1



ts+i Fk(s) (m) = |gm+1 (s) gm+2 (s) gm+k (s) g1 (s) g2 (s) gm (s)|.

i=0

(7.3.24)

7.3 The W (m) -Algorithm for GREP(m)

169

(j, p 1)
when

(j + 1, p 1)

p=1

(j, p)
Figure 7.3.1:

By making the necessary column permutations, it can be seen that the right-hand side of


(7.3.24) is nothing but (1)mk G (s)
m+k .
( j)

From Theorem 7.3.4, it is thus clear that, for p m + 1, D p can be computed by a

( j)
recursion relation of xed length m. This implies that the cost of computing D p is xed
and, therefore, independent of j and p. Consequently, the total cost of computing all the
( j)
D p , 1 j + p L, is now O(L 2 ) arithmetic operations, as opposed to O(L 3 ) for the
FS-algorithm.
( j)
( j)
As we compute the p (I ), we can also apply Theorem 3.4.1 to compute the pi , with

p

p
( j)
( j)
( j)
( j)
the help of which we can determine  p = i=0 | pi | and ! p = i=0 | pi | |a(t j+i |,
( j)
the quantities of relevance to the numerical stability of A p .
In summary, the results of relevance to the W(m) -algorithm are (3.3.6), (3.3.10),
(3.3.12), (3.4.1), (3.4.2), (7.3.11), (7.3.13), (7.3.14), (7.3.20), and (7.3.22). With the
( j)
A p ordered as in Table 7.3.1, from these results we see that the quantities at the ( j, p)
location in this table are related to others as in Figure 7.3.1 and Figure 7.3.2.
All this shows that the W(m) -algorithm can be programmed so that it computes
Table 7.3.1 row-wise, thus allowing the one-by-one introduction of the sets X l =
{tl , a(tl ), k (tl ), k = 1, . . . , m}, l = 0, 1, 2, . . . , that are necessary for the initial values 0(l) (a), 0(l) (I ), and 0(l) (gk ), k = 1, . . . , m + 1. Once a set X l is introduced, we
( j)
( j)
( j)
( j)
( j)
( j)
can compute all the p (gk ), p (a), p (I ), D p , D p (q), and  p (q) along the row
j + p = l in the order p = 1, 2, . . . , l. Also, before introducing the set X l+1 for the
next row of the table, we can discard some of these quantities and have the remaining
ones overwrite the corresponding quantities along the row j + p = l 1. Obviously,
this saves a lot of storage. More will be said on this later.
Below by initialize (l) we mean

(j + 1, p 2)

(j, p 1)
when p > 1.
(j + 1, p 1)

(j, p)
Figure 7.3.2:

170

7 Recursive Algorithms for GREP

{read tl , a(tl ), gk (l) = k (tl ), k = 1, . . . , m, and set

0(l) (a)

= a(l)/g1 (l), 0(l) (I ) = 1/g1 (l), 0(l) (gk ) = gk (l)/g1 (l), k = 2, . . . , m,

0(l) (gm+1 ) = tl , and 0(l) (1) = 1/tl }.

We recall that a(l) stands for a(tl ) A(yl ) and gk (l) = k (tl ) k (yl ), k = 1, . . . , m,
and tl = ylr .
Algorithm 7.3.5 (W(m) -algorithm)
initialize (0)
for l = 1 to L do
initialize (l)
for p = 1 to l do
j =l p
if p m then
( j)
( j+1)
( j)
D p = p1 (g p+1 ) p1 (g p+1 )
for k = p + 2 to m + 1 do
( j)
( j+1)
( j)
( j)
p (gk ) = [ p1 (gk ) p1 (gk )]/D p
endfor
endif
if p m 1 then
( j)
( j)
 p ( p + 1) = (1) p / p (gm+1 )
endif
for q = 1 to min{ p 1, m 1} do
( j)
( j+1)
( j)
( j+1)
D p (q) =  p2 (q)[1/ p1 (q) 1/ p1 (q)]
q = q + 1
( j)
( j+1)
( j)
( j)
 p (q  ) = [ p1 (q  )  p1 (q  )]/D p (q)
endfor
if p > m then
( j)
( j)
( j+1)
( j)
( j+1)
D p = D p (m) =  p2 (m)[1/ p1 (m) 1/ p1 (m)]
endif
( j)
( j+1)
( j)
( j)
 p (1) = [ p1 (1)  p1 (1)]/D p
( j)

( j+1)

( j)

( j)

( j)

( j+1)

( j)

( j)

p (a) = [ p1 (a) p1 (a)]/D p

p (I ) = [ p1 (I ) p1 (I )]/D p
( j)

( j)

( j)

A p = p (a)/ p (I )
endfor
endfor
From the initialize (l) statement, it is clear that 1 (tl ) = g1 (l) = 0 must hold for all
l in the algorithm.
As can be seen, not every computed quantity has to be saved throughout the course of
computation. Before l is incremented in the statement for l = 1 to L do the following
( j)
newly computed quantities are saved: (i) p (gk ), p + 2 k m + 1, for p m;
( j)
( j)
( j)
(ii)  p (q), 1 q min{ p + 1, m}; and (iii) p (a) and p (I ); all with j 0,

7.3 The W (m) -Algorithm for GREP(m)

171

p 0, and j + p = l. With suitable programming, these quantities can occupy the

storage locations of those computed in the previous stage of this statement. None of the
( j)
( j)
D p and D p (q) needs to be saved. Thus, we need to save approximately m + 2 vectors
of length L when L is large.
As for the operation count, we rst observe that there are L 2 /2 + O(L) lattice points
( j)
( j)
( j)
( j)
( j, p) for 0 j + p L. At each point, we compute A p , p (a), p (I ),  p (q)
( j)
for 1 q min{ p + 1, m}, and D p (q) for 1 q min{ p 1, m}. The total number of arithmetic operations then is (5m + 5)L 2 /2 + O(L), including the rest of the
computation. It can be made even smaller by requiring that only A(0)
p be computed.
We nally mention that SUBROUTINE WMALGM that forms part of the FORTRAN
77 program given in Ford and Sidi [87, Appendix B] implements the W(m) -algorithm
exactly as in Algorithm 7.3.5. This program (with slight changes, but with the same
SUBROUTINE WMALGM) is given in Appendix I of this book.

7.3.4 Simplications for the Cases m = 1 and m = 2

The following result is a consequence of Theorem 7.3.2 and (7.3.23).
( j)

Theorem 7.3.6 With the normal ordering of the k (t)t i , the  p (q) satisfy
mp

(1)


p

t j+i

i=0


m

 (pj) (i) = 1, p m 1.

(7.3.25)

i=1

Application to the Case m = 1

 p
 ( j)
If we let m = 1 in Theorem 7.3.6, we have (1) p
i=0 t j+i  p (1) = 1, p 0.
( j)
Thus, solving for  p (1) and substituting in (7.3.14) with q = 1 there, and recalling that
( j)
( j)
( j)
D p = D p (1) for p 2, we obtain D p = t j+ p t j for p 2. This result is valid also
for p = 1 as can be shown from (3.3.9). We therefore conclude that the W(1) -algorithm
is practically identical to the W-algorithm of the preceding section. The difference be( j)
tween the two is that the W-algorithm uses D p = t j+ p t j directly, whereas the W(1) ( j)
algorithm computes D p with the help of Theorems 7.3.1 and 7.3.4.
Application to the Case m = 2
 ( j)
 p
( j)
If we let m = 2 in Theorem 7.3.6, we have
i=0 t j+i  p (1) p (2) = 1, p 1.
( j)
( j)
( j)
Solving for  p (2) in terms of  p (1) = p (h 1 ), and substituting in (7.3.14) with
( j)
( j)
q = 2 there, and recalling that D p = D p (2) for p 3, we obtain
( j)

( j+1)

( j+1)

D (pj) = [t j p1 (h 1 ) t j+ p p1 (h 1 )]/ p2 (h 1 ),

(7.3.26)

which is valid for p 3. Using (3.3.9), we can show that this is valid for p = 2 as well.
As for p = 1, we have, again from (3.3.9),
( j)

D1 = g2 ( j + 1)/g1 ( j + 1) g2 ( j)/g1 ( j).

(7.3.27)

This allows us to simplify the W(2) -algorithm as follows: Given 1 (tl ) and 2 (tl ), l =
( j)
0, 1, . . . , use (7.3.27) and (3.3.10) to compute 1 (h 1 ). Then, for p = 2, 3, . . . , use

172

7 Recursive Algorithms for GREP

( j)

( j)

( j)

( j)

(7.3.26) to obtain D p and (3.3.10) to obtain p (a), p (I ), and p (h 1 ). For the sake
of completeness, we give below this simplication of the W(2) -algorithm separately.
Algorithm 7.3.7 (Simplied W(2) -algorithm)
( j)

( j)

( j)

1. For j = 0, 1, . . . , set 0 (a) = a(t j )/1 (t j ), 0 (I ) = 1/1 (t j ), and 0 (h 1 ) =

1/t j , and
( j)

D1 = 2 (t j+1 )/1 (t j+1 ) 2 (t j )/1 (t j ).

2. For j = 0, 1, . . . , and p = 1, 2, . . . , compute recursively
( j)

( j+1)

( j)

( j)

( j)

( j+1)

( j)

( j)

p (a) = [ p1 (a) p1 (a)]/D p ,

p (I ) = [ p1 (I ) p1 (I )]/D p ,
( j)

( j+1)

( j)

( j)

p (h 1 ) = [ p1 (h 1 ) p1 (h 1 )]/D p ,
( j)

( j)

( j+1)

D p+1 = [t j p (h 1 ) t j+ p+1 p
( j)

( j)

( j+1)

(h 1 )]/ p1 (h 1 ).

( j)

3. For all j and p, set A p = p (a)/ p (I ).

7.4 Implementation of the d (m) -Transformation by the W(m) -Algorithm

As mentioned in Chapter 6, the d (m) -transformation is actually a GREP(m) that can be implemented by the W(m) -algorithm. This means that in the initialize (l) statement of Algorithm 7.3.5, we need to have the input tl = 1/Rl , a(tl ) = A Rl , and gk (l) = Rlk k1 a Rl ,
k = 1, . . . , m. Nothing else is required in the rest of the algorithm.
Similarly, the W-algorithm can be used to implement the d (1) -transformation. In this
case, we need as input to Algorithm 7.2.4 tl = 1/Rl , a(tl ) = A Rl , and (tl ) = Rl a Rl .
As already mentioned, in applying the W(m) -algorithm we must make sure that 1 (tl ) =
g1 (l) = 0 for all l. Now, when this algorithm is used to implement the d (m) -transformation


k k1
an , k = 1, . . . , m, where
on an innite series
k=1 ak , we can set k (t) = n 
1
t = n , as follows from (6.1.18) and (6.2.1). Thus, 1 (t) = nan . It may happen that
an = 0 for some n, or even for innitely many values of n. [Consider, for instance,
the Fourier series of Example 6.1.7 with B = 1 and C = 0. When = /6, we have
a3+6i = 0, i = 0, 1, . . . .] Of course, we can avoid 1 (tl ) = 0 simply by choosing the Rl
such that a Rl = 0.
Another approach we have found to be very effective that is also automatic is as
follows: Set k (t) = n mk+1 mk an , k = 1, . . . , m. [Note that 1 (t), . . . , m (t) can
be taken as any xed permutation of n k k1 an , k = 1, . . . , m.] In this case, 1 (t) =
n m m1 an , and the chances of m1 an = 0 in general are small when {ak } b(m) and
an = 0. This argument applies also to the case m = 1, for which we already know that
an = 0 for all large n.
The FORTRAN 77 code in Appendix I implements the d (m) -transformation exactly
as we have just described.

7.5 The EW-Algorithm for an Extended GREP(1)

173

In any case, the zero terms of the series must be kept, because they play the same role
as the nonzero terms in the extrapolation process. Without them, the remaining sequence
of (nonzero) terms is no longer in b(m) , and hence the d (m) -transformation cannot be
effective.

7.5 The EW-Algorithm for an Extended GREP(1)

Let us consider the class of functions A(y) that have asymptotic expansions of the form
A(y) A + (y)

k y k as y 0+,

(7.5.1)

k=1

with
i = 0, i = 1, 2, . . . ; 1 < 2 < , and lim i = +.
i

(7.5.2)

This is the class of functions considered in Section 4.6 and described by (4.1.1) and
(4.6.1) with (4.6.2), and with m = 1. The extended GREP(1) for this class of A(y) is then
dened by the linear systems
A(yl ) = A(nj) + (yl )

n


k ylk , j l j + n.

(7.5.3)

k=1

When i are arbitrary, there does not seem to be an efcient algorithm analogous to the
W-algorithm. In such a case, we can make use of the FS-algorithm or the E-algorithm to
( j)
determine the An . An efcient algorithm becomes possible, however, when yl are not
arbitrary, but yl = y0 l , l = 1, 2, . . . , for some y0 (0, b] and (0, 1).
Let us rewrite (7.5.3) in the form
n

A(y j+i ) An
k
=
k y j+i
, 0 i n.
(y j+i )
k=1
( j)

(7.5.4)

We now employ the technique used in the proof of Theorem 1.4.5. Set k = ck , k =
1, 2, . . . , and let
Un (z) =

n
n


z ci

ni z i .
1 ci
i=1
i=0

(7.5.5)

Multiplying both sides of (7.5.4) by ni , summing from i = 0 to i = n, we obtain

n

i=0

n

A(y j+i ) An
=
k y j k Un (ck ) = 0,
(y j+i )
k=1
( j)

ni

from which we have

A(nj) =

n
( j)
Mn
i=0 ni [A(y j+i )/(y j+i )]

n
( j) .
Nn
i=0 ni [1/(y j+i )]

(7.5.6)

(7.5.7)

Therefore, Algorithm 1.3.1 that was used in the recursive computation of (1.4.5), can be
( j)
( j)
used for computing the Mn and Nn .

174

7 Recursive Algorithms for GREP

Algorithm 7.5.1 (EW-algorithm)

( j)

( j)

1. For j = 0, 1, . . . , set M0 = A(y j )/(y j ) and N0 = 1/(y j ).

( j)
( j)
2. For j = 0, 1, . . . , and n = 1, 2, . . . , compute Mn and Nn recursively from
( j+1)

Mn( j) =

( j)

( j+1)

( j)

cn Nn1
Mn1 cn Mn1
N
and Nn( j) = n1
.
1 cn
1 cn
( j)

( j)

(7.5.8)

( j)

3. For all j and n, set An = Mn /Nn .

Thus the EW-algorithm, just as the W-algorithm, constructs two tables of the form of
( j)
( j)
Table 7.2.1 for the quantities Mn and Nn .
( j)
( j)
We now go on to discuss the computation of n and !n . By (7.5.7), we have
( j)

ni =

ni /(y j+i )
( j)

Nn

, i = 0, 1, . . . , n.

(7.5.9)

As the ni are independent of j, they can be evaluated inexpensively from (7.5.5). They
( j)
( j)
can then be used to evaluate any of the n and !n as part of the EW-algorithm, since
( j)
Nn and (yi ) are already available. In particular, we may be content with the n(0) and
(0)
!(0)
n , n = 1, 2, . . . , associated with the diagonal sequence {An }n=0 .
( j)
It can be shown, by using (7.5.5) and (7.5.9), that n = 1 in the cases (i) ck are
positive and (yi ) alternate in sign, and (ii) ck are negative and (yi ) have the same sign.
It can also be shown that, when the ck are either all positive or all negative and the
( j)
( j)
(yi ) are arbitrary, n and !n can be computed as part of the EW-algorithm by adding
to Algorithm 7.5.1 the following:
(i) Add the following initial conditions to Step 1:
( j)

( j)

( j)

( j)

( j)

( j)

H0 = (1) j |N0 |, K 0 = (1) j |M0 |, if ck > 0, k = 1, 2, . . . ,

( j)

( j)

H0 = |N0 |, K 0 = |M0 |, if ck < 0, k = 1, 2, . . . .

( j+1)

( j)

( j+1)

( j)

cn K n1
Hn1 cn Hn1
K
and K n( j) = n1
to Step 2.
1 cn
1 cn



 H ( j) 
 K ( j) 
 n 
 n 
=  ( j)  and !(nj) =  ( j)  to Step 3.
 Nn 
 Nn 

(ii) Add Hn( j) =

(iii) Add n( j)

The proof of this can be accomplished by realizing that, when the ck are all positive,
Hn( j) =

n


ni

i=0

n

(1) j+i |A(y j+i )|
(1) j+i
and K n( j) =
,
ni
|(y j+i )|
|(y j+i )|
i=0

and, when the ck are all negative,

Hn( j) =

n

i=0

(cf. Theorem 7.2.3).

ni

n

|A(y j+i )|
1
and K n( j) =
.
ni
|(y j+i )|
|(y j+i )|
i=0

7.5 The EW-Algorithm for an Extended GREP(1)

175

Before closing this section, we would like to discuss briey the application of the
extended GREP(1) and the EW-algorithm to a sequence {Am }, for which
Am A + gm

k ckm as m ,

(7.5.10)

k=1

where
ck = 1, k = 1, 2, . . . , |c1 | > |c2 | > , and lim ck = 0.
k

(7.5.11)

The gm and the ck are assumed to be known, and A, the limit or antilimit of {Am }, is
sought. The extended GREP(1) is now dened through the linear systems
Al = A(nj) + gl

n


k ckl , j l j + n,

(7.5.12)

k=1

and it can be implemented by Algorithm 7.5.1 by replacing A(ym ) by Am and (ym ) by

gm in Step 1 of this algorithm.

8
Analytic Study of GREP(1) : Slowly Varying A(y) F(1)

( j)

8.1 Introduction and Error Formula for An

In Section 4.4, we gave a brief convergence study of GREP(m) for both Process I and
Process II. In this study, we treated the cases in which GREP(m) was stable. In addition,
we made some practical remarks on stability of GREP(m) in Section 4.5. The aim of
the study was to justify the preference given to Process I and Process II as the relevant
limiting processes to be used for approximating A, the limit or antilimit of A(y) as
y 0+. We also mentioned that stability was not necessary for convergence and that
convergence could be proved at least in some cases in which the extrapolation process
is clearly unstable.
In this chapter as well as the next, we would like to make more rened statements about
the convergence and stability properties of GREP(1) , the simplest form and prototype of
GREP, as it is being applied to functions A(y) F(1) .
Before going on, we mention that this chapter is an almost exact reproduction of the
recent paper Sidi [306]1 .
As we will be using the notation and results of Section 7.2 on the W-algorithm, we
believe a review of this material is advisable at this point. We recall that A(y) F(1) if
A(y) = A + (y)(y), y (0, b] for some b > 0,

(8.1.1)

where y can be a continuous or discrete variable, and ( ), as a function of the continuous

variable , is continuous in [0, ] for some > 0 and has a Poincare-type asymptotic
expansion of the form
( )

i ir as 0+, for some xed r > 0.

(8.1.2)

i=0
1/r
We also recall that A(y) F(1)
), as a function of the con if the function B(t) (t
r

tinuous variable t, is innitely differentiable in [0, ]. [Therefore, in the variable t,

i t i as t 0+.]
B(t) C[0, t] for some t > 0 and (8.1.2) reads B(t) i=0
r
Finally, we recall that we have set t = y , a(t) = A(y), (t) = (y), and tl = ylr , l =
( j)
0, 1, . . . , and that br t0 > t1 > > 0 and liml tl = 0. An is given by (7.2.4),
1

First published electronically in Mathematics of Computation, November 28, 2001, and later in Mathematics
of Computation, Volume 71, Number 240, 2002, pp. 15691596, published by the American Mathematical
Society.

176

( j)

8.1 Introduction and Error Formula for An

( j)

177
( j)

with the divided difference operator Dn as dened by (7.2.3). Similarly, n is given

by (7.2.12). Of course, (tl ) = 0 for all l.
We aim at presenting various convergence and stability results for GREP(1) in the
presence of different functions (t) and different collocation sequences {tl }. Of course,
(t) are not arbitrary. They are chosen to cover the most relevant cases of the D (1) transformation for innite integrals and the d (1) - and d(m) -transformations for innite series. The sequences {tl } that we consider satisfy (i) tl+1 = tl for all l; or
(ii) liml (tl+1 /tl ) = ; or (iii) tl+1 tl for all l, being a xed constant in (0, 1) in
all three cases; or (iv) liml (tl+1 /tl ) = 1, in particular, tl cl q as l , for some
c, q > 0. These are the most commonly used collocation sequences. Thus, by drawing
the proper analogies, the results of this chapter and the next apply very naturally to the
D (1) -, d (1) -, and d(m) -transformations. We come back to these transformations in Chapter 10, where we show how the conclusions drawn from the study of GREP(1) can be
used to enhance their performance in nite-precision arithmetic.
Surprisingly, GREP(1) is quite amenable to rigorous and rened analysis, and the
conclusions that we draw from the study of GREP(1) are relevant to GREP(m) with
arbitrary m, in general.
It is important to note that the analytic study of GREP(1) is made possible by the divided
( j)
( j)
difference representations of An and n that are given in Theorems 7.2.1 and 7.2.3.
With the help of these representations, we are able to produce results that are optimal
or nearly optimal in many cases. We must also add that not all problems associated
with GREP(1) have been solved, however. In particular, various problems concerning
Process II are still open.
In this chapter, we concentrate on functions A(y) that vary slowly as y 0+, while
in the next chapter we treat functions A(y) that vary quickly as y 0+. How A(y)
varies depends only on the behavior of (y) as y 0+ because (y) behaves polynomially in y r and hence varies slowly as y 0+. Specically, (y) 0 if 0 = 0, and
(y) s y sr for some s > 0, otherwise. Thus, A(y) and (y) vary essentially in the same
manner. If (y) behaves like some power y ( may be complex), then A(y) varies slowly.
If (y) behaves like some exponential function exp[w(y)], where w(y) ay , < 0,
then A(y) varies quickly.
In the remainder of this section, we present some preliminary results that will be useful
in the analysis of GREP(1) in this chapter and the next.
( j)
We start by deriving an error formula for An .
( j)

Lemma 8.1.1 The error in An is given by

( j)

A(nj) A =

Dn {B(t)}
( j)

Dn {1/(t)}

; B(t) (t 1/r ).

(8.1.3)

Proof. The result follows from a(t) A = (t)B(t) and from (7.2.4) in Theorem 7.2.1
( j)


and from the linearity of Dn .
It is clear from Lemma 8.1.1 that the convergence analysis of GREP(1) on F(1) is based
( j)
( j)
on the study of Dn {B(t)} and Dn {1/(t)}. Similarly, the stability analysis is based on

8 Analytic Study of GREP(1) : Slowly Varying A(y) F(1)

178

( j)

the study of n given in (7.2.12), which we reproduce here for the sake of completeness:
n( j) =

n


n


|Dn {1/(t)}|

i=0

( j)

|ni | =

i=0

( j)

n

|cni |
1
( j)
; cni =
. (8.1.4)
|(t j+i )|
t j+i t j+k
k=0
( j)

k=i

In some of our analyses, we assume the functions (t) and B(t) to be differentiable;
in others, no such requirement is imposed. Obviously, the assumption in the former case
is quite strong, and this makes some of the proofs easier.
( j)
The following simple result on An will become useful shortly.
Lemma 8.1.2 If B(t) C [0, t j ] and (t) 1/(t) C (0, t j ], then for any nonzero
complex number c,
A(nj) A =

[cB (n) (t jn,1 )] + i![cB (n) (t jn,2 )]

[c (n) (t jn,1 )] + i![c (n) (t jn,2 )]

for some t jn,1 , t jn,2 , t jn,1 , t jn,2 (t j+n , t j ).

(8.1.5)

Proof. It is known that if f C n [a, b] is real and a x0 x1 xn b, then the

divided difference f [x0 , x1 , . . . , xn ] satises
f [x0 , x1 , . . . , xn ] =

f (n) ( )
for some (x0 , xn ).
n!

Applying this to the real and imaginary parts of the complex-valued function u(t)
C n (0, t j ), we have
Dn( j) {u(t)} =


1  (n)
u (t jn,1 ) + i!u (n) (t jn,2 ) , for some t jn,1 , t jn,2 (t j+n , t j ).
n!
(8.1.6)

The result now follows.

The constant c in (8.1.5) serves us in the proof of Theorem 8.3.1 in the next section.
Note that, in many of our problems, it is known that B(t) C [0, t] for some t > 0,
whereas (t) C (0, t] only. That is to say, B(t) has an innite number of derivatives
at t = 0, and (t) does not. This is an important observation.
A useful simplication takes place in (8.1.5) for the case (t) = t. In this case, GREP(1)
is, of course, nothing but the polynomial Richardson extrapolation process that has been
studied most extensively in the literature, which we studied to some extent in Chapter 2.
Lemma 8.1.3 If (t) = t in Lemma 8.1.2, then, for some t jn,1 , t jn,2 (t j+n , t j ),
A(nj) A = (1)n

 n
[B (n) (t jn,1 )] + i![B (n) (t jn,2 )] 
n!

i=0


t j+i .

(8.1.7)

( j)

8.1 Introduction and Error Formula for An

179

In this case, it is also true that

An( j)

A = (1)


n

( j)
Dn+1 {a(t)}


t j+i ,

(8.1.8)

i=0

so that, for some t jn,1 , t jn,2 (t j+n , t j ),

A(nj) A = (1)n


 n
[a (n+1) (t jn,1 )] + i![a (n+1) (t jn,2 )] 
t j+i .
(n + 1)!
i=0

(8.1.9)

The rst part is proved by invoking in (8.1.3) the fact Dn {t 1 } = (1)n

Proof.
n
1
that was used in Chapter 7. We leave the rest to the reader.


i=0 t j+i
( j)

Obviously, by imposing suitable growth conditions on B (n) (t), Lemma 8.1.3 can be
turned into powerful convergence theorems.
The last result of this section is a slight renement of Theorem 4.4.2 concerning
Process I as it applies to GREP(1) .
( j)

Theorem 8.1.4 Let sup j n = $n < and (y j+1 ) = O((y j )) as j . Then,

with n xed,
n+

A(nj) A = O((t j )t j

) as j ,

(8.1.10)

where n+ is the rst nonzero i with i n.

Proof. Following the proof of Theorem 4.4.2, we start with
A(nj) A =

n


( j)

ni (t j+i )[B(t j+i ) u(t j+i )]; u(t) =

i=0

n1


k t k .

(8.1.11)

k=0

The result now follows by taking moduli on both sides and realizing that B(t) u(t)
n+ t n+ as t 0+ and recalling that t j > t j+1 > t j+2 > . We leave the details to
the reader.


Note that Theorem 8.1.4 does not assume that B(t) and (t) are differentiable. It does,
however, assume that Process I is stable.
As we will see in the sequel, (8.1.10) holds under appropriate conditions on the tl even
when Process I is clearly unstable.
In connection with Process I, we would like to remark that, as in Theorem 3.5.5, under
( j)
suitable conditions we can obtain a full asymptotic expansion for An A as j .
If we dene
( j)

( j)

n,k =

Dn {t k }
( j)

Dn {1/(t)}

, k = 0, 1, . . . ,

(8.1.12)

8 Analytic Study of GREP(1) : Slowly Varying A(y) F(1)

180

( j)

and recall that Dn {t k } = 0, for k = 0, 1, . . . , n 1, then this expansion assumes the

simple and elegant form
A(nj) A

( j)

k n,k as j .

(8.1.13)

k=n
( j)

If n+ is the rst nonzero i with i n, then An A also satises the asymptotic

equality
( j)

A(nj) A n+ n,n+ as j .

(8.1.14)

Of course, all this will be true provided that (i) {n,k }

k=n is an asymptotic se( j)
( j)
( j)
quence as j , that is, lim j n,k+1 /n,k = 0 for all k n, and (ii) An A

s1
( j)
( j)
k=n k n,k = O(n,s ) as j , for each s n. In the next sections, we aim at such
results whenever possible. We show that they are possible in most of the cases we treat.
We close this section with the well-known HermiteGennochi formula for divided
differences, which will be used later.
( j)

Lemma 8.1.5 (HermiteGennochi). Let f (x) be in C n [a, b], and let x0 , x1 , . . . , xn be

all in [a, b]. Then



n
(n)
f
i xi d1 dn ,
f [x0 , x1 , . . . , xn ] =
Tn

where

i=0

Tn = (1 , . . . , n ) : 0 i 1, i = 1, . . . , n,

n



i 1 ; 0 = 1

i=1

n


i .

i=1

For a proof of this lemma, see, for example, Atkinson [13]. Note that the argument

n
i xi of f (n) above is actually a convex combination of x0 , x1 , . . . , xn bez = i=0

n
i = 1. If we order the xi such that x0
cause 0 i 1, i = 0, 1, . . . , n, and i=0
x1 xn , then z [x0 , xn ] [a, b].
8.2 Examples of Slowly Varying a(t)
Our main concern in this chapter is with functions a(t) that vary slowly as t 0+. As
we mentioned earlier, by this we mean that (t) h 0 t as t 0+ for some h 0 = 0
and that may be complex in general. In other words, (t) = t H (t) with H (t) h 0


as t 0+. In most cases, H (t) i=0
h i t i as t 0 + . When  > 0, limt0+ a(t)
exists and is equal to A. When limt0+ a(t) does not exist, we have  0 necessarily,
and A is the antilimit of a(t) as t 0+ in this case, with some restriction on .
We now present practical examples of functions a(t) that vary slowly.
Example 8.2.1 If f A( ) strictly for some possibly complex = 1, 0, 1, 2, . . . ,
then we know from Theorem 5.7.3 that
 x
F(x) =
f (t) dt = I [ f ] + x f (x)g(x); g A(0) strictly.
0

8.3 Slowly Varying (t) with Arbitrary tl

181

This means that F(x) a(t), I [ f ] A, x 1 t, x f (x) (t) with (t) as above
and with = 1. (Recall that f B(1) in this case.)
( )

Example 8.2.2 If an = h(n) A0 strictly for some possibly complex = 1, 0, 1,

2, . . . , then we know from Theorem 6.7.2 that
An =

n


ak = S({ak }) + nan g(n); g A(0)

0 strictly.

k=1

This means that An a(t), S({ak }) A, n 1 t, nan (t) with (t) as above and
with = 1. (Recall that {an } b(1) in this case.)
( ,m)

strictly for some possibly complex = 1 +

Example 8.2.3 If an = h(n) A
0
i/m, i = 0, 1, . . . , and a positive integer m > 1, then we know from Theorem 6.6.6 that

An =

n


(0,m) strictly.
ak = S({ak }) + nan g(n); g A
0

k=1

This means that An a(t), S({ak }) A, n 1/m t, nan (t) with (t) as above
and with = 1. (Recall that {an } b (m) in this case.)
8.3 Slowly Varying (t) with Arbitrary tl
We start with the following surprising result that holds for arbitrary {tl }. (Recall that so
far the tl satisfy only t0 > t1 > > 0 and liml tl = 0.)
Theorem 8.3.1 Let (t) = t H (t), where is in general complex and = 0, 1,
2, . . . , and H (t) C [0, t] for some t > 0 with h 0 H (0) = 0. Let B(t) C [0, t],
and let n+ be the rst nonzero i with i n in (8.1.2). Then, provided n ,
we have
n+

A(nj) A = O((t j )t j

) as j .

(8.3.1)

( j)

Consequently, if n > , we have lim j An = A. All this is valid for arbitrary {tl }.
Proof. By the assumptions on B(t), we have
B (n) (t)

i(i 1) (i n + 1)i t in as t 0+,

i=n+

from which
B (n) (t) ( + 1)n n+ t as t 0+,

(8.3.2)

8 Analytic Study of GREP(1) : Slowly Varying A(y) F(1)

182

and by the assumptions on (t), we have for (t) 1/(t)

n  

n

(t) =
(n)

k=0

(k)

(n)

(t ) [1/H (t)](nk) (t )(n) /H (t) (t ) / h 0 as t 0+,

from which
n
(1)n ()n (t)t n as t 0 + .
(n) (t) (1)n h 1
0 ()n t

(8.3.3)

Also, Lemma 8.1.2 is valid for all sufciently large j under the present assumptions on
B(t) and (t), because t j < t for all sufciently large j. Substituting (8.3.2) and (8.3.3)
in (8.1.5) with |c| = 1 there, we obtain
A(nj) A = (1)n ( + 1)n

[(cn+ ) + o(1)](t jn,1 ) + i [!(cn+ ) + o(1)](t jn,2 )

[ jn,1 + o(1)](t jn,1 )n + i [! jn,2 + o(1)](t jn,2 )n

as j , (8.3.4)

with jn,s ch 0 1 ()n (t jn,s )i! and the o(1) terms uniform in c, |c| = 1. Here, we
have also used the fact that lim j t jn,s = lim j t jn,s = 0. Next, by 0 < t jn,s < t j

and 0, it follows that (t jn,s ) t j . This implies that the numerator of the quo
tient in (8.3.4) is O(t j ) as j , uniformly in c, |c| = 1. As for the denominator,
we start by observing that a = h 1
0 ()n = 0. Therefore, either a = 0 or !a = 0, and
we assume without loss of generality that a = 0. If we now choose c = (t jn,1 )i! , we
obtain jn,1 = a and hence  jn,1 = a = 0, as a result of which the modulus of the
denominator can be bounded from below by |a + o(1)|(t jn,1 )n , which in turn is
bounded below by |a + o(1)| t n
, since 0 < t jn,s < t j and  + n 0. The rej
sult now follows by combining everything in (8.3.4) and by invoking t = O((t)) as


t 0+, which follows from (t) h 0 t as t 0+.
( j)

Theorem 8.3.1 implies that the column sequence {An } j=0 converges to A if n > ;
it also gives an upper bound on the rate of convergence through (8.3.1). The fact that
convergence takes place for arbitrary {tl } and that we are able to prove that it does is
quite unexpected.
( j)
By restricting {tl } only slightly in Theorem 8.3.1, we can show that An A has
the full asymptotic expansion given in (8.1.13) and, as a result, satises the asymptotic
equality of (8.1.14) as well. We start with the following lemma that turns out to be very
useful in the sequel.
Lemma 8.3.2 Let g(t) = t u(t), where is in general complex and u(t) C [0, t] for
some t > 0. Pick the tl to satisfy, in addition to tl+1 < tl , l = 0, 1, . . . , also tl+1 tl

8.3 Slowly Varying (t) with Arbitrary tl

183

for all sufciently large l with some (0, 1). Then, the following are true:

(i) The nonzero members of {Dn {t +i }}i=0

form an asymptotic sequence as j .
( j)

( j)

(ii) Dn {g(t)} has the bona de asymptotic expansion

Dn( j) {g(t)}

gi Dn( j) {t +i } as j ; gi = u (i) (0)/i!, i = 0, 1, . . . , (8.3.5)

i=0

where the asterisk on the summation means that only those terms for which
( j)
Dn {t +i } = 0, that is, for which + i = 0, 1, . . . , n 1, are taken into account.
Remark. The extra condition tl+1 tl for all large l that we have imposed on the tl
is satised, for example, when liml (tl+1 /tl ) = for some (0, 1], and such cases
are considered further in the next sections.
 
Proof. Let be in general complex and = 0, 1, . . . , n 1. Denote n = M for simplicity of notation. Then, by (8.1.6), for any complex number c such that |c| = 1, we
have




cDn( j) {t } =  cM(t jn,1 )n + i! cM(t jn,2 )n
for some t jn,1 , t jn,2 (t j+n , t j ), (8.3.6)
from which we also have
|Dn( j) {t }| max{| [cM(t jn,1 )n ]|, |! [cM(t jn,2 )n ]|}.

(8.3.7)

Because M = 0, we have either M = 0 or !M = 0. Assume without loss of generality

that M = 0 and choose c = (t jn,1 )i! . Then,  [cM(t jn,1 )n ] = (M)(t jn,1 )n
and hence
|Dn( j) {t }| |M|(t jn,1 )n |M| min (t n ).
t[t j+n ,t j ]

(8.3.8)

Invoking in (8.3.8), if necessary, the fact that t j+n n t j , which is implied by the
conditions on the tl , we obtain
() n
()
|Dn( j) {t }| Cn1
tj
for all large j, with some constant Cn1
> 0. (8.3.9)

Similarly, we can show from (8.3.6) that

() n
()
|Dn( j) {t }| Cn2
tj
for all large j, with some constant Cn2
> 0. (8.3.10)

The proof of part (i) can now be achieved by using (8.3.9) and (8.3.10).
To prove part (ii), we need to show [in addition to part (i)] that, for any integer s for
( j)
which Dn {t +s } = 0, that is, for which + s = 0, 1, . . . , n 1, there holds
Dn( j) {g(t)}

s1

i=0



gi Dn( j) {t +i } = O Dn( j) {t +s } as j .

(8.3.11)

8 Analytic Study of GREP(1) : Slowly Varying A(y) F(1)

184
Now g(t) =

s1
i=0

gi t +i + vs (t)t +s , where vs (t) C [0, t]. As a result,

Dn( j) {g(t)}

s1


gi Dn( j) {t +i } = Dn( j) {vs (t)t +s }.

(8.3.12)

i=0

Next, by (8.1.6) and by the fact that

n  

n
[vs (t)](ni) (t +s )(i) vs (t)(t +s )(n) gs (t +s )(n) as t 0+,
[vs (t)t +s ](n) =
i
i=0
and by the additional condition on the tl again, we obtain
) = O(Dn( j) {t +s }) as j ,
Dn( j) {vs (t)t +s } = O(t +sn
j

(8.3.13)

with the last equality being a consequence of (8.3.9). Here we assume that gs = 0 without
loss of generality. By substituting (8.3.13) in (8.3.12), the result in (8.3.11) follows. This
completes the proof.


Theorem 8.3.3 Let (t) and B(t) be exactly as in Theorem 8.3.1, and pick the tl as in
( j)
Lemma 8.3.2. Then An A has the complete asymptotic expansion given in (8.1.13)
and hence satises the asymptotic equality in (8.1.14) as well. Furthermore, if n+ is
the rst nonzero i with i n, then, for all large j, there holds
n+

$1 |(t j )| t j

n+

|A(nj) A| $2 |(t j )| t j

, for some $1 > 0 and $2 > 0,

(8.3.14)

whether n  or not.
Proof. The proof of the rst part can be achieved by applying Lemma 8.3.2 to B(t) and
to (t) 1/(t). The proof of the second part can be achieved by using (8.3.9) as well.
We leave the details to the reader.


Remark. It is important to make the following observations concerning the behavior of
( j)
( j)
An A as j in Theorem 8.3.3. First, any column sequence {An } j=0 converges
( j)
at least as quickly as (or diverges at most as quickly as) the column sequence {An1 } j=0
that precedes it. In other words, each column sequence is at least as good as the one
preceding it. In particular, when m = 0 but m+1 = = s1 = 0 and s = 0, we
have
A(nj) A = o(A(mj) A) as j , m + 1 n s,
( j)

As+1 A = o(A(s j) A) as j .

(8.3.15)

In addition, for all large j we have

n1 |A(s j) A| |A(nj) A| n2 |A(s j) A|, m + 1 n s 1,
for some n1 , n2 > 0, (8.3.16)

8.4 Slowly Varying (t) with liml (tl+1 /tl ) = 1

185

( j)

which implies that the column sequences {An } j=0 , m + 1 n s, behave the same
way for all large j.
In the next sections, we continue the treatment of Process I by restricting the tl further, and we treat the issue of stability for Process I as well. In addition, we treat the
convergence and stability of Process II.

8.4 Slowly Varying (t) with liml (tl+1 /tl ) = 1

8.4.1 Process I with (t) = t H (t) and Complex
Theorem 8.4.1 Assume that (t) and B(t) are exactly as in Theorem 8.3.1. In addition,
( j)
pick the tl such that liml (tl+1 /tl ) = 1. Then, An A has the complete asymptotic
expansion given in (8.1.13) and satises (8.1.14) and hence satises also the asymptotic
equality
A(nj) A (1)n

( + 1)n
n+
n+ (t j )t j
as j ,
()n

(8.4.1)

where, again, n+ is the rst nonzero i with i n in (8.1.2). This result is valid whether
( j)
n  or not. In addition, Process I is unstable, that is, sup j n = .
Proof. First, Theorem 8.3.3 applies and thus (8.1.13) and (8.1.14) are valid.
Let us apply the HermiteGennochi formula of Lemma 8.1.5 to the function t , where
may be complex in general. By the assumption that liml (tl+1 /tl ) = 1, we have

n
i t j+i of the integrand in Lemma 8.1.5 satises z t j as
that the argument z = i=0
j . As a result, we obtain
 
n
t
as j , provided = 0, 1, . . . , n 1.
(8.4.2)
Dn( j) {t }
n j
Next, applying Lemma 8.3.2 to B(t) and to (t) 1/(t), and realizing that (t)


+i
as t 0+ for some constants i with 0 = h 1
i=0 i t
0 , and using (8.4.2) as
well, we have




n+

( j)
( j) i
( j) n+
i Dn {t } n+ Dn {t
}
n+ t j as j ,
Dn {B(t)}
n
i=n+
(8.4.3)
and
Dn( j) {(t)}


i=0

i Dn( j) {t +i }

( j)
h 1
0 Dn {t }

 

(t j )t n
as j .
j
n
(8.4.4)

The result in (8.4.1) is obtained by dividing (8.4.3) by (8.4.4).

For the proof of the second part, we start by observing that when liml (tl+1 /tl ) = 1
we also have lim j (t j+k /t j+i ) = 1 for arbitrary xed i and k. Therefore, for every

8 Analytic Study of GREP(1) : Slowly Varying A(y) F(1)

186

 > 0, there exists a positive integer J , such that




t j+k 
t j+i < t j+i t j for 0 i, k n and j > J. (8.4.5)
|t j+i t j+k | = 1
t j+i 
As a result of this,
( j)

|cni | =

n

k=0
k=i

|t j+i

1
> (t j )n for i = 0, 1, . . . , n, and j > J.
t j+k |

(8.4.6)

Next, by the assumption that H (t) h 0 as t 0+ and by lim j (t j+i /t j ) = 1, we

have that (t j+i ) (t j ) as j , from which |(t j+i )| K 1 |(t j )| for 0 i n
and all j, where K 1 > 0 is a constant independent of j. Combining this with (8.4.6), we
have
n


|cni | |(t j+i )| K 1 (n + 1)(t j )n |(t j )| for all j > J.

( j)

(8.4.7)

i=0

Similarly, |Dn {(t)}| K 2 |(t j )| t n

j for all j, where K 2 > 0 is another constant independent of j. (K 2 depends only on n.) Substituting this and (8.4.7) in (8.1.4), we obtain
( j)

n( j) Mn  n for all j > J, with Mn = (K 1 /K 2 )(n + 1) independent of  and j.

(8.4.8)
( j)

Since  can be chosen arbitrarily close to 0, (8.4.8) implies that sup j n = .

Obviously, the remarks following Theorem 8.3.3 are valid under the conditions of
Theorem 8.4.1 too. In particular, (8.3.15) and (8.3.16) hold. Furthermore, (8.3.16) can
now be rened to read
A(nj) A n (A(s j) A) as j , m + 1 n s 1, for some n = 0.
( j)

Finally, the column sequences {An } j=0 with n >  converge even though they
are unstable.
In Theorem 8.4.3, we show that the results of Theorem 8.4.1 remain unchanged if
we restrict the tl somewhat while we still require that liml (tl+1 /tl ) = 1 but relax the
conditions on (t) and B(t) considerably. In fact, we do not put any differentiability
requirements either on (t) or on B(t) this time, and we obtain an asymptotic equality
( j)
for n as well.
The following lemma that is analogous to Lemma 8.3.2 will be useful in the proof of
Theorem 8.4.3.


gi t +i as t 0+, where g0 = 0 and is in general
Lemma 8.4.2 Let g(t) i=0
complex, and let the tl satisfy
tl cl q and tl tl+1 cpl q1 as l , for some c > 0, p > 0, and q > 0.
(8.4.9)

8.4 Slowly Varying (t) with liml (tl+1 /tl ) = 1

187

Then, the following are true:

(i) The nonzero members of {Dn {t +i }}i=0

form an asymptotic sequence as j .
( j)
(ii) Dn {g(t)} has the bona de asymptotic expansion
( j)

Dn( j) {g(t)}

gi Dn( j) {t +i } as j ,

(8.4.10)

i=0

where the asterisk on the summation means that only those terms for which
( j)
Dn {t +i } = 0, that is, for which + i = 0, 1, . . . , n 1, are taken into account.
Remark. Note that liml (tl+1 /tl ) = 1 under (8.4.9), and that (8.4.9) is satised by
tl = c(l + )q , for example. Also, the rst part of (8.4.9) does not necessarily imply
the second part.
Proof. Part (i) is true by Lemma 8.3.2, because liml (tl+1 /tl ) = 1. In particular, (8.4.2)
holds. To prove part (ii), we need to show in addition that, for any integer s for which
( j)
Dn {t +s } = 0, there holds
Dn( j) {g(t)}

s1




gi Dn( j) {t +i } = O Dn( j) {t +s } as j .

(8.4.11)

i=0

m1 +i
gi t
+ vm (t)t +m , where |vm (t)| Cm for some constant Cm > 0
Now, g(t) = i=0
and for all t sufciently close to 0, and this holds for every m. Let us x s and take
m > max{s + n/q, }. We can write
Dn( j) {g(t)} =

s1


gi Dn( j) {t +i } +

m1


gi Dn( j) {t +i } + Dn( j) {vm (t)t +m }. (8.4.12)

i=s

i=0

Let us assume, without loss of generality, that gs = 0. Then, by part (i) of the lemma,
m1


gi Dn( j) {t +i } gs Dn( j) {t +s } gs

i=s



+ s +sn
as j .
tj
n



( j)
Therefore, the proof will be complete if we show that Dn {vm (t)t +m } = O t j +sn as
j . Using also the fact that t j+i t j as j , we rst have that
|Dn( j) {vm (t)t +m }| Cm

n

i=0

|cni | t +m
Cm t +m
j+i
j
( j)


n


( j)
|cni | .

(8.4.13)

i=0

Next, from (8.4.9),

t j+i t j+k cp(k i) j q1 p(k i) j 1 t j as j ,

(8.4.14)

8 Analytic Study of GREP(1) : Slowly Varying A(y) F(1)

188

as a result of which,
( j)
cni

n

k=0
k=i

t j+i


 
j n
1
i 1 n
(1)
, 0 i n, and
t j+k
n! i
pt j
n


( j)

|cni |

i=0

1
n!

2j
pt j

n
as j .

(8.4.15)

Substituting (8.4.15) in (8.4.13) and noting that (8.4.9) implies j (t j /c)1/q as j ,

we obtain
+mnn/q

+sn
) = O(t 
) as j ,
j

(8.4.16)

by the fact that  + m n n/q >  + s n. The result now follows.

Dn( j) {vm (t)t +m } = O(t j

Theorem 8.4.3 Assume that (t) = t H (t), with in general complex and = 0, 1,




h i t i as t 0+ and B(t) i=0
i t i as t 0+. Let us pick
2, . . . , H (t) i=0
( j)
the tl to satisfy (8.4.9). Then An A has the complete asymptotic expansion given in
(8.1.13) and satises (8.1.14) and hence also satises the asymptotic equality in (8.4.1).
( j)
In addition, n satises the asymptotic equality
 n
2j
1
( j)
n
as j .
(8.4.17)
|()n | p
That is to say, Process I is unstable.
( j)

Proof. The assertion concerning An can be proved by applying Lemma 8.4.2 to B(t)
and to (t) 1/(t) and proceeding as in the proof of Theorem 8.4.1.
( j)
We now turn to the analysis of n . To prove the asymptotic equality in (8.4.17), we

n
( j)
( j)
|cni ||(t j+i )| and Dn {(t)} as j .
need the precise asymptotic behaviors of i=0
By (8.4.15) and by the fact that (t j+i ) (t j ) as j for all xed i, we obtain




n
n

1 2j n
( j)
( j)
|cni | |(t j+i )|
|cni | |(t j )|
|(t j )| as j . (8.4.18)
n! pt j
i=0
i=0
Combining now (8.4.18) and (8.4.4) in (8.1.4), the result in (8.4.17) follows.

So far all our results have been on Process I. What characterizes these results is that
they are all obtained by considering only the local behavior of B(t) and (t) as t 0+.
( j)
The reason for this is that An is determined only by a(tl ), j l j + n, and that in
Process I we are letting j or, equivalently, tl 0, j l j + n. In Process II,
on the other hand, we are holding j xed and letting n . This means, of course, that
( j)
An is being inuenced by the behavior of a(t) on the xed interval (0, t j ]. Therefore,
we need to use global information on a(t) in order to analyze Process II. It is precisely
this point that makes Process II much more difcult to study than Process I.
An additional source of difculty when analyzing Process II with (t) = t H (t) is
complex values of . Indeed, except for Theorem 8.5.2 in the next section, we do not

8.4 Slowly Varying (t) with liml (tl+1 /tl ) = 1

189

have any results on Process II under the assumption that is complex. Our analysis in
the remainder of this section assumes real .
8.4.2 Process II with (t) = t
We now would like to present results pertaining to Process II. We start with the case
(t) = t. Our rst result concerns convergence and follows trivially from Lemma 8.1.3
as follows:
Assuming that B(t) C [0, t j ] and letting
B (n)  = max |B (n) (t)|

(8.4.19)




n
 D ( j) {B(t)}  B (n)  

 n
t j+i ,

 ( j)
 Dn {t 1 } 
n!
i=0

(8.4.20)

0tt j

we have

( j)

from which we have that limn An = A when (t) = t provided that



n
B (n) 
=o
as n .
t 1
j+i
n!
i=0

(8.4.21)

In the special case tl = c/(l + )q for some positive c, , and q, this condition reads
B (n)  = o((n!)q+1 cn n ( j+)q ) as n .

(8.4.22)

We are thus assured of convergence in this case under a very generous growth condition
on B (n) , especially when q 1.
Our next result in the theorem below pertains to stability of Process II.
Theorem 8.4.4 Consider (t) = t and pick the tl such that liml (tl+1 /tl ) = 1. Then,
( j)
( j)
Process II is unstable, that is, supn n = . If the tl are as in (8.4.9), then n
as n faster than n for every > 0. If, in particular, tl = c/(l + )q for some
positive c, , and q, then
 qn
e
for some E q( j) > 0, q = 1, 2.
(8.4.23)
n( j) > E q( j) n 1/2
q
( j)

Proof. We already know that, when (t) = t, we can compute the n by the recursion
relation in (7.2.27), which can also be written in the form
( j)

( j+1)

n( j) = n1 +

wn
1

( j)
wn



t j+n
( j)
( j+1)
n1 + n1 ; wn( j) =
< 1.
tj

(8.4.24)

Hence,
( j+1)

( j+2)

n( j) n1 n2 ,

(8.4.25)

190

8 Analytic Study of GREP(1) : Slowly Varying A(y) F(1)

( j)

( j+ns)

from which n s
for arbitrary xed s. Applying now Theorem 8.4.1, we have
( j+ns)
( j)
= for s 1, from which limn n s= follows. When the tl are
limn s
( j+ns)
s!1 2p n s as n . From this and
as in (8.4.9), we have from (8.4.17) that s
( j)
from the fact that s is arbitrary, we now deduce that n as n faster
than n for every > 0. To prove the last part, we start with
( j)

ni =

n

k=0
k=i

t j+k
, i = 0, 1, . . . , n,
t j+k t j+i

(8.4.26)

( j)

( j)

which follows from (7.2.26). The result in (8.4.23) follows from n > |nn |.

We can expand on the last part of Theorem 8.4.4 by deriving upper bounds on
( j)
n when tl = c/(l + )q for some positive c, , and q. In this case, we rst show
( j+1)
( j)
wn , from which we can prove by induction, and with the help of
that wn
( j)
( j+1)
( j)
( j+1)
(8.4.24), that n n . Using this in (8.4.24), we obtain the inequality n n1
( j)
( j)
[(1 + wn )/(1 wn )], and, by induction,
( j+n)

n( j) 0

n1


1 + wni

( j+i) 

i=0

1 wni

( j+i)

(8.4.27)

( j+n)

= 1 and bound the product in (8.4.27). It can be shown that, when

Finally, we set 0
n
( j)
q = 2, n = O(n 1/2 (e2 /3) ) as n . This result is due to Laurie [159]. On the
basis of this result, Laurie concludes that the error propagation in Romberg integration
with the harmonic sequence of stepsizes is relatively mild.

8.4.3 Process II with (t) = t H (t) and Real

We now want to extend Theorem 8.4.4 to the general case in which (t) = t H (t), where
is real and = 0, 1, 2, . . . , and H (t) C [0, t j ] with H (t) = 0 on [0, t j ]. To do
this, we need additional analytical tools. We make use of these tools in the next sections
as well. The results we have obtained for the case (t) = t will also prove to be very
useful in the sequel.
Lemma 8.4.5 Let 1 and 2 be two real numbers and 1 = 2 . Dene i (t) = t i , i =
1, 2. Then, provided 1 = 0, 1, 2, . . . ,
Dn( j) {2 (t)} =

(2 )n 1 2 ( j)
Dn {1 (t)} for some t jn (t j+n , t j ).
t
(1 )n jn

Proof. From Lemma 8.1.5, with z

n

i=0 i t j+i ,

 


Dn( j) {2 (t)} =

Tn

(n)
2 (z)d1 dn =

Tn

2(n) (z)
1(n) (z)


(n)
1 (z)d1 dn .

8.4 Slowly Varying (t) with liml (tl+1 /tl ) = 1

191

Because (n)
1 (z) is of one sign on Tn , we can apply the mean value theorem to the second
integral to obtain
Dn( j) {2 (t)} =
=

(n)
2 (t jn )
(n)
1 (t jn )


Tn

(n)
1 (z)d1 dn

(n)
2 (t jn ) ( j)
Dn {1 (t)} for some t jn (t j+n , t j ).
(n)
1 (t jn )

This proves the lemma.

Corollary 8.4.6 Let 1 > 2 in Lemma 8.4.5. Then, for arbitrary {tl },




( j)
 (2 )n  1 2
|Dn {2 (t)}|  (2 )n  1 2
t


 ( )  j+n
 ( )  t j
( j)
1 n
1 n
|Dn {1 (t)}|
from which we also have
( j)

|Dn {2 (t)}|

( j)

|Dn {1 (t)}|

K n 2 1 t j 1 2 = o(1) as j and/or as n ,

for some constant K > 0 independent of j and n. Consequently, for arbitrary real and

( j)
arbitrary {tl }, the nonzero members of {Dn {t +i }}i=0 form an asymptotic sequence as
j .
Proof. The rst part follows directly from Lemma 8.4.5, whereas the second part is
obtained by substituting in the rst the known relation
(b) ab
(b) (n + a)
(a)n

n
=
as n .
(b)n
(a) (n + b)
(a)

(8.4.28)

( j)

( j)

The next lemma expresses n and An A in factored forms. Analyzing each of the
factors makes it easier to obtain good bounds from which powerful results on Process II
can be obtained.
Lemma 8.4.7 Consider (t) = t H (t) with real and = 0, 1, 2, . . . . Dene
Dn {t 1 }
( j)

X n( j) =

( j)
Dn {t }

and Yn( j) =

( j)
Dn {t }
.
( j)
Dn {t /H (t)}

(8.4.29)

|cni | t
j+i .

(8.4.30)

Dene also
 n( j) () =

n


( j)

|Dn {t }|

i=0

( j)

8 Analytic Study of GREP(1) : Slowly Varying A(y) F(1)

192
Then

1
 n( j) () = |X n( j) |(t jn )  n( j) (1) for some t jn (t j+n , t j ),

(8.4.31)

1
n( j) = |Yn( j) | |H (t jn )|  n( j) () for some t jn (t j+n , t j ),

(8.4.32)

and
( j)

A(nj) A = X n( j) Yn( j)

Dn {B(t)}
( j)

Dn {t 1 }

(8.4.33)

In addition,
X n( j) =

n!
(tjn )1 for some tjn (t j+n , t j ),
()n

(8.4.34)

These results are valid for all choices of {tl }.

Proof. To prove (8.4.31), we start by writing (8.4.30) in the form
 n( j) () = |X n( j) |

n 



( j)
1
|cni | t 1
j+i t j+i .

( j)

|Dn {t 1 }|

i=0

The result follows by observing that, by continuity of t 1 for t > 0,



n 
n


1
( j)
( j) 1 
1
t jn
|cni | t 1
t
=
|c
|
t
for some t jn (t j+n , t j ),
ni
j+i
j+i
j+i
i=0

i=0

and by invoking (8.4.30) with = 1. The proof of (8.4.32) proceeds along the same
lines, while (8.4.33) is a trivial identity. Finally, (8.4.34) follows from Lemma 8.4.5.


In the next two theorems, we adopt the notation and denitions of Lemma 8.4.7. The
rst of these theorems concerns the stability of Process II, and the second concerns its
convergence.
Theorem 8.4.8 Let be real and = 0, 1, 2, . . . , and let the tl be as in (8.4.9).
Then, the following are true:
( j)
(i)  n () faster than n for every > 0, that is, Process II for (t) = t is
unstable.
(ii) Let (t) = t H (t) with H (t) C [0, t j ] and H (t) = 0 on [0, t j ]. Assume that

|Yn( j) | C1 n 1 for all n; C1 > 0 and 1 constants.

(8.4.35)

Then, n as n faster than n for every > 0, that is, Process II is

unstable.
( j)

8.5 Slowly Varying (t) with liml (tl+1 /tl ) = (0, 1)

Proof. Substituting (8.4.34) in (8.4.31), we obtain
 1
tjn
n!
( j)

n () =
 n( j) (1).
|()n | t jn

193

(8.4.36)

Invoking the asymptotic equality of (8.4.28) in (8.4.36), we have for all large n

|1|
( j)
1 t j+n

 n( j) (1) for some constant K () > 0. (8.4.37)

n () K ()n
tj
( j)
|1|
Now, t j+n c|1| n q|1| as n and, by Theorem 8.4.4,  n (1) as n
(
j)
faster than n for every > 0. Consequently,  n () as n faster than n
( j)

for every > 0 as well. The assertion about n can now be proved
by using this

1result
1
in (8.4.32) along with (8.4.35) and the fact that |H (t jn )| maxt[0,t j ] |H (t)|
>0
independently of n.


( j)

The purpose of the next theorem is to give as good a bound as possible for |An A|
in Process II. A convergence result can then be obtained by imposing suitable and liberal

n
( j)
( j)
growth conditions on B (n)  and Yn and recalling that Dn {t 1 } = (1)n /
i=0 t j+i .
Theorem 8.4.9 Assume that B(t) C [0, t j ] and dene B (n)  as in (8.4.19). Let (t)
be as in Theorem 8.4.8. Then, for some constant L > 0,



n
(n) 
( j)
( j)
1
1 B 
n
(8.4.38)
|An A| L |Yn |
max t
t j+i .
t[t j+n ,t j ]
n!
i=0
( j)

Note that it is quite reasonable to assume that Yn is bounded as n . The jus( j)

tication for this assumption is that Yn = (n) (t jn )/ (n) (t jn ) for some t jn and t jn

(t j+n , t j ), where (t) = t and (t) = 1/(t) = t /H (t), and that (n) (t)/ (n) (t)
H (0) as t 0+. Indeed, when 1/H (t) is a polynomial in t, we have precisely
( j)
( j)
Dn {t /H (t)} Dn {t }/H (0) as n , as can be shown with the help of Corol( j)
lary 8.4.6, from which Yn H (0) as n . See also Lemma 8.6.5. Next, with the tl
as in (8.4.9), we also have that (maxt[t j+n ,t j ] t 1 ) grows at most like n q|1| as n .
( j)
Thus, the product |Yn |(maxt[t j+n ,t j ] t 1 ) in (8.4.38) grows at most like a power of n as
( j)
n , and, consequently
the main behavior of |An A| as n is determined by


n
(B (n) /n!)
i=0 t j+i . Also note that the strength of (8.4.38) is primarily due to the
n
factor i=0 t j+i that tends to zero as n essentially like (n!)q when the tl satisfy
(8.4.9). Recall that what produces this important factor is Lemma 8.4.5.
8.5 Slowly Varying (t) with liml (tl+1 /tl ) = (0, 1)
As is clear from our results in Section 8.4, both Process I and Process II are unstable
when (t) is slowly changing and the tl satisfy (8.4.9) or, at least in some cases, when
the tl satisfy even the weaker condition liml (tl+1 /tl ) = 1. These results also show
that convergence will take place in Process II nevertheless under rather liberal growth
conditions for B (n) (t). The implication of this is that a required level of accuracy in the

194

8 Analytic Study of GREP(1) : Slowly Varying A(y) F(1)

( j)

numerically computed An may be achieved by computing the a(tl ) with sufciently high
accuracy. This strategy is quite practical and has been used successfully in numerical
calculation of multiple integrals.
( j)
In case the accuracy with which a(t) is computed is xed and the An are required
( j)
to have comparable numerical accuracy, we need to choose the tl such that the An can
be computed stably. When (t) = t H (t) with H (0) = 0 and H (t) continuous in a right
( j)
( j)
neighborhood of t = 0, best results for An and n are obtained by picking {tl } such
that tl 0 as l exponentially in l. There are a few ways to achieve this and each
of them has been used successfully in various problems.
Our rst results with such {tl } given in Theorem 8.5.1 concern Process I, and, like
those of Theorems 8.4.1 and 8.4.3, they are best asymptotically.
Theorem 8.5.1 Let (t) = t H (t) with in general complex and = 0, 1, 2, . . . ,
and H (t) H (0) = 0 as t 0+. Pick the tl such that liml (tl+1 /tl ) = for some
xed (0, 1). Dene
ck = +k1 , k = 1, 2, . . . .

(8.5.1)

Then, for xed n, (8.1.13) and (8.1.14) hold, and we also have


n
cn++1 ci
n+
n+ (t j )t j
A(nj) A
as j ,
1

c
i
i=1

(8.5.2)

where n+ is the rst nonzero i with i n in (8.1.2). This result is valid whether
n  or not. Also,
lim

n


( j)

ni z i =

i=0

n
n


z ci

ni z i ,
1

c
i
i=1
i=0

(8.5.3)

( j)

so that lim j n exists and

lim n( j) =

n

i=0

|ni | =

n

1 + |ci |
i=1

|1 ci |

(8.5.4)

and hence Process I is stable.

Proof. The proof can be achieved by making suitable substitutions in Theorems 3.5.3,
3.5.5, and 3.5.6 of Chapter 3.


Note that Theorem 8.5.1 is valid also when (t) satises (t) h 0 t | log t| as
t 0+ with arbitrary . Obviously, this is a weaker condition than the one imposed on
(t) in the theorem.
Upon comparing Theorem 8.5.1 with Theorem 8.4.1, we realize that the remarks that
follow the proof of Theorem 8.4.1 and that concern the convergence of column sequences
also are valid without any changes under the conditions of Theorem 8.5.1.
So far we do not have results on Process II with (t) and {tl } as in Theorem 8.5.1. We
are able to provide some analysis for the cases in which is real, however. This is the
subject of the next section.

8.5 Slowly Varying (t) with liml (tl+1 /tl ) = (0, 1)

195

We are able to give very strong results on Process II for the case in which {tl } is a
truly geometric sequence. The conditions we impose on (t) in this case are extremely
weak in the sense that (t) = t H (t) with complex in general and H (t) not necessarily
differentiable at t = 0.
Theorem 8.5.2 Let (t) = t H (t) with in general complex and = 0, 1, 2, . . . ,
and H (t) = H (0) + O(t ) as t 0+, with H (0) = 0 and > 0. Pick the tl such that
tl = t0 l , l = 0, 1, . . . , for some (0, 1). Dene ck = +k1 , k = 1, 2, . . . . Then,
for any xed j, Process II is both stable and convergent whether limt0+ a(t) exists or
( j)
not. In particular, we have limn An = A with
A(nj) A = O( n ) as n , for every > 0,

(8.5.5)

( j)

and supn n < with

lim n( j) =


1 + |ci |
i=1

|1 ci |

< .

(8.5.6)

The convergence result of (8.5.5) can be rened as follows: With B(t) C[0, t] for
some t > 0, dene


s1

i
s

s = max |B(t)
i t |/t , s = 0, 1, . . . ,
t[0,t]

(8.5.7)

i=0

or when B(t) C [0, t], dene

s = max (|B (s) (t)|/s!), s = 0, 1, . . . .
t[0,t]

(8.5.8)

If n or n is O(e n ) as n for some > 0 and < 2, then, for any  > 0 such
that +  < 1,


2
(8.5.9)
A(nj) A = O ( + )n /2 as n .

We refer the reader to Sidi [300] for proofs of the results in (8.5.5), (8.5.6), and (8.5.9).
We make the following observations about Theorem 8.5.2. First, note that all the results
in this theorem are independent of , that is, of the details of (t) H (0)t as t 0+.
( j)
Next, (8.5.5) implies that all diagonal sequences {An }n=0 , j = 0, 1, . . . , converge, and
( j)
the error An A tends to 0 as n faster than en for every > 0, that is, the
convergence is superlinear. Under the additional growth condition imposed on n or n ,
2
( j)
we have that An A tends to 0 as n at the rate of en for some > 0. Note
that this condition is very liberal and is satised in most practical situations. It holds,
for example, when n or n are O(( pn)!) as n for some p > 0. Also, it is quite
( j)
interesting that limn n is independent of j, as seen from (8.5.6).
Finally, note that Theorem 8.5.1 pertaining to Process I holds under the conditions
of Theorem 8.5.2 without any changes as liml (tl+1 /tl ) = is obviously satised
because tl+1 /tl = for all l.

196

8 Analytic Study of GREP(1) : Slowly Varying A(y) F(1)

8.6 Slowly Varying (t) with Real and tl+1 /tl (0, 1)

In this section, we consider the convergence and stability properties of Process II when
{tl } is not necessarily a geometric sequence as in Theorem 8.5.2 or liml (tl+1 /tl ) does
not necessarily exist as in Theorem 8.5.1. We are now concerned with the choice
tl+1 /tl , l = 0, 1, . . . , for some xed (0, 1).

(8.6.1)

If liml (tl+1 /tl ) = for some (0, 1), then given  > 0 such that = +  < 1,
there exists an integer L > 0 such that
 < tl+1 /tl < +  for all l L .

(8.6.2)

Thus, if t0 , t1 , . . . , t L1 are chosen appropriately, the sequence {tl } automatically satises (8.6.1). Consequently, the results of this section apply also to the case in which
liml (tl+1 /tl ) = (0, 1).

8.6.1 The Case (t) = t

The case that has been studied most extensively under (8.6.1) is that of (t) = t, and
we treat this case rst. The outcome of this treatment will prove to be very useful in the
study of the general case.
( j)
We start with the stability problem. As in Lemma 8.4.7, we denote the n corre(
j)
(
j)
(
j)
sponding to (t) = t by  n () and its corresponding ni by ni ().
Theorem 8.6.1 With (t) = t and the tl as in (8.6.1), we have for all j and n
 n( j) (1) =

n 
n





1 + i
1 + i
 ( j) 
<
< .
ni (1) $n
1 i
1 i
i=0
i=1
i=1

(8.6.3)

Therefore, both Process I and Process II are stable. Furthermore, for each xed i, we
( j)
have limn ni (1) = 0, with
( j)

ni (1) = O(n

/2+di n

) as n , di a constant.
( j)

(8.6.4)

( j)

Proof. Let tl = t0 l , l = 0, 1, . . . , and denote the ni and n appropriate for (t) = t

( j)
( j)
and the tl , respectively, by ni and  n . By (8.4.26), we have that
( j)

|ni (1)| =


i1

 
n

1 t j+i /t j+k
t /t
1
k=i+1 j+i j+k
 


i1
n
1
1

1 ik
ik 1
k=0
k=i+1

 

i1
n
1
1
( j)
=
= |ni |.

t
1

t
/
t
/
t

1
j+i
j+k
j+i
j+k
k=0
k=i+1
k=0

(8.6.5)

8.6 Slowly Varying (t) with Real and tl+1 /tl (0, 1)
( j)

( j)

197

( j)

Therefore,  n (1)  n . But  n = $n by Theorem 1.4.3. The relation in (8.6.4) is a

consequence of the fact that
( j)

|ni | =


n

(1 ck )

1

ck1 ckni ,

(8.6.6)

1k1 <<kni n

k=1

with ck = k , k = 1, 2, . . . , which in turn follows from

(1 ck ).

n
i=0

( j)

ni z i =

n

k=1 (z

ck )/

( j)

The fact that  n (1) is bounded uniformly both in j and in n was originally proved
by Laurent [158]. The rened bound in (8.6.3) was mentioned without proof in Sidi
[295].
Now that we have proved that Process I is stable, we can apply Theorem 8.1.4 and
( j)
conclude that lim j An = A with
n++1

A(nj) A = O(t j

) as j ,

(8.6.7)

without assuming that B(t) is differentiable in a right neighborhood of t = 0.

Since Process II satises the conditions of the SilvermanToeplitz theorem (Theo( j)
rem 0.3.3), we also have limn An = A. We now turn to the convergence issue for
Process II to provide realistic rates of convergence for it. We start with the following
important lemma due to Bulirsch and Stoer [43]. The proof of this lemma is very lengthy
and difcult and we, therefore, refer the reader to the original paper.
Lemma 8.6.2 With (t) = t and the tl as in (8.6.1), we have for each integer s
{0, 1, . . . , n}
n


( j)
|ni (1)| t s+1
j+i

 
n


t j+k

(8.6.8)

k=ns

i=0

for some constant M > 0 independent of j, n, and s.

This lemma becomes very useful in the proof of the convergence of Process II.
Lemma 8.6.3 Let us pick the tl to satisfy (8.6.1). Then, with j xed,
( j)

|Dn {B(t)}|
( j)

|Dn {t 1 }|

= O( n ) as n , for every > 0.

(8.6.9)

This result can be rened as follows: Dene s exactly as in Theorem 8.5.2. If n =

O(e n ) as n for some > 0 and < 2, then, for any  > 0 such that +  < 1,
( j)

|Dn {B(t)}|
( j)
|Dn {t 1 }|

= O(( + )n

/2

) as n .

(8.6.10)

8 Analytic Study of GREP(1) : Slowly Varying A(y) F(1)

198

Proof. From (8.1.11), we have for each s n

( j)

Q (nj)

|Dn {B(t)}|
( j)

|Dn {t 1 }|

n


( j)

|ni (1)| E s (t j+i ) t j+i ; E s (t) |B(t)

i=0

s1


k t k |.

k=0

(8.6.11)
By (8.1.2), there exist constants s > 0 such that E s (t) s t s when t [0, t] for some
t > 0 and also when t = tl > t. (Note that there are at most nitely many tl > t.) Therefore, (8.6.11) becomes
 

n
n

( j)
,
(8.6.12)
|ni (1)| t s+1

M
t
Q (nj) s
s
j+k
j+i
k=ns

i=0

the last inequality being a consequence of Lemma 8.6.2. The result in (8.6.9) follows

from (8.6.12) once we observe by (8.6.1) that nk=ns t j+k = O(n(s+1) ) as n with
s xed but arbitrary.
To prove the second part, we use the denition of s to rewrite (8.6.11) (with s = n)
in the form
 ( j)
 ( j)
|ni (1)| E n (t j+i ) t j+i + n
|ni (1)| t n+1
(8.6.13)
Q (nj)
j+i .
t j+i >t

t j+i t

k

Since E n (t) |B(t)| + n1

k=0 |k | t , |k | k for each k, and n grows at most like
( j)
n
n 2 /2+di n
for < 2, ni (1) = O(
) as n from (8.6.4), and there are at most
e
nitely many tl > t, we have that the rst summation on the right-hand side of (8.6.13) is
2
O(( + )n /2 ) as n , for any  > 0. Using Lemma 8.6.2, we obtain for the second
summation


n
n
 ( j)

( j)
n+1

|ni (1)| t n+1

(1)|
t

t
n
n
n
j+i ,
ni
j+i
j+i
t j+i t

i=0

which, by (8.6.1), is also O(( + )n

in (8.6.13), we obtain (8.6.10).

/2

i=0

) as n , for any  > 0. Combining the above

The following theorem is a trivial rewording of Lemma 8.6.3.

Theorem 8.6.4 Let (t) = t and pick the tl to satisfy (8.6.1). Then, with j xed,
( j)
limn An = A, and
A(nj) A = O( n ) as n , for every > 0.

(8.6.14)

This result can be rened as follows: Dene s exactly as in Theorem 8.5.2. If n =

O(e n ) as n for some > 0 and < 2, then, for any  > 0 such that +  < 1,
A(nj) A = O(( + )n

/2

) as n .
( j)

(8.6.15)

Theorem 8.6.4 rst implies that all diagonal sequences {An }n=0 converge to A and
( j)
that |An A| 0 as n faster than en for every > 0. It next implies that,

8.6 Slowly Varying (t) with Real and tl+1 /tl (0, 1)

199

( j)

with a suitable and liberal growth rate on the n , it is possible to achieve |An A| 0
2
as n practically like en for some > 0.

8.6.2 The Case (t) = t H (t) with Real and tl+1 /tl (0, 1)
We now come back to the general case in which (t) = t H (t) with real, = 0,
1, 2, . . . , and H (t) H (0) = 0 as t 0+. We assume only that H (t) C[0, t] and


H (t) = 0 when t [0, t] for some t > 0 and that H (t) i=0
h i t i as t 0+, h 0 = 0.


i
Similarly, B(t) C[0, t] and B(t) i=0 i t as t 0+, as before. We do not impose
any differentiability conditions on B(t) or H (t). Finally, unless stated otherwise, we
require the tl to satisfy
tl+1 /tl , l = 0, 1, . . . , for some xed and , 0 < < < 1,
(8.6.16)
instead of (8.6.1) only. Recall from the remark following the statement of Lemma
8.3.2 that the additional condition tl+1 /tl is naturally satised, for example, when
liml (tl+1 /tl ) = (0, 1); cf. also (8.6.2). It also enables us to overcome some problems in the proofs of our main results.
We start with the following lemma that is analogous to Lemma 8.3.2 and Lemma
8.4.2.


gi t +i as t 0+, where g0 = 0 and is real, such that
Lemma 8.6.5 Let g(t) i=0

g(t)t C[0, t] for some t > 0, and pick the tl to satisfy (8.6.16). Then, the following
are true:

form an asymptotic sequence both as

(i) The nonzero members of {Dn {t +i }}i=0
j and as n .
( j)
(ii) Dn {g(t)} has the bona de asymptotic expansion
( j)

Dn( j) {g(t)}

gi Dn( j) {t +i } as j ,

(8.6.17)

i=0

where the asterisk on the summation means that only those terms for which
( j)
Dn {t +i } = 0, that is, for which + i = 0, 1, . . . , n 1, are taken into account.
(iii) When = 0, 1, . . . , n 1, we also have
Dn( j) {g(t)} g0 Dn( j) {t } as n .

(8.6.18)

When < 0, the condition in (8.6.1) is sufcient for (8.6.18) to hold.

Proof. Part (i) follows from Corollary 8.4.6. For the proof of part (ii), we follow the
steps of the proof of part (ii) of Lemma 8.3.2. For arbitrary m, we have g(t) =

m1 +i
+ vm (t)t +m , where |vm (t)| Cm for some constant Cm > 0, whenever
i=0 gi t
t [0, t] and also t = tl > t. (Recall again that there are at most nitely many tl > t.)

8 Analytic Study of GREP(1) : Slowly Varying A(y) F(1)

200
Thus,

Dn( j) {g(t)} =

m1


gi Dn( j) {t +i } + Dn( j) {vm (t)t +m }.

(8.6.19)

i=0

Therefore, we have to show that Dn {vm (t)t +m } = O(Dn {t +m }) as j when

+ m = 0, 1, . . . , n 1.
( j)
( j)
( j) 1
By the fact that ni (1) = cni t 1
j+i /Dn {t }, we have
( j)

|Dn( j) {vm (t)t +m }|

n


( j)

|cni | |vm (t j+i )| t +m

j+i
( j)

i=0

Cm |Dn( j) {t 1 }|

n


+m+1
|ni (1)| t j+i
.
( j)

(8.6.20)

i=0

Now, taking s to be any integer that satises 0 s min{ + m, n}, and applying
Lemma 8.6.2, we obtain

 


n
n
n

( j)
( j)
+m+1
s+1
+ms
|ni (1)| t j+i

|ni (1)| t j+i t j

M
t j+k t +ms
. (8.6.21)
j
i=0

k=ns

i=0

Consequently, under (8.6.1) only,

|Dn( j) {vm (t)t +m }|

MCm |Dn( j) {t 1 }|

 
n

t j+k t j +ms .

(8.6.22)

k=ns


( j)
Recalling that |Dn {t 1 }| = ( nk=0 t j+k )1 and tl+1 tl , and invoking (8.3.9) that is
valid in the present case, we obtain from (8.6.22)
) = O(Dn( j) {t +m }) as j .
Dn( j) {vm (t)t +m } = O(t +mn
j

(8.6.23)

This completes the proof of part (ii).

As for part (iii), we rst note that, by Corollary 8.4.6,
lim

m1


gi Dn( j) {t +i }/Dn( j) {t } = g0 .

i=0

Therefore, the proof will be complete if we show that limn Dn {vm (t)t +m }/
( j)
Dn {t } = 0. By (8.6.22), we have
( j)

|Dn {vm (t)t +m }|

( j)

Tn( j)

( j)

|Dn {t }|

|Dn {t 1 }|
( j)

MCm

 
n


t j+k t +ms
. (8.6.24)
j

( j)

|Dn {t }|

k=ns

By Corollary 8.4.6, again,

|Dn {t 1 }|

( j)

( j)

|Dn {t }|

K1n

1+

max t

t[t j+n ,t j ]


,

(8.6.25)

8.6 Slowly Varying (t) with Real and tl+1 /tl (0, 1)

201

and by (8.6.1),
n


t j+k K 2 t sj t j+n ns ,

(8.6.26)

k=ns

where K 1 and K 2 are some positive constants independent of n. Combining these in

(8.6.24), we have
Tn( j) L Vn( j) n 1+ t j+n ns ; Vn( j) max t 1 ,

(8.6.27)

t[t j+n ,t j ]

;
for some constant L > 0 independent of n. Now, (a) for 1, Vn = t 1
j
( j)
( j)
1
1 n(1+ )
;
while
(c)
for

>
0,
V
=
t

(b) for 1 < < 0, Vn = t 1

n
j+n
j+n
j
( j)
by (8.6.16). Thus, (a) if 1, then Tn = O(n 1+ t j+n ns ) = o(1) as n ; (b) if
( j)
||
1 < < 0, then Tn = O(n 1+ t j+n ns ) = o(1) as n ; and (c) if > 0, then
( j)
Tn = O(n 1+ n(1+) ns ) = o(1) as n , provided we take s sufciently large in
this case, which is possible because m is arbitrary and n tends to innity. This completes
the proof.


( j)

Our rst major result concerns Process I.

Theorem 8.6.6 Let B(t), (t), and {tl } be as in the rst paragraph of this subsection.
( j)
( j)
n+
Then An A satises (8.1.13) and (8.1.14), and hence An A = O((t j )t j ) as
( j)
j . In addition, sup j n < , that is, Process I is stable.
( j)

Proof. The assertions about An A follow by applying Lemma 8.6.5 to B(t) and to
( j)
(t) 1/(t). As for n , we proceed as follows. By (8.4.31) and (8.4.34) and (8.6.16),
we rst have that


t j |1| ( j)
n!
n! n|1| ( j)

(8.6.28)
n (1)
n (1).
 n( j) ()
|()n | t j+n
|()n |
( j)
By Theorem 8.6.1, it therefore follows that sup j  n () < . Next, by Lemma 8.6.5
( j)
again, we have that Yn h 0 as j , and |H (t)|1 is bounded for all t close to 0.
( j)


Combining these facts in (8.4.32), it follows that sup j n < .

As for Process II, we do not have a stability theorem for it under the conditions
( j)
of Theorem 8.6.6. [The upper bound on  n () given in (8.6.28) tends to innity as
n .] However, we do have a strong convergence theorem for Process II.
Theorem 8.6.7 Let B(t), (t), and {tl } be as in the rst paragraph of this subsection.
Then, for any xed j, Process II is convergent whether limt0+ a(t) exists or not. In
( j)
particular, we have limn An = A with
A(nj) A = O( n ) as n , for every > 0.

(8.6.29)

202

8 Analytic Study of GREP(1) : Slowly Varying A(y) F(1)

This result can be rened as follows: Dene s exactly as in Theorem 8.5.2. If n =

O(e n ) as n for some > 0 and < 2, then for any  > 0 such that +  < 1
A(nj) A = O(( + )n

/2

) as n .

(8.6.30)

When > 0, these results are valid under (8.6.1).

( j)

Proof. First, by (8.4.29) and part (iii) of Lemma 8.6.5, we have that Yn H (0) as
n . The proof can now be completed by also invoking (8.4.34) and Lemma 8.6.3
in (8.4.33). We leave the details to the reader.

9
Analytic Study of GREP(1) : Quickly Varying A(y) F(1)

9.1 Introduction
In this chapter, we continue the analytical study of GREP(1) , which we began in the
preceding chapter. We treat those functions A(y) F(1) whose associated (y) vary
quickly as y 0+. Switching to the variable t as we did previously, by (t) varying
quickly as t 0+ we now mean that (t) is of one of the three forms
(a)

(t) = eu(t) h(t)

(b)

(t) = [(t s )] h(t)

(c)

(t) = [(t s )] eu(t) h(t),

(9.1.1)

where
(i) u(t) behaves like
u(t)

u k t ks as t 0+, u 0 = 0, s > 0 integer;

(9.1.2)

k=0

(ii) h(t) behaves like

h(t) h 0 t as t 0+, for some h 0 = 0 and ;

(9.1.3)

(iii) (z) is the Gamma function, > 0, and s is a positive integer; and, nally,
(iv) |(t)| is bounded or grows at worst like a negative power of t as t 0+. The
implications of this are as follows: If (t) is as in (9.1.1) (a), then limt0+ u(t) =
+. If (t) is as in (9.1.1) (c), then s s . No extra conditions are imposed when
(t) is as in (9.1.1) (b). Thus, in case (a), either limt0+ (t) = 0, or (t) is bounded
as t 0+, or it grows like t  when  < 0, whereas in cases (b) and (c), we have
limt0+ (t) = 0 always.
Note also that we have not put any restriction on that may now assume any real or
complex value.


k
As for the function B(t), in Section 9.3, we assume only that B(t)
k=0 k t as
t 0+, without imposing on B(t) any differentiability conditions. In Section 9.4, we
assume that B(t) C [0, t] for some t.
203

204

9 Analytic Study of GREP(1) : Quickly Varying A(y) F(1)

Throughout this chapter the tl are chosen to satisfy

tl cl q and tl tl+1 cpl q1 as l , for some c > 0, p > 0, and q > 0.
(9.1.4)
[This is (8.4.9).]
We also use the notation (t) 1/(t) freely as before.

9.2 Examples of Quickly Varying a(t)

We now present practical examples of functions a(t) that vary quickly.
Example 9.2.1 Let f (x) = e(x) w(x), where A(m) strictly for some positive integer
m and w A( ) strictly for some arbitrary and possibly complex . Then, we know from
Theorem 5.7.3 that
 x
f (t) dt = I [ f ] + x 1m f (x)g(x); g A(0) strictly.
F(x) =
a

This means that F(x) a(t), I [ f ] A, x 1 t, x 1m f (x) (t) with (t) as in

(9.1.1)(9.1.3) and with s = m and = m 1 . [Recall that f B(1) in this case
and that we can replace (t) by x f (x). This changes only.]
Example 9.2.2 Let an be of one of the three forms: (a) an = n w(n), or (b) an =
( )
[(n)] w(n), or (c) an = [(n)] n w(n). Here, w A0 strictly for some arbitrary and
possibly complex , is a negative integer, and is a possibly complex scalar that is
arbitrary in case (c) and that satises | | 1 and = 1 in case (a). Then, we know from
Theorem 6.7.2 that
n

ak = S({ak }) + an g(n); g A(0)
An =
0 strictly.
k=1

This means that An a(t), S({ak }) A, n 1 t, an (t) with (t) as in (9.1.1)

(9.1.3) and with = , s = 1, s = 1, and some . [Recall that {an } b(1) in this case
and that we can replace (t) by nan . This changes only.]
Example 9.2.3 Let an be of one of the three forms: (a) an = e (n) w(n), or (b) an =
(1r/m,m) strictly for some inte[(n)] w(n), or (c) an = [(n)] e(n) w(n). Here, A
0
(
,m)

gers m > 0 and r {0, 1, . . . , m 1}, w A

strictly for some arbitrary and pos0
sibly complex , and = /m with < 0 an integer. Then, we know from Theorem
6.6.6 that
n

(0,m) strictly,
ak = S({ak }) + n an g(n); g A
An =
0
k=1

where = /m for some {0, 1, . . . , m 1}. This means that An a(t), S({ak })
A, n 1/m t, n an (t) with (t) as in (9.1.1)(9.1.3) and with = , s = m,
s = m r and some . [Recall that {an } b (m) in this case and that again we can replace
(t) by nan . Again, this changes only.]

9.3 Analysis of Process I

205

9.3 Analysis of Process I

The rst two theorems that follow concern (t) as in (9.1.1) (a).


k
Theorem 9.3.1 Let (t) be as in (9.1.1) (a), and assume that B(t)
k=0 k t as
t 0+, and pick the constants c, p, and q in (9.1.4) such that q = 1/s and
exp(spu 0 /cs ) = 1. Then
A(nj) A

p n ( + 1)n
n+
n+ (t j )t j j n as j ,
(1 )n

(9.3.1)

where n+ is the rst nonzero i with i n. In addition, Process I is stable, and we

have


1 + | | n
as j .
(9.3.2)
n( j)
|1 |
( j)

Proof. First, Lemma 8.4.2 applies to B(t) and, therefore, Dn {B(t)} satises (8.4.3),
namely,
Dn( j) {B(t)}

( + 1)n

n+ t j as j .
n!

(9.3.3)

We next recall (8.4.14), from which we have

t rj t rj+i = ir pj 1 (cj q ) + o( j qr 1 ) as j , for every r.
r

(9.3.4)

Finally, we recall (8.4.15), namely,

( j)

cni (1)i


 
j n
1 n
as j .
n! i
pt j

(9.3.5)

All these are valid for any q > 0. Now, by (9.3.4), and after a delicate analysis, we have
with q = 1/s
s

s
1
1/s
u(t j ) u(t j+i ) u 0 (t s
) u 0 = ispu 0 cs as j .
j t j+i ) ispj (cj

Consequently,
exp[u(t j+i )] i exp[u(t j )] as j ,

(9.3.6)

from which we have

Dn( j) {1/(t)} =

n


cni eu(t j+i ) / h(t j+i )

( j)

i=0

n!
1

n!




j
pt j
j
pt j

n 
n
n

   u(t j )
n i e
(1)
as j

i
h 0 t j
i=0
i

(1 )n (t j ) as j .

(9.3.7)

9 Analytic Study of GREP(1) : Quickly Varying A(y) F(1)

206
Similarly,
n

i=0

( j)

|cni |/|(t j+i )| =

n


|cni | |eu(t j+i ) |/|h(t j+i )|

( j)

i=0

n 
 u(t j )
n  
n
|
i |e
| |
as j

i
|h
t
|
0 j
i=0


j n
1

(1 + | |)n |(t j )| as j .
n! pt j
1

n!

j
pt j

(9.3.8)

Combining (9.3.3) and (9.3.7) in (8.1.3), (9.3.1) follows, and combining (9.3.7) and
(9.3.8) in (8.1.4), (9.3.2) follows.


A lot can be learned from the analysis of GREP(1) for Process I when GREP(1) is being
applied as in Theorem 9.3.1. From (9.3.2), it is clear that Process I will be increasingly
stable as (as a complex number) gets farther from 1. Recall that = exp(spu 0 /cs ),
that is, is a function of both (t) and {tl }. Now, the behavior of (t) is determined
by the given a(t) and the user can do nothing about it. The tl , however, are chosen
by the user. Thus, can be controlled effectively by picking the tl as in (9.1.4) with
appropriate c. For example, if u 0 is purely imaginary and exp(spu 0 ) is very close to
1 (note that | exp(spu 0 )| = 1 in this case), then by picking c sufciently small we can
s
cause = [exp(spu 0 )]c to be sufciently far from 1, even though | | = 1. It is also
important to observe from (9.3.1) that the term (1 )n also appears as a factor in the
( j)
dominant behavior of An A. Thus, by improving the stability of GREP(1) , we are also
( j)
improving the accuracy of the An .
In the next theorem, we show that, by xing the value of q differently, we can cause
the behavior of Process I to change completely.
Theorem 9.3.2 Let (t) and B(t) be as in Theorem 9.3.1, and let u(t) = v(t) + iw(t),


ks
as t 0+, v0 = 0. Choose q > 1/s
with v(t) and w(t) real and v(t)
k=0 vk t
in (9.1.4). Then
n+ n

A(nj) A (1)n p n ( + 1)n n+ (t j+n )t j

as j ,

(9.3.9)

where n+ is the rst nonzero i with i n. In addition, Process I is stable, and we

have
n( j) 1 as j .

(9.3.10)

Proof. We start by noting that (9.3.3)(9.3.5) are valid with any q > 0 in (9.1.4). Next,
we note that by (9.1.2) and the condition on v(t) imposed now, we have
s

v(t j+1 ) v(t j ) spj 1 (cj q ) v0 = v0 (sp/cs ) j qs1 as j .

9.4 Analysis of Process II

207

Consequently,


 (t j ) 
s qs1


+ o( j qs1 )] as j .
 (t )  exp[v(t j+1 ) v(t j )] = exp[v0 (sp/c ) j
j+1
(9.3.11)
Since limt0+ u(t) = limt0+ v(t) = , we must have v0 < 0, and since q > 1/s,
we have qs 1 > 0, so that lim j |(t j )/(t j+1 )| = 0. This and (9.3.5) imply that


n

j n
(1)n
( j)
( j)
Dn( j) {1/(t)} =
cni (t j+i ) cnn
(t j+n )
(t j+n ) as j ,
n!
pt j
i=0
(9.3.12)
and
n


( j)

( j)
|cni | |(t j+i )| |cnn
| |(t j+n )|

i=0

1
n!

j
pt j

n
|(t j+n )| as j . (9.3.13)

The proof can be completed as before.

The technique we have employed in proving Theorem 9.3.2 can be used to treat the
cases in which (t) is as in (9.1.1) (b) and (c).
Theorem 9.3.3 Let (t) be as in (9.1.1) (b) or (c). Then, (9.3.9) and (9.3.10) hold if
q > 1/s .
Proof. First, (9.3.3) is valid as before. Next, after some lengthy manipulation of the
Stirling formula for the Gamma function, we can show that, in the cases we are considering, lim j |(t j )/(t j+1 )| = 0, from which (9.3.12) and (9.3.13) hold. The results
in (9.3.9) and (9.3.10) now follow. We leave the details to the reader.


Note the difference between the theorems of this section and those of Chapter 8
pertaining to process I when GREP(1) is applied to slowly varying sequences. Whereas in
( j)
( j)
Chapter 8 An A = O((t j )t nj ) as j , in this chapter An A = O((t j )t nj j n )
( j)
and An A = O((t j+n )t nj j n ) as j . This suggests that it is easier to accelerate
the convergence of quickly varying a(t).

9.4 Analysis of Process II

As mentioned before, the analysis of Process II turns out to be much more complicated
( j)
than that of Process I. What complicates things most appears to be the term Dn {1/(t)},
which, even for simple (t) such as (t) = t ea/t , turns out to be extremely difcult to
study. Because of this, we restrict our attention to the single case in which
sgn (tl ) (1)l ei as l , for some real ,
where sgn ei arg .

(9.4.1)

9 Analytic Study of GREP(1) : Quickly Varying A(y) F(1)

208

The results of this section are rened versions of corresponding results in Sidi [288].
The reader may be wondering whether there exist sequences {tl } that satisfy (9.1.4)
and ensure the validity of (9.4.1) at the same time. The answer to this question is in the
afrmative when limt0+ |!u(t)| = . We return to this point in the next section.
We start with the following simple lemma.
Lemma 9.4.1 Let a1 , . . . , an be real positive and let 1 , . . . , n be in [ ,  ] where
 and  are real numbers that satisfy 0   < /2. Then




n
 
 n
ik 
  )


a
e
cos(
a
k
k > 0.


k=1

k=1

Proof. We have
 n
2 
2  
2
n
n


ik 

ak e  =
ak cos k +
ak sin k

k=1

k=1

k=1

n 
n


ak al cos(k l )

k=1 l=1


n 
n



2
n
ak al cos(  ) =
ak cos(  )

k=1 l=1

k=1

The result now follows.

We address stability rst.

Theorem 9.4.2 Let (t) and {tl } be as described in Section 9.1 and assume that (9.4.1)
( j)
holds. Then, for each xed i, there holds limn ni = 0. Specically,
ni = O(en ) as n , for every > 0.
( j)

(9.4.2)

Furthermore, Process II is stable and we have

n( j) 1 as n .

(9.4.3)

( j)

If sgn [(t j+1 )/(t j )] = 1 for j J , then n = 1 for j J as well.

Proof. Let sgn (tl ) = (1)l eil . By (9.4.1), liml l = . Therefore, for arbitrary
(0, /4), there exists a positive integer M such that l ( , + ) when l M.
Let us dene
&1 =

M
j1


( j)

cni (t j+i ) and &2 =

n


( j)

cni (t j+i )

(9.4.4)

i=M j

i=0

and
$1 =

M
j1

i=0

( j)

|cni | |(t j+i )| and $2 =

n

i=M j

( j)

|cni | |(t j+i )|.

(9.4.5)

9.4 Analysis of Process II

209

Obviously, |&2 | $2 . But we also have from Lemma 9.4.1 that 0 < K $2 |&2 |,

K = cos 2 > 0. Next, let us dene ik = |t j+i t j+k |. Then

( j)
eni

 n1

 ( j)
 nk  (t j+n ) 
 cni (t j+i ) 


.

 ( j)
=
ik  (t j+i ) 
cnn (t j+n )
k=0

(9.4.6)

k=i

Now, because liml tl = 0, given  > 0, there exists a positive integer N for which
pq <  if p, q N . Without loss of generality, we pick  < i,i+1 and N > i. Also,
because {tl } 0 monotonically, we have ik i,i+1 for all k i + 1. Combining all
this in (9.4.6), we obtain with n > N


nN 
 N
1
 (t j+n ) 

nk
( j)


(9.4.7)
eni <
 (t ) .
ik
i,i+1
j+i
k=0
k=i

Since i is xed and (t j+n ) = O(tn ) = O(n q ) as n , and since  is arbitrary,

there holds
eni = O(en ) as n , for every > 0.
( j)

(9.4.8)

Substituting (9.4.8) in
M j1
$1
$1
|&1 |
1  ( j)

e ,
=
( j)
|&2 |
K $2
K i=0 ni
K |cnn | |(t j+n )|

(9.4.9)

we have limn &1 /&2 = 0 = limn $1 / $2 . With the help of this, we obtain for all
large n that
( j)

( j)

|ni | =

( j)

( j)

|cni /(t j+i )|

eni
|cni /(t j+i )|

, (9.4.10)
|&1 + &2 |
|&2 |(1 |&1 /&2 |)
K (1 |&1 /&2 |)

and this, along with (9.4.8), results in (9.4.2). To prove (9.4.3), we start with
1 n( j) =

1 1 + $ 1 / $2
$1 + $ 2

,
|&1 + &2 |
K 1 |&1 /&2 |

(9.4.11)

( j)

from which we obtain 1 lim supn n 1/K . Since can be picked arbitrarily
( j)
close to 0, K is arbitrarily close to 1. As a result, lim supn n = 1. In exactly the
( j)
( j)
same way, lim infn n = 1. Therefore, limn n = 1, proving (9.4.3). The last
( j)


assertion follows from the fact that ni > 0 for 0 i n when j J .
We close with the following convergence theorem that is a considerably rened version
of Theorem 4.4.3.
Theorem 9.4.3 Let (t) and {tl } be as in Section 9.1, and let B(t) C [0, t] for some
( j)
t > 0. Then, limn An = A whether A is the limit or antilimit of a(t) as t 0+. We
actually have
A(nj) A = O(n ) as n , for every > 0.

(9.4.12)

210

9 Analytic Study of GREP(1) : Quickly Varying A(y) F(1)

Proof. Following the proof of Theorem 4.4.3, we start with

A(nj) A =

n


( j)

ni (t j+i )[B(t j+i ) vn1 (t j+i )],

(9.4.13)

i=0

where vm (t) = m
k=0 f k Tk (2t/t 1) is the mth partial sum of the Chebyshev series of

B(t) over [0, t ]. Denoting Vn (t) = B(t) vn1 (t), we can write (9.4.13) in the form
 ( j)
 ( j)
A(nj) A =
ni (t j+i )Vn (t j+i ) +
ni (t j+i )Vn (t j+i ) (9.4.14)
t j+i >t

t j+i t

(If t j t, then the rst summation is empty.) By the assumption that B(t) C [0, t],
we have that maxt[0,t] |Vn (t)| = O(n ) as n for every > 0. As a result, in the
second summation in (9.4.14) we have maxt j+i t |Vn (t j+i )| = O(n ) as n for every > 0. Next, by (9.1.4) and (t) = O(t ) as t 0+, we have that max0in |(t j+i )|
is either bounded independently of n or grows at worst like n q (when  < 0). Next,


( j)
( j)
t j+i t |ni | n 1 as n , as we showed in the preceding theorem. Combining
all this, we have that
 ( j)
ni (t j+i )Vn (t j+i ) = O(n ) as n , for every > 0. (9.4.15)
t j+i t

As for the rst summation (assuming it is not empty), we rst note that the number of
( j)
terms in it is nite, and each of the ni there satises (9.4.2). The (t j+i ) there are
independent of n. As for Vn (t j+i ), we have
max |Vn (t j+i )| max |B(t j+i )| +

t j+i >t

t j+i >t

n1


| f k | |Tk (2t j /t 1)|.

(9.4.16)

k=0

Now, by B(t) C [0, t], f n = O(n ) as n for every > 0, and when z 
[1, 1], Tn (z) = O(en ) as n for some > 0 that depends on z. Therefore,
 ( j)
ni (t j+i )Vn (t j+i ) = O(en ) as n for every > 0. (9.4.17)
t j+i >t

Combining (9.4.15) and (9.4.17) in (9.4.14), the result in (9.4.12) follows.

9.5 Can {tl } Satisfy (9.1.4) and (9.4.1) Simultaneously?

We return to the question we raised at the beginning of the preceding section whether
the tl can satisfy (9.1.4) and (9.4.1) simultaneously. To answer this, we start with the


km
as t 0+, for some w0 = 0 and integer m s,
fact that !u(t) = w(t)
k=0 wk t

or of cos w(t)

or of a
and pick the tl to be consecutive zeros in (0, br ] of sin w(t)


km
w
t
.
Without
loss
of
generality,
linear combination of them, where w(t)

= m1
k
k=0
we assume that w0 > 0.
0 ) for some 0 . This
Lemma 9.5.1 Let t0 be the largest zero in (0, br ) of sin(w(t)
0 = 0 for some integer 0 . Next,
means that t0 is a solution of the equation w(t)

9.5 Can {tl } Satisfy (9.1.4) and (9.4.1) Simultaneously?

211

let tl be the smallest positive solution of the equation w(t)

0 = (0 + l) for each
l 1. Then, tl+1 < tl , l = 0, 1, . . . , and tl has the (convergent) expansion
tl =

ai l (i+1)/m for all large l; a0 = (w0 / )1/m > 0,

(9.5.1)

i=0

and hence
tl tl+1 =


i=0

a i l 1(i+1)/m for all large l; a 0 =

a0
,
m

(9.5.2)

so that
tl a0l 1/m and tl tl+1 a 0l 11/m as l .

(9.5.3)

Proof. We start by observing that w(t)

w0 t m and hence w(t)

+ as t 0+,
as a result of which the equation w(t)
0 = , for all sufciently large , has a
unique solution t( ) that is positive and satises t( ) (w0 / )1/m as . Therefore, being a solution of w(t)
0 = (0 + l), tl is unique for all large l and satises
tl (w0 / )1/m l 1/m as l . Letting  = l 1/m and substituting tl =  in this equa

k
tion, we see that satises the polynomial equation m
k=0 ck = 0, where cm = 1
k
m
and ck = wk  /[ + (0 + 0 ) ], k = 0, 1, . . . , m 1. Note that the coefcients
c0 , . . . , cm1 are analytic functions of  about  = 0. Therefore, is an analytic function of  about  = 0, and, for all  sufciently close to zero, there exists a convergent


ai  i . The ai can be obtained by substituting this expanexpansion of the form = i=0

m
k
sion in k=0 ck = 0 and equating the coefcients of the powers  i to zero for each i.
This proves (9.5.1). We can obtain (9.5.2) as a direct consequence of (9.5.1). The rest is
immediate.


Clearly, the tl constructed as in Lemma 9.5.1 satisfy (9.1.4) with c = (w0 / )1/m and
p = q = 1/m. They also satisfy (9.4.1), because exp[u(tl )] (1)l+0 ei(wm +0 ) eu(tl )
as l . As a result, when is real, (t) satises (9.4.1) with these tl . We leave the
details to the reader.

10
Efcient Use of GREP(1) : Applications to the D (1) -, d (1) -,
and d(m) -Transformations

10.1 Introduction
In the preceding two chapters, we presented a detailed analysis of the convergence and
stability properties of GREP(1) . In this analysis, we considered all possible forms of (t)
that may arise from innite-range integrals of functions in B(1) and innite series whose
terms form sequences in b(1) and b (m) . We also considered various forms of {tl } that
have been used in applications. In this chapter, we discuss the practical implications of
the results of Chapters 8 and 9 and derive operational conclusions about how the D (1) -,
d (1) -, and d(m) -transformations should be used to obtain the best possible outcome in
different situations involving slowly or quickly varying a(t). It is worth noting again that
the conclusions we derive here and that result from our analysis of GREP(1) appear to
be valid in many situations involving GREP(m) with m > 1 as well.
As is clear from Chapters 8 and 9, GREP(1) behaves in completely different ways
depending on whether (t) varies slowly or quickly as t 0+. This implies that different
strategies are needed for these two classes of (t). In the next two sections, we dwell
on this issue and describe the possible strategies pertinent to the D (1) -, d (1) -, and d(m) transformations.
Finally, the conclusions that we draw in this chapter concerning the d (1) - and d(m) transformations are relevant for other sequence transformations, as will become clear
later in this book.

10.2 Slowly Varying a(t)

We recall that a(t) is slowly varying when (t) i=0
h i t +i as t 0+, h 0 = 0, and
may be complex such that = 0, 1, 2, . . . . We also advise the reader to review
the examples of slowly varying a(t) in Section 8.2.
In Chapter 8, we discussed the application of GREP(1) to slowly varying a(t) with two
major choices of {tl }: (i) liml (tl+1 /tl ) = 1 and (ii) tl+1 /tl , l = 0, 1, . . . ,
for 0 < < 1. We now consider each of these choices separately.
10.2.1 Treatment of the Choice liml (tl+1 /tl ) = 1
The various results concerning the case liml (tl+1 /tl ) = 1 show that numerical insta( j)
bilities occur in the computation of the An in nite-precision arithmetic; that is, the
212

10.2 Slowly Varying a(t)

213

( j)

precision of the computed An is less than the precision with which the a(tl ) are computed, and it deteriorates with increasing j and/or n. The easiest way to remedy this
problem is to increase (e.g., double) the accuracy of the nite-precision arithmetic that
is used, without changing {tl }.
When we are not able to increase the accuracy of our nite-precision arithmetic, we
can deal with the problem by changing {tl } suitably. For example, when {tl } is chosen to
satisfy (8.4.9), namely,
tl cl q and tl tl+1 cpl q1 as l , for some c > 0, p > 0, and q > 0,
(10.2.1)
( j)

( j)

then, by Theorem 8.4.3, we can increase p to make n smaller because n is proportional to p n as j . Now, the easiest way of generating such {tl } is by taking
tl = c/(l + )q with some > 0. In this case p = q, as can easily be shown; hence, we
( j)
increase q to make n smaller for xed n and increasing j, even though we still have
( j)
( j)
lim j n = . Numerical experience suggests that, by increasing q, we make n
( j)
smaller also for xed j and increasing n, even though limn n = , at least in the
cases described in Theorems 8.4.4 and 8.4.8.

In applying the D (1) -transformation to the integral 0 f (t) dt in Example 8.2.1 with
this strategy, we can choose the xl according to xl = (l + )q /c with arbitrary c > 0,
q 1, and > 0 without any problem since the variable t is continuous in this case.
With this choice, we have p = q.


When we apply the d (1) -transformation to the series
k=1 ak in Example 8.2.2, however, we have to remember that t now is discrete and takes on the values 1, 1/2, 1/3, . . . ,
only, so that tl = 1/Rl , with {Rl } being an increasing sequence of positive integers. Sim

ilarly, when we apply the d(m) -transformation to the series
k=1 ak in Example 8.2.3,
1/m
t takes on the discrete values 1, 1/21/m , 1/31/m , . . . , so that tl = 1/Rl , with {Rl }
being again an increasing sequence of positive integers. The obvious question then is
whether we can pick {Rl } such that the corresponding {tl } satises (10.2.1). We can, of
course, maintain (10.2.1) with Rl = (l + 1)r , where and r are both positive integers.
But this causes Rl to increase very rapidly when r = 2, 3, . . . , thus increasing the cost
of extrapolation considerably. In view of this, we may want to have Rl increase like l r
with smaller (hence noninteger) values of r (1, 2), for example, if this is possible at
all. To enable this in both applications, we propose to choose the Rl as follows:
pick > 0, r 1, and > 0, and the integer R0 1, and set

Rl1 + 1 if (l + )r  Rl1 ,
l = 1, 2, . . . .
Rl =
(l + )r  otherwise,

(10.2.2)

Note that , , and r need not be integers now.

Concerning these Rl , we have the following results.
Lemma 10.2.1 Let {Rl } be as in (10.2.2), with arbitrary > 0 when r > 1 and with
1 when r = 1. Then, Rl = (l + )r  for all sufciently large l, from which it
follows that {Rl } is an increasing sequence and that Rl l r as l . In addition,
Rl = l r + r l r 1 + o(l r 1 ) as l , if r > 1,

(10.2.3)

214

10 Efcient Use of GREP(1)

and
Rl = l + R0 , l = 1, 2, . . . , if r = 1 and = 1, 2, . . . .

(10.2.4)

Proof. That {Rl } is an increasing sequence and Rl = (l + )r  for all sufciently large
l is obvious. Next, making use of the fact that x 1 < x x, we can write
(l + )r 1 < Rl (l + )r .
Dividing these inequalities by l r and taking the limit as l , we obtain
liml [Rl /(l r )] = 1, from which Rl l r as l follows. To prove (10.2.3), we
proceed as follows: First, we have
(l + )r = l r + r l r 1 + l ; l

r (r 1) 2
r 2
(1 + ) l r 2 for some (0, /l).
2

Note that r l r 1 as l , and that l > 0 for l 1. Next, from

l r + r l r 1 + l 1 < Rl l r + r l r 1 + l .
we obtain |Rl (l r + r l r 1 )| < l + 1, which implies (10.2.3). Finally, the result
in (10.2.4) is a trivial consequence of the fact that l is an integer under the conditions
imposed on and r there.


The following lemma is a direct consequence of Lemma 10.2.1. We leave its proof to
the reader.
Lemma 10.2.2 Let {Rl } be as in (10.2.2) with r > 1, or with r = 1 and = 1, 2, . . . .
Then, the following assertions hold:
(i) If tl = 1/Rl , then (10.2.1) is satised with c = 1/ and p = q = r .
1/m
where m is a positive integer, then (10.2.1) is satised with c =
(ii) If tl = 1/Rl
1/m
and p = q = r/m.
(1/)
The fact that the tl in Lemma 10.2.2 satisfy (10.2.1) guarantees that Theorems 8.4.1
and 8.4.3 hold for the d (1) - and d(m) -transformations as these are applied to
Example 8.2.2 and Example 8.2.3, respectively.
Note that the case in which Rl is as in (10.2.2) but with r = 1 and 1 not an integer
is not covered in Lemmas 10.2.1 and 10.2.2. In this case, we have instead of (10.2.4)
Rl = l + l , l = 0, 1, . . . ; 1 l .

(10.2.5)

As a result, we also have instead of (10.2.1)

tl cl q as l and K 1l q1 tl tl+1 K 2l q1 for some K 1 , K 2 > 0,
(10.2.6)
1/m

with q = 1 when tl = 1/Rl and with q = 1/m when tl = 1/Rl . In this case too, Theorem 8.4.1 holds when the d (1) - and d(m) -transformations are applied to Example 8.2.2
and Example 8.2.3, respectively. As for Theorem 8.4.3, with the exception of (8.4.17),

10.2 Slowly Varying a(t)

215

its results remain unchanged, and (8.4.17) now reads

G n1 j n n( j) G n2 j n for some G n1 , G n2 > 0, n xed.

(10.2.7)

All this can be shown by observing that Lemma 8.4.2 remains unchanged and that
( j)
L n1 ( j/t j )n |cni | L n2 ( j/t j )n for some L n1 , L n2 > 0, which follows from (10.2.6).
We leave the details to the interested reader.
From the preceding discussion, it follows that both the d (1) - and the d(m) transformations can be made effective by picking the Rl as in (10.2.2) with a suitable
and moderate value of r . In practice, we can start with r = 1 and increase it gradually if
( j)
needed. This strategy enables us to increase the accuracy of An for j or n large and also
( j)
improve its numerical stability, because it causes the corresponding n to decrease. That
( j)
is, by increasing r we can achieve more accuracy in the computed An , even when we
( j)
are limited to a xed precision in our arithmetic. Now, the computation of An involves


the terms ak , 1 k R j+n , of k=1 ak as it is dened in terms of the partial sums
A Rl , j l j + n. Because Rl increases like the power l r , we see that by increasing r
gradually, and not necessarily through integer values, we are able to increase the number
( j)
of the terms ak used for computing An gradually as well. Obviously, this is an advantage
offered by the choice of the Rl as in (10.2.2), with r not necessarily an integer.
( j)
( j)
Finally, again from Theorem 8.4.3, both n and |An A| are inversely proportional to |()n |. This suggests that it is easier to extrapolate when |()n | is large, as
( j)
( j)
this causes both n and |An A| to become small. One practical situation in which
this becomes relevant is that of small || but large |!|. Here, the larger |!|, the
( j)
better the convergence and stability properties of An for j , despite the fact that
( j)
+n
) as j for every value of !. Thus, extrapolation is easier
An A = O((t j )t j
when |!| is large. (Again, numerical experience suggests that this is so both for Process I

and for process II.) Consequently, in applying the D (1) -transformation to 0 f (t) dt in
q
Example 8.2.1 with large |! |, it becomes sufcient to use xl = (l + ) with a low
value of q; e.g., q = 1. Similarly, in applying the d (1) -transformation in Example 8.2.2
with large |! |, it becomes sufcient to choose Rl as in (10.2.2) with a low value of r ; e.g.,
r = 1 or slightly larger. The same can be achieved in applying the d(m) -transformation
in Example 8.2.3, taking r/m = 1 or slightly larger.
10.2.2 Treatment of the Choice 0 < tl+1 /tl < 1
( j)

Our results concerning the case 0 < tl+1 /tl < 1 for all l show that the An can
be computed stably in nite-precision arithmetic with such a choice of {tl }. Of course,
t0 l tl t0 l , l = 0, 1, . . . , and, therefore,
 tl 0 as l exponentially in l.
(1)
In applying the D -transformation to 0 f (t) dt, we can choose the xl according to
xl = /l for some > 0 and (0, 1). With this choice and by tl = 1/xl , we have
liml (tl+1 /tl ) = , and Theorems 8.5.1 and 8.5.2 apply. As is clear from (8.5.4) and
( j)
(8.5.6), n , despite its boundedness both as j and as n , may become large
if some of the ci there are very close to 0, 1, 2, . . . . Suppose, for instance, that 0.
Then, c1 = is close to 1 if we choose close to 1. In this case, we can cause c1 to
separate from 1 by making smaller, whether is complex or not.
In applying the d (1) - and d(m) -transformations with the present choice of {tl }, we
should again remember that t is discrete and tl = 1/Rl for the d (1) -transformation and

216

10 Efcient Use of GREP(1)

1/m
tl = 1/Rl for the d(m) -transformation, and {Rl } is an increasing sequence of positive
integers. Thus, the requirement that tl 0 exponentially in l forces that Rl ex

ponentially in l. This, in turn, implies that the number of terms of the series
k=1 ak
( j)
required for computing An , namely, the integer R j+n , grows exponentially with j + n.
To keep this growth to a reasonable and economical level, we should aim at achieving
Rl = O( l ) as l for some reasonable > 1 that is not necessarily an integer. The
following choice, which is essentially due to Ford and Sidi [87, Appendix B], has proved
very useful:

pick the scalar > 1 and the integer R0 1, and set


Rl1 + 1 if  Rl1  = Rl1 ,
Rl =
l = 1, 2, . . . .
 Rl1  otherwise,

(10.2.8)

(The Rl given in [87] are slightly different but have the same asymptotic properties,
which is the most important aspect.)
The next two lemmas, whose proofs we leave to the reader, are analogous to Lemmas
10.2.1 and 10.2.2. Again, the fact that x 1 < x x becomes useful in part of the
proof.
Lemma 10.2.3 Let {Rl } be as in (10.2.8). Then, Rl =  Rl1  for all sufciently large l,
from which it follows that g l Rl l for some g 1. Thus, {Rl } is an exponentially
increasing sequence of integers that satises liml (Rl+1 /Rl ) = .
Lemma 10.2.4 Let {Rl } be as in (10.2.8). Then, the following assertions hold:
(i) If tl = 1/Rl , then liml (tl+1 /tl ) = with = 1 (0, 1).
1/m
(ii) If tl = 1/Rl , where m is a positive integer, then liml (tl+1 /tl ) = with =
1/m (0, 1).
The fact that the tl in Lemma 10.2.4 satisfy liml (tl+1 /tl ) = with (0, 1)
guarantees that Theorems 8.5.1, 8.5.2, 8.6.1, 8.6.4, 8.6.6, and 8.6.7 hold for the d (1) and d(m) -transformations, as these are applied to Examples 8.2.2 and 8.2.3, respectively.
( j)
Again, in case n is large, we can make it smaller by decreasing (equivalently, by
increasing ).
Before closing this section, we recall that the d (m) - and d(m) -transformations, by their
denitions, are applied to subsequences {A Rl } of {Al }. This amounts to sampling the
sequence {Al }. Let us consider now the choice of the Rl as in (10.2.8). Then the sequence
{Rl } grows as a geometric progression. On the basis of this, we refer to this choice of
the Rl as the geometric progression sampling and denote it GPS for short.

10.3 Quickly Varying a(t)

We recall that a(t) is quickly varying when (t) is of one of three forms: (a)

(t) = eu(t) h(t), or (b) (t) = [(t s )] h(t), or (c) (t) = [(t s )] eu(t) h(t), where


h(t) h 0 t as t 0+ for some arbitrary and possibly complex , u(t) i=0
u i t is as
t 0+ for some positive integer s, (z) is the Gamma function, s is a positive integer,

10.3 Quickly Varying a(t)

217

> 0, and s s in case (c). In addition, (t) may be oscillatory and/or decaying as
t 0+ and |(t)| = O(t  ) as t 0+ at worst. At this point, we advise the reader to
review the examples of quickly varying a(t) given in Section 9.2.
In Chapter 9, we discussed the application of GREP(1) to quickly varying a(t) with
choices of {tl } that satisfy (10.2.1) only and were able to prove that both convergence
and stability prevail with such {tl }. Numerical experience shows that other choices of
{tl } are not necessarily more effective.

We rst look at the application of the D (1) -transformation to the integral 0 f (t) dt in
Example 9.2.1. The best strategy appears to be the one that enables the conditions of Theorems 9.4.2 and 9.4.3 to be satised. In this strategy, we choose the xl in accordance with


wi x mi as x , w0 > 0, in Example 9.2.1,
Lemma 9.5.1. Thus, if !(x) i=0

m1
we set w(x)

= i=0 wi x mi and choose x0 to be the smallest zero greater than a

of sin(w(x)

0 ) for some 0 . Thus, x0 is a solution of the polynomial equation

w(x)

0 = 0 for some integer 0 . Once x0 has been found, xl for l = 1, 2, . . . ,

is determined as the largest root of the polynomial equation w(x)

0 = (0 + l) .

is a polynomial; it is easiest when m = 1

Determination of {xl } is an easy task since w(x)
or m = 2. If we do not want to bother with extracting w(x)

from (x), then we can apply

the preceding strategy directly to ! (x) instead of w(x).

All this forms the basis of the

W -transformation for oscillatory innite-range integrals that we introduce later in this
book.
Let us consider the application of the d (1) - and d(m) -transformations to the series


k=1 ak in Examples 9.2.2 and 9.2.3, respectively. We again recall that tl = 1/Rl for
1/m
for the d(m) -transformation, where {Rl } is an
the d (1) -transformation and tl = 1/Rl
increasing sequence of integers. We choose the Rl as in (10.2.2), because we already
know from Lemma 10.2.2 that, with this choice of {Rl }, tl satises (10.2.1) when r > 1,
or when r = 1 but an are integers there. (When r = 1 but or is not an integer, we
have only tl cl q as l , as mentioned before.) We have only to x the parameter
q as described in Theorems 9.3.19.3.3.
Finally, we note that when Rl are as in (10.2.2) with r = 1, the sequence {Rl } grows
as an arithmetic progression. On the basis of this, we refer to this choice of the Rl
as the arithmetic progression sampling and denote it APS for short. We make use of
APS in the treatment of power series, Fourier series, and generalized Fourier series in
Chapters 12 and 13.
APS with integer was originally suggested in Levin and Sidi [165] and was incorporated in the computer program of Ford and Sidi [87]. It was studied rigorously in Sidi
[294], where it was shown when and how to use it most efciently.

11
Reduction of the D-Transformation for Oscillatory
D-,
W -,
Innite-Range Integrals: The D-,
and mW -Transformations

11.1 Reduction of GREP for Oscillatory A(y)

Let us recall that, when A(y) F(m) for some m, we have
A(y) A +

m


k (y)

k=1

ki y irk as y 0+,

(11.1.1)

i=0

where A is the limit or antilimit of A(y) as y 0+ and k (y) are known shape functions
that contain the asymptotic behavior of A(y) as y 0+. Consider the approximations
(m)
as the latter is applied to
A(m,0)
(,... ,) C , = 1, 2, . . . , that are produced by GREP

A(y). We consider the sequence {C }=1 , because it has excellent convergence properties.
Now C is dened via the linear system
A(yl ) = C +

m

k=1

k (yl )

1


ki ylirk , l = 0, 1, . . . , m,

(11.1.2)

i=0

and is, heuristically, the result of eliminating the m terms k (y)y irk , i = 0, 1, . . . ,
1, k = 1, . . . , m, from the asymptotic expansion of A(y) given in (11.1.1). Thus,
the number of the A(yl ) needed to achieve this elimination process is m + 1.
From this discussion, we conclude that, the smaller the value of m, the cheaper the
extrapolation process. [By cheaper we mean that the number of function values A(yl )
needed to eliminate terms from each k (y) is smaller.]
It turns out that, for functions A(y) F(m) that oscillate an innite number of times as
y 0+, by choosing {yl } judiciously, we are able to reduce GREP, which means that
we are able to use GREP(q) with suitable q < m to approximate A, the limit or antilimit
of A(y) as y 0+, thus saving a lot in the computation of A(y). This is done as follows:
As A(y) oscillates an innite number of times as y 0+, we have that at least
one of the form factors k (y) vanishes at an innite number of points y l , such that
y 0 > y 1 > > 0 and liml y l = 0. Suppose exactly m q of the k (y) vanish on
the set Y = { y 0 , y 1 , . . . }. Renaming the remaining q shape functions k (y) if necessary,
we have that
A(y) = A +

q


k (y)k (y), y Y = { y 0 , y 1 , . . . }.

(11.1.3)

k=1

In other words, when y is a discrete variable restricted to the set Y , A(y) F(q) with
218

11.1 Reduction of GREP for Oscillatory A(y)

219

q < m. Consequently, choosing {yl } Y , we are able to apply GREP(q) to A(y) and
obtain good approximations to A, even though A(y) F(m) to begin with.
Example 11.1.1 The preceding idea can be illustrated very
 xsimply via Examples 4.1.5
and 4.1.6 of Chapter 4. We have, with A(y) F(x) = 0 (sin t/t) dt and A I =
F(), that
F(x) = I +

cos x
sin x
H1 (x) +
H2 (x), H1 , H2 A(0) .
x
x

1
Thus, A(y) F(2)
with y x , 1 (y) cos x/x, and 2 (y) sin x/x. Here, y is
continuous.
Obviously, both 1 (y) = y cos(1/y) and 2 (y) = y sin(1/y) oscillate an innite number of times as y 0+, and 2 (y) = 0 when y = y i = 1/[(i + 1) ], i = 0, 1, . . . .
Thus, when y Y = { y 0 , y 1 , . . . }, A(y) F(1)
with A(y) = I + y cos(1/y)1 (y).


Example 11.1.2 As another illustration, consider theinnite integral I = 0 J0 (t) dt
x
that was considered in Example 5.1.13. With F(x) = 0 J0 (t) dt, we already know that
F(x) = I + x 1 J0 (x)g0 (x) + J1 (x)g1 (x), g0 , g1 A(0) .
1
Thus, A(y) F(2)
with y x , 1 (y) J0 (x)/x, 2 (y) J1 (x), and A I .
1
Let y l = xl , l = 0, 1, . . . , where xl are the consecutive zeros of J0 (x) that are greater
than 0. Thus, when y Y = { y 0 , y 1 , . . . }, A(y) F(1)
with A(y) = I + J1 (1/y)2 (y)

The purpose of this chapter is to derive reductions of the D-transformation for inte
grals 0 f (t) dt whose integrands have an innite number of oscillations at innity in
the way described. The reduced forms thus obtained have proved to be extremely efcient in computing, among others, integral transforms with oscillatory kernels, such as
Fourier, Hankel, and KontorovichLebedev transforms. The importance and usefulness
of asymptotic analysis in deriving these economical extrapolation methods are demonstrated several times throughout this chapter.
Recall that we are taking = 0 in the denition of the D-transformation;
we do so

with its reductions as well. In case the integral
 to be computed is a f (t) dt with a = 0,
we apply these reductions to the integral 0 f(t) dt, where f(x) = f (a + x) for x 0.
11.1.1 Review of the W-Algorithm for Innite-Range Integrals
As we will see in the next sections, the following linear systems arise from various
reductions of the D-transformation for some commonly occurring oscillatory inniterange integrals.
F(xl ) = A(nj) + (xl )

n1

i
i=0

xli

, j l j + n.

(11.1.4)

x

Here, F(x) = 0 f (t) dt and I [ f ] = 0 f (t) dt, and xl and the form factor (xl ) depend on the method being used.

220

11 Reduction of the D-Transformation

The W-algorithm for these equations assumes the following form:

1. For j = 0, 1, . . . , set
( j)

M0 =

F(x j )
,
(x j )

( j)

N0 =

1
,
(x j )

( j)

( j)

H0 = (1) j |N0 |,

( j)

( j)

K 0 = (1) j |M0 |.

2. For j = 0, 1, . . . , and n = 1, 2, . . . , compute

( j+1)

Q (nj) =
( j)

( j)

( j)

Q n1 Q n1
1
x 1
j+n x j

( j)

( j)

( j)

where Q n stand for Mn or Nn or Hn or K n .

3. For all j and n, set




( j)
 H ( j) 
 K ( j) 
Mn
 n 
 n 
( j)
( j)
( j)
An = ( j) , n =  ( j)  , and !n =  ( j)  .
 Nn 
 Nn 
Nn

11.2 The D-Transformation

11.2.1 Direct Reduction of the D-Transformation
We begin by reducing the D-transformation directly. When f B(m) and is as in Theorem
5.1.12, there holds
F(x) = I [ f ] +

m1


x k f (k) (x)gk (x),

(11.2.1)

k=0

x

where F(x) = 0 f (t) dt, I [ f ] = 0 f (t) dt, and k k + 1 are some integers that
satisfy (5.1.9), and gk A(0) . [Note that we have replaced the k in (5.1.8) by k , which
is legitimate since k k for each k.]
Suppose that f (x) oscillates an innite number of times as x and that there
exist xl , 0 < x0 < x1 < , liml xl = , for which
f (ki ) (xl ) = 0, l = 0, 1, . . . ; 0 k1 < k2 < < k p m 1.

(11.2.2)

Letting E = {0, 1, . . . , m 1} and E p = {k1 , . . . , k p } and X = {x0 , x1 , . . . }, (11.2.1)

then becomes

x k f (k) (x)gk (x), x X.
(11.2.3)
F(x) = I [ f ] +
kE\E p

Using (11.2.3), Sidi [274] proposed the D-transformation

as in the following denition.
Denition 11.2.1 Let f (x), E, E p , and X be as before, and denote q = m p. Then,
(q)

the D -transformation for the integral 0 f (t) dt is dened via the linear system
(q, j)
F(xl ) = D n +


kE\E p

x k f (k) (x)

n
k 1
i=0


ki
; N =
,
j

j
+
N
nk ,
xli
kE\E p
(11.2.4)


11.2 The D-Transformation

221

where n = (n s1 , n s2 , . . . , n sq ) with {s1 , s2 , . . . , sq } = E\E p . When the k in (11.1.1)

are known, we can choose k = k ; otherwise, we can take for k any known upper
bound of k . In case nothing is known about k , we can take k = k + 1, as was done
in the original denition of the D-transformation.

It is clear from Denition 11.2.1 that the D-transformation,

just as the Dtransformation, can be implemented by the W-algorithm (when q = 1) and by the W(q) algorithm (when q > 1). Both transformations have been used by Safouhi, Pinchon, and
Hoggan [253], and by Safouhi and Hoggan [250], [251], [252] in the accurate evaluation of some very complicated innite-range oscillatory integrals that arise in molecular
structure calculations and have proved to be more effective than others.

We now illustrate the use of the D-transformation

with two examples.
Example 11.2.2 Consider the case in which m = 2, f (x) = u(x)Q(x), and Q(x) vanishes and changes sign an innite number of times as x . If we choose the xl such that
Q(xl ) = 0, l = 0, 1, . . . , then f (xl ) = 0, l = 0, 1, . . . , too. Also, f  (xl ) = u(xl )Q  (xl )

for all l. The resulting D-transformation

is, therefore, dened via the equations
(1, j)

F(xl ) = D + xl 1 u(xl )Q  (xl )

1

i
i=0

xli

, j l j + ,

(11.2.5)

which can be solved by the W-algorithm. Note that only the derivative of Q(x) is needed
in (11.2.5), whereas that of u(x) is not. Also, the integer 1 turns out to be 0 in such
cases and thus can be replaced by 0. In the next sections, we give specic examples of
this application.

Example
is to integrals
 11.2.3 A practical application of the the D-transformation
( )
I [ f ] = 0 f (t) dt with f (x) = u(x)[M(x)] , where u A for some , M B(r )
for some r , and  1 is an integer. From Heuristic 5.4.3, we have that (M) B(m)
. Since u B(1) in addition, f B(m) as well. If we choose the
with m r +1

xl such that f (xl ) = 0, l = 0, 1, . . . , we also have f (k) (xl ) = 0, l = 0, 1, . . . , for

(m)
-transformation.
k = 1, . . . , 1. Therefore, I [ f ] can be computed by the D

11.2.2 Reduction of the s D-Transformation

Another form of the D-transformation

can be obtained by recalling the developments of
Section 5.3 on the simplied D (m) -transformation, namely, the s D (m) -transformation. In
case f (x) = u(x)Q(x), where u A( ) for some and Q(x) are simpler to deal with
than f (x), in the sense that its derivatives are available more easily than those of f (x),
we showed that
F(x) = I [ f ] +

m1


x k [u(x)Q (k) (x)]h k (x),

(11.2.6)

k=0

with k as before and for some h k A(0) . [Here, for convenience, we replaced the k of
(5.3.2) by their upper bounds k .] Because u(x) is monotonic as x , the oscillatory

222

11 Reduction of the D-Transformation

behavior of f (x) is contained in Q(x). Therefore, suppose there exist xl , 0 < x0 <
x1 < , liml xl = , for which
Q (ki ) (xl ) = 0, l = 0, 1, . . . ; 0 k1 < k2 < < k p m 1.

(11.2.7)

Letting E = {0, 1, . . . , m 1} and E p = {k1 , . . . , k p } and X = {x0 , x1 , . . . } as before,

(11.2.6) becomes

x k [u(x)Q (k) (x)]h k (x), x X.
(11.2.8)
F(x) = I [ f ] +
kE\E p

In view of (11.2.8), we give the following simplied denition of the D-transformation,

which was given essentially in Sidi [299].
Denition 11.2.4 Let f (x) = u(x)Q(x), E, E p , and X be as before, and denote q =
(q)
(q)
m p. Then, the s D -transformation, the simplied D -transformation for the inte
gral 0 f (t) dt is dened via the linear systems
(q, j)
F(xl ) = D n +

xlk [u(xl )Q (k) (xl )]

kE\E p

n
k 1
i=0


ki
; N =
,
j

j
+
N
nk ,
xli
kE\E p
(11.2.9)

where n = (n s1 , . . . , n sq ) with {s1 , . . . , sq } = E\E p . The k can be chosen exactly as

in Denition 11.2.1.
Obviously, the equations in (11.2.9) can be solved via the W(q) -algorithm.

11.3 Application of the D-Transformation

to Fourier Transforms
Let f (x) be of the form f (x) = T (x)u(x), where T (x) = A cos x + B sin x, |A| +
|B| = 0, and u(x) = e(x) h(x), where (x) is real and A(k) strictly for some integer
k 0 and h A( ) for some . Assume that, when k > 0, limx (x) = , so that
f (x) is integrable of innity. By the fact that T = f /u and T  + T = 0, it follows that
f = p1 f  + p2 f  with
p1 =

2(  + h  / h)
1
2

, p2 = ; w = 1 + (  + h  / h) (  + h  / h) . (11.3.1)
w
w

When k > 0, we have  A(k1) strictly. Therefore, w A(2k2) strictly. Consequently,

p1 A(k+1) and p2 A(2k+2) . When k = 0, we have  0, as a result of which,
w A(0) strictly, so that p1 A(1) and p2 A(0) . In all cases then, f B(2) and Theorem 5.1.12 applies and (11.2.1) holds with
0 = k + 1,
0 = 1,

1 = 2k + 2
1 = 0

when k > 0,
when k = 0.

(11.3.2)

Note that, in any case, 0 , 1 0. If we now pick the xl to be consecutive zeros of

(1)
T (x), 0 < x0 < x1 < , then the D -transformation can be used to compute I [ f ]


11.4 Application of the D-Transformation
to Hankel Transforms

223

via the equations

(1, j)

F(xl ) = D + xl 1 u(xl )T  (xl )

1

i
i=0

xli

, j l j + .

(11.3.3)
(1)

If we pick the xl to be consecutive zeros of T  (x), 0 < x0 < x1 < , then the s D transformation can be used via the equations
(1, j)

F(xl ) = D + xl 0 u(xl )T (xl )

1

i
i=0

xli

, j l j + .

(11.3.4)

These equations can be solved by the W-algorithm.

Note that, in both cases, xl = x0 + l, l = 0, 1, . . . , with suitable x0 .

11.4 Application of the D-Transformation

to Hankel Transforms
Let f (x) be of the form f (x) = C (x)u(x) with C (x) = A J (x) + BY (x), where
J (x) and Y (x) are the Bessel functions of (real) order of the rst and second
kinds, respectively, and |A| + |B| = 0, and u(x) = e(x) h(x), where (x) is real and
A(k) strictly for some integer k 0 and h A( ) for some . As in the preceding section, limx (x) = when k > 0, to guarantee that f (x) is integrable at
innity.
x
x2


By the fact that C (x) satises the Bessel equation C = 2 x
2 C + 2 x 2 C , and that


C = f /u, it follows that f = p1 f + p2 f with
p1 =

2x 2 (  + h  / h) x
x2
and p2 = ,
w
w

where
w = x 2 [(  + h  / h)2 (  + h  / h) ] x(  + h  / h) + x 2 2 .
From this, it can easily be shown that p1 A(i1 ) and p2 A(i2 ) with i 1 and i 2 exactly
as in the preceding section so that f B(2) . Consequently, Theorem 5.1.12 applies and
(11.2.1) holds with m = 2 and with 0 and 1 exactly as in the preceding section, namely,
0 = k + 1,
0 = 1,

1 = 2k + 2
1 = 0

when k > 0,
when k = 0.

(11.4.1)

Note that, in any case, 0 , 1 0.

(i) If we choose the xl to be consecutive zeros of C (x), 0 < x0 < x1 < , then the
D (1) -transformation can be used to compute I [ f ] via the equations
(1, j)

F(xl ) = D + xl 1 u(xl )C (xl )

1

i
i=0

xli

, j l j + .

(11.4.2)

Here, we can make use of the known fact that C (x) = (/x)C (x) C+1 (x), so
that C (xl ) = C+1 (xl ), which simplies things further.

224

11 Reduction of the D-Transformation

(ii) If we choose the xl to be consecutive zeros of C (x), 0 < x0 < x1 < , then the
s D (1) -transformation can be used to compute I [ f ] via the equations
(1, j)

F(xl ) = D + xl 0 u(xl )C (xl )

1

i
i=0

xli

, j l j + .

(11.4.3)

(iii) If we choose the xl to be consecutive zeros of C+1 (x), 0 < x0 < x1 < , then

we obtain another form of the s D-transformation

dened again via the equations
in (11.4.3). To see this, we rewrite (11.2.6) in the form


C (x) C+1 (x) h 1 (x)
F(x) = I [ f ] + x 0 u(x)C (x)h 0 (x) + x 1 u(x)
x
(11.4.4)
= I [ f ] + x 0 u(x)C (x)h 0 (x) + x 1 u(x)C+1 (x)h 1 (x),
where h 0 (x) = h 0 (x) + x 1 0 1 h 1 (x) A(0) since 1 0 1 0, and h 1 (x) =
h 1 (x) A(0) as well. Now, pick the xl to be consecutive zeros of C+1 (x) in (11.4.4)
to obtain the method dened via (11.4.3). [Note that the second expansion in (11.4.4) is
in agreement with Remark 5 in Section 4.3.]
These developments are due to Sidi [299]. All three methods above have proved to
be very effective for large, as well as small, values of . In addition, they can all be
implemented via the W-algorithm.

11.5 The D-Transformation

The D-transformation
of the previous sections is dened by reducing the Dtransformation with the help of the zeros of one or more of the f (k) (x). When these
zeros are not readily available or are difcult to compute, the D-transformation can be

reduced via the so-called D-transformation

of Sidi [274] that is a very exible device.
What we have here is really a general approach that is based on Remark 5 of Section 4.3,
within which a multitude of methods can be dened. What is needed for this approach
is some amount of asymptotic analysis of f (x) as x . Part of this analysis is quantitative, and the remainder is qualitative only.
Our starting point is again (11.2.1). By expressing f (x) and its derivatives as combinations of simple functions when possible, we rewrite (11.2.1) in the form
F(x) = I [ f ] +

m1


vk (x)g k (x),

(11.5.1)

k=0

where g k A(0) for all k. Here, the functions vk (x) have much simpler forms than
f (k) (x), and their zeros are readily available. Now, choose the xl , 0 < x0 < x1 < ,
liml xl = , for which
vk (xl ) = 0, l = 0, 1, . . . ; 0 k1 < k2 < < k p m 1.

(11.5.2)

Letting E = {0, 1, . . . , m 1} and E p = {k1 , . . . , k p } and X = {x0 , x1 , . . . } again,


11.6 Application of the D-Transformation
to Hankel Transforms

225

(11.5.1) becomes
F(x) = I [ f ] +

vk (x)g(x),
x X.

(11.5.3)

kE\E p

In view of (11.5.3), we give the following denition analogously to Denitions 11.2.1

and 11.2.4.
Denition 11.5.1 Let f (x), E, E p , and
X be as before, and denote q = m p. Then

the D (q) -transformation for the integral 0 f (t) dt is dened via the linear equations
(q, j)
F(xl ) = D n +


kE\E p

vk (xl )

n
k 1
i=0


ki
, j l j + N ; N =
n k , (11.5.4)
i
xl
kE\E p

where n = (n s1 , . . . , n sq ) with {s1 , . . . , sq } = E\E p .

transformation, like the D
It is obvious from Denition 11.5.1 that the Dtransformations, can be implemented by the W-algorithm (when q = 1) and by the
W(q) -algorithm (when q > 1).
The best way to clarify what we have done so far is with practical examples, to which
we now turn.

11.6 Application of the D-Transformation

to Hankel Transforms

Consider the integral 0 C (t)u(t) dt, where C (x) and u(x) are exactly as described in
Section 11.4. Now it is known that
C (x) = 1 (x) cos x + 2 (x) sin x, 1 , 2 A(1/2) .

(11.6.1)

The derivatives of C (x) are of precisely the same form as C (x) but with different
1 , 2 A(1/2) . Substituting these in (11.2.6) with Q(x) = C (x) there, we obtain
F(x) = I [ f ] + v0 (x)g 0 (x) + v1 (x)g 1 (x), g 0 , g 1 A(0) ,

(11.6.2)

where
v0 (x) = x 1/2 u(x) cos x, v1 (x) = x 1/2 u(x) sin x; = max{ 0 , 1 }, (11.6.3)
with 0 and 1 as in Section 11.4.
We now pick the xl to be consecutive zeros of cos x or of sin x. The xl are thus

equidistant with xl = x0 + l, x0 > 0. The equations that dene the D-transformation

then become
1/2
j)
+ (1)l xl
u(xl )
F(xl ) = D (1,

1

i
i=0

xli

, j l j + .

(11.6.4)

Here, we have a method that does not require specic knowledge of the zeros or
extrema of C (x) and that has proved to be very effective for low to moderate values of .
The equations in (11.6.4) can be solved via the W-algorithm.

226

11 Reduction of the D-Transformation

11.7 Application of the D-Transformation

to Integrals
of Products of Bessel Functions

Consider the integral 0 f (t) dt, where f (x) = C (x)C (x)u(x) with C (x) and u(x)
exactly as in Section 11.4 again. Because both C (x) and C (x) are in B(2) and u(x)
is in B(1) , we conclude, by part (i) of Heuristic 5.4.1, that f B(4) . Recalling that
C (x) = 1 (x) cos x + 2 (x) sin x, where i A(1/2) for i = 1, 2 and for all , we
see that f (x) is, in fact, of the following simple form:
f (x) = u(x)[w0 (x) + w1 (x) cos 2x + w2 (x) sin 2x], wi A(1) , i = 0, 1, 2.
(11.7.1)
+ (x)ei2x +
Now, the sum w1 (x) cos 2x + w2 (x) sin 2x can also be expressed in the form w
i2x
(1)
with w
A . Thus, f (x) = f 0 (x) + f + (x) + f (x) with f 0 (x) =
w
(x)e
(x)ei2x , f 0 , f B(1) . By part (ii) of Heuristic 5.4.1,
u(x)w0 (x), and f (x) = u(x)w
we therefore conclude
that f B(3) . Let us now write F(x) = F0 (x) + F+ (x) + F (x),
x
x
where F0 (x) = 0 f 0 (t) dt and F (x) = 0 f (t) dt. Recalling Theorem 5.7.3, we have
F(x) = I [ f ] + x 0 f 0 (x)h 0 (x) + x f + (x)h + (x) + x f (x)h (x),
where


0 =

1
if k = 0
, = k + 1, and h 0 , h A(0) ,
k + 1 if k > 0

(11.7.2)

(11.7.3)

which can be rewritten in the form

F(x) = I [ f ] + x 0 1 u(x)h 0 (x) + x 1 u(x)h 1 (x) cos 2x
+ x 1 u(x)h 2 (x) sin 2x, h 0 , h 1 , h 2 A(0) .

(11.7.4)

Now, let us choose the xl to be consecutive zeros of cos 2x or sin 2x. These are equidis
then
tant with xl = x0 + l/2, x0 > 0. The equations that dene the D-transformation
become
(2, j)
1
F(xl ) = D n + xl 0 u(xl )

n
1 1
i=0

n
2 1
1i
2i
l 1
+
(1)
x
u(x
)
, j l j + n1 + n2.
l
l
i
xli
i=0 xl

(11.7.5)
(For simplicity, we can take 0 = = 1 throughout.) Obviously, the equations in (11.7.5)
can be solved via the W(2) -algorithm.

Consider now the integral 0 f (t) dt, where f (x) = C (ax)C (bx)u(x) with a = b.
Using the preceding technique, we can show that

F(x) = I [ f ] + x 1 u(x) h c+ (x) cos(a + b)x + h c (x) cos(a b)x

+ h s+ (x) sin(a + b)x + h s (x) sin(a b)x , h c , h s A(0) . (11.7.6)
This means that f B(4) precisely. We can now choose the xl as consecutive zeros
of sin(a + b)x or of sin(a b)x and compute I [ f ] by the D (3) -transformation. If
(a + b)/(a b) is an integer, by choosing the xl as consecutive zeros of sin(a b)x

11.8 The W - and mW -Transformations for Very Oscillatory Integrals

227

[which make sin(a + b)x vanish as well], we can even compute I [ f ] by the D (2) transformation.
11.8 The W - and mW -Transformations for Very Oscillatory Integrals
11.8.1 Description of the Class B

The philosophy behind the D-transformation

can be extended further to cover a very
large family of oscillatory innite-range integrals. We start with the following denition
of the relevant family of integrands.
if it can be expressed
Denition 11.8.1 We say that a function f (x) belongs to the class B
in the form
r

u j ( j (x)) exp( j (x))h j (x),
(11.8.1)
f (x) =
j=1

where u j , j , j and h j are as follows:

1. u j (z) is either eiz or eiz or any linear combination of these (like cos z or sin z).
2. j (x) and j (x) are real and j A(m) strictly and j A(k) strictly, where m is a
positive integer and k is a nonnegative integer, and

+ ! j (x)
j (x) = (x) +  j (x) and j (x) = (x)

(11.8.2)

with

(x)
=

m1


i x mi and (x)
=

i=0

k1


i x ki and  j , ! j A(0) .

(11.8.3)

i=0

Also, when k 1, we have limx (x)

= . We assume, without loss of generality, that 0 > 0.
3. h j A( j ) for some possibly complex j such that j j  = integer for every j and
j  . Denote by that j whose real part is largest. (Note that ! j are all the same.)
Now, the class B is the union of two mutually exclusive sets: Bc and Bd , where Bc
that are integrable at innity, while Bd contains the ones
contains the functions in B
whose integrals diverge but are dened in the sense of Abel summability. (For this point,
see the remarks at the end of Section 5.7.)
When m = 1, f (x) is a linear combination of functions that have a simple sinusoidal
behavior of the form ei0 x as x , that is, the period of its oscillations is xed

as x . When m 2, however, f (x) oscillates much more rapidly [like ei (x) ] as

x , with the period of these oscillations tending to zero. In the sequel, we call
such functions f (x) very oscillatory.

In the remainder of this section, we treat only functions in B.

Note that, from Denition 11.8.1, it follows that f (x), despite its complicated appearance in (11.8.1), is of the form
f (x) = e(x) [ei(x) h + (x) + ei(x) h (x)], h A( ) .

(11.8.4)

228

11 Reduction of the D-Transformation

This is sobecause e(x) A(0) for A(0) . If we denote f (x) = e(x)i (x) h (x) and
x
F (x) = 0 f (t) dt, then, by Theorem 5.7.3, we have

F (x) = I [ f ] + x f (x)g (x), = 1 max{m, k}, g A(0) strictly. (11.8.5)

From this and from the fact that f (x) = f + (x) + f (x), it follows that

(0)

F(x) = I [ f ] + x + e(x) [cos((x))b

1 (x) + sin( (x))b2 (x)], b1 , b2 A . (11.8.6)


Here, I [ f ] is 0 f (t) dt when this integral converges; it is the Abel sum of 0 f (t) dt
when the latter diverges.
Before going on, we would like to note that the class of functions B is very comprehensive in the sense that it includes many integrands of oscillatory integrals that arise in
important engineering, physics, and chemistry applications.

11.8.2 The W - and mW -Transformations

We now use (11.8.6) to dene the W - and mW -transformations of Sidi [281] and [288],
respectively. Throughout, we assume that f (x) behaves in a regular manner, starting
with x = 0. By this, we mean that there do not exist intervals of (0, ) in which f (x),
relative to its average behavior, has large and sudden changes or remains almost constant.
[One simple way to test the regularity of f (x) is by plotting its graph.] We also assume

that (x)
and its rst few derivatives are strictly increasing on (0, ). This implies that

, 0 < x0 < x1 < , such that xl are consecutive

there exists a unique sequence {xl }l=0

roots of the equation sin (x) = 0 [or of cos (x)

= 0] and behave regularly as well.

= q
This also means that x0 is the unique simple root of the polynomial equation (x)
for some integer (or half integer) q, and, for each l > 0, xl is the unique simple root of
the polynomial equation (x) = (q + l). We also have liml xl = .
These assumptions make the discussion simpler and the W - and mW -transformations
more effective. If f (x) and (x) start behaving as described only for x  c for some
 c > 0,

then a reasonable strategy is to apply the transformations to

the
integral
0 f (t) dt,
c
where f(x) = f (c + x), and add to it the nite-range integral 0 f (t) dt that is computed
separately.
We begin with the W -transformation.
Denition 11.8.2 Choose the xl as in the previous paragraph, i.e., as consecutive positive

zeros of sin (x)

[or of cos (x)]. The W -transformation is then dened via the linear
equations
l)
+ (x

F(xl ) = Wn( j) + (1)l xl

n1

i
i=0

( j)

xli

, j l j + n,

(11.8.7)

( j)

where Wn is the approximation to I [ f ]. (Obviously, the Wn can be determined via

the W-algorithm.)
we need to
As is clear, in order to apply the W -transformation to functions in B,
analyze the j (x), j (x), and h j (x) in (11.8.1) quantitatively as x in order to

11.8 The W - and mW -Transformations for Very Oscillatory Integrals

229

extract (x),
(x),
and . The modication of the W -transformation, denoted the mW transformation, requires one to analyze only j (x); thus, it is very user-friendly. The
expansion in (11.8.6) forms the basis of this transformation just as it forms the basis of
the W -transformation.
Denition 11.8.3 Choose the xl exactly as in Denition 11.8.2. The mW -transformation
is dened via the linear equations
F(xl ) = Wn( j) + (xl )

n1

i
i=0

xli

, j l j + n,

where


(xl ) = F(xl+1 ) F(xl ) =

(11.8.8)

xl+1

f (t) dt.

(11.8.9)

xl
( j)

(These Wn can also be determined via the W-algorithm.)

Remarks.
1. Even though F(xl+1 ) F(xl ) is a function of both xl and xl+1 , we have denoted it
(xl ). This is not a mistake as xl+1 is a function of l, which, in turn, is a function of
xl (there is a 11 correspondence between xl and l), so that xl+1 is also a function
of xl .


2. Because F(xl ) = li=0 (xi1 ) with x1 = 0 in (11.8.9), the mW -transformation can


(xi1 ).
be viewed as a convergence acceleration method for the innite series i=0
(As we show shortly, when is real, this series is ultimately alternating too.) In this
sense, the mW -transformation is akin to a method of Longman [170], [171], in which
one integrates f (x) between its consecutive zeros or extrema y1 < y2 < to obtain
y +1
the integrals vi = yi i f (t) dt, where y0 = 0, and accelerates the convergence of the


vi by a suitable sequence transformation. Longman
alternating innite series i=0
used the Euler transformation for this purpose. Of course, other transformations,
such as the iterated 2 -process, the Shanks transformation, etc., can also be used. We
discuss these methods later.
 xl
f (t) dt without changing
3. The quantities (xl ) can also be dened by (xl ) = xl1
( j)
the quality of the Wn appreciably.
Even though the W -transformation was obtained directly from the expansion of F(x)
given in (11.8.6), the mW -transformation has not relied on an analogous expansion so
far. Therefore, we need to provide a rigorous justication of the mW -transformation.
For this, it sufces to show that, with (xl ) as in (11.8.9),
F(xl ) = I [ f ] + (xl )g(xl ), g A(0) .

(11.8.10)

As we discuss in the next theorem, this is equivalent to showing that this (xl ) is of the
form
l)
+ (x

(xl ) = (1)l xl

b(xl ), b A(0) .

(11.8.11)

230

11 Reduction of the D-Transformation

An important implication of (11.8.11) is that, when is real, the (xl ) alternate in sign
for all large l.
Theorem 11.8.4 With F(x), {xl }, and (xl ) as described before, (xl ) satises
(11.8.11). Consequently, F(xl ) satises (11.8.10).
Proof. Without loss of generality, let us take the xl as the zeros of sin (x). Let us
l ) = (1)q+l . Consequently,
set x = xl in (11.8.6). We have sin (xl ) = 0 and cos (x
(11.8.6) becomes
l)
+ (x

F(xl ) = I [ f ] + (1)q+l xl

b1 (xl ).

(11.8.12)

Substituting this in (11.8.9), we obtain



+
+
(xl ) = (1)q+l+1 xl e(xl ) b1 (xl ) + xl+1 e(xl+1 ) b1 (xl+1 )
l)
+ (x

= (1)q+l+1 xl
where
w(xl ) = R(xl )e

l)
(x


; R(xl ) =

xl+1
xl

b1 (xl )[1 + w(xl )],

+

(11.8.13)

b1 (xl+1 )
l ) = (x
l+1 ) (x
l ).
, (x
b1 (xl )
(11.8.14)

Comparing (11.8.13) with (11.8.11), we identify b(x)

= (1)q+1 b1 (x)[1 + w(x)]. We
(0)

have to show only that b A .

ai l (1i)/m for all
From Lemma 9.5.1 at the end of Chapter 9, we know that xl = i=0
1/m
> 0. From this, some useful results follow:
large l, a0 = (/0 )
xl+1 =

ai l (1i)/m , ai = ai , i = 0, 1, . . . , m 1, am = am + a0 /m,

i=0

a0
,
(11.8.15)
m
i=0
 



p p
xl+1 xl p
p
p
p
( p)
( p)
xl+1 xl = xl
a i l 1+( pi)/m , a 0 = a0 .
1 =
1+
xl
m
i=0
xl+1 xl =

a i l 1+(1i)/m , a 0 =

Using these, we can show after some manipulation that

R(xl ) 1 + l 1

di l i/m as l ,

(11.8.16)

i=0

and, for k = 0,
l) =
(x

k1

i=0

ki
i (xl+1
xlki ) = l 1+k/m


i=0

ei l i/m , e0 = 0

k k
a . (11.8.17)
m 0

l ) 0 too.]
[For k = 0, we have (x)
0 so that (x


(1i)/m
From the fact that xl = i=0 ai l
for all large l, we can show that any function


i l (r i)/m as l also has an
of l with an asymptotic expansion of the form i=0

11.8 The W - and mW -Transformations for Very Oscillatory Integrals

231

i xlr i as l , 0 = 0 /a0r . In addition, if
asymptotic expansion of the form i=0
one of these expansions converges, then so does the other. Consequently, (11.8.16) and
(11.8.17) can be rewritten, respectively, as
R(xl ) 1 + d1 xl1 + d2 xl2 + as l ,

(11.8.18)

and
l ) = xlkm
(x

ei xli , e0 =

i=0

k
0 a0m .
m

(11.8.19)

Thus, R A(0) strictly and  A(km) strictly. When k m, e(x) A(0) strictly

and e(x) 1 as x . Consequently, 1 + w(x) A(0) strictly, hence b A(0) . For

k > m, we have that e(x) 0 as x essentially like exp(e0 x km ) since

= due to the fact that e0 < 0 in (11.8.19). (Recall that a0 > 0 and
limx (x)
0 < 0 by assumption.) Consequently, 1 + w(x) A(0) strictly, from which b A(0)
again. This completes the proof of (11.8.11). The result in (11.8.10) follows from the ad

ditional observation that g(x) = 1/[1 + w(x)] A(0) whether k m or k > m.

Theorem 11.8.4 shows clearly that (xl ) is a true shape function for F(xl ), and that
the mW -transformation is a true GREP(1) .

11.8.3 Application of the mW -Transformation to Fourier,

Inverse Laplace, and Hankel Transforms
Fourier Transforms
If f (x) = u(x)T (x) with u(x) and T (x) exactly as in Section 11.3, then f B with

(x)
= x. Now, choose xl = x0 + l, l = 1, 2, . . . , x0 > 0, and apply the mW transformation.
Hasegawa and Sidi [127] devised
 an efcient automatic integration method for a large
class of oscillatory integrals 0 f (t) dt, such as Hankel transforms, in which these

integrals are expressed as sums of integrals of the form 0 eit g(t) dt, and the mW transformation just described is used to compute the latter. An important
 x ingredient
of this procedure is a fast method for computing the indenite integral 0 eit g(t) dt,
described in Hasegawa and Torii [128].

A variant of the mW -transformation for Fourier transforms 0 u(t)T (t) dt was proposed by Ehrenmark [73]. In this variant, the xl are obtained by solving some nonlinear
equations involving asymptotic information coming from the function u(x). See [73] for
details.
Inverse Laplace Transforms
An interesting application is to the inversion of the Laplace transform
by the Bromwich
 zt
is the Laplace transform of u(t), that is, u(z)

integral. If u(z)
= 0 e u(t) dt, then
u(t+) + u(t)
1
=
2
2i

c+i

ci

dz, t > 0,
e zt u(z)

232

11 Reduction of the D-Transformation

has all its singularwhere the contour of integration is the straight line z = c, and u(z)
ities to the left of this line. Making the substitution z = c + i in the Bromwich integral,
we obtain



ect
u(t+) + u(t)
i t
i t
+ i ) d +
i ) d .
=
e u(c
e u(c
2
2 0
0

Assume that u(z)

= ezt0 u 0 (z), with u 0 (z) A( ) for some . Now, we can apply the
mW -transformation to the last two integrals with l = (l + 1)/(t t0 ), l = 0, 1, . . . .
In case u(t) is real, only one of these integrals needs to be computed because now


ect
u(t+) + u(t)
i t
+ i ) d .
=

e u(c
2

This approach has proved to be very effective in inverting Laplace transforms when u(z)
is known for complex z.
Hankel Transforms
If f (x) = u(x)C (x) with u(x) and C (x) exactly as in Section 11.4, then f B with
(x) = x again. Choose xl = x0 + l, l = 1, 2, . . . , x0 > 0, and apply the mW transformation.
This approach was found to be one of the most effective means for computing Hankel
transforms when the order is of moderate size; see Lucas and Stone [189].
Variants for Hankel Transforms
Again, let f (x) = u(x)C (x) with u(x) and C (x) exactly as in Section 11.4. When , the
order of the Bessel function C (x), is large, it seems more appropriate to choose the xl
in the mW -transformation as the zeros of C (x) or of C (x) or of C+1 (x), exactly as was
and s D-transformations.

done in Section 11.4, where we developed the relevant DThis

use of the mW -transformation was suggested by Lucas
 and Stone [189], who chose the
xl as the zeros or extrema of J (x) for the integrals 0 u(t)J (t) dt. The choice of the xl
as the zeros of C+1 (x) was proposed by Sidi [299]. The resulting methods produce some
of the best results, as concluded in [189] and [299]. The same conclusion is reached in
the extensive comparative study of Michalski [211] for the so-called Sommerfeld type
integrals that are simply Hankel transforms.
The question, of course, is whether (xl ) is a true shape function for F(xl ). The


ai l 1i as l , a0 = .
answer to this question is in the afrmative as xl i=0
Even though this asymptotic expansion need not be convergent, the developments of
Theorem 11.8.4 remain valid. Let us show this for the case in which xl are consecutive
zeros of C (x). For this, we start with
F(x) = I [ f ] + x 0 u(x)C (x)b0 (x) + x 1 [u(x)C (x)] b1 (x),

(11.8.20)

which is simply (11.2.1). Letting x = xl , this reduces to

F(xl ) = I [ f ] + xl 1 u(xl )C (xl )b1 (xl ).

(11.8.21)

11.8 The W - and mW -Transformations for Very Oscillatory Integrals

233

Thus,

(xl ) = F(xl+1 ) F(xl ) = xl 1 u(xl )C (xl )b1 (xl )[w(xl ) 1],
where


w(xl ) =

xl+1
xl

 1

u(xl+1 ) b1 (xl+1 ) C (xl+1 )

.
u(xl ) b1 (xl ) C (xl )

(11.8.22)

(11.8.23)

Now, differentiating both sides of (11.6.1), we obtain

C (x) = 1 (x) cos x + 2 (x) sin x, 1 , 2 A(1/2) .

(11.8.24)

Recalling that xl l + a1 + a2l 1 + as l , and substituting this in (11.8.24),

we get
C (xl ) (1)l

vi l 1/2i as l ,

(11.8.25)


C (xl+1 )

1
+
i l i as l .
C (xl )
i=1

(11.8.26)

i=0

from which

Using the fact that u(x) = e(x) h(x), and proceeding as in the proof of Theorem 11.8.4
[starting with (11.8.15)], and invoking (11.8.25) as well, we can show that
w(x) 1 + w1 x 1 + w2 x 2 + as x if k 1,
w(x) = e P(x) , P(x)

pi x k1i as x , p0 < 0, if k > 1.

(11.8.27)

i=0

Combining (11.8.21), (11.8.22), and (11.8.26), we nally obtain

F(xl ) = I [ f ] + (xl )g(xl ),

(11.8.28)

where
g(x)

b i x i as x if k 1,

i=0

g(x) = 1 + e P(x) if k > 1.

(11.8.29)

Note that when k > 1, e P(x) 0 as x essentially like exp(e0 x k1 ) with e0 < 0.
Consequently, g A(0) in any case.
We have thus shown that (xl ) is a true shape function for F(xl ) with these variants
of the mW -transformation as well.
Variants for General Integral Transforms with Oscillatory Kernels
In view of the variants of the mW -transformation for Hankel transforms
 we have just
discussed, we now propose variants for general integral transforms 0 u(t)K (t) dt,
where u(x) does not oscillate as x or it oscillates very slowly, for example, like

234

11 Reduction of the D-Transformation

exp(ic(log x)s ), with c and s real, and K (x), the kernel of the transform, can be expressed in the form K (x) = v(x) cos (x) + w(x) sin (x), with (x) real and A(m)

for some positive integer m, and v, w A( ) for some . Let (x)

be the polynomial
part of the asymptotic expansion of (x) as x . Obviously, K (x) oscillates like
exp[i (x)] innitely many times as x .
Denition 11.8.5 Choose the xl to be consecutive zeros of K (x) or of K  (x) on (0, ),
and dene the mW -transformation via the linear equations
F(xl ) = Wn( j) + (xl )

n1

i
i=0

xli

, j l j + n,

where, with f (x) = u(x)K (x),

 xl

F(xl ) =
f (t) dt, (xl ) = F(xl+1 ) F(xl ) =
0

(11.8.30)

xl+1

f (t) dt. (11.8.31)

xl

These variants of the mW -transformation should be effective in case K (x) starts to

behave like exp[i (x)] only for sufciently large x, as is the case, for example, when
K (x) = C (x) with very large .

11.8.4 Further Variants of the mW -Transformation



Let f B as in Denition 11.8.1, and let I [ f ] be the value of the integral 0 f (t) dt
or its Abel sum. In case we do not want to bother with the analysis of j (x), we can
compute I [ f ] by applying variants of the mW -transformation that are again dened via
the equations (11.8.8) and (11.8.9), with xl now being chosen as consecutive zeros of
either f (x) or f  (x). Determination of these xl may be more expensive than those in
( j)
Denition 11.8.3. Again, the corresponding Wn can be computed via the W-algorithm.
These new methods can also be justied by proving a theorem analogous to Theorem 11.8.4 for their corresponding (xl ), at least in some cases for which xl satisfy


ai l (1i)/m as l , a0 = (/0 )1/m > 0.
xl i=0
The performances of these variants of the mW -transformation are similar to those of
and s D-transformations,

the Dwhich use the same sets of xl .

ai l (1i)/m
Another variant of the mW -transformation can be dened in case xl i=0

as l , a0 > 0, which is satised when the xl are zeros of sin (x) or of cos (x)
or of special functions such as Bessel functions. In such a case, any quantity l that


has an asymptotic expansion of the form l i=0
ei xli as l has an asymptotic


i/m
as l , and this applies to the function
expansion also of the form l i=0 ei l
g(x) in (11.8.10). The new variant of the mW -transformation is then dened via the linear
systems
F(xl ) = Wn( j) + (xl )

n1

i=0

( j)

i
, j l j + n.
(l + 1)i/m

(11.8.32)

Again, the Wn can be computed with the help of the W-algorithm. Note that the resulting

11.9 Convergence and Stability

235

method is analogous to the d(m) -transformation. Numerical results indicate that this new
method is also very effective.
11.8.5 Further Applications of the mW -Transformation
Recall that, in applying the mW -transformation, we consider only the dominant (polynomial) part (x) of the phase of oscillations of f (x) and need not concern ourselves
with the modulating factors h j (x)e j (x) . This offers a signicant advantage, as it suggests
(see Sidi [288]) that we could
 at least attempt to apply the mW -transformation to all
(very) oscillatory integrals 0 f (t) dt whose integrands are of the form
f (x) =

r


u j ( j (x))H j (x),

j=1

where u j (z) and j (x) are exactly as described in Denition 11.8.1, and H j (x) are
arbitrary functions that do not oscillate as x or that may oscillate slowly, that is,

slower than e j (x) . Note that not all such functions f (x) are in the class B.
Numerical results indicate that the mW -transformation is as efcient on such integrals
It was used successfully in the numerical
as it is on those integrals with integrands in B.
inversion of general Kontorovich-Lebedev transforms by Ehrenmark [74], [75], who also
provided a rigorous justication for this usage.
If we do not want to bother with the analysis of j (x), we can apply the variants of
the mW -transformation discussed in the preceding subsection to integrals of such f (x)
with the same effectiveness. Again, determination of the corresponding xl may be more
expensive than before.

11.9 Convergence and Stability

In this section, we consider briey the convergence and stability of the reduced transformations we developed so far. The theory of Section 4.4 of Chapter 4 applies in general,
because the transformations of this chapter are all GREPs.
For the D (1) -, s D (1) -, D (1) -, W -, and mW -transformations for (very) oscillatory integrals, the theory of Chapter 9 applies with t j = x 1
j throughout. For example, (i) the
D (1) -transformations for Fourier transforms dened in Section 11.3, (ii) all three D (1)
transformations for Hankel transforms in Section 11.4, (iii) the D-transformations
for
Hankel transforms in Section 11.5, and (iv) the W - and mW -transformations for inte can all be treated within the framework of Chapter 9.
grals of functions in the class B,
In particular, Theorems 9.3.19.3.3, 9.4.2, and 9.4.3 hold with appropriate (t) and
B(t) there. These can be identied from the form of F(xl ) I [ f ] in each case. We note
(1, j)
(1, j)
( j)
only that the diagonal sequences { D n }n=0 , { D n }n=0 , and {Wn }n=0 , when is real
everywhere, are stable and converge. Actually, we have
lim n( j) = 1 and A(nj) I [ f ] = O(n ) as n for every > 0,

( j)
(1, j)
(1, j)
( j)
as follows from Theorems 9.4.2 and 9.4.3. Here An stands for D n or D n or Wn ,
depending on the method being used.

236

11 Reduction of the D-Transformation

11.10 Extension to Products of Oscillatory Functions

In this section, we show how the reduced methods can be applied to integrands that are

given as products of different oscillating functions in B.

s
As an example, let us consider f (x) = u(x) r =1 fr (x), where
u(x) = e(x) h(x), (x) real and A(k) , k 0 integer, h A( ) , (11.10.1)
and
fr (x) = r(+) Ur(+) (x) + r() Ur() (x),

(11.10.2)

where r() are constants and Ur() (x) are of the form
Ur() (x) = evr (x)iwr (x) Vr() (x),
vr A(kr ) , wr A(m r ) , kr , m r 0 integers; vr (x), wr (x) real,
Vr() A(r ) for some r .

(11.10.3)

We assume that u(x), r() , and Ur() (x) are all known and that the polynomial parts


wri x m r i as x ,
w
r (x) of the wr (x) are available. In other words, if wr (x) i=0

m r 1
m r i
is known explicitly for each r .
then w
r (x) = i=0 wri x


(i) We now propose to compute 0 f (t) dt by expanding the product u rs =1 fr in
terms of the Ur() and applying to each term in this expansion an appropriate
GREP(1) . Note that there are 2s terms in this expansion and each of them is in
B(1) . This approach is very inexpensive and produces very high accuracy.
To illustrate the procedure above, let us look at the case s = 2. We have

f = u 1(+) 2(+) U1(+) U2(+) + 1(+) 2() U1(+) U2()

+ 1() 2(+) U1() U2(+) + 1() 2() U1() U2() .
Each of the four terms in this summation is in B(1) since u, U1() , U2() B(1) . It is
sufcient to consider the functions f (+,) uU1(+) U2() , the remaining functions
f (,) = uU1() U2() being similar. We have
f (+,) (x) = e(x)+i'(x) g(x),
with
 = + v1 + v2 , ' = w1 w2 , g = hV1(+) V2() .
Obviously,  A(K ) and ' A(M ) for some integers K 0, M 0, and g
A( +1 +2 ) .
Two different possibilities can occur:

(a) If M > 0, then f (+,) (x) is oscillatory, and we can apply to 0 f (+,) (t) dt the
1 (x)
D (1) - or the mW -transformation with xl as the consecutive zeros of sin[w
w
2 (x)] or cos[w
1 (x) w
2 (x)]. We can also apply the variants of the mW transformation by choosing the xl to be the consecutive zeros of  f (+,) (x) or
! f (+,) (x).

11.10 Extension to Products of Oscillatory Functions

237


(b) If M = 0, then f (+,) (x) is not oscillatory, and we can apply to 0 f (+,) (t) dt
the D (1) -transformation with xl = ecl for some > 0 and c > 0.
(ii) Another method we propose is based on the observation that the expansion of

u rs =1 fr can also be expressed as the sum of 2s1 functions, at least some of
Thus, the mW -transformation and its variants can be applied to
which are in B.
each of these functions.
To illustrate this procedure, we turn to the previous example and write f in the
form f = G + + G , where


G + = u 1(+) 2(+) U1(+) U2(+) + 1() 2() U1() U2()
and



G = u 1(+) 2() U1(+) U2() + 1() 2(+) U1() U2(+) .

with phase of oscillation w1 w2 . In this case, we can apIf M > 0, then G B


ply the mW -transformation to 0 G (t) dt by choosing the xl to be the consecutive
2 (x)] or cos[w
1 (x) w
2 (x)] or of (b) G (x) or
zeros of either (a) sin[w
1 (x) w
!G (x). If M 0, then G B(1) but is not oscillatory. In this case, we can apply

the D (1) -transformation to 0 G (t) dt.

products of two Bessel functions, namely,
 As an example, we consider integrals of()
()
()
()
0 u(t)C (r t)C (st) dt. In this example, U1 (x) = H (r x) and U2 (x) = H (sx),
(+)
()
where H (z) and H (z) stand for the Hankel functions of the rst and second kinds,
1 (x) =
respectively, namely, H() (z) = J (z) iY (z). Thus, v1 (x) = v2 (x) = 0 and w
r x and w
2 (x) = sx and 1 = 2 = 1/2. Therefore, we need to be concerned with
computation of the integrals of u(x)H(+) (r x)H(+) (sx), etc., by the mW -transformation.
 This approach was used by Lucas [188] for computing integrals of the form
0 u(t)J (r t)J (st) dt, which is a special case of those considered here.

12
Acceleration of Convergence of Power Series by the
d-Transformation: Rational d-Approximants

12.1 Introduction
In this chapter, we are concerned with the SidiLevin rational d-approximants and ef

k1
by the dcient summation of (convergent or divergent) power series
k=1 ck z
(m)
transformation. We assume that {cn } b for some m and analyze the consequences
of this.
One of the difculties in accelerating the convergence of power series has been the lack
of stability and acceleration near points of singularity of the functions f (z) represented
by the series. We show in this chapter via a rigorous analysis how to tackle this problem
and stabilize the acceleration process at will.
As we show in the next chapter, the results of our study of power series are useful for
Fourier series, orthogonal polynomial expansions, and other series of special functions.
Most of the treatment of the subject we give in this chapter is based on the work of
Sidi and Levin [312] and Sidi [294].
12.2 The d-Transformation on Power Series


k1
, where
Consider the application of the d-transformation to the power series
k=1 ck z
ck satisfy the (m + 1)-term recursion relation
cn+m =

m1


(s )

qs (n)cn+s , qs A0

strictly, s integer, s = 0, 1, . . . , m 1.

s=0

(12.2.1)
(This will be the case when {cn } b(m) in the sense of Denition 6.1.2.) We have the
following interesting result on the sequence {cn z n1 } :
Theorem 12.2.1 Let the sequence {cn } be as in the preceding paragraph. Then, the
sequence {an }, where an = cn z n1 , n = 1, 2, . . . , is in b(m) exactly in accordance with
Denition 6.1.2, except when z Z = {z 1 , . . . , z }, for some z i and 0 m. Actu

(0)
k
ally, for z  Z , there holds an = m
k=1 pk (n) an with pk A0 , k = 1, . . . , m.


Proof. Let us rst dene qm (n) 1 and rewrite (12.2.1) in the form m
s=0 qs (n)cn+s =
0. Multiplying this equation by z n+m1 and invoking ck = ak z k+1 and (6.1.7), we

238

12.2 The d-Transformation on Power Series

239

obtain
m


qs (n)z

ms

s=0

s  

s
k=0

k an = 0.

(12.2.2)

Interchanging the order of the summations, we have


m 
m  

s
qs (n)z ms k an = 0.
k
k=0 s=k
Rearranging things, we obtain an =

m
k=1

(12.2.3)

pk (n)k an with

m  s 
ms
Nk (n)
s=k k qs (n)z
, k = 1, . . . , m.

pk (n) =

m
ms
q
(n)z
D(n)
s=0 s

(12.2.4)

We realize that, when viewed as functions of n (z being held xed), the numerator Nk (n)
and the denominator D(n) of pk (n) in (12.2.4) have asymptotic expansions of the form
Nk (n)

rki (z)n k i and D(n)

i=0

ri (z)n i as n ,

(12.2.5)

i=0

where rki (z) and ri (z) are polynomials in z, of degree at most m k and m, respectively,
and are independent of n, and
k = max {s } and = max {s }; m = 0.
ksm

0sm

(12.2.6)
( )

Obviously, rk0 (z)  0 and r0 (z)  0 and k for each k. Therefore, Nk A0 k strictly
)
provided rk0 (z) = 0 and D A(
0 strictly provided r0 (z) = 0. Thus, as long as r0 (z) = 0
[note that r0 (z) = 0 for at most m values of z], we have pk A(i0 k ) , i k k 0;
( 1)
, and this may
hence, pk A(0)
0 . When z is such that r0 (z) = 0, we have that D A0


cause an increase in i k . This completes the proof.
Remarks.
1. One immediate consequence of Theorem 12.2.1 is that the d (m) -transformation can


k1
with k = 0, k = 0, 1, . . . , m.
be applied to the series
k=1 ck z
( )
2. Recall that if cn = h(n) with h A0 for some , and an = cn z n1 , then {an }
(0)
b(1) with an = p(n)an , p A0 as long as z = 1 while p A(1)
0 when z = 1. As


k1
c
z
converges
to
a function that is
mentioned in Chapter 6, if |z| < 1,
k
k=1
analytic for |z| < 1, and this function is singular at z = 1. Therefore, we conjecture
that the zeros of the polynomial r0 (z) in the proof of Theorem 12.2.1 are points of


k1
if this function has
singularity of the function f (z) that is represented by
k=1 ck z
singularities in the nite plane. We invite the reader to verify this claim for the case


k1
being
in which cn = n /n + n , = , the function represented by
k=1 ck z
1
1
z log(1 z) + (1 z) , with singularities at z = 1/ and z = 1/.

240

12 Acceleration of Convergence of Power Series by the d-Transformation

12.3 Rational Approximations from the d-Transformation

k1
assuming that
We now apply the d (m) -transformation to the innite series
k=1 ck z


k1
has a positive radius of convergence .
{cn } is as in Theorem 12.2.1 and that k=1 ck z


(n)k an with pk A(0)
Now, with an = cn z n1 , we have {an } b(m) and an = m
k=1 pk

0
k1
for all k, by Theorem 12.2.1. Therefore, when |z| < and hence
converges,
k=1 ck z
Theorem 6.1.12 holds, and we have the asymptotic expansion
An1 A +

m1


(k an )

k=0


gki
as n ,
ni
i=0

(12.3.1)

where Ar = rk=1 ak and A is the sum of
k=1 ak . Invoking (6.1.5), we can rewrite
(12.3.1) in the form
An1 A +

m1


an+k

k=0



gki
as n ,
ni
i=0

(12.3.2)

and redene the d (m) -transformation by the linear systems

A Rl 1 = dn(m, j) +

m


a Rl +k1

n
k 1

k=1

i=0

m

ki
, j l j + N; N =
n k , (12.3.3)
i
Rl
k=1
(m, j)

where n = (n 1 , . . . , n m ) as before and A0 0. Also, d(0,... ,0) = A j for all j.

12.3.1 Rational d-Approximants

From this denition of the d (m) -transformation, we can derive the following interesting
result:
(m, j)

Theorem 12.3.1 Let dn

is,

Al = dn(m, j) +

be as dened in (12.3.3) with Rl = l + 1, l = 0, 1, . . . , that

m


al+k

k=1
(m, j)

Then, dn

(m, j)

dn

n
k 1
i=0

ki
, l = j, j + 1, . . . , j + N .
(l + 1)i

(12.3.4)

(z) is of the form

dn(m, j) (z)
( j)

u(z)
=
=
v(z)

N

N i
A j+i
i=0 ni z

N ( j) N i
i=0 ni z
( j)

(12.3.5)

(m, j)

for some constants ni . Thus, dn

is a rational function in z, whose numerator and
denominator polynomials u(z) and v(z) are of degree at most j + N 1 and N , respectively.
(m, j)

to the sum of
Remark. In view of Theorem 12.3.1, we call the approximations dn


k1
a
z
(limit
or
antilimit
of
{A
})
that
are
dened
via
the
equations
in (12.3.4)
k
k
k=1
rational d-approximants.

12.3 Rational Approximations from the d-Transformation

241

Proof. Let us substitute an = cn z n1 in (12.3.4) and multiply both sides by z j+N l . This
results in the linear system
z j+N l Al = z j+N l dn(m, j) +

m


cl+k

k=1

n
k 1
i=0

ki
, l = j, j + 1, . . . , j + N ,
(l + 1)i
(12.3.6)

where ki = z j+N +k1 ki for all k and i are the new auxiliary unknowns. Making dn
the last component of the vector of unknowns, the matrix of this system assumes the
form
 N
 z

 z N 1

M(z) = M0  ..
, M0 : (N + 1) N and independent of z, (12.3.7)
 .
 z0
(m, j)

and the right-hand side is the vector

[z N A j , z N 1 A j+1 , . . . , z 0 A j+N ]T .
The result in (12.3.5) follows if we solve (12.3.6) by Cramers rule and expand the
resulting numerator and denominator determinants with respect to their last columns.
( j)
For each i, we can take ni to be the cofactor of z N i in the last column of M(z) in
(12.3.7).


( j)

(m, j)

is known completely. The

As is clear from (12.3.5), if the ni are known, then dn
( j)
ni can be determined in a systematic way as shown next in Theorem 12.3.2. Note that
( j)
the ni are unique up to a common multiplicative factor.
( j)

Theorem 12.3.2 Up to a common multiplicative factor, the coefcients ni of the de(m, j)

in (12.3.5) satisfy the linear system
nominator polynomial v(z) of dn
[M0 |w]T = e1 ,
( j)

( j)

(12.3.8)

( j)

where M0 is as in (12.3.7), = [n0 , n1 , . . . , n N ]T , and e1 = [1, 0, . . . , 0]T , and w

is an arbitrary column vector for which [M0 |w] is nonsingular.
Proof. Since ni can be taken as the cofactor of z N i in the last column of the matrix
M(z), we have that satises the N (N + 1) homogeneous system M0T = 0, namely,
( j)

N


( j)

ni c j+k+i /( j + i + 1)r = 0, 0 r n k 1, 1 k m.

(12.3.9)

i=0

Appending to this system the scaling w T = 1, we obtain (12.3.8).

12.3.2 Closed-Form Expressions for m = 1

(m, j)

Using the results of Section 6.3, we can give closed-form expressions for dn (z) when
(1, j)
m = 1. The d (1) -transformation now becomes the Levin L-transformation and dn (z)

242

12 Acceleration of Convergence of Power Series by the d-Transformation

( j)

becomes Ln (z). By (6.3.2) and (6.3.3), we have

n
 
( j) ni
n1
A j+i
( j)
i=0 ni z
( j)
i n ( j + i + 1)
; ni = (1)
. (12.3.10)
Ln (z) =
n
( j) ni
c j+i+1
i
i=0 ni z
A similar expression can be given with the d (1) -transformation replaced by the Sidi
S-transformation. By (6.3.4) and (6.3.5), we have

n
 
( j) ni
A j+i
( j)
i=0 ni z
( j)
i n ( j + i + 1)n1
;

=
(1)
. (12.3.11)
Sn (z) =

ni
( j) ni
n
i
c j+i+1
i=0 ni z
12.4 Algebraic Properties of Rational d-Approximants
The rational d-approximants of the preceding section have a few interesting algebraic
properties to which we now turn.
12.4.1 Pade-like Properties
(m, j)
dn

(m, j)

be the rational function dened via (12.3.4). Then d(1,... ,1) is

Theorem 12.4.1 Let


k1
.
the [ j + m 1/m] Pade approximant from the power series
k=1 ck z
Proof. Letting n 1 = = n m = 1 in (12.3.4), the latter reduces to
(m, j)

Al = d(1,... ,1) +

m


k al+k , l = j, j + 1, . . . , j + m,

(12.4.1)

k=1

and the result follows from the denition of the Pade approximants that are considered
later.


(m, j)

Our next result concerns not only the rational function dn (z) but all those rational
functions that are of the general form given in (12.3.5). Because the proof of this result
is straightforward, we leave it to the reader.
Theorem 12.4.2 Let A0 = 0 and Ar =
rational function of the form

r

U (z)
=
R(z) =
V (z)

k=1 ck z

k1

, r = 1, 2, . . . , and let R(z) be a

s
si
Aq+i
i=0 i z


.
s
si

z
i=0 i

(12.4.2)

Then
V (z)

ck z k1 U (z) = O(z q+s ) as z 0.

(12.4.3)

k=1

If s = 0 in addition, then


k=1

ck z k1 R(z) = O(z q+s ) as z 0.

(12.4.4)

12.4 Algebraic Properties of Rational d-Approximants

243

It is clear from (12.3.5) that Theorem 12.4.2 applies with q = j, s = N , and

(m, j)
R(z) = dn . A better result is possible, again with the help of Theorem 12.4.2, and this
is achieved in Theorem 12.4.3. The improved result of this theorem exhibits a Pade-like
(m, j)
property of dn . To help understand this property, we pause to give a one-sentence definition of Pade approximants: the [M/N ] Pade approximant f M,N (z) to the formal power


k1
, when it exists, is the rational function U (z)/V (z), with U (z)
series f (z) :=
k=1 ck z
and V (z) being polynomials of degree at most M and N , respectively, with V (0) = 1,
such that f (z) f M,N (z) = O(z M+N +1 ) as z 0. As we show later, f M,N (z) is determined (uniquely) by the terms c1 , c2 , . . . , c M+N +1 of f (z). Thus, all Pade approximants
R(z) to f (z) that are dened by the terms c1 , c2 , . . . , c L satisfy f (z) R(z) = O(z L )
(m, j)
we are alluding to, which we
as z 0. Precisely this is the Pade-like property of dn
prove next. Before we do that, we want to point out that Pade approximants, like rational
d-approximants, are of the form of R(z) in (12.4.2) with appropriate q and s.
(m, j)

Theorem 12.4.3 Let dn

be as in Theorem 12.3.1. Then

N ( j) N i
ni z
A j+m+i
(m, j)
dn (z) = i=0
,

N ( j) N i
i=0 ni z

(12.4.5)

( j)

with ni exactly as in (12.3.5) and as described in the proof of Theorem 12.3.1. Therefore,
v(z)

ck z k1 u(z) = O(z j+N +m ) as z 0.

(12.4.6)

k=1
( j)

If n N = 0 in addition, then

ck z k1 dn(m, j) (z) = O(z j+N +m ) as z 0.

(12.4.7)

k=1

Proof. We begin with the linear system in (12.3.4). If we add the terms al+1 , . . . , al+m
to both sides of (12.3.4), we obtain the system
Al+m = dn(m, j) +

m


al+k

k=1

ki

n
k 1
i=0


ki
, l = j, j + 1, . . . , j + N , (12.4.8)
(l + 1)i


k0

(m, j)
= ki , i = 0, and
= k0 + 1, k = 1, . . . , m, and dn
remains unwhere
changed. As a result, the system (12.3.6) now becomes

z j+N l Al+m = z j+N l dn(m, j) +

m

k=1

cl+k

n
k 1
i=0

ki
, l = j, j + 1, . . . , j + N ,
(l + 1)i
(12.4.9)


where ki = z j+N +k1 ki for all k and i are the new auxiliary unknowns, and (12.3.5)
( j)
becomes (12.4.5) with ni exactly as in (12.3.5) and as described in the proof of Theorem 12.3.1.
(m, j)
With (12.4.5) available, we now apply Theorem 12.4.2 to dn
with s = N and
q = j + m.

244

12 Acceleration of Convergence of Power Series by the d-Transformation

(m, j)

The reader may be led to think that the two expressions for dn (z) in (12.3.5) and
(12.4.5) are contradictory. As we showed in the proof of Theorem 12.4.3, they are not.
We used (12.3.5) to conclude that the numerator and denominator polynomials u(z) and
(m, j)
v(z) of dn (z) have degrees at most j + N 1 and N , respectively. We used (12.4.5) to
(m, j)
conclude that dn , which is obtained from the terms ck , 1 k j + N + m, satises
( j,m)
(12.4.6) and (12.4.7), which are Pade-like properties of dn .

12.4.2 Recursive Computation by the W(m) -Algorithm

(m, j)

satises the linear system in (12.3.6), the W(m) Because the approximation dn
( j)
algorithm can be used to compute the coefcients ni of the denominator polynomials
v(z) recursively, as has been shown in Ford and Sidi [87]. [The coefcients of the numerator polynomial u(z) can then be determined by (12.3.5) in Theorem 12.3.1.]
Let us rst write the equations in (12.3.6) in the form
z l Al = z l dn(m, j) +

m


cl+k

n
k 1

k=1

i=0

ki
, l = j, j + 1, . . . , j + N . (12.4.10)
(l + 1)i

For each l, let tl = 1/(l + 1), and dene the gk (l) = k (tl ) in the normal ordering
through [cf. (7.3.3)]
gk (l) = k (tl ) = cl+k , k = 1, . . . , m,
gk (l) = tl gkm (l), k = m + 1, m + 2, . . . .
( j)

( j)

( j)

(12.4.11)

( j)

Dening G p , D p , f p (b), and p (b) as in Section 3.3 of Chapter 3, with the present
(m, j)
is given by
gk (l), we realize that dn
( j)

dn(m, j) =

N ( )
( j)

N ()

(12.4.12)

where
n k = (N k)/m, k = 1, . . . , m, and N =

m


nk ,

(12.4.13)

k=1

and
(l) = z l Al and (l) = z l .

(12.4.14)

Since N () = f N ()/G N +1 and f N () is a polynomial in z 1 of degree N and G N +1

is independent of z, we have that
( j)

( j)

( j)

( j)

( j)

( j)

N () =

N


N i z ji ,
( j)

(12.4.15)

i=0
( j)

( j)

where N i = ni , i = 0, 1, . . . , N , up to scaling. By the fact that

( j+1)

p( j) () =

( j)

p1 () p1 ()
( j)

Dp

(12.4.16)

12.5 Prediction Properties of Rational d-Approximants

( j)

245

( j)

with D p independent of z, we see that the pi can be computed recursively from

( j+1)

( j)
pi

( j)

p1,i1 p1,i
( j)

Dp

, i = 0, 1, . . . , p,

(12.4.17)

( j)

with pi = 0 when i < 0 and i > p. It is clear from (12.4.17) that the W(m) -algorithm
( j)
is used only to obtain the D p .

12.5 Prediction Properties of Rational d-Approximants

So far we have used extrapolation methods to approximate (or predict) the limit or
antilimit S of a given sequence {Sn } from a nite number of the Sn . We now show
how they can be used to predict the terms Sr +1 , Sr +2 , . . . , when S1 , S2 , . . . , Sr are
given. An approach to the solution of this problem was rst formulated in Gilewicz


k1
[98], where the Maclaurin expansion of a Pade approximant f M,N (z) from
k=0 ck z
is used to approximate the coefcients ck , k M + N + 2. [Recall that f M,N (z) is

M+N +1
ck z k1 +
determined only by ck , k = 1, . . . , M + N + 1, and that f M,N (z) = k=1


k1
.] The approach we describe here with the help of the rational dk=M+N +2 ck z
approximants is valid for all sequence transformations and was rst given by Sidi and
Levin [312], [313]. It was applied recently by Weniger [355] to some known sequence
transformations that naturally produce rational approximations when applied to partial
sums of power series. (For different approaches to the issue of prediction, see Brezinski
and Redivo Zaglia [41, pp. 390] and Vekemans [345].)
Let c1 = S1 , ck = Sk Sk1 , k = 2, 3, . . . , and assume that {cn } is as in Section
12.2. By Theorem 12.2.1, we have that {cn z n1 } b(m) , so we can apply the d (m)

transformation to the sequence {Ar }, where Ar = rk=1 ck z k1 , to obtain the rational


( j)
(m, j)
d-approximant R N (z) dn , with n = (n 1 , . . . , n m ) and N = m
k=1 n k as before.


( j)
k1
c
z
, its Maclaurin
Because R N (z) is a good approximation to the sum of
k=1 k

k1
k1

is likely to be close to k=1 ck z

for small z. Indeed, from
expansion k=1 ck z
where k = j + N + m.
Theorem 12.4.3, we already know that ck = ck , k = 1, 2, . . . , k,
Thus,


k=1

( j)

ck z k1 R N (z) =

k z k1 ; k = ck ck , k k + 1,

(12.5.1)

k=k+1

and we expect at least the rst few of the k , k k + 1, to be very small. In case this
happens, we will have that for the rst few values of k k + 1, ck will be very close to

k
ci . Precisely this is the prediction
the corresponding ck , and hence Sk Sk + i=

k+1
property we alluded to above.
For a theoretical justication of the preceding procedure, see Sidi and Levin [312],
where the prediction properties of the rational d (m) - and Pade approximants are compared
by a nontrivial example. For a thorough and rigorous treatment of the case in which
m = 1, see Sidi and Levin [313], who provide precise rates of convergence of the ck
to the corresponding ck , along with convincing theoretical and numerical examples.
The results of [313] clearly show that ck ck very quickly under both Process I and
Process II.

246

12 Acceleration of Convergence of Power Series by the d-Transformation

( j)

Finally, it is clear that the rational function R N (z) can be replaced by any other
approximation (not necessarily rational) that has Pade-like properties.

12.6 Approximation of Singular Points

Assume that the series k=1 ck z k1 has a positive radius of convergence so that it is the
Maclaurin expansion of a function f (z) that is analytic for |z| < . In case is nite, f (z)
has singularities on |z| = . It may have additional singularities for |z| > . It has been
observed numerically that some or all of the poles of the rational functions obtained by
applying sequence transformations to the power series of f (z) serve as approximations
to the points of singularity of f (z). This is so for rational d-approximants and Pade
approximants as well. Sidi and Levin [312] use the poles of the rational d (m) - and Pade
approximants from the power series of a function f (z) that has both polar and branch
singularities to approximate these singular points.
In the case of Pade approximants, the generalized Koenig theorem, which is discussed
in Chapter 17, provides a theoretical justication of this use.
To clarify this point, we consider two examples of {cn } b(1) under Process I. In
(1, j)
(which are also Pade approximants)
particular, we consider the approximations d1


k1
that are given by
from k=1 ck z
(1, j)

d1
(1, j)

Thus, each d1

(z) =

c j+1 z A j c j A j+1
.
c j+1 z c j

(z) has one simple pole at z ( j) = c j /c j+1 .

( )

Example 12.6.1 When cn = h(n) A0 strictly, = 0, we have {cn } b(1) . As men

k1
converges for |z| < 1 and represents a function
tioned earlier, the series
k=1 ck z
analytic for |z| < 1 with a singularity at z = 1. We have
cj
c j+1

= 1 j 1 + O( j 2 ) as j .

(1, j)

That is, z ( j) , the pole of d1

have made.

(z), satises lim j z ( j) = 1, thus verifying the claim we

Example 12.6.2 When cn = (n!) h(n), where is a positive integer and h A0

k1
strictly for some , we have {cn } b(1) . In this case, the series
converges
k=1 ck z
for all z and represents a function that is analytic everywhere with singularities only at
innity. We have
( )

cj
c j+1
(1, j)

That is, z ( j) , the pole of d1

made.

j as j .

(z), satises lim j z ( j) = , verifying the claim we have

12.7 Efcient Application of the d-Transformation to Power Series with APS 247
12.7 Efcient Application of the d-Transformation to Power Series with APS


k1
have a positive but nite radius of convergence. As
Let the innite series
k=1 ck z
mentioned earlier, this series is the Maclaurin expansion of a function f (z) that is analytic
for |z| < and has singularities for some z with |z| = and possibly with |z| > as
well.
When a suitable convergence acceleration method is applied directly to the se

quence of the partial sums Ar = rk=1 ck z k1 , accurate approximants to f (z) can be
obtained as long as z is not close to a singularity of f (z). If z is close to a singularity, however, we are likely to face severe stability and accuracy problems in applying
convergence acceleration. In case the d (m) -transformation is applicable, the following
strategy involving APS (discussed in Section 10.3) has been observed to be very effective in coping with the problems of stability and accuracy: (i) For z not close to a
singularity, choose Rl = l + 1, l = 0, 1, . . . . (ii) For z close to a singularity, choose
Rl = (l + 1), l = 0, 1, . . . , where is a positive integer 2; the closer z is to a singularity, the larger should be. The d (1) -transformation with APS was used successfully
by Hasegawa [126] for accelerating the convergence of some slowly converging power
series that arise in connection with a numerical quadrature problem.
This strategy is not ad hoc by any means, and its theoretical justication has been


k1
given in [294, Theorems 4.3 and 4.4] for the case in which {cn } b(1) and
k=1 ck z
has a positive but nite radius of convergence so that f (z), the limit or antilimit of


k1
, has a singularity as in Example 12.6.1. Recall that the sequence of partial
k=1 ck z
sums of such a series is linear. We now present the mathematical treatment of APS that
was given by Sidi [294]. The next theorem combines Theorems 4.24.4 of [294].
( )

Theorem 12.7.1 Let cn = n h(n), where is some scalar and h A0 for some , and
and may be complex in general. Let z be such that |z| 1 and z = 1. Denote by


k1
f (z) the limit or antilimit of
. Then,
k=1 ck z
An = f (z) + cn z n g(n), g A(0)
(12.7.1)
0 strictly.


k1
If we apply the d (1) -transformation to the power series
with Rl = (l +
k=1 ck z
1), l = 0, 1, . . . , where is some positive integer, then
dn(1, j) f (z) = O(c j z j j 2n ) as j ,

(12.7.2)

an asymptotically optimal version of which reads

dn(1, j) f (z)

n ( + 1)n
gn+ c( j+1) z ( j+1) j 2n as j , (12.7.3)
(1 )n

where gn+ is the rst nonzero gi with i n in the asymptotic expansion g(n)


i
as n , and
i=0 gi n
= (z) ,
and


n( j)

1 + | |
|1 |

(12.7.4)

n
as j ,

(12.7.5)

248

12 Acceleration of Convergence of Power Series by the d-Transformation

(1, j)

is the solution to the equations A Rl =

provided, of course, that = 1. [Here dn

n1
(1, j)
i
Rl

dn + c Rl z
i=0 i /Rl , l = j, j + 1, . . . , j + N .]
Proof. The validity of (12.7.1) is already a familiar fact that follows from Theorem 6.7.2.
Because cn z n = (z)n h(n) = en log(z) h(n), (12.7.1) can be rewritten in the form
A(t) = A + (t)B(t), where t = n 1 , A(t) An , (t) = eu(t) t H (t) with u(t) =
log(z)/t, = , H (t) n h(n), and B(t) g(n).
We also know that applying the d-transformation is the same as applying GREP with
tl = 1/Rl . With Rl = (l + 1), these tl satisfy (9.1.4) with c = 1/ and p = q = 1 there.
From these observations, it is clear that Theorem 9.3.1 applies with =


exp( log(z)) = (z) , and this results in (12.7.2)(12.7.5).
From Theorem 12.7.1, it is clear that the d (1) -transformation accelerates the conver

k1
:
gence of the series
k=1 ck z

 (1, j)
 dn f (z) 
 = O( j 2n ) as j .
max 
0in A( j+i) f (z) 
We now turn to the explanation of why APS with the choice Rl = (l + 1) guarantees
more stability and accuracy in the application of the d (1) -transformation to the preceding
power series.
(1, j)
( j)
( j)
is determined by n . As n
As we already know, the numerical stability of dn
(1, j)
( j)
becomes large, dn
becomes less stable. Therefore, we should aim at keeping n
( j)
close to 1, its lowest possible value. Now, by (12.7.5), lim j n is proportional to
|1 |n , which, from = (z) , is unbounded as z 1 , the point of singularity
of f (z). Thus, if we keep xed ( = 1 say) and let z get close to 1 , then gets
close to 1, and this causes numerical instabilities in acceleration. On the other hand, if
we increase , we cause = (z) to separate from 1 in modulus and/or in phase so
( j)
that |1 | increases, thus causing n to stay bounded. This provides the theoretical
justication for the introduction of the integer in the choice of the Rl . We can even see
that, as z approaches the point of singularity 1 , if we keep = (z) approximately
xed by increasing gradually, we can maintain an almost xed and small value for
( j)
n .
(1, j)
which, by (12.7.3), can be written in the form
We now look at the error in dn
(1, j)
dn f (z) K n c( j+1) z ( j+1) j 2n as j . The size of K n gives a good indica( j)
tion of whether the acceleration is effective. Surprisingly, K n , just as lim j n , is
n
1
proportional to |1 | , as is seen from (12.7.3). Again, when z is close to , the
point of singularity of f (z), we can cause K n to stay bounded by increasing . It is thus
interesting that, by forcing the acceleration process to become stable numerically, we
(1, j)
are also preserving the quality of the theoretical error dn f (z).
If z is real and negative, then it is enough to take = 1. This produces excellent
( j)
results and n 1 as j . If z is real or complex and very close to 1, then we
need to take larger values of . In case {cn } is as in Theorem 12.7.1, we can use cn+1 /cn
as an estimate of , because limn cn+1 /cn = . On this basis, we can take (cn+1 /cn )z
to be an estimate of z and decide on an appropriate value for .

12.8 The d-Transformation on Factorial Series

249

k1
Table 12.7.1: Effect of APS with the d (1) -transformation on the series
/k.
k=1 z
(1,0)
The relevant Rl are chosen as Rl = (l + 1). Here, E n (z) = |dn (z) f (z)|,
(1, j)
(1, j)
and the dn (z) are the computed dn (z). Computations have been
done in quadruple-precision arithmetic
s

1
2
3
4
5
6
7

0.5
0.51/2
0.51/3
0.51/4
0.51/5
0.51/6
0.51/7

E n (z) smallest error

4.97D 30 (n
1.39D 25 (n
5.42D 23 (n
1.41D 21 (n
1.31D 19 (n
1.69D 18 (n
1.94D 17 (n

= 28)
= 35)
= 38)
= 41)
= 43)
= 43)
= 44)

E 28 (z) with = 1

E 28 (z) with = s

4.97D 30
5.11D 21
4.70D 17
1.02D 14
3.91D 13
5.72D 12
4.58D 11

4.97D 30
3.26D 31
4.79D 31
5.74D 30
1.59D 30
2.60D 30
4.72D 30

We end this section by mentioning that we can replace the integer by an arbitrary
real number 1, and change Rl = (l + 1), l = 0, 1, . . . , to
Rl = (l + 1), l = 0, 1, . . . ,

(12.7.6)

[cf. (10.2.2)]. It is easy to show that, with 1, there holds R0 < R1 < R2 < ,
and Rl l as l . Even though Theorem 12.7.1 does not apply to this case, the
numerical results obtained by the d-transformation with such values of appear to be
just as good as those obtained with integer values.
Example 12.7.2 We illustrate the advantage of APS by applying it to the power se

k1
/k whose sum is f (z) = z 1 log(1 z) when |z| 1, z = 1. The d (1) ries
k=1 z
transformation can be used in summing this series because {1/n} b(1) . Now, f (z) has
a singularity at z = 1. Thus, as z approaches 1 it becomes difcult to preserve numerical
stability and good convergence if the d (1) -transformation is used with Rl = l + 1. Use of
APS, however, improves things substantially, precisely as explained before. We denote
(1, j)
(1, j)
by dn (z) also the dn with APS.
In Table 12.7.1, we present the results obtained for z = 0.51/s , s = 1, 2, . . . . The computations have been done in quadruple-precision arithmetic. Let us denote the computed
(1, j)
(1, j)
dn (z) by dn (z) and set E n (z) = |d(1,0)
(z) f (z)|. Then, the d(1,0)
(z) associated with
n
n
the E n (z) in the third column seem to have the highest accuracy when = 1 in APS. The
results of the third and fth columns together clearly show that, as z approaches 1, the
(z), with n xed or otherwise, declines. The results of
quality of both dn(1,0) (z) and d(1,0)
n
the last column, on the other hand, verify the claim that, by applying APS with chosen
such that z remains almost xed, the quality of dn(1,0) (z), with n xed, is preserved.
1/s
In fact, the d(1,0)
28 (0.5 ) with = s turn out to be almost the best approximations in
quadruple-precision arithmetic and are of the same size for all s.

12.8 The d-Transformation on Factorial Series

Our next result concerns the application of the d-transformation to innite power series
whose partial sums form factorial sequences. This result essentially forms part (iii) of
Theorem 19.2.3, and Theorem 6.7.2 plays an important role in its proof.

250

12 Acceleration of Convergence of Power Series by the d-Transformation

(0)
(z) = f s,s (z) for
Table 12.9.1: Smallest errors in dn(m,0) (z), n = (, . . . , ), and 2s
r
the functions f (z) with z on the circles of convergence. The rst m( + 1) of the
(m,0)
coefcients ck are used in constructing d(,...
,) (z), and 2s + 1 coefcients are used in
(0)
constructing 2s (z). Computations have been done in quadruple-precision arithmetic

z = ei/6
z = 12 ei/6
z = 13 ei/6

|dn(1,0) (z) f 1 (z)| = 1.15D 24 ( = 42)

|dn(2,0) (z) f 2 (z)| = 1.96D 33 ( = 23)
|dn(4,0) (z) f 4 (z)| = 5.54D 33 ( = 20)

(0)
|160
(z) f 1 (z)| = 5.60D 34
(0)
|26 (z) f 2 (z)| = 4.82D 14
(0)
|26
(z) f 4 (z)| = 1.11D 09

Theorem 12.8.1 Let cn = [(n 1)!]r n h(n), where r is a positive integer, is some
( )
scalar, and h A0 for some , and and may be complex in general. Denote by


f (z) the limit of k=1 ck z k1 . Then
(12.8.1)
An = f (z) + n r cn z n g(n), g A(0)
0 strictly.


k1
with Rl = l + 1,
If we apply the d (1) -transformation to the power series
k=1 ck z
then

O(a j+n+1 j 2n ) as j , n r + 1,
(12.8.2)
dn(1, j) f (z) =
O(a j+n+1 j r 1 ) as j , n < r + 1,
and
n( j) 1 as j .

(12.8.3)

The d (1) -transformation of this theorem is nothing but the L-transformation, as mentioned earlier.

12.9 Numerical Examples

In this section, we illustrate the effectiveness of the d-transformation in accelerating


k1
such that {ck } b(m) for various
the convergence of power series f (z) :=
k=1 ck z
values of m. We compare the results obtained from the d-transformation with those
(m,0)

obtained from the Pade table. In particular, we compare the sequences {d(,...
,) (z)}=0

with the sequences { f N ,N (z)} N =0 of Pade approximants that have the best convergence
(m,0)
properties for all practical purposes. We have computed the d(,...
,) (z) with the help
of the computer code given in Appendix I after modifying the latter to accommodate
(m,0)
complex arithmetic. Thus, the equations that dene the d(,...
,) (z) are
(m,0)
Al = d(,...
,) (z) +

m

k=1

l k k1 al

1

ki
i=0

li

, l = 1, 2, . . . , m + 1.

Here, ak = ck z k1 for all k, as before. The f N ,N (z) have been computed via the algorithm of Wynn to which we come in Chapters 16 and 17. [We mention only that
( j)
the quantities k generated by the -algorithm on the sequence of the partial sums


( j)
k1
are related to the Pade table via 2s = f j+s,s (z).] In other words, we
of k=1 ck z

12.9 Numerical Examples

251

(0)
(z) = f s,s (z) for
Table 12.9.2: Smallest errors in dn(m,0) (z), n = (, . . . , ), and 2s
r
the functions f (z) with z outside the circles of convergence. The rst m( + 1) of the
(m,0)
coefcients ck are used in constructing d(,...
,) (z), and 2s + 1 coefcients are used in
(0)
constructing 2s (z). Computations have been done in quadruple-precision arithmetic

z = 32 ei/6
z = 34 ei/6
z = 12 ei/6

|dn(1,0) (z) f 1 (z)| = 6.15D 18 ( = 44)

|dn(2,0) (z) f 2 (z)| = 7.49D 27 ( = 26)
|dn(4,0) (z) f 4 (z)| = 1.62D 22 ( = 20)

(0)
|200
(z) f 1 (z)| = 4.25D 20
(0)
|26 (z) f 2 (z)| = 1.58D 09
(0)
|26
(z) f 4 (z)| = 1.48D 06

have not computed our approximations explicitly as rational functions. To have a good
comparative picture, all our computations were done in quadruple-precision arithmetic.
As our test cases, we chose the following three series:
f 1 (z) :=

z k1 /k

k=1


[1 (2)k ]z k1 /k,
f 2 (z) :=
k=1

f 4 (z) :=


[1 (2)k + (3i)k (3i)k ]z k1 /k.
k=1

The series f 1 (z), f 2 (z), and f 4 (z) converge for |z| < , where = 1, = 1/2, and
= 1/3, respectively, and represent functions that are analytic for |z| < . Denoting
these functions by f 1 (z), f 2 (z), and f 4 (z) as well, we have specically
f 1 (z) = z 1 log(1 z), |z| < 1,
f 2 (z) = z 1 [ log(1 z) + log(1 + 2z)], |z| < 1/2,
f 4 (z) = z 1 [ log(1 z) + log(1 + 2z) log(1 3iz) + log(1 + 3iz)], |z| < 1/3.
In addition, each series converges also when |z| = , except when z is a branch point of
the corresponding limit function. The branch points are at z = 1 for f 1 (z), at z = 1, 1/2
for f 2 (z), and at z = 1, 1/2, i/3 for f 4 (z).
As {ck } is in b(1) for f 1 (z), in b(2) for f 2 (z), and in b(4) for f 4 (z), we can apply the d (1) -,
(2)
d -, and d (4) -transformations to approximate the sums of these series when |z| but
z is not a branch point.
Numerical experiments suggest that both the d-approximants and the Pade approximants continue the functions f r (z) analytically outside the circles of convergence of
their corresponding power series f r (z). The functions f r (z) are dened via the principal
values of the logarithms log(1 z) involved. [The principal value of the logarithm of
is given as log = log | | + i arg , and < arg . Therefore, log(1 z) has
its branch cut along the ray ( 1 , ei arg ).] For the series f 1 (z), this observation is
consistent with the theory of Pade approximants from Stieltjes series, a topic we discuss
in Chapter 17. It is consistent also with the ndings of Example 4.1.8 and Theorem 6.8.8.
In Table 12.9.1, we present the best numerical results obtained from the dtransformation and the -algorithm with z on the circles of convergence. The partial

252

12 Acceleration of Convergence of Power Series by the d-Transformation

Table 12.9.3: Smallest errors in dn(m,0) (z), n = (, . . . , ), for
the functions f r (z) with z on the circles of convergence and close
to a branch point, with and without APS. Here Rl = (l + 1).
The rst (m + 1) + m 1 of the coefcients ck are used in
(m,0)
constructing d(,...
,) (z). Computations have been done
in quadruple-precision arithmetic
z = ei 0.05

|dn(1,0) (z) f 1 (z)| = 1.67D 16 ( = 1, = 48)

|dn(1,0) (z) f 1 (z)| = 2.88D 27 ( = 5, = 37)

z = 12 ei 0.95

|dn(2,0) (z) f 2 (z)| = 9.85D 19 ( = 1, = 37)

|dn(2,0) (z) f 2 (z)| = 6.99D 28 ( = 9, = 100)

z = 13 ei 0.45

|dn(4,0) (z) f 4 (z)| = 8.23D 19 ( = 1, = 33)

|dn(4,0) (z) f 4 (z)| = 8.86D 29 ( = 3, = 39)

sums converge extremely slowly for such z. The -algorithm, although very effective for
f 1 (z), appears to suffer from stability problems in the case of f 2 (z) and f 4 (z).
As Table 12.9.1 shows, the best result that can be obtained from the d (1) -transformation
for f 1 (z) with z = ei/6 has 24 correct digits. By using APS with Rl = 3(l + 1), we can
(1,0)
(z) f 1 (z)| = 3.15 1032 for this z,
achieve almost machine accuracy; in fact, |d33
and the number of terms used for this is 102.
Table 12.9.2 shows results obtained for z outside the circles of convergence. As we
increase , the results deteriorate. The reason is that the partial sums diverge quickly,
and hence the oating-point errors in their computation diverge as well.
From both Table 12.9.1 and Table 12.9.2, the d-transformations appear to be more
effective than the Pade approximants for these examples.
Finally, in Table 12.9.3, we compare results obtained from the d-transformation with
and without APS when z is on the circles of convergence and very close to a branch
point. Again, the partial sums converge extremely slowly. We denote the approximations
(m, j)
(m, j)
with APS also dn (z).
dn
For further examples, we refer the reader to Sidi and Levin [312].

13
Acceleration of Convergence of Fourier and Generalized
Fourier Series by the d-Transformation: The Complex
Series Approach with APS

13.1 Introduction
In this chapter, we extend the treatment we gave to power series in the preceding chapter
to Fourier series and their generalizations, whether convergent or divergent. In particular,
we are concerned with Fourier cosine and sine series, orthogonal polynomial expansions,
series that arise from SturmLiouville problems, such as FourierBessel series, and other
general special function series.
Several convergence acceleration methods have been used on such series, with limited
success. An immediate problem many of these methods face is that they do not produce
any acceleration when applied to Fourier and generalized Fourier series. The transformations of Euler and of Shanks discussed in the following chapters and the d-transformation
are exceptions. See the review paper by Smith and Ford [318] and the paper by Levin
and Sidi [165]. With those methods that do produce acceleration, another problem one
faces in working with such series is the lack of stability and acceleration near points of
singularity of the functions that serve as limits or antilimits of these series. Recall that
the same problem occurs in dealing with power series.
In this chapter, we show how the d-transformation can be used effectively to accelerate
the convergence of these series. The approach we are about to propose has two main
ingredients that can be applied also with some of the other sequence transformations.
(This subject is considered in detail later.) The rst ingredient involves the introduction
of what we call functions of the second kind. This decreases the cost of acceleration to
half of what it would be if extrapolation were applied to the original series. The second
ingredient involves the use of APS with Rl = (l + 1), where > 1 is an integer, near
points of singularity, as was done in the case of power series in the preceding chapter.
[We can also use APS by letting Rl = (l + 1), where > 1 and is not an integer
necessarily, as we suggested at the end Section 12.7.]
The contents of this chapter are based on the paper by Sidi [294]. As the approach
we describe here has been illustrated with several numerical examples [294], we do not
include any examples here, and refer the reader to Sidi [294]. See also the numerical
examples given by Levin and Sidi [165, Section 7] that are precisely of the type we treat
here.

253

254 13 Acceleration of Convergence of Fourier and Generalized Fourier Series

13.2 Introducing Functions of Second Kind
13.2.1 General Background
Assume we want to accelerate the convergence of an innite series F(x) given in the
form
F(x) :=

[bk k (x) + ck k (x)] ,

(13.2.1)

k=1

where the functions n (x) and n (x) satisfy

n (x) n (x) in (x) = einx gn (x),

(13.2.2)

being some xed real positive constant, and

gn (x) n 

j (x)n j as n ,

(13.2.3)

j=0

for some xed  that can be complex in general. (Here i = 1 is the imaginary unit,
as usual.) In other words, gn+ (x) and gn (x), as functions of n, are in A()
0 .

From (13.2.2) and (13.2.3) and Example 6.1.11, it is clear that {n (x)} b(1) . By the
fact that
n (x) =



1 +
1  +
(x) + n (x) and n (x) =
(x) n (x) ,
2 n
2i n

(13.2.4)

and by Theorem 6.8.7, we also have that {n (x)} b(2) and {n (x)} b(2) exactly in
accordance with Denition 6.1.2, as long as eix = 1.
The simplest and most widely treated members of the series above are the classical
Fourier series
F(x) :=

(bk cos kx + ck sin kx) ,

k=0

for which n (x) = cos nx and n (x) = sin nx, so that n (x) = einx and hence
gn (x) 1, n = 0, 1, . . . . More examples are provided in the next section.
In general, n (x) and n (x) may be (some linear combinations of) the nth eigenfunction of a SturmLiouville problem and the corresponding second linearly independent
solution of the relevant O.D.E., that is, the corresponding function of the second kind. In


most cases of interest, cn = 0, n = 1, 2, . . . , so that F(x) :=
k=1 bk k (x).

13.2.2 Complex Series Approach

The approach we propose for accelerating the convergence of the series F(x), assuming
that the coefcients bn and cn are known, is as follows:
1. Dene the series B (x) and C (x) by
B (x) :=


k=1

bk k (x) and C (x) :=


k=1

ck k (x),

(13.2.5)

13.2 Introducing Functions of Second Kind

255

and observe that

F (x) :=

bk k (x) =


1 +
B (x) + B (x) and
2

ck k (x) =


1  +
C (x) C (x) ,
2i

k=1

F (x) :=


k=1

(13.2.6)

and that
F(x) = F (x) + F (x).

(13.2.7)

2. Apply the d-transformation to the series B (x) and C (x) and then invoke (13.2.6)
and (13.2.7). Near the points of singularity of B (x) and C (x) use APS with Rl =
(l + 1).
When n (x) and n (x) are real, n (x) are necessarily complex. For this reason, we
call this the complex series approach.
In connection with this approach, we note that when the functions n (x) and n (x)
and the coefcients bn and cn are all real, it is enough to treat the two complex series
B + (x) and C + (x), as B (x) = B + (x) and C (x) = C + (x), so that F (x) = B + (x)
and F (x) = !C + (x) in such cases.
We also note that the series B (x) and C (x) can be viewed as power series because
B (x) :=





  k

bk gk (x) z k and C (x) :=
ck gk (x) z ; z = eix .

k=1

k=1

The developments of Section 12.7 of the preceding chapter, including APS, thus become
very useful in dealing with the series of this chapter.
Note that the complex series approach, in connection with the summation of classical
Fourier series by the Shanks transformation, was rst suggested by Wynn [374]. Wynn




k
proposed that a real cosine series
k=0 bk cos kx be written as ( k=0 bk z ) with
ix
z = e , and then the -algorithm be used to accelerate the convergence of the complex


k
power series
k=0 bk z . Later, Sidi [273, Section 3] proposed that the Levin transformations be used to accelerate the convergence of Fourier series in their complex power


k
(1)
series form
k=0 bk z when {bk } b , providing a convergence analysis for Process I
at the same time.

13.2.3 Justication of the Complex Series Approach and APS

We may wonder why the complex series approach is needed. After all, we can apply a
suitable extrapolation method directly to F(x). As we discuss next, this approach is economical in terms of the number of the bk and ck used in acceleration.
For simplicity, we consider the case in which F(x) = F (x), that is, cn = 0 for all n.
We further assume that the bn satisfy [cf. (12.2.1)]
bn+m =

m1

s=0

( )

qs (n)bn+s , qs A0 s , s integer, s = 0, 1, . . . , m 1. (13.2.8)

256 13 Acceleration of Convergence of Fourier and Generalized Fourier Series

Thus, bn n (x) = b n z n with z = eix and b n = bn gn (x). As a result, (13.2.8) becomes
b n+m =

m1


( )
q s (n)b n+s , q s A0 s , s = 0, 1, . . . , m 1,

(13.2.9)

s=0

(x)/gn+s
(x),
with the same integers s as in (13.2.8), because q s (n) = qs (n)gn+m

s = 0, 1, . . . , m 1. Therefore, Theorem 12.2.1 applies and we have {bn n (x)} =

{b n z n } b(m) , except for at most m values of x for which the limit or antilimit of


(m)
k
-transformation can
(the power series) B (x) :=
k=1 bk z is singular, and the d

be applied to accelerate the convergence of B (x). Finally, we use (13.2.6) to compute

F (x). Because {bn n (x)} b(m) and because n (x) = 12 [n+ (x) + n (x)], we have that
{bn n (x)} b(2m) by part (ii) of Heuristic 6.4.1, and the d (2m) -transformation can be
applied to accelerate the convergence of F (x) directly.
From our discussion in Subsection 4.4.5, we now conclude heuristically that, with
(m,0)
(m)
Rl = (l + 1) and xed , the approximations d(,...
,) obtained by applying the d
(2m,0)

transformation to the series B (x) and the approximation d(,... ,) obtained by applying the d (2m) -transformation directly to the series F (x) have comparable accuracy.
(m,0)
Now d(,...
,) is obtained by using the coefcients bk , 1 k m + + m 1, while
(2m,0)
d(,... ,) is obtained by using bk , 1 k 2m + + 2m 1. That is, if the rst M coefcients bk are needed to achieve a certain accuracy when applying the d-transformation
with he complex series approach, about 2M coefcients bk are needed to achieve the
same accuracy from the application of the d-transformation directly to F (x). This suggests that the cost of the complex series approach is about half that of the direct approach,
when costs are measured in terms of the number of coefcients used. This is the proper
measure of cost as the series are dened solely via their coefcients, and the functions
n (x) and n (x) are readily available in most cases of interest.

13.3 Examples of Generalized Fourier Series

In the preceding section, we mentioned the classical Fourier series as the simplest example of the class of series that is characterized through (13.2.1)(13.2.3). In this section,
we give further examples involving nonclassical Fourier series, orthogonal polynomial expansions, and FourierBessel series. Additional examples involving other special
functions can also be given.

FT (x) :=

13.3.1 Chebyshev Series


dk Tk (x) and FU (x) :=
ek Uk (x), 1 x 1,

k=0

k=0

(13.3.1)

where Tk (x) and Uk (x) are the Chebyshev polynomials of degree k, of the rst and second
kinds respectively.
Dening x = cos , 0 , we have Tn (x) = cos nand Un (x) = sin(n + 1)/
sin . Therefore, n (x) = Tn (x) = cos n and n (x) = 1 x 2 Un1 (x) = sin n,

13.3 Examples of Generalized Fourier Series

n = 0, 1, . . . , with U1 (x) 0. Both series, FT (x) and
treated as ordinary Fourier series.

257

1 x 2 FU (x), can thus be

13.3.2 Nonclassical Fourier Series


F(x) :=
(bk cos k x + ck sin k x),

(13.3.2)

k=1

where
n n

j n j as n , 0 > 0,

(13.3.3)

j=0

so that n 0 n as n .
Functions n (x) and n (x) such as the ones here arise, for example, when one solves
an eigenvalue problem associated with a boundary value problem involving the O.D.E.
u  + 2 u = 0 on the interval [0, l], which, in turn, may arise when one uses separation of
variables in the solution of some appropriate heat equation, wave equation, or Laplaces
equation. For instance, the eigenfunctions of the problem
u  + 2 u = 0, 0 < x < l; u(0) = 0, u  (l) = hu(l), h > 0,
are sin n x, n = 1, 2, . . . , where n is the nth positive solution of the nonlinear equation
cos l = h sin l. By straightforward asymptotic techniques, it can be shown that


1
+ e1 n 1 + e2 n 2 + as n .
n n
2 l
Consequently, we also have that = 0 = /l and  = 0 in (13.2.2) and (13.2.3).

FP (x) :=


k=0

13.3.3 FourierLegendre Series


dk Pk (x) or FQ (x) :=
ek Q k (x), 1 < x < 1,

(13.3.4)

k=0

where Pn (x) is the Legendre polynomial of degree n and Q n (x) is the associated Legendre
function of the second kind of order 0 of degree n. They are both generated by the
recursion relation
Mn+1 (x) =

2n + 1
n
x Mn (x)
Mn1 (x), n = 1, 2, . . . ,
n+1
n+1

(13.3.5)

where Mn (x) is either Pn (x) or Q n (x), with the initial conditions

P0 (x) = 1, P1 (x) = x and
1+x
1
Q 0 (x) = log
, Q 1 (x) = x Q 0 (x) 1, when |x| < 1.
2
1x

(13.3.6)

We now show that, with = cos1 x, n () = Pn (x) and n () = 2 Q n (x) so that

= Pn (x) i 2 Q n (x), and = 1 in (13.2.2) and  = 1/2 in (13.2.3).

n ( )

258 13 Acceleration of Convergence of Fourier and Generalized Fourier Series

First, we recall that Pn (x) = Pn0 (x) and Q n (x) = Q 0n (x), where P (x) and Q (x) are
the associated Legendre functions of degree and of order . Next, letting x = cos ,
0 < < , and u = n + 1/2 in [223, p. 473, Ex. 13.3], it follows that, for any xed real
m, there exist two asymptotic expansions Am (; u) and Bm ( ; u),
Am (; u) :=

2

Am
s ( )
as n ,
2s
u
s=0

Bm (; u) :=


Bsm ( 2 )
as n ,
u 2s
s=0

(13.3.7)

such that
Pnm (cos )


1/2

2 m
1
i Q n (cos ) m

u
sin


()
Hm() (u )Am (; u) + Hm1
(u )Bm (u; ) .
u

(13.3.8)

Here Hm(+) (z) and Hm() (z) stand for the Hankel functions Hm(1) (z) and Hm(2) (z), respectively.
Finally, we also have
Hm() (z) eiz


Cmj
j=0

z j+1/2

as z +.

(13.3.9)

Substituting (13.3.7) and (13.3.9) in (13.3.8), letting m = 0 there, and invoking u =

n + 1/2, the result now follows. We leave the details to the interested reader.
It is now also clear that, in the series FP (x) and FQ (x), Pn (x) and Q n (x) can be

replaced by Pn (x) and Q n (x), where is an arbitrary real number.

13.3.4 FourierBessel Series

F(x) :=

[dk J (k x) + ek Y (k x)] , 0 < x r, some r,

(13.3.10)

k=1

where J (z) and Y (z) are Bessel functions of order 0 of the rst and second kinds, respectively, and n are scalars satisfying (13.3.3). Normally, such n result from boundary
value problems involving the Bessel equation

 

d
2
du
x
+ 2 x
u = 0,
dx
dx
x
and r and 0 are related through 0r = . For example, n can be the nth positive zero of
J (z) or of J (z) or of some linear combination of them. In these cases, for all , 0 =
in (13.3.3).
We now show that n (x) = J (n x) and n (x) = Y (n x) so that n (x) = J (n x)
iY (n x), and = 0 in (13.2.2) and  = 1/2 in (13.2.3). From (13.3.9) and the fact

13.5 Direct Approach

259

that n + as n , it follows that

n (x) = H() (n x) ein x

Cj

j=0

(n x) j+1/2

as n ,

(13.3.11)

which, when combined with the asymptotic expansion in (13.3.3), leads to (13.2.3).

13.4 Convergence and Stability when {bn } b(1)

( )

Let {bn } b(1) with bn = n h(n) for some and some h A0 , where and may
be complex in general. In this section, we consider the stability and convergence of


the complex series approach to the summation of the series F (x) :=
k=1 bk k (x).
We recall that in this approach we apply the d-transformation to the series B (x) :=

1
+

k=1 bk k (x) and then use the fact that F (x) = 2 [B (x) + B (x)].

(1)
(1)
We have seen that {bn n (x)} b so that the d -transformation can be used in
accelerating the convergence of B (x). As we saw earlier, B (x) can also be viewed


( +)
k

and
as power series B (x) :=
k=1 bk (z) where bn , as a function of n, is in A0
ix

. Thus, for || < 1, B (x) converge to analytic functions of , say G (x; ).

z=e
When || = 1 but z = 1, B (x) converge also when ( + ) < 0. When || = 1
but ( + ) 0, B (x) diverge, but they have antilimits by Theorem 6.7.2, and these

k
antilimits are Abelian means of B (x), namely, lim 1
k=1 bk k (x) . They are also
generalized functions in x. When || = 1, the functions G (x; ), as functions of x, are
singular when z = 1, that is, at x = x = (arg )/. For || > 1 the series B (x)
diverge, but we assume that G (x; ) can be continued analytically to || 1, with
singularities removed.
From this information, we conclude that when || = 1 and x is close to x or when
|| 1 and eix 1, we can apply the d (1) -transformation with APS via Rl = (l + 1)
to the series B (x), where is a positive integer whose size depends on how close eix
is to 1. This results in approximations that enjoy excellent stability and convergence
properties by Theorem 12.7.1. They also have a low cost.

13.5 Direct Approach

In the preceding sections, we introduced the complex series approach to the summation
of Fourier and generalized Fourier series of the form (13.2.1)(13.2.3). We would like to
emphasize that this approach can be used when the coefcients bn and cn are available.
In fact, the complex series approach coupled with APS is the most efcient approach in
this case.
When bn and cn are not available by themselves, but we are given an = bn n (x) +
cn n (x) in one piece and there is no way of determining the bn and cn separately,
the complex series approach is not applicable. In such a case, we are forced to apply


extrapolation methods directly to
k=1 ak . Again, the d-transformation can be applied


directly to the series k=1 ak with APS through Rl = (l + 1); the size of depends on


how close x is to the singularities of the limit or antilimit of
k=1 ak .

260 13 Acceleration of Convergence of Fourier and Generalized Fourier Series

13.6 Extension of the Complex Series Approach
The complex series approach of Section 13.2, after appropriate preparatory work, can
be applied to series of products of functions of the form
H (x, y) :=

[bk 1k (x)2k (y) + ck 1k (x)2k (y)

k=1

+ dk 1k (x)2k (y) + ek 1k (x)2k (y)] ,

(13.6.1)

where
in p x
g pn (x),

pn (x) pn (x) i pn (x) = e

(13.6.2)

with p real positive constants and

p
g
pn (x) n

as n ,
pj (x)n

(13.6.3)

j=0

for some xed  p that can be complex in general. Thus, as a function of n, g

pn (x) is a
( p )
function in A0 . We assume the coefcients bn , cn , dn , and en are available.
(2)
We have already seen that { pn (x)}
n=1 b . From Heuristic 6.4.1, we have that

(r )
{ pn (x)qn (y)}n=1 b , where r = 3 or r = 4. Thus, if the d-transformation can be
applied to H (x, y), this is possible with m 4 in general, and we need a large number
of the terms of H (x, y) to accelerate its convergence by the d-transformation.
Employing in (13.6.1) the fact that
pn (x) =



1 +
1 +
pn (x) +
pn (x)
pn (x) and pn (x) =
pn (x) ,
2
2i

(13.6.4)

we can express H (x, y) in the form

H (x, y) = H +,+ (x, y) + H +, (x, y) + H ,+ (x, y) + H , (x, y),

(13.6.5)

where
H , (x, y) :=

wk, 1k
(x)2k
(y),

k=1

H , (x, y) :=

wk, 1k
(x)2k
(y),

(13.6.6)

k=1

with appropriate coefcients wk, and wk, . For example, when H (x, y) :=


,
= wk, = 14 bk for all k.
k=1 bk 1k (x)2k (y), we have wk
By (13.6.2), we have that

(x)2n
(y) = ein(1 x+2 y) u , (n), u , A0(1 +2 ) ,
1n

1n
(x)2n
(y) = ein(1 x2 y) u , (n), u , A0(1 +2 ) .

(13.6.7)

Thus, {1n
(x)2n
(y)} and {1n
(x)2n
(y)} are in b(1) . This means that we can apply
(m)
the d -transformation with a low value of m to each of the four series H , (x, y)
( )
and H , (x, y). In particular, if bn = n h(n) with h(n) A0 , where and are in

13.7 The H-Transformation

261

general complex, all four series can be treated by the d (1) -transformation. As long as
ei(1 x+2 y) = 1 [ei(1 x2 y) = 1], the series B , (x, y) [B , (x, y)] are oscillatory and hence can be treated by applying the d (1) -transformation with APS. When
ei(1 x+2 y) = 1 [ei(1 x2 y) = 1], however, B , (x, y) [B , (x, y)] are slowly
varying and can be treated by applying the d (1) -transformation, with GPS if necessary.
This use of the d-transformation is much more economical than its application directly
to H (x, y).
Finally, the approach of this section can be extended further to products of three or
more functions pn (x) and/or pn (x) without any conceptual changes.

13.7 The H-Transformation

A method, called the H-transformation, to accelerate the convergence of Fourier sine
and cosine series was proposed by Homeier [135]. We include this transformation in this
chapter, as it is simply a GREP(2) and a variant of the d (2) -transformation.
Let
F(x) :=


(bk cos kx + ck sin kx),
k=0

be the given Fourier series and let its partial sums be

Sn =

n


(bk cos kx + ck sin kx), n = 0, 1, . . . ,

k=0
( j)

Then, the approximation Hn to the sum of this series is dened via the linear system

n1

Sl = Hn( j) + rl cos lx
i=0


n1

i
i
+
sin
lx
, j l j + 2n, (13.7.1)
(l + )i
(l + )i
i=0

where

rn = (n + 1)M(bn , cn ), M( p, q) =

p if | p| > |q|,
q otherwise,

(13.7.2)

and is some xed constant. As before, i and i are additional auxiliary unknowns.
Homeier has given an elegant recursive algorithm for implementing the H-transformation
that is very economical.
Unfortunately, this transformation has two drawbacks:
1. The class of Fourier series to which it applies successfully is quite limited. This can
be seen as follows: The equations in (13.7.1) should be compared with those that
(2, j)
dene the d(n,n) , namely,
(2, j)

S Rl = d(n,n) + a Rl

n1
n1


i
i
+
a
, j l j + 2n,
Rl
i
i
i=0 Rl
i=0 Rl

(13.7.3)

262 13 Acceleration of Convergence of Fourier and Generalized Fourier Series

where an = bn cos nx + cn sin nx, with the special choice of the Rl , namely, Rl =
(2, j)
( j)
l + 1, l = 0, 1, . . . . Thus, d(n,n) and Hn use almost the same number of terms
of F(x).
The equations in (13.7.1) immediately suggest that the H-transformation can be
effective when





i
i
+ sin nx
Sn S + rn cos nx
as n ,
ni
ni
i=0
i=0
that is, when Sn is associated with a function A(y) F(2) . This situation is possible
only when {bn } and {cn } are both in b(1) . In view of this, it is clear that, when either
{bn } or {cn } or both are in b(s) with s > 1, the H-transformation ceases to be effective.
In contrast, the d (m) -transformation for some appropriate value of m > 2 is effective,
as we mentioned earlier.


As an example, let us consider the cosine series F(x) :=
k=0 bk cos kx, where
bn = Pn (t) are the Legendre polynomials. Because {bn } b(2) , we have that
{bn cos nx} b(4) . The d (4) -transformation can be applied directly to F(x). The d (2) transformation with the complex series approach can also be applied at about half the
cost of the direct approach. The H-transformation is ineffective.
2. By the way rn is dened, it is clear that the bn and cn are assumed to be available.
In this case, as explained before, the d (1) -transformation with Rl = l + 1 (that is
nothing but the Levin transformation) coupled with the complex series approach
achieves the required accuracy at about half the cost of the H-transformation, when
the latter is applicable. (As mentioned in Section 13.2, the application of the Levin
transformation with the complex series approach was suggested and analyzed earlier
in Sidi [273, Section 3].) Of course, better stability and accuracy is achieved by the
d (1) -transformation with APS near points of singularity.

14
Special Topics in Richardson Extrapolation

14.1 Conuence in Richardson Extrapolation

14.1.1 The Extrapolation Process and the SGRom-Algorithm
In Chapters 1 and 2, we considered functions A(y) that satisfy (1.1.2). In this section,
we consider functions that we now denote B(y) and that have asymptotic expansions of
the form
B(y) B +

Q k (log y)y k as y 0+,

(14.1.1)

k=1

where y is a discrete or continuous variable, k are distinct and in general complex

scalars satisfying
k = 0, k = 1, 2, . . . ; 1 2 ; lim k = +,
k

(14.1.2)

and Q k (x) are some polynomials given as

Q k (x) =

qk


ki x i for some integer qk 0.

(14.1.3)

i=0

From (14.1.2), it is clear that there can be only a nite number of k with equal real parts.
When 1 > 0, lim y0+ B(y) exists and is equal to B. When 1 0 and Q 1 (x)  0,
however, lim y0+ B(y) does not exist, and B in this case is the antilimit of B(y) as
y 0+.
We assume that B(y) is known (and hence is computable) for all possible y > 0 and
that the k and qk are also known. Note that qk is an upper bound for Q k , the degree of
Q k (x), and that Q k need not be known exactly. We assume that B and the ki are not
necessarily known and that B is being sought.
Clearly, the problem we have here generalizes that of Chapter 1 (i) by allowing the k
to have equal real parts and (ii) by replacing the constants k in (1.1.2) by the polynomials
Q k in log y.


Q k (log y)y k as being obtained by
Note that we can also think of the expansion
k=1


qk
ki
letting ki k , i = 0, 1, . . . , qk , in the expansion
k=1
i=0 ki y . Thus, we view
the extrapolation process we are about to propose for determining B as a Richardson
extrapolation process with conuence.
263

264

14 Special Topics in Richardson Extrapolation

Let us order the functions y k (log y)i as follows:

i (y) = y 1 (log y)i1 ,

1 i 1 q1 + 1,

1 +i (y) = y 2 (log y)i1 ,

1 i 2 q2 + 1,

1 +2 +i (y) = y (log y)

i1

1 i 3 q3 + 1,

(14.1.4)

and so on.
Let us choose y0 > y1 > > 0 such that liml yl = 0. Then we dene the generalization of the Richardson extrapolation process for the present problem through the
linear systems
B(yl ) = Bn( j) +

n


k k (yl ), j l j + n.

(14.1.5)

k=1

n

n
( j)
( j)
( j)
As always, we have Bn = i=0
ni B(y j+i ) with i=0
ni = 1, and we dene


( j)
( j)
n
|ni |. (Note that we have changed our usual notation slightly and writ'n = i=0
( j)
( j)
( j)
( j)
ten ni instead of ni and 'n instead of n .)
In this section, we summarize the treatment given to this problem by Sidi [298] with
yl = y0 l , l = 0, 1, . . . , for some (0, 1). For details and numerical examples, we
refer the reader to [298].
( j)
We start with the following recursive algorithm for computing the Bn . This algorithm
is denoted the SGRom-algorithm.
Algorithm 14.1.1 (SGRom-algorithm)
1. Let ck = k , k = 1, 2, . . . , and set
i = c1 ,

1 i 1 ,

1 +i = c2 ,

1 i 2 ,

1 +2 +i = c3 ,

1 i 3 ,

(14.1.6)

and so on.
2. Set
( j)

B0 = B(y j ), j = 0, 1, . . . .
( j)

3. Compute Bn by the recursion relation

( j+1)

Bn( j) =

( j)

Bn1 n Bn1
, j = 0, 1, . . . , n = 1, 2, . . . .
1 n
( j)

Our rst result concerns the ni .

( j)

Theorem 14.1.2 The ni are independent of j and satisfy

n

i=0

( j)

ni z i =

n

z i
Un (z).
1 i
i=1

(14.1.7)

14.1 Conuence in Richardson Extrapolation

265

( j)

Hence 'n is independent of j. In addition,

'(nj)

n

1 + |i |
i=1

|1 i |

for all j, n,

(14.1.8)

with equality in (14.1.8) when the ck all have the same phase or, equivalently, when the
k all have the same imaginary part.

14.1.2 Treatment of Column Sequences

Let us rst note that, for any integer n 0, there exist two unique nonnegative integers
t and s, such that
n=

t


k + s, 0 s t+1 1,

(14.1.9)

k=1

where k = qk + 1 as before. We also dene tk=1 k to be zero when t = 0. With n, t,
and s as in (14.1.9), we next dene the two sets of integers Sn and Tn as in
Sn = {(k, r ) : 0 r qk , 1 k t, and 0 r s 1, k = t + 1},
Tn = {(k, r ) : 0 r qk , k 1} \ Sn .

(14.1.10)

Theorem 14.1.3 concerns the convergence and stability of the column sequences
( j)
k
for all k and that the ordering of the k in (14.1.2)
{Bn }
j=0 . Note only that |ck | = e
implies
ck = 1, k = 1, 2, . . . ; |c1 | |c2 | ; lim ck = 0,
k

(14.1.11)

and that |ci | = |c j | if and only if i =  j . Also, with yl = y0 l , the asymptotic

expansion in (14.1.1) assumes the form
B(ym ) B +

qk


k=1


ki m

ckm as m ,

(14.1.12)

i=0

where ki depend linearly on the kr , i r qk , and k,qk = k,qk y0k (log )qk .
It is clear that the extrapolation method of this section can be applied via the SGRomalgorithm with no changes to sequences {X m } when the asymptotic expansion of X m is
exactly as in the right-hand side of (14.1.12).
Theorem 14.1.3
(i) With Un (z) as in (14.1.7), and Sn and Tn as in (14.1.10), for xed n, we have the
complete asymptotic expansion






d r  j
( j)

z Un (z) z=ck as j . (14.1.13)
kr z
Bn B
dz
(k,r )Tn

266

14 Special Topics in Richardson Extrapolation

If we let be that integer for which |ct+1 | = = |ct+ | > |ct++1 | and let q =
max{qt+1 s, qt+2 , . . . , qt+ }, then (14.1.13) gives
Bn( j) B = O(|ct+1 | j j q ) as j .
( j)

(14.1.14)
( j)

(ii) The process is stable in the sense that sup j 'n < . [Recall that 'n is independent of j and satises (14.1.8).]
When = 1 and t+1,qt+1 = 0, (14.1.13) gives the asymptotic equality


qt+1
j+s
( j)
Bn B t+1,qt+1
Un(s) (ct+1 )ct+1 j qt+1 s as j .
s

(14.1.15)

This is the case, in particular, when |c1 | > |c2 | > and k,qk = 0 for all k.
In any case, Theorem 14.1.3 implies that each column in the extrapolation table is at
least as good as the one preceding it.
14.1.3 Treatment of Diagonal Sequences
The treatment of the diagonal sequences turns out to be much more involved, as usual. To
proceed, we need to introduce some new notation. First, let 1 = k1 < k2 < k3 < be
the (smallest) integers for which
ki < ki+1 and m = ki , ki m ki+1 1, i = 1, 2, . . . , (14.1.16)
which implies that
|cki | > |cki+1 | and |cm | = |cki |, ki m ki+1 1, i = 1, 2, . . . . (14.1.17)
Then, let
Ni =

ki+1
1


r , i = 1, 2, . . . .

(14.1.18)

r =ki

In other words, Ni is the sum of the multiplicities r of all the cr that have modulus
equal to |cki |.
Part (i) of the following theorem is new and its proof can be achieved as that of part
(ii) of Theorem 1.5.4. Part (ii) is given in [298].
Theorem 14.1.4 Assume that the k satisfy
ki+1 ki d > 0 for all i.

(14.1.19)

Assume also that the k and qk are such that there exist constants E > 0 and b 0 for
which
lim sup Ni /i b = E.

(14.1.20)

(i) Under these conditions, the sequence {Bn }

n=0 converges to B as indicated in
( j)

Bn( j) B = O( ) as n for every > 0.

( j)

(ii) The process is stable in the sense that supn 'n < .

(14.1.21)

14.1 Conuence in Richardson Extrapolation

267

It is worth mentioning that the proof of this theorem relies on the fact that i=1
|i |
converges under the conditions in (14.1.19) and (14.1.20).
By imposing suitable growth conditions on the ki and k and assuming, in addition
to (14.1.20), that
lim inf Ni /i a = D > 0, 0 a b, a + 2 > b,
i

(14.1.22)

Sidi [298] shows that

|Bn( j) B| (K + )r

(a+2)

(L + )n

(a+2)/(b+1)

for all large n,

(14.1.23)

where  > 0 is arbitrary, r is that integer for which kr +1 t + 1 < kr +2 , and



Dd
Dd b + 1 (a+2)/(b+1)

; L= , =
. (14.1.24)
K = , =
a+2
a+2
E

We note that in many cases of interest a = b in general and a = b = 0 in particular.

For example, when Ni Fi v as i , we have a = b = v and D = E = F. In such
u
( j)
cases, (14.1.23) shows that Bn B = O(en ) as n , for some > 0, with
u = (a + 2)/(a + 1) > 1. This is a renement of (14.1.21).
n
( j)
|i |) as n .
We also note that, for all practical purposes, Bn B is O( i=1
Very realistic information about the error can be obtained by analyzing the behavior of
n
the product i=1
|i | for large n. See Sidi [301] for examples.
It is useful at this point to mention that the trapezoidal rule approximation T (h) for
the singular integral in Example 3.1.2 has all the characteristics of the functions B(y)
we have discussed so far. From the asymptotic expansion in (3.1.6) and from (3.1.7), it
is clear that, in this example, qk = 0 or qk = 1, a = b = 0, and D = 1 and E = 2. This
example has been generalized in Sidi [298].

14.1.4 A Further Problem

Before we end this section, we would like to mention that a problem similar to the one
considered in Subsections 14.1.114.1.3, but of a more general nature, has been treated
by Sidi [297]. In this problem, we consider a function B(y) that has the asymptotic
expansion
B(y) B +

k (y)Q k (log y) as y 0+,

(14.1.25)

k=1

where B is the limit or antilimit of B(y) as y 0+, Q k (x) is a polynomial in x of

degree at most qk , k = 1, 2, . . . , as before, and
k+1 (y) = O(k (y)) as y 0+, k = 1, 2, . . . .

(14.1.26)

The yl are chosen to satisfy liml (yl+1 /yl ) = (0, 1), and it is assumed that
k (yl+1 )
= ck = 1, k = 1, 2, . . . , and ci = c j if i = j. (14.1.27)
l k (yl )
lim

268

14 Special Topics in Richardson Extrapolation

Thus, |c1 | |c2 | |c3 | . [It also follows that if |ck+1 | < |ck |, then (14.1.26) is
actually k+1 (y) = o(k (y)) as y 0+.] In addition, we assume that there are nitely
many ck with the same modulus.
The Richardson extrapolation is generalized to this problem in a suitable fashion in
[297] and the column sequences are analyzed with respect to convergence and stability.


( j)
In particular, the approximation Bn to B, where n = tk=1 (qk + 1), are dened through
B(yl ) = Bn( j) +

t


k (yl )

k=1

qk


ki (log yl )i , j l j + n.

(14.1.28)

i=0

n

n

n
( j)
( j)
( j)
( j)
( j)
Again, Bn = i=0
ni A(y j+i ) with i=0
ni = 1, and we dene 'n = i=0
|ni |.
( j)
We then have the following convergence result for the column sequence {Bn } j=0 :
Bn( j) B = O(Rt (y j )) as j ,

(14.1.29)

where
Rt (y) = B(y) B

t


k (y)

k=1

qk


Q k (log y).

(14.1.30)

i=0

Note that
Rt (y) = O(t+1 (y)(log y)q ) as y 0+; q = max{qk : |ck | = |ct+1 |, k t + 1}.
(14.1.31)
The process is also stable because
lim

n


( j)
ni z i

i=0


t 

z ck qk +1
k=1

1 ck

from which we have

lim '(nj) =

n

i=0

| ni |

n


ni z i ,


t 

1 + |ck | qk +1
k=1

|1 ck |

(14.1.32)

i=0

< .

(14.1.33)
( j)

These results are special cases of those proved in Sidi [297], where Bn are dened and
analyzed for all n = 1, 2, . . . . For details see [297].
It is clear that the function B(y) in (14.1.1) and (14.1.2) considered in Subsections 14.1.114.1.3 is a special case of the one we consider in this subsection with
k (y) = y k . Because the yl are chosen to satisfy liml (yl+1 /yl ) = in the extrapolation process, we also have ck = k in (14.1.27). Thus, the theory of [297] in general
and (14.1.29)(14.1.33) in particular provide additional results for the function B(y) in
(14.1.1)(14.1.3).

14.2 Computation of Derivatives of Limits and Antilimits

In some common applications, functions B(y) (and their limits or antilimits B) of the form
we treated in the preceding section arise as derivatives with respect to some parameter
of functions A(y) (and their limits or antilimits A) that we treated in Chapter 1. The

14.2 Computation of Derivatives of Limits and Antilimits

269

trapezoidal rule approximation T (h) of Example 3.1.2, which we come back to shortly,
is one such function.
Also, numerical examples and the rigorous theoretical explanation given in Sidi [301]
show that, when A(y) and B(y) have asymptotic expansions that are essentially different,
it is much cheaper to use extrapolation on A(y) to approximate its limit or antilimit A than
on B(y) to approximate its limit or antilimit B. By this we mean that, for a given level
of required accuracy, more function values B(y) than A(y) need to be computed. The
question that arises then is whether it is possible to reduce the large computational cost of
d
d
A(y) and B = d
A.
extrapolating B(y) to approximate B when we know that B(y) = d
This question was raised recently by Sidi [301], who also proposed an effective approach to its solution. In this section, we summarize the approach of [301] and give
the accompanying theoretical results that provide its justication. For the details and
numerical examples, we refer the reader to this paper.
Let us denote by E0 the extrapolation process used on A(y) in approximating A. When
d
A(y) is essentially different from that of A(y), it
the asymptotic expansion of B(y) = d
is proposed to differentiate with respect to the approximations to A produced by E0 on
A(y) and take these derivatives as approximations to B. As a way of implementing this
procedure numerically, it is also proposed to differentiate with respect to the recursion
relations used in implementing E0 . (In doing that we should also differentiate the initial
conditions.)
Of course, the process proposed here can be applied to computation of higher-order
derivatives of limits and antilimits as well. It can also be applied to computation of partial
derivatives with respect to several variables.
These ideas can best be demonstrated via the Richardson extrapolation process of
Chapter 1.

14.2.1 Derivative of the Richardson Extrapolation Process

Let A(y) be exactly as in Chapter 1, that is,
A(y) A +

k y k as y 0+,

(14.2.1)

k=1

where
k = 0, k = 1, 2, . . . ; 1 < 2 < ; lim k = ,
k

(14.2.2)

and assume that A(y), A, the k and the k depend on a parameter in addition. Let
d

A(y) has an asymptotic expansion as y 0+ that is

us also assume that A(y)
d
obtained by differentiating that of A(y) termwise, that is,

A(y)
A +


( k + k k log y)y k as y 0 + .

(14.2.3)

k=1
d
d k = k and d k = k . Note that when the k
A = A,
Here we have also denoted d
d
d
do not all vanish, the asymptotic expansion in (14.2.3) is essentially different from that

270

14 Special Topics in Richardson Extrapolation

in (14.2.1). Also, the asymptotic expansion of A(y)

in (14.2.3) is exactly of the form
given in (14.1.1) for B(y) with qk = 1 for all k in the latter.
Let us apply the Richardson extrapolation process, with yl = y0 l , l = 0, 1, . . . , to
( j)
( j)
the function A(y) to obtain the approximations An to A. Next, let us differentiate the An
(
j)
The necessary algorithm for this is obtained by
to obtain the approximations A n to A.
( j)
differentiating the recursion relation satised by the An that is given in Algorithm 1.3.1.
d
cn = (log ) n cn , we obtain the following recursion
Thus, letting cn = n and cn = d
( j)

relation for the An :

( j)
( j)
j ), j = 0, 1, . . . ,
A0 = A(y j ) and A 0 = A(y
( j+1)

( j)

A(nj) =

An1 cn An1
and
1 cn

A (nj) =

( j+1)
( j)
A n1 cn A n1
cn
( j)
+
(A( j) An1 ), j = 0, 1, . . . , n = 1, 2, . . . .
1 cn
1 cn n

l ), j l j + n, for determining
Remark. It is clear that we need both A(yl ) and A(y
( j)
A n , and this may seem expensive at rst. However, in most problems of interest, the
l ) can be done simultaneously with that of A(yl ), and at almost no
computation of A(y
( j)
additional cost. Thus, in such problems, the computation of A n entails practically the
( j)
same cost as that of An , in general. This makes the proposed approach desirable.

n
( j)
( j)
ni A(y j+i ), we have that
Because An = i=0
A (nj) =

n


( j)
ni A(y
j+i ) +

i=0

n


( j)

ni A(y j+i ),

i=0

as a result of which we conclude that the propagation of errors (roundoff and other) in
l ), j l j + n, into A (nj) is controlled by the quantity $(nj) that is
the A(yl ) and A(y
dened as in
$(nj) =

n

i=0

( j)

|ni | +

n


( j)

|ni |.

(14.2.4)

i=0

The following theorem summarizes the convergence and stability of the column and
( j)
diagonal sequences in the extrapolation table of the A n . In this theorem, we make use of
( j)
( j)
the fact that the ni and hence the ni are independent of j in the case being considered.
Theorem 14.2.1
( j)
(i) For xed n, the error A n A has the complete asymptotic expansion



d
( j)

[Un (ck )k ] + Un (ck )k k log y j y j k as j , (14.2.5)

An A
d
k=n+1
( j)
from which we conclude that A n A = O( j|cn+1 | j ) as j . Here Un (z) =
n
( j)
( j)
i=1 (z ci )/(1 ci ). In addition, sup j $n < since $n are all independent of
j. That is, column sequences are stable.

14.2 Computation of Derivatives of Limits and Antilimits

271

(ii) Let us assume, in addition to (14.2.2), that the k also satisfy

k+1 k d > 0, k = 1, 2, . . . , for some xed d,
and that

(14.2.6)

( j)
|ci | < . Then, for xed j, the error A n A satises

i=1

A (nj) A = O( ) as n , for every > 0.

(14.2.7)

( j)

Also, supn $n < , which implies that diagonal sequences are stable.
( j)
Part (ii) of Theorem 14.2.1 says that, as n , A n A = O(en ) for every > 0.
By imposing some mild growth conditions on the k and the k , and assuming further
that |ci | K i |ci | for all i, such that K i = O(i a ) as i , for some a 1, the result
in (14.2.7) can be improved to read

( + )dn 2 /2 as n , with arbitrary  > 0.

| A (nj) A|

(14.2.8)

2
( j)
This means that A n A = O(en ) as n for some > 0.
( j)
just like A(nj) A, is
We also note that, for all practical purposes, A n A,
n
O( i=1 |ci |) as n . Very realistic information on both can be obtained by anan
|ci | as n .
lyzing the behavior of the product i=1
The important conclusion we draw from both parts of Theorem 14.2.1 is that the
( j)
( j)
accuracy of A n as an approximation to A is almost the same as that of An as an
approximation to A. This clearly shows that the approach we suggested for determining
A is very efcient. In fact, we can now show rigorously that it is more efcient than
the approach of the preceding section involving the SGRom-algorithm. (However, we
should keep in mind that the problem we are treating here is a special one and that the
approach with the SGRom-algorithm can be applied to a larger class of problems.)

Let us denote A(y)

= B(y) and A = B in (14.2.3) and apply the SGRom-algorithm
( j)
to B(y) to obtain the approximations Bn to B. Let us assume that the conditions of
Theorem 14.2.1 are satised. Then, the conditions of Theorem 14.1.4 are satised as
well, and Ni 2 for all i there. Assuming that Ni = 2 for all i, we have D = E = 2
( j)
and a = b = 0, so that L = d/4 in (14.1.24). As a result, by (14.1.23), Bn B is of
2
( j)

order dn /4 as n , for all practical purposes. For the approximations A n to A,

2
( j)
dn /2

as n , for all practical

on the other hand, we have that An A is of order
( j)
( j)
purposes. It is clear that B2n will have an accuracy comparable to that of A n ; its
computational cost will, of course, be higher.

An Application to Numerical Quadrature

1
Let us consider the numerical approximation of the integral B = 0 (log x)x g(x)d x,

d
where A = 1 x g(x)d x. Let us
 > 1, with g C [0, 1]. Clearly, B = d
A A,
0
set h = 1/n, where n is a positive integer, and dene the trapezoidal rule approximations
with stepsize h to A and B, respectively, by
!
n1

1
G( j h) + G(1) ; G(x) x g(x),
(14.2.9)
A(h) = h
2
j=1

272

14 Special Topics in Richardson Extrapolation

and
B(h) = h

n1

j=1

!
1
H ( j h) + H (1) ; H (x) (log x)x g(x).
2

(14.2.10)

Note that B(h) = A(h)

because H (x) = G(x).
Then we have the following extensions
of the classical EulerMaclaurin expansion for A(h) and B(h):
A(h) A +

ai h 2i +

i=1

bi h +i+1 as h 0,

(14.2.11)

i=0

and
B(h) B +


i=1

ai h 2i +


(b i + bi log h)h +i+1 as h 0,

(14.2.12)

i=0

with
ai =

B2i (2i1)
( i) (i)
G
g (0), i = 0, 1, . . . ,
(1), i = 1, 2, . . . ; bi =
(2i)!
i!
(14.2.13)

d
d
ai and b i = d
bi . As before, Bk are the Bernoulli numbers and should not
and ai = d
( j)
be confused with B(h) or with Bn below. The expansion in (14.2.12) is obtained by
differentiating that in (14.2.11). [Note that G(x) depends on but g(x) does not.] For
these expansions, see Appendix D.
Let us now consider the case 1 <  < 0. Then A(h) is of the form described in
(14.2.1) and treated throughout, with 1 , 2 , . . . , as in

3i2 = + 2i 1, 3i1 = + 2i, 3i = 2i, i = 1, 2, . . . ,

(14.2.14)

so that (14.2.6) is satised with d = min(, 1 +  ) > 0.

Let us also apply the Richardson extrapolation process to the sequence {A(h l )} with
h l = l , l = 0, 1, . . . , for some {1/2, 1/3, . . . }. (Recall that other more economical choices of {h l } are possible; but we stick with h l = l here as it enables us to use
the theory of [301] to make rigorous statements about convergence, convergence rates,
and stability of diagonal sequences.)
n
( j)
|ci |) as n . This implies that
Recall now that |An A| is practically O( i=1

3m
( j)
i = 3m 2 +
|A3m A| 0 as m practically like &m , where &m = i=1
( j)
3m 2
O(m) as m . Thus, |A3m A| 0 as m practically like .
Since 3i2 = 3i1 = 1, 3i = 0, i = 1, 2, . . . , we have c3i2 = (log )c3i2 ,


c3i1 = (log )c3i1 , c3i = 0, i = 1, 2, . . . , and thus i=1
|ci | < and K i | log |
( j)
0 as
for all i. Consequently, Theorem 14.2.1 applies, and we have that | A 3m A|
( j)
3m 2
practically, just like |A3m A|.
m like

Similarly, B(h) = A(h)

is of the form given in (14.1.1) with i as in (14.2.14) and
q3i2 = q3i1 = 1, q3i = 0, i = 1, 2, . . . . Let us also apply the generalized Richardson
extrapolation process of the previous section (via the SGRom-algorithm) to the sequence
( j)
{B(h l )}, also with h l = l , l = 0, 1, . . . . By Theorem 14.1.4, the sequence {Bn }
n=0
5m
3m
( j)
qi +1
converges to B, and |B5m B| is practically O( i=1 |i |), hence O( i=1 |ci |
), as

14.2 Computation of Derivatives of Limits and Antilimits

273

m . This implies that |B5m B| 0 as m practically like !m , where !m =

3m
( j)
2
5m 2
) as
i=1 (qi + 1)(i ) = 5m + O(m) as m . Therefore, |B5m B| is O(
m practically speaking.
( j)
and |Bn( j) B| tend to 0 as n like n 2 /3 and n 2 /5 , respectively,
Thus, | A n A|
( j)
for all practical purposes. That is to say, of the two diagonal sequences { A n }n=0 and
( j)
( j)
{Bn }n=0 , the former has superior convergence properties. Also, B5/3 n has an accuracy
( j)
( j)
comparable to that of A n . Of course, B5/3 n is much more expensive to compute
(
j)
than A n . [Recall that the computation of A(yl ) and/or B(yl ) involves l integrand
evaluations.]
The preceding
 1 comparative study suggests, therefore, that computation of integrals
of the form 0 (log x)x g(x)d x by rst applying the Richardson extrapolation process
1
to the integral 0 x g(x)d x and then differentiating the resulting approximations with
respect to may be a preferred method if we intend to use extrapolation methods in
the rst place. This approach may be used in multidimensional integration of integrands
that have logarithmic corner, or surface, or line singularities, for which appropriate
extensions of the EulerMaclaurin expansion can be found in the works of Lyness [196],
Lyness and Monegato [201], Lyness and de Doncker [199], and Sidi [283]. All these
expansions are obtained by term-by-term differentiation of other simpler expansions, and
this is what makes the approach of this section appropriate. Since the computation of the
trapezoidal rule approximations for multidimensional integrals becomes very expensive
as the dimension increases, the economy that can be achieved with this approach should
make it especially attractive.
approach can also be used to compute the singular integrals Ir =
 1The new
dr
r
(log
x)
x
g(x)d
x, where r = 2, 3, . . . , by realizing that Ir = d
r A. (Note that B = I1 .)
0
The approximations produced by the generalized Richardson extrapolation process have
( j)
dr
convergence properties that deteriorate in quality as r becomes large, whereas the d
r An
( j)
maintain the high-quality convergence properties of the An . For application of the generalized Richardson extrapolation process to such integrals, see Sidi [298]. Again, the
extension to multidimensional singular integrals is immediate.
Before closing, we make the following interesting observation that is analogous to an
observation of Bauer, Rutishauser, and Stiefel [20] about Romberg integration (see also
( j)
Davis and Rabinowitz [63]): The approximation A n to B can be expressed as a sort of
numerical quadrature formula with stepsize h j+n of the form
( j)

A (nj) =

j+n


w (0)
jnk H (kh j+n ) +

k=0

j+n


w(1)
jnk G(kh j+n ); = 1/,

(14.2.15)

k=0

(1)
in which the weights w(0)
jnk and w jnk depend on j and n, and satisfy
j+n

k=0

w (0)
jnk = 1 and

j+n


w (1)
jnk = 0.

(14.2.16)

k=0

n

n
These follow from the facts that i=0
ni = 1 and i=0
ni = 0. [We have obtained
(14.2.15) and (14.2.16) by adding the terms 12 G(0)h and 12 H (0)h to the right-hand
sides of (14.2.9) and (14.2.10), respectively, with the understanding that G(0) 0 and

274

14 Special Topics in Richardson Extrapolation

H (0) 0.] Also, these formulas are stable numerically in the sense that
j+n 



(1)
|w (0)
|
+
|w
|
$(nj) < for all j and n.
jnk
jnk

(14.2.17)

k=0

14.2.2

d
GREP(1) :
d

Derivative of GREP(1)

We end by showing how the approach of Sidi [301] can be applied to GREP(1) . The
( j)
d
GREP(1) . The following
resulting method that produces the A n has been denoted d
material is from Sidi [302].
If a(t) has an asymptotic expansion of the form
a(t) A + (t)

i t i as t 0+,

(14.2.18)

i=0
d
has an asymptotic expansion
a(t) a(t)
and a(t), A, (t), and the i depend on , and d
that can be obtained by termwise differentiation of that in (14.2.18), then

A + (t)
a(t)

i t i + (t)

i=0

i t i as t 0 + .

(14.2.19)

i=0

Recalling the developments of Section 7.2 concerning the W-algorithm, we observe

( j)
( j) d
d
Dn {g(t)} = Dn { d
g(t)}. This fact is now used to differentiate the W-algorithm
that d
( j)
d
and to derive what we call the d
W-algorithm for computing the A n :
d
Algorithm 14.2.2 ( d
W-algorithm)

1. For j = 0, 1, . . . , set
( j)

M0 =

a(t j )
1
( j)
( j)
( j)
, N0 =
, H0 = (1) j |N0 |, and
(t j )
(t j )

j ) a(t j )(t
j ) ( j)
a(t
(t
j)
( j)
( j)
( j)

M 0 =
, N0 =
, H 0 = (1) j | N 0 |.
(t j )
[(t j )]2
[(t j )]2
( j)
( j)
( j)
( j)
( j)
( j)
2. For j = 0, 1, . . . , and n = 1, 2, . . . , compute Mn , Nn , Hn , M n , N n , and H n
recursively from
( j+1)

Q (nj) =

( j)

Q n1 Q n1
.
t j+n t j

3. For all j and n, set

( j)

A(nj) =

( j)

Mn

|Hn |

Nn

|Nn |

, n( j) =
( j)

( j)

, and


( j)
( j)
( j)
( j) 
M n
N n
(nj) = | H n | + 1 + | N n | n( j) .
A (nj) = ( j) A(nj) ( j) , $
( j)
( j)
Nn
Nn
|Nn |
|Nn |
( j)

( j)

n is an upper bound on $n dened as in (14.2.4), and it turns out to be quite

Here $
tight.

14.2 Computation of Derivatives of Limits and Antilimits

275

The convergence and stability of column sequences for the case in which the tl are
chosen such that
lim (tl+1 /tl ) = for some (0, 1)

(14.2.20)

and (t) satises

lim (tl+1 )/(tl ) = for some (complex) = 0, 1, 2, . . . , (14.2.21)

and
(t)
= (t)[K log t + L + o(1)] as t 0+, for some K = 0 and L , (14.2.22)
have been investigated in a thorough manner in [302], where an application to the d (1) transformation is also provided. Recall that we have already encountered the condition
in (14.2.21) in Theorem 8.5.1. The condition in (14.2.22) was formulated in [302] and
d
GREP(1) . Theorem 14.2.3 summarizes the main results of
is crucial to the analysis of d
Sidi [302, Section 3].
Theorem 14.2.3 Under the conditions given in (14.2.20)(14.2.22), the following hold:
( j)

( j)

(i) The ni and ni have well-dened and nite limits as j . Specically,

lim

j
n


lim

n

n


( j)

ni z i = Un (z),

i=0

ni z i = K (log )[Un (z)Un (1) zUn (z)],

( j)

i=0

where Un (z) = i=1 (z ci )/(1 ci ) with ci = +i1 , i = 1, 2, . . . , and Un (z) =

( j)
d
U (z). Consequently, sup j $n < , implying that the column sequences
dz n
( j)
{ A n }
j=0 are stable.
(
j)
(ii) For A n we have that
A (nj) A = O((t j )t nj log t j ) as j .
For more rened statements of the results of Theorem 14.2.3 and for additional developments, we refer the reader to Sidi [302].
An Application to the d (1) -Transformation: The

d (1)
d -Transformation
d

The preceding approach can be applied to the problem of determining the derivative with
respect to a parameter of sums of innite series, whether convergent or divergent.


Consider the innite series
k=1 vk , where
vn


i=0

and let us dene Sn =

i n i as n ; 0 = 0, + 1 = 0, 1, 2, . . . ,

n
k=1

(14.2.23)

vk , n = 1, 2, . . . . Then we already know that

Sn S + nvn


i=0

i n i as n .

(14.2.24)

276

14 Special Topics in Richardson Extrapolation

Here S = limn Sn when the series converges; S is the antilimit of {Sn } otherwise.
Because {vn } b(1) as well, excellent approximations to S can be obtained by applying


the d (1) -transformation to
k=1 vk , with GPS if necessary. Let us denote the resulting
( j)
approximations to S by Sn .
If the asymptotic expansion in (14.2.24) can be differentiated with respect to term
by term, then we have
Sn S + n v n


i=0

i n i + nvn

i n i as n .

(14.2.25)

i=0

( j)
d
d
Here Sn = d
Sn , S = d
S, etc. Let us now differentiate the Sn with respect to and
(
j)
In other words, let us make in the d W-algorithm the
take Sn as the approximations to S.
d
j ) = S R j , (t j ) = R j v R j , and (t
substitutions t j = 1/R j , a(t j ) = S R j , a(t
j ) = R j v R j for
( j)
( j)
( j)
( j)

the input, and the substitutions An = Sn and An = S n for the output. We call the
d (1)
d -transformation.
extrapolation method thus obtained the d
( j)
( j)

The S n converge to S as quickly as Sn converge to S when the vn satisfy

v n = vn [K  log n + L  + o(1)] as n , for some K  = 0 and L  . (14.2.26)

In addition, good stability and accuracy is achieved by using GPS, that is, by choosing
the Rl as in

Rl1 + 1 if  Rl1  = Rl1 ,
R0 1, Rl =
l = 1, 2, . . . ; > 1.
 Rl1  otherwise,
(For = 1, we have Rl = l + 1, and we recall that, in this case, the d (1) -transformation
reduces to the Levin u-transformation.)
As we have seen earlier, tl = 1/Rl satisfy liml (tl+1 /tl ) = 1 . Consequently, from
Theorem 8.5.1 and Theorem 14.2.3, there holds
) as j ,
Sn( j) S = O(v R j R n+1
j
S(nj) S = O(v R j R n+1
log R j ) as j .
j
d (1)
d -transformation is to the summation of series of
An immediate application of the d


the form k=1 vk log k, where vn are as in (14.2.23). This is possible because vk log k =
d
d
[v k ]| =0 . Thus, we should make the following substitutions in the d
W-algorithm:
d k

j ) = R j v R j log R j , where
t j = 1/R j , a(t j ) = S R j , a(t j ) = S R j , (t j ) = R j v R j , and (t






Sn = nk=1 vk , Sn = nk=1 vk log k. The series
k=1 vk log k can also be summed by
the d (2) -transformation, but this is more costly.
For further examples, see Sidi [302].
One important assumption we made here is that the asymptotic expansion of Sn can
be differentiated with respect to term by term. This assumption seems to hold in
general. In Appendix E, we prove rigorously that it does for the partial sums Sn =

n

of the generalized Zeta function (, ).

k=1 (k + 1)

Part II
Sequence Transformations

15
The Euler Transformation, Aitken 2 -Process, and
Lubkin W -Transformation

15.1 Introduction
In this chapter, we begin the treatment of sequence transformations. As mentioned in the
Introduction, a sequence transformation operates on a given sequence {An } and produces
another sequence { A n } that hopefully converges more quickly than the former. We also
mentioned there that a sequence transformation is useful only when A n is constructed
from a nite number of the Ak .
Our purpose in this chapter is to review briey a few transformations that have been in
existence longer than others and that have been applied successfully in various situations.
These are the Euler transformation, which is linear, the Aitken 2 -process and Lubkin
W -transformation, which are nonlinear, and a few of the more recent generalizations
of the latter two. As stated in the Introduction, linear transformations are usually less
effective than nonlinear ones, and they have been considered extensively in other places.
For these reasons, we do not treat them in this book. The Euler transformation is an
exception to this in that it is one of the most effective of the linear methods and also
one of the oldest acceleration methods. What we present here is a general version of the
Euler transformation known as the EulerKnopp transformation. A good source for this
transformation on which we have relied is Hardy [123].

15.2 The EulerKnopp (E, q) Method

15.2.1 Derivation of the Method
Given the innite sequence {An }
n=0 , let us set a0 = A0 and an = An An1 , n =


1, 2, . . . . Thus, An = nk=0 ak , n = 0, 1, . . . . We give two different derivations of
the EulerKnopp transformation:
1. Let us dene the operator E by Eak = ak+1 and E i ak = ak+i for all i = 0, 1, . . . .
Thus, formally,


k=0

ak =




E k a0 = (1 E)1 a0 .

k=0

279

(15.2.1)

280

15 The Euler Transformation

Next, let us expand (1 E)1 formally as in

(1 E)1 = [(1 + q) (q + E)]1



q + E 1 
(q + E)k
1
1
= (1 + q)
=
.
k+1
1+q
k=0 (1 + q)

(15.2.2)

Since
(q + E)k a0 =



k  
k  

k ki i
k ki
q E a0 =
q ai ,
i
i
i=0
i=0

(15.2.3)

k  

k ki
1
q ai .
k+1
i
(1 + q)
i=0

(15.2.4)

we nally have formally

ak =

k=0


k=0

The double sum on the right-hand side of (15.2.4) is known as the (E, q) sum of




the latter converges or not. If the (E, q) sum of
k=0 ak , whether
k=0 ak is nite,


we say that k=0 ak is summable (E, q).


ai x i+1 with small x. Consider the bilinear trans2. Consider the power series i=0
formation x = y/(1 qy). Thus, y = x/(1 + q x), so that y 0 as x 0 and


ai x i+1 and expanding in powers
y (1 + q)1 as x 1. Substituting this in i=0
of y, we obtain

ai x i+1 =




i=0

k=0

x
1 + qx

k+1 
k  
k ki
q ai ,
i
i=0

(15.2.5)

which, upon setting x = 1, becomes (15.2.4).

By setting q = 1 in (15.2.4), we obtain

ak =

k=0


(1)k
k=0

2k+1

(k b0 ); bi = (1)i ai , i = 0, 1, . . . .

(15.2.6)

The right-hand side of (15.2.6), the (E, 1) sum of
k=0 ak , is known as the Euler transformation of the latter.
From (15.2.4), it is clear that the (E, q) method produces a sequence of approximations


A n to the sum of
k=0 ak , where
A n =

n

k=0

k  

k ki
1
q ai , n = 0, 1, . . . .
(1 + q)k+1 i=0 i

(15.2.7)

Obviously, A n depends solely on a0 , a1 , . . . , an (hence on A0 , A1 , . . . , An ), which

makes the (E, q) method an acceptable sequence transformation.

15.2 The EulerKnopp (E, q) Method

281

15.2.2 Analytic Properties

It follows from (15.2.7) that the (E, q) method is a linear summability method, namely,


n

n + 1 ni
1

An =
ni Ai , ni =
q , 0 i n. (15.2.8)
(1 + q)n+1 i + 1
i=0
Theorem 15.2.1 Provided q > 0, the (E, q) method is a regular summability method,
i.e., if {An } has limit A, then so does { A n }.
Proof. The proof can be achieved by showing that the SilvermanToeplitz theorem
(Theorem 0.3.3) applies.



n

n
|ni | = i=0
ni = 1 (1 + q)n1 < 1 for
Actually, when q > 0, we have i=0
all n, which implies that the (E, q) method is also a stable summability method. In the
sequel, we assume that q > 0.
The following result shows that the class of EulerKnopp transformations is closed
under composition.
Theorem 15.2.2 Denote by { A n } the sequence obtained by the (E, q) method on {An },
and denote by { A n } the sequence obtained by the (E, r ) method on { A n }. Then { A n } is
also the sequence obtained by the (E, q + r + qr ) method on {An }.
The next result shows the benecial effect of increasing q.
Theorem 15.2.3 If {An } is summable (E, q), then it is also summable (E, q  ) with q  > q.
The proof of Theorem 15.2.2 follows from (15.2.8), and the proof of Theorem 15.2.3
follows from Theorem 15.2.2. We leave the details to the reader.
We next would like to comment on the convergence and acceleration properties of the
EulerKnopp transformation. The geometric series turns out to be very instructive

k for this
q+z
1
n
k

. The sepurpose. Letting ak = z in (15.2.7), we obtain An = (q + 1)

k=0 q+1
quence { A n } converges to (1 z)1 provided z Dq = B(q; q + 1), where B(c; ) =


k
{z : |z c| < }. Since
D D , we
k=0 z converges for D0 = B(0; 1) and since

0 k q
see that the (E, q) method has enlarged the domain of convergence of
k=0 z . Also,
Dq expands as q increases, since Dq Dq  for q < q  . This does not mean that the


k
(E, q) method always accelerates the convergence of
k=0 z , however. Acceleration

takes place only when |q + z| < (q + 1)|z|, that is, only when z is in the exterior of D,

where D = B(c; ) with c = 1/(q + 2) and = (q + 1)/(q + 2). Note that D0 D,

which means that not for every z D0 the (E, q) method accelerates convergence. For
{ A n } converges
{An } and { A n } converge at the same rate, whereas for z D,
z D,

less rapidly than {An }. Let us consider the case q = 1. In this case, D = B(1/3; 2/3), so


k
that for z = 1/4, z = 1/3, and z = 1/2, the series
(E, 1) sum are,
k=0 z and its

1
k
k
k
k
respectively, k=0 (1/4) and 2 k=0 (3/8) , k=0 (1/3) and 12
k=0 (1/3) , and

282

15 The Euler Transformation

k
k
(1/2)k and 12
k=0 (1/4) . Note that
k=0 z is an alternating series for these
values of z.
Another interesting example for which we can give A n in closed form is the logarithmic


k+1
/(k + 1), which converges to log (1 z)1 for |z| 1, z = 1. Using
series
k=0 z
1
the fact that ai = z i+1 /(i + 1) = z i+1 0 t i dt, we have

k=0

 1
k  

k ki
q k+1
(q + z)k+1

, k = 0, 1, . . . ,
(q + zt)k dt =
q ai = z
i
k+1
k+1
0
i=0
so that
A n =

n

k=0

1
k+1



q+z
q +1

k+1

q
q +1

k+1 
, n = 0, 1, . . . .

The (E, q) method thus enlarges the domain of convergence of {An } from B(0; 1)\{1}
to B(q; q + 1)\{1}, as in the previous example. We leave the issue of the domain of
acceleration to the reader.
The following result, whose proof is given in Knopp [152, p. 263], gives a sufcient
condition for the (E, 1) method to accelerate the convergence of alternating series.
Theorem 15.2.4 Let ak = (1)k bk and let the bk be positive and satisfy limk bk = 0
and (1)i i bk 0 for all i and k. In addition, assume that bk+1 /bk > 1/2, k =
0, 1, . . . . Then the sequence { A n } generated by the (E, 1) method converges more rapidly
than {An }. If A = limn An , then
|An A|

1
1
bn+1 b0 n+1 and | A n A| b0 2n1 ,
2
2

so that
 
| A n A|
1 1 n

.
|An A|
2
Remark. Sequences {bk } as in Theorem 15.2.4 are called totally monotonic and we treat
them in more detail in the next chapter. Here, we state only the fact that if bk = f (k),
where f (x) C [0, ) and (1)i f (i) (x) 0 on [0, ) for all i, then {bk } is totally
monotonic.
Theorem 15.2.4 suggests that the Euler transformation is especially effective on al

k
ternating series
k+1 /bk 1 as k . As an illustration, let us
k=0 (1) bk when b


k
apply the (E, 1) method to the series
k=0 (1) /(k + 1) whose sum is log 2. This re
k1
/(k + 1) that converges much more quickly. In this case,
sults in the series k=0 2
| A n A|/|An A| = O(2n ) as n , in agreement with Theorem 15.2.4.
Being linear,
the Euler
 transformation is also effective on innite series of the


p
(i)
(i)
form k=0
i=1 i ck , where each of the sequences {ck }k=0 , i = 1, . . . , p, is as in
Theorem 15.2.4.

15.3 The Aitken 2 -Process

283

Further results on the EulerKnopp (E, q) method as this is applied to power series
can be found in Scraton [261], Niethammer [220], and Gabutti [90].

15.2.3 A Recursive Algorithm

The sequence { A n } generated by the EulerKnopp (E, q) method according to (15.2.7)
can also be computed by using the following recursive algorithm due to Wynn [375],
which resembles very much Algorithm 1.3.1 for the Richardson extrapolation process:
( j)

A1 = A j1 , j = 0, 1, . . . ; (A1 = 0)
( j+1)

A(nj)

( j)

+ q An1
A
, j, n = 0, 1, . . . .
= n1
1+q

(15.2.9)

It can be shown by induction that

A(nj)




n 
q k k
(q) j 
j aj
= A j1 +
 (1) j ,
1 + q k=0 1 + q
q

(15.2.10)

with an as before. It is easy to verify that, for each j 0, {An }

n=0 is the sequence ob

tained by applying the (E, q) method to the sequence {A j+n }n=0 . In particular, A(0)
n = An ,

n = 0, 1, . . . , with An as in (15.2.7). It is also clear from (15.2.9) that the An generated

by the (E, 1) method (the Euler transformation) are obtained from the An by the repeated
application of a simple averaging process.
( j)

15.3 The Aitken 2 -Process

15.3.1 General Discussion of the 2 -Process
We began our discussion of the Aitken 2 -process already in Example 0.1.1 of the
Introduction. Let us recall that the sequence { A m } generated by applying the 2 -process
to {Am } is dened via


 Am Am+1 


 Am Am+1 
Am Am+2 A2m+1
A m = m ({As }) =
(15.3.1)
=
.
 1
Am 2Am+1 + Am+2
1 

 Am Am+1 
Drummond [68] has shown that A m can also be expressed as in
(Am /Am )
A m = m ({As }) =
.
(1/Am )

(15.3.2)

Computationally stable forms of A m are

(Am )2
(Am )(Am+1 )
A m = Am 2
= Am+1
.
 Am
2 A m

(15.3.3)

It is easy to verify that A m is also the solution of the equations Ar +1 A m = (Ar A m ),

r = m, m + 1, where is an additional auxiliary unknown. Therefore, when {Am } is

284

15 The Euler Transformation

such that Am+1 A = (Am A) , m = 0, 1, 2, . . . , (equivalently, Am = A + Cm ,

m = 0, 1, 2, . . . ), for some = 0, 1, we have A m = A for all m.
The 2 -process has been analyzed extensively for general sequences {Am } by Lubkin
[187] and Tucker [339], [340]. The following results are due to Tucker.
1. If {Am } converges to A, then there exists a subsequence of { A m } that converges to A
too.
2. If {Am } and { A m } converge, then their limits are the same.
3. If {Am } converges to A and if |(Am /Am1 ) 1| > 0 for all m, then { A m }
converges to A too.
Of these results the third is quite easy to prove, and the second follows from the rst.
For the rst, we refer the reader to Tucker [339]. Note that none of these results concerns
convergence acceleration. Our next theorem shows convergence acceleration for general
linear sequences. The result in part (i) of this theorem appears in Henrici [130, p. 73,
Theorem 4.5], and that in part (ii) (already mentioned in the Introduction) is a special
case of a more general result of Wynn [371] on the Shanks transformation that we discuss
in the next chapter.
Theorem 15.3.1
(i) If limm (Am+1 A)/(Am A) = for some = 0, 1, then limm ( A m A)/
(Am A) = 0, whether {Am } converges or not.
(ii) If Am = A + am + bm + O( m ) as m , where a, b = 0, || > ||, and
)2 m as m ,
, = 0, 1, and || < min{1, ||}, then A m A b(
1
whether {Am } converges or not.
Proof. We start with the error formula
A m A =



1  Am A Am+1 A
Am+1 
2 Am  Am

(15.3.4)

that follows from (15.3.1). By elementary row transformations, we next have



A m A
1 1 Rm 
Am+1 A
Am+1
; Rm
=
, rm
. (15.3.5)


1
r
Am A
rm 1
Am A
Am
m
Now limm Rm = = 0, 1 implies limm rm = as well, since rm = Rm (Rm+1 1)/
(Rm 1). This, in turn, implies that the right-hand side of the equality in (15.3.5) tends to
0 as m . This proves part (i). The proof of part (ii) can be done by making suitable
substitutions in (15.3.4).


Let us now consider the case in which the condition || > || of part (ii) of
Theorem 15.3.1 is not satised. In this case, we have || = || only, and we cannot make a denitive statement on the convergence of { A m }. We can show instead
that at least a subsequence of { A m } satises A m A = O(m ) as m ; hence,

15.3 The Aitken 2 -Process

285

2
no convergence
This
 acceleration
 follows from the fact that  Am =

2 takes place.


eim + O |/|m as m , where is real and dened
a( 1)2 m 1 + ab 1
1
i
by e = /.
Before proceeding further, we give the following denition, which will be useful in
the remainder of the book. At this point, it is worth reviewing Theorem 6.7.4.

Denition 15.3.2
(i) We say that {Am } b(1) /LOG if
Am A +

i m i as m , = 0, 1, . . . , 0 = 0.

(15.3.6)

i=0

(ii) We say that {Am } b(1) /LIN if

Am A + m

i m i as m , = 1, 0 = 0.

(15.3.7)

i=0

(iii) We say that {Am } b(1) /FAC if

Am A + (m!)r m

i m i as m , r = 1, 2, . . . , 0 = 0. (15.3.8)

i=0

In cases (i) and (ii), {Am } may be convergent or divergent, and A is either the limit or
antilimit of {Am }. In case (iii), {Am } is always convergent and A = limm Am .
Note that, if {Am } is as in Denition 15.3.2, then {Am } b(1) . Also recall that the
sequences in Denition 15.3.2 satisfy Theorem 6.7.4. This fact is used in the analysis of
the different sequence transformations throughout the rest of this work.
The next theorem too concerns convergence acceleration when the 2 -process is
applied to sequences {Am } described in Denition 15.3.2. Note that the rst of the
results in part (ii) of this theorem was already mentioned in the Introduction.
Theorem 15.3.3


0
wi m i as m , w0 = 1
= 0.
(i) If {Am } b(1) /LOG, then A m A i=0

(1)
m
2i

as m , w0 =
(ii) If {Am } b /LIN, then (a) Am A
im
i=0 w

2
) = 0, if = 0 and (b) A m A m i=0
wi m 3i as m ,
0 ( 1
2
) = 0, if = 0 and 1 = 0.
w0 = 21 ( 1


(1)
(iii) If {Am } b /FAC, then A m A (m!)r m i=0
wi m 2r 1i as m ,
2
w0 = 0 r = 0.
Here we have adopted the notation of Denition 15.3.2.
Proof. All three parts can be proved by using the error formula in (15.3.4). We leave the
details to the reader.

286

15 The Euler Transformation

It is easy to show that limm ( A m A)/(Am+2 A) = 0 in parts (ii) and (iii) of

this theorem, which implies convergence acceleration for linear and factorial sequences.
Part (i) shows that there is no convergence acceleration for logarithmic sequences.
Examples in which the 2 -process behaves in peculiar ways can also be constructed.


k/2
/(k + 1), m = 0, 1, . . . . Thus, Am is the mth partial sum of
Let Am = m
k=0 (1)
the convergent innite series 1 + 1/2 1/3 1/4 + 1/5 + 1/6 whose sum is
A = 4 + 12 log 2. The sequence { A m } generated by the 2 -process does not converge in
this case. We actually have
A 2m = A2m + (1)m

2m + 3
2m + 4
and A 2m+1 = A2m+1 + (1)m+1
,
(2m + 2)(4m + 5)
2m + 3

from which it is clear that the sequence { A m } has A, A + 1, and A 1 as its limit
points. (However, the Shanks transformation that we discuss in the next chapter and the
d (2) -transformation are effective on this sequence.) This example is due to Lubkin [187,
Example 7].
15.3.2 Iterated 2 -Process
The 2 -process on {Am } can be iterated as many times as desired. This results in the
following method, which we denote the iterated 2 -process:
( j)

B0 = A j , j = 0, 1, . . . ,
( j)

Bn+1 = j ({Bn(s) }), n, j = 0, 1, . . . .

(15.3.9)

( j)
( j)
[Thus, B1 = A j with A j as in (15.3.1).] Note that Bn is determined by Ak , j k
j + 2n.
The use of this method can be justied with the help of Theorems 15.3.1 and 15.3.3
in the following cases:


m
1. If Am A +
k=1 k k as m , where k = 1 for all k, |1 | > |2 | > ,
limk k = 0 and k = 0 for all k, then it can be shown by expanding A m properly


m
that A m A +
k=1 k k as m , where k are related to the k and satisfy
|1 | > |2 | > . . . , limk k = 0, and 1 = 2 , in addition. Because { A m } converges
more rapidly (or diverges less rapidly) than {Am } and because A m has an asymptotic
expansion of the same form as that of Am , the 2 -process can be applied to { A m }
very effectively, provided k = 1 for all k.
2. If {Am } b(1) /LIN or {Am } b(1) /FAC, then, by parts (ii) and (iii) of Theorem 15.3.3,
A m has an asymptotic expansion of precisely the same form as that of Am and converges more quickly than Am . This implies that the 2 -process can be applied to { A m }
very effectively. This is the subject of the next theorem.

Theorem 15.3.4
(i) If {Am } b(1) /LOG, then as j
Bn( j) A


i=0

wni j i , wn0 =

0
= 0.
(1 )n

15.3 The Aitken 2 -Process

287

(ii) If {Am } b(1) /LIN with = 0, 2, . . . , 2n 2, then as j

Bn( j)

wni j

2ni

, wn0 = (1) 0
n

i=0


2n

( 2i)
= 0.
1
i=0

n1


For the remaining values of we have Bn A = O( j j 2n1 ).

(iii) If {Am } b(1) /FAC, then as j
( j)

Bn( j) A ( j!)r j

wni j (2r +1)ni , wn0 = (1)n 0 ( 2r )n = 0.

i=0

Here we have adopted the notation of Denition 15.3.2.

It can be seen from this theorem that the sequence {Bn+1 }
j=0 converges faster than
( j)
{Bn } j=0 in parts (ii) and (iii), but no acceleration takes place in part (i).
We end with Theorem IV of Shanks [264], which shows that the iterated 2 -process
may also behave peculiarly.
( j)

Theorem 15.3.5 Let {Am } be the sequence of partial sums of the Maclaurin series for
f (z) = 1/[(z z 0 )(z z 1 )], 0 < |z 0 | < |z 1 |, and dene zr = z 0 (z 1 /z 0 )r , r = 2, 3, . . . .
Apply the iterated 2 -process to {Am } as in (15.3.9). Then (i) provided |z| < |z n | and
( j)
( j)
z = zr , r = 0, 1, . . . , Bn f (z) as j , (ii) for z = z 0 and
 z = z 1 , Bn
(
j)
z
z
0 1
as j , and (iii) for z = z 2 = z 12 /z 0 and n 2, Bn 1 (z +z
f (z 2 ) =
2
0
1)
f (z 2 ) as j .
Note that in this theorem, Am = A + am + bm with A = f (z), = z/z 0 , and
( j)
= z/z 1 , || > ||, and suitable a = 0 and b = 0. Then A j = B1 is of the form


( j)
j
k1
B1 = A + k=1 k k , where k = (/) , k = 1, 2, . . . , and k = 0 for all k.


( j)
j
If z = z 2 , then 2 = 2 / = 1, and this implies that B1 = B +
k=1 k k , where
B = A + 2 = A and 1 = 1 and k = k+1 , k = 2, 3, . . . , and k = 1 for all k. Fur( j)
thermore, | k | < 1 for k = 2, 3, . . . , so that B2 B as j for z = z 2 . This
( j)
explains why {Bn } j=0 converges, and to the wrong answer, when z = z 2 , for n 2.
15.3.3 Two Applications of the Iterated 2 -Process
Two common applications of the 2 -process are to the power method for the matrix
eigenvalue problem and to the iterative solution of nonlinear equations.
In the power method for an N N matrix Q, we start with an arbitrary vector x0 and
generate x1 , x2 , . . . , via xm+1 = Qxm . For simplicity, assume that Q is diagonalizable.

p
Then, the vector xm is of the form xm = k=1 vk m
k , where Qvk = k vk for each k and
k are distinct and nonzero and p N . If |1 | > |2 | |3 | , and if the vector y
is such that y v1 = 0, then

p


k m+1
y xm+1
k

m =
=
k=1
=

+
k m
1
p
k ; k = y vk , k = 1, 2, . . . .
m
y xm

k
k
k=1
k=1

288

15 The Euler Transformation

where k are related to the k and 1 > |1 | |2 | and 1 = 2 /1 . Thus, the

iterated 2 -process can be applied to {m } effectively to produce good approximations
to 1 .
If Q is a normal matrix, then better results can be obtained by choosing y = xm . In
this case, m is called a Rayleigh quotient, and

p

2m

xm xm+1
k=1 k |k | k
=

=

+
k |k |2m ; k = vk vk , k = 1, 2, . . . ,
m =
1
p
2m
xm xm

|
|
k
k
k=1
k=1
with the k exactly as before. Again, the iterated 2 -process can be applied to {m }, but
it is much more effective than before.
In the xed-point iterative solution of a nonlinear equation x = g(x), we begin with an
arbitrary approximation x0 to the solution s and generate the sequence of approximations
{xm } via xm+1 = g(xm ). It is known that, provided |g  (s)| < 1 and x0 is sufciently close
to s, the sequence {xm } converges to s linearly in the sense that limm (xm+1 s)/
(xm s) = g  (s). There is, however, a very elegant result concerning the asymptotic
expansion of the xm when g(x) is innitely differentiable in a neighborhood of s, and
this result reads
xm s +

k km as m , 1 = 0, = g  (s),

k=1

for some k that depend only on g(x) and x0 . (See de Bruijn [42, pp. 151153] and also
Meinardus [210].) If {x m } is the sequence obtained by applying the 2 -process to {xm },
then by a careful analysis of x m it follows that
x m s +

k (k+1)m as m .

k=1

It is clear in this problem as well that the iterated 2 -process is very effective, and we
have
Bn( j) s = O((n+1) j ) as j .
In conjunction with the iterative method for the nonlinear equation x = g(x), we
would like to mention a different usage of the 2 -process that reads as follows:
Pick u 0 and set x0 = u 0 .
for m = 1, 2, . . . , do
Compute x1 = g(x0 ) and x2 = g(x1 ).
Compute u m = (x0 x2 x12 )/(x0 2x1 + x2 ).
Set x0 = u m .
end do
This is known as Steffensens method. When g(x) is twice differentiable in a neighborhood of s, provided x0 is sufciently close to s and g  (s) = 1, the sequence {u m }
converges to s quadratically, i.e., limn (u m+1 s)/(u m s)2 = C = .

15.3 The Aitken 2 -Process

289

15.3.4 A Modied 2 -Process for Logarithmic Sequences

For sequences {Am } b(1) /LOG, the 2 -process is not effective, as we have already
seen in Theorem 15.3.3. To remedy this, Drummond [69] has proposed (in the notation
of Denition 15.3.2) the following modication of the 2 -process, which we denote the
2 ( )-process:
1 (Am )(Am+1 )
A m = m ({As }; ) = Am+1
, m = 0, 1, . . . . (15.3.10)

2 A m
[Note that we have introduced the factor ( 1)/ in the Aitken formula (15.3.3).] We
now give a new derivation of (15.3.10), through which we also obtain a convergence
result for { A m }.
Let us set a0 = A0 and am = Am Am1 , m 1, and dene p(m) = am /am , m
0. Using summation by parts, we obtain [see (6.6.16) and (6.6.17)],
Am =

m


p(k)ak = p(m)am+1

k=0

m

[p(k 1)]ak , p(k) = 0 if k < 0. (15.3.11)
k=0

( 1)

1
strictly, so that p(m) A(1)
Now am = h(m) A0
0 strictly with p(m) 1 m +


1
i
as m . As a result, p(m) = 1
+ q(m), where q(m) A(2)
. Thus,
i=0 ci m
0
(15.3.11) becomes

Am = p(m)am+1

m

1
Am
q(k 1)ak ,
1
k=0

from which
Am =

m
1
1 
p(m)am+1 + Bm , Bm =
q(k 1)ak .

k=0

(15.3.12)

( 3)

and applying Theorem 6.7.2, we rst

Noting that q(m 1)am = q(m 1)h(m) A0
( 2)
.
obtain Bm = B + H (m), where B is the limit or antilimit of {Bm } and H (m) A0
( )
( )
1
Because p(m)am+1 A0 strictly, we have that p(m)am+1 + H (m) A0 strictly.
Invoking now that = 0, 1, . . . , we therefore have that B is nothing but A, namely, the
limit or antilimit of {Am }. Thus, we have
Am

1
p(m)am+1 = A + H (m) = A + O(m 2 ) as m . (15.3.13)

Now the left-hand side of (15.3.13) is nothing but A m1 in (15.3.10), and we have also
proved the following result:
Theorem 15.3.6 Let {Am } b(1) /LOG in the notation of Denition 15.3.2 and A m be
as in (15.3.10). Then
A m A +


i=0

wi m 2i as m .

(15.3.14)

290

15 The Euler Transformation

In view of Theorem 15.3.6, we can iterate the 2 ( )-process as in the next theorem,
whose proof is left to the reader.
Theorem 15.3.7 With m ({As }; ) as in (15.3.10), dene
( j)

B0 = A j , j = 0, 1, . . . ,
( j)

Bn+1 = j ({Bn(s) }; 2n), n, j = 0, 1, . . . ,

(15.3.15)

When {Am } b(1) /LOG, we have that

Bn( j) A

wni j 2ni as j , n xed.

(15.3.16)

i=0

Here wni = 0, 0 i I , for some I 0 is possible.

The use of the 2 ( )-process of (15.3.10) in iterated form was suggested by
Drummond [69], who also alluded to the result in (15.3.16). The results in (15.3.14)
and (15.3.16) were also given by Bjrstad, Dahlquist, and Grosse [26].
We call the method dened through (15.3.15) the iterated 2 ( )-process.
It is clear from (15.3.10) and (15.3.15) that to be able to use the 2 ( )- and iterated
2
 ( )-processes we need to have precise knowledge of . In this sense, this method is
less user-friendly than other methods (for example, the Lubkin W -transformation, which
we treat in the next section) that do not need any such input. To remedy this deciency,
Drummond [69] proposed to approximate in (15.3.10) by m dened by
m = 1

(am )(am+1 )
1
.
=1+
2
(am /am )
am am+2 am+1

(15.3.17)

It is easy to show (see Bjrstad, Dahlquist, and Grosse [26]) that

m +

ei m 2i as m .

(15.3.18)

i=0

Using an approach proposed by Osada [225] in connection with some modications

of the Wynn -algorithm, we can use the iterated 2 ( )-process as follows: Taking
(15.3.18) into account, we rst apply the iterated 2 (2)-process to the sequence {m }
to obtain the best possible estimate to . Following that, we apply the iterated 2 ( )process to the sequence {Am } to obtain approximations to A. Osadas work is reviewed
in Chapter 20.

15.4 The Lubkin W -Transformation

The W -transformation of Lubkin [187], when applied to a sequence {Am }, produces the
sequence { A m }, whose members are given by
(Am+1 )(1 rm+1 )
Am+1
A m = Wm ({As }) = Am+1 +
, rm
. (15.4.1)
1 2rm+1 + rm rm+1
Am

15.4 The Lubkin W -Transformation

291

It can easily be veried (see Drummond [68] and Van Tuyl [343]) that A m can also be
expressed as in
(Am+1 (1/Am ))
2 (Am /Am )
=
.
A m = Wm ({As }) =
2
 (1/Am )
2 (1/Am )
( j)

(15.4.2)

( j)

Note also that the column {2 } j=0 of the -algorithm and the column {u 2 } of the Levin
u-transformation are identical to the Lubkin W -transformation.
Finally, with the help of the rst formula in (15.4.2), it can be shown that Wm
Wm ({As }) can also be expressed in terms of the k k ({As }) of the 2 -process as
follows:
Wm =

1 rm
m+1 qm m
, qm = rm+1
.
1 qm
1 rm+1

(15.4.3)

The following are known:

1. A m = A for all m = 0, 1, . . . , if and only if Am is of the form
Am = A + C

m

ak + b + 1
k=1

ak + b

, C = 0, a = 1, ak + b = 0, 1, k = 1, 2, . . . .

Obviously, Am = A + Cm , = 0, 1, is a special case. This result on the kernel of

the Lubkin transformation is due to Cordellier [56]. We obtain it as a special case of
a more general one on the Levin transformations later.
2. Wimp [366] has shown that, if |rm | < for all m, where 0 < || < 1 and
1/2
= [| 1|2 + (|| + 1)2 ] (|| + 1), and if {Am } A, then { A m } A too.
3. Lubkin [187] has shown that, if limm (Am+1 A)/(Am A) = for some =
0, 1, then limm ( A m A)/(Am A) = 0. [This can be proved by combining
(15.4.3), part (i) of Theorem 15.3.3, and the fact that limm rm = .]
For additional results of a general nature on convergence acceleration, see Lubkin [187]
and Tucker [339], [340]. The next theorem shows that the W -transformation accelerates
the convergence of all sequences discussed in Denition 15.3.2 and is thus analogous
to Theorem 15.3.3. We leave its proof to the reader. Note that parts (i) and (ii) of this
theorem are special cases of results corresponding to the Levin transformation that were
given by Sidi [273]. Part (i) of this theorem was also given by Sablonni`ere [248] and
Van Tuyl [343]. Part (iii) was given recently by Sidi [307]. For details, see [307].
Theorem 15.4.1


wi m i as m , such that
(i) If {Am } b(1) /LOG, then A m A i=0
2

is an integer 2. Thus Am A = O(m ) as m .

wi m i as m , such that
(ii) If {Am } b(1) /LIN, then A m A m i=0
is an integer 3. Thus A m A = O( m m 3 ) as m .


(iii) If {Am } b(1) /FAC, then A m A (m!)r m i=0
wi m 3r 2i as m ,
3
r m 3r 2

w0 = 0 r (r + 1) = 0. Thus, Am A = O((m!) m
) as m .
Here we have adopted the notation of Denition 15.3.2.

292

15 The Euler Transformation

It is easy to show that limm ( A m A)/(Am+3 A) = 0 in all three parts of this

theorem, which implies convergence acceleration for linear, factorial, and logarithmic
sequences.
Before we proceed further, we note that, when applied to the sequence {Am }, Am =

m
k/2
/(k + 1), m = 0, 1, . . . , the W -transformation converges, but is not better
k=0 (1)
than the sequence itself, as mentioned in [187].
It follows from Theorem 15.4.1 that the Lubkin W -transformation can also be
iterated. This results in the following method, which we denote the iterated Lubkin
transformation:
( j)

B0 = A j , j = 0, 1, . . . ,
( j)

Bn+1 = W j ({Bn(s) }), n, j = 0, 1, . . . .

(15.4.4)

( j)
( j)
[Thus, B1 = A j with A j as in (15.3.1).] Note that Bn is determined by Ak , j k
j + 3n.
The results of iterating the W -transformation are given in the next theorem. Again,
part (i) of this theorem was given in [248] and [343], and parts (ii) and (iii) were given
recently in [307].

Theorem 15.4.2
(i) If {Am } b(1) /LOG, then there exist constants k such that 0 = and k k1
are integers 2, for which, as j ,
Bn( j) A

wni j n i , wn0 = 0.

i=0

(ii) If {Am } b(1) /LIN, then there exist constants k such that 0 = and k k1
are integers 3, for which, as j ,
Bn( j) A j

wni j n i , wn0 = 0.

i=0

(iii) If {Am } b(1) /FAC, then as j

Bn( j) A ( j!)r j

wni j (3r +2)ni , wn0 = 0 [ 3r (r + 1)]n = 0.

i=0

Here we have adopted the notation of Denition 15.3.2.

It can be seen from this theorem that the sequence {Bn+1 }
j=0 converges faster than
( j)
{Bn }
for
all
sequences
considered.
j=0
( j)

15.5 Stability of the Iterated 2 -Process and Lubkin Transformation

293

15.5 Stability of the Iterated 2 -Process and Lubkin Transformation

15.5.1 Stability of the Iterated 2 -Process
From (15.3.1), we can write A m = 0 Am + 1 Am+1 , where 0 = Am+1 /2 Am and
( j)
1 = Am /2 Am . Consequently, we can express Bn as in
( j)

Bn+1 = (nj) Bn( j) + (nj) Bn( j+1) ,

(15.5.1)

(nj) = Bn( j+1) /2 Bn( j) and (nj) = Bn( j) /2 Bn( j) ,

(15.5.2)

where

and

( j)
Bn

( j+1)
Bn

( j)
Bn

for all j and n. Thus, we can write

Bn( j) =

n


( j)

ni A j+i ,

(15.5.3)

i=0
( j)

where the ni satisfy the recursion relation

( j)

( j)

( j+1)

n+1,i = (nj) ni + (nj) n,i1 , i = 0, 1, . . . , n + 1.

(15.5.4)

( j)

( j)

( j)

Here we dene ni = 0 for i < 0 and i > n. In addition, from the fact that n + n = 1,

n
( j)
ni = 1.
it follows that i=0
Let us now dene
n
n


( j)
( j)
ni z i and n( j) =
|ni |.
(15.5.5)
Pn( j) (z) =
i=0

i=0
( j)
n

As we have done until now, we take

as a measure of propagation of the errors in
( j)
( j)
( j)
the Am into Bn . Our next theorem, which seems to be new, shows how Pn (z) and n
behave as j . As before, we adopt the notation of Denition 15.3.2.
Theorem 15.5.1
(i) If {Am } b(1) /LIN, with = 0, 2, . . . , 2n 2, then




z n
1 + | | n
and lim n( j) =
.
lim Pn( j) (z) =
j
j
1
|1 |
(ii) If {Am } b(1) /FAC, then
lim Pn( j) (z) = z n and lim n( j) = 1.

( j)

Proof. Combining Theorem 15.3.4 and (15.5.2), we rst obtain that (i) n /( 1)
( j)
( j)
and n 1/( 1) as j for all n in part (i), and (ii) lim j n = 0 and
( j)


lim j n = 1 for all n in part (ii). We leave the rest of the proof to the reader.
Note that we have not included the case in which {Am } b(1) /LOG in this theorem.
The reason for this is that we would not use the iterated 2 -process on such sequences,
and hence discussion of stability for such sequences is irrelevant. For linear and factorial
sequences Theorem 15.5.1 shows stability.

294

15 The Euler Transformation

15.5.2 Stability of the Iterated Lubkin Transformation

From the second formula in (15.4.2), we can write A m = 1 Am+1 + 2 Am+2 , where 1 =
(1/Am )/2 (1/Am ) and 2 = (1/Am+1 )/2 (1/Am ). Consequently, we can
( j)
express Bn as in
( j)

Bn+1 = (nj) Bn( j+1) + (nj) Bn( j+2) ,

(15.5.6)

where
( j)

(nj) =
( j)

( j+1)

and Bn = Bn

( j+1)

(1/Bn )
( j)
2 (1/Bn )

and (nj) =

(1/Bn

( j)
2 (1/Bn )

(15.5.7)

( j)

Bn for all j and n. Thus, we can write

Bn( j) =

n


( j)

(15.5.8)

( j+2)

(15.5.9)

ni A j+n+i ,

i=0
( j)

where the ni satisfy the recursion relation

( j)

( j+1)

n+1,i = (nj) ni

+ (nj) n,i1 , i = 0, 1, . . . , n + 1.

( j)

( j)

( j)

Here we dene ni = 0 for i < 0 and i > n. Again, from the fact that n + n = 1,

n
( j)
ni = 1.
there holds i=0
Let us now dene again
Pn( j) (z) =

n


( j)

ni z i and n( j) =

i=0

n


( j)

|ni |.

(15.5.10)

i=0

( j)

( j)

Again, we take n as a measure of propagation of the errors in the Am into Bn . The

( j)
( j)
next theorem, due to Sidi [307], shows how Pn (z) and n behave as j . The
proof of this theorem is almost identical to that of Theorem 15.5.1. Only this time we
invoke Theorem 15.4.2.
Theorem 15.5.2
(i) If {Am } b(1) /LOG, then

n1
n1
 1
 1
( j)
n n
( j)

Pn (z)
k
(1 z) j and n  k  (2 j)n as j ,
k=0

k=0

where k are as in part (i) of Theorem 15.4.2.

(ii) If {Am } b(1) /LIN, then




z n
1 + | | n
( j)
( j)
lim Pn (z) =
and lim n =
.
j
j
1
|1 |
(iii) If {Am } b(1) /FAC, then
lim Pn( j) (z) = z n and lim n( j) = 1.

15.7 Further Convergence Results

295

Thus, for logarithmic sequences the iterated Lubkin transformation is not stable,
whereas for linear and factorial sequences it is.

15.6 Practical Remarks

As we see from Theorems 15.5.1 and 15.5.2, for both of the iterated transformations,
( j)
n is proportional to (1 )n as j when {Am } b(1) /LIN. This implies that, as
approaches 1, the singular point of A, the methods suffer from diminishing stability. In
( j)
addition, the theoretical error Bn A for such sequences, as j , is proportional
2n
in the case of the iterated 2 -process and to (1 )n in the case of the
to (1 )
iterated Lubkin transformation. (This is explicit in Theorem 15.3.4 for the former, and it
can be shown for the latter by rening the proof of Theorem 15.4.2 as in [307].) That is,
( j)
( j)
as 1, the theoretical error Bn A and n grow simultaneously. We can remedy
both problems by using arithmetic progression sampling (APS), that is, by applying
the methods to a subsequence {Am+ }, where and are xed integers with 2.
The justication for this follows from the fact that the factor (1 )n is now replaced
everywhere by (1 )n , which can be kept at a moderate size by a judicious choice of
that removes from 1 in the complex plane sufciently. Numerical experience shows
that this strategy works very well.
From Theorems 15.3.4 and 15.4.2, only the iterated Lubkin transformation is useful
for sequences {Am } b(1) /LOG. From Theorem 15.5.2, this method is unstable even
though it is convergent mathematically. The most direct way of obtaining more accuracy
( j)
from the Bn in nite-precision arithmetic is by increasing the precision being used
(doubling it, for example). Also, note that, when |! | is sufciently large, the product
n1
( j)
( j)
k=0 |k | increases in size, which makes n small (even though n as j ).
( j)
This implies that good numerical accuracy can be obtained for the Bn without having
to increase the precision of the arithmetic being used in such a case.
As for sequences {Am } b(1) /FAC, both methods are effective and stable numerically,
as the relevant theorems show.
( j)
Finally, the diagonal sequences {Bn }
n=0 with xed j (e.g., j = 0) have better con( j)
vergence properties than the column sequences {Bn }
j=0 with xed n.

15.7 Further Convergence Results

In this section, we state two more convergence results that concern the application of
the iterated modied 2 -process (in a suitably modied form) and the iterated Lubkin
transformation to sequences {Am } for which
Am A +

i m i/ p as m , p 2 integer,

i=0

=

i
, i = 0, 1, . . . , 0 = 0.
p

(15.7.1)

( , p) strictly. (Recall that the case with p = 1 is already

Obviously, Am A = h(m) A
0
treated in Theorems 15.3.7 and 15.4.2.)

296

15 The Euler Transformation

The following convergence acceleration results were proved by Sablonni`ere [248]

with p = 2 and by Van Tuyl [343] with general p.
Theorem 15.7.1 Consider the sequence {Am } described in (15.7.1) and, with
m ({As }; ) as dened in (15.3.10), set
( j)

B0 = A j , j = 0, 1, . . . ,
( j)

Bn+1 = j ({Bn(s) }; n/ p), n, j = 0, 1, . . . .

(15.7.2)

Then
Bn( j) A

wni j (n+i)/ p ) as j .

(15.7.3)

i=0

Here wni = 0, 0 i I , for some I 0 is possible.

We denote the method described by (15.7.2) the iterated 2 ( , p)-process. It is
important to realize that this method does not reduce to the iterated 2 ( )-process upon
setting p = 1.
( j)

Theorem 15.7.2 Consider the sequence {Am } described in (15.7.1) and let Bn be as
dened via the iterated Lubkin transformation in (15.4.4). Then there exist scalars k
such that 0 = and (k k1 ) p are integers 1, k = 1, 2, . . . , for which
Bn( j) A


i=0

wni j n i/ p as j , wn0 = 0.

(15.7.4)

16
The Shanks Transformation

16.1 Derivation of the Shanks Transformation

In our discussion of algorithms for extrapolation methods in the Introduction, we presented a brief treatment of the Shanks transformation and the -algorithm as part of
this discussion. This transformation was originally derived by Schmidt [258] for solving
linear systems by iteration. After being neglected for a long time, it was resurrected by
Shanks [264], who also gave a detailed study of its remarkable properties. Shanks paper
was followed by that of Wynn [368], in which the -algorithm, the most efcient implementation of the Shanks transformation, was presented. The papers of Shanks and
Wynn made an enormous impact and paved the way for more research in sequence
transformations.
In this chapter, we go into the details of the Shanks transformation, one of the most
useful sequence transformations to date. We start with its derivation.
Let {Am } be a given sequence with limit or antilimit A. Assume that {Am } satises
Am A +

k m
k as m ,

(16.1.1)

k=1

for some nonzero constants k and k independent of m, with k distinct and k = 1 for
all k, and |1 | |2 | , such that limk k = 0. Assume, furthermore, that the
k are not necessarily known. Note that the condition that limk k = 0 implies that
there can be only a nite number of k that have the same modulus.
Obviously, when |1 | < 1, the sequence {Am } converges and limm Am = A. When
|1 | = 1, {Am } diverges but is bounded. When |1 | > 1, {Am } diverges and is unbounded.
In case the summation in (16.1.1) is nite and, therefore,
Ar = A +

n


k rk , r = 0, 1, . . . ,

(16.1.2)

k=1

we can determine A and the 2n parameters k and k by solving the following system
of nonlinear equations:
Ar = A +

n


k rk , r = j, j + 1, . . . , j + 2n, for arbitrary xed j. (16.1.3)

k=1

297

298

16 The Shanks Transformation

n
Now, 1 , . . . , n are the zeros of some polynomial P() = i=0
wi i , w0 = 0 and
wn = 0. Here the wi are unique up to a multiplicative constant. Then, from (16.1.3), we
have that
n


wi (Ar +i A) = 0, r = j, j + 1, . . . , j + n.

(16.1.4)

i=0

Note that we have eliminated the parameters 1 , . . . , n from equations (16.1.3). It

follows from (16.1.4) that
n


wi Ar +i =


n

i=0


wi A, r = j, j + 1, . . . , j + n.

(16.1.5)

i=0

n
wi = P(1) = 0. This enables us to scale
Now, because k = 1 for all k, we have i=0

n
the wi such that i=0 wi = 1, as a result of which (16.1.5) can be written in the form
n


wi Ar +i = A, r = j, j + 1, . . . , j + n;

i=0

n


wi = 1.

(16.1.6)

i=0

This is a linear system in the unknowns A and w0 , w1 , . . . , wn . Using the fact that
Ar +1 = Ar + Ar , we have that
n

i=0

wi Ar +i =


n

n1


wi Ar +

i=0

w
i Ar +i ; w
i =

i=0

n


w p , i = 0, 1, . . . , n 1,

p=i+1

(16.1.7)
which, when substituted in (16.1.6), results in
A = Ar +

n1


w
i Ar +i , r = j, j + 1, . . . , j + n.

(16.1.8)

i=0

Finally, by making the substitution i = w

i1 in (16.1.8), we obtain the linear system
Ar = A +

n


i Ar +i1 , r = j, j + 1, . . . , j + n.

(16.1.9)

i=1

On the basis of (16.1.9), we now give the denition of the Shanks transformation,
hoping that it accelerates the convergence of sequences {Am } that satisfy (16.1.1).
Denition 16.1.1 Let {Am } be an arbitrary sequence. Then the Shanks transformation
on this sequence is dened via the linear systems
Ar = en (A j ) +

n


i Ar +i1 , r = j, j + 1, . . . , j + n,

(16.1.10)

i=1

where en (A j ) are the approximations to the limit or antilimit of {Am } and i are additional
auxiliary unknowns.

16.1 Derivation of the Shanks Transformation

299

Solving (16.1.10) for en (A j ) by Cramers rule, we obtain the following determinant

( j)
representation for An :



Aj
A j+1 . . .
A j+n 

 A j A j+1 . . . A j+n 




..
..
..


.
.
.


 A

j+n1 A j+n . . . A j+2n1
en (A j ) = 
(16.1.11)
.


1
1 ...
1


 A j A j+1 . . . A j+n 




..
..
..


.
.
.


 A

A
. . . A
j+n1

j+n

j+2n1

[Obviously, e1 (A j ) = j ({As }), dened in (15.3.1), that is, the e1 -transformation is the
Aitken 2 -process.] By performing elementary row transformations on the numerator
and denominator determinants in (16.1.11), we can express en (A j ) also in the form
( j)

en (A j ) =

Hn+1 ({As })
( j)

Hn ({2 As })

where H p(m) ({u s }) is a Hankel determinant dened by


 u m u m+1 . . . u m+ p1

 u m+1 u m+2 . . . u m+ p

H p(m) ({u s }) = 
..
..
..

.
.
.

u
u
... u
m+ p1

m+ p

m+2 p2

(16.1.12)






(m)
 ; H0 ({u s }) = 1.




(16.1.13)

Before proceeding further, we mention that the Shanks transformation as given in

(16.1.11) was derived by Tucker [341] through an interesting geometric approach.
Tuckers approach generalizes that of Todd [336, p. 260], which concerns the 2 -process.
The following result on the kernel of the Shanks transformation, which is completely
related to the preceding developments, is due to Brezinski and Crouzeix [40]. We note
that this result also follows from Theorem 17.2.5 concerning the Pade table for rational
functions.
Theorem 16.1.2 A necessary and sufcient condition for en (A j ) = A, j = J, J +
1, . . . , to hold with minimal n is that there exist constants w0 , w1 , . . . , wn such that

n
wi = 0, and that
wn = 0 and i=0
n


wi (Ar +i A) = 0, r J,

(16.1.14)

i=0

which means that

Ar = A +

t


Pk (r )rk , r = J, J + 1, . . . ,

k=1

where Pk (r ) is a polynomial in r of degree exactly pk ,

distinct and k = 1 for all k.

t

i=1 ( pi

(16.1.15)

+ 1) = n, and k are

300

16 The Shanks Transformation

Proof. Let us rst assume that en (A j ) = A for every j J . Then the equations in

n
i Ar +i1 , j r j + n. The i are thus the
(16.1.10) become Ar A = i=1
solution of n of these n + 1 equations. For j = J , we take these n equations to
be those for which j + 1 r j + n, and for j = J + 1, we take them such that
j r j + n 1. But these two systems are identical. Therefore, the i are the same
for j = J and for j = J + 1. By induction, the i are the same for all j J . Also, since
n is smallest, n = 0 necessarily. Thus, (16.1.10) is the same as (16.1.9) for any j J,
with the i independent of j and n = 0.
Working backward from this, we reach (16.1.4), in which the wi satisfy wn = 0 and

n
as (16.1.14). Let
i=0 wi = 0 and are independent of j when j J ; this is the same

n
wi = 0. This
us next assume, conversely, that (16.1.14) holds with wn = 0 and i=0
implies (16.1.4) for all j J . Working forward from this, we reach (16.1.9) for all
j J , and hence (16.1.10) for all j J , with en (A j ) = A. Finally, because (16.1.14)
is an (n + 1)-term recursion relation for {Ar A}r J with constant coefcients, its
solutions are all of the form (16.1.15).


Two types of sequences are of interest in the application of the Shanks transformation:
1. {en (A j )}
j=0 with n xed. In analogy with the rst generalization of the Richardson
extrapolation process, we call them column sequences.
2. {en (A j )}
n=0 with j xed. In analogy with the rst generalization of the Richardson
extrapolation process, we call them diagonal sequences.
In general, diagonal sequences appear to have much better convergence properties than
column sequences.
Before closing, we recall that in case the k in (16.1.1) are known, we can also use the
Richardson extrapolation process for innite sequences of Section 1.9 to approximate
A. This is very effective and turns out to be much less expensive than the Shanks
transformation. [Formally, it takes 2n + 1 sequence elements to eliminate the terms
k m
k , k = 1, . . . , n, from (16.1.1) by the Shanks transformation, whereas the same task
is achieved by the Richardson extrapolation process with only n + 1 sequence elements.]
When the k are not known and can be arbitrary, the Shanks transformation appears to
be the only extrapolation method that can be used.
We close this section with the following result due to Shanks, which can be proved by
applying appropriate elementary row and column transformations to the numerator and
denominator determinants in (16.1.11).


k
Theorem 16.1.3 Let Am = m
k=0 ck z , m = 0, 1, . . . . If the Shanks transformation is
applied to {Am }, then the resulting en (A j ) turns out to be f j+n,n (z), the [ j + n/n] Pade


k
approximant from the innite series f (z) :=
k=0 ck z .
Note. Recall that f m,n (z) is a rational function whose numerator and denominator
polynomials have degrees at most m and n, respectively, that satises, and is uniquely
determined by, the requirement f m,n (z) f (z) = O(z m+n+1 ) as z 0. The subject of
Pade approximants is considered in some detail in the next chapter.

16.2 Algorithms for the Shanks Transformation

301

16.2 Algorithms for the Shanks Transformation

Comparing (16.1.10), the equations dening en (A j ), with (3.1.4), we realize that en (A j )
can be expressed in the form
( j)

en (A j ) =

f n (a)
( j)
f n (I )

|g1 ( j) gn ( j) a( j)|
,
|g1 ( j) gn ( j) I ( j)|

(16.2.1)

where
a(l) = Al , and gk (l) = Ak+l1 for all l 0 and k 1.

(16.2.2)

It is possible to use the FS-algorithm or the E-algorithm to compute the en (A j ) recursively.

Direct application of these algorithms without taking the special nature of the gk (l) into
account results in expensive computational procedures, however. When A0 , A1 , . . . , A L
are given, the costs of these algorithms are O(L 3 ), as explained in Chapter 3. When
the special structure of the gk (l) in (16.2.2) is taken into account, these costs can be
reduced to O(L 2 ) by the rs- and FS/qd-algorithms that are discussed in Chapter 21 on the
G-transformation.
The most elegant and efcient algorithm for implementing the Shanks transformation
is the famous -algorithm of Wynn [368]. As the derivation of this algorithm is very
complicated, we do not give it here. We refer the reader to Wynn [368] or to Wimp [366,
pp. 244247] instead. Here are the steps of the -algorithm.
Algorithm 16.2.1 (-algorithm)
( j)

( j)

1. Set 1 = 0 and 0 = A j , j = 0, 1, . . . .
( j)
2. Compute the k by the recursion
( j)

( j+1)

k+1 = k1 +

( j+1)
k

( j)

, j, k = 0, 1, . . . .

Wynn has shown that

( j)

( j)

2n = en (A j ) and 2n+1 =

1
for all j and n.
en (A j )

(16.2.3)

( j)

[Thus, if {Am } has a limit and the 2n converge, then we should be able to observe that
( j)
|2n+1 | both as j and as n .]
( j)
Commonly, the k are arranged in a two-dimensional array as in Table 16.2.1. Note
( j)
that the sequences {2n } j=0 form the columns of the epsilon table, and the sequences

( j)
{2n }n=0 form its diagonals.
It is easy to see from Table 16.2.1 that, given A0 , A1 , . . . , A K , the -algorithm com( j)
( j)
putes k for 0 j + k K . As the number of these k is K 2 /2 + O(K ), the cost of
this computation is K 2 + O(K ) additions, K 2 /2 + O(K ) divisions, and no multiplications. (We show in Chapter 21 that the FS/qd-algorithm has about the same cost as the
-algorithm.)

302

16 The Shanks Transformation

Table 16.2.1: The -table
(0)
1
(1)
1
(2)
1

(3)
1
..
.
..
.
..
.
..
.
..
.

0(0)
0(1)
0(2)
0(3)
..
.
..
.
..
.
..
.

1(0)

2(0)

1(1)

3(0)

2(1)

1(2)

..

..

..

3(1)

2(2)
1(3)
..
.
..
.
..
.

..

3(2)
2(3)
..
.
..
.

3(3)
..
.

( j)

( j)

Since we are interested only in the 2n by (16.2.3), and since the 2n+1 are auxiliary
quantities, we may ask whether it is possible to obtain a recursion relation among the
( j)
2n only. The answer to this question, which is in the afrmative, was given again by
Wynn [372], the result being the so-called cross rule:
1
( j1)
2n+2

( j)
2n

1
( j+1)
2n2

( j)
2n

1
( j1)
2n

( j)
2n

1
( j+1)
2n

( j)

2n

with the initial conditions

( j)

( j)

2 = and 0 = A j , j = 0, 1, . . . .
Another implementation of the Shanks transformation proceeds through the qdalgorithm that is related to continued fractions and that is discussed at length in the
next chapter. The connection between the - and qd-algorithms was discovered and
analyzed by Bauer [18], [19], who also developed another algorithm, denoted the algorithm, that is closely related to the -algorithm. We do not go into the -algorithm
here, but refer the reader to [18] and [19]. See also the description given in Wimp [366,
pp. 160165].

16.3 Error Formulas

Before we embark on the error analysis of the Shanks transformation, we need appropriate
( j)
error formulas for en (A j ) = 2n .
Lemma 16.3.1 Let Cm = Am A, m = 0, 1, . . . . Then
( j)

( j)

2n A =

Hn+1 ({Cs })
( j)

Hn ({2 Cs })

(16.3.1)

16.4 Analysis of Column Sequences When Am A +

k=1

k m
k

303

If we let Cm = m Dm , m = 0, 1, . . . , then
( j)

( j)

2n A = j+2n

Hn+1 ({Ds })
( j)

Hn ({E s })

; E m = Dm 2 Dm+1 + 2 Dm+2 for all m. (16.3.2)

Proof. Let us rst subtract A from both sides of (16.1.11). This changes the rst row of
the numerator determinant to [C j , C j+1 , . . . , C j+n ]. Realizing now that As = Cs in
the other rows, and adding the rst row to the second, the second to the third, etc., in this
( j)
( j)
order, we arrive at Hn+1 ({Cs }). We already know that the denominator is Hn ({2 As }).
But we also have that 2 Am = 2 Cm . This completes the proof of (16.3.1). To prove
(16.3.2), substitute Cm = m Dm everywhere and factor out the powers of from the
rows and columns in the determinants of both the numerator and the denominator. This
completes the proof of (16.3.2).


( j)

Both (16.3.1) and (16.3.2) are used in the analysis of the column sequences {2n } j=0
in the next two sections.
16.4 Analysis of Column Sequences When Am A +

k=1

k m
k

Let us assume that {Am } satises

Am A +

k m
k as m ,

(16.4.1)

k=1

with k = 0 and k = 0 constants independent of m and

k distinct, k = 1 for all k; |1 | |2 | , lim k = 0.
k

(16.4.2)

What we mean by (16.4.1) is that, for any integer N 1, there holds

Am A

N 1


m
k m
k = O( N ) as m .

k=1

Also, limk k = 0 implies that there can be only a nite number of k with the same
modulus. We say that such sequences are exponential.
Because the Shanks transformation was derived with the specic intention of accelerating the convergence of sequences {Am } that behave as in (16.4.1), it is natural to rst
analyze its behavior when applied to such sequences.


We rst recall from Theorem 16.1.2 that, when Am = A + nk=1 k m
k for all m, we
( j)
have A = 2n for all j. From this, we can expect the Shanks transformation to perform
well when applied to the general case in (16.4.1). This indeed turns out to be the case,
but the theory behind it is not simple. In addition, the existing theory pertains only to
column sequences; nothing is known about diagonal sequences so far.
( j)
We carry out the analysis of the column sequences {2n }
j=0 with the help of the
following lemma of Sidi, Ford, and Smith [309, Lemma A.1].
Lemma 16.4.1 Let i 1 , . . . , i k be positive integers, and assume that the scalars vi1 ,... ,ik are
odd under an interchange of any two of the indices i 1 , . . . , i k . Let ti, j , i 1, 1 j k,

304

16 The Shanks Transformation

be scalars. Dene Ik,N and Jk,N by

Ik,N =

N


i 1 =1

and

Jk,N

N 
k

i k =1


ti p , p vi1 ,... ,ik

(16.4.3)

p=1


 ti1 ,1

 ti1 ,2


=
 .
 .
1i 1 <i 2 <<i k N  .
t


ti2 ,1 tik ,1 
ti2 ,2 tik ,2 
..
..  vi1 ,... ,ik .
.
. 
t t 

i 1 ,k i 2 ,k

(16.4.4)

i k ,k

Then
Ik,N = Jk,N .

(16.4.5)

Proof. Let &k be the set of all permutations of the index set {1, 2, . . . , k}. Then, by
the denition of determinants,


k


(sgn )
ti( p) , p vi1 ,... ,ik .
(16.4.6)
Jk,N =
1i 1 <i 2 <<i k N &k

p=1

Here sgn , the signature of , is +1 or 1 depending on whether is even or odd,

respectively. The notation ( p), where p {1, 2, . . . , k}, designates the image of
as a function operating on the index set. Now 1 ( p) = p when 1 p k, and
sgn 1 = sgn for any permutation &k . Hence


k


Jk,N =
(sgn 1 )
ti ( p) , p vi 1 (1) ,... ,i 1 (k) . (16.4.7)
1i 1 <i 2 <<i k N &k

p=1

By the oddness of vi1 ,... ,ik , we have

(sgn 1 )vi 1 (1) ,... ,i 1 (k) = vi (1) ,... ,i (k) .

(16.4.8)

Substituting (16.4.8) in (16.4.7), we obtain

Jk,N =

k
 

1i 1 <i 2 <<i k N &k


ti( p) , p vi (1) ,... ,i (k) .

(16.4.9)

p=1

Because vi1 ,... ,ik is odd under an interchange of the indices i 1 , . . . , i k , it vanishes when
any two of the indices are equal. Using this fact in (16.4.3), we see that Ik,N is just the
sum over all permutations of the distinct indices i 1 , . . . , i k . The result now follows by
comparison with (16.4.9).


We now prove the following central result:
Lemma 16.4.2 Let { f m } be such that
fm


k=1

ek m
k as m ,

(16.4.10)

16.4 Analysis of Column Sequences When Am A +

k=1

k m
k

305

( j)

with the k exactly as in (16.4.2). Then, Hn ({ f s }) satises


n

Hn( j) ({ f s })

1k1 <k2 <<kn


j
ek p k p [V (k1 , . . . , kn )]2 as j . (16.4.11)

p=1

( j)

Proof. Let us substitute (16.4.10) in Hn



j


k1 ek1 k1

j+1
 k2 ek2 k2
( j)

Hn ({ f s })  .
 ..


 k ekn kj+n1
n
n

({ f s }). We obtain






j+1
j+n1 
ek1 k1
ek1 k1

k
k
1
1





j+2
j+n


k2 ek2 k2
k2 ek2 k2
 , (16.4.12)
..
..


.
.




j+n
j+2n2 

kn ekn kn
kn ekn kn

where we have used k to mean
k=1 and we have also used P( j) Q( j) to mean
P( j) Q( j) as j . We continue to do so below.
By the multilinearity property of determinants with respect to their rows, we can
j+i1
from the ith row
move the summations outside the determinant. Factoring out eki ki
of the remaining determinant, and making use of the denition of the Vandermonde
determinant given in (3.5.4), we obtain
Hn( j) ({ f s })


k1

n
 
kn

p1

k p

 
k

p=1



j
ek p k p V (k1 , . . . , kn ) . (16.4.13)

p=1

Because the term inside the square brackets is odd under an interchange of any two
of the indices k1 , . . . , kn , Lemma 16.4.1 applies, and by invoking the denition of the
Vandermonde determinant again, the result follows.


The following is the rst convergence result of this section. In this result, we make
use of the fact that there can be only a nite number of k with the same modulus.
Theorem 16.4.3 Assume that {Am } satises (16.4.1) and (16.4.2). Let n and r be positive
integers for which
|n | > |n+1 | = = |n+r | > |n+r +1 |.

(16.4.14)

Then
( j)
2n

A=

n+r


p=n+1


n
i=1

p i
1 i

2
j

pj + o(n+1 ) as j .

(16.4.15)

Consequently,
( j)

2n A = O(n+1 ) as j .
All this is valid whether {Am } converges or not.

(16.4.16)

306

16 The Shanks Transformation

( j)

Proof. Applying Lemma 16.4.2 to Hn+1 ({Cs }), we obtain

( j)
Hn+1 ({Cs })

 n+1


1k1 <k2 <<kn+1


j
k p k p

2
V (k1 , k2 , . . . , kn+1 ) . (16.4.17)

p=1

The most dominant terms in (16.4.17) are those with k1 = 1, k2 = 2, . . . , kn = n, and

kn+1 = n+i , i = 1, . . . , r , their number being r . Thus,


 
n
n+r

2
( j)
j
j
j
Hn+1 ({Cs }) =
pp
p p V (1 , . . . , n , p ) + o(n+1 ) as j .
p=1

p=n+1

(16.4.18)
Observing that
2 Cm

2
k m
k as m ; k = k (k 1) for all k,

(16.4.19)

k=1

and applying Lemma 16.4.2 again, we obtain



n


2
j
k p k p V (k1 , k2 , . . . , kn ) . (16.4.20)
Hn( j) ({2 Cs })
1k1 <k2 <<kn

p=1

The most dominant term in (16.4.20) is that with k1 = 1, k2 = 2, . . . , kn = n. Thus,



n
( j)
2
j
Hn ({ Cs })
p p [V (1 , . . . , n )]2 .
(16.4.21)
p=1

Dividing (16.4.18) by (16.4.21), which is allowed because the latter is an asymptotic

equality, and recalling (16.3.1), we obtain (16.4.15), from which (16.4.16) follows
immediately.


This theorem was stated, subject to the condition that either 1 > 2 > > 0 or
1 < 2 < < 0, in an important paper by Wynn [371]. The general result in (16.4.15)
was given by Sidi [296].
Corollary 16.4.4 If r = 1 in Theorem 16.4.3, i.e., if |n | > |n+1 | > |n+2 |, then
(16.4.15) can be replaced by the asymptotic equality

2
n
n+1 i
( j)
j
n+1 as j .
(16.4.22)
2n A n+1
1

i
i=1
It is clear from Theorem 16.4.3 that the column sequence {2n }
j=0 converges when
( j)

|n+1 | < 1 whether {Am } converges or not. When |n+1 | 1, {2n }

j=0 diverges, but
more slowly than {Am }.
( j)

Note. Theorem 16.4.3 and Corollary 16.4.4 concern the convergence of {2n }
j=0 subject
to the condition |n | > |n+1 |, but they do not apply when |n | = |n+1 |, which is the
( j)
only remaining case. In this case, we can show at best that a subsequence of {2n }
j=0
( j)

16.4 Analysis of Column Sequences When Am A +

k=1

( j)

k m
k

307

converges under certain conditions and there holds 2n A = O(n+1 ) as j for

this subsequence. It can be shown that such a subsequence exists when |n1 | > |n | =
|n+1 |.
We see from Theorem 16.4.3 that the Shanks transformation accelerates the convergence of {Am } under the prescribed conditions. Indeed, assuming that |1 | > |2 |, and
using the asymptotic equality Am A 1 m
1 as m that follows from (16.4.1)
and (16.4.2), and observing that |n+1 /1 | < 1 by (16.4.14), we have, for any xed i,
( j)

2n A
= O(|n+1 /1 | j ) = o(1) as j ,
A j+i A
The next result, which appears to be new, concerns the stability of the Shanks transformation under the conditions of Theorem 16.4.3. Before stating this result, we recall
Theorems 3.2.1 and 3.2.2 of Chapter 3, according to which
( j)

2n =

n


( j)

ni A j+i ,

(16.4.23)

i=0

and


1

 A j


..

.

 A
n

j+n1
( j)
ni z i = 

1
i=0

 A j


..

.

 A
j+n1

z
...
A j+1 . . .
..
.
A j+n . . .
1 ...
A j+1 . . .
..
.
A j+n . . .









( j)
A j+2n1 
Rn (z)
 ( j) .

1
Rn (1)

A j+n 

..

.


A
zn
A j+n
..
.

(16.4.24)

j+2n1

Theorem 16.4.5 Under the conditions of Theorem 16.4.3, there holds

lim

n


( j)

ni z i =

i=0

n
n


z i

ni z i .
1 i
i=
i=0

(16.4.25)

( j)

That is, lim j ni = ni , i = 0, 1, . . . , n. Consequently, the Shanks transformation

is stable with respect to its column sequences, and we have
lim n( j) =

n

i=0

|ni |

n

1 + |i |
i=1

|1 i |

(16.4.26)

Equality holds in (16.4.26) when 1 , . . . , n have the same phase. When k are all real
( j)
negative, lim j n = 1.
( j)

Proof. Applying to the determinant Rn (z) the technique that was used in the proof
( j)
( j)
of Lemma 16.4.2 in the analysis of Hn ({ f s }), we can show that Rn (z) satises the

308

16 The Shanks Transformation

asymptotic equality


n
( j)
j
Rn (z)
p ( p 1) p V (1 , . . . , n )V (z, 1 , . . . , n ) as j . (16.4.27)
p=1

Using this asymptotic equality in (16.4.24), the result in (16.4.25) follows. The rest can
be completed as the proof of Theorem 3.5.6.


As the results of Theorems 16.4.3 and 16.4.5 are best asymptotically, they can be used
to draw some important conclusions about efcient use of the Shanks transformation. We
( j)
( j)
see from (16.4.15) and (16.4.26) that both 2n A and n are large if some of the k
( j)
are too close to 1. That is, poor stability and poor accuracy in 2n occur simultaneously.
Thus, making the transformation more stable results in more accuracy as well. [We
reached the same conclusion when we treated the application of the d-transformation to
power series and (generalized) Fourier series close to points of singularity of their limit
functions.]
In most cases of interest, 1 and possibly a few of the succeeding k are close to 1.
q
For some positive integer q, the numbers k separate from 1. In view of this, we propose
to apply the Shanks transformation to the subsequence {Aqm+s }, where q > 1 and s 0
are some xed integers. This achieves the desired result of increased stability as
Aqm+s A +

s
k m
k as m ; k = k , k = k k for all k, (16.4.28)

k=1

by which the k in (16.4.15), (16.4.16), (16.4.22), (16.4.25), and (16.4.26) are replaced
by the corresponding k , which are further away from 1 than the k . This strategy is
nothing but arithmetic progression sampling (APS).
Before leaving this topic, we revisit two problems we discussed in detail in Section 15.3
of the preceding chapter in connection with the iterated 2 -process, namely, the power
method and xed-point iterative solution of nonlinear equations.
Consider the sequence {m } generated by the power method for a matrix Q. Recall


m
that, when Q is diagonalizable, m has an expansion of the form m = +
k=1 k k ,
where is the largest eigenvalue of Q and 1 > |1 | |2 | . Therefore, the Shanks
transformation can be applied to the sequence {m }, and Theorem 16.4.3 holds.
Consider next the equation x = g(x), whose solution we denote s. Starting with x0 ,
we generate {xm } via xm+1 = g(xm ). Recall that, provided 0 < |g  (s)| < 1 and x0 is
sufciently close to s, there holds
xm s +

k km as m , 1 = 0; = g  (s).

k=1

Thus, the Shanks transformation can be applied to {xm } successfully. By Theorem 16.4.3,
this results in
( j)

2n s = O((n+1) j ) as j ,
whether some of the k vanish or not.

16.4 Analysis of Column Sequences When Am A +

k=1

k m
k

309

16.4.1 Extensions
So far, we have been concerned with analyzing the performance of the Shanks transformation on sequences that satisfy (16.4.1). We now wish to extend this analysis to
sequences {Am } that satisfy
Am A +

Pk (m)m
k as m ,

(16.4.29)

k=1

where
k distinct; k = 1 for all k; |1 | |2 | ; lim k = 0, (16.4.30)
k

and, for each k, Pk (m) is a polynomial in m of degree exactly pk 0 and with leading
coefcient ek = 0. We recall that the assumption that limk k = 0 implies that there
can be only a nite number of k with the same modulus. We say that such sequences
are exponential with conuence.


Again, we recall from Theorem 16.1.2 that, when Am = A + tk=1 Pk (m)m
k for all

t
( j)
m, and n is chosen such that n = k=1 ( pk + 1), then 2n = A for all j. From this, we
expect the Shanks transformation also to perform well when applied to the general case
in (16.4.29). This turns out to be the case, but its theory is much more complicated than
we have seen in this section so far.
We state in the following without proof theorems on the convergence and stability of
column sequences generated by the Shanks transformation. These are taken from Sidi
[296]. Of these, the rst three generalize the results of Theorem 16.4.3, Corollary 16.4.4,
and Theorem 16.4.5.
Theorem 16.4.6 Let {Am } be exactly as in the rst paragraph of this subsection and let
t and r be positive integers for which
|1 | |t | > |t+1 | = = |t+r | > |t+r +1 |,

(16.4.31)

and order the k such that

p pt+1 = = pt+ > pt++1 pt+r .

(16.4.32)

k = pk + 1, k = 1, 2, . . . ,

(16.4.33)

Set

and let
n=

t


k .

(16.4.34)

k=1

Then
( j)

2n A = j p

t+

s=t+1
j

es


 
t 
s i 2i j
j
s + o( j p t+1 ) as j ,
1

i
i=1

= O( j p t+1 ) as j .

(16.4.35)

310

16 The Shanks Transformation

Corollary 16.4.7 When = 1 in Theorem 16.4.6, the result in (16.4.35) becomes


 
t 
t+1 i 2i pt+1 j
( j)
2n A et+1
(16.4.36)
j t+1 as j .
1 i
i=1
Theorem 16.4.8 Under the conditions of Theorem 16.4.6, there holds

n 
n
n


z i i 
( j)
lim
ni z i =

ni z i .
j
1

i
i=0
i=1
i=0

(16.4.37)

( j)

That is, lim j ni = ni , i = 0, 1, . . . , n. Consequently, the Shanks transformation

is stable with respect to its column sequences, and we have

n
n 


1 + |i | i
( j)
|ni |
.
(16.4.38)
lim n =
j
|1 i |
i=0
i=1
Equality holds in (16.4.38) when 1 , . . . , t have the same phase. When k are all
( j)
negative, lim j n = 1.
( j)

Note that the second of the results in (16.4.35), namely, 2n A = O( j p t+1 ) as

j , was rst given by Sidi and Bridger [308, Theorem 3.1 and Note on p. 42].
Now, Theorem 16.4.6 covers only the cases in which n is as in (16.4.34). It does not




cover the remaining cases, namely, those for which tk=1 k < n < t+r
k=1 k , which turn
out to be more involved but very interesting. The next theorem concerns the convergence
of column sequences in these cases.
Theorem 16.4.9 Assume that the k are as in Theorem 16.4.6 with the same notation.
Let n be such that
t


k < n <

k=1

t+r


(16.4.39)

k=1

and let
=n

t


(16.4.40)

k=1

This time, however, also allow t = 0 and dene

nonlinear integer programming problem
maximize g(% ); g(% ) =

t+r


0
k=1

k = 0. Denote by IP( ) the

(k k k2 )

k=t+1

subject to

t+r


k = and 0 k k , t + 1 k t + r , (16.4.41)

k=t+1

and denote by G( ) the (optimal) value of g(% ) at the solution to IP( ).

( j)
Provided IP( ) has a unique solution for k , k = t + 1, . . . , t + r, 2n satises
2n A = O( j G( +1)G( ) t+1 ) as j ,
( j)

(16.4.42)

16.4 Analysis of Column Sequences When Am A +

k=1

k m
k

311

whether {Am } converges or not. [Here IP( + 1) is not required to have a unique
solution.]
Corollary 16.4.10 When r = 1, i.e., |t | > |t+1 | > |t+2 |, pt+1 > 1, and


n < t+1
k=1 k , a unique solution to IP( ) exists, and we have

t

2n A C j pt+1 2 t+1 as j ,
( j)

where = n

t
k=1

k=1

k <

(16.4.43)

k , hence 0 < < t+1 , and C is a constant given by


2 
 
t 
t+1
t+1 i 2i
pt+1 ! !
et+1
C = (1)
. (16.4.44)
( pt+1 )!
1 t+1
1 i
i=1

( j)

( j)

Therefore, the sequence {2n } j=0 is better then {2n2 } j=0 . In particular, if |1 | > |2 | >
|3 | > , then this is true for all n = 1, 2, . . . .
All the above hold whether {Am } converges or not.
It follows from Corollary 16.4.10 that, when 1 > |1 | > |2 | > , all column se
( j)
( j)
( j)
quences {2n }
j=0 converge, and {2n } j=0 converges faster than {2n2 } j=0 for each n.
Note. In case the problem IP( ) does not have a unique solution, the best we can say is
( j)
that, under certain conditions, there may exist a subsequence of {2n } j=0 that satises
(16.4.42).
In connection with IP( ), we would like to mention that algorithms for its solution
have been given by Parlett [228] and by Kaminski and Sidi [148]. These algorithms also
enable one to decide in a simple manner whether the solution is unique. A direct solution
of IP( ) has been given by Liu and Saff [169]. Some properties of the solutions to IP( )
have been given by Sidi [292], and we mention them here for completeness.
Denote J = {t + 1, . . . , t + r }, and let k , k J, be a solution of IP( ):

t+r
k .
1. k = k k , k J , is a solution of IP(  ) with  = k=t+1
2. If k  = k  for some k  , k  J , and if k  = 1 and k  = 2 in a solution to IP( ),
1 = 2 , then there is another solution to IP( ) with k  = 2 and k  = 1 . Consequently, a solution to IP( ) cannot be unique unless k  = k  . One implication
of this is that, for t+1 = = t+r = > 1, IP( ) has a unique solution only
for = qr, q = 1, . . . , 1, and in this solution k = q, k J . For t+1 = =
t+r = 1, no unique solution to IP( ) exists with 1 r 1. Another implication
is that, for t+1 = = t+ > t++1 t+r , < r, no unique solution to
IP( ) exists for = 1, . . . , 1, and a unique solution exists for = , this solution being t+1 = = t+ = 1, k = 0, t + + 1 k t + r .
3. A unique solution to IP( ) exists when k , k J , are all even or all odd, and


= qr + 12 t+r
k=t+1 (k t+r ), 0 q t+r . This solution is given by k = q +
1
(

),
k J.
k
t+r
2
4. Obviously, when r = 1, a unique solution to IP( ) exists for all possible and is
given as t+1 = . When r = 2 and 1 + 2 is odd, a unique solution to IP( ) exists
for all possible , as shown by Kaminski and Sidi [148].

312

16 The Shanks Transformation

16.4.2 Application to Numerical Quadrature

Sequences of the type treated in this section arise naturally when one approximates niterange integrals with endpoint singularities via the trapezoidal rule or the midpoint rule.
Consequently, the Shanks transformation has been applied to accelerate the convergence
of sequences of these approximations.
1
Let us go back to Example 4.1.4 concerning the integral I [G] = 0 G(x) d x, where
G(x) = x s g(x), s > 1, s not an integer, and g C [0, 1]. Setting h = 1/n, n a
positive integer, this integral is approximated by Q(h), where Q(h) stands for either
the trapezoidal rule or the midpoint rule that are dened in (4.1.3), and Q(h) has the
asymptotic expansion given by (4.1.4). Letting h m = 1/2m , m = 0, 1, . . . , we realize
that Am Q(h m ) has the asymptotic expansion
Am A +

k m
k as m ,

k=1

where A = I [G], and the k are obtained by ordering 4i and 2si , i = 1, 2, . . . , in

decreasing order according to their moduli. Thus, the k satisfy
1 > |1 | > |2 | > |3 | > .
If we now apply the Shanks transformation to the sequence {Am }, Corollary 16.4.4
applies, and we have that each column of the epsilon table converges to I [G] faster than
the one preceding it.
1
Let us next consider the approximation of the integral I [G] = 0 G(x) d x, where
G(x) = x s (log x) p g(x), with s > 1, p a positive integer, and g C [0, 1]. [Recall
that the case p = 1 has already been considered in Example 3.1.2.] Let h = 1/n, and de
n1
G(i h). Then, T (h) has an asymptotic
ne the trapezoidal rule to I [G] by T (h) = h i=1
expansion of the form

p




T (h) I [G] +
ai h 2i +
bi j (log h) j h s+i+1 as h 0,
i=1

i=0

j=0

where ai and bi j are constants independent of h. [For p = 1, this asymptotic expansion is

nothing but that of (3.1.6) in Example 3.1.2.] Again, letting h m = 1/2m , m = 0, 1, . . . ,
we realize that Am Q(h m ) has the asymptotic expansion
Am A +

Pk (m)m
k as m ,

k=1

where A = I [G] and the k are precisely as before, while Pk (m) are polynomials in
m. The degree of Pk (m) is 0 if k is 4i for some i, and it is at most p if k is 2si
for some i. If we now apply the Shanks transformation to the sequence {Am }, Corollaries 16.4.7 and 16.4.10 apply, and we have that each column of the epsilon table converges to I [G] faster than the one preceding it. For the complete details of this example
with p = 1 and s = 0, and for the treatment of the general case in which p is an arbitrary
positive integer, we refer the reader to Sidi [296, Example 5.2].
1
This use of the Shanks transformation for the integrals 0 x s (log x) p g(x) d x with
p = 0 and p = 1 was originally suggested by Chisholm, Genz, and Rowlands [49] and

16.5 Analysis of Column Sequences When {Am } b(1)

313

by Kahaner [147]. The only convergence result known at that time was Corollary 16.4.4,
which is valid only for p = 0, and this was mentioned by Genz [94]. The treatment of
the general case with p = 1, 2, . . . , was given later by Sidi [296].
16.5 Analysis of Column Sequences When {Am } b(1)
In the preceding section, we analyzed the behavior of the Shanks transformation on
sequences that behave as in (16.4.1) and showed that it is very effective on such sequences. This effectiveness was expected in view of the fact that the derivation of
the transformation was actually based on (16.4.1). It is interesting that the effectiveness of the Shanks transformation is not limited to sequences that are as in (16.4.1).
In this section, we present some results pertaining to those sequences {Am } that were
discussed in Denition 15.3.2 and for which {Am } b(1) . These results show that the
Shanks transformation is effective on linear and factorial sequences, but it is ineffective on logarithmic sequences. Actually, they are completely analogous to the results of
Chapter 15 on the iterated 2 -process. Throughout this section, we assume the notation
of Denition 15.3.2.

16.5.1 Linear Sequences

The following theorem is due to Garibotti and Grinstein [92].
Theorem 16.5.1 Let {Am } b(1) /LIN. Provided = 0, 1, . . . , n 1, there holds
( j)

2n A (1)n 0

n! [ ]n j+2n 2n

j
as j ,
( 1)2n

(16.5.1)

where [ ]n = ( 1) ( n + 1) when n > 0 and [ ]0 = 1, as before. This result

is valid whether {Am } converges or not.
Proof. To prove (16.5.1), we make use of the error formula in (16.3.2) with Dm =
(Am A) m precisely as in Lemma 16.3.1. We rst observe that


 um
u m . . .  p1 u m 

 u m 2 u m . . .  p u m 


(m)
(m)
sm
H p ({u s }) = H p ({ u m }) = 
 , (16.5.2)
..
..
..


.
.
.


  p1 u  p u . . . 2 p2 u 
m
m
m
which can be proved by performing a series of elementary row and column transformations on H p(m) ({u s }) in (16.1.13). As a result, (16.3.2) becomes
( j)

( j)

2n A = j+2n

Hn+1 ({s j D j })
( j)
Hn ({s j E j })

; E m = Dm 2 Dm+1 + 2 Dm+2 for all m.

(16.5.3)

314

16 The Shanks Transformation

Next, we have Dm

i=0

i m i as m , so that

r Dm 0 [ ]r m r as m ,
r E m ( 1)2 0 [ ]r m r ( 1)2 Dm as m .

(16.5.4)

( j)

Substituting (16.5.4) in the determinant Hn+1 ({s j D j }), and factoring out the powers
of j, we obtain
Hn+1 ({s j D j }) 0n+1 j n K n as j ,
( j)

where n =

n

i=0 (

(16.5.5)

2i) and

 [ ]0

 [ ]1

Kn =  .
 ..

 [ ]
n

[ ]1
[ ]2
..
.
[ ]n+1


. . . [ ]n 
. . . [ ]n+1 
,
..

.

. . . [ ] 

(16.5.6)

2n

provided, of course, that K n = 0. Using the fact that [x]q+r = [x]r [x r ]q , we factor
out [ ]0 from the rst column, [ ]1 from the second, [ ]2 from the third, etc. (Note that
all these factors are nonzero by our assumption on .) Applying Lemma 6.8.1 as we did
in the proof of Lemma 6.8.2, we obtain




n
n
[ ]i V ( , 1, . . . , n) =
[ ]i V (0, 1, 2, . . . , n).
Kn =
i=0

i=0

(16.5.7)
Consequently,
( j)

Hn+1 ({s j D j }) 0n+1


n


[ ]i V (0, 1, . . . , n) j n as j . (16.5.8)

i=0

Similarly, by (16.5.4) again, we also have

Hn( j) ({s j E j }) ( 1)2n Hn( j) ({s j D j }) as j .
Combining (16.5.8) and (16.5.9) in (16.5.3), we obtain (16.5.1).

(16.5.9)

Note that (2n A)/(A j+2n A) K j 2n as j for some constant K . That is,
the columns of the epsilon table converge faster than {Am }, and each column converges
faster than the one preceding it.
The next result that appears to be new concerns the stability of the Shanks transformation under the conditions of Theorem 16.5.1.
( j)

Theorem 16.5.2 Under the conditions of Theorem 16.5.1, there holds



n
n

z n 
( j) i
ni z =

ni z i .
lim
j
1
i=0
i=0

(16.5.10)

16.5 Analysis of Column Sequences When {Am } b(1)

315

( j)

That is, lim j ni = ni , i = 0, 1, . . . , n. Consequently, the Shanks transformation

is stable with respect to its column sequences and we have


1 + | | n
.
(16.5.11)
lim n( j) =
j
|1 |
( j)

When = 1, we have lim j n = 1.

( j)

Proof. Substituting Am = 0 ( 1) m Bm in the determinant Rn (z), and performing

elementary row and column transformation, we obtain



1
w ...
wn 


 Bj
B j . . . n B j 

2
n+1


Rn( j) (z) = [0 ( 1)]n  B j  B j . . .  B j  ,
(16.5.12)


..
..
..


.
.
.


 n1 B n B . . . 2n1 B 
j

n1
where = n + i=0
( j + 2i) and w = z/ 1. Now, since Bm m as m , we
also have k Bm [ ]k m k as m . Substituting this in (16.5.12), and proceeding
as in the proof of Theorem 16.5.1, we obtain
Rn( j) (z) L (nj) (z/ 1)n as j ,

(16.5.13)

( j)

for some L n that is nonzero for all large j and independent of z. The proof of (16.5.10)
can now be completed easily. The rest is also easy and we leave it to the reader.


As Theorem 16.5.1 covers the convergence of the Shanks transformation for the cases
in which = 0, 1, . . . , n 1, we need a separate result for the cases in which is an
integer in {0, 1, . . . , n 1}. The following result, which covers these remaining cases,
is stated in Garibotti and Grinstein [92].
Theorem 16.5.3 If is an integer, 0 n 1, and +1 = 0, then
( j)

2n A +1

(n 1)!(n + + 1)! j+2n 2n1

j
as j . (16.5.14)
( 1)2n

Important conclusions can be drawn from these results concerning the application of
the Shanks transformation to sequences {Am } b(1) /LIN.
( j)
As is obvious from Theorems 16.5.1 and 16.5.3, all the column sequences {2n } j=0
converge to A when | | < 1, with each column converging faster than the one preceding
( j)
it. When | | = 1 but = 1, {2n } j=0 converges to A (even when {Am } diverges) provided
(i) n >  /2 when = 0, 1, . . . , n 1, or (ii) n + 1 when is a nonnegative
( j)
integer. In all other cases, {2n } j=0 diverges. From (16.5.1) and (16.5.11) and (16.5.14),
( j)
we see that both the theoretical accuracy of 2n and its stability properties deteriorate
when approaches 1, because ( 1)2n as 1. Recalling that q for a positive integer q is farther away from 1 when is close to 1, we propose to apply the Shanks

316

16 The Shanks Transformation

transformation to the subsequence {Aqm+s }, where q > 1 and s 0 are xed integers.
( j)
This improves the quality of the 2n as approximations to A. We already mentioned that
this strategy is APS.

16.5.2 Logarithmic Sequences

The Shanks transformation turns out to be completely ineffective on logarithmic sequences {Am } for which {Am } b(1) .
Theorem 16.5.4 Let {Am } b(1) /LOG. Then, there holds
( j)

2n A (1)n 0

n!
j as j .
[ 1]n

(16.5.15)

In other words, no acceleration of convergence takes place for logarithmic sequences.

The result in (16.5.15) can be proved by using the technique of the proof of Theorem 16.5.1. We leave it to the interested reader.

16.5.3 Factorial Sequences

We end with the following results concerning the convergence and stability of the Shanks
transformation on factorial sequences {Am }. These results appear to be new.
Theorem 16.5.5 Let {Am } b(1) /FAC. Then
( j)

2n A (1)n 0r n

j+2n 2nr n
j
as j ,
( j!)r

(16.5.16)

and
lim

n


( j)

ni z i = z n and lim n( j) = 1.

(16.5.17)

i=0

Note that (2n A)/(A j+2n A) K j n as j for some constant K . Thus, in

this case too the columns of the epsilon table converge faster than the sequence {Am },
and each column converges faster than the one preceding it.
The technique we used in the proofs of Theorems 16.5.1 and 16.5.4 appears to be too
difcult to apply to Theorem 16.5.5, so we developed a totally different approach. We
start with the recursion relations of the -algorithm. We rst have
( j)

( j)

e1 (A j ) = 2 =

A j+1 (1/A j ) + 1
.
(1/A j )

( j)

(16.5.18)

( j)

Substituting the expression for 2n+1 in that for 2n+2 and invoking (16.5.18), we obtain
( j)

( j)

2n+2 =

( j)

( j+1)

e1 (2n )(1/2n ) + 2n
( j)

2n+1

( j+1)

2n1

(16.5.19)

16.6 The Shanks Transformation on Totally Monotonic

317

From this, we obtain the error formula

( j)

( j)
2n+2

A=

( j)

( j+1)

[e1 (2n ) A](1/2n ) + [2n

( j+1)

A]2n1

( j)

2n+1

(16.5.20)

Now use induction on n along with part (iii) of Theorem 15.3.3 to prove convergence.
To prove stability, we begin by paying attention to the fact that (16.5.19) can also be
expressed as
( j)

( j)

( j+1)

2n+2 = (nj) 2n + (nj) 2n

( j)

(16.5.21)

( j)

with appropriate n and n . Now proceed as in Theorems 15.5.1 and 15.5.2.

This technique can also be used to prove Theorem 16.5.1. We leave the details to the
reader.

16.6 The Shanks Transformation on Totally Monotonic

and Totally Oscillating Sequences
We now turn to application of the Shanks transformation to two important classes of real
sequences, namely, totally monotonic sequences and totally oscillating sequences. The
importance of these sequences stems from the fact that they arise in many applications.
This treatment was started by Wynn [371] and additional contributions to it were made
by Brezinski [34], [35].

16.6.1 Totally Monotonic Sequences

Denition 16.6.1 A sequence {m }
m=0 is said to be totally monotonic if it is real and if
(1)k k m 0, k, m = 0, 1, . . . , and we write {m } T M.

The sequences {(m + )1 }m=0 , > 0, and {m }

m=0 , (0, 1), are totally
monotonic.
The following lemmas follow from this denition in a simple way.
Lemma 16.6.2 If {m } T M, then
(i) {m }
m=0 is a nonnegative and nonincreasing sequence and hence has a nonnegative
limit,

(ii) {(1)k k m }m=0 T M, k = 1, 2, . . . .

Lemma 16.6.3
(i) If {m } T M and {m } T M, and , > 0, then {m + m } T M as well.
Also, {m m } T M. These can be extended to an arbitrary number of sequences.

T M and ci 0, i = 0, 1, . . . , and that i=0

ci (i)
(ii) Suppose that {(i)
m }m=0

0 con
verges, and dene m = i=0
ci (i)
{m } T M.
m , m = 0, 1, . . . . Then


(iii) Suppose that {m } T M and ci 0, i = 0, 1, . . . , and i=0
ci i0 converges, and


i
dene m = i=0 ci m , m = 0, 1, . . . . Then {m } T M.

318

16 The Shanks Transformation

Obviously, part (iii) of Lemma 16.6.3 is a corollary of parts (i) and (ii).
By Lemma 16.6.3, the sequence {m /(m + )}
m=0 with (0, 1) and > 0 is totally monotonic. Also, all functions f (z) that are analytic at z = 0 and have Maclaurin


ci z i with ci 0 for all i render { f (m )} T M when {m } T M, provided
series i=0


i
i=0 ci 0 converges.
The next theorem is one of the fundamental results in the Hausdorff moment problem.
Theorem 16.6.4 The sequence {m }
m=0 is totally monotonic if and only if there
exists
a
function
(t)
that
is
bounded
and nondecreasing in [0, 1] such that m =
1 m
t
d(t),
m
=
0,
1,
.
.
.
,
these
integrals
being dened in the sense of Stieltjes.
0

The sequence {(m + ) }m=0 with > 0 and > 0 is totally monotonic, because
1
1
(m + ) = 0 t m+1 (log t 1 ) dt/ ().
As an immediate corollary of Theorem 16.6.4, we have the following result on Hankel
determinants that will be of use later.
Theorem 16.6.5 Let {m } T M. Then H p(m) ({s }) 0 for all m, p = 0, 1, 2, . . . . If
1
the function (t) in m = 0 t m d(t), m = 0, 1, . . . , has an innite number of points
of increase on [0, 1], then H p(m) ({s }) > 0, m, p = 0, 1, . . . .
Proof. Let P(t) =

k

i=0 ci t

k
k 


be an arbitrary polynomial. Then



m+i+ j ci c j =

t m [P(t)]2 d(t) 0,

i=0 j=0

with strict inequality when (t) has an innite number of points of increase. The result
now follows by a well-known theorem on quadratic forms.


An immediate consequence of Theorem 16.6.5 is that m m+2 2m+1 0 for all m
= limm m in this case, then {m }
T M too. Therefore,
if {m } T M. If

(m+1 )
2 0 for all m, and hence
(m )(
m+2 )
0<

2
1

1,
0

provided m =
for all m. As a result of this, we also have that
lim

m+1

= (0, 1].
m

16.6.2 The Shanks Transformation on Totally Monotonic Sequences

The next theorem can be proved by invoking Theorem 16.1.3 on the relation between the
Shanks transformation and the Pade table and by using the so-called two-term identities
in the Pade table. We come back to these identities later.

16.6 The Shanks Transformation on Totally Monotonic

319

Theorem 16.6.6 When the Shanks transformation is applied to the sequence {Am }, the
(m)
, provided the
following identities hold among the resulting approximants ek (Am ) = 2k
(m)
relevant 2k exist:
( j)

( j)

[Hn+1 ({As })]

( j)

2n+2 2n =

(16.6.1)

({As })
,
( j)
(
j+1)
Hn ({2 As })Hn
({2 As })

(16.6.2)

( j)

( j)

Hn ({2 As })Hn+1 ({2 As })

( j)

( j+1)

2n

( j)

2n =

( j+1)

Hn+1 ({As })Hn

( j)

( j)

( j+1)

2n+2 2n

( j+1)

Hn+1 ({As })Hn+1 ({As })

( j)

( j+1)

Hn+1 ({2 As })Hn

( j+1)

( j)

( j+2)

2n+2 2n

({2 As })

[Hn+1 ({As })]

( j)

( j+2)

Hn+1 ({2 As })Hn

(16.6.3)

(16.6.4)

({2 As })

An immediate corollary of this theorem that concerns {Am } T M is the following:

( j)

Theorem 16.6.7 Let {Am } T M in the previous theorem. Then 2n 0 for each j and n.
( j)
( j)
In addition, both the column sequences {2n } j=0 and the diagonal sequence {2n }n=0 are
( j)
nonincreasing and converge to A = limm Am . Finally, (2n A)/(A j+2n A) 0
both as j , and as n , when limm (Am+1 A)/(Am A) = = 1.
( j)

Proof. That 2n 0 follows from (16.1.12) and from the assumption that {Am } T M.
To prove the rest, it is sufcient to consider the case in which A = 0 since {Am
( j)
A} T M and en (A j ) = A + en (A j A). From (16.6.3), it follows that 0 2n
( j+1)
( j+n)
( j)
( j)
2n2 0
= A j+n . Thus, lim j 2n = 0 and limn 2n = 0. That
( j)
( j)
{2n }n=0 and {2n } j=0 are nonincreasing follows from (16.6.1) and (16.6.2), respectively. The last part follows from part (i) of Theorem 15.3.1 on the Aitken 2 -process,
( j)
which says that 2 A = o(A j A) as j under the prescribed condition.

It follows from Theorem 16.6.7 that, if Am = cBm + d, m = 0, 1, . . . , for some
constants c = 0 and d, and if {Bm } T M, then the Shanks transformation on {Am }
( j)
( j)
produces approximations that satisfy lim j 2n = A and limn 2n = A, where
( j)
A = limm Am . Also, (2n A)/(A j+2n A) 0 both as j and as n ,
when limm (Am+1 A)/(Am A) = = 1.
It must be noted that Theorem 16.6.7 gives almost no information about rates of
convergence or convergence acceleration. To be able to say more, extra conditions need
to be imposed on {Am }. As an example, let us consider the sequence {Am } T M with
Am = 1/(m + ), m = 0, 1, . . . , > 0, whose limit is 0. We already know from Theorem 16.5.4 that the column sequences are only as good as the sequence {Am } itself. Wynn
( j)
[371] has given the following closed-form expression for 2n :
( j)

2n =

1
.
(n + 1)( j + n + )

320

16 The Shanks Transformation

It is clear that the diagonal sequences are only slightly better than {Am } itself, even
( j)
though they converge faster; 2n n 2 as n . No one diagonal sequence converges
more quickly than the one preceding it, however. Thus, neither the columns nor the
diagonals of the epsilon table are effective for Am = 1/(m + ) even though {Am } is
totally monotonic. (We note that the -algorithm is very unstable when applied to this
sequence.)
Let us now consider the sequence {Am }, where Am = m+1 /(1 m+1 ), m =


k(m+1)
, which by
0, 1, . . . , where 0 < < 1. It is easy to see that Am =
k=1
Lemma 16.6.3 is totally monotonic. Therefore, Theorem 16.6.7 applies. We know from
( j)
Corollary 16.4.4 that 2n = O( (n+1) j ) as j ; that is, each column of the epsilon
table converges (to 0) faster than the one preceding it. Even though we have no convergence theory for diagonal sequences that is analogous to Theorem 16.4.3, in this example
( j)
we have a closed-form expression for 2n , namely,
( j)

2n = A j(n+1)+n(n+2) .
This expression is the same as that given by Brezinski and Redivo Zaglia [41, p. 288] for
/
the epsilon table of the sequence {Sm }, where S0 = 1 and Sm+1 = 1 + a/Sm for some a
( j)
( j)
j(n+1)+n(n+2)+1
as n , so that limn 2n = 0.
( 1/4]. It follows that 2n
( j)
In other words, the diagonal sequences {2n }n=0 converge superlinearly, while {Am }
m=0
converges only linearly. Also, each diagonal converges more quickly than the one preceding it.
We end this section by mentioning that the remark following the proof of Theorem


ci x i for which
16.6.7 applies to sequences of partial sums of the convergent series i=0

m
c xi =
{cm } T M with limm cm = 0 and x > 0. This is so because Am = i=0



i i
i
A Rm , where Rm = i=m+1 ci x , and {Rm } T M. Here, A is the sum of i=0 ci x .

i
x /i.
An example of this is the Maclaurin series of log(1 x) = i=1

16.6.3 Totally Oscillating Sequences

m
Denition 16.6.8 A sequence {m }
m=0 is said to be totally oscillating if {(1) m }m=0
is totally monotonic, and we write {m } T O.

Lemma 16.6.9 If {m } T O, then {(1)k k m } T O as well. If {m }

m=0 is convergent, then its limit is 0.
The following result is analogous to Theorem 16.6.5 and follows from it.
Theorem 16.6.10 Let {m } T O. Then (1)mp H p(m) ({s }) 0 for all m, p = 0,
1
1, . . . . If the function (t) in (1)m m = 0 t m d(t), m = 0, 1, . . . , has an innite number of points of increase on [0, 1], then (1)mp H p(m) ({s }) > 0 for all m,
p = 0, 1, . . . .

16.7 Modications of the -Algorithm

321

An immediate consequence of Theorem 16.6.10 is that m m+2 (m+1 )2 0 for

all m if {m } T O; hence,
1
2

1.
0>
0
1
As a result, we also have that
m+1
= [1, 0).
m m
lim

16.6.4 The Shanks Transformation on Totally Oscillating Sequences

The following results follow from (16.1.12) and Theorem 16.6.10. We leave their proof
to the interested reader.
( j)

Theorem 16.6.11 Let {Am } T O in Theorem 16.6.6. Then (1) j 2n 0 for each j
( j)
( j)
and n. Also, both the column sequences {2n } j=0 and the diagonal sequences {2n }n=0

( j)
converge (to 0) when {Am } converges, the sequences {2n }n=0 being monotonic. In
( j)
addition, when {Am } converges, (2n A)/(A j+2n A) 0 both as j and as
n .
From this theorem, it is obvious that, if Am = cBm + d, m = 0, 1 . . . , for some constants c = 0 and d, and if {Bm } T O converges, then the Shanks transformation on
( j)
( j)
{Am } produces approximations that satisfy lim j 2n = d and limn 2n = d. For

( j)
( j)
each n, the sequences {2n }n=0 tend to d monotonically, while the sequences {2n } j=0
oscillate about d. (Note that limm Am = d because limm Bm = 0.)
It must be noted that Theorem 16.6.11, just as Theorem 16.6.7, gives almost no
information about rates of convergence or convergence acceleration. Again, to obtain
results in this direction, more conditions need to be imposed on {Am }.
16.7 Modications of the -Algorithm
As we saw in Theorem 16.5.4, the Shanks transformation is not effective on logarithmic
sequences in b(1) /LOG. For such sequences, Vanden Broeck and Schwartz [344] suggest
modifying the -algorithm by introducing a parameter that can be complex in general.
This modication reads as follows:
( j)

( j)

1 = 0 and 0 = A j , j = 0, 1, . . . ,
( j)

( j)

( j+1)

2n+1 = 2n1 +
( j+1)

2n+2 = 2n

( j+1)
2n

( j)

2n

, and

1
( j+1)
2n+1

( j)

2n+1

, n, j = 0, 1, . . . .

(16.7.1)

When = 1 the -algorithm of Wynn is recovered, whereas when = 0 the iterated 2 process is obtained. Even though the resulting method is dened only via the recursion

322

16 The Shanks Transformation

relations of (16.7.1) when is arbitrary, it is easy to show that it is quasi-linear in the

sense of the Introduction.
( j)
As with the -algorithm, with the present algorithm too, the k with odd k can be
eliminated, and this results in the identity
1
( j1)
2n+2

( j)
2n

( j+1)
2n2

( j)

( j)
2n

1
( j1)
2n

( j)
2n

1
( j+1)
2n

( j)

2n

( j)

(16.7.2)

( j)

Note that the new 2 and Wynns 2 are identical, but the other 2n are not.
The following results are due to Barber and Hamer [17].
Theorem 16.7.1 Let {Am } be in b(1) /LOG in the notation of Denition 15.3.2. With
= 1 in (16.7.1), we have
4 A = O( j 2 ) as j .
( j)

Theorem 16.7.2 Let {Am } be such that

 

, m = 0, 1, . . . ; = 0, 1, 2, . . . .
Am = A + C(1)m
m
Then, with = 1 in (16.7.1), 4 = A for all j. (Because Am A [C/ ()]m 1
as m , A = limm Am when  > 1. Otherwise, A is the antilimit of {Am }.)
( j)

We leave the proof of these to the reader.

Vanden Broeck and Schwartz [344] demonstrated by numerical examples that their
modication of the -algorithm is effective on some strongly divergent sequences when
is tuned properly. We come back to this in Chapter 18 on generalizations of Pade
approximants.
Finally, we can replace in (16.7.1) by k ; that is, a different value of can be used
to generate each column in the epsilon table.
Further modications of the -algorithm suitable for some special sequences {Am } that
behave logarithmically have been devised in the works of Sedogbo [262] and Sablonni`ere
[247]. These modications are of the form
( j)

( j)

1 = 0 and 0 = A j , j = 0, 1, . . . ,
gk
( j)
( j+1)
k+1 = k1 + ( j+1)
, k, j = 0, 1, . . . .
( j)
2n+1 2n+1

(16.7.3)

Here gk are scalars that depend on the asymptotic expansion of the Am . For the details
we refer the reader to [262] and [247].

17
The Pade Table

17.1 Introduction
In the preceding chapter, we saw that the approximations en (A j ) obtained by applying
the Shanks transformation to {Am }, the sequence of the partial sums of the formal power


k
e approximants corresponding to this power series. Thus, Pade
series
k=0 ck z , are Pad
approximants are a very important tool that can be used in effective summation of power
series, whether convergent or divergent. They have been applied as a convergence acceleration tool in diverse engineering and scientic disciplines and they are related to
various topics in classical analysis as well as to different methods of numerical analysis,
such as continued fractions, the moment problem, orthogonal polynomials, Gaussian
integration, the qd-algorithm, and some algorithms of numerical linear algebra. As such,
they have been the subject of a large number of papers and books. For these reasons,
we present a brief survey of Pade approximants in this book. For more information
and extensive bibliographies, we refer the reader to the books by Baker [15], Baker
and Graves-Morris [16], and Gilewicz [99], and to the survey paper by Gragg [106].
The bibliography by Brezinski [39] contains over 6000 items. For the subject of continued fractions, see the books by Perron [229], Wall [348], Jones and Thron [144], and
Lorentzen and Waadeland [179]. See also Henrici [132, Chapter 12]. For a historical
survey of continued fractions, see the book by Brezinski [38].
We start with the modern denition of Pade approximants due to Baker [15].


k
Denition 17.1.1 Let f (z) :=
k=0 ck z , whether this series converges or diverges.
The [m/n] Pade approximant corresponding to f (z), if it exists, is the rational function
f m,n (z) = Pm,n (z)/Q m,n (z), where Pm,n (z) and Q m,n (z) are polynomials in z of degree
at most m and n, respectively, such that Q m,n (0) = 1 and
f m,n (z) f (z) = O(z m+n+1 ) as z 0.

(17.1.1)

It is customary to arrange the f m,n (z) in a two-dimensional array, which has been
called the Pade table and which looks as shown in Table 17.1.1.
The following uniqueness theorem is quite easy to prove.
Theorem 17.1.2 If f m,n (z) exists, it is unique.
323

324

17 The Pade Table

Table 17.1.1: The Pade table
[0/0]
[0/1]
[0/2]
[0/3]
..
.

[1/0]
[1/1]
[1/2]
[1/3]
..
.

[2/0]
[2/1]
[2/2]
[2/3]
..
.

[3/0]
[3/1]
[3/2]
[3/3]
..
.

..
.

As follows from (17.1.1), the polynomials Pm,n (z) and Q m,n (z) also satisfy
Q m,n (z) f (z) Pm,n (z) = O(z m+n+1 ) as z 0, Q m,n (0) = 1.
If we now substitute Pm,n (z) =
tain the linear system
min(i,n)


m
i=0

ai z i and Q m,n (z) =

n
i=0

(17.1.2)

bi z i in (17.1.2), we ob-

ci j b j = ai , i = 0, 1, . . . , m,

(17.1.3)

ci j b j = 0, i = m + 1, . . . , m + n; b0 = 1.

(17.1.4)

j=0
min(i,n)

j=0

Obviously, the bi can be obtained by solving the equations in (17.1.4). With the bi
available, the ai can be obtained from (17.1.3). Using Cramers rule to express the bi , it
can be shown that f m,n (z) has the determinant representation
 n
 z Smn (z) z n1 Smn+1 (z)

 cmn+1
cmn+2

 cmn+2
cmn+3


.
..
..

.


cm
cm+1
u m,n (z)
=
f m,n (z) =

vm,n (z)
 zn
z n1 . . .

 cmn+1 cmn+2 . . .

 cmn+2 cmn+3 . . .

 .
..
 ..
.

 c
c
...
m

m+1


. . . z 0 Sm (z) 

. . . cm+1 
. . . cm+2 

..

.

. . . cm+n 
,

z0 

cm+1 
cm+2 
.. 
. 

c

(17.1.5)

m+n

p

where S p (z) = k=0 ck z k , p = 0, 1, . . . , and ck = 0 for k < 0.

n
bi z i and from (17.1.5), we see that
From Q m,n (z) = i=0

n
f m,n (z) =

bi z i Smi (z)

n
,
i
i=0 bi z

i=0

(17.1.6)

that is, once the bi have been determined, f m,n (z) can be determined completely without
actually having to determine the ai .

17.2 Algebraic Structure of the Pade Table

325

17.2 Algebraic Structure of the Pade Table

Denition 17.2.1 We say that the Pade approximant f m,n (z) is normal if it occurs exactly
once in the Pade table. We say that the Pade table is normal if all its entries are normal;
that is, no two entries are equal.
Let us dene

Cm,0 = 1; Cm,n



 cmn+1 cmn+2 . . . cm 


 cmn+2 cmn+3 . . . cm+1 


= .
 , m 0, n 1, (17.2.1)
...
... 
 ..


 c
cm+1 . . . cm+n1 
m

where ck = 0 for k < 0, as before. Thus, we also have that

C0,n = (1)n(n1)/2 c0n , n 0.

(17.2.2)

Let us arrange the determinants Cm,n in a two-dimensional table, called the C-table,
in complete analogy to the Pade table itself. The rst row of this table is occupied by
Cm,0 = 1, m = 0, 1, . . . , and the second row is occupied by Cm,1 = cm , m = 0, 1, . . . .
The rst column is occupied by C0,n = (1)n(n1)/2 c0n , n = 0, 1, . . . . When Cm,n = 0
for all m and n, the rest of the C-table can be computed recursively by using the Frobenius
identity
Cm,n+1 Cm,n1 = Cm+1,n Cm1,n (Cm,n )2 .

(17.2.3)

This identity can be proved by applying the Sylvester determinant identity to Cm,n+1 .
The normality of the Pade approximants is closely related to the zero structure of the
C-table as the following theorems show.
Theorem 17.2.2 The following statements are equivalent:
(i) f m,n (z) is normal.
(ii) The numerator Pm,n (z) and the denominator Q m,n (z) of f m,n (z) have degrees exactly
m and n, respectively, and the expansion of Q m,n (z) f (z) Pm,n (z) begins exactly
with the power z m+n+1 .
(iii) The determinants Cm,n , Cm,n+1 , Cm+1,n , and Cm+1,n+1 do not vanish.
Theorem 17.2.3 A necessary and sufcient condition for the Pade table to be normal is
that Cm,n = 0 for all m and n. In particular, Cm,1 = cm = 0, m = 0, 1, . . . , must hold.
Theorem 17.2.4 In case the Pade table is not normal, the vanishing Cm,n in the C-table
appear in square blocks, which are entirely surrounded by nonzero entries (except at
innity). For the r r block of zeros in the C-table, say, C, = 0, m + 1 m + r,
n + 1 n + r , we have that f , (z) = f m,n (z) for m, n, and +
m + n + r , while the rest of the f , (z) in the (r + 1) (r + 1) block, m m + r,
n n + r , do not exist.

326

17 The Pade Table

k
Theorem 17.2.5 Let
k=0 ck z be the Maclaurin series of the rational function R(z) =
p(z)/q(z), where p(z) and q(z) are polynomials of degree exactly m and n, respectively,
and have no common factor. Then, f , (z) = R(z) for all m and n. In this case,
the C-table has an innite zero block.
The next theorem shows the existence of an innite number of Pade approximants (i)
along any row, (ii) along any column, and (iii) along any diagonal, and it is proved by
considering the zero structure of the C-table.


k
Theorem 17.2.6 Given a formal power series
k=0 ck z with c0 = 0, in its correspond
for arbitrary xed n, (ii)
ing Pade table, there exist innite sequences (i) { f i ,n (z)}i=0

{ f m,i (z)}i=0 for arbitrary xed m, and (iii) { f i +k,i (z)}i=0 for arbitrary xed k.
The Pade table enjoys some very interesting duality and invariance properties that are
very easy to prove.


k
Theorem 17.2.7 Let f (z) :=
k=0 ck z be a formal power series with c0 = 0. Denote


k
g(z) = 1/ f (z) := k=0 dk z . [That is, dk are the solution of the triangular system of


equations c0 d0 = 1 and kj=0 ck j d j = 0, k = 1, 2, . . . .] If f , (z) and g, (z) denote
the [/] Pade approximants corresponding to f (z) and g(z), respectively, and if f m,n (z)
exists, then gn,m (z) exists and gn,m (z) = 1/ f m,n (z).
Corollary 17.2.8 If 1/ f (z) = f (z), then f m,n (z) = 1/ f n,m (z). Consequently, if
f m,n (z) = Pm,n (z)/Q m,n (z) with Q m,n (0) = 1, then Pm,n (z) = c0 Q n,m (z) as well.
A most obvious application of this corollary is to f (z) = e z . Also, 1/ f (z) = f (z)
holds when f (z) = s(z)/s(z) with any s(z).


Theorem 17.2.9 Let f (z) :=
c z k be a formal power series and let g(z) = [A +

k=0k k
B f (z)]/[C + D f (z)] := k=0 dk z . Then, provided C + Dc0 = 0 and provided f m,m (z)
exists, gm,m (z) exists and there holds gm,m (z) = [A + B f m,m (z)]/[C + D f m,m (z)].
The result of the following theorem is known as homographic invariance under an
argument transformation.


k
be a formal power series. Dene the
Theorem 17.2.10 Let f (z) :=
k=0 ck z
origin-preserving transformation w = Az/(1 + Bz), A = 0, and let g(w) f (z) =


k
e approximants corref (w/(A Bw)) :=
k=0 dk w . If f , (z) and g, (w) are Pad
sponding to f (z) and g(w), respectively, then gm,m (w) = f m,m (z), provided f m,m (z)
exists.

17.3 Pade Approximants for Some Hypergeometric Functions

327

17.3 Pade Approximants for Some Hypergeometric Functions

As can be concluded from the developments of Section 17.1, in general, it is impossible
to obtain closed-form expressions for Pade approximants. Such expressions are possible
for some special cases involving hypergeometric series, however. In this section, we
treat a few such cases by using a technique of the author that avoids the pain of going
through the determinant representation given in (17.1.5). Lemmas 17.3.1 and 17.3.2 that
follow are due to Sidi [277] and they form the main tools of this technique. We provide
the proof of the rst lemma only; that of the second is similar. We make use of Lemma
17.3.1 again in Chapter 19 in the derivation of a simple closed-form expression for the
S-transformation of Sidi.
Lemma 17.3.1 Let T and i , i = 0, 1, . . . , n 1, be dened by the linear equations

Ar = br T + cr 0 +

n1

i=1


i
, r = k, k + 1, . . . , k + n, (17.3.1)
+r +i 1

or
Ar = br T + cr

n1

i=0

i
, r = k, k + 1, . . . , k + n,
( + r )i

(17.3.2)

where (x)0 = 1 and (x)i = x(x + 1) (x + i 1), i 1. Provided + r + i 1 = 0

for 1 i n 1 and k r k + n, T is given by


n
j n
n (( + k)n1 Ak /ck )
j=0 (1) j ( + k + j)n1 Ak+ j /ck+ j

=
n
. (17.3.3)
T = n
j n
 (( + k)n1 bk /ck )
j=0 (1) j ( + k + j)n1 bk+ j /ck+ j
Proof. Let us multiply both sides of (17.3.1) and (17.3.2) by ( + r )n1 /cr . We obtain
( + r )n1 (Ar /cr ) = ( + r )n1 (br /cr )T + P(r ), r = k, k + 1, . . . , k + n,
where P(x) is a polynomial in x of degree at mostn 1. Next, let us write r = k + j in
this equation and multiply both sides by (1)n j nj and sum from j = 0 to j = n (the



number of equations is exactly n + 1). By the fact that n f k = nj=0 (1)n j nj f k+ j ,
this results in
n (( + k)n1 Ak /ck ) = T n (( + k)n1 bk /ck ) + n P(k).
Now, n P(k) = 0 because the polynomial P(x) is of degree at most n 1. The result
now follows.


Lemma 17.3.2 Let T and i , i = 0, 1, . . . , n 1, be dened by the linear equations
Ar = br T + cr

n1

i=0

i pi (r ), r = k, k + 1, . . . , k + n,

(17.3.4)

328

17 The Pade Table

where pi (x) are polynomials in x, each of degree n 1. Then T is given by



n
j n
n (Ak /ck )
j=0 (1) j Ak+ j /ck+ j

T = n
=
n
.
(17.3.5)
j n
 (bk /ck )
j=0 (1) j bk+ j /ck+ j


k
Let us now turn to Pade approximants corresponding to f (z) :=
k=0 ck z . From the
determinantal expression for f m,n (z) given in (17.1.5), it follows that f m,n (z), along with
the additional parameters 1 , . . . , n , satises
z mr Sr (z) = z mr f m,n (z) +

n


i cr +i , m n r m,

(17.3.6)

i=1

p
where S p (z) = k=0 ck z k , p = 0, 1, . . . .
When ck have the appropriate form, we can solve (17.3.6) for f m,n (z) in closed form
with the help of Lemmas 17.3.1 and 17.3.2.


j
Example 17.3.3 f (z) =1 F1 (1; ; z) =
j=0 z /() j , = 0, 1, 2, . . . . Using the
fact that () p+q = () p ( + p)q , the equations in (17.3.6) can be written in the form
z mr Sr (z) = z mr f m,n (z) +

n1
i
1 
, m n r m.
()r +1 i=0 ( + r + 1)i

Thus, Lemma 17.3.1 applies, and after some manipulation it can be shown that


n
j n
n j
Smn+ j (z)
j=0 (1) j ()m+ j z


.
f m,n (z) =

n
j n
n j
j=0 (1) j ()m+ j z
For = 1, we have f (z) = e z , and Corollary 17.2.8 also applies. Thus, we have

m m 
j
j=0 j (m + n j)! z
f m,n (z) =
n n 
j
j=0 j (m + n j)! (z)


k
as the [m/n] Pade approximant for e z =
k=0 z /k! with very little effort.


j
Example 17.3.4 f (z) =2 F1 (1, ; ; z) =
j=0 [() j /() j ]z , , = 0, 1, 2, . . . .
Again, using the fact that (x) p+q = (x) p (x + p)q , we have in (17.3.6) that
n

i=1

n
( + r + 1)i1
()r +1 
i
()r +1 i=1 ( + r + 1)i1


n1

i
()r +1
=
0 +
for some i .
()r +1
+r +i
i=1

i cr +i =

The last equality results from the partial fraction decomposition of the rational function

n
i=1 i ( + r + 1)i1 /( + r + 1)i1 . Thus, f m,n (z) satises
z mr Sr (z) = z mr f m,n (z) +



n1

()r +1
i
0 +
, m n r m.
()r +1
+r +i
i=1

17.4 Identities in the Pade Table

329

Therefore, Lemma 17.3.1 applies, and we obtain

n
f m,n (z) =

n 

()m+ j
z n j Smn+ j (z)
()mn+ j+1
  ()m+ j

n
j n
z n j
j=0 (1) j
()mn+ j+1

j=0

(1) j

Special cases of this are obtained by assigning to and specic values:

(i) For = 1, f (z) =1 F0 (; z) = (1 z) .


k
(ii) For = + 1, f (z) =
k=0 z /( + k). With = 1, this reduces to f (z) =
z 1 log(1 z). With = 1/2, it reduces to f (z) = z 1/2 tanh1 z 1/2 .


j
Example 17.3.5 f (z) =2 F0 (1, ; z) =
j=0 () j z , = 0, 1, 2, . . . . Again, using
the fact that (x) p+q = (x) p (x + p)q , we have in (17.3.6) that
n


i cr +i = ()r +1

i=1

n


i ( + r + 1)i1 ,

i=1

the summation on the right being a polynomial in r of degree at most n 1. Thus, f m,n (z)
satises
z mr Sr (z) = z mr f m,n (z) + ()r +1

n1


i r i , m n r m.

i=0

Therefore, Lemma 17.3.2 applies, and we obtain

n
f m,n (z) =

n 

1
z n j Smn+ j (z)
()mn+ j+1
.


n
1
j n
n j
(1)
z
j=0
j
()mn+ j+1

j=0

(1) j

Setting = 1, we obtain the Pade approximant for the Euler

 series, which is related to the exponential integral E 1 (z 1 ), where E 1 ( ) = (et /t)dt and E 1 ( )


k
k
(e / )
as  +.
k=0 (1) (k!)
Some of the preceding examples have also been treated by different techniques by Luke
[193] and by Iserles [139]. Pade approximants from so-called q-series have been obtained
in closed form by Wynn [373]. For additional treatment of q-elementary functions, see
Borwein [28].

17.4 Identities in the Pade Table

Various identities relate neighboring entries in the Pade table. We do not intend to go into
these in any detail here. We are content to derive one of the so-called two-term identities to
which we alluded in our discussion of the Shanks transformation in the previous chapter.
There, we mentioned, in particular, that the four identities (16.6.1)(16.6.4) among the
different entries of the epsilon table could be derived from these two-term identities.

330

17 The Pade Table

Consider f , (z) = u , (z)/v, (z), with u , (z) and v, (z) being, respectively,
the numerator and denominator determinants in (17.1.5). Now, by (17.1.1) we have
f m+1,n (z) f m,n+1 (z) = O(z m+n+2 ) as z 0. Next, we have f m+1,n (z) f m,n+1 (z) =
N (z)/D(z), where N (z) = u m+1,n (z)vm,n+1 (z) u m,n+1 (z)vm+1,n (z) and D(z) =
vm+1,n (z)vm,n+1 (z). Because N (z) is a polynomial of degree at most m + n + 2 and
D(0) = 0, we thus have that N (z) = z m+n+2 for some constant . Obviously, is the
product of the leading coefcients of u m+1,n (z) and vm,n+1 (z), which are both Cm+1,n+1 .
Summarizing, we have the two-term identity
f m+1,n (z) f m,n+1 (z) =

(Cm+1,n+1 )2 z m+n+2
.
vm+1,n (z)vm,n+1 (z)

(17.4.1)

Note that the identity in (16.6.4) pertaining to the epsilon table is obtained directly
from the two-term identity in (17.4.1) by recalling Theorem 16.1.3, which relates the
Shanks transformation to the Pade table.
Using the approach that led to (17.4.1), we can obtain additional two-term identities
involving f m+1,n+1 (z) f m,n (z), f m+1,n (z) f m,n (z), and f m,n+1 (z) f m,n (z), from
which we can obtain the identities given in (16.6.1)(16.6.3). We leave the details to
the interested reader.
For additional identities involving more entries in the Pade table, see Baker [15].
We close this section by stating the ve-term identity that follows directly from Wynns
cross rule, which we encountered in the previous chapter. This identity reads
1
1
+
=
f m+1,n (z) f m,n (z)
f m1,n (z) f m,n (z)
1
1
+
, (17.4.2)
f m,n+1 (z) f m,n (z)
f m,n1 (z) f m,n (z)
and is known as Wynns identity. Of course, it is valid when f m,n (z) is normal.

17.5 Computation of the Pade Table

Various methods have been developed for computing the Pade table corresponding to


k
a given power series f (z) :=
k=0 ck z when this table is normal. The most straightforward of these methods is provided by Wynns ve-term identity given in (17.4.2).
Obviously, this is a recursive method that enables us to compute the Pade table
row-wise or columnwise. The initial conditions for this method are f m,0 (z) = Sm (z),
f m,1 (z) = Sm1 (z) + cm z m /(1 (cm+1 /cm )z), m = 0, 1, . . . , and f 0,n (z) = 1/Tn (z),


k
n = 0, 1, . . . , where S p (z) and T p (z) are the pth partial sums of f (z) :=
k=0 ck z and


k
1/ f (z) := k=0 dk z , respectively. Obviously, the dk can be obtained from c0 d0 = 1


and kj=0 ck j d j = 0, k = 1, 2, . . . , recursively, in the order d0 , d1 , . . . . Without the
f 0,n (z), it is possible to compute half the Pade table, namely, those f m,n (z) with m n.
When the Pade table is not normal, we can use an extension of Wynns identity due to
Cordellier [57]. For a detailed treatment, see Baker and Graves-Morris [16].
We next present a method due to Longman [174] for recursive determination


(m,n) k
of the coefcients of the numerator polynomial Pm,n (z) = m
z and the
k=0 pk

17.5 Computation of the Pade Table

331

denominator polynomial Q m,n (z) = nk=0 qk(m,n) z k of f m,n (z) with the normalization
(m,n)
= 1, which implies p0(m,n) = c0 . This method too is valid for normal
condition q0
tables. We start with
Q m+1,n (z) f (z) Pm+1,n (z) = O(z m+n+2 )
Q m,n+1 (z) f (z) Pm,n+1 (z) = O(z m+n+2 )

(17.5.1)

that follow from (17.1.1). Upon subtraction, we obtain from (17.5.1) that




Q m,n+1 (z) Q m+1,n (z) f (z) Pm,n+1 (z) Pm+1,n (z) = O(z m+n+2 ).
Now, both terms inside the square brackets are polynomials that vanish at z = 0. Dividing
both sides by z, and invoking (17.1.1) again, we realize that
Pm,n+1 (z) Pm+1,n (z)

 = Pm,n (z),
q1(m,n+1) q1(m+1,n) z

(17.5.2)

Q m,n+1 (z) Q m+1,n (z)


 = Q m,n (z).
q1(m,n+1) q1(m+1,n) z

(17.5.3)

Let us denote by { pk , qk }, { pk , qk }, and { pk , qk } the { pk , qk } corresponding to

f m,n (z), f m,n+1 (z), and f m+1,n (z), respectively. Then, it follows from (17.5.2) and (17.5.3)
that
(,)

(,)


pm+1
pk pk
= pm ,

 = pk1 , k = 1, . . . , m; 
q1 q1
q1 q1

(17.5.4)


qn+1
qk qk
=
q
,
k
=
1,
.
.
.
,
n;
= qn .
k1
q1 q1
q1 q1

(17.5.5)

and

Eliminating q1 q1 , we can rewrite (17.5.4) and (17.5.5) in the form
p0 = c0 and pk = pk


pm+1
pk1 , k = 1, . . . , m,
pm

(17.5.6)


qn+1
qk1 , k = 1, . . . , n.
qn

(17.5.7)

and
q0 = 1 and qk = qk
(,)

The pk
can be computed with the help of (17.5.6) with the initial conditions
(,0)
(,)
= ck , k = 0, 1, . . . , , and the qk
can be computed with the help of (17.5.7)
pk
(0,)

= d k , k = 0, 1 . . . , , where the dk are the Maclaurin

with the initial conditions qk
series coefcients

for c0 /f (z) and thus can be computed recursively from d0 = 1
k1

and d k =
i=0 cki d i /c0 , k = 1, 2, . . . . Thus, given the terms c0 , c1 , . . . , c M ,
this algorithm enables us to obtain the coefcients of f , (z) for 0 + M.
For additional methods that also treat the issue of numerical stability, see Baker and
Graves-Morris [16].

332

17 The Pade Table

Finally, Pade approximants can also be computed via their connection to continued
fractions with the help of the quotient-difference (qd) algorithm. We turn to this in the
next section.

17.6 Connection with Continued Fractions

17.6.1 Denition and Algebraic Properties of Continued Fractions
By a continued fraction, we mean an expression of the form
 
an
a1
b0 +
b0 + K
,
a2
bn
b1 +
a3
b2 +
b3 + . .
.
which we write in the typographically more convenient form
 
an
a1 a2 a3
.
b0 + K
= b0 +
bn
b1 + b2 + b3 +

(17.6.1)

(17.6.2)

Let us set
A0
b0 A1
a1
= ,
= b0 + , and
B0
1
B1
b1
an
a1
An
= b0 +
, n = 2, 3, . . . .
Bn
b1 + + bn

(17.6.3)

We call an and bn , respectively, the nth partial numerator and the nth partial denominator
of b0 + K (an /bn ), and An /Bn is its nth convergent.
In case an = 0, n = 1, . . . , N , a N +1 = 0, the continued fraction terminates and
its value is A N /B N . If an = 0 and bn = 0 for all n, then it is innite. In this case, if
limn (An /Bn ) = G exists, then we say that b0 + K (an /bn ) converges to G.
It is easy to show by induction that the An and Bn satisfy the recursion relations
An+1 = bn+1 An + an+1 An1 , n = 1, 2, . . . , A1 = 1, A0 = b0 ,
Bn+1 = bn+1 Bn + an+1 Bn1 , n = 1, 2, . . . , B1 = 0, B0 = 1.

(17.6.4)

We call An and Bn , respectively, the nth numerator and the nth denominator of the
continued fraction b0 + K (an /bn ).
By (17.6.4), we have
An+1 Bn An Bn+1 = an+1 (An Bn1 An1 Bn ),
repeated application of which gives
An+1 Bn An Bn+1 = (1)n a1 a2 an+1 .

(17.6.5)

An+1 /Bn+1 An /Bn = (1)n a1 a2 an+1 /(Bn Bn+1 ),

(17.6.6)

Therefore,

17.6 Connection with Continued Fractions

333

and hence
An
a1 a2 an
a1
a1 a2
= b0 +

+ + (1)n1
.
Bn
B0 B1
B1 B2
Bn1 Bn

(17.6.7)

This last result can be used to prove some interesting convergence theorems for continued fractions. One of these theorems says that K (1/n ), with n > 0, n = 1, 2, . . . ,


converges if and only if
n=1 n diverges. See Henrici [132, Theorem 12.1c].
We say two continued fractions are equivalent if they have the same sequence of
convergents. It is easy to show that, for any sequence of nonzero complex numbers
{cn }
n=0 , c0 = 1, the continued fractions b0 + K (an /bn ) and b0 + K (cn1 cn an /cn bn )
are equivalent. In particular, by choosing cn = 1/bn , n = 1, 2, . . . , b0 + K (an /bn )
becomes equivalent to b0 + K (an /1), where a1 = a1 /b1 , an = an /(bn1 bn ), n =
2, 3, . . . , provided that bn = 0, n = 1, 2, . . . . Next, by choosing c1 = 1/a1 and
cn = 1/(an cn1 ), n = 2, 3, . . . , b0 + K (an /bn ) becomes equivalent to b0 + K (1/bn ),
where bn = bn cn , n = 1, 2, . . . .
By the even (odd) part of b0 + K (an /bn ), we mean a continued fraction whose se
quence of convergents is {A2n /B2n }
n=0 ({A2n+1 /B2n+1 }n=0 ). For the continued fraction
b0 + K (an /1), we have by a few applications of (17.6.4)
Cn+1 = (1 + an + an+1 )Cn1 an an1 Cn3 ,

(17.6.8)

where Cn stands for either An or Bn . Thus, the even part of b0 + K (an /1) is
b0 +

a2 a3
a4 a5
a6 a7
a1

1 + a2 1 + a3 + a4 1 + a5 + a6 1 + a7 + a8

(17.6.9)

and the odd part is

(b0 + a1 )

a3 a4
a5 a6
a1 a2
.
1 + a2 + a3 1 + a4 + a5 1 + a6 + a7

(17.6.10)

17.6.2 Regular C-Fractions and the Pade Table

By a regular C-fraction, we mean an innite continued fraction z 1 K (an z/1), namely,
a1 a2 z a3 z
.
1 + 1 + 1 +

(17.6.11)

Here, z is a complex variable. A regular C-fraction for which an > 0, n = 1, 2, . . . , is

called an S-fraction. To emphasize the dependence of the nth numerator and denominator
An and Bn on z, we denote them by An (z) and Bn (z), respectively.
By (17.6.4), it is clear that An (z) and Bn (z) are polynomials in z, and hence the
convergents An (z)/Bn (z) are rational functions. It is easy to see that A2n (z), B2n (z),
A2n+1 (z), and B2n+1 (z) have degrees at most n 1, n, n, and n, respectively. By
(17.6.5), we also have
An+1 (z)Bn (z) An (z)Bn+1 (z) = (1)n a1 a2 an+1 z n .

(17.6.12)

From this and from the fact that An (0) = a1 = 0 and Bn (0) = 1 for all n, it can be shown

334

17 The Pade Table

Table 17.6.1: The staircase in the Pade
table from the C-fraction of (17.6.11)
[0/0]
[0/1]

[1/1]
[1/2]

[2/2]
[2/3]

...
...

...

that An (z) and Bn (z) have no common zeros. It is also clear that
A2n (z)
A2n+1 (z)

= O(z 2n ) as z 0,
B2n+1 (z)
B2n (z)

(17.6.13)

A2n+2 (z)
A2n+1 (z)

= O(z 2n+1 ) as z 0.
B2n+2 (z)
B2n+1 (z)

(17.6.14)

and

From these, it follows that the Maclaurin expansion of the convergent A2n (z)/B2n (z)

2n1 i
ci z + O(z 2n ), whereas that of A2n+1 (z)/B2n+1 (z) has the form
has the form i=0

2n
i
2n+1
), where the coefcients ci are independent of n for all n that sati=0 ci z + O(z
ises 2n i. Thus, we conclude that A2n+1 (z)/B2n+1 (z) is the [n/n] Pade approximant


ci z i , and A2n (z)/B2n (z) is the [n 1/n]
corresponding to the formal power series i=0
Pade approximant. In other words, the convergents Ar (z)/Br (z), r = 1, 2, . . . , of the


ci z i , which
regular C-fraction z 1 K (an z/1) form the staircase in the Pade table of i=0
is shown in Table 17.6.1.
We now consider the converse problem: Given a formal power series f (z) :=


k
e approxk=0 ck z , does there exists a regular C-fraction whose convergents are Pad
imants? The following theorem, whose proof can be found in Henrici [132, Theorem
12.4c], summarizes everything.


k
Theorem 17.6.1 Given a formal power series f (z) :=
k=0 ck z , there exists at most
one regular C-fraction corresponding to f (z). There exists precisely one such fraction if
and only if Hn(m) ({cs }) = 0 for m = 0, 1 and n = 1, 2, . . . . [Here Hn(m) ({cs }) are Hankel
determinants dened in (16.1.13).]
Theorem 17.6.1 and the developments preceding it form the basis of the qd-algorithm
to which we alluded in the previous section.
Assuming that Hn(m) ({cs }) = 0 for m = 0, 1 and n = 1, 2, . . . , we now know that,


k
given f (z) :=
k=0 ck z , there exists a corresponding regular C-fraction, which we
choose to write as
F0 (z) :=

c0 q1(0) z e1(0) z q2(0) z e2(0) z

.
1 1 1 1 1

(17.6.15)

Similarly, if also Hn(m) ({cs }) = 0 for all m = 2, 3, . . . , and n = 1, 2, . . . , there exist

17.6 Connection with Continued Fractions

335

Table 17.6.2: The staircase in the Pade table

from the C-fraction of (17.6.16)
[k 1/0]

[k/0]
[k/1]

[k + 1/1]
[k + 1/2]

[k + 2/2]
...

...

corresponding regular C-fractions of the form

Fk (z) :=

k1


ci z i +

i=0

ck z k q1(k) z e1(k) z q2(k) z e2(k) z

,
1 1 1 1 1

(17.6.16)

for every k = 1, 2, . . . , and the convergents Ar (z)/Br (z), r = 0, 1, . . . , of Fk (z) form

the staircase in the Pade table of f (z), which is shown in Table 17.6.2.
It is seen that the even convergents of Fk+1 (z) and the odd convergents of Fk (z) are
identical. Now, the even part of Fk+1 (z) is the continued fraction
e
Fk+1
(z) :=

k


ci z i

i=0

ck+1 z k+1

q1(k+1) e1(k+1) z 2

q2(k+1) e2(k+1) z 2

,
1 q1(k+1) z 1 (e1(k+1) + q2(k+1) )z 1 (e2(k+1) + q3(k+1) )z . . .
(17.6.17)

and the odd part of Fk (z) is the continued fraction

Fko (z) :=

k


ci z i

i=0

ck q1(k) z k+1

e1(k) q2(k) z 2

e2(k) q3(k) z 2

1 (q1(k) + e1(k) )z 1 (q2(k) + e2(k) )z 1 (q3(k) + e3(k) )z

. (17.6.18)

e
(z), we obtain the relations
By equating Fko (z) and Fk+1

ck+1 = ck q1(k) , q1(k+1) = q1(k) + e1(k) , and

(k) (k)
(k+1)
(k)
(k)
qn(k+1) en(k+1) = qn+1
en , en(k+1) + qn+1
= qn+1
+ en+1
, n = 1, 2, . . . . (17.6.19)

It is easy to see that the qn(k) and en(k) can be computed recursively from these relations,
which form the basis of the qd-algorithm given next.
Algorithm 17.6.2 (qd-algorithm)
1. Given c0 , c1 , c2 , . . . , set e0(k) = 0 and q1(k) = ck+1 /ck , k = 0, 1, . . . .
2. For n = 1, 2, . . . , and k = 0, 1, . . . , compute en(k) and qn(k) recursively by
(k+1)
en(k) = en1
+ qn(k+1) qn(k) ,

(k)
qn+1
= qn(k+1) en(k+1) /en(k) .

The quantities qn(k) and en(k) can be arranged in a two-dimensional array as in Table
17.6.3. This table is called the qd-table.

336

17 The Pade Table

Table 17.6.3: The qd-table

e0(1)
e0(2)
e0(3)

e0(4)
..
.

q1(0)
q1(1)
q1(2)
q1(3)
..
.

e1(0)
e1(1)
e1(2)

q2(0)
q2(1)

e2(0)

q2(3)
..
.

..

..

e2(1)

q2(2)
e1(3)
..
.

..

e2(2)
..
.

With the qn(k) and en(k) available, we can now compute the convergents of the continued
fractions Fk (z).
It can be shown that (see Henrici [131, Theorem 7.6a])
qn(k) =

(k)
Hn1
Hn(k+1)
(k+1)
Hn(k) Hn1

and en(k) =

(k)
(k+1)
Hn+1
Hn1

Hn(k) Hn(k+1)

(17.6.20)

where we have used Hn(k) to denote the Hankel determinant Hn(k) ({cs }) for short.
The qd-algorithm was developed by Rutishauser in [243] and discussed further in
[242] and [244]. For a detailed treatment of it, we refer the reader to Henrici [131],
[132].
17.7 Pade Approximants and Exponential Interpolation
Pade approximants have a very close connection with the problem of interpolation by
a sum of exponential functions. The problem, in its simple form, is to nd a function


u(x; h) = nj=1 j e j x that satises the 2n interpolation conditions u(x0 + i h; h) = ci ,
i = 0, 1, . . . , 2n 1. Here, j and j are parameters to be determined. In case a solution exists, it can be constructed with the help of a method due to Prony [232]. It has
been shown by Weiss and McDonough [352] that the method of Prony is closely related
to the Pade table. As shown in [352], one rst constructs the [n 1/n] Pade approx

k
imant Fn1,n (z) from F(z) = 2n1
fraction expank=0 ck z . If Fn1,n (z) has the partial



n
(xx )/ h
sion j=1 A j /(z z j ), assuming simple poles only, then u(x; h) = nj=1 A j j 0 ,
where A j = A j /z j and j = 1/z j , j = 1, . . . , n .
In case Fn1,n (z) has multiple poles, the solution of Prony is no longer valid. This case
has been treated in detail by Sidi [280]. In such a case, the function u(x; h) needs to be
chosen from the set


j
r 
r

x/ h
B j,k x k1 j : j distinct, < arg j ,
j n .
Unh =
j=1 k=1

j=1

The following theorem, which concerns this problem, has been proved in [280]:

17.7 Pade Approximants and Exponential Interpolation

337

Theorem 17.7.1
(i) There exists a unique function u(x; h) in Unh that solves the interpolation problem
u(x0 + i h; h) = ci , i = 0, 1, . . . , 2n 1, if and only if the [n 1/n] Pade approx
2n1
imant Fn1,n (z) from F(z) = k=0
ck z k exists and satises limz Fn1,n (z) = 0.
(ii) In case Fn1,n (z) exists and is as in part (i), it has the partial fraction expansion
Fn1,n (z) =

j
s 

j=1 k=1

s

A j,k
,
j n.
(z z j )k j=1

Then, u(x; h) is given by

u(x; h) =

j
s 


E j,k

j=1 k=1

where

 p
i



k + (x x0 )/ h 1 (xx0 )/ h
j
,
k1

= [ p( p 1) ( p i + 1)]/i! is the binomial coefcient,

k
k
j = z 1
j , E j,k = (1) A j,k z j , 1 k j , 1 j s.

Another interpolation problem treated by Sidi [280] concerns the case in which h 0.
The result relevant to this problem also involves Pade approximants. To treat this case,
we dene the function set Un via


j
r 
r

Un =
B j,k x k1 e j x : j distinct,
j n .
j=1 k=1

j=1

Theorem 17.7.2
(i) There exists a unique function v(x) in Un that solves the interpolation problem
v (i) (x0 ) = i , i = 0, 1, . . . , 2n 1, if and only if the [n 1/n] Pade approximant

2n1
k k exists.
Vn1,n ( ) from V ( ) = k=0
(ii) In case Vn1,n ( ) exists, V n1,n ( ) = 1 Vn1,n ( 1 ) satises lim V n1,n ( ) =
0 and has the partial fraction expansion
V n1,n ( ) =

j
s 

j=1 k=1

s

B j,k
,
j n.
( j )k j=1

Then, v(x) is given by

v(x) =

j
s 

j=1 k=1

B j,k
(x x0 )k1 e j (xx0 ) .
(k 1)!

Integral representations for both u(x; h) and v(x) are given in [280], and they involve
Fn1,n (z) and Vn1,n ( ) directly. These representations are used in [280] to prove the
following theorem.
Theorem 17.7.3 Let f (x) be 2n 1 times differentiable in a neighborhood of x0 , and
set ci = f (x0 + i h) and i = f (i) (x0 ), i = 0, 1, . . . , 2n 1. Finally, assume that


k
the [n 1/n] Pade approximant Vn1,n ( ) from 2n1
k=0 k exists, its denominator

338

17 The Pade Table

polynomial has degree exactly n, and Vn1,n ( ) is not reducible. Then the following
hold:
(i) The interpolant v(x) of Theorem 17.7.2 exists.
(ii) The interpolant u(x; h) of Theorem 17.7.1 exists for all small h, and satises
lim u(x; h) = v(x).

h0

In a separate paper by Sidi [284], the interpolation problems above are extended to the
cases in which some of the exponents are preassigned. The solutions to these problems
are achieved by the so-called Pade-type approximants, which we discuss in the next
chapter. See [284] for details.
The method of Prony can be generalized by replacing the Pade approximant Fn1,n (z)

n1  i
n
ai z / j=0 bj z j , b0 = 1, with the bj determined
by the rational function n (z) = i=0
via the least-squares solution of the overdetermined linear system
n


ci j bj = 0, i = n, n + 1, . . . , N ,

j=0

where N is signicantly larger than 2n [cf. (17.1.4)], and the ai computed from
i


ci j bj = ai , i = 0, 1, . . . , n 1,

j=0

once the bj are determined [cf. (17.1.3)]. If n (z) = nj=1 A j /(z z j ) (simple poles as

(xx )/ h
sumed for simplicity), then u(x; h) = nj=1 A j j 0 , where A j = A j /z j and
j = 1/z j , j = 1, . . . , n, is the desired exponential approximation that satises u(x0 +
i h; h) ci , i = 0, 1, . . . , N . Essentially this approach and some further extensions of
it have been used in problems of signal processing in electrical engineering. (Note that,
in such problems, the ci are given with some errors, and this causes the original method
of Prony to perform poorly.)
17.8 Convergence of Pade Approximants from Meromorphic Functions
In this and the next sections, we summarize some of the convergence theory pertaining to
the Pade table. In this summary, we do not include the topics of convergence in measure
and convergence in capacity. Some of the theorems we state clearly show convergence
acceleration, whereas others show only convergence. For proofs and further results, we
refer the reader to the vast literature on the subject.
In this section, we are concerned with the convergence of rows of the Pade table from
the Maclaurin series of meromorphic functions.
17.8.1 de Montessuss Theorem and Extensions
e approximants
The classic result for row sequences { f m,n (z)}
m=0 (with n xed) of Pad
of meromorphic functions is de Montessuss theorem, which is a true convergence acceleration result and which reads as follows:

17.8 Convergence of Pade Approximants from Meromorphic Functions

339

Theorem 17.8.1 Let f (z) be analytic at z = 0 and meromorphic in the disk K = {z :

|z| < R} and let it have q poles in K counting multiplicities. Then, the row sequence
{ f m,q (z)}
m=0 converges to f (z) uniformly in any compact subset of K excluding the poles
of f (z), such that
lim sup | f (z) f m,q (z)|1/m |z/R|.

(17.8.1)

This theorem was originally proved by de Montessus de Ballore [64] and follows
from the work of Hadamard [120]. Different proofs of it have been given in Baker [15],
Karlsson and Wallin [150], Saff [249], and Sidi [292].
In case the singularities of f (z) on the boundary of the disk K , namely, on K = {z :
|z| = R}, are all poles, Theorem 17.8.1 can be improved substantially. This improvement,
presented originally in Sidi [292, Theorem 3.3], is quantitative in nature, and we state it
next.
Theorem 17.8.2 Let f (z) be analytic at z = 0 and meromorphic in the disk K and on
its boundary K . Let z 1 , . . . , z t be the poles of f (z) in K and let 1 , . . . , t be their re


spective multiplicities. Dene Q(z) = tj=1 (1 z/z j ) j and q = tj=1 j . Similarly,
let z 1 , . . . , z r be the poles of f (z) on K , and let 1 , . . . , r be their respective multiplicities. Thus, for each j {1, . . . , r }, the Laurent expansion of f (z) about z = z j is
given by
f (z) =

j

i=1

a ji
+ ' j (z); a j j = 0, ' j (z) analytic at z j . (17.8.2)
(1 z/z j )i

Let us now order the z j on K such that

1 = 2 = = > +1 r ,

(17.8.3)

and set p = 1 1. Then there holds

f (z) f m,q (z) =


  

a j j
Q(z j ) 2 z m+q+1
m p 
+ o(m p |z/R|m )
j=1 1 z/z j Q(z)
p!
z j

= O(m p |z/R|m ) as m ,

(17.8.4)

uniformly in any compact subset of K \{z 1 , . . . , z t }. This result is best possible as

m .
Theorem 17.8.2 also shows that, as m , f m,q (z) diverges everywhere on K . We
note that the result in (17.8.4) is best possible in the sense that its right-hand side gives
the rst term in the asymptotic expansion of f (z) f m,q (z) as m explicitly.
Now, Theorem 17.8.1 concerns the row sequence { f m,n (z)}
m=0 , where n = q, q being
the number of poles of f (z) in K , counted according to their multiplicities. It does not
apply to the sequences { f m,n (z)}
m=0 with n > q, however. Instead, we have the following
weaker result due to Karlsson and Wallin [150] on convergence of subsequences of
{ f m,n (z)}
m=0 .

340

17 The Pade Table

Theorem 17.8.3 Let f (z) be as in Theorem 17.8.1 with poles 1 , . . . , q in K that are
not necessarily distinct, and let n > q. Then, there exist n q points q+1 , . . . , n and
a subsequence { f m k ,n (z)}
k=0 that converges to f (z) uniformly in any compact subset of
K \{1 , . . . , n }, such that
lim sup | f (z) f m k ,n (z)|1/m k |z/R|.

(17.8.5)

It must be clear that a priori we do not have any knowledge of q+1 , . . . , n and the
integers m k in this theorem.
It is easy to construct examples for which one can show denitely that the sequence
{ f m,n (z)}
m=0 does not converge under the conditions of Theorem 17.8.3, but a subsequence does precisely as described in Theorem 17.8.3.
Treatment of Intermediate Rows


Similarly, Theorem 17.8.2 is not valid when q < n < q + rj=1 j . Rows of the Pade
table for which n takes on such values are called intermediate rows. Note that intermediate
rows may appear not only when n > q; when f (z) has multiple poles and/or a number
of poles with equal modulus in K , they appear with n < q as well. Thus, intermediate
rows are at least as common as those treated by de Montessuss theorem.
The convergence problem of intermediate rows was treated partially (for some special
cases) in a series of papers by Wilson [358], [359], [360]. Preliminary work on the
treatment of the general case was presented by Lin [168]. The complete solution for the
general case was given only recently by Sidi [292].
The following convergence result pertaining to the convergence of intermediate rows
in the most general case is part of Sidi [292, Theorem 6.1], and it gives a surprisingly
simple condition sufcient for the convergence of the whole sequence { f m,n (z)}
m=0 . This
condition involves the nonlinear integer programming problem IP( ) we discussed in
detail in the preceding chapter.
Theorem 17.8.4 Let f (z) be precisely as in Theorem 17.8.2 and let n = q + with


0 < < rj=1 j . Denote by IP( ) the nonlinear integer programming problem
maximize g(% ); g(% ) =

r

( k k k2 )
k=1

subject to

r


k = and 0 k k , 1 k r.

(17.8.6)

k=1

Then, { f m,n (z)}

m=0 converges uniformly to f (z) in any compact subset of K \{z 1 , . . . , z t },
provided IP( ) has a unique solution for 1 , . . . , r . If we denote by G( ) and G( + 1)
the (optimal) value of g(% ) at the solutions to IP( ) and IP( + 1), respectively, then
there holds
f (z) f m,n (z) = O(m G( +1)G( ) |z/R|m ) as m .

(17.8.7)

[Note that IP( + 1) need not have a unique solution.]

Obviously, when IP( ) does not have a unique solution, Theorem 17.8.3 applies, and
we conclude that a subsequence { f m k ,n (z)}
k=0 converges to f (z) uniformly, as explained

17.8 Convergence of Pade Approximants from Meromorphic Functions

341

in Theorem 17.8.3. Theorems 17.8.2 and 17.8.4, together with Theorem 17.8.3, present a
complete treatment of the convergence of row sequences in the Pade table of meromorphic
functions with polar singularities on their circles of meromorphy.
Interestingly, in the paper by Liu and Saff [169], the existence of a unique solution to
IP( ) features prominently as a sufcient condition for convergence of the intermediate
rows of Walsh arrays of best rational approximations as well.

17.8.2 Generalized Koenigs Theorem and Extensions

The following result concerning the poles of Pade approximants is known as the generalized Koenigs theorem.
Theorem 17.8.5 Let f (z) be as in Theorem 17.8.1 and denote its poles by 1 , . . . , q .
Here, the i are not necessarily distinct and are ordered such that |1 | |q |.
q
Dene Q(z) = j=1 (1 z/ j ). Let Q m,n (z) be the denominator of f m,n (z), normalized
such that Q m,n (0) = 1. Then
lim sup |Q m,q (z) Q(z)|1/m |q /R| as m .

(17.8.8)

The special case in which q = 1 was proved originally by Koenig [153]. The general
case follows from a closely related theorem of Hadamard [120], and was proved by
Golomb [100], and more recently, by Gragg and Householder [108].
If the poles of f (z) in K are as in Theorem 17.8.2, then the result in (17.8.8) can be
rened, as shown in Sidi [292], and reads
Q m,q (z) Q(z) = O(m |q /R|m ) as m , 0 some integer. (17.8.9)
Of course, what Theorem 17.8.5 implies is that, for all large m, f m,q (z) has precisely
q poles that tend to the poles 1 , . . . , q of f (z) in K . If we let the poles of f (z)
and their multiplicities and p be as in Theorem 17.8.2, then, for each j {1, . . . , t},
f m,q (z) has precisely j poles z jl (m), l = 1, . . . , j , that tend to z j . Also, the p j th
derivative of Q m,q (z), the denominator of f m,q (z), has a zero z j (m) that tends to z j . More
specically, we have the following quantitative results, whose proofs are given in Sidi
[292, Theorem 3.1].
Theorem 17.8.6
(i) In Theorem 17.8.1,
1/

1/m 
= z j /R  j ,
lim sup z jl (m) z j 
m

 
1/m
 1 j




z jl (m) z j 
= z j /R  ,
lim sup
m j l=1

1/m 

lim sup z j (m) z j 
= z j /R  .
m

(17.8.10)

342

17 The Pade Table

(ii) In Theorem 17.8.2, we obtain the following more rened results:

z jl (m) z j = O([m p |z j /R|m ]1/ j ) as m ,
j
1 
z jl (m) z j = O(m p |z j /R|m ) as m ,
j l=1

z j (m) z j = O(m p |z j /R|m ) as m .

(17.8.11)

The rst of the results in (17.8.10) and (17.8.11) were given earlier by Goncar [101].
The version of (17.8.11) that is given in [292] is actually more rened in that it provides
the rst term of the asymptotic expansion of z jl (m) z j :
z jl (m) z j + E jl (m)(m p |z j /R|m )1/ j as m ,

(17.8.12)

where {E jl (m)}
m=0 is some bounded sequence with a subsequence that has a nonzero
limit.
A similar result related to intermediate rows of Pade approximants in Theorem 17.8.4
exists and is given as part of [292, Theorem 6.1]. For additional references concerning
the theorem of Koenig and its generalizations, we refer the reader to Sidi [292].
Recently, results that form a sort of inverse to the generalized Koenigs theorem have
been of interest. The essential question now is the following: Suppose that the function


k
e
f (z) has a formal power series
k=0 ck z and that the poles of some sequence of Pad
approximants from this series converge to a set X . Does it follow that f (z) (or some
continuation of it) is singular on X ? Is f (z) analytic off X ? The rst theorem along
these lines that concerns the poles of the sequence { f m,1 (z)}
m=0 was given by Fabry [81].
Fabrys result was generalized to the sequences { f m,n (z)}
m=0 with arbitrary xed n in the
works of Goncar [101] and of Suetin [329], [330]. It is shown in [329] and [330], in parconverge to some complex numbers
ticular, that if the poles of the sequence { f m,n (z)}

m=0
k
1 , . . . , n , not necessarily distinct, then k=0 ck z represents a function f (z) analytic at
0 and meromorphic in the disk K n = {z : |z| < Rn }, where Rn = max{|1 |, . . . , |n |}. In
addition, it is shown that if K = {z : |z| < R} is the actual disk of meromorphy of f (z),
then those i in the interior of K are poles of f (z), while those on the boundary of K
are points of singularity of f (z). For additional results and references on this interesting
topic, see also Karlsson and Saff [149]. This paper treats both rows and columns of Pade
tables of nonmeromorphic as well as meromorphic functions.

17.9 Convergence of Pade Approximants from Moment Series

Note that all our results about Pade approximants from meromorphic functions have
been on row sequences. No denitive results are currently known about diagonal sequences. However, there is a detailed convergence theory of diagonal sequences of Pade
approximants from moment series associated with Stieltjes and Hamburger functions
(also called Markov functions). This theory is also closely related to orthogonal polynomials and Gaussian integration. We present a summary of this subject in this section. For
more details and further results, we refer the reader to Baker and Graves-Morris [16] and
Stahl and Totik [321]. For orthogonal polynomials and related matters, see the books

17.9 Convergence of Pade Approximants from Moment Series

343

by Szego [332] and Freud [88]. For the moment problem, see also the book by Widder
[357].
Denition 17.9.1 Let (a, b) be a real interval and let (t) be a real function that is nondecreasing on (a, b) with innitely many points of increase there.
(a) Dene the function f (z) via

f (z) =
a

d(t)
,
1 + tz

(17.9.1)

where the integral is dened in the sense of Stieltjes. If 0 a < b , f (z) is

called a Stieltjes function, and if a < 0 < b , it is called a Hamburger
function.
(b) Let (t) be such that its moments f k dened by
 b
t k d(t), k = 0, 1, . . . ,
(17.9.2)
fk =
a

k
all exist. Then the formal ( convergent or divergent) power series
k=0 f k (z) is
said to be the moment series associated with (t). It is called a Stieltjes series if
0 a < b and a Hamburger series if a < 0 < b .
It is easy to see that a Stieltjes (or Hamburger) function is real analytic in the complex
z-plane cut along the real interval [a 1 , b1 ] (or along the real intervals (, b1 ]
and [a 1 , +)).


k
It is also easy to see that the moment series
k=0 f k (z) represents f (z) asymptotically as z 0, that is,
f (z)

f k (z)k as z 0.

(17.9.3)

k=0

k
and nite radius of convergence
Clearly, if (a, b) is nite,
k=0 f k (z) has a positive


k
and is the Maclaurin expansion of f (z). Otherwise,
k=0 f k (z) diverges everywhere.
If { f k } is a moment sequence as in Denition 17.9.1, then
Hn(m) ({ f s }) > 0, m = 0, 1 and n = 0, 1, . . . , for 0 a < b , (17.9.4)
and
Hn(m) ({ f s }) > 0, m = 0 and n = 0, 1, . . . , for a < 0 b . (17.9.5)
Conversely, if { f k } is such that (17.9.4) [or (17.9.5)] holds, then there exists a function
(t) as described in Denition 17.9.1 for which f k are as given in (17.9.2) with 0 a <
b (or a < 0 < b ). The function (t) is unique if


k=0

1/(2k)

fk

= when 0 a < b ,

(17.9.6)

344

17 The Pade Table

and

1/(2k)

f 2k

= when a < 0 < b .

(17.9.7)

k=0

The conditions in (17.9.6) and (17.9.7) are known as Carlemans conditions. It is easy


k
to verify that they are satised automatically if
k=0 f k (z) has a nonzero radius of
convergence. What is implied by Carlemans condition in (17.9.6) [or (17.9.7)] is that


k
there exists a unique Stieltjes (or Hamburger) function that admits
k=0 f k (z) as its
asymptotic expansion as z 0.


k
1
K (n z/1)
Given a Stieltjes series
k=0 f k (z) , there exists an S-fraction, namely, z
with n > 0, n = 1, 2, . . . , whose convergents are the [n/n], [n/n + 1], n = 0, 1, . . . ,


k
e table in question is
entries in the Pade table of
k=0 f k (z) . In addition, the Pad
normal. The poles of Pade approximants f m,n (z) from a Stieltjes series are all simple
and lie on the negative real axis, and the corresponding residues are all positive. Thus,
f n+ j,n (z) =

j


f k (z)k + (z) j+1

k=0

n

i=1

Hi
, ti > 0 distinct and Hi > 0. (17.9.8)
1 + ti z

Let us also denote by Q m,n (z) the denominator polynomial of f m,n (z). Then, with
j 1, j,n (t) t n Q n+ j,n (t 1 ) is the nth orthogonal polynomial with respect to
the inner product
 b
(F, G)
F(t)G(t)t j+1 d(t),
(17.9.9)
a

where (t) is the function that gives rise to the moment sequence { f k } as in (17.9.2).
The poles and residues of f n+ j,n (z) are also related to numerical integration. Specically,

n
the sum i=1
Hi g(ti ) is the n-point Gaussian quadrature
with the ti and Hi as in (17.9.8),
b
formula for the integral a g(t)t j+1 d(t).
Concerning the Pade approximants from Hamburger series, results analogous to those
of the preceding paragraph can be stated. For example, all f n+2 j1,n (z) exist for j 0
and there holds
f n+2 j1,n (z) =

2
j1
k=0

f k (z)k + z 2 j

n

i=1

Hi
, ti real distinct and Hi > 0. (17.9.10)
1 + ti z

We now state two convergence theorems for the diagonal sequences of Pade approx

k
imants from Stieltjes and Hamburger series
k=0 f k (z) . We assume that, in case


k
k=0 f k (z) diverges everywhere, the f k satisfy the suitable Carleman condition in
(17.9.6) or (17.9.7). We recall that, with this condition, the associated Stieltjes function
or Hamburger function f (z) in (17.9.1) is unique. Theorem 17.9.2 concerns the case


k
in which
k=0 f k (z) has a positive radius of convergence, while Theorem 17.9.3
concerns the case of zero radius of convergence.
Theorem 17.9.2
(i) If < a < 0 < b < and j 0, the sequence { f n+2 j1,n (z)}
n=0 converges to
f (z) in the open set D0 formed from the complex z-plane cut along (, b1 ]

17.10 Convergence of Pade Approximants from Polya Frequency Series

345

and [a 1 , +), the pointwise rate of convergence being given by



 z 1 + b z 1 + a 


1/n
lim sup | f (z) f n+2 j1,n (z)| 

 < 1 (17.9.11)
 z 1 + b + z 1 + a 
n

with the phase convention that z 1 + b and z 1 + a are positive for z 1 > a.
The convergence is uniform in any compact subset of D0 .
(ii) If 0 a < b < and j 1, the sequence { f n+ j,n (z)}
n=0 converges to f (z) in
the open set D+ formed from the z-plane cut along (, b1 ], the pointwise rate
of convergence being given by


 1 + bz 1 


(17.9.12)
lim sup | f (z) f n+ j,n (z)|1/n 
<1

1 + bz + 1 
n

with the convention that 1 + bz > 0 for z > b1 . The convergence is uniform
in any compact subset of D+ .
Theorem 17.9.3
(i) When (a, b) = (, ) and j 0, the sequence { f n+2 j1,n (z)}
n=0 converges to
f (z) uniformly in {z : |z| R and |!z| > 0} for arbitrary R > 0 and small .
(ii) When (a, b) = (0, ) and j 1, the sequence { f n+ j,n (z)}
n=0 converges to f (z)
uniformly in {z : |z| R and (a) |!z| if z 0 and (b) |z| if z 0} for
arbitrary R > 0 and small > 0.

17.10 Convergence of Pade Approximants from Polya Frequency Series

k
olya
Denition 17.10.1 A formal power series f (z) :=
k=0 ck z is said to be a P
frequency series if
(1)m(m1)/2 Cm,n > 0, m, n = 0, 1, . . . .

(17.10.1)

Note that the sign of Cm,n is independent of n, and, for m = 0, 1, . . . , the sign pattern
+ + + + prevails. It is known (see Schonberg [260] and Edrei [72]) that all
Polya frequency series are Maclaurin series of functions of the form
f (z) = a0 e z


1 + i z
i=1

1 i z

(17.10.2)

where
a0 > 0, 0, i 0, i 0, i = 1, 2, . . . ,


(i + i ) < . (17.10.3)
i=1

Obviously, the Pade table from a Polya frequency series is normal. The following convergence theorem of Arms and Edrei [11] concerns rays of entries in the Pade
table.

346

17 The Pade Table

Theorem 17.10.2 Let f (z) be as in (17.10.2) and (17.10.3). Choose sequences of integers
{m k } and {n k } such that limk (m k /n k ) = . Then, the ray sequence { f m k ,n k (z)}
k=0
converges to f (z) uniformly in any compact set of the z-plane excluding the poles of
f (z) if there are such. Specically, with f m,n (z) = Pm,n (z)/Q m,n (z), Q m,n (0) = 1, we
have
lim Pm k ,n k (z) = a0 exp[ z/(1 + )]

(1 + i z),

i=1

lim Q m k ,n k (z) = exp[ z/(1 + )]

(1 i z),

i=1

uniformly in any compact set of the z-plane.

Note that Theorem 17.10.2 applies to the function f (z) = e z .

17.11 Convergence of Pade Approximants from Entire Functions

As has been shown by Perron [229, Chapter 4], convergence of Pade approximants from
arbitrary entire functions is not guaranteed. Indeed, we can have rows of the Pade table
not converging in any given open set of the complex plane.
In this section, we give convergence theorems for Pade approximants from entire
functions of very slow and smooth growth. For these and additional theorems on this
subject, we refer the reader to the papers by Lubinsky [180], [181], [182], [183], [184].
Our rst theorem is from [180].
Theorem 17.11.1 Let f (z) =

k=0 ck z

be entire with innitely many ck = 0 and

2

lim sup |ck |1/k = <

k

1
.
3

(17.11.1)

Then, there exists an increasing sequence of positive integers {m 0 , m 1 , . . . } such that

{ f m k ,n k (z)}
k=0 converges to f (z) uniformly in every compact set of the z-plane, where
{n 0 , n 1 , . . . } is arbitrary. This applies, in particular, with n k = m k + j and arbitrary j
(diagonal sequences). It also applies with n 0 = n 1 = (row sequences).
Theorem 17.11.1 has been improved in [184] by weakening the condition in (17.11.1)
as in Theorem 17.11.2 that follows.
Theorem 17.11.2 Let f (z) =

k=0 ck z

be entire with innitely many ck = 0 and

2

lim sup |ck |1/k = < 1.

(17.11.2)

Then, the diagonal sequence { f n,n (z)} converges to f (z) uniformly in every compact set
of the z-plane.

17.11 Convergence of Pade Approximants from Entire Functions

347

The next theorem gives another improvement of Theorem 17.11.1 and was proved in
[181].


k
Theorem 17.11.3 Let f (z) =
k=0 ck z be entire with ck = 0, k = 0, 1, . . . , and assume


ck1 ck+1 /c2  2 , k = 1, 2, . . . ,
(17.11.3)
k
0
where 0 = 0.4559 is the positive root of the equation
2

k = 1.

(17.11.4)

k=1

Then, the Pade table of f (z) is normal and, for any nonnegative sequence of integers
{n 0 , n 1 , . . . }, the sequence { f m,n m (z)}
m=0 converges to f (z) uniformly in any compact
set of the z-plane. In addition, the constant 0 in (17.11.3) is best possible.
2

Note that the ck in Theorem 17.11.3 satisfy lim supk |ck |1/k 0 . It is also in this
sense that Theorem 17.11.3 is an improvement over Theorem 17.11.1.
Lubinsky [183] showed that both rows and diagonals of the Pade table converge when
the condition in (17.11.3) is replaced by
lim ck1 ck+1 /ck2 = q, |q| < 1,

for some possibly complex scalar q.

(17.11.5)

18
Generalizations of Pade Approximants

18.1 Introduction
In this chapter, we consider some of the many generalizations of Pade approximants. We
describe the general ideas and show how the relevant approximations are constructed. We
do not go into their algebraic properties and the theory of their convergence, however. For
these subjects and an extensive bibliography, we refer the reader to Baker and GravesMorris [16].
The generalizations we mention are the so-called Pade-type approximants, multipoint Pade approximants, algebraic and differential HermitePade approximants, Pade
approximants from orthogonal polynomial expansions, BakerGammel approximants,
and PadeBorel approximants.
What we present here in no way exhausts the existing arsenal of approaches and
methods. For example, we leave out the vector and matrix Pade approximants. For these
topics, again, we refer the reader to [16].

18.2 Pade-Type Approximants

k
e
We begin with the formal power series f (z) :=
k=0 ck z and denote its [m/n] Pad
approximant by f m,n (z) as before. We recall that f m,n (z) is completely determined by
the equations in (17.1.3) and (17.1.4). Clearly, all the zeros and poles of f m,n (z) are
determined by these equations as well. In some situations, in addition to the ck , we may
have information about some or all of the zeros and/or poles of the function represented
by the series f (z), and we would like to incorporate this information in our rational approximants. This can be achieved via the so-called Pade-type approximants dened next.


k
Denition 18.2.1 Let the formal power series f (z) :=
k=0 ck z be given. The [m/n]
Pade-type approximant corresponding to f (z) and having known zeros z 1 , . . . , z and
known poles 1 , . . . , , with respective multiplicities 1 , . . . , and 1 , . . . , , such
that z i = 0 and i = 0, if it exists, is the rational function fm,n (z) = u(z)[ p(z)/q(z)],


(1 z/i )i and p(z) and q(z) are polynomials of
where u(z) = i=1 (1 z/z i )i / i=1
degree at most m and n, respectively, such that q(0) = 1 and and
fm,n (z) f (z) = O(z m+n+1 ) as z 0.
348

(18.2.1)

18.2 Pade-Type Approximants

349

The following elementary result that generalizes another due to Sidi [284] shows that,
when the known zeros and poles are xed, no new theory or justication is needed for
Pade-type approximants and that the known theory of Pade approximants applies directly.
It also shows that no new algorithms are needed to compute the Pade-type approximants
since the algorithms for ordinary Pade approximants discussed in Chapter 17 can be used
for this purpose.
Theorem 18.2.2 Let fm,n (z) be precisely as in Denition 18.2.1. The rational function


k
p(z)/q(z) is the [m/n] Pade approximant gm,n (z) from the power series
k=0 dk z of
the quotient g(z) = f (z)/u(z). Thus, fm,n (z) is unique as well.
Proof. Dividing both sides of (18.2.1) by u(z), we observe that p(z)/q(z) must satisfy
p(z)/q(z) g(z) = O(z m+n+1 ) as z 0.
The result now follows from the denition of Pade approximants, namely, from Denition 17.1.1.


Now, to compute p(z) and q(z), we need to know the dk . Obviously, the dk can be
computed by multiplying the power series f (z) by the Maclaurin series of 1/u(z).
As a result of Theorem 18.2.2, we have
(18.2.2)
fm,n (z) f (z) = [gm,n (z) g(z)]u(z),




k
where, as before, f (z) and g(z) are the functions represented by k=0 ck z k and
k=0 dk z
respectively, and g(z) = f (z)/u(z), of course. Because u(z) is completely known, we
can make statements on fm,n (z) f (z) by applying the known convergence theory of
Pade approximants to gm,n (z) g(z).
Finally, note that this idea can be extended with no changes whatsoever to the case in
which some or all of the z i and i are algebraic branch points, that is, the corresponding
i and i are not integers. We can thus dene fm,n (z) = u(z)[ p(z)/q(z)], with u(z),
p(z), and q(z) precisely as in Denition 18.2.1. Theorem 18.2.2 applies, in addition. Of
course, this time fm,n (z) is not a rational function, but has also algebraic branch points.
The approximations mentioned above (and additional ones involving multipoint Pade
approximants that we discuss later in this chapter) have been employed with success in
the treatment of several problems of interest in the literature of uid mechanics by Frost
and Harper [89].
A different and interesting use of Pade-type approximants has been suggested and
analyzed in a series of papers by Ambroladze and Wallin [6], [7], [8], [9]. This use
differs from the one just described in that the preassigned poles and their number are not
necessarily xed. They are chosen to enhance the quality of the Pade-type approximants
that can be obtained from a number of the coefcients ck .
For example, Ambroladze and Wallin [6] consider the Pade-type approximation of a
function
 b
w(t)
d(t),
f ( ) =

t
a

350

18 Generalizations of Pade Approximants

where (t) is a real function that is nondecreasing on the nite interval (a, b) with
innitely many points of increase
 b kthere, and w(t) is an entire function. The moment series,


k+1
c
/
,
where
c
=
k
k=0 k
a t w(t)d(t), k = 0, 1, . . . , converges to the function
f ( ) for all large . The Pade-type approximants are constructed from the moment
series of f ( ), and they are of the form rn ( ) = pn ( )/qn ( ), where, for each n, pn ( )
is a polynomial of degree at most n 1and qn ( ) is the nth orthogonal polynomial with
b
respect to the inner product (F, G) = a F(t)G(t) d(t), and pn ( ) is determined by
requiring that
f ( ) rn ( ) = O( n1 ) as .
That is, the preassigned poles of rn ( ) are the n zeros of qn ( ) [which are distinct and lie
in (a, b)] and thus their number tends to innity as n . It is shown in [6] that the
sequence {rn ( )}
n=1 converges to f ( ) uniformly in any compact subset of the extended
complex plane cut along the interval [a, b]. Furthermore, when w(t) 0 for t (a, b),
the upper bound on the error in rn ( ) is the same as that in the diagonal ordinary Pade


k+1
, whose numerator and denominator degrees are
approximant Rn ( ) from
k=0 ck /
at most n 1 and n, respectively, and satises
f ( ) Rn ( ) = O( 2n1 ) as .
[Recall that the convergence of the sequence {Rn ( )}
n=1 is covered completely by Theorem 17.9.2, where we set z = 1/ , when w(t) 0 for t (a, b).] Note that it takes


k+1
to determine Rn ( ) as opposed to only n for
2n coefcients of the series
k=0 ck /
rn ( ). This implies that, if c0 , c1 , . . . , c2n are available, we can construct both Rn ( )
and r2n ( ), but r2n ( ) is a better approximation to f ( ) than Rn ( ) is. In fact, the upper
bounds on the errors in Rn ( ) and in r2n ( ) suggest that the latter converges to f ( ) as
n twice as fast as the former.
Note that no theory of uniform convergence of {Rn ( )}
n=1 exists in case w(t) changes
sign on (a, b). In fact, examples can be constructed for which {Rn ( )}
n=1 does not
converge locally uniformly anywhere. See Stahl [320].

18.3 Vanden BroeckSchwartz Approximations

Let f (z) = k=0 ck z k as before. In Section 16.7, we introduced a generalization of the

-algorithm due to Vanden Broeck and Schwartz [344] that contains a parameter . Let

j
( j)
( j)
( j)
us set in (16.7.1) 0 = k=0 ck z k , j = 0, 1, . . . , and generate the k . The entries 2n ,
which we now denote fm,n (z), are rational approximations to the sum of the series f (z).
By (16.7.2), the fm,n (z), for m n, can be obtained recursively from
1
fm+1,n (z) fm,n (z)

1
fm1,n (z) fm,n (z)

1
+
.
fm,n+1 (z) fm,n (z)
fm,n1 (z) fm,n (z)

18.4 Multipoint Pade Approximants

351

Concerning fm,n (z), we observe the following:

( j)
1. Because k , k = 0, 1, 2, are independent of , the approximations fm,0 (z) and fm,1 (z)
are simply the ordinary Pade approximants f m,0 (z) and f m,1 (z) respectively. The rest
of the fm,n (z) are different from the f m,n (z) when = 1.
2. Just like f m,n (z), fm,n (z) too is determined from the m + n + 1 coefcients
c0 , c1 , . . . , cm+n and satises

fm,n (z) f (z) = O(z m+n+1 ) as z 0,

(18.3.1)

(m1,n)
(m,n1)
cm+n
= cm+n and cm+n
= cm+n ,

(18.3.2)

provided

where ck(m,n) are the coefcients of the Maclaurin expansion of fm,n (z), that is,


(m,n) k
fm,n (z) =
z . This can be shown by induction on n. The conditions in
k=0 ck
(18.3.2) can be violated only in exceptional circumstances because neither fm,n1 (z)
nor fm1,n (z) depends on cm+n .
Vanden Broeck and Schwartz [344] show with a numerical example that the diagonal approximations fn,n (z) with appropriate complex are quite effective in summing
everywhere-divergent asymptotic expansions also on the branch cuts of functions they
represent. They report some results for the function (1/z)e1/z E 1 (1/z), which we also de

k
k
note by f (z), which has the (everywhere-divergent) Euler series
 tk=0 (1) k!z as its
asymptotic expansion when z , z > 0. Here E 1 ( ) = (e /t) dt is the exponential integral. [Note also that this series is a Stieltjes series that satises the Carleman
condition, and f (z) is the corresponding Stieltjes function.] The function f (z) is analytic
in the z-plane cut along the negative real axis. Now, when the ck are real and is chosen to
be real, the fm,n (z) are real analytic, that is, they are real for z real. Thus, the fm,n (z) from
the Euler series are real when both z > 0 and z < 0. Although f (z) is real for z > 0, it
is not real for z < 0, because !E 1 (x i0) = for x > 0. We can obtain a complex
approximation in this case if we choose to be complex. We refer the reader to [344] for
more details. To date we are not aware of any research on the convergence properties of
this interesting method of approximation, which certainly deserves serious consideration.
18.4 Multipoint Pade Approximants
We dened a Pade approximant to be a rational function whose Maclaurin expansion


k
agrees with a given power series
k=0 ck z as far as possible. This idea can be generalized
as follows:


k
Denition 18.4.1 Let the formal power series Fr (z) :=
k=0 cr k (z z r ) , r =
% = (1 , . . . , q ). Then the rational function
1, . . . , q, with zr distinct, be given. Dene
f,n
% (z) with degrees of numerator and denominator at most m and n respectively, such

q
that m + n + 1 = r =1 (r + 1), is the q-point Pade approximant of type (1 , . . . , q )
q
from {Fr (z)}r =1 if it satises
r +1
f,n
) as z zr , r = 1, . . . , q.
% (z) Fr (z) = O((z z r )

(18.4.1)

352

18 Generalizations of Pade Approximants

m

n
i
i
Let us write f,n
% (z) =
i=0 i z / i=0 i z . From (18.4.1), it is clear that, if we set
0 = 1, then the i and i can be determined from the linear system of m + n + 1
equations that result from (18.4.1).
Multipoint Pade approximants may be very useful when, for each r , the power series
Fr (z) represents asymptotically as z zr a single function f (z). In such a case, f,n
% (z)
may be a very good approximation to f (z) in a large domain of the complex plane,
assuming that f (z) has suitable analyticity properties.
When r = 0 for all r , f,n
% (z) becomes simply a rational interpolant that assumes
given values at z 1 , z 2 , . . . , z q . This rational interpolation problem is known as the
CauchyJacobi problem and it can be treated numerically by, for example, Thieles
e
reciprocal difference algorithm. When q = 1, f,n
% (z) reduces to the [1 n/n] Pad
approximant from F1 (z).
For arbitrary q and r , a determinant expression for f,n
% (z) is known. See Baker [15].
(z)
as

with n xed can be treated

The convergence of the approximants f,n
%
r
by applying the following important theorem of Saff [249] from which de Montessuss
theorem can be derived as a special case.
Theorem 18.4.2 Let E be a closed bounded set in the z-plane whose complement K ,
including the point at innity, is connected and possesses a Greens function G(z) with
pole at innity. For each > 1, let  denote the locus G(z) = log , and let E denote
the interior of  . Let the sequence of (not necessarily distinct) interpolation points

{k(s) , k = 0, . . . , s}s=0 be given such that it has no limit points in K and


1/s
 s

(s) 

(z k ) = cap(E) exp{G(z)},
lim
s 
k=0

uniformly in z on each compact subset of K , where cap(E) is the capacity of E. Let f (z)
be analytic in E and meromorphic with precisely n poles, counting multiplicities, in E
for some > 1. Denote by D the region obtained from E by deleting the n poles of
f (z). Then, for all m sufciently large, there exists a unique rational function Rm,n (z)
with numerator of degree at most m and denominator of degree n that interpolates f (z)
at k(m+n) , k = 0, . . . , m + n. Each Rm,n (z) has n nite poles that converge to the poles
of f (z) in E . Furthermore, limm Rm,n (z) = f (z) throughout D and uniformly on
every compact subset of D . Specically, if S is a compact subset of D such that
S E , 1 < < , then

1/m

< 1.
lim sup max | f (z) Rm,n (z)|
zS

m
I n par ticular

1/m

1
lim sup max | f (z) Rm,n (z)|
< 1.
zE

The capacity of E is dened via

1/n


cap(E) = lim

min max | p(z)|

pPn zE

where Pn is the set of all polynomials of degree exactly n with leading coefcient unity.

18.4 Multipoint Pade Approximants

353

When k(s) = 0 for all k and s in Theorem 18.4.2, Rm,n (z) is nothing but the Pade
approximant f m,n (z) from the Maclaurin series of f (z). In this case, E = {z : |z| r }
for some r > 0, and hence cap(E) = r , G(z) = log(|z|/r ), and E = {z : |z| < r }, and
Saffs theorem reduces to de Montessuss theorem.

18.4.1 Two-Point Pade Approximants

The case that has received more attention than others is that of q = 2. This case can be
standardized by sending the points z 1 and z 2 to 0 and by a Moebius transformation.
In its symmetric form, this case can thus be formulated as follows:
Denition 18.4.3 Let the function f (z) satisfy
c0
+ c1 z + c2 z 2 + as z 0,
f (z)
2
c

c1
c2
0
+
+ 2 + as z ; c0 = 0.
f (z)
(18.4.2)
2
z
z
For any two integers i and j such that i + j is even, we dene the two-point Pade
approximant fi, j (z) to be the rational function

m
k z k
fi, j (z) = P(z) =
k=0
, 0 = 1, m = (i + j)/2,
(18.4.3)
m
k
Q(z)
k=0 k z
that satises
fi, j (z) f (z) = O(z i , z j1 )

O(z i )
as z 0,
j1
) as z ,
O(z

(18.4.4)

provided fi, j (z) exists.

Note that, in case f (0) + f () = 0, the asymptotic expansions of f (z) do not have
the symmetric form of (18.4.2). The symmetric form can be achieved simply by adding




i z i as z 0 and (z) i=0
i /z i
a constant to f (z). For example, if (z) i=0
as z , and 0 + 0 = 0, then f (z) = (z) (0 + 0 )/2 has asymptotic expansions
as in (18.4.2), with c0 = 0 0 and ci = i and ci = i , i = 1, 2, . . . .
The following results and their proofs can be found in Sidi [276].
Theorem 18.4.4 When fi, j (z) exists, it is unique .
Theorem 18.4.5 Let g(z) = 1/ f (z) in the sense that g(z) has asymptotic expansions
as z 0 and z obtained by inverting those of f (z) in (18.4.2) appropriately. If
g i, j (z) is the two-point Pade approximant from g(z) precisely as in Denition 18.4.3,
then g i, j (z) = 1/ fi, j (z).
Obviously, k and k can be obtained by solving the linear equations that result from
(18.4.4). In particular, the k satisfy the system
m

s=0

cr s s = 0, r = i m, i m + 1, . . . , i 1.

(18.4.5)

354

18 Generalizations of Pade Approximants

The approximant fi, j (z) has the determinant representation



 Sim (z) zSim1 (z) z m S j (z)


 ci1
ci2
cim1 

 ci2
ci3
cim2 


 .
.
..
..

 ..
.


 c
cim1
c j 
im
fi, j (z) =
,


 1
z zm 


 ci1 ci2 cim1 


 ci2 ci3 cim2 


 .
..
.. 
 ..
.
.


c

c
c
im

im1

(18.4.6)

where
c0
, Sk (z) = Sk1 (z) + ck z k , k = 1, 2, . . . .
(18.4.7)
2

k

k1
[Note that Sk (z) = c0 /2 + i=1
ci z i and Sk (z) = c0 /2 i=1
ci z i for k =
1, 2, . . . .] This representation is obtained by unifying the ones for the cases i j and
i j given in Sidi [276, Theorem 3]. As a consequence of (18.4.6), we also have that

m
k z k Simk (z)
fi, j (z) = k=0

.
(18.4.8)
m
k
k=0 k z
S0 (z) =

This implies that, once the k have been determined, the approximant fi, j (z) is known
for all practical purposes.
Two-point Pade approximants are convergents of certain continued fractions that are
known as M-fractions and T -fractions. (See Sidi [276] and McCabe and Murphy [208].)
For example, the approximants fr,r (z), fr +1,r 1 (z), r = 1, 2, . . . , are consecutive convergents of a continued fraction of the form
c+

1 w1 z 2 w2 z
dz
; i + i = 1 for all i,
+
1 + ez 1 + 1 + 2 + 1 + . . .

with f1,1 (z) = c + dz/(1 + ez), etc.

Being convergents of M-fractions, the fi, j (z) can be computed by the F-G algorithm
of McCabe and Murphy [208] that is of the quotient-difference type. They can also be
computed by a different method proposed in Sidi [276] that is based on the recursive
computation of the s .

18.5 HermitePade Approximants

We can view the Pade approximant f m,n (z) as being obtained in the following manner:
Determine the polynomials Pm,n (z) and Q m,n (z) by solving (17.1.2), namely,
Q m,n (z) f (z) Pm,n (z) = O(z m+n+1 ) as z 0, Q m,n (0) = 1,

18.5 HermitePade Approximants

355

and then obtain f m,n (z) as the solution to the equation Q m,n f m,n Pm,n = 0. This has
been generalized with HermitePade polynomials in two different ways, which we now
present.
Denition 18.5.1 Let h 1 (z), . . . , h r (z) be given formal power series. The HermitePade
polynomials [Q 1 (z), . . . , Q r (z)] of type (1 , . . . , r ), where Q k (z) is a polynomial of
degree at most k , k = 1, . . . , r, are the solution of
r


Q k (z)h k (z) = O(z 1

) as z 0,
=

r

(k + 1).

k=1

k=1

18.5.1 Algebraic HermitePade Approximants

Let f (z) := k=0 ck z k be a given formal series. Dene the HermitePade polynomials
[Q 0 (z), Q 1 (z), . . . , Q s (z)] of type (m 0 , m 1 , . . . , m s ) via
s


Q k (z)[ f (z)]k = O(z m1

) as z 0, m =

k=0

s

(m k + 1).

(18.5.1)

k=0

m k
Thus, Q k (z) = i=0
qki z i for each k and we set qs0 = 1. (Obviously, the qki satisfy a
linear system of m 1 equations.) With the Q k (z) available, solve the equation
s


Q k (z) k = 0

(18.5.2)

k=0

for . Obviously, this is a polynomial equation in that has s solutions that are functions


k
of z. The solution whose Maclaurin expansion agrees with
k=0 ck z as far as possible is
the algebraic HermitePade approximant of type m
% = (m 0 , m 1 , . . . , m s ), and we denote
it fm% (z). Obviously, we recover the ordinary Pade approximant f m 0 ,m 1 (z) when s = 1.
When s = 2, after determining Q 0 (z), Q 1 (z), and Q 2 (z), we obtain



= (z) = Q 1 (z) + [Q 1 (z)]2 4Q 0 (z)Q 2 (z) /[2Q 2 (z)],
whose singularities may be poles and branch points of square-root type. Provided we
pick the right branch for the square-root, we obtain fm% (z) = (z). The fm% (z) obtained
this way are known as the quadratic approximants of Shafer [263].

18.5.2 Differential HermitePade Approximants

Let f (z) := k=0 ck z k be a given formal series. Dene the HermitePade polynomials
[Q 1 (z), Q 0 (z), . . . , Q s (z)] of type (m 1 , m 0 , . . . , m s ) via
s

k=0

Q k (z) f (k) (z) Q 1 (z) = O(z m1

) as z 0, m =

s

k=1

(m k + 1). (18.5.3)

m k
Thus, Q k (z) = i=0
qki z i for each k and we set qs0 = 1. What is meant by f ( j) (z) is


k j
the formal power series
that is obtained by differk= j k(k 1) (k j + 1)ck z
entiating f (z) formally termwise. (The qki satisfy a linear system of m 1 equations in

356

18 Generalizations of Pade Approximants

this case too.) With the Q k (z) determined, solve the linear ordinary differential equation
s


Q k (z)y (k) = Q 1 (z), y ( j) (0) = c j ( j!), j = 0, 1, . . . , s 1,

(18.5.4)

k=0

for y(z), provided the Q k (z) allow such a solution to exist. When it exists, y(z) is the
differential HermitePade approximant of type m
% = (m 1 , m 0 , . . . , m s ), and we denote

it fm% (z). It must be noted that a priori it is not clear what one should take for the integers
m k to guarantee that (18.5.4) has a solution. Clearly, fm% (z) is not necessarily a rational
function. Obviously, we recover the ordinary Pade approximant f m 1 ,m 0 (z) when s = 0.
Differential HermitePade approximants seem to have originated in so-called series
analysis in statistical mechanics, for the case s = 2; the resulting approximants are
known as GammelGuttmannGauntJoyce (G3 J) approximants. These were given in
the works of Guttmann and Joyce [119], Joyce and Guttmann [146], and Gammel [91].
For a detailed review, we refer the reader also to Guttmann [118].

18.6 Pade Approximants from Orthogonal Polynomial Expansions

Let (x) be a real function that is nondecreasing in [a, b] with innitely many points
of increase there. Let k (x) be the kth orthogonal polynomial with respect to the inner
product ( , ) dened by
 b
F(x)G(x) d (x).
(18.6.1)
(F, G) =
a

When a formal expansion f (x) := k=0 ck k (x) is given, we would like to nd a

rational function Rm,n (x) = Pm,n (x)/Q m,n (x), with degrees of Pm,n (x) and Q m,n (x) at
most m and n, respectively, such that Rm,n (x) is a good approximation to the sum of the
given expansion when the latter converges. Two different types of rational approximation
procedures have been discussed in the literature.

18.6.1 Linear (Cross-Multiplied) Approximations

Dene Rm,n (x) by demanding that
Q m,n (x) f (x) Pm,n (x) :=

c k k (x).

(18.6.2)

k=m+n+1

In other words, we are requiring Q m,n f Pm,n to be orthogonal to k , k = 0, 1, . . . ,

m + n, with respect to ( , ) in (18.6.1). Thus, (18.6.2) is the same as
(Q m,n f Pm,n , k ) = 0, k = 0, 1, . . . , m,
(Q m,n f, k ) = 0, k = m + 1, . . . , m + n,

(18.6.3)

supplemented by the normalization condition

Q m,n (0 ) = 1 for some 0 [a, b].

(18.6.4)

18.6 Pade Approximants from Orthogonal Polynomial Expansions

357

[Obviously, (18.6.4) makes sense as the existence of Rm,n (x) depends also on its being
free of singularities on [a, b].]
The Rm,n (x) dened this way are called linear or cross-multiplied approximants. They
were originally developed by Maehly [205] for Chebyshev polynomial expansions.
Cheney [47] generalized them to series of arbitrary orthogonal polynomials, and
Holdeman [134] considered more general expansions. Fleischer [84] applied these approximations to Legendre polynomial expansions. For their convergence theory see
Lubinsky and Sidi [185] and Goncar, Rakhmanov, and Suetin [102]. [Note that Lubinsky
and Sidi [185] allow d (x) to have (a nite number of) sign changes on (a, b).]
Expressing Pm,n (x) and Q m,n (x) in the form
Pm,n (x) =

m


pi i (x) and Q m,n (x) =

i=0

n


qi i (x),

(18.6.5)

i=0

and using the fact that

i+ j


i (x) j (x) =

i jk k (x), i jk constants,

(18.6.6)

k=|i j|

it can be shown that, unlike the [m/n] Pade approximant from a power series, Rm,n (x)
is determined by the rst m + 2n + 1 terms of the series f (x), namely, by c0 , c1 , . . . ,
cm+2n . To see this, we start by noting that
f (x)k (x) :=

Akl l (x); Akl =

l=0

l+k


c j jkl ,

(18.6.7)

j=|lk|

which follows by invoking (18.6.6). Next, observing that (Q m,n f, k ) = (Q m,n , f k ),

and assuming for simplicity of notation that the k (x) are normalized such that
(k , k ) = 1 for all k, we realize that (18.6.3) and (18.6.4) can be expressed as in
pk =

n


Akl ql , k = 0, 1, . . . , m,

l=0
n


Akl ql = 0, k = m + 1, . . . , m + n,

l=0
n


ql l (0 ) = 1.

(18.6.8)

l=0

It is clear that the Akl in these equations are constructed from ck , 0 k m + 2n. It is
also clear that the ql are computed rst, and the pk are computed with the help of the ql .
Making use of the fact that
Pm,n (x) =



n
m
n


ql
Akl k (x) and Q m,n (x) =
ql l (x),
l=0

and of the conditions

n
l=0

k=0

(18.6.9)

l=0

Akl ql = 0, k = m + 1, . . . , m + n, we can conclude that

358

18 Generalizations of Pade Approximants

Rm,n (x) has the interesting determinant representation


 Sm,0 (x) Sm,1 (x) . . .

 Am+1,0 Am+1,1 . . .

 Am+2,0 Am+2,1 . . .

 .
..
 ..
.

A
A
m+n,0
m+n,1 . . .
Rm,n (x) = 
 0 (x) 1 (x) . . .

 Am+1,0 Am+1,1 . . .

 Am+2,0 Am+2,1 . . .

 .
..
 ..
.

A
A
...
m+n,0

m+n,1


Sm,n (x)

Am+1,n 
Am+2,n 
.. 
. 
Am+n,n 
,
n (x) 

Am+1,n 
Am+2,n 
.. 
. 

A

(18.6.10)

m+n,n

where
Sm,l (x) =

m


Akl k (x).

(18.6.11)

k=0

18.6.2 Nonlinear (Properly Expanded) Approximations

The linear approximations were derived by working with the expansion of Q m,n f Pm,n .
The nonlinear ones are derived by considering the expansion of f Rm,n directly. We
now require, instead of (18.6.2), that
f (x)


Pm,n (x)
:=
c k k (x).
Q m,n (x)
k=m+n+1

(18.6.12)

This is equivalent to
ck (k , k ) = (k , f ) = (k , Pm,n /Q m,n ), k = 0, 1, . . . , m + n, (18.6.13)
and these equations are nonlinear in the coefcients of the polynomials Pm,n and Q m,n .
Furthermore, only the rst m + n + 1 coefcients of f (x), namely, c0 , c1 , . . . , cm+n , are
needed now.
The approximations Rm,n (x) dened this way are called nonlinear or properly expanded approximants and were developed by Fleischer [86]. They are also called
by the names of the orthogonal polynomials involved; for example, LegendrePade,
ChebyshevPade, etc. They turn out to be much more effective than the linear approximations. For their convergence theory, see Suetin [328], Lubinsky and Sidi [185], and
Goncar, Rakhmanov, and Suetin [102]. Some results on the convergence of the so-called
ChebyshevPade approximations are also given in Gragg [107].
One way of determining Rm,n is by expressing it as a sum of partial fractions and
solving for the poles and residues. Let us assume, for simplicity, that m n 1 and that
all the poles of Rm,n (x) are simple. Then
Rm,n (x) = r (x) +

n

j=1

mn

Hj
, r (x) =
rk k (x).
x j
k=0

(18.6.14)

18.6 Pade Approximants from Orthogonal Polynomial Expansions

359

Therefore, the equations in (18.6.13) become

n

H j k ( j ) = ck (k , k ), k = 0, 1, . . . , m n,
rk (k , k ) +
j=1
n


H j k ( j ) = ck (k , k ), k = m n + 1, . . . , m + n,

(18.6.15)

j=1

where k ( ) are the functions of the second kind that are dened by
 b
k (x)
d (x),  [a, b].
k ( ) =
x

(18.6.16)

[Note again that j  [a, b] because otherwise Rm,n (x) does not exist.] Clearly, the H j
and j can be determined from the last 2n equations in (18.6.15) and, following that, the
rk can be computed from the rst m n + 1 of these equations.
Because the equations for the H j and j are nonlinear, more than one solution for them
may be obtained. Numerical computations with Legendre polynomial expansions have
shown that only one solution with j  [a, b] is obtained. This point can be understood
with the help of the following uniqueness theorem due to Sidi [269].


Theorem 18.6.1 Let f (x) be a real function in C[a, b] such that f (x) :=
k=0 ck k (x),
ck = (k , f )/(k , k ) for all k. Let Rm,n (x) be the [m/n] nonlinear Pade approximant
to f (x) such that Rm,n (x) has no poles along [a, b]. Then Rm,n (x) is unique.
[The more basic version of this theorem is as follows: There exists at most one real
rational function Rm,n (x) with degree of numerator at most m and degree of denominator
at most n and with no poles in [a, b], such that (k , Rm,n ) = ck , k = 0, 1, . . . , m + n,
where the ck are given real numbers.]
ChebyshevPade Table
A very elegant solution for Rm,n (x) has been given by Clenshaw and Lord [53] for
Chebyshev polynomial expansions. This solution circumvents the nonlinear equations in

m

n


ck Tk (x) and Rm,n (x) = i=0
pi Ti (x)/ i=0
q T (x),
a clever fashion. If f (x) := k=0

 si i
q0 = 2, where Tk (x) is the kth Chebyshev polynomial of the rst kind and
i=0 i =

s
1
+ i=1 i , then the coefcients pi and qi can be determined as follows:
2 0


1. Solve ns=0 s c|r s| = 0, r = m + 1, . . . , m + n; 0 = 1.

n

ns
i s+i , s = 1, . . . , n; 1 = 12 i=0
i2 .
2. Compute qs = i=0


1
n
3. Compute pr = 2
s=0 qs (cr +s + c|r s| ), r = 0, 1, . . . , m.
The expensive part of this procedure for increasing n is determining the s s(m,n) .
Clenshaw and Lord [53] have given a recursive algorithm for the s(m,n) that enables
their computation for m n. A different algorithm has been given by Sidi [267] and it
enables s(m,n) to be computed for all m and n. Here are the details of this algorithm.
1. Set
(m,n)
(m,n)
= 0 = n+1
for all m, n.
0(m,n) = 1 and 1

360

18 Generalizations of Pade Approximants

2. Compute

n

(m,n+1)

s=0

n

s(m+1,n) c|m+1s|

s=0

s(m,n) c|ms|

3. Compute s(m,n+1) recursively from

(m,n)
s(m,n+1) = s(m+1,n) + (m,n+1) s1
, s = 0, 1, . . . , n + 1.


Of course, this algorithm is successful as long as ns=0 s(m,n) c|ms| = 0. A similar
condition can be formulated for the algorithm of Clenshaw and Lord [53].


Another way of obtaining the Rm,n (x) from f (x) := k=0
ck Tk (x) with m n makes
use of the fact that

Tk (x) =

1 k
(t + t k ), k = 0, 1, . . . ; x = cos , t = ei .
2

k
Let us rewrite f (x) in the form f (x) = 12 [g(t) + g(t 1 )], where g(t) := k=0
ck t ,
and compute the [m/n] Pade approximant gm,n (t) to g(t). Then Rm,n (x) = 12 [gm,n (t) +
gm,n (t 1 )]. According to Fleischer [85], this approach to the ChebyshevPade table was
rst proposed by Gragg. It is also related to the LaurentPade table of Gragg [107]. We
do not go into the details of the latter here.
The block structure of the ChebyshevPade table has been analyzed by Geddes [93]
and by Trefethen and Gutknecht [337], [338].

18.7 BakerGammel Approximants

Another approach to accelerating the convergence of orthogonal polynomial expansions
makes use of the generating functions of these polynomials and the resulting approximations are called BakerGammel approximants. This approach can be applied to any


formal expansion f (x) :=
k=0 ck k (x), when some generating function G(z, x) for

{k (x)}k=0 , namely,
G(z, x) =

k z k k (x), k constants,

(18.7.1)

k=0

is known and it is analytic in a neighborhood of z = 0, with x restricted appropriately.

Therefore,
"
1
z k1 G(z, x) dz, k = 0, 1, . . . ,
(18.7.2)
k k (x) =
2i |z|=
such that G(z, x) is analytic for |z| . Substituting (18.7.2) in f (x) and interchanging
formally the summation and the integration, we obtain


"

1
ck k1
f (x) :=
G(z, x) dz.
(18.7.3)
z
2i |z|= k=0 k


k
Let h m,n (t) be the [m/n] Pade approximant for the series h(t) :=
k=0 (ck /k )t . Assume
that the contour |z| = in (18.7.3) can be deformed if necessary to a different contour

18.7 BakerGammel Approximants

361

C1 whose interior contains all the poles of z 1 h m,n (z 1 ). Then the [m/n] BakerGammel
approximant from f (x) is dened via
"
1
z 1 h m,n (z 1 )G(z, x) dz.
(18.7.4)
Rm,n (x) =
2i C1
If h m,n (t) has the partial fraction expansion (only simple poles assumed for simplicity)
h m,n (t) = r (t) +

n

j=1

mn

Hj
, r (t) =
rk t k ,
1 tz j
k=0

(18.7.5)

then, by the residue theorem, Rm,n (x) becomes

Rm,n (x) =

mn


k rk k (x) +

k=0

n


H j G(z j , x).

(18.7.6)

j=1

From the way Rm,n (x) is constructed, it is clear that we need to determine the partial
fraction expansion of h m,n (t) in (18.7.5) numerically.
Obviously, if G(z, x) is not a rational function in x, Rm,n (x) is not either. Interestingly,
if we expand Rm,n (x) formally in terms of the k (x), invoking (18.7.1) and using the fact
that h m,n (t) h(t) = O(t m+n+1 ) as t 0, we obtain
Rm,n (x) f (x) :=

c k k (x),

(18.7.7)

k=m+n+1

analogously to (18.6.12) for nonlinear Pade approximants.

Let us now illustrate this approach with a few examples:
1. G(z, x) = 1/(1 zx), k = 1, k (x) = x k . The resulting Rm,n (x) is nothing but the


k
[m/n] Pade approximant from f (x) :=
k=0 ck x .
2
2. G(z, x) = (1 x z)/(1 2x z + z ), k = 1, k (x) = Tk (x) (Chebyshev polynomials of the rst kind). The resulting Rm,n (x) is a rational function and, more importantly,


it is the [m/n] ChebyshevPade approximant from
k=0 ck Tk (x).
A different generating function for the Tk (x) is G(z, x) = 1 12 log(1 2x z + z 2 )
for which 0 = 1 and k = 1/k for k 1. Of course, the Rm,n (x) produced via this
G(z, x) has logarithmic branch singularities in the x-plane.
3. G(z, x) = 1/(1 2x z + z 2 ), k = 1, k (x) = Uk (x) (Chebyshev polynomials of the
second kind). The resulting Rm,n (x) is a rational function and, more importantly, it is


the [m/n] ChebyshevPade approximant from
k=0 ck Uk (x).
4. G(z, x) = (1 2x z + z 2 )1/2 , k = 1, k (x) = Pk (x) (Legendre polynomials). The



n
resulting Rm,n (x) is of the form Rm,n (x) = mn
k=0 rk Pk (x) +
j=1 H j (1 2x z j +
z 2j )1/2 and thus has branch singularities of the square-root type in the x-plane.
5. G(z, x) = e x z , k = 1/k!, k (x) = x k . The resulting Rm,n (x) is of the form Rm,n (x) =

n

mn
k
xz j
(i)
. In this case, (18.7.7) is equivalent to Rm,n
(0) =
k=0 (rk /k!)x +
j=1 H j e
(i)
f (0), i = 0, 1, . . . , m + n. This implies that the approximation Rn1,n (x) is nothing
but the solution to the exponential interpolation problem treated in Theorem 17.7.2.
Note that the assumption that the poles of h m,n (t) be simple is not necessary. We made
this assumption only for the sake of simplicity. The treatment of multiple poles poses no
difculty in determining Rm,n (x) dened via (18.7.4).

362

18 Generalizations of Pade Approximants

Finally, we propose to modify the BakerGammel approximants Rm,n (x) by replacing

k
the Pade approximants h m,n (t) from h(t) :=
k=0 (ck /k )t , whenever possible, by other
suitable rational approximations, such as the SidiLevin rational d-approximants we
discussed earlier. This may produce excellent results when, for example, {cn /n } b(m)
for some m.

18.8 PadeBorel Approximants

Closely related to BakerGammel approximants are the PadeBorel approximants,
which have proved to be useful in summing divergent series with zero radius of convergence. These approximations were introduced in a paper by Graf, Grecchi, and Simon
[103].


k
k
Let f (z) :=
k=0 ck z and assume it is known that ck = dk [( pk)!] anddk = O( )
t
as k for some p > 0 and > 0. Invoking in f (z) the fact that ! = 0 e t dt,
 > 1, and interchanging the integration and summation formally, we obtain



et g(zt p ) dt, g(u) :=
dk u k .
(18.8.1)
f (z) :=
0

k=0

Note that the series g(u) has a nonzero radius of convergence r and represents a function
that is analytic for u < r . Let us denote this function by g(u) as well and assume that it can
be continued analytically to all u > 0. If, in addition, the integral in (18.8.1) converges,
its value is called the Borel sum of f (z). The PadeBorel approximant fm,n (z) of f (z)
is dened by

et gm,n (zt p ) dt,
(18.8.2)
fm,n (z) =
0

where gm,n (u) is the [m/n] Pade approximant from g(u). The integral on the right-hand
side needs to be computed numerically.
Applying Watsons lemma, it can be shown that
fm,n (z) f (z) = O(z m+n+1 ) as z 0,

(18.8.3)

analogously to ordinary Pade approximants.

Of course, the Pade approximant gm,n (u) can be replaced by any other suitable rational
approximation in this case too.

19
The Levin L- and Sidi S-Transformations

19.1 Introduction
In this and the next few chapters, we discuss some nonlinear sequence transformations
that have proved to be effective on some or all types of logarithmic, linear, and factorial
sequences {Am } for which {Am } b(1) . We show how these transformations are derived, and we provide a thorough analysis of their convergence and stability with respect
to columns in their corresponding tables, as we did for the iterated 2 -process, the iterated Lubkin transformation, and the Shanks transformation. (Analysis of the diagonal
sequences turns out to be very difcult, and the number of meaningful results concerning
this has remained very small.)
We recall that the sequences mentioned here are in either b(1) /LOG or b(1) /LIN or
(1)
b /FAC described in Denition 15.3.2. In the remainder of this work, we use the
notation of this denition with no changes, as we did in previous chapters.
Before proceeding further, let us dene
a1 = A1 and am = Am1 = Am Am1 , m = 2, 3, . . . .

(19.1.1)

Consequently, we also have

Am =

m


ak , m = 1, 2, . . . .

(19.1.2)

k=1

19.2 The Levin L-Transformation

19.2.1 Derivation of the L-Transformation
We mentioned in Section 6.3 that the LevinSidi d (1) -transformation reduces to the Levin
u-transformation when the Rl in Denition 6.2.2 are chosen to be Rl = l + 1. We now
treat the Levin transformations in more detail.
In his original derivation, Levin [161] considered sequences {Am } that behave like
Am = A + m f (m), f (m)


i
as m ,
mi
i=0

(19.2.1)

for some known {m }, and dened the appropriate extrapolation method, which we now

363

19 The Levin L- and Sidi S-Transformations

364

call the L-transformation, as was done in Chapter 6, via the linear equations
Ar = L(nj) + r

n1

i
i=0

ri

, J r J + n; J = j + 1.

(19.2.2)

On multiplying both sides by r n1 and dividing by r , these equations become

r n1 Ar /r = L(nj)r n1 /r +

n1


i r n1r , J r J + n; J = j + 1. (19.2.3)

i=0

n1

is a polynomial in r of degree at most n 1, we can now

Because i=0 i r
proceed as in the proof of Lemma 17.3.1 to obtain the representation




n
i n
n1
A J +i / J +i
n J n1 A J / J
i=0 (1) i (J + i)
( j)

 =
n
n 
Ln =
; J = j + 1.
n
n1
i
n1
 J / J
/ J +i
i=0 (1) i (J + i)
n1r

(19.2.4)
Levin considered three different choices for the m and dened three different sequence transformations:
1. m = am (t-transformation)
2. m = mam (u-transformation)
3. m = am am+1 /(am+1 am ) (v-transformation)
Of these, the u-transformation appeared much earlier in work by Bickley and Miller [24,
p. 764].
Levin in his paper [161] and Smith and Ford in [317] and [318] (in which they presented
an exhaustive comparative study of acceleration methods) concluded that the u- and vtransformations are effective on all three types of sequences, whereas the t-transformation
is effective on linear and factorial sequences only. [Actually, all three transformations


are the best convergence acceleration methods on alternating series
k=1 (1)|ak | with
{an } b(1) .] The theoretical justication of these conclusions can be supplied with the
help of Theorem 6.7.2. That the t-transformation will be effective for linear and factorial
sequences is immediate by the fact that = 0 in Theorem 6.7.2 for such sequences. That
the u-transformation will be effective on logarithmic sequences is obvious by the fact that
= 1 in Theorem 6.7.2 for such sequences. These also explain why the t-transformation
is not effective on logarithmic sequences and why the u-transformation is effective for
linear and factorial sequences as well.
The justication of the conclusion about the v-transformation is a little involved. (a) In
the case of logarithmic sequences, {Am } b(1) /LOG, we have Am A = G(m), G
( )
A0 , = 0, 1, 2, . . . , and we can show that m = mam q(m) for some q A(0)
0 strictly.
We actually have q(m) ( 1)1 as m . (b) In the case of linear sequences,
( )
{Am } b(1) /LIN, we have Am A = m G(m), = 1, G A0 , with arbitrary , and
(0)
we obtain m = am q(m) for some q A0 strictly. We actually have q(m) ( 1)1
as m . (c) Finally, in the case of factorial sequences, {Am } b(1) /FAC, we have
( )
Am A = [(m)!]r m G(m), G A0 , r = 1, 2, . . . , and and are arbitrary, and
we can show that m = am+1 q(m), where q A(0)
0 strictly. Actually, q(m) 1 as
m . Now, from Theorem 6.7.2, we have that Am A = am+1 g(m + 1), where

19.2 The Levin L-Transformation

365

strictly. Substituting here am+1 = m /q(m) and noting that g(m + 1)/
g A(0)
0
(0)
q(m) A0 strictly, we obtain Am A = m f (m) with f A(0)
0 as well.
19.2.2 Algebraic Properties
Letting m = mam in (19.2.4) (u-transformation), we realize that




n J n2 A J /a J
n J n2 A j /A j
( j)

 =

 , J = j + 1,
Ln =
n J n2 /a J
n J n2 /A j

(19.2.5)

( j)

where the second equality holds for n 2. From (19.2.5), we see that L2 = W j ({As }),
where {W j ({As })} is the sequence produced by the Lubkin transformation. This observation is due to Bhowmick, Bhattacharya, and Roy [23].
The next theorem concerns the kernel of the u-transformation; it also provides the
kernel of the Lubkin transformation discussed in Chapter 15 as a special case.
( j)

( j)

Theorem 19.2.1 Let Ln be produced by the u-transformation on {Am }. We have Ln =

A for all j = 0, 1, . . . , and xed n, if and only if Am is of the form
Am = A + C

m
n1


P(k) + 1
, P(k) =
i k 1i ,
P(k)
k=2
i=0

C = 0, 0 = 1, P(k) = 0, 1, k = 2, 3, . . . . (19.2.6)
( j)
( j)
Proof. Let us denote the i in the equations in (19.2.2) that dene Ln by ni . We rst

n1 ( j) i
( j)
note that, by the fact that i=0 ni t is a polynomial of degree at most n 1, the ni
are uniquely determined by those equations with the index r = J + 1, . . . , J + n, (n in
( j+1)
( j+1)
when Ln
= A. This
number) in (19.2.2). The same equations determine the ni
( j)
( j+1)
( j)

forces ni = ni , 0 i n 1. Therefore, ni = i , 0 i n 1, for all j. As
a result,

Am = A + (Am1 )Q(m), Q(m) =

n1


i m 1i ,

(19.2.7)

i=0

must hold for all m. Writing (19.2.7) in the form

Am A = [(Am1 A)]Q(m),

(19.2.8)

and solving for Am A, we obtain

P(m) + 1
Am A
=
, P(m) = Q(m) 1,
Am1 A
P(m)
from which (19.2.6) follows. The rest is left to the reader.

(19.2.9)

Smith and Ford [317] have shown that the family of sequences of the partial sums
of Euler series is contained in the kernel of the u-transformation. As this is difcult to
conclude from (19.2.6) in Theorem 19.2.1, we provide a separate proof of it in Theorem 19.2.2 below.

19 The Levin L- and Sidi S-Transformations

366

k
Theorem 19.2.2 Let Am = m
k=1 k z , m = 1, 2, . . . , where is a nonnegative integer
( j)
on {Am }. Provided n + 2,
and z = 1. Let Ln be produced by the u-transformation
 d  1
( j)
there holds Ln = A for all j, where A = z dz 1z .
Proof. We start by observing that
 m



d 
d z z m+1
k
Am = z
, m = 1, 2, . . . .
z = z
dz
dz
1z
k=1
Next, it can be shown by induction on that


d z m+1
R(m, z)
z
= z m+1
,
dz 1 z
(1 z)+1
where R(m, z) is a polynomial in m and in z of degree . From this, we conclude that


Am satises (19.2.7). Substituting this in (19.2.5), the result follows.
( j)

We now turn to algorithms for computing the Ln . First, we can use (19.2.4) as is
for this purpose. Being a GREP(1) , the L-transformation can also be implemented very
conveniently by the W-algorithm of Sidi [278] discussed in Chapter 7. For this, we need
only to let tl = (l + 1)1 , a(tl ) = Al+1 , and (tl ) = l+1 , l = 0, 1, . . . , in our input for
the W-algorithm. Another recursive algorithm was given independently by Longman
[178] and by Fessler, Ford, and Smith [83]. This algorithm was called HURRY in [83],
where a computer program that also estimates error propagation in a thorough manner
is supplied. It reads as follows:
1. With J = j + 1 throughout, and for j = 0, 1, . . . , set
( j)

( j)

P0 = A J / J and Q 0 = 1/ J .
( j)

( j)

2. For j = 0, 1, . . . , and n = 1, 2, . . . , compute Pn and Q n recursively from



J + n 1 n2 ( j)
J
( j+1)
Un( j) = Un1
Un1 ,
J +n
J +n
( j)

( j)

( j)

where Un stands either for Pn or for Q n .

3. For all j and n, set
L(nj) = Pn( j) /Q (nj) .
Note that here
Pn( j) =
( j)

n (J n1 A J / J )
n (J n1 / J )
and Q (nj) =
.
n1
(J + n)
(J + n)n1
( j)

This prevents the Pn and Q n from becoming too large as n increases.

The L-transformation was extended slightly by Weniger [353] by replacing r i in
( j)
(19.2.2) by (r + )i for some xed . It is easy to see that the solution for Ln is now
obtained by replacing the factors J n1 and (J + i)n1 in the numerator and denominator of (19.2.4) by (J + )n1 and (J + + i)n1 , respectively. The effect of on the
quality of the resulting approximations is not clear at this time. (See Remark 1 following
Denition 6.2.1 in Chapter 6.)

19.2 The Levin L-Transformation

367

19.2.3 Convergence and Stability

We next summarize the convergence and stability properties of the u-transformation for
( j)
(1)
(1)
Process I (concerning {Ln }
j=0 ) as it is applied to sequences in b /LOG and b /LIN
(1)
and b /FAC.
Theorem 19.2.3 Let Am be as in (19.2.1), and let n+ be the rst nonzero i with i n,
( j)
and let Ln be as dened above.


(i) If m i=0
h i m i as m , h 0 = 0, = 0, 1, 2, . . . , then
L(nj) A (1)n
(ii) If m m

i=0

( + 1)n
(2 j)n
as j .
n+ j j n and n( j)
()n
|()n |

h i m i as m , h 0 = 0, = 1, then


L(nj) A ( + 1)n

n

n+ j+1 j 2n and

n( j)

(iii) If m (m!)r m

i=0

1 + | |
|1 |

n
as j .

h i m i as m , h 0 = 0, r = 1, 2, . . . , then

L(nj) A (1)n ( + 1)n n+ j+n+1 j 2n and n( j) 1 as j .

Part (i) of this theorem was given in Sidi [270], part (ii) was given in Sidi [273],
and part (iii) is new. These results can also be obtained by specializing those given in
Theorems 8.4.1, 8.4.3, 9.3.1, and 9.3.2. Finally, they can also be proved by analyzing
directly


n J n1 f (J )
( j)
 ; J = j + 1,
Ln A = n  n1
 J / J
and

n
n( j)



(J + i)n1 /| J +i |


; J = j + 1.
|n J n1 / J |

n
i=0 i

The analysis of Process II (concerning {Ln }

n=0 ) turns out to be quite difcult. We can
make some statements by making the proper substitutions in the theorems of Sections 8.5
and 9.4. The easiest case concerns the sequences {Am } in b(1) /LIN and b(1) /FAC and
was treated in [270].
( j)

Theorem 19.2.4 Let Am be as in (19.2.1), with m as in part (ii) or part (iii) of Theorem 19.2.3 and m m+1 < 0 for m > j. Assume that B(t) f (t 1 ) C [0, t] for
( j)
( j)
some t > 0. Then Ln A = O(n ) as n for every > 0, and n = 1.
A different approach to thegeneral case was proposed by Sidi [273]. In this approach,

one assumes that f (m) = m 0 emt (t) dt, where (t) is analytic in a strip about the

19 The Levin L- and Sidi S-Transformations

368

positive real axis in the t-plane. Subsequently, one analyzes

 J t t
e (e 1)n (n) (t) dt
( j)


Ln A = 0
; J = j + 1.
n J n1 / J
This can be achieved by using some of the properties of Laplace transforms. (See
Appendix B.)
This approach was used successfully in the analysis of some special cases by
 Sidi [273]

and [291]. Note that the difcult part of the analysis turns out to be that of n J n1 / J .
For example, the analysis of the special case in which m = m /m is the subject of a
long paper by Lubinsky and Sidi [186]. Writing


n J n1 / J = z j+1 n( j) (z); J = j + 1, z = 1 ,
with
n( j) (z) =

n


(1)ni

i=0

 
n
( j + i + 1)n z i ,
i

it is shown in [186] that

n!e(z)  (z) n
n(0) (z)
as n ,
ze
2n(z)
where (z) is the unique solution of the equation zev (1 v) = 1 in the region A dened
by
A = {v = a + ib : a 0, b (, ), 0 < |v 1|2 < (b/ sin b)2 }.
It is interesting (see Sidi [275] and Sidi and Lubinsky [314]) that (i) all the zeros of the
( j)
( j)
polynomials n (z) are simple and lie in (0, 1), and (ii) n (z) are orthogonal to (log z)k ,
0 k n 1, in the sense that
 1
z j n( j) (z)(log z)k dz = 0, 0 k n 1.
0

( j)

Thus, if we dene the scalars ni via the relation

 
n 
n

1
( j)
ni i ,

=
j
+
i
i=1
i=0
( j)

and the polynomials !n (y) via

!(nj) (y) =
( j)

( j)

( j)
n


(1)i ni y i ,
i!
i=0

then the n (z) and !n (log z) form a biorthogonal set of functions, in the sense that
 1
( j)
z j n( j) (z)!n  (log z) dz = 0, n = n  .
0

19.3 The Sidi S-Transformation

369

19.3 The Sidi S-Transformation

We again consider sequences {Am } that behave as in (19.2.1). Now observe that if




/m i as m , then K m i=0
i /(m)i as m , with (m)0 = 1
K m i=0
ii1
and (m)i = k=0 (m + k) for i 1, as before. It is clear that 0 = 0 , and, for each
i > 0, i is uniquely determined by 1 , . . . , i . Thus, (19.2.1) can be rewritten in the
form


i /(m)i m .
(19.3.1)
Am = A + m f (m), f (m)
i=0

We can now dene the S-transformation by truncating the asymptotic expansion in

(19.3.1) via the linear system
Ar = Sn( j) + r

n1

i
, J r J + n; J = j + 1.
(r )i
i=0

(19.3.2)

( j)

By Lemma 17.3.1, the solution of this system for Sn is given by



n
i n
n ((J )n1 A J / J )
i=0 (1) i (J + i)n1 A J +i / J +i
( j)

Sn =
; J = j + 1.
=
n
i n
n ((J )n1 / J )
i=0 (1) i (J + i)n1 / J +i
(19.3.3)
The m can be chosen exactly as in the L-transformation. The resulting sequence transformations have numerical properties similar to those of the t-, u-, and v-transformations,
except that they are quite inferior on sequences in b(1) /LOG. The S-transformation is
very effective on sequences {Am } in b(1) /LIN/FAC; its performance on such sequences
is quite similar to that of the L-transformation. Nevertheless, the L-transformation


k
appears to be the best method for handling alternating series
k=1 (1) ck , ck > 0
for all k.
As acknowledged also by Weniger [354, Section 2], the S-transformation was rst
used as a bona de convergence acceleration method in the M.Sc. thesis of Shelef [265],
which was done under the supervision of the author. The method was applied in [265] to
certain power series for the purpose of deriving some new numerical quadrature formulas
for the Bromwich integral. (We consider this topic in some detail in Section 25.7.)
Later, Weniger [353] observed that this transformation is a powerful accelerator for
some asymptotic power series with zero radius of convergence, such as the Euler series


k
k
k=0 (1) k!z . We discuss this in the next section. The name S-transformation was
introduced by Weniger [353] as well.
( j)
Computation of the Sn can be done by direct use of the formula in (19.3.3). It can also
be carried out via the following recursive algorithm that was developed by Weniger [353]:
1. With J = j + 1 throughout, and for j = 0, 1, . . . , set
( j)

( j)

P0 = A J / J and Q 0 = 1/ J .
( j)

( j)

2. For j = 0, 1, . . . , and n = 1, 2, . . . , compute Pn and Q n recursively from

( j+1)

Un( j) = Un1
( j)

( j)

( j + n 1)( j + n)
( j)
U ,
( j + 2n 2)( j + 2n 1) n1
( j)

where Un stands for Pn or for Q n .

19 The Levin L- and Sidi S-Transformations

370

3. For all j and n, set

Sn( j) = Pn( j) /Q (nj) .
Note that here
n ((J )n1 A J / J )
n ((J )n1 / J )
and Q (nj) =
.
(J + n)n1
(J + n)n1

Pn( j) =
( j)

( j)

This prevents the Pn and Q n from becoming too large as n increases.

The treatment of the kernel of the S-transformation is identical to that of the Ltransformation given in Theorem 19.2.1. A necessary and sufcient condition under
( j)
which Sn = A for all j = 0, 1, . . . , when m = mam , is that
Am = A + C

m
n1


P(k) + 1
, P(k) =
i k/(k)i ,
P(k)
k=2
i=0

C = 0, 0 = 1, P(k) = 0, 1, k = 2, 3, . . . .
We leave the details of its proof to the reader.
We next state a theorem on the convergence and stability of the S-transformation
under Process I. The proof can be accomplished in exactly the same way as those of
Theorem 19.2.3 and Theorem 19.2.4.

be the rst nonzero i with
Theorem 19.3.1 Let Am be as in (19.3.1), and let n+
( j)
i n, and let Sn be as dened above. Then, all the results of Theorem 19.2.3 and that
( j)
( j)

, respectively.
of Theorem 19.2.4 hold with Ln and n+ there replaced by Sn and n+

The results of this theorem pertaining to the convergence of the method on sequences
{Am } in b(1) /LOG and b(1) /LIN were mentioned by Weniger [353], while that pertaining
to sequences in b(1) /FAC is new.
For the proof of Theorem 19.3.1, we start with
Sn( j) A =
and

n
n( j)

n ((J )n1 f (J ))
; J = j + 1,
n ((J )n1 / J )



n
i=0 i

(J + i)n1 /| J +i |
; J = j + 1,
(Jn1 / J ) |

|n

and proceed as before.

The S-transformation was extended slightly by Weniger [353] by replacing (r )i in
(19.3.2) by (r + )i for some xed . This is also covered by Lemma 17.3.1 with
( j)
(17.3.2) and (17.3.3). Thus, the solution for Sn is now obtained by replacing the factors
(J )n1 and (J + i)n1 in the numerator and denominator of (19.3.3) by (J + )n1 and
(J + + i)n1 , respectively. The effect of on the quality of the resulting approximations is not clear at this time. (See Remark 1 following Denition 6.2.1 in Chapter 6
again.)

19.4 A Note on Factorially Divergent Sequences

371

A more substantial extension was also given by Weniger [354]. For this, one starts by
rewriting (19.3.1) in the form
Am = A + m f (m), f (m)

i /(c[m + ])i as m ,

(19.3.4)

i=0

which is legitimate. (As before, 0 = 0 and, for each i > 0, i is uniquely determined
by 1 , . . . , i .) Here, c and are some xed constants. We next truncate this expansion
to dene the desired extension:
Ar = Cn( j) + r

n1

i=0

i
, J r J + n; J = j + 1.
(c[r + ])i

(19.3.5)

These equations can also be solved by applying Lemma 17.3.1 to yield

Cn( j)

n ((c[J + ])n1 A J / J )
=
=
n ((c[J + ])n1 / J )



n

i n
i=0 (1) i (c[J +


n
i n
i=0 (1) i (c[J

+ i])n1 A J +i / J +i

.
+ + i])n1 / J +i
(19.3.6)

Weniger reports that the size of c may inuence the performance of this transformation.
For details, we refer the reader to [354].

19.4 A Note on Factorially Divergent Sequences

All our results concerning the L- and S-transformations so far have been on sequences
{Am } in b(1) /LOG, b(1) /LIN, and b(1) /FAC. We have not treated those sequences that
diverge factorially, in the sense that
Am (m!)r m

i m i as m ; r > 0 integer.

(19.4.1)

i=0

In keeping with Denition 15.3.2, we denote the class of such sequences b(1) /FACD.


Such sequences arise also from partial sums of innite series
k=1 ak , where
am = cm m , cm = (m!)r h(m), h(m)

i m i as m ; r > 0 integer.

i =0

(19.4.2)
We show the truth of this statement in the following lemma.


Lemma 19.4.1 Let am be as in (19.4.2) and Am = m
k=1 ak . Then




i m r i as m , 0 = 1/.
A m am 1 +

(19.4.3)

i=0

Proof. We start with

Am =



am1
a1
.
ak = am 1 +
+ +
am
am
k=1

m


(19.4.4)

19 The Levin L- and Sidi S-Transformations

372

Now, for each xed s,


ams

s,i m sr i as m , s,0 = 1/ s .
am
i=0

Therefore, for xed , there holds

r
1

am(1)
am1
+ +
=
i m r i + O(m r ) as m ,
am
am
i=0

(19.4.5)

and i , i = 0, 1, . . . r 1, remain unchanged if we increase . Also,

am
am(+1)
a1
+
+ +
= O(mam /am ) = O(m 1r ) as m . (19.4.6)
am
am
am
Combining (19.4.5) and (19.4.6) in (19.4.4), and realizing that is arbitrary, we obtain
the result in (19.4.3).


The following theorem on b(1) /FACD class sequences states that the two transformations diverge under Process I but denitely less strongly than the original sequences.


Theorem 19.4.2 Let Am be either as in (19.4.1) or Am = nk=1 ak with am as in (19.4.2).
With m = Am Am1 or m = am in the L- and S-transformations, and provided
n r + 1, there holds
Tn( j) K n j 2ns A j+1 as j , for some K n > 0 and integer s 0, (19.4.7)
( j)

( j)

( j)

where Tn stands for either Ln or Sn .

Proof. First, if Am is as in (19.4.1), then




r i
i m
m = Am Am1 Am 1 +
as m , 0 = 1/.
If Am =

n
k=1

i=0

ak with am as in (19.4.2), we have from Lemma 19.4.1 similarly





r i
i m
A m am 1 +
as m , 0 = 1/.
i=0

In either case, m Am as m and





Am
r i
1+
i m
as m , 0 = 1/ = 0.
m
i=0
In the case of the L-transformation, we then have
n (J n1 A J / J ) K J n1s and n (J n1 / J ) (1)n J n1 / J as j ,
for some integer s 0 and positive constant K . We have the same situation in the case
of the S-transformation, namely,
n ((J )n1 A J / J ) K J n1s and n ((J )n1 / J ) (1)n J n1 / J as j .
The result now follows by invoking (19.2.4) and (19.3.3).

19.4 A Note on Factorially Divergent Sequences

373

Table 19.4.1: Absolute errors in Sn(0) (z) and L(0)

n (z) on the Euler series


k
k
(1)
k!z
for
z
=
1/2,
1,
3.
Here
E
(z;
S)
= |Sn(0) (z) U (z)| and
n
k=0

x

e
E n (z; L) = |L(0)
n (z) U (z)|, where U (z) = 0 1+x/z d x is the Borel sum of the series
n

E n (1/2; S)

E n (1/2; L)

E n (1; S)

E n (1; L)

E n (3; S)

E n (3; L)

2
4
6
8
10
12
14
16
18
20
22
24
26
28
30
32
34
36
38
40

4.97D 02
9.67D 04
4.81D 05
4.62D 06
5.32D 07
1.95D 08
9.74D 09
7.74D 10
1.07D 10
3.74D 11
3.80D 12
4.44D 13
2.37D 13
3.98D 14
2.07D 16
1.77D 15
5.25D 16
7.22D 17
2.84D 17
2.38D 16

4.97D 02
4.81D 05
1.12D 04
1.76D 05
2.38D 06
3.25D 07
4.64D 08
7.00D 09
1.12D 09
1.86D 10
3.19D 11
5.55D 12
9.61D 13
1.61D 13
2.48D 14
2.91D 15
2.87D 14
8.35D 13
4.19D 11
2.35D 10

2.49D 02
3.38D 04
8.66D 06
1.30D 07
2.50D 08
2.49D 10
1.31D 10
4.37D 12
9.61D 13
9.23D 14
5.58D 15
1.70D 15
6.84D 17
2.22D 17
3.90D 18
3.87D 20
8.36D 20
1.51D 20
3.74D 20
1.66D 19

2.49D 02
9.85D 04
5.18D 05
1.26D 06
7.92D 08
1.77D 08
2.23D 09
2.45D 10
2.58D 11
2.70D 12
2.84D 13
3.05D 14
3.39D 15
3.96D 16
4.99D 17
6.89D 18
4.57D 18
2.06D 16
1.05D 15
1.09D 13

3.64D 03
2.92D 05
1.96D 07
1.73D 09
6.44D 12
5.06D 13
2.13D 14
2.73D 16
1.56D 17
8.42D 19
6.31D 21
1.97D 21
2.04D 23
5.57D 24
1.28D 25
2.08D 26
1.83D 27
2.75D 28
6.64D 27
2.43D 26

3.64D 03
1.74D 04
4.47D 06
1.52D 08
3.20D 09
7.67D 11
2.42D 12
2.66D 13
9.82D 15
8.76D 17
4.16D 17
3.58D 18
1.87D 19
3.41D 21
6.22D 22
1.04D 22
9.71D 24
8.99D 24
5.68D 23
3.52D 22

For a large class of {am } that satisfy (19.4.2), it is shown by Sidi [285] that the




k
series
k=1 ak =
k=1 ck have (generalized) Borel sums A( ) that are analytic in
the -plane cut along the real interval [0, +). This is the case, for example, when

h(m) = m 0 emt (t) dt for some integer 0 and some (t) of exponential order.
As mentioned in [285], the numerical results of Smith and Ford [318] suggest that the
L-transformation (under Process II) produces approximations to these Borel sums. The
same applies to the S-transformation, as indicated by the numerical experiments of
Weniger [353].
The numerical testing done by Grotendorst [116] suggests that, on the factorially divergent series considered here, the S-transformation is more effective than the
L-transformation, which, in turn, is more effective than the Shanks transformation
z ek, and
(equivalently, the Pade table). According to Weniger [354] and Weniger, C
Vinette [356], computations done in very high precision (up to 1000 digits) suggest
that, when applied to certain very wildly divergent asymptotic power series that arise in
the Rayleigh-Schrodinger perturbation theory, the L-transformation eventually diverges,
while the S-transformation converges.
Unfortunately, currently there is no mathematical theory that explains the observed
numerical behavior of the L- and S-transformations on factorially divergent sequences
under Process II; this is also true of other methods that can be applied to such sequences.


k
The one exception to this concerns the diagonals of the Pade table from
k=0 ck
(equivalently,
the Shanks transformation on the sequence of the partial sums) when

ck = 0 t k d(t), where the function (t) is nondecreasing on (0, ) and has an innite number of points of increase there. By Theorem 17.9.3, provided {ck } satises the

374

19 The Levin L- and Sidi S-Transformations

Carleman condition, the diagonals (analogous to Process II) converge to the corresponding Stieltjes function, which can be shown to be the Borel sum of the series in many
instances.
We close with a numerical example in which we demonstrate the performances of the
L- and S-transformations on such series under Process II.
Example 19.4.3 Let us consider the Euler series 0! 1!z 1 + 2!z 2 3!z 3 + .
Obviously, the elements  of this series satisfy (19.4.2). Its Borel sum is U (z) =

ze z E 1 (z), where E 1 (z) = z t 1 et dt is the exponential integral, and z  (, 0]. In

(0)
Table 19.4.1, we give the errors in the L(0)
for z = 1/2,1, 3. The value
n and Sn
ex
d x nuof U (z) is obtained with machine precision by computing U (z) = 0 1+x/z
merically. We have U (1/2) = 0.4614553162 , U (1) = 0.5963473623 , U (3) =
0.7862512207 . Here we have taken m = mam so the L-transformation becomes
the u-transformation.
We would like to note that, when applied to the Euler series, the iterated 2 -process,
the iterated Lubkin transformation, and the Shanks transformation discussed earlier, and
the -algorithm we discuss in Chapter 20 are able to produce fewer correct signicant
digits than the S- and L-transformations. Numerical experiments suggest that this seems
to be the case generally when these methods are applied to other factorially divergent
series.
In connection with the summation of the Euler series, we would like to refer the reader
to the classical paper by Rosser [241], where the Euler transformation is applied in a
very interesting manner.

20
The Wynn - and Brezinski -Algorithms

20.1 The Wynn -Algorithm and Generalizations

20.1.1 The Wynn -Algorithm
As we saw in Chapter 16, the Shanks transformation does not accelerate the convergence
of logarithmic sequences in b(1) /LOG. Simultaneously with his paper [368] on the
-algorithm for the Shanks transformation, Wynn published another paper [369] in which
he developed a different method and its accompanying algorithm. This method is very
effective on sequences {Am } for which
Am A +

i m i as m .

(20.1.1)

i=1

Of course, the set of all such sequences is a subset of b(1) /LOG.

The idea and motivation behind this method is the following: Since Am = h(m) A(0)
0 ,
h(m) has a smooth behavior as m . Therefore, near m = Am = h(m) can be
approximated very efciently by a rational function in m, R(m) say, with degree of
numerator equal to degree of denominator, and limm R(m) can serve as a good approximation for A = limm h(m) = limm Am . In particular, R(m) can be chosen
to interpolate h(m) at 2n + 1 points.
Denoting the rational function with degree of numerator and degree of denominator
( j)
equal to n that interpolates f (x) at the points x j , x j+1 , . . . , x j+2n by R2n (x), we can use
( j)
( j)
Thieles continued fraction to compute limx R2n (x) 2n recursively via
( j)

( j)

1 = 0, 0 = f (x j ), j = 0, 1, . . . ,
( j)

( j+1)

k+1 = k1 +

x j+k+1 x j
( j+1)

( j)

, j, k = 0, 1, . . . .

(20.1.2)

In Wynns method, the function we interpolate is h(x), and xi = i, i = 0, 1, . . . . In

( j)
other words, 2n is dened by the equations

n1
Am =

( j)

i
n
i=0 ei m + 2n m
, j m j + 2n.

n1
i
n
i=0 f i m + m

Of course, the ei and f i are the additional (auxiliary) unknowns. Thus, we have the
following computational scheme that is known as Wynns -algorithm:
375

20 The Wynn - and Brezinski -Algorithms

376

Algorithm 20.1.1 (-algorithm)

1. Set
( j)

( j)

1 = 0, 0 = A j , j = 0, 1, . . . .
( j)

2. Compute the k recursively from

( j)

k+1

( j+1)

k+1 = k1 +

( j+1)
k

( j)

, j, k = 0, 1, . . . .

( j)

The k can be arranged in a two-dimensional table that is the same as that correspond( j)
ing to the -algorithm. We mention again that only the 2n are the desired approximations
( j)
to A. Wynn has given a determinantal expression for 2n in [369]. We also note that this
method is quasi-linear, as can be seen easily. Finally, the kernel of the -algorithm is
the set of convergent sequences {Am } for which Am is a rational function of m. Specifically, if Am = P(m)/Q(m), where P(m) and Q(m) have no common factors, P(m) is
( j)
of degree at most n, and Q(m) is of degree exactly n, then 2n = limm Am for all
j = 0, 1, . . . .
It must be emphasized that the -algorithm is effective only on sequences {Am } that
are as in (20.1.1).

20.1.2 Modications of the -Algorithm

From the way the -algorithm is derived, we can conclude that it will not be effective
on sequences {Am } b(1) /LOG for which
Am A +

i m i as m ; 0 = 0,

(20.1.3)

i=0

where is not a negative integer. (Recall that = 0, 1, . . . , must always be true.) The
reason for this is that now Am = h(m) is not a smooth function of m at m = (it has
a branch singularity of the form m there), and we are trying to approximate it by a
rational function in m that is smooth at m = . Actually, the following theorem about
the behavior of the -algorithm can be proved by induction.
Theorem 20.1.2 Let Am be as in (20.1.3), where is either not an integer or = s,
s being a positive integer.
(i) When is not an integer, there holds for every n
2n A K 2n j , 2n+1 K 2n+1 j +1 as j ,
( j)

( j)

(20.1.4)

where
K 2n = 0

n
n

i +
i +1
n+1
, K 2n+1 =
.
i

i +
0
i=1
i=1

(20.1.5)

20.1 The Wynn -Algorithm and Generalizations

377

(ii) When = s, (20.1.4) holds for n = 1, 2, . . . , s 1, while for n = s, there holds

2s A = O( j s1 ) as j .
( j)

(20.1.6)

In other words, when is not an integer the -algorithm induces no convergence

acceleration, and when = s, where s is a positive integer, convergence acceleration
( j)
begins to take place with the column 2s .
To x this problem, Osada [225] took an approach that ignores the fact that the algorithm was obtained from rational interpolation. In this approach, Osada generalized
( )
Algorithm 20.1.1 to make it effective for the cases in which Am A = g(m) A0 with
arbitrary = 0, 1, . . . . The generalized -algorithm on the sequence {Am }, which we
denote the ( )-algorithm, reads as follows:
Algorithm 20.1.3 [( )-algorithm]
1. Set
( j)

( j)

1 = 0, 0 = A j , j = 0, 1, . . . .
( j)

2. Compute the k recursively from

( j)

( j+1)

k+1 = k1 +

( j+1)
k

( j)

, j, k = 0, 1, . . . .

Obviously, the (1)-algorithm is exactly the Wynn -algorithm.

The following theorem by Osada [225] shows that the ( )-algorithm accelerates
( )
convergence when Am A = g(m) A0 strictly.
( )

Theorem 20.1.4 Let Am A = g(m) A0 strictly, with = 0, 1, . . . , and apply the

( )-algorithm to {Am }. Then
( j)

2n A

wni j 2ni as j .

(20.1.7)

i=0

Of course, we need to have precise knowledge of to be able to use this algorithm.

( )
In case we know that Am A = g(m) A0 , but we do not know , Osada proposes
to apply the ()-algorithm to {Am } with chosen as a good estimate of , obtained by
applying the (2)-algorithm to the sequence {m }, where m is given by
m = 1

(am )(am+1 )
1
=1+
.
2
 (am /am )
am am+2 am+1

Here a0 = A0 and am+1 = Am , m = 0, 1, . . . . [Recall that, at the end of Section 15.3,
.]
we mentioned that m = u(m) A(2)
0

20 The Wynn - and Brezinski -Algorithms

378

Osada proposes the following algorithm, which he calls the automatic generalized
-algorithm. We call it the automatic ( )-algorithm.
Algorithm 20.1.5 [Automatic ( )-algorithm]
2
. If r =
(i) Given the terms A0 , A1 , . . . , Ar , apply the (2)-algorithm to {m }rm=0
(0)
(1)


2s + 2, set = 2s , whereas if r = 2s + 3, set = 2s .
( j)
(ii) Apply the (  )-algorithm to {Am } to obtain the approximation 2n for 0 j +
2n r .

We end this discussion by noting that the philosophy of the automatic ( )-algorithm
can be used in conjunction with the repeated generalized 2 -process of Section 15.3, as
we have already discussed.
A further modication of the -algorithm that is actually a generalization of the ( )algorithm for sequences {Am } such that
Am A +

i m i/ p as m ; p 2 integer,

i=0

0 = 0, =

i
, i = 0, 1, . . . .
p

(20.1.8)

was given by Van Tuyl [343]. We denote this algorithm the ( , p)-algorithm. It reads
as follows:
Algorithm 20.1.6 [( , p)-algorithm]
1. Set
( j)

( j)

1 = 0, 0 = A j , j = 0, 1, . . . .
( j)

2. Compute the k recursively from

( j)

( , p)

Ck

( j+1)

k+1 = k1 +

( j+1)

( j)

, j, k = 0, 1, . . . ,

where
( , p)

C2n

= +

n
n
( , p)
, C2n+1 = + + 1.
p
p

The next theorem by Van Tuyl [343] shows that the ( , p)-algorithm accelerates
( , p) strictly as in (20.1.8).
convergence when Am A = h(m) A
0
Theorem 20.1.7 With Am as in (20.1.8), apply the ( , p)-algorithm to {Am }. Then
( j)

2n A


i=0

wni j (n+i)/ p as j .

20.2 The Brezinski -Algorithm

379

20.2 The Brezinski -Algorithm

A different approach to accelerating the convergence of sequences in b(1) /LOG was
given by Brezinski [32]. In this approach, Brezinski modies the -algorithm in a clever
way we discuss next.
( j)
( j)
Changing our notation from k to k , we would like to modify the recursion relation
( j)
for k , by introducing some parameters n , to read
( j)

( j)

( j)

( j+1)

( j)

( j+1)

1 = 0, 0 = A j , j = 0, 1, . . . ,
( j)

( j)

( j)

2n+1 = 2n1 + D2n ; Dk = 1/k , for all j, k 0,

( j)

2n+2 = 2n
( j)

( j+1)

+ n D2n+1 , n, j = 0, 1, . . . .

( j)

Here Fk = Fk
Fk for all j and n. Now we would like to choose the n to
induce acceleration of convergence. (Of course, with n = 1 for all n we have nothing but
the -algorithm.) Let us apply the difference operator  to both sides of the recursion for
( j)
( j+1)
2n+2 and divide both sides of the resulting relation by 2n . We obtain
( j)

( j+1)

2n+2 /2n

( j)

( j+1)

= 1 + n D2n+1 /2n

We now require that {2n+2 }

j=0 converge more quickly than {2n } j=0 . Assuming
(1)
for the moment that both sequences are in b /LOG, this is the same as requiring
( j)
( j+1)
( j)
that lim j (2n+2 A)/(2n A) = 0. But the latter implies that lim j 2n+2 /
( j+1)
2n = 0 and, therefore,
( j)

( j)

( j+1)

n = lim 2n
j

( j)

/D2n+1 .

Now, we do not know this limit to be able to determine n . Therefore, we replace n by

( j+1)

n( j) = 2n

( j)

/D2n+1 .

As a result, we obtain the -algorithm of Brezinski [32].

Algorithm 20.2.1 ( -algorithm)
1. For j = 0, 1, . . . , set
( j)

( j)

1 = 0, 0 = A j .
( j)

2. For j, n = 0, 1, . . . , compute the k recursively from

( j)

( j+1)

( j)

( j)

( j)

2n+1 = 2n1 + D2n ; Dk = 1/k , for all j, k 0,

( j+1)

( j)
2n+2

( j+1)
2n

( j)

2n

( j)
D2n+1

( j)

D2n+1 .

(20.2.1)

Note that only the 2n are the relevant approximations to the limit or antilimit of
( j)
( j)
{Am }. For given j and n, 2n is determined from Ak , j k j + 3n, just like Bn in
the iterated Lubkin transformation.

20 The Wynn - and Brezinski -Algorithms

380

( j)

It is easy to see that the -algorithm is quasi-linear. As mentioned before, {2 } is

( j)
nothing but the sequence produced by the Lubkin transformation, that is, 2 = W j ({As })
( j)
in the notation of Section 15.4. It is also the sequence {L2 } produced by the Levin utransformation. This last observation is due to Bhowmick, Bhattacharya, and Roy [23].
The derivation of the -algorithm we gave here is only heuristic, as is the case with
all sequence transformations. We need to show that the -algorithm is a bona de convergence acceleration method.

20.2.1 Convergence and Stability of the -Algorithm

Because of the complexity of the recursion relation in (20.2.1), the analysis of the algorithm turns out to be quite involved. Specically, we have two different types of

( j)
( j)
sequences to worry about, namely, {2n } j=0 and {2n+1 } j=0 , and the two are coupled
nonlinearly. Luckily, a rigorous analysis of both types of sequences can be given, and
we turn to it now.
The rst result pertaining to this method was given by Van Tuyl [343] and concerns
the convergence of the -algorithm on sequences in b(1) /LOG. Theorems pertaining to
convergence of the method on sequences in b(1) /LIN and b(1) /FAC have been obtained
recently by Sidi in [307], where the issue of stability is also given a detailed treatment.
Our treatment of the -algorithm here follows [307].
( j)
We begin by expressing 2n in forms that are different from that given in Algorithm 20.2.1, as this enables us to present its analysis in a convenient way.
( j)

Lemma 20.2.2 2n+2 can be expressed as in

#

( j)

2n+2 =

( j)
2 2n+1

( j)

( j+1)

(s)
W j ({2n
}) 2 (1/2n ) + (2n

( j+1)

2n1 )

(20.2.2)

and
( j+1)

( j)

2n+2 =

(2n

( j)

2n+1 )
( j)

2 2n+1

(20.2.3)

Proof. From Algorithm 20.2.1, we rst have

( j+1)

( j)

2n+2 =
( j+1)

2n

( j)

( j+1)

2 2n+1 + 2n

( j+1)

2n+1

( j)

2 2n+1

( j+2)

( j+1)

( j)

( j+1)

(20.2.4)
( j)

Substituting 2n = 2n 2n and 2 2n+1 = 2n+1 2n+1 in the numer( j)

( j+1)
( j)
ator of (20.2.4), we obtain (20.2.3). Next, substituting 2n+1 = 2n1 + 1/2n in the
numerator of (20.2.3), we have
( j+1)

( j)

2n+2 =

(2n

( j)

( j+1)

(1/2n )) + (2n
( j)

( j)

( j)

2 2n+1
( j)

( j+1)

2n1 )

(20.2.5)

Letting now n = 0 and recalling that 1 = 0, 0 = A j , and hence 1 = 1/A j , we

20.2 The Brezinski -Algorithm

381

have from (20.2.5) and (15.4.2) that

( j)

2 =

(A j+1 (1/A j ))

= W j ({As }).
2 (1/A j )

(20.2.6)

Using this in (20.2.5), the result in (20.2.2) follows.

The next theorem explains the convergence of the sequences {2n }
j=0 .
( j)

Theorem 20.2.3 Let us assume that the sequence {Am } is as in Denition 15.3.2 with
exactly the same notation.
(i) If {Am } b(1) /LOG, then there exist constants k such that 0 = and k k1
are integers 2, for which, as j ,
( j)

2n A

wni j n i , wn0 = 0,

i=0
( j)

2n+1

h ni j n i+1 , h n0 = 1/(n wn0 ) = 0.

i=0

(ii) If {Am } b(1) /LIN, then there exist constants k such that 0 = and k k1
are integers 3, for which, as j ,
( j)

2n A j

wni j n i , wn0 = 0,

i=0

2n+1 j
( j)

h ni j n i , h n0 = 1/[( 1)wn0 ] = 0.

i=0

(iii) If {Am } b(1) /FAC, then, as j ,

( j)

2n A

j 
n
wni j n i , wn0 = 0 [ 3r (r + 1)] = 0,
( j!)r i=0

2n+1 j ( j!)r
( j)

h ni j n i , h n0 = 1/wn0 = 0,

i=0

where k = k(3r + 2), k = 0, 1, . . . .

The proof of this theorem can be carried out by induction on n. For this, we use the
error expression
( j)

2n+2 A =

1
( j)

2 2n+1

( j)

( j+1)

(s)
[W j ({2n
}) A]2 (1/2n ) + [(2n
( j)

$
( j+1)
A)(2n1 )] ,

which follows from (20.2.2), and the denition of 2n+1 given in Algorithm 20.2.1, and
Theorem 15.4.1 on the convergence of the Lubkin transformation. We leave the details
to the reader.

20 The Wynn - and Brezinski -Algorithms

382

Note the similarity of Theorem 20.2.3 on the -algorithm to Theorem 15.4.2 on

the iterated Lubkin transformation. It must be noted that the k in parts (i) and (ii) of
these theorems are not necessarily the same. However, in parts (iii) they are the same,
( j)
( j)
and we have lim j (2n A)/(Bn A) = 1. Finally, we see that the -algorithm
accelerates the convergence of all three types of sequences in Theorem 20.2.3.
( j)
We now turn to the stability of the -algorithm. We rst note that, by (20.2.3), 2n+2
can be expressed as in
( j)

( j+1)

2n+2 = (nj) 2n

( j+2)

+ (nj) 2n

( j)

(nj) =

( j+1)

2n+1
( j)

2 2n+1

and (nj) =

2n+1
( j)

2 2n+1

(20.2.7)

( j)

( j)

( j)

Starting from (20.2.7), by 0 = A j , j = 0, 1, . . . , and by the fact that n + n = 1,

we can easily see by induction on n that
( j)

2n =

n


n


( j)

ni A j+n+i ;

i=0

( j)

ni = 1,

(20.2.8)

i=0

( j)

and that the ni satisfy the recursion relation

( j)

( j+1)

n+1,i = (nj) ni
( j)

( j+2)

+ (nj) n,i1 , i = 0, 1, . . . , n + 1,

(20.2.9)

( j)

( j)

where we have dened n,1 = n,n+1 = 0. Next, let us dene the polynomials Pn (z) =

n
( j) i
( j)
i=0 ni z . It is easy to see from (20.2.9) that the Pn (z) satisfy the recursion
( j)

Pn+1 (z) = (nj) Pn( j+1) (z) + (nj) z Pn( j+2) (z).

(20.2.10)

Then we have the following theorem on the stability of the -algorithm.

Theorem 20.2.4 Let us assume that the sequence {Am } is exactly as in Theorem 20.2.3.
(i) If {Am } b(1) /LOG, then
Pn( j) (z)

n1


1
k

(1 z) j

n n

and

k=0

n( j)

1
n1
 

 k  (2 j)n as j .
k=0

(20.2.11)
(ii) If {Am } b(1) /LIN, then

lim

Pn( j) (z)

z
1

n
and lim

n( j)

1 + | |
|1 |

n
.

(20.2.12)

( j)

(Of course, when = 1, n 1 as j .)

(iii) If {Am } b(1) /FAC, then
lim Pn( j) (z) = z n and lim n( j) = 1.

(20.2.13)

20.2 The Brezinski -Algorithm

383
( j)

The proof of this theorem can be achieved by rst noting that (i) n j/n and
( j)
( j)
j/n as j , n 0, in part (i), (ii) n /( 1) and n 1/( 1)
( j)
( j)
as j , n 0, in part (ii), and (iii) n = o(1) and n 1 as j , n 0, in
part (iii). All these follow from Theorem 20.2.3. Next, we combine these with (20.2.9)
and (20.2.10) and use induction on n. We leave the details to the reader.
Again, note the similarity of Theorem 20.2.4 on the stability of the -algorithm with
Theorem 15.5.2 on the stability of the iterated Lubkin transformation.
In view of the results of Theorem 20.2.4, we conclude that the -algorithm is stable
on linear and factorial sequences but not on logarithmic sequences. The remarks we
made in Section 15.6 about the effective use of the iterated Lubkin transformation in
nite-precision arithmetic are valid for the -algorithm without any modications. In
particular, good accuracy can be achieved on logarithmic sequences by increasing the
precision of the arithmetic used. Good accuracy is possible on logarithmic sequences
( j)
when |! | is sufciently large, even though sup j n = . As for linear sequences, in
( j)
( j)
case is very close to 1, n is large, even though sup j n < . In this case, it is best
to use arithmetic progression sampling (APS), that is, we should apply the -algorithm
to a subsequence {Am+ }, where and are xed integers with 2. For more details,
we refer the reader to Section 15.6.
( j)
n

20.2.2 A Further Convergence Result

We have shown that the -algorithm accelerates the convergence of sequences {Am } for
which {Am } b(1) . In case {Am } b(m) with m > 1, however, the -algorithm is
ineffective in general. It is effective in case Am satises (20.1.8). For this application,
Van Tuyl [343] provides the following convergence acceleration result.
Theorem 20.2.5 With Am as in (20.1.8), apply the -algorithm to {Am }. Then there exist
scalars k such that 0 = and (k k1 ) p is an integer 1, k = 1, 2, . . . , for
which


( j)
2n A
wni j n i/ p as j , wn0 = 0,
i=0
( j)

2n+1

h ni j n i/ p+1 as j , h n0 = 1/(n wn0 ) = 0.

i=0

The proof of this theorem can be carried out as that of Theorem 20.2.3, and it is left
to the reader.

21
The G-Transformation and Its Generalizations

21.1 The G-Transformation

The G-transformation was designed by Gray and Atchison [110] for the purpose of

evaluating innite integrals of the form a f (t) dt. It was later generalized in different
ways in [12] and [111], the ultimate generalization being given by Gray, Atchison,
and McWilliams [112]. The way it is dened by Gray and Atchison [110], the Gtransformation produces an approximation to I [ f ] that is of the form
G(x; h) =
where


F(x) =

F(x + h) R(x, h)F(x)

,
1 R(x, h)

f (t) dt and R(x, h) = f (x + h)/ f (x).

(21.1.1)

(21.1.2)

It is easy to see that G(x; h) can also be expressed in the form



 F(x) F(x + h) 


 f (x) f (x + h) 
G(x; h) = 
,
 1

1


 f (x) f (x + h) 

(21.1.3)

and hence is the solution of the linear system

F(x) = G(x; h) + f (x)
F(x + h) = G(x; h) + f (x + h)

(21.1.4)

Thus, the G-transformation is simply the D -transformation with 0 = 0 in Denition 5.2.1. By Theorem 5.7.3, we see that it will be effective on functions f (x) that vary
exponentially as x , and that it will be ineffective on those f (x) that vary like some
power of x at innity. This is the subject of the following theorem.
(1)

Theorem 21.1.1
(i) If the function f (x) is of the form f (x) = ecx H (x) with c = 0 and c 0 and
H A( ) for arbitrary , then, with h xed such that ech = 1, there holds
G(x; h) I [ f ]
= O(x 2 ) as x .
F(x) I [ f ]
384

21.2 The Higher-Order G-Transformation

385

(ii) If the function f (x) is of the form f (x) = H (x) with H A( ) for some =
1, 0, 1, 2, . . . , then
G(x; h) I [ f ]
= O(1) as x .
F(x) I [ f ]
Proof. We start with the error formula


 F(x) I [ f ] F(x + h) I [ f ] 




f (x)
f (x + h)
G(x; h) I [ f ] =
,


 1

1


 f (x) f (x + h) 
From Theorem 5.7.3, we then have


 x f (x)g(x) (x + h) f (x + h)g(x + h) 




f (x)
f (x + h)
,
G(x; h) I [ f ] =



 1
1


 f (x) f (x + h) 
with = 0 in part (i) and = 1 in part (ii) and g(x) A(0) . The results follow from a
simple analysis of the right-hand side of this equality as x .


Let us dene the kernel of the G-transformation to be the collection of all functions
f (x) C [a, ) such that G(x; h) = I [ f ] for
 all x and h. It is easy to show that f (x)
is in this kernel if f (x) = ecx with c = 0 and a f (t) dt is dened in the sense of Abel
summability. (This implies that c 0 must hold.)

21.2 The Higher-Order G-Transformation

The higher-order G-transformation of Gray, Atchison, and McWilliams [112] is obtained by generalizing the determinantal representation of the G-transformation as
follows:




F(x)
F(x + h)
F(x + nh)




f (x)
f (x + h)
f (x + nh)




.
.
.
..
..
..




 f (x + (n 1)h) f (x + nh) f (x + (2n 1)h) 
G n (x; h) = 
 . (21.2.1)


1
1

1




f (x)
f (x + h)
f (x + nh)




..
..
..


.
.
.


 f (x + (n 1)h) f (x + nh) f (x + (2n 1)h) 

386

21 The G-Transformation and Its Generalizations

As such, G n (x; h) is also the solution of the linear system

F(x + i h) = G n (x; h) +

n


k f (x + (i + k 1)h), i = 0, 1, . . . , n, (21.2.2)

k=1

where k are additional unknowns. This fact was noted and exploited in the computation
of G n (x; h) by Levin and Sidi [165].
Let us dene the kernel of the higher-order G-transformation to be the collection of
all functions f (x) C [a, ) such that G n (x; h) = I [ f ] for all x and h and for some
appropriate n. It has been shown in [112] that f (x) is in this kernel if it is integrable
at innity in the sense of Abel and satises a linear homogeneous ordinary differential
equation with constant coefcients. Thus, f (x) is in this kernel if it is of the form f (x) =

r
ck x
, where ck = 0 are distinct and ck 0, and Pk (x) are polynomials. If
k=1 Pk (x)e


pk is the degree of Pk (x) for each k, and if rk=1 ( pk + 1) = n, then G n (x; h) = I [ f ]
for all x and h. On the basis of this result, Levin and Sidi [165] concluded that the


higher-order G-transformation is effective on functions of the form sk=1 eck x h k (x),
where h k (x) A(k ) with arbitrary k .
We end this section by mentioning that the papers of Gray, Atchison, and McWilliams
[112] and Levin [161] have been an important source of inspiration for the D- and
d-transformations.

21.3 Algorithms for the Higher-Order G-Transformation

The rst effective algorithm for implementing the higher-order G-transformation was
given by Pye and Atchison [233]. Actually, these authors consider the more general
( j)
problem in which one would like to compute the quantities An dened via the linear
equations
Al = A(nj) +

n


k u k+l1 , l = j, j + 1, . . . , j + n,

(21.3.1)

k=1

where the Ai and u i are known scalars, and the k are not necessarily known. Before
proceeding further, we note that these equations are the same as those in (3.7.1) with
gk (l) = u k+l1 , l = 0, 1, . . . . This suggests that the E- and FS-algorithms of Chapter 3
( j)
can be used for computing the An . Of course, direct application of these algorithms
without taking into account the special nature of the gk (l) is very uneconomical. By
( j)
taking the nature of the gk (l) into consideration, fast algorithms for the An can be
derived. Here we consider two such algorithms: (i) that of Pye and Atchison [233] that
has been denoted the rs-algorithm and (ii) the FS/qd-algorithm of the author that is new.

21.3.1 The rs-Algorithm

( j)

( j)

The rs-algorithm computes the An with the help of two sets of auxiliary quantities, rn
( j)
and sn . These quantities are dened by
( j)

rn( j) =

( j)

Hn

K n+1

Kn

Hn

, sn( j) =
( j)

( j)

(21.3.2)

21.3 Algorithms for the Higher-Order G-Transformation

( j)

( j)

where Hn , the Hankel determinant Hn

as in

 uj

 u j+1

( j)
Hn =  .
 ..

u

j+n1

and

K n( j)

( j)

({u s }) associated with {u s }, and K n are given


u j+1 u j+n1 
u j+2 u j+n 
,
..
..

.
.

u j+n u j+2n2 


 1
1

 uj
u
j+1


=  u j+1 u j+2
 .
..
 ..
.

u
j+n2 u j+n1
( j)

387

( j)

1
u j+n1
u j+n
..
.
u j+2n3







.





(21.3.3)

(21.3.4)

( j)

The rs-algorithm computes the An , rn , and sn simultaneously by efcient recursions

as follows:
Algorithm 21.3.1 (rs-algorithm)
1. For j = 0, 1, . . . , set
( j)

( j)

( j)

s0 = 1, r1 = u j , A0 = A j , j = 0, 1, . . . .
2. For j = 0, 1, . . . , and n = 1, 2, . . . , compute recursively
 ( j+1)
 ( j+1)


( j+1) r n
( j)
( j)
( j+1) sn
sn = sn1
1 , rn+1 = rn
1 .
( j)
( j)
rn
sn
3. For j = 0, 1, . . . , and n = 1, 2, . . . , set
( j)

A(nj)
( j)

( j+1)

( j+1)

rn An1 rn
( j)

( j)

An1

( j+1)

rn rn

( j)

We now realize that rn = (1)n1 n1 (gn ) in the notation of the E-algorithm of

( j)
Chapter 3. Thus, the rs-algorithm is simply the E-algorithm in which the n1 (gn ),
whose determination forms the most expensive part of the E-algorithm, are computed by a fast recursion. For this point and others, see Brezinski and Redivo Zaglia
[41, Section 2.4].

21.3.2 The FS/qd-Algorithm

In view of the close connection between the rs- and E-algorithms, it is natural to investigate the possibility of designing another algorithm that is related to the FS-algorithm.
This is worth the effort, as the FS-algorithm is more economical than the E-algorithm to
begin with. We recall that the most expensive part of the FS-algorithm is the determina( j)
tion of the Dn , and we would like to reduce its cost. Fortunately, this can be achieved

388

21 The G-Transformation and Its Generalizations

( j)

( j)

( j)

once we realize that, with G n dened as in (3.3.4), we have G n = Hn in the present

( j)
case, where Hn is as dened in (21.3.3). From this, (3.3.9), and (17.6.20), we obtain
the surprising result that
Dn( j) =

( j)

( j+1)

( j)

( j+1)

Hn+1 Hn1

= en( j) .

Hn Hn

(21.3.5)

( j)

Here, en is a quantity computed by the qd-algorithm (Algorithm 17.6.2) of Chapter 17,

( j)
along with the quantities qn given as in
qn( j) =

( j)

( j+1)

( j)

( j+1)

Hn1 Hn

Hn Hn1

(21.3.6)

This observation enables us to combine the FS- and qd-algorithms to obtain the
following economical implementation, the FS/qd-algorithm, for the higher-order Gtransformation. This has been done recently by Sidi [303]. For simplicity of notation,
( j)
( j)
( j)
( j)
we let n (a) = Mn and n (I ) = Nn in the FS-algorithm, as we did with the Walgorithm.
Algorithm 21.3.2 (FS/qd-algorithm)
1. For j = 0, 1, . . . , set
( j)

( j)

e0 = 0, q1 =

u j+1
Aj
1
( j)
( j)
, M0 =
, N0 = .
uj
uj
uj

2. For j = 0, 1, . . . , and n = 1, 2, . . . , compute recursively

( j+1)

( j+1)

( j)

en( j) = qn( j+1) qn( j) + en1 , qn+1 =

( j+1)

Mn( j) =

( j)

Mn1 Mn1
( j)

en

en

( j)

en

( j+1)

, Nn( j) =

qn( j+1) ,
( j)

Nn1 Nn1
( j)

en

3. For j, n = 0, 1, . . . , set
( j)

A(nj) =
( j)

Mn

( j)

Nn

( j)

We recall that the en and qn are ordered as in Table 17.6.3 (the qd-table) and this
( j)
( j)
( j)
( j)
table can be computed columnwise in the order {e1 }, {q2 }, {e2 }, {q3 }, . . . .
( j)
When A0 , A1 , . . . , A L , and u 0 , u 1 , . . . , u 2L1 are given, we can determine An for
(0)
0 j + n L. Algorithm 21.3.2 will compute all these except A L , because A(0)
L =
,
which
in
turn
requires
u
.
To
avoid
this,
we
M L(0) /N L(0) , and M L(0) and N L(0) require e(0)
2L
L
(0)
act exactly as in (3.3.13). That is, we do not compute e L , but we compute all the A(0)
n
(including A(0)
)
by
L
A(0)
n =

(1)
(0)
Mn1
Mn1
(1)
(0)
Nn1
Nn1

(21.3.7)

21.3 Algorithms for the Higher-Order G-Transformation

389

Table 21.3.1: Operation counts of the rs- and FS/qd-algorithms

Algorithm
FS/qd
rs

No. of Multiplications

No. of Additions

No. of Divisions

L 2 + O(L)
3L 2 + O(L)

3L 2 + O(L)
3L 2 + O(L)

5L 2 /2 + O(L)
5L 2 /2 + O(L)

21.3.3 Operation Counts of the rs- and FS/qd-Algorithms

Let us now compare the operation counts of the two algorithms. First, we note that the
( j)
( j)
( j)
( j)
rn and sn can be arranged in a table similar to the qd-table of the en and qn . Thus,
( j)
given A0 , A1 , . . . , A L , and u 0 , u 1 , . . . , u 2L1 , we can compute An for 0 j + n L.
( j)
Now, the number of the en in the relevant qd-table is L 2 + O(L) and so is that of the
( j)
( j)
( j)
( j)
qn . A similar statement can be made about the rn and sn . The number of the An is
( j)
( j)
2
L /2 + O(L), and so are the numbers of the Mn and the Nn . Consequently, we have
the operation counts given in Table 21.3.1.
In case only the A(0)
n are needed (as they have the best convergence properties), the
number of divisions in the FS/qd-algorithm can be reduced from 5L 2 /2 + O(L) to
2L 2 + O(L). In any case, we see that the operation count of the rs-algorithm is about
30% more than that of the FS/qd-algorithm.
Finally, we observe that, with the substitution u k = Ak , the higher-order Gtransformation reduces to the Shanks transformation, which was discussed in Chapter 16.
( j)
Actually, An = en (A j ) by Denition 16.1.1 in this case. Thus, the rs- and FS/qdalgorithms can be used for computing the en (A j ). Of course, the best-known algorithm
for implementing the Shanks transformation is the -algorithm, which was given again
in Chapter 16 as Algorithm 16.2.1. Given A0 , A1 , . . . , A2L , the -algorithm computes
( j)
(0)
for 0 n L, at a cost
2n for 0 j + 2n 2L, and the diagonal approximants 2n
2
2
of 4L + O(L) additions, 2L + O(L) divisions, and no multiplications. Comparing
this with the counts in Table 21.3.1, we see that the FS/qd-algorithm, when applied
(0)
for 0 n L, at a cost of 3L 2 + O(L) additions,
with u k = Ak , computes the 2n
2
2
L + O(L) multiplications, and 2L + O(L) divisions; thus it compares very favorably
with the -algorithm.

22
The Transformations of Overholt and Wimp

22.1 The Transformation of Overholt

In Chapter 15, we gave a detailed discussion of the Aitken 2 -process. There we saw
that one of the uses of this method is in accelerating the convergence of xed-point
iterative procedures for the solution of nonlinear equations. In this chapter, we come
back to this problem again and discuss two extensions of the 2 -process within the same
context.
Recall that, in the iterative solution of a nonlinear equation x = (x), we begin with an
arbitrary approximation x0 to the solution s and generate the sequence of approximations
{xm } via xm+1 = (xm ). It is known that, provided 0 < |  (s)| < 1 and x0 is sufciently
close to s, the sequence {xm } converges to s linearly in the sense that limm (xm+1 s)/
(xm s) =  (s), and that, provided (x) is innitely differentiable in a neighborhood
of s, xm has an asymptotic expansion of the form
xm s +

k km as m , 1 = 0, =  (s),

(22.1.1)

k=1

for some k that depend only on and x0 . Now, if were known, we could use the
Richardson extrapolation process for innite sequences (see Section 1.9) and approxi( j)
mate s via the z n that are given by
( j)

z 0 = x j , j = 0, 1, . . . ,
( j+1)

z n( j) =

( j)

z n1 n z n1
, j = 0, 1, . . . , n = 1, 2, . . . .
1 n

(22.1.2)

Since we do not know , let us replace it by some suitable approximation. Now can
be approximated in terms of the xm in different ways. By (22.1.1), we have that
xm

k (k 1)km as m .

(22.1.3)

k=1

From this and from the fact that 1 = 0, we see that limm xm+1 /xm = . On
the basis of this, we choose to approximate n in (22.1.2) by (x j+n /x j+n1 )n . This

390

22.2 The Transformation of Wimp

391

results in the method of Overholt [227], the rst extension of the 2 -process:
( j)

x0 = x j , j = 0, 1, . . . ,
( j+1)

xn( j) =

( j)

(x j+n1 )n xn1 (x j+n )n xn1

, j = 0, 1, . . . , n = 1, 2, . . . . (22.1.4)
(x j+n1 )n (x j+n )n

A variant of Overholts method was given by Meinardus [210]. This time, n in

(22.1.2) is approximated by (x j+1 /x j )n . This results in the following method:
( j)

x0 = x j , j = 0, 1, . . . ,
( j+1)

xn( j) =

( j)

(x j )n xn1 (x j+1 )n xn1

, j = 0, 1, . . . , n = 1, 2, . . . . (22.1.5)
(x j )n (x j+1 )n

Note that both methods are quasi-linear. Obviously, they can be applied to arbitrary
( j)
sequences {Am }, provided we replace the xm by the corresponding Am and the xn
( j)
by An .
( j)
Finally, if we let n = 1 in (22.1.4) and (22.1.5), we obtain x1 = j ({xs }) with j ({xs })
( j)
as dened in (15.3.1). That is, {x1 } is the sequence generated by the 2 -process
on {xm }.

22.2 The Transformation of Wimp

Let (x), s, {xm }, and be as in the preceding section. Starting with (22.1.3), let us


cki (k+i)m as m ,
note that, by the fact that || < 1, there holds (xm )k i=0
k
k
km
ck0 = [1 ( 1)] = 0, so that (xm ) ck0 as m . From this, we conclude
that (i) {(xm )k }
k=1 is an asymptotic scale as m , and (ii) we can reexpress (22.1.1)
in the form
xm s +

k (xm )k as m , 1 = 1/( 1).

(22.2.1)

k=1

Here k is uniquely determined by 1 , . . . , k for each k. On the basis of (22.2.1), it is

easy to see that we can apply the polynomial Richardson extrapolation to the sequence
{xm }. This results in the following scheme that was rst presented by Wimp in [361] and
rediscovered later by Germain-Bonne [96]:
( j)

x0 = x j , j = 0, 1, . . . ,
( j+1)

xn( j)

( j)

(x j )xn1 (x j+n )xn1

=
, j = 0, 1, . . . , n = 1, 2, . . . . (22.2.2)
x j x j+n

Note that Wimps method can also be obtained by approximating n in (22.1.2) by

x j+n /x j .
In addition to these developments, Wimps method also has its basis in the solution
of the nonlinear equation f (x) = (x) x = 0 by inverse interpolation. This is also

392

22 The Transformations of Overholt and Wimp

discussed in the book by Wimp [366, pp. 7375, pp. 105112]. For the subject of inverse
interpolation, see, for example, Ralston and Rabinowitz [235]. Assuming that the inverse
function x = h(y) f 1 (y) exists in a neighborhood of y = 0, we have that h(0) = s.
Let us denote by qn, j (y) the polynomial of interpolation of degree at most n to h(y) at the
points y j , y j+1 , . . . , y j+n in this neighborhood. From the NevilleAitken interpolation
formula, we thus have
qn, j (y) =

(y y j )qn1, j+1 (y) (y y j+n )qn1, j (y)

.
y j+n y j
( j)

Letting y = 0 in this formula and dening xn = qn, j (0), we obtain

( j+1)

xn( j) =

( j)

y j xn1 y j+n xn1

.
y j y j+n

(22.2.3)

Now, if the yi are sufciently close to 0, we will have qn, j (0) h(0) = s. In other words,
( j)
the xn will be approximations to s. What remains is to choose the yi appropriately.
With the sequence {xm } generated by xi+1 = (xi ), i = 0, 1, . . . , let us take yi = f (xi ),
i = 0, 1, . . . . Thus, yi = xi+1 xi = xi for each i. [Furthermore, because {xm } converges to s when x0 is sufciently close to s, we have that limi yi = 0.] Combining
this with (22.2.3), we obtain the method given in (22.2.2).
Note that, just like the methods of Overholt and Meinardus, the method of Wimp is also
quasi-linear. It can also be applied to arbitrary sequences {Am } with proper substitutions
in (22.2.2).
In addition, the kernel of the method of Wimp is the set of all sequences {Am } for
which
Am = A +

n


k (Am )k , m = 0, 1, . . . .

k=1
( j)

Finally, if we let n = 1 in (22.2.2), we obtain x1 = j ({xs }) with j ({xs }) as dened

( j)
in (15.3.1). That is, {x1 } is the sequence generated by the 2 -process on {xm }.

22.3 Convergence and Stability

Because of the way they are derived, we expect the methods we discussed in the preceding
two sections to be effective on sequences {Am } that satisfy
Am A +

k km as m ; 1 = 0, || < 1.

(22.3.1)

k=1

Therefore, we are going to investigate the convergence and stability of these methods on
such sequences.

22.3.1 Analysis of Overholts Method

We start with the following lemma.

22.3 Convergence and Stability

393

Lemma 22.3.1 Let r be a positive integer, a complex scalar, || < 1, and let {Bm }
and {Cm } be such that
Bm

bi (r +i)m as m ,

i=0

Cm

ci (r +i)m as m ; c0 = 0.

i=0

Then


Bm Cm+1 Bm+1 Cm
B m
b i (r +i+1)m as m ,

Cm+1 Cm
i=0

(22.3.2)

where
b0 c1 b1 c0 r ( 1)
.
b 0 =
c0
r 1
In case b0 c1 b1 c0 = 0, we have b 0 = 0 and hence B m b 0 (r +1)m as m . In case
bi = 0, 0 i p 1, b p = 0, for some integer p > 0, we have b i = 0, 0 i p 1,
b p = 0 and hence B m b p (r + p)m as m . In any case, we have B m = O((r +1)m )
as m .
Proof. We rst observe that

B m =

r +i



 Bm Bm+1 


Cm Cm+1 
Cm

Xm
.
Ym

(r +i)m

Now, Ym i=0 ci ( 1)
as m , and because c0 = 0 and || < 1, we
r
rm
also have that Ym c0 ( 1) as m . Next, applying Lemma 16.4.1 to the
determinant X m , we obtain




bi b j  ( j i )r (2r +i+ j)m as m .
Xm
ci c j 
0i< j
The result in (22.3.2) follows by combining the two asymptotic expansions. We leave
the rest of the proof to the reader.


Using Lemma 22.3.1 and induction on n, we can prove the following theorem on
( j)
the convergence of the column sequences {An }
j=0 in the methods of Overholt and of
Meinardus.
( j)

Theorem 22.3.2 Let the sequence {Am } satisfy (22.3.1). Let An be as in (22.1.4) or
(22.1.5). Then
A(nj) A

ani (n+i+1) j as j .

i=0
( j)

Thus, An A = O((n+1) j ) as j .

(22.3.3)

394

22 The Transformations of Overholt and Wimp

As for stability of column sequences, we proceed exactly as in Section 15.5 on the

iterated 2 -process. We rst write
( j)

An+1 = (nj) A(nj) + (nj) A(nj+1) ,

(22.3.4)

where
( j)

(nj) =

( j)
n

and (nj) =

1
( j)

1 n

(22.3.5)

( j)

( j)

where n = (A j+n+1 /A j+n )n+1 for Overholts method and n = (A j+1 /
A j )n+1 for Meinarduss method. Consequently, we can write
A(nj) =

n


( j)

ni A j+i ;

i=0

n


( j)

ni = 1,

(22.3.6)

i=0

( j)

where the ni satisfy the recursion relation

( j)

( j)

( j+1)

n+1,i = (nj) ni + (nj) n,i1 , i = 0, 1, . . . , n + 1.

(22.3.7)

( j)

Here, we dene ni = 0 for i < 0 and i > n. As we did before, let us dene
Pn( j) (z) =

n


( j)

ni z i and n( j) =

n


i=0

( j)

|ni |.

(22.3.8)

i=0

( j)

From the fact that lim j n = n+1 , we reach the following result on stability of
column sequences.
( j)

( j)

( j)

Theorem 22.3.3 Let {Am } and An be as in Theorem 22.3.2, and let Pn (z) and n be
as in (22.3.8). Then
lim Pn( j) (z) =

n
n


z i

ni z i and
1 i
i=1
i=0

lim n( j) (z) =

n


|ni |

i=0

n

1 + ||i
i=1

|1 i |

(22.3.9)

Therefore, the column sequences are stable. Equality holds in (22.3.9) when is real
positive.

22.3.2 Analysis of Wimps Method

( j)

It is easy to show that the An from the method of Wimp satisfy Theorems 22.3.2 and
( j)
( j)
22.3.3. In these theorems, we now have n = A j+n+1 /A j and lim j n = n+1
again.
These results can be obtained by observing that the method of Wimp is also what
we denoted in Chapter 3, a rst generalization of the Richardson extrapolation process,
for which we now have gk (l) = (Al )k , l = 0, 1, . . . , and k = 1, 2, . . . . We also have
lim j gk (l + 1)/gk (l) = k for all k 1. Thus, Theorems 3.5.33.5.6 of Chapter 3

22.3 Convergence and Stability

395

k
concerning the column sequences {An }
j=0 all hold with ck = , k = 1, 2, . . . , there.
Concerning the convergence of column sequences, we have the result


n
n+r i
(A j )n+r as j ,
A(nj) A n+r
i
1

i=1
( j)

= O((n+r ) j ) as j ,

(22.3.10)

where r is that integer for which n+r is the rst nonzero k with k n + 1 in the
asymptotic expansion
Am A +

k (Am )k as m , 1 = 1 /( 1).

(22.3.11)

k=1

We recall that this asymptotic expansion follows from (22.3.1).

( j)
In this case, we can also give a thorough analysis of diagonal sequences {An }
n=0
since the yl = Al satisfy liml (yl+1 /yl ) = . These results can be obtained from
Chapter 8. For example, in case is real positive, for any > 0 such that + < 1,
there exists an integer J , such that yl+1 /yl + < 1 for all l J . Then, with j J ,
( j)
{An }
n=0 is stable and converges to A, by Theorems 8.6.1 and 8.6.4.
We can also give a different convergence theory for Wimps method on the diagonal
( j)

sequences {xn }
n=0 obtained from the xed-point iteration sequence {x m }n=0 that is
based on the error formula
n

xn( j) s = (1)n h[0, y j , y j+1 , . . . , y j+n ]
y j+i ; yl = xl , l = 0, 1, . . . ,
i=0

derived in Chapter 2. In case max yI |h (n) (y)| = O(e n ) as n for some > 0 and
2
( j)
< 2, where I is some interval containing all of the yl , we have that xn s = O(en )
as n for some > 0.
In [366, p. 110], the transformations of Wimp and of Overholt are applied to two
sequences {xm } from two different xed-point iteration functions for some polynomial
equation f (x) = 0. One of these sequences converges to the solution s of f (x) = 0, while
the other diverges and has two limit points different from s. Numerical results indicate
that both methods perform equally well on the convergent sequence, in the sense that
both the columns and diagonals of their corresponding tables converge. The method of
Overholt diverges (or it is unstable at best) on the divergent sequence along columns
and/or diagonals, whereas the method of Wimp appears to converge along diagonals to
the solution s of the equation, although it too suffers from instability ultimately.
For comparison purposes, we have also applied the Shanks transformation to the
same sequences. It appears that the Shanks transformation performs similarly to the
methods of Wimp and of Overholt but is inferior to them. In connection with the Shanks
transformation on {xm } as in (22.1.1), we recall that column sequences converge. In fact,
( j)
there holds 2n s = O((n+1) j ) as j , just as is the case for the methods of this
chapter.

23
Conuent Transformations

23.1 Conuent Forms of Extrapolation Processes

23.1.1 Derivation of Conuent Forms
In Chapters 14, we were concerned with extrapolation methods on functions A(y) that
were assumed to be known (or computable) for y = 0. All these processes made use of
A(yl ) for a decreasing sequence {yl }. Of course, these yl are distinct. In this chapter, we
are concerned with what happens to the extrapolation processes when the functions A(y)
are differentiable as many times as needed and the yl coalesce. To understand the subject
in a simple way, we choose to study it through the rst generalization of the Richardson
extrapolation process of Chapter 3.
Recall that we assume

A(y) A +

k k (y) as y 0+,

(23.1.1)

k=1

where A and the k are some scalars independent of y and {k (y)} is an asymptotic
sequence as y 0+; that is, it satises
k+1 (y) = o(k (y)) as y 0+, k = 1, 2, . . . .

(23.1.2)

Here we are interested in A, the limit or antilimit of A(y) as y 0+. Recall also that
( j)
An , the approximation to A, is dened via the linear system

A(yl ) = A(nj) +

n


k k (yl ), j l j + n,

(23.1.3)

k=1

1 , . . . , n being the additional (auxiliary) unknowns, with y0 > y1 > , and

( j)
liml yl = 0. Using Cramers rule, we can express An in the determinantal form
given in (3.2.1). We reproduce (3.2.1) here explicitly in terms of the functions

396

23.1 Conuent Forms of Extrapolation Processes

397

k (y):

A(nj)



 1 (y j ) n (y j ) A(y j ) 


 1 (y j+1 ) n (y j+1 ) A(y j+1 ) 




..
..
..


.
.
.


 (y ) (y ) A(y ) 
1 j+n
n j+n
j+n
=
 .

 1 (y j ) n (y j ) 1 


 1 (y j+1 ) n (y j+1 ) 1 



..
..
.. 

.
.
.

 (y ) (y ) 1 
1

j+n

(23.1.4)

j+n

By performing elementary row transformations on both determinants in (23.1.4), we

obtain


 1 (y j )
n (y j )
A(y j ) 


 ( j)
( j)
( j)
 D1 {1 (y)} D1 {n (y)} D1 {A(y)} 


..
..
..




.
.
.

 ( j)
(
j)
(
j)
 Dn {1 (y)} Dn {n (y)} Dn {A(y)} 
( j)


An =
,
(23.1.5)
 1 (y j )
n (y j ) 1 


 ( j)
( j)
 D1 {1 (y)} D1 {n (y)} 0 

..
.. 
..


.
.
.

 ( j)
 Dn {1 (y)} Dn( j) {n (y)} 0 
where Dk(s) {g(y)} stands for g[ys , ys+1 , . . . , ys+k ], the divided difference of g(y) over the
set of points {ys , ys+1 , . . . , ys+k }. Let us now assume that A(y) and the k (y) are all differentiable functions of y, and let all yl tend to the same value, say y , simultaneously. Then
Dk(s) {A(y)} A(k) ( y )/k!, provided A(y) C k (I ), where I is an interval containing y ;
similarly, for the k (y). Consequently, the limits of the determinants in the numerator
( j)
and the denominator of (23.1.5), and hence the limit of An , all exist. We summarize
this discussion in the following theorem, where we also replace y by y for simplicity.
( j)

Theorem 23.1.1 Let An be dened via (23.1.3). If A(y) and the k (y) are n times
( j)
differentiable in y for y in some right neighborhood of 0, then the limit of An as yl y,
j l j + n, exists. Denoting this limit by Q n (y), we have Q n (y) = !n [A(y)], where


 f (y) 1 (y) n (y) 
 

 f (y) 1 (y) n (y) 


 ..
..
.. 
 .
.
. 

 f (n) (y) (n) (y) (n) (y) 
n
1
.
(23.1.6)
!n [ f (y)]
 

 1 (y) n (y) 


 ..
.. 
 .
. 
 (n)
 (y) (n) (y) 
n
1

398

23 Conuent Transformations

We call this method that generates the Q n (y) the rst conuent form of the rst
generalization of the Richardson extrapolation process. It is easy to verify that Q n (y) is
also the solution of the linear system
A(y) = Q n (y) +

n


k k (y),

k=1

A(i) (y) =

n


k k(i) (y), i = 1, . . . , n,

(23.1.7)

k=1

where the k are the additional (auxiliary) unknowns. Note that this linear system is
obtained by differentiating the asymptotic expansion in (23.1.1) formally term by term
i times, truncating the summation at the term n n (y), replacing by =, A by Q n (y),
and the k by k , and setting i = 0, 1, . . . , n.
The recursive algorithms of Chapter 3 can be used to obtain Q n (y) of (23.1.6),
once we realize that the equations in (23.1.7) can be rewritten
in the form

n

l  l  (s)
A
+

g
(l),
l
=
0,
1,
.
.
.
,
n,
with
a(l)
=
(y)
and gk (l) =
a(l) = A(0)
k
k
k=1
s=0 s

l  l n (s)
(0)
s=0 s k (y) and An = Q n (y). See Brezinski and Redivo Zaglia [41, p. 267]. Of
course, the most direct way to compute Q n (y) is by solving the last n of the equations in (23.1.7) numerically for the k and substituting these in the rst equation.
Note that, for computing Q n (y) = !n [A(y)], we need A(y) and its derivatives
of order 1, . . . , n. In addition, the quality of Q n (y) improves with increasing n.
Therefore, to obtain high accuracy by the rst conuent form, we need to compute
a large number of derivatives of A(y). Consequently, the rst conuent form can be of
practical value provided the high-order derivatives of A(y) can be obtained relatively
easily.
We now propose another method that requires A(y) and the k (y) and their rst order
derivatives only. We derive this new method in a way that is similar to the derivation of
the rst conuent form via the linear system in (23.1.7).
Let us differentiate (23.1.1) once term by term as before, truncate the summation at
( j)
the term n n (y), replace by =, A by Q n (y), and the k by k , and collocate at the
points yl , j l j + n 1. This results in the linear system
A(y) = Q (nj) (y) +

n


k k (y),

k=1

A (yl ) =

n


k k (yl ), j l j + n 1,

(23.1.8)

k=1

where y in the rst of these equations can take on any value, and the yl are chosen to
satisfy y0 > y1 > and liml yl = 0 as before.
As we can set y = yl for l { j, j + 1, . . . , j + n 1} in (23.1.8), we call this method
( j)
that generates the Q n (y) the second conuent form of the rst generalization of the
Richardson extrapolation process.

23.1 Conuent Forms of Extrapolation Processes

( j)

399

( j)

Using Cramers rule to solve (23.1.8), we have Q n (y) = !n [A(y)], where




f (y)
1 (y)
n (y) 

 f  (y j )
1 (y j ) n (y j ) 



.
..
..
..


.
.



 f  (y


j+n1 ) 1 (y j+n1 ) n (y j+n1 )
( j)
.
(23.1.9)
!n [ f (y)] =


 1 (y j ) n (y j ) 




..
..


.
.


  (y
)  (y
)
1

j+n1

j+n1

The following result can be obtained by analyzing the determinantal representation

( j)
of Q n (y). We leave its proof to the reader.
( j)

Theorem 23.1.2 Let Q n (y) be dened via (23.1.8). If A(y) and the k (y) are n times
( j)
differentiable in y for y in some right neighborhood of 0, then the limit of Q n (y) as
yl y, j l j + n 1, exists, and satises
lim
y y

l
jl j+n1

Q (nj) (y) = Q n (y),

(23.1.10)

where Q n (y) is the rst conuent form dened above.

( j)

To determine Q n (y) numerically we may proceed exactly as proposed in the case of

the rst conuent form. Thus, we rst solve the last n of the equations in (23.1.8) for the
( j)
k , and substitute these in the rst equation to compute Q n (y) for some y, possibly in
the set {y j , y j+1 , . . . , y j+n1 }.
As the yl and y are an important part of the input to the second conuent form, a
question that arises naturally is whether one can nd a best set of these that will give
( j)
the highest accuracy in Q n (y). This seems to be an interesting research problem.
23.1.2 Convergence Analysis of a Special Case
We now present a convergence theory of the two preceding conuent methods for the
special case in which k (y) = y k , k = 1, 2, . . . , that was treated in Chapters 1 and 2.
This theory is analogous to those we saw in the previous chapters and that pertain to column sequences, and it shows that both of the conuent methods accelerate convergence
in this mode.
We rst note that
Q n (y) A = !n [A(y) A],

Q (nj) (y) A = !(nj) [A(y) A]. (23.1.11)

Next, in our analysis we assume for each i = 1, 2, . . . , that (i) {k(i) (y)}
k=1 is an asymptotic sequence, and (ii) A(i) (y) exists and has an asymptotic expansion that is obtained
by differentiating that of A(y) given in (23.1.1) term by term i times. Thus,


k k(i) (y) as y 0+, i = 1, 2, . . . .
(23.1.12)
A(i) (y)
k=1

400

23 Conuent Transformations
( j)

Substituting (23.1.1) and (23.1.12) in (23.1.11), recalling that lim j y j = 0 in Q n (y j ),

we obtain, formally, the asymptotic expansions
Q n (y) A

k !n [k (y)] as y 0+,

k=n+1

Q (nj) (y j ) A

k !(nj) [k (y j )] as j .

(23.1.13)

k=n+1

[Note that both summations in (23.1.13) start with k = n + 1 since !n [k (y)] = 0 and
( j)
!n [k (y j )] = 0 for k = 1, . . . , n, as is clear from (23.1.6) and (23.1.9).] As usual,
it is necessary to prove that these are valid asymptotic expansions. For instance, for
Q n (y) A, it must be shown that (i) {!n [k (y)]}
k=n+1 is an asymptotic scale as
y 0+, and (ii) for each positive integer N n + 1, there holds Q n (y) A

N 1
( j)
k=n+1 k !n [k (y)] = O(!n [ N (y)]) as y 0+. Similarly, for Q n (y j ) A.
Going back to k (y) = y k , k = 1, 2, . . . , we rst have
A(y) A +

k y k as y 0+,

(23.1.14)

k=1

where
k = 0, k = 1, 2, . . . ; 1 < 2 < ; lim k = .
k

(23.1.15)

By the fact that k(i) (y) = [k ]i y k i , it follows from (23.1.15) that {k(i) (y)}
k=1 , i =
1, 2, . . . , are asymptotic sequences as y 0+. Here [x]0 = 1 and [x]i = x(x 1)
(x i + 1) for i = 1, 2, . . . , as usual. We also assume that, for each i 1, A(i) (y) has
an asymptotic expansion as y 0+, which can be obtained by differentiating that of
A(y) term by term i times. Thus,
A(i) (y)

[k ]i k y k i as y 0+, i = 1, 2, . . . .

(23.1.16)

k=1
( j)

The following lemma concerns !n [k (y)] and !n [k (y j )]. We leave its proof to the
reader. (Of course, the condition imposed on the yl in this lemma is relevant only for the
second conuent form.)
Lemma 23.1.3 With k (y) = y k , and the k as in (23.1.15), and with yl = y0 l ,
l = 1, 2, . . . , for some (0, 1) and y0 , we have
!n [k (y)] = n,k y k ; n,k =

!(nj) [k (y j )] = n,k y j k ; n,k

n

i k
,
i
i=1


1/k
 0
 ck
 1
k
 c
=
 k
V (c1 , . . . , cn )  ..
 .

 cn1
k


1/1 1/n 

c10 cn0 
c11 cn1  . (23.1.17)
..
.. 
.
. 
cn1 cn1 
1

23.2 Conuent Forms of Sequence Transformations

401

Thus, {!n [k (y)]}

k=n+1 is an asymptotic scale as y 0+ and {!n [k (y j )]}k=n+1 is
k
an asymptotic scale as j . Here ck = , k = 1, 2, . . . , and V (1 , . . . , n ) is the
Vandermonde determinant as usual.
( j)

We next give the main convergence results that concern the rst and second conuent
forms.
Theorem 23.1.4 With A(y) as in (23.1.14)(23.1.16) and with yl as in Lemma 23.1.3,
( j)
Q n (y) A and Q n (y j ) A have the complete asymptotic expansions given in
(23.1.13). That is,
Q n (y) A

n,k k y k as y 0+,

k=n+1

Q (nj) (y j ) A

n,k k y j k as j ,

(23.1.18)

k=n+1

with n,k and n,k as in (23.1.17). Thus, if n+ is the rst nonzero k with k n + 1,
( j)
then Q n (y) A and Q n (y j ) A satisfy the asymptotic equalities
Q n (y) A n,n+ n+ y n+ as y 0+,

Q (nj) (y j ) A n,n+ n+ y j n+ as j .

(23.1.19)

We leave the proof of the results in (23.1.18) and (23.1.19) to the reader.

23.2 Conuent Forms of Sequence Transformations

23.2.1 Conuent -Algorithm
The idea of conuent methods rst appeared in a paper by Wynn [370], and it was based
on algorithms for sequence transformations. To illustrate the idea, let us consider the
Shanks transformation and the -algorithm, as was done by Wynn [370].
We start by replacing the sequence Am by some function F(x), making the analogy
( j)
( j)
limm Am limx F(x), when both limits exist. Let us next replace 2n and 2n+1
1
in Algorithm 16.2.1 by 2n (x + j h) and h 2n+1 (x + j h), respectively. Finally, let us
allow h 0. As a result, we obtain the conuent -algorithm of Wynn that reads as
follows:
Algorithm 23.2.1 (Conuent -algorithm)
1. Set
1 (x) = 0 and 0 (x) = F(x).
2. Compute the k+1 (x) by the recursion
k+1 (x) = k1 (x) +

,
k (x)

k = 0, 1, . . . .

402

23 Conuent Transformations

The relevant approximations to limx F(x) are the 2n (x).

The following result is due to Wynn [370].
Theorem 23.2.2 For 2n (x) and 2n+1 (x) we have
2n (x) =

(0)
(x)
Hn+1

Hn(2) (x)

and 2n+1 (x) =

Hn(3) (x)
(1)
Hn+1
(x)

(23.2.1)

( j)

where, for all j 0, we dene H0 (x) = 1 and



 F ( j) (x) F ( j+1) (x) F ( j+n1) (x) 

 ( j+1)
 F
(x) F ( j+2) (x) F ( j+n) (x) 

( j)
Hn (x) = 
 , n 1.
..
..
..


.
.
.


 F ( j+n1) (x) F ( j+n) (x) F ( j+2n2) (x)

(23.2.2)

Consequently, 2n (x) is the solution of the linear system

F(x) = 2n (x) +

n


k F (k) (x),

k=1

F (i) (x) =

n


k F (k+i) (x), i = 1, . . . , n.

(23.2.3)

k=1
( j)

Note that the determinants Hn (x) are analogous to the Hankel determinants introduced in (16.1.13) in connection with the Shanks transformation. Note also that the linear
system in Theorem (23.2.3) is completely analogous to that in (23.1.7), and 2n (x) can
be computed as the solution of this system. Another way of computing the 2n (x) is via
( j)
(23.2.1), with the Hn (x) being determined from the recursion
( j)

( j)

H0 (x) = 1 and H1 (x) = F ( j) (x), j = 0, 1, . . . ,

( j)

( j+2)

Hn+1 (x)Hn1 (x) = Hn( j) (x)Hn( j+2) (x) [Hn( j+1) (x)]2 , j 0, n 1. (23.2.4)
This can be proved by applying Sylvesters determinant identity to the determinant
( j)
Hn+1 (x). For yet another algorithm, see Wynn [376].
Some of the algebraic properties of the conuent -algorithm and its application to
functions F(x) that are completely monotonic are discussed by Brezinski [33]. See
also Brezinski [36] and Brezinski and Redivo Zaglia [41]. Unfortunately, the conuent
-algorithm cannot be very practical, as it requires knowledge of high-order derivatives
of F(x).

23.2.2 Conuent Form of the Higher-Order G-Transformation

The conuent form of the higher-order G-transformation described in Chapter 21 is
obtained by letting h 0 in (21.2.1). By suitable row and column transformations on

23.2 Conuent Forms of Sequence Transformations

403

the determinants in (21.2.1), we rst obtain




F(x)
F(x)/ h
n F(x)/ h n 


f (x)
 f (x)/ h
n f (x)/ h n 



..
..
..


.
.
.


n1 f (x)/ h n1 n f (x)/ h n 2n1 f (x)/ h 2n1 
G n (x; h) =
, (23.2.5)


  f (x)/ h
n f (x)/ h n 



..
..


.
.


n f (x)/ h n 2n1 f (x)/ h 2n1 
where g(x) = g(x + h) g(x), 2 g(x) = (g(x)), and so on. Next, by letting
x h 0
in (23.2.5) and using the fact that f (x) = F  (x), which follows from F(x) = 0 f (t) dt,
and also the fact that limh0 k g(x)/ h k = g (k) (x), we obtain
Bn (x) = lim G n (x; h) =
h0

(0)
(x)
Hn+1

Hn(2) (x)

(23.2.6)

( j)

Here Hn (x) is exactly as dened in (23.2.2). Thus, Bn (x) is nothing but 2n (x) and can
be computed by the conuent -algorithm.
All these developments are due to Gray, Atchison, and McWilliams [112], where the
convergence acceleration properties of Bn (x) as x are also discussed.

23.2.3 Conuent -Algorithm

Wynn [370] applies the preceding approach to the -algorithm. In other words, he re( j)
( j)
places 2n and 2n+1 in Algorithm 20.1.1 by 2n (x + j h) and h 1 2n+1 (x + j h) respectively, and lets h 0. This results in the conuent -algorithm of Wynn, which reads
as follows:
Algorithm 23.2.3 (Conuent -algorithm)
1. Set
1 (x) = 0 and 0 (x) = F(x).
2. Compute the k+1 (x) by the recursion
k+1 (x) = k1 (x) +

k+1
, k = 0, 1, . . . .
k (x)

The relevant approximations to limx F(x) are the 2n (x).

The following result is due to Wynn [370].

404

23 Conuent Transformations

Theorem 23.2.4 For 2n (x) and 2n+1 (x) we have

2n (x) =

(0)
(x)
H n+1
H n(3) (x)
and

,
(x)
=
2n+1
(1)
H n(2) (x)
H n+1
(x)

(23.2.7)

( j)
( j)
where, for each j 0 and n 0, H n (x) is obtained from Hn (x) by replacing F (i) (x)
(i)
in the latter by F (x)/i!, i = 0, 1, . . . .
( j)

It is clear that 2n (x) can be computed from (23.2.7), where the Hn (x) can be determined by using a recursion relation similar to that in (23.2.4). For further results,
see Wynn [376]. See also [41]. It is obvious that the conuent -algorithm, just like
the conuent -algorithm, is not very practical as it requires knowledge of high-order
derivatives of F(x).

23.2.4 Conuent Overholt Method

Applying the technique above to the method of Overholt, Brezinski and Redivo Zaglia
[41] derive the following conuent form:
Algorithm 23.2.5 (Conuent Overholt method)
1. Set V0 (x) = F(x).
2. Compute Vn+1 (x) by the recursion
Vn+1 (x) = Vn (x)

F  (x) Vn (x)

, n = 0, 1, . . . .
F  (x) n + 1

The relevant approximations to limx F(x) are the Vn (x).

The following theorem, whose proof can be achieved by induction on n, is given in
[41, p. 255, Theorem 5.1].
Theorem 23.2.6 Assume that {[F  (x)]k }
k=1 is an asymptotic sequence as x , and
that
F(x) A +

k [F  (x)]k as x .

(23.2.8)

k=1

Assume, in addition, that both sides of (23.2.8) can be differentiated term by term. Then



n

k
(1 k/s) [F  (x)]k as x .
(23.2.9)
Vn (x) A
k=n+1

s=1

We do not know of another more convenient way of dening the Vn (x) except through
the recursion relation of Algorithm 23.2.5. This recursion relation, however, requires us
to rst obtain Vn (x) in closed form and then differentiate it. Of course, this task can be
achieved only in some cases, and by using symbolic computation. Thus, the conuent
form of the method of Overholt is not very useful.

23.3 Conuent D (m) -Transformation

405

The conuent form of the -algorithm is obtained similarly in [41]. As mentioned

there, this conuent form suffers from the same deciency as the conuent form of the
method of Overholt; hence, it is not very useful.

23.3 Conuent D (m) -Transformation

With the approach of the preceding sections, it is possible to derive conuent forms
of additional transformations. For example, Levin and Sidi [165, Section
3] give the

conuent form of the D (m) -transformation for innite-range integrals 0 f (t) dt (called
the C-transformation in [165]) that is obtained in the same way as the rst conuent
form of the Richardson extrapolation process. This derivation reads as follows:
F(x) = Cn (x) +

m


x k f (k1) (x)

n
k 1

k=1

F (s) (x) =

ds
dxs


m

ki x i ,

i=0

x k f (k1) (x)

n
k 1

k=1


ki x i , s = 1, 2, . . . , N ,

(23.3.1)

i=0

x
where F(x) = 0 f (t) dt hence F (s) (x) = f (s1) (x) for all s 1, n = (n 1 , . . . , n m ), and

m
N = k=1 n k . Gray and Wang [114] provide convergence results concerning Cn (x) as
x . See also Gray and Wang [113].
Now, this rst conuent form requires the computation of the derivatives of f (x) of
order as high as N + m 1, which may be inconvenient as N is a large integer generally.
Here, we propose the second conuent form, which circumvents this inconvenience
entirely. From the formalism of Section 23.1, this reads as follows:
F(x) = Cn( j) (x) +

m

k=1

x k f (k1) (x)

n
k 1

ki x i ,

i=0


 m
n
k 1

d 
k (k1)
i 

x f
(x)
, j l j + N 1, (23.3.2)
F (xl ) =
ki x

d x k=1
x=xl
i=0


where F(x), n, and N are as before and the xl are chosen to satisfy 0 < x0 < x1 < ,
and liml xl = . Once the ki are determined from the last N of the equations in
( j)
(23.3.2), they can be substituted in the rst equation to compute Cn (x), with x = x j ,
for example.
Obviously, this transformation requires the computation of only one (nite-range)
integral F(x), and knowledge of the derivatives f (k) (x), k = 1, 2, . . . , m, independently
of the size of N . As before, the question about the best choice of x and the xl is of
interest.

23.3.1 Application to the D (1) -Transformation and Fourier Integrals

As a special case, let us consider application
 of the second conuent form of the
D (1) -transformation to the Fourier integral 0 eit g(t) dt, where g A( ) . Replacing
the k = 1 term in the summations of the equations in (23.3.2) by f (x) = eix g(x) (recall

406

23 Conuent Transformations

that 0 = 0 for such integrals), letting m = 1, these equations become

F(x) = Cn( j) (x) + f (x)

n1


i x i

i=0

f (xl ) = f (xl )

n1

i=1

i i xli1 + f  (xl )

n1


i x i , j l j + n 1. (23.3.3)

i=0

Substituting f  (x) = [i + g  (x)/g(x)] f (x) in the last n equations, we see that the s
can be obtained by solving the linear system

n1 

s 1
g  (xl )

(23.3.4)
s = 1, j l j + n 1.
i +
g(xl )
xl xls
s=0
Obviously, this is a rather inexpensive way of approximating the integral in question,
since it requires the computation of only one nite-range integral, namely, the integral
F(x) in (23.3.3).

24
Formal Theory of Sequence Transformations

24.1 Introduction
The purpose of this chapter is to present a formal theory of sequence transformations
that was begun recently by Germain-Bonne [96], [97]. The theory of Germain-Bonne
covers very few cases. It was later extended by Smith and Ford [317] to cover more
cases. Unfortunately, even after being extended, so far the formal theory includes a very
small number of cases of interest and excludes the most important ones. In addition, for
the cases it includes, it has produced results relevant to column sequences only, and these
results are quite weak in the sense that they do not give any information about rates of
acceleration. Nevertheless, we have chosen to present its present achievements briey
here for the sake of completeness. Our treatment of the subject here follows those of
Smith and Ford [317] and Wimp [366, Chapter 5, pp. 101105].
Let us denote the approximations that result by applying an extrapolation method
ExtM to the sequence {Am }
m=0 by Sn, j , where
Sn, j = G n, j (A j , A j+1 , . . . , A j+n )

(24.1.1)

for some function G n, j (x0 , x1 , . . . , xn ). We assume that this function satises

G n, j (x0 + , x1 + , . . . , xn + ) = G n, j (x0 , x1 , . . . , xn ) + (24.1.2)
for all scalars and . This, of course, means that ExtM is a quasi-linear sequence
transformation, in the sense dened in the Introduction. It is clear that
G n, j (0, 0, . . . , 0) = 0, G n, j (, , . . . , ) = .
Using (24.1.2), we can write
G n, j (x0 , x1 , . . . , xn ) = x0 + (x1 x0 )G n, j (0, Y1 , Y2 , . . . , Yn ),

(24.1.3)

where we have dened

Yk =

xk x0
, k = 1, 2, . . . .
x1 x0

(24.1.4)

Letting
xk = xk+1 xk and X k =
407

xk
,
xk1

(24.1.5)

408

24 Formal Theory of Sequence Transformations

we see that
Y1 = 1, Yk = 1 +

k1


X 1 X 2 X i , k = 2, 3, . . . .

(24.1.6)

i=1

Using (24.1.4)(24.1.6), and dening the function gn, j through

gn, j (X 1 , . . . , X n1 ) = G n, j (0, 1, Y2 , . . . , Yn ),

(24.1.7)

we now write (24.1.3) in the form

G n, j (x0 , x1 , . . . , xn ) = x0 + (x0 )gn, j (X 1 , X 2 , . . . , X n1 ).

(24.1.8)

Let S stand for the limit or antilimit of {Am }, and dene

Rm =

Am+1 S
Am+1
, rm =
,
Am S
Am

m = 0, 1, . . . .

(24.1.9)

Going back to Sn, j in (24.1.1), and invoking (24.1.8), we thus have that
Sn, j = A j + (A j ) gn, j (r j , . . . , r j+n2 ),

(24.1.10)

A j
Sn, j S
=1+
gn, j (r j , . . . , r j+n2 ).
Aj S
Aj S

(24.1.11)

and hence

Before going on, we mention that, in the theory of Germain-Bonne, the functions G n, j
and hence gn, j do not depend on j explicitly, that is, they are the same for all j. It is this
aspect of the original theory that makes it relevant for only a limited number of cases.
The explicit dependence on j that was introduced by Smith and Ford allows more cases
to be covered. We present examples of both types of gn, j in the next section.

24.2 Regularity and Acceleration

We now investigate the regularity and acceleration properties of Sn, j as j , while
n is being held xed. The following results are direct consequences of (24.1.10) and
(24.1.11).
Proposition 24.2.1
(i) Provided the sequence {gn, j (r j , . . . , r j+n2 )}
j=0 is bounded, ExtM is regular on
{Am } as j , that is, lim j Sn, j = lim j A j = S.
(ii) {Sn, j }
j=0 converges faster than {Am }, that is, lim j (Sn, j S)/(A j S) = 0, if
and only if
lim (1 R j ) gn, j (r j , . . . , r j+n2 ) = 1.

(24.2.1)

24.2 Regularity and Acceleration

409

24.2.1 Linearly Convergent Sequences

In case the functions gn, j for xed n are all the same function gn , and the sequence {Am }
converges linearly, the previous proposition can be rened considerably. In the present
context, {Am } is linearly converging if, with Rm as dened in (24.1.9), there holds
lim Rm = for some , 0 < || < 1.

(24.2.2)

As mentioned in the proof of of Theorem 15.3.1, (24.2.2) implies that limm rm = as

well, where rm is as dened in (24.1.9). Note also that the family of sequences b(1) /LIN
is a subset of the set of linearly convergent sequences.
Using these in Proposition 24.2.1, we can now state the following theorem, whose
proof we leave to the reader.
Theorem 24.2.2 Let {Am } be a linearly convergent sequence as in (24.2.2). In the
sequence transformation ExtM, let
lim gn, j (X 1 , . . . , X n1 ) = gn (X 1 , . . . , X n1 ),

(24.2.3)

and assume that gn is continuous in a neighborhood of (, , . . . , ), = 1, and satises

gn (, , . . . , ) =

1
.
1

(24.2.4)

Then Proposition 24.2.1 applies, that is, ExtM is regular and accelerates the convergence
of {Am } in the sense described there.
Of course, (24.2.3) is automatically satised in case gn, j = gn for every j. When this
is the case, under the rest of the conditions of Theorem 24.2.2, the sequence of the partial

i
, = 1, is in the kernel of ExtM, in the sense that
sums of the geometric series i=0
Sn, j = S = 1/(1 ) for every j.
We now illustrate Theorem 24.2.2 with a few examples.
The conditions of Theorem 24.2.2 hold when ExtM is the 2 -process, because in this
case g2, j (x) = g2 (x) = 1/(1 x), as can be shown with the help of the formula S2, j =
A j (A j )2 /(2 A j ). (We already proved in Theorem 15.3.1 that the 2 -process is
regular for and accelerates the convergence of linearly convergent sequences.)
The conditions of Theorem 24.2.2 hold also when ExtM is the W -transformation
of Lubkin. In this case, g3, j (x, y) = g3 (x, y) = (1 2y + x)/(1 2y + x y), and the
singularities of this function occur only along y = 1/(2 x), which meets y = x
only at (1, 1). Therefore, g3 (x, y) will be continuous in any neighborhood of (, )
with = 1. This has been stated by Smith and Ford [317]. (We already stated prior
to Theorem 15.4.1 that he Lubkin transformation is regular for and accelerates the
convergence of linearly convergent sequences.)
Smith and Ford [317] show that Proposition 24.2.1 applies to the functions gn, j associated with the Levin t-, u-, and v-transformations. Therefore, these transformations
are regular on and accelerate the convergence of linearly convergent sequences as
j . Here we treat the u-transformation, the treatment of the rest being similar.

410

24 Formal Theory of Sequence Transformations

( j)

Because Ln in (19.2.4), with m = mam there, depends on A j , A j+1 , . . . , A j+n+1 ,

we have that

i1
 
n

s=1 X 1 X s
i n
n2 1 +
(1)
( j + i + 1)
i
X1 X2 Xi
i=1
gn+1, j (X 1 , . . . , X n ) =
. (24.2.5)
 
n

1
i n
n2
(1)
( j + i + 1)
i
X1 X2 Xi
i=0
Thus,

i1
 
n

s=1 X 1 X s
i n 1+
(1)
i
X1 X2 Xi
i=1
.
lim gn+1, j (X 1 , . . . , X n ) = gn (X 1 , . . . , X n ) =


n

j
1
i n
(1)
i X1 X2 Xi
i=0
(24.2.6)
Substituting X 1 = = X n = in (24.2.6), it can be shown that (24.2.4) is satised,
and hence Theorem 24.2.2 applies.
With the technique we used for the Levin transformations, it can be shown that Proposition 24.2.1 also applies to the functions gn, j associated with the Sidi S-transformation.
For this transformation too, the functions gn, j depend on j explicitly. Actually, with
( j)
Sn as in (19.3.3), and m = mam there, the function gn+1, j (X 1 , . . . , X n ) is obtained from (24.2.5) by replacing the expression ( j + i + 1)n2 by ( j + i + 1)n2 for
each i. Consequently, (24.2.6) holds without any change, and hence Theorem 24.2.2
applies.
For additional examples, see Weniger [353].

24.2.2 Logarithmically Convergent Sequences

So far our treatment has been concerned with linearly convergent sequences as dened in (24.2.2). We have seen that convergence acceleration of all such sequences
can be achieved by more than one method. We now turn to logarithmically convergent
sequences. In this context, a sequence {Am } converges logarithmically if it satises
lim Rm = 1 and

lim rm = 1.

(24.2.7)

Here Rm and rm are as dened in (24.1.9). Note that the family of sequences b(1) /LOG
is a subset of the set of logarithmically convergent sequences.
Such sequences are very difcult to treat numerically and to analyze analytically. This
fact is also reected in the problems one faces in developing a formal theory for them.
The main results pertaining to linearly convergent sequences were given by Delahaye
and Germain-Bonne [66], [67]. These authors rst dene a property they call generalized
remanence:

24.3 Concluding Remarks

411

Denition 24.2.3 A set M of real convergent sequences is said to possess the property
of generalized remanence if the following conditions are satised:
1. There exists a convergent sequence { Sm } with limit S such that Sm = S for all m, and
such that
(i) there exists {Sm0 } M such that limm Sm0 = S0 ,
(ii) for any m 0 0, there exists p0 m 0 and {Sm1 } M such that limm Sm1 = S1
and Sm1 = Sm0 for m p0 ,
(iii) for any m 1 > p0 , there exists p1 m 1 and {Sm2 } M such that limm Sm2 = S2
and Sm2 = Sm1 for m p1 ,
(iv) .........................
2. The sequence {S00 , S10 , . . . , S 0p0 , S 1p0 +1 , S 1p0 +2 , . . . , S 1p1 , . . . } is in M.
(Note that the notion of generalized remanence was given in [67] and it was preceded
in [66] by the notion of remanence.)
Delahaye and Germain-Bonne next prove that a sequence set M that has the property
of generalized remanence cannot be accelerated, in the sense that there does not exist
a sequence transformation that accelerates the convergence of all sequences in M. Following this, they prove that the set of logarithmically convergent sequences possesses
the property of generalized remanence and therefore cannot be accelerated.
Techniques other than that involving (generalized) remanence but similar to it have
been used to determine further sets of sequences that cannot be accelerated. See
Kowalewski [155], [156], and Delahaye [65]. For a list of such sets, see Brezinski and
Redivo Zaglia [41, pp. 4041].
The fact that there does not exist a sequence transformation that can accelerate the
convergence of all sequences in a certain set means that the set is too large. This suggests
that one should probably investigate the possibility of nding (proper) subsets of this set
that can be accelerated. It would be interesting to know what the largest such subsets are.
Similarly, it would be interesting to know the smallest subsets that cannot be accelerated.
For some progress in this direction, see Delahaye [65] and Osada [224].

24.3 Concluding Remarks

As mentioned in the beginning of this chapter, the formal theory of sequence transformations, despite its elegance, is of limited scope. Indeed, its positive results concern mostly
sequences that converge linearly [in the sense of (24.2.2)], whereas its results concerning
sequences that converge logarithmically [in the sense of (24.2.7)] are mostly negative.
We have seen that this theory, as it applies to linearly convergent sequences, shows
only regularity and acceleration for {Sn, j }
j=0 , but gives no information on rates of convergence. Thus, it does not tell anything about the relative efciencies of Sn, j and Sn+1, j ,

that is, whether {Sn+1, j }

j=0 converges more quickly than {Sn, j } j=0 . In addition, so far
no results about the diagonal sequences {Sn, j }
n=0 have been obtained within the framework of the formal theory. (This should be contrasted with the rather complete results
we obtained for sequences in b(1) /LIN and b(1) /LOG.) It seems that these subjects may
serve as new research problems for the formal theory.

412

24 Formal Theory of Sequence Transformations

One way of enlarging the scope of the formal theory can be by investigating sets other
than those mentioned so far. An interesting set can be, for example, that containing linear
combinations of linearly convergent sequences, such as real Fourier series. The reason
for this is that such series are not necessarily linearly convergent in the sense of (24.2.2),
as can be veried with simple examples. Recall that we considered such sequences in
Chapter 6. It would be interesting to see, for example, whether it is possible to obtain
positive results analogous to Theorem 24.2.2.

Part III
Further Applications

25
Further Applications of Extrapolation Methods
and Sequence Transformations

In Parts I and II of this book, we studied in some detail the Richardson extrapolation and
its generalizations and various important sequence transformations. We also mentioned
several applications of them. Actually, we discussed in detail the Romberg integration
of nite-range integrals of regular integrands, numerical differentiation, and the computation of innite-range integrals by the D-transformation. We discussed the application
of the various generalizations of the D-transformation to the computation of oscillatory
innite-range integrals, including some important integral transforms. We also treated
in detail the acceleration of convergence of innite series, including power series and
Fourier series and their generalizations, by the d-transformation and other methods,
such as the Shanks transformation, the -algorithm, the BakerGammel approximants
and their extensions, and so on. In connection with acceleration of convergence of power
series, we also discussed in some detail the subject of prediction via the d-transformation
and mentioned that the approach presented can be used with any sequence transformation. In this chapter, we add further applications of special interest.
We would like to note that extensive surveys and bibliographies covering the application of extrapolation methods to numerical integration can be found in Joyce [145],
Davis and Rabinowitz [63], and Rabinowitz [234].

25.1 Extrapolation Methods in Multidimensional Numerical Quadrature

25.1.1 By GREP and d-Transformation
Multidimensional EulerMaclaurin Expansions
One of the important uses of the Richardson extrapolation process is in computation of
nite-range integrals over the s-dimensional hypercube C or hypersimplex S, namely,
of the integrals

IC [ f ] =


f (x) dx, I S [ f ] =

f (x) dx; x = (x1 , . . . , xs ), dx =

C

s

i=1

Here C and S are given, respectively, by

&
%
C = (x1 , . . . , xs ) : 0 xi 1, i = 1, . . . , s
415

d xi .

416

25 Further Applications of Extrapolation Methods

and
&
%
S = (x1 , . . . , xs ) : 0 xi 1, i = 1, . . . , s; x1 + + xs 1 .
Let TC (h) and TS (h) be approximations to IC [ f ] and I S [ f ], respectively, that are
obtained by applying the trapezoidal rule in each of the variables x1 , . . . , xs , with stepsize
h = 1/n, where n is a positive integer. (We restrict our attention to this simple rule for
simplicity. Other more sophisticated rules can also be dened.) Thus, TC (h) and TS (h)
are given by
TC (h) = h s

n
n




i 1 =0

i 2 =0

n



f (i 1 h, i 2 h, . . . , i s h)

i s =0

and
TS (h) = h s

ni
n

1 

i 1 =0

ni 1 
i s1

i 2 =0



f (i 1 h, i 2 h, . . . , i s h),

i s =0

k1

 k
ai = 12 a0 + i=1
ai + 12 ak . Let us denote C or S by $, IC [ f ] or I S [ f ] by
where i=0
I [ f ], and TC (h) or TS (h) by Q n .
In case f (x) C ($), the generalized EulerMaclaurin expansions for Q n read
Qn I [ f ] +

k n 2k as n ,

k=1

for some constants k independent of h. Thus, the polynomial Richardson extrapolation

can be applied to {Q n } effectively. See Lyness and McHugh [200] and Lyness and
Puri [203] for these results. In a recent work, Lyness and Rude [204] consider double
integrals whose integrands involve derivatives of known functions; they derive quadrature
formulas Q n based on function values only and show that Q n have asymptotic (Euler
Maclaurin) expansions in powers of n 2 as well. Thus, the polynomial Richardson
extrapolation can be applied to {Q n } to obtain approximations of high accuracy to such
integrals too.
In case f (x) is a C function in the interior of the set $ but has algebraic and/or
logarithmic singularities at corners and/or along edges and/or on surfaces of $, the
EulerMaclaurin expansions assume the following more general form:

qi
p



p=0 ikp (log n)
Qn I [ f ] +
as n , qi 0 integers.
n i +k
i=1 k=0
Here, Q n are approximations to I [ f ] constructed by modifying the rules TC (h) and TS (h)
suitably by avoiding the singularities of f (x) on the boundary. The i and qi depend on
the nature of the singularities, and the ikp are constants independent of n. Generally
speaking, is the number of the different types of singularities of f (x).
We do not go into the details of each expansion here. For specic results about corner
singularities, we refer the reader to the original works by Lyness [196] and Lyness and
Monegato [201]; for results about line and edge singularities, see Sidi [283], and for
those concerning full (line and corner) singularities, see Lyness and de Doncker [199].

25.1 Extrapolation Methods in Multidimensional Numerical Quadrature

417

Direct Application of GREP to {Q n }

In case i and qi are known, we can apply GREP(m) of Denition 4.2.1 to the se

quence {Q n } rather, to the function A(y), where A(y) Q n with m = i=1 (qi + 1),
y = n 1 , and with rk = 1 and the k (y) chosen as (log n) p n i , 0 p qi , 1 i .
The method can then be efciently implemented by the W(m) -algorithm. Now, the computation of Q n for given n requires O(n s ) integrand evaluations. Therefore, to keep the
cost of computation in check, it is appropriate to choose yl = 1/(l + 1), l = 0, 1, . . . .
The numerical instability that results from this choice of the yl can be controlled by using
high-precision oating-point arithmetic, as was discussed in Section 2.1. This approach
was originally suggested by Lyness [195] for integrals over hypercubes of functions with
a corner singularity. Its use for functions with line singularities was considered by Sidi
in [271], an earlier version of [283]. See also Davis and Rabinowitz [63] and Sidi [287]
for brief reviews.
Application of the d-Transformation for Innite Sequences to {Q n }
We next propose using the d (m) -transformation for innite sequences (as in Denition 6.2.2) to accelerate the convergence of the sequence {Q n } in case the i and qi are
not all known. We give a theoretical justication of the proposed approach next.
Let us rst consider the case in which qi = 0 for each i. In this case, Q n = I [ f ] +


i)
(n) A(
, i = 1, . . . , . Thus, Theorem 6.8.5 applies to {Q n },
i=1 Hi (n), where Hi

0

k
k
that is, Q n = I [ f ] + k=1 n ( Q n )gk (n) for some gk A(0)
0 , k = 1, . . . , , and we
conclude that the d () -transformation for innite sequences can be used on this sequence.


In general, when not all qi are zero, we rst observe that Q n = I [ f ] + i=1 G (i)
n ,


qi
(i )
p
=
u
(n)(log
n)
with
u
(n)

A
,
0

q
.
Thus,
we
have
from
where G (i)
ip
i
n
p=0 i p
0
b(qi +1) , and so is {G (i)
} by Proposition 6.1.6.
Example 6.4.9 that, for each i, {G (i)
n } is in
n


Consequently, by the fact that Q n = i=1 G (i)
n and by part (ii) of Heuristic 6.4.1,

{Q n } b(m) with m = i=1 (qi + 1). We can thus approximate I [ f ] effectively by
applying the d (m) -transformation for innite sequences to the sequence {Q n }.
To keep the cost of computation under control, again it is appropriate to choose
Rl = l + 1, l = 0, 1, . . . , in Denition 6.2.2, and use high-precision oating-point arithmetic. Note that the only input needed for this application is the integrand f (x) and the
integer m. In case m is not known, we can start with m = 1 and increase it if necessary
until acceleration takes place. We already know no extra cost is involved in this strategy. Numerical experiments show that this approach produces approximations of high
accuracy to I [ f ].
One may wonder whether other sequence transformations can be used to accelerate
the convergence of the sequence {Q n } when the i and qi are not all known. Judging
from the form of the EulerMaclaurin expansion of Q n , and invoking Theorems 16.4.6
and 16.4.9, it becomes clear that the only other transformation that is relevant is that
of Shanks, provided it is applied to the sequence {Q r n }, where r 1 and 2 are
integers. Clearly, this approach is very costly even for moderate s. In view of all this, the
d (m) -transformation, when applied as proposed here, seems to be the most appropriate
sequence transformation for computing singular multidimensional integrals over hypercubes and hypersimplices when the i and qi in the expansion of Q n are not available.

418

25 Further Applications of Extrapolation Methods

Another Use of GREP

Finally, we mention an approach introduced in two papers by Espelid [78], [79] that
uses extrapolation in a different way.
 1 approach can be explained in a simple
 1 This
way via the double integral I [ f ] = 0 0 f (x, y) d x d y, where f (x, y) = x g(x, y)
and g C ([0, 1] [0, 1]). (When is not an integer, this integral has an edge
singularity along
y-axis.) Here we compute the sequence of nonsingular inte 1 the
1h
[
f (x, y) d x]dy for different values of h. Then I [ f ] F(h) =
grals
F(h)
=
0 0
h
1
x
G(x)
d
x,
where
G(x)
= 0 g(x, y) dy C [0, 1]. Expanding G(x) at x = 0, we
0


+k
as h 0. Now apply
obtain the asymptotic expansion F(h) I [ f ] +
k=1 k h
(1)
the Richardson extrapolation process (GREP in this particular case) to {F(h l )}, where
h 0 > h 1 > , and liml h l = 0. For good numerical results, we should compute
the integrals F(h l ) with sufcient accuracy. The generalization to arbitrary dimension
and edge or corner singularities is now clear. For details and numerical examples, see
Espelid [78], [79]. We note that the approach of Espelid seems to be an
 1extension of that
of Evans, Hyslop, and Morgan [80] for one-dimensional integrals 0 f (x) d x, where
f (x) has a singularity at x = 0. The approach of [80] is heuristic, and the extrapolation
method employed is the -algorithm.

25.1.2 Use of Variable Transformations

The performance of GREP and the d-transformation (and the method of Espelid as well)
in numerical integration over the hypercube C can be enhanced by rst transforming the
integral via so-called variable transformations, namely, xi = i (t), i = 1, . . . , s, where
the functions (t) = i (t), for each i, possess the following properties:
(a) (t) C [0, 1] maps [0, 1] to [0, 1], such that (0) = 0 and (1) = 1 and  (t) > 0
on (0, 1).
(b)  (1 t) =  (t) so that (1 t) = 1 (t).
(c) ( p) (0) = ( p) (1) = 0, p = 1, . . . , p , for some p , 1 p . In other words,
when p is nite,
(t)

ci t p +i+1 as t 0+, (t) 1

i=0

ci (1 t) p +i+1 as t 1,

i=0

while for p = these asymptotic expansions are empty.

Following these variable transformations, we obtain

s

I[ f ] =
g(t) dt, g(t) = f (1 (t1 ), . . . , s (ts ))
i (ti ).
C

i=1

s

with t = (t1 , . . . , ts ) and dt = i=1 dti , as usual. Now let h = 1/n and approximate
the transformed integral C g(t)dt (using the trapezoidal rule in each of the variables ti )
by
Q n = TC (h) = h s

n1 
n1

i 1 =1 i 2 =1

n1

i s =1

g(i 1 h, i 2 h, . . . , i s h).

25.1 Extrapolation Methods in Multidimensional Numerical Quadrature

419

[Here we have used the fact that g(t) = 0 when, for each i, ti = 0 or ti = 1, because


i (0) = i (1) = 0.] Then, Q n has the asymptotic expansion

qi



ikp (log n) p
p=0
Q n I [ f ] +
as n , qi 0 integers.
n i +k
i=1 k=0
By property (c) of the i (t), the new integrand g(t) will be such that i > i , so that

Q n I [ f ] tends to 0 faster than Q n I [ f ]. This also means that, when GREP or the
d-transformation is used to accelerate the convergence of { Q n }, the computational effort
spent to obtain a required level of accuracy will be smaller than that spent on {Q n } for
( p)
( p)
the same purpose. In case i (0) = i (1) = 0 for all p 1, the asymptotic expansion
of Q n I [ f ] is even empty. In such a case, the convergence of { Q n } to I [ f ] is very
quick and extrapolation does not improve things.
The EulerMaclaurin expansions for the Q n and the application of GREP to the
sequences { Q n } have been considered recently by Verlinden, Potts, and Lyness [347].
We close this section by giving a list of useful variable transformations x = (t).
1. The Korobov [154] transformation:

  t
2m
[u(1 u)]m du.
(t) = (2m + 1)
m
0

Thus, ( p) (0) = ( p) (1) = 0 for p = 1, . . . , m. Analysis of the trapezoidal and midpoint rules following the Korobov transformation was given by Sidi [293] for regular
integrands and by Verlinden, Potts, and Lyness [347] for integrands with endpoint
singularities.
2. The Sag and Szekeres [254] tanh-transformation:



c 1
1
1
1

+ , c > 0.
(t) = tanh
2
2 t
1t
2
In this case, we have ( p) (0) = ( p) (1) = 0 for all p 1.
3. The IMT-transformation of Iri, Moriguti, and Takasawa [138]:
t


(u) du
c
(t) =  01
, (t) = exp
, c > 0.
t(1 t)
(u)
du
0
In this case too, we have ( p) (0) = ( p) (1) = 0 for all p 1. For a most elegant
theory of this transformation, we refer the reader to Iri, Moriguti, and Takasawa [138].
4. The double exponential transformation of Mori [214]:

 

1
1
1
1

(t) = tanh a sinh b

+ , a, b > 0.
2
1t
t
2
In this case too, ( p) (0) = ( p) (1) = 0 for all p 1.
5. The Sidi [293] class Sm transformations: These transformations satisfy properties (a)
and (b). They also satisfy property (c) with p = m and with the renement
 (t)


i=0

ci t m+2i as t 0+,  (t)


i=0

ci (1 t)m+2i as t 1 .

420

25 Further Applications of Extrapolation Methods

It is this renement that makes class Sm transformations very useful, especially when
m is chosen to be an even integer. For the rigorous analysis of the trapezoidal and
midpoint rules following these transformations see [293].
A special representative of class Sm transformations, given already by Sidi [293],
is the sinm -transformation:
t
(t) =  01
0

(u) du
(u) du


m
, (t) = sin t .

Denoting (t) by m (t), it is easy to show by integration parts that the m (t) can be
computed recursively via


t 2
,
0 (t) = t, 1 (t) = sin
2
( m )
m (t) = m2 (t) 2 m+1 (sin t)m1 cos t, m = 2, 3, . . . .
2 ( 2 )
In particular, 2 (t) = t (sin 2t)/(2) is quite effective.
Additional transformations in Sm for some values of m can be obtained by composition of several transformations in Sm i . In particular, if  = 1 2 r with

i Sm i , then  S M with M = ri=1 (m i + 1) 1. Also, M is even if and only if
m i are all even. See [293].
Another transformation that is in the class Sm and similar to the sinm -transformation
(with even m only) was recently given by Elliott [76], and the analysis given by
Sidi [293] covers it completely. (Note that the lowest-order transformation of [76]
is nothing but the sin2 -transformation of [293].) Finally, Laurie [160] has given a
polynomial transformation that satises the rened property (c) partially. We refer
the interested reader to the original works for details.
It follows from the results of Sidi [293, p. 369, Remarks] that, when applied to regular
integrands, class Sm variable transformations with even m are much more effective than
the Korobov transformation with the same m, even though they both behave in the same
(polynomial) fashion asymptotically as t 0+ and as t 1. In addition, numerical
work with class Sm variable transformations is less prone to overows and underows
than that with the tanh-, IMT-, and the double exponential transformations that behave
exponentially as t 0+ and as t 1. See [293] for details.
Finally, we would like to mention that variable transformations were originally
considered within the context of multidimensional integration over the hypercube by
the so-called lattice methods. These methods can be viewed as generalizations of
the one-dimensional trapezoidal rule to higher dimensions. It turns out that the sinm transformations with even m are quite effective in these applications too. See, for example, Sloan and Joe [316] and Hill and Robinson [133] and Robinson and Hill [239]. Of
these, [316] is an excellent source of information for lattice methods.

25.2 Extrapolation of Numerical Solutions

421

25.2 Extrapolation of Numerical Solutions of Ordinary Differential Equations

Another useful application of the Richardson extrapolation process is to numerical solution of ordinary differential equations by nite-difference methods. We explain the
general idea via the solution of initial value problems by linear multistep methods.
Consider the initial value problem Y  = f (x, Y ) with Y (a) = y0 , whose solution we
denote Y (x). Let xn = a + nh, n = 0, 1, . . . , for some h > 0, and let {yn } be the numerical solution to this problem obtained by a linear multistep method with the xed
stepsize h. That is, yn Y (xn ), n = 1, 2, . . . .
Let us recall that a linear multistep method (of stepnumber k) is of the form
k

j=0

j yn+ j = h

k


j f n+ j ; f m f (xm , ym ),

j=0

for which the initial values y1 , . . . , yk1 should be provided by the user with sufcient
accuracy. Here j and j are xed constants and k = 0 and |0 | + |0 | = 0. When
k = 0, the method is said to be explicit; otherwise, it is implicit.
Let us x x such that x = a + mh. Let us also assume, for simplicity, that f (x, y) is
innitely differentiable with respect to x and y. Provided the linear multistep method
satises certain suitable conditions, there holds
ym Y (x) +


i=0

ci (x)h p+i as m and h 0, m

lim mh = x a. (25.2.1)
h0

Here p is some positive integer that depends on the local error of the linear multistep
method and the ci (x) are independent of h. Provided the method satises some symmetry condition, (25.2.1) assumes the rened form
ym Y (x) +


i=0

di (x)h p+2i as m and h 0, m

lim mh = x a. (25.2.2)
h0

When {yn } is generated by the Euler method, that is, yn+1 = yn + h f (xn , yn ), (25.2.1)
holds with p = 1. When {yn } is generated by the trapezoidal rule, that is, yn+1 = yn +
h
[ f (xn , yn ) + f (xn+1 , yn+1 )], then (25.2.2) holds with p = 2. The same is true when
2
the implicit midpoint rule is used, that is, when yn+1 = yn1 + h f ( xn +x2 n+1 , yn +y2 n+1 ).
(Note that this last method is not a linear multistep method.)
The existence of the asymptotic expansions in (25.2.1) and (25.2.2) immediately
suggests that the Richardson extrapolation process can be used to improve the accuracy
of the numerical solutions. Let us rst denote the yn by yn (h). Now we start with a stepsize
h 0 and xn = a + nh 0 , n = 0, 1, . . . , and apply the linear multistep method to compute
the approximations yn (h 0 ). We next apply the same method with stepsizes h i = h 0 /2i ,
i = 1, 2, . . . , to obtain the approximations yn (h i ). [Obviously, y2i n (h i ) Y (xn ) and
limi y2i n (h i ) = Y (xn ) for each xed n.] Finally, for each n = 1, 2, . . . , we apply

the Richardson extrapolation process to the sequence {y2i n (h i )}i=0

, to obtain better and
better approximations to Y (xn ), n = 1, 2, . . . .

422

25 Further Applications of Extrapolation Methods

Clearly, the most important research topics of this subject are (i) classication of
those difference methods that give rise to expansions of the forms described in (25.2.1)
and (25.2.2), and (ii) explicit construction of these asymptotic expansions. The rest is
immediate.
This interesting line of research was initiated by Gragg [104], [105]. [Before the work
of Gragg, the existence of the asymptotic expansions in (25.2.1) and (25.2.2) was tacitly
assumed.] Important contributions to this topic have been made by several authors. See,
for example, the works Stetter [323], [324], [325], and Hairer and Lubich [121]. See
also Stoer and Bulirsch [326], Marchuk and Shaidurov [206], and Hairer, Nrsett, and
Wanner [122]. For introductions and summaries, see also Lambert [157, Chapter 6] and
Walz [349, Chapter 3].
Asymptotic expansions of the forms given in (25.2.1) and (25.2.2) have also been
derived for difference solutions of two-point boundary value problems in ordinary differential equations and linear and nonlinear integral and integro-differential equations of
Volterra and Fredholm types. Again, the resulting numerical solutions can be improved
by applying the Richardson extrapolation process precisely as described here. We do not
consider these problems here but refer the reader to the relevant literature.

25.3 Romberg-Type Quadrature Formulas for Periodic Singular and

Weakly Singular Fredholm Integral Equations
25.3.1 Description of Periodic Integral Equations
The numerical solution of Fredholm integral equations
 b
K (t, x) f (x) d x = g(t), a t b,
f (t) +

(25.3.1)

is of practical interest in different disciplines. (Such equations are of the rst or the second
kind depending on whether = 0 or = 1, respectively.) In this section, we consider a
special class of such equations that arise from so-called boundary integral formulation of
two-dimensional elliptic boundary value problems in a bounded domain $. The integral
term in (25.3.1) in such a case is actually a line integral along the boundary curve $ of
the domain $. These equations have the following important features: (i) Their kernel
functions K (t, x) are singular along the line x = t. (ii) The input functions K (t, x) and
g(t) and the solution f (t) are all periodic with period T = b a. (iii) When the curve
$ is innitely smooth and the function g is innitely smooth along $, that is, when
g(t) C (, ), so is the solution f .
In case K (t, x) has an integrable singularity across x = t, (25.3.1) is said to be weakly
singular.
In case K (t, x) c/(x t) as x t for some constant c = 0, and the integral
b
K
(t,
x)
f (x) d x is dened only in the Cauchy principal value sense, it is said to be
a
singular.
Here we consider those integral equations with the following properties:
(i) The kernel K (t, x) is periodic both in t and in x with period T = b a and is innitely differentiable in (, ) \ {t + kT }
k= . It either has a polar singularity

25.3 Romberg-Type Quadrature Formulas

(PS) and can be expressed as in
H1 (t, x)
+ H2 (t, x), (PS),
K (t, x) =
x t

423

(25.3.2)

or it has an algebraic singularity (AS) and can be expressed as in

K (t, x) = H1 (t, x)|t x|s + H2 (t, x), s > 1, (AS),

(25.3.3)

or it has a logarithmic singularity (LS) and can be expressed as in

K (t, x) = H1 (t, x) log |t x| + H2 (t, x), (LS),

(25.3.4)

or has an algebraic-logarithmic singularity (ALS) and can be expressed as in

K (t, x) = H1 (t, x)|t x|s log |t x| + H2 (t, x), s > 1, (ALS), (25.3.5)
and H1 (t, x) and H2 (t, x) in all cases are innitely differentiable for all t, x [a, b]
(including t = x), but are not necessarily periodic. In all cases, we assume that
H1 (t, t)  0.
(ii) The function g(t) is periodic in t with period T and innitely differentiable in
(, ).
(iii) Finally, we assume that the solution f (t) exists uniquely and is periodic in t with
period T and innitely differentiable in (, ). (This assumption seems to hold
always. For a heuristic justication, see Sidi and Israeli [310, Section 1].)
A common approach to the numerical solution of integral equations is via so-called
quadrature methods, where one replaces the integral in (25.3.1) by a numerical quadrature
formula and then collocates the resulting equation at the abscissas xi of this formula, thus
obtaining a set of equations for the approximations to the f (xi ). When treating singular
and weakly singular integral equations, such an approach produces low accuracy because
of the singularity in the kernel. Here we would like to describe an approach due to Sidi
and Israeli [310] that is suitable precisely for such (periodic) integral equations and that
makes use of appropriate Romberg-type numerical quadrature formulas of high accuracy.
(See also Section D.6 of Appendix D, where the relevant EulerMaclaurin expansions
are summarized.)

25.3.2 Corrected Quadrature Formulas

Let n be a positive integer and set h = T /n and xi = a + i h, i = 1, . . . , n. In addition,
let t {x1 , . . . , xn }. Following Section D.6 of Appendix D, let us now
 b dene the corrected trapezoidal rule approximations I [h; t] to the integral I [t] = a K (t, x) f (x) d x
as in
n

wn (t, xi ) f (xi ),
(25.3.6)
I [h; t] =
i=1

where
wn (t, x) = h K (t, x) for x = t, h = T /n,

(25.3.7)

424
and

25 Further Applications of Extrapolation Methods

0
(PS)

h[H (t, t) 2 (s)H (t, t)h s ]

(AS)
2
1
(25.3.8)
wn (t, t) =
h )]

(t,
t)
+
H
(t,
t)
log(
h[H
(LS)
2
1

h[H2 (t, t) + 2{  (s) (s) log h}H1 (t, t)h s ] (ALS)

[Here, (s) and  (s) can be computed by accelerating the convergence of the series



z
z
and
log k for suitable z and by using Riemanns reection formula
k=1 k
k=1 k
when necessary. See Appendix E.] These I [h; t] have the asymptotic expansions
I [h; t] = I [t] + (t)h + O(h ) as h 0, for every > 0,
I [h; t] I [t] +

(PS)

(25.3.9)

k (t; s)h s+2k+1 as h 0,

(AS)

(25.3.10)

k (t; 0)h 2k+1 as h 0,

(LS)

(25.3.11)

k=1

I [h; t] I [t] +


k=1

I [h; t] I [t] +


[k (t; s) log h + k (t; s)]h s+2k+1 as h 0, (ALS) (25.3.12)
k=1

where (t), the k (t; s), and the k (t; s) depend only on t but are independent of h:


[H1 (t, x) f (x)]x=t ,

x

(s 2k) 2k
[H1 (t, x) f (x)]x=t , k = 1, 2, . . . ,
(25.3.13)
k (t; s) = 2
2k
(2k)!
x

 (s 2k) 2k
k (t; s) = 2
[H1 (t, x) f (x)]x=t , k = 1, 2, . . . .
2k
(2k)!
x
(t) = H2 (t, t) f (t)

From these expansions, it follows that I [h; t] I [t] is O(h) for PS, O(h s+3 ) for AS,
O(h 3 ) for LS, and O(h s+3 log h) for ALS.
When s is a positive even integer in (25.3.5) (ALS), say s = 2 p with p = 1, 2, . . . ,
we have k (t; 2 p) = 0 for all k. Therefore, in this case,
wn (t, t) = h[H2 (t, t) + 2  (2 p)H1 (t, t)h 2 p ],
I [h; t] I [t] +

k (t; 2 p)h 2 p+2k+1 as h 0,

(25.3.14)
(25.3.15)

k=1

from which we also have I [h; t] I [t] = O(h 2 p+3 ) as h 0. [The case that normally
arises in applications is that with s = 2, and the (ALS) formula I [h; t] in (25.3.6) with
(25.3.7) and (25.3.14), and with p = 1, has been used in a recent paper by Christiansen
[52]. In this case, we have I [h; t] I [t] = O(h 5 ) as h 0.]
Note that, in constructing I [h; t] for the weakly singular cases, we do not need to
know H1 (t, x) and H2 (t, x) for all t and x, but only for x = t. We can obtain H1 (t, t) and
H2 (t, t) simply by expanding K (t, x) for x t. For the singular case, neither H1 (t, t)
nor H2 (t, t) needs to be known.

25.3 Romberg-Type Quadrature Formulas

425

The quadrature method based on any of the rules I [h; t] is now dened by the equations
fk + I [h; xk ] = g(xk ), k = 1, 2, . . . , n.

(25.3.16)

More explicitly, these equations are

fk +

n


wn (xk , xi ) fi = g(xk ), k = 1, 2, . . . , n,

(25.3.17)

i=1

where fi is the approximation to f (xi ). In general, the accuracy of the fi is the same as
that of the underlying numerical quadrature formula, which is I [h; t] in this case. We can
increase the accuracy of the quadrature method by increasing that of I [h; t], which we
propose to achieve by using extrapolation. What makes this possible is the periodicity
of the integrand K (t, x) f (x) as a function of x. We turn to this subject next.

25.3.3 Extrapolated Quadrature Formulas

Treatment of the Singular Case
We start by illustrating this point for the case of PS, where one extrapolation sufces
to remove the single term (t)h from the expansion of I [h; t]. Let us choose h = T /n
for some even integer n and let xi = a + i h, i = 0, 1, . . . , n. Performing this single
extrapolation, we obtain the Romberg-type quadrature rule
J [h; t] = 2I [h; t] I [2h; t]

(25.3.18)

as the new approximation to I [t]. Consequently, we also have

J [h; t] = I [t] + O(h ) as h 0, for every > 0.

(25.3.19)

That is, as n , the error in J [h; t] tends to zero faster than any negative power of
n. The quadrature method for (25.3.1) based on J [h; t] is thus
fk + J [h; xk ] = g(xk ), k = 1, 2, . . . , n.

(25.3.20)

More explicitly,
fk + 2h

n


k,i K (xk , xi ) fi = g(xk ), k = 1, 2, . . . , n,

(25.3.21)

i=1

where we have dened


k,i =

1 if k i odd,
0 if k i even.

(25.3.22)

Surprisingly, J [h; t] is the midpoint rule approximation (with n/2 abscissas) for the
integral

I [t] =

t+T /2

tT /2


G(x) d x =

T /2

T /2

G(t + ) d ; G(x) K (t, x) f (x).

426

25 Further Applications of Extrapolation Methods

From symmetry, this can also be written as

 T /2
1
G e ( ) d ; G e ( ) [G(t + ) + G(t )].
I [t] =
2
T /2
Note that G e ( ) is periodic with period T and has no singularities in (, ). In view of
this, the error term given in (25.3.19) can be improved substantially in case the integrand
G(x) possesses certain analyticity properties. This improvement was given by Sidi and
Israeli [310, Theorem 9], and we include it here for completeness.
Theorem 25.3.1 Let K (t, z) f (z) G(z), for xed t and variable complex z, be analytic in the strip |!z| < for some > 0, except at the simple poles t + kT ,
k = 0, 1, 2, . . . . Dene


M( ) = max max |G e (x + i )|, max |G e (x i )| ,
<x<

<x<

where G e ( ) is as dened above. Then

|J [h; t] I [t]| 2T M(  )

exp(n  /T )
, for every  (0, ).
1 exp(n  /T )

Simply put, this theorem says that the error in J [h; t] tends to zero as n exponentially in n, like en /T , for all practical purposes.
Treatment of the Weakly Singular Case
For the case of LS, we start by using only one extrapolation to eliminate the term
1 (t; 0)h 3 from the asymptotic expansion of I [h; t]. Let us choose h = T /n for some
even integer n and let xi = a + i h, i = 0, 1, . . . , n. Performing this single extrapolation,
we obtain the Romberg-type quadrature rule
J1 [h; t] =

1
8
I [h; t] I [2h; t]
7
7

(25.3.23)

as the new approximation to I [t]. We also have

J1 [h; t] I [t] +


23 22k+1
k=2

k (t; 0)h 2k+1 as h 0,

(25.3.24)

hence J1 [h; t] I [t] = O(h 5 ) as h 0. The quadrature method for (25.3.1) based on
J1 [h; t] is thus
fk + J1 [h; xk ] = g(xk ), k = 1, 2, . . . , n.

(25.3.25)

More explicitly,
fk +


1 (1)
wn (xk , xi ) k,i wn/2 (xk , xi ) fi = g(xk ), k = 1, 2, . . . , n,
7
7

n 

8
i=1

(25.3.26)

25.3 Romberg-Type Quadrature Formulas

where

427

1 if k i even,
0 if k i odd.

(1)
k,i
=

(25.3.27)

By applying two extrapolations, we can remove the terms k (t; 0)h 2k+1 , k = 1, 2, from
the asymptotic expansion of I [h; t]. This time we choose h = T /n for an integer n that is
divisible by 4, and let xi = a + i h, i = 0, 1, . . . , n. Performing the two extrapolations,
we obtain the Romberg-type quadrature rule
32
1
J1 [h; t]
J1 [2h; t]
31
31
40
1
256
I [h; t]
I [2h; t] +
I [4h; t]
=
217
217
217

J2 [h; t] =

(25.3.28)

as the new approximation to I [t]. We also have

J2 [h; t] I [t] +


23 22k+1

k=3

25 22k+1
k (t; 0)h 2k+1 as h 0, (25.3.29)
31

hence J2 [h; t] I [t] = O(h 7 ) as h 0. The quadrature method for (25.3.1) based on
J2 [h; t] is thus
fk + J2 [h; xk ] = g(xk ), k = 1, 2, . . . , n.

(25.3.30)

More explicitly,
fk +

n 

256
i=1

217

wn (xk , xi )


40 (1)
1 (2)
k,i wn/2 (xk , xi ) +
k,i wn/4 (xk , xi ) fi = g(xk ),
217
217
k = 1, 2, . . . , n,

(1)
where k,i
are as before and
(2)
=
k,i

1 if k i divisible by 4,
0 otherwise.

(25.3.31)

(25.3.32)

For the development of Romberg-type formulas of all orders for all types of weak
singularities, we refer the reader to Sidi and Israeli [310].

25.3.4 Further Developments

Once the f i have been obtained, we can construct a trigonometric polynomial Pn (t)
in cos(2kt/T ), sin(2kt/T ), k = 0, 1, . . . , that satises the interpolation conditions Pn (xi ) = fi , i = 1, 2, . . . , n. As the fi are good approximations to the f (xi )
for all i = 1, . . . , n, we expect Pn (t) to be a good approximation to f (t) throughout
[a, b].
It is easy to see that the methodology presented here can be applied to systems of
periodic integral equations in several unknown functions, where the integral terms may

428

25 Further Applications of Extrapolation Methods

contain both singular and weakly singular kernels of the forms discussed. All these
kernels have their singularities only along x = t. Such systems occur very frequently in
applications.
In case g(t) 0 in (25.3.1), we have an eigenvalue problem. It is clear that our
methodology can be applied without any changes to such problems too.
The approach of Sidi and Israeli [310] to the solution of periodic singular and weakly
singular Fredholm integral equations outlined partially here has been used successfully
in different applications involving boundary integral equation formulations. See, for example, Almgren, Dai, and Hakim [5], Coifman et al. [55], Fainstein et al. [82], Haroldsen
and Meiron [124], Hou, Lowengrub, and Shelley [136], McLaughlin, Muraki, and
Shelley [209], Nie and Tian [219], Nitsche [221], Shelley, Tian, and Wlodarski [266],
and Tyvand and Landrini [342].
For the case of LS [with a special kernel K (t, x) only], Christiansen [51] derived a
numerical quadrature rule that has the same appearance as the rule I [h; t] given through
(25.3.6) and (25.3.7). It differs from our I [h; t] in its wn (t, t), which is more complicated
than our wn (t, t) in (25.3.8) (LS). The asymptotic expansion of the error in Christiansens
rule was given in Sidi [289], where it was shown that this rule too has an error that is
O(h 3 ) and its asymptotic expansion contains all the powers h 3+k , k = 0, 1, . . . . This
should be compared with the error of our I [h; t], whose asymptotic expansion has only
the odd powers h 3+2k , k = 0, 1, . . . . We are thus led to conclude that the Romberg-type
quadrature formulas based on the I [h; t] presented here will be more effective than those
based on the rule of Christiansen [51]. This conclusion has also been veried numerically
in Sidi [289].

25.4 Derivation of Numerical Schemes for Time-Dependent Problems

from Rational Approximations
Rational approximations have been used in deriving numerical schemes (nite differences, nite elements, etc.) for time-dependent problems. Let us consider the solution
of the problem
U
= AU,
t

(25.4.1)

where t denotes time and A is an operator independent of t. At least formally, we can

write



 k


(t)k
=
exp
t

U (t) = exp(t A)U (t).

U
(t)
U (t + t) =
t k
k!
t
k=0
(25.4.2)
Let us now approximate exp(z) by a rational function r (z) = p(z)/q(z), and use this to
replace (25.4.2) by the approximate equation
U (t + t) r (t A)U (t) q(t A)U (t + t) p(t A)U (t). (25.4.3)

25.4 Derivation of Numerical Schemes

429

K

L
If we write p(z) = i=0
ai z i and q(z) = i=0
bi z i , and denote U (k) U (kt), then
(25.4.3) can be written in the form
L


bi (t)i Ai U (k+1)

i=0

K


ai (t)i Ai U (k) .

(25.4.4)

i=0

Finally, replacing the operator A by a suitable approximation A if necessary, we obtain

from the approximate equation in (25.4.4) the numerical scheme
L


bi (t)i A i u (k+1) =

i=0

K


ai (t)i A i u (k) ,

(25.4.5)

i=0

where u (k) is an approximation for U (k) . We expect the numerical solution u (k) to have
high accuracy, provided r (z) and A are high-accuracy approximations for exp(z) and A,
respectively.
As an example, let us consider the one-dimensional heat equation
2U
U
= 2 , 0 < x < 1, t > 0,
t
x

(25.4.6)

subject to the boundary and initial conditions

U (0, t) = f 0 (t), U (1, t) = f 1 (t); U (x, 0) = g(x).

(25.4.7)

First, choosing r (z) = (1 + 12 z)/(1 12 z), the [1/1] Pade approximant to exp(z) for
which exp(z) r (z) = O(z 3 ), we obtain [cf. (25.4.4)]




1
1
2
(k+1)
I (t)A U
I + (t)A U (k) ; A 2 .
(25.4.8)
2
2
x
Letting x = 1/N and xi = ix, i = 0, 1, . . . , N , and denoting U (xi , t) = Ui (t) for
short, we next approximate A via the central difference of order 2
Ui+1 (t) 2Ui (t) + Ui1 (t)
2
Ui (t)
, i = 1, . . . , N 1.
x2
(x)2

(25.4.9)

Consequently, the equations in (25.4.5) become

1  (k+1)
1  (k)
(k+1) 
(k) 
= u i(k) + u i+1
,
2u i(k+1) + u i1
2u i(k) + u i1
u i(k+1) u i+1
2
2
t
, (25.4.10)
i = 1, . . . , N 1, k = 0, 1, . . . ;
(x)2
with the boundary and initial conditions
(k)
(0)
u (k)
0 = f 0 (kt), u N = f 1 (kt); u i = g(x i ), i = 0, 1, . . . , N . (25.4.11)

Here u i(k) is an approximation to Ui(k) = Ui (kt) = U (ix, kt). The resulting (implicit) nite-difference method is known as the CrankNicolson method and can be
found in standard books on numerical solution of partial differential equations. See, for
example, Ames [10] or Iserles [141]. It is unconditionally stable, that is, x and t can

430

25 Further Applications of Extrapolation Methods

be reduced independently. Let us express the difference equations in (25.4.10) in the

form
Fi,k (u) = 0,
where, with vi(k) = v(ix, kt) for arbitrary v(x, t),
Fi,k (v)

 (k)
(k)
(k)
(k+1)
(k+1) 
2vi(k+1) + vi1
vi+1
vi(k+1) vi(k)
vi+1 2vi + vi1
.
+
+
t
2
(x)2
(x)2

Then, the local truncation error of the CrankNicolson method is given as

Fi,k (U ) = O((x)2 ) + O((t)2 ).
[Recall that U (x, t) is the exact solution of (25.4.6).] Thus, the method is second-orderaccurate in both x and t.
Rational approximations, Pade approximants and others, to exp(z) have been used to
derive numerical methods for the solution of ordinary differential equations as well. For
such problems, the relevant rational approximations r (z) are required to be A-acceptable,
that is, they are required to satisfy |r (z)| < 1 for z < 0. (This is the same as requiring
that the corresponding numerical method be A-stable.) An important technique that has
been used in the study of A-acceptability is that of order stars, developed rst in Wanner,
Hairer, and Nrsett [350]. These authors applied the technique to Pade approximants
and showed that the [m/n] Pade approximants are A-acceptable if and only if m n
m + 2. These results were generalized to approximations other than Pade by Iserles
[140]. See also Iserles and Nrsett [142].

25.5 Derivation of Numerical Quadrature Formulas

from Sequence Transformations
Let (t) be a nondecreasing function on an interval [a, b] with innitely many points of
increase there, and let the integral
 b
f (x) d(x)
(25.5.1)
I[ f ] =
a

be dened in the sense of Stieltjes. Let us approximate this integral by the n-point
numerical quadrature formula
In [ f ] =

n


wn,k f (xn,k ),

(25.5.2)

k=1

where the xn,k and wn,k are, respectively, the abscissas and weights of this formula.
Furthermore, xn,k [a, b]. One way of viewing the numerical quadrature formula In [ f ]
is as follows: Let f (z) be analytic in a domain $ of the z-plane that contains the interval
[a, b] in its interior. Then, we can write


1
1
H (z) f (z) dz, In [ f ] =
Hn (z) f (z) dz,
(25.5.3)
I[ f ] =
2i C
2 i C

25.5 Derivation of Numerical Quadrature Formulas

431

where C is a closed contour that is in the interior of $ and that contains [a, b] in its
interior, and the functions H (z) and Hn (z) are dened via
 b
n

d(x)
wn,k
, Hn (z) =
.
(25.5.4)
H (z) =
z xn,k
a zx
k=1
We see that H (z) is analytic in the z-plane cut along the interval [a, b] and that Hn (z)
is a rational function of z, its numerator and denominator polynomials being of degree
n 1 and n, respectively. From the error expression

1
I [ f ] In [ f ] =
[H (z) Hn (z)] f (z) dz,
(25.5.5)
2i C
it is clear that, for In [ f ] to converge to I [ f ], Hn (z) must converge to H (z) in the z-plane
cut along the interval [a, b].
Now, H (z) has the asymptotic expansion
 b


i z i1 as z ; i =
x i d(x), i = 0, 1, . . . . (25.5.6)
H (z)
i=0

In view of (25.5.6), one effective way of approximating H (z) is via Pade approximants,
that is, by choosing Hn (z) = z 1 H n1,n (z 1 ), where H m,n ( ) is the [m/n] Pade approx

i i . As we saw in Section 17.9, the approximation
imant from the power series i=0
In [ f ] that results from this is nothing but the n-point Gaussian quadrature formula for
I [ f ], for which In [ f ] = I [ f ] for all f (x) that are polynomials of degree at most 2n 1.


i i are also obtained by applying the
Recall that Pade approximants from i=0

n
i i }
Shanks transformation to the sequence (of partial sums) { i=0
n=0 . This means
that numerical quadrature formulas other than Gaussian can be derived by applying to
this sequence suitable sequence transformations other than that of Shanks. For example,
we may use the Levin transformations or appropriate modications of them for this
purpose, provided {n } b(1) .
Let us rst observe that
 b n1
k1

x
1
d(x); Ak (z) =
i z i1 , k = 1, 2, . . . .
H (z) An1 (z) = n1
z
a zx
i=0
(25.5.7)
If the integral on the right-hand side of (25.5.7) has an asymptotic expansion of the form


cn i=0
i n i as n , then we can approximate H (z) by applying GREP(1) to the
sequence {An (z)} via the equations
Ar 1 (z) = A(nj) (z) +

n1
i
cr 
, r = j + 1, . . . , j + n + 1.
r
1
z
ri
i=0

(25.5.8)

[Here we have dened A0 (z) = 0.] The approximation A(0)

n (z) to H (z) is given by

n


i
n1
i=0 i z Ai (z)
i n (i + 1)

n
A(0)
;

=
(1)
, i = 0, 1, . . . , n. (25.5.9)
i
n (z) =
i
i
ci+1
i=0 i z

432

25 Further Applications of Extrapolation Methods

Obviously, when the ci are independent of z, so are the i , and hence A(0)
n (z) is a
rational function whose numerator and denominator polynomials are of degree n 1
abscissas
and n, respectively. Therefore, we can choose Hn (z) to be A(0)
n (z). Then, the

n
xn,k of In [ f ] are the poles of Hn (z) [equivalently, the zeros of the polynomial i=0
i z i ],
(0)
while the weights wn,k of In [ f ] are the residues of An (z) at the xn,k , namely,

n
wn,k =

i=1


i1

r 1 
i z i
r =0 r z


n
, k = 1, . . . , n,

i1
i
z
i
z=xn,k
i=1

(25.5.10)

provided the poles xn,k are simple. This approach to the derivation of numerical quadrature formulas was proposed by Sidi [275] for nite-range integrals, and it was extended to
innite-range integrals in Sidi [279], [282]. See also Davis and Rabinowitz [63, p. 307].
We show in the following how this approach is applied to nite-range integrals with
algebraic-logarithmic endpoint singularities.
When [a, b] = [0, 1] and d(x) = w(x)d x, where w(x) = (1 x) x ( log x) ,
+ > 1, > 1, we have that {n } b(1) indeed. By making the transformation
of variable x = et in (25.5.7), we obtain
H (z) An1 (z) =

1
z n1


0

ent

t (1 et ) et
dt.
z et

(25.5.11)

which, upon applying Watsons lemma, yields the asymptotic expansion

H (z) An1 (z)


1
u i (z)n i as n ,
z n1 n ++1 i=0

(25.5.12)

. Note that this asymptotic expansion is valid for all complex

where u 0 (z) = (++1)
z1
(++1)
.
z  [0, 1]. Thus, we can construct Hn (z) = A(0)
n (z) as in (25.5.9) with cn = n
As is shown by Sidi [275] for integer + and by Sidi and Lubinsky [314] for arbitrary

n
i z i of (25.5.9) has n simple real zeros xn,k in the open
+ , the polynomial i=0
interval (0, 1). Furthermore,
 1 these zeros are clustered near 0. (There are similar formulas
for integrals of the form 1 w(x) f (x) d x, with w(x) = (1 x 2 ) , for example, that have
their abscissas in (1, 1) and located symmetrically with respect to 0. See Sidi [275] for
details.)
The resulting formulas In [ f ] are remarkable in that their abscissas are independent
of and depend only on + . Thus, by setting cn = n 1 in (25.5.9) and (25.5.10), we
are able to obtain numerical quadrature formulas for w(x) = x for any . Furthermore,
cn = n 1 can also be used to obtain formulas for w(x) = x ( log x) , where is a
small integer such as 1, 2, . . . . All these formulas have the same set of abscissas.
More generally, by setting cn = n in (25.5.9) and (25.5.10), we are able to obtain
numerical quadrature formulas for w(x) = (1 x) x ( log x) , where is arbitrary
and + + 1 = + s, where s is a small integer like 1, 2, . . . . All these formulas too
have the same set of abscissas. Numerical experiments show that these rules are very
effective. Their accuracy appears to be comparable to that of the corresponding Gaussian
formulas, at least for moderate n.

25.6 Computation of Integral Transforms with Oscillatory Kernels

433

When we apply the formulas corresponding

to w(x) = 1 [that is, with cn = n 1
1
1
and n = (n + 1) ] to integrals I [ f ] = 0 f (x) d x, where f (x) has an integrable
singularity at x = 0 and is innitely differentiable everywhere in (0, 1], the new rules
seem to be more effective than the corresponding Gaussian rules, as suggested by the
numerical examples in [275]. In view of this observation, let us transform the variable
x via x = 2(u/2), where (t) is precisely as in Subsection 25.1.2. This results
1
in I [ f ] = 0 g(u) du, where g(u) f (2(u/2))  (u/2). In case (t) is innitely
differentiable on [0, 1], the transformed integrand g(u) is innitely differentiable on
(0, 1] and may have a singularity at u = 0 that is weaker than the singularity of f (x)
1
at x = 0. Thus, the new rules can be much more effective on 0 g(u) du than on
1
0 f (x) d x. (This idea was proposed by Johnston [143], who used Gaussian quadrature.
Johnston [143] also demonstrated via numerical experiments that class Sm variable
transformations are very appropriate for this purpose.)
25.6 Computation of Integral Transforms with Oscillatory Kernels
25.6.1 Via Numerical Quadrature Followed by Extrapolation
In Chapter 11, we considered in detail the application of various powerful extrapolation
D-,
s D-, and mW -transformations and their variants to integrals
methods such as the D-,

of the form I [ f ] = 0 f (t) dt, where f (x) = u(x)K (x), u(x) either does not oscillate
at innity or oscillates very slowly there, and the kernel function K (x) oscillates about
zero innitely many times as x with its phase of oscillation being polynomial
ultimately. Here we recall briey the main points of Chapter 11 on this topic.
To set the background, we begin with the use of (the variant of ) the mW -transformation
on such integrals. In this method, we rst choose a sequence of points xl , 0 < x0 <
or of K  (x). We next compute the intex1 < , as the
 xl+1consecutive zeros of K(x)
xl
grals (xl ) = xl f (t) dt and F(xl ) = 0 f (t) dt, l = 0, 1, . . . . Finally, we dene
( j)

the approximations Wn to I [ f ] via the linear systems

F(xl ) = Wn( j) + (xl )

n1

i
i=0

( j)

xli

, j l j + n.

These systems can be solved for the Wn very efciently via the W-algorithm, as described in Subsection 11.1.1.
 x0


Since (in case of convergence) I [ f ] =
k=0 ck , where c0 = 0 f (t) dt and ck =
(xk1 ), k = 1, 2, . . . , we can approximate I [ f ] by accelerating the convergence of


the series
k=0 ck . In view of the fact that this series is alternating, several sequence
transformations are effective for this purpose. For example, we can apply the iterated
2 -process, the iterated Lubkin transformation, the Shanks transformation (the
-algorithm), and the Levin t- and u-transformations successfully. This line of research
was begun by Longman [170], [171], who used the Euler transformation. Later the
use of the iterated Shanks transformations was demonstrated by Alaylioglu, Evans, and
Hyslop [3], and the use of the Shanks and Levin transformations was demonstrated in
the survey paper by Blakemore, Evans, and Hyslop [27].

434

25 Further Applications of Extrapolation Methods

In their surveys on the numerical computation of Bessel function integrals [the case
K (x) = J (x), > 0, > 0], Lucas and Stone [189] and Michalski [211] test a large
battery of methods and conclude that the mW -transformation and its variants are among
the most effective. In particular, the mW -transformation with equidistant xl , namely,
xl = x0 + l/, l = 0, 1, . . . , produces very good results for moderate values of . For
large values of , the mW -transformation, with the xl chosen as the zeros of K (x) or
K  (x), produces very high accuracy. The work of Lucas and Stone [189] is concerned
with the case of large values of . The numerical experiments of Sidi [299] show the

D-transformation
and its variants to be equally effective for small and for large .
In connection with Bessel function integrals, we would like to also mention that the
use of equidistant xl as above was rst proposed by Sidi [274], [281] in connection
and W -transformations. The same xl were later used by
with application of the DLyness [197] in connection with application of the Euler transformation.
Finally, we recall the variant of the mW -transformation given in Subsection 11.8.4,
which is dened via the linear systems
F(xl ) = Wn( j) + (xl )

n1

i=0

i
, j l j + n,
(l + 1)i/m

when xl i=0 ai l (1i)/m as l , a0 > 0. Here xl and (xl ) are as before. As

mentioned in Subsection 11.8.4, this method is comparable to other variants of the mW transformation as far as its performance is concerned, and it can also be implemented
via the W-algorithm.

25.6.2 Via Extrapolation Followed by Numerical Quadrature

We continue with other uses of sequence transformations that are relevant to Fourier
and inverse Laplace transforms. Let us consider the case in which K (x) is periodic with
period 2 and assume that K (x + ) = K (x) for all x 0. [This is the case when
K (x) = sin( x/ ), for example.] Let us choose xl = (l + 1) , l = 0, 1, . . . . We can
write



k
(1)
u(k + )K ( ) d.
I[ f ] =
0

k=0

Upon interchanging the order of integration and summation, we obtain




S( )K ( ) d, S( ) =
(1)k u(k + ).
I[ f ] =
0

k=0

Being an innite series that is ultimately alternating, S( ) canbe summed by an ap

propriate sequence transformation. Following that, the integral 0 S( )K ( ) d can be
approximated by a low-order Gaussian quadrature formula, for example.
The xl can be chosen to be consecutive zeros of K (x) or of K  (x) too. Obviously,
xl+1 xl = for all l. In this case, we can write

 x0


u( )K ( ) d +
(1)k
u(x0 + k + )K (x0 + ) d.
I[ f ] =
0

k=0

25.6 Computation of Integral Transforms with Oscillatory Kernels

435

Upon interchanging the order of integration and summation, we obtain


I[ f ] =
S( ) =

x0

u( )K ( ) d +

S( )K (x0 + ) d,

(1)k u(x0 + k + ).

k=0

Now we proceed as before.

25.6.3 Via Extrapolation in a Parameter


An extrapolation method for (convergent) oscillatory integrals I [ f ] = 0 f (t) dt, completely different from the ones discussed earlier, was given by Toda and Ono [335]. In

this method, one rst computes the integral A( ) = 0 e t f (t) dt, > 0, for a decreasing sequence of values k , k = 0, 1, . . . , tending to zero. (Toda and Ono choose
k = 2k .) Next, one applies the polynomial Richardson extrapolation to the sequence
{A(k )} assuming that A( ) has the asymptotic expansion
A( ) I [ f ] +

k k as 0+; k constants.

(25.6.1)

k=1

Intuitively,
this method is likely to be useful because,
 x t
 x for > 0, the integral
e
f
(t)
dt
converges
as
x

more
quickly
than
0
0 f (t) dt, thus allowing the
integral A( ) to be computed more easily than I [ f ] itself.
Toda
by example that their method works well for integrals such
 and Ono showed

as 0 sin t/t dt, 0 cos t/t dt, and 0 (cos t cos 2t)/t dt. As part of their method,
Toda and Ono also suggest that A( ) be approximated as follows: Letting t = u,

rst rewrite A( ) in the form A( ) = 0 eu f (u/ ) du/. Next, transform this integral by letting u = (v), where (v) = exp(v ev ), one of the
 double exponential formulas of Takahasi and Mori [333]. This results in A( ) = Q(v) dv, where
Q(v) = exp((v)) f ((v)/ )  (v)/ . Finally, this integral is approximated by the


trapezoidal rule T (h) = h
m= Q(mh). It is clear that the doubly innite summation
dening T (h) converges quickly for all small h.
An approach similar to that of Toda and Ono [335] was later used by Lund [194] in
computing Hankel transforms.
The theoretical justication of the method of Toda and Ono was given in a paper
by Sugihara [331]. This paper shows that limh0 T (h) = A( ) and that the asymptotic
expansion in (25.6.1) is valid for a quite general class of kernel functions that includes,
for example, K (x) = eix and K (x) = J (x) with integer . Here we provide a different
proof
 of the validity of (25.6.1) in the context of Abel summability, whether the integral
Sugihara [331].
0 f (t) dt converges or not. This proof is simpler than the one given
 by
t
We recall only the denition of Abel summability:
 If lim0+ a e g(t) dt = for
some nite , then we say that the integral a g(t)dt exists in the sense of Abel

summability and is its Abel sum. Of course, in case a g(t) dt exists in the ordinary
sense, its value and its Abel sum are the same. We begin with the following simple lemma.

436

25 Further Applications of Extrapolation Methods

Lemma 25.6.1 Let f C[a, b] and f  C(a, b] such that limxa+ f  (x) = . Then
f C 1 [a, b] as well, and f  (a) = .
Proof. By the mean value theorem, we have
f (x) f (a)
= f  (x ), for some x (a, x).
x a
Letting x a+ on both sides of this equality, and noting that limxa+ x = a, we have
lim

xa+

f (x) f (a)
= ,
x a

from which it follows that f  (x) is continuous at x = a from the right and that
f  (a) = .


Theorem 25.6.2 Let the function f (t) be such that | f (t)| is integrable on any nite
interval (a, x), a 0, and f (t) = O(t c ) as t , for
 some real constant c. In addition,
for each k = 0, 1, . . . , assume that the integrals a t k f (t) dt exist in the sense of
k
Abel summability,
 t and denote their corresponding Abel sums by (1) k . Dene also
A( ) = a e
f (t) dt, > 0. Then
A( ) I [ f ] +


k
k=1

k!

k as 0+; I [ f ] = 0 .

(25.6.2)

Proof. First, we realize that A( ) is the Laplace transform of H (t a) f (t), where H (x)
is the Heaviside unit step function, and that A( ) is an analytic function of for  > 0.

Next, we recall that A(k) ( ) = (1)k a e t t k f (t) dt for  > 0 and all k. It thus fol
lows from the Abel summability of the integrals a t k f (t) dt that lim 0+ A(k) ( ) = k ,
k = 0, 1, . . . . Let us now dene A(0) = 0 . This makes A( ) continuous at = 0 from
the right. Applying now Lemma 25.6.1 to A( ), we conclude that A ( ) is also continuous
at = 0 from the right with A (0) = 1 . By repeated application of this lemma, we conclude that A( ) is innitely differentiable at = 0 from the right and that A(k) (0) = k
for each k. Therefore, A( ) has the Maclaurin series
A( ) =

N 1

k
k=0

k!

k +

A(k) (  ) N
, for some  (0, ).
N!

By the fact that N is arbitrary and that lim 0+  = 0, we have that



N 1

N
k k N
A(N ) (0)
lim A( )

=
,
=
0+
k!
N!
N!
k=0
from which the result in (25.6.2) follows.

(25.6.3)

(25.6.4)

It is easy to see that Theorem 25.6.2 is valid for all functions f (x) in the class B
dened in Section 11.8.
A similar treatment for the case in which K (x) = eiq x , with real q, was given by
and Rice [190]. In this method, one rst computes the integral B( ) =
Lugannani
2 t 2 /2
e
f (t) dt for a decreasing sequence of values k , k = 0, 1, . . . , tending to
a

25.7 Computation of Inverse Laplace Transforms

437

 x 2 t 2 /2
f (t) dt converges as x
zero. Again, because,
for

>
0,
the
integral
0 e
x
more quickly than 0 f (t) dt, the integral B( ) can be computed more easily than I [ f ]
itself. Again, it can be shown that
B( ) I [ f ] +

k 2k as 0+; k constants.

(25.6.5)

k=1

As a result, we can apply the polynomial Richardson extrapolation process to the sequence {B(k )} to approximate I [ f ]. See Lugannani and Rice [190] for details.

25.7 Computation of Inverse Laplace Transforms

In this section, we discuss various methods, based on extrapolation processes and sequence transformations, that have been used in the numerical inversion of Laplace
transforms.

25.7.1 Inversion by Extrapolation of the Bromwich Integral

First, we concentrate on the extrapolation methods used in computing the Bromwich
is the Laplace transform of u(t), that is,
integral. In Section 11.8, we showed that if u(z)
zt

if u(z)
= 0 e u(t) dt, then the Bromwich integral for the inverse transform u(t) can
be expressed as in
ect
u(t+) + u(t)
=
2
2



+ i ) d +
ei t u(c


i ) d .
ei t u(c

Here c is real and we have assumed that u(z)

is analytic for z c. We mentioned

in Section 11.8 that, in case u(z)

= ezt0 u 0 (z) with u 0 (z) A( ) for some [and even
in more general cases where u 0 (z) oscillates very slowly as !z ], the mW transformation with l = (l + 1)/(t t0 ), l = 0, 1, . . . , is very effective.
The P-transformation developed by Levin [162] is very similar to the mW transformation but uses the points l = (l + 1), l = 0, 1, . . . . Because these points
do not take into consideration the fact that the phase of oscillation of the integrands
i ) is proportional to t t0 , this transformation will be costly when |t t0 |
ei t u(c
is too small or too large.

25.7.2 Gaussian-Type Quadrature Formulas for the Bromwich Integral

Expressing the Bromwich integral in the form
1
u(t+) + u(t)
=
u(t) =
2
2 ti

ct+i

/t) d,
e u(

(25.7.1)

cti

we can develop numerical quadrature formulas of Gaussian type for this integral. In

/t) = t s V ( ). Of course, V ( ) A(0) , and we can

case u(z)
A(s) , let us write u(

438

25 Further Applications of Extrapolation Methods

rewrite (25.7.1) in the form

u(t) =

1
2i

ct+i

w( )V ( ) d, w( ) = e s .

(25.7.2)

cti

Here w( ) is viewed as a (complex-valued) weight function. Salzer [255], [256], [257]

showed that it is possible to obtain numerical quadrature formulas of Gaussian type for
this integral that are of the form
u n (t) =

n


wn,k V (n,k ) = t 1

k=1

n


s
n,k /t)
u(
wn,k n,k

(25.7.3)

k=1

with complex abscissas n,k and weights wn,k . These formulas are obtained by demanding
that
 ct+i
n

1
wn,k p(n,k ) =
w( ) p( ) d,
(25.7.4)
2i cti
k=1
for all p( ) that are polynomials of degree at most 2n 1 in 1 . As shown by Zakian
[377], [378], their abscissas and weights are related to the partial fraction expansions of


k
the [n 1/n] Pade approximants Wn1,n ( ) from the power series W ( ) :=
k=0 k ,
where
 ct+i
1
1
, k = 0, 1, . . . .
(25.7.5)
w( ) k d =
k =
2 i cti
(s + k)
[Note that W ( ) converges for all and hence represents an entire function, which we
denote W ( ) as well. For s = 1, we have W ( ) = e .] Actually, there holds
Wn1,n ( ) =

n

k=1

wn,k
.
1 /n,k

(25.7.6)

It turns out that the abscissas n,k are in the right half plane z > 0. For even n, they
are all complex and come in conjugate pairs. For odd n, only one of the abscissas is real
and the rest appear in conjugate pairs. See Piessens [230].

ci t s1+i as t 0+.
Now the assumption that u(z)
A(s) means that u(t) i=0


si

as z , as can
With this u(t), we actually have that u(z)
i=0 ci (s + i)z
be veried with the help of Watsons lemma. Substituting this in (25.7.3), and invoking
(25.7.5), it is easy to show that
u n (t) u(t) = O(t s+2n1 ) as t 0 + .

(25.7.7)

In other words, the Gaussian formulas produce good approximations to u(t) for small to
moderate values of t.
For an extensive list of references on this subject, see Davis and Rabinowitz [63,
pp. 266270].
Note that rational approximations other than Pade approximants can be used to derive
numerical quadrature formulas of the form (25.7.3). Because the sequence {k } with k
as in (25.7.5) is in b(1) , we can use the Levin t- or the Sidi S-transformation to replace the
Pade approximants from the series W ( ). It turns out that the abscissas and weights of
the formulas that are obtained from the t-transformation grow in modulus very quickly,

25.7 Computation of Inverse Laplace Transforms

439

which ultimately causes numerical instabilities in nite-precision arithmetic. The ones

obtained from the S-transformation grow much more slowly; actually, their growth
rates are similar to those of the Gaussian-type formulas. (Interestingly, for s = 1 these
formulas are the same as the corresponding Gaussian-type formulas.) The abscissas and
weights that result from the S-transformation on the series W ( ) satisfy

n
Sn(0) ( )

i=0

 
n
r

i ni ri1
wn,k
=0 r
i n (i + 1)n1

n
=
; i = (1)
.
ni
i
1 /n,k
i
i=0 i
k=1
(25.7.8)

n

In other words, the n,k are the zeros of the polynomial i=0 i ni [equivalently, the
poles of Sn(0) ( )], and the wn,k are given by

i1

n

ni
r
1
i=0 i
r =0 r 
wn,k =

.
(25.7.9)
n
ni1 

=n,k
i=0 (n i)i
This approach to Laplace transform inversion was used in the M.Sc. thesis of
Shelef [265], which was done under the supervision of the author.

25.7.3 Inversion via Rational Approximations

We continue by describing a method for computing inverse Laplace transforms that

is based on rational approximations in the complex plane. Let u(z)

be the Laplace
transform of u(t). One way of approximating the inverse transform u(t) is by generating

and inverting each n (z) exactly

a sequence of rational approximations { n (z)} to u(z)
to obtain the sequence {n (t)}. Of course, we hope that {n (t)} will converge to u(t)
quickly in the z-plane. Also, as
quickly. For this to happen, { n (z)} should tend to u(z)

= 0, we must require that limz n (z) = 0 as well. This means that the
limz u(z)
degree of the numerator polynomial Pn (z) of n (z) should be strictly smaller than that
of its denominator polynomial Q n (z) for each n. Let the degree of Q n (z) be exactly n
and assume, for simplicity, that the zeros 1 , . . . , n of Q n (z) are simple. Then, n (z)
has the partial fraction expansion
n (z) =

n

k=1

Ak
Pn (k )
, k = 1, . . . , n,
; Ak = 
z k
Q n (k )

(25.7.10)

and n (t) is given by

n (t) =

n


A k e k t .

(25.7.11)

k=1

The rational functions n (z) can be obtained by applying a sequence transformation

is analytic.
to the Taylor series of u(z)
at a point z 0 in the right half plane where u(z)
Thus, we can use the [n 1/n] Pade approximants for this purpose. We can also use
the SidiLevin rational d-approximants when appropriate. In particular, we can use the
rational approximations obtained by applying the Levin L-transformation or the Sidi


S-transformation whenever this is possible. If z 0 = 0 and i=0
ci z i is the Taylor series

440

25 Further Applications of Extrapolation Methods

of u(z),
then the appropriate L-approximants are given by

n
L(0)
n (z) =

 
r
n1
i z ni ri1
=0 cr z
i n (i + 1)

n
;

=
(1)
.
i
ni
i
ci
i=0 i z

i=0

(25.7.12)

at z = 0, the n (t) will produce

Because the n (z) are obtained from information on u(z)
good approximations for large t.
assumes the form u(z)

Sometimes u(z)
= z 1 v(z), where v(z) is analytic at z = 0. In
1

this case, we choose n (z) = z vn,n (z), where vn,n (z) are the [n/n] Pade approximants
from the expansion of v(z) at z = 0. Again, other rational approximations can be used
instead of Pade approximants.

di z i1 as z , then we can choose n (z) = z 1 Un1,n (z 1 ),

If u(z)
i=0
where Un1,n ( ) is the [n 1/n] Pade approximant from the power series U ( ) :=




i
i
i=0 di . Because in this case u(t)
i=0 di t /i! as t 0+, it follows that
n (t) u(t) = O(t 2n ) as t 0 + .

(25.7.13)

We can also choose n (z) as a two-point Pade approximant from the expansions

u(z)
= i=0
ci z i and u(z)
i=0
di z i1 as z and invert these exactly to obtain
the n (t). In this case, we expect the n (t) to approximate u(t) well both for small and
for large values of t.

The preceding methods should be very practical in case u(z)

is difcult to compute
accurately, but its expansions as z 0 and/or z can be obtained relatively easily.
The general ideas were suggested by Luke and were developed further and used extensively (through continued fractions, Pade approximants, and rational approximations
from the Levin transformation) by Longman in the solution of problems in theoretical seismology and electrical circuit theory. See, for example, Luke [191], [192], and
Longman and [173], [175], [176], [177].
Two-point Pade approximants were used also by Grundy [117] but in a different
manner. In Grundys approach, the approximations n (t) are not sums of exponential
functions but two-point Pade approximants obtained from two asymptotic expansions of
the inverse transform u(t) in some xed power of t. One of these expansions is at t = 0
at z = , which we have already discussed. The other
and is obtained from that of u(z)

expansion is at some nite point t0 that is determined by the singularity structure of u(z).
For details and examples, see Grundy [117].

25.7.4 Inversion via the Discrete Fourier Transform

Finally, we mention a method that was originally introduced by Dubner and Abate [70]
and improved by Durbin [71] that uses the discrete Fourier transform. Taking u(t) to be
real for simplicity, Durbin gives the following approximation to u(t):


 







k
eat 1
kt
k
k t
+
u a + i
u(a)
cos
!u a + i
sin
.
u T (t) =
T 2
T
T
T
T
k=1
(25.7.14)

25.8 Simple Problems Associated with {an } b(1)

441

is analytic for z a. The error in this

Here a is a real constant chosen such that u(z)
approximation is given by
u T (t) u(t) =

e2kaT u(2kT + t),

(25.7.15)

k=1

and, provided |u(t)| C for all t, it satises

|u T (t) u(t)| Ce2aT for all t [0, 2T ].

(25.7.16)

We see from (25.7.16) that, by choosing aT large, we make the error in u T (t) small.
However, in general we can only approximate the sum of the Fourier series inside the
square brackets in (25.7.14), and the error we commit in this approximation is magnied
by the factor eat , which may be large when t is of order T . This suggests that we should
obtain the sum of this series as accurately as possible. This can be achieved by using
appropriate sequence transformations. Crump [58] uses the Shanks transformation. For
the same problem Kiefer and Weiss [151] also use summation by parts. Of course, the
d-transformation with the complex series approach and APS can be used to give very
high accuracy, as was done by Sidi [294]. [The latter is discussed in Chapter 12.] In this
case, it is easy to see that (25.7.14) can be rewritten as in
u T (t) =




 



1
t
k k
eat

 u(a)
+
u a + i
, exp i
, (25.7.17)
T
2
T
T
k=1

and this is very convenient for the complex series approach. When u(z)
A(s) for some
s, we can use the d (1) -transformation and other sequence transformations such as the

-algorithm. For more complicated u(z),

however, we can use only the d (m) - and Shanks
transformations.

25.8 Simple Problems Associated with {an } b(1)

In this section, we treat two simple problems associated with sequences {an } that are
in b(1) .
( )

1. Given that an = h(n) A0 strictly for some = 0, we would like to nd . We can

easily show that

an
ci
h(n) 

=
as n ; c0 = .
an
h(n)
n i+1
i=0

Thus, Un = n(an /an ) satises

Un +


ci
as n .
i
n
i=1

[Compare this with the asymptotic expansion given in (15.3.18).]

442

25 Further Applications of Extrapolation Methods

( )

2. Given that an = n h(n), with = 1 and h(n) A0 strictly for some , we would
like to nd and . This time, we have that




h(n)
ci
an+1
h(n + 1)
= 1+
+
=
as n .
Vn =
an
h(n)
h(n)
n i+1
i=0
Similarly, we can show that

Vn
ei
(h(n + 1)/ h(n)) 

=
as n ; e0 = .
i+2
Vn
h(n + 1)/ h(n)
n
i=0

Thus, Wn = n 2 (Vn /Vn ) satises

Wn +


ei
as n .
ni
i=1

It is thus clear that in the rst problem and and in the second problem can be
determined by applying the polynomial Richardson extrapolation to the sequences {Un },
{Vn }, and {Wn }. They can also be determined via the Levin u-transformation, the iterated
Lubkin transformation, the -algorithm, and the -algorithm.

25.9 Acceleration of Convergence of Innite Series with Special Sign Patterns

k/q
Let us rst consider innite series of the form
vk , where q is a positive ink=0 (1)
(r )
teger and {vn } b for some r . Note that the sequence of the factors (1)k/q comprises
q (+1)s, followed by q (1)s, followed by q (+1)s, and so on. We have already seen one


k/2
/(k + 1),
such example (with q = 2 and r = 1), namely, Lubkins series
k=0 (1)
2
in our discussion of the  -process in Chapter 15.
In case q 2 or r 2 or both, such series cannot be handled by the iterated
2 -process, the iterated Lubkin transformation, the Levin transformations, and the
-algorithm. When the term (1)n/q vn has no nonoscillatory part in its asymptotic
expansion as n , the transformations of Euler and of Shanks can be applied effectively. Otherwise, they are not useful. The d (m) -transformation with m = qr is always
effective, however. We now discuss the reason for this.
Denote ck = (1)k/q . Then the ck satisfy the linear (q + 1)-term
 recursion relation

q
cn+q + cn = 0, hence the linear difference equation cn = k=1 12 qk k cn . Therefore,

q
{cn } b(q) . In fact, cn is of the form cn = s=1 s sn , where s = exp[i (2s 1)/q],
s = 1, . . . , q, and the constants s are determined from the initial conditions c0 = =
cq1 = 1. As a result of this, we have that {(1)n/q vn } b(m) for some m qr , by
Heuristic 6.4.1. Hence, the d (qr ) -transformation is effective.
To illustrate
this, let us consider Lubkins series again. For this example, cn =
(1)n/2 = 2 cos(n/2 /4). Therefore, the term (1)n/2 /(n + 1) has no
nonoscillatory part as n . Consequently, the Euler, Shanks, and d (2) -transformations
are effective on this example. Others are not, as mentioned in Wimp [366, p. 171].
Incidentally, after some complicated manipulations involving the functions (x) =

(1)k
d
k=0 x+k and (x) = d x log (x), it can be shown for the generalization of Lubkins

25.10 Acceleration of Convergence of Rearrangements of Innite Series

443

series, where (1)k/2 is replaced by (1)k/q , that


(1)k/q
k=0

k+1

q1
1

k
log 2 +
, q = 1, 2, . . . ,
tan
q
2q k=1
2q

(25.9.1)

which seems to be new. This series can be summed very efciently by the d (q) transformation.

25.9.1 Extensions

(r )
Let us now consider the more general case of the series
k=0 ck vk , where {vn } b for
some r , as before, and the cn are either +1 or 1 [but not necessarily (1)n/q ], such
that cn+q = cn . This means that c0 , c1 , . . . , cq1 take on the values +1 or 1 in any
order, and the remaining ck satisfy cn+q = cn . (For example, with q = 4, the ck may be
such that |ck | = 1 and have the sign pattern + + + + + + + + .)
Since these cn satisfy the recursion cn+q + cn = 0 too, we see that {cn vn } b(m) for some
m qr , as before. Therefore, the d (m) -transformation can be applied with no changes
to these series.


Finally, the argument of the preceding paragraph is valid for those series
k=0 ck vk ,
where {vn } b(r ) for some r , as before, and the cn are either +1 or 1, such that
cn+q = cn . This means that c0 , c1 , . . . , cq1 take on the values +1 or 1 in any order,
and the remaining ck satisfy cn+q = cn , n = 0, 1, . . . . (For example, with q = 4, the ck
may be such that |ck | = 1 and have the sign pattern + + + + + + .) Since these
cn satisfy the recursion cn+q cn = 0 too, we see that {cn vn } b(m) for some m qr ,
as before. Therefore, the d (m) -transformation can be applied with no changes to these
series as well.

25.10 Acceleration of Convergence of Rearrangements of Innite Series

An interesting series that was considered by Lubkin [187, Example 2] is S :=
k=0 ak =
1/2 + 1 1/4 + 1/3 1/6 + 1/5 , which is a rearrangement of the series




k
T :=
k=0 u k =
k=0 (1) /(k + 1) = 1 1/2 + 1/3 1/4 + , the sum of T being log 2. Despite the fact that T converges conditionally, it is easy to see that the rearrangement S has log 2 as its sum too.
Since {(1)n /(n + 1)} b(1) , the series T can be summed very accurately by most
sequence transformations we discussed earlier, such as the iterated 2 -process, the iterated Lubkin transformation, the transformations of Euler, Shanks, and Levin, and the
-algorithm. None of these methods accelerates the convergence of the rearrangement
series S, as numerical computations show, however. The only method that is effective
on S appears to be the d (2) -transformation. The reason for this is that the sequence {an }
is in b(2) , as we show for a more general case next.


Let {u n } b(1) , and consider the rearrangement series S :=
k=0 ak = u 1 + u 0 +
u 3 + u 2 + u 5 + u 4 + . Thus,
a2k = u 2k+1 , and a2k+1 = u 2k , k = 0, 1, . . . .

444

25 Further Applications of Extrapolation Methods

We would like to show that {an } b(2) in the relaxed sense of Section 6.4, that is, that
there exists a 3-term recursion relation of the form
an+2 + (n)an+1 + (n)an = 0, n = 0, 1, . . . ,

(25.10.1)

with (n) A(r0 ) and (n) A(s)

0 for some integers r and s. Letting n = 2k and n = 2k + 1
in (25.10.1), the latter can be written in terms of the u n as follows:
u 2k+3 + (2k)u 2k + (2k)u 2k+1 = 0, n = 2k,
u 2k+2 + (2k + 1)u 2k+3 + (2k + 1)u 2k = 0, n = 2k + 1.

(25.10.2)

(q)

Now, we know that u n+1 = c(n)u n for some c(n) A0 , q an integer. Therefore, u n+2 =
c(n + 1)c(n)u n , u n+3 = c(n + 2)c(n + 1)c(n)u n , etc. Substituting these in (25.10.2), we
obtain the equations
c(2k + 2)c(2k + 1)c(2k) + (2k) + (2k)c(2k) = 0,
c(2k + 1)c(2k) + (2k + 1)c(2k + 2)c(2k + 1)c(2k) + (2k + 1) = 0. (25.10.3)
Letting 2k = n in the rst of these equations, and 2k + 1 = n in the second, we nally
obtain the following linear system for (n) and (n):
(n) + c(n)(n) = c(n + 2)c(n + 1)c(n)
c(n + 1)c(n)c(n 1)(n) + (n) = c(n)c(n 1)

(25.10.4)

(q)

Since c(n) A0 , q an integer, the elements of the matrix of these equations and the
)
right-hand-side vector are also in A(
0 for some integers . By Cramers rule, so are (n)
and (n), which implies that {an } b(2) .

25.10.1 Extensions

We can use this approach to treat other rearrangement series, S :=
k=0 ak , such as


u
with
even
index
followed by q
that obtained alternately from p terms of T :=
k=0 k
terms with odd index, namely, S := u 0 + u 2 + + u 2 p2 + u 1 + u 3 + + u 2q1 +
u 2 p + . Using the same technique, we can now show that {an } b( p+q) when
{u n } b(1) . As a result, the only acceleration method that is effective on such series
is the d ( p+q) -transformation. Thus, the d (3) -transformation sums the series 1 + 1/3
1/2 + 1/5 + 1/7 1/4 + very efciently to 32 log 2. The d (5) -transformation sums
the series 1 + 1/3 + 1/5 1/2 1/4 + 1/7 + 1/9 + 1/11 1/6 1/8 + very
efciently to log 2 + 12 log(3/2).
All this can be generalized in a straightforward manner to the case in which {u n } b (m) .
We leave the details to the reader.

25.11 Acceleration of Convergence of Innite Products

We now consider the efcient computation of some classes of innite products by extrapolation. Let {An } be the sequence of the partial products of the convergent innite product
n

A=
k=1 (1 + vk ). That is to say, An =
k=1 (1 + vk ), n = 1, 2, . . . . We propose to

25.11 Acceleration of Convergence of Innite Products

445

accelerate the convergence of the sequence {An }. As we show next, this can be achieved
provided the vk have suitable properties.


First, the innite product converges if and only if
k=1 vk converges. Next, let
(s/m,m)

strictly for some integer s, that is, vn

us assume that vn = w(n) A0


(s+i)/m
as n , 0 = 0, and convergence implies that s m + 1.
i=0 i n
Now, An = An1 (1 + vn ), from which we obtain An1 = vn An1 . If we let a1 = A1


and an = An An1 , n 2, so that An = nk=1 ak , n 1, then the latter equality
becomes an /vn = An1 . Applying  to both sides of this last equality, we
 have
(an /vn ) = an , which can be written in the form an = p(n)an , where p(n) = vn+1 +
1
(1,m) strictly, with p(n) (m/s)n
vn /vn . A careful analysis shows that p(n) A
0
as n . By Theorem 6.6.5, this implies that {an } b (m) ; hence, both the d(m) - and the
d (m) -transformations can be applied successfully. In addition, for best numerical results
we can use GPS. The iterated Lubkin transformation and the -algorithm can also be
applied to {An }.

25.11.1 Extensions


We now extend this approach to innite products of the form A =
k=1 (1 + ck vk ),
(s/m,m) strictly for some integer s, as before, and the cn are such
where vn = w(n) A
0
that cn+q = cn for all n. This means that c1 , . . . , cq take on arbitrary nonzero values,
and the remaining ck are determined by cn+q = cn .
As an illustration, let us consider the case in which the cn are either +1 or 1, such that
cn+q = cn . Two simple examples of this are cn = (1)n for q = 2, and cn = (1)n/2
for q = 4. Another more complex example with q = 5 is one in which the ck are such
that |ck | = 1 and have the sign pattern + + + + + + .
n+q
In this case, we have An+q = An k=n+1 (1 + ck vk ). Replacing n by qn, denoting
Aqn = An , and using the fact that cqn+k = ck for all k, this becomes
An+1 = An

qn+q


(1 + ck vk ) = An

k=qn+1

q

(1 + ck vqn+k ).
k=1

(s/m,m)

Now, by the fact that vn = w(n) A

0

, there holds

q


(s /m,m) strictly,
(1 + ck vqn+k ) = 1 + vn , vn = w  (n) A
0
k=1

q

q
for some integer s  s. (Note that s  = s when k=1 ck = 0, but s  > s for k=1 ck = 0.)
Thus, we have shown that {An } is the sequence of partial products of the innite product


A=
k=1 (1 + vk ) and that this innite product is precisely of the form treated in the
beginning of this section. Therefore, we can accelerate the convergence of {An } in exactly
the same form described there.
New nonlinear methods for accelerating the convergence of innite products of the


(s)
k
for some posiforms
k=1 (1 + vk ) and
k=1 [1 + (1) vk ], with vn = w(n) A0
tive integer s, have recently been devised in a paper by Cohen and Levin [54]. These
methods are derived by using an approach analogous to that used in deriving the

446

25 Further Applications of Extrapolation Methods

L- and the d (2) -transformations. Unlike the d-transformations, they require knowledge
of s, however.
One interesting problem treated by Cohen and Levin [54] is that of approximating

(s/m,m) strictly for
the limit of the product A(z) =
k=1 (1 + vk z), with vn = w(n) A0
some integer s m + 1, as before. Here A(z), as a function of z, vanishes at the points
z k = 1/vk , k = 1, 2, . . . , and we would like to nd approximations to A(z) that will
vanish at the rst zeros z 1 , . . . , z . Such approximations can be obtained as follows:

First, accelerate the convergence of the sequence of partial products of
k=+1 (1 + vk z)
( j)
via the d (m) - or d(m) -transformations. Call the resulting approximations An (z; ). Next,

( j)
approximate A(z) by [ k=1 (1 + vk z)]An (z; ). This procedure is analogous to the one
proposed in [54].

25.12 Computation of Innite Multiple Integrals and Series

The problem of accelerating the convergence of innite multiple series and integrals by
extrapolation methods has been of some interest recently.
The rst work in this eld seems to be due to Streit [327] who considered the generalization of the Aitken 2 -process to double series.
The rst paper in multiple power series acceleration was published by Chisholm [48].
In this paper, Chisholm denes the diagonal Pade approximants to double power se


c x i y j . These approximants are of the form f n,n (x, y) =
ries f (x, y) := i=0

nj=0
i nj

n
n
i j
i j
i=0
j=0 u i j x y /
i=0
j=0 vi j x y .
The nondiagonal approximants f m,n (x, y) were later dened by Graves-Morris,
Hughes Jones, and Makinson [109]. The diagonal approximants of [48] and the nondiagonal ones of [109] were generalized to power series in an arbitrary number of variables
by Chisholm and McEwan [50] and by Hughes Jones [137], respectively. General order
Pade approximants for multiple power series were dened by Levin [163] and further
developed by Cuyt [59], [60], [61].
A general discussion on accelerating the convergence of innite double series and
integrals was presented by Levin [164], who also gave a generalization of the Shanks
transformation. Generalizations of the Levin and the Shanks transformations to multiple
innite series were also considered in Albertsen, Jacobsen, and Srensen [4].
A recent paper by Greif and Levin [115] combines the general idea of Levin [164] with
an approach based on the D-transformation for one-dimensional innite-range integrals
and the d-transformation for one-dimensional innite series. Earlier, Sidi [268, Chapter 4,
Section 6] proposed an approach in which one uses of the d-transformation sequentially
in summation of multiple series and provided some theoretical justication for it at the
same time. The same approach can be used to compute multiple innite-range integrals.
In a more recent work by Levin and Sidi [166], multidimensional generalizations of
GREP and of the D- and d-transformations are reviewed and some new methods are
derived. We do not go into these methods here, as this would require new denitions and
notation. However, we do describe the approach of Sidi [268], because it is simple to
explain and use; it is very effective as well. Our description here is precisely that given
by Levin and Sidi [166].

25.12 Computation of Innite Multiple Integrals and Series

447

25.12.1 Sequential D-Transformation for s-D Integrals


Let us consider the s-dimensional (s-D) integral I [ f ] = IR s f (t) dt, where we have de0

noted IR s0 = {(t1 , . . . , ts ) : ti 0, i = 1, . . . , s}, t = (t1 , . . . , ts ), and dt = sj=1 dt j ,
and let us dene
H1 (t1 , . . . , ts ) = f (t) = f (t1 , . . . , ts ),

Hk+1 (tk+1 , . . . , ts ) =
Hk (tk , . . . , ts ) dtk , k = 1, . . . , s 1.
0

Then I [ f ] = 0 Hs (ts ) dts . Let us now assume that, for each k and for xed tk+1 , . . . , ts ,
and as a function of tk , Hk (tk , . . . , ts ) B(m k ) for some integer m k . [This assumption
seems to hold when f (t), as a function of the variable tk the rest of the variables being
held xed is in B(m k ) .] This means we can compute Hk+1 (tk+1 , . . . , ts ) by applying the

D (m k ) -transformation to the integral 0 Hk (tk , . . . , ts ) dtk . The computation

 of I [ f ] can
thus be completed by applying the D (m s ) -transformation to the integral 0 Hs (ts ) dts .
It is very easy to see that the preceding assumption is automatically satised when

f (x) = sj=1 f j (x j ), with f j B(m j ) for some integers m j . This then serves as the
motivation for sequential use of the D-transformation.
As an example, consider the function f (x, y) = eax u(y)/(x + g(y)), where a is
a constant with a > 0, u(y) B(q) , and g(y) A(r ) for some positive integer r and
g(y) > 0 for all large y. [We have q = 2 when u(y) = cos by or u(y) = J (by), for
(x, y) is in B(q)
example.] First, f (x, y) is in B(1) as a function of x (with xed y) and
 fc
as a function of y (with xed x). Next, invoking the relation 1/c = 0 e d , c > 0,
we can show that


H2 (y) =
f (x, y) d x = u(y)
e g(y) /(a + ) d.
0

Applying Watsons lemma to this integral, we see that H2 (y) has an asymptotic expansion
of the form
H2 (y) u(y)

i [g(y)]i1 u(y)

i=0

i y ir as y .

i=0

This implies that H2 (y) B(q) .

25.12.2 Sequential d-Transformation for s-D Series

Sequential use of the d-transformation for computing s-D innite series is analogous to
the use of the D-transformation for s-D integrals. Let us now consider the s-D innite


s
series S({ai }) =
iZZ s+ ai , where ZZ + = {(i 1 , . . . , i s ) : i j integer 1, j = 1, . . . , s},
and dene
L 1 (i 1 , . . . , i s ) = ai = ai1 ,... ,is ,
L k+1 (i k+1 , . . . , i s ) =


i k =1

L k (i k , . . . , i s ), k = 1, . . . , s 1.

448

25 Further Applications of Extrapolation Methods

Table 25.12.1: Numerical results by the
sequential d-transformation on the double
power series of Example 25.12.1
x

approximation

0.5
1.0
1.5
2.0
2.5

0.5
1.0
1.5
2.0
2.5

0.3843515211843
0.3149104237
0.26744390
0.2337732
0.207640

Hence, S({ai }) = is =1 L s (i s ). Let us assume that, for each k and for xed i k+1 , . . . , i s ,
the sequence {L k (i k , . . . , i s )}ik =1 is in b(m k ) for some integer m k . [This assumption
seems to hold when {ai }ik =1 b(m k ) , for each k and for i k+1 , . . . , i s xed.] Therefore,
we can compute L k+1 (i k+1 , . . . , i s ) by applying the d (m k ) -transformation to the series


of S({ai }) can be completed by applying the
i k =1 L k (i k , . . . , i s ), and the computation


d (m s ) -transformation to the series is =1 L s (i s ).
What motivates this approach to the summation of s-D series is the fact that the

( j)
( j)

preceding assumption is automatically satised when ai = sj=1 ai j , with {ai }i=1

b(m j ) for some integers m j .



j
As an example, consider the double series
j=1
k=1 a j,k , where a j,k = x u k /
(r )
(q)
( j + g(k)), where |x| < 1, {u k } b , and g(k) A0 for some positive integer r and
g(k) > 0 for all large k. [We have q = 2 when u k = cos k or u k = Pk (y), the kth

(1)
Legendre polynomial, for example.] First, {a j,k }
j=1 b with xed k, while {a j,k }k=1

b(q) with xed j. Next, invoking the relation 1/c = 0 ec d , c > 0, we can show
that



a j,k = xu k
e g(k) /(e x) d.
L 2 (k) =
j=1

Applying Watsons lemma, we can show that L 2 (k) has the asymptotic expansion
L 2 (k) u k

i [g(k)]i1 u k

i=0

i k ir as k .

i=0

Therefore, {L 2 (k)} b(q) .

The following examples have been taken from Sidi [268]:
Example 25.12.1 Consider the double power series



j=1 k=1

c j,k x j1 y k1 , c j,k =

j2

1
.
+ k3

(1)
(1)
and {c j,k x j1 y k1 }
Because {c j,k x j1 y k1 }
j=1 b
k=1 b , we apply the sequential
d-transformation with p = 1 and q = 1. Using about 100 terms of the series, in doubleprecision arithmetic, this method produces the results shown in Table 25.12.1.

25.13 A Hybrid Method: The RichardsonShanks Transformation

449

Note that the series diverges when |x| > 1 or |y| > 1, but the method produces its
sum very efciently. (The accuracy decreases as the rate of divergence increases, since
the absolute errors in the partial sums of the series increase in nite-precision arithmetic
in this case.) The series converges very slowly when |x| = 1 or |y| = 1, and the method
produces very accurate results for such x and y.
Example 25.12.2 Consider the double Fourier sine series







j x
k y
32
1
sin
, c j,k = 4
.
c j,k sin
U (x, y) =
2
2
a
b

jk( j /a + k 2 /b2 )
j=1 k=1
The function U (x, y) is the solution of the 2-D Poisson equation U = 2 for
(x, y) R, where R = {(x, y) : 0 < x < a, 0 < y < b}, with homogeneous boundary
conditions on R. Obviously, this double series converges very slowly. It is easy to see
(2)
(2)
that {c j,k sin( j x/a) sin(k y/b)}
and {c j,k sin( j x/a) sin(k y/b)}
j=1 b
k=1 b .
Therefore, we apply the sequential d-transformation with p = 2 and q = 2. Using about
400 terms of this series, we can obtain its sum to 13-digit accuracy in double-precision
arithmetic. The exact value of U (x, y) can easily be obtained from the simple series



n x
cosh[n(2y b)/(2a)]
8a 2 
sin
,
U (x, y) = x(a x) 3
n=1
n 3 cosh[nb/(2a)]
a
n odd

which converges very quickly for 0 < y < b.

25.13 A Hybrid Method: The RichardsonShanks Transformation

In Section 16.4, we saw via Theorems 16.4.3, 16.4.6, and 16.4.9 that the Shanks transformation is a very effective extrapolation method for sequences {Am } that satisfy (16.4.1)
and (16.4.2), namely, for
Am A +

k m
k as m ,

(25.13.1)

k=1

k distinct, k = 1 for all k; |1 | |2 | ; lim k = 0. (25.13.2)

k

We now describe a hybrid approach that reduces the cost of this transformation when
some of the largest k are available. We assume that 1 , . . . , s are available.
The rst step of this approach consists of eliminating the known k explicitly from
the asymptotic expansion of Am in (25.13.1), to obtain a new sequence { A m }. This is
achieved by applying to {Am } the Richardson extrapolation process as follows:
( j)

x0 = A j , j = 0, 1, . . . ,
( j+1)

x (pj) =

( j)

x p1 p x p1
1 p

, j = 0, 1, . . . , p = 1, 2, . . . , s.

A j = xs( j) , j = 0, 1, . . . .

(25.13.3)

450

25 Further Applications of Extrapolation Methods

From Theorem 1.5.1, it follows that the new sequence satises

A m A +

k m
k = k
k as m ;

k=s+1

s

k i
.
1 i
i=1

(25.13.4)

Thus, the Richardson extrapolation process functions as a genuine linear lter in this
step.
The second step consists of applying the Shanks transformation to { A m }, which will
( j)
be effective judging from (25.13.4). Let us denote the resulting approximations 2n by
( j)
2n ({Ak }). Then, Theorem 16.4.3 says that
( j)

2n ({Ak }) A = O(s+n+1 ) as j ,

(25.13.5)

provided |s+n | > |s+n+1 |.

It is clear that, formally speaking, the elimination of the terms k m
k , k = 1, . . . ,
s + n, from (25.13.1) by this procedure requires using of Am , j m j + s + 2n,
whereas application of the Shanks transformation directly to {Am } for eliminating the
same terms requires using Am , j m j + 2s + 2n.
We denote the procedure developed here the RichardsonShanks transformation
(R-EPS). This approach and its application as in the next subsection are due to Sidi
and Israeli [311]. Obviously, it can be applied using APS when necessary.

25.13.1 An Application
An interesting application of R-EPS is to problems in which a sequence {Bm } satises
Bm B +

s


k m
k +

k=1

k km as m ; |1 | |2 | , (25.13.6)

k=1

such that 1 , . . . , s are known, and we are required to determine

B(m)
B+

s


r rm .

(25.13.7)

r =1

This means we need to nd 1 , . . . , s , in addition to B.

First, B can be approximated by applying R-EPS with Am = Bm and k = k for
k = 1, . . . , s, in (25.13.1) and (25.13.2). Thus, the approximations to B produced by
( j)
R-EPS are 2n ({Bk }).
To approximate r , r {1, . . . , s}, we propose the following course of action: First,
multiply Bm by rm , r = 1, 2, . . . , and denote Bm rm = Am and r = A. Therefore,
 m 
 m
 m 
s

k
k
1
+
k
+
k
as m . (25.13.8)
Am A + B
r
r
r
k=1
k=1
k=r

Next, apply R-EPS to the sequence {Am } with k = k /r for 1 k s, k = r , and

( j)
r = 1/r , to obtain the approximations 2n ({Bk rk }). Then,
2n ({Bk rk }) r = O(|n+1 /r | j ) as j ,
( j)

provided |n | > |n+1 |.

(25.13.9)

25.13 A Hybrid Method: The RichardsonShanks Transformation

451

Once B and the r are approximated, we form

( j)
( j)
B n (m) 2n ({Bk }) +

s

r =1

2n ({Bk rk })rm
( j)

(25.13.10)

as an approximation to B(m).
When |n | > |n+1 |, we also have
( j)

= O(|n+1 /| j ) as j ; = min{1, |1 |, . . . , |s |}.

B n (m) B(m)
(25.13.11)

Sequences of the type described here arise, for example, when solving numerically
time-dependent problems with steady-state solutions that are periodic in time. This
may be the result of the periodicity being built directly into the associated equations
and/or of the presence of boundary conditions that are periodic in time. (The numerical
solutions involve marching in time with a xed time increment.) For such problems,
it turns out that |k | = 1 for k = 1, . . . , s, and |k | < 1 for all k = 1, 2, . . . , so that


steady
steady
+ o(1) as m , where Bm
= B + sk=1 k m
Bm = Bm
k is the numerical
steady
steady-state solution, and we are interested in determining Bm .
Important Remark. In some problems, we may be given not the whole sequence {Bm }
but only {Bm }
m= j for some possibly unknown j, and we may be asked to approxi
mate B(m). This can be achieved by applying R-EPS to the sequence {Bk }
k=0 , where
( j)
Bk = B j+k , k = 0, 1, . . . , exactly as before. Then, with B n (m) as dened in (25.13.10),
there holds
s

( j)
(0)
(0)
B n ( j + p) = 2n
({Bk }) +
2n
({Bk rk })rp .
(25.13.12)
r =1

The key to this is the fact that R-EPS, being a composition of two quasi-linear processes, the Richardson extrapolation and the Shanks transformation, is itself quasi( j)
linear too. That is, for any two constants a = 0 and b, there holds 2n ({a Ak + b}) =
( j)
a 2n ({Ak }) + b. Therefore,
(0)
({Bk rk }), r = 1, . . . , s. (25.13.13)
2n ({Bk rk }) = r j 2n ({Bk rjk }) = r j 2n
( j)

( j)

In addition,
(0)
({Bk }).
2n ({Bk }) = 2n
( j)

(25.13.14)

The validity of (25.13.12) is now a consequence of (25.13.10), (25.13.13), and (25.13.14).

We would like emphasize that the procedure described here does not require knowledge
1
2
of j because {Bk rk }
k=0 = {B j , r B j+1 , r B j+2 , . . . } in (25.13.12).
Example 25.13.1 As a model, consider the linear system of ordinary differential equations
y (t) = Cy(t) + f(t), t > 0; y(0) = y0 .

(25.13.15)

Here y(t) = [y1 (t), . . . , yr (t)]T , and C is a constant r r matrix whose eigenvalues have
negative real parts, and the forcing function f(t) = [ f 1 (t), . . . , fr (t)]T is periodic with

452

25 Further Applications of Extrapolation Methods

period . Using the fact that

y(t) = eCt y0 +

eC(ts) f(s) ds,

(25.13.16)

it can be shown that y(t) = ytrans (t) + ysteady (t), where ytrans (t) is the transient, that is, it
satises limt ytrans (t) = 0, and ysteady (t) is periodic with period . Let us now solve
this system numerically by a linear multistep method with a xed time step t = h. For
simplicity of presentation, we use the Euler method, namely,
ym+1 = Qym + hfm , m = 0, 1, . . . ; Q = I + hC, fm = f(mh), ym y(mh).
(25.13.17)
Here we choose h = /M, M = 2K + 1 for some integer K . Then, fm+ pM = fm for
integers p. We also choose h small enough so that all eigenvalues of Q are smaller than
unity in modulus; hence, the matrix cI Q, where |c| = 1, is nonsingular. By induction,
it follows that
ym = Q m y0 + h

m1


Q m1 j f j , m = 0, 1, . . . .

(25.13.18)

j=0
K
be the discrete Fourier transform of {f j }2K
Letting {gk }k=K
j=0 , we can write

fj =

K



gk jk , j = 0, 1, . . . ; = exp i

k=K


2
.
2K + 1

(25.13.19)

Of course, just like f j , gk are r -dimensional vectors. Substituting (25.13.19) in (25.13.18),

we can show after some manipulation that

ytrans
m

ym = ytrans
+ ysteady
;
m
m


K
K


= Q m y0 h
(k I Q)1 gk , ysteady
=h
(k I Q)1 gk km .
m
k=K

k=K

(25.13.20)
Obviously, ytrans
is the transient since limm ytrans
= 0 by the fact that the eigenvalues
m
m
steady
is the periodic steady-state solution
of Q are all less than unity in modulus, and ym
steady
steady
for every integer p. If we assume, for simplicity, that C is dibecause ym+ pM = ym
agonalizable, then Q is diagonalizable as well. Let us denote its distinct eigenvalues by

q
= k=1 dk km , where dk is an eigenvector correk , k = 1, . . . , q. Then we have ytrans
m

K
steady
= k=K
bk (k )m , where bk = h(k I Q)1 gk
sponding to k . We also have ym
for each k. The vectors bk and dk are independent of m. Combining all this in (25.13.20),
we nally have that
ym =

K

k=K

bk (k )m +

q


dk km .

(25.13.21)

k=1

We now apply R-EPS to the sequence {ym } componentwise to determine the vectors bk ,
steady
K k K , hence ym . It is clear that, for errors that behave like |n+1 | j , we need
to store and use the vectors ym , m = j, j + 1, . . . , j + 2K + 2n.

25.14 Application of Extrapolation Methods to Ill-Posed Problems

453

Let us consider the special case in which

f(t) =

L


ck exp(i 2kt/ )

k=L

= c0 +

L
L


(ck + ck ) cos(2k t/ ) +
i (ck ck ) sin(2k t/ ),
k=1

k=1

for some integer L. Choosing K L, we have in this case f j =

0, 1, . . . , with as in (25.13.19). Consequently,
ym =

L


bk (k )m +

k=L

q


dk km .

L
k=L

ck jk , j =

(25.13.22)

k=1

If L is considerably small compared with K , then the cost of applying R-EPS to the
sequence {ym } in (25.13.22) is much smaller than that incurred by applying R-EPS to
the sequence in (25.13.21). For errors that behave like |n+1 | j , this time we need to store
and use the vectors ym , m = j, j + 1, . . . , j + 2L + 2n.
Such a situation can also arise (approximately) in case f(t) is innitely differentiable
on (, ). Now, f(t) can be approximated with very high accuracy via a small number

L
ck exp(i 2kt/ ) with a small L, so
of terms of its Fourier series, namely, f(t) k=L
that
ym

L


bk (k )m +

k=L

q


dk km .

k=1

A different approach to the problem described here was suggested by Skelboe [315]. In
steady
is obtained by applying the Shanks transformation to the sequences
this approach, ym
steady

,
{y Mm+i }m=0 , i = 0, 1, . . . , M 1. The limits of these sequences are, of course, yi
steady
completely. With large M, this api = 0, 1, . . . , M 1, and these determine ym
proach seems to have a larger cost than the R-EPS approach we have proposed.

25.14 Application of Extrapolation Methods to Ill-Posed Problems

An interesting application of extrapolation methods to ill-posed problems appears in
Brezinski and Redivo Zaglia [41, Section 6.3]. Consider the linear system
Ax = b,

(25.14.1)

where the matrix A is nonsingular. As is well known, the numerical solution of this
system will have poor accuracy if the condition number of A is large. To avoid this
problem partially, we solve instead of (25.14.1) the perturbed problem
(A +  B)x() = b

(25.14.2)

with some small , such that the matrix A +  B is better conditioned. This is the socalled Tikhonov regularization technique. Of course, x() may be quite different from
x = x(0). The problem now is how to apply this technique and still obtain a reasonable
numerical approximation to x.

454

25 Further Applications of Extrapolation Methods

From (25.14.2), we have

x() = (A +  B)1 b = (I +  M)1 x, M = A1 B.

(25.14.3)

Provided  < 1/(M), where (M) is the spectral radius of M, we obtain from (25.14.3)
the convergent expansion
x() = x +


[(1)k M k x] k .

(25.14.4)

k=1

It is now easy to see from (25.14.4) that we can apply the polynomial Richardson extrapolation or the rational extrapolation to x() componentwise to obtain approximations to x.
Practically, the methods are applied to a sequence {x(m )}, where 0 > 1 > , and
limm m = 0.
Ill-posed problems arise frequently from the discrete solution of Fredholm integral
equations of the rst kind. They may also arise in the solution of the least squares
problems
minimize Ax b,
where, this time, A may be a rectangular matrix of full column rank. If this problem is
replaced by a regularized one of the form
minimize (Ax b2 + Bx2 ),

where B is some suitable matrix (for example, B = I ), and y = y y is the standard
Euclidean norm, then the solution x() to this problem can be shown to satisfy the linear
system
(A A +  B B)x() = A b
that is precisely of the form discussed above.
For numerical examples and for additional applications of similar nature, including
the relevant bibliography, we refer the reader to Brezinski and Redivo Zaglia [41].

25.15 Logarithmically Convergent Fixed-Point Iteration Sequences

We close this chapter by mentioning a rare and yet interesting type of sequences that
arise from xed-point iterative methods applied to a nonlinear equation x = g(x), for
which g  (s) = 1, where s denotes the solution. (Recall that we discussed the case in
which |g  (s)| < 1 when we studied the 2 -process.) These sequences have been studied
in detail in de Bruijn [42, pp. 153175].
Let us pick x0 and compute x1 , x2 , . . . , according to xm+1 = g(xm ). Let us denote em =
xm s, m = 0, 1, . . . . Assume that g(x) is innitely differentiable in a neighborhood
of s, so that
g(x) s = u +

p+i u p+i , p 1 integer, p+1 < 0, u x s > 0.

i=1

(25.15.1)

25.15 Logarithmically Convergent Fixed-Point Iteration Sequences

455

Then, provided x0 > s is sufciently close to s, {xm } converges to s. Actually, {xm }

converges logarithmically, and there holds xm s ( p p+1 m)1/ p as m .
As follows from [42, p. 175, Exercise 8.11], the equation
(g (u)) = 1 + (u), g (u) g(s + u) s,

(25.15.2)

has a unique solution (u) that has an asymptotic expansion of the form
(u)

p


ci u i + c0 log u +

i=1

ci u i as u 0+, c p = 0. (25.15.3)

i=1

The coefcients ci are determined by the expansion of g(x) in (25.15.1) and from
(25.15.2).
The asymptotic behavior of xm can be obtained by analyzing the relation
(em ) = m + (e0 ), m = 1, 2, . . . .

(25.15.4)

Taking s = 0 for simplicity, Sablonni`ere [246] gives the following results for the cases
p = 1, 2:
When p = 1,
xm s +

k m k as m ; (c0 = 0),

k=1

xm s +


k1

k=1


k,i (log m)i m k as m ; (c0 = 0),

i=0

and when p = 2 and g(x) is odd,

xm s +

k m k+1/2 as m ; (c0 = 0),

k=1

xm s +


k1

k=1


k,i (log m)i m k+1/2 as m ; (c0 = 0).

i=0

When p = 2 and 4 = 0, hence g(x) is not necessarily odd,



k1


i
k,i (log m) m k+1/2
xm s +
k=1


k1

i=0

k,i
(log m)i


m


as m ; (c0 = 0).

i=0

The k , k,i , etc. depend on the k . Explicit expressions for the rst few of them are
given by Sablonni`ere [246].
We see that whether c0 vanishes or not makes a big difference in the nature of the
asymptotic expansion of xm . When c0 = 0, xm has powers of log m in its asymptotic
expansion, and this will surely cause problems when convergence acceleration methods
are applied to {xm }. Surprisingly, numerical and theoretical results given by Sablonni`ere
[246], [247] and Sedogbo [262] show that, when c0 = 0, the iterated Lubkin transformation and the -algorithm are very effective when applied to {xm }. For example, when

456

25 Further Applications of Extrapolation Methods

p = 1, the kth column in these methods will eliminate all the terms in the summation

k1
k,i (log m)i ]m k simultaneously. The iterated 2 -process and the -algorithm,
[ i=0
when modied suitably as described in [246], [247], and [262], perform similarly. We
refer the reader to these papers for more results and details.
Before we end, we recall that g  (s) = 1 implies that s is a multiple root of the equation
f (x) = x g(x) = 0. It is known that, when applied to f (x) = 0, the Newton-Raphson
method converges linearly. It can even be modied slightly to converge quadratically.
See, for example, Ralston and Rabinowitz [235]. [The sequence {xm } generated via the
xed-point iterations xm+1 = g(xm ) treated here converges logarithmically, hence it is
inferior to that generated by the Newton-Raphson method.]

Part IV
Appendices

A
Review of Basic Asymptotics

As the subject of extrapolation methods depends heavily on asymptotics, it is important and

useful to review some of the fundamentals of the latter. It is correct to say that, without a good
understanding of asymptotics, it is impossible to appreciate the beauty, relevance, and intricacies,
both mathematical and practical, of the research that has been done in the area of extrapolation
methods.
For thorough treatments of asymptotic methods, we refer the reader to the books by Olver [223],
Murray [215], Erdelyi [77], de Bruijn [42], and Bender and Orszag [21]. See also Henrici [132,
Chapter 11].

A.1 The O, o, and Symbols

Denition A.1.1 Let x0 0 be nite or innite. Let f (x) and g(x) be two functions dened on an
interval I that is (i) of the form (x0 , x0 + ), > 0, when x0 is nite and (ii) of the form (X, ),
X > 0, when x0 is innite. Finally, let limxx0 stand for limxx0 + when x0 is nite and for limx
when x0 is innite.
(i) We write
f (x) = O(g(x)) as x x0 if | f (x)/g(x)| is bounded on I.
In words, f is of order not exceeding g.
(ii) We write
f (x) = o(g(x)) as x x0 if lim f (x)/g(x) = 0.
xx0

In words, f is of order less than g.

(iii) We write
f (x) g(x) as x x0 if lim f (x)/g(x) = 1.
xx0

In words, f is asymptotically equal to g.

Special cases of these denitions are f (x) = O(1) as x x0 , which means that f (x) is bounded
as x x0 , and f (x) = o(1) as x x0 , which means that f (x) 0 as x x0 .
The notations O(g) and o(g) can also be used to denote the classes of functions f that satisfy
f = O(g) and f = o(g), respectively, or to denote generically unspecied functions f with these
properties.
Thus, in terms of the o-notation,
f (x) g(x) as x x0 if and only if f (x) = g(x)[1 + o(1)] as x x0 .
Here are some important consequences of Denition A.1.1:
1. Obviously, f = o(g) implies f = O(g), but the converse of this is not true.
2. We must also realize that f = O(g) does not imply g = O( f ). As a consequence, f = O(g)
does not imply 1/ f = O(1/g) either. The latter holds provided g = O( f ).

459

460

A Review of Basic Asymptotics

3. On the other hand, if f g, then g f too. Of course, we have 1/ f 1/g as well. Clearly,
f g implies both f = O(g) and g = O( f ), and hence both 1/ f = O(1/g) and 1/g =
O(1/ f ).
4. From what we have seen so far, we understand that the amount of information contained
in f g is largest; this is followed by f = O(g) and g = O( f ) simultaneously, which is
followed by f = O(g) only. Also, f = o(g) contains more information than f = O(g). In
case g = o(h), the relation f = O(g) contains more information than f = o(h). Thus, as
x 0+, f (x) x 3 is better than cx 3 | f (x)| d x 3 for some c > 0 and d > 0, which is
better than f (x) = O(x 3 ), which is better than f (x) = o(x 2 ).
If the functions f (x) and g(x) in Denition A.1.1 are replaced by the sequences {an } and {bn },
we can analogously dene an = O(bn ), an = o(bn ), and an bn as n . Only this time, we
should keep in mind that n through integer values only.

A.2 Asymptotic Expansions

Denition A.2.1 The sequence of functions {k (x)}
k=0 is called an asymptotic sequence or an
asymptotic scale as x x0 if
k+1 (x)
= o(1) as x x0 , k = 0, 1, . . . .
k (x)
Two most important examples of asymptotic sequences are (i) {(x x0 )k } when x0 is nite, and
(ii) {x k } when x0 = .


Denition A.2.2 We say that the (convergent or divergent) series
k=0 ak k (x) represents the
function f (x) asymptotically as x x0 [or that it is an asymptotic expansion of f (x) as x x0 ],
and write


f (x)
ak k (x) as x x0 ,
k=0

if

{k (x)}
k=0

is an asymptotic sequence and if for every integer n 0 there holds

f (x)

n1


ak k (x) = O(n (x)) as x x0 .

k=0

Theorem A.2.3 Let {k (x)}

k=0 be an asymptotic sequence. Then the following three statements
are equivalent:


(i) The innite expansion
k=0 ak k (x) represents the function f (x) asymptotically as x x 0 ;
i.e.,


ak k (x) as x x0 .
f (x)
k=0

(ii) For every integer n 0, there holds

n

ak k (x) = o(n (x)) as x x0 .
f (x)
k=0

(iii) For every integer n 0,

lim

xx0

f (x)

n1
k=0

n (x)

ak k (x)

exists and equals an .

When x0 is nite, the sequence {(x x0 )k }

as already
mentioned.
k=0 is asymptotic as x x 0 ,

f (k) (x0 )
If f C [x0 , x0 + ], > 0, then its Taylor series about x0 , namely,
(x x0 )k ,
k=0
k!
whether convergent or not, represents f (x) asymptotically as x x0 +. This can be proved by

A.3 Operations with Asymptotic Expansions

461

using the Taylor series with the remainder. As is known, if f (x) is analytic at x0 , then its Taylor
series at x0 converges when |x x0 | < for some > 0 and f (x) is equal to the sum of this
series when |x x0 | < . In case f (x) is not analytic at x0 , its Taylor series at x0 diverges for
all x = x0 .
All this remains valid when x0 = , provided we replace the sequence {(x x0 )k }
k=0 by
{x k }
k=0 , and make other suitable modications.


An immediate consequence of the preceding theorem is that, if the series k=0 ak k (x) is an
asymptotic expansion of the function f (x) as x x0 , then it is unique. (The converse is not necessarily true; that is, more than one function may be represented
same asymptotic expansion.

by the
k
The question of the uniqueness of f (x) when f (x)
as x is discussed at
k=0 ak x
length in Hardy [123, Section 8.11], where it is assumed that f (z) is analytic in some sector of the
z-plane.)

A.3 Operations with Asymptotic Expansions

The following results on the algebra of asymptotic expansions can be proved with Theorem A.2.3.




Theorem A.3.1 Let f (x)
k=0 ak k (x) and g(x)
k=0 bk k (x) as x x 0 . Then, for arbitrary constants and , there holds
f (x) + g(x)


(ak + bk )k (x) as x x0 .
k=0

Theorem
Let k (x) = (x

x0 )k when x0 is nite and k (x) = x k when x0 = . Let

A.3.2

f (x) k=0 ak k (x) and g(x)

k=0 bk k (x) as x x 0 . Then the following are true:
(i) The product f (x)g(x) is represented asymptotically as x x0 by the Cauchy product of the
asymptotic expansions of f (x) and g(x). That is,


k

aki bi k (x) as x x0 .
f (x)g(x)
k=0

i=0

(ii) If a0 = 0, then the function g(x)/ f (x) has an asymptotic representation given by
g(x)/ f (x)

dk k (x) as x x0 ,

k=0

where the dk are determined recursively from the equations

k


aki di = bk , k = 0, 1, . . . .

i=0

In particular, the function 1/ f (x) has an asymptotic representation given by

1/ f (x)

ck k (x) as x x0 ,

k=0

where the ck are determined recursively from the equations

c0 a0 = 1 and

k


aki ci = 0, k = 1, 2, . . . .

i=0

In the next theorem, we discuss the asymptotic expansion of the composition of two functions.
For the sake of clarity, we restrict ourselves only to the case in which x0 = and k (x) = x k ,
k = 0, 1, . . . .

462

A Review of Basic Asymptotics

Theorem A.3.3 Assume that f (x) a1 x 1 + a2 x 2 + as x . Assume also that g(w)

b0 + b1 w + b2 w2 + as w 0. Then the function g( f (x)) admits an asymptotic expansion of
the form
g( f (x))

dk x k as x .

k=0

for some constants dk . In particular, we have d0 = b0 , d1 = b1 a1 , d2 = b1 a2 + b2 a12 , etc.

We now make a few important remarks about other operations with asymptotic expansions such
as integration and differentiation.


k
Theorem A.3.4 Let f (x)
as x . Then
k=2 ak x



ak k+1
x
f (t) dt
as x .
k
1
x
k=2


k
as x , then
If f (x)
k=0 ak x
 x


ak
x k+1 as x ,
f (t) dt K + a0 x + a1 log x +
k + 1
c
k=2

where c > 0 and K = c [ f (t) a0 a1 t 1 ] dt a0 c a1 log c. Thus, term-by-term integration
of the asymptotic expansion of f (x) is permitted.
Term-by-term differentiation of asymptotic expansions is not always permitted. Concerning this
problem, we do have the following results, however.


k
Theorem A.3.5 Let f (x)

as x . Assume that f  (x) is continuous for all
k=0 ak x

large x and that f  (x) k=1 bk x k1 as x holds. Then bk = kak , k = 1, 2, . . . . Thus,

term-by-term differentiation of the asymptotic expansion of f (x) is permitted in this case.
Theorem A.3.6 Let f (z)

be analytic in the closed annular sector S = {z : |z| > 0,
k
arg z }, and let f (z)
as z in S. This asymptotic expansion can be differk=0 ak z
entiated term-by-term any number of times in the interior of S.
Other operations on asymptotic expansions likewise are not straightforward and need to be
carried out with care. Similarly, if L is an operator and f = O(g), for example, then L( f ) =
O(L(g)) does not necessarily hold. This remark applies equally to f = o(g) and to f g.
To illustrate this point, consider the behavior as x of L( f ; x) dened by


 f (x)  f (x) 

 ;  f (x) = f (x + 1) f (x), 2 f (x) = ( f (x)).
L( f ; x) = 
 f (x) 2 f (x) 
If g(x) = x , real and not an integer for simplicity, then we have L(g; x) x 2 2 . If f (x)
x , we cannot be sure that L( f ; x) x 2 2 , however. (i) When f (x) = x + O(x 1 ), the
best we can say is that L( f ; x) = O(x 2 1 ). (ii) When f (x) = x + ax 1 + O(x 2 ), the best
we can say is that L( f ; x) = O(x 2 2 ). (iii) Only when f (x) = x + ax 1 + bx 2 + o(x 2 )
do we have that L( f ; x) x 2 2 . Observe that, in all three cases, f (x) x .

B
The Laplace Transform and Watsons Lemma

B.1 The Laplace Transform

Denition B.1.1 The Laplace transform L[ f (t); z] f(z) of the function f (t) is dened by the
integral

ezt f (t) dt
(B.1.1)
f(z) =
0

whenever this integral exists.

For simplicity of treatment, in this book we assume that
(i) f (t) is integrable in the sense of Riemann on any interval (0, T ), T > 0, and that
(ii) there exist positive constants M, a, and t0 for which
| f (t)| Meat for every t t0 .
Functions with these properties are said to be of exponential order.
With these restrictions on f (t), the integral in (B.1.1) converges absolutely for each z that satises
z > a. In addition, f(z) is analytic in the right half plane z > a, and limz f(z) = 0. The
derivatives of f(z) are given by

ezt t n f (t) dt,
(B.1.2)
f(n) (z) = (1)n
0

and these integral representations converge for z > a too.

If f (k) (t), k = 1, . . . , n 1, are continuous for all t 0, and f (n) (t) is of exponential order,
then the Laplace transform of f (n) (t) is given in terms of f(z) by
L[ f (n) (t); z] = z n f(z)

n1


z ni1 f (i) (0+).

i=0

The following inversion formula is known as the Bromwich integral:

 c+i
1
f (t+) + f (t)
=
e zt f(z) dz,
2
2i ci
where the path of integration is the straight line z = c, which is parallel to the !z-axis, and f(z)
has all its singularities to the left of z = c.
For an extensive treatment of Laplace transforms, see Sneddon [319].

B.2 Watsons Lemma

One of the most important tools of asymptotic analysis that we make use of in this book is
Watsons lemma. This lemma forms the basis of the method of Laplace and the method of

463

464

B The Laplace Transform and Watsons Lemma

steepest descent for the asymptotic expansion of integrals. For the proof, see Olver [223] or
Murray [215].
Theorem B.2.1 Let the function f (t) be of exponential order and satisfy
f (t)

ck t k as t 0+,

k=0

where 1 < 0 < 1 < , and limk k = +. Then, for any > 0,



ck (k + 1)
f(z) =
ezt f (t) dt
as z , | arg z | < /2 .
z k +1
0
k=0

C
The Gamma Function

The Gamma function (z) is dened as in


et t z1 dt, z > 0,
(z) =
0

and hence is analytic in the right half plane z > 0.

For positive integer values of z, it assumes the special values
(n) = (n 1)!, n = 1, 2, . . . .
By integrating by parts m times, we obtain from the preceding integral representation the equality
(z) =

(z + m)
.
z(z + 1) (z + m 1)

But the right-hand side of this equality is dened and analytic as well in the right half plane z >
m except at z = 0, 1, 2, . . . , m + 1 . Thus, through this equality, (z) can be continued
analytically to the whole complex plane except the points z = n, n = 0, 1, . . . , where it has
simple poles, and the residue at z = n is (1)n /n! for each such n.
Another representation of (z) that is also valid for all z is Eulers limit formula
(z) = lim

n!n z
.
z(z + 1)(z + 2) (z + n)

Two important identities that result from Eulers limit formula are the reection formula
(z)(1 z) =

, z = 0, 1, 2, . . . ,
sin z

and the duplication formula

(2z) =

22z1
(z)(z + 12 ), 2z = 0, 1, 2, . . . .
1/2

The following innite product representation is valid for all z:

#

1
z  z/k $
= zeC z
e
1+
.
(z)
k
k=1

Here, C = 0.5772156 is Eulers constant dened by



n
1
C = lim
log n .
n
k
k=1
From this product representation, it is obvious that the function 1/ (z) is entire; i.e., it is analytic
everywhere in the complex plane. Therefore, (z) has no zeros.

465

466

C The Gamma Function

The logarithmic derivative of (z), namely,   (z)/ (z), is called the Psi function and is denoted
(z). From the product representation, it follows that


1 
1
1
(z) = C +

, z = 0, 1, 2, . . . ,
z
k
k+z
k=1
from which it also follows that
  (1) = (1) = C.
The following asymptotic expansion is known as Stirlings formula:



1
1
+

as z , |argz| < .
(z) 2 z z1/2 ez 1 +
12z
288z 2
In particular, we have




(x)
x+1/2
(x + 1) = 2 x
for x > 0; (0 < (x) < 1).
exp x +
12x

One of the useful consequences of Stirlings formula is the asymptotic expansion





(z + a)
z ab 1 +
ck z k as z ,
(z + b)
k=1
along any curve joining z = 0 and z = providing z + a and z + b are different from 0, 1,
2, . . . . Here ck are some constants that depend only a and b.
For more details, see Olver [223] or Abramowitz and Stegun [1].

D
Bernoulli Numbers and Polynomials and
the EulerMaclaurin Formula

D.1 Bernoulli Numbers and Polynomials

The Bernoulli numbers Bn , n = 0, 1, . . . , are dened via


t
tn
=
Bn ,
t
e 1
n!
n=0
 t 
dn
i.e., Bn = dt n et 1 |t=0 , n = 0, 1, . . . . They can be computed from the recursion relation

n 

n+1
B0 = 1 and
Bk = 0, n = 1, 2, . . . .
k
k=0
The rst few of the Bernoulli numbers are thus B0 = 1, B1 = 1/2, B2 = 1/6, B4 =
1/30, . . . , and B3 = B5 = B7 = = 0. The Bernoulli numbers B2n are related to the Riemann
Zeta function (z) as in


1
(2 )2n B2n
(2n) =
, n = 1, 2, . . . ,
= (1)n1
2n
k
2(2n)!
k=1
which also shows that (1)n1 B2n > 0, n = 1, 2, . . . , and also that B2n /(2n)! = O((2 )2n )
as n .
The Bernoulli polynomials Bn (x), n = 0, 1, . . . , are dened via


tn
te xt
=
,
B
(x)
n
et 1
n!
n=0

and they can be computed from

Bn (x) =

n  

n
k=0

Bk x nk , n = 0, 1, . . . .

Thus, B0 (x) = 1, B1 (x) = x 12 , B2 (x) = x 2 x + 16 , etc., and Bn (0) = Bn , n = 0, 1, . . . .

They satisfy
Bn (x + 1) Bn (x) = nx n1 and Bn (1 x) = (1)n Bn (x), n = 1, 2, . . . ,
from which
Bn (1) = (1)n Bn , n = 1, 2, . . . ,
and B1 ( 12 ) = B3 ( 12 ) = B5 ( 12 ) = = 0. They also satisfy
B2n ( 12 ) = (1 212n )B2n , n = 1, 2, . . . ,
and
Bn (x) = n Bn1 (x),

Bn (x) d x = 0, n = 1, 2, . . . .
0

467

468

D Bernoulli Numbers and Polynomials and the EulerMaclaurin Formula

Finally,
|B2n (x)| < |B2n | when x (0, 1), n = 1, 2, . . . ,
so that, for n = 1, 2, . . . , the sign of B2n B2n (x) on (0, 1) is the same as that of B2n ; thus,
(1)n1 [B2n B2n (x)] 0 on [0, 1]. Actually, B2n B2n (x) = 0 only for x = 0 and x = 1. In
addition, (1)n1 [B2n B2n (x)] achieves its maximum on [0, 1] at x = 1/2, as a result of which,
(1)n [B2n ( 12 ) B2n (x)] 0 on [0, 1] too.
For each n, the periodic Bernoullian function B n (x) is dened to be the 1-periodic extension of
the Bernoulli polynomial Bn (x). That is,
B n (x) = Bn (x i), when x [i, i + 1), i = 0, 1, 2, . . . .
Thus, B 1 (x) is a piecewise linear sawtooth function, and B n (x) C n2 (, ), n = 2, 3, . . . .
They also satisfy
 a+1


Bn (x) = n B n1 (x) and

B n (x) d x = 0 for every a, n = 1, 2, . . . .
a

For more details, see Abramowitz and Stegun [1] and Steffensen [322].

D.2 The EulerMaclaurin Formula

D.2.1 The EulerMaclaurin Formula for Sums
An important tool of asymptotic analysis is the famous EulerMaclaurin formula. In the following
theorem, we give the most general form of this formula with its remainder.
Theorem D.2.1 Let F(t) C m [r, ), where r is an integer, and let [0, 1] be xed. Then, for
any integer n > r ,
 n
n1
m



Bk ( )  (k1)
F
F(i + ) =
F(t) dt +
(n) F (k1) (r ) + Rm (n; ),
k!
r
i=r
k=1
where the remainder term Rm (n; ) is given by
 n
B m ( t)
dt.
F (m) (t)
Rm (n; ) =
m!
r
For the important special cases in which = 0 and = 1/2, Theorem D.2.1 assumes the
following forms:
Theorem D.2.2 Let F(t) C 2m [r, ), where r is an integer. Then, for any integer n > r ,
 n
n
m1



B2k  (2k1)

F
F(i) =
F(t) dt +
(n) F (2k1) (r ) + Rm (n),
(2k)!
r
i=r
k=1
where the double prime on the summation on the left-hand side means that the rst and last terms
in the summation are halved, and the remainder term Rm (n) is given by
 n
B2m B 2m (t)
dt
F (2m) (t)
Rm (n) =
(2m)!
r
= (n r )

B2m (2m)
F (m,n ) for some m,n (r, n).
(2m)!

Theorem D.2.3 Let F(t) C 2m [r, ), where r is an integer. Then, for any integer n > r ,
 n
n1
m1



B2k ( 12 )  (2k1)
F(i + 12 ) =
F(t) dt +
(n) F (2k1) (r ) + Rm (n),
F
(2k)!
r
i=r
k=1

D.2 The EulerMaclaurin Formula

469

where the remainder term Rm (n) is given by

 n
B2m ( 12 ) B 2m (t + 12 )
dt
F (2m) (t)
Rm (n) =
(2m)!
r
= (n r )

B2m ( 12 ) (2m)
F (m,n ) for some m,n (r, n).
(2m)!

n
These theorems can be used to nd the asymptotic expansion of the sum i=r
F(i + ) as
n or to approximate its limit as n when this limit exists. We give one such example
in the next section.

D.2.2 The EulerMaclaurin Formula for Integrals

Taking r = 0 and making the transformation of variable t = n(x a)/(b a) in Theorem D.2.1,
and dening f (x) F(n(x a)/(b a)), we obtain the following EulerMaclaurin formula for
the offset trapezoidal rule approximation of integrals:
Theorem D.2.4 Let f (x) C m [a, b] and let [0, 1] be xed. Set h = (b a)/n for n integer
and dene
 b
n1

Tn () = h
f (a + i h + h) and I =
f (x) d x.
a

i=0

Then,
Tn () = I +

m


Bk ( )  (k1)
f
(b) f (k1) (a) h k + Um (h; ),
k!
k=1

where the remainder term Um (h; ) is given by



 b
xa
B m n ba
m
(m)
d x.
f (x)
Um (h; ) = h
m!
a
Obviously, Um (h; ) = O(h m ) as h 0 [equivalently, Um (h; ) = O(n m ) as n ].
If we let = 0 and = 1/2 in Theorem D.2.4, then half the terms in the summation on k there
disappear, and we obtain the following special forms
 b of the EulerMaclaurin expansion for the
trapezoidal and midpoint rule approximations of a f (x) d x:
Theorem D.2.5 Let f (x) C 2m [a, b] and dene
Tn = Tn (0) B1 [ f (b) f (a)] h = h

n



f (a + i h) and I =

i=0

f (x) d x,
a

where the double prime on the summation on the right-hand side means that the rst and last terms
in this summation are halved. Then,
Tn = I +

m1


B2k  (2k1)
f
(b) f (2k1) (a) h 2k + Um (h),
(2k)!
k=1

with the remainder term Um (h) being given by

 b
xa
)
B2m B 2m (n ba
Um (h) = h 2m
dx
f (2m) (x)
(2m)!
a
= (b a)

B2m (2m)
f
(m,n ) h 2m for some m,n (a, b).
(2m)!

470

D Bernoulli Numbers and Polynomials and the EulerMaclaurin Formula

Theorem D.2.6 Let f (x) C 2m [a, b] and dene

Mn = Tn ( 12 ) = h

n1



f (a + i h + 12 h) and I =

i=0

f (x) d x.
a

Then,
Mn = I +

m1


B2k ( 12 )  (2k1)
f
(b) f (2k1) (a) h 2k + Um (h),
(2k)!
k=1

where the remainder term Um (h) is given by



Um (h) = h 2m

f (2m) (x)

xa
)
B2m ( 12 ) B 2m ( 12 + n ba

(2m)!

= (b a)

dx

B2m ( 12 ) (2m)
f
(m,n ) h 2m for some m,n (a, b).
(2m)!

When f (x) C [a, b] in

Theorems D.2.4D.2.6, the nite summations on k can be replaced
by the innite summations
 b the remainder can be deleted, and this gives asymptotic
k=1 and
expansions of the approximations to a f (x) d x as h 0 (equivalently, as n ). For example,
the result of Theorem D.2.4 becomes
Tn ( ) I +



Bk ()  (k1)
f
(b) f (k1) (a) h k as h 0.
k!
k=1

For more details, see Steffensen [322].

D.3 Applications of EulerMaclaurin Expansions

D.3.1 Application to Harmonic Numbers
We continue by applying Theorem D.2.2

to the function F(t) = 1/t to derive an asymptotic
n
expansion for the harmonic number Hn = i=1
1/i as n . Taking r = 1 in Theorem D.2.2,
we rst obtain
Hn = log n +

m1
m1

1
1
B2k 
B2k 2k
+
+

n
+ Rm (n),
2 2n
2k
2k
k=1
k=1

where


Rm (n) =
1

B2m B 2m (t)
dt.
t 2m+1

Because B2m B 2m (t) is bounded for all t, Rm () exists, and hence


B2m B 2m (t)
Rm (n) = Rm ()
dt = Rm () + O(n 2m ) as n .
t 2m+1
n
By the well-known fact that Hn log

n C as n , where C = 0.5772156 is Eulers
constant, we realize that Rm () + m1
k=1 B2k /(2k) + 1/2 = C, independently of m. Thus, we
have obtained the asymptotic expansion
Hn log n + C +


1
B2k 2k
as n .

n
2n
2k
k=1

D.4 A Further Development

471

D.3.2 Application to Cauchy Principal Value Integrals

f (x) = g(x)/(x t) for some t (a, b), and assume that g(t) = 0. Then the integral
Let
b
f (x) d x does not exist in the ordinary sense. Provided g(x) is differentiable at x = t, it is
a
dened in the Cauchy principal value sense; that is,
 t  b 
I = lim
f (x) d x
+
0

t+

exists. For such integrals, the following result has been given by Sidi and Israeli [310, Theorem 4
and Corollary]:
Theorem D.3.1 Let h = (b a)/n for some integer n 1 and xi = a + i h, i = 0, 1, . . . , n. Let
t be xed and t {x1 , . . . , xn1 }. Dene the trapezoidal rule approximation to I by
Tn = h

n



f (xi ) + g  (t)h.

i=0
xi =t

Provided g(x) C 2m [a, b], there holds

Tn = I +

m1


B2k  (2k1)
f
(b) f (2k1) (a) h 2k + U m (h),
(2k)!
k=1

where U m (h) = O(h 2m ) as h 0 (such that t {x1 , . . . , xn1 } always), and U m (h) is given by
the Cauchy principal value integral
 b
xa
)
B2m B 2m (n ba
d x.
U m (h) = h 2m
f (2m) (x)
(2m)!
a
This result follows in a nontrivial fashion from Theorem D.2.5.

D.4 A Further Development

We now use Theorems D.2.4D.2.6 to prove the following interesting result that appears to be
new. We take [a, b] = [0, 1] for simplicity.
Theorem D.4.1 Set t = n 2 in Theorem D.2.5. If f C 2m [0, 1], then the trapezoidal rule approximation Tn can be continued to a function w(t) dened for all real t [0, 1] [i.e., w(n 2 ) = Tn ,
n = 1, 2, . . . ] such that w(t) C q [0, 1], where q = (2m/3) 1; i.e., q is the greatest integer
less than 2m/3. Thus, if f C [0, 1], then w(t) C [0, 1] as well.
The same holds when Tn is replaced by the midpoint rule approximation Mn of Theorem D.2.6.
A similar result holds for the general offset trapezoidal rule approximation Tn () of
Theorem D.2.4 (with = 0, 1/2, 1). Only this time, t = n 1 , and q = (m/2) 1 when f (x)
C m [0, 1].
Proof. We give the proof for Tn only. That for Mn is identical and that for Tn () is similar.
By Theorem D.2.5, it seems natural to try
w(t) =
where c0 =

1
0

m


ck t k + Wm (t),

k=0

f (x) d x and ck = B2k [ f (2k1) (1) f (2k1) (0)]/(2k)!, k = 1, 2, . . . , and

 1
B 2m (xt 1/2 )
Wm (t) = t m
d x.
f (2m) (x)
(2m)!
0

Obviously, mk=0 ck t k , the polynomial part of w(t), is in C [0, 1]. By the fact that B 2m (x)
2m2
C
(, ), Wm (t) is in C 2m2 (0, 1]; hence, we have to analyze its behavior at t = 0. First,
provided m 1, Wm (t) is continuous at t = 0 because limt0 Wm (t) = 0. Next, provided m 2,

472

D Bernoulli Numbers and Polynomials and the EulerMaclaurin Formula

we can differentiate the integral expression with respect to t for t > 0. Using also the fact that

B k (x) = k B k1 (x), we obtain
Wm (t) = mt m1

f (2m) (x)
0

 1
B 2m (xt 1/2 )
B 2m1 (xt 1/2 )
d x + t m3/2
d x,
x f (2m) (x)
(2m)!
2(2m 1)!
0

from which limt0 Wm (t) = 0, implying that Wm (t) is continuous at t = 0. Continuing this
(t) + W
(t), where
way, and provided m > 3 p/2, we obtain Wm( p) (t) = W
 1
B 2m p (xt 1/2 )
(t) = (1) p+1 t m3 p/2
d x,
x p f (2m) (x) p
W
2 (2m p)!
0
(t) = 0, and W
(t) is smoother than W
(t) at t = 0. This completes the proof.
hence limt0 W

D.5 The EulerMaclaurin Formula for Integrals with Endpoint Singularities

Theorems D.2.4D.2.6 concerned integrands f (x) that were sufciently smooth on [a, b]. We now
want to treat functions f (x) that are smooth on (a, b) but have singularities at one or both endpoints
x = a and x = b. Here we give Navots extensions of the EulerMaclaurin formula for integrals
with algebraic and/or logarithmic end point singularities. Navots results were rederived by Lyness
and Ninham [202] by a different technique that uses generalized functions. For a generalization
of Navots results to integrands with other singularities, see Waterman, Yos, and Abodeely [351].
For an extension to Hadamard nite part integrals, see Monegato and Lyness [213].
These results have been extended to multidimensional integrals over hypercubes and hypersimplices in the works of Lyness [196], Lyness and Monegato [201], Lyness and de Doncker [199],
and Sidi [283]. Recently, numerical integration over curved surfaces and accompanying Euler
Maclaurin expansions have been considered. See the works by Georg [95], Verlinden and Cools
[346], and Lyness [198], for example.

D.5.1 Algebraic Singularity at One Endpoint

We start by stating a result of fundamental importance that is due to Navot [216] and that generalizes
Theorem D.2.4 to the case in which f (x) has an algebraic singularity at x = a. As before, h =
(b a)/n throughout this section too.
Theorem D.5.1 Let g(x) C m [a, b] and let be a constant such that = 0, 1, 2, . . . , and
 > 1. Let also (0, 1] be xed. Set f (x) = (x
a) g(x). Then, with Tn () as in
g ( j) (a)
Theorem D.2.4, and with p = ( + m), sm (x) = m1
(x a) j+ and m (x) =
j=0
j!
f (x) sm (x), we have
Tn () = I +

m
m1


g ( j) (a) + j+1
Bk ( ) (k1)
f
h
(b) h k +
( j, )
+ Um, p (h; ),
k!
j!
k=1
j=0

where the remainder term Um, p (h; ) is given by


Um, p (h; ) = h m

(m)
m (x)
a

xa
B m ( n ba
)

m!

dx


xa
p

)
B p ( n ba
Bk ( ) (k1)
sm (b) h k + h p
d x,
sm( p) (x)
k!
p!
b
k=m+1

where (z, ) is the generalized Zeta function dened by (z, ) =

and continued analytically to the complex z-plane.

n=0 (n

+ )z for z > 1

D.5 The EulerMaclaurin Formula for Integrals with Endpoint Singularities 473
The generalized Zeta function has properties very similar to those of the standard Riemann
Zeta function, and both are considered in some detail in Appendix E.
Obviously, Um, p (h; ) = O(h m ) as h 0 [equivalently, Um, p (h; ) = O(n m ) as n ].
Also, when 1 <  < 0, we can take p = m.
Using the fact that
( j, ) =

B j+1 ( )
, j = 0, 1, . . . ,
j +1

it can be shown that the result of Theorem D.5.1 reduces precisely to that of Theorem D.2.4
as 0.
When g(x) C [a, b], we have the asymptotic expansion
Tn ( ) I +



g ( j) (a) + j+1
Bk ( ) (k1)
f
h
(b) h k +
( j, )
as h 0.
k!
j!
k=1
j=0

If we let = 1 and = 1/2 in Theorem D.5.1, half the terms in the summation on k disappear.
Let us now dene
Tn = Tn (1) B1 (1) f (b) h = h

n



f (a + i h),

i=1

where the prime on the summation on the right-hand side means that the last term in the summation
is halved.
Thus, when g(x) C [a, b], the asymptotic expansions of Tn and Mn become, respectively,
Tn I +



g ( j) (a) + j+1
B2k (2k1)
f
h
(b) h 2k +
( j)
as h 0,
(2k)!
j!
k=1
j=0

because (z, 1) is nothing but (z), and

Mn I +



B2k ( 12 ) (2k1)
g ( j) (a) + j+1
f
h
(b) h 2k +
( j, 12 )
as h 0.
(2k)!
j!
k=1
j=0

D.5.2 Algebraic-Logarithmic Singularity at One Endpoint

Theorem D.5.1 enables us to treat integrals of functions u(x) with an algebraic-logarithmic sin
m
gularity at x = a of the form u(x)
 b = (log(x a))(x b a) g(x) with g(x) C [a, b].
d
We rst observe that, if J = a u(x) d x and I = a (x a) g(x) d x, then J = d
I . This suggests that we differentiate with respect to the EulerMaclaurin expansion associated with I given
d
in Theorem D.5.1. A careful analysis reveals that d
Um, p (h; ) = O(h m ) as h 0. Thus, if we
dene
Sn () = h

n1


u(a + i h + h),

i=0

then
m

Bk () (k1)
u
(b) h k
k!
k=1
m1 ( j)


g (a) 
( j, ) log h  ( j, ) h + j+1 + O(h m ) as h 0,
+
j!
j=0

Sn () = J +

474

D Bernoulli Numbers and Polynomials and the EulerMaclaurin Formula

where  (z, ) =

When g(x) C [a, b], we have the asymptotic expansion

d
(z, ).
dz


Bk () (k1)
u
(b) h k
k!
k=1



g ( j) (a) 
( j, ) log h  ( j, ) h + j+1 as h 0.
+
j!
j=0

Sn ( ) J +

Simplications take place in this expansion when we take = 1 and = 1/2 as before. Further
simplications take place when = 0. All these developments too are due
 bto Navot [217].
Needless to say, this approach can be used to treat those integrals J = a u(x) d x with u(x) =
(log(x a))r (x a) g(x), where r is a positive integer. In case g(x) C [a, b], this results in
an asymptotic expansion of the form


Bk () (k1)
u
(b) h k
k!
k=1

 r


g ( j) (a) 
+
c ji (log h)i h + j+1 as h 0,
j!
j=0
i=0

Sn ( ) J +

independent of g(x) and n; they depend on (s) ( j, ), s = 0, 1, . . . , r .

where c ji are constants
ds
Here, (s) (z, ) = dz
s (z, ).

D.5.3 Algebraic-Logarithmic Singularities at Both Endpoints

We end by discussing briey the case in which f (x) = (x a) (b x) g(x), where , =
0, 1, . . . , and ,  > 1 and g(x) is sufciently smooth on [a, b]. For simplicity, we assume
that g(x) C [a, b]. With (0, 1) and Tn ( ) as in Theorem D.2.4, we then have
Tn ( ) I +

( j, )

j=0

ga( j) (a) + j+1

h
j!

( j)


g (b) + j+1
h
+
(1) j ( j, 1 ) b
as h 0,
j!
j=0

where we have dened ga (x) = (b x) g(x) and gb (x) = (x a) g(x).

This result was rst stated
 b Ninham [202]. It can be obtained by applying Theo c by Lyness and
rem D.5.1 to the integrals a u(x) d x and c u(x) d x, where c = a + kh for some integer k n/2,
and by adding the resulting EulerMaclaurin expansions, and recalling that Bk (1 ) =
(1)k Bk () for all k. We leave the details to the interested reader.
When = 1, this result should be modied as follows: Dene
Tn = h

n1


f (a + i h).

i=1

Then,
Tn I +

( j)

j=0

ga( j) (a) + j+1

h
j!
( j)

(1) j ( j)

j=0

gb (b) + j+1
h
as h 0.
j!

By repeated differentiation with respect to and , we can extend these results to the case in
which the integrand is of the form
u(x) = (log(x a))r (x a) (log(b x))s (b x) g(x),
where r and s are some nonnegative integers.

D.6 Application to Singular Periodic Integrands

475

D.6 Application to Singular Periodic Integrands

b
An interesting application of the results of the previous sections is to integrals I = a f (x) d x,
where f (x) is periodic with period T = b a, and innitely differentiable on (, ) except at the points x = t + kT , k = 0, 1, 2, . . . , where it may have polar or algebraic and/or
logarithmic singularities.

(i) When f (x) has a polar singularity at t (a, b), it is of the form f (x) = g(x)/(x t) + g(x)
for x [a, b], and g(t) = 0.
(ii) When f (x) has an algebraic singularity at t (a, b), it is of the form f (x) = g(x)|x t|s +

g(x),
s > 1, for x [a, b].
(iii) When f (x) has a logarithmic singularity in t (a, b), it is of the form f (x) = g(x) log |x

t| + g(x)
for x [a, b],
(iv) When f (x) has an algebraic-logarithmic singularity at t (a, b), it is of the form f (x) =

g(x)|x t|s log |x t| + g(x),

s > 1, for x [a, b].
b
In the case of the polar singularity, a f (x) d x does not exist asan ordinary integral but is dened
b
in the Cauchy principal value sense. In the remaining cases, a f (x) d x exists because f (x) is
integrable across x = t, and we say that these singularities are weak.
Of course, in all four cases, g, g C [a, b], but g, g  C (, ), and they are not necessarily periodic. Such integrals arise as part of periodic singular and/or weakly singular Fredholm
integral equations. For these integrals, Sidi and Israeli [310, Theorem 7] derived corrected trapezoidal rule approximations T (h; t), and obtained asymptotic expansions for T (h; t) as h 0,
where h = (b a)/n = T /n as usual. (See also Sidi [289] for further considerations and comparisons with other works.)
(i) In case of a polar singularity,
T (h; t) = h

n1


f (t + i h),

i=1

+ g  (t)]h + O(h ) as h 0, for every > 0.

T (h; t) = I [g(t)
(ii) In case of an algebraic singularity,
T (h; t) = h

n1


f (t + i h) + g(t)h
2 (s)g(t)h s+1 ,

i=1

T (h; t) I + 2


(s 2k)
k=1

(2k)!

g (2k) (t)h s+2k+1 as h 0.

(iii) In case of a logarithmic singularity,

T (h; t) = h

n1


f (t + i h) + g(t)h
+ g(t)h log

i=1

T (h; t) I 2


 (2k)
k=1

(2k)!


h
,
2

g (2k) (t)h 2k+1 as h 0.

(iv) In case of an algebraic-logarithmic singularity,

T (h; t) = h

n1


f (t + i h) + g(t)h
+ 2[  (s) (s) log h]g(t)h s+1 ,

i=1

T (h; t) I + 2


(s 2k) log h  (s 2k)
k=1

(2k)!

g (2k) (t)h s+2k+1 as h 0.

476

D Bernoulli Numbers and Polynomials and the EulerMaclaurin Formula

b
 t+T
Here we
f (x) d x = t
f (x) d x. This is so because, by the periodicity
 b used the fact
 bthat

a
of f (x), a f (x) d x = a  f (x) d x for any a  and b such that b a  = T . We refer the reader to
Sidi and Israeli [310] for details and for further results.
Three types of the integrals treated here occur commonly in applications:
+ g  (t)]h + O(h ) as
1. Those with a polar singularity, for which we have T (h; t) I = [g(t)
h 0, for every > 0.
2. Those with a logarithmic singularity, for which we have T (h; t) I = O(h 3 ) as h 0.
3. Those with an algebraic-logarithmic singularity with s = 2. [In this case too the singularity is
only logarithmic in nature, because |x t|s = (x t)s is analytic for all x including x = t when
s is a positive even integer.] Now in Appendix E it is stated that (2 j) = 0 for j = 1, 2, . . . .
Therefore, for s = 2 p, p > 0 an integer, T (h; t) and its asymptotic expansion become
T (h; t) = h

n1


f (t + i h) + g(t)h
+ 2  (2 p)g(t)h 2 p+1 ,

i=1

T (h; t) I 2


 (2 p 2k)
k=1

(2k)!

g (2k) (t)h 2 p+2k+1 as h 0.

Consequently, T (h; t) I = O(h 2 p+3 ) as h 0. In particular, when s = 2, i.e., when p = 1,

we have T (h; t) I = O(h 5 ) as h 0.

n1

wetake T (h; t) = h i=0

f (t + i h); that
When s = 1, 2, . . . , in f (x) = g(x)|x t|s + g(x),
b
is, T (h; t) is simply the trapezoidal rule approximation to a f (x) d x. When s is an even integer,
f (x) has no singularity across x = t. In this case, T (h; t) I has an empty asymptotic expansion,
which implies that T (h; t) = I + O(h ) as h 0, for every > 0. When s is an odd integer, the
derivatives of f (x) have nite jump discontinuities
across x = t, and T (h; t) has an asymptotic


2k
expansion of the form T (h; t) I +
as h 0, as has been shown by Averbuch
k=1 k (t)h
et al. [14].

E
The Riemann Zeta Function and the Generalized
Zeta Function

E.1 Some Properties of (z)

The
Zeta function (z) is dened to be the sum of the (convergent) Dirichlet series

Riemann
z
for z > 1. From the theory of Dirichlet series, it is known that (z) is an analytic
k=1 k
function of z for z > 1. It turns out that (z) can be continued analytically to the whole z-plane
except z = 1, where it has a simple pole with residue 1, and we denote its analytic continuation
again by (z). The reection formula due to Riemann, namely,
(1 z) = 21z z cos( 12 z)(z) (z),
can be used to continue (z) analytically. It is interesting to note the following special values that
this analytic continuation assumes:
1
B2n
(0) = ; (2n) = 0 and (1 2n) =
, n = 1, 2, . . . .
2
2n
We also have
1
 (0) = log(2),
2
 (2n) = (1)n 22n1 2n (2n + 1) (2n + 1), n = 1, 2, . . . .
Let us, in addition, recall the connection between (2n) and B2n , namely,
(2n) =

2n

1
n1 (2) B2n
, n = 1, 2, . . . .
=
(1)
k 2n
2(2n)!
k=1

The Zeta function has no zeros in the right half plane z > 1, and its only zeros in the left half
plane z < 0 are 2, 4, 6, . . . . As for the zeros in the remaining strip 0 z 1, a famous
and still unproved conjecture by Riemann claims that they all lie on the midline z = 12 .

E.2 Asymptotic Expansion of

n1

k=0 (k

+ )z

z
We now derive the asymptotic
expansion as n of the partial sum Sn (z) = n1
k=0 (k + )


z
with > 0 whether k=0 (k + ) converges or not. Simultaneously, we also obtain
the
analytic


z
structure of the generalized Zeta function (z, ), which is dened via (z, ) =
k=0 (k + )
for z > 1 and then continued analytically to the complex z-plane, and hence of the Riemann
Zeta function (z) as well. [Observe that (z, 1) = (z).] This can be achieved by using the Euler
Maclaurin expansion, as was done in Section D.3 of Appendix D. Our approach here is quite
different in that it involves the Laplace transform and thus enables us to conclude also that the
partial sum Sn (z) is associated with a function A(y) in F(1)
.

477

478

E The Riemann Zeta Function and the Generalized Zeta Function



Using the fact that a z =

Sn (z) =

0
n1



eat t z1 dt / (z), a > 0 and z > 0, we obtain

(k + )z =

k=0

which, by the additional fact that

1
Sn (z) =
(z)

n1
k=0

1
(z)

t z1


n1


e(k+ )t dt,

k=0

k = (1 n )/(1 ), gives

1
t z1 e(1)t
dt
et 1
(z)

nt


t z1 e(1 )t
dt, z > 1.
et 1

(E.2.1)

Because t z1 e(1)t /(et 1) = t z2 + O(t z1 ) as t 0+, the second integral in (E.2.1) is

O(n z+1 ), hence tends to 0 as n , as can be shown by using Watsons lemma. Therefore, we
conclude that
 z1 (1)t
1
t e
dt, z > 1,
(z, ) =
(z) 0
et 1
and hence that
(z, ) = Sn (z) +

1
(z)


0


ent


te(1)t z2
t dt, z > 1.
et 1

(E.2.2)

Now, from Appendix D, we have for any positive integer m,

m1

te(1 )t
Bs (1 ) s
=
t + Wm (t), Wm (t) = O(t m ) as t 0.
t
e 1
s!
s=0

Substituting this in (E.2.2) and making use of the facts that (z + k)/ (z) = (z)k = z(z + 1)
(z + k 1) and Bk (1 ) = (1)k Bk ( ) for all k, we obtain for z > 1
(z, ) = Sn (z) +

m1
(z)s1 Bs () sz+1
n z+1 
1
+
n
(1)s
+
z1
s!
(z)
s=1

ent Wm (t)t z2 dt.

(E.2.3)
The rst term on the right-hand side of (E.2.3), namely, the term Sn (z), is an entire function of z.
The second term, n z+1 /(z 1), is analytic everywhere

except at z = 1, where it has a simple pole
with residue 1. The third term, namely, the summation m1
s=1 , being a polynomial in z multiplied
by n z , is entire just as the rst one. Finally, because Wm (t)t z2 is(i) O(t m+z3 ) as t , and

(ii) O(t m+z2 ) as t 0, and because 1/ (z) is entire, the last term, 0 ent Wm (t)t z2 dt / (z),
mz+1
exists and is analytic for z > m + 1. In addition, it is O(n
) as n , by Watsons
lemma. Putting everything together, we conclude that the right-hand side of (E.2.3) is analytic
in the half plane z > m + 1 except at z = 1, where it has a simple pole with residue 1, and
it is thus the analytic continuation of the left-hand side. We have thus shown that (z, ) can be
continued analytically to the whole complex plane except z = 1, where it has a simple pole with
residue 1.
When z = 0, 1, 2, . . . , m + 2, the integral term on the right-hand side of (E.2.3) vanishes
because 1/ (z) = 0 for these values of z. Thus, for z = p, p = 0, 1, . . . , (E.2.3) reduces to
n1

k=0


( p)s1 Bs () ps+1
n p+1

n
(1)s
+ ( p, ).
p + 1 s=1
s!
p+1

(k + ) p =

n1
(k + ) p , which is obtained
Comparing this with the EulerMaclaurin formula for the sum k=0
p
by applying Theorem D.2.1 to the function F(t) = t , we obtain the following well-known
result:
B p+1 ( )
, p = 0, 1, . . . .
(E.2.4)
( p, ) =
p+1

E.2 Asymptotic Expansion of

n1

k=0 (k

+ )z

479

[Note that the relations (2n) = 0 and (1 2n) = B2n /(2n), n = 1, 2, . . . , mentioned in the
preceding section can be obtained by letting = 1 in (E.2.4).]
We have also shown that, for all other values of z, Sn (z) has the asymptotic expansion
Sn (z) (z, )

n z+1 
(z)s1 Bs ( ) zs+1

n
(1)s
as n , z = 1,
z1
s!
s=1

(E.2.5)

)z converges or not.
whether
k=0 (k +

z
Note that, when
converges, that is, when z > 1, this asymptotic expansion can
k=0 (k + )
be obtained by applying Watsons lemma to the second integral in (E.2.1).
Let z = 1 and z > m + 1 for some positive integer m. We can rewrite (E.2.3) in the form
Sn (z) = (z, ) + n z+1 g(n), where g(x) is given by

m1

(z)s1 Bs ( ) s x z1 xt
x
(1)s
e Wm (t)t z2 dt,
g(x) = (z 1)1
s!
(z)
0
s=1
( )
that are
and hence g(x) A(0)
0 . It is clear from one of the examples of functions in the class A
given in Section 5.1 that g(x) A(0) . This shows that A(y) (z, ) + y z1 g(y 1 ) is not only in
F(1) but also in F(1)
.
d
Sn (z) =
We continue our approach to show that the asymptotic expansion of dz


n1
k=0 (k + )z log(k + ) as n can be obtained by differentiating that of Sn (z) given in
(E.2.5) term by term. This can be done by differentiating both sides of (E.2.3), and realizing that


d
ent Wm (t)t z2 dt =
ent log t Wm (t)t z2 dt = O(n mz+1 log n) as n .
dz 0
0

This last assertion follows from the fact that,when f (t) is of exponential order and also satis
es f (t) = O(t p ) as t 0+, p > 1, then 0 et log t f (t) dt = O( p+1 log ) as ,
| arg | < /2 , for any > 0. Thus,
Sn (z)  (z, ) +




cs (z) log n cs (z) n zs+1 as n , for any z = 1,

(E.2.6)

s=0

where c0 (z) = 1/(z 1) and cs (z) = (1)s (z)s1 Bs ( )/s!, s = 1, 2, . . . , again whether

d
z
log(k + ) converges or not. Here, g  (z) stands for dz
g(z) in general.
k=0 (k + )
For more details on the Zeta function, see Olver [223], Abramowitz and Stegun [1], or Titchmarsh
[334].

F
Some Highlights of Polynomial Approximation Theory

F.1 Best Polynomial Approximations

Let f (x) be dened on the nite interval [a, b], and let #n denote the set of all polynomials of
degree at most n. The best polynomial approximation of degree at most n in the L -norm on [a, b]
to f (x) is that polynomial p (x) #n that satises
max | f (x) p (x)| max | f (x) p(x)| for all p #n .

x[a,b]

x[a,b]

It is known that, when f C[a, b], p (x) exists and is unique.

Let us denote
E n ( f ) = max | f (x) p (x)|.
x[a,b]

What interests us here are the convergence properties of the sequence of best polynomial approximations when f (x) is innitely smooth. We recall one of Jacksons theorems:
Theorem F.1.1 Let { pn (x)} be the sequence of best polynomial approximations to f (x) on [a, b],
where pn (x) #n . If f C k [a, b], then
E n ( f ) = O(n k ) as n .
The following result is an immediate consequence of Jacksons theorem.
Theorem F.1.2 Let { pn (x)} be the sequence of best polynomial approximations to f (x) on [a, b],
where pn (x) #n . If f C [a, b], then
E n ( f ) = O(n ) as n , for every > 0.
A theorem of Bernstein says that, if f (z) is analytic in an open domain that contains the interval
[a, b] in its interior, then the result of Theorem F.1.2 can be improved to read
E n ( f ) = O(en ) as n , for some > 0.
depends on the location of the singularity of f (z) that is closest to [a, b] in some well-dened
sense, which we omit.
If f (z) is an entire function, that is, it is analytic in the entire z-plane, an additional improvement
takes place, and we have
E n ( f ) = O(en ) as n , for every > 0.

F.2 Chebyshev Polynomials and Expansions

Let x = cos for [0, ]. The Chebyshev polynomials are dened by
Tn (x) = cos(n ), n = 0, 1, . . . .

480

F.2 Chebyshev Polynomials and Expansions

481

Thus, T0 (x) = 1, T1 (x) = x, T2 (x) = 2x 2 1, . . . . From this denition, it follows that the Tn (x)
satisfy the three-term recursion relation
Tn+1 (x) = 2x Tn (x) Tn1 (x), n = 1, 2, . . . ,
from which it can be shown that Tn (x) is a polynomial in x of degree exactly n and that Tn (x) is
even or odd depending on whether n is an even or an odd integer.
Because the transformation x = cos is one-to-one between 1 x 1 and 0 , we
see from the denition of the Tn (x) that
|Tn (x)| 1 for x [1, 1].
In other words, for x [1, 1], Tn (x) assumes values between 1 and 1 only.
When x is real and |x| > 1, the Tn (x) can be shown to satisfy

Tn (x) = (sign x)n cosh(n), where > 0 and e = |x| + x 2 1 > 1.
As a result, we have that
Tn (x)


1
1
(sign x)n en = (sign x)n (|x| + x 2 1)n as n .
2
2

In other words, when |x| > 1, the sequence {|Tn (x)|} increases to innity like en for some > 0
that depends on x.
The Tn (x) also satisfy the orthogonality property

 1
0 if m = n
Tm (x)Tn (x)
d x = 12 if m = n = 0.

if m = n = 0
1 x2
1
1
If f (x) satises 1 (1 x 2 )1/2 | f (x)|2 d x < , then it can be expanded in a series of
Chebyshev polynomials in the form



2 1 f (x)Tn (x)

an Tn (x); an =
d x, n = 0, 1, . . . ,
f (x) =

1
1 x2
n=0
where the prime on the summation indicates that the n = 0 term is to be multiplied by 1/2. It is
known that the sequence of the partial sums converges to f (x) in the sense that

2
 1
n

1

lim
f (x)
ai Ti (x) d x = 0.

n 1
1 x2
i=0
As we are interested in approximation of functions on an arbitrary nite interval [a, b], we
consider the Chebyshev expansions of functions on such an interval in the sequel. First, it is clear
that the expansion of f (x) in a Chebyshev series on [a, b] can be achieved by transforming [a, b]
to [1, 1] by the linear transformation x = (t) = (b + a)/2 + (b a)t/2, so that t [1, 1]
when x [a, b]. Then, with (x) = (2x a b)/(b a), we have



2 1 f ( (t))Tn (t)

f (x) =
an Tn ( (x)); an =
dt, n = 0, 1, . . . .

1
1 t2
n=0
Let us denote



n




ai Ti ( (x)).
Wn ( f ) = max  f (x)
x[a,b]
i=0

Theorem F.2.1 Let f C k [a, b]. Then

an = o(n k ) and Wn ( f ) = o(n k+1 ) as n .
Thus, for k 2 f (x) has a Chebyshev expansion that converges uniformly on [a, b].
The following result is an immediate consequence of Theorem F.2.1.

482

F Some Highlights of Polynomial Approximation Theory

Theorem F.2.2 Let f C [a, b]. Then

an = O(n ) and Wn ( f ) = O(n ) as n , for every > 0.
If f (z) is analytic in an open domain that contains the interval [a, b] in its interior, then the
result of Theorem F.2.2 can be improved to read
an = O(en ) and Wn ( f ) = O(en ) as n , for some > 0.
depends on the location of the singularity of f (z) that is closest to [a, b] in some well-dened
sense, which we omit.
If f (z) is an entire function, an additional improvement takes place, and we have
an = O(en ) and Wn ( f ) = O(en ) as n , for every > 0.
For the preceding and more, see, for example, the books by Cheney [47] and Davis [62].

G
A Compendium of Sequence Transformations

For the sake of convenience, we collect here the denitions of the various sequence transformations
and the algorithms used in their implementation. {Am } is the sequence we wish to extrapolate.

1. Richardson Extrapolation Process (REP)

Given the distinct scalars c1 , c2 , . . . , such that ck = 1 for all k, dene A(nj) through
Ar = A(nj) +

n


k ckr , j r j + n.

k=1

The

A(nj)

can be computed recursively via Algorithm 1.3.1; that is,

( j+1)

( j)

A0 = A j , j 0; A(nj) =

( j)

An1 cn An1
, j 0, n 1.
1 cn

2. Polynomial Richardson Extrapolation Process [REP-POL( )]

Given the scalar = 0, choose integers Rl such that R0 < R1 < R2 < , and dene A(nj) through
A Rl = A(nj) +

n

k

, j r j + n.

k=1

Rl

Letting (nj) = (R j /R j+n ) , the A(nj) can be computed recursively via Algorithm 2.2.1; that is,
( j+1)

( j)

A0 = A j , j 0; A(nj) =

( j)

An1 (nj) An1

1

( j)
n

, j 0, n 1.

3. Richardson Extrapolation Process with Conuence (REP-CONF)

Given the distinct scalars c1 , c2 , . . . , such that ck = 1 for all k, and the nonnegative integers
q1 , q2 , . . . , rst let
i = c1 ,

gi (m) = c1m m i1 , 1 i 1 q1 + 1,

1 +i = c2 ,

g1 +i (m) = c2m m i1 , 1 i 2 q2 + 1,

1 +2 +i = c3 , g + +i (m) = cm m i1 , 1 i 3 q3 + 1,
2
1
2
and so on. Then, dene A(nj) through
Ar = A(nj) +

n


k gk (r ), j r j + n.

k=1

483

484

G A Compendium of Sequence Transformations

The A(nj) can be computed recursively via the SGRom-algorithm (Algorithm 14.1.1); that is,
( j+1)

( j)

A0 = A j , j 0; A(nj) =

( j)

An1 n An1
, j 0, n 1.
1 n

4. Iterated 2 -Process (IDELTA)

Dene rst
m ({As }) = Am+1

(Am )(Am+1 )
.
2 Am

Next, dene Bn( j) recursively from

( j)

( j)

B0 = A j , j 0; Bn+1 = j ({Bn(s) }), j, n 0.

5. Iterated 2 ( )- and 2 ( , p)-Processes [IMDELTA( , p)]

Given the scalar and the integer p 1, dene rst
m ({As }; ) = Am+1

1 (Am )(Am+1 )
.

2 A m

Next, dene Bn( j) recursively from

( j)

( j)

( j)

( j)

B0 = A j , j 0; Bn+1 = j ({Bn(s) }; 2n), j, n 0, for p = 1,

B0 = A j , j 0; Bn+1 = j ({Bn(s) }; n/ p), j, n 0, for p 2.
For p 2, IMDELTA( , p) is the iterated 2 ( , p)-process [see (15.7.2)], whereas for p = 1, it
is the iterated 2 ( )-process [see (15.3.15)].

6. Iterated Lubkin Transformation (ILUBKIN)

Dene rst
Wm ({As }) =

2 (Am /Am )
.
2 (1/Am )

Next, dene Bn( j) recursively from

( j)

( j)

B0 = A j , j 0; Bn+1 = W j ({Bn(s) }), j, n 0.

7. Shanks Transformation (EPSILON)

Dene en (A j ) through
Ar = en (A j ) +

n


k Ar +k1 , j r j + n.

k=1
( j)

Next, dene the k recursively via the -algorithm (Algorithm 16.2.1); that is,
( j)

( j)

( j)

( j+1)

1 = 0, 0 = A j , j 0; k+1 = k1 +

1
( j+1)
k

( j)

, j, k 0.

( j)

Then, en (A j ) = 2n for all j and n.

8. Higher-Order G-Transformation (G-TRAN)

Given the scalars u 0 , u 1 , . . . , dene A(nj) through
Ar = A(nj) +

n

k=1

k u r +k1 , j r j + n.

G A Compendium of Sequence Transformations

485

The A(nj) can be computed with the help of the rs-algorithm (Algorithm 21.3.1) or the FS/qdalgorithm (Algorithm 21.3.2). Here are the steps of the FS/qd-algorithm:
( j)

( j)

e0 = 0, q1 =

u j+1
Aj
1
( j)
( j)
, M0 =
, N0 =
, j 0.
uj
uj
uj
( j+1)

( j)

en( j) = qn( j+1) qn( j) + en1 , qn+1 =

( j+1)

Mn( j) =

( j)

Mn1 Mn1
( j)
en

( j+1)

, Nn( j) =

en( j+1)
( j)

en

( j)

Nn1 Nn1
( j)
en

, A(nj) =

qn( j+1) ,
Mn( j)
( j)

Nn

, j 0, n 1.

9. The L-Transformation (L-TRAN)

Dene

L(nj)

through
Ar = L(nj) + r

n1

i
i=0

ri

, J r J + n; J = j + 1.

Then, L(nj) is given by the closed-form expression





n
i n
n1
A J +i / J +i
n J n1 A J / J
i=0 (1) i (J + i)
( j)

 =
n
n 
; J = j + 1.
Ln =
i
n1 /
(J
+
i)
n J n1 / J
(1)
J +i
i=0
i
Here the choices m = Am1 and m = mAm1 give rise to the t- and u-transformations,
respectively. The choice m = mAm1 (u-transformation) is the more useful.

10. The S-Transformation (S-TRAN)

Dene Sn( j) through
Ar = Sn( j) + r

n1

i
, J r J + n; J = j + 1.
(r
)i
i=0

Then, Sn( j) is given by the closed-form expression



n
i n
n ((J )n1 A J / J )
i=0 (1) i (J + i)n1 A J +i / J +i
( j)

Sn =
=
n
; J = j + 1.
i n
n ((J )n1 / J )
i=0 (1) i (J + i)n1 / J +i
Here too we can make the choices m = Am1 or m = mAm1 to obtain transformations
analogous to, respectively, the t- or the u-transformation.

11. The d (1) -Transformation (d (1) -TRAN)

Pick integers Rl such that 1 R0 < R1 < R2 < , and dene dn(1, j) = A(nj) through
A Rl = A(nj) + Rl (A Rl 1 )

n1

i
, j l j + n.
Rli
i=0

The A(nj) and the corresponding n( j) and !(nj) can be computed recursively via the W-algorithm
(Algorithm 7.2.4). We give here the steps of the resulting algorithm:
1. For j 0, set
( j)

M0 =

AR j
R j (A R j 1 )

( j)

, N0 =

1
( j)
( j)
( j)
( j)
, H0 = (1) j |N0 |, K 0 = (1) j |M0 |.
R j (A R j 1 )

2. For j 0 and n 1, compute Mn( j) , Nn( j) , Hn( j) , and K n( j) recursively from

( j+1)

Q (nj) =

( j)

Q n1 Q n1
1
R 1
j+n R j

where Q (nj) stand for Mn( j) or Nn( j) or Hn( j) or K n( j) .

486

G A Compendium of Sequence Transformations

3. For j, n 0, set
A(nj) =

 ( j) 
 ( j) 
 Hn 


( j)

, !( j) =  K n .
,

=
n
n
 ( j) 
 ( j) 
( j)
Nn
Nn
Nn

Mn( j)

When Rl = l + 1, l = 0, 1, . . . , we have dn(1, j) = L(nj) with m = mAm1 , that is, the d (1) transformation reduces to the u-transformation.

12. The d (m) -Transformation (d (m) -TRAN)

Pick integers Rl such that 1 R0 < R1 < R2 < , let n = (n 1 , n 2 , . . . , n m ), and dene dn(m, j)
through
A Rl = dn(m, j) +

m


Rlk (k A Rl 1 )

k=1

n
k 1
i=0

m

ki
,
j

j
+
N
;
N
=
nk .
Rli
k=1

The dn(m, j) can be computed recursively via the W(m) -algorithm (Algorithm 7.3.5) when m > 1.

13. The d(m) -Transformation (d(m) -TRAN)

j)
Pick integers Rl such that 1 R0 < R1 < R2 < , and dene d(m,
= A(nj) through
n

A Rl = A(nj) + Rl (A Rl 1 )

n1

ki
i/m

i=0

Rl

, j l j + n.

The A(nj) and the corresponding n( j) and !(nj) can be computed recursively via the W-algorithm
(Algorithm 7.2.4). We give here the steps of the resulting algorithm:
1. For j 0, set
AR j

( j)

M0 =

1
( j)
( j)
( j)
( j)
, H0 = (1) j |N0 |, K 0 = (1) j |M0 |.
R j (A R j 1 )

( j)

R j (A R j 1 )

, N0 =

2. For j 0 and n 1, compute Mn( j) , Nn( j) , Hn( j) , and K n( j) recursively from

( j+1)

Q (nj) =

( j)

Q n1 Q n1
1/m

1/m

R j+n R j

where Q (nj) stand for Mn( j) or Nn( j) or Hn( j) or K n( j) .

3. For j, n 0, set
 ( j) 
 ( j) 
H 
K 
M ( j)
A(nj) = n( j) , n( j) =  n( j) , !(nj) =  n( j) .
Nn
Nn
Nn

14. The -, ( )-, and ( , p)-Algorithms [RHO( , p)]

Given the scalar and the positive integer p, let
( , p)

Ck
( , p)

C2n

= +

= + k if p = 1,

n
n
( , p)
and C2n+1 = + + 1 if p 2,
p
p

( j)

and dene k recursively via the recursion relation

( , p)

( j)

( j)

( j)

( j)

Ck

( j+1)

1 = 0, 0 = A j , j 0; k+1 = k1 +

( j+1)

( j)

, j, k 0.

Here, the relevant quantities are the 2n . For p 2, RHO( , p) is the ( , p)-algorithm
(Algorithm 20.1.6), and for p = 1 it reduces to the ( )-algorithm (Algorithm 20.1.3). For p = 1
and = 1, RHO( , p) reduces to the -algorithm (Algorithm 20.1.1).

G A Compendium of Sequence Transformations

15. The -Algorithm (THETA)
Dene

( j)
k

recursively via Algorithm 20.2.1; that is,

( j)

( j)

( j)

( j+1)

1 = 0, 0 = A j , j 0;
( j)

( j)

( j)

2n+1 = 2n1 + D2n ; Dk = 1/k

( j)

( j+1)

2n+2 = 2n

( j+1)
2n
( j)
D2n+1

( j)

D2n+1 ,

for all j, k 0,

j, n 0.

( j)

Here, the relevant quantities are the 2n .

16. The Overholt Transformation (OVERH)

Let

(nj)

= (A j+n /A j+n1 )n , and dene A(nj) recursively as in (22.1.4); that is,
( j+1)

( j)

A0 = A j , j 0; A(nj) =

( j)

An1 (nj) An1

( j)

1 n

, j 0, n 1.

17. The Wimp Transformation (WIMP)

Let (nj) = A j+n /A j , and dene A(nj) recursively as in (22.2.2); that is,
( j+1)

( j)

A0 = A j , j 0; A(nj) =

( j)

An1 (nj) An1

1

( j)
n

, j 0, n 1.

487

H
Efcient Application of Sequence
Transformations: Summary

In this appendix, we summarize the types of sequences {Am } and series
k=1 ak treated in this
book and point to the sequence transformations appropriate for each type. Our conclusions are
based on numerical comparisons carried out in double- and quadruple-precision arithmetic. It is
worth emphasizing that the differences between the various methods become more pronounced
in quadruple-precision arithmetic. Therefore, we urge the reader to use quadruple-precision arithmetic in comparing the methods.
For convenience, we adopt the shorthand names introduced in Appendix G. In addition, by
L-TRAN and S-TRAN, we mean the L- and S-transformations with r = rar .

1. Exponential Sequences
Let {Am } be such that
Am A +

k ckm as m ,

k=1

where k and ck are scalars independent of m and

ck distinct, ck = 1 for all k; c1 | |c2 | ; lim ck = 0.
k

In case the ck are not known, the only methods that can be used to accelerate the convergence
of {Am } are EPSILON and IDELTA. We must remember, however, that application of the latter
may be problematic in some cases, as exemplied in Theorem 15.3.5.
If the ck are known, the more appropriate method is REP, because it is less expensive than
EPSILON and IDELTA.

2. Exponential Sequences with Conuence

Let {Am } be such that
Am A +

Pk (m)ckm as m ,

k=1

where Pk (m) are polynomials in m, and ck are constants independent of m satisfying

ck distinct, ck = 1 for all k; c1 | |c2 | ; lim ck = 0.
k

In case the ck are not known, the only method that can be used to accelerate the convergence of
{Am } is EPSILON.
If the ck and the degrees of the polynomials Pk (m) (or some upper bounds for them), say qk , are
known, the more appropriate method is REP-CONF, because it is less costly than EPSILON.

488

H Efcient Application of Sequence Transformations: Summary

489

3. Exponential Sequences (Contd)

Am A +

k km as m ,

k=1

where k are constants independent of m and || < 1.

If is not known, only OVERH, WIMP, EPSILON, and IDELTA can be used effectively. Of
these, OVERH and WIMP appear to be the most effective.
If is known, we can also apply REP with ck = k .

4. Asymptotically Polynomial Sequences

Let {Am } be such that
Am A +

k m k as m ,  > 0,

k=1

and is known. The appropriate method for this case is REP-POL( ).

5. Sequences in b(1) /LOG

Let {Am } be such that
Am A +

i m i as m , = 0, 1, . . . , 0 = 0.

i=0

Such sequences also arise as partial sums of innite series

am

k=1

ak , where

ei m i1 as m .

i=0

If is not known, the methods appropriate for such sequences are ILUBKIN, THETA, and
L-TRAN. Here L-TRAN appears to be the most effective. When is known, we can also apply
IMDELTA( , 1) and RHO( , 1).
When approximations with highest possible accuracy in nite-precision arithmetic are desired,
d (1) -TRAN in the GPS mode, that is, with Rl chosen as in

Rl1 + 1 if  Rl1  = Rl1 ,
R0 1, Rl =
l = 1, 2, . . . ; > 1,
 Rl1  otherwise,
produces the best results.

6. Sequences in b(1) /LIN

Let {Am } be such that
Am A + m

i m i as m , = 1, 0 = 0.

i=0

Such sequences also arise as partial sums of innite series

am m

k=1

ak , where

ei m i as m .

i=0

The methods appropriate for such sequences are IDELTA, ILUBKIN, EPSILON, THETA, LTRAN, and S-TRAN. Of these, the last two appear to be the most effective.
When approximations with highest possible accuracy in nite-precision arithmetic are desired,
all these methods can be applied in the APS mode; that is, they can be applied to the subsequences
{Am+ }, where 1 and 0 are appropriate integers. For the same purpose, d (1) -TRAN in the
APS mode, that is, with Rl = (l + 1), 1 integer, produces the best results. For d (1) -TRAN,

490

H Efcient Application of Sequence Transformations: Summary

it is not necessary to choose as an integer, however. Excellent results are produced in the APS
mode also when 1 is not necessarily an integer, and the Rl are chosen as follows:
Rl = (l + 1), l = 0, 1, . . . .

7. Sequences in b(1) /FAC

Let {Am } be such that
Am A + (m!)r m

i m i as m , r > 0 integer, 0 = 0.

i=0

Such sequences also arise as partial sums of innite series

am (m!)r m

k=1

ak , where

ei m i+r as m .

i=0

The methods appropriate for such sequences are IDELTA, ILUBKIN, EPSILON, THETA, LTRAN, and S-TRAN. Of these, the last two appear to be the most effective.

8. Factorially Divergent Sequences in b(1) /FACD

Let {Am } be the sequence of the partial sums of the innite series
am = (m!)r m h(m), h(m)

k=1

ak , where

ei m i as m , r > 0 integer, e0 = 0.

i=0

Then,

k=1

ak is divergent, and
Am (m!)r m

i m i as m .

i=0


mt
When h(m) is independent of and h(m) = m

0 e (t) dt for some integer 0 and some
(t) of exponential order, the divergent series k=1 ak has a (generalized) Borel sum, which, as a
function of , is analytic in the -plane cut along the real interval [0, +). In such a case, IDELTA,
ILUBKIN, EPSILON, THETA, L-TRAN, and S-TRAN can be applied to produce approximations
to the (generalized) Borel sum. Of these transformations, S-TRAN appears to produce the best
results and is followed by L-TRAN.

9. Sums of Sequences in b(1) /LOG/LIN/FAC

When {Am } is the sum of different sequences in b(1) /LOG, b(1) /LIN, and b(1) /FAC, the only
appropriate methods that produce good results appear to be d ( p) -TRAN with some p > 1 always
and EPSILON in certain cases. (IDELTA, ILUBKIN, THETA, L-TRAN, and S-TRAN do not
produce any acceleration in general.)
Important special cases of such sequences are
(i) Am = A +

p


( )

h k (m), h k (m) A0 k , k distinct, k = 0, 1, . . . ,

k=1

(ii) Am = A +

p


( )

km h k (m), h k (m) A0 k , k distinct, k = 1.

k=1

EPSILON is effective in case (ii) but does not produce any acceleration in case (i). It is also not
effective when {Am } is a sum of sequences from both b(1) /LOG and b(1) /LIN.
The APS and GPS strategies can be used effectively wherever necessary. Highest possible accuracy for sequences described in case (i), for example, can be achieved only by d ( p) -TRAN with
GPS.

H Efcient Application of Sequence Transformations: Summary

491

Now, sequences {Am } for which

Am A +

i m i/ p as m , =

i=0

i
, i = 0, 1, . . . , p > 0 integer, 0 = 0,
p

are special instances of case (i) with

k = (k 1)/ p, k = 1, . . . , p. Such sequences also
arise as partial sums of innite series
k=1 ak , where
am

ei m 1i/ p as m .

i=0

( p) , which is discussed in the following. For these sequences,

The sequence {am } is also in b
ILUBKIN and THETA are also effective. When p is known, d( p) -TRAN is effective as well.
When as well as p are known, IMDELTA( , p) and RHO( , p) are successful too. Highest possible accuracy for such sequences too can be achieved only by d ( p) -TRAN and d( p) -TRAN with
GPS.

10. Series

k=1

ak with {ak } b ( p)

The most appropriate methods for accelerating the convergence of the series
( p) , p > 1, that is, with
b
am (m!)r/ p e Q(m)

ei m i/ p as m , r integer, Q(m) =

i=0

p1


k=1

ak with {ak }

i m 1i/ p ,

i=0

seem to be d( p) -TRAN and d ( p) -TRAN. When (i) r < 0, or (ii) r = 0 and limm Q(m) < ,
or (iii) r = 0 and Q(m) 0 and = 1 + i/ p, i = 0, 1, . . . , these am give rise to partial sums
Am that satisfy
Am A + (m!)r/ p e Q(m)

i m +1i/ p as m .

i=0

11. Series

k=1

ak with {ak } b( p)

The only methods that are useful in accelerating the convergence of the series
k=1 ak with
{ak } b( p) , p > 1, appear to be d ( p) -TRAN always and EPSILON in certain cases.
In many instances, am can be shown to be the sum of products of terms that form sequences
in b( pi ) for various values of pi . Heuristics 6.4.16.4.3 can be used to determine the pi and p
conveniently.

12. Power Series

k=1 ck z

k1

with {ck } b( p)

k1
The only methods that are useful in accelerating the convergence of the series
with
k=1 ck z
( p)
( p)
{ck } b , p > 1, appear to be d -TRAN (i.e., rational d-approximants) and EPSILON (i.e.,
Pade approximants).
IDELTA, ILUBKIN, THETA, L-TRAN, and S-TRAN do not produce any acceleration when
p > 1, in general. They can be used when p = 1.
APS can be used effectively wherever necessary.

13. (Generalized) Fourier Series and Series of Special Functions

( p)
Series of the form
for some p, and {k (x)} is a sequence of
k=1 ck k (x), where {ck } b
trigonometric functions [such as k (x) = sin kx] or of special functions [such as k (x) = Pk (x),
Legendre polynomials], which is in b(2) in most cases of interest, can be treated directly by d (2 p) TRAN and EPSILON. Using the complex series approach of Chapter 13, they can be treated at
half the cost by d ( p) -TRAN and EPSILON. (This cost can be reduced further by using the extended
complex series approach when possible.)
IDELTA, ILUBKIN, THETA, L-TRAN, and S-TRAN do not produce any acceleration when
p > 1, in general. When p = 1, they can be used only with the complex series approach.

492

H Efcient Application of Sequence Transformations: Summary

APS can be used effectively wherever necessary.

When k (x) are orthogonal polynomials, the special-purpose methods of Chapter 18, such as
Pade approximants and BakerGammel approximants from orthogonal polynomial expansions,
and their generalizations, can also be used. Their use may be computationally more involved,
however.

Concluding Remarks on Sequence Transformations

We end by collecting some of the information about those sequence transformations that require
no information except the elements of the sequences to be transformed. These are d (m) -TRAN,
d(m) -TRAN, with m = 1, 2, . . . , L-TRAN, S-TRAN, EPSILON, RHO(1, 1), THETA, IDELTA,
ILUBKIN, OVERH, and WIMP.
1. For {Am } b(1) /LOG: L-TRAN is the best, followed by ILUBKIN and THETA, while S-TRAN
. The rest of
is inferior. RHO(1, 1) is effective only for sequences that satisfy {Am } A(2)
0
the methods are useless. For such sequences, all relevant methods are numerically unstable.
For numerically stable results that have very high accuracy, d (1) -TRAN with GPS should be
used.
2. For {Am } b(1) /LIN/FAC: L-TRAN and S-TRAN are the best, followed by THETA and
ILUBKIN, which are followed by EPSILON and IDELTA. In case {Am } b(1) /LIN, all these
methods are numerically unstable near points of singularity of the limit or antilimit. In such
situations, d (1) -TRAN with APS should be used or L-TRAN and S-TRAN should be applied
to some subsequence {Am+ }, where and are integers.
3. For {Am } b(1) /FACD: S-TRAN is the best, followed by L-TRAN. THETA, ILUBKIN, EPSILON, and IDELTA are inferior.
4. For {Am } b ( p) , limm Am+1 /Am = 1, p 2: d ( p) -TRAN and d( p) -TRAN are the
best, followed by THETA and ILUBKIN. The other methods are useless. Again, numerically
stable and high-accuracy approximations are obtained from d ( p) -TRAN and d( p) -TRAN with
GPS.

p
5. For {Am }, with Am = k=1
Hm(k) , {Hm(k) } b(1) /LOG, p 2: d ( p) -TRAN is the only useful
method.

p
6. For {Am }, with Am = k=1
Hm(k) , {Hm(k) } b(1) /LIN, p 2: d ( p) -TRAN is the best, followed
by EPSILON. The other methods are useless.
7. For general {Am } b( p) , p 2: d ( p) -TRAN is the only useful method.
8. For general exponential sequences (with or without conuence):
EPSILON is the only useful


km
method. For exponential sequences for which Am A +
k=1 k , || < 1, such as those
arising from the xed-point iterative solution of nonlinear equations, WIMP and OVERH may
be preferable. For all types of exponential sequences, the remaining methods are useless.
It is clear that, to accelerate the convergence of the different classes of sequences mentioned
here, it is sufcient to have available the methods d (m) -TRAN, d(m) -TRAN, with m = 1, 2, . . . ,
EPSILON, L-TRAN, S-TRAN, and OVERH or WIMP. Recall also that L-TRAN (as the utransformation) is already d (1) -TRAN (and also d(1) -TRAN), with Rl = l + 1.
When additional quantitative information on the nature of {Am } is available, provided they are
applicable, REP, REP-CONF, GREP(m) , and REP-POL( ) are more economical than others.

I
FORTRAN 77 Program for the d (m) -Transformation

I.1 General Description

In this appendix, we provide a

FORTRAN 77 code that implements the d (m) -transformation on
(m)
real innite series of the form
-algorithm. After modifying the type declak=1 ak via the W
rations in a suitable fashion, this code can be adapted to complex series as well. Here we use the
notation and terminology of Section 7.3.
The most important part of the code we give here is SUBROUTINE WMALG, which implements
the W(m) -algorithm with the normal ordering of the k (t)t i . This subroutine is exactly the same as
that given originally in Ford and Sidi [87, Appendix B] and can be included in any implementation
of GREP(m) , such as the D (m) -, D (m) -, and D (m) -transformations. (Recall that, when m = 1, the
W(m) -algorithm becomes the W-algorithm, as mentioned in Chapter 7. Recall also that, when m = 1
and Rl = l + 1, l = 0, 1, . . . , the d (m) -transformation reduces to the Levin u-transformation.)
SUBROUTINE WMALG produces the approximations APPROX(J,P), 0 J+P LMAX, where
APPROX(J,P)=A(PJ) .
Computation of the Rl and gk (l) is carried out in SUBROUTINE MLTAG, which is somewhat
different from that of Ford and Sidi [87, Appendix B] with regard to how the k (t) are ordered and
how the Rl are prescribed:
To avoid (as much as possible) division by vanishing g1 (l), which may arise when some of the
ak may be zero (this may happen with Fourier series, for example), the k (t) are ordered as
follows:
k (t) = n mk+1 mk an , k = 1, . . . , m.
Thus, 1 (t) = n m m1 an , with t = 1/n.
The Rl in the d (m) -transformation are assigned in one of the following two ways:
1. Arithmetic Progression Sampling (APS). For some 1, set
Rl = (l + 1), l = 0, 1, . . . .

(I.1.1)

2. Geometric Progression Sampling (GPS). For some > 1, set


R0 = 1, Rl =

Rl1 + 1 if  Rl1  = Rl1 ,

l = 1, 2, . . . .
 Rl1  otherwise,

(I.1.2)

Finally, the elements ak , k = 1, 2, . . . , of the innite series are computed by FUNCTION CF

via ak = CF(K).
The user is to supply the parameters m, , and . In the computer program, we have m = M,
= KAPPA, and = SIGMA. We note that, to invoke APS via (I.1.1), SIGMA = 1 must be
prescribed. When invoking GPS via (I.1.2), KAPPA can be assigned any value.

493

I FORTRAN 77 Program for the d (m) -Transformation

494

The computer code we give here has in it the following convergent series as test cases, with the
appropriate m, , and .

1
2
(1)
= 6
m=1
n2

n=1

1
1
(2)
(
+
)
=
4.25730941553371477982098273457009

m=2
n 3/2
n2



n=1
cos n


m=2
= log 2 sin 2
(3)
n=1
n

(1)n1
(4)
=
log
2
m=1
n=1
n

(1)(n1)/2
(5)
= S2
m=2
n=1
n

(1)(n1)/3
= S3
m=3
(6)
n=1
n

(1)(n1)/4
= S4
m=4
(7)
n=1
n


exp(0.1n 1/2 )
e0.1
m=2
(8)
n=1 (cn cn+1 ) = c1 = 2 ; cn =
1+n 1/2


exp(0.1n 2/3 +0.2n 1/3 )
e0.1
m=3
(9)
n=1 (cn cn+1 ) = c1 = 2 ; cn =
1+n 1/3
(10) 12 + 1

1
4

1
3

1
6

(11)

1+

1+

1
7
1
4

(12)

1
5
1
2

1
3
1
3

1
2
1
5

1
4
1
7

= log 2

1
5

+ =
+

1
9

1
11

3
2

m=2
m=3

log 2

1
6

1
8

+ =

1
2

log 6.

m=5

In series (3), the value of should be provided by the user through the variable THETA in the
main program.
In the series (5)(7), S2 , S3 , and S4 are special cases of (25.9.1), namely,
Sq =


(1)k/q
k=0

k+1

q1
1

k
log 2 +
, q = 1, 2, . . . .
tan
q
2q k=1
2q

The series (10)(12) are rearrangements of series (4), which converges conditionally.
For gaining experience with the use of the d-transformation, we urge the reader to run this code
in both double- and quadruple-precision arithmetic, with the values of KAPPA and SIGMA as
recommended in the main program and with other values as well. (As the code is already in double
precision, it can be run in quadruple precision without any changes.)
Following these series, we suggest that the reader apply the code to other series that diverge,
such as

1/2
(13)
n
antilimit: ( 12 )
m=1

n=1

1
n1 1/2
3/2
n
antilimit: (1 2 ) ( 2 ) m = 1
(14)
n=1 (1)

(2n+1)(s)n
3
(15)
P
(x),
s

antilimit: (1 x)s
m=2
n=0 2s (1s)n+1 n
4
In series (15), Pn (x) are the Legendre polynomials and, as before,
(u)0 = 1, (u)n = u(u + 1) (u + n 1), n = 1, 2, . . . .
For s < 3/4, this series converges to (1 x)s provided x [1, 1). For all other x in the complex
plane cut along [1, +), it diverges for every s with antilimit (1 x)s . See Davis [62, p. 327,
Exercise 16].

I.2 The Code

CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
C
THIS PROGRAM APPLIES THE LEVIN-SIDI d^(m)-TRANSFORMATION TO INFINITE
C
SERIES (SUM OF CF(I) FOR I=1 UP TO I=INFINITY) OF VARIOUS FORMS.
C
C
THE d^(m)-TRANSFORMATION IS IMPLEMENTED BY THE W^(m)-ALGORITHM OF
C
FORD AND SIDI.
C
C
THE FOLLOWING ARE NEEDED AS INPUT AND SHOULD BE PROVIDED BY THE
C
USER VIA THE PARAMETER STATEMENTS: NP, M, LMAX, POSITIVE INTEGERS,
C
KAPPA, SIGMA, DOUBLE PRECISION CONSTANTS .GE. 1D0.

I.2 The Code

C
WHEN NP=3 THETA SHOULD ALSO BE PROVIDED.
C
C
THE USER IS TO PROVIDE CF(I) AS A FUNCTION SUBPROGRAM.
C
C
MDIM.GE.M AND LDIM.GE.LMAX SHOULD ALWAYS BE SATISFIED. MDIM AND
C
LDIM CAN BE INCREASED BY THE USER TO ACCOMMODATE LARGE VALUES OF
C
M AND LMAX.
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
C
THE FOLLOWING CASES ARE INCLUDED IN THE PRESENT CF(I):
C
IF NP=1, THEN M=1, KAPPA=1D0, SIGMA=1.3D0
C
IF NP=2, THEN M=2, KAPPA=1D0, SIGMA=1.3D0
C
IF NP=3, THEN M=2, KAPPA=?, SIGMA=1D0 (THIS IS A FOURIER SERIES.)
C
(KAPPA DEPENDS ON X=ABS(THETA), WHERE THETA IS THE ANGLE.)
C
(FOR X LARGE KAPPA=1D0 IS ENOUGH.)
C
(FOR X SMALL TRY KAPPA>1D0)
C
(E.G., FOR X=PI/3, KAPPA=1D0)
C
(E.G., FOR X=PI/6, KAPPA=2D0)
C
(E.G., FOR X=PI/50, KAPPA=20D0)
C
IF NP=4, THEN M=1, KAPPA=1D0, SIGMA=1D0
C
IF NP=5, THEN M=2, KAPPA=1D0, SIGMA=1D0
C
IF NP=6, THEN M=3, KAPPA=1D0, SIGMA=1D0
C
IF NP=7, THEN M=4, KAPPA=1D0, SIGMA=1D0
C
IF NP=8, THEN M=2, KAPPA=1D0, SIGMA=1.15D0
C
IF NP=9, THEN M=3, KAPPA=1D0, SIGMA=1.15D0
C
IF NP=10, THEN M=2, KAPPA=1D0, SIGMA=1D0
C
IF NP=11, THEN M=3, KAPPA=2D0, SIGMA=1D0
C
IF NP=12, THEN M=5, KAPPA=4D0, SIGMA=1D0
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
C
NOTE THAT IRL(L)=R_L , L=0,1,..., OF THE d-TRANSFORMATION, AND
C
APPROX(J,P) IS THE APPROXIMATION RETURNED BY THE d^(M)-TRANSFORMATION
C
THAT IS OBTAINED FROM THE PARTIAL SUMS A_{R_0}, A_{R_1},...,
C
A_{R_{J+P}} OF THE GIVEN SERIES.
C
C
IN THE OUTPUT ERROR(L,0) IS THE ABSOLUTE ERROR IN THE PARTIAL SUM
C
A_{R_L}, WHILE ERROR(0,L) IS THE ABSOLUTE ERROR IN APPROX(0,L).
C
THE EXACT SUM OF THE SERIES IS CONTAINED IN THE VARIABLE RESULT.
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
IMPLICIT DOUBLE PRECISION (A-H,O-Z)
DOUBLE PRECISION KAPPA,KAPPAP
PARAMETER (MDIM=6,LDIM=120,EPSDIV=1D-77)
PARAMETER (NP=12,M=5,LMAX=100,KAPPA=4D0,SIGMA=1.0D0)
DIMENSION G(MDIM),PSIAI(0:LDIM,2,2),BIGPSI(0:LDIM,MDIM,2)
DIMENSION PSIG(0:MDIM,2:MDIM+1,2),APPROX(0:LDIM,0:LDIM)
EXTERNAL MLTAG
COMMON /SIGKAP/SIGMAP,KAPPAP
COMMON /NP/NPP/THETA/THETA
COMMON /RL/IRL(0:LDIM)
WRITE(6,111)
111 FORMAT(SUMMATION OF CF(I) FROM I=1 TO I=INFINITY,//)
NPP=NP
PI=DACOS(-1D0)
THETA=PI/50D0
SIGMAP=SIGMA
KAPPAP=KAPPA
CALL EXACT(RESULT)
CALL WMALGM(MDIM,LDIM,M,LMAX,MLTAG,G,PSIAI,BIGPSI,PSIG,
*
APPROX,EPSDIV)
WRITE(6,121)

495

496
121

20
131

I FORTRAN 77 Program for the d (m) -Transformation

FORMAT(/,3X,L,5X,R_L,7X,ERROR(L,0),5X,ERROR(0,L))
DO 20 L=0,LMAX
ER1=DABS(APPROX(L,0)-RESULT)
ER2=DABS(APPROX(0,L)-RESULT)
WRITE(6,131) L,IRL(L),ER1,ER2
CONTINUE
FORMAT(I4,2X,I6,2X,1P,2D15.3)
WRITE (6,*) EXACT SUM =,RESULT
STOP
END

SUBROUTINE WMALGM(MDIM,LDIM,M,LMAX,MLTAG,G,PSIAI,BIGPSI,PSIG,
*
APPROX,EPSDIV)
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
C
THIS SUBROUTINE GIVES THE IMPLEMENTATION OF SIDIS GENERALIZED
C
RICHARDSON EXTRAPOLATION PROCESS GREP^(m) VIA THE W^(m)-ALGORITHM
C
OF FORD AND SIDI.
C
C
THE APPROXIMATIONS TO THE LIMIT OR ANTILIMIT OF THE SEQUENCE IN
C
QUESTION ARE CONTAINED IN THE ARRAY APPROX. IN PARTICULAR,
C
APPROX(0,L), L=0,1,..., SEEM TO BE THE BEST IN GENERAL.
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
IMPLICIT DOUBLE PRECISION(A-H,O-Z)
INTEGER CUR,TEMP,P,PM,Q,QP
DIMENSION G(MDIM),PSIAI(0:LDIM,2,2)
DIMENSION BIGPSI(0:LDIM,MDIM,2),PSIG(0:MDIM,2:MDIM+1,2)
DIMENSION APPROX(0:LDIM,0:LDIM)
CUR=1
TEMP=2
CALL MLTAG(M,0,T,A,G)
APPROX(0,0)=A
PSIAI(0,1,CUR)=A/G(1)
PSIAI(0,2,CUR)=1D0/G(1)
BIGPSI(0,1,CUR)=1D0/T
DO 10 K=2,M
PSIG(0,K,CUR)=G(K)/G(1)
10
CONTINUE
PSIG(0,M+1,CUR)=T
DO 80 L=1,LMAX
CALL MLTAG(M,L,T,A,G)
APPROX(L,0)=A
PSIAI(0,1,TEMP)=A/G(1)
PSIAI(0,2,TEMP)=1D0/G(1)
BIGPSI(0,1,TEMP)=1D0/T
DO 20 K=2,M
PSIG(0,K,TEMP)=G(K)/G(1)
20
CONTINUE
PSIG(0,M+1,TEMP)=T
SIGN=-1D0
DO 60 P=1,L
IF (P.LE.M) THEN
D=PSIG(P-1,P+1,TEMP)-PSIG(P-1,P+1,CUR)
DO 30 I=P+2,M+1
PSIG(P,I,TEMP)=(PSIG(P-1,I,TEMP)-PSIG(P-1,I,CUR))/D
30
CONTINUE
END IF
IF (P.LT.M) THEN
BIGPSI(P,P+1,TEMP)=SIGN/PSIG(P,M+1,TEMP)

I.2 The Code

40

50
60

101
70

80

SIGN=-SIGN
END IF
PM=MIN0(P-1,M-1)
DO 40 Q=1,PM
PS=BIGPSI(P-2,Q,CUR)
DQ=PS/BIGPSI(P-1,Q,CUR)-PS/BIGPSI(P-1,Q,TEMP)
QP=Q+1
BIGPSI(P,QP,TEMP)=(BIGPSI(P-1,QP,TEMP)-BIGPSI(P-1,QP,CUR))/DQ
CONTINUE
IF (P.GT.M) THEN
PS=BIGPSI(P-2,M,CUR)
D=PS/BIGPSI(P-1,M,CUR)-PS/BIGPSI(P-1,M,TEMP)
END IF
BIGPSI(P,1,TEMP)=(BIGPSI(P-1,1,TEMP)-BIGPSI(P-1,1,CUR))/D
DO 50 I=1,2
PSIAI(P,I,TEMP)=(PSIAI(P-1,I,TEMP)-PSIAI(P-1,I,CUR))/D
CONTINUE
CONTINUE
DO 70 P=1,L
J=L-P
IF (DABS(PSIAI(P,2,TEMP)).GE.EPSDIV) THEN
APPROX(J,P)=PSIAI(P,1,TEMP)/PSIAI(P,2,TEMP)
ELSE
APPROX(J,P)=1D75
WRITE(6,101)J,P
FORMAT(1X,APPROX(,I3,,,I3,) IS NOT DEFINED)
END IF
CONTINUE
JJ=CUR
CUR=TEMP
TEMP=JJ
CONTINUE
RETURN
END

SUBROUTINE MLTAG(M,L,T,A,G)
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
C
THIS SUBROUTINE IS CALLED BY SUBROUTINE WMALGM AND PROVIDES THE
C
LATTER WITH THE NECESSARY INPUT FOR THE d-TRANSFORMATION.
C
THE CONSTANT LDIM IN THE PARAMETER STATEMENT BELOW MUST BE THE SAME
C
AS THAT IN THE MAIN PROGRAM.
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
IMPLICIT DOUBLE PRECISION(A-H,O-Z)
DOUBLE PRECISION KAPPA
PARAMETER (LDIM=120)
DIMENSION G(M)
COMMON /SIGKAP/SIGMA,KAPPA
COMMON /RL/IRL(0:LDIM)
IF (SIGMA.EQ.1D0) THEN
LSUMP=KAPPA*L+1D-10
LSUM=KAPPA*(L+1)+1D-10
END IF
IF (SIGMA.GT.1D0) THEN
IF (L.EQ.0) THEN
LSUMP=0
LSUM=1
ELSE
LSUM=1

497

498

10

20

30

40
50

60

70

I FORTRAN 77 Program for the d (m) -Transformation

LSUMP=1
DO 10 I=1,L
IR=SIGMA*LSUM+1D-10
IF (IR.LE.LSUM) THEN
LSUM=LSUM+1
ELSE
LSUM=IR
END IF
IF (I.EQ.L-1) LSUMP=LSUM
CONTINUE
END IF
END IF
IRL(L)=LSUM
IF (L.EQ.0) A=0
DO 20 I=LSUMP+1,LSUM
A=A+CF(I)
CONTINUE
P=LSUM
T=1D0/P
DO 30 K=1,M
G(K)=CF(LSUM+K-1)
CONTINUE
DO 50 I=2,M
DO 40 J=M,I,-1
G(J)=G(J)-G(J-1)
CONTINUE
CONTINUE
DO 60 K=1,M
G(K)=G(K)*P
P=P*LSUM
CONTINUE
DO 70 K=1,M/2
ST=G(K)
G(K)=G(M-K+1)
G(M-K+1)=ST
CONTINUE
RETURN
END

FUNCTION CF(I)
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
C
CF(I) IS THE I-TH TERM OF THE INFINITE SERIES, I=1,2,...
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
IMPLICIT DOUBLE PRECISION (A-H,O-Z)
COMMON /NP/NP/THETA/THETA
UU(X)=DEXP(-0.1D0*DSQRT(X))/(1+DSQRT(X))
VV(X)=DEXP(-0.1D0*X**(2D0/3D0)+0.2D0*X**(1D0/3D0))
*
/(1+X**(1D0/3D0))
FI=DFLOAT(I)
II=I-1
IF (NP.EQ.1) CF=1D0/(FI)**2
IF (NP.EQ.2) CF=1D0/(FI)**1.5D0+1D0/(FI)**2
IF (NP.EQ.3) CF=DCOS((FI)*THETA)/(FI)
IF (NP.EQ.4) CF=(-1)**(II)/(FI)
IF (NP.EQ.5) CF=(-1)**(II/2)/(FI)
IF (NP.EQ.6) CF=(-1)**(II/3)/(FI)
IF (NP.EQ.7) CF=(-1)**(II/4)/(FI)
IF (NP.EQ.8) CF=UU(FI)-UU(FI+1)

I.2 The Code

IF (NP.EQ.9) CF=VV(FI)-VV(FI+1)
IF (NP.EQ.10) THEN
IF (I-2*(I/2).EQ.1) CF=-1D0/(I+1)
IF (I-2*(I/2).EQ.0) CF=1D0/(I-1)
END IF
IF (NP.EQ.11) THEN
IF (I-3*(I/3).EQ.1) CF=3D0/(4*I-1)
IF (I-3*(I/3).EQ.2) CF=3D0/(4*I+1)
IF (I-3*(I/3).EQ.0) CF=-3D0/(2*I)
END IF
IF (NP.EQ.12) THEN
IF (I-5*(I/5).EQ.1) CF=5D0/(6*I-1)
IF (I-5*(I/5).EQ.2) CF=5D0/(6*I+3)
IF (I-5*(I/5).EQ.3) CF=5D0/(6*I+7)
IF (I-5*(I/5).EQ.4) CF=-5D0/(4*I-6)
IF (I-5*(I/5).EQ.0) CF=-5D0/(4*I)
END IF
RETURN
END
SUBROUTINE EXACT(RESULT)
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
C
RESULT IS THE EXACT SUM OF THE INFINITE SERIES.
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
IMPLICIT DOUBLE PRECISION (A-H,O-Z)
COMMON /NP/NP/THETA/THETA
PI=DACOS(-1D0)
IF (NP.EQ.1) RESULT=PI**2/6D0
IF (NP.EQ.2) RESULT=4.25730941553371477982098273457009D0
IF (NP.EQ.3) RESULT=-DLOG(DABS(2D0*DSIN(THETA/2)))
IF (NP.EQ.4) RESULT=DLOG(2D0)
IF (NP.EQ.5) RESULT=DLOG(2D0)/2+PI/4
IF (NP.EQ.6) RESULT=DLOG(2D0)/3+2*PI/(3D0**1.5D0)
IF (NP.EQ.7) RESULT=DLOG(2D0)/4
*
+PI/8*(DTAN(PI/8)+DTAN(PI/4)+DTAN(3*PI/8))
IF (NP.EQ.8) RESULT=DEXP(-0.1D0)/2
IF (NP.EQ.9) RESULT=DEXP(0.1D0)/2
IF (NP.EQ.10) RESULT=DLOG(2D0)
IF (NP.EQ.11) RESULT=1.5D0*DLOG(2D0)
IF (NP.EQ.12) RESULT=DLOG(6D0)/2
RETURN
END

499

Bibliography

[1] M. Abramowitz and I.A. Stegun. Handbook of Mathematical Functions with Formulas,
Graphs, and Mathematical Tables. Number 55 in Nat. Bur. Standards Appl. Math. Series.
US Government Printing Ofce, Washington, D.C., 1964.
[2] A.C. Aitken. On Bernoullis numerical solution of algebraic equations. Proc. Roy. Soc.
Edinburgh, 46:289305, 1926.
[3] A. Alaylioglu, G.A. Evans, and J. Hyslop. The evaluation of oscillatory integrals with
innite limits. J. Comp. Phys., 13:433438, 1973.
[4] N.C. Albertsen, G. Jacobsen, and S.B. Srensen. Nonlinear transformations for
accelerating the convergence of M-dimensional series. Math. Comp., 41:623634, 1983.
[5] R. Almgren, W.-S. Dai, and V. Hakim. Scaling behavior in anisotropic Hele-Shaw ow.
Phys. Rev. Lett., 71:34613464, 1993.
[6] A. Ambroladze and H. Wallin. Convergence rates of Pade and Pade-type approximants.
J. Approx. Theory, 86:310319, 1996.
[7] A. Ambroladze and H. Wallin. Convergence of rational interpolants with preassigned
poles. J. Approx. Theory, 89:238256, 1997.
[8] A. Ambroladze and H. Wallin. Pade-type approximants of Markov and meromorphic
functions. J. Approx. Theory, 88:354369, 1997.
[9] A. Ambroladze and H. Wallin. Extremal polynomials with preassigned zeros and rational
approximants. Constr. Approx., 14:209229, 1998.
[10] W.F. Ames. Numerical Methods for Partial Differential Equations. Academic Press,
New York, second edition, 1977.
[11] R.J. Arms and A. Edrei. The Pade tables and continued fractions generated by totally
positive sequences. In H. Shankar, editor, Mathematical Essays, pages 121, Athens,
Ohio, 1970. Ohio University Press.
[12] T.A. Atchison and H.L. Gray. Nonlinear transformations related to the evaluation of
improper integrals II. SIAM J. Numer. Anal., 5:451459, 1968.
[13] K.E. Atkinson. An Introduction to Numerical Analysis. Wiley, New York, second edition,
1989.
[14] A. Averbuch, E. Braverman, R. Coifman, M. Israeli, and A. Sidi. Efcient computation of
oscillatory integrals via adaptive multiscale local Fourier bases. Appl. Comput. Harmonic
Anal., 9:1953, 2000.
[15] G.A. Baker, Jr. Essentials of Pade Approximants. Academic Press, New York, 1975.
[16] G.A. Baker, Jr. and P.R. Graves-Morris. Pade Approximants. Cambridge University Press,
Cambridge, second edition, 1996.
[17] M.N. Barber and C.J. Hamer. Extrapolation of sequences using a generalized
epsilon-algorithm. J. Austral. Math. Soc., Series B, 23:229240, 1982.
[18] F.L. Bauer. The quotient-difference and epsilon algorithms. In R.E. Langer, editor, On
Numerical Approximation, pages 361370. The University of Wisconsin Press, 1959.
[19] F.L. Bauer. Nonlinear sequence transformations. In H.L. Garabedian, editor,
Approximation of Functions, pages 134151. Elsevier, 1965.

501

502

Bibliography

[20] F.L. Bauer, H. Rutishauser, and E. Stiefel. New aspects in numerical quadrature. In
Experimental Arithmetic, High Speed Computing, and Mathematics, pages 199218,
Providence, Rhode Island, 1963. American Mathematical Society.
[21] C.M. Bender and S.A. Orszag. Advanced Mathematical Methods for Scientists and
Engineers. McGraw-Hill, New York, 1978.
[22] R. Bhattacharya, D. Roy, and S. Bhowmick. On the regularity of the Levin u-transform.
Comput. Phys. Comm., 55:297301, 1989.
[23] S. Bhowmick, R. Bhattacharya, and D. Roy. Iterations of convergence accelerating
nonlinear transforms. Comput. Phys. Comm., 54:3146, 1989.
[24] W.G. Bickley and J.C.P. Miller. The numerical summation of slowly convergent series of
positive terms. Phil. Mag., 7th Ser., 22:754767, 1936.
[25] G.D. Birkhoff and W.J. Trjitzinsky. Analytic theory of singular difference equations. Acta
Math., 60:189, 1932.
[26] P.E. Bjrstad, G. Dahlquist, and E.H. Grosse. Extrapolation of asymptotic expansions by a
modied Aitken 2 -formula. BIT, 21:5665, 1981.
[27] M. Blakemore, G.A. Evans, and J. Hyslop. Comparison of some methods for evaluating
innite range oscillatory integrals. J. Comp. Phys., 22:352376, 1976.
[28] P.B. Borwein. Pade approximants for the q-elementary functions. Constr. Approx.,
4:391402, 1988.
[29] J.P. Boyd. The rate of convergence of Chebyshev polynomials for functions which have
asymptotic power series about one endpoint. Math. Comp., 37:189195, 1981.
[30] J.P. Boyd. A Chebyshev polynomial rate-of-convergence theorem for Stieltjes functions.
Math. Comp., 39:201206, 1982.
[31] C. Brezinski. Application du -algorithme a la quadrature numerique. C. R. Acad. Sci.
Paris, 270 A:12521253, 1970.
[32] C. Brezinski. Acceleration de suites a` convergence logarithmique. C. R. Acad. Sci. Paris,
273 A:727730, 1971.
[33] C. Brezinski. Convergence dune forme conuente de l-algorithme. C. R. Acad. Sci.
Paris, 273 A:582585, 1971.

[34] C. Brezinski. Etudes

sur les - et -algorithmes. Numer. Math., 17:153162, 1971.
[35] C. Brezinski. L-algorithme et les suites totalement monotones et oscillantes. C. R. Acad.
Sci. Paris, 276 A:305308, 1973.
[36] C. Brezinski. Acceleration de la Convergence en Analyse Numerique. Springer-Verlag,
Berlin, 1977.
[37] C. Brezinski. A general extrapolation algorithm. Numer. Math., 35:175187, 1980.
[38] C. Brezinski. History of Continued Fractions and Pade Approximants. Springer-Verlag,
Berlin, 1990.
[39] C. Brezinski. A Bibliography on Continued Fractions, Pade Approximation, Extrapolation
and Related Subjects. Prensas Universitarias de Zaragoza, Zaragoza, 1991.
[40] C. Brezinski and M. Crouzeix. Remarques sur le procede 2 dAitken. C. R. Acad. Sci.
Paris, 270 A:896898, 1970.
[41] C. Brezinski and M. Redivo Zaglia. Extrapolation Methods: Theory and Practice.
North-Holland, Amsterdam, 1991.
[42] N.G. de Bruijn. Asymptotic Methods in Analysis. North-Holland, Amsterdam, 1970.
[43] R. Bulirsch and J. Stoer. Fehlerabschatzungen und Extrapolation mit rationalen
Funktionen bei Verfahren vom Richardson-Typus. Numer. Math., 6:413427, 1964.
[44] R. Bulirsch and J. Stoer. Asymptotic upper and lower bounds for results of extrapolation
methods. Numer. Math., 8:93104, 1966.
[45] R. Bulirsch and J. Stoer. Numerical treatment of ordinary differential equations by
extrapolation methods. Numer. Math., 8:113, 1966.
[46] R. Bulirsch and J. Stoer. Numerical quadrature by extrapolation. Numer. Math.,
9:271278, 1967.
[47] E.W. Cheney. Introduction to Approximation Theory. McGraw-Hill, New York, 1966.
[48] J.S.R. Chisholm. Rational approximants dened from double power series. Math. Comp.,
27:841848, 1973.
[49] J.S.R. Chisholm, A. Genz, and G.E. Rowlands. Accelerated convergence of sequences of
quadrature approximations. J. Comp. Phys., 10:284307, 1972.

Bibliography

503

[50] J.S.R. Chisholm and J. McEwan. Rational approximants dened from power series in n
variables. Proc. Roy. Soc. London, A 336:421452, 1974.
[51] S. Christiansen. Numerical solution of an integral equation with a logarithmic kernel. BIT,
11:276287, 1971.
[52] S. Christiansen. Derivation and investigation of fth order quadrature formulas for
biharmonic boundary integral operators. Appl. Numer. Math., 37:145159, 2001.
[53] C.W. Clenshaw and K. Lord. Rational approximations from Chebyshev series. In
B.K.P. Scaife, editor, Studies in Numerical Analysis, pages 95113, London, 1974.
Academic Press.
[54] A.M. Cohen and D. Levin. Accelerating innite products. Numer. Algorithms,
22:157165, 1999.
[55] R. Coifman, M. Goldberg, T. Hrycak, M. Israeli, and V. Rokhlin. An improved operator
expansion algorithm for direct and inverse scattering computations. Waves Random
Media, 9:441457, 1999.
[56] F. Cordellier. Caracterisation des suites que la premi`ere e tape du -algorithme. C. R. Acad.
Sci. Paris, 284 A:389392, 1977.
[57] F. Cordellier. Demonstration algebrique de lextension de lidentite de Wynn aux tables de
Pade non normales. In L. Wuytack, editor, Pade Approximation and Its Applications,
number 765 in Lecture Notes in Mathematics, pages 3660, Berlin, 1977.
Springer-Verlag.
[58] K.S. Crump. Numerical inversion of Laplace transforms using a Fourier series
approximation. Journal of the ACM, 23:8996, 1976.
[59] A. Cuyt. Pade Approximations for Operators: Theory and Applications. Springer-Verlag,
Berlin, 1984.
[60] A. Cuyt. A multivariate convergence theorem of the de Montessus de Ballore type.
J. Comp. Appl. Math., 32:4757, 1990.
[61] A. Cuyt. Multivariate partial NewtonPade and NewtonPade type approximations.
J. Approx. Theory, 72:301316, 1993.
[62] P.J. Davis. Interpolation and Approximation. Dover, New York, 1975.
[63] P.J. Davis and P. Rabinowitz. Methods of Numerical Integration. Academic Press,
New York, second edition, 1984.
[64] R. de Montessus de Ballore. Sur les fractions continue algebriques. Bull. Soc. Math.
France, 30:2836, 1902.
[65] J.P. Delahaye. Sequence Transformations. Springer-Verlag, Berlin, 1988.
[66] J.P. Delahaye and B. Germain-Bonne. Resultats negatifs en acceleration de la
convergence. Numer. Math., 35:443457, 1980.
[67] J.P. Delahaye and B. Germain-Bonne. The set of logarithmically convergent sequences
cannot be accelerated. SIAM J. Numer. Anal., 19:840844, 1982.
[68] J.E. Drummond. A formula for accelerating the convergence of a general series. Bull.
Austral. Math. Soc., 6:6974, 1972.
[69] J.E. Drummond. Summing a common type of slowly convergent series of positive terms.
J. Austral. Math. Soc., Series B, 19:416421, 1976.
[70] R. Dubner and J. Abate. Numerical inversion of Laplace transforms by relating them to
the nite Fourier cosine transform. Journal of the ACM, 15:115123, 1968.
[71] F. Durbin. Numerical inversion of Laplace transforms: an efcient improvement of
Dubner and Abates method. Computer J., 17:371376, 1968.
[72] A. Edrei. Proof of a conjecture of Schonberg on the generating function of a totally
positive sequence. Canad. J. Math., 5:8694, 1953.
[73] U.T. Ehrenmark. A note on an extension of extrapolative techniques for a class of innite
oscillatory integrals. BIT, 30:152155, 1990.
[74] U.T. Ehrenmark. On computing uniformly valid approximations for viscous waves on a
plane beach. J. Comp. Appl. Math., 50:263281, 1994.
[75] U.T. Ehrenmark. The numerical inversion of two classes of KontorovichLebedev
transforms by direct quadrature. J. Comp. Appl. Math., 61:4372, 1995.
[76] D. Elliott. Sigmoidal transformations and the trapezoidal rule. J. Austral. Math. Soc.,
Series B (E), 40:E77E137, 1998.
[77] A. Erdelyi. Asymptotic Expansions. Dover, New York, 1956.

504

Bibliography

[78] T.O. Espelid. Integrating singularities using non-uniform subdivision and extrapolation. In
H. Brass and G. Hammerlin, editors, Numerical Integration IV, number 112 in ISNM,
pages 7789, Basel, 1993. Birkhauser.
[79] T.O. Espelid. On integrating vertex singularities using extrapolation. BIT, 34:6279, 1994.
[80] G.A. Evans and J. Hyslop A.P.G. Morgan. An extrapolation procedure for the evaluation
of singular integrals. Intern. J. Computer Math., 12:251265, 1983.
[81] E. Fabry. Sur les points singuliers dune fonction donnee par son developpement en serie
et sur limpossibilite du prolongement analytique dans les cas tr`es gen`eraux. Ann. Ecole
Normale (3), 13:107114, 1896.
[82] G. Fainstein, A. Sidi, M. Israeli, and Y. Tsur-Lavie. Application of boundary integral
equations to the solution of stresses around a shallow circular hole. In I.W. Farmer,
J.J.K. Daemen, C.S. Desai, C.E. Glass, and S.P. Neumann, editors, 28th U.S. Symposium
in Rock Mechanics, Tucson, Arizona, pages 745755, 1987.
[83] T. Fessler, W.F. Ford, and D.A. Smith. HURRY: An acceleration algorithm for scalar
sequences and series. ACM Trans. Math. Software, 9:346354, 1983.
[84] J. Fleischer. Analytic continuation of scattering amplitudes and Pade approximants.
Nuclear Phys. B, 37:5976, 1972.
[85] J. Fleischer. Generalizations of Pade approximants. In P.R. Graves-Morris, editor, Pade
Approximants: Lectures delivered at a summer school held at the University of Kent,
July 1972, pages 126131, Bath, 1973. Pitman Press.
[86] J. Fleischer. Nonlinear Pade approximants for Legendre series. J. Math. Phys.,
14:246248, 1973.
[87] W.F. Ford and A. Sidi. An algorithm for a generalization of the Richardson extrapolation
process. SIAM J. Numer. Anal., 24:12121232, 1987.
[88] G. Freud. Orthogonal Polynomials. Pergamon Press, New York, 1971.
[89] P.A. Frost and E.Y. Harper. An extended Pade procedure for constructing global
approximations from asymptotic expansions: An application with examples. SIAM Rev.,
18:6291, 1976.
[90] B. Gabutti. An algorithm for computing generalized Eulers transformations of series.
Computing, 34:107116, 1985.
[91] J.L. Gammel. Review of two recent generalizations of the Pade approximant. In P.R.
Graves-Morris, editor, Pade Approximants and Their Applications, pages 39, New York,
1973. Academic Press.
[92] C.R. Garibotti and F.F. Grinstein. Recent results relevant to the evaluation of innite
series. J. Comp. Appl. Math., 9:193200, 1983.
[93] K.O. Geddes. Block structure in the ChebyshevPade table. SIAM J. Numer. Anal.,
18:844861, 1981.
[94] A. Genz. Application of the -algorithm to quadrature problems. In P.R. Graves-Morris,
editor, Pade Approximants and Their Applications, pages 105116, New York, 1973.
Academic Press.
[95] K. Georg. Approximation of integrals for boundary element methods. SIAM J. Sci. Statist.
Comput., 12:443453, 1991.
[96] B. Germain-Bonne. Transformations de suites. RAIRO, R1:8490, 1973.
[97] B. Germain-Bonne. Estimation de la limite de suites et formalisation de procedes
dacceleration de la convergence. PhD thesis, Universite de Lille I, 1978.
[98] J. Gilewicz. Numerical detection of the best Pade approximant and determination of the
Fourier coefcients of the insufciently sampled functions. In P.R. Graves-Morris, editor,
Pade Approximants and Their Applications, pages 99103, New York, 1973. Academic
Press.
[99] J. Gilewicz. Approximants de Pade. Number 667 in Lecture Notes in Mathematics.
Springer-Verlag, New York, 1978.
[100] M. Golomb. Zeros and poles of functions dened by power series. Bull. Amer. Math. Soc.,
49:581592, 1943.
[101] A.A. Goncar. Poles of rows of the Pade table and meromorphic continuation of functions.
Math. USSR-Sbornik, 43:527546, 1982.
[102] A.A. Goncar, E.A. Rakhmanov, and S.P. Suetin. On the rate of convergence of Pade
approximants of orthogonal expansions. In A.A. Goncar and E.B. Saff, editors, Progress
in Approximation Theory, pages 169190, New York, 1992. Springer-Verlag.

Bibliography

505

[103] S. Graf, V. Grecchi, and B. Simon. Borel summability: Application to the anharmonic
oscillator. Phys. Lett., 32B:631634, 1970.
[104] W.B. Gragg. Repeated extrapolation to the limit in the numerical solution of ordinary
differential equations. PhD thesis, University of California at Los Angeles, 1964.
Supervised by P. Henrici.
[105] W.B. Gragg. On extrapolation algorithms for ordinary initial value problems. Journal of
the Society for Industrial and Applied Mathematics: Series B, Numerical Analysis,
2:384403, 1965.
[106] W.B. Gragg. The Pade table and its relation to certain algorithms of numerical analysis.
SIAM Rev., 14:162, 1972.
[107] W.B. Gragg. Laurent, Fourier and ChebyshevPade tables. In E.B. Saff and R.S. Varga,
editors, Pade and Rational Approximation, pages 6172, New York, 1977. Academic
Press.
[108] W.B. Gragg and A.S. Householder. On a theorem of Konig. Numer. Math., 8:465468,
1966.
[109] P.R. Graves-Morris, R. Hughes Jones, and G.J. Makinson. The calculation of rational
approximants in two variables. J. Inst. Maths. Applics., 13:311320, 1974.
[110] H.L. Gray and T.A. Atchison. Nonlinear transformations related to the evaluation of
improper integrals I. SIAM J. Numer. Anal., 4:363371, 1967.
[111] H.L. Gray and T.A. Atchison. The generalized G-transform. Math. Comp., 22:595605,
1968.
[112] H.L. Gray, T.A. Atchison, and G.V. McWilliams. Higher order G-transformations. SIAM
J. Numer. Anal., 8:365381, 1971.
[113] H.L. Gray and S. Wang. An extension of the LevinSidi class of nonlinear transformations
for accelerating convergence of innite integrals and series. Appl. Math. Comp., 33:7587,
1989.
[114] H.L. Gray and S. Wang. A new method for approximating improper integrals. SIAM
J. Numer. Anal., 29:271283, 1992.
[115] C. Greif and D. Levin. The d2 -transformation for innite double series and the
D2 -transformation for innite double integrals. Math. Comp., 67:695714, 1998.
[116] J. Grotendorst. A Maple package for transforming sequences and functions. Comput.
Phys. Comm., 67:325342, 1991.
[117] R.E. Grundy. Laplace transform inversion using two-point rational approximants. J. Inst.
Maths. Applics., 20:299306, 1977.
[118] A.J. Guttmann. Asymptotic analysis of power-series expansions. In C. Domb and J.L.
Lebowitz, editors, Phase Transitions and Critical Phenomena, volume 13, pages 1234,
New York, 1989. Academic Press.
[119] A.J. Guttmann and G.S. Joyce. On a new method of series analysis in lattice statistics.
J. Phys. A, 5:L81L84, 1972.
[120] J. Hadamard. Essai sur letude des fonctions donnees par leur developpement de Taylor.
J. Math. Pures Appl. (4), 8:101186, 1892.
[121] E. Hairer and Ch. Lubich. Asymptotic expansions of the global error of xed-stepsize
methods. Numer. Math., 45:345360, 1984.
[122] E. Hairer, S.P. Nrsett, and G. Wanner. Solving Ordinary Differential Equations I: Nonstiff
Problems. Springer-Verlag, New York, second edition, 1993.
[123] G.H. Hardy. Divergent Series. Clarendon Press, Oxford, 1949.
[124] D.J. Haroldsen and D.I. Meiron. Numerical calculation of three-dimensional interfacial
potential ows using the point vortex method. SIAM J. Sci. Comput., 20:648683,
1988.
[125] J.F. Hart, E.W. Cheney, C.L. Lawson, H.J. Maehly, C.K. Mesztenyi, J.R. Rice,
H.J. Thacher, Jr., and C. Witzgall, editors. Computer Approximations. SIAM Series in
Applied Mathematics. Wiley, New York, 1968.
[126] T. Hasegawa. Numerical integration of functions with poles near the interval of
integration. J. Comp. Appl. Math., 87:339357, 1997.
[127] T. Hasegawa and A. Sidi. An automatic integration procedure for innite range integrals
involving oscillatory kernels. Numer. Algorithms, 13:119, 1996.
[128] T. Hasegawa and T. Torii. Indenite integration of oscillatory functions by the Chebyshev
series expansion. J. Comp. Appl. Math., 17:2129, 1987.

506

Bibliography

[129] T. Havie. Generalized Neville type extrapolation schemes. BIT, 19:204213, 1979.
[130] P. Henrici. Elements of Numerical Analysis. Wiley, New York, 1964.
[131] P. Henrici. Applied and Computational Complex Analysis, volume 1. Wiley, New York,
1974.
[132] P. Henrici. Applied and Computational Complex Analysis, volume 2. Wiley, New York,
1977.
[133] M. Hill and I. Robinson. d2lri: A nonadaptive algorithm for two-dimensional cubature.
J. Comp. Appl. Math., 112:121145, 1999.
[134] J.T. Holdeman, Jr. A method for the approximation of functions dened by formal series
expansions in orthogonal polynomials. Math. Comp., 23:275287, 1969.
[135] H.H.H. Homeier. A Levin-type algorithm for accelerating the convergence of Fourier
series. Numer. Algorithms, 3:245254, 1992.
[136] T.Y. Hou, J.S. Lowengrub, and M.J. Shelley. Boundary integral methods for
multicomponent uids and multiphase materials. J. Comp. Phys., 169:302362, 2001.
[137] R. Hughes Jones. General rational approximants in N variables. J. Approx. Theory,
16:201233, 1976.
[138] M. Iri, S. Moriguti, and Y. Takasawa. On a certain quadrature formula. Kokyuroku of Res.
Inst. for Math. Sci. Kyoto Univ., 91:82118, 1970. In Japanese. English translation in
J. Comp. Appl. Math., 17:320, (1987).
[139] A. Iserles. A note on Pade approximations and generalized hypergeometric functions. BIT,
19:543545, 1979.
[140] A. Iserles. Generalized order star theory. In M.G. de Bruin and H. van Rossum, editors,
Pade Approximation and Its Applications Amsterdam 1980, pages 228238, New York,
1981. Springer-Verlag.
[141] A. Iserles. A First Course in the Numerical Analysis of Differential Equations. Cambridge
Texts in Applied Mathematics. Cambridge University Press, Cambridge, 1996.
[142] A. Iserles and S.P. Nrsett. Order Stars. Chapman & Hall, London, 1991.
[143] P.R. Johnston. Semi-sigmoidal transformations for evaluating weakly singular boundary
element integrals. Intern. J. Numer. Methods Engrg., 47:17091730, 2000.
[144] W.B. Jones and W.J. Thron. Continued Fractions: Analytic Theory and Applications.
Addison-Wesley, London, 1980.
[145] D.C. Joyce. Survey of extrapolation processes in numerical analysis. SIAM Rev.,
13:435490, 1971.
[146] G.S. Joyce and A.J. Guttmann. A new method of series analysis. In P.R. Graves-Morris,
editor, Pade Approximants and Their Applications, pages 163167, New York, 1973.
Academic Press.
[147] D.K. Kahaner. Numerical quadrature by the -algorithm. Math. Comp., 26:689693, 1972.
[148] M. Kaminski and A. Sidi. Solution of an integer programming problem related to
convergence of rows of Pade approximants. Appl. Numer. Math., 8:217223, 1991.
[149] J. Karlsson and E.B. Saff. Singularities of functions determined by the poles of Pade
approximants. In M.G. de Bruin and H. van Rossum, editors, Pade Approximation and Its
Applications Amsterdam 1980, pages 239254, New York, 1981. Springer-Verlag.
[150] J. Karlsson and H. Wallin. Rational approximation by an interpolation procedure in
several variables. In E.B. Saff and R.S. Varga, editors, Pade and Rational Approximation,
pages 83100, New York, 1977. Academic Press.
[151] J.E. Kiefer and G.H. Weiss. A comparison of two methods for accelerating the
convergence of Fourier series. Comp. & Maths. with Applics., 7:527535, 1981.
[152] K. Knopp. Theory and Application of Innite Series. Hafner, New York, 1947.

[153] J. Koenig. Uber

eine Eigenschaft der Potenzreichen. Math. Ann., 23:447449, 1884.
[154] N.M. Korobov. Number-Theoretic Methods of Approximate Analysis. GIFL, Moscow,
1963. Russian.
[155] C. Kowalewski. Acceleration de la convergence pour certain suites a` convergence
logarithmique. In M.G. de Bruin and H. van Rossum, editors, Pade Approximation and Its
Applications Amsterdam 1980, pages 263272, New York, 1981. Springer-Verlag.
[156] C. Kowalewski. Possibilites dacceleration de la convergence logarithmique. PhD thesis,
Universite de Lille I, 1981.
[157] J.D. Lambert. Computational Methods in Ordinary Differential Equations. Wiley,
New York, 1973.

Bibliography

507

[158] P.-J. Laurent. Un theor`eme de convergence pour le procede dextrapolation de Richardson.

C. R. Acad. Sci. Paris, 256:14351437, 1963.
[159] D.P. Laurie. Propagation of initial rounding error in Romberg-like quadrature. BIT,
15:277282, 1975.
[160] D.P. Laurie. Periodizing transformations for numerical integration. J. Comp. Appl. Math.,
66:337344, 1996.
[161] D. Levin. Development of non-linear transformations for improving convergence of
sequences. Intern. J. Computer Math., B3:371388, 1973.
[162] D. Levin. Numerical inversion of the Laplace transform by accelerating the convergence
of Bromwichs integral. J. Comp. Appl. Math., 1:247250, 1975.
[163] D. Levin. General order Pade-type rational approximants dened from double power
series. J. Inst. Maths. Applics., 18:18, 1976.
[164] D. Levin. On accelerating the convergence of innite double series and integrals. Math.
Comp., 35:13311345, 1980.
[165] D. Levin and A. Sidi. Two new classes of nonlinear transformations for accelerating the
convergence of innite integrals and series. Appl. Math. Comp., 9:175215, 1981.
Originally appeared as a Tel Aviv University preprint in 1975.
[166] D. Levin and A. Sidi. Extrapolation methods for innite multiple series and integrals.
J. Comp. Meth. Sci. & Engrg., 1:167184, 2001.
[167] M.J. Lighthill. Introduction to Fourier Analysis and Generalized Functions. Cambridge
University Press, Cambridge, 1978.
[168] Jing Lin. Extension of Wilsons theorem. Computations of Mathematics, 1:4348, 1987.
In Chinese.
[169] Xiaoyan Liu and E.B. Saff. Intermediate rows of the Walsh array of best rational
approximations to meromorphic functions. Methods Appl. Anal., 2:269284, 1995.
[170] I.M. Longman. Note on a method for computing innite integrals of oscillatory functions.
Proc. Cambridge Phil. Soc., 52:764768, 1956.
[171] I.M. Longman. Tables for the rapid and accurate numerical evaluation of certain innite
integrals involving Bessel functions. Mathematical Tables and Other Aids to
Computation, 11:166180, 1957.


[172] I.M. Longman. A short table of x J0 (t)t n dt and x J1 (t)t n dt (in Technical Notes
and Short Papers). Mathematical Tables and Other Aids to Computation, 13:306311,
1959.
[173] I.M. Longman. The application of rational approximations to the solution of problems in
theoretical seismology. Bull. Seism. Soc. Amer., 56:10451065, 1966.
[174] I.M. Longman. Computation of the Pade table. Intern. J. Computer Math., Section B,
3:5364, 1971.
[175] I.M. Longman. Approximate Laplace transform inversion applied to a problem in
electrical network theory. SIAM J. Appl. Math., 23:439445, 1972.
[176] I.M. Longman. Numerical Laplace transform inversion of a function arising in
viscoelasticity. J. Comp. Phys., 10:224231, 1972.
[177] I.M. Longman. On the generation of rational approximations for Laplace transform
inversion with an application to viscoelasticity. SIAM J. Appl. Math., 24:429440, 1973.
[178] I.M. Longman. Difculties in convergence acceleration. In M.G. de Bruin and H. van
Rossum, editors, Pade Approximation and Its Applications Amsterdam 1980,
pages 273289, New York, 1981. Springer-Verlag.
[179] L. Lorentzen and H. Waadeland. Continued Fractions with Applications. North-Holland,
Amsterdam, 1992.
[180] D.S. Lubinsky. Pade tables of a class of entire functions. Proc. Amer. Math. Soc.,
94:399406, 1985.
[181] D.S. Lubinsky. Pade tables of entire functions of very slow and smooth growth. Constr.
Approx., 1:349358, 1985.
[182] D.S. Lubinsky. Uniform convergence of rows of the Pade table for functions with smooth
Maclaurin series coefcients. Constr. Approx., 3:307330, 1987.
[183] D.S. Lubinsky. Pade tables of entire functions of very slow and smooth growth II. Constr.
Approx., 4:321339, 1988.
[184] D.S. Lubinsky. On the diagonal Pade approximants of meromorphic functions.
Indagationes Mathematicae, 7:97110, 1996.

508

Bibliography

[185] D.S. Lubinsky and A. Sidi. Convergence of linear and nonlinear Pade approximants from
series of orthogonal polynomials. Trans. Amer. Math. Soc., 278:333345, 1983.
[186] D.S. Lubinsky and A. Sidi. Strong asymptotics for polynomials biorthogonal to powers of
log x. Analysis, 14:341379, 1994.
[187] S. Lubkin. A method of summing innite series. J. Res. Nat. Bur. Standards, 48:228254,
1952.
[188] S.K. Lucas. Evaluating innite integrals involving products of Bessel functions of
arbitrary order. J. Comp. Appl. Math., 64:269282, 1995.
[189] S.K. Lucas and H.A. Stone. Evaluating innite integrals involving Bessel functions of
arbitrary order. J. Comp. Appl. Math., 64:217231, 1995.
[190] R. Lugannani and S. Rice. Use of Gaussian convergence factors in numerical evaluation of
slowly convergent integrals. J. Comp. Phys., 37:264267, 1980.
[191] Y.L. Luke. On the approximate inversion of some Laplace transforms. In Proceedings of
4th U.S. National Congress on Applied Mechanics, pages 269276, 1962.
[192] Y.L. Luke. Approximate inversion of a class of Laplace transforms applicable to
supersonic ow problems. Quart. J. Mech. Appl. Math., 17:91103, 1964.
[193] Y.L. Luke. Algorithms for the Computation of Special Functions. Academic Press,
New York, 1977.
[194] J. Lund. Bessel transforms and rational extrapolation. Numer. Math., 47:114, 1985.
[195] J.N. Lyness. Applications of extrapolation techniques to multidimensional quadrature of
some integrand functions with a singularity. J. Comp. Phys., 20:346364, 1976.
[196] J.N. Lyness. An error functional expansion for N -dimensional quadrature with an
integrand function singular at a point. Math. Comp., 30:123, 1976.
[197] J.N. Lyness. Integrating some innite oscillating tails. J. Comp. Appl. Math.,
12/13:109117, 1985.
[198] J.N. Lyness. Quadrature over curved surfaces by extrapolation. Math. Comp., 63:727740,
1994.
[199] J.N. Lyness and E. de Doncker. Quadrature error expansions II. The full corner singularity.
Numer. Math., 64:355370, 1993.
[200] J.N. Lyness and B.J. McHugh. Integration over multidimensional hypercubes. Computer
J., 6:264270, 1963.
[201] J.N. Lyness and G. Monegato. Quadrature error functional expansions for the simplex
when the integrand function has singularities at vertices. Math. Comp., 34:213225, 1980.
[202] J.N. Lyness and B.W. Ninham. Numerical quadrature and asymptotic expansions. Math.
Comp., 21:162178, 1967.
[203] J.N. Lyness and K.K. Puri. The EulerMaclaurin expansion for the simplex. Math. Comp.,
27:273293, 1973.
[204] J.N. Lyness and U. Rude. Cubature of integrands containing derivatives. Numer. Math.,
78:439461, 1998.
[205] H.J. Maehly. Rational approximations for transcendental functions. In Proceedings of the
International Conference on Information Processing, pages 5762, Butterworth, London,
1960.
[206] G.I. Marchuk and V.V. Shaidurov. Difference methods and their extrapolations.
Springer-Verlag, New York, 1983.
[207] A.C. Matos and M. Prevost. Acceleration property for the columns of the E-algorithm.
Numer. Algorithms, 2:393408, 1992.
[208] J.H. McCabe and J.A. Murphy. Continued fractions which correspond to power series at
two points. J. Inst. Maths. Applics., 17:233247, 1976.
[209] D.W. McLaughlin, D.J. Muraki, and M.J. Shelley. Self-focussed optical structures in a
nematic liquid crystal. Physica D, 97:471497, 1996.

[210] G. Meinardus. Uber

das asymptotische Verhalten von Iterationsfolgen. Z. Angew. Math.
Mech., 63:7072, 1983.
[211] K.A. Michalski. Extrapolation methods for Sommerfeld integral tails. IEEE Trans.
Antennas Propagat., 46:14051418, 1998.
[212] G.F. Miller. On the convergence of the Chebyshev series for functions possessing a
singularity in the range of representation. SIAM J. Numer. Anal., 3:390409, 1966.
[213] G. Monegato and J.N. Lyness. The EulerMaclaurin expansion and nite-part integrals.
Numer. Math., 81:273291, 1998.

Bibliography

509

[214] M. Mori. An IMT-type double exponential formula for numerical integration. Publ. Res.
Inst. Math. Sci. Kyoto Univ., 14:713729, 1978.
[215] J.D. Murray. Asymptotic Analysis. Clarendon Press, Oxford, 1974.
[216] I. Navot. An extension of the EulerMaclaurin summation formula to functions with a
branch singularity. J. Math. and Phys., 40:271276, 1961.
[217] I. Navot. A further extension of the EulerMaclaurin summation formula. J. Math. and
Phys., 41:155163, 1962.
[218] G. Nemeth. Chebyshev expansions for Fresnel integrals. Numer. Math., 7:310312, 1965.
[219] Q. Nie and F.R. Tian. Singularities in HeleShaw ows driven by a multipole. SIAM J.
Appl. Math., 62:385406, 2001.
[220] W. Niethammer. Numerical application of Eulers series transformation and its
generalizations. Numer. Math., 34:271283, 1980.
[221] M. Nitsche. Singularity formation in a cylindrical and a spherical vortex sheet. J. Comp.
Phys., 173:208230, 2001.
[222] N.E. Norlund. Vorlesungen u ber Differenzenrechnung. Springer-Verlag, Berlin, 1924.
[223] F.W.J. Olver. Asymptotics and Special Functions. Academic Press, New York, 1974.
[224] N. Osada. Accelerable subsets of logarithmic sequences. J. Comp. Appl. Math.,
32:217227, 1990.
[225] N. Osada. A convergence acceleration method for some logarithmically convergent
sequences. SIAM J. Numer. Anal., 27:178189, 1990.
[226] N. Osada. The E-algorithm and the Ford-Sidi algorithm. J. Comp. Appl. Math.,
122:223230, 2000.
[227] K.J. Overholt. Extended Aitken acceleration. BIT, 5:122132, 1965.
[228] B.N. Parlett. Global convergence of the basic QR algorithm on Hessenberg matrices.
Math. Comp., 22:803817, 1968.
[229] O. Perron. Die Lehre von den Kettenbruchen. Teubner, Stuttgart, third edition, 1957. In
two volumes.
[230] R. Piessens. Gaussian quadrature formulas for the numerical integration of Bromwichs
integral and the inversion of the Laplace transforms. J. Engrg. Math., 5:19, 1971.
[231] R.E. Powell and S.M. Shah. Summability Theory and Its Applications. Van Nostrand
Rheinhold, London, 1972.
[232] R. de Prony. Essai experimental et analytique: sur les lois de la dilatabilite de uides
e lastiques et sur celles de la force expansive de la vapeur de leau et de la vapeur de

lalkool, a` differentes temperatures. Journal de lEcole

Polytechnique Paris, 1:2476,
1795.
[233] W.C. Pye and T.A. Atchison. An algorithm for the computation of the higher order
G-transformation. SIAM J. Numer. Anal., 10:17, 1973.
[234] P. Rabinowitz. Extrapolation methods in numerical integration. Numer. Algorithms,
3:1728, 1992.
[235] A. Ralston and P. Rabinowitz. A First Course in Numerical Analysis. McGraw-Hill,
New York, second edition, 1978.
[236] L.F. Richardson. The approximate arithmetical solution by nite differences of physical
problems involving differential equations, with an application to the stress in a masonry
dam. Phil. Trans. Roy. Soc. London, Series A, 210:307357, 1910.
[237] L.F. Richardson. Theory of the measurement of wind by shooting spheres upward. Phil.
Trans. Roy. Soc. London, Series A, 223:345382, 1923.
[238] L.F. Richardson. The deferred approach to the limit, I: Single lattice. Phil. Trans. Roy.
Soc. London, Series A, 226:299349, 1927.
[239] I. Robinson and M. Hill. Algorithm 816: r2d2lri: An algorithm for automatic
two-dimensional cubature. ACM Trans. Math. Software, 28:75100, 2002.
[240] W. Romberg. Vereinfachte numerische Integration. Det. Kong. Norske Videnskabers
Selskab Forhandlinger (Trondheim), 28:3036, 1955.
[241] J.B. Rosser. Transformations to speed the convergence of series. J. Res. Nat. Bur.
Standards, 46:5664, 1951.
[242] H. Rutishauser. Anwendungen des Quotienten-Differenzen-Algorithmus. Z. Angew. Math.
Phys., 5:496508, 1954.
[243] H. Rutishauser. Der Quotienten-Differenzen-Algorithmus. Z. Angew. Math. Phys.,
5:233251, 1954.

510

Bibliography

[244] H. Rutishauser. Der Quotienten-Differenzen-Algorithmus. Birkhauser, Basel, 1957.

[245] H. Rutishauser. Ausdehnung des Rombergschen Prinzips. Numer. Math., 5:4854, 1963.
[246] P. Sablonni`ere. Convergence acceleration of logarithmic xed point sequences. J. Comp.
Appl. Math., 19:5560, 1987.
[247] P. Sablonni`ere. Comparison of four algorithms accelerating the convergence of a subset of
logarithmic xed point sequences. Numer. Algorithms, 1:177197, 1991.
[248] P. Sablonni`ere. Asymptotic behaviour of iterated modied 2 and 2 transforms on some
slowly convergent sequences. Numer. Algorithms, 3:401409, 1992.
[249] E.B. Saff. An extension of Montessus de Ballore theorem on the convergence of
interpolating rational functions. J. Approx. Theory, 6:6367, 1972.
[250] H. Safouhi and P.E. Hoggan. Efcient evaluation of Coulomb integrals: the nonlinear

D- and D-transformations.
J. Phys. A: Math. Gen., 31:89418951, 1998.
[251] H. Safouhi and P.E. Hoggan. Non-linear transformations for rapid and efcient evaluation
of multicenter bielectronic integrals over B functions. J. Math. Chem., 25:259280,
1999.
[252] H. Safouhi and P.E. Hoggan. Three-center two-electron Coulomb and hybrid integrals

evaluated using non-linear D- and D-transformations.

J. Phys. A: Math. Gen.,
32:62036217, 1999.
[253] H. Safouhi, D. Pinchon, and P.E. Hoggan. Efcient evaluation of integrals for density
functional theory: nonlinear D transformations to evaluate three-center nuclear attraction
integrals over B functions. Intern. J. Quantum Chem., 70:181188, 1998.
[254] T.W. Sag and G. Szekeres. Numerical evaluation of high-dimensional integrals. Math.
Comp., 18:245253, 1964.
[255] H.E. Salzer. Orthogonal polynomials arising in the numerical evaluation of inverse Laplace
transforms. Mathematical Tables and Other Aids to Computation, 9:164177, 1955.
[256] H.E. Salzer. Tables for numerical calculation of inverse Laplace transforms. J. Math. and
Phys., 37:89109, 1958.
[257] H.E. Salzer. Additional formulas and tables for orthogonal polynomials originating from
inversion integrals. J. Math. and Phys., 40:7286, 1961.
[258] J.R. Schmidt. On the numerical solution of linear simultaneous equations by an iterative
method. Phil. Mag., 7:369383, 1941.
[259] C. Schneider. Vereinfachte Rekursionen zur Richardson-Extrapolation in Spezialfallen.
Numer. Math., 24:177184, 1975.
[260] I.J. Schonberg. On the Polya frequency functions I: The totally positive functions and their
Laplace transforms. J. dAnal. Math., 1:331374, 1951.
[261] R.E. Scraton. The practical use of the Euler transformation. BIT, 29:356360, 1989.
[262] G.A. Sedogbo. Convergence acceleration of some logarithmic sequences. J. Comp. Appl.
Math., 32:253260, 1990.
[263] R.E. Shafer. On quadratic approximation. SIAM J. Numer. Anal., 11:447460, 1974.
[264] D. Shanks. Nonlinear transformations of divergent and slowly convergent sequences.
J. Math. and Phys., 34:142, 1955.
[265] R. Shelef. New numerical quadrature formulas for Laplace transform inversion by
Bromwichs integral. Masters thesis, TechnionIsrael Institute of Technology, 1987. In
Hebrew. Supervised by A. Sidi.
[266] M.J. Shelley, F.-R. Tian, and K. Wlodarski. Hele-Shaw ow and pattern formation in a
time-dependent gap. Nonlinearity, 10:14711495, 1997.
[267] A. Sidi. Computation of the ChebyshevPade table. J. Comp. Appl. Math., 1:6971,
1975.
[268] A. Sidi. Exponential function approximation to Laplace transform inversion and
development of non-linear methods for accelerating the convergence of innite integrals
and series. PhD thesis, Tel Aviv University, 1977. In Hebrew. Supervised by
I.M. Longman.
[269] A. Sidi. Uniqueness of Pade approximants from series of orthogonal polynomials. Math.
Comp., 31:738739, 1977.
[270] A. Sidi. Convergence properties of some nonlinear sequence transformations. Math.
Comp., 33:315326, 1979.
[271] A. Sidi. EulerMaclaurin expansions for a double integral with a line of singularities in
the domain of integration. Technical Report 161, Computer Science Dept.,
TechnionIsrael Institute of Technology, 1979.

Bibliography

511

[272] A. Sidi. Some properties of a generalization of the Richardson extrapolation process.

J. Inst. Maths. Applics., 24:327346, 1979.
[273] A. Sidi. Analysis of convergence of the T -transformation for power series. Math. Comp.,
35:833850, 1980.
[274] A. Sidi. Extrapolation methods for oscillatory innite integrals. J. Inst. Maths. Applics.,
26:120, 1980.
[275] A. Sidi. Numerical quadrature and nonlinear sequence transformations; unied rules for
efcient computation of integrals with algebraic and logarithmic endpoint singularities.
Math. Comp., 35:851874, 1980.
[276] A. Sidi. Some aspects of two-point Pade approximants. J. Comp. Appl. Math., 6:917,
1980.
[277] A. Sidi. A new method for deriving Pade approximants for some hypergeometric
functions. J. Comp. Appl. Math., 7:3740, 1981.
[278] A. Sidi. An algorithm for a special case of a generalization of the Richardson
extrapolation process. Numer. Math., 38:299307, 1982.
[279] A. Sidi. Converging factors for some asymptotic moment series that arise in numerical
quadrature. J. Austral. Math. Soc., Series B, 24:223233, 1982.
[280] A. Sidi. Interpolation at equidistant points by a sum of exponential functions. J. Approx.
Theory, 34:194210, 1982.
[281] A. Sidi. The numerical evaluation of very oscillatory innite integrals by extrapolation.
Math. Comp., 38:517529, 1982.
[282] A. Sidi. Numerical quadrature rules for some innite range integrals. Math. Comp.,
38:127142, 1982.
[283] A. Sidi. EulerMaclaurin expansions for integrals over triangles and squares of functions
having algebraic/logarithmic singularities along an edge. J. Approx. Theory, 39:3953,
1983.
[284] A. Sidi. Interpolation by a sum of exponential functions when some exponents are
preassigned. J. Math. Anal. Appl., 112:151164, 1985.
[285] A. Sidi. Borel summability and converging factors for some everywhere divergent series.
SIAM J. Math. Anal., 17:12221231, 1986.
[286] A. Sidi. Extrapolation methods for divergent oscillatory innite integrals that are dened
in the sense of summability. J. Comp. Appl. Math., 17:105114, 1987.
[287] A. Sidi. Generalizations of Richardson extrapolation with applications to numerical
integration. In H. Brass and G. Hammerlin, editors, Numerical Integration III, number 85
in ISNM, pages 237250, Basel, 1988. Birkhauser.
[288] A. Sidi. A user-friendly extrapolation method for oscillatory innite integrals. Math.
Comp., 51:249266, 1988.
[289] A. Sidi. Comparison of some numerical quadrature formulas for weakly singular periodic
Fredholm integral equations. Computing, 43:159170, 1989.
[290] A. Sidi. On a generalization of the Richardson extrapolation process. Numer. Math.,
57:365377, 1990.
[291] A. Sidi. On rates of acceleration of extrapolation methods for oscillatory innite integrals.
BIT, 30:347357, 1990.
[292] A. Sidi. Quantitative and constructive aspects of the generalized Koenigs and de
Montessuss theorems for Pade approximants. J. Comp. Appl. Math., 29:257291, 1990.
[293] A. Sidi. A new variable transformation for numerical integration. In H. Brass and
G. Hammerlin, editors, Numerical Integration IV, number 112 in ISNM, pages 359373,
Basel, 1993. Birkhauser.
[294] A. Sidi. Acceleration of convergence of (generalized) Fourier series by the
d-transformation. Annals Numer. Math., 2:381406, 1995.
[295] A. Sidi. Convergence analysis for a generalized Richardson extrapolation process with an
application to the d (1) -transformation on convergent and divergent logarithmic sequences.
Math. Comp., 64:16271657, 1995.
[296] A. Sidi. Extension and completion of Wynns theory on convergence of columns of the
epsilon table. J. Approx. Theory, 86:2140, 1996.
[297] A. Sidi. Further results on convergence and stability of a generalization of the Richardson
extrapolation process. BIT Numerical Mathematics, 36:143157, 1996.
[298] A. Sidi. A complete convergence and stability theory for a generalized Richardson
extrapolation process. SIAM J. Numer. Anal., 34:17611778, 1997.

512

Bibliography

[299] A. Sidi. Computation of innite integrals involving Bessel functions of arbitrary order by

the D-transformation.
J. Comp. Appl. Math., 78:125130, 1997.
[300] A. Sidi. Further convergence and stability results for the generalized Richardson
extrapolation process GREP(1) with an application to the D (1) -transformation for innite
integrals. J. Comp. Appl. Math., 112:269290, 1999.
[301] A. Sidi. Extrapolation methods and derivatives of limits of sequences. Math. Comp.,
69:305323, 2000.
[302] A. Sidi. The generalized Richardson extrapolation process GREP(1) and computation of
derivatives of limits of sequences with applications to the d (1) -transformation. J. Comp.
Appl. Math., 122:251273, 2000.
[303] A. Sidi. A new algorithm for the higher order G-transformation. Preprint, Computer
Science Dept., TechnionIsrael Institute of Technology, 2000.
[304] A. Sidi. Numerical differentiation, polynomial interpolation, and Richardson
extrapolation. Preprint, Computer Science Dept., TechnionIsrael Institute of Technology,
2000.
[305] A. Sidi. The Richardson extrapolation process with a harmonic sequence of collocation
points. SIAM J. Numer. Anal., 37:17291746, 2000.
[306] A. Sidi. New convergence results on the generalized Richardson extrapolation process
GREP(1) for logarithmic sequences. Math. Comp., 71:15691596, 2002. Published
electronically on November 28, 2001.
[307] A. Sidi. A convergence and stability study of the iterated Lubkin transformation and
the -algorithm. Math. Comp., 72:419433, 2003. Published electronically on May 1,
2002.
[308] A. Sidi and J. Bridger. Convergence and stability analyses for some vector extrapolation
methods in the presence of defective iteration matrices. J. Comp. Appl. Math., 22:3561,
1988.
[309] A. Sidi, W.F. Ford, and D.A. Smith. Acceleration of convergence of vector sequences.
SIAM J. Numer. Anal., 23:178196, 1986. Originally appeared as NASA TP-2193,
(1983).
[310] A. Sidi and M. Israeli. Quadrature methods for periodic singular and weakly singular
Fredholm integral equations. J. Sci. Comput., 3:201231, 1988. Originally appeared as
Technical Report No. 384, Computer Science Dept., TechnionIsrael Institute of
Technology, (1985), and also as ICASE Report No. 86-50 (1986).
[311] A. Sidi and M. Israeli. A hybrid extrapolation method: The Richardson-Shanks
transformation, 1989. Unpublished research.
[312] A. Sidi and D. Levin. Rational approximations from the d-transformation. IMA J. Numer.
Anal., 2:153167, 1982.
[313] A. Sidi and D. Levin. Prediction properties of the t-transformation. SIAM J. Numer. Anal.,
20:589598, 1983.
[314] A. Sidi and D.S. Lubinsky. On the zeros of some polynomials that arise in numerical
quadrature and convergence acceleration. SIAM J. Numer. Anal., 20:400405,
1983.
[315] S. Skelboe. Computation of the periodic steady-state response of nonlinear networks by
extrapolation methods. IEEE Trans. Circuits and Systems, 27:161175, 1980.
[316] I.H. Sloan and S. Joe. Lattice Methods in Multiple Integration. Clarendon Press, Oxford,
1994.
[317] D.A. Smith and W.F. Ford. Acceleration of linear and logarithmic convergence. SIAM J.
Numer. Anal., 16:223240, 1979.
[318] D.A. Smith and W.F. Ford. Numerical comparisons of nonlinear convergence accelerators.
Math. Comp., 38:481499, 1982.
[319] I.H. Sneddon. The Use of Integral Transforms. McCraw-Hill, New York, 1972.
[320] H. Stahl. On the divergence of certain Pade approximants and the behavior of the
associated orthogonal polynomials. In C. Brezinski, A. Ronveaux, A. Draux,
A.P. Magnus, and P. Maroni, editors, Polynomes Orthogonaux et Applications,
Proceedings, Bar-le-Duc, 1984, number 1171 in Springer Lecture Notes in Mathematics,
pages 321330, New York, 1985. Springer-Verlag.
[321] H. Stahl and V. Totik. General Orthogonal Polynomials. Cambridge University Press,
Cambridge, 1992.

Bibliography

513

[322] J.F. Steffensen. Interpolation. Chelsea, New York, 1950.

[323] H.J. Stetter. Asymptotic expansions for the error of discretization algorithms for
non-linear functional equations. Numer. Math., 7:1831, 1965.
[324] H.J. Stetter. Symmetric two-step algorithms for ordinary differential equations.
Computing, 5:267280, 1970.
[325] H.J. Stetter. Analysis of Discretization Methods for Ordinary Differential Equations.
Springer-Verlag, Berlin, 1973.
[326] J. Stoer and R. Bulirsch. Introduction to Numerical Analysis. Springer-Verlag, New York,
1980.
[327] R.F. Streit. The evaluation of double series. BIT, 12:400408, 1972.
[328] S.P. Suetin. On the convergence of rational approximations to polynomial expansions in
domains of meromorphy. Math. USSR-Sbornik, 34:367381, 1978.
[329] S.P. Suetin. On the poles of the mth row of a Pade table. Math. USSR-Sbornik,
48:493497, 1984.
[330] S.P. Suetin. On the inverse problem for the mth row of the Pade table. Math.
USSR-Sbornik, 52:231244, 1985.
[331] M. Sugihara. Methods of numerical integration of oscillatory functions by the DE-formula
with the Richardson extrapolation. J. Comp. Appl. Math., 17:4768, 1987.
[332] G. Szego. Orthogonal Polynomials. American Mathematical Society, Providence, Rhode
Island, 1939.
[333] H. Takahasi and M. Mori. Double exponential formulas for numerical integration. Publ.
Res. Inst. of Math. Sci. Kyoto Univ., 9:721741, 1974.
[334] E.C. Titchmarsh. Theory of the Riemann Zeta Function. Cambridge University Press,
Cambridge, 1930.
[335] H. Toda and H. Ono. Some remarks for efcient usage of the double exponential formulas.
Kokyuroku of Res. Inst. for Math. Sci. Kyoto Univ., 339:74109, 1978. In Japanese.
[336] J. Todd, editor. Survey of Numerical Analysis. McGraw-Hill, New York, 1962.
[337] L.N. Trefethen and M.H. Gutknecht. On convergence and degeneracy in rational Pade and
Chebyshev approximation. SIAM J. Math. Anal., 16:198210, 1985.
[338] L.N. Trefethen and M.H. Gutknecht. Pade, stable Pade, and ChebyshevPade
approximation. In J.C. Mason and M.G. Cox, editors, Algorithms for Approximation,
pages 227264, Oxford, 1987. Oxford University Press.
[339] R.R. Tucker. The 2 -process and related topics. Pacic J. Math., 22:349359, 1967.
[340] R.R. Tucker. The 2 -process and related topics II. Pacic J. Math., 28:455463, 1969.
[341] R.R. Tucker. A geometric derivation of Daniel Shanks ek transform. Faculty Rev. Bull.
N.C. A and T State Univ., 65:6063, 1973.
[342] P.A. Tyvand and M. Landrini. Free-surface ow of a uid body with an inner circular
cylinder in impulsive motion. J. Engrg. Math., 40:109140, 2001.
[343] A.H. Van Tuyl. Acceleration of convergence of a family of logarithmically convergent
sequences. Math. Comp., 63:229246, 1994.
[344] J.-M. Vanden Broeck and L.W. Schwartz. A one-parameter family of sequence
transformations. SIAM J. Math. Anal., 10:658666, 1979.
[345] D. Vekemans. Algorithm for the E-prediction. J. Comp. Appl. Math., 85:181202,
1997.
[346] P. Verlinden and R. Cools. Proof of a conjectured asymptotic expansion for the
approximation of surface integrals. Math. Comp., 63:717725, 1994.
[347] P. Verlinden, D.M. Potts, and J.N. Lyness. Error expansions for multidimensional
trapezoidal rules with Sidi transformations. Numer. Algorithms, 16:321347, 1997.
[348] H.S. Wall. Analytic Theory of Continued Fractions. Van Nostrand, New York, 1948.
[349] G. Walz. Asymptotics and Extrapolation. Akademie Verlag, Berlin, 1996.
[350] G. Wanner, E. Hairer, and S.P. Nrsett. Order stars and stability theorems. BIT,
18:475489, 1978.
[351] P.C. Waterman, J.M. Yos, and R.J. Abodeely. Numerical integration of non-analytic
functions. J. Math. and Phys., 43:4550, 1964.
[352] L. Weiss and R. McDonough. Pronys method, Z-transforms, and Pade approximation.
SIAM Rev., 5:145149, 1963.
[353] E.J. Weniger. Nonlinear sequence transformations for the acceleration of convergence and
the summation of series. Comput. Phys. Rep., 10:189371, 1989.

514

Bibliography

[354] E.J. Weniger. Interpolation between sequence transformations. Numer. Algorithms,

3:477486, 1992.
[355] E.J. Weniger. Prediction properties of Aitken iterated 2 process, of Wynns epsilon
algorithm, and of Brezinskis iterated theta algorithm. J. Comp. Appl. Math.,
122:329356, 2000.
z ek, and F. Vinette. The summation of the ordinary and renormalized
[356] E.J. Weniger, J. C
perturbation series for the ground state energy of the quartic, sextic, and octic anharmonic
oscillators using nonlinear sequence transformations. J. Math. Phys., 34:571609, 1993.
[357] D.V. Widder. The Laplace Transform. Princeton University Press, Princeton, 1946.
[358] R. Wilson. Divergent continued fractions and polar singularities. Proc. London Math.
Soc., 26:159168, 1927.
[359] R. Wilson. Divergent continued fractions and polar singularities II. Boundary pole
multiple. Proc. London Math. Soc., 27:497512, 1928.
[360] R. Wilson. Divergent continued fractions and polar singularities III. Several boundary
poles. Proc. London Math. Soc., 28:128144, 1928.
[361] J. Wimp. Derivative-free iteration processes. SIAM J. Numer. Anal., 7:329334, 1970.
[362] J. Wimp. The summation of series whose terms have asymptotic representations.
J. Approx. Theory, 10:185198, 1974.
[363] J. Wimp. Toeplitz arrays, linear sequence transformations and orthogonal polynomials.
Numer. Math., 23:117, 1974.
[364] J. Wimp. Acceleration methods. In Encyclopedia of Computer Science and Technology,
volume I, pages 181210, New York, 1975. Dekker.
[365] J. Wimp. New methods for accelerating the convergence of sequences arising in Laplace
transform theory. SIAM J. Numer. Anal., 14:194204, 1977.
[366] J. Wimp. Sequence Transformations and Their Applications. Academic Press, New York,
1981.
[367] L. Wuytack. A new technique for rational extrapolation to the limit. Numer. Math.,
17:215221, 1971.
[368] P. Wynn. On a device for computing the em (Sn ) transformation. Mathematical Tables and
Other Aids to Computation, 10:9196, 1956.
[369] P. Wynn. On a procrustean technique for the numerical transformation of slowly
convergent sequences and series. Proc. Cambridge Phil. Soc., 52:663671, 1956.
[370] P. Wynn. Conuent forms of certain nonlinear algorithms. Arch. Math., 11:223234, 1960.
[371] P. Wynn. On the convergence and stability of the epsilon algorithm. SIAM J. Numer. Anal.,
3:91122, 1966.
[372] P. Wynn. Upon systems of recursions which obtain among the quotients of the Pade table.
Numer. Math., 8:264269, 1966.
[373] P. Wynn. A general system of orthogonal polynomials. Quart. J. Math. Oxford, 18:8196,
1967.
[374] P. Wynn. Transformations to accelerate the convergence of Fourier series. In Gertrude
Blanche Anniversary Volume, pages 339379, Wright Patterson Air Force Base, 1967.
[375] P. Wynn. A note on the generalized Euler transformation. Computer J., 14:437441, 1971.
[376] P. Wynn. Upon some continuous prediction algorithms. I. Calcolo, 9:197234, 1972.
[377] V. Zakian. Properties of I M N approximants. In P.R. Graves-Morris, editor, Pade
Approximants and Their Applications, pages 141144, New York, 1973. Academic Press.
[378] V. Zakian. Properties of I M N and JM N approximants and applications to numerical
inversion of Laplace transforms. J. Math. Anal. Appl., 50:191222, 1975.

Index

A( ) class functions, 96
integral properties of, 117
( )
A0 class functions, 122
( ,m) class functions, 140
A
0
summation properties of, 141
Abel sum, 120, 227, 228, 385, 435
Abel summability, see Abel sum
Abelian mean, 147, 259
Acceleration factor, 3
Acceleration of convergence, 3
of innite products, 444
of innite series
rearrangements of, 443
with special sign patterns, 442
of logarithmic xed-point sequences, 454
Acceleration of convergence of (generalized)
Fourier series
by d-transformation, 253262
complex series approach, 254
direct approach, 259
extension of complex series approach, 260
functions of second kind, 254
with APS, 255
Acceleration of convergence of power series
by rational d-approximants, 238250
with APS, 247
Aitken 2 -process, 2, 15, 283290
Alternating series, 151
Antilimits, 2
examples of, 45
APS, 217
Archimedes method for , 22
by Richardson extrapolation, 46
Arithmetic Progression Sampling, see APS
Asymptotic equality, 459
Asymptotic expansion, 460
Asymptotic scale, see asymptotic sequence
Asymptotic sequence, 460
B(1) class functions, 117120
characterization of, 118
integral properties of, 118
B(m) class functions, 95103
denition of, 98
integral properties of, 100103
proof of, 112117

class functions, 227

B
Bc class functions, 227
Bd class functions, 227
Abel sum of integrals of, 227
b(1) class sequences, 150152
characterization of, 150
factorial, 151, 152
factorially divergent, 152
linear, 151, 152
logarithmic, 150, 152
summation properties of, 150152
sums of linear, 155157
sums of logarithmic, 153155
sums of mixed, 157
(1)
b /FACD class sequences, 371
b(1) /FAC class sequences, 285
b(1) /LIN class sequences, 285
b(1) /LOG class sequences, 285
b(m) class sequences, 121130
denition of, 123
summation properties of, 126130
b (m) class sequences, 140149
characterization of, 142145
denition of, 142
summation properties of, 145
BakerGammel approximants, 360
Bernoulli numbers, 467
Bernoulli polynomials, 467
Best polynomial approximation, 480
Bromwich integral, 231, 437, 463
C-fraction, 333
C-table, 325
Carlemans conditions, 344
C-transformation, 371
Chebyshev expansion, 481
Chebyshev polynomial series, see Chebyshev
expansion
Chebyshev polynomials, 480
ChebyshevPade table, 359
algorithms for, 359
Complex series approach, 254
Conuent D (m) -transformation, 405
Conuent -algorithm, 401
Conuent forms of extrapolation processes, 396401
convergence analysis for, 399
derivation of, 396

515

516

Index

Conuent forms of sequence transformations,

401405
Conuent G-transformation, 402
Conuent Overholt method, 404
Conuent -algorithm, 403
Continuation of midpoint rule, 471
Continuation of trapezoidal rule, 471
Continued fraction, 332
Convergence acceleration methods, 2
CrankNicolson method, 429
Cross-multiplied Pade approximants, see Linear Pade
approximants
D-transformation, 95120
D (m) -transformation, 103
kernel of, 105
s D (m) -transformation, 105

D-transformation,
220

s D-transformation,
221

D-transformation,
224
d-transformation, 121157
on integrals over hypercubes and hypersimplices,
155, 417
(1)
d -transformation, 132
d (m) -transformation, 130
factorial, 132
for innite sequences, 131
kernel of, 131
d(m) -transformation, 147
d (1)
d d -transformation, 275
de Montessuss theorem, 338
extensions of, 339, 340
Deferred approach to the limit, see Richardson
extrapolation process
2 -process, see Aitken 2 -process
2 ( )-process, 289
2 ( , p)-process, 296
Derivatives of extrapolation methods, 268276
of GREP(1) , 274
application to d (1) -transformation, 275
of Richardson extrapolation, 269
application to numerical quadrature, 271
Derivatives of limits and antilimits, see Derivatives
of extrapolation methods
Divergent integrals
Abel sum of, 120
Hadamard nite part of, 120
-algorithm, 9, 301
modications of, 321
Euler series, 329, 351, 374
Euler transformation, 280
Eulers constant, 465
EulerKnopp (E, q) method, 279
analytic properties of, 281
recursive algorithm for, 283
EulerMaclaurin expansion
for multi-dimensional integrals, 416
for periodic singular integrands, 25
for regular integrands, 24
for singular integrands, 58, 82
EulerMaclaurin formula
application to harmonic numbers, 470
for Cauchy principal value integrals, 471
for nonsingular integrals, 469470

for singular integrals, 472474

of periodic integrands, 475
singularities at both endpoints, 474
singularity at one endpoint, 472
for sums, 468
EW-algorithm, 173175
Exponential integral, 97, 329, 351, 374
Exponential interpolation, 336
method of Prony, 336
Exponential order, 463
Exponential sequences, 303
with conuence, 309
Extrapolated quadrature formulas for periodic
singular integrals, 425427
Extrapolation methods, 2
algebraic properties of, 58
algorithms for, 8
iteration of, 14
stable, 12
unstable, 12
Extrapolation methods on
ill-posed problems, 453
innite multiple integrals, 447
innite multiple series, 447
innite oscillatory integrals, 433
integrals over hypercubes and hypersimplices, 415
numerical solutions of integral equations, 421
numerical solutions of ordinary differential
equations, 421
F(1) class functions, 176
(1)
F class functions, 176
F(m) class functions, 81
(m)
F class functions, 81
Factorial d-transformation, 132
Factorial sequences, 285
Factorially divergent sequences, 152, 371
First conuent form, 396
First generalization of Richardson extrapolation,
5780
algebraic properties of, 5961
algorithms for, 6165
E-algorithm, 64
FS-algorithm, 62
analysis of columns, 6777
conditioning, 73
convergence and stability, 6973
rst conuent form, 396398
second conuent form, 398399
Fixed-point iterative method, 288
with iterated 2 -process, 288
with Shanks transformation, 308
Form factor, see Shape function
FS/qd-algorithm, 387
Function of exponential order, 463
Functions of second kind, 254
G-transformation, 384
Gamma function, 465
Gaussian quadrature, 343
Generalized Fourier series, 254
examples of, 256
Generalized function, 4
Generalized Koenigs theorem, 341
extensions of, 341

Index
Generalized Zeta function, 477
Geometric Progression Sampling, see GPS
GPS, 216
GREP, 8194
GREP(1) , 159, 177
analysis for quickly varying A(y), 205211
analysis for slowly varying A(y), 181202
efcient application of, 212217
with APS, 217
with GPS, 216
error formulas for, 177180
( j)
expression for n , 161
( j)
expression for An , 159
GREP(m) , 8586, 164
convergence of, 8892
Process I, 88
Process II, 88
denition of, 85
extensions of, 93, 173
stability of, 92
H-transformation, 261
Hadamard nite part, 5, 26, 117, 120
Hamburger function, 343
Hamburger series, 343
Hankel determinant, 299, 318
Harmonic numbers, 470
HermiteGennochi formula, 180
HermitePade approximants
algebraic, 355
differential, 355
quadratic, 355
HermitePade polynomials, 355
Higher-order G-transformation, 385
algorithms for, 386
FS/qd-algorithm, 387
rs-algorithm, 386
Ill-posed problems, 453
solution via extrapolation, 453
Intermediate rows, see Pade table
IP( ), 310, 340
Iterated 2 -process, 286
convergence analysis for, 286
on xed-point iterations, 288
on power method, 287
stability analysis for, 293
Iterated 2 ( )-process, 290
Iterated 2 ( , p)-process, 296
Iterated Lubkin transformation, 292
convergence analysis for, 292
stability analysis for, 294
Iteration of extrapolation methods, 14
Kernel, 16
Koenigs theorem, see Generalized Koenigs
theorem
L-transformation, 133, 363
t-transformation, 133
u-transformation, 133
algorithms for, 366
convergence and stability of, 367
kernel of, 365
on factorially divergent sequences, 371
rational approximations from, 242

517

Laplace transform, 463

Laplace transform inversion, 437441
by extrapolation of Bromwich integral, 437
by quadrature formulas on Bromwich integral, 437
via discrete Fourier transform, 440
via rational approximations, 439
Linear monotone series, 151
Linear Pade approximants, 356
Linear sequences, 285
Logarithmic sequences, 285
Lubkin W -transformation, 290
kernel of, 291
Markov function, see Stieltjes function, Hamburger
function
Meinarduss method, 391
Method of Prony, 336
Midpoint rule, 470
Modied 2 -process, 289
Moment series, 343
Multiple innite integrals
computation by extrapolation methods, 447
Multiple innite series
computation by extrapolation methods, 447
Multipoint Pade approximants, 351
mW -transformation, 229
NevilleAitken interpolation algorithm, 44
extension of, 50
Nonlinear Pade approximants, 358
ChebyshevPade approximants, 359
Normal ordering, 164
Normal Pade table, 325
Numerical differentiation, 23
and Richardson extrapolation, 4852
Numerical quadrature
formulas from sequence transformations, 430
midpoint rule, 24
Romberg integration, 52
trapezoidal rule, 24
endpoint singularities, 58, 82
periodic singular integrands, 25
variable transformations in, 418
Numerical quadrature formulas for periodic singular
integrals, 423424
O and o symbols, 459
Offset trapezoidal rule, 469
Orthogonal polynomial expansions
BakerGammel approximants from, 360
Pade approximants from
linear (cross-multiplied), 356
nonlinear (properly expanded), 358
Orthogonal polynomials, 343
Oscillatory innite integrals, 218237
Overholts method, 391
analysis of, 392
Polya frequency series, 345
Pade approximants
and exponential interpolation, 336
method of Prony, 336
denition of, 323
for hypergeometric functions, 327
from orthogonal polynomial expansions, 356
linear (cross-multiplied), 356
nonlinear (properly expanded), 358

518

Index

Pade table, 323

algebraic structure of, 325
analysis for entire functions, 346
analysis for meromorphic functions, 338342
de Montessuss theorem, 338
generalized Koenigs theorem, 341
intermediate rows, 340
analysis for moment series, 342345
connection with Gaussian quadrature, 343
connection with orthogonal polynomials, 343
analysis for Polya frequency series, 345
computation of, 330
by Longmans method, 330
connection with continued fractions, 333
qd-algorithm, 335
identities in, 329
normal, 325
staircase in, 334, 335
PadeBorel approximants, 362
Pade-type approximants, 348
Periodic Bernoullian functions, 468
Periodic integral equations
numerical solution by quadrature methods, 422
singular, 422
weakly singular, 422
Polynomial Richardson extrapolation, 4355
algorithm for, 44
error and stability in, 44
Power method, 287
with iterated 2 -process, 287
with Shanks transformation, 308
Prediction properties
of rational d-approximants, 245
of sequence transformations, 245
Process I, 88, 89
Process II, 88, 90
Properly expanded Pade approximants, see Nonlinear
Pade approximants
Psi function, 466
qd-algorithm, 335
Quadratic approximants, 355
Quadrature methods, 423
Quadrature methods via extrapolated formulas for
periodic singular integral equations, 425
periodic weakly singular integral equations, 426
Quasi-linearity, 6
of 2 -process, 7
of Shanks transformation, 10
Quickly varying A(y), 203
Rational d-approximants, 240
d (1) -approximants on factorial power series, 249
algebraic properties of, 242
approximation of singular points by, 246
efcient application with APS, 247
analysis of d (1) -approximants, 247
prediction properties of, 245
recursive computation of, 244
Rational extrapolation, 5556
Reduction of D-transformation, 220237
application to integrals of products of oscillatory
functions, 236

D-transformation,
220

s D-transformation,
221
application to integral transforms, 222

D-transformation,
224
application to oscillatory integrals, 225
W -transformation, 228
mW -transformation, 229
application to integral transforms, 231
theoretical justication of, 230
Reduction of GREP, 218
Regular C-fraction, 333
-algorithm, 375
modications of, 376
( )-algorithm, 377
( )-algorithm, automatic, 378
( , p)-algorithm, 378
Richardson extrapolation process, 2741
algebraic properties of, 29
algorithm for, 28
as a summability method, 3941
conditioning of diagonals, 74
convergence analysis for, 3337
for sequences, 41
polynomial, 28
stability analysis for, 3738
with harmonic collocation points, 42
Richardson extrapolation with conuence, 263268
convergence and stability analysis for, 265267
generalization of, 267
SGRom-algorithm, 264
RichardsonShanks transformation, 449453
application of, 450
to time-periodic steady states, 451
description of, 449
Riemann Zeta function, 25, 84, 477
computation by Richardson extrapolation, 38
Romberg integration, 52
Romberg table, 29
rs-algorithm, 386
S-fraction, 344
S-transformation, 133, 369
algorithms for, 369
convergence and stability of, 370
kernel of, 370
on factorially divergent sequences, 371
rational approximations from, 242
s D (m) -transformation, 105
Second conuent form, 398
Sequence transformations, 2
formal theory of, 407412
generalized remanence, 410
iteration of, 14
Sequential D-transformation, 447
Sequential d-transformation, 447
SGRom-algorithm, 264
Shanks transformation, 9, 297322
algorithms for, 301
-algorithm, 301
analysis of columns
for exponential sequences, 303313
for factorial sequences, 316
for linear sequences, 313
for logarithmic sequences, 316
application to numerical quadrature, 312

Index
derivation of, 297
error formulas for, 302
kernel of, 299
on xed-point iterations, 308
on power method, 308
on totally monotonic sequences, 318
on totally oscillating sequences, 321
Shape function, 87
SilvermanToeplitz theorem, 8
Singular integral equations, 422
Slowly varying A(y), 180
Stability of extrapolation methods, 10
Staircase in Pade table, 334, 335
Steffensens method, 288
Stieltjes function, 343
Stieltjes series, 343
Stirlings formula, 466
Summability methods, 6
linear, 6
regular, 8
Sylvester determinant identity, 62
-algorithm, 379
convergence and stability of, 380383
Time-periodic steady state, 451
Totally monotonic sequences, 317
Totally oscillating sequences, 320
Trapezoidal rule, 469
Two-point Pade approximants, 353
determinant representation for, 354

Uniqueness of
nonlinear Pade approximants, 359
Pade approximants, 323
Pade-type approximants, 349
two-point Pade approximants, 353
Vanden BroeckSchwartz approximations,
350
Vandermonde determinant, 68
Variable transformations, 418
Very oscillatory innite integrals, 227
W-algorithm, 159164, 219
W(m) -algorithm, 164173
normal ordering, 164
the cases m = 1, 2, 171
W -transformation, 228
mW -transformation, 229
variants of, 234
Watsons lemma, 463
Weakly singular integral equations, 422
Wimps method, 391
analysis of, 394
kernel of, 392
X( ) class functions, 98
( )
X0 class functions, 123
Zeta function, 477
Generalized Zeta function, 477

519