Runge-Kutta Methods
Runge-Kutta Methods
(Note: the above equations have different but equivalent definitions in different texts.[4])
Here is the RK4 approximation of , and the next value ( ) is determined by the present value (
) plus the weighted average of four increments, where each increment is the product of the size of the interval,
h, and an estimated slope specified by function f on the right-hand side of the differential equation.
is the slope at the beginning of the interval, using (Euler's method);
is the slope at the midpoint of the interval, using and ;
is again the slope at the midpoint, but now using and ;
is the slope at the end of the interval, using and .
In averaging the four slopes, greater weight is given to the slopes at the midpoint. If is independent of , so that
the differential equation is equivalent to a simple integral, then RK4 is Simpson's rule.[5]
The RK4 method is a fourth-order method, meaning that the local truncation error is on the order of , while
the total accumulated error is on the order of .
In many practical applications the function is independent of (so called autonomous system, or time-invariant
system, especially in physics), and their increments are not computed at all and not passed to function , with
only the final formula for used.
where[6]
(Note: the above equations may have different but equivalent definitions in some texts.[4])
To specify a particular method, one needs to provide the integer s (the number of stages), and the coefficients aij
(for 1 ≤ j < i ≤ s), bi (for i = 1, 2, ..., s) and ci (for i = 2, 3, ..., s). The matrix [aij] is called the Runge–Kutta matrix,
while the bi and ci are known as the weights and the nodes.[7] These data are usually arranged in a mnemonic
device, known as a Butcher tableau (after John C. Butcher):
A Taylor series expansion shows that the Runge–Kutta method is consistent if and only if
There are also accompanying requirements if one requires the method to have a certain order p, meaning that the
local truncation error is O(hp+1). These can be derived from the definition of the truncation error itself. For
example, a two-stage method has order 2 if b1 + b2 = 1, b2c2 = 1/2, and b2a21 = 1/2.[8] Note that a popular
condition for determining coefficients is [8]
This condition alone, however, is neither sufficient, nor necessary for consistency. [9]
In general, if an explicit -stage Runge–Kutta method has order , then it can be proven that the number of stages
must satisfy and if , then .[10] However, it is not known whether these bounds are sharp in
all cases. In some cases, it is proven that the bound cannot be achieved. For instance, Butcher proved that for
, there is no explicit method with stages.[11] Butcher also proved that for , there is no
explicit Runge-Kutta method with stages. [12] In general, however, it remains an open problem what the
precise minimum number of stages is for an explicit Runge–Kutta method to have order . Some values which
are known are:[13]
The provable bound above then imply that we can not find methods of orders that require fewer
stages than the methods we already know for these orders. The work of Butcher also proves that 7th and 8th order
methods have a minimum of 9 and 11 stages, respectively.[11][12] An example of an explicit method of order 6
with 7 stages can be found in Ref.[14] Explicit methods of order 7 with 9 stages[11] and explicit methods of order
8 with 11 stages[15] are also known. See Refs.[16][17] for a summary.
Examples
The RK4 method falls in this framework. Its tableau is[18]
0
1/2 1/2
1/2 0 1/2
1 0 0 1
1/6 1/3 1/3 1/6
A slight variation of "the" Runge–Kutta method is also due to Kutta in 1901 and is called the 3/8-rule.[19] The
primary advantage this method has is that almost all of the error coefficients are smaller than in the popular
method, but it requires slightly more FLOPs (floating-point operations) per time step. Its Butcher tableau is
0
1/3 1/3
2/3 −1/3 1
1 1 −1 1
1/8 3/8 3/8 1/8
However, the simplest Runge–Kutta method is the (forward) Euler method, given by the formula
. This is the only consistent explicit Runge–Kutta method with one stage. The
corresponding tableau is
0
1
0
1/2 1/2
0 1
The midpoint method is not the only second-order Runge–Kutta method with two stages; there is a family of such
methods, parameterized by α and given by the formula[20]
In this family, gives the midpoint method, is Heun's method,[5] and is Ralston's method.
Use
As an example, consider the two-stage second-order Runge–Kutta method with α = 2/3, also known as Ralston
method. It is given by the tableau
0
2/3 2/3
1/4 3/4
with the corresponding equations
During the integration, the step size is adapted such that the estimated error stays below a user-defined threshold:
If the error is too high, a step is repeated with a lower step size; if the error is much smaller, the step size is
increased to save time. This results in an (almost), optimal step size, which saves computation time. Moreover,
the user does not have to spend time on finding an appropriate step size.
which is . The Butcher tableau for this kind of method is extended to give the values of :
The Runge–Kutta–Fehlberg method has two methods of orders 5 and 4. Its extended Butcher tableau is:
0
1/4 1/4
3/8 3/32 9/32
12/13 1932/2197 −7200/2197 7296/2197
1 439/216 −8 3680/513 -845/4104
1/2 −8/27 2 −3544/2565 1859/4104 −11/40
16/135 0 6656/12825 28561/56430 −9/50 2/55
25/216 0 1408/2565 2197/4104 −1/5 0
However, the simplest adaptive Runge–Kutta method involves combining Heun's method, which is order 2, with
the Euler method, which is order 1. Its extended Butcher tableau is:
0
1 1
1/2 1/2
1 0
Other adaptive Runge–Kutta methods are the Bogacki–Shampine method (orders 3 and 2), the Cash–Karp
method and the Dormand–Prince method (both with orders 5 and 4).
Runge–Kutta–Nyström methods
Runge–Kutta–Nyström methods are specialized Runge–Kutta methods that are optimized for second-order
differential equations.[22][23] A general Runge–Kutta–Nyström method for a second-order ODE system
with order is with the form
Two fourth-order explicit RKN methods are given by the following Butcher tables:
These two schemes also have the symplectic-preserving properties when the original equation is derived from a
conservative classical mechanical system, i.e. when
The instability of explicit Runge–Kutta methods motivates the development of implicit methods. An implicit
Runge–Kutta method has the form
where
[26]
The difference with an explicit method is that in an explicit method, the sum over j only goes up to i − 1. This
also shows up in the Butcher tableau: the coefficient matrix of an explicit method is lower triangular. In an
implicit method, the sum over j goes up to s and the coefficient matrix is not triangular, yielding a Butcher tableau
of the form[18]
See Adaptive Runge-Kutta methods above for the explanation of the row.
The consequence of this difference is that at every step, a system of algebraic equations has to be solved. This
increases the computational cost considerably. If a method with s stages is used to solve a differential equation
with m components, then the system of algebraic equations has ms components. This can be contrasted with
implicit linear multistep methods (the other big family of methods for ODEs): an implicit s-step linear multistep
method needs to solve a system of algebraic equations with only m components, so the size of the system does not
increase as the number of steps increases.[27]
Examples
The simplest example of an implicit Runge–Kutta method is the backward Euler method:
Another example for an implicit Runge–Kutta method is the trapezoidal rule. Its Butcher tableau is:
The trapezoidal rule is a collocation method (as discussed in that article). All collocation methods are implicit
Runge–Kutta methods, but not all implicit Runge–Kutta methods are collocation methods.[28]
The Gauss–Legendre methods form a family of collocation methods based on Gauss quadrature. A Gauss–
Legendre method with s stages has order 2s (thus, methods with arbitrarily high order can be constructed).[29] The
method with two stages (and thus order four) has Butcher tableau:
[27]
Stability
The advantage of implicit Runge–Kutta methods over explicit ones is their greater stability, especially when
applied to stiff equations. Consider the linear test equation . A Runge–Kutta method applied to this
equation reduces to the iteration , with r given by
[30]
where e stands for the vector of ones. The function r is called the stability function.[31] It follows from the
formula that r is the quotient of two polynomials of degree s if the method has s stages. Explicit methods have a
strictly lower triangular matrix A, which implies that det(I − zA) = 1 and that the stability function is a
polynomial.[32]
The numerical solution to the linear test equation decays to zero if | r(z) | < 1 with z = hλ. The set of such z is
called the domain of absolute stability. In particular, the method is said to be absolute stable if all z with Re(z) < 0
are in the domain of absolute stability. The stability function of an explicit Runge–Kutta method is a polynomial,
so explicit Runge–Kutta methods can never be A-stable.[32]
If the method has order p, then the stability function satisfies as . Thus, it is of
interest to study quotients of polynomials of given degrees that approximate the exponential function the best.
These are known as Padé approximants. A Padé approximant with numerator of degree m and denominator of
degree n is A-stable if and only if m ≤ n ≤ m + 2.[33]
The Gauss–Legendre method with s stages has order 2s, so its stability function is the Padé approximant with m =
n = s. It follows that the method is A-stable.[34] This shows that A-stable Runge–Kutta can have arbitrarily high
order. In contrast, the order of A-stable linear multistep methods cannot exceed two.[35]
B-stability
The A-stability concept for the solution of differential equations is related to the linear autonomous equation
. Dahlquist (1963) proposed the investigation of stability of numerical schemes when applied to
nonlinear systems that satisfy a monotonicity condition. The corresponding concepts were defined as G-stability
for multistep methods (and the related one-leg methods) and B-stability (Butcher, 1975) for Runge–Kutta
methods. A Runge–Kutta method applied to the non-linear system , which verifies
, is called B-stable, if this condition implies for two
numerical solutions.
A Runge–Kutta method is said to be algebraically stable[36] if the matrices and are both non-negative
definite. A sufficient condition for B-stability [37] is: and are non-negative definite.
where:
We develop the derivation[38] for the Runge–Kutta fourth-order method using the general formula with
evaluated, as explained above, at the starting point, the midpoint and the end point of any interval ;
thus, we choose:
and otherwise. We begin by defining the following quantities:
and for the previous relations we can show that the following equalities hold up to :
where:
See also
Euler's method
List of Runge–Kutta methods
Numerical methods for ordinary differential equations
Runge–Kutta method (SDE)
General linear methods
Lie group integrator
Notes
1. "Runge-Kutta method" (https://www.dictionary.com/browse/runge-kutta-method). Dictionary.com.
Retrieved 4 April 2021.
2. DEVRIES, Paul L.; HASBUN, Javier E. A first course in computational physics. Second edition.
Jones and Bartlett Publishers: 2011. p. 215.
3. Press et al. 2007, p. 908; Süli & Mayers 2003, p. 328
4. Atkinson (1989, p. 423), Hairer, Nørsett & Wanner (1993, p. 134), Kaw & Kalu (2008, §8.4) and
Stoer & Bulirsch (2002, p. 476) leave out the factor h in the definition of the stages. Ascher &
Petzold (1998, p. 81), Butcher (2008, p. 93) and Iserles (1996, p. 38) use the y values as stages.
5. Süli & Mayers 2003, p. 328
6. Press et al. 2007, p. 907
7. Iserles 1996, p. 38
8. Iserles 1996, p. 39
9. As a counterexample, consider any explicit 2-stage Runge-Kutta scheme with and
and randomly chosen. This method is consistent and (in general) first-order convergent. On the
other hand, the 1-stage method with is inconsistent and fails to converge, even though it
References
Runge, Carl David Tolmé (1895), "Über die numerische Auflösung von Differentialgleichungen" (http
s://zenodo.org/record/2178704), Mathematische Annalen, 46 (2), Springer: 167–178,
doi:10.1007/BF01446807 (https://doi.org/10.1007%2FBF01446807), S2CID 119924854 (https://api.
semanticscholar.org/CorpusID:119924854).
Kutta, Wilhelm (1901), "Beitrag zur näherungsweisen Integration totaler Differentialgleichungen" (htt
ps://archive.org/details/zeitschriftfrma12runggoog/page/434/), Zeitschrift für Mathematik und
Physik, 46: 435–453.
Ascher, Uri M.; Petzold, Linda R. (1998), Computer Methods for Ordinary Differential Equations and
Differential-Algebraic Equations, Philadelphia: Society for Industrial and Applied Mathematics,
ISBN 978-0-89871-412-8.
Atkinson, Kendall A. (1989), An Introduction to Numerical Analysis (2nd ed.), New York: John Wiley
& Sons, ISBN 978-0-471-50023-0.
Butcher, John C. (May 1963), "Coefficients for the study of Runge-Kutta integration processes",
Journal of the Australian Mathematical Society, 3 (2): 185–201, doi:10.1017/S1446788700027932
(https://doi.org/10.1017%2FS1446788700027932).
Butcher, John C. (May 1964), "On Runge-Kutta processes of high order", Journal of the Australian
Mathematical Society, 4 (2): 179–194, doi:10.1017/S1446788700023387 (https://doi.org/10.1017%
2FS1446788700023387)
Butcher, John C. (1975), "A stability property of implicit Runge-Kutta methods", BIT, 15 (4): 358–
361, doi:10.1007/bf01931672 (https://doi.org/10.1007%2Fbf01931672), S2CID 120854166 (https://
api.semanticscholar.org/CorpusID:120854166).
Butcher, John C. (2000), "Numerical methods for ordinary differential equations in the 20th century",
J. Comput. Appl. Math., 125 (1–2): 1–29, Bibcode:2000JCoAM.125....1B (https://ui.adsabs.harvard.
edu/abs/2000JCoAM.125....1B), doi:10.1016/S0377-0427(00)00455-6 (https://doi.org/10.1016%2F
S0377-0427%2800%2900455-6).
Butcher, John C. (2008), Numerical Methods for Ordinary Differential Equations, New York: John
Wiley & Sons, ISBN 978-0-470-72335-7.
Cellier, F.; Kofman, E. (2006), Continuous System Simulation, Springer Verlag, ISBN 0-387-26102-
8.
Dahlquist, Germund (1963), "A special stability problem for linear multistep methods", BIT, 3: 27–
43, doi:10.1007/BF01963532 (https://doi.org/10.1007%2FBF01963532), hdl:10338.dmlcz/103497
(https://hdl.handle.net/10338.dmlcz%2F103497), ISSN 0006-3835 (https://search.worldcat.org/issn/
0006-3835), S2CID 120241743 (https://api.semanticscholar.org/CorpusID:120241743).
Forsythe, George E.; Malcolm, Michael A.; Moler, Cleve B. (1977), Computer Methods for
Mathematical Computations, Prentice-Hall (see Chapter 6).
Hairer, Ernst; Nørsett, Syvert Paul; Wanner, Gerhard (1993), Solving ordinary differential equations
I: Nonstiff problems, Berlin, New York: Springer-Verlag, ISBN 978-3-540-56670-0.
Hairer, Ernst; Wanner, Gerhard (1996), Solving ordinary differential equations II: Stiff and
differential-algebraic problems (2nd ed.), Berlin, New York: Springer-Verlag, ISBN 978-3-540-
60452-5.
Iserles, Arieh (1996), A First Course in the Numerical Analysis of Differential Equations, Cambridge
University Press, ISBN 978-0-521-55655-2.
Lambert, J.D (1991), Numerical Methods for Ordinary Differential Systems. The Initial Value
Problem, John Wiley & Sons, ISBN 0-471-92990-5
Kaw, Autar; Kalu, Egwu (2008), Numerical Methods with Applications (http://numericalmethods.eng.
usf.edu/topics/textbook_index.html) (1st ed.), autarkaw.com.
Press, William H.; Teukolsky, Saul A.; Vetterling, William T.; Flannery, Brian P. (2007), "Section 17.1
Runge-Kutta Method" (http://apps.nrbook.com/empanel/index.html#pg=907), Numerical Recipes:
The Art of Scientific Computing (3rd ed.), Cambridge University Press, ISBN 978-0-521-88068-8.
Also, Section 17.2. Adaptive Stepsize Control for Runge-Kutta (http://apps.nrbook.com/empanel/ind
ex.html#pg=910).
Stoer, Josef; Bulirsch, Roland (2002), Introduction to Numerical Analysis (3rd ed.), Berlin, New
York: Springer-Verlag, ISBN 978-0-387-95452-3.
Süli, Endre; Mayers, David (2003), An Introduction to Numerical Analysis, Cambridge University
Press, ISBN 0-521-00794-1.
Tan, Delin; Chen, Zheng (2012), "On A General Formula of Fourth Order Runge-Kutta Method" (htt
p://msme.us/2012-2-1.pdf) (PDF), Journal of Mathematical Science & Mathematics Education, 7
(2): 1–10.
advance discrete maths ignou reference book (code- mcs033)
John C. Butcher: "B-Series : Algebraic Analysis of Numerical Methods", Springer(SSCM, volume
55), ISBN 978-3030709556 (April, 2021).
Butcher, J.C. (1985), "The non-existence of ten stage eighth order explicit Runge-Kutta methods" (h
ttps://link.springer.com/article/10.1007/BF01935372), BIT Numerical Mathematics, 25 (3): 521–540,
doi:10.1007/BF01935372 (https://doi.org/10.1007%2FBF01935372).
Butcher, J.C. (1965), "On the attainable order of Runge-Kutta methods" (https://www.ams.org/journ
als/mcom/1965-19-091/S0025-5718-1965-0179943-X/), Mathematics of Computation, 19 (91):
408–417, doi:10.1090/S0025-5718-1965-0179943-X (https://doi.org/10.1090%2FS0025-5718-1965
-0179943-X).
Curtis, A.R. (1970), "An eighth order Runge-Kutta process with eleven function evaluations per
step" (https://link.springer.com/article/10.1007/BF02219778), Numerische Mathematik, 16 (3): 268–
277, doi:10.1007/BF02219778 (https://doi.org/10.1007%2FBF02219778).
Cooper, G.J.; Verner, J.H. (1972), "Some Explicit Runge–Kutta Methods of High Order" (https://epu
bs.siam.org/doi/abs/10.1137/0709037?journalCode=sjnaam), SIAM Journal on Numerical Analysis,
9 (3): 389–405, doi:10.1137/0709037 (https://doi.org/10.1137%2F0709037).
Butcher, J.C. (1996), "A History of Runge-Kutta Methods" (https://dx.doi.org/10.1016/0168-9274%2
895%2900108-5), Applied Numerical Mathematics, 20 (3): 247–260, doi:10.1016/0168-
9274(95)00108-5 (https://doi.org/10.1016%2F0168-9274%2895%2900108-5).
External links
"Runge-Kutta method" (https://www.encyclopediaofmath.org/index.php?title=Runge-Kutta_method),
Encyclopedia of Mathematics, EMS Press, 2001 [1994]
Runge–Kutta 4th-Order Method (http://numericalmethods.eng.usf.edu/topics/runge_kutta_4th_meth
od.html)
Tracker Component Library Implementation in Matlab (https://github.com/USNavalResearchLaborat
ory/TrackerComponentLibrary/tree/master/Mathematical_Functions/Differential_Equations) —
Implements 32 embedded Runge Kutta algorithms in RungeKStep, 24 embedded Runge-Kutta
Nyström algorithms in RungeKNystroemSStep and 4 general Runge-Kutta Nyström algorithms in
RungeKNystroemGStep.