Content deleted Content added
Undid revision 567273160: this is just recursion with an explicit stack, and doesn't add anything mathematically or practically to the description of the algorithm |
m MOS:ELLIPSES (via WP:JWB) |
||
(25 intermediate revisions by 16 users not shown) | |||
Line 2:
first1=Nicholas J. | last1=Higham | journal=[[SIAM Journal on Scientific Computing]] |
volume=14 | issue=4 | pages=783–799 | doi=10.1137/0914050 | year=1993
| citeseerx=10.1.1.43.3535 }}</ref> Although there are other techniques such as [[Kahan summation]] that typically have even smaller round-off errors, pairwise summation is nearly as good (differing only by a logarithmic factor) while having much lower computational cost—it can be implemented so as to have nearly the same cost (and exactly the same number of arithmetic operations) as naive summation.
In particular, pairwise summation of a sequence of ''n'' numbers ''x<sub>n</sub>'' works by [[recursion (computer science)|recursively]] breaking the sequence into two halves, summing each half, and adding the two sums: a [[divide and conquer algorithm]]. Its worst-case roundoff errors grow [[Big O notation|asymptotically]] as at most ''O''(ε log ''n''), where ε is the [[machine precision]] (assuming a fixed [[condition number]], as discussed below).<ref name=Higham93/> In comparison, the naive technique of accumulating the sum in sequence (adding each ''x<sub>i</sub>'' one at a time for ''i'' = 1, ..., ''n'') has roundoff errors that grow at worst as ''O''(ε''n'').<ref name=Higham93/> [[Kahan summation]] has a [[error bound|worst-case error]] of roughly ''O''(ε), independent of ''n'', but requires several times more arithmetic operations.<ref name=Higham93/> If the roundoff errors are random, and in particular have random signs, then they form a [[random walk]] and the error growth is reduced to an average of <math>O(\varepsilon \sqrt{\log n})</math> for pairwise summation.<ref name=Tasche>Manfred Tasche and Hansmartin Zeuner ''Handbook of Analytic-Computational Methods in Applied Mathematics'' Boca Raton, FL: CRC Press, 2000).</ref>
A very similar recursive structure of summation is found in many [[fast Fourier transform]] (FFT) algorithms, and is responsible for the same slow roundoff accumulation of those FFTs.<ref name="Tasche
==The algorithm==
In [[pseudocode]], the pairwise summation algorithm for an [[Array data type|array]]
''s'' = '''pairwise'''(''x''[1
'''if''' ''n'' ≤ ''N''
''s'' =
'''for''' ''i'' =
''s'' = ''s'' + ''x''[''i'']
'''else
''m'' = [[Floor and ceiling functions|floor]](''n'' / 2)
''s'' = '''pairwise'''(''x''[1
For some sufficiently small
In an algorithm of this sort (as for [[Divide and conquer algorithm#Choosing the base cases|divide and conquer algorithm]]s in general<ref>Radu Rugina and Martin Rinard, "[http://people.csail.mit.edu/rinard/paper/lcpc00.pdf Recursion unrolling for divide and conquer programs]," in ''Languages and Compilers for Parallel Computing'', chapter 3, pp. 34–48. ''Lecture Notes in Computer Science'' vol. 2017 (Berlin: Springer, 2001).</ref>), it is desirable to use a larger base case in order to [[Amortized analysis|amortize]] the overhead of the recursion. If ''N'' = 1, then there is roughly one recursive subroutine call for every input, but more generally there is one recursive call for (roughly) every ''N''/2 inputs if the recursion stops at exactly ''n'' = ''N''. By making ''N'' sufficiently large, the overhead of recursion can be made negligible (precisely this technique of a large base case for recursive summation is employed by high-performance FFT implementations<ref name=JohnsonFrigo08/>).
Line 48:
Note that the <math>1 - \varepsilon \log_2 n</math> denominator is effectively 1 in practice, since <math>\varepsilon \log_2 n</math> is much smaller than 1 until ''n'' becomes of order 2<sup>1/ε</sup>, which is roughly 10<sup>10<sup>15</sup></sup> in double precision.
In comparison, the relative error bound for naive summation (simply adding the numbers in sequence, rounding at each step) grows as <math>O(\varepsilon n)</math> multiplied by the condition number.<ref name=Higham93/> In practice, it is much more likely that the rounding errors have a random sign, with zero mean, so that they form a random walk; in this case, naive summation has a [[root mean square]] relative error that grows as <math>O(\varepsilon \sqrt{n})</math> and pairwise summation
==Software implementations==
Pairwise summation is the default summation algorithm in [[NumPy]]<ref>[https://github.com/numpy/numpy/pull/3685 ENH: implement pairwise summation], github.com/numpy/numpy pull request #3685 (September 2013).</ref> and the [[Julia (programming language)|Julia technical-computing language]],<ref>[https://github.com/JuliaLang/julia/pull/4039 RFC: use pairwise summation for sum, cumsum, and cumprod], github.com/JuliaLang/julia pull request #4039 (August 2013).</ref> where in both cases it was found to have comparable speed to naive summation (thanks to the use of a large base case).
Other software implementations include the HPCsharp library<ref>https://github.com/DragonSpit/HPCsharp HPCsharp nuget package of high performance C# algorithms</ref> for the [[C Sharp (programming language)|C#]] language and the standard library summation<ref>{{Cite web|title=std.algorithm.iteration - D Programming Language|url=https://dlang.org/phobos/std_algorithm_iteration.html#sum|access-date=2021-04-23|website=dlang.org}}</ref> in [[D (programming language)|D]].
==References==
{{reflist}}
[[Category:Computer arithmetic]]
|