Numerical Integration and

Monte Carlo Integration

Elementary schemes for integration over one variable
Multi-dimensional integration
- dimension-by dimension
Problems with multi-dimensional numerical integrations

Monte Carlo sampling of high-dimensional integrals

- including some aspects of analysis of statistical data
Numerical integration in one dimension
Function of one variable x, assume no singularities
Z b
<latexit sha1_base64="zRzIGDBfZOR7QXMHfsY9MDDCqVQ=">AAAB+3icbVDLSsNAFL3xWeur1qWbwSLUTUmkqBuh4MbuKtgHtDFMJpN26GQSZibSEvorblwo4tYfceffOH0stPXAhcM593LvPX7CmdK2/W2trW9sbm3ndvK7e/sHh4WjYkvFqSS0SWIey46PFeVM0KZmmtNOIimOfE7b/vB26refqFQsFg96nFA3wn3BQkawNpJXKNZvekxoDz/6KCyPzlEw8golu2LPgFaJsyAlWKDhFb56QUzSiApNOFaq69iJdjMsNSOcTvK9VNEEkyHu066hAkdUudns9gk6M0qAwliaEhrN1N8TGY6UGke+6YywHqhlbyr+53VTHV67GRNJqqkg80VhypGO0TQIFDBJieZjQzCRzNyKyABLTLSJK29CcJZfXiWti4pzWaneV0u1+iKOHJzAKZTBgSuowR00oAkERvAMr/BmTawX6936mLeuWYuZY/gD6/MHYlWTZw==</latexit>
<latexit sha1_base64="sJSn+CGaT6gGlSgJqDYdUNuqNXQ=">AAAB63icbVBNSwMxEJ2tX7V+VT16CRahXsqulOqxIIjHCvYD2qVk02wbmmSXJCuWpeAv8OJBEa/+IW/+G7NtD9r6YODx3gwz84KYM21c99vJra1vbG7ltws7u3v7B8XDo5aOEkVok0Q8Up0Aa8qZpE3DDKedWFEsAk7bwfg689sPVGkWyXsziakv8FCykBFsMiksP573iyW34s6AVom3ICVYoNEvfvUGEUkElYZwrHXXc2Pjp1gZRjidFnqJpjEmYzykXUslFlT76ezWKTqzygCFkbIlDZqpvydSLLSeiMB2CmxGetnLxP+8bmLCKz9lMk4MlWS+KEw4MhHKHkcDpigxfGIJJorZWxEZYYWJsfEUbAje8surpHVR8WqV6l21VL95mseRhxM4hTJ4cAl1uIUGNIHACJ7hFd4c4bw4787HvDXnLCI8hj9wPn8AloGOXg==</latexit>

f (x)
I= f (x)dx
a
Discretize the x-axis
- n+1 equally spaced points including a,b:

Consider groups of m+1 points (m intervals of size h)

Construct the order-m polynomials tting the m+1 points
a b
<latexit sha1_base64="EddW87na/N41TigdLPZrpGWd8Wc=">AAAB6HicbVDLSgNBEOyNrxhfUY9eBoPgKexKUI8BQTwmYB6QLGF20puMmZ1dZmaFsAS8e/GgiFc/yZt/4+Rx0MSChqKqm+6uIBFcG9f9dnJr6xubW/ntws7u3v5B8fCoqeNUMWywWMSqHVCNgktsGG4EthOFNAoEtoLRzdRvPaLSPJb3ZpygH9GB5CFn1FipHvSKJbfszkBWibcgJVig1it+dfsxSyOUhgmqdcdzE+NnVBnOBE4K3VRjQtmIDrBjqaQRaj+bHTohZ1bpkzBWtqQhM/X3REYjrcdRYDsjaoZ62ZuK/3md1ITXfsZlkhqUbL4oTAUxMZl+TfpcITNibAllittbCRtSRZmx2RRsCN7yy6ukeVH2LsuVeqVUvX2ax5GHEziFc/DgCqpwBzVoAAOEZ3iFN+fBeXHenY95a85ZRHgMf+B8/gDvA41z</latexit>

<latexit sha1_base64="GylYjbmqkFiIcBaLkD+c3ZarhnE=">AAAB6HicbVDLSgNBEOyNrxhfUY9eBoPgKexKUI8BQTwmYB6QLGF20puMmZ1dZmaFsAS8e/GgiFc/yZt/4+Rx0MSChqKqm+6uIBFcG9f9dnJr6xubW/ntws7u3v5B8fCoqeNUMWywWMSqHVCNgktsGG4EthOFNAoEtoLRzdRvPaLSPJb3ZpygH9GB5CFn1FipTnvFklt2ZyCrxFuQEixQ6xW/uv2YpRFKwwTVuuO5ifEzqgxnAieFbqoxoWxEB9ixVNIItZ/NDp2QM6v0SRgrW9KQmfp7IqOR1uMosJ0RNUO97E3F/7xOasJrP+MySQ1KNl8UpoKYmEy/Jn2ukBkxtoQyxe2thA2poszYbAo2BG/55VXSvCh7l+VKvVKq3j7N48jDCZzCOXhwBVW4gxo0gAHCM7zCm/PgvDjvzse8NecsIjyGP3A+fwDtf41y</latexit>

h
<latexit sha1_base64="rTY8A/41fKd2STNcY2eiXTDuu2Y=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkqMeiF48t2FpoQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fCoreNUMWyxWMSqE1CNgktsGW4EdhKFNAoEPgTj25n/8IRK81jem0mCfkSHkoecUWOl5qhfrrhVdw6ySrycVCBHo1/+6g1ilkYoDRNU667nJsbPqDKcCZyWeqnGhLIxHWLXUkkj1H42P3RKzqwyIGGsbElD5urviYxGWk+iwHZG1Iz0sjcT//O6qQmv/YzLJDUo2WJRmApiYjL7mgy4QmbExBLKFLe3EjaiijJjsynZELzll1dJ+6LqXVZrzVqlfpPHUYQTOIVz8OAK6nAHDWgBA4RneIU359F5cd6dj0VrwclnjuEPnM8f0OeM9Q==</latexit>

h
<latexit sha1_base64="O7Xu1s/0XnX9z4tvwLA2vstsv+M=">AAAB6HicbVDLSgNBEOyNrxhfUY9eBoPgKexKUI8BQTwmYB6QLGF20puMmZ1dZmaFsAS8e/GgiFc/yZt/4+Rx0MSChqKqm+6uIBFcG9f9dnJr6xubW/ntws7u3v5B8fCoqeNUMWywWMSqHVCNgktsGG4EthOFNAoEtoLRzdRvPaLSPJb3ZpygH9GB5CFn1FipPuwVS27ZnYGsEm9BSrBArVf86vZjlkYoDRNU647nJsbPqDKcCZwUuqnGhLIRHWDHUkkj1H42O3RCzqzSJ2GsbElDZurviYxGWo+jwHZG1Az1sjcV//M6qQmv/YzLJDUo2XxRmApiYjL9mvS5QmbE2BLKFLe3EjakijJjsynYELzll1dJ86LsXZYr9Uqpevs0jyMPJ3AK5+DBFVThDmrQAAYIz/AKb86D8+K8Ox/z1pyziPAY/sD5/AH4G415</latexit>

h
<latexit sha1_base64="rTY8A/41fKd2STNcY2eiXTDuu2Y=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkqMeiF48t2FpoQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fCoreNUMWyxWMSqE1CNgktsGW4EdhKFNAoEPgTj25n/8IRK81jem0mCfkSHkoecUWOl5qhfrrhVdw6ySrycVCBHo1/+6g1ilkYoDRNU667nJsbPqDKcCZyWeqnGhLIxHWLXUkkj1H42P3RKzqwyIGGsbElD5urviYxGWk+iwHZG1Iz0sjcT//O6qQmv/YzLJDUo2WJRmApiYjL7mgy4QmbExBLKFLe3EjaiijJjsynZELzll1dJ+6LqXVZrzVqlfpPHUYQTOIVz8OAK6nAHDWgBA4RneIU359F5cd6dj0VrwclnjuEPnM8f0OeM9Q==</latexit>

Simple formulas exist to construct the polynomials Pi(x)

Integrate the polynomials exactly and add up
Leads to simple integration formulas (sums) for small m
Error for one window typically of order O(hm+1) or O(hm+2)
Simplest case; m=1 (trapezoidal rule)

Here we also see that the error (“step error”) is O(h3)

For the total error, we have to sum up step errors from (general m)

Assuming no “lucky” error cancellations (from sign oscillations)

- error is O(hm+2)O(h-1) ~ O(hm+1)
Second order; m=2 (Simpson’s rule)

Solve for a,b,c, integrate polynomial →

What is the order of the error?

- from the polynomial it may seem O(h4) (from missing integrated 3 term)
Write expansion around x1 instead:

When integrated in the symmetric window, all odd powers give 0

Same formula for I1 as above when a,b,c terms included, d term gives 0
Error is O(h5), becomes O(h4) for range [x0,xn]
Extended formulas

trapezoid

Simpson

For integrands with singularities at the end point(s); open interval formulas
midpoint rule

Alternative:

- interior points have O(h3) errors, sum to O(h2), end points contribute O(h2) errors
Comments on singularities
Open-interval formulas can be used
- singular point(s) should be at end(s); divide up interval in parts if needed
- but convergence with number of points n may be very slow
Divergent part can some times be subtracted and solved analytically
More sophisticated methods exist for dif cult cases
Other methods
Gaussian quadrature:
- non-uniform grid points; n+1 points → exact result for polynomial of order n
- several Julia packages, e.g., FastGaussQuadrature.jl
Gauss-Kronrod quadrature:
- uses two Gaussian quad. evaluations for different n, similarly to Romberg
- package QuadGK.jl uses a version of this method
Adaptive grid (adaptive mesh):
- dynamically adapted to be more dense where most needed
In nite integration range
Change variables to make range nite
fi
fi
Multi-Dimensional integration

Can be carried out numerically dimension-by-dimension

Example, function of two variables

Integrating numerically over x rst, gives a function of y:

This has to be done for values of y on a grid, to be used in

Very time consuming for large dimensionality D; scaling MD of effort

- M represents ~mean number of grid points for 1D integrals

fi
Monte Carlo Integration
An integral over a nite volume V:
- is (by de nition) the mean value of the function times the volume

The mean value <f> can be estimated by sampling

- generate N random (uniformly distributed) x values xi in the range, then

For nite N, there is a statistical error:

interepretation of the mean error:
If the “simulation” is repeated many times,
the averaged squared error (variance) tends
The statistical result for the to a value a/N, for with a some constant
integral should be expressed as

Computing the “error bar” is an important aspect of the sampling method

fi
fi
fi
𝜎
Standard ilustration of MC integration; estimate of
Consider a circle of radius 1, centered at (x,y)=0. De ne a function:
mean value inside
>
<latexit sha1_base64="g3ff9RqB3A+7Sv0CqKOnEcQpLr8=">AAAB6HicbVDLSgNBEOyNrxhfUY9eBoPgKexKUE8S8KK3BMwDkiXMTnqTMbOzy8ysEEK+wIsHRbz6Sd78GyfJHjSxoKGo6qa7K0gE18Z1v53c2vrG5lZ+u7Czu7d/UDw8auo4VQwbLBaxagdUo+ASG4Ybge1EIY0Cga1gdDvzW0+oNI/lgxkn6Ed0IHnIGTVWqt/0iiW37M5BVomXkRJkqPWKX91+zNIIpWGCat3x3MT4E6oMZwKnhW6qMaFsRAfYsVTSCLU/mR86JWdW6ZMwVrakIXP198SERlqPo8B2RtQM9bI3E//zOqkJr/0Jl0lqULLFojAVxMRk9jXpc4XMiLEllClubyVsSBVlxmZTsCF4yy+vkuZF2bssV+qVUvU+iyMPJ3AK5+DBFVThDmrQAAYIz/AKb86j8+K8Ox+L1pyTzRzDHzifP5NajNI=</latexit>

the surrounding box

Use MC sampling to compute:

Expected fraction of “hits”
inside circle = /4 The error after
N steps

Four repetitions
of a simulation,
dots showing
partial results as
the mean value
evolves

We should compute the statistical error properly

fi
𝜋