1

Comparing Monte Carlo and Riemann approximation

Abstract Computers can approximate integrals that are difcult or impossible to solve analytically. This paper demonstrates Riemann and Monte Carlo integration, two basic methods of numeric integration. We begin by integrating functions with simple analytic solutions, then focus on more difcult cases. Finally, both methods are used to compute the unit sphere volume. We discuss why Monte Carlo integretion is preferable in two special cases: integrating very spiky functions and high dimensional integration. Keywords: High dimensional integration, simulation, smooth function, rapidly oscillating function 1. Introduction The concept of Riemann sums is familiar from introductory calculus. To approximate the area under a curve dened by f (x) in the interval (a, b), one creates a staircase of n columns that ascend and descend alongside the curve. Each column, which is centered over Xi for i = 1, ..., n, has height f (Xi ) and width w = (b a)/n. The Riemann sum is the sum of the column areas, wf (Xi ). As n increases, the Riemann sum becomes a better approximation of the area under the curve. An alternative method, called Monte Carlo approximation, begins with generating the Xi at random from the UNIF(0,1) distribution. One then computes the area of the rectangle Ai = (b a)f (Xi ), and the Monte Carlo approximation is the mean of n areas, n Ai /n. As with the Riemann approximation, the Monte i=1 Carlo approximation improves with larger n. 2. Comparing approximation methods for a smooth curve In general we are interested in how large n has to be for each method to give a reasonable approximation. It turns out that the answer depends partially on the shape of the curve. Consider, for instance, the smooth 1 curve described by J = 0 xd dx with d = 1/2. A trace plot of both methods (Figures 1a and 1b) shows that the Riemann sum converges much faster than the Monte Carlo approximation. Using the Riemann method, you can obtain a very good approximation with n = 10. When using the Monte Carlo method, the usual practice is to decide beforehand on an acceptable margin of error and use a value of n that achieves it. Figure 1c is a trace of the approximate 95% margin of error. Judging from the trace plot, n should be at least 2,000.

3. Important conditions for the Monte Carlo method An important assumption for Monte Carlo approximation is that f (x) is bounded. In practice, however, because Xi UNIF(0,1) and P (Xi = a) = 0 for an unbounded value f (a), and for this probabilistic reason we can sometimes get reasonable results even if this assumption is violated. Figures 2 and 3 plot traces for d = 1/2 and d = 1. Our function f (x) is unbounded in both cases at x = 0; in fact, because J is undened for d = 1, an attempt at approximation is nonsensical. In the former case the integral is dened with J = 2, but the Monte Carlo method sometimes gives rather poor approximations. These poor approximations are caused by an Xi occurring very close to zero, and the problem is exaggerated for d = 1. Riemann approximation is preferable for this function.

r1[1:10, 3]

0.67

0.69

2 0.50 0.65 0.80

6 r1[1:10, 1]

10

r1[, 4]

2000

4000 r1[, 1]

6000

8000

10000

0.0 0.2 0.4

r1[, 6]

2000

4000

6000

8000

10000

Figure 1: Traces of the denite integral for each method and Monte Carlo margin of error for J(x) =

r1[1:1000, 3]

1.4

1.7

2.0

200

400 r1[1:1000, 1]

600

800

1000

r1[, 4]

0 12

2000

4000 r1[, 1]

6000

8000

10000

r1[, 6]

2000

4000

6000

8000

10000
1 x

Figure 2: Traces of the denite integral for each method and Monte Carlo margin of error for J(x) =

r1[1:1000, 3]

200

400 r1[1:1000, 1]

600

800

1000

r1[, 4]

10000

2000

4000 r1[, 1]

6000

8000

10000

r1[, 6]

20000

2000

4000

6000

8000

10000
1 x

Figure 3: Traces of the denite integral for each method and Monte Carlo margin of error for J(x) = 4. A rapidly oscillating function

A curve consisting of thin spikes presents difculties for any area approximation method. Many subintervals (large n) will be required to achieve a reasonably accurate approximation. A good example of such a curve is represented by h(x) = | sin(kx)| where k is a large positive integer. Figure 4 shows this type of function where k is only 4. We need to do a little calculus to get the exact value for comparison. Examining plots of h(x) for several k (Figure 1), it is clear that the denite integral 1 1/k sin(kx)dx. We can easily nd the exact value of this expression: 0 h(x)dx can be rewritten k 0

1/k

k
0

sin(kx)dx = k[

1 1/k cos(kx)]0 k 1 k 1 1 1 2 = cos( ) + cos(0) = + = k

In Figure 5, we again plot traces to nd good values of n. Moreover, to discover how trustworthy our simulated margins of error are, we can use calculus to nd the theoretical variance of h(x).

V (Y ) = V (h(x)) = V (h(U )) = E[h2 (U )] {E[h(U )]}2 We use the change-of-variable technique to nd the expected value of a function of a random variable. The expected value is the product of the function and the underlying distribution integrated over the relevant region.

0.0 0.0

0.4

0.8

0.2

0.4

0.6

0.8

1.0

Figure 4: Plot of h(x) = |sin(4x)|

V (Y ) =
0

h2 (u)U (u)du
0

h(u)U (u)du

Now since U (u) = 1/(b a) = 1, we have

V (Y ) =
0

h2 (u)du
0

h(u)du

We already know the second term is (2/)2 = 4/ 2 . The rst term is

sin2 (ku)du =
0

u sin(2ku) 2 4k

=
0

1 sin(2k) 1 = 2 4k 2

So V (Y ) = 1/2 4/ 2 = 0.0947 and the theoretical margin of error for n =10,000 is 0.0062. This appears to be very consistent with the trace plots. Moreover, the traces clearly show that Monte Carlo converges to a reasonable approximate much sooner than Riemann, which sometimes produces very poor estimates. Because Monte Carlo relies on averages, it is very robust to extreme function values. 5. Approximating an integral with no analytic solution Our results so far suggest that we should prefer the Riemann method for smooth functions and the Monte Carlo method for spiky functions. Let us apply this heuristic in approximating an integral with no analytic 1 solution, such as J = 0 sin2 (1/x)dx . Unfortunately, the plot of J in Figure 2 shows a half-spiky, halfsmooth function, so our heuristic does not help much. It turns out that we can transform J into an equivalent integral-one with equal area but a smooth boundary. Given 0 sin2 (1/x)dx = /2, we can show that J = /2 the latter equation:
1 1 1 2 2 0 x sin (x)dx.

We substitute integrals in

sin2 (1/x)dx =
0 0 1

sin2 (1/x)dx
0

sin2 x dx x2
1

=
0

sin2 (1/x)dx +
1

sin2 (1/x)dx
0

sin2 x dx x2

r2[1:1000, 3]

0.0

0.4

0.8

200

400 r2[1:1000, 1]

600

800

1000

r2[, 4]

0.4

0.7

2000

4000 r2[, 1]

6000

8000

10000

r2[, 6]

0.0

0.3

0.6

2000

4000

6000

8000

10000

Figure 5: Traces of the denite integral for each method and Monte Carlo margin of error for h(x) = |sin(5000x)| This yields

sin2 (1/x)dx =
1 0

sin2 x dx x2

Using software, we nd that the R.H.S. is

x sin2 (1/x) Si(2/x)| = lim [x sin2 (1/x) Si(2/x)] [sin2 (1) Si(2)] 1
x

where Si is the sine integral. The limiting term vanishes, leaving Si(2)sin2 (1). The L.H.S. is This expression vanishes as x tends to zero, leaving

1 2xSi(2x)+cos(2x)1 . 2x 0

2Si(2) + (2) 1 1 = Si(2) + (cos2 1) = Si(2) sin2 (1) 2 2 by the trigonometric identities. Using the transformed denition, the Riemann produces a good estimate at about n = 25. This behavior is consistent with our heuristic. As with the other smooth curves we looked at, the Monte Carlo method seems to need n = 2, 000 to produce a stable result. Riemann is slightly more convenient here.

0.655

0.670

50

100 r3[1:200, 1]

150

200

0.60

0.70

2000

4000 r3[, 1]

6000

8000

10000

0.00

0.10

2000

4000

6000

8000

10000

Figure 6: Trace plots for transformed J 6. Approximating a volume Lastly, we compare how well each method approximates the unit sphere volume. The implementations are straightforward. For the Riemann method, we generate a two-dimensional grid, and for each grid coordinate, we compute the z-coordinate of the unit sphere. A 3D plot of the z-coordinates will show a sphere surface as in Figure 4a. For the Monte Carlo method, we generate a bivariate uniform distribution constrained to a quarter-circle. The resulting cloud looks like a quarter-pipe where the density decreases along the pipe length. As we might expect, the Monte Carlo estimate turns out to be a mean of cube volumes. The trace plots look remarkably similar. In general, it is known that the Monte Carlo method performs better in high dimensional integration. The reason is that the Riemann method systematically over- or underestimates the region of integration, a problem that becomes worse in higher dimensions (Seuss 2010). Interestingly, in this case, the smoothness of the unit sphere favors Riemann and the three-dimensional aspect favors Monte Carlo. The two opposing forces seem to balance each other. 7. Summary Some useful generalizations can be made about the comparative advantages of these two methods. For a smooth function such as f (x) = xd , the Riemann method tends to converge sooner. For a very spiky function such as h(x) = | sin(5000x)|, the Monte Carlo method converges sooner and is robust to extreme function values, unlike Riemann. We have seen how to capitalize on the advantages of each method in approximating an integral with no known analytic solution and multiple formulations. Finally, we analyzed the convergence behaviors for smooth three-dimensional function, and discussed the advantage of the Monte Carlo method in a high dimension setting.

Figure 7: Riemann integration for the unit sphere

Figure 8: Monte Carlo integration for the unit sphere

3.9

4.2

4.5

2000

4000 r4[1:10000, 1]

6000

8000

10000

2 3 4 5 6

2000

4000 r4[, 1]

6000

8000

10000

0.0

0.2

0.4

2000

4000

6000

8000

10000

Figure 9: Trace plots for the unit sphere volume References Seuss, E. A.; Trumbo, B. E.: Introduction to Probability Simulation and Gibbs Sampling with R. Springer, New York (2010).