Error Estimates

Associated Reading
Computational Methods of Neutron Transport by E.E. Lewis and W.F. Miller.
Chapter 7

Expectation Value
x: random variable
x̄: true mean of x
xn : random sample
x̂: an unbiased estimator of the x̄

Z +∞
E[x] = xf (x)dx = x̄ (1)
−∞
E[xn ] = E[x] (2)
Z +∞
E[g(x)] = ḡ = g(x)f (x)dx (3)
−∞
E[g(xn )] = E[g(x)] (4)
if g(x) = c1 g1 (x) + c2 g2 (x) (5)
E[g(x)] = c1E[g1 (x)] + c2 E[g2 (x)] (6)
Z +∞ Z +∞
E[g(x, y)] = dx dyg(x, y)f (x, y)dx (7)
−∞ −∞
E[g(xn , yn′ )] = E[g(x, y)] (8)

" N
# N
1 X 1 X
E[x̂] = E xn = E[xn ] = E[x] = x̄ (9)
N n=1 N n=1

1
Variance

g(x) = (x − x̄)2 (10)

variance is defined as,

Z +∞
σ 2 (x) = E[(x − x̄)2 ] = dx(x − x̄)2 f (x) (11)
−∞
Z +∞ Z +∞ Z +∞
2 2 2
σ (x) = dxx f (x) − 2x̄ dxxf (x) + x̄ dxf (x) (12)
−∞ −∞ −∞
σ 2 (x) = (x¯2 ) − (x̄)2 (13)
Z +∞
¯ n
(x) = dxxn f (x) (14)
−∞
q
σ(x) = (x¯2 ) − (x̄)2 (15)

What is σ 2 (x̂)?

σ 2 (x̂) = E[(x̂ − x̄)2 ] (16)


N
!2 
1 X
σ 2 (x̂) = E  xn − x̄  (17)
N n=1

N
!2 
1 X
σ 2 (x̂) = E  (xn − x̄)  (18)
N n=1

N
!2 
N

1 X 1 X XX
(xn − x̄) = 2 (xn − x̄)2 + (xn − x̄)(xn′ − x̄) (19)
N n=1 N n=1 ′ n̸=n

(20)

N
2 1 X  1 XX
E (xn − x̄)2 + 2

σ (x̂) = 2 E [(xn − x̄)(xn′ − x̄)] (21)
N n=1 N ′ n̸=n

E[(xn − x̄)2 ] = E[x2n ] − 2x̄E[xn ] + x̄2 = σ 2 (x) (22)

2
 if xn and xn′ are independent, then, f (x, x′ ) = f (x)f (x′ ) and,

E[(xn − x̄)(xn′ − x̄) = E[xn xn′ ] − x̄E[xn ] − x̄E[xn′ ] + x̄2 (23)

Z +∞ 2
E[xn xn′ ] = dxxf (x) = x̄2 (24)
−∞

1 2
σ 2 (x̂) = σ (x) (25)
N
1
σ(x̂) = √ σ(x) (26)
N

Unbiased estimator of σ 2 (x)

Assert, E[s2 ] = σ 2 (x), and make a guess for s2

N
1 X
s2 = (xn − x̂)2 (27)
N − 1 n=1
"N #
1 X
E[s2 ] = E (xn − x̂)2 (28)
N −1 n=1

xn − x̂ = (xn − x̄) − (x̂ − x̄) (29)

N
X N
X
(xn − x̂)2 = (xn − x̄)2 − 2(xn − x̄)(x̂ − x̄) + (x̂ − x̄)2
 
(30)
n=1 n=1
N
X
−2(x̂ − x̄) (xn − x̄) = −2N (x̂ − x̄)2 (31)
n=1
N
X N
X
(xn − x̂)2 = (xn − x̄)2 − N (x̂ − x̄)2
 
(32)
n=1 n=1

N
!
2 1 X  2
  2

E[s ] = E (xn − x̄) − N E (x̂ − x̄) (33)
N −1 n=1
 
2 1 1
E[s ] = N σ (x) − N σ 2 (x)
2
(34)
N −1 N
E[s2 ] = σ 2 (x) (35)

3
 In Monte Carlo we don’t tally (xn − x̂)2 because we don’t know x̂ until the
end. Instead,

N N
1 X 1 X 2
(xn − x̂)2 = (x − 2xn x̂ + x̂2 ) (36)
N − 1 n=1 N − 1 n=1 n
" N N
#
N 1 X 2 1 X 2
= x − x̂ (37)
N − 1 N n=1 n N n=1
N h ˆ2 i
= (x ) − (x̂)2 (38)
N −1
N
1 X 2
(xˆ2 ) = x (39)
N n=1 n

Notice that

N
lim =1 (40)
N →∞ N − 1

4
Central Limit Theorem

(x̂ − x̄)2
 
1
lim fN (x̂) = p exp − (41)
N →∞ 2πσ 2 (x̂) 2σ 2 (x̂)

for large N we may substitute σ(x̂) ≈ √sN .

The CLT allows to calculate approximate confidence intervals for x̂ with
respect to x̄.
Notes:

1. Must have enough N

2. Some distributions don’t have a variance, therefore CLT does not apply

3. Must have i.i.d. variables. CLT variants exist for non identically dis-
tributed variables.

5
Error Estimates Example
• ν=2
• Σf = 0.1
• Σc = 0.1
• Σs = 0.1

The transport process leads to a Bernoulli random variable for the estimator
of k.
Sample ν̄ with probability p
Sample 0 with probability (1 − p)
For example,
k1 = 0
k2 = 2
k3 = 0
k4 = 0
k5 = 2
k6 = 2
k7 = 0

Analytic Calculation of True values (not accessible)

k̄ = ν̄p (42)
σk2n = E[(kn − k̄) ] 2
(43)
2 2
= p(ν̄ − ν̄p) + (1 − p)(0 − ν̄p) (44)
= ν̄ 2 p(1 − p)2 + ν̄ 2 (1 − p)p2 (45)
2 2 2 2
= ν̄ p(1 − 2p + p ) + ν̄ (1 − p)p (46)
σk2n = 2
ν̄ p(1 − p) (47)

6
Estimator of k̄ for N = 2
2
1X
k̂ = kn (48)
2 n=1

Sample k̂ = 2 with probability 25%

Sample k̂ = 1 with probability 50%
Sample k̂ = 0 with probability 25%
For example,
k̂1 = 2
k̂2 = 0
k̂3 = 1
k̂4 = 1

Binomial distribution (not accessible)

 
ν̄ N x
p(k̂) = p (1 − p)N −x (49)
N x

where, x is the number of fission events.

E[k̂] = ν̄p (50)

2
ν̄
σk̂2 = p(1 − p) (51)
N
However the distribution is still binomial and not normal. Therefore, 1σ is
not a 67% confidence interval at all.
You can estimate the mean and variance but without knowledge of the dis-
tribution you cannot construct a confidence interval.

k̂-Estimator of k̄ for N -large (e.g. N = 103 )

Still a (discrete) binomial distribution (i.e. max value is limited by ν̄N ) but
now it is well approximated by a normal distribution and confidence intervals
from a normal distributions are good approximations. This is true for (almost)
any underlying distribution.