FC_intervals_bounded_gaussian

July 4, 2024

1 Feldman-Cousins Confidence Interval Tutorial

Anil Kumar | [email protected] | 3 July, 2024
Ref.: A Unified Approach to the Classical Statistical Analysis of Small Signals, Gary J. Feldman,
Robert D. Cousins, PRD 57 (1998) 3873-3889, arXiv:physics/9711021
Suppose that we wish to make an inference about a parameter 𝜇 whose true value 𝜇𝑡 is unknown.
Assume that we do this by making a single measurement of an observable 𝑥 such that the probability
density function (pdf) for obtaining the value 𝑥 depends on the unknown parameter 𝜇 in a known
way: we call this pdf 𝑃 (𝑥|𝜇). Now suppose that the single measurement of 𝑥 yields the value 𝑥0 .
One substitutes this value of 𝑥 into 𝑃 (𝑥|𝜇) to obtain 𝑃 (𝑥0 |𝜇), known as the likelihood function,
which we denote as ℒ(𝑥0 |𝜇).
In the frequentist approach, we obtain a confidence interval [𝜇1 , 𝜇2 ], which is a member of a set,
such that the set has the property that

𝑃 (𝜇 ∈ [𝜇1 , 𝜇2 ] = 𝛼. (1)

Here 𝜇1 and 𝜇2 are functions of the measured 𝑥, and this equation refers to the varying confidence
intervals [𝜇1 , 𝜇2 ] from an ensemble of experiments with fixed 𝜇. For a set of confidence intervals, this
equation is true for every allowed 𝜇. Thus, in particular, the intervals contain the fixed unknown
𝜇𝑡 in a fraction 𝛼 of experiments.

1.0.1 Example: Gaussian with Boundary at Origin

Consider the observable 𝑥 which is simply the measured value of 𝜇 in an experiment with a Gaussian
resolution function with known fixed rms deviation 𝜎 = 1

1
𝑃 (𝑥|𝜇) = √ exp (−(𝑥 − 𝜇)2 /2) (2)
2𝜋

We consider an interesting case where only non-negative values for 𝜇 are physically allowed. For
example, if 𝜇 is a mass. Therefore, the graph does not exist for 𝜇 < 0.
In this tutorial, we will study the following type of confidence intervals: - Central confidence interval
- Upper confidence limit - Confidence interval based on Flip-flopping policy - Feldmann-Cousins
confidence interval

1
[1]: import math
import numpy as np
import pandas as pd
from scipy.stats import norm
from matplotlib import pyplot as plt
import matplotlib
matplotlib.rcParams.update({'font.size': 18})

2 Gaussian distribution with unit 𝜎

Here, we plot the Gaussian pdf 𝑃 (𝑥|𝜇) as a function of 𝑥 for a given value of 𝜇 with known fixed
rms deviation 𝜎 = 1

1
𝑃 (𝑥|𝜇) = √ exp (−(𝑥 − 𝜇)2 /2) . (3)
2𝜋
We select an interval [𝑥1 , 𝑥2 ] such that

𝑃 (𝑥 ∈ [𝑥1 , 𝑥2 ]|𝜇) = 𝛼. (4)

Here, we choose 𝛼 to be 90% confidence level (CL). We refer to the interval [𝑥1 , 𝑥2 ] as the “accep-
tance region” or the “acceptance inteval” for this 𝜇. In order to specify uniquely the acceptance
region, one must choose auxiliary criteria. One has total freedom to make this choice, if the choice
is not influenced by the data 𝑥0 .

2.1 Central confidence intervals

The choice for “central confidence intervals” is

𝑃 (𝑥 < 𝑥1 |𝜇) = 𝑃 (𝑥 > 𝑥2 |𝜇) = (1 − 𝛼)/2, (5)

which satisfy

𝑃 (𝜇 < 𝜇1 ) = 𝑃 (𝜇 > 𝜇2 ) = (1 − 𝛼)/2. (6)

[2]: mean = 2 # mean of Gaussian pdf

std = 1 # sigma of Gaussian pdf
alpha = 0.9 # 90% CL

#print(f' mean = {mean}\n sigma = {std}')

xmin, xmax = mean-3*std, mean+3*std # we plot x over 3 sigma range

x = np.linspace(xmin, xmax, 100)
y = norm.pdf(x, mean, std) # Gaussian pdf for x with given mean␣
↪and sigma

2
lower_CL = 0.5*(1 - alpha) # lower limit for central confidence␣
↪intervals

upper_CL = alpha + 0.5*(1 - alpha) # upper limit for central confidence␣

↪intervals

x1 = norm.ppf(lower_CL, mean, std) # lower limit of x for central␣

↪confidence intervals

x2 = norm.ppf(upper_CL, mean, std) # upper limit of x for central␣

↪confidence intervals

print(f' x1 (90% CL) = {x1:6.2f} \n x2 (90% CL) = {x2:6.2f}')

fig, ax = plt.subplots(figsize=(6, 6))

plt.plot(x, y) # plot Gaussian pdf
plt.fill_between( # shade acceptance region for 90%␣
↪central CL

x= x,
y1= y,
where= ((x > x1) & (x < x2)), # shade region between x1 and x2
color= "k",
alpha= 0.2,
label = "90% CL")

plt.text(x1, 0.02, r'$x_1$', horizontalalignment='right')

plt.text(x2, 0.02, r'$x_2$', horizontalalignment='left')
plt.xlabel('Measured Mean x')
plt.ylabel(r'Probability density function (pdf)')
plt.xlim([xmin, xmax])
plt.ylim(bottom=0)
plt.legend()
plt.show()

x1 (90% CL) = 0.36

x2 (90% CL) = 3.64

3
 2.2 Central confidence intervals
Now, we plot the parameter 𝜇 vs. the measured quantity 𝑥. For each value of 𝜇, one examines
𝑃 (𝑥|𝜇) along the horizontal line through 𝜇. We select an interval [𝑥1 , 𝑥2 ] which corresponds to the
confidence level of 𝛼. The construction is complete when horizontal acceptance intervals (gray) are
drawn for each value of 𝜇.
Upon performing an experiment to measure 𝑥 and obtaining the value 𝑥0 , one draws a vertical line
through 𝑥0 on the horizontal axis. The confidence interval is the union of all values of 𝜇 for which
the corresponding horizontal interval is intercepted by the vertical line; typically this is a simply
connected interval [𝜇1 , 𝜇2 ].

[3]: # Fig. 3

std = 1
alpha = 0.9

4
lower_CL = 0.5*(1 - alpha) # lower limit for central confidence␣
↪intervals

upper_CL = alpha + 0.5*(1 - alpha) # upper limit for central confidence␣

↪intervals

ymin, ymax = 0, 6 # only positive values of mean

mean = np.linspace(ymin, ymax, 100)
x1 = norm.ppf(lower_CL, mean, std) # lower limit of x for central␣
↪confidence intervals

x2 = norm.ppf(upper_CL, mean, std) # upper limit of x for central␣

↪confidence intervals

fig, ax = plt.subplots(figsize=(6, 6))

plt.plot(x1, mean, color = 'k')
plt.plot(x2, mean, color = 'k')
plt.fill_between( # shade acceptance region for 90%␣
↪central CL between x1 and x2

x= x1,
y1= mean,
color= "k",
alpha= 0.2,
label = "90% CL")

plt.fill_between( # shade acceptance region for 90%␣

↪central CL between x1 and x2

x= x2,
y1= mean,
color= "w",
alpha= 1,
)

plt.text(-1, 1, r'$x_1$', horizontalalignment='right')

plt.text(3, 1, r'$x_2$', horizontalalignment='left')

plt.xlabel('Measured Mean x')

plt.ylabel(r'Mean $\mu$')
plt.xlim([-2, 4])
plt.ylim([ymin, ymax])
plt.legend()
plt.show()

5
 2.3 Upper confidence limit
For the “upper confidence limit”, the choice is

𝑃 (𝑥 < 𝑥1 |𝜇) = 1 − 𝛼 (7)

which satisfy

𝑃 (𝜇 > 𝜇2 ) = 1 − 𝛼. (8)

[4]: mean = 2
std = 1
alpha = 0.9

6
#print(f' mean = {mean}\n sigma = {std}')

xmin, xmax = mean-3*std, mean+3*std

x = np.linspace(xmin, xmax, 100)
y = norm.pdf(x, mean, std)

lower_CL = (1 - alpha) # lower limit for central␣

↪confidence intervals

x1 = norm.ppf(lower_CL, mean, std) # lower limit of x for central␣

↪confidence intervals

print(f' x1 (90% CL) = {x1:6.2f}')

fig, ax = plt.subplots(figsize=(6, 6))

plt.plot(x, y)
plt.fill_between(
x= x,
y1= y,
where= (x > x1), # shade acceptance region for␣
↪90% central CL for x > x1

color= "k",
alpha= 0.2,
label = "90% CL")

plt.xlabel('Measured Mean x')

plt.ylabel(r'Probability density function (pdf)')
plt.text(x1, 0.02, r'$x_1$', horizontalalignment='right')
plt.xlim([xmin, xmax])
plt.ylim(bottom=0)
plt.legend()
plt.show()

x1 (90% CL) = 0.72

7
 2.4 Upper confidence limit
For each value of 𝜇, one examines 𝑃 (𝑥|𝜇) along the horizontal line through 𝜇 and select an interval
𝑥1 which corresponds to the confidence level of 𝛼. Here, 𝑥2 can be thought of equal to infinity.
The vertical line through the measured value 𝑥0 provide upper confidence limit 𝜇2 .

[5]: # Fig. 2

std = 1
alpha = 0.9

lower_CL = (1 - alpha) # lower limit for␣

↪central confidence intervals

8
ymin, ymax = 0, 6 # only positive values␣
↪of mean

mean = np.linspace(ymin, ymax, 100)

x1 = norm.ppf(lower_CL, mean, std) # lower limit of x for␣
↪central confidence intervals

fig, ax = plt.subplots(figsize=(6, 6))

plt.plot(x1, mean, color = 'k')
plt.fill_between( # shade acceptance␣
↪region for 90% central CL for x > x1

x= x1,
y1= mean,
color= "k",
alpha= 0.2,
label = "90% CL")

plt.text(-0.5, 1, r'$x_1$', horizontalalignment='right')

plt.xlabel('Measured Mean x')
plt.ylabel(r'Mean $\mu$')
plt.xlim([-2, 4])
plt.ylim([ymin, ymax])
plt.legend()
plt.show()

9
 2.5 Confidence interval based on flip-flopping
Let us suppose, for example, that Physicist X takes the following attitude in an experiment designed
to measure a small quantity: “If the result 𝑥 is less then 3𝜎, I will state an upper limit from the
standard tables. If the result is greater than 3𝜎, I will state a central confidence interval from the
standard tables.” We call this policy “flip-flopping” based on the data. Furthermore, Physicist X
may say, “If my measured value of a physically positive quantity is negative, I will pretend that I
measured zero when quoting a confidence interval”, which introduces some conservatism.
[6]: # Fig. 4

std = 1
alpha = 0.9

10
ymin, ymax = 0, 6
mean = np.linspace(ymin, ymax, 100)

lower_UL = (1 - alpha) # lower limit for upper␣

↪confidence intervals

x1_UL= norm.ppf(lower_UL, mean, std) # lower limit for x for␣

↪upper confidence intervals

lower_central_CL = 0.5*(1 - alpha) # lower limit for central␣

↪confidence intervals

upper_central_CL = alpha + 0.5*(1 - alpha) # upper limit for central␣

↪confidence intervals

x1_central = norm.ppf(lower_central_CL, mean, std) # lower limit of x for␣

↪central confidence intervals

x2_central = norm.ppf(upper_central_CL, mean, std) # upper limit of x for␣

↪central confidence intervals

## selecting interval based on flip-flopping

x1_FF = [] # x based on flip-flopping

mean1_FF = [] # mean1 based on␣
↪flip-flopping

mean2_FF = [] # mean1 based on␣

↪flip-flopping

for x in np.arange(-2, 4.01, 0.01):

if x <= 0: # For negative␣
↪measurements, pretending it to be zero

y1 = np.interp(0, x1_UL, mean)

y2 = 0
x1_FF.append(x)
mean1_FF.append(y1)
mean2_FF.append(y2)

if (x >= 0) & (x <= 3): # For small signals,␣

↪selecting upper confidence intervals
y1 = np.interp(x, x1_UL, mean)
y2 = 0
x1_FF.append(x)
mean1_FF.append(y1)
mean2_FF.append(y2)

if (x >= 3): # For large enough␣

↪signals, selecting central confidence intervals
x1_FF.append(x)
y1 = np.interp(x, x1_central, mean)
y2 = np.interp(x, x2_central, mean)

11
 mean1_FF.append(y1)
mean2_FF.append(y2)

fig, ax = plt.subplots(figsize=(6, 6))

plt.plot(x1_FF, mean1_FF, color='k') # upper bound for mean,␣
↪based on flip-flopping

plt.plot(x1_FF, mean2_FF, color='k') # lower bound for mean,␣

↪based on flip-flopping

plt.fill_between( # Shading 90% confidence␣

↪interval based on flip-flopping

x= x1_FF,
y1= mean1_FF,
y2= mean2_FF,
color= "k",
alpha= 0.2,
label = "90% CL")

plt.text(-0.5, 1.5, r'$x_1$', horizontalalignment='right')

plt.text(3.5, 1.5, r'$x_2$', horizontalalignment='left')

plt.xlabel('Measured Mean x')

plt.ylabel(r'Mean $\mu$')
plt.xlim([-2, 4])
plt.ylim([0, 6])
plt.legend()
plt.show()

12
2.6 Feldmann-Cousins confidence interval
For a particular 𝑥, we let 𝜇best be the physically allowed value of 𝜇 for which 𝑃 (𝑥|𝜇) is maximum.
For Gaussian distribution 𝜇best = 𝑚𝑎𝑥(0, 𝑥), and

√
1/ 2𝜋, 𝑥>0
𝑃 (𝑥|𝜇best ) = { √ (9)
exp (−𝑥2 /2)/ 2𝜋, 𝑥<0

We then compute 𝑅

𝑃 (𝑥|𝜇)
𝑅(𝑥) = (10)
𝑃 (𝑥|𝜇best )

During our Neymann construction of confidence intervals, the values of 𝑥 are added to the accep-
tance region at a particular value of 𝜇 in the decreasing order of 𝑅. In practice, this means that

13
 for a given value of 𝜇, one finds the interval [𝑥1 , 𝑥2 ] such that 𝑅(𝑥1 ) = 𝑅(𝑥2 ) and

𝑥2
∫ 𝑃 (𝑥|𝜇)𝑑𝑥 = 𝛼 (11)
𝑥1

We solve for 𝑥1 and 𝑥2 numerically to the desired precision, for each 𝜇 in a grid. With the acceptance
regions all constructed, we then read off the confidence intervals [𝜇1 , 𝜇2 ] for each 𝑥0 .

[7]: # defining a fuction to calculate FC confidence intervals [x_1, x_2] for a␣

↪given mean

def FC_range(mean, std = 1, alpha = 0.9):

dx = 0.01 # step size for x

x_list = []
R_list = []
pdf_list = []
for x in np.arange(mean - 5*std, mean + 5*std, dx):
pdf = norm.pdf(x, mean, std) # probability␣
↪distribution function at x for a given mean and sigma

mean_best = max(0,x) # For Gaussian, the␣

↪maximum pdf occur at this

pdf_best = norm.pdf(x, mean_best, std) # maximum probability␣

↪distribution function at x_best

R = pdf/pdf_best # Feldmann-Cousins␣
↪ordering ratio

x_list.append(x)
R_list.append(R)
pdf_list.append(pdf)

df = pd.DataFrame({'x': x_list, 'P': pdf_list, 'R': R_list} )

df_sorted = df.sort_values(by='R', ascending=False) # sorting the R in␣
↪decreasing order

x1, x2 = None, None

# finding x_1 and x_2 for 90% CL where x with highest R are accepted first

Prob_sum = 0
for index, row in df_sorted.iterrows(): # Loop index progresses␣
↪in decreasing order of R

x = row["x"]
P = row["P"]
R = row["R"]
Prob_sum += P*dx # integration over pdf
if x <= mean :
x1 = x # lower CL

14
 else:
x2 = x # upper CL
if Prob_sum >= alpha: # stop when total␣
↪probability is equal to or

print(f'{mean:6.3f} {x1:6.3f} {x2:6.3f}') # greater than desired␣

↪CL

break

return x1, x2

[8]: # calculating FC confidence intervals [x_1, x_2] for a list of means

mean_list = []
FC_x_range = []

print(f'mean \t x1 \t x2')
for mean in np.arange(0, 6, 0.1):
x_range = FC_range(mean)
mean_list.append(mean)
FC_x_range.append(list(x_range))

mean x1 x2
0.000 -4.930 1.280
0.100 -4.900 1.380
0.200 -3.990 1.480
0.300 -2.620 1.590
0.400 -2.010 1.730
0.500 -1.650 1.880
0.600 -1.400 2.020
0.700 -1.190 2.170
0.800 -1.020 2.310
0.900 -0.860 2.440
1.000 -0.730 2.570
1.100 -0.600 2.690
1.200 -0.480 2.810
1.300 -0.370 2.920
1.400 -0.250 3.030
1.500 -0.140 3.140
1.600 -0.040 3.240
1.700 0.060 3.340
1.800 0.160 3.440
1.900 0.260 3.540
2.000 0.360 3.640
2.100 0.460 3.740
2.200 0.560 3.840
2.300 0.660 3.940
2.400 0.760 4.040

15
 2.500 0.860 4.140
2.600 0.960 4.240
2.700 1.060 4.340
2.800 1.160 4.440
2.900 1.260 4.540
3.000 1.360 4.640
3.100 1.460 4.740
3.200 1.560 4.840
3.300 1.660 4.940
3.400 1.760 5.040
3.500 1.860 5.140
3.600 1.960 5.240
3.700 2.060 5.340
3.800 2.160 5.440
3.900 2.260 5.540
4.000 2.360 5.640
4.100 2.460 5.740
4.200 2.560 5.840
4.300 2.660 5.940
4.400 2.760 6.040
4.500 2.860 6.140
4.600 2.960 6.240
4.700 3.060 6.340
4.800 3.160 6.440
4.900 3.260 6.540
5.000 3.360 6.640
5.100 3.460 6.740
5.200 3.560 6.840
5.300 3.660 6.940
5.400 3.760 7.040
5.500 3.860 7.140
5.600 3.960 7.240
5.700 4.060 7.340
5.800 4.160 7.440
5.900 4.260 7.540

[9]: # plotting FC confidence intervals

fig, ax = plt.subplots(figsize=(6, 6))

plt.plot(np.array(FC_x_range)[:, 0], mean_list, color ='k') # plotting␣
↪x1 for FC confidence interval

plt.plot(np.array(FC_x_range)[:, 1], mean_list, color ='k') # plotting␣

↪x2 for FC confidence interval

plt.fill_between( # shading␣
↪90% confidence interval

x= np.array(FC_x_range)[:, 0],

16
 y1= mean_list,
color= "k",
alpha= 0.2,
label = "90% CL")

plt.fill_between( # shading␣
↪90% confidence interval

x= np.array(FC_x_range)[:, 1],
y1= mean_list,
color= "w",
alpha= 1,
)

plt.text(-0.5, 1.5, r'$x_1$', horizontalalignment='right')

plt.text(3.5, 1.5, r'$x_2$', horizontalalignment='left')

plt.xlabel('Measured Mean x')

plt.ylabel(r'Mean $\mu$')
plt.xlim([-2, 4])
plt.ylim([0, 6])
plt.legend()
plt.show()

17
[10]: # calculating FC confidence intervals [\mu_1, \mu_2] for a list of measured x_0␣
↪(same as in Table X)

x0_list = np.arange(-3, 3.1, 0.1)

mu1_list = np.interp(x0_list, np.array(FC_x_range)[:, 1], mean_list)
mu2_list = np.interp(x0_list, np.array(FC_x_range)[:, 0], mean_list)

print('x0 \t mu1 \t mu2')

for (x0, mu1, mu2) in zip(x0_list, mu1_list, mu2_list):
print(f'{x0: 5.2f} {mu1: 5.2f} {mu2: 5.2f}')

x0 mu1 mu2
-3.00 0.00 0.27
-2.90 0.00 0.28
-2.80 0.00 0.29

18
-2.70 0.00 0.29
-2.60 0.00 0.30
-2.50 0.00 0.32
-2.40 0.00 0.34
-2.30 0.00 0.35
-2.20 0.00 0.37
-2.10 0.00 0.39
-2.00 0.00 0.40
-1.90 0.00 0.43
-1.80 0.00 0.46
-1.70 0.00 0.49
-1.60 0.00 0.52
-1.50 0.00 0.56
-1.40 0.00 0.60
-1.30 0.00 0.65
-1.20 0.00 0.70
-1.10 0.00 0.75
-1.00 0.00 0.81
-0.90 0.00 0.88
-0.80 0.00 0.95
-0.70 0.00 1.02
-0.60 0.00 1.10
-0.50 0.00 1.18
-0.40 0.00 1.27
-0.30 0.00 1.36
-0.20 0.00 1.45
-0.10 0.00 1.54
0.00 0.00 1.64
0.10 0.00 1.74
0.20 0.00 1.84
0.30 0.00 1.94
0.40 0.00 2.04
0.50 0.00 2.14
0.60 0.00 2.24
0.70 0.00 2.34
0.80 0.00 2.44
0.90 0.00 2.54
1.00 0.00 2.64
1.10 0.00 2.74
1.20 0.00 2.84
1.30 0.02 2.94
1.40 0.12 3.04
1.50 0.22 3.14
1.60 0.31 3.24
1.70 0.38 3.34
1.80 0.45 3.44
1.90 0.51 3.54
2.00 0.59 3.64

19
 2.10 0.65 3.74
2.20 0.72 3.84
2.30 0.79 3.94
2.40 0.87 4.04
2.50 0.95 4.14
2.60 1.03 4.24
2.70 1.11 4.34
2.80 1.19 4.44
2.90 1.28 4.54
3.00 1.37 4.64

2.7 Feldmann-Cousins intervals using Toy Monte Carlo method

2.7.1 Example: Gaussian with Boundary at Origin
Ref.: Feldman-Cousins Confidence Levels - Toy MC Method, Till Moritz Karbach, arXiv:1109.0714
The method is based on toy Monte Carlo (MC) experiments, which are datasets drawn from the
assumed pdf. Here, we shall use it to compute the confidence belt for the trivial example of a unit
Gaussian bound to positive values, 𝜇 > 0.
Feldmann-Cousins method uses ratio of likelihood:

𝑃 (𝑥|𝜇)
𝑅(𝑥, 𝜇) = (12)
𝑃 (𝑥|𝜇best )

In pratice, it is convenient to take the logarithm of the likelihood ratio

−2 ln 𝑅 ≡ Δ𝜒2 = 𝜒2 − 𝜒2best (13)

2 2
= 𝜒 (𝑥, 𝜇) − 𝜒 (𝑥, 𝜇best ), (14)

because one often prefers to minimize −2 ln 𝑅 rather than to maximize 𝑃 . The algorithm to
contruct FC intervals is as following:
• At the considered value of the true parameter 𝜇0 , generate a toy experiment by drawing a
value of 𝑥toy from the pdf. That is, draw from the unit Gaussian 𝐺(𝜎 = 1, 𝜇 = 𝜇0 ).
• Compute Δ𝜒2toy for the toy experiment. For 𝜒2 (𝑥, 𝜇), 𝑥 is set to 𝑥toy and 𝜇 is set to 𝜇0 . For
𝜒2 (𝑥, 𝜇best ), 𝑥 = 𝑥toy and 𝜇best = 𝑚𝑎𝑥(0, 𝑥toy ) implementing the boundary at zero.
• Find the value Δ𝜒2𝑐 , such that 𝛼 of the toy experiment have Δ𝜒2toy < Δ𝜒2𝑐 .
• The interval [𝑥1 , 𝑥2 ] is given by all values of 𝑥 such that Δ𝜒2 (𝑥, 𝜇0 ) < Δ𝜒2𝑐 .

[11]: def get_FC_x_range(mean):

std = 1 # sigma of Gaussian pdf
alpha = 0.9 # 90% CL
total_events = 10000 # Total number of pseudo␣
↪experiments

x_toy = norm.rvs(mean, std , size = total_events) # Randomly generated␣

↪events following Gaussian distribution

20
 P = norm.pdf(x_toy, mean,std) # P(x|u) = P(x_toy | u)
chi2 = -2*np.log(P) # converting P(x|u) to␣
↪chi^2(x,u)

mean_best = np.maximum(np.zeros(len(x_toy)), x_toy)# calculating the best␣

↪mean [max(0,x_toy)] for which P(x|u) is maximum
P_best = norm.pdf(x_toy, mean_best, std) # maximum pdf =␣
↪P(x_toy|u_best)

chi2_best = -2*np.log(P_best) # converting␣

↪P(x_toy|u_best) to chi^2(x_toy,u_best)

delta_chi2 = chi2 - chi2_best # Delta_chisq =␣

↪chi^2(x_toy,u) - chi^2(x_toy,u_best)

df = pd.DataFrame({'x_toy': x_toy, 'delta_chi2': delta_chi2} )

df_sorted_delta_chi2 = df.sort_values(by='delta_chi2') # sorting the␣
↪delta_chi2 in increasing order

# finding x_1 and x_2 for 90% CL where x with highest R are accepted first

count = 0
for index, row in df_sorted_delta_chi2.iterrows(): # Loop index progresses␣
↪in increasing order of delta_chi2

x = row["x_toy"]
d = row["delta_chi2"]
count += 1 # integration over pdf
if x <= mean :
x1 = x # lower CL
else:
x2 = x # upper CL
if count/total_events >= alpha: # stop when total␣
↪probability is equal to or

print(f'u = {mean:6.3f}, x1 = {x1:6.3f}, x2 = {x2:6.3f},␣

↪delta_chi2_c = {d:6.3f}') # greater than desired CL
break

return x1, x2, d, df_sorted_delta_chi2

2.7.2 Toy experiment with Gaussian pdf

• Distribution of 𝑥toy for Gaussian pdf 𝐺(𝜎 = 1, 𝜇 = 𝜇0 ) with 𝜇0 = 0.5.
• Solid blue curve showing likelihood ratio in terms of Δ𝜒2 as a function 𝑥toy
• Dashed blue curve showing Δ𝜒2𝑐 at 90% CL
• The intersection of solid and dashed curve gives 𝑥1 and 𝑥2 .

21
[12]: mean = 0.5
x1, x2, delta_chi2_c, df_toy = get_FC_x_range(mean) # Toy experiment␣
↪based on Gaussian pdf with mean = 0.5

fig, ax1 = plt.subplots(figsize=(6, 6))

x_ = ax1.hist(df_toy["x_toy"], bins = 20, histtype='step', color = 'k') #␣

↪Histogram of x_toy

ax1.set_ylabel("Number")
ax1.set_xlabel("Measured Mean x")

df_sorted_x_toy = df_toy.sort_values(by='x_toy') # Sorting with␣

↪increasing x_toy for plotting smooth curve

ax2 = ax1.twinx()
color = 'b'
ax2.plot(df_sorted_x_toy["x_toy"], df_sorted_x_toy["delta_chi2"] , color =␣
↪color) # plot delta_chi2 as a function of x_toy

ax2.axhline(delta_chi2_c, color = color, linestyle='--') # plot delta_chi2 at␣

↪90 %CK

ax2.text(3.5, 2.3, r'$\Delta\chi^2_c$', color=color)

ax2.text(1.1*x1, delta_chi2_c, r'$x_1$', horizontalalignment='right',␣

↪verticalalignment="top", color = color)

ax2.text(x2, delta_chi2_c, r'$x_2$', horizontalalignment='left',␣

↪verticalalignment="top", color = color)

ax2.set_ylabel(r'$\Delta\chi^2$', color=color)
ax2.tick_params(axis='y', labelcolor=color)
ax2.set_ylim(bottom=0)

u = 0.500, x1 = -1.637, x2 = 1.874, delta_chi2_c = 1.887

[12]: (0.0, 15.802095940000012)

22
[13]: mean_list = np.arange(0.1, 6, 0.1)
FC_x_range = []
for u in mean_list:
x1, x2, _, _ = get_FC_x_range(u)
FC_x_range.append([x1, x2])

u = 0.100, x1 = -3.415, x2 = 1.377, delta_chi2_c = 1.631

u = 0.200, x1 = -3.805, x2 = 1.486, delta_chi2_c = 1.655
u = 0.300, x1 = -2.772, x2 = 1.630, delta_chi2_c = 1.768
u = 0.400, x1 = -1.937, x2 = 1.708, delta_chi2_c = 1.711
u = 0.500, x1 = -1.577, x2 = 1.851, delta_chi2_c = 1.827
u = 0.600, x1 = -1.381, x2 = 2.020, delta_chi2_c = 2.018
u = 0.700, x1 = -1.197, x2 = 2.170, delta_chi2_c = 2.165
u = 0.800, x1 = -1.018, x2 = 2.306, delta_chi2_c = 2.269
u = 0.900, x1 = -0.848, x2 = 2.428, delta_chi2_c = 2.336
u = 1.000, x1 = -0.739, x2 = 2.574, delta_chi2_c = 2.478
u = 1.100, x1 = -0.615, x2 = 2.700, delta_chi2_c = 2.562
u = 1.200, x1 = -0.472, x2 = 2.804, delta_chi2_c = 2.574
u = 1.300, x1 = -0.368, x2 = 2.926, delta_chi2_c = 2.646

23
u = 1.400, x1 = -0.232, x2 = 3.016, delta_chi2_c = 2.610
u = 1.500, x1 = -0.150, x2 = 3.143, delta_chi2_c = 2.700
u = 1.600, x1 = -0.030, x2 = 3.228, delta_chi2_c = 2.655
u = 1.700, x1 = 0.063, x2 = 3.338, delta_chi2_c = 2.683
u = 1.800, x1 = 0.158, x2 = 3.442, delta_chi2_c = 2.697
u = 1.900, x1 = 0.264, x2 = 3.539, delta_chi2_c = 2.685
u = 2.000, x1 = 0.370, x2 = 3.631, delta_chi2_c = 2.659
u = 2.100, x1 = 0.470, x2 = 3.730, delta_chi2_c = 2.656
u = 2.200, x1 = 0.559, x2 = 3.845, delta_chi2_c = 2.706
u = 2.300, x1 = 0.676, x2 = 3.928, delta_chi2_c = 2.651
u = 2.400, x1 = 0.788, x2 = 4.012, delta_chi2_c = 2.600
u = 2.500, x1 = 0.824, x2 = 4.173, delta_chi2_c = 2.809
u = 2.600, x1 = 0.945, x2 = 4.254, delta_chi2_c = 2.741
u = 2.700, x1 = 1.034, x2 = 4.364, delta_chi2_c = 2.774
u = 2.800, x1 = 1.161, x2 = 4.438, delta_chi2_c = 2.687
u = 2.900, x1 = 1.285, x2 = 4.515, delta_chi2_c = 2.609
u = 3.000, x1 = 1.366, x2 = 4.636, delta_chi2_c = 2.677
u = 3.100, x1 = 1.457, x2 = 4.743, delta_chi2_c = 2.700
u = 3.200, x1 = 1.561, x2 = 4.841, delta_chi2_c = 2.691
u = 3.300, x1 = 1.682, x2 = 4.917, delta_chi2_c = 2.619
u = 3.400, x1 = 1.744, x2 = 5.056, delta_chi2_c = 2.743
u = 3.500, x1 = 1.833, x2 = 5.166, delta_chi2_c = 2.777
u = 3.600, x1 = 1.940, x2 = 5.259, delta_chi2_c = 2.755
u = 3.700, x1 = 2.069, x2 = 5.330, delta_chi2_c = 2.660
u = 3.800, x1 = 2.166, x2 = 5.433, delta_chi2_c = 2.670
u = 3.900, x1 = 2.262, x2 = 5.539, delta_chi2_c = 2.688
u = 4.000, x1 = 2.357, x2 = 5.643, delta_chi2_c = 2.699
u = 4.100, x1 = 2.475, x2 = 5.729, delta_chi2_c = 2.653
u = 4.200, x1 = 2.564, x2 = 5.835, delta_chi2_c = 2.677
u = 4.300, x1 = 2.680, x2 = 5.920, delta_chi2_c = 2.626
u = 4.400, x1 = 2.739, x2 = 6.062, delta_chi2_c = 2.762
u = 4.500, x1 = 2.858, x2 = 6.142, delta_chi2_c = 2.696
u = 4.600, x1 = 2.977, x2 = 6.224, delta_chi2_c = 2.637
u = 4.700, x1 = 3.048, x2 = 6.352, delta_chi2_c = 2.730
u = 4.800, x1 = 3.160, x2 = 6.440, delta_chi2_c = 2.689
u = 4.900, x1 = 3.257, x2 = 6.544, delta_chi2_c = 2.701
u = 5.000, x1 = 3.350, x2 = 6.650, delta_chi2_c = 2.724
u = 5.100, x1 = 3.456, x2 = 6.744, delta_chi2_c = 2.703
u = 5.200, x1 = 3.581, x2 = 6.821, delta_chi2_c = 2.629
u = 5.300, x1 = 3.651, x2 = 6.950, delta_chi2_c = 2.723
u = 5.400, x1 = 3.758, x2 = 7.041, delta_chi2_c = 2.695
u = 5.500, x1 = 3.854, x2 = 7.143, delta_chi2_c = 2.710
u = 5.600, x1 = 3.945, x2 = 7.256, delta_chi2_c = 2.742
u = 5.700, x1 = 4.052, x2 = 7.348, delta_chi2_c = 2.717
u = 5.800, x1 = 4.194, x2 = 7.406, delta_chi2_c = 2.581
u = 5.900, x1 = 4.248, x2 = 7.553, delta_chi2_c = 2.731

24
[14]: fig, ax = plt.subplots(figsize=(6, 6))
plt.plot(np.array(FC_x_range)[:, 0], mean_list, color = 'k') # plotting␣
↪x1 for FC confidence interval

plt.plot(np.array(FC_x_range)[:, 1], mean_list, color = 'k') # plotting␣

↪x2 for FC confidence interval

plt.fill_between( # shading␣
↪90% confidence interval

x= np.array(FC_x_range)[:, 0],
y1= mean_list,
color= "k",
alpha= 0.2,
label = "90% CL")

plt.fill_between( # shading␣
↪90% confidence interval

x= np.array(FC_x_range)[:, 1],
y1= mean_list,
color= "w",
alpha= 1,
)

plt.text(-0.5, 1.5, r'$x_1$', horizontalalignment='right')

plt.text(3.5, 1.5, r'$x_2$', horizontalalignment='left')

plt.xlabel('Measured Mean x')

plt.ylabel(r'Mean $\mu$')
plt.xlim([-2, 4])
plt.ylim([0, 6])
plt.legend()
plt.show()

25
[ ]:

26