Coursework - Probability for Statistics

CID: 02091191

November 23, 2021

1 First exercise
Let Ω be a nonempty set.

(a) Let C = {A, B, C} be a partition of Ω. The σ-algebra generated by the partition C noted
σ(C) is
σ(C) = {∅, {A, B, C}, {A}, {B}, {C}, {B, C}, {A, C}, {A, B}}
□

(b) Suppose that C is a countable partition of Ω. Let us prove that every element of σ(C) is a
countable union of elements of C.

Let us write C = {Ci , i ∈ N} a countable partition of Ω. Let Γ = i∈I Ci , I ⊆ N be the set

S

of all countable unions of elements of C. We first show that Γ is a σ-algebra:

• Ω ∈ Γ: as C is a partition of Ω, i Ci = Ω. Hence, Γ is not empty.

S

• Γ is closed under countable unions: since every element of Γ are sets of countable unions of
elements of C, Γ is closed under countable unions: let K be a countable set and (Ak )k∈K
be a sequence of elements
S of Γ. Then,
S forSall k ∈ K, there exists Jk countable such that
Ak = j∈Jk Cj . Hence, k∈K Ak = k∈K j∈Jk Cj which is a countable union of elements
S
of C.

• Γ is closed under complement: let AS ∈ Γ. Thus,

S there exists I ⊆ N countable such that
A = i∈I Ci . Then, A = Ω \ A = i∈N Ci \ i∈I Ci = i∈N, i∈I
c
/ Ci which is a countable
S S
union of elements of C and thus Ac ∈ Γ.

Hence, Γ is a σ-algebra. Also, as Γ is the set of all countable unions of elements of C, we have
C ⊆ Γ: thus, Γ is a σ-algebra containing C. Moreover, by definition, σ(C) is the intersection of
all σ-algebras containing C (i.e. the smallest σ-algebra containing C), thus σ(C) ⊆ Γ. Hence,
every element of σ(C) is a countable union of elements of C. □

Remark: As σ(C) is the σ-algebra generated by C, σ(C) contains all countable unions of elements
of C, therefore σ(C) contains all the elements of Γ. Hence, Γ ⊆ σ(C) and therefore, Γ = σ(C).

(c) Let f : Ω →
− R be a function. Let us show that f is measurable with respect to (Ω, σ(C)) if
and only if f is constant on each member of that partition.

f is measurable with respect to (Ω, σ(C)) if and only if, for all y ∈ R, f −1 ({y})S ∈ σ(C), i.e.
f −1 ({y}) is a countable union of elements of C as shown in (b), i.e. f −1 ({y}) = i∈I Ci where
I ⊆ N. Hence, f is constant on each member of the partition C. □

1
2 Second exercise
Let (Ω, F, P) be a probability space and X be a non-negative random variable on (Ω, F, P) with
distribution function F (·).

(a) First, let us show that for all p ∈ N

Z ∞
E[X p ] = psp−1 (1 − F (s))ds
0

By definition, for a non-negative random variable,

Z Z ∞
p p
E[X ] = (X(ω)) dP(ω) = xp dF (x)
Ω 0

However, notice that Z x

pup−1 du = xp
0
Thus, Z ∞Z x
p
E[X ] = pup−1 dudF (x)
0 0
Then, since the Lebesgue measure is a σ-finite measure on ([0, ∞), B[0,∞) ) and given that g :
(u, x) 7−→ pup−1 is non-negative and measurable on B[0,∞) × B[0,∞) , Tonelli’s theorem allows to
write Z ∞Z ∞
p
E[X ] = dF (x)pup−1 du
Z0 ∞ u
p
E[X ] = pup−1 (1 − F (u))du
0
□

(b) Let X1 , . . . , Xp be a set of independent identically distributed random variables with the
same distribution as X. Let us show that
Z ∞
E[min(X1 , . . . , Xp )] = (1 − F (s))p ds
0

This can be shown by several ways, but we will here use the result found in (a) to prove this
new result. Consider the distribution function of the random variable min(X1 , . . . , Xp ) defined
as, for all x in R
Fmin(X1 ,...,Xp ) (x) = P[min(X1 , . . . , Xp ) ≤ x]
= 1 − P[min(X1 , . . . , Xp ) > x]
p
Y
=1− P[Xi > x]
i=1
Yp
=1− (1 − F (x))
i=1
Fmin(X1 ,...,Xp ) (x) = 1 − (1 − F (x))p
Therefore, we can now use the result proved in (a) as min(X1 , . . . , Xp ) is a non-negative random
variable Z ∞
1
E[min(X1 , . . . , Xp ) ] = 1 · s1−1 (1 − Fmin(X1 ,...,Xp ) (s))ds
0

2
Thus, Z ∞
E[min(X1 , . . . , Xp )] = (1 − (1 − (1 − F (s)))p )ds
Z0 ∞
E[min(X1 , . . . , Xp )] = (1 − F (s))p ds
0
□

Remark: the
R∞same result could have been found using that, for all non negative random variable
X, E[X] = 0 P[X > x]dx. Applying this result to the random variable min(X1 , . . . , Xp ) would
give Z ∞
E[min(X1 , . . . , Xp )] = P[min(X1 , . . . , Xp ) > x]dx
Z0 ∞
= P[X1 > x, . . . , Xp > x]dx
0
Z ∞
= P[X1 > x] . . . P[Xp > x]dx
0
Z ∞
= (1 − P[X ≤ x])p dx
Z0 ∞
E[min(X1 , . . . , Xp )] = (1 − F (x))p dx
0

3 Third exercise
Let Xn for n ∈ N be sequence of random variables on (Ω, F, P) and let X be a random variable
on the same probability space.
Lr
(a) Suppose that Xn −→ X where r ≥ 1, i.e.

lim E [|Xn − X|r ] = 0

n→∞

Using Minkowski’s inequality,

(E|Xn + (X − Xn )|r )1/r ≤ (E|Xn |r )1/r + (E|X − Xn |r )1/r

(E|X|r )1/r − (E|Xn |r )1/r ≤ (E|X − Xn |r )1/r

In the same way,
(E|Xn |r )1/r − (E|X|r )1/r ≤ (E|X − Xn |r )1/r
And combining both gives

0 ≤ |(E|Xn |r )1/r − (E|X|r )1/r | ≤ (E|Xn − X|r )1/r

Thus,
lim (E|Xn |r )1/r − (E|X|r )1/r = 0
n→∞

Hence,
lim (E|Xn |r )1/r = (E|X|r )1/r
n→∞

Then, we can apply f : x 7→ xr with r ≥ 1 on both sides, and we get by continuity of f

lim (E|Xn |r ) = (E|X|r )

n→∞

3
 L1
(b) Suppose that Xn −→ X, i.e.
lim E [|Xn − X|] = 0
n→∞
Notice that, using Jensen’s inequality with φ : x 7→ |x| providing φ(E[Xn − X]) ≤ E[φ(Xn − X)]
as φ is convex on R, we have
E[|Xn − X|] ≥ |E[Xn − X]|
= |E[Xn ] − E[X]|
Hence, for all n,
0 ≤ |E[Xn ] − E[X]| ≤ E[|Xn − X|]
And as limn→∞ E [|Xn − X|] = 0,
lim E[Xn ] − E[X] = 0 i.e. lim E[Xn ] = lim E[X]
n→∞ n→∞ n→∞
□

L2
(c) Suppose that Xn −→ X. Consider
Var[Xn ] = E[Xn2 ] − E[Xn ]2
By (a) with r = 2, we show that on the one hand,
lim E[Xn2 ] = E[X 2 ]
n→∞

L2
On the other hand, as convergence in Lq implies convergence in Lp for 1 ≤ p ≤ q, Xn −→ X =⇒
L1
Xn −→ X and using (b) by continuity of x 7→ x2
lim E[Xn ] = E[X] =⇒ lim E[Xn ]2 = E[X]2
n→∞ n→∞
Hence,
lim Var[Xn ] = E[X 2 ] − E[X]2 = Var[X]
n→∞
□

L1
(d) Suppose limn→∞ E[Xn ] = E[X]. We do not have necessarily Xn −→ X. For issue, let (Xn )
be a sequence of random variables defined as
1 1
P[Xn = 0] = 1 − and ∀n ≥ 1, P[Xn = n] =
n n
And let X be defined as
P[X = 1] = 1
Then, for all n,
E[Xn ] = 0 · P[Xn = 0] + n · P[Xn = n]
 
1 1
=0· 1− +n·
n n
=1
Also, E[X] = 1. Hence, E[Xn ] →
− E[X]. However,
1 1
P[|Xn − X| = 1] = 1 − and P[|Xn − X| = n − 1] =
n n
Therefore,  
1 1
E[|Xn − X|] = 1 · 1 − + (n − 1) ·
n n
2
=2− → − 2 ̸= 0
n
Hence, Xn does not converge to X in L1 . □

4
4 Fourth exercise
Consider an aircraft engine being driven by a engine control unit and especially a total of N
replica processing units running in parallel. Given a probability space (Ω, F, P), we model
the time of failure of the ith unit as the random variable Xi on (Ω, F, P). We assume that
X1 , . . . , XN are independent and identically distributed. We define the system failure time
MN = max(X1 , . . . , XN ) i.e. the entire system fails when all the units have failed.

(a) Let us show that MN is a random variable on (Ω, F, P), i.e. show that for all t ∈ R, {Mn ≤ t}
sets are measurable. Let ω ∈ Ω,

ω ∈ {MN ≤ t} ⇐⇒ ω ∈ {max(X1 . . . , XN ) ≤ t}
⇐⇒ (X1 (ω) ≤ t) ∩ . . . ∩ (XN (ω) ≤ t)
⇐⇒ ω ∈ {X1 ≤ t} ∩ . . . ∩ {XN ≤ t}
N
\
⇐⇒ ω ∈ {Xi ≤ t}
i=1

where all {Xi ≤ t} are measurable sets as Xi are random variables on (Ω, F, P). Hence, {Mn ≤ t}
is measurable as finite intersection of measurable sets. Thus, MN is a random variable on
(Ω, F, P). □

(b) Suppose that X1 , . . . , XN are exponentially distributed random variables with rate λ. Let
FMN be the distribution function of MN . By definition, for all x in [0, ∞)

FMN (x) = P [MN ≤ x]

= P [max(X1 , . . . , XN ) ≤ x]
= P [X1 ≤ x, . . . , XN ≤ x]
N
Y
= P [Xi ≤ x] by independence
i=1
YN
= FXi (x)
i=1
YN  
= 1 − e−λx
i=1
 N
FMN (x) = 1 − e−λx

We then obtain the probability density function of MN noted fMN defined by, for all x in [0, ∞)

d
fMN (x) = FM (x)
dx N
 N −1
fMN (x) = N λe−λx 1 − e−λx

5
 1.0

λ=1
λ=2
0.8

1.5
λ=5
0.6

1.0
FMN

fMN
0.4

0.5
λ=1
0.2

λ=2
λ=5
0.0

0.0
0 2 4 6 8 10 0 2 4 6 8 10

x x

(a) Distribution functions, N = 10 (b) Density functions, N = 10

1.0

λ=1
λ=2
0.8

1.5
λ=5
0.6

1.0
FMN

fMN
0.4

0.5

λ=1
0.2

λ=2
λ=5
0.0

0.0

0 2 4 6 8 10 0 2 4 6 8 10

x x

(c) Distribution functions, N = 100 (d) Density functions, N = 100

Figure 4.1: Examples of distribution and density functions for some values of λ and N

Figure 4.1 shows the effect of parameters λ and N on the shape and location of the distribution
of MN . As for a simple exponential distribution, as λ is getting smaller, the distribution and
density functions show that high values of x are more likely to occur than for higher values of
λ. Also, as N increases, the curves move to the right which is consistent since it corresponds
to increase the number of experiments Xi , and thus the probability to have large values of MN
increases.

(c) Consider the distribution function of the sequence of random variables MN − 1

λ log(N ), for
all x in [0, ∞)

6
    
1 1
P MN − log(N ) ≤ x = P max(X1 , . . . , XN ) ≤ x + log(N )
λ λ
 
1 1
= P X1 ≤ x + log(N ), . . . , XN ≤ x + log(N )
λ λ
N  
Y 1
= P Xi ≤ x + log(N )
λ
i=1
N  
1
1 − e−λ(x+ λ log(N ))
Y
=
i=1
 N
= 1 − e−λx−log(N )
N
e−λx

−λx
= 1− −−−−→ e−e
N N →∞

Hence, MN − λ1 log(N ) converges in distribution to a random variable X on (Ω, F, P) with dis-

−λx
tribution F (x) = e−e for all x in [0, ∞). □

(d) Suppose we wish to study the system failure time of the engine under increasing redundancy.
To do so, we need to generate independent realisations of MN with N very large. Assume we
can generate independent identically distributed U [0, 1] random variables U1 , U2 , . . .. We can
then implement two different algorithms, one exact but inefficient, and another one approximate
but efficient.

Exact but inefficient algorithm: Assume we want to generate p realisations of MN =

max(X1 , . . . , XN ) where X1 , . . . , XN are independent identically exponentially distributed ran-
dom variables with parameter λ. The idea is to generate N exponentially distributed samples,
compute the maximum and repeat this procedure p times. We can generate independent expo-
nentially distributed observations from U [0, 1] observations by solving
1 1
y = 1 − e−λx ⇐⇒ x = − log(1 − y) or similarly x = − log y
λ λ
This algorithm is inefficient as it requires to generate p × N uniform observations, apply trans-
formations and find a maximum p times over N observations.

Algorithm 1 Exact but inefficient

1: Generate N samples from U [0, 1] distribution, noted U
2: Transform U components to Exp(λ) distribution X = − λ 1
log(U )
3: Generate MN from the X observations by MN = max(X1 , . . . , XN )
4: Repeat steps 1 to 3 p times

Not exact but efficient algorithm: Assume again we want to generate p realisations of MN =
max(X1 , . . . , XN ). The idea of this approximate algorithm is to consider that MN − λ1 log(N )
−λx
has distribution e−e on [0, ∞), which is only approximate. Then, we could write that MN
−λx
has (approximate) distribution e−N e , and we could then generate directly MN samples by
inverting the distribution:
 
1 −λx 1 log y
y = e−λ(x− λ log N ) = e−N e ⇐⇒ x = − log −
λ N
This method is called inversion method and consists in computing the quantile function (ana-
lytically or numerically). Hence, this method only requires to generate p uniform observations
and apply a transformation.

7
Algorithm 2 Approximate but efficient
1: Generate p samples from U [0, 1] distribution, noted U
2: Transform U components to MN approximate distribution using X = − λ1 log(− logNU )

(e) Now that we have written two different algorithms to generate an empirical distribution of p
observations of MN for some N , we may compare the quality of sampling for each algorithm. The
idea we have in the following is to proceed to Kolmogorov-Smirnov tests in R in order to compare

N
the samples to the real distribution, with exact distribution function FMN (x) = 1 − e−λx as
computed in (b). Doing such a test is more powerful than just comparing some statistic (like
only comparing the mean, the median or the mode for example). The formal hypothesis of the
KS tests are here: the null hypothesis H0 : samples from algorithm k are from MN distribution
against the alternative hypothesis H1 : samples from algorithm k are not from MN distribution.
We will do these tests for multiple values of N and for both algorithms k = 1, 2.

We need to be careful as we use samples from a uniform distribution U , which would thus lead
to different results for different samples. First, as comparing for only one situation would not
be very consistent, we will run the procedure 100 times so that we will obtain 100 different p-
values for each chosen N and therefore have a more precise idea of the accuracy of the sampling
algorithms in general and not only in one case. Second, we will fix the seed in R so that for
all values of N , we will use exactly the same uniform distributions. The full procedure may be
summarised as follows:
Algorithm 3 Multiple KS tests for a given N
1: Fix the seed
2: Initialise λ, N , p (the number of samples) and P s (an empty list to store all p-values)
3: for i in 1:100 do
4: U ∼ U [0, 1]
5: Compute Mn1 samples using the exact Algorithm 1
6: Compute Mn2 samples using the approximate Algorithm 2
7: Proceed to KS test for Mn1 and Mn2 comparing them separately, in two different tests, to
the known cdf FMN
8: Add to P s the p-values of each test
9: end for
10: Compute the mean p-values from each algorithm (mean of P s columns) ▷ P s is here a
matrix with dimensions (100,2)
11: Repeat for all chosen N

Using Algorithm 3, we could generate the following results (the whole following will use λ = 3
and p = 104 ).

Exact Approximate
N = 10 0.5305 3.821 · 10−6
N = 102 0.4727 0.4399
N = 103 0.5026 0.4949
N = 104 0.5069 0.4213

Table 1: Mean p-values from KS tests obtained from comparing both sampling algorithms
(exact and approximate) to real cdf FMN , for N = 10, 102 , 103 , 104

Table 1 shows that for each value of N , the exact sampling Algorithm 1 provides samples that
come from real MN distribution: in fact, as all p-values are greater than the threshold 0.05, we
fail to reject the null hypothesis. Therefore, we may think that Algorithm 1 has been correctly

8
designed and implemented. However and as expected, this is not the case for Algorithm 2: with
small values of N (especially here N = 10), this algorithm doesn’t provide good samples of MN
as the p-value is less than 0.05, and therefore we reject the null hypothesis that samples have the
same distribution as MN . However, as N increases, Algorithm 2 seems more and more efficient
as the obtained p-values for N = 102 , 103 , 104 are greater than 0.05: thus, we fail to reject the
null hypothesis. This is consistent since we have shown in (c) that the approximation we used
for FMN is true when N → − ∞, this is why Algorithm 2 seems better and better as N increases
(even if the mean p-value obtained for N = 104 is lower than for N = 103 , it is still high enough
to accept H0 ).

Exact algorithm Approximate algorithm Exact algorithm Approximate algorithm

1.0

1.0

1.0

1.0
0.8

0.8

0.8

0.8
0.6

0.6

0.6

0.6
0.4

0.4

0.4

0.4
0.2

0.2

0.2

0.2
0.0

0.0

0.0

0.0
(a) p-values, N = 10 (b) p-values, N = 102

Exact algorithm Approximate algorithm Exact algorithm Approximate algorithm

1.0

1.0

1.0

1.0
0.8

0.8

0.8

0.8
0.6

0.6

0.6

0.6
0.4

0.4

0.4

0.4
0.2

0.2

0.2

0.2
0.0

0.0

0.0

0.0

(c) p-values, N = 103 (d) p-values, N = 104

Figure 4.2: Box plots of obtained p-values for each iteration (from 1 to 100), for
N = 10, 102 , 103 , 104

Figure 4.2 shows the distribution of obtained p-values as box plots for all chosen N . As N
increases, the accuracy of generating process used in Algorithm 2 seems to increase, whereas the
accuracy of Algorithm 1 is always good enough (in general) to consider this algorithm as exact.
We may notice that even if Algorithm 2 is only approximate, it still converges with acceptable
p-values at around N = 102 .

9
 1.0

1.2
1.0
0.8

0.8
0.6

Density
Fn(x)

0.6
Exact samples
0.4

Approx. samples

0.4
Exact samples
Approx. samples Exact cdf
0.2

Exact cdf

0.2
0.0

0.0
0.0 0.5 1.0 1.5 2.0 2.5 3.0 0 1 2 3 4

x x

(a) Distribution functions, N = 5 (b) Density functions, N = 5

1.0

1.0
0.8

0.8
0.6

0.6
Density
Fn(x)

Exact samples
Approx. samples
0.4

0.4

Exact samples Exact cdf

Approx. samples
0.2

0.2

Exact cdf
0.0

0.0

2 3 4 5 6 2 3 4 5 6

x x

(c) Distribution functions, N = 104 (d) Density functions, N = 104

Figure 4.3: Densities and distributions for N = 5 and N = 104 obtained empirically from
Algorithms 1 and 2, compared to the real ones

Furthermore, Figure 4.3 shows that whatever value of N , Algorithm 1 should generate samples
that are coming from MN distribution as the difference between the real density and cumulative
distribution functions and its empirical density is almost zero (red and black curves are similar).
However, this is not true for Algorithm 2 whose distribution and density for N = 5 are not
exactly the same as the theoretical ones. Moreover, as N increases (here N = 104 ), we can state
that Algorithm 2 generates samples from almost same distribution as MN .

To study more in deep the convergence of Algorithm 2 samples to the real distribution, we may
focus on "small" values of N and look at the mean p-values obtained from distributions with
repeated sampling.

10
 1.0
Exact algorithm
Approx. algorithm

0.8
0.05 threshold

Mean p−values

0.6
0.4
0.2
0.0

10 20 30 40 50

Figure 4.4: Evolution of mean p-values of KS tests on samples distribution (H0 : the samples
have the same distribution as MN ) versus N for both algorithms

Figure 4.4 represents the mean p-values obtained from KS tests after sampling from both algo-
rithms multiple times (50 times). It can then be inferred that, given a 0.05 p-value threshold, we
may consider that for N > 25, Algorithm 2 generates accurate samples from MN distribution
as the mean p-values obtained are greater than the threshold. This plot also shows clearly that
as N increases, the quality of sampling from Algorithm 2 is getting better and better, whereas
the quality of sampling from Algorithm 1 remains constant and "perfect". □

References
[1] Dr Andrew. Duncan (October 2021) MATH70082 - Probability for Statistics 2021-2022
Lecture Notes, Imperial College London MSc Statistics resources

11
Appendices
FMn <- function(x, lambda, N) (1-exp(-lambda*x))^N
fMn <- function(x, lambda, N) N*lambda*exp(-lambda*x)*(1-exp(-lambda*x))^(N-1)

x <- seq(0, 10, by=0.01)

plot(x, FMn(x, lambda=1, N=100), type="l", col="orange", lwd=2, xlab=expression(x),

ylab=expression(F[M[N]]))
lines(x, FMn(x, lambda=2, N=100), col="purple", lwd=2)
lines(x, FMn(x, lambda=5, N=100), col="lightblue", lwd=2)
legend(7, 0.3, c(expression(lambda == 1), expression(lambda == 2),
expression(lambda == 5)), col=c("orange", "purple", "lightblue"),
lty=1:1)

plot(x, fMn(x, lambda=1, N=100), type="l", col="orange", lwd=2, xlab=expression(x),

ylab=expression(f[M[N]]), ylim=c(0, 1.9))
lines(x, fMn(x, lambda=2, N=100), col="purple", lwd=2)
lines(x, fMn(x, lambda=5, N=100), col="lightblue", lwd=2)
legend(7, 1.8, c(expression(lambda == 1), expression(lambda == 2),
expression(lambda == 5)), col=c("orange", "purple", "lightblue"),
lty=1:1)

FMn <- function(x, lambda, N) (1-exp(-lambda*x))^N

fMn <- function(x, lambda, N) N*lambda*exp(-lambda*x)*(1-exp(-lambda*x))^(N-1)

x <- seq(0, 10, by=0.01)

plot(x, FMn(x, lambda=1, N=10), type="l", col="orange", lwd=2, xlab=expression(x),

ylab=expression(F[M[N]]))
lines(x, FMn(x, lambda=2, N=10), col="purple", lwd=2)
lines(x, FMn(x, lambda=5, N=10), col="lightblue", lwd=2)
legend(7, 0.3, c(expression(lambda == 1), expression(lambda == 2),
expression(lambda == 5)), col=c("orange", "purple", "lightblue"),
lty=1:1)

plot(x, fMn(x, lambda=1, N=10), type="l", col="orange", lwd=2, xlab=expression(x),

ylab=expression(f[M[N]]), ylim=c(0, 1.9))
lines(x, fMn(x, lambda=2, N=10), col="purple", lwd=2)
lines(x, fMn(x, lambda=5, N=10), col="lightblue", lwd=2)
legend(7, 1.8, c(expression(lambda == 1), expression(lambda == 2),
expression(lambda == 5)), col=c("orange", "purple", "lightblue"),
lty=1:1)

library(pbapply)

set.seed(4567)
N <- 20
p <- 1e4
lambda <- 3
F <- function(x) (1-exp(-lambda*x))^N
f <- function(x) N*lambda*exp(-lambda*x)*(1-exp(-lambda*x))^(N-1)

P <- c()
for(N in 5:50){
F <- function(x) (1-exp(-lambda*x))^N
f <- function(x) N*lambda*exp(-lambda*x)*(1-exp(-lambda*x))^(N-1)

12
 Ps <- c()
for(i in 1:50){
U <- runif(p)

# Exact but inefficient

allMn.exact <- pbreplicate(p, max(-1/lambda * log(runif(N))))

# Approximate but efficient

f <- function(u) -1/lambda * log(-log(u)/N)
allMn.approx <- f(U)

Ps <- rbind(Ps, c(ks.test(allMn.exact, "F")$p.value,

ks.test(allMn.approx, "F")$p.value))
}
P <- rbind(P, c(mean(Ps[,1]), mean(Ps[,2])))
}
plot(5:50, P[,1], col="red", ylim=c(0,1), pch=16, xlab="N", ylab="Mean p-values")
points(5:50, P[,2], col="blue", pch=16)
lines(5:50, rep(0.05, 46), lwd=2)
legend(30, 1, c("Exact algorithm", "Approx. algorithm", "0.05 threshold"),
col=c("red", "blue", "black"), lty=1:1)

Ps <- c()
for(i in 1:50){
U <- runif(p)

# Exact but inefficient

allMn.exact <- pbreplicate(p, max(-1/lambda * log(runif(N))))

# Approximate but efficient

f <- function(u) -1/lambda * log(-log(u)/N)
allMn.approx <- f(U)

Ps <- rbind(Ps, c(ks.test(allMn.exact, "F")$p.value,

ks.test(allMn.approx, "F")$p.value))
}

mean(Ps[,1])
mean(Ps[,2])

par(mfrow=c(1,2))
boxplot(Ps[,1], ylim=c(0,1), main="Exact algorithm", col="steelblue1")
boxplot(Ps[,2], ylim=c(0,1), main="Approximate algorithm", col="tan1")

allMn.exact <- pbreplicate(p, max(-1/lambda * log(runif(N))))

f <- function(u) -1/lambda * log(-log(u)/N)
allMn.approx <- f(U)

par(mfrow=c(1,1))
plot(ecdf(allMn.exact), col="red", main="", xlim=c(2,6))
plot(ecdf(allMn.approx), add=TRUE, col="blue")
curve(F, 0, 4, col="black", add=TRUE)
legend(4, 0.4, c("Exact samples", "Approx. samples", "Exact cdf"),
col=c("red", "blue", "black"), lty=1:1)

par(mfrow=c(1,1))
plot(density(allMn.exact), col="red", main="", xlab="x")

13
lines(density(allMn.approx), col="blue")
curve(f, 0, 4, col="black", add=TRUE)
legend(2.5, 0.6, c("Exact samples", "Approx. samples", "Exact cdf"),
col=c("red", "blue", "black"), lty=1:1)

14