Scribd
Upload
162 views

Exercises - MIT Assignments

This document contains 4 problems from Problem Set 2 of the course 18.S997 High-dimensional Statistics at MIT in Spring 2015. Problem 1 asks to show that a matrix is positive definite given properties of a random variable. Problem 2 defines weak lq norms and asks several questions about them. Problem 3 asks to analyze the mean squared error of a modified BIC estimator for a linear model with subgaussian errors. Problem 4 is not included in the summary.

Uploaded by

Deep Ghose
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
162 views

Exercises - MIT Assignments

This document contains 4 problems from Problem Set 2 of the course 18.S997 High-dimensional Statistics at MIT in Spring 2015. Problem 1 asks to show that a matrix is positive definite given properties of a random variable. Problem 2 defines weak lq norms and asks several questions about them. Problem 3 asks to analyze the mean squared error of a modified BIC estimator for a linear model with subgaussian errors. Problem 4 is not included in the summary.

Uploaded by

Deep Ghose
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 6

18.

S997 Spring 2015: Problem Set 1

Problem 1.1
A random variable X has χ2n (chi-squared with n degrees of freedom) if it has
the same distribution as Z12 + . . . + Zn2 , where Z1 , . . . , Zn are iid N (0, 1).
(a) Let Z ∼ N (0, 1). Show that the moment generating function of Y = Z 2 −1
satisfies
−s

 sY   √ e if s < 1/2
φ(s) := E e =
 1 − 2s
∞ otherwise
(b) Show that for all 0 < s < 1/2,
 s2 
φ(s) ≤ exp .
1 − 2s
(c) Conclude that √
IP(Y > 2t + 2 t) ≤ e−t
√
[Hint: you can use the convexity inequality 1 + u ≤ 1 + u/2].
(d) Show that if X ∼ χ2n , then, with probability at least 1 − δ, it holds
p
X ≤ n + 2 n log(1/δ) + 2 log(1/δ) .

Problem 1.2
Let A = {Ai,j } 1≤i≤n be a random matrix such that its entries are iid sub-
1≤j ≤m
Gaussian random variables with variance proxy σ 2 .
(a) Show that the matrix A is sub-Gaussian. What is its variance proxy?
(b) Let kAk denote the operator norm of A defined by
|Ax|2
max .
x∈IRm |x|2
Show that there exits a constant C > 0 such that
√ √
IEkAk ≤ C( m + n) .

1
Problem 1.3
Let K be a compact subset of the unit sphere of IRp that admits an ε-net Nε
with respect to the Euclidean distance of IRp that satisfies |Nε | ≤ (C/ε)d for all
ε ∈ (0, 1). Here C ≥ 1 and d ≤ p are positive constants. Let X ∼ subGp (σ 2 ) be
a centered random vector.
Show that there exists positive constants c1 and c2 to be made explicit such
that for any δ ∈ (0, 1), it holds
p p
max θ> X ≤ c1 σ d log(2p/d) + c2 σ log(1/δ)
θ∈K

with probability at least 1 − δ. Comment on the result in light of Theorem 1.19.

Problem 1.4
Let X1 , . . . , Xn be n independent and random variables such that IE[Xi ] = µ
and var(Xi ) ≤ σ 2 . Fix δ ∈ (0, 1) and assume without loss of generality that n
can be factored into n = K · G where G = 8 log(1/δ) is a positive integers.
¯ g denote the average over the gth group of k variables.
For g = 1, . . . , G, let X
Formally
gk
1 X
X̄g = Xi .
k
i=(g−1)k+1

1. Show that for any g = 1, . . . , G,


 2σ  1
IP X̄g − µ > √ ≤ .
k 4

¯1, . . . , X
2. Let µ̂ be defined as the median of {X ¯ G }. Show that

 2σ   G
IP µ̂ − µ > √ ≤ IP B ≥ ,
k 2

where B ∼ Bin(G, 1/4).


3. Conclude that r
 2 log(1/δ ) 
IP µˆ − µ > 4σ ≤δ
n
4. Compare this result with Corollary 1.7 and Lemma 1.4. Can you conclude
that µ̂ − µ ∼ subG(σ̄ 2 /n) for some σ̄ 2 ? Conclude.

2
MIT OpenCourseWare
http://ocw.mit.edu

18.S997 High-dimensional Statistics


Spring 2015

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.S997 Spring 2015: Problem Set 2

Problem 2.1
Let X = (1, Z, . . . , Z d−1 )> ∈ IRd be a random vector where Z is a random
variable. Show that the matrix IE(XX > ) is positive definite if Z admits a
probability density with respect to the Lebesgue measure on IR.

Problem 2.2
For any q > 0, a vector θ ∈ IRd is said to be in a weak `q ball of radius R if the
decreasing rearrangement |θ[1] | ≥ |θ[2] | ≥ . . . satisfies

|θ[j] | ≤ Rj −1/q .

Moreover, we define the weak `q norm of θ by

|θ|w`q = max j 1/q |θ[j ] |


1≤j≤d

(a) Give examples of θ, θ0 ∈ IRd such that

|θ + θ0 |w`1 > |θ|w`1 + |θ0 |w`1

What do you conclude?


(b) Show that |θ|w`q ≤ |θ|q .
(c) Show that if limd→∞ |θ|w`q < ∞, then limd→∞ |θ|q0 < ∞ for all q 0 > q.

(d) Show that, for any q ∈ (0, 2) if limd→∞ |θ|w`q = C, there exists a con-
stant Cq > 0 that depends on q but not on d and such that under the
assumptions of Theorem 2.11, it holds
 σ 2 log 2d 1− q2
|θˆhrd − θ∗ |22 ≤ Cq
n
with probability .99.

1
Problem 2.3
Assume that the linear model (Equation 2.2) with ε ∼ subGn (σ 2 ) and θ∗ =
6 0.
Show that the modified BIC estimator θˆ defined by
n1  ed o
θ̂ ∈ argmin |Y − Xθ|22 + λ|θ|0 log
θ ∈IRd n |θ|0

satisfies
ed

∗ 2
log |θ ∗ |0
MSE(Xθ̂) . |θ |0 σ .
n
with probability .99, for appropriately chosen λ. What do you conclude?

2
MIT OpenCourseWare
http://ocw.mit.edu

18.S997 High-dimensional Statistics


Spring 2015

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

You might also like