Scribd
Upload
2 views2 pages

Sheet 4

This document is a problem set for the course 'Foundations in Data Analysis' at the Technical University of Munich, focusing on two exercises. Exercise 4.1 explores the convergence of the power method and the spectral norm related to singular value decomposition, while Exercise 4.2 discusses the convergence of the alternating projection method in data analysis. The document outlines specific mathematical tasks and proofs required to demonstrate the convergence properties of these methods.

Uploaded by

Hollo
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
2 views2 pages

Sheet 4

This document is a problem set for the course 'Foundations in Data Analysis' at the Technical University of Munich, focusing on two exercises. Exercise 4.1 explores the convergence of the power method and the spectral norm related to singular value decomposition, while Exercise 4.2 discusses the convergence of the alternating projection method in data analysis. The document outlines specific mathematical tasks and proofs required to demonstrate the convergence properties of these methods.

Uploaded by

Hollo
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

School of Computation, Information and Technology

Technical University of Munich

Prof. Felix Krahmer Summer Term 2025


Dr. Alessandro Scagliotti

Foundations in Data Analysis [MA4800]

Problem Set 4

Exercise 4.1 (The Power Method):

As discussed in the lecture, the speed of convergence of the power method depends on the gap
between the leading singular value, σ1 , and the next singular value that differs from σ1 . In this
exercise, we will quantify this process of convergence.
r
X
Given some matrix A ∈ Rm×n and its singular value decomposition A = σi ui viT , consider
i=1
r
X
matrix B = AT A with its singular (eigen-) value decomposition B = σi2 vi viT . Let E be the
i=1
eigenspace related to the leading eigenvalue of matrix B with dimension d = dim(E), d ≥ 1.

Bℓ PE
a) Show that Pr 2ℓ
→ as ℓ → ∞, where PE is the projection onto eigenspace E.
i=1 σi d

Bℓ PE
b) Quantify the convergence process above in the spectral norm, i.e. Pr 2ℓ
− , in
i=1 σi d
terms of the so-called spectral gap
σi
δ := max .
i,σi ̸=σ1 σ1

In other words, find an upper bound for the spectral norm in terms of δ and ℓ.

1
Exercise 4.2 (Alternating Projection):
The methods based on the procedure of alternating projection are often used in data analysis.
In this task, we will prove that under certain conditions the algorithm of alternating projection
converges. To this end, we consider the problem setup and follow the steps, which are indicated
below.

Let C and D be nonempty closed convex sets in Rd . Denote by PC and PD the orthogonal
projection onto C and D, respectively. Let x0 ∈ C be arbitrary. Define sequences {xk } ⊂ C and
{yk } ⊂ D by
yk := PD (xk ) and xk+1 := PC (yk ), k ∈ N0 .

a) Show that for any x ∈ Rd and any c ∈ C,

∥PC (x) − x∥22 ≤ ∥x − c∥22 − ∥PC (x) − c∥22 .

(Hint: the inequality that characterizes the projections may be useful here.)

b) Suppose C ∩ D ̸= ∅. Let x ∈ C ∩ D. Show that

∥yk − x∥22 ≤ ∥xk − x∥22 − ∥yk − xk ∥22 , k ∈ N0 ,

and
∥xk+1 − x∥22 ≤ ∥yk − x∥22 − ∥xk+1 − yk ∥22 , k ∈ N0 .

c) Using the results from a) and b), show that

∥xk − x∥2 ≤ ∥x0 − x∥2 and ∥yk − x∥2 ≤ ∥x0 − x∥2 , k ∈ N.

d) Show that the sequence {xk } is bounded and has an accumulation point x∗ ∈ C exploiting
the results in (c).

e) Exploit again the results from (c) to conclude that the two sequences {∥yk − xk ∥2 } and
{∥xk+1 − yk ∥2 } converge to zero.

f) Show that x∗ ∈ D and thus x∗ ∈ C ∩ D.

g) Establish that ∥xk − x∗ ∥2 → 0 and ∥yk − x∗ ∥2 → 0 as k → ∞.

This sheet will be discussed in the week of May 26, 2025.

You might also like