Scribd
Upload
6 views

Convex-projection-Note5

The document discusses the projection of points onto closed convex sets, establishing the existence and uniqueness of the projection. It introduces key concepts such as the projection property, subdifferential calculus, and hyperplane separation theorems, which are essential in convex analysis and optimization. Theorems are provided to demonstrate conditions for separating hyperplanes between convex sets.

Uploaded by

roger.chemoul86
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
6 views

Convex-projection-Note5

The document discusses the projection of points onto closed convex sets, establishing the existence and uniqueness of the projection. It introduces key concepts such as the projection property, subdifferential calculus, and hyperplane separation theorems, which are essential in convex analysis and optimization. Theorems are provided to demonstrate conditions for separating hyperplanes between convex sets.

Uploaded by

roger.chemoul86
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

1.

5 Projection to Convex Sets


Given a set C ⊆ Rn , the distance of a point to C is defined by

d(x; C) := inf{||x − y|| | y ∈ C}

For closed convex sets, an important consequence is the following projection


property.
Proposition:(Projection Property) Let C be a nonempty, closed convex
subset of Rn . For each x ∈ Rn , there exists an unique w ∈ C such that

||x − w|| = d(x; C)

w is called the projection of x to C, and is denoted by PC (x).


Proof. By definition of d(x; C), there exists wk ∈ C such that
1
d(x; C) ≤ ||x − wk || < d(x; C) +
k
It follows that {wk } is a bounded sequence. Hence it has a converging subse-
quence {wkl } which converges to a point w. Since C is closed, w ∈ C.
Considering the limit of
1
d(x; C) ≤ ||x − wkl || < d(x; C) +
kl
Hence d(x; C) = ||x − w||.
Now suppose w1 6= w2 ∈ C satisfy

||x − w1 || = ||x − w2 || = d(x; C)

Then we have,

w1 + w2 2 ||w1 − w2 ||2
2||x − w1 ||2 = ||x − w1 ||2 + ||x − w2 ||2 = 2||x − || +
2 2
w1 +w2
Since C is convex, 2 ∈ C. This gives,

w1 + w2 2 ||w1 − w2 ||2
||x − || = ||x − w1 ||2 − < ||x − w1 ||2 = d(x; C)2
2 4
w1 +w2
But since C is convex, 2 ∈ C, this is a contradiction.
Proposition: Let C be a nonempty, closed convex set, then w = PC (x) if and
only if
hx − w, u − wi ≤ 0, ∀u ∈ C

1
Proof. Suppose w = PC (x).
Let u ∈ C, λ ∈ (0, 1). Since C is convex, λu + (1 − λ)w ∈ C. Then

||x−w||2 = d(x; C)2 ≤ ||x−w−λ(u−w)||2 = ||x−w||2 −2λhx−w, u−wi+λ2 ||u−w||2 .

That is
2hx − w, u − wi ≤ λ||u − w||2
Letting λ → 0+ , we have
hx − w, u − wi ≤ 0
Conversely, suppose
hx − w, u − wi ≤ 0, ∀u ∈ C
Then

||x − u||2 = ||x − w||2 + 2hx − w, w − ui + ||w − u||2


≥ ||x − w||2 − 2hx − w, u − wi ≥ ||x − w||2

Hence ||x − w|| ≤ ||x − u|| for all u ∈ C and w = PC (x).

Figure 1: Projection to a convex set

2
2 Subdifferential Calculus
2.1 Convex Separation
The separating theorems are of fundamental importance in convex analysis and
optimization. This section provides some of the useful results.

Definition:(Hyperplane Separation) Two sets C1 , C2 are said to be sep-


arated by a hyperplane if there exists a 6= 0 such that

sup ha, xi ≤ inf ha, yi


x∈C1 y∈C2

C1 , C2 are said to be strictly separated if there exists a 6= 0 such that

sup ha, xi < inf ha, yi


x∈C1 y∈C2

If x is a relative boundary point of C, a hyperplane that separates C and {x}


is called a supporting hyperplane at x.

We will focus on the separation of two convex sets. To proof the existence
of such separation, we start with two lemmas.
Lemma: Let C be an nonempty, closed convex set and x ∈ / C. Then there
exists nonzero a such that
sup ha, xi < ha, xi
x∈C

Proof. Let w = PC (x) (which exists by the projection property). Then

hx − w, xi ≤ hx − w, wi for all x ∈ C.

Let a = x − w 6= 0, then

ha, xi ≤ ha, wi = ha, xi − ||x − w||2 < ha, xi

Lemma: Let C be a nonempty, convex subset of Rn with x ∈ C\ri(C). Then


there exists {xk } such that xk → x while xk ∈
/ C for all k.
Proof. Since ri(C) is nonempty, pick x0 ∈ri(C).
x0
Let xk = k+1
k x− k .
Clearly, xk → x. It remains to show that xk ∈/ C. Suppose otherwise, then by
the Line Segment property,
1 k k+1 x0
x= x0 + ( x − ) ∈ ri(C)
k+1 k+1 k k

This is a contradiction. Hence xk ∈


/ C for all k.

3
Theorem:(Supporting Hyperplane Theorem) Let C be a nonempty, con-
vex set. Suppose x ∈rel ∂C = C\ri(C). Then there exists a 6= 0 such that

sup ha, xi ≤ ha, xi


x∈C

Proof. Since x ∈rel ∂C. Then there exists xk ∈


/ C with xk → x.
By lemma, there exists ak 6= 0 such that

sup hak , xi < hak , xk i


x∈C

By dividing kak k, we may assume {ak } is bounded.


Since {ak } is bounded, it has a converging subsequence.
Without loss of generality (considering the subsequence), we may assume that
ak → a 6= 0
Taking the limit, we have for all x ∈ C

ha, xi ≤ ha, xi

Theorem:(Separating Hyperplane Theorem) Let C1 , C2 be two convex


sets. Suppose C1 ∩ C2 = ∅. Then there exists a hyperplane that separates C1
and C2 .
Proof. Consider C := C1 − C2 . Since C1 ∩ C2 = ∅, 0 ∈
/ C.
There are two cases:
Case (1): 0 ∈ C.
By the supporting hyperplane theorem, there exists a 6= 0 such that

ha, xi ≤ ha, 0i = 0, for all x ∈ C

That is
ha, x1 i ≤ ha, x2 i
Case (2): 0 ∈
/C
The result follows directly from the previous lemma.

In order to get strict separation, we need more assumptions.

Theorem:(Strict Hyperplane Separation) Let C1 , C2 be nonempty, closed


convex sets with C1 ∩ C2 = ∅. Suppose at least one of the two sets is also
bounded. Then there exists a 6= 0 such that

sup ha, x1 i < inf ha, x2 i


x1 ∈C1 x2 ∈C2

4
Proof. Let C := C1 − C2 . Then C is a nonempty, closed convex set with 0 ∈
/ C.
Then there exists a 6= 0 such that

γ := sup ha, xi < 0


x∈C

Then for all x1 ∈ C1 , x2 ∈ C2 , we have ha, x1 i ≤ γ + ha, x2 i. Then

sup ha, x1 i ≤ γ + inf ha, x2 i < inf ha, x2 i


x1 ∈C1 x2 ∈C2 x2 ∈C2

You might also like