Weak Convergence

Omar Khalil, Ruikun Luo, Anthony Della Pella, Xianan Huang

December 06, 2016

Definition 1. A sequence {xn } in a normed linear vector space X is said to converge weakly to
x ∈ X if for every x∗ ∈ X ∗ we have x∗ (xn ) → x∗ (x). In this case we write xn → x weakly.
Definition 2. (Strong Convergence) In a normed linear space an infinite sequence of vectors {xn }
is said to converge to a vector x if the sequence {kx − xn k} of real numbers converges to zero. In
this case, xn → x.
Proposition 1. If xn → x strongly, then xn → x weakly.

Proof. Since X ∗ is the space of bounded linear functionals with |x∗ (x)| ≤ kx∗ kkxk, then

|x∗ (xn ) − x∗ (x)| ≤ kx∗ kkxn − xk → 0.

Example 1. In X = `2 consider the element xn = {0, 0, . . . , 0, 1, 0, . . .} with 1 in the n−th position.

Let x∗ = {η1 , η2 , . . .} ∈ `2 = X ∗ . Since the `p space is finite, i.e.
∞
X
|ηn |p < ∞
n=1

with p = 2, then the norm of an element x∗ ∈ X ∗

v
u∞
uX
∗
kx k = t |ηn |2 < ∞.
n=1

This implies that ηn → 0 as n → ∞. We have x∗ (xn ) = hx∗ , xn i = ηn → 0 as n → ∞ and xn → 0

weakly.
However, xn 6→ 0 strongly.

Proof. We prove by contradiction. Let xn → x with x = 0. By the definition of strong convergence,

kxn − 0k → 0 as n → ∞. This implies that kxn k → 0 as n → ∞, which is a contradiction since
kxn k = 1.

Definition 3. A sequence {x∗n } in X ∗ is said to converge weak-star (or weak*) to the element x∗
if for every x ∈ X, x∗n (x) → x∗ (x). In this case, we write x∗n → x∗ weak*.
We give an example to demonstrate that weak* convergence does not imply weak convergence
in X ∗ .

1
Example 2. Let X = c0 be the space of infinite sequences x = {ζi }∞ i=1 of real numbers convergent
to zero, with norm on c0 being kxk = maxi |ζi |. Then X = `1 , X = `∞ , where X ∗∗ is the dual of
∗ ∗∗

X ∗ . In X ∗ = `1 , let x∗n = {0, 0, 0, . . . , 0, 1, 0, 0, . . .} with 1 in the n−th position. Then x∗n (x) picks
the nth sequence ζn and
x∗n (x) = ζn → 0 as n → ∞
since ζn ∈ c0 , the space of infinite sequences convergent to zero. Thus, x∗n (x) → 0 as n → ∞ and
x∗n → 0 weak*.
However, for x∗∗ = {1, 1, 1, . . .}, x∗∗ (x∗n ) = 1 6= 0. Hence, x∗∗ (x∗n ) 6→ 0 as n → ∞ and x∗n 6→ 0
weakly.
Definition 4. A set K ⊂ X ∗ is said to be weak* compact if every infinite sequence from K contains
a weak* convergent subsequence.
Remind 1. A set D is said to be dense in a normed space X if for each element x ∈ X and each
 > 0 there exists d ∈ D with ||x − d|| < .
Remind 2. A normed space is separable if it contains a countable dense set.
Theorem 1. (Alaoglu) Let X be a real normed linear vector space. The closed unit ball in X ∗ is
weak* compact.
Proof: This theorem is to say every finite sequence {x∗n } in the closed unit ball (which means
that ||x∗n || ≤ 1) contains a weak* convergent subsequence. Thus we need to find such subsequence.
Here we try to prove this theorem in the case that X is separable(although X ∗ need not be sepa-
rable.).

Let {x∗n } be an infinite sequence in X ∗ such that ||x∗n || ≤ 1. Let {xk } be a sequence in X which
is the dense set in the separable space X.

The sequence {x∗n (x1 )} of real numbers is bounded (∵ |x∗n (x1 )| ≤ ||x∗n ||||x1 || ≤ ||x1 ||) and thus
contains a convergent subsequence which we denote as {x∗n1 (x1 )}(Bolzano-Weierstrass). Similarly,
{x∗n1 (x2 )} is also bounded and contains a convergent subsequence {x∗n2 (x2 )}. Continuing in this
fashion to extract convergent subsequencs {x∗nk (xk )}, we then form the diagonal sequence {x∗nn }
in X ∗ which is a subsequence of {x∗n }. Note that as the way we build {x∗nn }, we have {x∗nn (xk )}
converges. Thus the sequence {x∗nn } converges on the dense subset {xk } of X.

Next we need to prove that {x∗nn } converges weak* to an element x∗ ∈ X ∗ .

Fix x ∈ X and  > 0. Then for any n, m, k, we have

|x∗nn (x) − x∗mm (x)| ≤ |x∗nn (x) − x∗nn (xk )| + |x∗nn (xk ) − x∗mm (xk )| + |x∗mm (xk ) − x∗mm (x)|

For |x∗nn (x) − x∗nn (xk )| = ||x∗nn (x − xk )| ≤ ||x∗nn ||||x − xk || ≤ ||x − xk ||. Similarly, we have
|x∗mm (xk ) − x∗mm (x)| ≤ ||x − xk ||. As {xk } is in the dense subset of X. Thus choose k so that
||xk − x|| < 3 . As {x∗nn (xk )} converges, {x∗nn (xk )} is Cauchy. Thus there exists N such that for all
n, m > N , we have |x∗nn (xk ) − x∗mm (xk )| < 3 . Thus we have |x∗nn (x) − x∗mm (x)| < . Then we know
that {x∗nn (x)} is Cauchy and converges to a real number.

2
 Now we define a functional f such that f (x) = limn x∗nn (x). We need to prove that this func-
tional f is in X∗ which means that it is linear and bounded.

Let x1 , x2 ∈ X and α1 , α2 ∈ R. We have: f (α1 x1 + α2 x2 ) = limn x∗nn (α1 x1 + α2 x2 ) =

limn x∗nn (α1 x1 ) + x∗nn (α2 x2 ) = limn x∗nn (α1 x1 ) + limn x∗nn (α2 x2 ) = f (α1 x1 ) + f (α2 x2 ). Thus f
is linear.
To prove that f is bounded, we have

||f || = sup |f||x||

(x)|
x6=0
∗
= sup | lim||x||
xnn (x)|
x6=0
∗
= sup lim |x nn (x)|
||x||
x6=0
∗
≤ sup lim ||x||x||
nn ||||x||

x6=0
||x||
≤ sup lim||x||
x6=0
= 1

Thus f is linear and bounded. Denote f as x∗ . We have x∗nn (x) converges to x∗ (x) for all x.
Thus {x∗nn } converges weak* and the closed unit ball in X ∗ is weak* compact.

Definition 5. A functional (possibly nonlinear) defined on a normed space X is said to be weakly

continuous at x0 if given  > 0 there is a δ > 0 and a finite collection {x∗1 , . . . , x∗n } from X ∗ such
that |f (x) − f (x0 )| <  for all x such that |x∗i (x − x0 )| < δ for i = 1, 2, . . . , n. Weak∗ continuity of
a functional defined on X ∗ is defined analagously with the roles of X and X ∗ interchanged.

Proposition 2. If a functional f : X ∗ → R is weak∗ continuous, then x∗i → x in the weak∗ sense

implies that f (x∗i ) → f (x).

Theorem 2 (Heine-Borel). For a subset K of real numbers, the following are equivalent

• K is compact.

• K is closed and bounded.

Theorem 3. Let f be a weak∗ continuous real-valued functional on a weak∗ compact subset S of

X ∗ . Then f is bounded on S and achieves its maximum on S.

Proof. The theorem is proven true if we can show that the image of S under f is compact in R
for any weak∗ continuous f . Let f be as such and suppose {f (x∗i )} is a sequence in f (S) (i.e.
{x∗i } ⊆ S). Since S is weak∗ compact, we can find a weak∗ convergent subsequence {x∗ij } of {x∗i },
such that x∗ij → x in weak∗ . But, by assumption f is weak∗ continous, and so by the proposition
above, f (x∗ij ) → f (x). But, by weak∗ compactness of S, the limit x is also a member of S and so
{f (x∗i )} has a convergent subsequence {f (x∗ij } as desired. By the Heine Borel theorem above, f (S)
is a closed and bounded set of real numbers, and thus attains (by f (S) closed) a maximum (by
f (S) bounded).

3
 Noting again that the notion of compactness in certain spaces is very restrictive, Theorem 3
provides us with hope when trying to optimize certain functionals. For example, it is known that
in infinite dimensional Banach spaces, the unit ball is never compact. This could be troubling for
instance in the following example.
Rb
Example 3. Consider L2 [a, b] = {g | g : [a, b] → R and a |g|2 < ∞}. Then, we can set up the
following maximization problem:
minimize hx, x∗ i
x∗ ∈L2 [a,b]∗
(1)
subject to ||x∗ ||L2 ≤ 1.
where h·, ·i denotes the usual inner product in L2 [a, b]. Recall that L2 is a Hilbert space, and so
∗
L2 [a, b] = L2 [a, b]. Moreover, note that hx, x∗ i is a weak∗ continuous functional on X ∗ (as strong
∗
continuity implies weak∗ continuity). By Alaoglu’s theorem the unit ball S = {x∗ ∈ L2 | ||x∗ ||L2 ≤ 1
is compact, and so by Theorem 3 we can in fact find a solution to problem (1) above.
Definition 6 (Notations in Measure Theory). Focus analysis on X :Normed Space
σ-algebra Σ: set of subsets of X such that
φ ∈ Σ, X ∈ Σ; ∀A ∈ Σ ⇒ C
T A ∈ ΣS
Ai ∈ Σ, i = 1, . . . , n ⇒ Ai ∈ Σ, Ai ∈ Σ
Borel σ-algebra BX : smallest σ-algebra containing all open subsets of X
Borel Measure µ on X: set function, BX → [0, ∞)
Measure is regular: ∀ε > 0, ∀A ∈ Σ, ∃open set O and closed set P such that P ⊂ A ⊂ O, µ(O −
P) < ε 
µ(A) = inf{µ(O)|A ⊂ O, O open}
Alternatively, ∀A ∈ Σ,
µ(A) = sup{µ(P )|A ⊃ P, P compact} S P
Measure is finitely additive: E1 , . . . , En ∈ Σ are disjoint subsets of X, then µ ( Ei ) = µ(Ei )
Measure is countably additive:S countable
P sequence E1 , . . . , En , . . . ∈ Σ are pairwise disjoint
subset of X, then one have µ ( Ei ) = µ(Ei )
Measure bounded: bounded with respect to total variance norm
Total variation norm:
Upper variation: W̄ (µ, E) = sup(µ(A)|A ∈ Σ, A ⊂ E)
Lower variation: W (µ, E) = inf(µ(A)|A ∈ Σ, A ⊂ E)
Absolute variation: |µ|(E) = W̄ (µ, E) + |W (µ, E)|
Total variation norm: kµk = |µ|(X)
Definition 7 (Convergence in Distribution). Let {µn }n∈N be the sequence of probability mea-
sures
R on (X,
R BX ), µn converges in distribution (converge weakly) to a probability µ on (X, BX ) if
f dµn → f dµ, ∀f ∈ Cb (X)
The definition of convergence in distribution is equivalent to weak* convergence of probability
measure. Borel probability measure is bounded countabily additive measure. Typically, countably
additive can imply finitely additive, but not vice versa. However, if defined in metric space and
Borel σ-algebra, regulatity and finite additivity can imple countably additive. In this way, in order
to show the equivalence, we only need to show that the dual space of Cb (X) is the space of regular
bounded finitely additive measure rba(X).
Theorem 4. The dual space of bounded functions B(X) on normed space X with Borel σ-algebra
is the space of bounded finitely additive measure ba(X)

4
Theorem 5. Measure µ is finitely additive if for disjoint sets A and B

µ(A ∪ B) = µ(A) + µ(B)

Theorem 6. The dual space of continuous bounded functions Cb (X) on normed space X with Borel
σ-algebra is the space of regular bounded finitely additive measure rba(X) [1]
Z
x∗ (f ) = f (x)dµ(x)
X

Proof. First prove ∀µ ∈ rba(X) defines a linear functional x∗ (f ) = X f (x)dµ(x)

R

∵ f (x) is bounded continuous function defined on X

∴ Range set of f (x) is a bounded set, can be covered with a finite set of open sets G1 , . . . , Gn
with diameter less than ε > 0
Define disjoint sets Ai from Gi : A1 = G1 , Aj = Gj − j−1
S
i=1 Gi
For each Ai , select a point αi ∈ Ai , if Ai = φ, αi = 0
Since f is continuous, Bj = f −1 (Aj ) ⊂ X, in the domain of µ, in this way, a ε-approximation
of f can be defined with the indicator function XBj of Bj by
X
fε = αj XBj

Since the measure is regular and finitely additive, the integral is defined as
Z X
fε dµ = αj µ(Bj )

∵ |αj |6  |fP
R supx∈X (x)| = kf k∞
∴  fε dµ  = | αj µ(Bj )| 6 kf k∞ |µ|
∴ ∀µ ∈ rba(X) defines a linear functional x∗ (f ) = X f (x)dµ(x) R
R

Next prove ∀x∗ ∈ Cb (X)∗ , x∗ (f ) can be represented with x∗ (f ) = X f (x)dµ(x) with µ ∈ rba(X)
∵ Cb (X) is a subspace of space of bounded function B(X)
∴Apply Hahn-Banach theorem extension form Corollary 1 and Theorem 4
∃λ ∈ ba(X) such that x∗ (f ) = X f (x)dλ(x)
R
R R
The goal is to find µ ∈ rba(X) such that X f (x)dµ(x) = X f (x)dλ(x) ∀f ∈ Cd (X)
Denote F as general closed subset, G as general open subset, E as general subset of X. Define
set function µ1 and µ2 as

µ1 (F ) = inf λ(G), µ2 (E) = sup µ1 (F )

G⊃F F ⊂E

By definition, µ1 and µ2 are nonnegative and nondecreasing. Let G1 be open set and F1 be closed
set, if G ⊃ (F1 − G1 ), then (G ∪ G1 ) ⊃ F1 , λ(G ∪ G1 ) 6 λ(G) + λ(G1 ). Since G is arbitary open
set containing F1 − G1 , we have

µ1 (F1 ) 6 λ(G1 ) + µ1 (F1 − G1 )

By allowing G1 range over all open sets containing F ∩ F1 ,

µ1 (F1 ) 6 µ1 (F ∩ F1 ) + µ2 (F1 − F )

5
Let F1 range over all closed subsets of E which is arbitary subset of X

µ2 (E) 6 µ2 (F ∩ E) + µ2 (E − F ) = µ2 (E ∩ F ) + µ2 (E ∩ F c )

Let F1 and F2 be disjoint closed sets, there exists disjoint open neighborhoods G1 and G2 . G is an
arbitary neighborhood of F1 ∪ F2 , then λ(G) > λ(G ∩ G1 ) + λ(G ∩ G2 ), thus

µ1 (F1 ∪ F2 ) > µ1 (F1 ) + µ1 (F2 )

Let F and E be arbitary subsets of X with F closed. Let F1 range over all closed subsets of E ∩ F ,
F2 range over all closed subsets of E ∩ F c

µ2 (E) > µ2 (E ∩ F ) + µ2 (E ∩ F c )

Restrict µ to µ2 , since µ(E) = µ(E ∩ F ) + µ(E ∩ F c ), by Theorem 5, µ is finitely additive. By

definition, µ2 (F ) = µ1 (F ) = µ(F ), thus regular. Let F3 be largest closed subset of X, µ(X) =
µ1 (F3 ) 6 λ(X), thus µ is bounded, µ ∈ rba(X). R Since µ andR λ are bounded, they can be scaled
such that µ(X) = λ(X). Remaining to show X f (x)dµ(x) = X f (x)dλ(x), ∀f ∈ Cb (X)
Since f is continuous and bounded. WLOG let 0 6 f (x) 6 1
Since f is continuous, ∀ε > 0, ∃disjoint paritions E1 , . . . , En of X, ai = inf x∈Ei (f (x))
Xn Z
ai µ(Ei ) + ε > f (x)dµ(x)
i=1 X

Since µ is regular, i.e.∀ε > 0, ∀A ∈ Σ, ∃open set O and closed set P such that P ⊂ A ⊂ O, µ(O −
P ) < ε Let Fi , i = 1, . . . , n be closed subsets of Ei such that
Xn Z
ai µ(Fi ) + 2ε > f (x)dµ(x)
i=1 X

Since f is continuous, there are open sets Gi , i = 1, . . . , n; Gi ⊃ Fi such that

n Z
ε X
bi = inf f (x) > ai − , hence bi µ(Gi ) + 3ε > f (x)dµ(x)
x∈Gi nkµk X
i=1

For a open set G containing F , µ(F ) 6 λ(G). Since µ is regular, µ(G) 6 λ(G), therefore
Z Xn n
X Z
f (x)dµ(x) 6 bi µ(Gi ) 6 bi λ(Gi ) 6 f (x)dλ(x)
X i=1 i=1 X

Since f ∈ Cd (X), 0 6 f 6 1 ⇒ 1 − f ∈ Cd (X), therefore

Z Z
(1 − f )(x)dµ(x) 6 (1 − f )(x)dλ(x)
X X

Since it’s scaled such that µ(X) = λ(X)

Z Z Z Z
(−f )(x)dµ(x) 6 (−f )(x)dλ(x) ⇒ f (x)dµ(x) > f (x)dλ(x)
X X X X
R R
Thus ∀f ∈ Cd (X), ∃µ ∈ rba(X) such that X f (x)dµ(x) = X f (x)dλ(x)

6
References
[1] N. Dunford and J. T. Schwartz, “Linear operators, vol. i,” Interscience, New York, vol. 1963,
1958.

[2] D. G. Luenberger, Optimization by vector space methods. John Wiley & Sons, 1969.