Gradient Flow PDF
Gradient Flow PDF
Time
1
0
Pos
itio 0.0
0.5
n -0.5
Example:
metric energy functional gradient flow
d 1 2 d
(R , | · |) E(x) = x x(t) = x(t)
2 dt
Given x(0) ∈ ℝd, x(t) = x(0)e-t is unique solution of the gradient flow.
2
gradient flow in finite dimensions
Gradient flows often arise when solving optimization problems:
min E(x)
x2Rd
3
gradient flow in finite dimensions
If E(x) is λ-convex, then…
4
gradient flow
5
gradient flow
5
gradient flow with different metrics
In general, given a complete metric space (X,d), a curve x(t): ℝ → X is the
gradient flow of an energy E: X → ℝ if
“ d
x(t) = rX E(x(t)) ’’
dt
Examples: Euclidean L2
metric (X,d) (Rd , | · |) (L2 (Rd ), k · kL2 )
Examples: Euclidean W2
metric (X,d) (Rd , | · |) (P2 (Rd ), W2 )
Examples: Euclidean W2
metric (X,d) (Rd , | · |) (P2 (Rd ), W2 )
Examples: Euclidean W2
metric (X,d) (Rd , | · |) (P2 (Rd ), W2 )
Examples: Euclidean W2
metric (X,d) (Rd , | · |) (P2 (Rd ), W2 )
Examples: Euclidean W2
metric (X,d) (Rd , | · |) (P2 (Rd ), W2 )
L2 geodesic W2 geodesic
⇢(t) = (1 t)⇢0 + t⇢1 ⇢(t) = ((1 t)id + tT⇢⇢01 )#⇢0
10
interpolating with different metrics
The same dichotomy between values of a function and mass of a
function is also present in the geodesics.
Def: A constant speed geodesic between two points ρ0 and ρ1 in a
metric space (X,d) is any curve ρ:[0,1]→X s.t.
⇢(0) = ⇢0 , ⇢(1) = ⇢1 , d(⇢(t), ⇢(s)) = |t s|d(⇢0 , ⇢1 )
L2 geodesic W2 geodesic
⇢(t) = (1 t)⇢0 + t⇢1 ⇢(t) = ((1 t)id + tT⇢⇢01 )#⇢0
10
interpolating with different metrics
The same dichotomy between values of a function and mass of a
function is also present in the geodesics.
Def: A constant speed geodesic between two points ρ0 and ρ1 in a
metric space (X,d) is any curve ρ:[0,1]→X s.t.
⇢(0) = ⇢0 , ⇢(1) = ⇢1 , d(⇢(t), ⇢(s)) = |t s|d(⇢0 , ⇢1 )
L2 geodesic W2 geodesic
⇢(t) = (1 t)⇢0 + t⇢1 ⇢(t) = ((1 t)id + tT⇢⇢01 )#⇢0
10
gradient flow in the Wasserstein metric
Examples:
energy functional gradient flow
Z
d
E(⇢) = ⇢ log ⇢ ⇢= ⇢
dt
Z
1 d
E(⇢) = ⇢m ⇢ = ⇢m
m 1 dt
Z
d
E(⇢) = V ⇢ ⇢ = r · (rV ⇢)
dt
Z
d
E(⇢) = (K ⇤ ⇢)⇢ ⇢ = r · (r(K ⇤ ⇢)⇢)
dt
All Wasserstein gradient flows are of the form
d
⇢ + r · (v⇢) = 0
dt
continuity equation 11
gradient flow in the Wasserstein metric
Examples:
energy functional gradient flow
Z r⇢
d
E(⇢) = ⇢ log ⇢ ⇢= ⇢ v=
dt ⇢
Z
1 d
E(⇢) = ⇢m ⇢ = ⇢m
m 1 dt
Z
d
E(⇢) = V ⇢ ⇢ = r · (rV ⇢)
dt
Z
d
E(⇢) = (K ⇤ ⇢)⇢ ⇢ = r · (r(K ⇤ ⇢)⇢)
dt
All Wasserstein gradient flows are of the form
d
⇢ + r · (v⇢) = 0
dt
continuity equation 11
gradient flow in the Wasserstein metric
Examples:
energy functional gradient flow
Z r⇢
d
E(⇢) = ⇢ log ⇢ ⇢= ⇢ v=
dt ⇢
Z
1 d
E(⇢) = ⇢m ⇢ = ⇢m v= m⇢m 2
r⇢
m 1 dt
Z
d
E(⇢) = V ⇢ ⇢ = r · (rV ⇢)
dt
Z
d
E(⇢) = (K ⇤ ⇢)⇢ ⇢ = r · (r(K ⇤ ⇢)⇢)
dt
All Wasserstein gradient flows are of the form
d
⇢ + r · (v⇢) = 0
dt
continuity equation 11
gradient flow in the Wasserstein metric
Examples:
energy functional gradient flow
Z r⇢
d
E(⇢) = ⇢ log ⇢ ⇢= ⇢ v=
dt ⇢
Z
1 d
E(⇢) = ⇢m ⇢ = ⇢m v= m⇢m 2
r⇢
m 1 dt
Z
d
E(⇢) = V ⇢ ⇢ = r · (rV ⇢) v= rV
dt
Z
d
E(⇢) = (K ⇤ ⇢)⇢ ⇢ = r · (r(K ⇤ ⇢)⇢)
dt
All Wasserstein gradient flows are of the form
d
⇢ + r · (v⇢) = 0
dt
continuity equation 11
gradient flow in the Wasserstein metric
Examples:
energy functional gradient flow
Z r⇢
d
E(⇢) = ⇢ log ⇢ ⇢= ⇢ v=
dt ⇢
Z
1 d
E(⇢) = ⇢m ⇢ = ⇢m v= m⇢m 2
r⇢
m 1 dt
Z
d
E(⇢) = V ⇢ ⇢ = r · (rV ⇢) v= rV
dt
Z
d
E(⇢) = (K ⇤ ⇢)⇢ ⇢ = r · (r(K ⇤ ⇢)⇢) v = r(K ⇤ ⇢)
dt
All Wasserstein gradient flows are of the form
d
⇢ + r · (v⇢) = 0
dt
continuity equation 11
gradient flow in the Wasserstein metric
Examples:
energy functional gradient flow
Z r⇢
d
E(⇢) = ⇢ log ⇢ ⇢= ⇢ v=
dt ⇢
Z
1 d
E(⇢) = ⇢m ⇢ = ⇢m v= m⇢m 2
r⇢
m 1 dt
Z
d
E(⇢) = V ⇢ ⇢ = r · (rV ⇢) v= rV
dt
Z
d
E(⇢) = (K ⇤ ⇢)⇢ ⇢ = r · (r(K ⇤ ⇢)⇢) v = r(K ⇤ ⇢)
dt
All Wasserstein gradient flows are of the form
d @E
⇢ + r · (v⇢) = 0 v= r
dt @⇢
continuity equation 11
gradient flow in the Wasserstein metric
aggregation, drift, and degenerate diffusion:
d
⇢ = r · ((rK ⇤ ⇢)⇢) + r · (rV ⇢) + ⇢m K, V : Rd ! R, and m 1
dt
| {z } |{z} |{z}
self|interaction
{z } drift |{z}
diffusion
|{z}
Z Z Z
1 1
E(⇢) = K ⇤ ⇢d⇢ + V d⇢ + ⇢m
2 m 1
12
biological chemotaxis
a colony of slime mold [Gregor, et. al]
13
biological chemotaxis
a colony of slime mold [Gregor, et. al]
13
gradient flow in the Wasserstein metric
aggregation, drift, and degenerate diffusion:
d
⇢ = r · ((rK ⇤ ⇢)⇢) + r · (rV ⇢) + ⇢m K, V : Rd ! R, and m 1
dt
14
gradient flow in the Wasserstein metric
aggregation, drift, and degenerate diffusion:
d
⇢ = r · ((rK ⇤ ⇢)⇢) + r · (rV ⇢) + ⇢m K, V : Rd ! R, and m 1
dt
14
gradient flow in the Wasserstein metric
aggregation, drift, and degenerate diffusion:
d
⇢ = r · ((rK ⇤ ⇢)⇢) + r · (rV ⇢) + ⇢m K, V : Rd ! R, and m 1
dt
14
gradient flow in the Wasserstein metric
aggregation, drift, and degenerate diffusion:
d
⇢ = r · ((rK ⇤ ⇢)⇢) + r · (rV ⇢) + ⇢m K, V : Rd ! R, and m 1
dt
14
gradient flow in the Wasserstein metric
aggregation, drift, and degenerate diffusion:
d
⇢ = r · ((rK ⇤ ⇢)⇢) + r · (rV ⇢) + ⇢m K, V : Rd ! R, and m 1
dt
K(x) = |x|3 /3
14
gradient flow in the Wasserstein metric
aggregation, drift, and degenerate diffusion:
d
⇢ = r · ((rK ⇤ ⇢)⇢) + r · (rV ⇢) + ⇢m K, V : Rd ! R, and m 1
dt
K(x) = |x|3 /3
14
gradient flow in the Wasserstein metric
aggregation, drift, and degenerate diffusion:
d
⇢ = r · ((rK ⇤ ⇢)⇢) + r · (rV ⇢) + ⇢m K, V : Rd ! R, and m 1
dt
3
K(x) = |x| /3 |x| K(x) = |x|3 /3
14
gradient flow in the Wasserstein metric
aggregation, drift, and degenerate diffusion:
d
⇢ = r · ((rK ⇤ ⇢)⇢) + r · (rV ⇢) + ⇢m K, V : Rd ! R, and m 1
dt
3
K(x) = |x| /3 |x| K(x) = |x|3 /3
14
gradient flow in the Wasserstein metric
aggregation, drift, and degenerate diffusion:
d
⇢ = r · ((rK ⇤ ⇢)⇢) + r · (rV ⇢) + ⇢m K, V : Rd ! R, and m 1
dt
15
gradient flow in the Wasserstein metric
aggregation, drift, and degenerate diffusion:
d
⇢ = r · ((rK ⇤ ⇢)⇢) + r · (rV ⇢) + ⇢m K, V : Rd ! R, and m 1
dt
15
gradient flow in the Wasserstein metric
aggregation, drift, and degenerate diffusion:
d
⇢ = r · ((rK ⇤ ⇢)⇢) + r · (rV ⇢) + ⇢m K, V : Rd ! R, and m 1
dt
15
gradient flow in the Wasserstein metric
aggregation, drift, and degenerate diffusion:
d
⇢ = r · ((rK ⇤ ⇢)⇢) + r · (rV ⇢) + ⇢m K, V : Rd ! R, and m 1
dt
16
gradient flow in the Wasserstein metric
aggregation, drift, and degenerate diffusion:
d
⇢ = r · ((rK ⇤ ⇢)⇢) + r · (rV ⇢) + ⇢m K, V : Rd ! R, and m 1
dt
16
gradient flow in the Wasserstein metric
aggregation, drift, and degenerate diffusion:
d
⇢ = r · ((rK ⇤ ⇢)⇢) + r · (rV ⇢) + ⇢m K, V : Rd ! R, and m 1
dt
16
numerical simulation of W2 grad flow
JKO scheme
• Pros: reduces simulating grad flow to solving a sequence of optimization
problems involving W2 distance; leverages state of the art W2 solvers.
• Cons: current methods lose convexity/stability properties of gradient flow
Particle methods
17
particle methods
d
Goal: Approximate a solution to ⇢(x, t) + r · (v(x, t)⇢(x, t)) = 0
dt
Assume: v(x,t) comes from a Wasserstein gradient flow and is “nice”
• But even the interaction term can slow down convergence if it has a
strong singularity at the origin…
20
v(x,t) is often not nice v(x,t)
• But even the interaction term can slow down convergence if it has a
strong singularity at the origin…
20
v(x,t) is often not nice v(x,t)
• But even the interaction term can slow down convergence if it has a
strong singularity at the origin…
20
particle
a blob method
methods
for aggregation
dd
Goal: Approximate a solution to ⇢⇢(x,
= rt)· + r · (v(x,
((rK t)⇢(x,
⇤ ⇢)⇢) v =t))rK
= 0⇤ ⇢
dt
dt
N
X
(3) ⇢N (x, t) = xi (t) mi is a gradient
Z flow of the regularized energy
i=1
E✏ (⇢) = (K ✏ ⇤ ⇢)⇢
21
a blob method for aggregation
Theorem [C., Bertozzi 2014]: If ε = hq, 0<q<1, the blob method
two dimensions,
converges as h →0 . aggregation
K(x) = log |x|/2π K(x) = |x|2 /2 K(x) = |x|3 /3
Time
1
0
Pos
itio 0.0
0.5
n -0.5
22
aggregation + ?
aggregation, drift, and diffusion:
d m
⇢ = r · ((rK ⇤ ⇢)⇢) + r · (rV ⇢) + ⇢ K, V : Rd ! R, and m 1
dt
25
numerics: Keller-Segel (d=2)
subcritical mass critical mass supercritical mass
Evolution of Density
26
numerics: Keller-Segel (d=2)
subcritical mass critical mass supercritical mass
Evolution of Density
26
numerics: Keller-Segel (d=2)
subcritical mass critical mass supercritical mass
Evolution of Density
26
Future work
• Convergence for 1 ≤ m < 2?