Scribd
Upload
76 views2 pages

SVM Dual Problem Proof

1. The document describes deriving the dual problem formulation for hard margin support vector machines (SVMs). 2. It starts with the primal formulation, then defines the Lagrangian function and takes its partial derivatives to obtain the dual problem. 3. The dual problem is formulated as maximizing the Lagrangian subject to the constraints that all dual variables must be greater than or equal to 0 and their sum must equal 0.

Uploaded by

Sat Adha
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
76 views2 pages

SVM Dual Problem Proof

1. The document describes deriving the dual problem formulation for hard margin support vector machines (SVMs). 2. It starts with the primal formulation, then defines the Lagrangian function and takes its partial derivatives to obtain the dual problem. 3. The dual problem is formulated as maximizing the Lagrangian subject to the constraints that all dual variables must be greater than or equal to 0 and their sum must equal 0.

Uploaded by

Sat Adha
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

1 Hard Margin SVM:

1 T
minimizew,w0 w w
2
s.t. ∀i : yi (wT xi + w0 ) ≥ 1 (1)

2 Deriving Dual Problem for Hard Margin SVM


. The Lagrangian Function can be written as
m
1 T X    
L(w, b, α) = w w− α(i) t(i) wT x(i) + b − 1
2 i=1 (2)
(i)
with ∀i : α ≥0
Partial derivatives of the Lagrangian Function w.r.t w and b implies
m
X
∆w L(w, b, α) = w − α(i) t(i) x(i)
i=1
m
(3)
∂ X
L(w, b, α) = − α(i) t(i)
∂b i=1

When these Partial Derivatives are equal to zero, we have


m
X
ŵ = α̂(i) t(i) x(i)
i=1
m
(4)
X
(i) (i)
α̂ t =0
i=1

If the obtained stationary points are substituted into the Lagrangian,

m m
1 T 1 X (i) (i) (i) T X (i) (i) (i) 1 X X (i) (j) (i) (j)
w w= ( α t x ) ( α t x )= α α t t < x(i) x(j) >
2 2 i=1 i=1
2 i j
(5)
m
X   
 X X X
α(i) t(i) wT x(i) + b − 1 = α(i) t(i) (( α(j) t(j) x(j) )T x(i) + b) − α(i)
i=1 i j i
X X XX
(i) (i)
α t ( α(j) t(j) x(j) )T x(i) = α(i) α(j) t(i) t(j) < x(i) x(j) >
i j i j
(6)
Substituting (5), (6) in (2) gives us,
X 1 X X (i) (j) (i) (j)
L(w, b, α) = α(i) − α α t t < x(i) x(j) > (7)
i
2 i j

1
The Optimization for Dual Problem:
X 1 X X (i) (j) (i) (j)
maxα α(i) − α α t t < x(i) x(j) >
i
2 i j

s.t ∀i : α(i) ≥ 0 (8)


X
α̂(i) t(i) = 0
i

You might also like