Primal-Dual Interior-Point Methods for Linear

Programming
based on Newton’s Method

Robert M. Freund

March, 2004

2004 Massachusetts Institute of Technology.

1
1 The Problem

The logarithmic barrier approach to solving a linear program dates back to

the work of Fiacco and McCormick in 1967 in their book Sequential Un­
constrained Minimization Techniques, also known simply as SUMT. The
method was not believed then to be either practically or theoretically in-
teresting, when in fact today it is both! The method was re-born as a
consequence of Karmarkar’s interior-point method, and has been the sub-
ject of an enormous amount of research and computation, even to this day.
In these notes we present the basic algorithm and a basic analysis of its
performance.

Consider the linear programming problem in standard form:

P : minimize cT x

s.t. Ax = b
x ≥ 0,

where x is a vector of n variables, whose standard linear programming dual

problem is:

D : maximize bT π

s.t. AT π + s = c
s ≥ 0.

Given a feasible solution x of P and a feasible solution (π, s) of D, the du-

ality gap is simply

2
 cT x − bT π = xT s ≥ 0.

We introduce the following notation which will be very convenient

for manipulating equations, etc. Suppose that x > 0. Define the matrix
X to be the n × n diagonal matrix whose diagonal entries are precisely the
components of x. Then X looks like:

⎛ ⎞
x1 0 ... 0
⎜ 0 x2 ... 0 ⎟
⎜ ⎟
⎜ .. .. .. .. ⎟.
⎝ . . . . ⎠
0 0 . . . xn

Notice that X is positive definite, and so is X 2 , which looks like:

⎛ ⎞
(x1 )2 0 ... 0
⎜ 0 (x2 )2 ... 0 ⎟
⎜ ⎟
⎜ . .. .. .. ⎟.
⎝ .. . . . ⎠
0 0 . . . (xn )2

Similarly, the matrices X −1 and X −2 look like:

⎛ ⎞
(1/x1 ) 0 ... 0
⎜ 0 (1/x2 ) . . . 0 ⎟
⎜ ⎟
⎜ ⎟
⎝ ... ... ..
. ... ⎠
0 0 . . . (1/xn )

3
and

⎛ ⎞
1/(x1 )2 0 ... 0
⎜ 0 1/(x2 )2 ... 0 ⎟
⎜ ⎟
⎜ .. .. .. .. ⎟.
⎝ . . . . ⎠
0 0 . . . 1/(xn )2

Let us introduce a logarithmic barrier term for P . We obtain:

n

P (θ) : minimize cT x − θ ln(xj )
j=1

s.t. Ax = b
x > 0.

Because the gradient of the objective function of P (θ) is simply c − θX −1 e,

(where e is the vector of ones, i.e., e = (1, 1, 1, ..., 1)T ), the Karush-Kuhn-
Tucker conditions for P (θ) are:

⎧
⎪
⎨ Ax = b, x > 0
(1)
⎪
⎩ c − θX −1 e = AT π .

If we define s = θX −1 e, then

1
Xs = e ,
θ

4
equivalently

1
XSe = e ,
θ

and we can rewrite the Karush-Kuhn-Tucker conditions as:

⎧
⎪
⎪ Ax = b, x > 0
⎪
⎪
⎪
⎨
AT π + s = c (2)
⎪
⎪
⎪
⎪
⎪
⎩ 1
θ XSe − e = 0 .

From the equations of (2) it follows that if (x, π, s) is a solution of (2), then
x is feasible for P , (π, s) is feasible for D, and the resulting duality gap is:

xT s = eT XSe = θeT e = θn.

This suggests that we try solving P (θ) for a variety of values of θ as θ → 0.

However, we cannot usually solve (2) exactly, because the third

equation group is not linear in the variables. We will instead define a
“β-approximate solution” of the Karush-Kuhn-Tucker conditions (2). A
“β-approximate solution” of P (θ) is defined as any solution (x, π, s) of

⎧
⎪
⎪ Ax = b, x > 0
⎪
⎪
⎪
⎨
AT π + s = c (3)
⎪
⎪
⎪
⎪
⎪
⎩
 1θ Xs − e ≤ β .

Here the norm  ·  is the Euclidean norm.

5
Lemma 1.1 If (¯ x, π,
¯ s̄) is a β-approximate solution of P (θ) and β < 1,
then x
¯ is feasible for P , (¯
π, s)
¯ is feasible for D, and the duality gap satisfies:

nθ(1 − β) ≤ cT x
¯ − bT π ¯T s̄ ≤ nθ(1 + β).
¯=x (4)

Proof: Primal feasibility is obvious. To prove dual feasibility, we need to

show that s̄ ≥ 0. To see this, note that the third equation system of (3)
implies that

x
¯j s¯j
−β ≤ −1≤β
θ

which we can rearrange as:

¯j s¯j ≤ θ(1 + β).

θ(1 − β) ≤ x (5)

¯j s¯j ≥ (1 − β)θ > 0, which implies x

Therefore x ¯j s¯j > 0, and so s¯j > 0. From
(5) we have

n n n
nθ(1 − β) = θ(1 − β) ≤ x ¯T s̄ ≤
¯j s¯j = x θ(1 + β) = nθ(1 + β).
j=1 j=1 j=1

2 The Primal-Dual Algorithm

Based on the analysis just presented, we are motivated to develop the fol-
lowing algorithm:

Step 0 . Initialization. Data is (x0 , π 0 , s0 , θ0 ). k = 0. Assume that

(x0 , π 0 , s0 ) is a β-approximate solution of P (θ0 ) for some known value of β
that satisfies β < 12 .

6
Step 1. Set current values. (¯ ¯ s̄) = (xk , π k , sk ), θ = θk .
x, π,
�
Step 2. Shrink θ. Set θ = αθ for some α ∈ (0, 1).

Step 3. Compute the primal-dual Newton direction. Compute the

Newton step (∆x, ∆π, ∆s) for the equation system (2) at (x, π, s) = (¯
x, π,
¯ s̄)
�
for θ = θ , by solving the following system of equations in the variables
(∆x, ∆π, ∆s): ⎧
⎪
⎪ A∆x = 0
⎪
⎪
⎪
⎨
AT ∆π + ∆s = 0 (6)
⎪
⎪
⎪
⎪
⎪
⎩ ¯
S̄∆x + X∆s ¯ ¯ − θe .
= XSe
Denote the solution to this system by (∆x, ∆π, ∆s).

Step 4. Update All Values.

� � �
(x , π , s ) = (¯
x, π,
¯ s̄) + (∆x, ∆π, ∆s)

� � �
Step 5. Reset Counter and Continue. (xk+1 , π k+1 , sk+1 ) = (x , π , s ).
�
θk+1 = θ . k ← k + 1. Go to Step 1.

Figure 1 shows a picture of the algorithm.

Some of the issues regarding this algorithm include:

• how to set the approximation constant β and the fractional decrease

3
parameter α. We will see that it will be convenient to set β = 40 and
1
8
α=1− 1 √ .
5 + n

• the derivation of the primal-dual Newton equation system (6)

7
 θ=0

θ = 1/10

θ = 10

θ = 100

Figure 1: A conceptual picture of the interior-point algorithm.

8
 • whether or not successive iterative values (xk , π k , sk ) are β-approximate
solutions to P (θk )

• how to get the method started

3 The Primal-Dual Newton Step

We introduce a logarithmic barrier term for P to obtain P (θ):

n

P (θ) : minimizex cT x − θ ln(xj )
j=1
s.t. Ax = b
x > 0.

Because the gradient of the objective function of P (θ) is simply c − θX −1 e,

(where e is the vector of ones, i.e., e = (1, 1, 1, ..., 1)T ), the Karush-Kuhn-
Tucker conditions for P (θ) are:

⎧
⎪
⎨ Ax = b, x > 0
(7)
⎪
⎩ c − θX −1 e = AT π .

We define s = θX −1 e, whereby

1
Xs = e ,
θ
equivalently

1
XSe = e ,
θ

9
and we can rewrite the Karush-Kuhn-Tucker conditions as:

⎧
⎪ Ax = b, x > 0
⎪
⎪
⎪
⎪
⎨
AT π + s = c (8)
⎪
⎪
⎪
⎪
⎪
⎩ 1
θ XSe − e = 0 .

Let (¯
x, π,
¯ s̄) be our current iterate, which we assume is primal and
dual feasible, namely:

A¯
x = b, x
¯>0, AT π
¯ + s¯ = c , s¯ > 0 . (9)

Introducing a direction (∆x, ∆π, ∆s), the next iterate will be (¯

x, π,
¯ s̄) +
(∆x, ∆π, ∆s), and we want to solve:

⎧
⎪
⎪ A(¯
x + ∆x) = b, x
¯ + ∆x > 0
⎪
⎪
⎪
⎨
AT (¯
π + ∆π) + (¯
s + ∆s) = c
⎪
⎪
⎪
⎪
⎪
⎩ 1 ¯
θ (X + ∆X)(S + ∆S)e − e = 0 .
¯

Keeping in mind that (¯

x, π,
¯ s̄) is primal-dual feasible and so satisfies (9), we
can rearrange the above to be:

⎧
⎪
⎪ A∆x = 0
⎪
⎪
⎪
⎨
AT ∆π + ∆s = 0
⎪
⎪
⎪
⎪
⎪
⎩ ¯
S̄∆x + X∆s = θe − XSe
¯ ¯ − ∆X∆Se .

Notice that the only nonlinear term in the above system of equations in
(∆x, ∆π, ∆s) is term the term “ ∆X∆Se” in the last system. If we erase
this term, which is the same as the linearized version of the equations, we
obtain the following primal-dual Newton equation system:

10
 ⎧
⎪ A∆x = 0
⎪
⎪
⎪
⎪
⎨
AT ∆π + ∆s = 0 (10)
⎪
⎪
⎪
⎪
⎪
⎩ ¯
S̄∆x + X∆s = θe − XSe
¯¯ .

The solution (∆x, ∆π, ∆s) of the system (10) is called the primal-
dual Newton step. We can manipulate these equations to yield the following
formulas for the solution:

 −1 
 
∆x ← ¯ S̄ −1 AT AX
I −X ¯ S̄ −1 AT A ¯ + θS̄ −1 e ,
−x

 −1  
∆π ← ¯ S̄ −1 AT
AX ¯ − θS¯−1 e ,
A x (11)

 −1  
∆s ← AT AX
¯ S̄ −1 AT ¯ + θS̄ −1 e .
A −x

Notice, by the way, that the computational effort in these equations

lies primarily in solving a single equation:

   
¯ S̄ −1 AT ∆π = A x
AX ¯ − θS̄ −1 e .

Once this system is solved, we can easily substitute:

∆s ← −AT ∆π
(12)
∆x ← −x
¯+ θS̄ −1 e − S̄ −1 X∆s
¯ .

However, let us instead simply work with the primal-dual Newton system
(10). Suppose that (∆x, ∆π, ∆s) is the (unique) solution of the primal-dual
Newton system (10). We obtain the new value of the variables (x, π, s) by
taking the Newton step:

11
 � � �
(x , π , s ) = (¯
x, π,
¯ s)
¯ + (∆x, ∆π, ∆s) .

We have the following very powerful convergence theorem which demon-

strates the quadratic convergence of Newton’s method for this problem,
with an explicit guarantee of the range in which quadratic convergence takes
place.

Theorem 3.1 (Explicit Quadratic Convergence of Newton’s Method).

Suppose that (¯ ¯ s̄) is a β-approximate solution of P (θ) and β < 12 . Let
x, π,
(∆x, ∆π, ∆s) be the solution to the primal-dual Newton equations (10), and
let:
� � �
(x , π , s ) = (¯
x, π,
¯ s̄) + (∆x, ∆π, ∆s) .
� � �
 
1+β
Then (x , π , s ) is a (1−β)2
β 2 -approximate solution of P (θ).

Proof: Our current point (¯

x, π,
¯ s̄) satisfies:
⎧
⎪
⎪ A¯
x = b, x
¯>0
⎪
⎪
⎪
⎨
AT π
¯ + s¯ = c (13)
⎪
⎪
⎪
⎪
⎪
⎩
 1θ XSe
¯ ¯ − e ≤ β .

Furthermore the primal-dual Newton step (∆x, ∆π, ∆s) satisfies:

⎧
⎪
⎪ A∆x = 0
⎪
⎪
⎪
⎨
AT ∆π + ∆s = 0 (14)
⎪
⎪
⎪
⎪
⎪
⎩ ¯
S̄∆x + X∆s = θe − XSe
¯¯ .

Note from the first two equations of (14) that ∆xT ∆s = 0. From the third
equation of (13) we have
s¯j x
¯j
1−β ≤ ≤ 1 + β, j = 1, . . . , n, (15)
θ

12
which implies:

(1 − β)θ (1 − β)θ
¯j ≥
x and s¯j ≥ , j = 1, . . . , n . (16)
s̄j x̄j

As a result of this we obtain:

¯ −1 ∆x2 = θ(1 − β)∆xT X
θ(1 − β)X ¯ −1 X
¯ −1 ∆x
≤ ∆xT X ¯ −1 S∆x
¯
 
T ¯ −1
= ∆x X θe − XSe ¯ ¯ − X∆s
¯

= ∆xT X ¯ −1 θe − XSe
¯¯
¯ −1
≤ X ∆xθe − XSe ¯¯
≤ X −1
¯ ∆xβθ .

From this it follows that

¯ −1 ∆x ≤ 1 − β < 1 .
X
β
Therefore
�
x =x ¯ +X
¯ + ∆x = X(e ¯ −1 ∆x) > 0 .

We have the exact same chain of inequalities for the dual variables:

θ(1 − β)S̄ −1 ∆s2 = θ(1 − β)∆sT S̄ −1 S̄ −1 ∆s

≤ ∆sT S¯−1 X∆s
¯
 
T −1
= ∆s S̄ θe − XSe ¯ ¯ − S∆x¯

= ∆sT S̄ −1 θe − XSe
¯¯
≤ S̄ −1 ∆sθe − XSe
¯¯
−1
≤ S̄ ∆sβθ .

From this it follows that

1−β
S̄ −1 ∆s ≤ <1.
β
Therefore
�
¯ + S̄ −1 ∆s) > 0 .
s = s¯ + ∆s = S(e
Next note from (14) that for j = 1, . . . , n we have:
� �
xj sj = (¯
xj +∆xj )(¯
sj +∆sj ) = x
¯j s¯j +¯
xj ∆sj +∆xj s̄j +∆xj ∆sj = θ+∆xj ∆sj .

13
Therefore  
1 � � ∆xj ∆sj
e− X S e =− .
θ j θ
From this we obtain:
� � � �
e − 1θ X S e ≤ e − 1θ X S e1
 |∆xj ∆sj |
= nj=1 θ
 |∆x | ∆s | x
¯ s¯
= nj=1 x̄j j s̄jj jθ j
 |∆x | |∆s |
≤ nj=1 x¯j j s̄j j (1 + β)
¯ −1 ∆xS̄ −1 ∆s(1 + β)
≤ X
 2
β
≤ 1−β (1 + β)

4 Complexity Analysis of the Algorithm

Theorem 4.1 (Relaxation Theorem). Suppose that (¯

x, π,
¯ s̄) is a β =
3
40 -approximate solution of P (θ). Let
1
8
α=1− 1 √
5 + n
� �
and let θ = αθ. Then (¯
x, π, ¯ is a β = 15 -approximate solution of P (θ ).
¯ s)

Proof: The triplet (¯x, π,

¯ s)
¯ satisfies A¯ ¯ > 0, and AT π
x = b, x ¯ + s¯ = c, and
so it remains to show that
 
1 
¯ − e ≤ 1 .
 � Xs̄
θ  5
We have
         
1     
¯ − e =  1 Xs̄
 � Xs̄ ¯ − e =  1 1 Xs̄¯ − e − 1 − 1 e
θ   αθ  α θ α 
    
1    
¯ − e +  1 − α  e
 1 Xs̄
≤
α θ   α 

14
 3   3 √
40 1−α √ 40 + n √ 1
≤ + n = − n= .
α α α 5

Theorem 4.2 (Convergence Theorem). Suppose that (x0 , π 0 , s0 ) is a

3
β = 40 -approximate solution of P (θ0 ) . Then for all k = 1, 2, 3, . . ., (xk , π k , sk )
3
is a β = 40 -approximate solution of P (θk ).

Proof: By induction, suppose that the theorem is true for iterates 0, 1, 2, ..., k.
3
Then (xk , π k , sk ) is a β = 40 -approximate solution of P (θk ). From the Re-
laxation Theorem, (x , π , s ) is a 15 -approximate solution of P (θk+1 ) where
k k k

θk+1 = αθk .
From the Quadratic Convergence Theorem, (xk+1 , π k+1 , sk+1 ) is a β-approximate
solution of P (θk+1 ) for
1  2
1+ 5 1 3
β= 2 = .
1− 1 5 40
5

Therefore, by induction, the theorem is true for all values of k.

Figure 2 shows a better picture of the algorithm.

Theorem 4.3 (Complexity Theorem). Suppose that (x0 , π 0 , s0 ) is a

3
β = 40 -approximate solution of P (θ0 ). In order to obtain primal and dual
feasible solutions (xk , π k , sk ) with a duality gap of at most , one needs to
run the algorithm for at most

  
√ 43 (x0 )T s0
k = 10 n ln
37 
iterations.

Proof: Let k be as defined above. Note that

1
8 1 1
α=1− 1 √ =1− 
8 √  ≤ 1 − 10√n .
+ n
5 5 +8 n

15
 θ = 80

x^

~
x θ = 90

_
x θ = 100

Figure 2: Another picture of the interior-point algorithm.

16
Therefore  k
1
θk ≤ 1 − √ θ0 .
10 n
This implies that
 k  
1 43
cT xk − bT π k = (xk )T sk ≤ θk n(1 + β) ≤ 1 − √ n θ0 )
10 n 40
 k   
1 43 (x0 )T s0
≤ 1− √ n 37 ,
10 n 40 40 n
from (4). Taking logarithms, we obtain
   
T k T k 1 43 0 T 0
ln(c x − b π ) ≤ k ln 1 − √ + ln (x ) s
10 n 37
 
−k 43 0 T 0
≤ √ + ln (x ) s
10 n 37
   
43 (x0 )T s0 43 0 T 0
≤ − ln + ln (x ) s = ln().
37  37
Therefore cT xk − bT π k ≤ .

5 An Implementable Primal-Dual Interior-Point

Algorithm

Herein we describe a more implementable primal-dual interior-point algo-

rithm. This algorithm differs from the previous method in the following
respects:

• We do not assume that the current point is near the central path. In
fact, we do not assume that the current point is even feasible.
1
• The fractional decrease parameter α is set to α = 10 rather than the
conservative value of
1
8
α=1− 1 √ .
5 + n

17
 • We do not necessarily take the full Newton step at each iteration, and
we take different step-sizes in the primal and dual.

Our current “point” is (¯x, π,

¯ s)
¯ for which x
¯ > 0 and s¯ > 0, but quite possibly
A¯ T
x = b and/or A π¯ + s¯ = c. We are given a value of the central path barrier
parameter θ > 0. We want to compute a direction (∆x, ∆π, ∆s) so that
x + ∆x, π
(¯ ¯ + ∆π, s̄ + ∆s) approximately solves the central path equations.
We set up the system:

(1) A(x̄ + ∆x) = b

(2) AT (¯
π + ∆π) + (¯
s + ∆s) = c

¯ + ∆X)(S¯ + ∆S)e = θe .
(3) (X

We linearize this system of equations and rearrange the terms to

obtain the Newton equations for the current point (¯
x, π,
¯ s̄):

(1) A∆x = b − Ax̄ =: r1

(2) AT ∆π + ∆s = c − AT π̄ − s̄ =: r2

¯
(3) S̄∆x + X∆s = θe − XSe
¯ ¯ =: r3

We refer to the solution (∆x, ∆π, ∆s) to the above system as the primal-
dual Newton direction at the point (¯ x, π, ¯ It differs from that derived
¯ s).
earlier only in that earlier it was assumed that r1 = 0 and r2 = 0.
Given our current point (¯
x, π,
¯ s̄) and a given value of θ, we compute
the Newton direction (∆x, ∆π, ∆s) and we update our variables by choosing
primal and dual step-sizes αP and αD to obtain new values:

(˜ ˜ s̃) ← (¯
x, π, x + αP ∆x, π
¯ + αD ∆π, s̄ + αD ∆s) .

18
 In order to ensure that x ˜ > 0 and s˜ > 0, we choose a value of r
satisfying 0 < r < 1 (r = 0.99 is a common value in practice), and determine
αP and αD as follows:

  
x
¯j
αP = min 1, r min
∆xj <0 −∆xj
  
s̄j
αD = min 1, r min .
∆sj <0 −∆sj

These step-sizes ensure that the next iterate (˜

x, π,
˜ s)
˜ satisfies x
˜>0
and s̃ > 0.

5.1 Decreasing the Path Parameter θ

We also want to shrink θ at each iteration, in order to (hopefully) shrink the

duality gap. The current iterate is (¯
x, π, ¯ and the current values satisfy:
¯ s),

x̄T s¯
θ≈
n

We then re-set θ to
  T 
1 x
¯ s̄
θ← ,
10 n
1
where the fractional decrease 10 is user-specified.

5.2 The Stopping Criterion

We typically declare the problem solved when it is “almost” primal feasible,

“almost” dual feasible, and there is “almost” no duality gap. We set our

19
tolerance  to be a small positive number, typically  = 10−8 , for example,
and we stop when:

(1) Ax̄ − b ≤ 

(2) AT π
¯ + s¯ − c ≤ 

(3) s̄T x
¯≤

5.3 The Full Interior-Point Algorithm

1. Given (x0 , π 0 , s0 ) satisfying x0 > 0, s0 > 0, and θ0 > 0, and r satisfying

0 < r < 1, and  > 0. Set k ← 0.
2. Test stopping criterion. Check if:

(1) Axk − b ≤ 

(2) AT π k + sk − c ≤ 

(3) (sk )T xk ≤  .

If so, STOP. If not, proceed.

  
1 (xk )T (sk )
3. Set θ ←
10 n
4. Solve the Newton equation system:

(1) A∆x = b − Axk =: r1

(2) AT ∆π + ∆s = c − AT π k − sk =: r2

(3) S k ∆x + X k ∆s = θe − X k S k e =: r3

20
 5. Determine the step-sizes:

  
xkj
θP = min 1, r min
∆xj <0 −∆xj
  
skj
θD = min 1, r min .
∆sj <0 −∆sj

6. Update values:

(xk+1 , π k+1 , sk+1 ) ← (xk + αP ∆x, π k + αD ∆π, sk + αD ∆s) .

Re-set k ← k + 1 and return to (b).

5.4 Remarks on Interior-Point Methods

• The algorithm just described is almost exactly what is used in com-

mercial interior-point method software.

• A typical interior-point code will solve a linear or quadratic optimiza-

tion problem in 25-80 iterations, regardless of the dimension of the
problem.

• These days, interior-point methods have been extended to allow for

the solution of a very large class of convex nonlinear optimization
problems.

6 Exercises on Interior-Point Methods for LP

1. (Interior Point Method) Verify that the formulas given in (11) indeed
solve the equation system (10).

21
2. (Interior Point Method) Prove Proposition 6.1 of the notes on New-
ton’s method for a logarithmic barrier algorithm for linear program-
ming.

3. (Interior Point Method) Create and implement your own interior-point

algorithm to solve the following linear program:

LP:
minimize cT x

s.t. Ax = b
x ≥ 0,
where    
1 1 1 0 100
A= , b= ,
1 −1 0 1 50
and  
cT = −9 −10 0 0 .

In running your algorithm, you will need to specify the starting point
(x0 , π 0 , s0 ), the starting value of the barrier parameter θ0 , and the
value of the fractional decrease parameter α. Use the following values
and make eight runs of the algorithm as follows:

• Starting Points:
– (x0 , π 0 , s0 ) = ((1, 1, 1, 1), (0, 0), (1, 1, 1, 1))
– (x0 , π 0 , s0 ) = ((100, 100, 100, 100), (0, 0), (100, 100, 100, 100))
• Values of θ0 :
– θ0 = 1000.0
– θ0 = 10.0
• Values of α:
1√
– α=1− 10 n
1
– α= 10

4. (Interior Point Method) Suppose that x ¯ > 0 is a feasible solution of

the following logarithmic barrier problem:

22
 P (θ):
n

minimize cT x − θ ln(xj )
j=1

s.t. Ax = b
x ≥ 0,
and that we wish to test if x̄ is a β-approximate solution to P (θ) for a
given value of θ. The conditions for x̄ to be a β-approximate solution
to P (θ) are that there exist values (π, s) for which:
⎧
⎪
⎪ A¯
x = b, x
¯>0
⎪
⎪
⎪
⎨
AT π + s = c (17)
⎪
⎪
⎪
⎪
⎪
⎩
 1θ Xs
¯ − e ≤ β.

Construct an analytic test for whether or not such (π, s) exist by solv-
ing an associated equality-constrained quadratic program (which has
a closed form solution). HINT: Think of trying to minimize  1θ Xs
¯ − e
over appropriate values of (π, s).
5. Consider the logarithmic barrier problem:
n

P (θ) : min cT x − θ ln xj
j=1
s.t. Ax = b

x > 0.

Suppose x
¯ is a feasible point, i.e., x
¯ > 0 and A¯ ¯ = diag(¯
x = b. Let X x).
a. Construct the (primal) Newton direction d at x̄. Show that d can
be written as:
 
¯ T (AX
¯ − XA
d = X(I ¯ e − 1 Xc
¯ 2 AT )−1 AX) ¯ .
θ
¯ T (AX
b. Show in the above that P = (I − XA ¯ 2 AT )−1 AX)
¯ is a pro-
jection matrix.

23
c. Suppose AT π ¯ + s¯ = c and s¯ > 0 for some π,
¯ s.
¯ Show that d can
also be written as:
 
¯ T (AX
¯ − XA
d = X(I ¯ e − 1 Xs̄
¯ 2 AT )−1 AX) ¯ .
θ

d. Show that if there exists (¯ ¯ with AT π

π, s) ¯ + s¯ = c and s¯ > 0, and
1 ¯ 1 −1 1
¯ ≤ 4 , then X d ≤ 4 , and then the Newton iterate
e − θ X s ¯
satisfies x̄ + d > 0.

24