ISyE 6669 HW 2

January 26, 2023

1. Expand the following summations:

(For example, the answer to part (a) is x1 + x2 + x3 .)
P3 P2 P4
(a) i=1 xi (d) j=2 (xij − yi )
P3 x2t Pi=1
1
(b) (e) (2k + 1)x2k−1
Pt=1
3 Pt!
i i+j
Pk=−1
4 Pn+2
(c) i=1 j=1 x (f) n=2 m=n xn ym
Note that by definition t! = 1 · 2 · · · (t − 1) · t for integer t ≥ 1.
Solution:

(a) x1 + x2 + x3
x4 x6
(b) x2 + 2 + 6
(c) x2 + x + x + x4 + x5 + x6
3 4

(d) (x12 − y1 ) + (x13 − y1 ) + (x14 − y1 ) + (x22 − y2 ) + (x23 − y2 ) + (x24 − y2 )

(e) −x2−2 + x2−1 + 3x20
(f) x2 y2 + x2 y3 + x2 y4 + x3 y3 + x3 y4 + x3 y5 + x4 y4 + x4 y5 + x4 y6
  
1 3
2. Consider the following two vectors: x =  2  , y =  2 , and a matrix
3 1
 
1 −2 3
A = −2 3 −1.
3 −1 2

(a) Let n be the dimension of x and y. What is the value of n?

(b) Compute 2x − y.
(c) Compute the inner product x> y.
pPn
(d) Compute the Euclidean norm kx − yk2 = 2
i=1 (xi − yi ) . Also
called the `2 -norm.
Pn
(e) Compute the `1 -norm kx − yk1 = i=1 |xi − yi |.
(f) Compute the `∞ -norm kx − yk∞ = max1≤i≤n |xi − yi |.
(g) Compute x> Ay.

1
 Solution:
(a) The dimension of x and of y are 3. Thus, n is 3.
(b) 2x − y = (−1, 2, 5)>
(c) x> y = 10
√ √ √
(d) kx − yk = −22 + 02 + 22 = 8=2 2
(e) kx − yk1 = | − 2| + |0| + |2| = 4
(f) kx − yk∞ = max{2, 0, 2} = 2
(g) x> Ay = 27

3. State whether each of the following sets is convex or not. Explain your
reasoning.
(a) X = {(x1 , x2 ) ∈ R2 | x21 + x22 ≥ 1}.
(b) X = {x ∈ R | x2 = 2}.
x2
(c) X = {(x1 , x2 ) | (x1 −1) ≤ 1, x1 ≥ 1}.

Solution:

(a) X is not convex. Consider X1 = (0, 1), X2 = (0, −1) X1 , X2 ∈ X but

the convex combination X3 = ( 21 · (0) + 12 · (0), 21 · (1) + 21 · (−1)) =
(0, 0) 6∈ X.
√ √ √
(b) X = {− 2, 2} and thus is not convex because 0 = 12 · (− 2) + 12 ·
√
( 2) 6∈ X
(c) Method 1: Notice that points for which x1 = 1 do not belong to
the set, because first inequality is not defined for those points. Then
from x1 > 1 we have x1 − 1 > 0 and hence for points (x1 , x2 ) ∈ X
we have x1x−1
2
≤ 1 ⇔ x2 ≤ x1 − 1 With that in mind we can rewrite
set as:

X = {(x1 , x2 ) : x2 ≤ x1 − 1, x1 > 1} =
= {(x1 , x2 ) : x2 ≤ x1 − 1} ∩ {(x1 , x2 ) : x1 > 1}

Since both of the two sets are defined using only linear constraints
they are convex, and hence X is convex as intercept of two convex
sets.
Method 2: Again we will use the fact that:

X = {(x1 , x2 ) : x2 ≤ x1 − 1, x1 > 1}
We will show that this set is convex using the definition. Let a =
(a1 , a2 ), b = (b1 , b2 ) ∈ X. That means:

2
 a1 , b1 > 1
a2 ≤ a1 − 1
b2 ≤ b1 − 1

We need to show that c = (c1 , c2 ) = λa + (1 − λ)b ∈ X i.e that

c1 > 1
c2 ≤ c1 − 1

We have

c = (c1 , c2 ) = λa + (1 − λ)b = (λa1 + (1 − λ)b1 , λa2 + (1 − λ)b2 )

From a1 , b1 > 1 we have:

c1 = λa1 + (1 − λ)b1 > λ + (1 − λ) = 1

From a2 ≤ a1 − 1 and b2 ≤ b1 − 1 we have:

c2 = λa2 + (1 − λ)b2 ≤ λ(a1 − 1) + (1 − λ)(b1 − 1) = λa1 + (1 − λ)b1 − 1

Hence c ∈ X, and thus set X is convex. We will show that this set is
convex using the definition. Let a = (a1 , a2 ), b = (b1 , b2 ) ∈ X. That
means:

a1 , b1 > 1
a2 ≤ a1 − 1
b2 ≤ b1 − 1

We need to show that c = (c1 , c2 ) = λa + (1 − λ)b ∈ X i.e that

c1 > 1
c2 ≤ c1 − 1

We have

c = (c1 , c2 ) = λa + (1 − λ)b = (λa1 + (1 − λ)b1 , λa2 + (1 − λ)b2 )

3
 From a1 , b1 > 1 we have:

c1 = λa1 + (1 − λ)b1 > λ + (1 − λ) = 1

From a2 ≤ a1 − 1 and b2 ≤ b1 − 1 we have:

c2 = λa2 + (1 − λ)b2 ≤ λ(a1 − 1) + (1 − λ)(b1 − 1) = λa1 + (1 − λ)b1 − 1

Hence c ∈ X, and thus set X is convex.

Method 3: Plot of the set X is below:

Figure 2: Set X
We can see that set X is convex.

4. State whether the following problems are convex programs or not. Explain
your reasoning.
(a) min{x21 + 2x22 : x1 ≤ 0, x2 ≤ 0}.
(b) min{x1 · x2 : x21 + x22 ≤ 1}.
Pn xi Pn
(c) min{ i=1 i!i : i=1 xi ≥ 5}.

Solution:
(a) This is a convex program. The feasible region R2− is a convex set. The
objective f (x1 , x2 ) = x21 + 2x22 is a convex function over R2− because
it is sum of two convex functions g(x1 , x2 ) = x21 and h(x1 , x2 ) = 2x22 .
Alternatively it can be shown
 that function is convex by calculating
2 0
its Hessian ∇2 f =  0.
0 4

4
 (b) This is not a convex program. The feasible region f (x1 , x2 ) : x21 +
x22 ≤ 1 is a ball, and thus a convex set. However  objective
 function
0 1
x1 ẋ2 is not convex because its Hessian ∇2 f = is not positive
1 0
semidefinite.
Pn is a not a convex program. The feasible region {(x1 , ...., xn ) :
(c) This
i=1 xi ≥ 5}. is a half space and thus a convex set. The objective
Pn xi
function f (x1 , ...., xn ) = i=1 i!i is a not a convex function because
it not always a positive sum of convex functions.
 Forexample, when
0 0 0
n = 3 the objective function has ∇2 f = 0 1 0 . This matrix
0 0 x3
does not always have non-negative eigenvalues subject to the x1 +
x2 + x3 ≥ 5 constraint.

5. A quantity y is known to depend upon another quantity x. A set of n

data pairs {yi , xi }ni=1 has been collected.
(a) Formulate an optimization model for fitting the “best” straight line
y = a + bx to the data set, where best is with respect to the sum of
absolute deviations. What kind of an optimization model is it?
(b) Re-formulate the optimization model in part (a) where best is with
respect to the maximum absolute deviation. What kind of an opti-
mization model is it?
(c) Formulate an optimization model for fitting the “best” quadratic
curve y = a + bx + cx2 to the data set, where best is with respect
to the maximum absolute deviations. What kind of an optimization
model is it ?
Solution:
(a) The optimization problem can be directly formulated as
n
X
min | yi − a − bxi |
∀a,b∈R
i=1

which is an unconstrained convex nonlinear program.

(b) The optimization problem can be directly formulated as
min max | yi − a − bxi |
∀a,b∈R i=1,...,n

which is an unconstrained convex nonlinear program.

(c) The optimization problem can be directly formulated as
min max | yi − a − bxi − cx2i |
∀a,b,c∈R i=1,...,n

which is an unconstrained convex nonlinear program.