Nonlinear Programming, Chapter 2, Part II.

Quadratic
Functions

Majid Soleimani-damaneh

University of Tehran

Winter 2023

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 1 / 14
Main Reference

The main reference of this presentation is

Amir Beck, Introduction to Nonlinear Optimization: Theory, Algorithms,
and Applications with MATLAB, MOS-SIAM Series on Optimization,
SIAM, Philadelphia, 2014.

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 2 / 14
 Quadratic Functions

Quadratic Functions
Definition 1
A quadratic function over Rn is a function of the form
f (x) = x T Ax + 2b T x + c (∗)
where A ∈ R n×n is a symmetric matrix, b ∈ Rn , and c ∈ R.

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 3 / 14
 Quadratic Functions

Quadratic Functions
Definition 1
A quadratic function over Rn is a function of the form
f (x) = x T Ax + 2b T x + c (∗)
where A ∈ R n×n is a symmetric matrix, b ∈ Rn , and c ∈ R.

Theorem 1
Consider f (x) = x T Ax + 2b T x + c with symmetric A. For each x ∈ Rn ,

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 3 / 14
 Quadratic Functions

Quadratic Functions
Definition 1
A quadratic function over Rn is a function of the form
f (x) = x T Ax + 2b T x + c (∗)
where A ∈ R n×n is a symmetric matrix, b ∈ Rn , and c ∈ R.

Theorem 1
Consider f (x) = x T Ax + 2b T x + c with symmetric A. For each x ∈ Rn ,
Xn Xn n
X
(i) f (x) = aij xi xj + 2 bi xi + c.
i=1 j=1 i=1

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 3 / 14
 Quadratic Functions

Quadratic Functions
Definition 1
A quadratic function over Rn is a function of the form
f (x) = x T Ax + 2b T x + c (∗)
where A ∈ R n×n is a symmetric matrix, b ∈ Rn , and c ∈ R.

Theorem 1
Consider f (x) = x T Ax + 2b T x + c with symmetric A. For each x ∈ Rn ,
Xn Xn n
X
(i) f (x) = aij xi xj + 2 bi xi + c.
i=1 j=1 i=1
(ii) ∇f (x) = 2(Ax + b).

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 3 / 14
 Quadratic Functions

Quadratic Functions
Definition 1
A quadratic function over Rn is a function of the form
f (x) = x T Ax + 2b T x + c (∗)
where A ∈ R n×n is a symmetric matrix, b ∈ Rn , and c ∈ R.

Theorem 1
Consider f (x) = x T Ax + 2b T x + c with symmetric A. For each x ∈ Rn ,
Xn Xn n
X
(i) f (x) = aij xi xj + 2 bi xi + c.
i=1 j=1 i=1
(ii) ∇f (x) = 2(Ax + b).
(iii) ∇2 f (x) = 2A (Independent of x).

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 3 / 14
 Quadratic Functions

Quadratic Functions
Definition 1
A quadratic function over Rn is a function of the form
f (x) = x T Ax + 2b T x + c (∗)
where A ∈ R n×n is a symmetric matrix, b ∈ Rn , and c ∈ R.

Theorem 1
Consider f (x) = x T Ax + 2b T x + c with symmetric A. For each x ∈ Rn ,
Xn Xn n
X
(i) f (x) = aij xi xj + 2 bi xi + c.
i=1 j=1 i=1
(ii) ∇f (x) = 2(Ax + b).
(iii) ∇2 f (x) = 2A (Independent of x).

Proof.
In class.
Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 3 / 14
 Quadratic Functions

Example 1
f (x) = x12 − 6x22 + 3x32 + x1 x2 − 5x2 x3 + 6x1 + 7x2 + x3 + 20 is a
quadratic function.

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 4 / 14
 Quadratic Functions

Example 1
f (x) = x12 − 6x22 + 3x32 + x1 x2 − 5x2 x3 + 6x1 + 7x2 + x3 + 20 is a
quadratic function.
 
2x1 + x2 + 6
∇f (x) = −12x2 + x1 − 5x3 + 7
6x3 − 5x2 + 1

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 4 / 14
 Quadratic Functions

Example 1
f (x) = x12 − 6x22 + 3x32 + x1 x2 − 5x2 x3 + 6x1 + 7x2 + x3 + 20 is a
quadratic function.
 
2x1 + x2 + 6
∇f (x) = −12x2 + x1 − 5x3 + 7
6x3 − 5x2 + 1
1 12 3
     
0 x1
1 −5    7
=2 2 −6 x 2 + 2 
 
2
0 −5 2 3 x3 1
2

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 4 / 14
 Quadratic Functions

Example 1
f (x) = x12 − 6x22 + 3x32 + x1 x2 − 5x2 x3 + 6x1 + 7x2 + x3 + 20 is a
quadratic function.
 
2x1 + x2 + 6
∇f (x) = −12x2 + x1 − 5x3 + 7
6x3 − 5x2 + 1
1 12 3
     
0 x1
1 −5    7
=2 2 −6 x 2 + 2 
 
2
0 −5 2 3 x3 1
2
1
 
1 2 0
∇2 f (x) = 2  12 −6 −5 2

−5
0 2 3

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 4 / 14
 Quadratic Functions

Example 1
f (x) = x12 − 6x22 + 3x32 + x1 x2 − 5x2 x3 + 6x1 + 7x2 + x3 + 20 is a
quadratic function.
 
2x1 + x2 + 6
∇f (x) = −12x2 + x1 − 5x3 + 7
6x3 − 5x2 + 1
1 12 3
     
0 x1
1 −5    7
=2 2 −6 x 2 + 2 
  
2
0 −5 2 3 x3 1
2
1
 
1 2 0
∇2 f (x) = 2  12 −6 −5 2

−5
0 2 3
Indeed,   
1 12 0
 
x1 x1
1 −5 
f (x) = (x1 , x2 , x3 )  2 −6 2  x2  + 2(3, 27 , 12 ) x2  + 20
0 −52 3 x3 x3
Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 4 / 14
 Quadratic Functions

Why quadratic functions deserve special attention?

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 5 / 14
 Quadratic Functions

An important quadratic function

Quadratic function in least squares problems

Example 2
If A ∈ Rm×n and b ∈ Rm , then f (x) = ∥Ax − b∥2 is a quadratic function,
because

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 6 / 14
 Quadratic Functions

An important quadratic function

Quadratic function in least squares problems

Example 2
If A ∈ Rm×n and b ∈ Rm , then f (x) = ∥Ax − b∥2 is a quadratic function,
because
f (x) = ∥Ax − b∥2
= (Ax − b)T (Ax − b)

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 6 / 14
 Quadratic Functions

An important quadratic function

Quadratic function in least squares problems

Example 2
If A ∈ Rm×n and b ∈ Rm , then f (x) = ∥Ax − b∥2 is a quadratic function,
because
f (x) = ∥Ax − b∥2
= (Ax − b)T (Ax − b)
= (x T AT − b T )(Ax − b)

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 6 / 14
 Quadratic Functions

An important quadratic function

Quadratic function in least squares problems

Example 2
If A ∈ Rm×n and b ∈ Rm , then f (x) = ∥Ax − b∥2 is a quadratic function,
because
f (x) = ∥Ax − b∥2
= (Ax − b)T (Ax − b)
= (x T AT − b T )(Ax − b)
= x T AT Ax − x T AT b − b T Ax + b T b

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 6 / 14
 Quadratic Functions

An important quadratic function

Quadratic function in least squares problems

Example 2
If A ∈ Rm×n and b ∈ Rm , then f (x) = ∥Ax − b∥2 is a quadratic function,
because
f (x) = ∥Ax − b∥2
= (Ax − b)T (Ax − b)
= (x T AT − b T )(Ax − b)
= x T AT Ax − x T AT b − b T Ax + b T b
= x T (AT A)x − 2b T Ax + ∥b∥2

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 6 / 14
 Quadratic Functions

An important quadratic function

Quadratic function in least squares problems

Example 2
If A ∈ Rm×n and b ∈ Rm , then f (x) = ∥Ax − b∥2 is a quadratic function,
because
f (x) = ∥Ax − b∥2
= (Ax − b)T (Ax − b)
= (x T AT − b T )(Ax − b)
= x T AT Ax − x T AT b − b T Ax + b T b
= x T (AT A)x − 2b T Ax + ∥b∥2
= x T (AT A)x + 2(−AT b)T x + ∥b∥2 .

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 6 / 14
 Quadratic Functions

Remark 1
If f (x) = x T Ax + 2b T x + c, while A ∈ Rn×n is NOT symmetric, then
one can replace A with a symmetric matrix.

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 7 / 14
 Quadratic Functions

Remark 1
If f (x) = x T Ax + 2b T x + c, while A ∈ Rn×n is NOT symmetric, then
one can replace A with a symmetric matrix.

▶ We have
x T Ax = (x T Ax)T

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 7 / 14
 Quadratic Functions

Remark 1
If f (x) = x T Ax + 2b T x + c, while A ∈ Rn×n is NOT symmetric, then
one can replace A with a symmetric matrix.

▶ We have
x T Ax = (x T Ax)T = x T AT x

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 7 / 14
 Quadratic Functions

Remark 1
If f (x) = x T Ax + 2b T x + c, while A ∈ Rn×n is NOT symmetric, then
one can replace A with a symmetric matrix.

▶ We have
1 T
x T Ax = (x T Ax)T = x T AT x = x Ax + 12 x T AT x
2

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 7 / 14
 Quadratic Functions

Remark 1
If f (x) = x T Ax + 2b T x + c, while A ∈ Rn×n is NOT symmetric, then
one can replace A with a symmetric matrix.

▶ We have
1 T A + AT
x T Ax = (x T Ax)T = x T AT x = x Ax + 12 x T AT x = x T ( )x.
2 2

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 7 / 14
 Quadratic Functions

Remark 1
If f (x) = x T Ax + 2b T x + c, while A ∈ Rn×n is NOT symmetric, then
one can replace A with a symmetric matrix.

▶ We have
1 T A + AT
x T Ax = (x T Ax)T = x T AT x = x Ax + 12 x T AT x = x T ( )x.
2 2
A + AT A + AT
▶ So, f (x) = x T ( )x + 2b T x + c in which is symmetric.
2 2

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 7 / 14
 Quadratic Functions

Remark 1
If f (x) = x T Ax + 2b T x + c, while A ∈ Rn×n is NOT symmetric, then
one can replace A with a symmetric matrix.

▶ We have
1 T A + AT
x T Ax = (x T Ax)T = x T AT x = x Ax + 12 x T AT x = x T ( )x.
2 2
A + AT A + AT
▶ So, f (x) = x T ( )x + 2b T x + c in which is symmetric.
2 2
Therefore, any function with the form f (x) = x T Ax + 2b T x + c, even
if A is not symmetric, can be represented as (∗) satisfying Definition
1, and it is a quadratic function.

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 7 / 14
 Quadratic Functions

Theorem 2
Consider the quadratic function f (x) = x T Ax + 2b T x + c, where
A ∈ Rn×n is a symmetric matrix, b ∈ Rn , and c ∈ R. Then:

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 8 / 14
 Quadratic Functions

Theorem 2
Consider the quadratic function f (x) = x T Ax + 2b T x + c, where
A ∈ Rn×n is a symmetric matrix, b ∈ Rn , and c ∈ R. Then:
(i) x̄ is a stationary point of f if and only if Ax̄ = −b.

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 8 / 14
 Quadratic Functions

Theorem 2
Consider the quadratic function f (x) = x T Ax + 2b T x + c, where
A ∈ Rn×n is a symmetric matrix, b ∈ Rn , and c ∈ R. Then:
(i) x̄ is a stationary point of f if and only if Ax̄ = −b.
(ii) If A ⪰ 0, then x ∗ is a global minimizer of f over Rn if and only if
Ax ∗ = −b.

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 8 / 14
 Quadratic Functions

Theorem 2
Consider the quadratic function f (x) = x T Ax + 2b T x + c, where
A ∈ Rn×n is a symmetric matrix, b ∈ Rn , and c ∈ R. Then:
(i) x̄ is a stationary point of f if and only if Ax̄ = −b.
(ii) If A ⪰ 0, then x ∗ is a global minimizer of f over Rn if and only if
Ax ∗ = −b.
(iii) If A ≻ 0, then x ∗ = −A−1 b is a strict global minimizer of f over Rn .

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 8 / 14
 Quadratic Functions

Theorem 2
Consider the quadratic function f (x) = x T Ax + 2b T x + c, where
A ∈ Rn×n is a symmetric matrix, b ∈ Rn , and c ∈ R. Then:
(i) x̄ is a stationary point of f if and only if Ax̄ = −b.
(ii) If A ⪰ 0, then x ∗ is a global minimizer of f over Rn if and only if
Ax ∗ = −b.
(iii) If A ≻ 0, then x ∗ = −A−1 b is a strict global minimizer of f over Rn .
(iv) If A ⪯ 0, then x ∗ is a global maximizer of f over Rn if and only if
Ax ∗ = −b.

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 8 / 14
 Quadratic Functions

Theorem 2
Consider the quadratic function f (x) = x T Ax + 2b T x + c, where
A ∈ Rn×n is a symmetric matrix, b ∈ Rn , and c ∈ R. Then:
(i) x̄ is a stationary point of f if and only if Ax̄ = −b.
(ii) If A ⪰ 0, then x ∗ is a global minimizer of f over Rn if and only if
Ax ∗ = −b.
(iii) If A ≻ 0, then x ∗ = −A−1 b is a strict global minimizer of f over Rn .
(iv) If A ⪯ 0, then x ∗ is a global maximizer of f over Rn if and only if
Ax ∗ = −b.
(v) If A ≺ 0, then x ∗ = −A−1 b is a strict global maximizer of f over Rn .

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 8 / 14
 Quadratic Functions

Theorem 2
Consider the quadratic function f (x) = x T Ax + 2b T x + c, where
A ∈ Rn×n is a symmetric matrix, b ∈ Rn , and c ∈ R. Then:
(i) x̄ is a stationary point of f if and only if Ax̄ = −b.
(ii) If A ⪰ 0, then x ∗ is a global minimizer of f over Rn if and only if
Ax ∗ = −b.
(iii) If A ≻ 0, then x ∗ = −A−1 b is a strict global minimizer of f over Rn .
(iv) If A ⪯ 0, then x ∗ is a global maximizer of f over Rn if and only if
Ax ∗ = −b.
(v) If A ≺ 0, then x ∗ = −A−1 b is a strict global maximizer of f over Rn .
(vi) If A is indefinite and Ax ∗ = −b, then x ∗ is a saddle point of f over
Rn .

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 8 / 14
 Quadratic Functions

Theorem 2
Consider the quadratic function f (x) = x T Ax + 2b T x + c, where
A ∈ Rn×n is a symmetric matrix, b ∈ Rn , and c ∈ R. Then:
(i) x̄ is a stationary point of f if and only if Ax̄ = −b.
(ii) If A ⪰ 0, then x ∗ is a global minimizer of f over Rn if and only if
Ax ∗ = −b.
(iii) If A ≻ 0, then x ∗ = −A−1 b is a strict global minimizer of f over Rn .
(iv) If A ⪯ 0, then x ∗ is a global maximizer of f over Rn if and only if
Ax ∗ = −b.
(v) If A ≺ 0, then x ∗ = −A−1 b is a strict global maximizer of f over Rn .
(vi) If A is indefinite and Ax ∗ = −b, then x ∗ is a saddle point of f over
Rn .

Proof.
In class.
Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 8 / 14
 Quadratic Functions

Remark 2
When A ≻ 0, the vector x ∗ = −A−1 b is a strict global minimizer of f over
Rn . Hence, the minimum value of f equals
f (−A−1 b)

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 9 / 14
 Quadratic Functions

Remark 2
When A ≻ 0, the vector x ∗ = −A−1 b is a strict global minimizer of f over
Rn . Hence, the minimum value of f equals
f (−A−1 b)= (−A−1 b)T A(−A−1 b) + 2b T (−A−1 b) + c

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 9 / 14
 Quadratic Functions

Remark 2
When A ≻ 0, the vector x ∗ = −A−1 b is a strict global minimizer of f over
Rn . Hence, the minimum value of f equals
f (−A−1 b)= (−A−1 b)T A(−A−1 b) + 2b T (−A−1 b) + c
= b T (A−1 )T AA−1 b − 2b T A−1 b + c

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 9 / 14
 Quadratic Functions

Remark 2
When A ≻ 0, the vector x ∗ = −A−1 b is a strict global minimizer of f over
Rn . Hence, the minimum value of f equals
f (−A−1 b)= (−A−1 b)T A(−A−1 b) + 2b T (−A−1 b) + c
= b T (A−1 )T AA−1 b − 2b T A−1 b + c= c − b T A−1 b.

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 9 / 14
 Quadratic Functions

Due to Theorem 2, solving the unconstrained quadratic problems is

equivalent to solving the linear system Ax = −b. Not difficult! It is
polynomially solvable.

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 10 / 14
 Quadratic Functions

Due to Theorem 2, solving the unconstrained quadratic problems is

equivalent to solving the linear system Ax = −b. Not difficult! It is
polynomially solvable.
If A ⪰ 0, then the quadratic function f (x) = x T Ax + 2b T x + c is a
convex function (We will prove it later).

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 10 / 14
 Quadratic Functions

Due to Theorem 2, solving the unconstrained quadratic problems is

equivalent to solving the linear system Ax = −b. Not difficult! It is
polynomially solvable.
If A ⪰ 0, then the quadratic function f (x) = x T Ax + 2b T x + c is a
convex function (We will prove it later).
For constrained quadratic problems (quadratic objective function over
linear constraints), the convex case is polynomially solvable. But for
nonconvex case (Indefinite Hessian), the matter is different.

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 10 / 14
 Quadratic Functions

Due to Theorem 2, solving the unconstrained quadratic problems is

equivalent to solving the linear system Ax = −b. Not difficult! It is
polynomially solvable.
If A ⪰ 0, then the quadratic function f (x) = x T Ax + 2b T x + c is a
convex function (We will prove it later).
For constrained quadratic problems (quadratic objective function over
linear constraints), the convex case is polynomially solvable. But for
nonconvex case (Indefinite Hessian), the matter is different.
See for instance
▶ P.M. Pardalos and S.A. Vavasis, Quadratic programming with one
negative eigenvalue is NP-hard. Journal of Global Optimization,
1(1):15-22, 1991.

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 10 / 14
 Quadratic Functions

Due to Theorem 2, solving the unconstrained quadratic problems is

equivalent to solving the linear system Ax = −b. Not difficult! It is
polynomially solvable.
If A ⪰ 0, then the quadratic function f (x) = x T Ax + 2b T x + c is a
convex function (We will prove it later).
For constrained quadratic problems (quadratic objective function over
linear constraints), the convex case is polynomially solvable. But for
nonconvex case (Indefinite Hessian), the matter is different.
See for instance
▶ P.M. Pardalos and S.A. Vavasis, Quadratic programming with one
negative eigenvalue is NP-hard. Journal of Global Optimization,
1(1):15-22, 1991.
▶▶ S. Burer and A.N. Letchford, On nonconvex quadratic
programming with box constraints. SIAM Journal on Optimization,
20(2):1073-1089, 2009.

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 10 / 14
 Quadratic Functions

Due to Theorem 2, solving the unconstrained quadratic problems is

equivalent to solving the linear system Ax = −b. Not difficult! It is
polynomially solvable.
If A ⪰ 0, then the quadratic function f (x) = x T Ax + 2b T x + c is a
convex function (We will prove it later).
For constrained quadratic problems (quadratic objective function over
linear constraints), the convex case is polynomially solvable. But for
nonconvex case (Indefinite Hessian), the matter is different.
See for instance
▶ P.M. Pardalos and S.A. Vavasis, Quadratic programming with one
negative eigenvalue is NP-hard. Journal of Global Optimization,
1(1):15-22, 1991.
▶▶ S. Burer and A.N. Letchford, On nonconvex quadratic
programming with box constraints. SIAM Journal on Optimization,
20(2):1073-1089, 2009.
▶▶▶ M. Zamani, New bounds for nonconvex quadratically
constrained quadratic programming. Journal of Global Optimization,
85:595-613, 2023.
Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 10 / 14
 Quadratic Functions

Theorem 3
(Coerciveness of quadratic functions):

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 11 / 14
 Quadratic Functions

Theorem 3
(Coerciveness of quadratic functions): Consider the quadratic function
f (x) = x T Ax + 2b T x + c, where A ∈ Rn×n is a symmetric matrix,
b ∈ Rn , and c ∈ R. Then:

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 11 / 14
 Quadratic Functions

Theorem 3
(Coerciveness of quadratic functions): Consider the quadratic function
f (x) = x T Ax + 2b T x + c, where A ∈ Rn×n is a symmetric matrix,
b ∈ Rn , and c ∈ R. Then: f is coercive if and only if A ≻ 0.

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 11 / 14
 Quadratic Functions

Theorem 3
(Coerciveness of quadratic functions): Consider the quadratic function
f (x) = x T Ax + 2b T x + c, where A ∈ Rn×n is a symmetric matrix,
b ∈ Rn , and c ∈ R. Then: f is coercive if and only if A ≻ 0.

Proof.
In class.

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 11 / 14
 Quadratic Functions

Theorem 3
(Coerciveness of quadratic functions): Consider the quadratic function
f (x) = x T Ax + 2b T x + c, where A ∈ Rn×n is a symmetric matrix,
b ∈ Rn , and c ∈ R. Then: f is coercive if and only if A ≻ 0.

Proof.
In class.

Corollary 1
Consider the quadratic function f (x) = x T Ax + 2b T x + c, where
A ∈ Rn×n is a symmetric matrix, b ∈ Rn , and c ∈ R.

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 11 / 14
 Quadratic Functions

Theorem 3
(Coerciveness of quadratic functions): Consider the quadratic function
f (x) = x T Ax + 2b T x + c, where A ∈ Rn×n is a symmetric matrix,
b ∈ Rn , and c ∈ R. Then: f is coercive if and only if A ≻ 0.

Proof.
In class.

Corollary 1
Consider the quadratic function f (x) = x T Ax + 2b T x + c, where
A ∈ Rn×n is a symmetric matrix, b ∈ Rn , and c ∈ R.
If A ≻ 0, then f attains its global minimum over any nonempty closed
subset of Rn .

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 11 / 14
 Quadratic Functions

Theorem 4
(Characterization of the nonnegativity of quadratic functions):

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 12 / 14
 Quadratic Functions

Theorem 4
(Characterization of the nonnegativity of quadratic functions):
Consider the quadratic function f (x) = x T Ax + 2b T x + c, where
A ∈ Rn×n is a symmetric matrix, b ∈ Rn , and c ∈ R. Then:

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 12 / 14
 Quadratic Functions

Theorem 4
(Characterization of the nonnegativity of quadratic functions):
Consider the quadratic function f (x) = x T Ax + 2b T x + c, where
A ∈ Rn×n is a symmetric matrix,
 b∈ Rn , and c ∈ R. Then: f (x) ≥ 0 for
A b
each x ∈ Rn if and only if T ⪰ 0.
b c

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 12 / 14
 Quadratic Functions

Theorem 4
(Characterization of the nonnegativity of quadratic functions):
Consider the quadratic function f (x) = x T Ax + 2b T x + c, where
A ∈ Rn×n is a symmetric matrix,
 b∈ Rn , and c ∈ R. Then: f (x) ≥ 0 for
A b
each x ∈ Rn if and only if T ⪰ 0.
b c

Proof.
In class.

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 12 / 14
 Quadratic Functions

Theorem 4
(Characterization of the nonnegativity of quadratic functions):
Consider the quadratic function f (x) = x T Ax + 2b T x + c, where
A ∈ Rn×n is a symmetric matrix,
 b∈ Rn , and c ∈ R. Then: f (x) ≥ 0 for
A b
each x ∈ Rn if and only if T ⪰ 0.
b c

Proof.
In class.

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 12 / 14
 References

Main Reference

A. Beck, Introduction to Nonlinear Optimization, MOS-SIAM series in

Optimization, 2014.

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 13 / 14
 Thanks

Majid Soleimani-damaneh (UT) NLP, Chapter 2, Part II. Quadratic Functions Winter 2023 14 / 14