0% found this document useful (0 votes)
315 views7 pages

HW1 Solution

The document is a homework assignment for Math 2050 that is due on October 28th and is worth 30 points. It contains 4 questions on topics including subspaces, generalized Cauchy-Schwartz inequalities, orthogonalization, and linear functions. The questions involve showing properties of sets and vectors, finding bases, expressing functions as scalar products, and determining gradients. Solutions to each part are provided.

Uploaded by

Lê Quân
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
315 views7 pages

HW1 Solution

The document is a homework assignment for Math 2050 that is due on October 28th and is worth 30 points. It contains 4 questions on topics including subspaces, generalized Cauchy-Schwartz inequalities, orthogonalization, and linear functions. The questions involve showing properties of sets and vectors, finding bases, expressing functions as scalar products, and determining gradients. Solutions to each part are provided.

Uploaded by

Lê Quân
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 7

Math 2050 Fall 2022

Homework 1

Homework 1 is due by Friday, 28th Oct at the recitation class. The total number of points is 30.

1 Subspaces

1. (2 points) Consider the set S of points x ∈ R3 such that

x1 + 2x2 + 3x3 = 0, 3x1 + 2x2 + x3 = 0.

Show that S is a subspace. Determine its dimension, and find a basis for it.
Solution:
By solving the system of 2 linear equations, we obtain:
(
x1 = x3
x2 = −2x1
So for every x ∈ S, x = (x1 , −2x1 , x1 ) = x1 (1, −2, 1)
Therefore, S is a subspace of R3 , has dimension 1 and S = span(1, −2, 1).
2. (4 points) Consider the set in R3 , defined by the equation

P := {x ∈ R3 : x1 + 2x2 + 3x3 = 1}.

(a) Show that the set P is an affine subspace of dimension 2. To this end, express it as x0 +
span(x1 , x2 ), where x0 in P , and x1 , x2 are independent vectors.
Solution:
We have: x1 = 1 − 2x2 − 3x3
So x = (1 − 2x2 − 3x3 , x2 , x3 ) = (1, 0, 0) + x2 (−2, 1, 0) + x3 (−3, 0, 1)
Furthermore, the origin is not in P so P is an affine subspace of dimension 2.
(b) Find the minimum Euclidean distance from 0 to the set P . Find a point that achieves the
minimum distance. (Hint: using the Cauchy-Schwartz inequality, prove that the minimum-
distance point is proportional to a := (1, 2, 3)).
Solution:
We need to find the minimum of x21 + x22 + x23 such that x1 + 2x2 + 3x3 = 1
Apply Cauchy-Schwartz inequality for x = (x1 , x2 , x3 ) and a = (1, 2, 3):

(xT a)2 ≤ ∥x∥2 . ∥a∥2

This is equivalent to
(x1 + 2x2 + 3x3 )2 ≤ 14. ∥x∥2
√ √
Therefore ∥x∥ ≥ 1/ 14, and ∥x∥ = 1/ 14 when x proportional to a = (1, 2, 3).
We can find this point by assuming x is of the form x = α(1, 2, 3) for some α ∈ R, to be
1
determined as follows. Replace the coordinate of x in x1 + 2x2 + 3x3 = 1, we obtain α = 14
1
so x = 14 (1, 2, 3).

1
2 Generalized Cauchy-Schwartz inequalities

1. (2 points)
Show that the following inequalities hold for any vector x:

∥x∥∞ ≤ ∥x∥1 ≤ n ∥x∥∞ .


Solution:
This is obvious, it follows from the definition of these norms.

2. (2 points) Show that following inequalities hold for any vector x:


∥x∥2 ≤ ∥x∥1 ≤ n ∥x∥2 .
Hint: use the Cauchy-Schwartz inequality for the second inequality.
Solution:
For the first inequality, we just need to square both sides.
For the second inequality: Apply Cauchy-Schwartz inequality for x and a = (1, . . . , 1), we have:

|xT a| ≤ ∥x∥2 . ∥a∥2


This is equivalent to: √
∥x∥1 ≤ n ∥x∥2

3. (2 points) In a generalized version of the above inequalities, show that for any non-zero vector x,

∥x∥21
≤ Card(x),
∥x∥22
where Card(x) is the cardinality of the vector x, defined as the number of non-zero elements in x.
For which vectors x is the equality attained?
Solution:

Consider vector a such that, for i ∈ {1, . . . , n}:


(
ai = 1 if xi ̸= 0,
ai = 0 if xi = 0.

Apply Cauchy-Schwartz inequality to x and a, we have:

∥x∥21 ≤ ∥x∥22 .Card(x).

The equality attained when all the coordinates that different from 0 of x are equal.

2
3 Orthogonalization

(2 points) Let x, y ∈ Rn be two unit-norm vectors, that is, such that ∥x∥2 = ∥y∥2 = 1. Show that the
vectors x − y and x + y are orthogonal. Use this to find an orthogonal basis for the subspace spanned by
x and y.
Solution:
We calculate the inner product of x − y and x + y:

(x − y)T (x + y) = ∥x∥22 + (xT y − y T x) − ∥y∥22 = 0.

We also have
1
x = [(x + y) + (x − y)],
2
1
y = [(x + y) − (x − y)].
2
Therefore, x − y and x + y is an orthogonal basis for the subspace spanned by x and y.

3
4 Linear Functions

1. (2 points) For a n-vector x, with n = 2m − 1 odd, we define the median of x as xm . Now consider
the function f : Rn → R, with values

n
1X
f (x) = xm − xi .
n i=1

Express f as a scalar product, that is, find a in Rn such that f (x) = aT x for every x. Find a basis
for the set of points x such that f (x) = 0.
Solution:
Vector a satisfies:
(
− n1 if i ̸= m
ai = n−1
n
if i = m

For the second part, we have f (x) = 0 so:


n
1X
xm = xi
n i=1

equivalent to:
n
1 X
xm = xi
n − 1 i=1,i̸=m

Therefore, the basis of the space orthogonal to a is the set of n − 1 vectors which are the column
vectors of the following matrix (n × (n − 1) dimensions):
 
1 0 ... 0
 0 1 ... 0 
 
... ... ... ...
A=
 1 1 1 

 n−1 n−1
... n−1 
... ... ... ...
0 0 ... 1
1
where n−1
is the value for all elements of row m. For example, when n = 5:
 
1 0 0 0
0 1 0 0
1 1 1 1

A=
4 4 4 4
.
0 0 1 0 
0 0 0 1

2. (2 points) For α in R2 , we consider the ”power law” function f : R2++ → R, with values

f (x) = xα1 1 xα2 2 .

4
Justify the statement: ”the coefficients αi provide the ratio between the relative error in f to a relative
error in xi ”.
Solution:
We have:
df ∆f
= α1 xα1 1 −1 xα2 2 =
dx1 ∆x1
Thus:
∆f x1 α1 .f
= = α1
f ∆x1 f
Similarly, we obtain:
∆f x2
= α2
f ∆x2

3. (2 points) Find the gradient of the function f : R2 → R that gives the distance from a given point p
in R2 to a point x in R2 .
Solution:
Let p = (p1 , p2 ), x = (x1 , x2 ). We have:
p
f (x1 , x2 ) = g(x), where g(x) := (x1 − p1 )2 + (x2 − p2 )2 .

We have
∇g = (2(x1 − p1 ), 2(x2 − p2 )) = 2(x − p).
Since g(x) = f (x)2 , we have
∇g(x) = 2f (x)∇f (x).
Therefore:
1 x−p
∇f = ((x1 − p1 ), (x2 − p2 )) = .
f ∥x − p∥

5
5 Norm and Distance

(1) (3 points) Nearest point to a line. Let a and b be the different n-vectors. The line passing through a
and b is given by the set of vectors of the norm (1 − θ)a + θb, where θ is a scalar that determines the
particular point on the line.
Let x be any n-vector. Find a formula for the point p on the line that is closest (in the Euclidean norm
sense) to x. The point p is called the projection of x onto the line. Show that (p − x) ⊥ (a − b), and
draw a simple picture illustrating this in 2-D. Hint. Work with the square of the distance between a
point on the line and x, i.e. ∥(1 − θ)a + θb − x∥2 . Expand this, and minimize over θ.
Solution:

Let:
f (θ) = ∥(1−θ)a+θb−x∥2 = (1−θ)2 ∥a∥2 +θ2 ∥b∥2 +∥x∥2 +2θ(1−θ)aT b−2(1−θ)aT x−2θbT x

f ′ (θ) = −2(1 − θ) ∥a∥2 + 2θ ∥b∥2 + 2abT (1 − 2θ) + 2axT − 2bxT

If f ′ = 0, we have:
(aT − xT )(a − b)
θ=
∥a − b∥2
Thus:
(xT − bT )(a − b)
1−θ =
∥a − b∥2
Inner product of a − b and p − x as follows:
(a − b)T (p − x) = (xT − bT )a + (aT − xT )b − (aT − bT )x = 0

(2) (3 points) Regression model sensitivity. Consider the regression model ŷ = xT β + v, where ŷ is the
prediction, x is a feature vector, β is a coefficient vector, and v is the offset term. If x and x̃ are
feature vectors with corresponding prediction ŷ and ỹ, show that |ŷ − ỹ| ≤ ∥β∥∥x − x̃∥. This means
that when ∥β∥ is small, the prediction is not very sensitive to a change in the feature vector.
Solution:
We have |ŷ − ỹ| = |(xT β + v) − (x̃T β + v)| = |(x − x̃)T β|. Apply the Cauchy-Schwartz inequality,
we obtain:
|ŷ − ỹ| = |(x − x̃)T β| ≤ ∥x − x̃∥ . ∥β∥ .

6
(3) (3 points) Changes in coefficient vector in a scoring function. We consider a scoring function f
defined as
f : x ∈ Rn → aT x + b
where a ∈ Rn , b ∈ R. Assume that the coefficients of the vector a are only known up to intervals.
Precisely, for every i ∈ {1, . . . , n}, we have |ai − âi | ≤ σi , where â ∈ Rn and σ ∈ Rn+ are given.
For a given x ∈ Rn , what are the corresponding largest and smallest values f (x) can take? It may
be useful to define |x|, the n-vector with entries equal to the absolute values in x.
Solution:
We just need to find the maximum and minimum of aT x. We have:

|ai − âi | ≤ σi , i = 1, . . . , n.

Multiplying both sides with |xi |, we obtain:

|(ai − âi )xi | ≤ σi |xi |, i = 1, . . . , n.

Thus:
âi xi − σi |xi | ≤ ai xi ≤ âi xi + σi |xi |, i = 1, . . . , n.
Summing:
n
X n
X n
X
(âi xi − σi |xi |) ≤ (ai xi ) ≤ (âi xi + σi |xi |).
i=1 i=1 i=1

Let |x| = (|x1 |, |x2 |, . . . , |xn |). We can write

âT x − σ T |x| ≤ aT x ≤ âT x + σ T |x|.

You might also like