Scribd
Upload
29 views2 pages

HW10 Solutions

Uploaded by

teju1996cool
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
29 views2 pages

HW10 Solutions

Uploaded by

teju1996cool
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

APMA 1660 Homework 10 Solutions

Problem 1. Let K # (x, x′ ) and K ∗ (x, x′ ) be two functions defined on [0, 1] × [0, 1]. Please show the
following: If K # (x, x′ ) and K ∗ (x, x′ ) are kernels, then K(x, x′ ) = K # (x, x′ ) + K ∗ (x, x′ ) is a
kernel.
Solution. To show that K(x, x′ ) = K # (x, x′ ) + K ∗ (x, x′ ) is a kernel, given that both
K # (x, x′ ) and K ∗ (x, x′ ) are kernels, we need to verify that K(x, x′ ) satisfies the definition of
a kernel, namely that it is symmetric and positive semi-definite.

Symmetry of K(x, x′ ): Since K # (x, x′ ) and K ∗ (x, x′ ) are kernels, they are symmetric, i.e.,

K # (x, x′ ) = K # (x′ , x) and K ∗ (x, x′ ) = K ∗ (x′ , x) for all x, x′ ∈ [0, 1].

Given the symmetry of K # (x, x′ ) and K ∗ (x, x′ ), it follows that

K(x, x′ ) = K # (x, x′ ) + K ∗ (x, x′ ) = K # (x′ , x) + K ∗ (x′ , x) = K(x′ , x).

Hence, K(x, x′ ) is symmetric.


Positive Semi-definiteness of K(x, x′ ): Since K # (x, x′ ) and K ∗ (x, x′ ) are kernels, they are
positive semi-definite. Therefore, for any set of points x1 , . . . , xn ∈ [0, 1] and any coefficients
c1 , . . . , cn ∈ R, we have
Xn X n
ci cj K # (xi , xj ) ≥ 0,
i=1 j=1
n X
X n
ci cj K ∗ (xi , xj ) ≥ 0.
i=1 j=1

Consider any set of points x1 , . . . , xn ∈ [0, 1] and coefficients c1 , . . . , cn ∈ R. Then,


n X
X n n X
X n  
ci cj K(xi , xj ) = ci cj K # (xi , xj ) + K ∗ (xi , xj )
i=1 j=1 i=1 j=1

n X
X n n X
X n
= ci cj K # (xi , xj ) + ci cj K ∗ (xi , xj ).
i=1 j=1 i=1 j=1

Since both sums on the right-hand side are non-negative (due to the positive semi-definiteness
of K # (x, x′ ) and K ∗ (x, x′ )), their sum is also non-negative:
n X
X n
ci cj K(xi , xj ) ≥ 0.
i=1 j=1

Thus, since K(x, x′ ) is symmetric and positive semi-definite, it is a kernel.

1
Problem 2. Given the definitions of ĉĉ, U , V , λ and Y , show the following formula:

V )−1U T Y
U T U + λV
ĉ = (U

Solution. In the setup described, we are required to find the minimizer b


c for the optimization
problem involving the objective function

J(c) = (Y − U c)T (Y − U c) + λcT V c.

The objective function can be expanded as follows:

J(c) = Y T Y − Y T U c − cT U T Y + cT U T U c + λcT V c.

To find the minimizer, we take the gradient of J(c) with respect to c and set it to zero:

∇c J(c) = −2U T Y + 2U T U c + 2λV c = 0.

This simplifies to:


U T U c + λV c = U T Y .

Assuming the matrix U T U + λV is invertible, we can solve for c:

c = (U T U + λV )−1 U T Y .

This gives us the expression for b


c as required.

You might also like