MATH 3310 Computational and Applied Mathematics

Final Exam Practice Exercises

1 FFT
1. Draw the butterfly diagram for F4 . Hence, show the procedure of computing y3 in FFT, where
(y0 , y1 , y2 , y3 )T = F4 (x0 , x1 , x2 , x3 )T .

2 Iterative method
2. Consider the linear system:

x1 + 2x2 2x3 = 1
x1 + x2 + x3 = 1
2x1 + 2x2 + x3 = 1

(a) Write down the Jacobi’s iterative method and Gauss-Seidel iterative method to solve the linear
system, in the form of xk+1 = Bxk + c. Starting with x0 = (0, 0, 0)T . Write down the first
two iterations for each iterative scheme.
(b) Determine whether the Jacobi’s method and the Gauss-Seidel’s method to solve the above
linear system converge.

3. Consider the real matrix: 0 1

1 0 0 a
B 1 1 0 b C
B
A=@ C.
2 1 1 c A
3 2 1 1
where a, b and c are positive real constants.

(a) Find a necessary and sufficient condition in term of a, b and c such that the Gauss-Seidel
method for solving the linear system Ax = b converges. Explain your answer.
(b) Prove that if a = b = c/11, then the Gauss-Seidel method for solving Ax = b converges in
finitely many iterations.

4. Consider the linear system Ax = b, where A is a SPD. Does the Jacobi method in solving the
linear system always converge? If not, find a counter example.
0 1
4 1 0 0
B 1 4 1 0 C
5. Let A = B C
@ 0 1 4 1 A. State the Householder-John theorem. Prove that the A is SPD. Using
0 0 1 4
the Householder-John theorem. Show that both the Jacobi method and Gauss-Seidel method
converge.

6. (a) Consider the linear system Ax = b where

0 1 01
2 1 1 4
A=@ 2 3 1 A b=@ 8 A
1 1 2 5

Write down the SOR iterative scheme to solve the linear system, in the form of xk+1 = Bxk +c.

1
 (b) Note that A is strictly diagonally dominant. In general, suppose A is a strictly diagonally
dominant n ⇥ n matrix. Prove that the SOR algorithm converges for 0 < !  1. (Hint: Prove
that all eigenvalues of B in the iterative scheme has magnitude straightly less than 1. You
may use without proof the fact that: the iterative scheme converges if the spectral radius
⇢(B) of B satisfies ⇢(B) < 1.)
7. Consider the linear system Ax = b (x 2 R3 ), where
0 1 0 1
4 3 0 1
A=@ 3 4 1 A and b = @ 2 A .
0 1 4 3

We solve the linear system using successive over-relaxation (SOR) method with an initial guess
x(0) = (0, 0, 0)T .

(a) Write down the numerical algorithm of the SOR method to solve the above linear system.
(b) Prove that the SOR method converges for solving the above linear system if and only if
0 < ! < 2 (! is the parameter in the SOR method).
(c) Find the optimal choice of ! such that the SOR method converges at the fastest rate. (Note :
You may use the theorems in the lecture notes without proof.)

8. Let A be an invertible n ⇥ n upper triangular matrix. Consider the linear system Ax = b.

(a) Show that both the Jacobi’s method and Gauss-Seidel’s method to solve the linear system
converge.
(b) Consider the SOR method with relaxation parameter !. Find the optimal ! so that the SOR
method has the fastest convergency rate.
(c) Now, consider the linear system AT x = b. Determine whether the the Jacobi’s method and
Gauss-Seidel’s method to solve the linear system converge.
0 1
1 a 1
9. Prove that A = @ a 1 a A is positive definite when 1/2 < a < 1. Also, find a necessary and
a a 1
sufficient condition on a such that the Jacobi method converges.
10. Let A be n ⇥ n real matrix with real eigenvalues 1 , 2 , ..., n (may not be distinct). Assume that
0 < ↵  1  2  ...  n  . Consider the iterative scheme:

xk+1 = xk + !(b Axk ), k = 0, 1, 2, ...

(a) Prove that the iterative scheme converges to the solution of Ax = b if 0 < ! < 2/ .
(b) Assume that 1 = 2 = ... = n. Prove that the iterative scheme converges in n iterations if
! = 1/ 1 .

3 Eigenvalues Problem
11. Let 0 1
3 0 0
A=@ 2 4 1 A
2 5 2

(a) Starting with x0 = (1, 1, 1)T . Compute all iterations of the power method to approximate
the dominant eigenvector of A. Does the power method converge in this case? Explain your
answer.
(b) By considering the power method on A2 with x0 = (1, 1, 1)T , determine the dominant eigen-
values of A and their corresponding eigenvectors. Explain your answer.

2
12. Find the QR factorization of 0 1
2 1 3
B=@ 1 0 7 A
0 1 1

13. Define the Rayleign quotient R(v, A). Find a upper bound and lower bound for R(v, A) in term
of the eigenvalues of A. Describe the RQI algorithm.
✓ ◆
3 2
14. Let A = . Describe the inverse power method. Compute the first two iterations of the
4 5
inverse power method. (You may use calculator for this example. In the final exam, the numbers
are nice and no calculator is needed.)
0 1
2 1 0
15. Let A = @ 1 3 1 A. We use inverse power method with shift to find the eigenvalue closest to
0 1 4
1.3. Describe the inverse power method with shift algorithm. Compute the first two iterations of
the inverse power method with shift. (You may use calculator for this example. In the final exam,
the numbers are nice and no calculator is needed.)
16. Describe how to use power method to find the smallest eigenvalues of a non-singular matrix A.
(Hint: Inverse power method with shift.)
17. Let 0 1
✓ ◆ 3 1 0
4 5
A= ; B=@ 1 3 0 A
6 5
0 0 2

(a) Starting with x0 = (1, 1)T . Compute the first two iterations of the power method to approx-
imate the dominant eigenvector of A.
(b) Starting with x0 = (1, 1, 1)T . Compute the first two iterations of the power method to
approximate the dominant eigenvector of B.
(c) Find the eigenvalues and eigenvectors of A and B. Determine whether the iterations in (a)
and (b) converge to the dominant eigenvector of A and B respectively. Explain your answer.
18. Consider an invertible n ⇥ n symmetric matrix A with eigenvalues 1, 2 , ..., n, where:

| 1| | 2| ... | n|

Let x1 , x2 , ..., xn be the eigenvectors associated with 1, 2 , ..., n respectively, which form a basis
of Rn .

(a) Show that if | 1 | = | 2 |, the power method with infinity norm does not converge in general.
(Hint: Show that if 1 = - 2 , the power method with infinity norm does not converges.)
(b) Determine whether the power method with infinity norm converge if 1 = 2 ( 1 > 0).
Explain you answer.

19. Let A 2 Mn⇥n (R) be real symmetric positive definite matrix with eigenvalues | 1| >| 2| | 3|
... | n |. Consider: x(k+1) = Ax(k) (k = 1, 2, ...). Prove that:

Ax(k) · x(k) 2 2k
= 1 + O(| | )
x(k) · x(k) 1

20. Let 0 1
1 3 2 1
B 1 1 4 1 C
A=B
@ 1
C
3 4 1 A
1 1 2 3
(a) Find the QR factorization of A.
(b) Using (a), solve the equation Ax = (1, 2, 3, 4)T .

3
 (c) Compute the first iteration of the QR method to approximate the eigenvalues of A.
21. Let: 0 1 0 1
3 2 1
A=@ 0 3 A and b = @ 3 A .
4 4 5
We consider the least square problem: find x that minimizes E(x) = ||Ax b||2 .

(a) Find the QR factorization of A. Hence, solve the above least square problem.
0 1
3 2 12
(b) Let B = @ 0 3 20 A.
4 4 9
Given that (12, 20, 9)T = ( 2, 3, 4)T ⇥ (3, 0, 4)T (cross product). Find the QR factorization
of B.
(c) Describe the QR method to approximate the eigenvalues of B. Using (b), compute the first
iteration of the QR method to approximate the eigenvalues of B.

22. Let {Ak }1

k=1 be the sequence obtained from the QR method on A. Show that

Ak = QT AQ

for some orthogonal matrix Q.

23. Does QR method always converge to a upper triangular matrix? Explain your answer.

4 Steepest descent and Conjugate gradient method

24. Consider the conjugate gradient method to solve the linear system Ax = b:
.

x(k+1) = x(k) + ↵k dk .

Show that hd1 , d2 i = 0 and r1 · r2 = 0.

25. Let {di }1 i 1
i=1 and {r }i=1 be the sequences of search direction and residual obtained by the conjugate
gradient method respectively. Show that:

span(d0 , d1 , ...dm ) = span(r0 , r1 , ..., rm )

for m = 0, 1, 2, .... (Hint: Use Mathematical Induction.)

26. Consider the linear system Ax = b, where A is a n ⇥ n symmetric positive definite real matrix and
b 2 Rn . The conjugate gradient descent algorithm can be described as follow:

x(k+1) = x(k) + ↵k d(k)

where ↵k 2 R is the time step and d(k) is the descent direction.

For the conjugate gradient descent algorithm, d(k+1) = r(k+1) + kd
(k)
. Suppose r(k) 6= 0. Prove
that:
r(k+1) · r(k+1)
k = .
r(k) · r(k)

5 Conformal mapping: optional

27. (Optional: Just for fun. Will not be covered in the final exam) Consider a connected domain ⌦ in
R2 , which is defined as:

⌦ = {(xex cos y yex sin y, xex sin y + yex cos y) 2 R2 : x2 + y 2  0.25}.