MATH-GA 2010.001/CSCI-GA 2420.

001

Fall 2024: Numerical Methods I

Assignment 1 (due Sep. 23, 2024 before class)

Objectives. This class is for you and you should try to get the most out of it for yourself. The
methods we study are the basis for more complicated algorithms and will help you to understand
functions that are available in numerical software packages. The problems are meant to be challenging
and to leave room for discussion. You are not expected to get all of them 100% right. Please make
sure you discuss your results whenever that is required; a meaningful discussion that illustrates that
you understand the point is usually more important than obtaining the correct result.
Homework submission. Homework assignments must be before 5.10pm on the day of the listed
due date. Use LATEX typesetting (try LyX if you are not yet familiar with LATEX) for your homework.
You need to upload your homework as a single PDF to Brightspace. If you are required to hand in
code listings (together with the PDF), this will explicitly be stated on that homework assignment.
Collaboration, and acknowledging sources. NYU’s integrity policies will be enforced. You are
encouraged to discuss the problems with other students. However, you must write (i.e., type) every
line of code yourself and also write up your solutions independently. Copying of any portion of someone
else’s solution/code or allowing others to copy your solution/code is considered cheating. Please cite
every source you use for your homework, even if it is just Wikipedia.
Plotting and formatting. Plot figures carefully and think about what you want to illustrate with
a plot. Choose proper ranges and scales (semilogx, semilogy, loglog), and always label axes.
Sometimes, using a table can be useful, but never submit pages of numbers—numerical mathematics
is not about producing numbers, but about analyzing, learning from and gaining insight through
computations. Discuss what we can observe in and learn from a plot. If you do print numbers, use
fprintf to format the output nicely. Use format compact and other format commands to control
how MATLAB prints things.
Exporting figures. When you create figures using MATLAB (or Python/Octave), please always
export them in a vector graphics format (.eps, .pdf, .dxf) rather than raster graphics or bitmaps (.jpg,
.png, .gif, .tif). Vector graphics-based plots avoid pixelation and look much cleaner. They can be
scaled without losing resolution, and look much more professional. They can directly be embedded
in LATEX documents. Many tutorials on making professional looking figures and visualizations in
MATLAB/Python can be found on the web when you search for MATLAB professional looking plots.
To achieve reproducible prints from within a script/function, I recommend using the print command
in MATLAB rather than printing from the GUI. For instance, use
print -depsc myfigure.eps
to print a color figure in .eps format to a file named myfigure.eps.
Coding. It is only possible to fully understand and appreciate a numerical method once you have
implemented it yourself. We will use MATLAB’s build-in methods to compare with, but it is a critical
part of this class to implement, debug and to verify your own (small) programs. After you are finished
with an implementation, make sure that it is correct and does what you think it does. Use meaningful
variable names, try to write clean, concise and easy-to-read code and use comments for explanation.
If you feel that you are struggling with MATLAB (or Python/Octave), please spend some time as
soon as possible to brush up your programming skills using, for instance, Cleve Moler’s book. This
will eventually save you time and you will benefit more from the class.

1
 1. [Condition numbers, 1+3+3+3+5pt] Compute absolute and relative condition numbers (for
infinitesimal perturbations) for the following problems
√
(a) G(x) = x for x ∈ R, x ≥ 0.
(b) G : R2 → R2 defined by  
x2 + 1
G(x) = ,
x21 x2
where x = (x1 , x2 )T ∈ R2 . Compute the condition numbers when using the ∥ · ∥1 and the
∥ · ∥∞ norms in R2 .
(c) Consider two lines in R2 given by the equations y = 0 and ax + y = b, with a, b ∈ R.
Compute the intersection point S of the two lines as a function of a, b. When you change b,
for which values of a is S most sensitive?
(d) Let f, g : R → R denote two problems with the absolute condition numbers κf (x) and
κg (x), respectively, at x ∈ R. The functions f and g are differentiable. Derive the absolute
condition number of the composed problem h = g ◦ f at x ∈ R.
(e) Consider the polynom p(x) = ni=1 ai xi with coefficients ai and roots xj . Proof that the
P
relative condition number κj (ai ) of the root xj with respect to the coefficient ai is

|δxj | |δai | |ai xi−1

j |
κj (ai ) = / = ′ .
|xj | |ai | |p (xj )|
Hint: Take the derivative of the problem p(xj (ai ); ai ) = 0 with respect to ai with the chain
rule.
2. [What error to expect? 5pt] Consider solving the equation (x − 1)3 = d over R where d = 0.
Assume we know the right-hand side d only up to an absolute error |δd| ≤ 10−6 . This means we
solve (x − 1)3 = dˆ with |d − d|
ˆ ≤ 10−6 to compute x̂. Bound the absolute error |x − x̂|. Would
you consider this problem to be poorly conditioned?
3. [Can we trust the numerical result? 3 + 3 + 4pt] Consider the polynomial function p(x) =
(x − 2)9 with coefficient representation

p(x) = x9 − 18x8 + 144x7 − 672x6 + 2016x5 − 4032x4 + 5376x3 − 4608x2 + 2304x − 512.

(a) Plot p(x) for x = 1.920, 1.921, . . . , 2.080 using the coefficient presentation.
(b) Plot p(x) using the representation (x − 2)9 .
(c) Which of the two plots do you trust and why?
4. [Computing in finite precision, 4+3+3(+1)pt]
(a) Due to the finite precision in floating point number representation, there are gaps between
consecutive numbers. The size of these gaps depends on the size of the number and on
the precision (e.g., double or single precision). MATLAB provides the function eps(·) (the
analogue in NumPy is the function spacing), which returns, for a number, the distance to
the next floating point number in the same precision. Using the form of double and single
floating point number representation, explain the values you find for
eps(1),1
1
This value, eps(1) = eps ≈ 2.22 × 10−16 is usually referred to as machine epsilon, and it gives an indication of the
rounding error when dealing with double precision floating point numbers.

2
 eps(single(1)),2
eps(240 ),
eps(single(240 )).
(b) Try the following experiment and explain the behavior:
> a = 0.5;
> b = 0.7-0.2;
> a == b3
> sprintf(’%20.18f’,a)4
> sprintf(’%20.18f’,b)

(c) Give examples of nonzero numbers a, b, c whose floating point representations satisfy:
> b + a = a;
> (a + b) + c ̸= a + (b + c);

(d) [1pt extra credit] In Matlab the command realmin gives the smallest positive floating-point
number. We might expect that realmin/2 gives 0. However, we obtain
1: >> realmin /2
2:
3: ans =
4:
5: 1.1125 e -308

Explain this behavior by explicitly writing down the floating-point representation (sign bit,
mantisse, exponent) and performing the division in this format. Hint: It might be helpful to
learn about denormalized numbers, which we have not covered in class.

5. [Finite differences in finite precision, 10+10pt] Often, derivatives of functions can only be
computed numerically. For a three times differentiable function f : R → R one can use Taylor
expansions around the point x0 to show the following approximations:
f (x0 + h) − f (x0 )
f ′ (x0 ) = D1 f (x0 )+O(|h|) with D1 f (x0 ) := (one-sided finite differences),
h
f (x0 + h) − f (x0 − h)
f ′ (x0 ) = + O(|h|2 ) (centered finite differences).
2h
In these approximations, one subtracts very similar numbers from each other, which is badly
conditioned and can lead to significant cancellation errors. Here, we discuss an alternative.

(a) Assuming the function f can also be defined for complex arguments, use a Taylor expansion
of f (x0 + ih) to show that
Im(f (x0 + ih))
f ′ (x0 ) = + O(|h|2 ),
h
2
Note that the MATLAB command single switches to single precision.
3
The double equal sign checks if the numbers to its left and right are the same (i.e., every bit coincides!). An answer
of 0 means false (not the same), and 1 means true. Note that you should never do this for floating point numbers! You
should always test if the difference between two numbers is small (e.g., 10*eps(1)) rather than if they are equal.
4
This outputs a large number of digits and helps to be show rounding errors due to the finite precision of the computer
representation of numbers. Use help sprintf to read up on this MATLAB syntax (which is the same as in other languages,
such as C). The analogue output in Python is generated by print "%20.18f’’ % a.

3
 where “Im” denotes the imaginary part of a complex number. This is called complex step
differentiation, and it obviously avoids numerical cancellation.
(b) Compare these three approximations to compute the derivative of the function

exp(x)
f (x) =
cos(x)3 + sin(x)3

at x0 = π/4. In order to do so, use progressively smaller perturbations h = 10−k for

k = 1, . . . 16. Present the errors in the resulting approximations in a log-log plot, i.e.,
use a logarithmic scale to plot the values of h on the x-axis, and a logarithmic scale to
plot the errors between the finite difference approximations and the exact value, which is
f ′ (x0 ) = 3.101766393836051, on the y-axis. Discuss your result.

6. [Rounding and Taylor series, 6+7+7pt] Write a program that approximates exp(x) with its
Taylor series:
n
X xi
exp(x) ≈ (1)
i!
i=0

and plot, for x = −5.5 the difference to the exact value obtained with n = 1, . . . , 30, where you
use the (truncated) Taylor expansion in three different ways:

• Use the formula (1) directly to compute exp(−5.5).

• First use that exp(−5.5) = 1/ exp(5.5), and then use (1).
• Use that exp(−5.5) = (1/ exp(0.5))11 and then use (1).

Discuss the results, which show that a different mathematical formulation can significantly influ-
ence the accuracy of a computation.

7. Condition [5pt + 5pt] Let ẋ(t) = v(x(t)), v : R → R be an ordinary differential equation with
initial condition x(0) = z ∈ R. Let x(t; z) be the solution at time t, where the dependence on
the initial condition is explicitly denoted. Derive the absolute and relative condition number of
the solution map f (z) = x(t; z) for a fixed time t > 0 with respect to the initial condition z for
the following differential equations

(a) v(x) = λx , λ ∈ R, and

(b) v(x) = x2 .

Hint: First calculate the solution of the ordinary differential equation in closed form and then
derive the condition number.