Numerical Analysis

Lecture-1
Numerical Analysis is a branch of Mathematics concerned with the theoretical founda-
tions of numerical algorithm which provides a complete set of procedures for finding an
approximate solution of problems that arise in scientific applications. In this course, we will
focus on the development of algorithms to solve some class of problems and mathematically
analyse these techniques. Since we will obtain approximate solutions to our problems, we
will focus our attention on the issues of how accurate our solution is?, what is the error that
is incurred at each step of our computation and see how we can get reliable results correct
to the desired degree of accuracy.
A good knowledge of the tools from calculus is necessary for understanding of the nu-
merical algorithms and we will review these tools before discussing the main topics.
In scientific and engineering applications, we come across situations where we have
to solve equations which are linear, nonlinear or involve derivatives or integrals and a
combination of these. It may not be possible to write an exact solution of these using
elementary functions. The topics that we discuss and understand from this course will help
us to construct approximate solutions to such otherwise intractable problems. while doing
this, we will compare and assess different methods for solving a class of problems with our
focus on efficiency and accuracy of these methods.
Numerical Analysis lies at the heart of Scientific Computing or computational sciences
and engineering domains and it includes the development of high performing computing
skills. In the course, we focus our attention on laying a strong foundation and knowledge
of various methods used for obtaining approximate solutions of a certain class of problems
with emphasis on error analysis and postpone the part of developing numerical codes for
implementation to another course in future.
In this course, we will consider the topics on polynomial interpolation, numerical dif-
ferentiation, numerical integration, numerical solution of first order ordinary differential
equations governing initial value problems. Enclosure methods and fixed point iteration
methods for obtaining a simple root of algebraic and transcendental equations will be dis-
cussed. Decomposition method and Iterative methods will be discussed for solving a system
of linear algebraic equations and error analysis will be considered. This will be followed by
learning some simple methods for solving algebraic eigenvalue problems.

1 Polynomial Interpolation
We begin our study with the problems of polynomial interpolation. Polynomial interpolation
is one of the most fundamental problems in numerical methods. A function y = f (x) is
known at (n + 1) discrete points x0 , x1 , x2 , . . . , xn ; that is, f (x0 ), f (x1 ), f (x2 ), . . . , f (xn ) are
given or we may not know the function and the data that is experimentally collected at
the points x0 , x1 , x2 , . . . , xn namely y0 , y1 , y2 , . . . , yn respectively may be known. In either
case, our goal is to construct a function that passes through these points; that is, to find a
function p(x) such that the conditions

p(xi ) = f (xi ) (or yi ) for i = 0, 1, . . . , n

1
are satisfied. We are interested in seeking a smooth function. We therefore look for a
function that is a polynomial. The points are called interpolation points. By ”interpolating
the data”, we mean that the polynomial passes through the points (xi , f (xi )) such that
pn (xi ) = f (xi ), i = 0, 1, 2, . . . , n. The function that interpolates the data, pn (x) is an
interpolating polynomial.
We shall develop methods for constructing interpolation polynomials in this course.
Note that the interpolating polynomial pn (x) and the underlying function f (x) are gen-
erally different except that they agree at the interpolation points. So, we focus on error
in interpolation which is a measure on the difference between the two functions f (x) and
pn (x).

2 Numerical Differentiation
We next consider numerical approximation of derivatives. Why do we need to approximate
derivatives?
The function values may be known only at a set of discrete points and we may have
to obtain a derivative of the function and understand the changes in the data that are
related to derivatives, or the computation of the derivative of the given known function
may be complicated and it may be easier to approximate the derivative. For example, if
you measure the position of a moving vehicle every minute through some mechanism (say,
GPS) and you want to compute its speed. The position of the vehicle is known only at
discrete times and differentiation is not possible. So, we require a numerical method for
computing the derivative of a function whose values are known at a set of discrete points.
For the above and other reasons, we wish to develop methods for approximating various
order derivatives.

3 Numerical Integration
Integration is a basic Mathematical operations having a wide range of applications in science
and engineering. Most of the integrals can not be evaluated by analytical methods that you
have learnt in school. Also, you may know function values (such as speed of a vehicle) at
a set of discrete points (times) and you may want to compute the integral of this function
(distance travelled by the vehicle in unit time). The solution to this problem is to use a
numerical method to evaluate the required integral. We shall consider a number of methods
for evaluating an integral and while doing so, we shall understand a general strategy for
developing such methods. This will help you to develop new techniques that you may need
for your applications. The techniques developed allow us to keep track of the error incurred
in these methods and we perform error analysis and also determine a bound on the error.

4 Numerical Solution of Ordinary Differential Equa-

tions
Ordinary Differential Equations(ODE’s) of order one occur as mathematical models in many
branches of science, engineering and in other branches. These equations may not belong to
the class of equations for which exact analytical solutions can be obtained and therefore the

2
solutions can not be expressed in closed form. Therefore, we seek approximate solutions
by means of numerical methods. These methods provide us a way of obtaining accurate
results with a reliable bound on the error. We will focus on the development and analysis
of numerical techniques for solving Initial value problems governed by first order ODE’s.
We begin with some preliminary results that give us the conditions under which an Initial
value Problem(IVP) possesses a unique solution in an interval containing x0 (an initial
point). The methods that we consider are for IVP’s that satisfy the conditions stated in
the existence and uniqueness theorem on solutions of IVP’s.

5 Root finding problem: solving f(x)=0

We then move onto developing methods that help us to find an approximation to a simple
root of an equation f (x) = 0, where f (x) is an algebraic or transcendental function. The
methods developed belong to either a class of enclosure methods or fixed point iteration
methods.The iterative methods will generate a sequence of approximations to a root of
f (x) = 0 and we need to discuss the conditions under which this sequence converges and so
we perform convergence analysis and determine the order of convergence of these methods.
We will also consider in detail the error estimation of each of these methods that we develope
and see the advantages and disadvantages of each of these methods.

6 Solutions of System of equations: A(x) = b

We next consider two main classes of methods for solving linear systems:A(x) = b, where A
is an n × n matrix, x is an n × 1 vector and b is an n × 1 vector, namely, the direct methods
and iterative methods. we consider only simple iterative methods in this course as it is
important for a student to be able to use these methods efficiently before studying more
efficient and modern methods. In the case of modern methods, we consider decomposition
methods and understand conditions under which Gauss Elimination Method is guaranteed
to yield a solution in the absence of round-off errors. We will also consider the effects of
such errors and find an estimate on the error bound.
We finally consider the algebraic eigenvalue problems which are important in stability
analysis. We begin with a brief description of the eigenvalue problem and present a simple
but powerful method of computing the dominant eigenvalue of a matrix and corresponding
eigenvector.
The lecture notes and the material presented in the classes are based on the content from
the following books. The presentation and explanations are excellent in these books and I
have enjoyed reading and understanding Numerical Analysis from these books. Some subtle
points are explained very well and I have closely followed the text books and prepared the
lecture notes. I sincerely thank and acknowledge the authors of these text books for their
clear and excellent presentation in their books. I have also given below the book references
from where I have collected problems.

References
[1] David Kincaid and Ward Cheney, Numerical Analysis: Mathematics of Scientific Com-
puting, ISBN-13:978-0-8218-4788-6 (AMS), Pure and Applied Undergraduate texts, Vol

3
 2, (2002).

[2] R.L. Burden and J.D. Faires, Numerical Analysis, 7th Edition, Brooks/Cole, (2001).

[3] E. Süli and Mayers, An Introduction to Numerical Analysis, Cambridge University Press,
(2003).

[4] M.K. Jain, S.R.K. Iyengar and R.K. Jain, Numerical Methods for Scientific and Engi-
neering Computation, New Age International (P) Limited, Publishers New Delhi.

[5] Robert J. Schilling and Sandra L. Harris, Applied Numerical Methods for Engineers
using MATLAB and C, Brooks/Cole, New york.

[6] M.K. Jain, S.R.K. Iyengar and R.K. Jain, Numerical Methods: Problems and Solutions,
New Age International, (2007)