Fakultät Bauingenieurwesen Institut für Bauinformatik

Module BIWO-04

Numerical Methods

Lecture notes (part 1/2)

1
Numerical analysis

▪ discipline of mathematics
▪ design, development and study of algorithms that use numerical approximation for
solution to the problems of continuous mathematics, for which :

- an explicite solution does not exist

- an explicite solution exists but its application is too costly (not efficient)

formal description of the solution, so that a transformation of the solution

into a machine readable language is possible

examples: ▪ roots of polynomials of degree 1, 2, 3, and 4 are explicitely calculable, but

roots of polynomials of higher degrees are only numericaly calculable

▪ integration

2
Contents: 1. Introduction Seite 4

2. Programming with Fortran Seite 6

3. Linear algebra Seite 15

• vector and matrix calculus
• system of linear equations
• eigenvalue problems

4. Systems of non-linear equations Seite 44

5. Numerical integration Seite 47

6. Approximation Seite 53

7. Computer graphics Seite 62

• coordinate transformation
• projection

8. Basics of Optimization (see Lecture Notes 2/2)

9. Basics of Articial Neural Networks (see Lecture Notes 2/2)

3
1. Introduction

▪ 2 types of algorithm:
- direct methods compute the exact solution to a problem in a finite
number of steps (unlimited computer precision assumed)
(e. g. Gaussian elimination method)
- iterative methods form successive approximations that converge to the
exact solution only in the limit
(e. g. Newton's approximation method)
▪ criteria:
runtime: - number of calculation steps to solve the problem
stability: - an algorithm is a) stable
b) weakly stable
c) unstable

if a arithmetic or rounding error occured in the mth calculation step

a) decrease
b) is about the same size
c) increase
in the next calculation steps

- associated with
4
▪ condition of the problem:
- dependency of the solution of the problem on a variation or error of the input
data

- a problem is ill-conditioned if small changes of the input data result in large

changes of the solution

problem algorithm

input data changed

input data

▪ reasons for arithmetic or rounding errors:

- finite description of floating-point numbers by computers

approximation of real numbers
- approximations by the algorithm

- absolute error:

- relative error:

5
2. Programming with Fortran

▪ FORmula TRANSlator
general-purpose programming language that is especially suited to
numeric computation and scientific computing
▪ source code machine code
Compiler
.f90 .exe

▪ history: - 1957: first Fortan Compiler (IBM)

revision
- 1966: FORTRAN 66
of the
- 1978: FORTRAN 77
standard
- 1992: Fortan 90
▪ structur of a main program (analogous : subroutine, function)

- program header
- declaration part
- procedure body
- end of program
! keine
Unterscheidung

6
- symbol set: 26 upper case letters A–Z
26 lower case letters a–z ! no
differentiation
10 numerals 0–9
1 underline _
5 arithmetic symbols + - * / **
17 further symbols ( ) . = , ‘ $ : ! “ % & ; < > ? space

- basic elements: - values e.g.: 1.0

- symbolic names (maximum of 31 characters) e. g.: alpha
- commands e. g.: write
- operators e. g.: +

▪ program header: program name main program

[subroutine name (…) subprogram]

▪ declaration part: - constants and variables needed within the program

definite data type and name

integer - integer number e. g.: 10 -3

real - real number e. g.: 1.3 -1.0e-6
logical - logical value e. g.: .true. .false.
character - character string e. g.: ‘name‘

- data arrays access mode column-wise

7
 ▪ implicit declaration
all names with initial letters i … n: integer
else: real

▪ explicit declaration (recommended!)

implicit none
data arrays

explicit declaration

8
▪ procedure body:
- value assignments: variable name = term

assignment operator constants

variables
values
arithmetic & logical operators

- arithmetic operators:

hierarchy
pyramid of the
operators () brackets

** exponential

* / multiplication, division

+ - addition, subtraction

direction of priority of coequal

arithmetic operators

9
- logical operators:

comparison: == or .eq.
/= “ .ne.
> “ .gt.
>= “ .ge.
< “ .lt.
<= “ .le.

example result
3<4 or 3.lt.4 .true.
3<=4 or 3.le.4 .true.
3==4 or 3.eq.4 .false.
hierarchy
pyramid of the
== /= < <= > >=
operators

composition: .and. logical AND

.or. logical OR
.eqv. .true., if both terms have the same logical value
.neqv. .true., if both terms have a different logical value
log. Wert haben

10
- functions:
exp(x) exponential function
log(x) natural logarithm
log10(x) common logarithm
sqrt(x) square root
abs(x) absolute value
sin(x) sine radian
asin(x) arc sine
int(x) real value into integer by truncation
nint(x) real value into integer by rounding
real(i) integer value into real
min(a,b) minimum value of a and b
max(a,b) maximum value of an and b

11
- conditional statements:

if (logical condition) then

program statement 1
program statement 2
…
end if

if (logical condition 1) then

program statement 1
program statement 2
…
else if (logical condition 2) then
program statement m
program statement n
…
else
program statement s
…
end if

12
- loops:

do loop variable = initial value, final value, increment

program statement 1
program statement 2
…
end do

do while (logical condition)

program statement 1
program statement 2
…
end do

do
program statement 1
program statement
…
if (logical condition) exit
program statement m
program statement n
…
end do
13
- input and output:

write (*,*) a,b,c,‘Text‘ screen

read (*,*) a,b,c keyboard

location format

- input and output files:

open (unit,file=‘data.txt‘,access=‘sequential‘) - unit = 10,11, 12, …

- data.txt is located
in the working directory
read (unit,*) a,b,c

write (unit,*) a,b,c

close (unit)

14
3. Linear algebra
3.1 Vector calculus

▪ dot product

- properties

- application
a) perpendicular vectors
b) length/norm of a vector
c) angle between two vectors
d) distance of two points given by
their position vector
15
▪ cross product (in )

- properties

- application

a) linearly dependent vectors

b) area of the parallelogram spanned by two
vectors
c) and and form a right-handed
coordinate system

16
▪ linear dependency of vectors

- are linearly independent, if not one can be represented as a

linear combination of the others
e. g.

- that is, if it holds , then are linearly dependent

- In two linearly independent vectors span a plane

▪ triple product (in )

- properties

- application
volume of the parallelepiped spanned by three vectors

17
3.2 Matrix calculus

▪ notation columns

rows

▪ quadratic matrix
main diagonal

secondary diagonal

▪ transposed matrix

▪ symmetric matrix
18
▪ matrix of the same dimension

▪ addition and subtraction

- properties

▪ decomposition of quadratic matrices

19
▪ multiplication with a scalar

▪ multiplication of matrices

number of columns of = number of rows of

- FALK‘s scheme

20
▪ properties

▪ diagonal matrix
all elements outside the main diagonal = 0

= quadratic
= symmetric

21
- properties all elements of the th row of are
multiplied with

-“- th column -“-

if for

if for identity matrix

▪ permutation matrix

identity matrix: permutation of row (column) with row (column)

permutation matrix

- properties
permutation of row and of

permutation of column and of

22
- example

▪ addition matrix identity matrix: element at the point

addition matrix

- example

23
▪ tridiagonal matrix

▪ band matrix for and

band width:
24
▪ triangular matrix
upper lower

▪ inverse (reciprocal) matrix:

if exists, then is invertible, i. e. nonsingular

▪ orthogonal matrix, if it holds:

25
▪ determinant
- a special number associated with any square matrix

- in : absolute value of the determinant = area of the parallelogram spanned by

the (real-valued) column (row) vectors
- in : -“- = volume of the parallelepiped -“-
-“-

26
 Laplace's formula

- along the th column :

- along the th row :

removing the th row and the th column

▪ example (along the first column):

note: det(triangular matrix ) = product of the elements of the main diagonal

27
▪ properties

1. if results from by:

a. interchanging two rows (columns)

b. adding a multiple of one row (column) to another row (column)

c. multiplying one row (column) with a number

2. rows (columns) of are linearly dependent

3.

4.

5.

▪ inverse of a matrix

is invertible, if

for

2
▪ example ( ):

▪ rank of a matrix
- a special number associated with any matrix
- maximal number of linearly independent columns (rows)
- column rank = row rank

▪ rank of an augmented matrix

- properties

1. if results from by:

a. interchanging two rows (columns)

b. adding a multiple of one row (column) to another row (column)

c. multiplying one row (column) with a number

29
3.3 Systems of linear equations

▪ linear equation with

▪ system

matrix notation
homogeneous
inhomogeneous

▪ example: - displacement method

stiffness unknown vector of vector of

matrix displacements loads on internal forces
of the nodes the at the end of
nodes the bars

30
▪ solvability : a) no solution
b) one solution
c) infinitely many solutions

homogeneous inhomogeneous

1. one solution : one solution :

2. infinitely many solutions a)

infinitely many solutions
b)
no solution

▪ condition: Hadamard‘s formula

with

ill-conditioned

31
Algorithms

Gaussian elimination

reduce a matrix by a) adding a (multiple of) one row to another row

b) interchanging two rows (columns)

to triangular form:

e.g.

number of linearly independent equations = rank

32
▪ procedure : row 2 to :

row 3 to : analogous,

▪ properties

- in theory exact, but in practice rounding errors!

numerically stable by pivoting:
- interchange rows until the element with the largest absolute value of the current row
(= partial pivoting) or
- interchange rows and columns -“- of the remaining matrix
(= complete pivoting)
is main diagonal element of the current row
multipliers < 1 reduction of the rounding errors

▪ example

33
Cramer's rule

▪ precondition

- unique solvable quadratic system of linear equations

-th column of replaced by

▪ properties

- (too) costly, because determinants are to compute

inapplicable for
- for systems with algorithm is instable, because division with very small
numbers yields increasing errors

34
Cholesky decomposition

▪ precondition

- symmetric and positive definite matrix of the system of linear equations

positive definite :

- eigenvalue criterion: later

- determinant criterion : for

▪ algorithm :

with and

lower triangular matrix

35
▪ example ( ):

- equating coefficients:
for

for

for

- solve :

- solve :

36
▪ properties
- numerically stable
- by extracting the square root large main diagonal elements become larger,
i.e. a reduction of the loss of decimal places
good for ill-conditioned problems
- less costly than Gaussian elimination
- necessary for positive definiteness : easy to negative it
- a band structure of is transferred to
- minor effort for further

3.4 Specific eigenvalue problem

or

- eigenvector (≠ zero vector): direction is not changed by the transformation

- eigenvector is only scaled by a factor of λ
- scaling factor = eigenvalue

geometrical interpretation:

ellipse:

37
 normal vector:

principal axis

formal interpretation:

Transformation:

real-valued symmetric orthogonal modal matrix

matrix

38
 Algorithms

eigenvalues as roots of the characteristic polynomial:

▪ homogeneous system of linear equations:

nontrivial solution, if

polynomial of degree
roots

set one component equal to 1

solution of the characteristic polynomial for large matrices almost impossible

power iteration:

▪ computation of the eigenvalue with the largest absolute value

- standardization: (reduction of the rounding errors!)

39
 termination, if

- Termination condition:

▪ computation of the eigenvalue with the smallest absolute value

▪ iterative computation of further eigenvalues ( = symmetric)

40
 - iteration:

- standardization:

- in general:

▪ further methods: - QR-algorithm

- inverse iteration
- Lanczos algorithm
- Arnoldi iteration

41
▪ properties:

if is an eigenvalue of corresponding to the eigenvector ,

then is eigenvalue of corresponding to

= real-valued, symmetric
eigenvalues are real-valued
eigenvectors corresponding to different eigenvalues are orthogonal to each other

= real, symmetric, a.) positive or b.) negative definite

all eigenvalues are real-valued and a.) greater or b.) lower than zero

Rayleigh quotient:

for = real, symmetric and corresponding eigenvalues it holds:

42
 3.5 Generalized eigenvalue problem
or
▪ statics/dynamics:

stiffness matrix eigen mass matrix eigenmode

(symmetric, angular (symmetric,
positive frequency positive definite,
definite) mostly diagonal)

▪ ▪ in general: - transformation to a specific eigenvalue problem

(Cholesky decomposition)

similarity
transformation
(eigenvalues
remain unchanged) 43
4. Systems of nonlinear equations

▪ system of nonlinear equations

▪ iterative algorithms:
improve approximate solution successively until a required accuracy is reached

▪ one-dimensional, continuous problems

4.1 Bisection method

- find root of within the interval (precondition: change of sign)

- repeatedly bisects an interval then selects a subinterval in which a root must lie

44
4.2 Secant method

rounding error close to , because

termination: or

4.3 Regula falsi method

combination of the bisection method and the secant method

45
 4.4 Newton's method

▪ multidimensional, continuous problems

▪ numerical: - system of linear equations

- often is not given explicitely

approximation of the derivatives, e.g. by finite differences

46
5. Numerical integration

- algorithms for the approximate solution of definite integrals

- an explicite solution often does not exist for integrals:
- no antiderivative is available
function is only known on discrete points

▪ one-dimensional problem
5.1 Quadrature rules based on interpolating functions

rectangle method

mit

▪ properties:

- convergence
- error decreases proportionally to

47
trapezoidal rule

mit

▪ properties:

- convergence
- error decreases proportionally to

Simpson's rule

with
▪ properties:

- convergence
- even
- error decreases proportionally to
48
Newton-Cotes-Formeln

- interpolation of by polynomials of degree

- integration of these polynomials yields approximate solution
- for subintervals polynomials of degree are exactly integrable
- for rectangle method, trapezoidal rule, Simpson's rule
- numerical: for unfeasible (rounding errors, subtractive cancellation)

Gaussian quadrature

- for nodes polynomials of degree are exactly integrable

for

weights (≥ 0) specific nodes

tabulated

49
▪ multidimensional poblems
5.2 Monte Carlo integration (simulation)

▪ example:

50
„random numbers“

- random number generators generate uniformly distributed realizations within the interval [0, 1]
necessary: transformation into the required intervals [a1, b1], [a2, b2], …

pseudo-random numbers low-discrepancy numbers grid numbers

- periodic deterministic - deterministic series of - deterministic series

series of numbers numbers (non-periodic) of numbers
- regions of higher and - low discrepancy - „inflexible “
lower density - uniform distribution is
- „moderate“ uniform always given
distribution

51
▪ example:

52
6. Approximation

-of a: - implicitly given solution, which is too complex to solve analytically

- explicitly given (difficult) function, in order to reduce the complexity of the problem

function approximation

= a difficult function approximated by a function, which is easier to handle

▪ Taylor series

- one-dimensional problem:

times differentiable

▪ example:

53
 n=1 n=3 n=5 n=15 n=oo

0
-8 -6 -4 -2 0 2 4 6 8
-1

-2

-3

-4

discrete data

interpolation vs. regression analysis

- finitely many, discrete data are approximated by a continuous function

- function is not given analytically, only a set of points is provided
- by measurements
- by computation with discrete methods (Deformationmethode, FEM, …)
- functional characteristics at arbitrary intermediate points are estimated
- (also in terms of function approximation applicable)

interpolation function: an exact fit regression function: a "smooth" function that

to the data (points) is required approximately fits the data (points)
54
▪ one-dimensional problem:

linear interpolation
- one of the simplest methods
- two data points and are interpolated by a linear function

polynomial interpolation
- for mutually distinct data points with
one specific interpolating polynomial of maximum degree exists
(fundamental theorem of algebra)

with

and

- computation of the coefficients = solution of a system of linear equations

disadvantage: - if polynomial degree is high many extremas = heavy oscillations

advantage: - easy to integrate and to differentiate
55
▪ Lagrange polynomial:

advantage: - arbitrary step size

disadvantage : - consideration of further nodes during computation is not possible

▪ Newton polynomial:

mit

advantage: - arbitrary step size

- consideration of further nodes during computation is possible 56
piecewise interpolation
- polynomials of higher degrees ( ) disadvantageous (see above)
- desired: „smooth“ interpolation function
- piecewise interpolation: - linear polygon
- polynomials with spline interpolation.
- continuity and differentiability at the nodes:
- linear first derivative at the nodes discontinuous („kinks“)
- splines differentiable

▪ cubic spline polynomials:

for and

nodes intervals per 4 coefficients conditional equations needed

► continuity ( equations):

► 2-times differentiability at the inner nodes ( equations):

57
► 2 boundary conditions (variable):

- natural spline (free boundaries):

- periodic spline ( ):

- fixed boundaries:

- not-a-knot:

one cubic polynomial in the intervals

and
▪ algorithm (natural spline):
it holds:

1.
2.

3.

with:

58
 ▪ properties: - tridiagonal
- symmetric
- diagonally dominant
- positive definite
- only positive elements
- well-conditioned
- invertible

4.

5.

59
▪ algorithm (periodic spline):

it holds:

1.

2.

with:

3.

60
 4.

5.

▪ application of splines: - graphical and CAD software

- fonts
- tracks of roller coasters
- railway high-speed route

▪ multidimensional poblems:

- approximation by polynomials, artificial neural networks, …

response surface = metamodel

61
7. Computer graphics
7.1 Coordinate transformation
▪ geometric transformations of three-dimensional (spatial) objects:
- translation vector addition
- rotation
- scaling matrix multiplication
- shear

▪ example: - translation and rotation of a unit cube

e. g. conversion of coordinates of the coordinate system of a (virtual) object

into coordinates of a screen (dependent on the movement of a virtual camera)

62
- avoiding of rounding errors
- reducing computational effort
combining successive geometric transformations to one single transformation matrix

- translation? extension of the spatial coordinate system by a fourth dimension

homogeneous coordinates: - means for a standardized representation of geometric

transformations by multiplication with matrices of size 4x4
- reversal of the transformations by matrix inversion

▪ example: - translation and rotation of a unit cube

63
translation:

scaling:

shear:

64
rotation about Z-axis:

rotation about X-axis:

rotation about Y-axis:

7.2 Projection

orthogonal projection:

65
perspective projection: + standardization (division
by the fourth component) !!

central projection: vanishing point:

+ standardization (division
by the fourth component) !!

66