Simulation and modeling of natural processes

Week 2: Introduction to programming with Python 3

Orestis Malaspinas

1/64
Introduction to high performance
computing for modeling

2/64
Why do we need programming in this course?

• We want to represent nature through complex mathematical models.

• These models are too complex to be solved analytically.
• Either they do not possess analytical solutions or it would take too long to compute it.
• Numerical methods exist to approximate the mathematical models used to represent nature.
• Computers are very efficient in doing large amount of computations.
• We need an efficient way to make the computer use the numerical methods.

3/64
Computers are fast but... we always want more

For a computer model to be satisfying it must...

• Represent with good accuracy the process it is supposed to model.
• Give a solution in a reasonable amount of time.

For a program to be faster one can...

• Wait for the hardware to become faster (Moore’s law).
• Optimize the algorithm (bubble sort, quick sort).
• Optimize the way algorithm implemented.

4/64
 End of module
Introduction to high performance
computing

Coming next
Concepts of code optimization
5/64
Concepts of code optimization

6/64
Computers are fast but... we always want more

Example: sum of the numbers stored in an array.

Mathematical representation Graphical representation
n−1
X
A = {xi }n−1
i=0 , xi ∈ N, s= xi .
i=0

How do we implement this on a computer?

7/64
Computers are fast but... we always want more

Performed in 3 different ways with Python 3 (there are more)

• The array is read linearly.

• The array is read randomly.

• The sum is performed with sum.

8/64
Benchmark: Sum of n = 106 integers

Efficiency depends on the algorithm.

linear random sum
time [s] 0.0594 0.270 0.00654
t/tsum 9.08 41.3 1

Efficiency depends on the programming language

Python 3 (sum) NumPy C++ (linear)
time [s] 0.00654 0.000634 0.000478
t/tC++ 13.7 1.3264 1

By coding naively C++ is 100 times faster than Python but with NumPy the performance is
equivalent.

9/64
 End of module
Concepts of code optimization

Coming next
Concepts of parallelism

10/64
Concepts of parallelism

11/64
A computer is fast... many computers together
are faster

Shared memory (OpenMP) Distributed memory (MPI)

Performance
• One modern processor ∼ 100 Gflops = 1011 flops.
• The fastest supercomputer (as of 4.1.2015) ∼ 52 Pflops = 52 · 1018 flops.

12/64
A computer is fast... many computers together
are faster

Not all problems are parallelizable

Parallelizable problem Non-parallelizable problem
Can be decomposed into pieces that Fibonacci series computation
can be executed simultaneously.
F(i) = F(i − 1) + F(i − 2),
F(1) = 1, F(2) = 1.

13/64
 End of module
Concepts of parallelism

Coming next
Palabos, a parallel lattice Boltzmann
solver
14/64
Palabos, a parallel lattice Boltzmann
solver

15/64
The Palabos library
Palabos, http://www.palabos.org
• C++ parallel open-source library. • Fluid flow solver based on LBM.
• Parallelism completely transparent. • Multi-physics modules.

Example: Air-conditioning

16/64
Parallelism in Palabos: Multi-Block approach

• Spatial domain cut into many blocks.

• Blocks are dispatched on different CPUs.
• Communication is done through the
interfaces.
• Efficient if communication  computation
and the load on each processor is well
balanced.

17/64
 End of module
Palabos, a parallel lattice Boltzmann
solver

Coming next
An introduction to Python 3
18/64
An introduction to Python 3

19/64
Introduction to Python 3

• Multi-platform programming language used in this course (along with NumPy).

• With NumPy good trade-off between efficiency and simplicity.
• Large collection of advanced open-source libraries.
• Only Python 3 course (not compatible with Python 2).
• Introduction intended for beginners.
• Prerequisite installation of Python 3 (see https://www.python.org/downloads/)
and NumPy (see http://www.scipy.org/scipylib/download.html/).

20/64
Literature on Python 3

• Python 3 documentation : https://docs.python.org/3/

• Dive into Python 3: http://www.diveintopython3.net/
• Rapid GUI Programming with Python and Qt: The Definitive Guide to PyQt Programming,
Mark Summerfield, ISBN-10: 0132354187 – ISBN-13: 978-0132354189
• NumPy documentation : http://docs.scipy.org/doc/numpy/index.html

21/64
Interpreted vs Compiled languages

Interpreted Compiled
• Program is directly run by the interpreter. • Program transformed into machine code and
• The interpreter translates a code into its saved into an executable file.
subroutines that are precompiled. • The executable file can then be run.
• Examples: Python, JavaScript, Ruby, ... • Examples: C/C++, Fortran, ...

22/64
Interpreted vs Compiled languages

Advantages of each approach (only tendencies)

• Compiled code tends to be faster. • Interpreted code can be dynamically

• Interpreted code tends to be easier and typed.
faster to develop.
• Interpreted code tends to be more
portable.

23/64
 End of module
An introduction to Python 3

Coming next
Running a Python program

24/64
Running a Python program

25/64
The interactive shell

Python 3 interactive shell

modeling@mooc:~$ python3
Python 3.4.2 (default, Oct 8 2014, 10:45:20)
[GCC 4.9.1] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

Hello World! Calculator Errors Quit

>>> print("Hello World!") >>> 4+8*12 >>> print(Hello World!) >>> quit()
Hello World! 100 File "<stdin>", line 1 modeling@mooc:~$
>>> "Hello World!" >>> 56+3*90 print(Hello World!)
’Hello World!’ 326 ^
>>> >>> SyntaxError: invalid syntax
>>>

26/64
Spyder 3

27/64
Running a Python script

Needed when programs become too long to use the interpreter.

• Create a file “myscript.py” and edit it with your favorite text editor or IDE.
• Add the following line in it
print("Hello World!")

With text editor With Spyder

In terminal or command prompt: Hit the run button!
modeling@mooc:~$ python3 myscript.py runfile(’~/myscript.py’, wdir=’~/’)
Hello World! Hello World!

28/64
 End of module
Running a Python program

Coming next
Variables and data types

29/64
Variables and data types

30/64
Variables
Container to store values that can accessed and changed.

In Python (not the case in C/C++, Java for example):

• No need to declare the data type of a variable.
• Variables can change data type.
Example Discussion
a = 10 1 1: The value 10 is stored in a (a is the identifier).
print(a) 2
10 3
4: a is taking a new value which is a+1.
a = a+1 4
print(a) 5
7: The string “Hello World!” is stored in a.
11 6
a = "Hello World!" 7
print(a) 8
Hello World! 9

31/64
Variables: identifiers
Rules for variable identifiers
1. The first character can be ANY Unicode letter or the underscore.
2. The other characters can be ANY Unicode letter or digit or the underscore.
3. All the Python keywords are forbidden.
4. Identifiers are case sensitive.
Examples

1 >>> àél3_aerkcàéal = 10 1 >>> True = 10

2 >>> 2àél3_aerkcàéal = 10 2 File "<stdin>", line 1
3 File "<stdin>", line 1 3 SyntaxError: can’t assign to keyword
4 2él3_aerkcàéal = 10 4 >>> a,A = 10,20
5 ^ 5 >>> print(a,A)
6 SyntaxError: invalid syntax 6 10 20

32/64
Data types
Built-in types
Numbers Strings
1. Integers (arbitrary size) Immutable chain of Unicode characters.
>>> type(2378) >>> ’Single line strings defined with single quotes’ 1
<class ’int’> ’Single line strings defined with single quotes’ 2
>>> "Or with double quotes" 3
’Or with double quotes’ 4
2. Floating-point numbers >>> c = ’’’Triple quotes for multiline 5

>>> type(1.23456e-6) ... and can contain ’ or " ’’’ 6

<class ’float’> >>> c[3] = "a" 7

Traceback (most recent call last): 8
File "<stdin>", line 1, in <module> 9
3. Complex numbers TypeError: ’str’ object does not support item assignment 10

>>> type(3+6j)
<class ’complex’>

33/64
Data types (continued)

Built-in types
Other types Examples
1. Boolean type 1 >>> type(True)
>>> print(type(True),type(False)) 2 <class ’bool’>
<class ’bool’> <class ’bool’> 3 >>> bool("")
4 False
5 >>> bool("hi")
2. None Type 6 True
7 >>> bool(0)
>>> type(None)
8 False
<class ’NoneType’>
9 >>> bool(10)
10 True
11 >>> print(True+True,True+False)
12 2 1

34/64
Data types (continued)
Built-in types
Sequence Types Examples
1 >>> a = [1,7,29,1,39]; a[2]
1. list : mutable sequence. 2 29
>>> type([1,’is an int’,2.0,’is a float’]) 3 >>> a[3] = "a"; print(a)
<class ’list’> 4 [1, 7, 29, ’a’, 39]
5 >>> b=(’njd’,[1,1],1.0)
6 >>> b[0] = 1
2. tuple : immutable sequence. 7 Traceback (most recent call last):
>>> type((1,’is an int’,2.0,’is a float’)) 8 File "<stdin>", line 1, in <module>
<class ’tuple’> 9 TypeError: ’tuple’ object does not
10 support item assignment
11 >>> b[1][0] = 4
3. range : immutable number sequence. 12 >>> print(b)
13 (’njd’, [4, 1], 1.0)
>>> type(range(10)) 14 >>> print(list(range(4)))
<class ’range’> 15 [0, 1, 2, 3]

35/64
 End of module
Variables and data types

Coming next
Operators

36/64
Operators

37/64
Operators: on numeric types

Operators (numeric type) Examples

• Addition +, Subtraction -, 1
2
>>> 12+15-37
-10
Multiplication *. 3 >>> 12*15*37
4 6660
• Real division /,Integer division //, 5 >>> 10/3
Modulo %. 6 3.3333333333333335
7 >>> 10//3
• Exponentiation ** . 8 3
9 >>> 10%3
• Unary +, -. 10 1
11 >>> 2.0**10
12 1024.0
13 >>> +2, -2
14 (2, -2)

38/64
Operators: on sequence types
Operators (sequence types) Examples
• Concatenation + . 1
2
>>>
>>>
a = [1,2,3,4,"b"]
a[0]
• Copies of a sequence * . 3 1
4 >>> a[0] = 9; print(a)
• Access to element [] . 5 [9, 2, 3, 4, ’b’]
6 >>> a+a
• Slice : . 7 [9, 2, 3, 4, ’b’, 9, 2, 3, 4, ’b’]
8 >>> 2*a
9 [9, 2, 3, 4, ’b’, 9, 2, 3, 4, ’b’]
10 >>> a[1:4]
11 [2, 3, 4]
12 >>> a[0:4:2]
13 [9, 3]
14 >>> a[2:]
15 [3, 4, ’b’]
16 >>> a[:2]
17 [9, 2]

39/64
Operators: Boolean operations and comparisons

Operators Examples More examples

• <, <=, >, >=, ==, !=. 1
2
>>> 2<1
False
1
2
>>> 2 or 3
2
• or, and, not . 3 >>> a,b = 1,2 3 >>> 2 and 3
4 >>> a == b 4 3
• in or not in a sequence. 5 False 5 >>> "a" == "a"
6 >>> 1 != 2 6 True
7 True 7 >>> "a" > "b"
8 >>> 1 != 2 and 1 == 1 8 False
Remark 9 True 9 >>> "a" and "b"
10 >>> 1 != 2 and not 1 == 1 10 ’b’
• or and and always return 11 False 11 >>> a = [1,2,3,4,"b"]
one of their operands. 12 >>> "b" in a
13 True

40/64
 End of module
Operators

Coming next
Control structures: conditional
statements
41/64
Control structures: conditional
statements

42/64
Control structures: conditional statements

Conditional branching based on boolean information.

The if statement Remarks

if condition_1:
statements_1
• There can be zero or more elif .
[elif condition_2: • There can be at most one else .
statements_2
... • The statements can be of more than one
elif condition_n:
statements_n] line.
<else:
statements>
• Indentation is important!
if condition: if condition:
statement_1 statement_1
statement_2 statement_2

43/64
 Conditional statements (example)

Velocity on the motorway Output 1

x = int(input("Enter your velocity : ")) 1 Enter your velocity : 60
2 Going too slow. Velocity increased by 20
if x >= 120: 3
y = x-120 4
x = 120 5
print("Going too fast. Velocity reduced by ", y) 6 Output 2
elif x <= 80: 7
y = 80-x 8 Enter your velocity : 151
x = 80 9 Going too fast. Velocity reduced by 31
print("Going too slow. Velocity increased by ", y) 10
else: 11
print("Everything is fine.") 12
Output 3
Enter your velocity : 89
Everything is fine.

44/64
Control structures: conditional statements

The ternary (or inline) if Remark

a = x if condition else y • x is returned if condition is True,
else y is returned.
• No elif allowed.

Example Output
1 x = int(input("Input an integer: ")) Input an int: 10
2 a = "even" if x%2 == 0 else "odd" 10 is an even number.
3 print(x, " is an ", a, " number.")
Input an int: 15
15 is an odd number.

45/64
 End of module
Control structures: conditional
statements

Coming next
Control structures: loops
46/64
Control structures: loops

47/64
Control structures: for loop
Loop over any kind of ordered sequence.
The for statement Example
for variable in sequence: animals = ["cat", "dog", "cow", "bird"] 1
statements_1 for x in animals: 2
[else: if x=="cow": 3
statements_2] break 4
print(x) 5
else: 6
Remarks print("no cow in my list") 7
8
• Indentation is important! cat 9
dog 10
• The sequence can be of any type.
• If sequence is numbers use range.
• The else statement is optional.
• break statement exits the loop.

48/64
Control structures: while loop
Perform an action as long as a condition is satisfied.

The while statement Example Output

while condition: 1 x = int(input("Enter a number : ")) Enter a number : 5
statements_1 2 ary = list(range(x)) Total ended normally.
[else: 3 tot = 0 The sum is: 10
statements_2] 4 i = 0
5 while i < len(ary):
6 tot += ary[i]
Remarks 7 i += 1
8 else:
• Indentation is important! 9 print("Total ended normally.")
10
• The else statement is 11 print("The sum is: ",tot)
optional.
• The loop can be exited
with the break.

49/64
 End of module
Control structures: loops

Coming next
Functions

50/64
Functions

51/64
Functions
Definition and use
√
• Provided some input parameters compute one or more results (f : x → x).
• More generally a set of statements that can be reused in a program.
• A basic building block of a program.
Syntax Remarks
def function_name(parameter_list): • The return and comments statements are optional.
[""" comments (doc) """]
statements • The return statement ends the function and returns
[return]
the result.
• Functions always return None by default.
• The number of parameters can be zero or more.
• Indentation is important.

52/64
Functions (example)

Code Result
1 def isPrime(number): isPrime(number)
2 """Check if number is prime""" Check if number is prime
3 if number <= 1:
4 return False 0 is not a prime number
5 1 is not a prime number
6 for i in range(2,number): 2 is a prime number
7 if number % i == 0: 3 is a prime number
8 return False 4 is not a prime number
9 return True 5 is a prime number
10 6 is not a prime number
11 help(isPrime) 7 is a prime number
12 for j in range(0,13): 8 is not a prime number
13 if (isPrime(j)): 9 is not a prime number
14 print(j, " is a prime number") 10 is not a prime number
15 else: 11 is a prime number
16 print(j, " is not a prime number") 12 is not a prime number

53/64
Functions: Scope of variables
Variables have the scope of the block they are declared in and can be used after their point of
declaration.
Global variable Error
1 def foo(): 1 def baz():
2 print(a) 2 print(a)
3 3 a="Hi!"
4 a = "Hello!" 4 print(a)
5 foo() 5
6 Hello! 6 a = "Hello!"
7 baz()
Local variable 8 Traceback (most recent call last):
9 File "scope.py", line 27, in <module>
1 def bar(): 10 foo()
2 a="Hi!" 11 File "scope.py", line 22, in foo
3 print(a) 12 print(a)
4 13 UnboundLocalError: local variable ’a’
5 a = "Hello!" 14 referenced before assignment
6 bar()
7 Hi!

54/64
Functions: Mutability/Immutability
Number List String, Tuple
1 def foo(a): 1 def bar(a): 1 def baz(a):
2 a=4 2 a[1]=-2 2 a[1]="n"
3 print(a) 3 print(a) 3 print(a)
4 4 4
5 a=2 5 a=[2,4,5] 5 a="abcdef"
6 foo(a) 6 bar(a) 6 baz(a)
7 print(a) 7 print(a) 7 print(a)
8 4 8 [2, -2, 5] 8 Traceback (most recent call last):
9 2 9 [2, -2, 5] 9 File "immutable_mutable.py",
10 line 37, in <module>
11 foo(a)
12 File "immutable_mutable.py",
13 line 33, in foo
14 a[1]="n"
15 TypeError: ’str’ object does not support
16 item assignment

55/64
Modules

Modular programming Example

• Complex programs can be separated into • Save isPrime function into a file
many simple parts. named checkPrime.py.
• Parts can be assembled to construct different >>> import checkPrime
>>> checkPrime.isPrime(8)
programs. False

• Using alias
Creation and use
>>> import checkPrime as cp
• Saving the functions into .py files. >>> cp.isPrime(8)
False
• Using the import...[as]... keyword.
• All math functions are in math
module

56/64
End of module
Functions

Coming next
NumPy

57/64
NumPy

58/64
NumPy

• Open-source library (module) extension to Python.

• Fast precompiled numerical routines.
• Support for large multi-dimensional arrays (Matlab-like).
• Linear algebra, Fourier transform, and random number features.
• Integration of C/C++ or Fortran code.
• With SciPy even more advanced mathematical functions.
• Installation guide on http://www.numpy.org/.
• If using Spyder NumPy is already available.
59/64
NumPy (continued)
Example Execution
1 import time l = 10000000 1
2 import numpy print("t List = ",list_ver(l)) 2
3 print("t NumPy = ",numpy_ver(l)) 3
4 def list_ver(length): t List = 0.999732255935669 4
5 X = list(range(length)) t Numpy = 0.029507875442504883 5
6 Y = list(range(length))
7 t1 = time.time()
8 Z = [0]*length
9 for i in range(length): Discussion
10 Z[i] = X[i] + Y[i]
11 return time.time() - t1 • Creation of two arrays (X, Y) of size 10000000.
12
13 def numpy_ver(length): • X and Y are summed and the result is stored in Z.
14 X = numpy.arange(length)
15 Y = numpy.arange(length) • These sums are performed with Lists or NumPy arrays.
16 t1 = time.time()
17 Z = X + Y • The performance is measured for the two cases.
18 return time.time() - t1
• With NumPy no for loop and more efficient.

60/64
NumPy: ndarray manipulations

Matrix-Scalar manipulations
Example NumPy Example List
1 import numpy as np 1 Y = [[0]*2, [0]*2]
2 2 X = [[1]*2, [1]*2]
3 X = np.ones((2,2)) 3 for i in range(2):
4 print(1+2.5*X) 4 for j in range(2):
5 [[ 3.5 3.5] 5 Y[i][j]=1+2.5*X[i][j]
6 [ 3.5 3.5]] 6
7 print(Y)
8 [[ 3.5 3.5], [ 3.5 3.5]]

NumPy applies the multiplication to the whole array.

61/64
NumPy: ndarray manipulations
Matrix-Matrix manipulations
Example NumPy Example List
1 import numpy as np 1 X = [list(range(2)), list(range(3,5))]
2 2 Y = [list(range(2)), list(range(3,5))]
3 X = np.arange(4).reshape(2,2) 3 Z = [[0]*2, [0]*2]
4 Y = np.arange(4).reshape(2,2) 4 for i in range(2):
5 Z = X+Y 5 for j in range(2):
6 print(Z) 6 Z[i][j]=X[i][j]+Y[i][j]
7 print(Z*Z) 7 print(Z)
8 [[0 2] 8 ZZ = [[0]*2, [0]*2]
9 [4 6]] 9 for i in range(2):
10 [[ 0 4] 10 for j in range(2):
11 [16 36]] 11 ZZ[i][j]=Z[i][j]*Z[i][j]
12 print(ZZ)
13 [[0, 2], [4, 6]]
14 [[0, 4], [16, 36]]

The addition/multiplication element-wise to the whole array.

62/64
NumPy: linalg

Basic linear algebra And many more

1
2
>>> import numpy as np
>>> A = np.arange(4).reshape(2,2)
• Determinant, Condition number.
3 >>> b = np.arange(1,3) • Cholesky decomposition, QR
4 >>> print(A,b,np.dot(A,b))
5 [[0 1] decomposition.
6 [2 3]] [1 2] [2 8]
7 >>> invA = np.linalg.inv(A) • Eigenvalue decomposition.
8 >>> print(np.dot(invA,A))
9 [[ 1. 0.] Among others.
10 [ 0. 1.]]
11 >>> print(np.dot(invA,b))
12 [-0.5 1. ]
13 >>> print(np.linalg.solve(A,b))
14 [-0.5 1. ]

63/64
 End of module
NumPy

End of Week 2
Thank you for your attention!

64/64