Pythonize Yourself Intro Tools Language Packages4Science Examples 1/XX

1/77

Pythonize Yourself

Pythonize Yourself

Michele Dei
[email protected]

Integrated Circuits and Systems (ICAS)

Instituto de Microelectrónica de Barcelona, IMB-CNM(CSIC)

This work ’as-is’ we provide.

No warranty express or implied.
May 2016
We’ve done our best,
to debug and test.
Liability for damages denied.

Permission is granted hereby,

to copy, share, and modify.
Use as is fit,
free or for profit.
These rights, on this notice, rely.

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 2/77

1 Introduction:
Matlabholics Anonymous - How to redeem

2 Tools:
How to get things done: Tools, IDEs, Installation

3 Language:
Enjoy the Lego
R

4 Packages4Science:
Four Horsemen of the Apocalypse:
NumPy, SciPy, Matplotlib, Sympy

5 Examples:
Dirty work (but someone has to do it)

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 2/77

1 Introduction:
Matlabholics Anonymous - How to redeem

2 Tools:
How to get things done: Tools, IDEs, Installation

3 Language:
Enjoy the Lego
R

4 Packages4Science:
Four Horsemen of the Apocalypse:
NumPy, SciPy, Matplotlib, Sympy

5 Examples:
Dirty work (but someone has to do it)

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 3/77

Targeting my audience, being a target too

I You already know what is a(n):

15
algorithm, array, argument, assignment, boolean, code,
conditional statement (if-then-else), function,
program, variable 25

I Don’t expect me to: 10 10

20
I be formal, be unbiased, be a guru
15
I port your MATLAB/GNU Octave scripts.
Some tools already have been designed to it, for ex.:
https://github.com/victorlei/smop
https://sourceforge.net/projects/ 10
libermate/
You can also run your M-file directly into Python:
https://pypi.python.org/pypi/oct2py

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 4/77

What is this about?

I Introduce Python and its core scientific packages

I Promote open source for scientific computing

I Present a Python integrated development environment

(IDE) similar to that of MATLAB
R

I Discuss the potentiality of Python through some practical

examples of common tasks in the scientific routine

The content of this presentation is inspired on:

I ”Python for MATLAB Users” material by Kitware Staff
http://www.kitware.com/cvpr2012.html
I Python Course by Bernd Klein
http://www.python-course.eu/

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 5/77

Ethics!

”Universalism requires that sci-

ence be independent of race, color,
or creed and that it should be
essentially international”

”Communalism requires that sci-

entific knowledge should be pub-
lic knowledge; [...] there should
be freedom of exchange of sci-
entific information between sci-
entists everywhere”

”Disinterestedness requires that

the results of bona fide scien-
tific research should not be ma-
nipulated to serve [...] personal
profit, ideology, or expediency” ”Organized skepticism requires
that statements should not be
accepted on the word of author-
ity, but that scientists should be
R.H. Brown. The Wisdom of Science. free to question them [...]”
Cambridge University Press. 1986.
M, Dei, IMB-CNM(CSIC) ICAS Tutorials
 Pythonize Yourself Intro Tools Language Packages4Science Examples 5/77

1 from mpl_toolkits.mplot3d import Axes3D

2 import matplotlib.pyplot as pp
3
4 fig = pp.figure(figsize=(5,5))
5 ax = Axes3D(fig)
6
7 # https://en.wikipedia.org/wiki/Tetrahedron
8 X = ( 1, 1, 1), ( 1, -1, -1),\
9 (-1, 1, -1), (-1, -1, 1)
10
11 keys = ’s’, ’color’, ’ha’, ’va’, ’size’, ’bbox’
12 bbox_props = dict(boxstyle="round4,pad=0.3",\
13 fc="w", ec="1.0", alpha=0.7)
14 labels = ’Universalism’, ’Communalism’,\
15 ’Desiterestedness’, ’Organized\n skeptisism’
16
17 def pt2ln(p1, p2, **kwargs):
18 s = (’xs’,’ys’,’zs’,’color’,’linewidth’,’alpha’)
19 p = lambda i: [p1[i], p2[i]]
20 return dict(zip(s, [p(0),p(1),p(2),’g’,8,0.5]))
21
22 for i1 in range(len(X)):
23 for i2 in range(i1+1, len(X)):
24 ax.plot(**pt2ln(X[i1], X[i2]))
25 ax.text(*X[i1], **dict(zip(keys, [labels[i1],
26 ’g’, ’center’, ’center’, 18, bbox_props])))
27
28 pp.title(’Mertonian paradigm’,fontsize=16,fontweight=’bold’)
29 ax.xaxis.set_ticklabels([])
30 ax.yaxis.set_ticklabels([])
31 ax.zaxis.set_ticklabels([])
32 pp.show()
mertonian paradigm.py

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 6/77

Why not MATLAB

R?
Nor MATLAB-like programs

I Expensive
I Influential establishment held by a pri-
vate company
”MATLAB , R the language of technical computing, is a pro-
gramming environment for algorithm development, data anal-
ysis, visualization, and numeric computation.”

> 5000 academias as customers

> 1 million users of MATLAB worldwide
> 500 third-party solutions built on MATLAB/Simulink
> 1700 MATLAB based books in 28 languages

I Structurally poor language

A major flaw is that promotes cut+paste coding by:

Function-name = file-name.
Ok for small code bases but discourages writing modular pro-
grams suitable for grown up projects, and at last the capability
of your code to be shared and be useful for the rest of the
community

http://www.mathworks.com/company/factsheet.pdf
http://www.mathworks.com/company/aboutus/index.html?s_tid=
gn_loc_drop
M, Dei, IMB-CNM(CSIC) ICAS Tutorials
 Pythonize Yourself Intro Tools Language Packages4Science Examples 7/77

Why not MATLAB

R?
Nor MATLAB-like programs

I Exercise #1

Provided that I have convinced you that MATLAB

R is:

1. an influential establishment hold by a private company;

2. a structurally poor language;
3. expensive,
in which sense its use in science offends the Mertonian paradigm:
universalism, communalism, disinterestedness, organized skeptisism?
I Exercise #2
Given that GNU Octave:
1. treats incompatibility with MATLAB
R as a bug;

2. is distributed under a GLPv3-”Free as in Freedom” licence,

discuss the following statement:

GNU Octave is a free-MATLAB, but not MATLAB-free

http://wiki.octave.org/FAQ
M, Dei, IMB-CNM(CSIC) ICAS Tutorials
 Pythonize Yourself Intro Tools Language Packages4Science Examples 8/77

The Python remedy

I Holistic (not just domain specific)
I Open and encouraging your vocation to share (algorithms, methods, results...)
I Exhibits excellent readability (if you can read it you can reuse it)
I Implement modular programming since functions and classes are organized in names-
paces
I Has powerful built-in data structures
I Scientific modules (and not only) are supported by a large and active community

1 # -*- coding: utf-8 -*-

2 import numpy as np
3 import matplotlib.pyplot as pp
4
x = np.linspace(0, 2, 1000)
Side effects?

5
6 pp.xkcd()
7 pp.figure(figsize=(3.5,3.5))
8 pp.plot(x, np.sqrt(x),color=’m’,\
9 label=r"Skiing: $\sqrt{t}$")
10 pp.plot(x,x**2,color=’g’,\
11 label=r"Snowboarding: $tˆ2$")
12 pp.xlabel("Time $t$") ; pp.ylabel("Skill")
13 pp.legend(loc=’upper center’,fontsize=14,\
14 fancybox=True,framealpha=0.25)
15 pp.show()
learning curve.py

https://www.stat.washington.edu/˜hoytak/blog/whypython.html
M, Dei, IMB-CNM(CSIC) ICAS Tutorials
 Pythonize Yourself Intro Tools Language Packages4Science Examples 9/77

More & less

I Cython: optimising static compiler for Python, allows calls back and forth from and to
C or C++ code natively at any point. easily tune readable Python code into plain C
performance by adding static type declarations
I Python can be integrated with other languages too (Fortran, Java, PHP ...):
https://wiki.python.org/moin/IntegratingPythonWithOtherLanguages
I What is Python used for? Google, Yahoo, Youtube, gaming...
https://wiki.python.org/moin/OrganizationsUsingPython
I Is used as extension language in Inkscape, Freecad, Klayout, Glade, Pymol:
https://en.wikipedia.org/wiki/List_of_Python_software#Embedded_as_a_scripting_language

I Build a compiled application optimizing speed and simplifying dependencies:

https://sourcecontribute.com/2015/05/02/compiling-python-to-standalone-executables-on-linux-using-nu

There is no 1 to 1 replacement of Simulink in Python

Anyway we can take advantage of the interoperability of Scilab/Python and use the Xcos to
design dynamical systems models. https://www.scilab.org/scilab/features/
xcos
The last example in this presentation is focused on this issue.

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 10/77

1 Introduction:
Matlabholics Anonymous - How to redeem

2 Tools:
How to get things done: Tools, IDEs, Installation

3 Language:
Enjoy the Lego
R

4 Packages4Science:
Four Horsemen of the Apocalypse:
NumPy, SciPy, Matplotlib, Sympy

5 Examples:
Dirty work (but someone has to do it)

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 11/77

Get the tools

DIE HARD
Build Python (latest version) , config Vi(m) or Emacs
https://docs.python.org/2/using/unix.html

apt-get install python, than pip modules. Get Ipython qt-

console and configure it with fancy text editors

The same as above but using Synaptic (Ubuntu based Linux) or

other software manager. Main modules can also be installed from
there (Numpy, Scipy, Matplotlib, Sympy...)
Spyder is a complete IDE that can be used, integrating both: an
editor tool and Ipython qt-console(s).

You can rely on a Python package distribution provided by third par-

ties: Continuum R Analytics Anaconda, Pyzo or Enthought Canopy.
Download, Install and Launch. (Batteries included)
Marshmallow https://www.continuum.io/downloads

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 12/77

IDE = Editor + Console

I Python on terminal
you may want to use it as command line
tool

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 12/77

IDE = Editor + Console

I Python on terminal
you may want to use it as command line
tool

I IPython on terminal
Interactive with graphical capabilities,
shell syntax, tab completion, and history.
It has %magic commands

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 12/77

IDE = Editor + Console

I Python on terminal
you may want to use it as command line
tool

I IPython on terminal
Interactive with graphical capabilities,
shell syntax, tab completion, and history.
It has %magic commands

I IDLE
Syntax highlighting, debugger

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 12/77

IDE = Editor + Console

I Python on terminal
you may want to use it as command line
tool

I IPython on terminal
Interactive with graphical capabilities,
shell syntax, tab completion, and history.
It has %magic commands

I IDLE
Syntax highlighting, debugger

I Spyder
Syntax highlighting, debugger, intro-
spection for code completion, support
for multiple Python consoles (including
IPython), has the ability to explore and
edit variables from a GUI

IDE >Editor + Console

A plenty of IDEs: https://wiki.python.org/moin/IntegratedDevelopmentEnvironments

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 13/77

1 Introduction:
Matlabholics Anonymous - How to redeem

2 Tools:
How to get things done: Tools, IDEs, Installation

3 Language:
Enjoy the Lego
R

4 Packages4Science:
Four Horsemen of the Apocalypse:
NumPy, SciPy, Matplotlib, Sympy

5 Examples:
Dirty work (but someone has to do it)

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 14/77

About the language

I It is interpreted1and dynamically typed2

I It is case sensitive and indentation sensitive.
I Python has two basic modes:
Script: as a command line tool to run scripts
Interactive (>>>, [1]:): immediate feedback for each statement.
The user provides statements line by line. quit() or Ctrl-D to exit

Let’s start to scratch the surface...

1 Notcompiled, like C. The interpreter executes instructions directly, without previously

compiling a program into machine-language instructions. The interpreter executes the
program directly, translating each statement into a sequence of one or more subroutines
already compiled into machine code.
2 Variables do not have a fixed type. Differently from statically typed languages, variables
are not a segment of the memory where some value is written, they are tags or names
pointing to objects. It is therefore possible for the variable a to be set to the value 1, then
to the value "a string", then to a function.

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 15/77

Calculator
>>> 17 / 3 # Comment: int / int -> int
5
>>> 17 / 3.0 # int / float -> float
5.666666666666667
>>> 17 // 3.0 # explicit floor division discards the fractional part
5.0
>>> 17 % 3 # the % operator returns the remainder of the division
2
>>> _ ** 3 # Exponentiation, the underscore is the last returned value
8
>>> (-1 + 3j)*2j # Complex numbers, natively supported
(-6-2j)

Numerical Special Container

int, (long) bool str

limited, unlimited
values: True or False string of characters
precision (Python 2)

float None tuple

Floating point, fixed sequence of
Null object
(Some) Built-in Types: double precision objects

complex list
mutable sequence of
real and imaginary floats
objects

dict
maps one set of objects to
another

The world ”object” here appears several times, please be patient...

M, Dei, IMB-CNM(CSIC) ICAS Tutorials
 Pythonize Yourself Intro Tools Language Packages4Science Examples 16/77

Playing around
>>> type(’float’) # String using single quotes
<type ’str’>
>>> type(8), type(8.1) # A sequence of inputs separated by a comma
(<type ’int’>, <type ’float’>)
>>> bool(-3), bool(0.0) # How much truth in a number?
(True, False)
>>> float(7), complex(3.1) # Conversion of numerical types
(7.0, (3.1+0j))
>>> a = complex(3, -4) # Assignment using complex() constructor
>>> abs(a) # Absolute function, consistent with the argument type
5.0
>>> int(a) # Will throw an exception ’TypeError’ since it is ambiguous
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can’t convert complex to int ’
>>> str(0.278), str(True), int(’23’) # Back and forth from str
(’0.278’, ’True’, 23)

Boolean operators: and, or, not

Comparisons: <, <=, >, >=, ==, !=, is, is not
>>> x, y = 3, 4.4
>>> x == y , x is y, id(x), id(y)
(False, False, 35246376, 35317432)
>>> x = y
>>> x == y , x is y, id(x), id(y) Multiple assignment
(True, True, 35317432, 35317432)
>>> z = 4.4 Identity:
>>> z == y, z is y Every Python object has an identity. The function
(True, False) id(object) returns a unique identifying integer

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 17/77

Lists & Tuples

Sequences of items, of any type, mutable and immutable, respectively.
I Tuple: comma separated items, enclosed in parentheses (), sometimes omitted
I List: comma separated items, enclosed in square brackets [], mandatory
>>> t0 = 5, ’ciao’, (3.14, None) # The last item is a tuple of two items
>>> t0[0], t0[2] # can be accessed by indexing (0-based)
(5, (3.14, None))
>>> t0[1] = ’hola’ # They are immutable!
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: ’tuple’ object does not support item assignment
>>> l0 = list(t0) # Cast the tuple into a list
>>> l0[1] = ’hola’ # No complaints now
>>> l0 # Note the output:
[5, ’hola’, (3.14, None)]

Common operations and functions Lists have more capabilities (methodsa)

to both: than tuples:
>>> ’hola’ in t0, ’hola’ in l0 # test inclusion >>> dir([])
(False, True) [’append’, ’count’, ’extend’, ’index’,
>>> t0 + (’hola’,) # Concatenate item ’insert’, ’pop’, ’remove’, ’reverse’,
(5, ’ciao’, (3.14, None), ’hola’) ’sort’]
>>> t0 * 2 # Concatenate shallow copies
(5, ’ciao’, (3.14, None), 5, ’ciao’, (3.14, None)) Tuples don’t, but simplicity makes them
>>> len(l0) # Length of sequence computationally efficient.
3
a
Again, be patient

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 18/77

Indexing and slicing

Zero based indexing3.
>>>
>>>
a = range(1,6)
a[0], a[-1]
# Create a sequence of integers
# Indexing
Index:
(1, 5) 0 1 2 3 4 5
>>> a[1:3] # Slicing: index 1 to 3, all within
[2,
>>>
3]
a[:-1] # Slicing: index 0 to last-1
Item: 1 2 3 4 5
[1, 2, 3, 4]
>>> a[:-1:2] # Slicing: same as above, step 2 -5 -4 -3 -2 -1
[1, 3] range(stop), return a list of integers in the half
>>> a[-3:-1] # Slicing: index -3 to -1, all within open interval [0; stop)
[3, 4]
>>> a[3:1:-1] # Beware! range(start, stop[, step]) list of inte-
[4, 3] gers in the half open interval [start; stop),
step specifies the increment/decrement
Slice objects:
>>> s = slice(1, 4, 2)
>>> a[s] slice(stop)
[2, 4] slice(start, stop[, step])
>>> s = slice(3, len(a))
>>> a[s] len
[4, 5] number of items of a sequence or collection

a.index(x) index of the first occurrence of x in a

Very usefull methods:
a.count(x) total number of occurrences of x in a

3 http://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/
EWD831.html

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 19/77

Strings

I Immutable sequences of characters

I Many built-in methods:
>>> s = "Hello World"
>>> s.upper() # upper case
’HELLO WORLD’
>>> s[::-1].split() # splitting
[’dlroW’, ’olleH’]
>>> "_".join((s,s[::-1])) # join with a given separator
’Hello World_dlroW olleH’
>>> s.replace("Hello", "Goodbye") # substring replacement
’Goodbye World’
>>> s + ’ %g’%(1/3.0) # sprintf-like formatter
’Hello World 0.333333’
>>> s = """ # triple quotes string
... This is typically used in docstrings
... to document a segment of code.
... They are important metada!
... """

I Many more sophisticated ways to format strings:

https://docs.python.org/2/library/string.html

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 20/77

Dictionaries

I Mutable mapping of one object to another

I key:value pairs are comma separated and enclosed in curly braces {}
I Can be created using the dict() constructor
>>> a = dict(one=1, two=2, three=3)
>>> b = {’one’: 1, ’two’: 2, ’three’: 3}
>>> c = dict(zip([’one’, ’two’, ’three’], [1, 2, 3]))
>>> d = dict([(’two’, 2), (’one’, 1), (’three’, 3)])
>>> e = dict({’three’: 3, ’one’: 1, ’two’: 2})
>>> a == b == c == d == e
True
>>> ’four’ in a, ’one’ in b
(False, True)
>>> c[’two’] = 22
>>> c.values()
[3, 22, 1]
>>> c.keys()
[’three’, ’two’, ’one’]

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 21/77

Flow control4
if <condition1>: for <item> in <iterable>: while <condition>:
... ... ...
elif <condition2>:
... for <item> in <iterable>: while <condition1>:
... ... ...
elif <conditionN>: if <condition>: if <condition2>:
... continue continue
else: ... ...
...
for <item> in <iterable>: while <condition1>:
... ...
if <condition>: if <condition2>:
break break
... ...

Iterable: An object capable of returning its members one at a time. Examples: all Error handling
sequence types (list, str, and tuple) and some non-sequence types like dict Sometimes: ”it’s better to beg forgive-
and file. ness than ask permission”
XRange Type: The xrange type is an immutable sequence which is commonly used try:
for looping. The advantage of the xrange type over range is that the former will 1/0
always take the same amount of memory, no matter the size of the range it represents. except ZeroDivisionError:
range(100000), is list of 100000 elements! xrange(100000) not. print(’1/0!’)
>>> tuple(enumerate([6, 4, 5, 9])) assert(1 > 0)
((0, 6), (1, 4), (2, 5), (3, 9))
Enumerate: >>> for i, v in enumerate([6, 4, 5, 9]): assert for checking inputs or interme-
... print(i, v) diate results. If not asserted, raises an
AssertionError

4 https://docs.python.org/2/tutorial/controlflow.html
https://docs.python.org/2/tutorial/errors.html

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 22/77

Functions
Suppose you want a function that calculates the roots of ax2 + bx + c = 0.
1 def quadratic_formula(a, b, c): >>> quadratic_formula(1, -3, 2)
2 dis = (b**2 - 4*a*c)**0.5 (1.0, 2.0)
3 return (-dis-b)/(2*a), (dis-b)/(2*a)
quadratic0.py >>> quadratic_formula(1, 3, 3)
Traceback (most recent call last):
...
ValueError: negative number cannot be raised to
a fractional power

NOTE: def, :, 4-spaces indentation, return.

When the return statement is not declared, the function returns a Null object.
Let’s fix the ValueError:
1 def quadratic_formula(a, b, c): >>> quadratic_formula(1, 3, 3)
2 dis = (b**2 - 4*a*c + 0j)**0.5 ((-1.5-0.8660254037844386j),
3 return (-dis-b)/(2*a), (dis-b)/(2*a) (-1.5+0.8660254037844386j))
quadratic1.py
>>> quadratic_formula(1, -3, 2)
((1-0j), (2+0j))

I What if we want only real solutions?

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 23/77

Functions ...
Optional arguments:
1 def quadratic_formula(a, b, c, real=False): >>> quadratic_formula(1, -3, 2)
2 dis = (b**2 - 4*a*c + 0j)**0.5 ((1-0j), (2+0j))
3 if real:
4 if dis.imag == 0: >>> quadratic_formula(1, -3, 2, real=True)
5 return (-dis.real-b)/(2*a), (1.0, 2.0)
6 (dis.real-b)/(2*a)
7 else: a, b, c are positional arguments
8 return (-dis-b)/(2*a), (dis-b)/(2*a) real is a keyword argument
quadratic2.py keyword arguments come always after positional arguments

I Exercise: What is returned by quadratic_formula(1, 3, 3, real=True)?

Before continuing let’s do the following experiment on a new python console:

dir()
>>> dir() If called without an argument, return the names in the
[’__builtins__’, ’__doc__’, ’__name__’, ’__package__’] current scope.
>>> __name__
’__main__’

I The variable name is already defined, and set to the value ’ main ’.
I This allows the programmer to control the behaviour of the script whether it is the main
program executed or not.

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 24/77

Script behaviours: !#

1 #! /usr/bin/env python
#!
2 def quadratic_formula(a, b, c, real=False): (shebang) directive:
3 """ instructs the loader which program to look for
4 Returns the roots of: a*x**2 + b*x + c = 0 executing the current script.
5 Only real solutions are returned if real is True
6 """ For Python scripts:
7 dis = (b**2 - 4*a*c + 0j)**0.5 #!/usr/bin/python
8 if real:
9 if dis.imag == 0: or, even bettera
10 return (-dis.real-b)/(2*a), (dis.real-b)/(2*a)
11 else: #!/usr/bin/env python
12 return (-dis-b)/(2*a), (dis-b)/(2*a)
13 The interpreter ignores it, as it is a comment
14 if __name__ == ’__main__’: a
https://mail.python.org/pipermail/
15 print(’This script contains the definition of a function’) tutor/2007-June/054816.html
16 else:
17 print(__name__)
quadratic3.py

Your script is now ready to be used in a shell, once you have changed its privileges to be
executable:
$ chmod a+x quadratic3.py
$ ./quadratic3.py
This script contains the definition of a function
$ _

What about the else part?

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 25/77

Script behaviours: import

>>> import quadratic3

quadratic3 On import the else part is executed.

>>> dir(quadratic3)
[’__builtins__’, ’__doc__’, ’__file__’, quadratic3 has its own attributes:
’__name__’, ’__package__’, ’quadratic_formula’] it has a name...

>>> quadratic3.__name__
’quadratic3’

>>> quadratic3.__file__
’quadratic3.py’

>>> help(quadratic_formula) The docstrings allows the help() function

Help on function quadratic_formula in module __main__: to return them if queried

quadratic_formula(a, b, c, real=False)
Returns the roots of: a*x**2 + b*x + c = 0
Only real solutions are returned if real is True

>>> quadratic_formula(1, -2, 1, True) quadratic_formula() is now available for

(1.0, 1.0) use

Functions, classes, and more generally, objects, can be defined and reused im-
porting them. From now on cut+paste code from/to your scripts is capital crime!

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 26/77

import is very important

I How to use simultaneously the definition of quadratic formula() from quadratic1.py

and quadratic3.py?
from ... import ... as ...
In [1]: from quadratic1 import quadratic_formula as qf1
In [2]: from quadratic3 import quadratic_formula as qf3 %timeit This magic command provided by
quadratic3 the IPython interactive interpreter provide a
In [3]: %timeit [qf1(1,0,i) for i in xrange(1000)] simple way to time small bits of Python code
1000 loops, best of 3: 1.69 ms per loop [... for ...] ? Be patient!
In [4]: %timeit [qf3(1,0,i) for i in xrange(1000)]
1000 loops, best of 3: 1.76 ms per loop
The import quadratic1 statement will look for the proper file, which is quadratic1.py in the same directory as the caller. If it is
not found, the Python interpreter will search “path” recursively and raise an ImportError exception if it is not found.
from some_module import *
Very bad: [...]
Is some function part of some module? A built-in? Defined above? x = some_function(4)

from some_module import some_function

Better: [...]
some function may be part of some module, if not redefined in between x = some_function(4)

import some_module
Best: [...]
some function visibility as part of some module namespace x = some_module.some_function(4)

I as allows renaming in the local namespace

http://docs.python-guide.org/en/latest/writing/structure/
https://ipython.org/ipython-doc/dev/interactive/magics.html

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 27/77

Scripts, passing arguments

I Suppose you want your script with quadratic formula() be accessible directly at
command line level. You may want to pass additional arguments to the interpreter.
$ python script_name.py argument(s) The script name and additional arguments thereafter are turned into a list
of strings and assigned to the argv variable in the sys module.
1 #! /usr/bin/env python
2 import sys $ python quadratic4.py 4 0 -100
3 import quadratic3 as q3 quadratic3
4 ((-5+0j), (5+0j))
5 if __name__ == ’__main__’:
6 arg = sys.argv $ python quadratic4.py 4 0 -100 True
7 try: quadratic3
8 assert(len(arg) >= 4) (-5.0, 5.0)
9 a,b,c = float(arg[1]),float(arg[2]),float(arg[3])
10 real = False $ python quadratic4.py 4 Z -100 True
11 if len(arg) > 4: quadratic3
12 real = True if arg[4] == ’True’ else ’False’ Invalid arguments
13 if a == 0:
14 try: $ python quadratic4.py
15 print(-c/b) Help on function quadratic_formula in
16 except ZeroDivisionError: module quadratic3:
17 print(’No unknown to solve for’)
18 else: quadratic_formula(a, b, c, real=False)
19 print(q3.quadratic_formula(a, b, c, real)) Returns the roots of: a*x**2 + b*x
20 except AssertionError: + c = 0
21 help(q3.quadratic_formula) Only real solutions are returned
22 except ValueError: if real is True
23 print(’Invalid arguments’) (END)
quadratic4.py

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 28/77

More on passing arguments

Argument: value passed to a function (or method) when calling the function.
There are two types of arguments:
I Keyword argument:
an argument preceded by an identifier (e.g. name=) in a function call or >>> complex(real=3, imag=5)
passed as a value in a dictionary preceded by **.
For example, 3 and 5 are both keyword arguments in the following calls to >>> complex(**{’real’: 3, ’imag’: 5})
complex():

I Positional argument:
an argument that is not a keyword argument. Positional arguments can >>> complex(3, 5)
appear at the beginning of an argument list and/or be passed as elements
of an iterable preceded by *. >>> complex(*(3, 5))
For example, 3 and 5 are both positional arguments in the following calls:

Some remarks:
>>> def fun0(*args, **kwargs): >>> x, y = (1,2,3), (4,5,6)
... print(args) >>> z = zip(x,y); print(z)
... print(kwargs) [(1, 4), (2, 5), (3, 6)]
>>> xx, yy = zip(*z)
>>> fun0(1, 2, third=3, forth=4) >>> print(xx, yy)
(1, 2) ((1, 2, 3), (4, 5, 6))
{’forth’: 4, ’third’: 3}
zip()
Note nor args neither kwargs are known previously
from function side. args, kwargs are conventional
names.

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 29/77

Functional programming5

I a, b, c = float(arg[1]), float(arg[2]), float(arg[3]) : yuck!

List comprehensions: [ expression for item in list (if conditional) ]
>>> o = [i*i for i in range(10)]

Adding a simple condition:
>>> o = [] 

>>> for i in range(10) >>> o = [i*i for i in range(10) if i%2==0]
... o.append(i*i)
−→
Adding a multiple condition:


>>> o = [i*i if i%2==0 else -i*i for i in range(10)]

Mapping a function to a sequence: map() and the lambda operator

>>> sq = lambda x: x*x Beware!
>>> o = map(sq, range(10)) >>> cb = lambda x: x*x*x
>>> f = [sq, cb]
>>> o = map(lambda x: x*x, range(10)) >>> o2 = [map(lambda y: y(x), f) for x in range(10)]

What about the if? filter()

o = filter(lambda x: x%2-1, range(10))
o = map(lambda x: x*x, filter(lambda x: x%2-1, range(10))) # Beware!

Want more? reduce(), applies a function to a sequence iteratively.

See for ex. http://www.python-course.eu/lambda.php
5 However this is a controversial topic:
http://stackoverflow.com/questions/1017621/why-isnt-python-very-good-for-functional-programming

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 30/77

Classes
I What is a class? It’s about the definition of things. def: defines a function, class
defines a logical grouping of data and functions. We refer to them as attributes and
methods, respectively.

A rapid search in the web for ”class examples in Python” gives:

bank accounts, vehicles, persons, pets...

that’s because objects created from a class aim to mimic real world ”objects”.

How to spot the usefulness of classes in a (probably) more familiar context?

https://xkcd.com/26/
You have a big database of signals:
I some of them are represented in the time domain, others in frequency domain.
Each of them comes with its time (∆t) or frequency resolution (∆f ), for simplicity uni-
formly spaced. You may wish to switch from one representation to another. Direct and
inverse fft()
I you may want to analyse them from a power spectral density (psd) point of view, and
associate to each signal its proper windowing function, or more advanced filtering to ex-
tract the signal from unwanted interference. psd(), window, averages number,
filt(), filt params = { ... }
I you may also want some specialized visualization tools for your data: psd plot, an his-
togram plotting the statistical distribution of amplitudes, etc. psd plot(), histo()

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 31/77

Classes ...
A skecth of your class would look like:
After defining your class you would use it simply like:
1 class Signal():
2 def __init__(self, x, domain, res): >>> x = Signal(raw_data, ’freq’, 1e2)
3 assert(domain in [’time’, ’freq’]) >>> x.filt(5e3, 50e3) # use a method!
4 TF = domain == ’time’ >>> x.histo()
5 self.xt = x if TF else self.fft(x, res, -1) ...
6 self.xf = self.fft(x, res) if TF else x
7
x is now an object with all its attributes (x.xt, x.xf
8 def fft(self, x, dx, direction = 1): ...) tidily bundled and organized.
9 pass # to be implemented! It has capabilities (methods) like histo(), psd(), etc.
10
that operates on it, and are specialized functions that op-
11 def psd(self, window): erate in a well defined environment.
12 self.window = window
13 pass # to be implemented! __init__()? Initialization method, needed to initialize
14 the instance.
15 def filt(self, **filt_params):
16 pass # to be implemented! self?
17 used to declare a method or an a attribute of the instance!
18 def psd_plot(self): Without it all would look like local variables and the code
19 pass # to be implemented! would be ambiguous
20
21 def histo(self):
22 pass # to be implemented!

Inheritance? Operator overloading? Polymorphism? Decorators? You can dig more6!

6 For example: http://python-textbok.readthedocs.io/en/1.0/Classes.html

https://jeffknupp.com/blog/2014/06/18/
improve-your-python-python-classes-and-object-oriented-programming/
https://julien.danjou.info/blog/2013/guide-python-static-class-abstract-methods

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 32/77

1 Introduction:
Matlabholics Anonymous - How to redeem

2 Tools:
How to get things done: Tools, IDEs, Installation

3 Language:
Enjoy the Lego
R

4 Packages4Science:
Four Horsemen of the Apocalypse:
NumPy, SciPy, Matplotlib, Sympy

5 Examples:
Dirty work (but someone has to do it)

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 33/77

Stack/Ecosystem

NumPy, the fundamental package for numerical computation.

It defines the numerical array and matrix types and basic
operations on them.

The SciPy library, a collection of numerical algorithms and

domain-specific toolboxes, including signal processing,
optimization, statistics and much more.

Matplotlib, a mature and popular plotting package, that

provides publication-quality 2D plotting as well as rudimentary
3D plotting.

SymPy, for symbolic mathematics and computer algebra.

Anyway the ”ecosystem” includes much more tools, for ex.:

Pandas (high-performance data structures), Mayavi (3D visualization), Cython (C extensions and integration), Numba for just-in-time
code compilation, etc.

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 34/77

Numpy

I Open source extension module for Python: import numpy as np

I Provides fast precompiled numerical routines for (large) multi-dimensional arrays
Create
>>> a = np.array([[1, 2, 3], [4, 5, 6]])
array([[1, 2, 3], arange([start,] stop[, step,], dtype=None)
[4, 5, 6]]) array of evenly spaced values within the half-open interval
[start, stop)
>>> np.arange(1,10,2)
linspace(start, stop, num=50, endpoint=True,
array([1, 3, 5, 7, 9])
retstep=False)
Returns num evenly spaced samples, calculated over the
>>> np.linspace(0, 10, 4, endpoint=False)
interval [start, stop]
array([ 0. , 2.5, 5. , 7.5])
>>> np.linspace(0, 10, 4, endpoint=True)
logspace(start, stop, num=50, endpoint=True,
array([ 0. , 3.33333333, 6.66666667, 10. ])
base=10.0)
samples uniformly distributed in log space
>>> np.logspace(0, 3, 4)
array([ 1., 10., 100., 1000.]) meshgrid
N-dimensional coordinate arrays useful to evaluate func-
>>> x, y = np.arange(-5, 5, .1), np.arange(-5, 5, .1) tions on a grid
>>> xx, yy = meshgrid(x, y, sparse=True)
>>> z = np.sin(xx**2 + yy**2) / (xx**2 + yy**2)

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 35/77

Create
>>> np.ones((2,3))
array([[ 1., 1., 1.], ones(shape, dtype=None, ...)
[ 1., 1., 1.]]) zeros(shape, dtype=None, ...)
>>> np.ones(4) ones_like(a, dtype=None, ...)
array([ 1., 1., 1., 1.]) zeros_like(a, dtype=None, ...)
>>> a = np.zeros(3)
>>> np.ones_like(a)
eye(N, M=None, k=0, dtype=<type ’float’>)
array([ 1., 1., 1.])
Return a 2-D array with ones on the diagonal and zeros
...
elsewhere. M columns, k index of diagonal.
>>> np.eye(3)
array([[ 1., 0., 0.],
[ 0., 1., 0.], k: -1 0 1
[ 0., 0., 1.]]) 11 12 13
>>> np.eye(3, k=-2) + np.eye(3, k=1)
array([[ 0., 1., 0.], 21 22 23
[ 0., 0., 1.], 31 32 33
[ 1., 0., 0.]])

Simple tests
>>> a = np.array([[1, 2], [3, 4]]) dtype
>>> type(a), a.dtype, len(a), np.shape(a), np.size(a) shape
(<type ’numpy.ndarray’>, dtype(’int64’), 2, (2, 2), 4) size
>>> np.unique(a)
unique
array([1, 2, 3, 4])
Extraction of unique elements
>>> a.diagonal() diagonal
array([1, 4])

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 36/77

Simple manipulation
>>> a.transpose() >>> a In multidimensional arrays you
array([[1, 3], transpose array([[1, 2],
both as method and function have the axis optional argu-
[2, 4]]) [3, 4]]) ment:
>>> np.transpose(a)
array([[1, 3], >>> a.sum()
[2, 4]]) 10

axis=0
>>> a.T >>> a.sum(axis=0)
array([[1, 3], array([4, 6])

2
[2, 4]]) >>> a.prod(axis=1)

=
is
array([ 2, 12])

ax
>>> a.reshape(4,1) reshape, flatten axis=1
array([[1], both as methods and functions >>> a.mean(axis=1)
sum, prod
[2], array([ 1.5, 3.5])
[3], mean, var, std, max, min
[4]]) >>> a.argmin(axis=1)
>>> a.flatten() array([0, 0]) argmax, argmin
array([1, 2, 3, 4]) >>> a.argmax(axis=0) array indices of the minimum
array([1, 1]) and maximum values
>>> a.clip(2,3) clip, fill
array([[2, 2], >>> b = a copy
[3, 3]]) >>> b.fill(0); a
>>> a.fill(9); a array([[0, 0],
array([[9, 9], [0, 0]])
[9, 9]])
>>> a = np.copy(b)
>>> a = np.eye(2); b
array([[0, 0],
[0, 0]])

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 37/77

More refined manipulation and testing

>>> a = np.where(a>2, 0, a)
array([[1, 2], where Tests returning boolean values:
[0, 0]]) any() checks if at least one el-
ement of the array is True or
>>> 2 in a in greater than zero
True all() checks if all elements are
True of greater than zero
>>> np.nonzero(a) nonzero isnan() checks if the element if
(array([0, 0]), array([0, 1])) Return the indices of the ele- not-a-number NaN
>>> a.nonzero() ments that are non-zero isinf() infinity check
(array([0, 0]), array([0, 1]))

Element wise agebra and broadcasting Element wise functions

+ - * / % ** == != > >= < <=
abs, sign, sqrt, log, log10, exp,
sin, cos, tan, arcsin, arccos, arctan,
11 12 13 1 2 3 12 14 16 sinh, cosh, tanh, arcsinh, arccosh, andarctanh.
21 22 23 1 2 3 22 24 26 floor ceil, rint
31 32 33 1 2 3 32 34 36

11 12 13 1 1 1 12 13 14
21 22 23 2 2 2 23 24 25
Exercises
Which is the result of:
31 32 33 3 3 3 34 35 36
I np.array([[1, 2, 3]]).T*np.array([1, 2, 3])?
11
1 1 1 1 2 3 2 3 4 I np.arange(1,10).reshape((3,3))**2?
2 2 2 1 2 3 3 4 5
3 3 3 1 2 3 4 5 6

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 38/77

A simple speed test

Using the IPython console, we have the %timeit command which
1 # -*- coding: utf-8 -*- returns also statistical information:
2 import time
3 import numpy as np In [1]: %timeit fahrenheit =
4 [x*9/5 + 32 for x in Tl]
5 Tm = 25.0 # mean temperature 1000 loops, best of 3: 717 us per loop
6 Td = 5.0 # temperature standard deviation
7 ns = 1000 # number of samples In [2]: %timeit fahrenheit = T*9/5 + 32
8 100000 loops, best of 3: 14.1 us per loop
9 # Generate the a normally distributed array
10 # of temperatures
11 T = np.random.normal(Tm, Td, ns)
12 Tl = list(T) # convert to list
13
14 # list approach:
15 t0 = time.time()
16 fahrenheit = [x*9/5 + 32 for x in Tl]
17 t_list = time.time() - t0
18
19 # numpy approach:
20 t0 = time.time()
21 fahrenheit = T*9/5 + 32
22 t_array = time.time() - t0
23
24 # print ratio
25 print str(t_list/t_array)
list vs nparray.py

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 39/77

Indexing and slicing

>>> a = np.arange(10); a a[start:stop:step]
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

>>> a[0]
0-based indexing
0

>>> a[-1] The -1, (-2, -3, ...)

9

>>> a[2:6] Slicing

array([2, 3, 4, 5])

>>> a[::-1] Reverse ordering

array([9, 8, 7, 6, 5, 4, 3, 2, 1, 0])

>>> b = a[1:].reshape((3,3)) Indexing multidimensional arrays

>>> b[1][1], b[1,1]
5, 5

I Whereas slicings on lists and tuples create new objects, a slicing operation on an array
creates a view on the original array. So we get an another possibility to access the
array, or better a part of the array. From this follows that if we modify a view, the
original array will be modified as well.

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 40/77

Array selectors
I Selection by booleans I Selection by indexes
>>> a = np.array([[6, 4], [5, 9]], float) >>> a = np.array([2, 4, 6, 8], float)
>>> a >= 6 >>> b = np.array([0, 0, 1, 3, 2, 1], int)
array([[ True, False], >>> a[b]
[False, True]], dtype=bool) array([ 2., 2., 4., 8., 6., 4.])
>>> a[a >= 6]
array([ 6., 9.]) >>> a[np.where(a>5)]
>>> a[np.logical_and(a > 5, a < 9)] array([ 6., 8.])

>>> a[np.nonzero(a>5)]
array([ 6., 8.])

Structured arrays
>>> dtype = np.dtype([(’month’, ’S10’), (’day’, ’S10’), (’number’, ’i4’)])

>>> calendar = np.array([(’April’, ’Saturday’, 30),(’May’, ’Sunday’, 1)], dtype=dtype)

>>> calendar[’month’]
array([’April’, ’May’],
dtype=’|S10’)
dtype() declares a data type. ’S10’ stands for a string of 10 characters, ’i4’ stands for integer of 4 bytes.
Detailed description of primitive types:
http://docs.scipy.org/doc/numpy-1.5.x/reference/arrays.scalars.html

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 41/77

Array algebra
>>> a, b = np.array([1, 2, 3]), np.array([0, 1, 1])
>>> np.dot(a,b) dot()
5 dot product

NumPy also comes with a number of built-in routines for linear algebra calculations. These
can be found in the sub-module linalg.

>>> a = np.eye(8) - 4*np.ones((8,8)) det()

>>> np.linalg.det(a) Determinant
-31.0 eig()
Eigenvectors, eigenvalues
>>> vals, vecs = np.linalg.eig(a) inv()
Inverse
>>> b = np.linalg.inv(a) (+ Singular value decomposition, and more)

Polynomials
poly() Given a set of roots, returns the polynomial co-
>>> np.poly([-1, 1, 1, 10]) efficients
poly() Given a set of polynomial coefficients, returns
>>> np.roots([1, 4, -2, 3]) the roots
polyadd(), polysub(), polymul(), polydiv()
>>> np.polyint([1, 1, 1, 1]) Polynomial algebra
polyder(), polyint() Derivation, integration (con-
>>> np.polyder([1./4., 1./3., 1./2., 1., 0.]) stant of unknown term is 0)
polyval() Evaluates a polynomial at a particular point
... polyfit() fot data to polynomial (least-squares error)

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 42/77

Random numbers: random submodule

>>> np.random.seed(293423)
random() Return random floats in the half-open interval
>>> np.random.random() [0.0, 1.0)
rand() An array of random numbers in the half-open in-
>>> np.random.rand(5) terval [0.0, 1.0)
>>> np.random.rand(2, 3) randint() An array of random integers in the half open
specified interval
>>> np.random.randint(5, 10) normal() Gaussian probability distribution with µ mean
and σ standard deviation
>>> np.random.normal(1.5, 4.0) uniform() Uniform probability distribution in the speci-
fied half-open interval
>>> np.random.uniform(1.5, 4.0)
%

NumPy also includes generators for many other distributions:

Beta, binomial, chi-square, Dirichlet, exponential, F, Gamma, geometric, Gumbel, hyper-
geometric, Laplace, logistic, log- normal, logarithmic, multinomial, multivariate, negative
binomial, noncentral chi-square, noncentral F, normal, Pareto, Poisson, power, Rayleigh,
Cauchy, Student’s t, triangular, von Mises, Wald, Weibull, and Zipf distributions.

It also allows: (Weigthed) Random Choices Random Intervals Random Samples

http://www.python-course.eu/python_numpy_probability.php

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 43/77

Scipy?

Scipy is composed of task-specific sub-modules:

scipy.cluster Vector quantization / Kmeans

scipy.constants Physical and mathematical constants
scipy.fftpack Fourier transform
scipy.integrate Integration routines
scipy.interpolate Interpolation
scipy.io Data input and output
scipy.linalg Linear algebra routines
scipy.ndimage n-dimensional image package
scipy.odr Orthogonal distance regression
scipy.optimize Optimization
scipy.signal Signal processing
scipy.sparse Sparse matrices
scipy.spatial Spatial data structures and algorithms
scipy.special Any special mathematical functions
scipy.stats Statistics
http://docs.scipy.org/doc/scipy/reference/
M, Dei, IMB-CNM(CSIC) ICAS Tutorials
 Pythonize Yourself Intro Tools Language Packages4Science Examples 44/77

Matplotlib Backends

Spyder: Tools > Preferences Or interactively/scipting

through the magic commands
in IPython:
Activate the Qt backend:
>>> %matplotlib qt

Back to the inline backend:

>>> %matplotlib inline

Pylab?
from pylab import *

includes the core parts of

numpy, scipy, and matplotlib in
the default namespace.
Useful for interactive work,
discouraged for non interactive
flow (i.e. scripting)

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 45/77

Matplotlib Pyplot

I matplotlib.pyplot is a module in matplotlib: Provides a MATLAB-like plotting

framework by interfacing to the underlying plotting library.
For example figure objects and axes object are implicitly created to achieve the desired
plot when the method plot is called.
I in scripting mode it is usually imported as plt:
import matplotlib.pyplot as plt

Controlling the interactive updating of plots7

I plt.isinteractive(): returns the interactive setting True|False
I plt.ion(): turns interactive mode on
I plt.ioff(): turns interactive mode off
I plt.draw(): forces a figure redraw
When working with a big figure or a number of figures in which drawing is expensive, you
may want to turn matplotlib’s interactive setting off temporarily to avoid the performance
degradation (expecially when data processing is not separated with respect to the plotting
portion of your code)
7 http://matplotlib.org/users/shell.html:

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 46/77

Matplotlib Van der Pol phase portrait #1

1 # -*- coding: utf-8 -*- 8 # Computation part: Van der Pol equation
2 from numpy import * 9 def vanderpol(u, t=0., mu=.5):
3 from scipy import * 10 x = u[0]
4 from scipy.integrate import odeint 11 v = u[1]
5 from matplotlib.pyplot import * 12 dx = v
6 matplotlib.rc(’axes’, edgecolor = ’black’) 13 dv = mu*(1. - x**2)*v - x
van der pol mod1.py 14 return dx, dv
15
With respect to the original codea here the computational
16 t = linspace(-5.5, 5.5, 300)
and the graphical parts are separated.
17 u0 = array([1. , 1.])
I numpy, scipy, matplotlib.pyplot imported di- 18 mu = [ .25, .5, 1., 1.5, 2.0]
rectly into the main namespace. 19 u = []
20 for m in mu:
I The odeint solve numerically systems of ordinary 21 u.append(odeint(vanderpol, u0, t, args=(m,)))
differential equation: 22
23 # Computational part: attractors
dx 24 x = linspace(-3, 3, 15)
= f (x, t0 , . . .)
dt 25 y = linspace(-4, 4, 15)
26 x, y = meshgrid(x, y)
I The Van der Pol equation is: 27 X, Y = vanderpol([x, y])
(
dx
28 M = (hypot(X, Y))
d2 x 2 dx dt
=v 29 M[M == 0] = 1.
−µ(1−x ) +x = 0 −→ dv
dt2 dt dt
= µ(1 − x2 )v − x 30 X, Y = X/M, Y/M

https://commons.wikimedia.org/wiki/
a van der pol mod1.py
File:Van_der_pols_equation_phase_portrait.
svg

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 47/77

Matplotlib Van der Pol phase portrait #1 (...)

32 # Graphical part: Van der Pol equation

33 fig = figure(figsize=(5.5, 7))
34 ax = subplot(111)
35 for i, m in enumerate(mu):
36 ax.plot(u[i][:,0], u[i][:,1], lw=1.5, label=r’$\mu=%.2f$’%m)
37 fig.savefig("van_der_pol_equation_phase_portrait_1_p1.svg",
38 bbox_inches="tight", pad_inches=.15)
van der pol mod1.py

I Declaration of the figure object, and its size

I Declaration of subplot: (111), 1 row, 1 column, start at 1

I Each plot will be referenced in the legend, being label at-

tached individually

I The figure is saved as svg in this case, most backends support

also png, pdf, ps, eps

savefig (fname, dpi=None, face-

color=’w’, edgecolor=’w’, orien-
tation=’portrait’, papertype=None,
format=None, transparent=False,
bbox inches=None, pad inches=0.1,
frameon=None)

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 48/77

Matplotlib Van der Pol phase portrait #1 (...)

40 # Graphical part: attractors

41 ax.quiver(x, y, X, Y, M, pivot = ’mid’, cmap = cm.Pastel1)
42 fig.savefig("van_der_pol_equation_phase_portrait_1_p2.svg",
43 bbox_inches="tight", pad_inches=.15)
van der pol mod1.py

I Quiver plot, using the previously calculated vanderpol on the

meshgrid

I Arrow length normalization (see line 23-30)

I Color map: cmap http://matplotlib.org/examples/
color/colormaps_reference.html

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 49/77

Matplotlib Van der Pol phase portrait #1 (...)

I legend() http://matplotlib.org/users/legend_
guide.html

I It is also possible to have multiple legends on the same axes

I setp() is used to set the properties of graphical objects.
To get a list of settable line properties, call the setp() function
with a line or lines as argument.
For example in this case:
>>> setp(ax)
adjustable: [ ’box’ | ’datalim’ | ...
agg_filter: unknown
alpha: float (0.0 transparent through ...
anchor: unknown
animated: [True | False]
aspect: unknown
autoscale_on: unknown
autoscalex_on: unknown
autoscaley_on: unknown
axes: an :class: matplotlib.axes.Axes ...
axes_locator: unknown
...

45 # Legend
46 ax.legend(loc = ’upper left’)
47 setp(ax.get_legend().get_texts(), fontsize = 14)
48 fig.savefig("van_der_pol_equation_phase_portrait_1_p3.svg",
49 bbox_inches="tight", pad_inches=.15)
van der pol mod1.py

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 50/77

Matplotlib Van der Pol phase portrait #1 (...)

I grid(): True or False

I text(): position text on figure with given coordinates, text
properties have been passed using a dictionary and the **
operator

50 # Grids
51 ax.grid(True)
52 ax.minorticks_on()
53
54 # Text on figure
55 txt_dic = {’fontsize’: 18, ’backgroundcolor’: ’w’}
56 text(0.0, -3.2, r’$\dot{x} = v$’, **txt_dic)
57 text(0.0, -3.7, r’$\dot{v} = \mu(1 - xˆ2)v - x$’, **txt_dic)
58 fig.savefig("van_der_pol_equation_phase_portrait_1_p4.svg",
59 bbox_inches="tight", pad_inches=.15)
van der pol mod1.py

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 51/77

Matplotlib Van der Pol phase portrait #1 (...)

I xlabel(), ylabel(): axis labels

I title(): title

61 # Lables and title

62 xlabel(r’$\dot{x}$’, fontsize = 18)
63 ylabel(r’$\dot{v}$’, fontsize = 18)
64 title(’Van der Pol\’s Phase portrait’)
65 fig.savefig("van_der_pol_equation_phase_portrait_1.svg",
66 bbox_inches="tight", pad_inches=.15)
van der pol mod1.py

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 52/77

Matplotlib Van der Pol phase portrait #2

van der pol mod2.py

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 53/77

Matplotlib Van der Pol phase portrait #3

van der pol mod3.py

Plotting contest: http://nellev.github.io/tmp/jhepc/gallery.
html
M, Dei, IMB-CNM(CSIC) ICAS Tutorials
 Pythonize Yourself Intro Tools Language Packages4Science Examples 54/77

Sympy8
I Sympy adds symbolic math for a Maple or
$ ipython notebook --profile=sympy
Mathematica alternative. In conjunction with
a IPython Notebook, it has similar functional- >>> from sympy import *
ity. However we will explore its features im-
porting it as a module.
I Regarding Sage9:
– Sage includes Sympy
– Sage is huge and glues together many mathematical open source packages (also
non-Python)
– Sympy is a lightweight normal Python module. No external dependencies.

 Geometry Module
Linear Algebra Module




Numerics Module




I It has various submodules: Plotting Module

 Concrete Mathematics Module
Polynomials Module





 Statistics Module
Pretty Printing
8 Extensive documentation: http://sympy.googlecode.com/svn/api/sympy.html
9 https://github.com/sympy/sympy/wiki/SymPy-vs.-Sage

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 55/77

SymPy: Symbols
Built-in numeric types: Real for arbitrary-precision floating-point numbers, Rational
Represents integers and rational numbers p/q of any size
>>> Rational(3), Rational(1,2)
(3, 1/2)
>>> Rational("3/5"), Rational("1.23") # from a string
(3/5, 123/100)
>>> Rational("1.23").p, Rational("1.23").q # Access nominator and denominator as .p and .q
(123, 100)

Complex numbers
>>> exp(I*x).expand(complex=True)
I*exp(-im(x))*sin(re(x)) + exp(-im(x))*cos(re(x))
>>> x = Symbol("x", real=True)
>>> exp(I*x).expand(complex=True)
I*sin(x) + cos(x)

Explicit declaration of symbolic variables:

>>> x = Symbol(’x’) >>> symbols(’a0:3’)
>>> y = Symbol(’y’) (a0, a1, a2)
>>> ((x+y)**2).expand()
x**2 + 2*x*y + y**2 Note: an arbitrary number of symbols can be created at run-
>>> ((x+y)**2).subs(x, 1)
(y + 1)**2 time without knowing them in advance by manipulating the
>>> ((x+y)**2).subs(x, y) string ’a0:3’.
4*y**2

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 56/77

SymPy: Calculus
Limits Algebraic equations
>>> limit(sin(x)/x, x, 0) In [19]: solve(Eq(x**4, 1), x)
1 Out[19]: [1, -1, -I, I]
>>> limit(1/x, x, oo) # infinity symbol: oo
0 Integration
>>> integrate(log(x), x) # indefinite
Differentiation x*log(x) - x
>>> diff(tan(x), x) >>> integrate(sin(x), (x, 0, pi/2)) # definite
tan(x)**2 + 1 1
>>> limit((tan(x+y)-tan(x))/y, y, 0) >>> integrate(exp(-x), (x, 0, oo)) # improper
tan(x)**2 + 1 1
# Higher derivatives diff(func, var, n):
>>> diff(sin(2*x), x, 2) Series expansion
-4*sin(2*x)
>>> cos(x).series(x, 0, 10)
1 - x**2/2 + x**4/24 - x**6/720 + x**8/40320 +
+ O(x**10)

Differential equations
In [2]: Eq(f(x).diff(x, x) + f(x), 0)
Out[2]:
2
d
-----(f(x)) + f(x) = 0
dx dx

In [3]: dsolve(Eq(f(x).diff(x, x) + f(x), 0), f(x))

Out[3]: C1*sin(x) + C2*cos(x)

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 57/77

SymPy
Linear Algebra: Matrix
>>> from sympy.matrices import Matrix
>>> A = Matrix([[1,x], [y,1]])
>>> A
[1, x]
[y, 1]

>>> A**2
[x*y + 1, 2*x]
[ 2*y, x*y + 1]

Printing
Python build-in way to print an expression is simply through the use of str(expression)
or repr(expression). More readable printing is available through the Pretty print-
ing submodule:

I pretty(expr), pretty print(expr), pprint(expr)

Return or print, respectively, a pretty representation of expr.
I latex(expr), print latex(expr)
Return or print, respectively, a LATEX representation of expr.
I To enable LATEX output directly on the qt console:
>>> init_printing(pretty_print=True)

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 58/77

SymPy
Simplifications
Simplify: uses heuristics to determine the ”simplest” result. Combinations of expand(),
factor(), collect(), cancel() may return the desired result10.
>>> simplify(sin(x)**2 + cos(x)**2)
1

Partial fractions
>>> f = 1/(x**2*(x**2 + 1))
>>> f
1
2 2
x (x + 1)
>>> apart(f)
1 1
− 2 + 2
x +1 x
>>> together(_)
1
2 2
x (x + 1)

Evaluations:
evalf evaluates a sympy expression numerically with arbitrary number of digits. lamb-
dify transform sympy expressions to lambda functions which can be used to calculate
numerical values very fast. Example:
>>> pi.evalf(n=30)
3.14159265358979323846264338328

10 http://docs.sympy.org/latest/tutorial/simplification.html#simplify

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 59/77

SymPy, Plot
The plotting module allows you to make 2-dimensional and 3-dimensional plots. Plots are
rendered using matplotlib as a backend. The following functions are available:
I plot: Plots 2D line plots.
I plot parametric: Plots 2D para-
metric plots.
I plot implicit: Plots 2D implicit 15
and region plots.
10
I plot3d: Plots 3D plots of functions 5
in two variables.
0
I plot3d parametric line: Plots 5
3D line plots, defined by a parame-
10
ter.
15
I plot3d parametric surface: 10
Plots 3D parametric surface plots. 5
10 0
Example: 5
0 5
5
>>> from sympy.plotting import plot3d
10 10
>>> x, y = symbols(’x y’)
>>> plot3d(cos(x*3)*cos(y*5)-y*cos(x))

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 60/77

1 Introduction:
Matlabholics Anonymous - How to redeem

2 Tools:
How to get things done: Tools, IDEs, Installation

3 Language:
Enjoy the Lego
R

4 Packages4Science:
Four Horsemen of the Apocalypse:
NumPy, SciPy, Matplotlib, Sympy

5 Examples:
Dirty work (but someone has to do it)

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 61/77

Gini Coefficient, part #1: gini coeff.py

This example is designed to show how:
I Import data from a simple CSV file to a structured array
I Play with list comprehensions and some Numpy array methods

Suppose we are interested in the socio-economic inequality issue, we find some data on
the internet11 and we propose to analyse them.

We save our CSV file, with its predifined name DP LIVE 26042016225351582.csv lo-
cally into a data subfolder relative to our script path.

The data we found comes as a comma-separated-value file (CSV). There is no well-defined

standard for CSV files, but we know how many lines form the header, how many columns
and the structure of the data.
$ head DP_LIVE_26042016225351582.csv -n 3
"LOCATION","INDICATOR","SUBJECT","MEASURE","FREQUENCY","TIME","Value","Flag Codes"
"AUT","INCOMEINEQ","GINI","INEQ","A","2007",0.285,
"AUT","INCOMEINEQ","GINI","INEQ","A","2008",0.281,

We want to be able to view the data by country (LOCATION), by year (TIME), and of course
by the value of the Gini coefficient (VALUE).

11 https://data.oecd.org/inequality/income-inequality.htm

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 62/77

Gini Coeffi5cient, part #1: gini coeff.py ...

1 # -*- coding: utf-8 -*-
2 import numpy as np
3 import os
4 import csv

6 csv_file = (
7 ’DP_LIVE_26042016225351582.csv’,
8 ’data’,
9 os.path.dirname(os.path.abspath(__file__))
10 )
11 data_struct = {
12 ’field’ : (’Country’, ’Year’, ’Value’),
13 ’dtype’ : (’S3’, np.int16, np.float16),
14 ’ncol’ : (0, 5, 6),
15 ’heads’ : 1
16 }

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 63/77

Gini Coefficient, part #1: gini coeff.py ...

18 # Create a void structured array
19 dtype = zip(data_struct[’field’], data_struct[’dtype’])
20 data = np.array( [], dtype=dtype )

22 # Open the file to fecth the data

23 f = os.sep.join(csv_file[::-1])
24 with open(f, ’rb’) as fopen:
25 reader = csv.reader(fopen, delimiter=’,’)
26 skip_header_count = data_struct[’heads’]
27 for row in reader:
28 # skip the header
29 if skip_header_count > 0:
30 skip_header_count -= 1
31 # fill the data array
32 else:
33 data0 = np.zeros(1, dtype=dtype)
34 for i, f in enumerate(data_struct[’field’]):
35 data0[f] = row[data_struct[’ncol’][i]]
36 data = np.hstack( (data, data0) )

>>> print(data[1:8])
[(’AUT’, 2008, 0.281005859375) (’AUT’, 2009, 0.2890625)
(’AUT’, 2010, 0.280029296875) (’AUT’, 2011, 0.281982421875)
(’AUT’, 2012, 0.27587890625) (’BEL’, 2005, 0.280029296875)
(’BEL’, 2006, 0.27099609375)]

I data could be large and difficult to read

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 64/77

Gini Coefficient, part #1: gini coeff.py ...

38 def filter_data(key, key_value):
39 # dtype, data, data_struct: outside the scope of this function
40 fdata = np.array( [], dtype=dtype )
41 for d in data:
42 if d[key] == key_value:
43 fdata = np.hstack( (fdata, d) )
44 return fdata

>>> d = filter_data(’Year’, 2005); print(d)

[(’BEL’, 2005, 0.280029296875) (’CZE’, 2005, 0.260986328125)
(’FIN’, 2005, 0.264892578125) (’GRC’, 2005, 0.345947265625)
(’IRL’, 2005, 0.323974609375) (’ITA’, 2005, 0.324951171875)
(’LUX’, 2005, 0.283935546875) (’POL’, 2005, 0.326904296875)
(’PRT’, 2005, 0.37890625) (’SVK’, 2005, 0.2890625)
(’GBR’, 2005, 0.35009765625) (’EST’, 2005, 0.337890625)
(’SVN’, 2005, 0.239990234375)]

I filter data() helps fetching interesting data, but it looks redundant

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 65/77

Gini Coefficient, part #1: gini coeff.py ...

46 def extract_data(key, key_value):
47 # dtype, data, data_struct: outside the scope of this function
48 fields = data_struct[’field’]
49 field0 = [f for f in fields if f != key]
50 type0 = [data_struct[’dtype’][fields.index(f)] for f in field0]
51 dtype0 = zip(field0, type0)
52
53 fdata = filter_data(key, key_value)
54 data0 = np.zeros(len(fdata), dtype=dtype0)
55 for f in field0:
56 data0[f] = fdata[f]
57 return data0

>>> d = extract_data(’Year’, 2005); print(d)

[(’BEL’, 0.280029296875) (’CZE’, 0.260986328125) (’FIN’, 0.264892578125)
(’GRC’, 0.345947265625) (’IRL’, 0.323974609375) (’ITA’, 0.324951171875)
(’LUX’, 0.283935546875) (’POL’, 0.326904296875) (’PRT’, 0.37890625)
(’SVK’, 0.2890625) (’GBR’, 0.35009765625) (’EST’, 0.337890625)
(’SVN’, 0.239990234375)]

Looks ok, but:

I This script is tailored to a specific CSV file
I How do I reuse this code without committing the cut+paste capital crime for
a (not so) different CSV file?

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 66/77

Gini Coefficient, part #2: csv2data.py

This example is designed to show how:
I Organize code in a class for reusability

1 # -*- coding: utf-8 -*-

2 import numpy as np
3 import csv
4
5 class csv2data():
6
7 def __init__(self, csv_file, data_struct):
8 self.csv_file = csv_file
9 self.data_struct = data_struct
10 self.dtype = zip(data_struct[’field’], data_struct[’dtype’])
11 self.data = np.array( [], dtype=self.dtype )

We want csv file, data struct, dtype, data to be accessible for all the methods
in the csv2data class, hence they are created at init as class attributes.
Now the class could be organized with the following functionalities (methods):
I fetch data in the given csv file, corresponding to the code between lines 22-26 of
gini coeff.py
I capability to filter by key, key value analogously to the function filter data()
I capability to extract data, as in extract data()

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 67/77

Gini Coefficient, part #2: csv2data.py ...

13 def fetch_data(self):
14 f = self.csv_file
15 try:
16 with open(f, ’rb’) as fopen:
17 reader = csv.reader(fopen, delimiter=’,’)
18 skip_header_count = self.data_struct[’heads’]
19 for row in reader:
20 # skip the header
21 if skip_header_count > 0:
22 skip_header_count -= 1
23 # fill the data array
24 else:
25 data0 = np.zeros(1, dtype=self.dtype)
26 for i, f in enumerate(self.data_struct[’field’]):
27 data0[f] = row[self.data_struct[’ncol’][i]]
28 self.data = np.hstack( (self.data, data0) )
29 except IOError:
30 print(’Cannot open ’ + f)

With respect to the gini coeff.py we added here a try/except construct to handle
input/output errors of the file (for example, file not found, etc.)

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 68/77

Gini Coefficient, part #2: csv2data.py ...

32 def filter_data(self, key, key_value):
33 fdata = np.array( [], dtype=self.dtype )
34 for d in self.data:
35 if d[key] == key_value:
36 fdata = np.hstack( (fdata, d) )
37 return fdata
38
39 def extract_data(self, key, key_value):
40 fields = self.data_struct[’field’]
41 field0 = [f for f in fields if f != key]
42 type0 = [self.data_struct[’dtype’][fields.index(f)] for f in field0]
43 dtype0 = zip(field0, type0)
44 fdata = self.filter_data(key, key_value)
45 data0 = np.zeros(len(fdata), dtype=dtype0)
46 for f in field0:
47 data0[f] = fdata[f]
48 return data0

Finally we add a call method to let any instance of the csv2data class to be used
as they were functions:
50 def __call__(self, key, key_value):
51 if len(self.data)==0:
52 self.fetch_data()
53 return self.extract_data(key, key_value)

call ? Enables the object to be called like a function.

>>> gini(file_name, my_data_struct) # initialize
>>> gini(’Country’, ’AUT’) # call
>>> gini.extract_data(’Country’, ’AUT’) # call a specific method

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 69/77

Gini Coefficient, part #2: test csv2data.py

Example of usage of the csv2data class:

Simple test of the csv2data class:

1 # -*- coding: utf-8 -*-
2 import os
3 import numpy as np
I first an instance is created
4 from csv2data import csv2data as c2d ( init ())
5 import matplotlib.pyplot as plt
6 I then is used through its
csv_file = (
7
8 ’DP_LIVE_26042016225351582.csv’,
call () method
9 ’data’,
10 os.path.dirname(os.path.abspath(__file__))
11 )
12 data_struct = {
13 ’field’ : (’Country’, ’Year’, ’Value’),
14 ’dtype’ : (’S3’, np.int16, np.float16),
15 ’ncol’ : (0, 5, 6),
16 ’heads’ : 1
17 }
18
19 f = os.sep.join(csv_file[::-1])
20 gini = c2d(f, data_struct)

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 70/77

Gini Coefficient, part #2: test csv2data.py ...

SVN LUX ITA PRT
22 countries = [’SVN’, ’LUX’, ’ITA’, ’PRT’] 0.40
23 colors = [’cyan’, ’blue’, ’magenta’, ’red’]
24
25 fig = plt.figure(figsize=(4,4))
26 ax = fig.add_subplot(111) 0.35

Gini coefficient
27 props = dict(width=0.3, alpha=0.5, linewidth=0)
28
29
30
for i, country in enumerate(countries):
g = gini(’Country’, country)
0.30
31 x = g[’Year’]
32 y = g[’Value’]
33 ax.bar(x+i*0.2, y, color=colors[i], label=country, **props) 0.25
34
35 plt.legend(bbox_to_anchor=(0., 1.02, 1., .102), loc=3,
ncol=4, mode="expand", borderaxespad=0., fontsize=10)
36
37 ax.set_ylim([0.2,0.4]) 0.20

05
06
07
08
09
10
11
12
38 ax.set_ylabel(’Gini coefficient’)

20
20
20
20
20
20
20
20
39 xTickMarks = [str(i) for i in range(min(g[’Year’]),max(g[’Year’])+1)]
40 xtickNames = ax.set_xticklabels(xTickMarks)
41 plt.setp(xtickNames, rotation=45)
42 plt.subplots_adjust(right=0.5)
43 plt.tight_layout()
test csv2data.py

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 71/77

Multiprocessing: multiproc.py
This example is designed to show how:
I include a very basic multiprocessing capability in Python scripts

When parallel processing has an How many parallel process is optimum

advantage over sequential processing? for performances?
I CPU-limited vs. I/O-limited I CPU cores
processes
I Inter-process communication
I Independent processes vs.
pipelined processes
f(x) will be the target function of the
1 # -*- coding: utf-8 -*- test: it will be executed independently by
2 import multiprocessing as mp the processes.
3 import time
4
5 def f(x):
6 "simple test function"
7 result = 0
8 for i in xrange(x):
9 result += x*x
10 return result

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 72/77

Multiprocessing: multiproc.py ...

I Heuristically:
12 if __name__ == ’__main__’: Number of processes ∝ Number of processor
13 units
14 # Inputs to f:
15 num_iter = 1000 I .cpu_count(): counts the number of process-
16 f_inputs = range(num_iter) ing units
17 .Pool() defines a pool of workers: each worker
18 # Parallel process: will start an independent parallel job
19 num_workers = mp.cpu_count()*4 .map() gives to the pool of workers what to do
20 pool = mp.Pool(num_workers) and with which arguments
21 t0 = time.time()
22 res_parallel = pool.map(f, f_inputs)
23 dt_parallel = time.time() - t0 I The main is protected by an if statement:
24 print("parallel [s]: ".rjust(18) + str(dt_parallel)) due to the way the new processes are started,
25
the child process needs to be able to import the
26 # Serial process: script containing the target function.
27 res_series = [] Wrapping the main part of the application in a
28 t0 = time.time() check for main ensures that it is not run re-
29 for x in f_inputs: cursively in each child as the module is imported.
30 res_series.append(f(x))
31 dt_serial = time.time() - t0 I This script sketches a very basic parallel pro-
32 print("serial [s]: ".rjust(18) + str(dt_serial)) cessing example that could be useful in a Monte
33 Carlo simulation batch, where each worker (a
34 # Resume Monte Carlo try) does not share data with other
35 check = all([res_parallel[i]==res_series[i]\ workers. Much more complicated (and interest-
36 for i in xrange(len(res_parallel))]) ing) parallel processing involving inter process
37 print(’_’*36) communication can actually be implemented!
38 print("Results check: ".rjust(18) + str(check))
39 print("Speed up: ".rjust(18) + str(dt_serial/dt_parallel))

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 73/77

Interfacing with Xcos: deltasigma1ct.py

This example is designed to show how:
I Use the capability of Scilab-Xcos inside Python

I Xcos, like Simulink provides a graphical programming environment for modelling, sim-
ulating and analysing multi-domain dynamic systems.
I You can script Xcos directly within the Scilab12 environment without Python. Anyway
Scilab and Python has a certain grade of interoperability
I From the Python-side, to interact with Scilab-Xcos, you need the scilab2py mod-
ule13. It is not present in the Anaconda bundle by default, but it is easy to install by
pip: $ pip install scilab2py

Useful links:
How to simulate a Xcos model from the scilab command window (batch mode):
https://help.scilab.org/docs/5.5.0/fr_FR/xcos_simulate.html
A (very short) documentation on how to use the scilab2py API can be found here:
http://blink1073.github.io/scilab2py/source/api.html

12 http://www.scilab.org/
13 https://pypi.python.org/pypi/scilab2py

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 74/77

Interfacing with Xcos: deltasigma1ct.py ...

The model can be set up from the Scilab graphical interface side: A very basic Scilab script to simulate the Xcos model:

1 importXcosDiagram(filename)
2 scs_m.props.context =
3 ["Ain=0.01" "fs=1" "fin=0.01"];
4 xcos_simulate(scs_m, 4);

We wish to have the scripting control from the Python

side with the ability to:
I Set model parameters (for ex.: Ain, fs, fin)
I Recollect data (for ex.: DOUT) to be post pro-
cessed

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself Intro Tools Language Packages4Science Examples 75/77

Interfacing with Xcos: deltasigma1ct.py ...

1 from scilab2py import Scilab2Py 33 # psd plot
2 import os 34 f, Pxx = signal.periodogram(dout.T, 1,
3 from scipy import signal 35 ’flattop’, scaling=’spectrum’)
4 import matplotlib.pyplot as plt 36 fig = plt.figure(figsize=(4,4))
5 37 plt.semilogy(f, Pxx[0])
6 sci = Scilab2Py() 38 plt.grid(True)
7 39 plt.xlabel(r’frequency $[Hz]$’)
8 xcos_file_path = ( 40 plt.ylabel(r’PSD $[Vˆ2/Hz]$’)
9 ’CT_DS_1.zcos’,
10 ’xcos’,
11 os.path.dirname(\
12 os.path.abspath(__file__))
13 )
14 filename = ’/’.join(xcos_file_path[::-1]) 101
100
15
16 # Put a variable or variables into the
17
18
# Scilab session:
sci.push(’filename’, filename)
10-1
19 # import Xcos diagram into Scilab session:
sci.eval("importXcosDiagram(filename)")
10-2

PSD [V 2 /Hz]
20
21 # set Xcos parameters (context): 10-3
22 sci.eval(’scs_m.props.context = ’ +
23 ’["Ain=0.01" "fs=1" "fin=0.01" "nclocks = 10-4
2048"];’)
24 # simulate:
25 sci.eval(’xcos_simulate(scs_m, 4);’) 10-5
# retrieve DOUT from scilab workscape:
10-6
26
27 dout = sci.pull(’DOUT.values’)
28
29
# Close session:
sci.close()
10-7
10-8
0.0 0.1 0.2 0.3 0.4 0.5
frequency [Hz]

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself 76/77

Resources

This document is downloadable at:

http://www.cnm.es/˜mdei/pythonize_yourself/pythonize_yourself.pdf

All the scripts presented are located at:

http://www.cnm.es/˜mdei/pythonize_yourself/scripts/

Web resources:

I http://stackoverflow.com/
I https://docs.python.org/2/
I https://wiki.python.org/moin/FrontPage

(and many many more)

M, Dei, IMB-CNM(CSIC) ICAS Tutorials

 Pythonize Yourself 77/77

Bonus
World clouds in Python: https://github.com/amueller/word_cloud
1 # /usr/bin/env python
2 # -*- coding: utf-8 -*-
3 from os import path
4 from scipy.misc import imread
5 import matplotlib.pyplot as plt
6 import random
7 from wordcloud import WordCloud, STOPWORDS
8
9 # custom color function
10 def py_color_func(word, font_size, position, orientation, random_state=None, **kwargs):
11 return "hsl(0, 65%%, %d%%)" % random.randint(60, 80)
12
13 # current script path
14 d = path.dirname(__file__)
15
16 # python logo
17 mask = imread(path.join(d, "pythonlogo45.png"), flatten = True)
18
19 # from the tex source of the session 3 of this presentation
20 text = open("language.tex").read()
21
22 # stopwords: exclude set of latex syntax words
23 stopwords = STOPWORDS.copy()
24 with open(path.join(d, ’latex_syntax.txt’), ’r’) as f:
25 rm = set(f.read().split())
26 stopwords = stopwords | rm
27
28 # create the cloud
29 wc = WordCloud(max_words=1000,
30 mask=mask,
31 stopwords=stopwords,
32 margin=1,
33 random_state=3,
34 width=794*10,
35 height=595*10,
36 font_path=path.join(d,’cmuntt.ttf’),
37 background_color=’white’).generate(text)
38
39 # plot
40 plt.figure()
41 plt.imshow(wc.recolor(color_func=py_color_func, random_state=3))
42 wc.to_file(’pythonwordcloud.png’)
43 plt.axis("off")
44 print(’File output: ’+d+os.sep+’pythonwordcloud.png’)

python world cloud.py

M, Dei, IMB-CNM(CSIC) ICAS Tutorials