Introduction to Python

Part 2

v0.7

Research Computing Services

Information Services & Technology
Tutorial Outline – Part 2

 Lists
 Tuples and dictionaries
 Modules
 numpy and matplotlib modules
 Script setup
 Development notes
Lists
 A Python list is a general purpose 1-dimensional container for variables.
 i.e. it is a row, column, or vector of things

 Lots of things in Python act like lists or use list-style notation.

 Variables in a list can be of any type at any location, including other lists.

 Lists can change in size: elements can be added or removed

Making a list and checking it twice…

 Make a list with [ ] brackets.

 Append with the append() function

 Create a list with some initial elements

 Create a list with N repeated elements

Try these out yourself!

Edit the file in Spyder and run it.
Add some print() calls to see the lists.
List functions

 Try dir(list_1)

 Like strings, lists have a number of

built-in functions

 Let’s try out a few…

 Also try the len() function to see how

many things are in the list: len(list_1)
List Indexing
 Elements in a list are accessed by an index number.

 Index #’s start at 0.

 List: x=['a', 'b', 'c', 'd' ,'e']

 First element: x[0]  'a'

 Nth element: x[2]  'c'
 Last element: x[-1] 'e'
 Next-to-last: x[-2] 'd'
 x=['a', 'b', 'c', 'd' ,'e']
List Slicing x[0:1]  ['a']
x[0:2]  ['a','b']
x[-3:]  ['c', 'd', 'e']
# Third from the end to the end
x[2:5:2]  ['c', 'e']

 Slice syntax: x[start:end:step]

 The start value is inclusive, the end value is exclusive.
 Start is optional and defaults to 0.
 Step is optional and defaults to 1.
 Leaving out the end value means “go to the end”
 Slicing always returns a new list copied from the existing list
List assignments and deletions

 Lists can have their elements overwritten or deleted (with the del) command.

x=['a', 'b', 'c', 'd' ,'e']

x[0] = -3.14  x is now [-3.14, 'b', 'c', 'd', 'e']

del x[-1]  x is now [-3.14, 'b', 'c', 'd']

DIY Lists

 In the Spyder editor try the following things:

 Assign some lists to some variables. a = [1,2,3] b = 3*[‘xyz’]

 Try an empty list, repeated elements, initial set of elements
 Add two lists: a + b What happens?

 Try list indexing, deletion, functions from dir(my_list)

 Try assigning the result of a list slice to a new variable

 More on Lists and Variables

 Open the sample file list_variables.py

but don’t run it yet!

 What do you think will be printed?

Variables and Memory Locations

 Variables refer to a value stored in memory.

 y = x does not mean “make a copy of the
list x and assign it to y” it means “make a
copy of the memory location in x and assign
it to y” x

 x is not the list it’s just a reference to it.

y
 This is how all objects in Python are
handled.
Copying Lists

 How to copy (2 ways…there are more!):

 y = x[:] or y=list(x)

 In list_variables.py uncomment the code at the bottom and run it.

While Loops
 While loops have a condition and a
code block.
 the indentation indicates what’s in the while loop.
 The loop runs until the condition is false.
 The break keyword will stop a while
loop running.

 In the Spyder edit enter in some

loops like these. Save and run them
one at a time. What happens with
the 1st loop?
For loops collection
In-loop reference
 for loops are a little different. They variable for each
collection element
loop through a collection of things.
 The for loop syntax has a collection
and a code block.
 Each element in the collection is accessed in
order by a reference variable The code block

 Each element can be used in the code block.

 The break keyword can be used in for

loops too.
Processing lists element-by-element
 A for loop is a convenient way to process every element in a list.

 There are several ways:

 Loop over the list elements
 Loop over a list of index values and access the list by index
 Do both at the same time
 Use a shorthand syntax called a list comprehension
 Open the file looping_lists.py
 Let’s look at code samples for each of these.
The range() function

 The range() function auto-generates sequences of numbers that can be

used for indexing into lists.

 Syntax: range(start, exclusive end, increment)

 range(0,4)  produces the sequence of numbers 0,1,2,3

 range(-3,15,3)  -3,0,3,6,9,12
 range(4,-3,2)  4,2,0,-2

 Try this: print(range(4))

 numbers.txt

Lists With Loops 38,83,37,21,98

50,53,55,37,97
39,7,81,87,82
 Open the file read_a_file.py
18,83,66,82,47
56,64,9,39,83
 This is an example of reading a file …etc…
into a list. The file is shown to the
right, numbers.txt

 We want to read the lines in the file • read_a_file_low_mem.py is a

modification that uses less memory
into a list of strings (1 string for each by processing the file line-by-line.
line), then extract separate lists of
the odd and even numbers.
Tutorial Outline – Part 2

 Lists
 Tuples and dictionaries
 Modules
 numpy and matplotlib modules
 Script setup
 Development notes
Tuples

 Tuples are lists whose elements can’t

be changed.
 Like strings they are immutable

 Indexing (including slice notation) is

the same as with lists.
Return multiple values from a function

 Tuples are more useful than they

might seem at first glance.

 They can be easily used to return

multiple values from a function.

 Python syntax can automatically

unpack a tuple return value.
Dictionaries

 Dictionaries are another basic Python data type that are tremendously
useful.

 Create a dictionary with a pair of curly braces:

x = {}
 Dictionaries store values and are indexed with keys

 Create a dictionary with some initial values:

x = {'a_key':55, 100:'a_value', 4.1:[5,6,7]}

Dictionaries
 Values can be any Python thing

 Keys can be primitive types (numbers), strings, tuples, and some custom
data types
 Basically, any data type that is immutable

 Lists and dictionaries cannot be keys but they can stored as values.

 Index dictionaries via keys:

x['a_key']  55
x[100]  'a_value'
 Try Out Dictionaries
 Create a dictionary in the Python console or
Spyder editor.

 Add some values to it just by using a new key as

an index. Can you overwrite a value?

 Try x.keys() and x.values()

 Try: del x[valid_key]  deletes a key/value

pair from the dictionary.
Tutorial Outline – Part 2

 Lists
 Tuples and dictionaries
 Modules
 numpy and matplotlib modules
 Script setup
 Development notes
Modules

 Python modules, aka libraries or packages, add functionality to the core

Python language.

 The Python Standard Library provides a very wide assortment of functions

and data structures.
 Check out their Brief Tour for a quick intro.

 Distributions like Anaconda provides dozens or hundreds more

 You can write your own libraries or install your own.

PyPI

 The Python Package Index is a central repository for Python software.

 Mostly but not always written in Python.

 A tool, pip, can be used to install packages from it into your Python setup.
 Anaconda provides a similar tool called conda

 Number of projects (as of September 2021): 326,278

 You should always do your due diligence when using software from a
place like PyPI. Make sure it does what you think it’s doing!
Python Modules on the SCC

 Python modules should not be confused with the SCC module command.

 For the SCC there are instructions on how to install Python software for
your account or project.

 Many SCC modules provide Python packages as well.

 Example: tensorflow, pycuda, others.

 Need help on the SCC? Send us an email: [email protected]

 Importing Libraries
 The import command is used to load a
library.

 The name of the library is prepended to

function names and data structures in the
module.
 The preserves the library namespace

 This allows different libraries to have the Try these out!

same function names – when loaded the

library name keeps them separate.
Fun with import
 The import command can strip away the module name:

from math import *

 Or it can import select functions:

from math import cos

from math import cos,sqrt

 Or rename on the import:

from math import sin as pySin

Easter Eggs

# Try to load curly braces for Python

from __future__ import braces

# Proof that Python programmers have more fun

import antigravity
 myfuncs.py
Fun with import def get_odds(lst):
''' Gets the odd numbers in a list.
 The import command can also load
lst: incoming list of integers
your own Python files. return: list of odd integers '''
odds = []
for elem in lst:
 The Python file to the right can be # Odd if there's a remainder when
# dividing by 2.
used in another Python script: if elem % 2 != 0:
odds.append(elem)
return odds
# Don't use the .py ending
import myfuncs
x = [1,2,3,4]
y = myfuncs.get_odds(x)
 Splitting your code into multiple files
helps with development and
organization.
 myfuncs.py
Import details def get_odds(lst):
''' Gets the odd numbers in a list.

 Python reads and executes a file lst: incoming list of integers

return: list of odd integers '''
when the file is: odds = []
 opened directly: python somefile.py for elem in lst:
# Odd if there's a remainder when
# dividing by 2.
 imported: import somefile
if elem % 2 != 0:
odds.append(elem)
return odds
 Lines that create variables, call
functions, etc. are all executed. x = [1,2,3,4]
y = get_odds(x)
print(y)
 Here these lines will run when it’s
imported into another script!
 The __name__ attribute
 Python stores object information in  Every file has one called __name__
hidden fields called attributes whose value depends on how the
file is used.

# in another Python
# script __name__  myfuncs
import myfuncs (i.e. the file name)

myfuncs.py
# called directly
__name__  __main__
python myfuncs.py
 myfuncs.py
The __name__ attribute def get_odds(lst):
''' Gets the odd numbers in a list.

 __name__ can be used to make a lst: incoming list of integers

Python scripts usable as a return: list of odd integers '''
odds = []
standalone program and as for elem in lst:
imported code. # Odd if there's a remainder when
# dividing by 2.
if elem % 2 != 0:
odds.append(elem)
 Now: return odds
 python myfuncs.py  __name__ has the
value of ‘__main__’ and the code in the if
if __name__=='__main__':
statement is executed. x = [1,2,3,4]
y = get_odds(x)
 import myfuncs  __name__ is ‘myfuncs’ print(y)
and the if statement does not run.
Tutorial Outline – Part 2

 Lists
 Tuples and dictionaries
 Modules
 numpy and matplotlib modules
 Script setup
 Development notes
A brief into to numpy and matplotlib

 numpy is a Python library that provides efficient multidimensional matrix

and basic linear algrebra
 The syntax is very similar to Matlab or Fortran

 matplotlib is a popular plotting library

 Remarkably similar to Matlab plotting commands!

 A third library, scipy, provides a wide variety of numerical algorithms:

 Integrations, curve fitting, machine learning, optimization, root finding, etc.
 Built on top of numpy
 Investing the time in learning these three libraries is worth the effort!!
numpy

 numpy provides data structures written in compiled C code

 Many of its operations are executed in compiled C or Fortran code, not
Python.

 Check out numpy_basics.py

numpy datatypes

 Unlike Python lists, which are generic

containers, numpy arrays are typed.

 If you don’t specify a type, numpy will assign

one automatically.

 A wide variety of numerical types are available.

 Proper assignment of data types can sometimes have a significant effect on

memory usage and performance.
Numpy operators

 Numpy arrays will do element-wise

arithmetic: + / - * **

 Matrix (or vector/matrix, etc.)

multiplication needs the .dot() function.

 Numpy has its own sin(), cos(), log(),

etc. functions that will operate element-
by-element on its arrays. Try these out!
Plotting with matplotlib

 Matplotlib is the most popular

Python plotting library
 Seaborn is another.

 Based on Matlab plotting.

Try these out!
 Plots can be made from lists,
tuples, numpy arrays, etc.
 Some sample images from matplotlib.org
 A vast array of plot types in 2D and 3D are available in
this library.
A numpy and matplotlib example

 numpy_matplotlib_fft.py is a short example on using numpy and matplotlib

together.

 Open numpy_matplotlib_fft.py

 This sample extracts signals from a noisy background.

Tutorial Outline – Part 2

 Lists
 Tuples and dictionaries
 Modules
 numpy and matplotlib modules
 Script setup
 Development notes
Writing Quality Pythonic Code
 Cultivating good coding habits pays off in many ways:
 Easier and faster to write
 Easier and faster to edit, change, and update your code
 Other people can understand your work

 Python lends itself to readable code

 It’s quite hard to write completely obfuscated code in Python.
 Exploit language features where it makes sense
 Contrast that with this sample of obfuscated C code.

 Here we’ll go over some suggestions on how to setup a Python script,

make it readable, reusable, and testable.
Compare some Python scripts

 Open up three files and let’s look at them.

 A file that does…something…

 bad_code.py
 Same code, re-organized:
 good_code.py
 Same code, debugged, with testing code:
 good_code_testing.py
 Command line arguments
 Try to avoid hard-coding file paths,
problem size ranges, etc. into your
program.

 They can be specified at the command

line.

 Look at the argparse module, part of

the Python Standard Library.
Tutorial Outline – Part 2

 Lists
 Tuples and dictionaries
 Modules
 numpy and matplotlib modules
 Script setup
 Development notes
Function, class, and variable naming

 There’s no word or character limit for names.

 It’s ok to use descriptive names for things.

 An IDE (like Spyder) will help you fill in longer names so there’s no extra
typing anyway.

 Give your functions and variables names that reflect their meaning.
 Once a program is finished it’s easy to forget what does what where
An example development process
 Work to develop your program.
 Do some flowcharts, work out algorithms, and so on.
 Write some Python to try out a few ideas.
 Get organized.

 Write a “1st draft” version that gets most of what’s needed done.

 Move hard-coded values into the if __name__==‘__main__’ section of your code.

 Once the code is testing well add command line arguments and remove hard-
coded values

 Finally (e.g. to run as an SCC batch job) test run from the command line.
Spyder command line arguments
 Click on the Run menu and choose
Configuration per file

 Enter command line arguments

Python from the command line on the SCC

 To run Python from the command line:

 After a Python module is loaded just type python followed by the script
name followed by script arguments.
Where to get help…

 The official Python Tutorial

 Automate the Boring Stuff with Python

 Focuses more on doing useful things with Python, not focused on scientific computing

 Full Speed Python tutorial

 Contact Research Computing: [email protected]