Table

of Contents
About

1.1

Introduction

1.2

Number and String data types

1.3

Functions

1.4

Getting User input

1.5

Executing external commands

1.6

Control Structures

1.7

Lists

1.8

Sequence, Set and Dict data types

1.9

Text Processing

1.10

File handling

1.11

Command line arguments

1.12

Exception Handling and Debugging

1.13

Docstrings

1.14

Further Reading

1.15

About

Python Basics
Introduction to Python - Syntax, working with Shell commands, Files, Text
Processing, and more...
Content here is for a one-day workshop for Python beginners
Python curated resources for more complete resources list, including tutorials
for beginners
For more related resources, visit scripting course

Chapters
Introduction
Installation, Hello World example, Python Interpreter, Python Standard
Library
Number and String data types
Numbers, String, Constants, Built-in Operators
Functions
def, print function, range function, type function, Variable Scope
Getting User input
Integer input, Floating point input, String input
Executing external commands
Calling Shell commands, Calling Shell commands with expansion,
Getting command output and redirections
Control Structures
Condition checking, if, for, while, continue and break
Lists
Assigning List variables, Slicing and Modifying Lists, Copying Lists, List
Methods and Miscellaneous, Looping, List Comprehension
Sequence, Set and Dict data types
Strings, Tuples, Set, Dictionary
Text Processing

About
String methods, Regular Expressions, Pattern matching and extraction,
Search and Replace, Compiling Regular Expressions, Further Reading
on Regular Expressions
File handling
open function, Reading files, Writing to files
Command line arguments
Known number of arguments, Varying number of arguments, Using
program name in code, Command line switches
Exception Handling and Debugging
Exception Handling, Syntax check, pdb, Importing program
Docstrings
Style guide, Palindrome example
Further Reading
Standard topics not covered, Useful links on coding, Python extensions

ebook
Read as ebook on gitbook
Download ebook for offline reading - link

Credits
Devs and Hackers - helpful slack group
Weekly Coders, Hackers & All Tech related thread - for suggestions and
critique
/r/learnpython/ - helpful forum for beginners and experienced programmers
alike

License
3

About
This work is licensed under a Creative Commons Attribution-NonCommercialShareAlike 4.0 International License

Introduction

Introduction
Installation
Hello World example
Python Interpreter
Python Standard Library
From wikipedia)
Python is a widely used high-level, general-purpose, interpreted, dynamic
programming language. Its design philosophy emphasizes code readability,
and its syntax allows programmers to express concepts in fewer lines of code
than possible in languages such as C++ or Java. The language provides
constructs intended to enable clear programs on both a small and large scale
Guido van Rossum is the author of Python programming language, and continues
to oversee the Python development process

Installation
Get Python for your OS from official website - https://www.python.org/
Examples presented here is for Unix-like systems, Python version 3 and
uses bash shell
You can also run Python code online
pythontutor - code execution in Python 2 and 3 versions, visualizing code
flow and sample programs are among its features
jupyter - web application that allows you to create and share documents
that contain live code, equations, visualizations and explanatory text
ideone - online compiler and debugging tool which allows you to compile
source code and execute it online in more than 60 programming
languages
Python Interpreter shell
It is assumed that you are familiar with command line. If not, check out this

Introduction
basic tutorial on ryanstutorials and this list of curated resources for Linux

Hello World example

Let's start with simple a Python program and how to run it
#!/usr/bin/python3
print("Hello World")

The first line has two parts

/usr/bin/python3 is the path of Python interpreter
#! called as shebang), directs the program loader to use the interpreter

path provided
The third line prints the message Hello World with a newline character added
by default by the print function
Running Python program
You can write the program using text editor like gedit, vim or other editors
After saving the file, give execute permission and run the program from a terminal
$ chmod +x hello_world.py
$ ./hello_world.py
Hello World

To find out path and version of Python in your system

$ type python3
python3 is /usr/bin/python3
$ python3 --version
Python 3.4.3

Introduction
If you happen to follow a book/tutorial on Python version 2 or coming with Perl
experience, it is a common mistake to forget () with print function
#!/usr/bin/python3
print "Have a nice day"

Depending on type of error, it may be easy to spot the mistake based on error
messages printed on executing the program
In this example, we get the appropriate Missing parentheses message
$ ./syntax_error.py
File "./syntax_error.py", line 3
print "Have a nice day"
^
SyntaxError: Missing parentheses in call to 'print'

single line comments start with #

#! has special meaning only on first line of program

we will see multiline comments in later chapters

#!/usr/bin/python3
# Greeting message
print("Hello World")

Further Reading
Python docs - version 3
Different ways of executing Python programs
Where is Python used?
Python docs - Errors and Exceptions
Common syntax errors

Python Interpreter
7

Introduction
It is generally used to execute snippets of Python language as a means to
learning Python or for debugging purposes
The prompt is usually >>>
Some of the topics in coming chapters will be complemented with examples
using the Python Interpreter
A special variable _ holds the result of last printed expression
One can type part of command and repeatedly press Up arrow key to match
commands from history
Press Ctrl+l to clear the screen, keeping any typed command intact
exit() to exit

$ python3
Python 3.4.3 (default, Oct 14 2015, 20:28:29)
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more inform
ation.
>>> print("hi")
hi
>>> abc
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'abc' is not defined
>>> num = 5
>>> num
5
>>> 3 + 4
7
>>> 12 + _
19
>>> exit()

Further Reading
Python docs - Using the Python Interpreter
Python docs - Interpreter example

Introduction

Python Standard Library

Python docs - library
pypi - repository of software for the Python programming language
The library contains built-in modules (written in C) that provide access to
system functionality such as file I/O that would otherwise be inaccessible to
Python programmers, as well as modules written in Python that provide
standardized solutions for many problems that occur in everyday
programming.
Some of these modules are explicitly designed to encourage and enhance the
portability of Python programs by abstracting away platform-specifics into
platform-neutral APIs

Number and String data types

Number and String data types

Numbers
String
Constants
Built-in Operators
Variable data type is automatically determined by Python. They only need to be
assigned some value before using it elsewhere - like print function or part of
expression

Numbers
Integer examples
>>> num1 = 7
>>> num2 = 42
>>> sum = num1 + num2
>>> print(sum)
49
>>> sum
49
>>> 34 ** 32
10170102859315411774579628461341138023025901305856
>>> 9 / 5
1.8
>>> 9 // 5
1
>>> 9 % 5
4

Floating point examples

10

Number and String data types

>>> appx_pi = 22 / 7
>>> area = 42.16
>>> appx_pi + area
45.30285714285714
>>> num1
7
>>> num1 + area
49.16

the E scientific notation can be used as well

>>> sci_num1 = 3.982e5
>>> sci_num2 = 9.32e-1
>>> sci_num1 + sci_num2
398200.932

Octal numbers are prefixed with 0o or 0O (i.e digit 0 followed by

lower/upper case letter o)
Similarly, Hexadecimal numbers are prefixed with 0x or 0X
>>> oct_num = 0o12
>>> hex_num = 0xF
>>> oct_num
10
>>> hex_num
15
>>> oct_num + hex_num
25

Further Reading
Python docs - numbers
decimal
fractions
complex
Python docs - keywords - do not use these as variables

11

Number and String data types

String
strings can be declared using single or double quotes
Use \ to escape quotes which are part of string itself if the string contains
both single and double quotes
>>> str1 = 'This is a string'
>>> str1
'This is a string'
>>> greeting = "Hello World!"
>>> greeting
'Hello World!'
>>> weather = "It's a nice and warm day"
>>> weather
"It's a nice and warm day"
>>> print(weather)
It's a nice and warm day
>>> weather = 'It\'s a nice and warm day'
>>> weather
"It's a nice and warm day"

Escape sequences like newline character \n can be used within string

declaration
>>> colors = 'Blue\nRed\nGreen'
>>> colors
'Blue\nRed\nGreen'
>>> print(colors)
Blue
Red
Green

Use r prefix (stands for raw) if you do not want escape sequences to be
interpreted

12

Number and String data types

>>> raw_str = r"tr '\n' ' ' < list.txt"

>>> raw_str
"tr '\\n' ' ' < list.txt"
>>> print(raw_str)
tr '\n' ' ' < list.txt
>>> normal_str = "tr '\n' ' ' < list.txt"
>>> print(normal_str)
tr '
' ' ' < list.txt

String concatenation and repetition

>>> str1
'This is a string'
>>> str2 = ' with concatenation'
>>> print(str1 + str2)
This is a string with concatenation
>>> style_char = '-'
>>> style_char * 10
'----------'
>>> word = 'buffalo '
>>> print(word * 8)
buffalo buffalo buffalo buffalo buffalo buffalo buffalo buffalo

Triple quoted strings

like single line strings, """ or ''' can be used as required as well as
escape characters using \

13

Number and String data types

#!/usr/bin/python3
"""
This line is part of multiline comment
This program shows examples of triple quoted strings
"""
# assigning multiple line string to variable
poem = """\
The woods are lovely, dark and deep,
But I have promises to keep,
And miles to go before I sleep,
And miles to go before I sleep.
"""
print(poem, end='')

Triple quoted strings are also serve as docstrings (documentation),

examples in later chapters
$ ./triple_quoted_string.py
The woods are lovely, dark and deep,
But I have promises to keep,
And miles to go before I sleep,
And miles to go before I sleep.
$

Further Reading
Python docs - strings
Python docs - List of Escape Sequences and more info on strings
Python docs - Binary Sequence Types
formatting triple quoted strings

Constants
14

Number and String data types

Paraphrased from Python docs - constants
None The sole value of the type NoneType
None is frequently used to represent the absence of a value
False The false value of the bool type
True The true value of the bool type

Python docs - Truth Value Testing

>>> bool(2)
True
>>> bool(0)
False
>>> bool('')
False
>>> bool('a')
True

Built-in Operators
arithmetic operators
+ addition
- subtraction
* multiplication
/ division (float output)
// division (integer output, result is not rounded)
** exponentiation
% modulo

string operators
+ string concatenation
* string repetition

comparison operators
== equal to
> greater than
< less than
!= not equal to

15

Number and String data types

>= greater than or equal to
<= less than or equal to

boolean logic
and logical and
or logical or
not logical not

bitwise operators
& and
| or
^ exclusive or
~ invert bits
>> right shift
<< left shift

and many more...

Further Reading
Python docs - Numeric types - complete list of operations and precedence
Python docs - String methods

16

Functions

Functions
def
print function
range function
type function
Variable Scope

def
#!/usr/bin/python3
# ----- function without arguments ----def greeting():
print("-----------------------------")
print(" Hello World ")
print("-----------------------------")
greeting()
# ----- function with arguments ----def sum_two_numbers(num1, num2):
sum = num1 + num2
print("{} + {} = {}".format(num1, num2, sum))
sum_two_numbers(3, 4)
# ----- function with return value ----def num_square(num):
return num * num
my_num = 3
print(num_square(2))
print(num_square(my_num))

17

Functions
The def keyword is used to define functions
Functions have to be defined before use
A common syntax error is leaving out : at end of def statement
Block of code for functions, control structures, etc are distinguished by
indented code
4-space indentation is recommended
Python docs - Coding Style
The default return value is None
How variables are passed to functions in Python
$ ./functions.py
---------------------------- Hello World
----------------------------3 + 4 = 7
4
9

Default valued arguments

#!/usr/bin/python3
# ----- function with default valued argument ----def greeting(style_char='-'):
print(style_char * 29)
print(" Hello World ")
print(style_char * 29)
print("Default style")
greeting()
print("\nStyle character *")
greeting('*')
print("\nStyle character =")
greeting(style_char='=')

18

Functions
Often, functions can have a default behavior and if needed changed by
passing relevant argument
$ ./functions_default_arg_value.py
Default style
---------------------------- Hello World
----------------------------Style character *
*****************************
Hello World
*****************************
Style character =
=============================
Hello World
=============================

Triple quoted comment describing function purpose is a usually followed

guideline
To avoid distraction from example code, docstrings for programs and
functions won't be generally used in this material
Example usage will be discussed briefly in a separate chapter
def num_square(num):
"""
returns square of number
"""
return num * num

Further Reading
There are many more ways to call a function and other types of declarations, refer
the below links for more info
Python docs - defining functions

19

Functions
Python docs - Built-in Functions

print function
By default, print function adds newline character
This can be changed by passing our own string to the end argument
>>> print("hi")
hi
>>> print("hi", end='')
hi>>>
>>> print("hi", end=' !!\n')
hi !!
>>>

Printing variables - requires explicit type conversion to str for numeric

variables when used for string concatenation

20

Functions

>>> greeting = 'Hello World'

>>> print(greeting)
Hello World
>>> num = 42
>>> print(num)
42
>>> print(greeting + '. We are learning Python')
Hello World. We are learning Python
>>> print("The Ultimate Answer to Life, The Universe and Everyth
ing is..." + num)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Can't convert 'int' object to str implicitly
>>> print("The Ultimate Answer to Life, The Universe and Everyth
ing is..." + str(num))
The Ultimate Answer to Life, The Universe and Everything is...42

str.format() can be used to style strings and handle multiple variables

more elegantly than string concatenation

>>> num1 = 42
>>> num2 = 7
>>> print("{} + {} = {}".format(num1, num2, num1 + num2))
42 + 7 = 49

using numbered arguments

>>> num1
42
>>> num2
7
>>> print("{0} + {1} * {0} = {2}".format(num1, num2, num1 + num2
* num1))
42 + 7 * 42 = 336

21

Functions
number formatting - specified using optional argument number, followed by
: and then the formatting style

>>> appx_pi = 22 / 7
>>> appx_pi
3.142857142857143
>>> print("{0:.2f}".format(appx_pi))
3.14
>>> print("{0:10.3f}".format(appx_pi))
3.143

different base
>>> num1
42
>>> print("42 in binary = {:b}".format(num1))
42 in binary = 101010
>>> print("42 in octal = {:o}".format(num1))
42 in octal = 52
>>> print("42 in hex = {:x}".format(num1))
42 in hex = 2a
>>> hex_str = "{:x}".format(num1)
>>> hex_str
'2a'
>>> int(hex_str, 16)
42

Python docs - formatstrings - for more info and examples

range function
By default start=0 and step=1 , so they can be skipped or defined as
appropriate
range(stop)
range(start, stop)

22

Functions

range(start, stop, step)

Note that range output doesn't include stop value - it is always upto
stop value but not including it

We'll see more about list function and data type in later chapters
>>> range(5)
range(0, 5)
>>> list(range(5))
[0, 1, 2, 3, 4]
>>> list(range(-2, 2))
[-2, -1, 0, 1]
>>> list(range(1, 15, 2))
[1, 3, 5, 7, 9, 11, 13]
>>> list(range(10, -5, -2))
[10, 8, 6, 4, 2, 0, -2, -4]

Python docs - Ranges - for more info and examples

type function
Useful to check data type of a variable or value

23

Functions

>>> type(5)
<class 'int'>
>>> type('Hi there!')
<class 'str'>
>>> type(range(7))
<class 'range'>
>>> type(None)
<class 'NoneType'>
>>> type(True)
<class 'bool'>
>>> arr = list(range(4))
>>> arr
[0, 1, 2, 3]
>>> type(arr)
<class 'list'>

Variable Scope
#!/usr/bin/python3
def print_num():
print("Yeehaw! num is visible in this scope, its value is: "
+ str(num))
num = 25
print_num()

Variables defined before function call are visible within the function scope too
Python docs - Default Argument Values - see description for when default
values are evaluated

24

Functions

$ ./variable_scope_1.py
Yeehaw! num is visible in this scope, its value is: 25

What happens when a variable declared within a block is used outside of it?
#!/usr/bin/python3
def square_of_num(num):
sqr_num = num * num
square_of_num(5)
print("5 * 5 = {}".format(sqr_num))

Here, sqr_num is declared inside square_of_num function and not

accessible outside the block
$ ./variable_scope_2.py
Traceback (most recent call last):
File "./variable_scope_2.py", line 7, in <module>
print("5 * 5 = {}".format(sqr_num))
NameError: name 'sqr_num' is not defined

One way to overcome this is to use the global keyword

#!/usr/bin/python3
def square_of_num(num):
global sqr_num
sqr_num = num * num
square_of_num(5)
print("5 * 5 = {}".format(sqr_num))

Now, we can access sqr_num even outside the function definition

25

Functions

$ ./variable_scope_3.py
5 * 5 = 25

If a variable name is same outside and within function definition, the one inside
the function will stay local to the block and not affect the one outside of it
#!/usr/bin/python3
sqr_num = 4
def square_of_num(num):
sqr_num = num * num
print("5 * 5 = {}".format(sqr_num))
square_of_num(5)
print("Whoops! sqr_num is still {}!".format(sqr_num))

Note that using global sqr_num will affect the sqr_num variable outside
the function
$ ./variable_scope_4.py
5 * 5 = 25
Whoops! sqr_num is still 4!

Further Reading
Python docs - scope example
Python docs - global statement

26

Getting User input

Getting User input

Integer input
Floating point input
String input

Integer input
#!/usr/bin/python3
usr_ip = input("Enter an integer number: ")
# Need to explicitly convert input string to desired type
usr_num = int(usr_ip)
sqr_num = usr_num * usr_num
print("Square of entered number is: {}".format(sqr_num))

Let us test the program by giving an integer number and a string

Python docs - integer-literals
$ ./user_input_int.py
Enter an integer number: 23
Square of entered number is: 529
$ ./user_input_int.py
Enter an integer number: abc
Traceback (most recent call last):
File "./user_input_int.py", line 6, in <module>
usr_num = int(usr_ip)
ValueError: invalid literal for int() with base 10: 'abc'

27

Getting User input

Floating point input

#!/usr/bin/python3
usr_ip = input("Enter a floating point number: ")
# Need to explicitly convert input string to desired type
usr_num = float(usr_ip)
sqr_num = usr_num * usr_num
# Limit the number of digits after decimal points to 2
print("Square of entered number is: {0:.2f}".format(sqr_num))

The E scientific notation can also be used when required

Python docs - floating-point-literals
Python docs - floatingpoint
$ ./user_input_float.py
Enter a floating point number: 3.232
Square of entered number is: 10.45
$ ./user_input_float.py
Enter a floating point number: 42.7e5
Square of entered number is: 18232900000000.00
$ ./user_input_float.py
Enter a floating point number: abc
Traceback (most recent call last):
File "./user_input_float.py", line 6, in <module>
usr_num = float(usr_ip)
ValueError: could not convert string to float: 'abc'

String input

28

Getting User input

#!/usr/bin/python3
usr_name = input("Hi there! What's your name? ")
usr_color = input("And your favorite color is? ")
print("{}, I like the {} color too".format(usr_name, usr_color))

No need any type conversion for string and no newline character to be taken
care unlike Perl
$ ./user_input_str.py
Hi there! What's your name? learnbyexample
And your favorite color is? blue
learnbyexample, I like the blue color too

29

Executing external commands

Executing external commands

Calling Shell commands
Calling Shell commands with expansion
Getting command output and redirections
The sample output shown in this chapter may be different based on your
username, working directories, etc

Calling Shell commands

#!/usr/bin/python3
import subprocess
# Executing external command 'date'
subprocess.call('date')
# Passing options and arguments to command
print("\nToday is ", end="", flush=True)
subprocess.call(['date', '-u', '+%A'])
# another example
print("\nSearching for 'hello world'", flush=True)
subprocess.call(['grep', '-i', 'hello world', 'hello_world.py'])

import statement here is used to load the subprocess module, which is

part of the Python standard library

the call function from subprocess module is one of the ways to execute
external commands
By passing True to flush argument (default is False ) we ensure that
our message is printed before subprocess.call
For passing arguments, list of strings is passed instead of single string

30

Executing external commands

$ ./calling_shell_commands.py
Tue Jun 21 18:35:33 IST 2016
Today is Tuesday
Searching for 'hello world'
print("Hello World")

Further Reading
Python docs - subprocess
Python docs - os.system
difference between os.system and subprocess.call
Python docs - import statement

Calling Shell commands with expansion

#!/usr/bin/python3
import subprocess
# Executing command without shell expansion
print("No shell expansion when shell=False", flush=True)
subprocess.call(['echo', 'Hello $USER'])
# Executing command with shell expansion
print("\nshell expansion when shell=True", flush=True)
subprocess.call('echo Hello $USER', shell=True)
# escape quotes if it is part of command
print("\nSearching for 'hello world'", flush=True)
subprocess.call('grep -i \'hello world\' hello_world.py', shell=
True)

By default, subprocess.call will not expand shell wildcards, perform

31

Executing external commands

command substitution, etc
This can be overridden by passing True value for shell argument
Note that the entire command is now passed as string and not as a list of
strings
Quotes need to be escaped if they clash between command string and
quotes within the command itself
Use shell=True only if you are sure of the command being executed, else
it could be a security issue
Python docs - subprocess.Popen
$ ./shell_expansion.py
No shell expansion when shell=False
Hello $USER
shell expansion when shell=True
Hello learnbyexample
Searching for 'hello world'
print("Hello World")

In certain cases, escaping quotes can be avoided by using combination of

single/double quotes as shown below
# use alternate quotes, like this
subprocess.call('grep -i "hello world" hello_world.py', shell=Tr
ue)
# or this
subprocess.call("grep -i 'hello world' hello_world.py", shell=Tr
ue)

Shell command redirections can be used as usual

# use variables for clarity and to avoid long strings within cal
l function
cmd = "grep -h 'test' report.log test_list.txt > grep_test.txt"
subprocess.call(cmd, shell=True)

32

Executing external commands

Workaround to avoid using shell=True
>>> import subprocess, os
>>> subprocess.call(['echo', 'Hello', os.environ.get("USER")])
Hello learnbyexample
0

os.environ.get("USER") gives back the value of environment variable

USER
0 is the exit status, meaning success. It is a caveat of python interpreter

which displays return value too

Getting command output and redirections

33

Executing external commands

#!/usr/bin/python3
import subprocess
# Output includes any error messages also
print("Getting output of 'pwd' command", flush=True)
curr_working_dir = subprocess.getoutput('pwd')
print(curr_working_dir)
# Get status and output of command executed
# Exit status other than '0' is considered as something gone wro
ng
ls_command = 'ls hello_world.py xyz.py'
print("\nCalling command '{}'".format(ls_command), flush=True)
(ls_status, ls_output) = subprocess.getstatusoutput(ls_command)
print("status: {}\noutput: '{}'".format(ls_status, ls_output))
# Suppress error messages if preferred
# subprocess.call() returns status of command which can be used
instead
print("\nCalling command with error msg suppressed", flush=True)
ls_status = subprocess.call(ls_command, shell=True, stderr=subpr
ocess.DEVNULL)
print("status: {}".format(ls_status))

Output of getstatusoutput() is of tuple data type, more info and

examples in later chapters
getstatusoutput() and getoutput() are legacy functions

Use newer functions for more features and secure options

Python docs - subprocess.check_output
Python docs - subprocess.run

34

Executing external commands

$ ./shell_command_output_redirections.py
Getting output of 'pwd' command
/home/learnbyexample/Python/python_programs
Calling command 'ls hello_world.py xyz.py'
status: 2
output: 'ls: cannot access xyz.py: No such file or directory
hello_world.py'
Calling command with error msg suppressed
hello_world.py
status: 2

35

Control Structures

Control Structures
Condition checking
if
for
while
continue and break

Condition checking
simple and combination of tests
>>> num = 5
>>> num > 2
True
>>> num > 3 and num <= 5
True
>>> 3 < num <= 5
True
>>> num % 3 == 0 or num % 5 == 0
True
>>> fav_fiction = 'Harry Potter'
>>> fav_detective = 'Sherlock Holmes'
>>> fav_fiction == fav_detective
False
>>> fav_fiction == "Harry Potter"
True

Testing variable or value by themselves

36

Control Structures

>>> bool(num)
True
>>> bool(fav_detective)
True
>>> bool(3)
True
>>> bool(0)
False
>>> bool("")
False
>>> bool(None)
False
>>> if -1:
... print("-1 evaluates to True in condition checking")
...
-1 evaluates to True in condition checking

The use of in operator in condition checking

Compare this way of checking
>>> def num_chk(n):
... if n == 10 or n == 21 or n == 33:
... print("Number passes condition")
... else:
... print("Number fails condition")
...
>>> num_chk(10)
Number passes condition
>>> num_chk(12)
Number fails condition

vs this one

37

Control Structures

>>> def num_chk(n):

... if n in (10,21,33):
... print("Number passes condition")
... else:
... print("Number fails condition")
...
>>> num_chk(12)
Number fails condition
>>> num_chk(10)
Number passes condition

(10,21,33) is a tuple data type, will be covered in later chapters

Python docs - Truth Value Testing

if

38

Control Structures

#!/usr/bin/python3
num = 45
# only if
if num > 25:
print("Hurray! {} is greater than 25".format(num))
# if-else
if num % 2 == 0:
print("{} is an even number".format(num))
else:
print("{} is an odd number".format(num))
# if-elif-else
# any number of elif can be used
if num < 0:
print("{} is a negative number".format(num))
elif num > 0:
print("{} is a positive number".format(num))
else:
print("{} is neither postive nor a negative number".format(n
um))

Block of code for functions, control structures, etc are distinguished by

indented code
4-space indentation is recommended
Python docs - Coding Style
A common syntax error is leaving out : at end of control structure
statements
Using () around conditions is optional
indented block can have any number of statements, including blank lines
$ ./if_elif_else.py
Hurray! 45 is greater than 25
45 is an odd number
45 is a positive number

39

Control Structures
if-else as conditional operator
#!/usr/bin/python3
num = 42
num_type = 'even' if num % 2 == 0 else 'odd'
print("{} is an {} number".format(num, num_type))

Python doesn't have ?: conditional operator like many other languages

Using if-else in single line like in this example is one workaround
More ways of simulating ternary conditional operator
$ ./if_else_oneliner.py
42 is an even number

for
#!/usr/bin/python3
number = 9
for i in range(1, 5):
mul_table = number * i
print("{} * {} = {}".format(number, i, mul_table))

traditional iteration based loop can be written using range function

recall that by default start=0 , step=1 and stop value is not
inclusive
iterating over variables like list, tuples, etc will be covered in later chapters
Python docs - itertools

40

Control Structures

$ ./for_loop.py
9 * 1 = 9
9 * 2 = 18
9 * 3 = 27
9 * 4 = 36

while
#!/usr/bin/python3
# continuously ask user input till it is a positive integer
usr_string = 'not a number'
while not usr_string.isnumeric():
usr_string = input("Enter a positive integer: ")

while loop allows us to execute block of statements until a condition is

satisfied
Python docs - string methods
$ ./while_loop.py
Enter a positive integer: abc
Enter a positive integer: 1.2
Enter a positive integer: 23
$

continue and break

The continue and break keywords are used to change the normal flow of
loops on certain conditions
continue - skip rest of statements in the loop and start next iteration

41

Control Structures

#!/usr/bin/python3
prev_num = 0
curr_num = 0
print("The first ten numbers in fibonacci sequence: ", end='')
for num in range(10):
print(curr_num, end=' ')
if num == 0:
curr_num = 1
continue
temp = curr_num
curr_num = curr_num + prev_num
prev_num = temp
print("")

continue can be placed anywhere in a loop block without having to worry

about complicated code flow

this example is just to show use of continue , check this for a more
Pythonic way
$ ./loop_with_continue.py
The first ten numbers in fibonacci sequence: 0 1 1 2 3 5 8 13 21
34

break - skip rest of statements in the loop (if any) and exit loop

42

Control Structures

#!/usr/bin/python3
import random
while True:
# as with range() function, 500 is not inclusive
random_int = random.randrange(500)
if random_int % 4 == 0 and random_int % 6 == 0:
break
print("Random number divisible by 4 and 6: {}".format(random_int
))

while True: is generally used as infinite loop

Python docs - random

$ ./loop_with_break.py
Random number divisible by 4 and 6: 168
$ ./loop_with_break.py
Random number divisible by 4 and 6: 216
$ ./loop_with_break.py
Random number divisible by 4 and 6: 24

in case of nested loops, continue and break only affect the immediate
parent loop
Python docs - else clauses on loops

43

Lists

Lists
Assigning List variables
Slicing and Modifying Lists
Copying Lists
List Methods and Miscellaneous
Looping
List Comprehension
Getting List as user input

Assigning List variables

Simple lists and retrieving list elements

44

Lists

>>> vowels = ['a', 'e', 'i', 'o', 'u']

>>> vowels
['a', 'e', 'i', 'o', 'u']
>>> vowels[0]
'a'
>>> vowels[2]
'i'
>>> vowels[10]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: list index out of range
>>> even_numbers = list(range(2, 11, 2))
>>> even_numbers
[2, 4, 6, 8, 10]
>>> even_numbers[-1]
10
>>> even_numbers[-2]
8
>>> even_numbers[-15]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: list index out of range

Mix of data types and multi-dimensional lists

>>> student = ['learnbyexample', 2016, 'Linux, Vim, Python']
>>> print(student)
['learnbyexample', 2016, 'Linux, Vim, Python']
>>> list_2D = [[1, 3, 2, 10], [1.2, -0.2, 0, 2]]
>>> list_2D[0][0]
1
>>> list_2D[1][0]
1.2

Python docs - lists

45

Lists

Slicing and Modifying Lists

Like the range() function, list index has start:stop:step format,
stop value being non-inclusive

The indexing format can be used to extract from list variable or modify itself

46

Lists

>>> books = ['Harry Potter', 'Sherlock Holmes', 'To Kill a Mocki

ng Bird']
>>> books[2] = "Ender's Game"
>>> print(books)
['Harry Potter', 'Sherlock Holmes', "Ender's Game"]
>>> prime = [2, 3, 5, 7, 11]
>>> prime[2:4]
[5, 7]
>>> prime[:3]
[2, 3, 5]
>>> prime[3:]
[7, 11]
>>> prime[-1]
11
>>> prime[-1:]
[11]
>>> prime[-2:]
[7, 11]

>>> prime[::1]
[2, 3, 5, 7, 11]
>>> prime[::2]
[2, 5, 11]
>>> prime[3:1:-1]
[7, 5]
>>> prime[::-1]
[11, 7, 5, 3, 2]
>>> prime[:]
[2, 3, 5, 7, 11]

Copying Lists
Variables in Python contain reference to objects

47

Lists
For example, when an integer variable is modified, the variable's reference is
updated with new object
the id() function returns the "identity" of an object
For variables referring to immutable types like integer and strings, this
distinction usually doesn't cause confusion in their usage
>>> a = 5
>>> id(a)
10105952
>>> a = 10
>>> id(a)
10106112
>>> b = a
>>> id(b)
10106112
>>> b = 4
>>> b
4
>>> a
10
>>> id(b)
10105920

But for variables referring to mutable types like lists, it is important to know
how variables are copied and passed to functions
When an element of list variable is modified, it does so by changing the value
(mutation) of object at that index

48

Lists

>>> a = [1, 2, 5, 4.3]

>>> a
[1, 2, 5, 4.3]
>>> b = a
>>> b
[1, 2, 5, 4.3]
>>> id(a)
140684757120264
>>> id(b)
140684757120264
>>> b[0] = 'xyz'
>>> b
['xyz', 2, 5, 4.3]
>>> a
['xyz', 2, 5, 4.3]

avoid copying lists using indexing format, it works for 1D lists but not for
higher dimensions

49

Lists

>>> prime = [2, 3, 5, 7, 11]

>>> b = prime[:]
>>> id(prime)
140684818101064
>>> id(b)
140684818886024
>>> b[0] = 'a'
>>> b
['a', 3, 5, 7, 11]
>>> prime
[2, 3, 5, 7, 11]
>>> list_2D = [[1, 3, 2, 10], [1.2, -0.2, 0, 2]]
>>> a = list_2D[:]
>>> id(list_2D)
140684818102344
>>> id(a)
140684818103048
>>> a
[[1, 3, 2, 10], [1.2, -0.2, 0, 2]]
>>> a[0][0] = 'a'
>>> a
[['a', 3, 2, 10], [1.2, -0.2, 0, 2]]
>>> list_2D
[['a', 3, 2, 10], [1.2, -0.2, 0, 2]]

use the copy module instead

>>> import copy
>>> list_2D = [[1, 3, 2, 10], [1.2, -0.2, 0, 2]]
>>> c = copy.deepcopy(list_2D)
>>> c[0][0] = 'a'
>>> c
[['a', 3, 2, 10], [1.2, -0.2, 0, 2]]
>>> list_2D
[[1, 3, 2, 10], [1.2, -0.2, 0, 2]]

50

Lists

List Methods and Miscellaneous

adding elements to list
>>> books = []
>>> books
[]
>>> books.append('Harry Potter')
>>> books
['Harry Potter']
>>> even_numbers
[2, 4, 6, 8, 10]
>>> even_numbers += [12, 14, 16, 18, 20]
>>> even_numbers
[2, 4, 6, 8, 10, 12, 14, 16, 18, 20]
>>> even_numbers = [2, 4, 6, 8, 10]
>>> even_numbers.extend([12, 14, 16, 18, 20])
>>> even_numbers
[2, 4, 6, 8, 10, 12, 14, 16, 18, 20]
>>> a = [[1, 3], [2, 4]]
>>> a.extend([[5, 6]])
>>> a
[[1, 3], [2, 4], [5, 6]]

deleting elements from a list - based on index

51

Lists

>>> prime = [2, 3, 5, 7, 11]

>>> prime.pop()
11
>>> prime
[2, 3, 5, 7]
>>> prime.pop(0)
2
>>> prime
[3, 5, 7]
>>> list_2D = [[1, 3, 2, 10], [1.2, -0.2, 0, 2]]
>>> list_2D[0].pop(0)
1
>>> list_2D
[[3, 2, 10], [1.2, -0.2, 0, 2]]
>>> list_2D.pop(1)
[1.2, -0.2, 0, 2]
>>> list_2D
[[3, 2, 10]]

clearing list and alternate method to delete elements

52

Lists

>>> prime.clear()
>>> prime
[]
>>> books = ['Harry Potter', 'Sherlock Holmes', 'To Kill a Mocki
ng Bird']
>>> del books[1]
>>> books
['Harry Potter', 'To Kill a Mocking Bird']
>>> list_2D = [[1, 3, 2, 10], [1.2, -0.2, 0, 2]]
>>> del list_2D[0][1]
>>> list_2D
[[1, 2, 10], [1.2, -0.2, 0, 2]]
>>> del list_2D[0]
>>> list_2D
[[1.2, -0.2, 0, 2]]

deleting elements from a list - based on value

>>> even_numbers = [2, 4, 6, 8, 10]
>>> even_numbers.remove(8)
>>> even_numbers
[2, 4, 6, 10]
>>> even_numbers.remove(12)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: list.remove(x): x not in list

inserting elements at a particular index

53

Lists

>>> books = ['Harry Potter', 'Sherlock Holmes', 'To Kill a Mocki

ng Bird']
>>> books.insert(2, "The Martian")
>>> books
['Harry Potter', 'Sherlock Holmes', 'The Martian', 'To Kill a Mo
cking Bird']

get index of an element

>>> even_numbers = [2, 4, 6, 8, 10]
>>> even_numbers.index(6)
2
>>> even_numbers.index(12)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: 12 is not in list

checking if an element is present

>>> prime = [2, 3, 5, 7, 11]
>>> 3 in prime
True
>>> 13 in prime
False

sorting
>>> a = [1, 5.3, 321, 0, 1, 2]
>>> a.sort()
>>> a
[0, 1, 1, 2, 5.3, 321]
>>> a = [1, 5.3, 321, 0, 1, 2]
>>> a.sort(reverse=True)
>>> a
[321, 5.3, 2, 1, 1, 0]

54

Lists
min and max

>>> a = [321, 899.232, 5.3, 2, 1, -1]

>>> min(a)
-1
>>> max(a)
899.232

reverse list in place

>>> prime = [2, 3, 5, 7, 11]
>>> id(prime)
140684818102088
>>> prime.reverse()
>>> prime
[11, 7, 5, 3, 2]
>>> id(prime)
140684818102088
>>> a = [1, 5.3, 321, 0, 1, 2]
>>> id(a)
140684818886024
>>> a = a[::-1]
>>> a
[2, 1, 0, 321, 5.3, 1]
>>> id(a)
140684818102664

size of lists
>>> prime
[2, 3, 5, 7, 11]
>>> len(prime)
5

summing numeric lists

55

Lists

>>> a
[321, 5.3, 2, 1, 1, 0]
>>> sum(a)
330.3

all and any

>>> conditions = [True, False, True]

>>> all(conditions)
False
>>> any(conditions)
True
>>> conditions[1] = True
>>> all(conditions)
True
>>> a = [321, 5.3, 2, 1, 1, 0]
>>> all(a)
False
>>> any(a)
True

comparing lists
>>> prime
[2, 3, 5, 7, 11]
>>> a = [4, 2]
>>> prime == a
False
>>> prime == [2, 3, 5, 11, 7]
False
>>> prime == [2, 3, 5, 7, 11]
True

building a list of lists can be quite useful

56

Lists

#!/usr/bin/python3
import random
north = ['aloo tikki', 'baati', 'khichdi', 'makki roti', 'poha']
south = ['appam', 'bisibele bath', 'dosa', 'koottu', 'sevai']
west = ['dhokla', 'khakhra', 'modak', 'shiro', 'vada pav']
east = ['hando guri', 'litti', 'momo', 'rosgulla', 'shondesh']
zones = ['North', 'South', 'West', 'East']
choose_dish = [north, south, west, east]
rand_zone = random.randrange(4)
rand_dish = random.randrange(5)
zone = zones[rand_zone]
dish = choose_dish[rand_zone][rand_dish]
print("Would you like to have '{}' speciality '{}' today?".forma
t(zone, dish))

Here we take advantage of the way list variables are referenced

A change in any of the four lists north, south, east and west will reflect
accordingly in choose_dish too
$ ./list_of_lists.py
Would you like to have 'West' speciality 'vada pav' today?
$ ./list_of_lists.py
Would you like to have 'South' speciality 'appam' today?
$ ./list_of_lists.py
Would you like to have 'South' speciality 'dosa' today?
$ ./list_of_lists.py
Would you like to have 'East' speciality 'momo' today?

Further Reading
Python docs - more on lists
Python docs - del
Python docs - len
Python docs - dir

57

Lists
Python docs - sort
Python docs - various techniques for sorting data
Python docs - all, any

Looping
#!/usr/bin/python3
numbers = [2, 12, 3, 25, 624, 21, 5, 9, 12]
odd_numbers = []
even_numbers = []
for num in numbers:
odd_numbers.append(num) if(num % 2) else even_numbers.append
(num)
print("numbers: {}".format(numbers))
print("odd_numbers: {}".format(odd_numbers))
print("even_numbers: {}".format(even_numbers))

usually, it is enough to deal with every element of list without needing index of
elements
$ ./list_looping.py
numbers: [2, 12, 3, 25, 624, 21, 5, 9, 12]
odd_numbers: [3, 25, 21, 5, 9]
even_numbers: [2, 12, 624, 12]

use enumerate() if both index and element is needed

58

Lists

#!/usr/bin/python3
north_dishes = ['Aloo tikki', 'Baati', 'Khichdi', 'Makki roti',
'Poha']
print("My favorite North Indian dishes:")
for idx, item in enumerate(north_dishes):
print("{}. {}".format(idx + 1, item))

enumerate() returns a Tuple data type, more info on tuples in later

chapters
$ ./list_looping_enumeration.py
My favorite North Indian dishes:
1. Aloo tikki
2. Baati
3. Khichdi
4. Makki roti
5. Poha

List Comprehension

59

Lists

#!/usr/bin/python3
import time
numbers = list(range(1,100001))
fl_square_numbers = []
# reference time
t0 = time.perf_counter()
# ------------ for loop -----------for num in numbers:
fl_square_numbers.append(num * num)
# reference time
t1 = time.perf_counter()
# ------- list comprehension ------lc_square_numbers = [num * num for num in numbers]
# performance results
t2 = time.perf_counter()
fl_time = t1 - t0
lc_time = t2 - t1
improvement = (fl_time - lc_time) / fl_time * 100
print("Time with for loop: {:.4f}".format(fl_time))
print("Time with list comprehension: {:.4f}".format(lc_time))
print("Improvement: {:.2f}%".format(improvement
))
if fl_square_numbers == lc_square_numbers:
print("\nfl_square_numbers and lc_square_numbers are equival
ent")
else:
print("\nfl_square_numbers and lc_square_numbers are NOT equ
ivalent")

List comprehensions is a Pythonic way for some of the common looping

60

Lists
constructs
Usually is a more readable and time saving option than loops
In this example, not having to call append() method also saves lot of time
in case of list comprehension
Time values in this example is indicative and not to be taken as absolute
It usually varies even between two runs, let alone different machines
$ ./list_comprehension.py
Time with for loop: 0.0142
Time with list comprehension: 0.0062
Improvement: 56.36%
fl_square_numbers and lc_square_numbers are equivalent

conditional list comprehension

# using if-else conditional in list comprehension
numbers = [2, 12, 3, 25, 624, 21, 5, 9, 12]
odd_numbers = []
even_numbers = []
[odd_numbers.append(num) if(num % 2) else even_numbers.append(nu
m) for num in numbers]
# or a more simpler and readable approach
numbers = [2, 12, 3, 25, 624, 21, 5, 9, 12]
odd_numbers = [num for num in numbers if num % 2]
even_numbers = [num for num in numbers if not num % 2]

Further Reading
For more examples, including nested loops, check these
Python docs - list comprehensions
Python List Comprehensions: Explained Visually
are list comprehensions and functional functions faster than for loops
Python docs - perf_counter
understanding perf_counter and process_time
Python docs - timeit

61

Lists

Getting List as user input

>>> b = list(input('Enter strings separated by space: ').split()
)
Enter strings separated by space: foo bar baz
>>> b
['foo', 'bar', 'baz']
>>> nums = [int(n) for n in input('Enter numbers separated by sp
ace: ').split()]
Enter numbers separated by space: 1 23 5
>>> nums
[1, 23, 5]
>>> primes = [int(n) for n in input('Enter prime numbers separat
ed by comma: ').split(',')]
Enter prime numbers separated by comma: 3,5,7
>>> primes
[3, 5, 7]

Since user input is all treated as string, need to process based on agreed
delimiter and required data type

62

Sequence, Set and Dict data types

Sequence, Set and Dict data types

Strings
Tuples
Set
Dictionary
We have already seen Sequence types in previous chapters - strings, ranges and
lists
We'll see some more operations on strings followed by Tuple, Set and Dict types
in this chapter

Strings
The indexing we saw for lists can be applied to strings as well
As strings are immutable, they can't be modified like lists though

63

Sequence, Set and Dict data types

>>> book = "Alchemist"

>>> book[0]
'A'
>>> book[3]
'h'
>>> book[-1]
't'
>>> book[10]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: string index out of range
>>> book[2:6]
'chem'
>>> book[:5]
'Alche'
>>> book[5:]
'mist'
>>> book[::-1]
'tsimehclA'
>>> book[:]
'Alchemist'
>>> list(book)
['A', 'l', 'c', 'h', 'e', 'm', 'i', 's', 't']
>>> import string
>>> string.ascii_lowercase[:10]
'abcdefghij'
>>> list(string.digits)
['0', '1', '2', '3', '4', '5', '6', '7', '8', '9']

looping over strings

64

Sequence, Set and Dict data types

>>> book
'Alchemist'
>>> for char in book:
... print(char)
...
A
l
c
h
e
m
i
s
t

Other operations

65

Sequence, Set and Dict data types

>>> book
'Alchemist'
>>> len(book)
9
>>> book.index('A')
0
>>> book.index('t')
8
>>> 'A' in book
True
>>> 'B' in book
False
>>> 'z' not in book
True
>>> min('zealous')
'a'
>>> max('zealous')
'z'

Tuples
Tuples can be thought of as sort of lists but immutable

66

Sequence, Set and Dict data types

>>> north_dishes = ('Aloo tikki', 'Baati', 'Khichdi', 'Makki rot

i', 'Poha')
>>> north_dishes
('Aloo tikki', 'Baati', 'Khichdi', 'Makki roti', 'Poha')
>>> north_dishes[0]
'Aloo tikki'
>>> north_dishes[-1]
'Poha'
>>> north_dishes[6]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: tuple index out of range
>>> north_dishes[::-1]
('Poha', 'Makki roti', 'Khichdi', 'Baati', 'Aloo tikki')
>>> north_dishes[0] = 'Poori'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment

Example operations

67

Sequence, Set and Dict data types

>>> 'roti' in north_dishes

False
>>> 'Makki roti' in north_dishes
True
>>> len(north_dishes)
5
>>> min(north_dishes)
'Aloo tikki'
>>> max(north_dishes)
'Poha'
>>> for dish in north_dishes:
... print(dish)
...
Aloo tikki
Baati
Khichdi
Makki roti
Poha

Tuple is handy for multiple variable assignment and returning more than one
value in functions
We have already seen example when using enumerate for iterating
over lists

68

Sequence, Set and Dict data types

>>> a = 5
>>> b = 20
>>> a, b = b, a
>>> a
20
>>> b
5
>>> def min_max(arr):
... return min(arr), max(arr)
...
>>> min_max([23, 53, 1, -34, 9])
(-34, 53)

using () is not always required

>>> words = 'day', 'night'
>>> words
('day', 'night')
>>> coordinates = ((1,2), (4,3), (92,3))
>>> coordinates
((1, 2), (4, 3), (92, 3))
>>> prime = [2, 3, 5, 7, 11]
>>> prime_tuple = tuple((idx + 1, num) for idx, num in enumerate
(prime))
>>> prime_tuple
((1, 2), (2, 3), (3, 5), (4, 7), (5, 11))

converting other types to tuples

similar to list()

69

Sequence, Set and Dict data types

>>> tuple('books')
('b', 'o', 'o', 'k', 's')
>>> a = [321, 899.232, 5.3, 2, 1, -1]
>>> tuple(a)
(321, 899.232, 5.3, 2, 1, -1)

data types can be mixed and matched in different ways

>>> a = [(1,2), ['a', 'b'], ('good', 'bad')]
>>> a
[(1, 2), ['a', 'b'], ('good', 'bad')]
>>> b = ((1,2), ['a', 'b'], ('good', 'bad'))
>>> b
((1, 2), ['a', 'b'], ('good', 'bad'))

Python docs - tuple

Set
Set is unordered collection of objects
Mutable data type
Typically used to maintain unique sequence, perform Set operations like
intersection, union, difference, symmetric difference, etc

70

Sequence, Set and Dict data types

>>> nums = {3, 2, 5, 7, 1, 6.3}

>>> nums
{1, 2, 3, 5, 6.3, 7}
>>> primes = {3, 2, 11, 3, 5, 13, 2}
>>> primes
{2, 3, 11, 13, 5}
>>> nums.union(primes)
{1, 2, 3, 5, 6.3, 7, 11, 13}
>>> primes.difference(nums)
{11, 13}
>>> nums.difference(primes)
{1, 6.3, 7}

Example operations
>>> len(nums)
6
>>> nums[0]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'set' object does not support indexing
>>> book
'Alchemist'
>>> set(book)
{'i', 'l', 's', 'A', 'e', 'h', 'm', 't', 'c'}
>>> set([1, 5, 3, 1, 9])
{1, 9, 3, 5}
>>> list(set([1, 5, 3, 1, 9]))
[1, 9, 3, 5]
>>> nums = {1, 2, 3, 5, 6.3, 7}
>>> nums
{1, 2, 3, 5, 6.3, 7}
>>> nums.pop()

71

Sequence, Set and Dict data types

1
>>> nums
{2, 3, 5, 6.3, 7}
>>> nums.add(1)
>>> nums
{1, 2, 3, 5, 6.3, 7}
>>> 6.3 in nums
True
>>> for n in nums:
... print(n)
...
1
2
3
5
6.3
7

Python docs - set

Python docs - frozenset
set tutorial

Dictionary
dict types can be thought of as unordered list of key:value pairs or a

named list of items

72

Sequence, Set and Dict data types

>>> marks = {'Rahul' : 86, 'Ravi' : 92, 'Rohit' : 75}

>>> marks
{'Ravi': 92, 'Rohit': 75, 'Rahul': 86}
>>> fav_books = {}
>>> fav_books['fantasy'] = 'Harry Potter'
>>> fav_books['detective'] = 'Sherlock Holmes'
>>> fav_books['thriller'] = 'The Da Vinci Code'
>>> fav_books
{'thriller': 'The Da Vinci Code', 'fantasy': 'Harry Potter', 'de
tective': 'Sherlock Holmes'}
>>> marks.keys()
dict_keys(['Ravi', 'Rohit', 'Rahul'])
>>> fav_books.values()
dict_values(['The Da Vinci Code', 'Harry Potter', 'Sherlock Holm
es'])

looping and printing

73

Sequence, Set and Dict data types

>>> for book in fav_books.values():

... print(book)
...
The Da Vinci Code
Harry Potter
Sherlock Holmes
>>> for name, mark in marks.items():
... print(name, mark, sep=': ')
...
Ravi: 92
Rohit: 75
Rahul: 86
>>> import pprint
>>> pp = pprint.PrettyPrinter(indent=4)
>>> pp.pprint(fav_books)
{ 'detective': 'Sherlock Holmes',
'fantasy': 'Harry Potter',
'thriller': 'The Da Vinci Code'}

modifying dicts and example operations

74

Sequence, Set and Dict data types

>>> marks
{'Ravi': 92, 'Rohit': 75, 'Rahul': 86}
>>> marks['Rajan'] = 79
>>> marks
{'Ravi': 92, 'Rohit': 75, 'Rahul': 86, 'Rajan': 79}
>>> del marks['Ravi']
>>> marks
{'Rohit': 75, 'Rahul': 86, 'Rajan': 79}
>>> len(marks)
3
>>> fav_books
{'thriller': 'The Da Vinci Code', 'fantasy': 'Harry Potter', 'de
tective': 'Sherlock Holmes'}
>>> "fantasy" in fav_books
True
>>> "satire" in fav_books
False

Further Reading
Python docs - dict
Python docs - pprint

75

Text Processing

Text Processing
String methods
Regular Expressions
Pattern matching and extraction
Search and Replace
Compiling Regular Expressions
Further Reading on Regular Expressions

String methods
translate string characters
maketrans() to get translation table
translate() to perform the string mapping based on translation table

the first argument to maketrans() is string characters to be replaced, the

second is characters to replace with and the third is characters to be mapped
to None
character translation examples

76

Text Processing

>>> greeting = '===== Have a great day ====='

>>> greeting.translate(greeting.maketrans('=','-'))
'----- Have a great day -----'
>>> greeting = '===== Have a great day!! ====='
>>> greeting.translate(greeting.maketrans('=','-', '!'))
'----- Have a great day -----'
>>> import string
>>> quote = 'SIMPLICITY IS THE ULTIMATE SOPHISTICATION'
>>> tr_table = quote.maketrans(string.ascii_uppercase,string.asc
ii_lowercase)
>>> quote.translate(tr_table)
'simplicity is the ultimate sophistication'
>>> sentence = "Thi1s is34 a senten6ce"
>>> sentence.translate(sentence.maketrans('', '', string.digits)
)
'This is a sentence'

removing leading/trailing/both characters

only consecutive characters from start/end string are removed
by default whitespace characters are stripped
if more than one character is specified, it is treated as a set and all
combinations of it are used

77

Text Processing

>>> greeting = ' Have a nice day :) '

>>> greeting.strip()
'Have a nice day :)'
>>> greeting.rstrip()
' Have a nice day :)'
>>> greeting.lstrip()
'Have a nice day :) '
>>> greeting.strip(') :')
'Have a nice day'
>>> greeting = '===== Have a great day!! ====='
>>> greeting.strip('=')
' Have a great day!! '

styling
width argument specifies total output string length
>>> ' Hello World '.center(40, '*')
'************* Hello World **************'

changing case and case checking

78

Text Processing

>>> sentence = 'thIs iS a saMple StrIng'

>>> sentence.capitalize()
'This is a sample string'
>>> sentence.title()
'This Is A Sample String'
>>> sentence.lower()
'this is a sample string'
>>> sentence.upper()
'THIS IS A SAMPLE STRING'
>>> sentence.swapcase()
'THiS Is A SAmPLE sTRiNG'
>>> 'good'.islower()
True
>>> 'good'.isupper()
False

check if string is made up of numbers

>>> '1'.isnumeric()
True
>>> 'abc1'.isnumeric()
False
>>> '1.2'.isnumeric()
False

check if character sequence is present or not

79

Text Processing

>>> sentence = 'This is a sample string'

>>> 'is' in sentence
True
>>> 'this' in sentence
False
>>> 'This' in sentence
True
>>> 'this' in sentence.lower()
True
>>> 'is a' in sentence
True
>>> 'test' not in sentence
True

matching character sequence at start/end of string

>>> sentence
'This is a sample string'
>>> sentence.startswith('This')
True
>>> sentence.startswith('The')
False
>>> sentence.endswith('ing')
True
>>> sentence.endswith('ly')
False

split string based on character sequence

returns a list
to split using regular expressions, use re.split() instead

80

Text Processing

>>> sentence = 'This is a sample string'

>>> sentence.split()
['This', 'is', 'a', 'sample', 'string']
>>> "oranges:5".split(':')
['oranges', '5']
>>> "oranges :: 5".split(' :: ')
['oranges', '5']
>>> "a e i o u".split(' ', maxsplit=1)
['a', 'e i o u']
>>> "a e i o u".split(' ', maxsplit=2)
['a', 'e', 'i o u']
>>> line = '(1.0 2.0 3.0)'
>>> nums = [float(s) for s in line.strip('()').split()]
>>> nums
[1.0, 2.0, 3.0]

joining list of strings

>>> str_list
['This', 'is', 'a', 'sample', 'string']
>>> ' '.join(str_list)
'This is a sample string'
>>> '-'.join(str_list)
'This-is-a-sample-string'
>>> '--'.join(str_list)
'This--is--a--sample--string'

replace characters
third argument specifies how many times replace has to be performed

81

Text Processing

>>> phrase = '2 be or not 2 be'

>>> phrase.replace('2', 'to')
'to be or not to be'
>>> phrase.replace('2', 'to', 1)
'to be or not 2 be'

Further Reading
Python docs - string methods
python string methods tutorial

Regular Expressions
Handy reference of regular expression elements
Meta characters

Description

anchor, match from beginning of string

anchor, match end of string

Match any character except newline character \n

OR operator for matching multiple patterns

()

for grouping patterns and also extraction

[]

Character class - match one character among many

\^

use \ to match meta characters like ^

82

Text Processing

Quantifiers

Description

Match zero or more times the preceding character

Match one or more times the preceding character

Match zero or one times the preceding character

{n}

Match exactly n times

{n,}

Match at least n times

{n,m}

Match at least n times but not more than m times

Character
classes

Description

[aeiou]

Match any vowel

[^aeiou]

^ inverts selection, so this matches any consonant

[a-f]

Match any of abcdef character

\d

Match a digit, same as [0-9]

\D

Match non-digit, same as [^0-9] or [^\d]

\w

Match alphanumeric and underscore character, same as [azA-Z_]

\W

Match non-alphanumeric and underscore character, same as

[^a-zA-Z_] or [^\w]

\s

Match white-space character, same as [\ \t\n\r\f\v]

\S

Match non white-space character, same as [^\s]

\b

word boundary, word defined as sequence of alphanumeric

characters

\B

not a word boundary

83

Text Processing

Compilation
Flags

Description

re.I

ignore case

re.M

multiline mode, ^ and $ anchors work on internal lines

re.S

singleline mode, . will also match \n

re.V

verbose mode, for better readability and adding

comments

Python docs - Compilation Flags - for more details and long names for flags

Variable

Description

\1, \2, \3 etc

backreferencing matched patterns

\g<1>, \g<2>,
\g<3> etc

backreferencing matched patterns, useful to differentiate

numbers and backreferencing

Pattern matching and extraction

matching/extracting sequence of characters
use re.search() to see if a string contains a pattern or not
use re.findall() to get a list of matching patterns
use re.split() to get a list from splitting a string based on a pattern
their syntax given below
re.search(pattern, string, flags=0)
re.findall(pattern, string, flags=0)
re.split(pattern, string, maxsplit=0, flags=0)

84

Text Processing

>>> import re
>>> string = "This is a sample string"
>>> bool(re.search('is', string))
True
>>> bool(re.search('this', string))
False
>>> bool(re.search('this', string, re.I))
True
>>> bool(re.search('T', string))
True
>>> bool(re.search('is a', string))
True
>>> re.findall('i', string)
['i', 'i', 'i']

using regular expressions

use the r'' format when using regular expression elements

85

Text Processing

>>> string
'This is a sample string'
>>> re.findall('is', string)
['is', 'is']
>>> re.findall('\bis', string)
[]
>>> re.findall(r'\bis', string)
['is']
>>> re.findall(r'\w+', string)
['This', 'is', 'a', 'sample', 'string']
>>> re.split(r'\s+', string)
['This', 'is', 'a', 'sample', 'string']
>>> re.split(r'\d+', 'Sample123string54with908numbers')
['Sample', 'string', 'with', 'numbers']
>>> re.split(r'(\d+)', 'Sample123string54with908numbers')
['Sample', '123', 'string', '54', 'with', '908', 'numbers']

backreferencing
>>> quote = "So many books, so little time"
>>> re.search(r'([a-z]{2,}).*\1', quote, re.I)
<_sre.SRE_Match object; span=(0, 17), match='So many books, so'>
>>> re.search(r'([a-z])\1', quote, re.I)
<_sre.SRE_Match object; span=(9, 11), match='oo'>
>>> re.findall(r'([a-z])\1', quote, re.I)
['o', 't']

86

Text Processing

Search and Replace

Syntax
re.sub(pattern, repl, string, count=0, flags=0)

simple substitutions
re.sub will not change value of variable passed to it, has to be explicity

assigned
>>> sentence = 'This is a sample string'
>>> re.sub('sample', 'test', sentence)
'This is a test string'
>>> sentence
'This is a sample string'
>>> sentence = re.sub('sample', 'test', sentence)
>>> sentence
'This is a test string'
>>> re.sub('/', '-', '25/06/2016')
'25-06-2016'
>>> re.sub('/', '-', '25/06/2016', count=1)
'25-06/2016'
>>> greeting = '***** Have a great day *****'
>>> re.sub('\*', '=', greeting)
'===== Have a great day ====='

backreferencing

87

Text Processing

>>> words = 'night and day'

>>> re.sub(r'(\w+)( \w+ )(\w+)', r'\3\2\1', words)
'day and night'
>>> line = 'Can you spot the the mistakes? I i seem to not'
>>> re.sub(r'\b(\w+) \1\b', r'\1', line, flags=re.I)
'Can you spot the mistakes? I seem to not'

using functions in replace part of re.sub()

>>> import math
>>> numbers = '1 2 3 4 5'
>>> def fact_num(n):
... return str(math.factorial(int(n.group(1))))
...
>>> re.sub(r'(\d+)', fact_num, numbers)
'1 2 6 24 120'
>>> re.sub(r'(\d+)', lambda m: str(math.factorial(int(m.group(1)
))), numbers)
'1 2 6 24 120'

call functions from re.sub

replace string pattern with output of function
lambda tutorial

Compiling Regular Expressions

88

Text Processing

>>> swap_words = re.compile(r'(\w+)( \w+ )(\w+)')

>>> swap_words
re.compile('(\\w+)( \\w+ )(\\w+)')
>>> words = 'night and day'
>>> swap_words.search(words).group()
'night and day'
>>> swap_words.search(words).group(1)
'night'
>>> swap_words.search(words).group(2)
' and '
>>> swap_words.search(words).group(3)
'day'
>>> swap_words.search(words).group(4)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: no such group
>>> bool(swap_words.search(words))
True
>>> swap_words.findall(words)
[('night', ' and ', 'day')]
>>> swap_words.sub(r'\3\2\1', words)
'day and night'
>>> swap_words.sub(r'\3\2\1', 'yin and yang')
'yang and yin'

Further Reading on Regular Expressions

Python docs - re module
Python docs - introductory tutorial to using regular expressions
developers.google - Regular Expressions tutorial
automatetheboringstuff - Regular Expressions
Comprehensive reference: What does this regex mean?

89

Text Processing
Practice tools
online regex tester shows explanations, has reference guides and ability
to save and share regex
regexone - interative tutorial
cheatsheet - one can also learn it interactively
regexcrossword - practice by solving crosswords, read 'How to play'
section before you start

90

File handling

File handling
open function
Reading files
Writing to files

open function
syntax: open(file, mode='r', buffering=-1, encoding=None, errors=None,
newline=None, closefd=True, opener=None)
>>> import locale
>>> locale.getpreferredencoding()
'UTF-8'

We'll be seeing these modes in this chapter

r open file for reading
w open file for writing
a open file for appending

default is text mode, so passing 'r' and 'rt' are equivalent

for binary mode, it would be 'rb', 'wb' and so on
locale.getpreferredencoding() gives default encoding being used

Python docs - open

Python docs - standard encodings

Reading files

91

File handling

#!/usr/bin/python3
# open file, read line by line and print it
filename = 'hello_world.py'
f = open(filename, 'r', encoding='ascii')
print("Contents of " + filename)
print('-' * 30)
for line in f:
print(line, end='')
f.close()
# 'with' is a simpler alternative, automatically handles file cl
osing
filename = 'while_loop.py'
print("\n\nContents of " + filename)
print('-' * 30)
with open(filename, 'r', encoding='ascii') as f:
for line in f:
print(line, end='')

default encoding is usually 'UTF-8', use 'ascii' where applicable

using with and file handle name as f is usual convention

92

File handling

$ ./file_reading.py
Contents of hello_world.py
-----------------------------#!/usr/bin/python3
print("Hello World")

Contents of while_loop.py
-----------------------------#!/usr/bin/python3
# continuously ask user input till it is a positive integer
usr_string = 'not a number'
while not usr_string.isnumeric():
usr_string = input("Enter a positive integer: ")

If file doesn't exist

#!/usr/bin/python3
with open('xyz.py', 'r', encoding='ascii') as f:
for line in f:
print(line, end='')

Error if file is not found

$ ./file_reading_error.py
Traceback (most recent call last):
File "./file_reading_error.py", line 3, in <module>
with open('xyz.py', 'r', encoding='ascii') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'xyz.py'
$ echo $?
1

read entire file content as single string using read()

93

File handling

>>> f = open('hello_world.py', 'r', encoding='ascii')

>>> f
<_io.TextIOWrapper name='hello_world.py' mode='r' encoding='asci
i'>
>>> print(f.read())
#!/usr/bin/python3
print("Hello World")

read line by line using readline()

>>> f = open('hello_world.py', 'r', encoding='ascii')
>>> print(f.readline(), end='')
#!/usr/bin/python3
>>> print(f.readline(), end='')
>>> print(f.readline(), end='')
print("Hello World")
>>> f.close()
>>> print(f.readline(), end='')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: I/O operation on closed file.

read all lines of file as list using readlines()

note the plural form
>>> f = open('hello_world.py', 'r', encoding='ascii')
>>> all_lines = f.readlines()
>>> all_lines
['#!/usr/bin/python3\n', '\n', 'print("Hello World")\n']

check if file is closed or not

94

File handling

>>> f = open('hello_world.py', 'r', encoding='ascii')

>>> f.closed
False
>>> f.close()
>>> f.closed
True

Writing to files
#!/usr/bin/python3
with open('new_file.txt', 'w', encoding='ascii') as f:
f.write("This is a sample line of text\n")
f.write("Yet another line\n")

Use the write() method to print a string to files

To add text to an existing file, use 'a' mode instead of 'w'
$ ./file_writing.py
$ cat new_file.txt
This is a sample line of text
Yet another line

95

Command line arguments

Command line arguments

Known number of arguments
Varying number of arguments
Using program name in code
Command line switches

Known number of arguments

#!/usr/bin/python3
import sys
if len(sys.argv) != 3:
sys.exit("Error: Please provide exactly two numbers as argum
ents")
else:
(num1, num2) = sys.argv[1:]
total = int(num1) + int(num2)
print("{} + {} = {}".format(num1, num2, total))

Command line arguments given to Python program are automatically saved

in sys.argv list
It is a good idea to let the user know something went wrong
As the program terminates with error message for wrong usage, use
sys.exit() for error message (exit status 1) or a custom exit status

number
Python docs - sys module

96

Command line arguments

$ ./sum_of_two_numbers.py 2 3
2 + 3 = 5
$ echo $?
0
$ ./sum_of_two_numbers.py 2 3 7
Error: Please provide exactly two numbers as arguments
$ echo $?
1
$ ./sum_of_two_numbers.py 2 'x'
Traceback (most recent call last):
File "./sum_of_two_numbers.py", line 9, in <module>
total = int(num1) + int(num2)
ValueError: invalid literal for int() with base 10: 'x'
$ echo $?
1

Varying number of arguments

97

Command line arguments

#!/usr/bin/python3
import sys, pathlib, subprocess
if len(sys.argv) < 2:
sys.exit("Error: Please provide atleast one filename as argu
ment")
input_files = sys.argv[1:]
files_not_found = []
for filename in input_files:
if not pathlib.Path(filename).is_file():
files_not_found.append("File '{}' not found".format(file
name))
continue
line_count = subprocess.getoutput('wc -l < ' + filename)
print("{0:40}: {1:4} lines".format(filename, line_count))
print("\n".join(files_not_found))

When dealing with filenames obtained as user input, it is good to do a sanity

check before processing
Python docs - pathlib

98

Command line arguments

$ ./varying_command_line_args.py
Error: Please provide atleast one filename as argument
$ echo $?
1
$ #selective output presented
$ ./varying_command_line_args.py *.py
calling_shell_commands.py : 14 lines
for_loop.py : 6 lines
functions_default_arg_value.py : 16 lines
functions.py : 24 lines
hello_world.py : 3 lines
if_elif_else.py : 22 lines
if_else_oneliner.py : 6 lines
shell_command_output_redirections.py : 21 lines
...
$ ./varying_command_line_args.py hello_world.py xyz.py for_loop.
py abc.py
hello_world.py : 3 lines
for_loop.py : 6 lines
File 'xyz.py' not found
File 'abc.py' not found

use os module instead of pathlib for file checking if your Python version
is not 3.4 and higher
import os
if not os.path.isfile(filename):
sys.exit("File '{}' not found".format(filename))

Using program name in code

99

Command line arguments

#!/usr/bin/python3
import sys, pathlib, subprocess, re
if len(sys.argv) != 2:
sys.exit("Error: Please provide exactly one filename as argu
ment")
program_name = sys.argv[0]
filename = sys.argv[1]
if not pathlib.Path(filename).is_file():
sys.exit("File '{}' not found".format(filename))
if re.search(r'line_count.py', program_name):
lc = subprocess.getoutput('wc -l < ' + filename)
print("No. of lines in '{}' is: {}".format(filename, lc))
elif re.search(r'word_count.py', program_name):
wc = subprocess.getoutput('wc -w < ' + filename)
print("No. of words in '{}' is: {}".format(filename, wc))
else:
sys.exit("Program name '{}' not recognized".format(program_n
ame))

Same program can be repurposed for different functionalities based on its

name

100

Command line arguments

$ ./line_count.py if_elif_else.py
No. of lines in 'if_elif_else.py' is: 22
$ ln -s line_count.py word_count.py
$ ./word_count.py if_elif_else.py
No. of words in 'if_elif_else.py' is: 73
$ ln -s line_count.py abc.py
$ ./abc.py if_elif_else.py
Program name './abc.py' not recognized
$ wc -lw if_elif_else.py
22 73 if_elif_else.py

Command line switches

#!/usr/bin/python3
import argparse, subprocess
parser = argparse.ArgumentParser()
parser.add_argument('-f', '--file', help="file to be sorted", re
quired=True)
parser.add_argument('-u', help="sort uniquely", action="store_tr
ue")
args = parser.parse_args()
if args.u:
subprocess.call(['sort', '-u', args.file, '-o', args.file])
else:
subprocess.call(['sort', args.file, '-o', args.file])

using argparse module is a simpler way to build programs that behave like
shell commands
By default it adds a help option -h (short form) and --help (long form)

101

Command line arguments

as well as handles wrong usage
In this example:
-f or --file option is declared as required option, it accepts an

argument (which we treat as input file name in code)

-u is an optional flag

the help argument is used to specify text to be displayed for those options
in help message

102

Command line arguments

$ ./sort_file.py
usage: sort_file.py [-h] -f FILE [-u]
sort_file.py: error: the following arguments are required: -f/-file
$ ./sort_file.py -h
usage: sort_file.py [-h] -f FILE [-u]
optional arguments:
-h, --help show this help message and exit
-f FILE, --file FILE file to be sorted
-u sort uniquely
$ ./sort_file.py -f test_list.txt
$ cat test_list.txt
async_test
basic_test
input_test
input_test
output_test
sync_test
$ ./sort_file.py -f test_list.txt -u
$ cat test_list.txt
async_test
basic_test
input_test
output_test
sync_test
$ ./sort_file.py -f xyz.txt
sort: cannot read: xyz.txt: No such file or directory

specifying positional arguments

103

Command line arguments

#!/usr/bin/python3
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('num1', type=int, help="first number")
parser.add_argument('num2', type=int, help="second number")
args = parser.parse_args()
total = args.num1 + args.num2
print("{} + {} = {}".format(args.num1, args.num2, total))

more readable approach than manually parsing sys.argv as well as

default help and error handling
type conversions required can be specified while building parser itself
$ ./sum2nums_argparse.py
usage: sum2nums_argparse.py [-h] num1 num2
sum2nums_argparse.py: error: the following arguments are require
d: num1, num2
$ ./sum2nums_argparse.py -h
usage: sum2nums_argparse.py [-h] num1 num2
positional arguments:
num1 first number
num2 second number
optional arguments:
-h, --help show this help message and exit
$ ./sum2nums_argparse.py 3 4
3 + 4 = 7
$ ./sum2nums_argparse.py 3 4 7
usage: sum2nums_argparse.py [-h] num1 num2
sum2nums_argparse.py: error: unrecognized arguments: 7

104

Command line arguments

Further Reading
Python docs - argparse tutorial
Python docs - argparse module
argparse examples with explanation
Python docs - getopt module

105

Exception Handling and Debugging

Exception Handling and Debugging

Exception Handling
Syntax check
pdb
Importing program

Exception Handling
We have seen plenty of errors in previous chapters when something goes
wrong or some input was given erroneously
For example:
$ ./user_input_int.py
Enter an integer number: abc
Traceback (most recent call last):
File "./user_input_int.py", line 6, in <module>
usr_num = int(usr_ip)
ValueError: invalid literal for int() with base 10: 'abc'

In such cases, it might be preferred to inform the user on the error and give a
chance to correct it
Python provides the try-except construct to achieve this

106

Exception Handling and Debugging

#!/usr/bin/python3
while True:
try:
usr_num = int(input("Enter an integer number: "))
break
except ValueError:
print("Not an integer, try again")
print("Square of entered number is: {}".format(usr_num * usr_num
))

except can be used for particular error (in this case ValueError ) or

without argument to handle any kind of error

$ ./user_input_exception.py
Enter an integer number: a
Not an integer, try again
Enter an integer number: 1.2
Not an integer, try again
Enter an integer number: 3
Square of entered number is: 9

Further Reading
Python docs - errors and exception handling
Python docs - raising exceptions
Python docs - built-in exceptions
Python docs - pass statement

Syntax check
Python's command line options can be used for variety of purposes
Syntax checking is one of them

107

Exception Handling and Debugging

$ python3 -m py_compile syntax_error.py

File "syntax_error.py", line 3
print "Have a nice day"
^
SyntaxError: Missing parentheses in call to 'print'

Useful to quickly catch syntax errors like missing : for if for with etc
and () for print statements
While this example might be trivial, real world program might have thousands
of lines and tougher to find typos
Python docs - cmdline
One-liners: #1, #2, #3

pdb
Invoking debugger is another use of cmdline
Use it instead of using print all over a program when something goes
wrong, plus one can use breakpoints and other features specific to debugging
programs

108

Exception Handling and Debugging

$ python3 -m pdb if_elif_else.py

> /home/learnbyexample/python_programs/if_elif_else.py(3)<module
>()
-> num = 45
(Pdb) p num
*** NameError: name 'num' is not defined
(Pdb) n
> /home/learnbyexample/python_programs/if_elif_else.py(6)<module
>()
-> if num > 25:
(Pdb) p num
45
(Pdb) l
1 #!/usr/bin/python3
2
3 num = 45
4
5 # only if
6 -> if num > 25:
7 print("Hurray! {} is greater than 25".format(num))
8
9 # if-else
10 if num % 2 == 0:
11 print("{} is an even number".format(num))
(Pdb) q

l prints code around the current statement the debugger is at, useful to

visualize the progress of debug effort

s execute current line, steps inside function calls
n execute current line and treats function as single execution step
c continue execution until next breakpoint
p variable print value of variable
h list of commands
h c help on c command
q quit the debugger

Further Reading
Python docs - pdb
109

Exception Handling and Debugging

common runtime errors
common beginner errors as a flowchart
Common pitfalls
Python docs - Basic Logging Tutorial

Importing program
One can also import a program directly in Interpreter to test functions
if __name__ == "__main__": construct

code inside that construct will be executed when program is intended to

be run - ex: executing the program from command line
code inside that construct will NOT be executed when program is
intended to be imported as module - ex: to use programs's functions
A good practice is to put all code outside of functions inside a main function
and call main() inside the if __name__ == "__main__": construct
Note that __name__ and __main__ have two underscore characters as
prefix and suffix

110

Exception Handling and Debugging

#!/usr/bin/python3
# ----- function without arguments ----def greeting():
print("-----------------------------")
print(" Hello World ")
print("-----------------------------")
# ----- function with arguments ----def sum_two_numbers(num1, num2):
sum = num1 + num2
print("{} + {} = {}".format(num1, num2, sum))
# ----- function with return value ----def num_square(num):
return num * num
# ----- main ----def main():
greeting()
sum_two_numbers(3, 4)
my_num = 3
print(num_square(2))
print(num_square(my_num))
if __name__ == "__main__":
main()

When run as a program

$ ./functions_main.py
---------------------------- Hello World
----------------------------3 + 4 = 7
4
9

111

Exception Handling and Debugging

When importing
>>> import functions_main
>>> functions_main.greeting()
---------------------------- Hello World
---------------------------->>> functions_main.num_square()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: num_square() missing 1 required positional argument:
'num'
>>> functions_main.num_square(5)
25
>>> functions_main.main()
---------------------------- Hello World
----------------------------3 + 4 = 7
4
9

Further Reading
Python docs - __main__
What does if __name__ == "__main__" do?
Python packaging guide
diveintopython3 - packaging

112

Docstrings

Docstrings
Style guide
Palindrome example

Style guide
Paraphrased from Python docs - coding style
Use 4-space indentation, and no tabs.
4 spaces are a good compromise between small indentation (allows
greater nesting depth) and large indentation (easier to read). Tabs
introduce confusion, and are best left out.
Wrap lines so that they dont exceed 79 characters.
This helps users with small displays and makes it possible to have
several code files side-by-side on larger displays.
Use blank lines to separate functions and classes, and larger blocks of code
inside functions.
When possible, put comments on a line of their own.
Use docstrings.
Use spaces around operators and after commas
Name your classes and functions consistently;
the convention is to use CamelCase for classes and
lower_case_with_underscores for functions and methods
Style guides
PEP 0008
Google - pyguide
elements of python style
The Hitchhikers Guide to Python - handbook of best practices

113

Docstrings

Palindrome example
#!/usr/bin/python3
"""
Asks for user input and tells if string is palindrome or not
Allowed characters: alphabets and punctuations .,;:'"-!?
Minimum alphabets: 3 and cannot be all same
Informs if input is invalid and asks user for input again
"""
import re
def is_palindrome(usr_ip):
"""
Checks if string is a palindrome
ValueError: if string is invalid
Returns True if palindrome, False otherwise
"""
# remove punctuations & whitespace and change to all lowerca
se
ip_str = re.sub(r'[\s.;:,\'"!?-]', r'', usr_ip).lower()
if re.search(r'[^a-zA-Z]', ip_str):
raise ValueError("Characters other than alphabets and pu
nctuations")
elif len(ip_str) < 3:
raise ValueError("Less than 3 alphabets")
else:
return ip_str == ip_str[::-1] and not re.search(r'^(.)\1
+$', ip_str)
def main():
while True:

114

Docstrings
try:
usr_ip = input("Enter a palindrome: ")
if is_palindrome(usr_ip):
print("{} is a palindrome".format(usr_ip))
else:
print("{} is NOT a palindrome".format(usr_ip))
break
except ValueError as e:
print('Error: ' + str(e))
if __name__ == "__main__":
main()

The first triple quoted strings marks the docstring for entire program
The second one inside is_palindrome() is specific for that function
$ ./palindrome.py
Enter a palindrome: as2
Error: Characters other than alphabets and punctuations
Enter a palindrome: "Dammit, I'm mad!"
"Dammit, I'm mad!" is a palindrome
$ ./palindrome.py
Enter a palindrome: a'a
Error: Less than 3 alphabets
Enter a palindrome: aaa
aaa is NOT a palindrome

Let's see how docstrings can be used as help

Notice how docstring gets automatically formatted

115

Docstrings

>>> import palindrome

>>> help(palindrome)
Help on module palindrome:
NAME
palindrome - Asks for user input and tells if string is pali
ndrome or not
DESCRIPTION
Allowed characters: alphabets and punctuations .,;:'"-!?
Minimum alphabets: 3 and cannot be all same
Informs if input is invalid and asks user for input again
FUNCTIONS
is_palindrome(usr_ip)
Checks if string is a palindrome
ValueError: if string is invalid
Returns True if palindrome, False otherwise
main()
FILE
/home/learnbyexample/python_programs/palindrome.py

One can get help on functions directly too

116

Docstrings

>>> help(palindrome.is_palindrome)
Help on function is_palindrome in module palindrome:
is_palindrome(usr_ip)
Checks if string is a palindrome
ValueError: if string is invalid
Returns True if palindrome, False otherwise

and of course test the functions

>>> palindrome.is_palindrome('aaa')
False
>>> palindrome.is_palindrome('Madam')
True
>>> palindrome.main()
Enter a palindrome: 3452
Error: Characters other than alphabets and punctuations
Enter a palindrome: Malayalam
Malayalam is a palindrome

Further Reading
docstring formats
exception message capturing

117

Further Reading

Further Reading
Standard topics not covered
Useful links on coding
Python extensions

Python curated resources - online courses, books, coding practice sites,

frameworks for GUI/Game/Web, cheatsheet, online help forums, etc..

Standard topics not covered

Python docs - classes
Python docs - virtual environment
Python docs - modules and packaging
Python packaging guide

Useful links on coding

Python docs - FAQ
list of common questions on stackoverflow
profiling Python code for performance
pylint
code patterns
Performance tips

Python extensions
Cython - optimising static compiler for both the Python programming
language and the extended Cython programming language

118

Further Reading
Jython - Python for the Java Platform
NumPy - fundamental package for scientific computing with Python

119