NPTEL MOOC
PROGRAMMING,
DATA STRUCTURES AND
ALGORITHMS IN PYTHON
Week 4, Lecture 1
Madhavan Mukund, Chennai Mathematical Institute
http://www.cmi.ac.in/~madhavan
O(n )
2 sorting algorithms
Selection sort and insertion sort are both O(n2)
O(n2) sorting is infeasible for n over 5000
A different strategy?
Divide array in two equal parts
Separately sort left and right half
Combine the two sorted halves to get the full array
sorted
Combining sorted lists
Given two sorted lists A and B, combine into a
sorted list C
Compare first element of A and B
Move it into C
Repeat until all elements in A and B are over
Merging A and B
Merging two sorted lists
32 74 89
21 55 64
Merging two sorted lists
32 74 89
21 55 64
21
Merging two sorted lists
32 74 89
21 55 64
21 32
Merging two sorted lists
32 74 89
21 55 64
21 32 55
Merging two sorted lists
32 74 89
21 55 64
21 32 55 64
Merging two sorted lists
32 74 89
21 55 64
21 32 55 64 74
Merging two sorted lists
32 74 89
21 55 64
21 32 55 64 74 89
Merge Sort
Sort A[0:n//2]
Sort A[n//2:n]
Merge sorted halves into B[0:n]
How do we sort the halves?
Recursively, using the same strategy!
Merge Sort
43 32 22 78 63 57 91 13
Merge Sort
43 32 22 78 63 57 91 13
43 32 22 78
Merge Sort
43 32 22 78 63 57 91 13
43 32 22 78 63 57 91 13
Merge Sort
43 32 22 78 63 57 91 13
43 32 22 78 63 57 91 13
43 32 22 78
Merge Sort
43 32 22 78 63 57 91 13
43 32 22 78 63 57 91 13
43 32 22 78 63 57 91 13
Merge Sort
43 32 22 78 63 57 91 13
43 32 22 78 63 57 91 13
43 32 22 78 63 57 91 13
43 32
Merge Sort
43 32 22 78 63 57 91 13
43 32 22 78 63 57 91 13
43 32 22 78 63 57 91 13
43 32 22 78
Merge Sort
43 32 22 78 63 57 91 13
43 32 22 78 63 57 91 13
43 32 22 78 63 57 91 13
43 32 22 78 63 57
Merge Sort
43 32 22 78 63 57 91 13
43 32 22 78 63 57 91 13
43 32 22 78 63 57 91 13
43 32 22 78 63 57 91 13
Merge Sort
43 32 22 78 63 57 91 13
43 32 22 78 63 57 91 13
32 43 22 78 63 57 91 13
43 32 22 78 63 57 91 13
Merge Sort
43 32 22 78 63 57 91 13
43 32 22 78 63 57 91 13
32 43 22 78 63 57 91 13
43 32 22 78 63 57 91 13
Merge Sort
43 32 22 78 63 57 91 13
43 32 22 78 63 57 91 13
32 43 22 78 57 63 91 13
43 32 22 78 63 57 91 13
Merge Sort
43 32 22 78 63 57 91 13
43 32 22 78 63 57 91 13
32 43 22 78 57 63 13 91
43 32 22 78 63 57 91 13
Merge Sort
43 32 22 78 63 57 91 13
22 32 43 78 63 57 91 13
32 43 22 78 57 63 13 91
43 32 22 78 63 57 91 13
Merge Sort
43 32 22 78 63 57 91 13
22 32 43 78 13 57 63 91
32 43 22 78 57 63 13 91
43 32 22 78 63 57 91 13
Merge Sort
13 22 32 43 57 63 78 91
22 32 43 78 13 57 63 91
32 43 22 78 57 63 13 91
43 32 22 78 63 57 91 13
Divide and conquer
Break up problem into disjoint parts
Solve each part separately
Combine the solutions efficiently
Merging sorted lists
Combine two sorted lists A and B into C
If A is empty, copy B into C
If B is empty, copy A into C
Otherwise, compare first element of A and B and
move the smaller of the two into C
Repeat until all elements in A and B have been
moved
Merging
def merge(A,B): # Merge A[0:m],B[0:n]
(C,m,n) = ([],len(A),len(B))
(i,j) = (0,0) # Current positions in A,B
while i+j < m+n: # i+j is number of elements merged so far
if i == m: # Case 1: A is empty
C.append(B[j])
j = j+1
elif j == n: # Case 2: B is empty
C.append(A[i])
i = i+1
elif A[i] <= B[j]: # Case 3: Head of A is smaller
C.append(A[i])
i = i+1
elif A[i] > B[j]: # Case 4: Head of B is smaller
C.append(B[j])
j = j+1
return(C)
Merging, wrong
def mergewrong(A,B): # Merge A[0:m],B[0:n]
(C,m,n) = ([],len(A),len(B))
(i,j) = (0,0) # Current positions in A,B
while i+j < m+n:
# i+j is number of elements merged so far
# Combine Case 1, Case 4
if i == m or A[i] > B[j]:
C.append(B[j])
j = j+1
# Combine Case 2, Case 3:
elif j == n or A[i] <= B[j}:
C.append(A[i])
i = i+1
return(C)
Merge Sort
To sort A[0:n] into B[0:n]
If n is 1, nothing to be done
Otherwise
Sort A[0:n//2] into L (left)
Sort A[n//2:n] into R (right)
Merge L and R into B
Merge Sort
def mergesort(A,left,right):
# Sort the slice A[left:right]
if right - left <= 1: # Base case
return(A[left:right])
if right - left > 1: # Recursive call
mid = (left+right)//2
L = mergesort(A,left,mid)
R = mergesort(A,mid,right)
return(merge(L,R))
NPTEL MOOC
PROGRAMMING,
DATA STRUCTURES AND
ALGORITHMS IN PYTHON
Week 4, Lecture 2
Madhavan Mukund, Chennai Mathematical Institute
http://www.cmi.ac.in/~madhavan
Merge sorted lists
Given two sorted lists A and B, combine into a
sorted list C
Compare first element of A and B
Move it into C
Repeat until all elements in A and B are over
Merging A and B
Analysis of Merge
How much time does Merge take?
Merge A of size m, B of size n into C
In each iteration, we add one element to C
Size of C is m+n
m+n ≤ 2 max(m,n)
Hence O(max(m,n)) = O(n) if m ≈ n
Merge Sort
To sort A[0:n] into B[0:n]
If n is 1, nothing to be done
Otherwise
Sort A[0:n//2] into L (left)
Sort A[n//2:n] into R (right)
Merge L and R into B
Analysis of Merge Sort …
T(n): time taken by Merge Sort on input of size n
Assume, for simplicity, that n = 2k
T(n) = 2T(n/2) + n
Two subproblems of size n/2
Merging solutions requires time O(n/2+n/2) = O(n)
Solve the recurrence by unwinding
Analysis of Merge Sort …
T(1) = 1
T(n) = 2T(n/2) + n
2 2
= 2 [ 2T(n/4) + n/2 ] + n = 2 T(n/2 ) + 2n
2 3 2 3 3
= 2 [ 2T(n/2 ) + n/2 ] + 2n = 2 T(n/2 ) + 3n
…
j j
= 2 T(n/2 ) + jn
j j
When j = log n, n/2 = 1, so T(n/2 ) = 1
log n means log2 n unless otherwise specified!
j j log n
T(n) = 2 T(n/2 ) + jn = 2 + (log n) n = n + n log n = O(n log n)
Variations on merge
Union of two sorted lists (discard duplicates)
While A[i] == B[j], increment j
Append A[i] to C and increment i
Intersection of two sorted lists
If A[i] < B[j], increment i
If B[j] < A[i], increment j
If A[i] == B[j]
While A[i] == B[j], increment j
Append A[i] to C and increment i
Exercise: List difference: elements in A but not in B
Merge Sort: Shortcomings
Merging A and B creates a new array C
No obvious way to efficiently merge in place
Extra storage can be costly
Inherently recursive
Recursive call and return are expensive
NPTEL MOOC
PROGRAMMING,
DATA STRUCTURES AND
ALGORITHMS IN PYTHON
Week 4, Lecture 3
Madhavan Mukund, Chennai Mathematical Institute
http://www.cmi.ac.in/~madhavan
Merge Sort: Shortcomings
Merging A and B creates a new array C
No obvious way to efficiently merge in place
Extra storage can be costly
Inherently recursive
Recursive call and return are expensive
Alternative approach
Extra space is required to merge
Merging happens because elements in left half
must move right and vice versa
Can we divide so that everything to the left is
smaller than everything to the right?
No need to merge!
Divide and conquer without merging
Suppose the median value in A is m
Move all values ≤ m to left half of A
Right half has values > m
This shifting can be done in place, in time O(n)
Recursively sort left and right halves
A is now sorted! No need to merge
T(n) = 2T(n/2) + n = O(n log n)
Divide and conquer without merging
How do we find the median?
Sort and pick up middle element
But our aim is to sort!
Instead, pick up some value in A — pivot
Split A with respect to this pivot element
Quicksort
Choose a pivot element
Typically the first value in the array
Partition A into lower and upper parts with respect
to pivot
Move pivot between lower and upper partition
Recursively sort the two partitions
Quicksort
High level view
43 32 22 78 63 57 91 13
Quicksort
High level view
43 32 22 78 63 57 91 13
Quicksort
High level view
43 32 22 78 63 57 91 13
Quicksort
High level view
13 32 22 43 63 57 91 78
Quicksort
High level view
13 22 32 43 57 63 78 91
Quicksort: Partitioning
43 32 22 78 63 57 91 13
Quicksort: Partitioning
43 32 22 78 63 57 91 13
Quicksort: Partitioning
43 32 22 78 63 57 91 13
Quicksort: Partitioning
43 32 22 78 63 57 91 13
Quicksort: Partitioning
43 32 22 78 63 57 91 13
Quicksort: Partitioning
43 32 22 78 63 57 91 13
Quicksort: Partitioning
43 32 22 78 63 57 91 13
Quicksort: Partitioning
43 32 22 78 63 57 91 13
Quicksort: Partitioning
43 32 22 78 63 57 91 13
Quicksort: Partitioning
43 32 22 78 63 57 91 13
Quicksort: Partitioning
43 32 22 13 63 57 91 78
Quicksort: Partitioning
13 32 22 43 63 57 91 78
Quicksort in Python
def Quicksort(A,l,r): # Sort A[l:r]
if r - l <= 1: # Base case
return ()
# Partition with respect to pivot, a[l]
yellow = l+1
for green in range(l+1,r):
if A[green] <= A[l]:
(A[yellow],A[green]) = (A[green],A[yellow])
yellow = yellow + 1
# Move pivot into place
(A[l],A[yellow-1]) = (A[yellow-1],A[l])
Quicksort(A,l,yellow-1) # Recursive calls
Quicksort(A,yellow,r)
NPTEL MOOC
PROGRAMMING,
DATA STRUCTURES AND
ALGORITHMS IN PYTHON
Week 4, Lecture 4
Madhavan Mukund, Chennai Mathematical Institute
http://www.cmi.ac.in/~madhavan
Quicksort
Choose a pivot element
Typically the first value in the array
Partition A into lower and upper parts with respect
to pivot
Move pivot between lower and upper partition
Recursively sort the two partitions
Quicksort in Python
def Quicksort(A,l,r): # Sort A[l:r]
if r - l <= 1: # Base case
return ()
# Partition with respect to pivot, a[l]
yellow = l+1
for green in range(l+1,r):
if A[green] <= A[l]:
(A[yellow],A[green]) = (A[green],A[yellow])
yellow = yellow + 1
# Move pivot into place
(A[l],A[yellow-1]) = (A[yellow-1],A[l])
Quicksort(A,l,yellow-1) # Recursive calls
Quicksort(A,yellow,r)
Analysis of Quicksort
Worst case
Pivot is either maximum or minimum
One partition is empty
Other has size n-1
T(n) = T(n-1) + n = T(n-2) + (n-1) + n
2
= … = 1 + 2 + … + n = O(n )
Already sorted array is worst case input!
Analysis of Quicksort
But …
Average case is O(n log n)
All permutations of n values, each equally likely
Average running time across all permutations
Sorting is a rare example where average case can
be computed
Quicksort: randomization
Worst case arises because of fixed choice of pivot
We chose the first element
For any fixed strategy (last element, midpoint), can
work backwards to construct O(n2) worst case
Instead, choose pivot randomly
Pick any index in range(0,n) with uniform probability
Expected running time is again O(n log n)
Quicksort in practice
In practice, Quicksort is very fast
Typically the default algorithm for in-built sort
functions
Spreadsheets
Built in sort function in programming
languages
Stable sorting
Sorting on multiple criteria
Assume students are listed in alphabetical order
Now sort students by marks
After sorting, are students with equal marks still
in alphabetical order?
Stability is crucial in applications like spreadsheets
Sorting column B should not disturb previous
sort on column A
Stable sorting …
Quicksort, as described, is not stable
Swap operation during partitioning disturbs
original order
Merge sort is stable if we merge carefully
Do not allow elements from right to overtake
elements from left
Favour left list when breaking ties
NPTEL MOOC
PROGRAMMING,
DATA STRUCTURES AND
ALGORITHMS IN PYTHON
Week 4, Lecture 5
Madhavan Mukund, Chennai Mathematical Institute
http://www.cmi.ac.in/~madhavan
Tuples
Simultaneous assignments
(age,name,primes) = (23,"Kamal",[2,3,5])
Can assign a “tuple” of values to a name
point = (3.5,4.8)
date = (16,7,2013)
Extract positions, slices
xcoordinate = point[0]
monthyear = date[1:]
Tuples are immutable
date[1] = 8 is an error
Generalizing lists
l = [13, 46, 0, 25, 72]
View l as a function, associating values to positions
l : {0,1,..,4} ⟶ integers
l(0) = 13, l(4) = 72
0,1,..,4 are keys
l[0],l[1],..,l[4] are corresponding values
Dictionaries
Allow keys other than range(0,n)
Key could be a string
test1["Dhawan"] = 84
test1["Pujara"] = 16
test1["Kohli"] = 200
Python dictionary
Any immutable value can be a key
Can update dictionaries in place —mutable, like lists
Dictionaries
Empty dictionary is {}, not []
Initialization: test1 = {}
Note: test1 = [] is empty list, test1 = () is
empty tuple
Keys can be any immutable values
int, float, bool, string, tuple
But not lists, or dictionaries
Dictionaries
Can nest dictionaries
score["Test1"]["Dhawan"] = 84
score["Test1"]["Kohli"] = 200
score["Test2"]["Dhawan"] = 27
Directly assign values to a dictionary
score = {"Dhawan":84, "Kohli":200}
score = {"Test1":{"Dhawan":84,
"Kohli":200}, "Test2":{"Dhawan":50}}
Operating on dictionaries
d.keys() returns sequence of keys of dictionary d
for k in d.keys():
# Process d[k]
d.keys() is not in any predictable order
for k in sorted(d.keys()):
# Process d[k]
sorted(l) returns sorted copy of l, l.sort()
sorts l in place
d.keys() is not a list —use list(d.keys())
Operating on dictionaries
Similarly, d.values() is sequence of values in d
total = 0
for s in test1.values():
total = total + test1
Test for key using in, like list membership
for n in ["Dhawan","Kohli"]:
total[n] = 0
for match in score.keys():
if n in score[match].keys():
total[n] = total[n] + score[match][n]
Dictionaries vs lists
Assigning to an unknown key inserts an entry
d = {}
d[0] = 7 # No problem, d == {0:7}
… unlike a list
l = []
l[0] = 7 # IndexError!
Summary
Dictionaries allow a flexible association of values to
keys
Keys must be immutable values
Structure of dictionary is internally optimized for key-
based lookup
Use sorted(d.keys()) to retrieve keys in
predictable order
Extremely useful for manipulating information from
text files, tables … — use column headings as keys
NPTEL MOOC
PROGRAMMING,
DATA STRUCTURES AND
ALGORITHMS IN PYTHON
Week 4, Lecture 6
Madhavan Mukund, Chennai Mathematical Institute
http://www.cmi.ac.in/~madhavan
Passing values to functions
Argument value is substituted for name
def power(x,n): power(3,5)
ans = 1 x = 3
for i in range(0,n): n = 5
ans = ans*x ans = 1
return(ans) for i in range..
Like an implicit assignment statement
Pass arguments by name
def power(x,n):
ans = 1
for i in range(0,n):
ans = ans*x
return(ans)
Call power(n=5,x=4)
Default arguments
Recall int(s) that converts string to integer
int("76") is 76
int("A5") generates an error
Actually int(s,b) takes two arguments, string s and
base b
b has default value 10
int("A5",16) is 165 (10 x 16 + 5)
Default arguments
def int(s,b=10):
. . .
Default value is provided in function definition
If parameter is omitted, default value is used
Default value must be available at definition time
def Quicksort(A,l=0,r=len(A)): does not
work
Default arguments
def f(a,b,c=14,d=22):
. . .
f(13,12) is interpreted as f(13,12,14,22)
f(13,12,16) is interpreted as f(13,12,16,22)
Default values are identified by position, must
come at the end
Order is important
Function definitions
def associates a function body with a name
Flexible, like other value assignments to name
Definition can be conditional
if condition:
def f(a,b,c):
. . .
else:
def f(a,b,c):
. . .
Function definitions
Can assign a function to a new name
def f(a,b,c):
. . .
g = f
Now g is another name for f
Can pass functions
Apply f to x n times
def apply(f,x,n): def square(x):
res = x return(x*x)
for i in range(n):
res = f(res) apply(square,5,2)
return(res)
square(square(5))
625
Passing functions
Useful for customizing functions such as sort
Define cmp(x,y) that returns -1 if x < y,
0 if x == y and 1 if x > y
cmp("aab","ab") is -1 in dictionary order
cmp("aab","ab”) is 1 if we compare by length
def sortfunction(l,cmpfn=defaultcmpfn):
Summary
Function definitions behave like other assignments
of values to names
Can reassign a new definition, define conditionally
…
Can pass function names to other functions
NPTEL MOOC
PROGRAMMING,
DATA STRUCTURES AND
ALGORITHMS IN PYTHON
Week 4, Lecture 7
Madhavan Mukund, Chennai Mathematical Institute
http://www.cmi.ac.in/~madhavan
Operating on lists
Update an entire list
for x in l:
x = f(x)
Define a function to do this in general
def applylist(f,l):
for x in l:
x = f(x)
Built in function map()
map(f,l) applies f to each element of l
Output of map(f,l) is not a list!
Use list(map(f,l)) to get a list
Can be used directly in a for loop
for i in map(f,l):
Like range(i,j), d.keys()
Selecting a sublist
Extract list of primes from list numberlist
primelist = []
for i in numberlist:
if isprime(i):
primelist.append(i)
return(primelist)
Selecting a sublist
In general
def select(property,l):
sublist = []
for x in l:
if property(x):
sublist.append(x)
return(sublist)
Note that property is a function that returns True or
False for each element
Built in function filter()
filter(p,l) checks p for each element of l
Output is sublist of values that satisfy p
Combining map and filter
Squares of even numbers from 0 to 99
list(map(square,filter(iseven,range(100))
def square(x):
return(x*x)
def iseven(x):
return(x%2 == 0)
List comprehension
Pythagorean triple: x2 + y2 = z2
All Pythagorean triples (x,y,z) with values below n
{ (x,y,z) | 1 ≤ x,y,z ≤ n, x2 + y2 = z2 }
In set theory, this is called set comprehension
Building a new set from existing sets
Extend to lists
List comprehension
Squares of even numbers below 100
[square(x) for i in range(100) if iseven(x)]
map generator filter
Multiple generators
Pythagorean triples with x,y,z below 100
[(x,y,z) for x in range(100)
for y in range(100)
for z in range(100)
if x*x + y*y == z*z]
Order of x,y,z is like nested for loop
for x in range(100):
for y in range(100):
for z in range(100):
Multiple generators
Later generators can depend on earlier ones
Pythagorean triples with x,y,z below 100, no
duplicates
[(x,y,z) for x in range(100)
for y in range(x,100)
for z in range(y,100)
if x*x + y*y == z*z]
Useful for initialising lists
Initialise a 4 x 3 matrix
4 rows, 3 columns
Stored row-wise
l = [ [ 0 for i in range(3) ]
for j in range(4)]
Warning
What’s happening here?
>>> zerolist = [ 0 for i in range(3) ]
>>> l = [ zerolist for j in range(4) ]
>>> l[1][1] = 7
>>> l
[[0,7,0],[0,7,0],[0,7,0],[0,7,0]]
Each row in l points to same list zerolist
Summary
map and filter are useful functions to manipulate
lists
List comprehension provides a useful notation for
combining map and filter
