© 2019 Michel Pasquier - all rights reserved CMP 305 Data Structures and Algorithms 01a Programming - Introduction 2

Data Structures and Algorithms

Chapter 1
Introduction
Sections 1.1, 1.2, 1.3

© 2019 Michel Pasquier - all rights reserved CMP 305 Data Structures and Algorithms 01a Programming - Introduction 3 © 2019 Michel Pasquier - all rights reserved CMP 305 Data Structures and Algorithms 01a Programming - Introduction 4

Overview Example: Selection Problem

What is this course about? Find the kth-largest (or smallest) number in a set of size N
Examples: find
Why the choice of algorithms is critical - the 1000 best customers (who spend the most) in a database
- the most active users (who post the most) in a social network
when dealing with large inputs
How would you solve this?
Basic mathematical background ... ... ...
Algorithm 1:
Detailed introduction to recursion Sort the N numbers and Pick the kth one
Intuitive and easy to implement, but
Review of basic C++ concepts Sorting requires many comparisons
Introduction to modern C++ features Much work is done comparing elements that have no chance
of being in position k
To sort N elements need N log2(N) comparisons/swaps
Unsorted database with 10,000,000 elements and 1,000
swaps/sec will result in 2.7 days to finish the task !
© 2019 Michel Pasquier - all rights reserved CMP 305 Data Structures and Algorithms 01a Programming - Introduction 5 © 2019 Michel Pasquier - all rights reserved CMP 305 Data Structures and Algorithms 01a Programming - Introduction 6

Another strategy Example: Word Puzzle

Algorithm 2 (better): Find all words, given a 2D Algorithm 2:
Sort only the first k elements in the array array and a word list. For each combination of
Insert elements (k+1) to N, one by one, discarding the letters in the 2D array
smallest element each time Check if the word is in
the word-list
Then pick the 808 ... ... ... 421
kth element What if the word list is the
In the order of N comparisons if k is small entire dictionary?
Now it takes “only” 2.8 hours to finish the task (23 times faster) (similar to the game of Boggle)
What if the puzzle / array
because of fewer comparisons/swaps: k log2(k) << N
Algorithm 1: is larger? e.g. 16x16
But: what if N = 10,000,000 and k = 5,000,000 ?
For each word in the list
Little or no improvement if k is large Algorithm analysis
Find if it is in the 2D array
A better algorithm can solve this in a second! Which data structure?
cf. chapters 6-7 (also 10) – using ad hoc data structures (heap…)

© 2019 Michel Pasquier - all rights reserved CMP 305 Data Structures and Algorithms 01a Programming - Introduction 7 © 2019 Michel Pasquier - all rights reserved CMP 305 Data Structures and Algorithms 01a Programming - Introduction 8

Example: N-Queens Problem

Place N queens on a NxN board
so that no queen can capture
another e.g. 8-queens solution

Many approaches
Brute force search: can solve up to N=10
Using a smart formulation: up to N=30
Ad hoc algorithm (CSP): up to N=100
Good heuristic algorithm: up to N=1000
Best algorithm: up to N=1,000,000 !

Data structure selection + Algorithm analysis

© 2019 Michel Pasquier - all rights reserved CMP 305 Data Structures and Algorithms 01a Programming - Introduction 9 © 2019 Michel Pasquier - all rights reserved CMP 305 Data Structures and Algorithms 01a Programming - Introduction 10

Proof by Induction Proof by Induction - example

Given a theorem Fibonacci Series
First prove a base case F0 = 1, F1 = 1, Fi = Fi-1 + Fi-2 , for i > 1
Show the theorem is true for some small degenerate
values Show that
Next assume an inductive hypothesis Fi < (5/3) i , for i ≥ 1
Assume the theorem is true for all cases up to some
limit k Base case: F0 = 1 = F1 < 5/3
Then prove that the theorem holds for the next Inductive Hypothesis
value (k+1) Fi < (5/3) i , for i = 1, 2, ..., k

© 2019 Michel Pasquier - all rights reserved CMP 305 Data Structures and Algorithms 01a Programming - Introduction 11 © 2019 Michel Pasquier - all rights reserved CMP 305 Data Structures and Algorithms 01a Programming - Introduction 12

Proof by Induction - example (contd.) Proof by Induction - Exercise

Now prove that Fk+1 < (5/3)k+1 N (N+1)

∑i=
From definition of Fibonacci Sequence 1. i=0..N
2
Fk+1 < Fk + Fk-1
Using inductive Hypothesis
∑ i 2 = N (N+1)(2N+1)
Fk+1 < (5/3) k + (5/3) k-1 6
2. i=0..N
< (5/3) k + (5/3) k-1
< (5/3) k+1 [ 3/5 + (3/5)2]
< (5/3) k+1 [24/25]
< (5/3) k+1
© 2019 Michel Pasquier - all rights reserved CMP 305 Data Structures and Algorithms 01a Programming - Introduction 13

Other types of proofs

Proof by counter-example
The statement Fk < k2 in the Fibonacci series is false
Proof: F11 = 144 > 112
Proof by Contradiction
Initially assume that the theorem is false
Then show that some known property would be false as well
Example: “There are an infinite number of prime numbers”
Proof: Assume the theorem is false
Let P1, P2, ..., Pk be all the primes in increasing order
N = P1•P2 • ⋅⋅⋅ •Pk + 1 is > Pk and so it is not a prime.
But it is also not divisible by any of the listed primes,
contradicting the factorization of integers into primes.
This is a counter-example