ITEC 2620

Introduction to
Data Structures
:Algorithms
Khairul Bashar
School of Information Technology
York University, Toronto.
What is an
Algorithm?
• A well-defined step by step instructions to
solve an independent problem.

• The input and output must be defined

clearly
• The steps should be clear and
unambiguous.
• An algorithm should not include any
programming code
Algorithm vs Pseudocode vs Program

Algorithm Pseudocode Program

• Systematic • Instruction in • Exact code

logical smallest written that
approach. steps. solves the
• Written in • Written in problem.
human plain English • Must follow
language but uses the rule of a
short phrases. programming
language.
Example
• Problem: Write a program to add two numbers.

Algorithm Pseudocode Program

Step 1: Start Step1: input a, b public double addNumbers(double

Step 2: Read values a and b Step 2: sum <- a + b a, double b) {
Step 3: Add a and b and assign the result Step 3: print sum double sum = a + b;
to sum. return sum;
Step 4: Return the value of sum }
Step 5: End
How to write
good code

http://xkcd.com/844/
Data Structures and Algorithms
Data structures and algorithms are interrelated concepts in computer science.

Data Structures are the building block of algorithms

A good algorithm makes a data structure efficient

Searching and sorting are two critical method/operation for most of the data structures.

An efficient algorithm for searching and sorting is the key to performance

 • Why is searching important?
• Computers process data
• Process data -> store data -> find data
• Searching is a fundamental task

Searching • ITEC-1620 example:

• Write a static method that determines the range
(the difference between the largest and smallest
element) for a fully populated array of ints.
 take arr[]
maxValue <- arr[0]
minValue <- arr[0]

for i = 1 to arr.length
Solution if arr[i] > maxValue
maxValue = arr[i]
else if arr[i] < minValue
minValue = arr[i]

Return maxValue - minValue

Searching
• How is the above example similar to searching?
• Find the largest and smallest value in a set of
elements

• Searching -> find a given value in a set of elements

Practice Problem
• Write a program that determines the location (the array index) for a
given value in a fully populated array of ints. Return -1 if the value is
not in the array.
What to program?
• Is this how we always search?
• Find Prof. Edo’s phone number in the York phone book
• Aby -> No
• Abner -> No
• Abram -> No
• Etc.
What to program?
• Let’s play a game
• I have a number in my mind between 1 to 100. I will say higher or lower on
your each guess, if not equal to the number in my mind.

• What is the optimal strategy?

• Can you always guess my number in 7 guesses?
• What is this strategy?
• Binary Search
Binary Search
• Make your first guess in the middle of the range
• If not the target element, determine which sub-range you have to
search
• Make your next guess in the middle of this sub-range
• Repeat until you find the number
Binary Search
2 5 9 12 14 16 17 18 21 23 24

• Find K = 12
• lower = 0, upper = 11 -> middle = 5, arr[5] == 16
• lower = 0, upper = 5 -> middle = 2, arr[2] == 9
• lower = 2, upper = 5 -> middle = 3, ar[3] == 12

• Return 3
Binary Search
take K and arr[]
lower <- 0
Upper <- arr.length
while lower < upper
middle <- (lower + upper)/2
if K == arr[middle]
return middle
else if K < arr[middle]
upper = middle
else // K > arr[middle]
lower = middle
return -1
Analysis
• What’s the best case for this search?
• Best case is when the number is the first element
• Linear Search -> 1 compare
• Binary Search -> 1 compare

• What’s the worst case for this search?

• Worst case is when the number is the last element
• Linear search -> n compares
• Binary Search -> logn compares
Analysis – Average case
• What’s the average case for search?
• Average case when the number is middle/weighted average element
• Linear Search -> n/2 compares
• Binary Search -> logn compares
Summary
• What’s the difference?
• Toronto Phone book -> 2 million names
• Average linear -> 1 million compares
• Average Binary -> 20 compares
• For 2 million records, binary search is 50,000 times faster on average case.

• What’s the cost?

• To do binary search, we need a sorted array
• Sorting
Searching in Real records
• A query for the existence and location of an element in a set of
distinct elements.
• Elements have searchable keys associated with a stored record
• To find a record – e.g. phone number
• Search on keys – e.g. name
• All keys must have a same data type

• Two goals of searching

• Existence -> does the person exists?
• Location -> Where are they in the phonebook?
Introduction to Sorting
• Why is sorting important?
• Easier to search in sorted data sets
• Searching and sorting are primary problems of computer science
• Sorting in General
• Arrange a set of elements by their “keys” in increasing/decreasing order
Sorting • A full deck
• Find the smallest value: A spades
• Find the next smallest value: 2 spades
• Sort Cards • Find the next smallest value: 3 spades
• A suite of a full deck • Repeat…
• Selection sort
• Select the next smallest value each time
Selection Sort
10 5 9 2 14 16 17 18 21 23 24

2 10 • Find the smallest value and move it into position

• What to do with the value that was previously there?
• Swap it to where the smallest value was
• Sorting can work on the original array
Selection sort: Algorithm
n <- arr.length
For i=0 to n-2
minIdx <- i
for j=i+1 to n-1
if arr[minIdx] > arr[j]
minIdx = j;
temp <- arr[minIdx]
arr[minIdx] <- arr[i]
arr[i] <- temp
Sorting
• Sort Cards • A set of random cards dealt to you
• Start with selecting the second index
• A set of random cards dealt • Compare values between selected and indices to
to you the left of the selected index
• Any smaller value moves one index right
• Insert the selected value in the index where no
value to the left smaller than the selected value
• Repeat
• Insertion sort
• Insert new value each time
Insertion sort
10 5 9 2 14 16 17 18 21 23 24

5 10 9 2 14 16 17 18 21 23 24

5 9 10 2 14 16 17 18 21 23 24

2 5 9 10 14 16 17 18 21 23 24

2 5 9 10 14 16 17 18 21 23 24

Elements in insertion sort can still move.

Insertion sort: Pseudocode
n <- arr.length
for i=1 to n-1
key <- arr[i]
j <- i -1

while j >=0 and arr[j] > key

arr[j+1] = arr[j]
j <- j-1
arr[j+1] = key
Cost of Sorting
• Selection sort: is there any best, worst and average case?
• n elements in the outer loop
• Average n/2 elements in the inner loop for each iteration of outer
loop
• n*n/2 compares
Cost of Sorting
• Insertion Sort
• Worst: next element swaps till end
• n*n/2 compares
• Best: Elements are already sorted
• n*1 compares
• Average: 50% values are sorted
• n*n/4 compares
Value of Sorting
• Current cost of sorting is roughly n2 compares
• Toronto phone book
• 2 million records
• 4 trillion compares to sort
Value of Sorting
• 2 million records • 100 searches
• Linear search • unsorted array, linear search
• 1 million compares • 100 million total compares
• Binary search • sorted array, binary search
• 20 compares • 4 trillion and 2 thousand total
compares
• sorting the array first is 4000 times
slower
That’s all for Today!
Thank you