Data Structures and Algorithms

Lecture 1

1
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Course objectives:
 Assess how the choice of data structures and
algorithm design methods impacts the performance
of programs.
 Choose the appropriate data structure and
algorithm design method for a specified application.
 Solve problems using data structures such as linear
lists, stacks, queues, hash tables,….
 Solve problems using algorithm design methods
such as the greedy method, divide and and
conquer.

2
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Agenda
 Administrative
 Course objective and outline

3
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Introduction
 What is Algorithm?
 a clearly specified set of simple instructions to be followed
to solve a problem
 Takes a set of values, as input and

 produces a value, or set of values, as output

 May be specified
 In English

 As a computer program

 As a pseudo-code

 Data structures
 Methods of organizing data
 Program = algorithms + data structures
4
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Objectives
 Analysis of algorithms
 The theoretical study of the computer-program
performance and resource usage.
 Design of algorithms
 data structures techniques

5
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Expectation
 Assumed Knowledge
 Some experience in Java or other similar OO language

 BEFORE each lecture

 Download and printout the lecture slides
 Read the related chapter (s) and lectures notes
 Make sure you keep up with the progress
 Consult course staff EARLY enough for difficulties and
problems!

6
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Our Textbook
 Anany Levitin, Introduction to
The Design & Analysis of
Algorithms, 2rd edition.

7
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Rule of Conducts
 Okay to discuss ideas and problem approaches
 All work must be your own creation
 Turn off your mobile phone during the class
 Be on time

8
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Questions?

9
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
What is an algorithm?
An algorithm is a sequence of unambiguous instructions for
solving a problem, i.e., for obtaining a required output for
any legitimate input in a finite amount of time.

problem

algorithm

input “computer” output

10
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Historical Perspective

 Muhammad ibn Musa al-Khwarizmi – 9th

century mathematician
www.lib.virginia.edu/science/parshall/khwariz.
html

11
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
 Example of computational problem:
sorting
 Statement of problem:
 Input: A sequence of n numbers <a1, a2, …, an>

 Output: A reordering of the input sequence <a´1, a´2, …, a

n> so that a i ≤ a j whenever i < j
´ ´ ´

 Instance: The sequence <5, 3, 2, 8, 3>

 Algorithms:
 Selection sort
 Insertion sort
 Merge sort
 (many others)

12
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Selection Sort
 Input: array a[1],…,a[n]

 Output: array a sorted in non-decreasing

order

 Algorithm:
 for i=1 to n
swap a[i] with smallest of a[i],…a[n]

13
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Analysis of Algorithms
 How good is the algorithm?
 Correctness
 Time efficiency
 Space efficiency

 Does there exist a better algorithm?

 Lower bounds
 Optimality

14
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
What is an algorithm?
 Recipe, process, method, technique, procedure, routine,… with
following requirements:
1. Finiteness
 terminates after a finite number of steps
2. Definiteness
 rigorously and unambiguously specified
3. Input
 valid inputs are clearly specified
4. Output
 can be proved to produce the correct output given a
valid input
 Effectiveness
 steps are sufficiently simple and basic

15
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Why study algorithms?
 Theoretical importance

 the core of computer science

 Practical importance

 A practitioner’s toolkit of known algorithms

 Framework for designing and analyzing algorithms

for new problems
16
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Analysis of algorithms
 How good is the algorithm?
 timeefficiency
 space efficiency

 Does there exist a better algorithm?

 lowerbounds
 optimality

17
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
 Important problem types
 sorting

 searching

 string processing

 graph problems

 combinatorial problems

 geometric problems

 numerical problems
18
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Fundamental data structures
 list  graph
 array  tree
 linked list  set and dictionary
 string

 stack

 queue

 priority queue
19
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Questions?

20
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Chapter

Fundamentals of the Analysis of

Algorithm Efficiency (1)

21
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Introduction
 What is Algorithm?
 a clearly specified set of simple instructions to be followed
to solve a problem
 Takes a set of values, as input and

 produces a value, or set of values, as output

 May be specified
 In English

 As a computer program

 As a pseudo-code

 Data structures
 Methods of organizing data
 Program = algorithms + data structures

A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Introduction
 Why need algorithm analysis ?
 writing a working program is not good enough
 The program may be inefficient!
 If the program is run on a large data set, then
the running time becomes an issue

A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Algorithm Analysis…
 Factors affecting the running time
 computer
 compiler
 algorithm used
 input to the algorithm
 The content of the input affects the running time

 typically, the input size (number of items in the input) is the

main consideration
 E.g. sorting problem  the number of items to be sorted
 E.g. multiply two matrices together  the total number of
elements in the two matrices
 Machine model assumed
 Instructions are executed one after another, with no
concurrent operations  Not parallel computers

A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Example
 Calculate N

i
i 1
3

1
1
2 2N+2
3 4N
4 1

 Lines 1 and 4 count for one unit each

 Line 3: executed N times, each time four units
 Line 2: (For… 1 for initialization, N+1 for all the tests, N
for all the increments) total 2N + 2
 total cost: 6N + 4

A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Example: Selection Problem
 Given a list of N numbers, determine the kth
largest, where k  N.
 Algorithm 1:
(1)   Read N numbers into an array
(2)   Sort the array in decreasing order by some
simple algorithm
(3)   Return the element in position k

A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Example: Selection Problem…
 Algorithm 2:
(1)   Read the first k elements into an array and
sort them in decreasing order
(2)   Each remaining element is read one by one
 If smaller than the kth element, then it is ignored
 Otherwise, it is placed in its correct spot in the
array, bumping one element out of the array.
(3)   The element in the kth position is returned
as the answer.

A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Example: Selection Problem…
 Which algorithm is better when
 N =100 and k = 100?
 N =100 and k = 1?
 What happens when N = 1,000,000 and k =
500,000?
 There exist better algorithms

A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
 Theoretical analysis of time
efficiency
Time efficiency is analyzed by determining the
number of repetitions of the basic operation as a
function of input size

Basic operation: the operation that contributes

most towards the running time of the algorithm
input size

running time

T(n) ≈ copC(n) Number of times

basic operation is
execution time executed 29

for basic operation A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Best-case, average-case, worst-case
For some algorithms efficiency depends on form of input:

 Worst case: Cworst(n) – maximum over inputs of size n

 Best case: Cbest(n) – minimum over inputs of size n

 Average case: Cavg(n) – “average” over inputs of size n

 Number of times the basic operation will be executed on typical input
 NOT the average of worst and best case
 Expected number of basic operations considered as a random
variable under some assumption about the probability distribution of al
possible inputs

30
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
 Input size and basic operation
examples
Problem Input size measure Basic operation

Searching for key in a Number of list’s items,

Key comparison
list of n items i.e. n

Multiplication of two Matrix dimensions or Multiplication of two

matrices total number of elements numbers

Checking primality of n’size = number of digits

Division
a given integer n (in binary representation)

Visiting a vertex or
Typical graph problem #vertices and/or edges
traversing an edge31
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
 Example: Sequential search

 Worst case

 Best case

 Average case 32
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
 Example: Sequential search (Count.)

 Worst case

The last number  n

33
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
 Example: Sequential search (Count.)

 Worst case

The last number  n

34
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Binary Search
 Used with a sorted list
 First check the middle list element
 If the target matches the middle element, we
are done
 If the target is less than the middle element,
the key must be in the first half
 If the target is larger than the middle element,
the key must be in the second half

A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Binary Search Example

A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Binary Search Algorithm
start = 1
end = N
while start ≤ end do
middle = (start + end) / 2
switch (Compare(list[middle], target))
case -1: start = middle + 1
break
case 0: return middle
break
case 1: end = middle – 1
break
end select
end while
return 0

A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Algorithm Review
 Each comparison eliminates about half of the
elements of the list from consideration
 If we begin with N = 2k – 1 elements in the list,
there will be 2k–1 – 1 elements on the second
pass, and 2k–2 – 1 elements on the third pass

A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Worst-Case Analysis
 In the worst case, we will either find the target
on the last pass, or not find the target at all
 The last pass will have only one element left
to compare, which happens when
21-1 = 1
 If N = 2k – 1, then there must be
k = log(N+1) passes

A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Questions?

A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1