Algorithms & Data Structures

Lecture 01
Algorithms, Correctness, and E ciency
Lukas Holik
[email protected]
(based on slides of Giovanni Bacci, thanks!)

1
ffi
 Outline

• Algorithms & pseudocode

• E ciency & order of growth

• Correctness of an algorithm

• Case Study: nd an element in a sequence of numbers

2
ffi
fi
Intended Learning Goals
KNOWLEDGE
• Mathematical reasoning on concepts such as recursion, induction,
concrete and abstract computational complexity
• Data structures, algorithm principles e.g., search trees, hash tables,
dynamic programming, divide-and-conquer
• Graphs and graph algorithms e.g., graph exploration, shortest path,
strongly connected components.
SKILLS
• Determine abstract complexity for speci c algorithms
• Perform complexity and correctness analysis for simple algorithms
• Select and apply appropriate algorithms for standard tasks
COMPETENCES
• Ability to face a non-standard programming assignment
• Develop algorithms and data structures for solving speci c tasks
• Analyse developed algorithms
3
fi
fi
 Problem Solving Procedure
Covered in this course
• Analysis Real-life problem
• Speci cation
• Correctness Computational
• E ciency Problem
• Design
• Computational Thinking
• Algorithm patterns Algorithms
• Data Structures
NOT about Coding
Blueprint
• Speci c programming languages
• Computer architectures
• Software architectures
t Program Source
• Software design and developmen Code
principles
• Software testing and veri cation 4
ffi
fi
fi
fi
 What are algorithms?
Input Output

• Informally, an algorithm is any well-de ned computational

procedure that takes some values, as input and produces
some values, as output

• An algorithm is a sequence of computational steps that

transforms the input into the output.

• Is a tool for solving a well-speci ed computational

problem. The statement of the problem speci es the
desired I/O relationship.
5
fi
fi
fi
Computational Problems
The sorting problem:
Input: A sequence of n numbers ⟨a1, a2, …, an⟩.
Output: A permutation (reordering) ⟨aπ(1), aπ(2), …, aπ(n)⟩ of the
input such that aπ(1) ≤ aπ(2) ≤ ⋯ ≤ aπ(n).

The element search problem:

Input: A sequence of n numbers ⟨a1, a2, …, an⟩ and a number a
Output: An index 1 ≤ i ≤ n such that a = ai if there exists such
an index, 0 otherwise.

Characteristics of a computational problem:

• The the format of the input and the output are speci ed
• The desired input/output relationship is not ambiguous
6
fi
 Computational Problems
The sorting problem:
Input: A sequence of n numbers ⟨a1, a2, …, an⟩.
Output: A permutation (reordering) ⟨aπ(1), aπ(2), …, aπ(n)⟩ of the
input such that aπ(1) ≤ aπ(2) ≤ ⋯ ≤ aπ(n).

Example:
• An input sequence ⟨32,5,8,9,20⟩ is a valid instance of
the sorting problem.
• Its associated output sequence is ⟨5,8,9,20,32⟩.
De nition: An instance of a problem consists of the input
(satisfying whatever constraints are imposed in the problem
statement) needed to compute a solution to the problem.
7
fi
 Quiz
The element search problem:
Input: A sequence of n numbers ⟨a1, a2, …, an⟩ and a number a
Output: An index 1 ≤ i ≤ n such that a = ai if there exists such
an index, 0 otherwise.

• Which of the following are valid instances of the element search

problem?
A. ⟨1,2,4,20,0, − 5⟩, 2
B. ⟨1,2,4,20,f, − 5⟩, f
C. ⟨1,7,30,5⟩
D. ⟨⟩, 4

8
 Quiz
The element search problem:
Input: A sequence of n numbers ⟨a1, a2, …, an⟩ and a number a
Output: An index 1 ≤ i ≤ n such that a = ai if there exists such
an index, 0 otherwise.

• Which of the following are correct I/O pairs for the ‘element
search problem’?
A. ⟨1,2,4,20,0, − 5⟩, 2 ↦0
B. ⟨1,2,4,20,2, − 5⟩, 2 ↦2
C. ⟨1,2,4,20,2, − 5⟩, 2 ↦5
D. ⟨1,7,30,5⟩, 20 ↦0
E. ⟨⟩, 4 ↦0

• How many I/O pairs do we need to test to establish that an

algorithms solves the ‘element search problem’?
9
 From Problems to Algorithms
• A problem can be solved in several di erent ways, therefore
there are many correct algorithms solving the same problem
• Which algorithm is best for a given application depends on —
among other factors —
• the size of the input,
• the actual con guration of the input,
• the data structures used,
• the computer architecture, and
• the algorithmic design
De nition: An algorithm is said to be correct if, for every input
instance, it halts with the correct output. We say that a correct
algorithm solves the given computational problem.

An incorrect algorithm might not halt on some input instances, or it

might halt with an incorrect answer.
10
fi
fi
ff
 Describing Algorithms
• We will typically describe algorithms as programs written
in a pseudocode (similar in many respects to C, C++,
Java, Python, or Pascal)

• In pseudocode we employ whatever expressive method is

most clear and concise (sometimes just English)

• We are NOT concerned with issues of software

engineering such as
• Data abstraction
• Modularity
• Error handling
11
 Finding an element
Here is presented the pseudocode for solving the element search
problem as a function called Find-Element, which takes as a
parameter an array A[1..n] representing a sequence of numbers,
and a number a representing the number to nd.

Find-Element(A, a)
Sketched Idea: scan the array 1 j=0
A[1..n] element by element and 2 for i = 1 to A . length
compare the current element
A[i] with a. If the two are equal, 3 if A[i] = a
store the current index. 4 j=i
5 return j

12
fi
 Finding an element
There are many alternative approaches for solving the same
problem. The following pseudocode describes one possible
alternative algorithm for solving the element search problem.

Sketched Idea: scan the array Find-ElementV2(A, a)

A[1..n] from the last element 1 i = A . length
and compare the current
2 while i > 0 and A[i] ≠a
element A[i] with a.
If the two are equal, terminate 3 i=i−1
returning the index. 4 return i

13
 Quiz
• Consider the pseudocode aside. Find-Element(A, a)
What will be the value of the
1 j=0
variables i and j after execution
on the following instances? 2 for i = 1 to A . length
A. ⟨1,2,4,20,0, − 5⟩, 2 3 if A[i] = a
B. ⟨1,2,4,2,2,3⟩, 2 4 j=i
C. ⟨1,7,30,5⟩, 3 5 return j
D. ⟨⟩, 4

• Does Find-Element solve the ‘element search’ problem? How do

you justify your answer?
• How does the value of i at the end of the execution relate with the
length of the given array A?

14
 Ef ciency

• Computers may be fast, but they are not in nitely fast.

And memory may be inexpensive, but it is not for free.

• When designing an algorithm one should use these

resources wisely, and algorithms that are e cient in
terms of time and space will help you do so.

• Di erent algorithms devised to solve the same problem

often di er dramatically in their e ciency

15
ff
fi
ff
ffi
ffi
fi
 Ef ciency Analysis
• Analysing the e ciency of algorithms deals with
predicting the resources that the algorithm requires.

• Most often is computational time that we want to

measure

• The time taken by an algorithm depends on the actual

input (e.g., di erent array size, or di erent array
con gurations)

• It is traditional to describe the running time of an

algorithm as a function of the size of its input.

16
fi
fi
ff
ffi
ff
 Input Size
The best notion of input size depends on the problem being
solved

• A natural notion is the number of items in the input (e.g.,

array size n for sorting or element search)

• For many other problems it is more appropriate to use the

total number of bits needed to represent the input (e.g.,
for multiplying two integers)

• Sometimes, it’s more appropriate to use two or more

numbers rather than one (e.g., the size of a graph is
described by the number of its vertices and edges)

17
 Running Time
• The running time of an algorithm on a particular input is
the number of primitive operations or ‘steps’ executed

• We will use a notion of ‘step’ so that it is as machine-

independent as possible

• The book uses as computational model a generic one-

processor, random-access machine (RAM) (page 23-24)

• For now, we assume a constant amount of time for

each line of pseudocode
• Di erent lines may take di erent time, yet constant
18
ff
ff
 Example
The running time of Find-Element on the instance (A[1..n], a) is the sum
of the running times for each statement executed.

Find-Element(A, a) cost times

1 j=0 c1 1 One more time
2 for i = 1 to A . length c2 n+1 than the loop
body!
3 if A[i] = a c3 n
4 j=i c4 na
5 return j c5 1

Where n is the number of elements of A, and na is the number of

occurrences of a in A.

T(n) = c1 + c2(n + 1) + c3n + nac4 + c5

19
 Example
Consider the exact running time of Find-Element

T(n) = c1 + c2(n + 1) + c3n + nac4 + c5

Even for input of a the same size, the running time may depend on which
speci c array of that size is given. linear func
tions of n.
Of the form
an + b for so
me
• The best case occurs if the number of occurrences ofstaantiss aminimal,
c o n and b that
is, when na = 0

T(n) = (c2 + c3)n + c1 + c2 + c5

• The worst case occurs if the number of occurrences of a is maximal,
that is na = n

T(n) = (c2 + c3 + c4)n + c1 + c2 + c5

20
fi
 Quiz
Describe the running time of Find-ElementV2 on a generic instance
(A[1..n], a) as a function T(n) where n is the number of elements of A.

Find-ElementV2(A, a) cost times

1 i = A . length c1 1
2 while i > 0 and A[i] ≠a c2 n−r+1
3 i=i−1 c3 n−r
4 return i c4 1

Remark: as done before with na, you can use additional variables to help
you describe the exact expression for T(n).

21
 Answer
Describe the running time of Find-ElementV2 on a generic instance
(A[1..n], a) as a function T(n) where n is the number of elements of A.

Find-ElementV2(A, a) cost times

1 i = A . length c1 1
2 while i > 0 and A[i] ≠a c2 n−r+1
3 i=i−1 c3 n−r
4 return i c4 1

Where the variable r ∈ {0,…, n} denotes the index returned by the

algorithm on the given input instance.

T(n) = c1 + c2(n − r + 1) + c3(n − r) + c4

= (c2 + c3)(n − r) + (c1 + c4 + c2)
22
 Worst-case analysis

• We will mainly, concentrate on analysing the worst-case

running time

• It is the longest running time for any input of size n

• Knowing it provides a guarantee that the algorithm will

never take longer than that to return the output.

• For some algorithms the worst case occurs fairly often

23
 Order of growth
• We already used some abstraction to describe T(n)
1. We ignored actual cost of statements by using generic
constants ci
2. Then, we expressed the worst-case running time as
an + b for some a, b > 0 (that depend on ci’s)
• We can abstract even further by only looking at the rate of
growth (or order of growth) of T(n)
• We consider only the leading term (i.e., an) and we also
ignore its constant coe cient for large values of n
• Find-Element has worst case running time of Θ(n)

24
ffi
 Quiz
Comparison of running times. For each function f(n) and time t in
the following table, determine the largest size n of an instance that can
be solved in time t, assuming the algorithms for solving the problem
takes f(n) microsecond (1 microsecond = 10−6 sec)

1 sec 1 min 1 hr
log n
n
2
n
n
2
25
 Learned Today
• Formal concepts of
• algorithm
• computational problem
• problem instance
• correctness of an algorithm

• Using pseudocode to describe algorithms

• Some tools for reasoning about algorithm e ciency

• Input size
• ‘Exact’ running time
• Worst case running time
• Rate of growth
26
ffi