Computational Thinking

Computational Thinking

Expressing and
Analyzing
Algorithms:
Part 2 Komarudin
Computational Thinking

Outline
Part 1:
✔ Finding extreme values
✔ Linear search
✔ Algorithm complexity

Part 2:
✔ More on Algorithm
Complexity
✔ Binary search
✔ Brute force algorithm
✔ Greedy algorithm
Computational Thinking

Characteristics of a good algorithm

Correctness Generality Finiteness

Modularity Maintainability User-friendliness

Extensibility
Computational Thinking

Correct and Efficient Algorithm

• An algorithm is correct (effective) only if it solves the problem
• Finite
• how accurate producing the result without error
• An algorithm is efficient in using resources
• Computation time
• Memory space
• Computers are very fast nowadays, why bother?
• scalability
• monetization
• computing cost
Computational Thinking

Analysis of Algorithm
• The process of evaluating the
performance of an algorithm
• time complexity: the time required
to execute an algorithm
• space complexity: the total
amount of memory space used by
an algorithm/program, including
the space of input values for
execution
Computational Thinking

Analysis of Algorithm
• The process of evaluating the
performance of an algorithm
Our • time complexity: the time required
Focus to execute an algorithm
• space complexity: the total
amount of memory space used by
an algorithm/program, including
the space of input values for
execution
Computational Thinking

Measuring the running time of algorithms

• Objective - analyze algorithms independently of
specific implementations, hardware, or data
• Observation - An algorithm’s execution time is
related to the number of operations it requires
• Solution: count the number of steps, i.e. constant
time, operations the algorithm will perform for an
input of given size
Computational Thinking

How many steps?

PUZZLE(x)
while x != 1
if x is even then
x = x / 2
else
x = 3x + 1

Sample run: 7, 22, 11, 34, 17, 52, 26, 13, 40, 20, 10, 5, 16, 8, 4, 2, 1
Computational Thinking

•
Time complexity
•
Computational Thinking

Growth rates
•

1 2 3 4 5 6 7 8

.5 2 4.5 8 12.5 18 24.5 32

15 20 25 30 35 40 45 50

50 100 1,000 10,000 100,000

1250 5,000 500,000 50,000,000 5,000,000,000

260 510 5,010 50,010 500,010

Computational Thinking

Asymptotic growth rates

•

Algorithm A
Algorithm B
Computational Thinking

Asymptotic Notation
• Running time of an algorithm as a function of input size n for
large n.
• Expressed using only the highest-order term in the expression
for the exact running time.

• Describes behavior of function in the limit.

• Written using Asymptotic Notation.

12
Computational Thinking

Worst and average case analysis

• Worst case
• just how bad can it get: the maximum number of steps
• our focus in this course
• Average case
• number of steps expected “usually”
• In this course we will hand wave when it comes to average case
• Best case
• The smallest number of steps, usually need luckiness

• Example: searching for an item in an unsorted array

Computational Thinking

Asymptotic Notation
• Threre are 5 asymptotic notations:
• Θ (theta)  average case
• O (Big Oh) worst case
• Ω (Big Omega)  best case
• o (little oh)  loose upper bound bound
• ω (little omega)  loose lower bound

• Always prepare for the worst! The worst case is often happened!
That’s why we will focus on Big Oh
 Computational Thinking

•
Big Oh
Computational Thinking

Simplify the time complexity

• We ignore small values of n because we’re interested in the
behavior for large problem sizes.
• We ignore the multiplier because it’s hard to gauge, and is
implementation dependent.
• Instead of the exact running time T(n) = 5n2+2n+4, say only O(n2).
• Instead of the exact running time T(n) = 5n + 10000, say only
O(n).
Computational Thinking
Computational Thinking

Examples
Complexity Example & notes
Constant • Any integer/double arithmetic/logic operation
• Accessing a variable or an element in an array
Logarithmic

Linear

Linearithmic • Merge sort

Quadratic • looping over an array
• comparing the current element with all other elements in the
array
Cubic • Matrix multiplication
• Inverse matrix
Exponential • Stupid Fibonacci algorithm
Factorial • brute force solution for the traveling salesman problem
Computational Thinking

Quadratic complexity
Computational Thinking

Flowchart for maxima finding

start

max := a1

i := 2

no
max < ai?

yes

max := ai i: = i + 1

no
i = n?

yes

end
Computational Thinking

Time complexity of algorithms

Counts the largest number of basic operations required to execute an algorithm.

Example: Maxima finding

procedure max (a1, a2, …, an: integers)
max := a1 1 operation
for i :=2 to n 1 operation i:=2
if max < a1 then max := ai {n-1 times}
{2 ops + 1 op to check if i > n + 1 op to increment i}
return max {the largest element}

The total number of operations is 4(n-1)+2 = 4n-2

Computational Thinking

Matrix multiplication
Input:
Output:
Computational Thinking

Standard algorithm
for i ← 1 to n
do for j ← 1 to n
do cij ← 0
for k ← 1 to n
do cij ← cij + aik ⋅bkj

Running time complexity = Θ(n3)

Computational Thinking

Algorithm design paradigms

1. Brute-force algorithm
An algorithmic paradigm is a general
2. Recursive algorithm approach based on a particular
3. Greedy algorithm concept for constructing algorithms to
solve a variety of problems.
4. Divide and conquer
5. Dynamic programming
6. Etc.
Computational Thinking

Algorithm design paradigms

1. Brute-force algorithm • Straightforward, without any smart strategy
• Relying on loop/iteration
• Every possible solutions/combinations must be
checked
• Carefully not missed anything
• Suitable for small problem

Examples:
• exploring all the paths to a nearby market

What is the probability of rolling two • Arranging the books in a rack using all the
dice to have a sum of 7? possibilities
Computational Thinking

Algorithm design paradigms

2. Recursive algorithm • calling a copy of itself and solving smaller
subproblems of the original problems.
• every time the function calls itself must be
with a simpler version of the original problem.
• can reduce the length of our code
• Sometimes make it easier to read and write,
sometimes more difficult

• More likely needs extra space in the memory.

Examples:
• Factorial calculation, Fibonacci number
• Tower of Hanoi algorithm
Computational Thinking

Algorithm design paradigms

3. Greedy algorithm • You take the best you can get right now, without regard
for future consequences.
• You hope that by choosing a local best/optimum at
each step, you will end up at a global best/optimum.
• Not necessarily the best solution
• Usually good result with relatively fast computation
time

Examples:
• Iteratively go to the nearest neighboring city, when
finding shortest path from Depok to Banyuwangi

https://www.simplilearn.com/tutorials/data-structure-tutorial/greedy-algorithm
• Sequencing jobs with the smallest processing time (SPT)
Computational Thinking

Algorithm design paradigms

4. Divide and conquer

• Recursively breaking down a complex

problem into smaller, more manageable
parts, solving each part individually,
and then combining the solutions to
solve the original problem.

Examples:
• Merge sort
• Strassen’s matrix multiplication
algorithm
Computational Thinking

Algorithm design paradigms

5. Dynamic programming

• Divide the main problem into smaller, independent

subproblems.
• Solve each subproblem and store the solution in a table
or array.
• Use the stored solutions to build up the solution to the
main problem.
• Avoid redundancy

Examples:
• The Floyd-Warshall algorithm for all-pair shortest path
• Partition a Set into Two Subsets of Equal Sum
Computational Thinking

Let’s study some algorithms

• Binary search
• Brute-force algorithm for checking
whether a number is prime or not
• Greedy algorithm for knapsack
problem

Study these 3 algorithms with your group.

Follow the instructions on Assignment 4.
The following slides may be included in
your assignment, but you must add other
information
End of today’s lecture