INTRODUCTION TO

ALGORITHMS
Dr. Prima Dewi Purnamasari
Introduction to Algorithms, 3rd ed, by Cormen, Leiserson,
Rivest & Stein. Chapter 1: The Role of Algorithms in
Computing
Introduction to the Designs and Analysis of Algorithm,
Anany Levitin. Chapter 1: Introduction
 WHAT IS AN ALGORITHM?
▪ Levitin:
▪ An algorithm is a sequence of unambiguous instructions for solving a
problem, i.e., for obtaining a required output for any legitimate input in a
finite amount of time.
▪ Cormen:
▪ A sequence of computational steps that transform the input into output.
▪ A tool for solving a well-specified computational problem.
▪ Problem specification includes what the input is, what the desired output should be.
▪ Algorithm describes a specific computational procedure for achieving the desired
output for a given input.

PDP2024 2
 WHAT IS AN ALGORITHM?
• Can be represented various forms

problem • Unambiguity/clearness
• Effectiveness
• Finiteness/termination
algorithm • Correctness

input “computer” output

PDP2024 3
 Studying
algorithms is about
studying efficiency

How fast does it ALGORITHMS

run?

Can I make it run

faster?

PDP2024 4
HISTORICAL PERSPECTIVE
▪ Muhammad ibn Musa al-Khwarizmi – 9th century mathematician
www.lib.virginia.edu/science/parshall/khwariz.html

PDP2024 5
EUCLID’S ALGORITHM FOR FINDING THE
GREATEST COMMON DIVISOR
▪ The greatest common divisor (GCD) of two nonnegative, not-both-zero integers m and
n, gcd (m, n), is defined as the largest integer that divides both m and n evenly, i.e., with
a remainder of zero.
▪ Euclid of Alexandria (third century b.c.) outlined an algorithm for solving this problem
in one of the volumes of his Elements most famous for its systematic exposition of
geometry.
▪ In modern terms, Euclid’s algorithm is based on applying repeatedly the equality
gcd (m, n) = gcd (n, m mod n),
where m mod n is the remainder of the division of m by n, until m mod n is equal to 0.
▪ Since gcd(m, 0) = m , the last value of m is also the greatest common divisor of the
initial m and n.
▪ For example, gcd(60, 24) can be computed as follows:
gcd (60, 24) = gcd (24, 12) = gcd (12, 0) = 12.

PDP2024 6
EUCLID’S ALGORITHM FOR COMPUTING
GCD(M, N)
▪ Step 1 If n = 0, return the value of m as the answer and stop; otherwise, proceed to
Step 2.
▪ Step 2 Divide m by n and assign the value of the remainder to r.
▪ Step 3 Assign the value of n to m and the value of r to n. Go to Step 1.

PDP2024 7
ALGORITHM: EUCLID(M, N)
//Computes gcd(m, n) by Euclid’s algorithm
//Input: Two nonnegative, not-both-zero integers m and n
//Output: Greatest common divisor of m and n
while n = 0 do
r ←m mod n
m←n
n←r
return m

PDP2024 8
CONSECUTIVE INTEGER CHECKING
ALGORITHM FOR COMPUTING GCD(M, N)
▪ Step 1 Assign the value of min{m, n} to t.
▪ Step 2 Divide m by t. If the remainder of this division is 0, go to Step 3; otherwise,
go to Step 4.
▪ Step 3 Divide n by t. If the remainder of this division is 0, return the value of t as the
answer and stop; otherwise, proceed to Step 4.
▪ Step 4 Decrease the value of t by 1. Go to Step 2.

PDP2024 9
MIDDLE-SCHOOL PROCEDURE FOR
COMPUTING GCD(M, N)
▪ Step 1 Find the prime factors of m.
▪ Step 2 Find the prime factors of n.
▪ Step 3 Identify all the common factors in the two prime expansions found in
▪ Step 1 and Step 2. (If p is a common factor occurring pm and pn times in m and n,
respectively, it should be repeated min{pm, pn } times.)
▪ Step 4 Compute the product of all the common factors and return it as the greatest
common divisor of the numbers given.

This one does not qualify, in the form presented, as a legitimate algorithm.
Why? Because the prime factorization steps are not defined unambiguously: they
require a list of prime numbers,
PDP2024 10
EXAMPLE ON THE SORTING PROBLEM
Input: A sequence of n numbers [a1, a2, … , an].
Output: A permutation or reordering [a'1, a'2, … , a'n ] of the input sequence
such that a'1  a'2  …  a'n .

An instance of the Sorting Problem:

Input: A sequence of 6 number [31, 41, 59, 26, 41, 58].

Expected output for given instance:

Expected Output:
The permutation of the input [26, 31, 41, 41, 58 , 59].

PDP2024 11
 • Sorting
• Searching
• Shortest paths in a graph
• Minimum spanning tree
SOME WELL- • Primality testing
KNOWN • Traveling salesman
COMPUTATIONAL problem
PROBLEMS • Knapsack problem
• Chess
• Towers of Hanoi
• Program termination

Some of these problems don’t have efficient algorithms, or

algorithms at all!
PDP2024 12
BASIC ISSUES RELATED TO ALGORITHMS

How to design How to express

Proving correctness
algorithms algorithms

Efficiency (or
complexity) analysis
• Theoretical analysis Optimality
• Empirical analysis

1-13
PDP2024
 • Brute force • Greedy approach
• Divide and conquer • Dynamic programming
• Decrease and conquer • Iterative improvement
• Transform and conquer • Backtracking
• Space and time tradeoffs • Branch-and-bound

ALGORITHM DESIGN STRATEGIES

PDP2024 14
 ALGORITHMS
▪ An algorithm is said to be correct, if, for
every input instance, it halts with the
correct output.
▪ A correct algorithm solves the given
computational problem.
▪ Focus will be on correct algorithms;
incorrect algorithms can sometimes be
useful.
▪ Algorithm specification may be in
Natural Language, as a computer
program, even as a hardware design.

PDP2024 15
ANALYSIS OF ALGORITHMS
▪ How good is the algorithm?
▪ Correctness
▪ Time efficiency
▪ Space efficiency

▪ Does there exist a better algorithm?

▪ Lower bounds
▪ Optimality

1-16
PDP2024
WHY DO WE NEED
TO EVALUATE
ALGORITHM?
 Computers may be fast, but they are
not infinitely fast.

Memory may be inexpensive, but it is

not free.

REASON
Computing time is therefore a
bounded resource, and so is space in
memory.

You should use these resources

wisely, and algorithms that are
efficient in terms of time or space will
help you do so.

PDP2024 18
EFFICIENCY Computer A
Faster
Computer B
Slower
Insertion sort Merge sort

TASK: 10.000 MIPS 10 MIPS

Sorting array of 10 Machine language High language program
million interger program with insufficient compiler
numbers 2n2 50n lg n

A:

B:

Merge sort is more efficient than insertion sort

PDP2024 19
ALGORITHM AS A TECHNOLOGY
▪ For example, consider a Web-based service that
determines how to travel from one location to
another.
▪ Its implementation would rely on:
▪ fast hardware
▪ graphical user interface
▪ wide-area networking,
▪ and also possibly on object orientation.

▪ However, it would also require algorithms for certain

operations, such as:
▪ finding routes,
▪ rendering maps,
▪ and interpolating addresses.

PDP2024 20
 ALGORITHM
DESIGN AND
ANALYSIS
PROCESS.

PDP2024 21
▪ sorting
▪ searching IMPORTANT
▪ string processing PROBLEM
▪ graph problems TYPES
▪ combinatorial problems
▪ geometric problems
▪ numerical problems

PDP2024 22
SORTING (I)
▪ Rearrange the items of a given list in ascending order.
▪ Input: A sequence of n numbers <a1, a2, …, an>
▪ Output: A reordering <a´1, a´2, …, a´n> of the input sequence such that a´1≤ a´2 ≤ … ≤
a´n.
▪ Why sorting?
▪ Help searching
▪ Algorithms often use sorting as a key subroutine.

▪ Sorting key
▪ A specially chosen piece of information used to guide sorting. E.g., sort student records
by names.

PDP2024 23
SORTING (II)
▪ Examples of sorting algorithms
▪ Selection sort
▪ Bubble sort
▪ Insertion sort
▪ Merge sort
▪ Heap sort …

▪ Evaluate sorting algorithm complexity: the number of key comparisons.

▪ Two properties
▪ Stability: A sorting algorithm is called stable if it preserves the relative order of any two
equal elements in its input.
▪ In place : A sorting algorithm is in place if it does not require extra memory, except,
possibly for a few memory units.

PDP2024 24
SELECTION SORT
Algorithm SelectionSort(A[0..n-1])
//The algorithm sorts a given array by selection sort
//Input: An array A[0..n-1] of orderable elements
//Output: Array A[0..n-1] sorted in ascending order
for i  0 to n – 2 do
min  i
for j  i + 1 to n – 1 do
if A[j] < A[min]
min  j
swap A[i] and A[min]

PDP2024 25
SEARCHING
▪ Find a given value, called a search key, in a given set.
▪ Examples of searching algorithms
▪ Sequential search
▪ Binary search …

Input: sorted array a_i < … < a_j and key x;

m (i+j)/2;
while i < j and x != a_m do
if x < a_m then j  m-1
else i  m+1;
if x = a_m then output a_m;

PDP2024 Time: O(log n) 26

STRING PROCESSING
▪ A string is a sequence of characters from an alphabet.
▪ Text strings: letters, numbers, and special characters.
▪ String matching: searching for a given word/pattern in a text.

Examples:
(i) searching for a word or phrase on WWW or in a
Word document
(ii) searching for a short read in the reference genomic
sequence

PDP2024 27
GRAPH PROBLEMS
▪ Informal definition
▪ A graph is a collection of points called vertices, some of which are connected by line
segments called edges.
▪ Modeling real-life problems
▪ Modeling WWW
▪ Communication networks
▪ Project scheduling …

▪ Examples of graph algorithms

▪ Graph traversal algorithms
▪ Shortest-path algorithms
▪ Topological sorting

PDP2024 28
 ▪ list
▪ array FUNDAMENTAL
▪ linked list
▪ string
DATA
▪ stack
STRUCTURES
▪ queue
▪ priority queue/heap
▪ graph
▪ tree and binary tree
▪ set and dictionary

29
PDP2024
LINEAR DATA STRUCTURES
array

▪ Arrays ▪ Arrays
▪ A sequence of n items of the same data ▪ fixed length (need preliminary
type that are stored contiguously in reservation of memory)
computer memory and made ▪ contiguous memory locations
accessible by specifying a value of the
array’s index. ▪ direct access
▪ Insert/delete
▪ Linked List
▪ A sequence of zero or more nodes ▪ Linked Lists
each containing two kinds of ▪ dynamic length
information: some data and one or ▪ arbitrary memory locations
more links called pointers to other
▪ access by following links
nodes of the linked list.
▪ Insert/delete

single linked list

PDP2024 30
double linked list
STACKS AND QUEUES
▪ Stacks
▪ A stack of plates
▪ insertion/deletion can be done only at the top.
▪ LIFO
▪ Two operations (push and pop)

▪ Queues
▪ A queue of customers waiting for services
▪ Insertion/enqueue from the rear and deletion/dequeue from the front.
▪ FIFO
▪ Two operations (enqueue and dequeue)

PDP2024 31
PRIORITY QUEUE AND HEAP
▪ Priority queues (implemented using heaps)
▪ A data structure for maintaining a set of elements, each associated with a key/priority,
with the following operations
▪ Finding the element with the highest priority
▪ Deleting the element with the highest priority
▪ Inserting a new element
▪ Scheduling jobs on a shared computer

PDP2024 32
GRAPHS
▪ Formal definition
▪ A graph G = <V, E> is defined by a pair of two sets: a finite set V of
items called vertices and a set E of vertex pairs called edges.
▪ Undirected and directed graphs (digraphs).
▪ What’s the maximum number of edges in an undirected graph with |V| vertices?
▪ Complete, dense, and sparse graphs
▪ A graph with every pair of its vertices connected by an edge is
called complete, K|V|

1-33
PDP2024
undirected graph digraph
GRAPH REPRESENTATION
▪ Adjacency matrix
▪ n x n boolean matrix if |V| is n.
▪ The element on the ith row and jth column is 1 if there’s an edge
from ith vertex to the jth vertex; otherwise 0.
▪ The adjacency matrix of an undirected graph is symmetric.

▪ Adjacency linked lists

▪ A collection of linked lists, one for each vertex, that contain all the
vertices adjacent to the list’s vertex.
▪ Which data structure would you use if the graph is a 100-node
star shape?

PDP2024 34
WEIGHTED GRAPHS
▪ Weighted graphs
▪ Graphs or digraphs with numbers assigned to the edges.

(a) Weighted graph. (b) Its weight matrix. (c) Its adjacency lists.

1-35
PDP2024
GRAPH PROPERTIES -- PATHS AND
CONNECTIVITY
▪ Paths
▪ A path from vertex u to v of a graph G is defined as a sequence of adjacent (connected by
an edge) vertices that starts with u and ends with v.
▪ Simple paths: All edges of a path are distinct.
▪ Path lengths: the number of edges, or the number of vertices – 1.

▪ Connected graphs
▪ A graph is said to be connected if for every pair of its vertices u and v there is a path from
u to v.
▪ Connected component
▪ The maximum connected subgraph of a given graph.

PDP2024 36
Graph that is not connected.
GRAPH PROPERTIES -- ACYCLICITY
▪ Cycle
▪ A simple path of a positive length that starts and ends a the same vertex.

▪ Acyclic graph
▪ A graph without cycles
▪ DAG (Directed Acyclic Graph)

1 2

3 4

PDP2024 37
TREES
(a) Tree. (b) Forest.
▪ Trees
▪ A tree (or free tree) is a connected acyclic graph.
▪ Forest: a graph that has no cycles but is not necessarily connected.

▪ Properties of trees
▪ For every two vertices in a tree there always exists exactly one simple path from one of
these vertices to the other. Why?
▪ Rooted trees: The above property makes it possible to select an arbitrary vertex in a free tree and
consider it as the root of the so called rooted tree.
▪ Levels in a rooted tree.

◼ |E| = |V| - 1

PDP2024 38
(a) Free tree. (b) Its transformation into a rooted tree.
ROOTED TREES (I)
▪ Ancestors
▪ For any vertex v in a tree T, all the vertices on the simple path from the root to that vertex are
called ancestors.
▪ Descendants
▪ All the vertices for which a vertex v is an ancestor are said to be descendants of v.

▪ Parent, child and siblings

▪ If (u, v) is the last edge of the simple path from the root to vertex v, u is said to be the parent of v
and v is called a child of u.
▪ Vertices that have the same parent are called siblings.

▪ Leaves
▪ A vertex without children is called a leaf.

▪ Subtree
▪ A vertex v with all its descendants is called the subtree of T rooted at v.

PDP2024 39
ROOTED TREES (II)
▪ Depth of a vertex
▪ The length of the simple path from the root to the vertex.

▪ Height of a tree
▪ The length of the longest simple path from the root to a leaf.

h=2
3

4 1 5

PDP2024 40
ORDERED TREES
(a) Binary tree. (b) Binary search tree.
▪ Ordered trees
▪ An ordered tree is a rooted tree in which all the children of each vertex are ordered.

▪ Binary trees
▪ A binary tree is an ordered tree in which every vertex has no more than two children and
each children is designated s either a left child or a right child of its parent.
▪ Binary search trees
▪ Each vertex is assigned a number.
▪ A number assigned to each parental vertex is larger than all the numbers in its left
subtree and smaller than all the numbers in its right subtree.
▪ log2n  h  n – 1, where h is the height of a binary tree and n the size.

PDP2024 41
SETS AND DICTIONARIES
▪ set can be described as an unordered collection (possibly empty) of distinct items
called elements of the set.Aspecific set is defined either by an explicit listing of its
elements (e.g., S = {2, 3, 5, 7}) or by specifying a property that all the set’s elements
and only they must satisfy (e.g., S = {n: n is a prime number smaller than 10}).
▪ list defined as an ordered collection of items. List can contain duplicative element
▪ bag is an unordered collection of items that are not necessarily distinct.
▪ dictionary, data structure that implements searching, adding a new item, and
deleting an item from the collection.

PDP2024 42
TI-02 (INDIVIDUAL, 1 WEEK)
1. Which of the following formulas can be considered an algorithm for
computing the area of a triangle whose side lengths are given positive
numbers a, b, and c?
a. S = p(p − a)(p − b)(p − c), where p = (a + b + c)/2
b. S = ½ b c sin A, where A is the angle between sides b and c
c. S = ½ a ha, where ha is the height to base a
2. Suppose we are comparing implementations of insertion sort and merge sort
on the same machine. For inputs of size n, insertion sort runs in 8n2 steps,
while merge sort runs in 64 n lg n steps. For which values of n does insertion
sort beat merge sort?

PDP2024 43
3. What is the smallest value of n such that an algorithm whose running time is 100n2
runs faster than an algorithm whose running time is 2n on the same machine?
4. Consider the algorithm for the sorting problem that sorts an array by counting,

a. Apply this algorithm to sorting the list 60, 35, 81, 98, 14, 47.
b. Is this algorithm stable?
c. Is it in-place?
44
PDP2024
5. For each of the following applications, indicate the most appropriate data structure:
a. answering telephone calls in the order of their known priorities
b. sending backlog orders to customers in the order they have been received
c. implementing a calculator for computing simple arithmetical expressions

6. Design an algorithm for checking whether two given words are anagrams, i.e.,
whether one word can be obtained by permuting the letters of the other. For
example, the words tea and eat are anagrams.

PDP2024 45