UCCC 2063

ALGORITHMS ANALYSIS
Lecturer & Tutor: Savio TSE
E-mail: [email protected]

Teaching Assistant: Ms Ng Wan Qing

Email: [email protected]
Lectures: Online
Tuesday, 12pm –2pm
Thursday, 8:00am – 9:00am

Tutorials: LDK 2
T1+T2: Monday, 1pm-2am
T5+T6: Monday, 2pm-3pm
T7+T8: Wednesday, 2pm-3pm
T4+T9: Wednesday, 3am-4am
T10+T11: Thursday, 12pm-1pm
T3+T12: Thursday, 1pm-2pm

Consultation hours: Savio + NgWQ

Tuesday, 2-4pm, online
Assessment
• 1 Midterm Test (15%)
• 1 Quiz (10%)
• 1 Assignment (15%)
• 1 Final Assessment (60%)

• In order to pass the course, you must

• get  40 out of 100 in BOTH coursework
and Final Assessment; AND
• get  50 out of 100 in overall grade
• From Week 1 to Week 10,  80% attendance
is needed. 1-3
Textbook: the best one
• Cormen, T.H., Leiserson, C.E.,
Rivest, R.L., Stein, C.
Introduction to Algorithms. (2nd
ed.). Cambridge, Massachusetts:
MIT Press.
• This book is good, and enough for
almost your whole life.
• My opinion: 2nd Ed. is better
than 3rd Ed.
1-4
Mode of Teaching
Lectures will be held online.
Tutorials will be held physically.
Consultation hours will be held online.

Information delivery is done by the official

channel is WBLE. You are required to
visit WBLE once for every three days.
Mode of Teaching
Consultations are done by MS Team in the
scheduled time.

Consultations can also be done by email.

Email me your problem. If the issues are
of common interest, I will post it to
WBLE.
Requirement
1. Programming
Concepts • Psychologically, be
2. Data Structures ready for tough
and Algorithms works.
3. Discrete
Mathematics • Patience 忍耐
4. Basic Algebra • Persistence 堅持
• Perseverance 毅力
5. Calculus
6. Probability
1-7
Expected from you:
A few days before each lecture or
tutorial, download the lecture
notes or tutorial handout, and
study it to your best.
In the class, tell me your
difficulties in understanding the
materials.
After the class, revise it.
1-8
Expected from you:
Other than lectures and tutorials,
spend 10 hours per week for this
course.

Note, it is less than 1.5 hours a

day. Not demanding at all.

Even if you take 5 such subjects,

7.5 hours are needed. After all,
not all subjects are like this one.
Expected from you:

Keep Algorithm in mind.

(Fairly speaking, do the

same to other subjects.)
Warranty & Guaranteed
1. No guarantee for a pass.

2. Ifyou work hard, it is

guaranteed that you will be
improving everyday, and finally
pushed to your capacity.

1-11
The most important
question:

What is this course

about?
Introduction
1. What is algorithm?
2. Why study algorithms?
3. What is the difference between this
subject and data structures and
algorithms?
4. Time vs Space Trade-off
5. Example of Computational Problem:
Sorting
6. Some Well-known Computational Problems
7. Basic Issues Related to Algorithms 1-13
What is algorithm?
Algorithm is a term for
describing the ways of
solving problems.

Algorithm is the name of such

a subject.
What is algorithm?
An algorithm is designed for
solving a problem.
A problem is a computational
problem in this course.
A problem is defined by input
domain, and output
requirement. (input,output)
Comparing with the course:
Data Structures and Algorithms
Algorithms Analysis should be algorithm
analysis.
It is an advanced subject of Data
Structures and Algorithms.
No C++ programming in final examination,
midterm test, and quiz. However, there
may be some C++ programming in
assignment. Not yet decided.
Sorting:
Input: An integer set S of
arbitrarily ordered elements.
Output: A sequence---b1, b2,
…, bn, which consists all
elements in S, such that
bi≤bi+1 where i[1,n-1] and
n=|S|.
Domains of Input and output must
be clearly defined.
What are the conditions of input
and output, respectively?

An algorithm will accept an

input, and give the required
output.
.
Basically, it is not a job of
an algorithm to check whether
the input is taken from the
correct domain.

We can always assume the inputs

are taken from the correct
domain.
Example:

After you write a sorting algorithm

(program) for sorting integers, your
boss blames you after he tries to use
strings as input.
Actually, it is NOT the fault of the
algorithm, but the design of input
domain. That is, the problem design.
Problem design is in Software
Engineering, not Algorithm Analysis.
Tell your boss that the problem he gave
is wrong, not the algorithm.
What is algorithm?
An algorithm can be a sequence of
unambiguous instructions for solving a
problem, i.e., for obtaining a required
output for any legitimate input in a
finite amount of time.

input algorithm output

1-21
What is algorithm?
 AnALGORITHM can be considered as a
tool for solving a computational
problem.

 AnALGORITHM can be considered as a

function from the set of possible
inputs into the set of possible
outputs.

1-22
What is algorithm?
Algorithm + Data Structure
=
Spirit of a program

1-23
What is algorithm?
1. terminates in a finite number of steps
2. rigorously & unambiguously specified
3. input & output domains are clearly
defined
4. Correctness can be proven
5. steps are sufficiently simple and
primitive

1-24
Why Theoretical Computer Science?
 Theoretical Computer Science
 Discrete Math, Algorithm, Data structure, …
 System
 Operating Systems, Networking, Language, …
 Software Engineering
 A subject for developing large system.
 Applications
 Without applications, computer is useless!
All these branches together form the core of
Computer Science.
You can decide which one is core to yourself.1-25
What is algorithm analysis? (1)

Algorithm Analysis
We want to analyze
the execution of the
algorithm

First, output must be correct.

Second, algorithm must be good.

1. using less time when executed
2. using less memory when executed
In order
3. having short code
4. is well documented

1-29
What is algorithm analysis? (2)

Algorithm Analysis
We want to analyze
the execution of the
algorithm

First, output must be correct.

Second, algorithm must be good.

1. using less time when executed
4. using less memory when executed
For
2. having short code
product
3. is well documented
manager

1-30
What is algorithm analysis? (3)

Algorithm Analysis
We want to analyze
the execution of the
algorithm

First, output must be correct.

Second, algorithm must be good.

3. using less time when executed
4. using less memory when executed
For
1. having short code
Software
2. is well documented
Engineers

1-31
What is algorithm analysis? (4)

Algorithm Analysis
We want to analyze
the execution of the
algorithm

First, output must be correct.

Second, algorithm must be good.

1. using less time when executed
4. using less memory when executed
For System
2. having short code
Researchers
3. is well documented

1-32
Time vs Space Trade-off

Here we will focus on TIME complexity first.

Two algorithms designed for solving the same problem won’t have such
a big difference in memory requirements. (normally)

Sometimes we have to find the golden middle between:

more space less space

but faster but slower

SPACE complexity plays a more theoretical role. Here we can make the
hierarchy of problem classes finer.

1-33
Basic Issues Related to Algorithms

 How to design algorithms

 How to express algorithms
Use pseudo codes
 Proving correctness
Theoretical analysis
 Efficiency
Theoretical analysis
Empirical analysis, not preferred
 Optimality
1-34
Important problem types
 Sorting
 Searching
 string processing
 graph problems
 combinatorial problems
 geometric problems
 numerical problems

1-35
Fundamental data structures

• array
• linked list
• tree
• I think, in the next 30 years,
there will only be these three
fundamental data structures. 1-36
Other common data structures

 list
 stack
 queue
 priority queue
 Binary tree
1-37
Some Well-known Computational Problems

 Sorting
 Searching
 Shortest paths in a graph
 Minimum spanning tree
 Primality testing
 Traveling salesman problem (NP hard)
 Knapsack problem (NP hard)
 Chess
 Towers of Hanoi
1-38
Fundamental Data Structures

Array

Linked-list

Tree
Array Revisit – linear search – O(n) time

Given an array A[1..N] of N elements of arbitrary

order, search for a target element x. If found,
output its first position of in A; otherwise,
output “unfound”.
In C/C++
For i=1 to N int A[100], x, i;
………
If A[i] = x
for(i=1, i<=N, i++)
Output i;
if (A[i]== x) {
terminate; cout << i;
If unfound break;
Output “unfound”; }
if (i>N) cout <<“unfound”;
return 0;
Linked-List Revisit

There are two styles of implementations.

With header node, and with head pointer.

Header node Null

Head pointer
Null

// for header node, if used

struct Node {
struct HeaderNode{
int element;
string info;
Node* next;
Node* next;
};
};
Linked-List Revisit

For empty linked-list,

Header node Null

Head pointer
Null
Linked-List Revisit

If header node is used, struct Node {

int element;
Header node Null Node* next;
};

// for header node, if used

struct HeaderNode{
string info;
Node* next;
};

HeadNode L;
L.Next = NULL;
// L is the header node.
Linked-List Revisit

If head pointer is used, struct Node {

int element;
Node* next;
};

Head pointer
Null Node* L = NULL;
// L is the head pointer,
// initialized to NULL
Linked-List Revisit

There are two styles of implementations.

With header node, and with head pointer.

Header node Null

Head pointer
Null

Which one is better?

Linked-List Revisit – linear search – O(n) time

We are given a linked-list of N arbitrarily-ordered

elements, with a head pointer L. Write code for
searching for a target integer x. If found, output
“found”; otherwise, output “unfound”.

temp = L; node* temp=L;

while (temp != NULL){
while temp  
if (temp->element== x){
If temp->element = x
cout << “found”;
Output “found”;
return;
terminate;
}
temp=temp->next;
temp=temp->next;
Output “unfound”; }
cout <<“unfound”;
return;
Tree – Revisit

When we program, we commonly program binary tree, not

general tree. Why?
Suppose a parent can have 1 to N children.

#define N 1000
struct node {
int element;
node* next[N];
};

Note, a parent has 1 child, then the pointers

next[1..999] will be a waste of storage.
Tree – Revisit

Logical tree  Physical tree

struct node {
int element;
node* child; // red
node* sibling; // black, horizontal
};
Tree – Revisit

Structurally, it is the same as binary tree

Physical tree

struct node { // N-ary struct node { // binary

int element; int element;
node* child; node* left;
node* sibling; node* right;
}; };
Tree Revisit

N-ary tree (a parant can have 1 to N children) is

better implemented by binary tree.

This is why programming binary tree is more important

than programming N-ary tree directly.

Study on binary tree is more important than N-ary

tree.
Binary tree Revisit
Two styles of implementations: Root pointer
treeheadnode rootptr

root root

struct node { struct TreeHeadNode{

int element; string info;
node* left; node* ptr;
node* right; };
}; TreeHeadNode treeheadnode;
node* rootptr;
Binary tree Revisit

Empty tree Representations:

rootptr
treeheadnode

NULL NULL
Binary tree Revisit

Traversal algorithms:

Preorder traversal
Inorder traversal
Postorder traversal
Binary tree Revisit

Preorder traversal – O(n)-time, n=#node

void preorder(node* ptr){

if (ptr==NULL)
return;
cout << ptr->element <<  ;
preorder(ptr->left);
preorder(ptr->right);
return;
}

Can there be two binary trees resulting the

same output?
Binary tree Revisit

Inorder traversal - O(n)-time, n=#node

void inorder(node* ptr){

if (ptr==NULL)
return;
inorder(ptr->left);
cout << ptr->element <<  ;
inorder(ptr->right);
return;
}

Can there be two binary trees resulting the

same output?
Binary tree Revisit

Postorder traversal - O(n)-time, n=#node

void postorder(node* ptr){

if (ptr==NULL)
return;
postorder(ptr->left);
postorder(ptr->right);
cout << ptr->element <<  ;
return;
}

Can there be two binary trees resulting the

same output?
Binary tree Revisit

Preorder traversal
Postorder traversal

Can there be two binary trees

resulting the same output in
BOTH preorder and postorder
traversals?
Binary tree Revisit

Preorder traversal
Inorder traversal

Can there be two binary trees

resulting the same output in
BOTH preorder and inorder
traversals?
Binary tree Revisit

Inorder traversal
Postorder traversal

Can there be two binary trees

resulting the same output in
BOTH inorder and postorder
traversals?
Binary SEARCH tree Revisit

A binary tree with sorted order.

x x

The equal signs can be applied to

both sides although we normally
place it on one side only.
Binary SEARCH tree Revisit
Time for insertion, searching, deletion, are
all O(h), where h is the height of the
tree, and h varies from log2n to n.

What is the definition of height?

Two definitions from the literature. One is

#step from the root to the farthest leaf.
The other is #node along the path from the
root to the farthest leaf. The difference
is only one. We choose the second
(highlighted) one in this course.