INFORMATION ABOUT TDA3231

Attendance

 80% (MUST)
 If absent, please justify with valid reason
 If sick, then provide MC
 If has parttime job, please get an official letter from your employer with
stamp and sign
 Inform within one week after absent
 If attendance is extremely low, either withdraw or bar
 ERU will run the attendance calculation on 9 December. Any
justification and attendance issue must solve before that
 It is not an faculty issue, it is the whole university, please treat it
seriously
 Replacement class
Original Date/Time Original Venue Replace Date/Time Replace Venue

1/5/2024 (Wed) Graphic Lab 20/5/2024 (Mon) Team

Meeting ID: 482 306 763 341
Lab 1A 7.30pm – 9.30pm Passcode:
GPY8BWnQUM

22/5/2024 (Wed) Graphic Lab 20/5/2024 (Mon) Team

Meeting ID: 482 306 763 341
Lab 1A 9.30 pm– 11.30 pm Passcode: BWnQUM

3/6/2024 (Mon) A1 27/5/2024 (Mon) Team

GPY9
Meeting ID: 480 934 713 074
Lecture 8.00 pm – 10.00pm Passcode: fQQ3zy
10am - 12pm

17/6/2024 (Mon) A1 18/6/2024 (Tues) Team

GPY10
Meeting ID: 432 734 390 701
Lecture 8.00 pm – 10.00pm Passcode: Z3ewkE
10am - 12pm
Slide 3

GPY8 https://teams.microsoft.com/l/meetup-join/19%3ameeting_NTUzZDAzNmQtNDY4Ni00NDUyLTg3YTYtMjBhNDEwOTExZWMy%40thread.v2/0?context=%7b%
Goh Pey Yun, 3/18/2024

GPY9 https://teams.microsoft.com/l/meetup-join/19%3ameeting_NmVjZTc3NWItYzJlNS00MzU1LTljZDQtODcxNmNlZDRmM2I1%40thread.v2/0?context=%7b%22T
Goh Pey Yun, 3/18/2024

GPY10 https://teams.microsoft.com/l/meetup-join/19%3ameeting_ZTU1ZWVkZGYtOWY4OC00M2U4LWE2ZjctMjE2Yjc5OWM4Njc0%40thread.v2/0?context=%7b%2
Goh Pey Yun, 3/18/2024
Learning Outcomes of the course

Learning Outcomes Assessment Method

LO1: Comprehend the essential concept behind algorithm Quiz,

design and analysis.
LO2: Solve the fundamental problems in algorithm design Assignment,Test
and analysis
LO3: Demonstrate the solution by using proper algorithm Project (2 parts, 1 review, 1 programming + presentation)
design and analysis methods in team.
Assessment Methods

Assessment Methods Marks(%)

Continuous Assessment
Quiz 20
Test 20
Assignment 20
Final Assessment
Project 40
 LECTURE 01
INTRODUCTION
ALL OF COMPUTER SCIENCE IS THE STUDY OF
ALGORITHMS
Books and Reference Sources

 Main
 Jon Kleinberg (2015). Algorithm Design (2nd ed.). Addison Wesley.
 Harsh Bhasin (2015). Algorithms: Design and Analysis. Oxford University
Press.
 Dwivedi, Gyanendra Kumar (2017). Analysis and Design of Algorithm.
University Science Press.
 Additional
 1. Robert Sedgewick and Philippe Flajolet (2013) Introduction to the Analysis
of Algorithms (2nd ed.). Addison Wesley.
 2. Michael Soltys-Kulinicz (2018). Introduction to the Analysis of Algorithms
(3rd ed.). World Scientific Publishing.
 3. Anany Levitin (2013). Introduction to the Design and Analysis of Algorithms
(3rd ed.). Addison Wesley. 4. Anany Levitin (2013). Introduction to the Design
and Analysis of Algorithms (3rd ed.). Addison Wesley.
 Importance of Algorithms

 To solve problem with proper procedures, just like the SOP for
covid19
 Some existing applications:
 To search by typing a query in Google
 Output: ordered set of results mostly relevant to the query are
presented
 How Google able to do it?
 Use page rank algorithm, ranks the relevant pages based on the
entered query
 To combat terrorism
 Develop a global database of DNA
 Use sorting and searching algorithms to mine information among
billions of DNAs
 Optimisation
 In business, minimizing cost and maximizing profit is a critical task
 Greedy algorithm helps in achieving the task of optimisation
Representative Problem: Stable Matching

 Problem: assign Interns to Companies

 Challenge: if everyone act to self interest, then breaking down cases will keep
occur
 E.g.: Jimmy obtains offer from company B. He accepts the offer. However,
his favorite is actually company C. One week later, company C offers Jimmy
the opportunity. Since Jimmy prefers company C, so he rejects company B
although he has accepted the offer. When this happen, it is called breaking
down. NOT STABLE
 Let us exchange the situation.
 If company C offers Jimmy. After that, a new candidate Mary is applying.
From the resume, company C prefers Mary rather than Jimmy. Thus,
company C cancel the offer to Jimmy. This is a breaking down too.
 Chaos will occur.

Reference source: Algorithm Design. Kleinberg, J. & Tardos, E. (2015)

Stable Matching

 Stable:
 Since Jimmy has accepted the offer from company B so he said no to
company C.
 Then, company C offers the internship to Mary.
 All outcomes are stable.
 Stable Matching

 Assume the following, the ranked preference list:

Least favourite Average Favourite Perfect match occurs when:

Student 1 Company A Company B Company C Each student gets exactly a
company.
Student 2 Company B Company A Company C
Each company gets exactly a
Student 3 Company A Company B Company C student.

Least favourite Average Favourite Unstable occur when:

There is a student X of the first matched
Company A Student 2 Student 1 Student 3
set prefers the given company Z of the
Company B Student 1 Student 2 Student 3 second matched over his/her match pair;
Company C Student 1 Student 2 Student 3 AND
Company Z also prefers X over its own
match pair.
Stable Matching

 Look at the matching of 1-C, 2-B, 3-A, is the match stable?

Favourite Average Least Favourite

Student 1 Company A Company

Company B B Company C NOT STABLE
Student 2 Company B Company A Company C

Student 3 Company A Company B Company C

Favourite Average Least Favourite

Company A Student 2 Student 1 Student 3

Company B Student
Student 1 1 Student 2 Student 3

Company C Student 1 Student 2 Student 3

Stable Matching

 Assume the following, the ranked preference list:

Change to 1-B, 2-A, and 3-C.
By following the sequence,
Least favourite Average Favourite since 1 has chosen B, 2 has
Student 1 Company A Company B Company C chosen A, so 3 left only C.
Student 2 Company B Company A Company C

Student 3 Company A Company B Company C

The match is stable now.

Least favourite Average Favourite

Company A Student 2 Student 1 Student 3

Company B Student 1 Student 2 Student 3

Company C Student 1 Student 2 Student 3

Stable Matching

 The goal of stable matching:

 No instability for a matching where it is perfect, no instability for the
match
Can you get the perfect match?
 Example:
m1: w1 w2 m1 w1
m2: w2 w1
w1: m1 m2
w2: m2 m1 m2 w2
Stable Matching

 Interesting contradiction that is always happening in the real life

m1: w1 w2 m1 w1
m2: w2 w1
w1: m2 m1
w2: m1 m2 m2 w2

Two perfect matches:

m1w1 & m2w2
m1w2 & m2w1
Designing the Algorithm

 Initially, no matching pair (assume each element has preference ranked

list)
 While there is any m that remain free, proceed
 w1= the highest preference on m list where m is yet to visit
 If m1 favours over w1 and w1 is free, match (m1, w1) and keep in an
intermediate stage (there is a possibility that w1 may reject, so we can
design to have an intermediate stage)
 Else w1 not free and has matched with m2
 If w1 favours m2 more than m1, then m1 remains free
 Otherwise, match m1, w1 as a pair
 Return all matched pairs
Analyse the algorithm

 We need to know:
 Will the algorithm terminate? ~ demo
 Will it runs as it is i.e. to provide a stable match? ~ proof by logic such
as contradiction
 Can it runs within an acceptable time frame? ~ proof by growth
function
Why study algorithm design and analysis?

 Enhancing the thinking process

 Formalizing real world problem include the domain in computer science,
business, finance, medical etc
 Model: graph and preference lists
 Mechanism: stability condition
 Efficient algorithms lead to efficient programs
 Efficient programs sell better
 Efficient programs make better use of hardware
 Programmers who write efficient programs are more marketable than those
who don’t
 There are many alternative solutions, to find a suitable algorithm to solve
problem is challenging
 Developing your own algorithm may appropriately solve the problem
Math you Need to Review: Summations

 Look at the following notation for a summation:


 Which of the following definition is equal to the notation?
 A.
 B.
 C.
Math you Need to Review: Summations

 Why you need to know that?

 When loop is iterating, it naturally give rise to summations.
 Another e.g.: geometric summations

 Where the theorem for the summation is:
 For any integer n ≥ 0 and any real number 0< a≠1
 Math you Need to Review: Summations
 Fun story about the genius mathematician: Carl Friedrich
Gauss when he was an elementary student (late 1700s)
 Teacher wanted to snap but must assigned students
with work
 Sum the number from 1 to 100
Know your problem  Within few minutes, Gauss gave the answer: 5050
 Gauss solved the problem in this way:
Define the
similar/general structure

Design and formulate the

solution

 Using the matching pair method, there are 50 pairs of 101

(the middle pair will be 50+51). Thus, 101*50, you get the
( )
answer. The summation formula is:
Math you Need to Review: Induction

 The claims about a running time or a space bound involve an integer

parameter n
 If you try to proof thru direct calculation, that is exhaustive
 To proof a claim is true for all the n values, induction is applied.
 For e.g., the summation of first n integer is:
( )
 1+2+3+4….+n =
 Is it true for all the n integers where n ≥ 0?
 Use induction to show that it is applicable to all the other n values.

Math you Need to Review: Induction

 Justify the summation of first n integer with induction:

( )
 Base case: n = 1.
 Induction hypothesis: assume that the claim holds for n = k.
 Induction step:
( )( )
 Show that this is true for k+1:
( )

( )( )
 is true.
 Proof.
Exercise: Induction (15 minutes)

 For all n Natural number.

Base case: n = 0. 2 = 2 −1=1

Induction hypothesis: assume that the claim holds for n = k.
Induction step:
Show that this is true: 2 + 2 + ⋯ 2 + 2 = 2 −1+2
2 −1+2 =2 2 −1=2 − 1 is true.
Proof.
Math you Need to Review: Logarithms & Exponents

 𝑙𝑜𝑔 𝑎 = 𝑐 if 𝑎 = 𝑏
 Logarithms: how many times a number b has been multiplied by itself in order to obtain value a
 When b = 2, it is not written.
 Let a, b, c be positive real numbers:
 𝑙𝑜𝑔 𝑎𝑐 = 𝑙𝑜𝑔 𝑎 + 𝑙𝑜𝑔 𝑐 - rule 1
 = 𝑙𝑜𝑔 𝑎 − 𝑙𝑜𝑔 𝑐 - rule 2
 𝑙𝑜𝑔 𝑎 = 𝑐𝑙𝑜𝑔 𝑎 - rule 3
( )
 𝑙𝑜𝑔 𝑎 = - rule 4
𝑏 =𝑎 - rule 5
 (𝑏 ) = 𝑏 - rule 6
𝑏 𝑏 =𝑏 - rule 7
 𝑏 /𝑏 = 𝑏 - rule 8
 Rule 1 – 4 ~ logarithms
 Rule 5 – 8 ~ exponents
Exercise: Logarithms (10 minutes)


 =? • log 2𝑛𝑙𝑜𝑔𝑛 = log 2 + log 𝑛 + log log 𝑛 = 1 +
log 𝑛 + log log 𝑛 (rule 1, twice)
 n=? • log = log 𝑛 − log 2 = log 𝑛 − 1 (rule 2)
• 𝑙𝑜𝑔 𝑛 = (log 𝑛)/ log 4 = (log 𝑛)/2 (rule 4)
 =? • log 2 = 𝑛 log 2=n (rule 3)
• 2 = (2 ) = 𝑛 (rules 5 & 6)
 • 𝑛 2 = 𝑛 𝑛 = 𝑛 (rules 5, 6 & 7)
 =? • 4 /2 = =2 =2

 ?
Math you Need to Review: Proof Techniques

 To make strong claims about an algorithm, such as:

 The algorithm runs fast
 The algorithm is accurately solving the problem
 Using Mathematic language to justify or proof the claims
 Logic proof such as De Morgan’s Law, distributive, associative etc.
 E.g.: Someone claim that 2i-1 will get a prime number when i >1. We can
use counterexample method to proof that the claim is wrong.
Counterexample means that we able to find an i integer that make the
claim becomes false.
 Let i = 4, 24-1 = 15 and 15 = 3*5, thus it is not a prime number.
 Math you Need to Review: Proof Techniques

 If ab is odd, then a is odd or b is even.

 Proof: we can consider contrapositive (reverse
statement). If a is even and b is odd, then ab is even.
 a = 2i for some integer i. p q p q p q
q 
 ab = (2i)b = 2(ib) p

 Thus, ab is even. T T F F T T
T F F T F F
 As a conclusion: If ab is odd, then a is odd or b is F T T F T T
even is true. F F T T T T

 Contrapositive is preferable when the original statement

is difficult to proof directly.
 The truth table is shown for contrapositive.
 pq and the contrapositive is q p
Math you Need to Review: Basic Probability

 Basic probability properties

 P(Ø)=0 – empty set
 P(S) = 1
 0≤P(A)≤1, for any AS
 If A, BS and AB = Ø, the P(AB) = P(A)+P(B)
 Conditional probability
 P(A|B) = P(A  B)/P(B) where P(B) > 0
 Expected value
 E(X) =
 E(X+Y) = E(X) + E(Y)
 Exercise (5 minutes)

 Probability of picking a red marble from a jar that contains 4 red marbles
and 6 blue marbles. What is the probability of picking a blue marble?

6/10
Problem Development Steps

 To solve computational problem:

 Problem definition
 Development of a model
 Specification of an algorithm
 Designing an algorithm
 Checking the correctness of an algorithm
 Analysis of an algorithm
 Implementation of an algorithm
 Program testing
 documentation
Algorithms
 An algorithm is:
 A step-by-step procedure for transforming input
objects into output objects
 Executable instructions with sequence to reach a
specific goal/to accomplish a particular task
 Aim to reach a goal or solve a problem in a finite
amount of time
Algorithms

 Computer algorithm:
 To solve a problem with a set of steps which describe precisely how
the computer can run
 Steps of operations
 Performing calculation or
 Data processing or
 Automated reasoning tasks
 Best way to represent the solution of a particular problem in a very
simple and efficient way
 Independent from any programming languages
Algorithms

 All algorithms must satisfy the following:

 Input
 Output
 Definiteness: clear and unambiguous (not confuse) instruction
 Finiteness: It will stop after certain number of steps
 Effectiveness: each step of the algorithm is run correctly (must be
basic)
Example

 Assume we want to count the parameter of a square

 Algorithm is used to reach the goal
 Input: we need the length (l) and width (w) to count
 Output: the parameter
 Definiteness
 To obtain the parameter of a square, by mathematics, it is w+w+l+l
 The formula can be simplified as 2w+2l or 2(w+l)
 Clear Instruction:
 Get w and l
 Parameter = (2*w)+(2*l) or 2*(w+l)
 Print the output Parameter
 Finiteness: after it prints the output, the algorithm solves the problem
 Effectiveness
 Make sure each step is correct (basic)
 The algorithm produces what it is suppose to produce (feasible)
 E.g. of not feasible: the formula is w+l, and the generated output does
not solve the parameter problem
 Quiz

 Given a task: to make a cup of tea

 Arrange the given steps in a proper sequence to form a proper
algorithm:

 Add sugar and serve it in cup How about the procedures to

 Boil water enter supermarket during
RMCO/MCO?
 Put in tea leaves
 Add water and milk to the cup
Expectation from an Algorithm

 Correctness
 Must produce correct result
 If cannot avoid the error/wrong result, need to know the possible
failures, for e.g.: 1 failure out of 1000 execution
 Approximation algorithm: optimization solution is available when exact
solution is not found
 Less resource
 Time
 space
Design and Analysis VS Analysis and Design

 Design and analysis

 To accomplish a task, a solution is needed
 Thus, design an algorithm to solve the problem
 For e.g.: to find the maximum element among a set of elements
 Design an algorithm that receive an array
 Let the Max equal to the first element in the array
 Search the others elements in the array that is greater than Max and replace it
 Otherwise, continue to search
 Analyse the time complexity of the algorithm
 Analysis and design
 To develop a software for client but they have existing hardware
 Analyse the capability of the existing hardware
 Use appropriate algorithm to solve the problem
 General approach is: design and analysis
 However, analysis and design is more practical and implementable
Process of Algorithm Design and Analysis

Understand the problem

Decide on:
Computational means, exact
vs. approximate solving,
algorithm design technique

Design an algorithm

Prove its correctness

Analyse the algorithm

Code the algorithm

Source: The Design & Analysis of Algorithms, A. Levitin, 2012

Algorithm Design

 Different approaches can be followed

 Efficient:
 Time consumption
 Memory space
 Impossible to optimize both time and memory together
 If lesser time is required to run algorithm then more memory is
needed
 To run with lesser memory, more time is needed
 Good algorithm designer: have knowledge of algorithmic techniques,
creativity, imagination
Why Analyse Algorithm?

 Classify problems and algorithms by difficulty

 Predict performance, compare algorithms, tune parameters
 Algorithm may not be efficient
 Running time is a problem if apply on big data set
 How much time for the execution
 In terms of application, for e.g., can I give a call before my phone runs
out of battery power?
 To know which algorithm is better?
 Better understand and improve implementations and algorithms
 Good algorithm analyst requires this skills: logic, care, mathematical
ability
Running Time of Algorithms

 Commonly the primary

measure is time 120

 Factors affect the real running 100

time could be:

Running Time
80

 Speed of the computer 60

 Programming language 40

 Skill of the programmer 20

 Different data set 0

1000 2000 3000 4000
 The steps/formula in the Input Size
algorithm etc best case average case worst case

 The running time of an

algorithm typically grows with
the input size.
Running Time of Algorithms

 Faster computer A  Slower computer B

 Running time grows n^2  Running time grows
nlogn
 The computer can
execute 10 billion  The computer can
execute 10 million
instructions per second instructions per second
(1000 times faster than (slower than A)
B)  Use an inefficient
 Use a good compiler that needs 50n
programming language log n to sort n input
which required 2n^2 to numbers
sort n input numbers  Time:
∗ ∗
 Time: 1163s =
∗( )
=20000s=5.5 hrs 30 minutes
An algorithm that is well designed do
have a great impact!!!
Running Time Analysis

 There are three different cases of analysis:

 Worst-case analysis
 Look at the worst possible scenario
 Best-case analysis
 Look at the ideal case
 Not useful (most of the time)
 Average-case analysis
 Probability distribution should be known (always unknown in real life)
 Often difficult to determine
 We focus on the worst case running time.
 Easier to analyze
 Gives a guaranteed upper bound on the running time for any input
 Crucial to applications such as games, finance and robotics

 To know the relationship between running time and the size of input n, we can use these techniques:
 Experimental Studies
 Theoretical Analysis
Experimental Studies

 Write a program 9000

implementing the algorithm
8000
 Run the program with 7000
inputs of varying size and
6000
composition

Time (ms)
5000
 Use programming 4000
language’s time method to
get an accurate measure of 3000

the actual running time 2000

 For e.g.: Sys.time() in R 1000

programming 0
0 50 100
 Plot the results Input Size
Limitations of Experimental Studies

 It is necessary to implement the algorithm, which may be difficult

 Results may not be indicative of the running time on other inputs not
included in the experiment.
 In order to compare two algorithms, the same hardware and software
environments must be used
The measure is not
independent
Theoretical Analysis
 Known as complexity of an algorithm
 Uses a high-level description of the algorithm
(pseudocode) instead of an implementation
 Characterizes running time as a function of the input
size, n.
 Look at the number of operations
 Advantages:
 Takes into account all possible inputs
 Allows us to evaluate the speed of an algorithm
independent of the hardware/software environment