COMP6049001

Algorithm Design and Analysis

Introduction to Algorithm, Analyzing and Designing Algorithms

Week 1, Session 1&2

Maria Seraphina Astriani

[email protected]
 Hello World
// Waw C++ ^_^

#include <iostream>

int main() {
std::cout << "Hello World! I am Sera (D3697)";
std::cout << "[email protected]";
return 0;
}
 Materials
• COMP6049001-Algorithm Design and Analysis

• https://binusianorg-
my.sharepoint.com/personal/seraphina_binus_ac_id/_layouts/15/gu
estaccess.aspx?share=EmrQ7rireUtIh2_4F-
3YG4UBCvgbEDtpFg9ubigTSBQF4A&e=hyKZoF
 Materials
• COMP6049001-Algorithm Design and Analysis

• https://binusianorg-
my.sharepoint.com/personal/seraphina_binus_ac_id/_layouts/15/gu
estaccess.aspx?share=EmrQ7rireUtIh2_4F-
3YG4UBCvgbEDtpFg9ubigTSBQF4A&e=hyKZoF

• http://astriani.com/ADA
Algorithm Design
and Analysis???

What does it mean?

 Learning Outcomes
• LO 1. Explain the most important algorithms used in various common
computer science applications
• LO 2. Apply algorithmic design techniques and methods to analyse
the performance of algorithms
• LO 3. Apply efficient algorithm design techniques and understand the
limitations of algorithms
Text and Other Resources
Text
• Cormen, T. H., Leiserson, C. H., Rivest, R. L. & Stein, C.
(2022). Introduction to algorithms (4th ed.).
Massachusetts Institute of Technology. ISBN: 978-
0262046305, 026204630X [T1]

Other Resources
• Sedgewick, R. (2002). Algorithms in C++. Addison-
Wesley Publishing Company. ISBN: 9780201510591
• Skiena, S. S. (1998). The Algorithm Design Manual.
Springer Science & Business Media. ISBN: 0387948600,
9780387948607
• Relevant academic papers
 Schedule
Week Topics References
1. • Introduction to Algorithm [T1] Ch. 1, 2
• Analyzing and Designing Algorithms
2. • Asymptotic Notations [T1] Ch. 3
3. • Divide and Conquer [T1] Ch. 4
4. • Sorting and Order Statistics [T1] Ch. 6, 7, 8, 9
5. • Data structures [T1] Ch. 10, 11, 12, 13
6. • Dynamic Programming [T1] Ch. 14
7. • Project Proposal Presentation N/A
8. • Greedy Algorithms [T1] Ch. 15
9. • Graph Algorithms [T1] Ch. 20, 21, 22
10. • Project Milestone Presentation N/A
11. • String Matching [T1] Ch. 32
12. • Summary/Quiz N/A
13. • Project Presentation N/A
 Assessment

No. Components Percentage Course Intended

Learning Outcomes
1. Assignments 20 % LO 1, LO 2, LO 3
2. Quiz 20 % LO 1, LO 2, LO 3
3. Final project 30 % LO 1, LO 2, LO 3
4. Final examination 30 % LO 1, LO 2, LO 3
Total 100 %
Project
Project presentation + demo +
documentation

Programming language? It’s up to you J

• Implement algorithm(s) to solve real life

problem (effective and efficient
algorithm)
Or
• Compare algorithms (cost, running time,
etc.)
 Group
• Max. 3 students
Q&A
 Session Learning Outcomes
Upon successful completion of this course, students are expected to be
able to:
• LO 1. Explain the most important algorithms used in various common
computer science applications
 Topics
• Introduction to Algorithm
• Analyzing and Designing Algorithms
Introduction to Algorithm
Algorithms
• What are algorithms?

• Why is the study of algorithms

worthwhile?

• What is the role of algorithms relative

to other technologies used in
computers?

What is an Algorithm_.mp4
 Algorithms
• Informally, an algorithm is any well-defined
computational procedure that takes some
value, or set of values, as input and
produces some value, or set of values, as
output.

• An algorithm is thus a sequence of

computational steps that transform the
input into the output.

http://www.digitaltechnologieshub.edu.au/images/default-source/topics/tile-images/algorithm.jpg
 Algorithms
• Algorithm as a tool for solving a well-specified computational
problem

• For example
• given the input sequence (31; 41; 59; 26; 41; 58),
• a sorting algorithm returns as output the sequence (26; 31; 41; 41; 58; 59)

• Such an input sequence is called an instance of the sorting problem.

• In general, an instance of a problem consists of the input (satisfying whatever
constraints are imposed in the problem statement) needed to compute a solution to
the problem.

http://www.trycomputing.org/sites/default/files/webform/upload/jumbled_numbers.jpg
 Algorithms
• An algorithm is said to be correct if, for every input instance, it halts
with the correct output
• We say that a correct algorithm solves the given computational
problem
• An incorrect algorithm might not halt at all on some input instances, or it
might halt with an incorrect answer

Contrary to what you might

expect, incorrect
algorithms can sometimes
be useful, if we can control
their error rate.

http://www.coolthings.com/wp-content/uploads/2011/07/wrongulator1.jpg
 Algorithms
• An algorithm can be specified in English, as a computer program, or
even as a hardware design.
 What kinds of problems are solved by
algorithms?
• Sorting is by no means the only computational problem for which
algorithms have been developed.

• Practical applications of algorithms are ubiquitous and include the

following examples:
• Biological problems – DNA
• Internet – finding good routes on which the data will travel
• E-commerce – privacy of personal information (credit card, password, bank
statements, etc.) cryptography and digital signature
• Manufacturing / commercial enterprises – allocate scarce resource, to
maximize its expected profit
 What kinds of problems are solved by
algorithms?
• Not every problem solved by algorithms has an easily identified set of
candidate solutions.

• They have many candidate solutions, the overwhelming majority of

which do not solve the problem at hand.
• Finding one that does, or one that is “best,” can present quite a
challenge.

• Many algorithms -> find the efficient one

Characteristics of an Algorithm

https://www.geeksforgeeks.org/introduction-to-algorithms/
 Group Discussion
Mentimeter

https://www.geeksforgeeks.org/introduction-to-algorithms/

• Please list down and explain the characteristics of an algorithm (you

may add more characteristics J)
https://www.educba.com/types-of-algorithms/
 Techniques
• You can use our course - reference book as a “cookbook” for
algorithms

• But……. you may someday encounter a problem for which you cannot
readily find a published algorithm

• This course will teach you techniques of algorithm design and analysis
so that you can develop algorithms on your own, show that they give
the correct answer, and understand their efficiency

https://flashdba.files.wordpress.com/2012/03/cookbook.jpg
 Hard problems
• Most of our course discussed about efficient algorithms.

• Our usual measure of efficiency is speed,

• i.e., how long an algorithm takes to produce its result.

• There are some problems, however, for which no efficient solution is

known (which are known as NP-complete)
• Longer computation time – make the solution better
 Parallelism
• In order to elicit the best performance from multicore computers, we
need to design algorithms with parallelism in mind

• “Multithreaded” algorithms, which take advantage of multiple cores

• This model has advantages from a theoretical standpoint, and it forms the
basis of several successful computer programs, including a championship
chess program.
 Please Guess! Mentimeter

• What other measures (other than speed) of efficiency

might one use in a real-world setting?
 https://visualign.files.wordpress.com/2012/07/shortestpath1.png

Do a Research
(you may discuss with your friends)

1. How are the shortest-path and traveling-salesman problems given

above similar?
2. How are they different?
 Do a Research - Answers
(you may discuss with your friends)

1. How are the shortest-path and traveling-salesman problems given above

similar?
2. How are they different?

Answers:
1. They are similar, because each of then has to walk a graph and find a
path in them.
2. The difference is the constraint on the solution.
• The shortest-path requires just a path between two points
• The traveling salesman requires a path between more points that returns to the first
point.
 Algorithms as a technology
• Suppose computers were infinitely fast and computer memory was
free.
• Would you have any reason to study algorithms?

You would probably want your implementation to be

within the bounds of good software engineering practice

(for example, your implementation should be well

designed and documented), but you would most often use
whichever method was the easiest to implement.

https://mir-s3-cdn-cf.behance.net/project_modules/disp/9fe5e714270143.56280a63e1ee4.png
 Algorithms as a technology
• Computing time is therefore a
bounded resource, and so is space
in memory.

• You should use these resources

wisely, and algorithms that are
efficient in terms of time or space
will help you do so.
 Efficiency
• Different algorithms devised to solve the same problem often differ
dramatically in their efficiency.
• These differences can be much more significant than differences due
to hardware and software.

http://www.qsys-ea.com/images/increased-efficiency.jpg
 https://ds055uzetaobb.cloudfront.net/image_optimizer/22630368cbc032ea43967b2610e73ded399e22a4.png

Efficiency
Example:
• Sort
• Bubble sort
• Insertion sort
• Merge sort
• Etc.
Radix sort
• Computer – sort an array of 10 million numbers Recall: Bubble Sort.mp4
• Computer A (faster computer) -> more efficient
Code: Bubble Sort
• Computer B (slower computer)
Psssttt… you can use
https://www.onlinegdb.com/
 Algorithms and other technologies
• Computer hardware as a technology

• Total system performance depends on choosing efficient algorithms

as much as on choosing fast hardware.
 Algorithms and other technologies
• You might wonder whether algorithms are truly that important on
contemporary computers in light of other advanced technologies,
such as
• advanced computer architectures and fabrication technologies,
• easy-to-use, intuitive, graphical user interfaces (GUIs),
• object-oriented systems,
• integrated Web technologies, and
• fast networking, both wired and wireless.
A lth o u g h so m e applications
.
The answer is yes a lg o ri th m ic c o ntent at
requir e
do not explicitly u c h a s so m e simple,
v e l (s
the application le
a se d a pp lic a ti o ns), many do.
Web-b
 Algorithms and other technologies
• Algorithms are at the core of most technologies used in
contemporary computers.
• Furthermore, with the ever-increasing capacities of computers, we
use them to solve larger problems than ever before.

Having a solid base of algorithmic

knowledge and technique is one
characteristic that separates the truly
skilled programmers from the novices.
With modern computing technology, you can
accomplish some tasks without knowing much
about algorithms, but with a good background in
algorithms, you can do much, much more.

https://media.licdn.com/mpr/mpr/p/6/005/08c/33f/1b25a7a.jpg
 Question
(you may googling ^_^)

• Give an example of an
application that requires
algorithmic content at the
application level, and discuss
the function of the
algorithms involved.
 Answer
• Give an example of an application that requires algorithmic content at
the application level, and discuss the function of the algorithms
involved.

• Google Maps when finding a route between two places.

• The algorithms are an essential part of this use case, since the route is
what the user cares for the most.
Analyzing and Designing
Algorithms
 Big O Notation
• “When we talk about algorithms, we are interested in their
performance. But instead of expressing an algorithm’s performance in
seconds or minutes, we use the Big-O notation.”

https://www.youtube.com/watch?v=biAB5iT3NXE
What is Big O Notation.mp4
 Big O Notation

https://static1.squarespace.com/static/506e28cee4b04973cff61716/t/518eb7a0e4b0d03df1fe0cbc/1368307617566/Big+O+Notation+Summary.jpg
http://javaconceptoftheday.com/wp-content/uploads/2015/05/Big-O-Notation.png
How To “Calculate” Your Code Growth Rate?

• Resource: Algorithms for dummies

(Part 1)_ Big-O Notation and Sorting
_ Adrian Mejia Blog
https://adrianmejia.com/algorithms-
for-dummies-part-1-sorting/
 Insertion Sort
• Insertion Sort -> an efficient
algorithm for sorting a small
number of elements.
• The numbers that we wish to sort
are also known as the keys.
• Although conceptually we are
sorting a sequence, the input
comes to us in the form of an
array with n elements. Sorting a hand of cards
using insertion sort.

• Describe algorithms as programs

written in a pseudocode
Insertion Sort

Notes - Pseudocode:
The symbol “//” indicates that the
remainder of the line is a comment.

Insertion sort in 2 minutes.mp4

Code: Insertion Sort

 Insertion Sort - Exercise
• Illustrate the operation of INSERTION-SORT on the array
A = (31, 41, 59, 26, 41, 58)
Insertion Sort - Exercise
 Searching Algorithm
• Linear search
• Binary search
• Sequential search
• Exponential search
• Depth-first search (DFS)
• Breadth-first search (BFS)
• Etc.
 Analyzing Algorithms
• Analyzing an algorithm has come to mean predicting the resources
that the algorithm requires.

• Occasionally, resources such as memory, communication bandwidth,

or computer hardware are of primary concern, but most often it is
computational time that we want to measure.
We shall assume a generic one-processor, random-access
machine (RAM) model of computation as our implementation
technology and understand that our algorithms will be
implemented as computer programs. In the RAM model,
instructions are executed one after another, with no concurrent
operations. Insertion Sort vs Bubble Sort
+ Some analysis.mp4
 Analyzing Algorithms
• The RAM model contains instructions commonly found in real
computers:
• arithmetic (such as add, subtract, multiply, divide, remainder, floor,
ceiling),
• Data movement (load, store, copy), and
• control (conditional and unconditional branch, subroutine call and
return).

• Each such instruction takes a constant amount of time

 Analyzing Algorithms
• Example
• When working with inputs of size n, we typically assume that integers are
represented by c lg n bits for some constant c >= 1.
• In most computers, shifting the bits of an integer by one position to the left is
equivalent to multiplication by 2, so that shifting the bits by k positions to the
left is equivalent to multiplication by 2 k
 Analysis of Insertion Sort

The running time of the algorithm is the sum of running

times for each statement executed; a statement that takes ci
steps to execute and executes n times will contribute cin to
the total running time
 Analysis of Insertion Sort

• To compute T(n), the running time of

INSERTION-SORT on an input of n
values, we sum the products of the
cost and times columns, obtaining
 Analysis of Insertion Sort

• In the worst case, the running time of INSERTION-

SORT is
 Worst-case and average-case analysis
• The worst-case running time of an algorithm gives us an upper bound
on the running time for any input. Knowing it provides a guarantee
that the algorithm will never take any longer.
• The “average case” is often roughly as bad as the worst case. The
resulting average-case running time turns out to be a quadratic
function of the input size, just like the worst-case running time.
 Further reading
• Order of growth
• Worst-case running time of (pronounced “theta of n-squared”)
 Discussion
• How can we modify almost any algorithm to have a good
best-case running time?
 Discussion
• How can we modify almost any algorithm to have a good
best-case running time?

We can modify it to handle the best-case efficiently.

For example, if we modify merge-sort to check if the array is

sorted and just return it, the best-case running time will be Θ(n).
 Designing algorithms
• We can choose from a wide range of algorithm design techniques.
• For insertion sort, we used an incremental approach

• We examine an alternative design approach, known as “divide and-

conquer”
 The divide-and-conquer approach
Teaser. More
• Many useful algorithms are recursive in structure discussion on week 3

• These algorithms typically follow a divide-and-conquer approach:

they break the problem into several sub-problems that are similar to
the original problem but smaller in size, solve the sub-problems
recursively, and then combine these solutions to create a solution to
the original problem.
 The divide-and-conquer approach
• Involves three steps at each level of the recursion

1. Divide the problem into a number of sub-problems that are smaller

instances of the same problem.
2. Conquer the sub-problems by solving them recursively. If the sub-
problem sizes are small enough, however, just solve the sub-
problems in a straight-forward manner.
3. Combine the solutions to the sub-problems into the solution for the
original problem
Analyzing divide-and-conquer
algorithms
 Discussion
• What is bubble sort
• Big O Notation of bubble sort
• Pseudocode of bubble sort

• Illustrate the operation of insertion sort and bubble sort on the array
A = (3, 41, 52, 26, 38, 57, 9, 49)
Which one is faster?

• Illustrate the operation of insertion sort and bubble sort on the array
A = (3, 9, 26, 29, 38, 49, 52, 49)
Which one is faster?
 References
• Cormen, T. H., Leiserson, C. H., Rivest, R. L. & Stein, C. (2022).
Introduction to algorithms (4th ed.). Massachusetts Institute of
Technology. ISBN: 978-0262046305, 026204630X

• Sedgewick, R. (2002). Algorithms in C++. Addison-Wesley Publishing

Company. ISBN: 9780201510591

• Skiena, S. S. (1998). The Algorithm Design Manual. Springer Science &

Business Media. ISBN: 0387948600, 9780387948607
Thank you