CSE-243: Algorithm Design and Analysis

 L01_30.09.2018_Introduction
Instructor Teaching Assistant
Md. Sabir Hossain
 Abdullah 1
Lecturer
Id: 1604
Dept. of CSE, CUET
 Abdullah 2
Mobile: 01737143868
Email: [email protected] Id: 1604

Acknowledgment: Most of the content of this slide is adopted from the book: Introduction to Algorithms, Third Edition
(International Edition), Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, Clifford Stein, ISBN-13: 978-0262033848 and
Algorithms, 4th Edition by Robert Sedgewick and Kevin Wayne, Addison-Wesley Professional, 2011, ISBN 0-321-57351-X.
 Today’s Target
• Expected Outcomes of CSE-243
• Topics Covered in Data Structure Course
• Text and Reference Books
• Introduction & definition
• Why Study Algorithm?
• Algorithms categories & types
• Pseudo-code
• Designing an algorithm
– Example: Max
• Analysis of algorithms
• Running time (Kind of Analysis)
 Expected Outcomes of CSE-243
• Student will able to
– Design Algorithms to solve any Computational Problems (by class
lectures and performing assignments)
– Analyze the proposed Algorithms by Standard Way
(Mathematically) (by class lectures and assignments)
– Compare alternative solutions to any specific problems to
select the best one (by class lectures)
– Implement Algorithms to solve Competitive Programming
Challenges (by practice and contest lab)
– Discover new era of Algorithmic implementation (by doing capstone
projects)
– Modify/Improve/Propose effective and efficient algorithms to
existing ones (by research work)
Prerequisites: CSE-241 : Data Structure

 Concepts and examples  Graphs,

of elementary data  heaps,
objects,
 Btrees,
 elementary data
structures,  Fibomacci heaps,

 array,  Recursion,

 stacks and  Memory management,

 queues.  Sorting and searching,

 Lists,  hash techniques.

 Trees,

Md. Sabir Hossain, Lecturer, Dept. of CSE, CUET 9/30/2018 4

Textbooks
 Algorithms, 4th  Introduction to  Fundamentals of Computer
Edition by Robert Algorithms, Third Algorithms, Second Edition,
Sedgewick and Edition (International Ellis Horowitz, Sanguthevar
Kevin Wayne, Edition), Thomas H. Rajasekaran, Sartaj Sahni, ISBN
Addison-Wesley Cormen, Charles E. 10: 8173716129 / ISBN 13:
Professional, Leiserson, Ronald L. 9788173716126, Published by
2011, ISBN 0-321- Rivest, Clifford Stein, Universities Press/Orient
57351-X. ISBN-13: 978- BlackSwan, 2008.
0262033848.

Md. Sabir Hossain, Lecturer, Dept. of CSE, CUET 9/30/2018 5

Book Site

 Lecture slides.
 Download code.
 Summary of content.

http://algs4.cs.princeton.edu

Md. Sabir Hossain, Lecturer, Dept. of CSE, CUET 9/30/2018 6

Ref. Books

 (Lecture Notes Series on Computing) M H Alsuwaiyel- Algorithms Design

Techniques and Analysis -World Scientific Publishing Company (2016)
 Anany Levitin, Introduction to the Design and Analysis of Algorithms, 3rd
Edition
 Steven S Skiena-The Algorithm Design Manual-Springer (2010)

Md. Sabir Hossain, Lecturer, Dept. of CSE, CUET 9/30/2018 7

 Computer Science is About Problem Solving
• A Problem is specified by
1. The givens (a formulation)
• A set of objects
• Relations between them
2. The query
• The information one is trying to extract from the formulation, the question to
answer
• An algorithm is a method or procedure that solves instances of a problem
• ‘Algorithm’ is a distortion of Al-Khawarizmi, a Persian mathematician

Real World  Computing World

Objects represented by… data Structures, ADTs, Classes
Relations implemented with… relations & functions (e.g., predicates)
Actions Implemented with… algorithms: a sequence of instructions
 Algorithms: Formal Definition
• Definition: An algorithm is a sequence of unambiguous
instructions for solving a problem.
• Properties of an algorithm
– Finite: the algorithm must eventually terminate
– Complete: Always give a solution when one exists
– Correct (sound): Always give a correct solution
• For an algorithm to be an acceptable solution to a problem, it
must also be effective. That is, it must give a solution in a
‘reasonable’ amount of time
• Efficient= runs in polynomial time. Thus, effective efficient
• There can be many algorithms to solve the same problem
Why study algorithms?

Their impact is broad and far-reaching.

Internet. Web search, packet routing, distributed file sharing, ...

Biology. Human genome project, protein folding, ...
Computers. Circuit layout, file system, compilers, ...
Computer graphics. Movies, video games, virtual reality, ...
Security. Cell phones, e-commerce, voting machines, ...

Multimedia. MP3, JPG, DivX, HDTV, face recognition, ...

Social networks. Recommendations, news feeds, advertisements, ...
Physics. N-body simulation, particle collision simulation, ...

Md. Sabir Hossain, CSE, CUET

Why study algorithms?

Old roots, new opportunities. 300 BCE

・Study of algorithms dates at least to Euclid.

・Formalized by Church and Turing in 1930s.
・Some important algorithms were discovered
by undergraduates in a course like this!

1920s
1930s
1940s
1950s
1960s
1970s
1980s
1990s
2000s
Md. Sabir Hossain, CSE, CUET
Why study algorithms?

To solve problems that could not otherwise be addressed.

Ex. Network connectivity.

Md. Sabir Hossain, CSE, CUET

Why study algorithms?

For intellectual stimulation.

F RO M T H E
E D I T O RS

THE JOY OF ALGORITHMS

“ For me, great algorithms are the poetry of computation. Just like Francis Sullivan, AssociateEditor-in-Chief

T HE THEME OF THIS FIRST-OF-THE-CENTURY ISSUE OF COMPUTING IN

SCIENCE & ENGINEERING IS ALGORITHMS. IN FACT, W E WE RE BOLD

ENOUGH—AND PERHAPS FOOLISH ENOUGH—TO CALL THE 10 EXAMPLES WE’VE SE-

L E C TE D “THE TOP 10 ALGORITHMS OF THE CENTURY.”

verse, they can be terse, allusive, dense, and even mysterious. Computational algorithms are probably as old as civilization. mysterious. But once unlocked, they cast a brilliant new light
Sumerian cuneiform, one of the most ancient written records, on some aspect of computing. A colleague recently claimed
consists partly of algorithm descriptions for reckoning in base that he’d done only 15 minutes of productive work in his
60. And I suppose wecould claim that the Druid algorithm for whole life. He wasn’t joking, because he was referring to the
estimating the start of summer is embodied in Stonehenge. 15 minutes during which he’d sketched out a fundamentalop- (That’s
really hard hardware!) timization algorithm. He regarded the previous years of Like
so many other things that technology affects, algo- thought and investigation as a sunk cost that might or might
rithms have advanced in startling and unexpected ways in the not have paid off.
20th century—at least it looks that way to us now. The algo- Researchers have cracked many hard problems since 1 Jan- rithms
we chose for this issue have been essential for progress uary 1900, but we are passing some even harder ones on to the in
communications, health care, manufacturing, economics, next century. In spite of a lot of good work, the question of weather

But once unlocked, they cast a brilliant new light on some

prediction, defense, and fundamental science. Con- how to extract information from extremely large masses of versely,
progress in these areas has stimulated the search for data is still almost untouched. There are still very big chal- ever-better
algorithms. I recall one late-night bull session on lenges coming from more “traditional” tasks, too. For exam- the Maryland
Shore when someone asked, “Who first ate a ple, we need efficient methods to tell when the result of a large crab? After all, they
don’t look very appetizing.’’ After the usualfloating-point calculation is likely to be correct. Think of the speculations about the
observed behavior of sea gulls, someone way that check sums function. The added computational cost gave what must be the right
answer—namely, “A very hungry is very small, but the added confidence in the answer is large. person first ate a crab.”
Is there an analog for things such as huge, multidisciplinary
The flip sideto “necessity is the mother of invention’’ is “in- optimizations? At an even deeper level is the issue of reason-
vention creates its own necessity.’’ Our need for powerful ma- able methods for solving specific cases of “impossible’’ prob-

aspect of computing. ” — Francis Sullivan

chines always exceeds their availability. Each significant com- lems. Instances of NP-complete problems crop up in at- putation
brings insights that suggest the next, usually much tempting to answer many practical questions. Are there
larger, computation to be done. New algorithms are an attempt efficient ways to attack them?
to bridge the gap between the demand for cycles and the avail- I suspect that in the 21st century, things will be ripe for an- able
supply of them. We’ve become accustomed to gaining the other revolution in our understanding of the foundations of Moore’s
Law factor of two every 18 months. In effect, Moore’s computational theory. Questions already arising from quan- Law changes
the constant in front of the estimate of running tum computing and problems associated with the generation time as a function
of problem size. Important new algorithms of random numbers seem to require that we somehow tie to- do not come along
every 1.5 years, but when they do, they can gether theories of computing, logic, and the nature of the change the exponent of
the complexity! physical world.
For me, great algorithms are the poetry of computation. The new century is not going to be very restful for us, but it Just like
verse, they can be terse, allusive, dense, and even is not going to be dull either!

2 C O MPUTIN G IN SCIEN CE & EN GINEERIN G

“ An algorithm must be seen to be believed. ” — Donald Knuth

Md. Sabir Hossain, CSE, CUET

Why study algorithms?

To become a proficient programmer.

“ I will, in fact, claim that the difference between a bad programmer

and a good one is whether he considers his code or his data structures
more important. Bad programmers worry about the code. Good
programmers worry about data structures and their relationships. ”
— Linus Torvalds (creator of Linux)

“ Algorithms + Data Structures = Programs. ” — Niklaus Wirth

Md. Sabir Hossain, CSE, CUET

Why study algorithms?

They may unlock the secrets of life and of the universe.

Computational models are replacing math models in scientific inquiry.

for (double t = 0.0; true; t = t + dt)

E.  mc 2 for (int i = 0; i < N; i++)
Gm1 m2
F  {
F.  ma r2 bodies[i].resetForce();
for (int j = 0; j < N; j++)
 ! 2 2  if (i != j)
   V (r)(r)  E (r) bodies[i].addForce(bodies[j]);
 2m  }

20th century science

21st century science
(formula based)
(algorithm based)

“ Algorithms: a common language for nature, human, and computer. ” — Avi Wigderson

Md. Sabir Hossain, CSE, CUET

Why study algorithms?

For fun and profit.

Md. Sabir Hossain, CSE, CUET

Why study algorithms?

・Their impact is broad and far-reaching.

・Old roots, new opportunities.
・To solve problems that could not otherwise be addressed.
・For intellectual stimulation.
・To become a proficient programmer.
・They may unlock the secrets of life and of the universe.
・For fun and profit.

Why study anything else?

Md. Sabir Hossain, CSE, CUET

 Algorithms: General Techniques
• There are many broad • Some general styles of
categories of algorithms algorithms include
– Deterministic versus – Brute force (enumerative
techniques, exhaustive search)
Randomized (e.g.,
– Divide & Conquer
Monte-Carlo)
– Transform & Conquer
– Exact versus (reformulation)
Approximation – Greedy Techniques
– Sequential/serial versus – Backtracking
Parallel, etc. – Dynamic Programming

Md. Sabir Hossain, CSE, CUET

 Good Pseudo-Code: Example
Intersection
Input: Two finite sets A, B
Output: A finite set C such that C = A  B
1. C0
2. If |A|>|B|
3. Then Swap(A,B)
4. End
5. For every x  A Do
6. If x  B
7. Then C  C  {x} Union(C,{x})
8. End
9. End
10. Return C
 Algorithms: Pseudo-Code
• Algorithms are usually presented using pseudo-code
• Bad pseudo-code
– gives too many details or
– is too implementation specific (i.e., actual C++ or Java code or giving
every step of a sub-process such as set union)
• Good pseudo-code
– Is a balance between clarity and detail
– Abstracts the algorithm
– Makes good use of mathematical notation
– Is easy to read and
– Facilitates implementation (reproducible, does not hide away
important information)
 Writing Pseudo-Code: Advice
• Input/output must properly defined
• All your variables must be properly initialized, introduced
• Variables are instantiated, assigned using 
• All `commands' (while, if, repeat, begin, end) bold face \bf
For i  1 to n Do
• All functions in small caps Union(s,t) \sc
• All constants in courier: pi  3.14 \tt
• All variables in italic: temperature  78 (\it, \em)
• LaTeX: Several algorithm formatting packages exist on WWW
 Designing an Algorithm
• A general approach to designing algorithms is as follows
– Understanding the problem, assess its difficulty
– Choose an approach (e.g., exact/approximate, deterministic/
probabilistic)
– Choose appropriate data structures
– Choose a strategy
– Prove
1. Termination
2. Completeness
3. Correctness/soundness
– Evaluate complexity
– Implement and test it
– Compare to other known approach and algorithms
 Algorithm Example: Max
• When designing an algorithm, we usually give a
formal statement about the problem to solve
• Problem
– Given: a set A={a1,a2,…,an} of integers
– Question: find the index i of the maximum integer ai
• A straightforward idea is
– Simply store an initial maximum, say a1
– Compare the stored maximum to every other integer in A
– Update the stored maximum if a new maximum is ever
encountered
 Pseudo-code of Max
Max
Input: A finite set A={a1,a2,…,an} of integers
Output: An index i such that ai = max{a1,a2,…,an}
1. index  1
2. For i =2 to n Do
3. If ai > aindex
4. Then index  i
5. End
6. End
7. Return aindex
 Tools: Pseudo Code to Flow Chart

• https://code2flow.com/app
 Tools: Flow Chart to Pseudo Code

• http://larp.marcolavoie.ca/en/ [Must Install and write Pseudo

Code]
 Written Assignment-01: Design Algorithm (write
pseudo code) for the following problems:
• Must use http://larp.marcolavoie.ca/en/ to perform
assignment
• Will be posted in Classroom
 Analysis of algorithms
The theoretical study of computer-program
performance and resource usage.
What’s more important than performance?
• modularity • user-friendliness
• correctness • programmer time
• maintainability • simplicity
• functionality • extensibility
• robustness • reliability
 Why study algorithms and
performance?
• Algorithms help us to understand scalability.
• Performance often draws the line between what
is feasible and what is impossible.
• Algorithmic mathematics provides a language
for talking about program behavior.
• Performance is the currency of computing.
• The lessons of program performance generalize
to other computing resources.
• Speed is fun!
 The problem of sorting

Input: sequence a1, a2, …, an of numbers.

Output: permutation a'1, a'2, …, a'n such

that a'1  a'2  …  a'n .

Example:
Input: 8 2 4 9 3 6
Output: 2 3 4 6 8 9
Example of insertion sort
8 2 4 9 3 6
Example of insertion sort
8 2 4 9 3 6
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
Example of insertion sort
8 2 4 9 3 6

2 8 4 9 3 6
2 4 8 9 3 6
Example of insertion sort
8 2 4 9 3 6

2 8 4 9 3 6
2 4 8 9 3 6
Example of insertion sort
8 2 4 9 3 6

2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
Example of insertion sort
8 2 4 9 3 6

2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
Example of insertion sort
8 2 4 9 3 6

2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
Example of insertion sort
8 2 4 9 3 6

2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
Example of insertion sort
8 2 4 9 3 6

2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
2 3 4 6 8 9 done
 Insertion sort
INSERTION-SORT (A, n) ⊳A[1 . . n]
for j ← 2 to n
do key ← A[ j]
i←j–1
“pseudocode” while i > 0 and A[i] > key
do A[i+1] ← A[i]
i←i–1
A[i+1] = key
 Insertion sort
INSERTION-SORT (A, n) ⊳A[1 . . n]
for j ← 2 to n
do key ← A[ j]
i←j–1
“pseudocode” while i > 0 and A[i] > key
do A[i+1] ← A[i]
i←i–1
A[i+1] = key
1 i j n
A:
key
sorted
 Running time

• The running time depends on the input:

an already sorted sequence is easier to
sort.
• Parameterize the running time by the size of
the input, since short sequences are easier
to sort than long ones.
• Generally, we seek upper bounds on the
running time, because everybody likes a
guarantee.
 Kinds of analyses
Worst-case: (usually)
• T(n) = maximum time of algorithm
on any input of size n.
Average-case: (sometimes)
• T(n) = expected time of algorithm
over all inputs of size n.
• Need assumption of statistical
distribution of inputs.
Best-case: (bogus)
• Cheat with a slow algorithm that
works fast on some input.
 Machine-independent time

What is insertion sort’s worst-case time?

• It depends on the speed of our computer:
• relative speed (on the same machine),
• absolute speed (on different machines).
BIG IDEA:
• Ignore machine-dependent constants.
• Look at growth of T(n) as n → ∞ .
“Asymptotic Analysis”
Insertion Sort Related Challenges
Insertion Sort Related Challenges
Insertion Sort Related Research

 2017: Bidirectional Conditional Insertion Sort algorithm; An efficient progress

on the classical insertion sort
 http://linkinghub.elsevier.com/retrieve/pii/S0167739X17301711
 https://arxiv.org/pdf/1608.02615.pdf

 2017: Based on this paper another work:

 Parallelization of Modified Merge Sort Algorithm
 http://www.mdpi.com/2073-8994/9/9/176/htm
More Research on Insertion Sort
More Research on Insertion Sort
More Research on Insertion Sort
 References
• http://www.unf.edu/~broggio/cop3530/3530pseu.htm
• https://visualgo.net/en/sorting
• https://www.cs.usfca.edu/~galles/visualization/Algorithms.html
• https://www.toptal.com/developers/sorting-algorithms/insertion-
sort
• https://www.geeksforgeeks.org/insertion-sort/
• Introduction to Algorithms, Third Edition (International Edition),
Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, Clifford
Stein, ISBN-13: 978-0262033848.
• https://www.hackerearth.com/practice/algorithms/sorting/inserti
on-sort/practice-problems/
• https://www.codechef.com/problems/SOPC03