REII 211 – Introduction to Algorithms

and Optimisation
Study Unit 1 – Introduction to Algorithms

Mr. Pieter Marx

13/02/2024
 Months February March April May June July

Days Date Event Date Event Date Event Date Event Date Event Date Event
Monday 1 RECESS 3 1

Tuesday 2 SU5 - LECTURE 7 4 2

Class Program
EXAM OPP 2
Wednesday 3 1 5 EXAM OPP 1 3
PRACTICAL SEMESTER TEST
Thursday 1 4 2 WEEK 6 4
PROBLEMS
RECESS
Friday 2 1 5 3 7 5 RECESS
Saturday 3 2 6 4 8 6
• 12 February – Class Begins Sunday 4 3 7 5 9 7
Monday 5 4 SU3 - LECTURE 5 8 SU5 - LECTURE 8 6 SU5 - LECTURE 10 10 8

• 22 February – Tutorial 1 (SU1 & SU2) Tuesday 6 5 SU3 - LECTURE 5 9 SU5 - LECTURE 8 7 SU5 - LECTURE 10 11 9
2nd
Wednesday 7 RECESS 6 10 8 12 EXAM OPP 1 10
• 07 March – Tutorial 2 (SU3) Thursday 8 7 TUTORIAL 2 11 TUTORIAL 4 9 SEMESTER TEST 3 13 11
SEMESTER

• 14 March – Semester Test 1 (L1-5) Friday 9 8 12 10 14 12

Saturday 10 9 13 11 15 13
Sunday 11 10 14 12 16 14
• 19 March – Tutorial 3 (SU4) Monday 12 WELCOME 11 SU4 - LECTURE 6 15 SU5 - LECTURE 9 13 17 15

• 21 Mar. – 01 Apr. – Recess Tuesday 13 SU1 - LECTURE 1 12 SU4 - LECTURE 6 16 SU5 - LECTURE 9 14 18
EXAM OPP 1
16
2nd
Wednesday 14 13 17 15 PRACTICAL DEMO 19 17 SEMESTER

• 02 April – Hand Out Practical (L9) Thursday 15

PRACTICAL
PROBLEMS
14 SEMESTER TEST 1 18
PRACTICAL
PROBLEMS
16 20 18

Friday 16 15 19 17 21 EXAM OPP 2 19

• 11 April – Tutorial 4 (SU5) Saturday 17 16 20 18 22 20
Sunday 18 17 21 19 23 21

• 22 Apr. – 03 May – MSA Week (L1-8) Monday 19 SU2 - LECTURE 2 18 SU4 - LECTURE 6 22 20 OPEN FOR HELP 24 22

Tuesday 20 SU2 - LECTURE 2 19 TUTORIAL 3 23 21 OPEN FOR HELP 25 23

• 09 May – Semester Test 3 Wednesday 21 20 24

SEMESTER TEST
WEEK 22 26 EXAM OPP 2 24
2nd
SEMESTER

(L1-9) Thursday 22 TUTORIAL 1 21

RECESS
25 23 OPEN FOR HELP 27 25

Friday 23 22 26 24 28 26
• 13 – 17 May – Demo Practical Saturday 24 23 27 SU4 - LECTURE 6 25 29 27
Sunday 25 24 28 26 30 28

• 28 May – 20 Jun. – 1st Opportunity Monday 26 SU3 - LECTURE 3 25 29

SEMESTER TEST
27 OPEN FOR HELP 29
WEEK
Exam
2nd
Tuesday 27 SU3 - LECTURE 4 26 30 28 30 SEMESTER
Wednesday 28 27 RECESS 29 31

• 21 Jun. – 04 Jul. – 2nd Opportunity Thursday 29

PRACTICAL
PROBLEMS
28 30
EXAM OPP 1

Exam Friday 29 31
Saturday 30 1
• 05 Jul. – Recess Sunday 31 2
Agenda
• What is an Algorithm?
• Optimisation of Algorithms and Heuristics
• Selecting the Right Jobs
• Reasoning about Correctness
• Induction and Recursion
• Modelling the Problem
STUDY UNIT 1 – Chapter 1 (Skiena, S. S.)
Study Outcomes
• Provide a Formal Definition of an Algorithm
• Describe What the Correctness of an Algorithm
Entails
• Evaluate the Correctness of a Provided Algorithm
• Construct and Implement an Algorithm to Address a
Real-World Problem

Fair Warning
This Study Unit May Feel a Bit Disjointed, But Is
Basically a Summary of All the Topics of the Module
 Think of Describing the Sets as:
or or , etc..
More Types:
What is an Algorithm? •
•
Numeric
Alphabetic
• Date Time

“An algorithm is a procedure to accomplish a specific task. An algorithm is

the idea behind any reasonable computer program.” – S. S. Skiena

Start with a General, Well-Specified Problem

• Describing the Complete Set of Instance Inputs, it Must Work On
• Describing the Complete Set of Instance Outputs, After Running on
One of These Instances
 Examples of General Algorithms:
• Sorting Algorithms

Example • Scheduling Algorithms

• Weighted Graph Traversal Algorithms
Context Examples:
• Streaming, Compression, Telecoms
• Route Planning
How to Specify a General Sorting Algorithm: • Delivery Management
• Manufacturing (Additive/Subtractive)
Problem: Sorting • 3D Modelling (CAD, Games, Movies)
• Render Shadows/Reflections/Light Arrays
Input: A Sequence of Keys • And Many More

Output: The Permutation (Reordering) of the Input Sequence Such that

When does a General Algorithm Become an Instance?:

• Sort a List of Only Numbers: {, , , , , }
• Sort a List of Only Names: {, , , , }, etc.
More than One Solution for an Algorithm
As in Life, There are Many Different Ways to Do the Same Thing
(In Coding, There are Many Different Ways to Write an Algorithm)
In Fact, You Will Learn to Implement Different Solutions for the Same
Algorithm (Thing You Want to Perform)
Take Sorting For Instance:
• Bubble Sort • Merge Sort
• Insertion Sort • Selection Sort
• Quick Sort • Heap Sort
• Bucket Sort • Etc.
Bubble Sorting Algorithm
Insertion Sorting Algorithm
Even Different Solutions for Insertion
Sorting
If You Would Act Out the Steps of
These Two Functions, You Will See
the Code is Written Differently, But
That the Ideas are Exactly the Same.
However, One is Superior…
Can You Guess Which One and Why?
Three Desirable Properties of Good
Algorithms
1. Correctness
Are the Answers Good Enough?
Always Returns the Desired Output for All Legal Instances of the Problem.

2. Efficiency
Does the Algorithm Slow the Bigger Picture Process Down too Much?
A Measure of the Average Execution Time Necessary for an Algorithm to
Complete Work on a Set of Data

3. Easily Implementable
Is the Complexity of the Algorithm Justifiable for Its Purpose?
Correctness of an Algorithm
• Def: Always Returns the Desired Output for All Legal Instances of the Problem.
• It is Seldom Obvious Whether an Algorithm Correctly Solves a Given Problem.
• It Might Work for Some or Most Instances, But Malfunction for other Instances.
• Usually, Not General/Well-Specified Enough (Think of Geeks For Geeks Code).
• Carefully Specifying Algorithm Problems is Key for Measuring Correctness.
• Correct Algorithms Usually Come with a Proof of Correctness:
An Explanation of Why We Know that the Algorithm Must Take Every Instance of
the Problem to the Desired Result
Takeaway
1. An algorithm is a procedure to accomplish a
specific task. An algorithm is the idea behind
any reasonable computer program

2. Formulizing a General, Well-Specified

Problem

3. The Desirable Properties of Good

Algorithms:
1. Correctness
2. Efficiency
3. Easily Implementable
 Answer: to Optimize:
Reasons
Optimisation of Algorithms • Increase
Stop at Last Node
• Increase
Route
Production
Direction.
and Reverse
Speed the
Throughput Before MTTF
and Heuristics • Basically,
What Problem Increase
Arises Then?
ROI and Profit

Robot Tour Optimization

Consider a Robot in an Industrial Chain that Needs to Visit a Set of
Coordinates/Vertices/Nodes and Return Back to the First Destination
(Creating a Loop) After Completion to Start All Over Again.
Why Would We Want to Optimize the Travel Distance of the Route?
How Can We Optimize the Route Even Further?
Where to Start Formulating the
Algorithm?
Specify a General Traversing Algorithm:
Problem: Robot Tour Optimization
Input: A Set of Points in the Plane
Output: What is the Shortest Cycle Tour that Visits Each Point in the Set ?
 This Makes it Wrong!!! I.e., NOT Correct! Only Works for Some Legal Input
Instances. Here, the Correct Point of Departure Will Solve the Problem.
However, Finding an Algorithm to Find Such Optimal Departure Point and
A Solution* Using it in Conjunction with the Nearest-Neighbor Algorithm is Not Simple.

The Nearest-Neighbor Heuristic: (Seldom the Most Efficient Algorithm,

However, One Of the Most Common Algorithms)
Starting from Some Point , We Walk First to its Nearest Neighbor . From ,
We Walk to its Nearest Unvisited Neighbor, thus Excluding only as a
Candidate. We Now Repeat this Process Until We Run out of Unvisited
Points, After Which We Return to to Close off the Tour.
A Revised Solution*
The Closest-Pair Heuristic
• Let’s Stop Restricting Ourselves to Go to the Nearest Neighbor
• Let’s Rather Connect the Closest Pair of Endpoints Whose Connection Will Not
Create a Problem, Such as Premature Termination of the Cycle.
1. Let Each Vertex Begin as Its Own Single Vertex Chain.
2. At any Step, We will have a Set of Single Vertices and the End of Vertex-
Disjoint Chains Available to Merge.
3. After Merging Everything Together, We Will End Up With a Single Chain
Containing All the Vertices in it.
4. Connecting the Final Two Endpoints Gives Us a Cycle Somewhat More Complicated
and Less Efficient that Previous
One, But at Least Gives the
Correct Answer in This Example
The Closest-Pair Heuristic (Continues)
It Is Not Always Correct!
Example: (l) is 20% Longer Travel Distance than (r)

The Correct Solution: Exhaustive Search

Consider All Permutations and Choose Best
 Such a Well-Known
Problem, We Refer to it as:
Traveling Salesman
The Correct Solution is Not Efficient! Problem (TSP)

Consider Even Just 20 Possible Orderings of 20 Points:

(Not Possible in a Day for a Good PC)

This Is Not Only Slow, But Not Pragmatic. Many Circuit Board Designs
Require Many More Nodes (Some even ).
“All of the world’s computers working full time wouldn’t come close to finishing
the problem before the end of the universe, at which point it presumably becomes
moot” – S. S. Skiena

• We Therefore Sometimes Sacrifice Correctness for Efficiency (Engineers).

• We Refer to Algorithms with This Trade-Off as Heuristics
Takeaway
1. There is a Fundamental Difference Between
Algorithms, Procedures that Always Produce a
Correct Result, and Heuristics, Which
May Usually Do a Good Job But Provide
No Guarantee of Correctness.
Selecting the Right Jobs
Let’s Play a Game:
You Want to Schedule As Many Movies to Act in as Possible, But You May Not Do Multiple
Movies at a Time.
Specify a General Scheduling Algorithm:
Problem: Movie Scheduling Problem
Input: A Set of Intervals on the Line
Output: What is the Largest Subset of Mutually Non-Overlapping Intervals that can be
Selected from ?

Optional Solutions*
• Earliest Job First
• Shortest Job First
• Exhaustive Scheduling
• Optimal Scheduling
Earliest Job First

• Premise: Work Whenever Work is Available

• Amount: 2 Movies
• Problem: Could Choose a Job That Overlaps Too Much
• Correctness: Wrong!
Shortest Job First

• Premise: Work for the Shortest Amount of Time

• Amount: 4 Movies
• Problem: Could Choose a Job That Overlaps Too Much
• Correctness: Wrong!
Exhaustive Scheduling

• Premise: Try All Possible Combinations

• Amount: 4 Movies
• Problem: Computations
• Efficiency: Could Be Too Slow to Implement
Optimal Scheduling

• Premise: Keep Choosing the First Job to Complete

• Amount: 4 Movies
• Correctness: Most Correct (Not Perfect But Covers Most Scenarios)
• Efficiency: Most Efficient
Takeaway
1. Reasonable-Looking Algorithms Can Easily be
Incorrect.

2. Ensuring the Optimal Answer Over All

Possible Inputs is a Difficult But Often
Achievable Goal.

3. Seeking Counterexamples that Break

Pretender Algorithms is an Important Part
of the Algorithm Design Process.
Reasoning About Correctness
Tools are Needed to Distinguish Correct Algorithms from Incorrect Ones
One Such Tool is a Proof, Which Consists of:
• Clear, Precise Statement of What You are Trying to Prove
• Set of Assumptions of Things Taken to Be True
• Chain of Reasoning to Get From These Assumptions to the Statement
You are Trying to Prove
• Finally, a ▪ or is Inserted to Denote that the Proof is Finished
Correct Algorithms Require Careful Exposition, and Efforts to Show Both
Correctness and Not Incorrectness.
Reasoning About Correctness
Careful Description of Formulating an Algorithmic Problem:
1. Set of Allowed Input Instances
2. Required Properties of Algorithm’s Output

Some Problem Specifications Allow too Broad a Class of Input Instances

An Important and Honorable Technique in Algorithm Design is to Narrow the Set of

Allowable Instances Until There is a Correct and Efficient Algorithm.

Most Common Traps Are:

• Ask Absolutely Defined Questions (Not Abstract Words Such as Best)
• Do Not Create Compound Goals (Too Complicated To Reason and Solve)
Expressing Algorithms
Three Most Common Forms of Algorithmic Notation:
1. English
2. Pseudocode (Code With No Complaints About Syntax Errors)
3. Real Programming Language
The Heart of Any Algorithm is an Idea. If Your Idea is Not Clearly
Revealed when You Express an Algorithm, then You are Using Too Low-
Level a Notation to Describe It.
Demonstrating Incorrectness
Counter-Examples: Best Way to Disprove the Correctness of Heuristics
Properties of Counter-Examples:
1. Verifiability – Provide a Better Answer than that of the Heuristic.
2. Simplicity – Precise and Simplified to Promote Reasoning.
Advice When Seeking Counter-Examples:
• Think of Small Examples – KISS
• Think Exhaustively – Analyse Classes of Instances to Account for All
• Hunt for the Weakness – Challenge the Correctness of Arbitrary Criteria
• Go For a Tie – Verify Expectations When You Provide Sets of Duplicate Data
• Seek Extremes – Use Combinations of Edge Cases
Example
Provide a Counter-Example for the Following Greedy Heuristic:
Keep Choosing the Movies with the Least Number of Overlaps First
• Premise: The Number of Overlaps are Considered Fines
• Amount: 4 Movies
• Problem: Could Choose a Job That Ripples Through
• Correctness: Wrong!
Demonstrating Incorrectness
Failure to Find a Counter-Example to a Given Algorithm Does Not
Necessarily Mean that the Algorithm is Correct
Mathematical Induction (and Proof By Contradiction) is a Very Useful
Method for Proving the Correctness of Recursive Algorithms
Recursion and Induction are the Same Basic Idea:
1. Basis Case
2. General Assumption
3. General Case
Takeaway
1. Disproving the Correctness of a Heuristic is Easier
than Proving Its Correctness.

2. Counter-Examples are the Best Way to

Disprove the Correctness of Heuristics.

3. Mathematical Induction and Proof By

Contradiction are Useful to Prove a
Heuristic’s Correctness.
Modeling the Problem
A Model:
• Is a Simplified Abstract Representation of Some More Complex Thing
• Can Become More Accurate By Adding Representative Complexity
• Can Become Too Complex and Suffer Some Other Trade-Off

In This Module, We Have Some Basic (Abstract) Tools Called Objects or

Structures that Will Aid Us in Modelling the Problem/Algorithm. They
are Referred to as Combinatorial/Recursive Objects
 Recursive Object:
Combinatorial Objects A Type of Object that
Contains Objects of the
Same Type

Combinatorial/Recursive Objects:
• Permutations – Arrangement, Tour, Ordering, Sequence
• Subsets – Cluster, Collection, Committee, Group, Packaging, Collection
• Trees – Hierarchy, Dominance/Descendant Relationship, Taxonomy
• Graphs – Network, Circuit, Web, Relationship
• Points – Locations, Sites, Positions, Data Records
• Polygons – Shapes, Regions, Configurations, Boundaries PolygonsTrees
Graphs
Permutations/Subsets
Points
• Strings – Text, Characters, Patterns, Labels
Strings
Takeaway
1. Modelling is a Process Where Representative
Complexity is Added to a Simplified Version of
Something Until Desired Trade-Offs
are Met.

2. We Will Be Using Combinatorial/

Recursive Objects Throughout this
Module to Model Problems/Algorithms.
Homework
Remember to Bring Along the
Following for Our Next Session:
• Your (Charged) Laptop
• If Possible, Your Own Internet
(NWUWIFI/nwu-guest Not
Always Available)
• Installed .C-Compiler/IDE
• Codeblocks
• Visual Studio Code
• Xcode
• Or Any Other
Questions?