Topic 4 Notes

4.1 Introduction to Algorithms

An algorithm is a general method for solving problems of a certain kind.
An algorithm is a step-by-step procedure for solving a problem in a finite amount of time.
Most algorithms transform input objects into output objects.
We describe algorithms using natural language and mathematical notation. Algorithms, as such,
cannot be executed by a computer. The formulation of an algorithm in a programming language is
called a program.

4.2 Developing an Algorithm

There are many steps involved in writing a computer program to solve a given problem.
The steps go from problem formulation and specification, to design of the solution, to
implementation, testing and documentation, and finally to evaluation of the solution.
Handling Problems
The first thing with which one is confronted when writing programs is the problem. Typically, you
are confronted with ``real-life'' problems and you want to make life easier by providing a program
for the problem. However, real-life problems are nebulous and the first thing you have to do is to
try to understand the problem to separate necessary from unnecessary details: You try to obtain
your own abstract view, or model, of the problem. This process of modeling is called abstraction.
The model defines an abstract view to the problem. This implies that the model focuses only on
problem related stuff and that you try to define properties of the problem. These properties include:
the data which are affected and the operations which are identified by the problem.
To sum up, abstraction is the structuring of a nebulous/vague problem into well-defined entities
by defining their data and operations.
Algorithms
Once we have a suitable mathematical model for our problem, we can attempt to find a solution in
terms of that model. Our initial goal is to find a solution in the form of an algorithm, which is a
finite sequence of instructions, each of which has a clear meaning and can be performed with a
finite amount of effort in a finite length of time.
An integer assignment statement such as x := y + z is an example of an instruction that can be
executed in a finite amount of effort.
In an algorithm instruction can be executed any number of times, provided the instructions
themselves indicate the repetition. However, we require that, no matter what the input values may
be, an algorithm terminates after executing a finite number of instructions. Thus, a program is an
algorithm as long as it never enters an infinite loop on any input.

4.3 Good programming

There are a substantial number of ideas we should bear in mind when designing an algorithm and
implementing it as a program.
Plan the design of a program.
When designing an algorithm first sketch the algorithm informally, then as a pseudo-program and
gradually refining the pseudo-program until it becomes executable code. This strategy of sketch-
then-detail tends to produce a more organized final program that is easier to debug and maintain.
Encapsulate.
Use procedures and ADT's to place the code for each principal operation and type of data in one
place in the program listing. Then, if changes become necessary, the section of code requiring
change will be localized.
Use or modify an existing program.
One of the chief inefficiencies in the programming process is that usually a project is tackled as if
it were the first program ever written. One should first look for an existing program that does all
or a part of the task. Conversely, when writing a program, one should consider making it available
to others for possibly unanticipated uses.
Be a tool-smith.
In programming, a tool is a program with a variety of uses.
When writing a program, consider whether it could be written in a somewhat more general way
with little extra effort.
Pick good variable names
Here are some things should consider when choosing variable names.
1. Meaningful names: The variable name should convey what kind of information is
contained in the object. Here are some examples:
Good Bad
Age foo
Student xwy
Name a
2. Multi-word names: Don't be afraid of making variable names with more than one word.
Just make sure that it's readable. There are two conventions for doing this:
studentAge
student_age
Please do not write 'studentage'.
Use constants
Whenever you have a value that should not change, always make it a constant. This way Ruby will
help you catch some possible errors.
Some examples:
Pi = 3.14159265
Electron_mass = 9.109e-31
Speed_of_light = 3e8
Earth_Sun_distance = 5.79e10
In general, do not write numerical values directly. Use constants to make your code more clear.
For instance, the formula for the area of a circle is A=πr2. Where r is the radius. Your code should
resemble this formula as much as possible.
PI = 3.14159265
Area = PI*radius**2
Instead of area = 3.1416*radius**2

4.4 Algorithm Representation

Algorithm representation requires to have some features in order for it to be a meaningful and
useful representation.
Show the Logic
Your algorithm representation should focus on the logic of the problem, and not the logic of the
eventual implementation. More specifically, the upper levels of your representation should, to the
degree possible, be devoid of implementation details - these should be relegated to the lower levels
of the representation.
Reveal the Flow
Most problems, especially if they are intended to be solved with the aid of a computer program,
involve flow control. In the "structured programming" paradigm, this flow control consists of
sequences, selections, and repetitions. At some point, as each of those tasks is developed, decisions
will have to be made and different steps taken depending on the outcome of those decisions. The
representation method used should be compatible with this and clearly show the points at which
decisions are made and what the various courses of action are that can result.
Be Expandable and Collapsible
Our algorithm representation should be flexible and allow us to readily collapse it so as to show
less detail and focus on the more abstract elements of the algorithm or to expand it so as to get as
detailed as necessary in order to actually implement the solution. Unstated in this is an
acknowledgement that, as we expand our algorithm and become more detailed, at some point we
have to get into the logic of the implementation issues.
Aid in Implementation
At the end of the day, the goal is usually to actually implement a solution to the problem being
solved. If our method of representing our algorithm does not lend itself to an orderly
implementation of that algorithm, then our method is seriously flawed. Conversely, if our method
of representation lends itself to a systematic implementation of the algorithm, then our method is
extremely useful.
By "systematic implementation" we mean that we should be able to take our represented algorithm
and break it into easily identifiable fragments each of which can be readily translated into one of
the structures, such as a while() loop or an if()/else block, available to us in our chosen
implementation scheme be it a C program, an assembly language routine, an electronic circuit, or
a physical mechanism.

4.5 Pseudocode
It is used primarily as a rather short-term communication between members working on a specific
project - the code itself and other documents used for long-term archival purposes. Pseudocode
tends to be much more informal and a case of "whatever works". Some people choose to write
their pseudocode very much as though it were a true programming language with very formal
constructs Some people write their pseudocode almost like a free-verse description of what the
program needs to do.

A Recommended Pseudocode Format

To aid in communication - particularly between you and the grader (the value of which should be
relatively obvious) it is recommended that the problem be decomposed into a set of hierarchical
tasks. The lowest-level tasks should either be tasks that are very straightforward to implement
directly (using your level of knowledge) or the specific instructions for performing the task should
be provided. By beginning each line with one of the keywords discussed below, the chance for
miscommunication between you and the grader is greatly diminished.
Keywords
Keywords describe what needs to be done or provides information about why something is being
done.
Task
A TASK statement is something that the program must perform but that is described at a level
more abstract than what can be coded directly. One way to think of it is that you break a problem
down into a set of TASKs. Each TASK can, in turn, be broken down into more narrowly defined
TASKs. At some point, the TASK can be described in terms of steps that can be directly
implemented
Rem
A REM statement is merely a remark or comment. They are useful if the TASK statement proves
to be insufficient to convey all the desired information or if the reason that something is done or
why it works is not obvious.
Action Keywords
Action keywords are the lines that actually do the work. There are three basic actions that can be
carried out: changing the value stored at some location in memory, getting input from some device,
or generating output to some device. We will use the SET, GET, and PUT keywords for these
actions respectively.
1. Set
This is an "action" keyword that denotes performing some operation that changes a value
in memory. The most common example would be the evaluation of some equation.
2. Put
This is an "action" keyword that denotes an output operation, generally to the screen. If the
destination is anything other than the screen, such as a file or the serial port, then that
should be explicitly stated.
3. Get
This is an "action" keyword that denotes in input operation, generally from the keyboard.
It is generally understood that there is an implied SET action involved where the value
brought in gets stored in some memory location. If the source is anything other than the
keyboard, such as a file or the serial port, then that should be explicitly stated.
Flow Control Keywords
While the action keywords perform the actual work, they are insufficient in and of themselves to
write all but the most trivial programs. Of the three structured programming constructs, the action
keywords are only sufficient to implement the first of them, namely a sequence of instructions.

Selection - Case 1
SEL: (test condition)
TRUE:
Statement(s) to be executed if test condition is TRUE
FALSE
Statement(s) to be executed if test condition is FALSE

Selection - Case 2
IF: (test condition)
Statement(s) to be executed if test condition is TRUE
ELSE:
Statement(s) to be executed if test condition is FALSE

Repetition - Case 1
LOOP:
WHILE: (test condition)
Statement(s) to be executed if test condition is TRUE

Repetition - Case 2
LOOP:
Statement(s) to be executed if test condition is TRUE
WHILE: (test condition)

4.6 Flowcharts
Flowcharts are a graphical means of representing an algorithm, as should be expected, they have
advantages and disadvantages compared to pseudocode.
The idea behind a flowchart is that it links together a series of blocks each of which perform some
specific task. Each of these tasks is represented by a block and has exactly one arrow leading to it
and, more importantly, one arrow exiting from it. This is key to the concept of a "structured
program".
The shape of the block may convey additional information about what is happening. For instance,
a rectangular block is frequently used to indicated that a computation is occurring while a slanted
parallelogram is used to indicate some type of input or output operation. The diversity of shapes
that can be used and what they mean is staggering - for instance a different shape can be used to
indicated output to a tape drive versus to a hard disk or to indicate output in text format verses
binary format. By using such highly specialized symbols, much of what is happening can be
conveyed by the symbols themselves. But the power of using these distinctions is generally only
useful to people that work with flowcharts continuously, professionally, and who are describing
very large and complex systems.

Basic Flowchart Shapes

Circle - Entry/Exit Point
The circle indicates the entry and exit point for the program - or for the current segment of the
program. The entry point has exactly one arrow leaving it and the exit point has exactly one arrow
entering it. Execution of the program - or of that segment of the program - always starts at the
entry point and finishes at the exit point.
Rectangle - Task
The rectangle represents a task that is to be performed. That task might be as simple as
incrementing the value of a single variable or as complex as you can imagine. The key point is that
it also has a single entry point and a single exit point.
Parallelogram - Input/Output
The parallelogram is used to indicate that some form in input/output operation is occurring. They
must also obey the single entry single exit point rule which makes sense given that they are a task-
block except with a slightly different shape for the symbol.
Diamond - Decision Point
The diamond represents a decision point within our program. A question is asked and depending
on the resulting answer, different paths are taken. Therefore a diamond has a single entry point but
more than one exit point. Usually, there are two exit points - one that is taken if the answer to the
question is "true" and another that is taken if the answer to the question is "false". This is sufficient
to represent any type of branching logic including both the typical selection statements and the
typical repetition statements.
Arrow - Interblock Flow
The arrows simply show which symbol gets executed next. The rule is that once an arrow leaves
a symbol, it must lead directly to exactly one other symbol - arrows can never fork and diverge.
They can, however, converge and join arrows coming from other blocks.

4.7 Algorithm analysis

Analysis is a field that is dedicated to understanding the complexity of algorithms. The purpose of
algorithm analysis is to test and then draw conclusions about how well a particular algorithm works
in general. This would be very difficult and time consuming to do on individual computers, so
researchers devise models of computer functioning to test algorithms.
More than one algorithm might work to perform the same operation, but some algorithms use more
memory and take longer to perform than others. Also, how do we know how well algorithms work
in general, given differences between computers and data inputs? This is where algorithm analysis
comes in.
One way to test an algorithm is to run a computer program and see how well it works. The problem
with this approach is that it only tells us how well the algorithm works with a particular computer
and set of inputs.
In general, algorithm analysis is most concerned with finding out how much time a program takes
to run, and how much memory storage space it needs to execute the program. In particular,
computer scientists use algorithm analysis to determine how the data imputed into a program
affects its total running time, how much memory space the computer needs for program data, how
much space the program’s code takes in the computer, whether an algorithm produces correct
calculations, how complex a program is, and how well it deals with unexpected results.
The performance of a computer is determined by the hardware: processor used (type and speed),
memory available (cache and RAM), and disk available; the programming language in which the
algorithm is specified; the language compiler/interpreter used; and the computer operating system
software.

The Running Time of a Program

The running time of a program depends on factors such as:
1. The input to the program,
2. The quality of code generated by the compiler used to create the object program,
3. The nature and speed of the instructions on the machine used to execute the program,
4. The time complexity of the algorithm underlying the program.

5. The hardware:

o processor used (type and speed),

o memory available (cache and RAM), and computer performance
o disk available
6. The programming language in which the algorithm is specified;
7. The language compiler/interpreter used; and
8. The computer operating system software.
The fact that running time depends on the input tells us that the running time of a program should
be defined as a function of the input.
4.8 Types of Algorithms
A simple recursive algorithm:
Solves the base cases directly
Recurs with a simpler subproblem
Does some extra work to convert the solution to the simpler subproblem into a solution to the given
problem
It’s called “simple” because several of the other algorithm types are inherently recursive.
Example recursive algorithms
To count the number of elements in a list:
If the list is empty, return zero; otherwise,
Step past the first element, and count the remaining elements in the list
Add one to the result

Backtracking algorithms
Based on a depth-first recursive search
A backtracking algorithm:
Tests to see if a solution has been found, and if so, returns it; otherwise
For each choice that can be made at this point,
Make that choice
Recur
If the recursion returns a solution, return it
If no choices remain, return
failure

Example backtracking algorithm

To color a map with no more than four colors:
color(Country n)
If all countries have been colored (n > number of countries) return success; otherwise,
For each color c of four colors,
If country n is not adjacent to a country that has been colored c
Color country n with color c
recursivly color country n+1
If successful, return success
Return failure (if loop exits)

Divide and Conquer

Consists of two parts:
i. Divide the problem into smaller subproblems of the same type, and solve these
subproblems recursively
ii. Combine the solutions to the subproblems into a solution to the original problem
•Traditionally, an algorithm is only called divide and conquer if it contains two or more
recursive calls
Examples
Quicksort:
Partition the array into two parts, and quicksort each of the parts –No additional
work is required to combine the two sorted parts
Mergesort:
Cut the array in half, and merge sort each half
Combine the two sorted arrays into a single sorted array by merging them

Dynamic programming algorithms

A dynamic programming algorithm remembers past results and uses them to find new results
Dynamic programming is generally used for optimization problems –Multiple
solutions exist, need to find the “best” one
Requires “optimal substructure” and “overlapping subproblems”
Optimal substructure: Optimal solution contains optimal solutions to subproblems
Overlapping subproblems: Solutions to subproblems can be stored and reused in a bottom-up
fashion
This differs from Divide and Conquer, where subproblems generally need not overlap
To find the nth Fibonacci number:
i. If n is zero or one, return one; otherwise,
ii. Compute, or look up in a table, fibonacci(n-1) and fibonacci(n-2)
iii. Find the sum of these two numbers
iv. Store the result in a table and return it
Since finding the nth Fibonacci number involves finding all smaller Fibonacci numbers, the second
recursive call has little work to do
The table may be preserved and used again later

Greedy algorithms
A “greedy algorithm” works well for optimization problems •Works in phases:
At each phase:
You take the best you can get right now, without regard for future consequences –You hope that
by choosing a local optimum at each step, you will end up at a global optimum
Example: Counting money
Suppose you want to count out a certain amount of money, using the fewest possible bills and
coins
A greedy algorithm would do this would be:
At each step, take the largest possible bill or coin that does not overshoot –Example: To make
$6.39, you can choose:
a $5 bill
a $1 bill, to make $6
a 25¢ coin, to make $6.25
A 10¢ coin, to make $6.35
four 1¢ coins, to make $6.39
For US money, the greedy algorithm always gives the optimum solution
A failure of the greedy algorithm

Branch and bound algorithms

Branch and bound algorithms are generally used for optimization problems
As the algorithm progresses, a tree of subproblems is formed
The original problem is considered the “root problem”
A method is used to construct an upper and lower bound for a given problem
At each node, apply the bounding methods
If the bounds match, it is deemed a feasible solution to that particular subproblem •If bounds do
not match, partition the problem represented by that node, and make the two subproblems into
children nodes
Continue, using the best-known feasible solution to trim sections of the tree, until all nodes have
been solved or trimmed

Example branch and bound algorithm

Travelling salesman problem: A salesman has to visit each of n cities (at least) once each, and
wants to minimize total distance travelled
Consider the root problem to be the problem of finding the shortest route through a set of cities
visiting each city once
Split the node into two child problems:
Shortest route visiting city A first
Shortest route not visiting city A first
Continue subdividing similarly as the tree grows

Brute force algorithm

A brute force algorithm simply tries all possibilities until a satisfactory solution is found
Such an algorithm can be:
Optimizing: Find the best solution. This may require finding all solutions, or if a value for the best
solution is known, it may stop when any best solution is found
Example: Finding the best path for a travelling salesman
Satisficing: Stop as soon as a solution is found that is good enough

Example: Finding a travelling salesman path that is within 10% of optimal

Randomized algorithms
A randomized algorithm uses a random number at least once during the computation to make a
decision
Example: In Quicksort, using a random number to choose a pivot –Example: Trying to factor a
large prime by choosing random numbers as possible divisors

4.9 Characteristics of a good algorithm

i. Finiteness. An algorithm must always terminate after a finite number of steps.
ii. Definiteness. Each step of an algorithm must be precisely defined; the actions to be carried
out must be rigorously and unambiguously specified for each case.
iii. Input. An algorithm has zero or more inputs, i.e, quantities which are given to it initially
before the algorithm begins.
iv. Output. An algorithm has one or more outputs i.e, quantities which have a specified relation
to the inputs.
v. Effectiveness. An algorithm is also generally expected to be effective. This means that all
of the operations to be performed in the algorithm must be sufficiently basic that they can
in principle be done exactly and in a finite length of time.
vi. Unambiguous. Algorithm should be clear and unambiguous. Each of its steps (or phases),
vii. Feasibility. Should be feasible with the available resources.
 viii. Independent. An algorithm should have step-by-step directions which should be
independent of any programming code.

Revision questions
1. What is the relationship between an algorithm and program?
2. Explain the process of developing and algorithm
3. What considerations should be taken into account when developing an algorithm
4. Describe the types of algorithms can be developed
5. Explain the importance of flowcharts in algorithm representation