Design and Analysis of Algorithm

"That fondness for science, ... that affability and condescension which God shows to the
learned, that promptitude with which he protects and supports them in the elucidation of
obscurities and in the removal of difficulties, has encouraged me to compose a short work on
calculating by al-jabr and al-muqabala , confining it to what is easiest and most useful in
arithmetic."

Abu Ja'far Muhammad ibn Musa Al-Khwarizmi

[Born: about 780 in Baghdad (now in Iraq). Died: about 850]
[al-jabr means "restoring", referring to the process of moving a subtracted quantity to the other
side of an equation; al-muqabala is "comparing" and refers to subtracting equal quantities from
both sides of an equation.]

Algorithms Lecture Notes

Prelims:
1. Introduction
o Etymology
2. Steps for Creating a Computer Program
3. Basic Computer Operations
o Input
o Output
o Calculate
o Assignment
o Selection
o Loop
o Execute a subprogram
4. Flowcharting
o Mastering the details
o Use flowcharts to clarify and communicate
o A special note on team flowcharting
o How to draw a flowchart
o Tracking 'handovers'
o A set of useful standard symbols
5. Pseudo code
o Introduction
o Examples
Fahrenheit to Celsius
Min
Gross Pay
Factorial
Sum N Numbers
Euclid's GCD Algorithm
CSU Football
Find Smallest
Pattern Matching
Descending List of Integers
Selection Sort
Quick Sort
6. Sorting Algorithms
o Sorting Algorithms
Bubble sort
Heap sort
Insertion sort
Merge sort
Quick sort
Selection sort
Shell sort

Prelims
An algorithm, named after the ninth century scholar Abu Jafar Muhammad Ibn Musu AlKhowarizmi, is defined as follows: Roughly speaking:

An algorithm is a set of rules for carrying out calculation either by hand or on a machine.
An algorithm is a finite step-by-step procedure to achieve a required result.
An algorithm is a sequence of computational steps that transform the input into the
output.
An algorithm is a sequence of operations performed on data that have to be organized
in data structures.
An algorithm is an abstraction of a program to be executed on a physical machine
(model of Computation).

The most famous algorithm in history dates well before the time of the ancient Greeks: this is
Euclid's algorithm for calculating the greatest common divisor of two integers.

Steps for Creating a Computer Program

1. Define the problem. This will generally be provided by the instructor. Don't be afraid to
ask for clarification if you don't understand the problem.
2. Devise and algorithm that solves the problem. Although the algorithm should be neat
and precise, it should not be written in any particular programming language. An
algorithm is intended to be read by a human and not a computer. Another way to
express this step would be "Design a solution to the problem"
3. Code the algorithm and translate the algorithm.
4. Compile, test and debug program.

Basic Computer Operations

Computers are capable of performing only a few basic operations. Your algorithms should use
these operations
1. Input: A computer can receive input from a user or other source such as a mouse or a
disk file. In an algorithm, express input like this:
read age, weight, height
or
input age, weight, height
2. Output: A computer can output text, the contents of a variable or the result of a
computation. Examples:
write "Please enter age"
output hatsize
output hatsize + 10
3. Calculate: A computer can do arithmetic. Use the following in you pseudocode: + - * /
( ). Examples:
sum = sum + 1
root = (2+x) * y
4. Assignment: A computer can store data in a variable. This is used to give a variable an
initial value, i.e. initialize a variable. Assignment is also used to store the result of a
computation in a variable. Examples:
sum gets 0 // initialize
sum <-- 4+ x / y
The first example is read "sum receives 0" or "sum is assigned 0" It should not be read
"sum equals zero" because the = symbol represents assignment and not equality.
5. Selection: A computer can compare two values and select one of two actions as a
result of the comparison. In almost all computer languages, this is accomplished with
and if - then - else control structure. Example:
if (sum <= 1000) then
interest = 0
else
interest = sum * RATE
endif

6. Loop: A computer can repeat a group of instructions. Most computer languages have
several looping structures including while, do-while and for-next. Example:
count = 10
while (count >=0)
sum = sum + value
count = count - 1
endwhile
7. Execute a sub-program: A computer can execute a sub-program and then return
where it left off. In C++ a sub-program is a function and a function is executed with a
function call. Example:
input accountNumber, balance
while there are more account numbers
interest = computeInterest(balance) // function call
input accountNumber, balance
endwhile
computeInterest(balance) // function definition
return (balance * (ANNUAL_RATE/12)

Flowcharting

Topics
Mastering the details
Use flowcharts to clarify and
communicate
A special note on team
flowcharting
How to draw a flowchart
Tracking 'handovers'
A set of useful standard
symbols

Mastering the details

Flowcharting is a tool for analysing processes. It allows you to break any process down
into individual events or activities and to display these in shorthand form showing the
logical relationships between them. Constructing flowcharts promotes better
understanding of processes, and better understanding of processes is a pre-requisite for
improvement.
Examples of processes are "Receiving orders and entering them into the computer
system" or "Converting dry-mix powder into tablet form" or "Following-up sales enquiries".
The events which make up a process, and which appear in the flowchart, may be of any
type. For example, they may be "taking a phone call", "completing an order form",
"printing a report", "deciding between a number of alternatives", and so on. The symbols
used to represent each event may take any form. They may be boxes, circles, diamonds
or other shapes, or events may simply be described in words. Connections between
events are always represented by lines; usually with arrowheads to show the direction or
order in which they occur. These lines represent the flow of activity in the process being
described; hence the name of the technique. (See the example in the `How to use it'
section of this document.)
There are specialised applications for flowcharting (such as in computer systems design
and programming, engineering, and science) which use standardised sets of symbols.
You must decide for yourself whether these need to be used in your particular
management application. In most circumstances this will either not be necessary or you
can rely on three or four simple symbols to cover most types of events. A useful set of
symbols is given in the `How to use it' section below. For special standardised symbols,
see the `Other references' section of this document for sources you can refer to.

Use flowcharts to clarify and communicate

Flowcharting to help clarify what actually happens or needs to happen
Organisational activity involves many separate tasks. These are often complex and they
change over time in response to new customer demands, new product and service
requirements, or new laws and regulations. These changes are often made in isolated,
reactive and piecemeal ways, which are not necessarily best for the company or the
people doing the work. In addition to external pressures for change, there is a constant
need to search for new and better ways to do things in order to maintain a competitive
edge, and to make life easier and more interesting for those who do the work.
The only way to control change, rather than have it control you, is to clarify what actually
happens and to decide whether this is the way you want it or not. By grouping tasks into
logical areas of activity (processes) and drawing flowcharts of the events which occur, it is
possible to get a concise picture of the way particular processes are completed within the
organisation. This makes it easier for you to move on to the next logical step which is to
make changes for the better. This is because the flowcharting exercise will point you in
the right direction to collect and analyse relevant statistics, examine other processes
which relate to the one flowcharted, and pursue critical policy or procedure problems.
A special note on team flowcharting
Flowcharting, as a tool for clarifying situations and thus improving knowledge and
understanding, is particularly useful when used by a group or team. This is because by
drawing a flowchart together, the team,

develops a common understanding of the situation

contributes a larger pool of knowledge than an individual can (assuming team
members are well chosen for their knowledge and experience)
can agree a common approach to solving problems, resolving ambiguities and
making improvements

Team leaders will find that drawing a flowchart progressively on a whiteboard or flipchart,
as team members contribute their information, opinions and ideas, will not only identify
problems and areas of confusion, but will automatically build consensus and commitment
by focusing the team's attention on a single shared view of their task. Be warned,
however, that while flowcharting looks easy to do, it takes practice to use effectively. It
forces users to identify process steps so that they are clear and logical, which is of course
its principal purpose.
Flowcharting to help communicate what actually happens or needs to happen
After taking stock of a process by flowcharting it, the final form of the flowchart (which
should describe the process clearly and unambiguously) can then be used to
communicate to others. Communication of processes is important for two reasons,

people who are new to particular processes need to learn them and be able to
refer to information about them
the organisation benefits considerably from standardised processes and these

must be clearly communicated to be effective

Flowcharts can either be distributed unchanged to communicate processes, or they can
be used to help prepare logically structured and clearly written policies and procedures.

How to draw a flowchart

There are no hard and fast rules for constructing flowcharts, but there are guidelines
which are useful to bear in mind.
Here are six steps which can be used as a guide for completing flowcharts.
1. describe the process to be charted (this is a one-line statement such as, "How to
fill the car's petrol tank")
2. start with a 'trigger' event
3. note each successive action concisely and clearly
4. go with the main flow (put extra detail in other charts)
5. make cross references to supporting information
6. follow the process through to a useful conclusion (end at a 'target' point)
The best way to illustrate the use of these guidelines is to look at a simple example (see
below) and follow how each step has been applied.
1. The first step is to identify the process to be flowcharted and to give the chart a
title. In this case, it is `How to fill the car's petrol tank'.
2. Begin to draw the chart by first describing the event which initiates the process (the
'trigger'). In the example this is `Low petrol warning light comes on'.
3. Then note down each successive action taken. Actions should be described in as
few words as possible, but make sure the description is not ambiguous or unclear.
4. When you reach a point at which the flowchart branches into a number of
alternatives, and the resulting complexity threatens to overwhelm the exercise,
choose the most important alternative to continue flowcharting with. The others can
simply be terminated and dealt with in separate flowcharts. Such a point is
illustrated in the example where a decision is required on how much petrol is to be
put in the tank.
5. Often you may need to make cross-references to important supporting information
(in this example cross references may be made to, say, a table of preferred brands
of petrol, or to a list of cars able to use unleaded petrol).
6. Continue describing each event, action or decision as it occurs in sequence, until
the process is concluded. In the example, this point is reached when the petrol is
paid for, the tank is recharged, and you are ready to drive off.

Flowcharts help to identify all the key tasks involved and the finished chart can be used,

as a springboard for further discussion of the process

to connect with other flowcharts explaining related activities
to identify points where data can be usefully collected and analysed
to isolate possible problem areas
to communicate the process to those unfamiliar with it

Tracking 'handovers'
When a flowchart describes a process in which a number of different people,
departments, or functional areas are involved, it is sometimes difficult to keep track of who
is responsible for each step. A useful additional technique for tracking this, and for
analysing the number of times a process is 'handed over' to different people, is to divide
the flowchart into columns. Head up each column with the name of the person or function
involved in the process, and each time they carry out an action show it in their column.
This is illustrated in the flowchart below which covers a simple purchasing process. It
shows how control of the process passes from the person initiating the purchase, to the
Purchasing Dept. and then to the Supplier.

A set of useful standard symbols

It is not strictly necessary to use boxes, circles, diamonds or other such symbols to
construct a flowchart, but these do help to describe the types of events in the chart more
clearly. Described below are a set of standard symbols which are applicable to most
situations without being overly complex.

Rounded box - use it to represent an event which occurs automatically. Such an

event will trigger a subsequent action, for example `receive telephone call, or describe a
new state of affairs.

Rectangle or box - use it to represent an event which is controlled within the

process. Typically this will be a step or action which is taken. In most flowcharts this will
be the most frequently used symbol.

Diamond - use it to represent a decision point in the process. Typically, the

statement in the symbol will require a `yes' or `no' response and branch to different parts
of the flowchart accordingly.

Circle - use it to represent a point at which the flowchart connects with another
process. The name or reference for the other process should appear within the symbol.

Question 6
Draw a flowchart to find the sum of first 50 natural numbers.

Question 7
Draw a flowchart to find the largest of three numbers A,B, and C.

Question 8
Draw a flowchart for computing factorial N (N!)
Where N! = 1 * 2 * 3 * N .

Question 9
Draw a flowchart on how to fill the petrol tank of a car

Question 10
prompt for num1, num2
input num1, num2
while not(num1 == 0 and num2 == 0)
sum = num1 + num2
product = num1 * num2
average = sum / 2
if (sum > 200)
then display sum, "*", product, average
else display sum, product, average
endif
prompt for num1, num2 input num1, num2
endwhile

Question 11

Pseudo code
Algorithms are often expressed in pseudocode. Pseudocode is an informal English-like
notation that is independent of any particular programming language. The examples above are
pseudocode. Note the following about pseudocode:
1. Although pseudocode is "informal", it is not messy. It should be neatly printed or typed
2. Writing an pseudocode algorithm is the design phase of the program development
cycle. You design something before you build it. You write the pseudocode algorithm
before you write the program. Your algorithm later becomes part of the documentation
of the completed program.
3. Pseudocode is not C++ code. The specifics of the language are ignored in pseudocode.
Placement of semicolons, braces, variable declarations, etc. are of no concern when
writing pseudocode.
4. Indentation is only used to indicate the body of control structures. See the above
examples of the if-then-else and the while loop. Indentation is not used to indicate
minor or less important areas of code. An algorithm is not an outline. Use blank lines
and a comment to indicate transitions from one area of the program to the next. For
example, separate the input area of the program from the computation area with several
blank lines.
Example
This algorithm prompts and reads pairs of numbers from the user and displays their sum,
product and average. If the calculated sum is over 200 then a asterisk is displayed beside the
sum
prompt for num1, num2
input num1, num2
while not(num1 == 0 and num2 == 0)
sum = num1 + num2
product = num1 * num2
average = sum / 2
if (sum > 200)
then display sum, "*", product, average
else display sum, product, average
endif
prompt for num1, num2 input num1, num2
endwhile

CSci 1200 -- Introduction to Computing, Algorithms, & Problem Solving

Imperative & Functional Pseudocode Examples -- Spring Quarter 1998
Scott W. Lewandowski

Examples:
i.
Fahrenheit to Celsius
ii.
Min
iii.
Gross Pay
iv. Factorial
v. Sum N Numbers
vi.
Euclid's GCD Algorithm
vii.
CSU Football
viii.
Find Smallest
ix.
Pattern Matching
x.
Descending List of Integers
xi.
Selection Sort
xii.
Quick Sort

Fahrenheit to Celsius
Imperative Pseudocode:
1. get a value for ftemp
2. set the value of ctemp to (5 / 9) * (ftemp - 32)
3. return the value of ctemp
Functional Pseudocode:
fahrenheit_to_celsius(ftemp)
= ctemp
where ctemp = (5 / 9) * (ftemp - 32)

Min
Imperative Pseudocode:
1. get a value for X, Y
2. if X <= Y then
a. return the value of X
3. else
a. return the value of Y
Functional Pseudocode:
min(x, y)
= x, if x <= y
= y, otherwise

Gross Pay (Exercise 11, pp 53)

Functional Pseudocode:
gross_pay(wage, hours)
= wage * hours, if hours <= 40
= (wage * 40) + (1.5 * wage * (hours - 40)), if hours <= 54
= (wage * 40) + (1.5 * wage * 14)
+ (2.0 * wage * (hours - 54)), otherwise
Given the function above, we can readily compute take home pay as well if we know the tax
rate to use...
take_home_pay(wage, hours, tax_rate)
= (1 - tax_rate) * gross_pay(wage, hours)

Factorial
Imperative Pseudocode:
1. get a value for n
2. set the value of fact to 1
3. while n > 0 do
a. set the value of fact to fact * n
b. set the value of n to n - 1
4. end of the loop
5. return the value of fact
Functional Pseudocode (matching the loop above):
factorial(n)
= fact_loop(n, 1)
fact_loop(n, fact)
= fact, if n = 0
= fact_loop(n - 1, fact * n), otherwise
Functional Pseudocode (standard recursive definition):
factorial(n)
= 1, if n = 0
= n * factorial(n - 1), otherwise

Sum N Numbers
Imperative Pseudocode:
1. get a value for n
2. set the value of sum to 0
3. while n > 0 do
a. set the value of sum to sum + n
b. set the value of n to n - 1
4. end of the loop
5. return the value of sum
Functional Pseudocode (matching the loop above):
sum_n_numbers(n)
= sum_loop(n, 0)
sum_loop(n, sum)
= sum, if n = 0
= sum_loop(n - 1, sum + n), otherwise
Functional Pseudocode (standard recursive definition):
sum_n_numbers(n)

= 0, if n = 0
= n + sum_n_numbers(n - 1), otherwise

Euclid's GCD Algorithm (Exercise 7, pp 21)

Imperative Pseudocode:
1. get a value for i, j (assume i >= j)
2. set the value of r to i mod j
3. while r > 0 do
a. set the value of i to j
b. set the value of j to r
c. set the value of r to i mod j
4. end of the loop
5. return the value of j
Functional Pseudocode (standard recursive definition):
gcd(i, j)
= j, if r = 0
= gcd(j, r), otherwise
where r = i mod j

CSU Football (Exercise 7, pp 53)

Imperative Pseudocode:
1. set the value of i to 1
2. set the values of wins, losses, and ties to 0
3. while i <= 10 do
a. get values for csu, opp
b. if csu > opp then set the value of wins to wins + 1
c. else if csu < opp then set the value of losses to losses + 1
d. else set the value of ties to ties + 1
e. set the value of i to i + 1
4. end of the loop
5. return the values of wins, losses, and ties

A tuple is a collection of two or more values of possibly different types which is

represented as {v1, ..., vn}
A list is a collection of zero or more values all of which have the same type which is
represented as [v1, ..., vn]

Functional Pseudocode (matching the loop above):

season_record(scores)
= tally_loop(scores, {0, 0, 0})
tally_loop([], {w, l, t})
= {w, l, t}
tally_loop([{csu, opp} : rest_of_scores], {w, l, t})
= tally_loop(rest_of_scores, {w + 1, l, t}), if csu > opp
= tally_loop(rest_of_scores, {w, l + 1, t}), if csu < opp
= tally_loop(rest_of_scores, {w, l, t + 1}), otherwise

Find Smallest

See also Find Largest (Fig 2.10, pp 45)

Imperative Pseudocode:
1. get a value for n
2. get a value for L1, ... , Ln
3. set the value of smallest to L1
4. set the value of i to 2
5. while i <= n do
a. if Li < smallest then
i. set the value of smallest to Li
b. set the value of i to i + 1
6. end of the loop
7. return the value of smallest
Functional Pseudocode (matching the loop above):
find_smallest([])
= error "Empty list not allowed"
find_smallest([num : rest_of_nums])
= smallest_loop(rest_of_nums, num)
smallest_loop([], smallest)
= smallest
smallest_loop([num : rest_of_nums], smallest)
= smallest_loop(rest_of_nums, num), if num < smallest
= smallest_loop(rest_of_nums, smallest), otherwise

Pattern Matching (Fig 2.12, pp 51)

Functional Pseudocode:
match([], pattern)
= false
match([t : rest_of_text], pattern)
= (prefix_match([t : rest_of_text], pattern)) or (match(rest_of_text, pattern))
prefix_match(text, [])
= true
prefix_match([], pattern)
= false
prefix_match([t : rest_of_text], [p : rest_of_pattern])
= false, if not(t = p)
= prefix_match(rest_of_text, rest_of_pattern), otherwise

Descending List of Integers

Imperative Pseudocode:
1. get a value for n
2. set the value of i to 1
3. while n > 0 do
a. set the value of Ai to n
b. set the value of n to n - 1
c. set the value of i to i + 1
4. end of the loop
5. return the value of A
Functional Pseudocode (matching the loop above):
descending_list_of_integers(n)
= build_loop(n, 1, [])
build_loop(n, i, list)
= list, if i > n
= build_loop(n, i + 1, [i : list]), otherwise
Functional Pseudocode (standard recursive definition):
descending_list_of_integers(n)
= [], if n = 0
= [n : descending_list_of_integers(n - 1)], otherwise

Selection Sort

Functional Pseudocode:
remove_item(x, [])
= []
remove_item(x, [hd : tl])
= tl, if x = hd
= [hd : remove_item(x, tl)], otherwise
selection_sort([])
= []
selection_sort(values)
= [smallest : selection_sort(remove_item(smallest, values))]
where smallest = find_smallest(values)

Quick Sort
Functional Pseudocode:
less_than(x, [])
= []
less_than(x, [hd : tl])
= less_than(x, tl), if hd > x
= [hd : less_than(x, tl)], otherwise
quick_sort([])
= []
quick_sort([pivot : rest_of_values])
= quick_sort(less_than(pivot, rest_of_values)) ++ [pivot] ++
quick_sort(greater_than(pivot, rest_of_values))

Midterms
Sorting Algorithms
One of the fundamental problems of computer science is ordering a list of items. There's a
plethora of solutions to this problem, known as sorting algorithms. Some sorting algorithms are
simple and intuitive, such as the bubble sort. Others, such as the quick sort are extremely
complicated, but produce lightening-fast results.
Below are links to algorithms, analysis, and source code for seven of the most common sorting
algorithms.
Sorting Algorithms
Bubble sort
Heap sort
Insertion sort
Merge sort
Quick sort
Selection sort
Shell sort
The common sorting algorithms can be divided into two classes by the complexity of their
algorithms. Algorithmic complexity is a complex subject (imagine that!) that would take too
much time to explain here, but suffice it to say that there's a direct correlation between the
complexity of an algorithm and its relative efficiency. Algorithmic complexity is generally written
in a form known as Big-O notation, where the O represents the complexity of the algorithm and
a value n represents the size of the set the algorithm is run against.
For example, O(n) means that an algorithm has a linear complexity. In other words, it takes ten
times longer to operate on a set of 100 items than it does on a set of 10 items (10 * 10 = 100).
If the complexity was O(n2) (quadratic complexity), then it would take 100 times longer to
operate on a set of 100 items than it does on a set of 10 items.
The two classes of sorting algorithms are O(n2), which includes the bubble, insertion, selection,
and shell sorts; and O(n log n) which includes the heap, merge, and quick sorts.
In addition to algorithmic complexity, the speed of the various sorts can be compared with
empirical data. Since the speed of a sort can vary greatly depending on what data set it sorts,
accurate empirical results require several runs of the sort be made and the results averaged
together. The empirical data on this site is the average of a hundred runs against random data
sets on a single-user 250MHz UltraSPARC II. The run times on your system will almost
certainly vary from these results, but the relative speeds should be the same - the selection
sort runs in roughly half the time of the bubble sort on the UltraSPARC II, and it should run in
roughly half the time on whatever system you use as well.

These empirical efficiency graphs are kind of like golf - the lowest line is the "best". Keep in
mind that "best" depends on your situation - the quick sort may look like the fastest sort, but
using it to sort a list of 20 items is kind of like going after a fly with a sledgehammer.
O(n2) Sorts

As the graph pretty plainly shows, the bubble sort is grossly inefficient, and the shell sort blows
it out of the water. Notice that the first horizontal line in the plot area is 100 seconds - these
aren't sorts that you want to use for huge amounts of data in an interactive application. Even
using the shell sort, users are going to be twiddling their thumbs if you try to sort much more
than 10,000 data items.

On the bright side, all of these algorithms are incredibly simple (with the possible exception of
the shell sort). For quick test programs, rapid prototypes, or internal-use software they're not
bad choices unless you really think you need split-second efficiency.
O(n log n) Sorts

Speaking of split-second efficiency, the O(n log n) sorts are where it's at. Notice that the time
on this graph is measured in tenths of seconds, instead hundreds of seconds like the O(n2)
graph.
But as with everything else in the real world, there are trade-offs. These algorithms are
blazingly fast, but that speed comes at the cost of complexity. Recursion, advanced data
structures, multiple arrays - these algorithms make extensive use of those nasty things.
In the end, the important thing is to pick the sorting algorithm that you think is appropriate for
the task at hand. You should be able to use the source code on this site as a "black box" if you
need to - you can just use it, without understanding how it works. Obviously taking the time to
understand how the algorithm you choose works is preferable, but time constraints are a fact
of life.

Bubble Sort
Algorithm Analysis
The bubble sort is the oldest and simplest sort in use. Unfortunately, it's also the slowest.
The bubble sort works by comparing each item in the list with the item next to it, and swapping
them if required. The algorithm repeats this process until it makes a pass all the way through
the list without swapping any items (in other words, all items are in the correct order). This
causes larger values to "bubble" to the end of the list while smaller values "sink" towards the
beginning of the list.
The bubble sort is generally considered to be the most inefficient sorting algorithm in common
usage. Under best-case conditions (the list is already sorted), the bubble sort can approach a
constant O(n) level of complexity. General-case is an abysmal O(n2).
While the insertion, selection, and shell sorts also have O(n2) complexities, they are
significantly more efficient than the bubble sort.
Pros: Simplicity and ease of implementation.
Cons: Horribly inefficient.
Empirical Analysis
Bubble Sort Efficiency

The graph clearly shows the n2 nature of the bubble sort.

A fair number of algorithm purists (which means they've probably never written software for a
living) claim that the bubble sort should never be used for any reason. Realistically, there isn't
a noticeable performance difference between the various sorts for 100 items or less, and the
simplicity of the bubble sort makes it attractive. The bubble sort shouldn't be used for repetitive
sorts or sorts of more than a couple hundred items.
Source Code
Below is the basic bubble sort algorithm.
void bubbleSort(int numbers[], int array_size)
{
int i, j, temp;
for (i = (array_size - 1); i >= 0; i--)
{
for (j = 1; j <= i; j++)
{
if (numbers[j-1] > numbers[j])
{
temp = numbers[j-1];
numbers[j-1] = numbers[j];
numbers[j] = temp;
}
}
}
}

Insertion Sort
Algorithm Analysis
The insertion sort works just like its name suggests - it inserts each item into its proper place in
the final list. The simplest implementation of this requires two list structures - the source list
and the list into which sorted items are inserted. To save memory, most implementations use
an in-place sort that works by moving the current item past the already sorted items and
repeatedly swapping it with the preceding item until it is in place.
Like the bubble sort, the insertion sort has a complexity of O(n2). Although it has the same
complexity, the insertion sort is a little over twice as efficient as the bubble sort.
Pros: Relatively simple and easy to implement.
Cons: Inefficient for large lists.
Empirical Analysis
Insertion Sort Efficiency

The graph demonstrates the n2 complexity of the insertion sort.

The insertion sort is a good middle-of-the-road choice for sorting lists of a few thousand items
or less. The algorithm is significantly simpler than the shell sort, with only a small trade-off in
efficiency. At the same time, the insertion sort is over twice as fast as the bubble sort and
almost 40% faster than the selection sort. The insertion sort shouldn't be used for sorting lists
larger than a couple thousand items or repetitive sorting of lists larger than a couple hundred
items.
Source Code
Below is the basic insertion sort algorithm.
void insertionSort(int numbers[], int array_size)
{
int i, j, index;
for (i=1; i < array_size; i++)
{
index = numbers[i];
j = i;
while ((j > 0) && (numbers[j-1] > index))
{
numbers[j] = numbers[j-1];
j = j - 1;
}
numbers[j] = index;
}
}

Selection Sort
Algorithm Analysis
The selection sort works by selecting the smallest unsorted item remaining in the list, and then
swapping it with the item in the next position to be filled. The selection sort has a complexity of
O(n2).
Pros: Simple and easy to implement.
Cons: Inefficient for large lists, so similar to the more efficient insertion sort that the insertion
sort should be used in its place.
Empirical Analysis
Selection Sort Efficiency

The selection sort is the unwanted stepchild of the n2 sorts. It yields a 60% performance
improvement over the bubble sort, but the insertion sort is over twice as fast as the bubble sort
and is just as easy to implement as the selection sort. In short, there really isn't any reason to
use the selection sort - use the insertion sort instead.

If you really want to use the selection sort for some reason, try to avoid sorting lists of more
than a 1000 items with it or repetitively sorting lists of more than a couple hundred items.
Source Code
Below is the basic selection sort algorithm.
void selectionSort(int numbers[], int array_size)
{
int i, j;
int min, temp;
for (i = 0; i < array_size-1; i++)
{
min = i;
for (j = i+1; j < array_size; j++)
{
if (numbers[j] < numbers[min])
min = j;
}
temp = numbers[i];
numbers[i] = numbers[min];
numbers[min] = temp;
}
}

Shell Sort
Algorithm Analysis
Invented by Donald Shell in 1959, the shell sort is the most efficient of the O(n2) class of
sorting algorithms. Of course, the shell sort is also the most complex of the O(n2) algorithms.
The shell sort is a "diminishing increment sort", better known as a "comb sort" to the unwashed
programming masses. The algorithm makes multiple passes through the list, and each time
sorts a number of equally sized sets using the insertion sort. The size of the set to be sorted
gets larger with each pass through the list, until the set consists of the entire list. (Note that as
the size of the set increases, the number of sets to be sorted decreases.) This sets the
insertion sort up for an almost-best case run each iteration with a complexity that approaches
O(n).
The items contained in each set are not contiguous - rather, if there are i sets then a set is
composed of every i-th element. For example, if there are 3 sets then the first set would
contain the elements located at positions 1, 4, 7 and so on. The second set would contain the
elements located at positions 2, 5, 8, and so on; while the third set would contain the items
located at positions 3, 6, 9, and so on.
The size of the sets used for each iteration has a major impact on the efficiency of the sort.
Several Heroes Of Computer ScienceTM, including Donald Knuth and Robert Sedgewick, have
come up with more complicated versions of the shell sort that improve efficiency by carefully
calculating the best sized sets to use for a given list.
Pros: Efficient for medium-size lists.
Cons: Somewhat complex algorithm, not nearly as efficient as the merge, heap, and quick
sorts.
Empirical Analysis
Shell Sort Efficiency

The shell sort is by far the fastest of the N2 class of sorting algorithms. It's more than 5 times
faster than the bubble sort and a little over twice as fast as the insertion sort, its closest
competitor.
The shell sort is still significantly slower than the merge, heap, and quick sorts, but its relatively
simple algorithm makes it a good choice for sorting lists of less than 5000 items unless speed
is hyper-critical. It's also an excellent choice for repetitive sorting of smaller lists.
Source Code
Below is the basic shell sort algorithm.
void shellSort(int numbers[], int array_size)
{
int i, j, increment, temp;
increment = 3;
while (increment > 0)
{
for (i=0; i < array_size; i++)
{
j = i;
temp = numbers[i];
while ((j >= increment) && (numbers[j-increment] > temp))
{
numbers[j] = numbers[j - increment];
j = j - increment;
}
numbers[j] = temp;
}
if (increment/2 != 0)
increment = increment/2;
else if (increment == 1)
increment = 0;
else
increment = 1;
}
}

Heap Sort
Algorithm Analysis
The heap sort is the slowest of the O(n log n) sorting algorithms, but unlike the merge and
quick sorts it doesn't require massive recursion or multiple arrays to work. This makes it the
most attractive option for very large data sets of millions of items.
The heap sort works as it name suggests - it begins by building a heap out of the data set, and
then removing the largest item and placing it at the end of the sorted array. After removing the
largest item, it reconstructs the heap and removes the largest remaining item and places it in
the next open position from the end of the sorted array. This is repeated until there are no
items left in the heap and the sorted array is full. Elementary implementations require two
arrays - one to hold the heap and the other to hold the sorted elements.
To do an in-place sort and save the space the second array would require, the algorithm below
"cheats" by using the same array to store both the heap and the sorted array. Whenever an
item is removed from the heap, it frees up a space at the end of the array that the removed
item can be placed in.

Pros: In-place and non-recursive, making it a good choice for extremely large data sets.
Cons: Slower than the merge and quick sorts.
Empirical Analysis
Heap Sort Efficiency

As mentioned above, the heap sort is slower than the merge and quick sorts but doesn't use
multiple arrays or massive recursion like they do. This makes it a good choice for really large
sets, but most modern computers have enough memory and processing power to handle the
faster sorts unless over a million items are being sorted.
The "million item rule" is just a rule of thumb for common applications - high-end servers and
workstations can probably safely handle sorting tens of millions of items with the quick or
merge sorts. But if you're working on a common user-level application, there's always going to
be some yahoo who tries to run it on junk machine older than the programmer who wrote it, so
better safe than sorry.

Source Code
Below is the basic heap sort algorithm. The siftDown() function builds and reconstructs the
heap.
void heapSort(int numbers[], int array_size)
{
int i, temp;
for (i = (array_size / 2)-1; i >= 0; i--)
siftDown(numbers, i, array_size);
for (i = array_size-1; i >= 1; i--)
{
temp = numbers[0];
numbers[0] = numbers[i];
numbers[i] = temp;
siftDown(numbers, 0, i-1);
}
}
void siftDown(int numbers[], int root, int bottom)
{
int done, maxChild, temp;
done = 0;
while ((root*2 <= bottom) && (!done))
{
if (root*2 == bottom)
maxChild = root * 2;
else if (numbers[root * 2] > numbers[root * 2 + 1])
maxChild = root * 2;
else
maxChild = root * 2 + 1;
if (numbers[root] < numbers[maxChild])
{
temp = numbers[root];
numbers[root] = numbers[maxChild];
numbers[maxChild] = temp;
root = maxChild;
}
else
done = 1;
}
}

Merge Sort
Algorithm Analysis
The merge sort splits the list to be sorted into two equal halves, and places them in separate
arrays. Each array is recursively sorted, and then merged back together to form the final sorted
list. Like most recursive sorts, the merge sort has an algorithmic complexity of O(n log n).
Elementary implementations of the merge sort make use of three arrays - one for each half of
the data set and one to store the sorted list in. The below algorithm merges the arrays in-place,
so only two arrays are required. There are non-recursive versions of the merge sort, but they
don't yield any significant performance enhancement over the recursive algorithm on most
machines.
Pros: Marginally faster than the heap sort for larger sets.
Cons: At least twice the memory requirements of the other sorts; recursive.
Empirical Analysis
Merge Sort Efficiency

The merge sort is slightly faster than the heap sort for larger sets, but it requires twice the
memory of the heap sort because of the second array. This additional memory requirement
makes it unattractive for most purposes - the quick sort is a better choice most of the time and
the heap sort is a better choice for very large sets.

Like the quick sort, the merge sort is recursive which can make it a bad choice for applications
that run on machines with limited memory.
Source Code
Below is the basic merge sort algorithm.
void mergeSort(int numbers[], int temp[], int array_size)
{
m_sort(numbers, temp, 0, array_size - 1);
}
void m_sort(int numbers[], int temp[], int left, int right)
{
int mid;
if (right > left)
{
mid = (right + left) / 2;
m_sort(numbers, temp, left, mid);
m_sort(numbers, temp, mid+1, right);
merge(numbers, temp, left, mid+1, right);
}
}
void merge(int numbers[], int temp[], int left, int mid, int right)
{
int i, left_end, num_elements, tmp_pos;
left_end = mid - 1;
tmp_pos = left;
num_elements = right - left + 1;
while ((left <= left_end) && (mid <= right))
{
if (numbers[left] <= numbers[mid])
{
temp[tmp_pos] = numbers[left];
tmp_pos = tmp_pos + 1;
left = left +1;
}
else
{
temp[tmp_pos] = numbers[mid];
tmp_pos = tmp_pos + 1;
mid = mid + 1;
}
}
while (left <= left_end)
{

temp[tmp_pos] = numbers[left];
left = left + 1;
tmp_pos = tmp_pos + 1;
}
while (mid <= right)
{
temp[tmp_pos] = numbers[mid];
mid = mid + 1;
tmp_pos = tmp_pos + 1;
}
for (i=0; i <= num_elements; i++)
{
numbers[right] = temp[right];
right = right - 1;
}
}

Quick Sort
Algorithm Analysis
The quick sort is an in-place, divide-and-conquer, massively recursive sort. As a normal person
would say, it's essentially a faster in-place version of the merge sort. The quick sort algorithm
is simple in theory, but very difficult to put into code (computer scientists tied themselves into
knots for years trying to write a practical implementation of the algorithm, and it still has that
effect on university students).
The recursive algorithm consists of four steps (which closely resemble the merge sort):
1. If there are one or less elements in the array to be sorted, return immediately.
2. Pick an element in the array to serve as a "pivot" point. (Usually the left-most element in
the array is used.)
3. Split the array into two parts - one with elements larger than the pivot and the other with
elements smaller than the pivot.
4. Recursively repeat the algorithm for both halves of the original array.
The efficiency of the algorithm is majorly impacted by which element is choosen as the pivot
point. The worst-case efficiency of the quick sort, O(n2), occurs when the list is sorted and the
left-most element is chosen. Randomly choosing a pivot point rather than using the left-most
element is recommended if the data to be sorted isn't random. As long as the pivot point is
chosen randomly, the quick sort has an algorithmic complexity of O(n log n).

Pros: Extremely fast.

Cons: Very complex algorithm, massively recursive.
Empirical Analysis
Quick Sort Efficiency

The quick sort is by far the fastest of the common sorting algorithms. It's possible to write a
special-purpose sorting algorithm that can beat the quick sort for some data sets, but for
general-case sorting there isn't anything faster.
As soon as students figure this out, their immediate implulse is to use the quick sort for
everything - after all, faster is better, right? It's important to resist this urge - the quick sort isn't
always the best choice. As mentioned earlier, it's massively recursive (which means that for
very large sorts, you can run the system out of stack space pretty easily). It's also a complex
algorithm - a little too complex to make it practical for a one-time sort of 25 items, for example.
With that said, in most cases the quick sort is the best choice if speed is important (and it
almost always is). Use it for repetitive sorting, sorting of medium to large lists, and as a default
choice when you're not really sure which sorting algorithm to use. Ironically, the quick sort has
horrible efficiency when operating on lists that are mostly sorted in either forward or reverse
order - avoid it in those situations.

Source Code
Below is the basic quick sort algorithm.
void quickSort(int numbers[], int array_size)
{
q_sort(numbers, 0, array_size - 1);
}
void q_sort(int numbers[], int left, int right)
{
int pivot, l_hold, r_hold;
l_hold = left;
r_hold = right;
pivot = numbers[left];
while (left < right)
{
while ((numbers[right] >= pivot) && (left < right))
right--;
if (left != right)
{
numbers[left] = numbers[right];
left++;
}
while ((numbers[left] <= pivot) && (left < right))
left++;
if (left != right)
{
numbers[right] = numbers[left];
right--;
}
}
numbers[left] = pivot;
pivot = left;
left = l_hold;
right = r_hold;
if (left < pivot)
q_sort(numbers, left, pivot-1);
if (right > pivot)
q_sort(numbers, pivot+1, right);
}

Finals
Greedy Algorithm
Greedy algorithms are simple and straightforward. They are shortsighted in their
approach in the sense that they take decisions on the basis of information at hand
without worrying about the effect these decisions may have in the future. They are
easy to invent, easy to implement and most of the time quite efficient. Many
problems cannot be solved correctly by greedy approach. Greedy algorithms are
used to solve optimization problems

Greedy Approach
Greedy Algorithm works by making the decision that seems most promising at any
moment; it never reconsiders this decision, whatever situation may arise
later.
As an example consider the problem of "Making Change".
Coins available are:

dollars (100 cents)

quarters (25 cents)
dimes (10 cents)
nickels (5 cents)
pennies (1 cent)

Problem
coins.

Make a change of a given amount using the smallest possible number of

Informal Algorithm

Start with nothing.

at every stage without passing the given amount.
o add the largest to the coins already chosen.

Formal Algorithm
Make change for n units using the least possible number of coins.
MAKE-CHANGE (n)
C {100, 25, 10, 5, 1} // constant.
Sol {};
// set that will hold the solution set.
Sum 0 sum of item in solution set
WHILE sum not = n
x = largest item in set C such that sum + x n
IF no such item THEN
RETURN "No Solution"
S S {value of x}
sum sum + x
RETURN S

Example Make a change for 2.89 (289 cents) here n = 2.89 and the solution
contains 2 dollars, 3 quarters, 1 dime and 4 pennies. The algorithm is greedy
because at every stage it chooses the largest coin without worrying about the
consequences. Moreover, it never changes its mind in the sense that once a coin
has been included in the solution set, it remains there.

Characteristics and Features of Problems solved by

Greedy Algorithms
To construct the solution in an optimal way. Algorithm maintains two sets. One
contains chosen items and the other contains rejected items.
The greedy algorithm consists of four (4) function.
1.
2.
3.
4.

A function that checks whether chosen set of items provide a solution.

A function that checks the feasibility of a set.
The selection function tells which of the candidates is the most promising.
An objective function, which does not appear explicitly, gives the value of a
solution.

Structure Greedy Algorithm

Initially the set of chosen items is empty i.e., solution set.

At each step
o item will be added in a solution set by using selection function.
o IF the set would no longer be feasible
reject items under consideration (and is never consider again).
o ELSE IF set is still feasible THEN
add the current item.

Definitions of feasibility
A feasible set (of candidates) is promising if it can be extended to produce not
merely a solution, but an optimal solution to the problem. In particular, the
empty set is always promising why? (because an optimal solution always
exists)
Unlike Dynamic Programming, which solves the subproblems bottom-up, a greedy
strategy usually progresses in a top-down fashion, making one greedy choice after
another, reducing each problem to a smaller one.
Greedy-Choice Property
The "greedy-choice property" and "optimal substructure" are two ingredients in the
problem that lend to a greedy strategy.
Greedy-Choice Property
It says that a globally optimal solution can be arrived at by making a locally optimal
choice.