Book 3: Internal Design and

Programming
Chapter 2: Algorithms
Introduction
This chapter describes the basics of
algorithms used for module logic design
and the algorithms used to solve typical
problems.
Technology Engineers No.3 Internal Design
and Programming Chapter 2
The basis of programming is algorithm design. In
designing the logical structure of a program, it is
important to use the most appropriate algorithm.
In choosing a program from a library, what
algorithm to use is one of the most important
criteria for choosing the most appropriate
program.
Overview
Objectives
Understand the basics of algorithms
Algorithm definition, design, relationships with
data structures, representation methods, etc.
Understand the characteristics and meaning
of representative search, sort, character
string processing, and file processing
algorithms
What is an Algorithm?
Definition of an algorithm
Defined in the Japanese Industrial Standard as:
A set of a limited number of clearly defined rules
that are applied a limited number of times to solve
problems
To be an algorithm, a set of rules must be
unambiguous (to solve a problem effectively
and efficiently) and have a clear stopping point
What is an Algorithm?
Algorithm and Programming
Programming describes data and algorithms in
a programming language to execute its given
tasks
Algorithm vs Program
A program consists of algorithms and data
An algorithm consists of logic and control
What is an Algorithm?
4 different types of programming:
Procedural programming
Most commonly used type of programming
E.g. FORTRAN, COBOL, Pascal, C, etc.
Characteristics:
Divided into modules (coding is done for each module)
Structured theorems are introduced
Limitations (as it is procedure oriented)
Beside declaring procedures, variables must be declared
Instructions are executed one by one in sequential manner
(parallel processing cannot be done)
Comparisons and calculations must be made to solve
problems
What is an Algorithm?
Functional programming
A type of function-oriented programming
E.g. used in field of Artificial Intelligence (AI), basic
calculation theories, other research tasks, etc.
Characteristics:
Expressions are built by nesting and they are replaced
with results of calculations to execute a program
Recursive calls are described more easily
Level of affinity with parallel processing is high
Calculation process or procedure does not need to be
considererd
What is an Algorithm?
Logic programming
Predicate based on facts and inferences
E.g. Prolog
Characteristics:
Facts are described based on the prejudice logic,
making the process of programming easier
Inferences and logical calculations can be made easily
Object-oriented programming
System is considered a group of objects
E.g. Smalltalk, C++, Java, etc.

Algorithm and the Data Structure
1. Data Structure
A procedure that a program follows to store data and
perform given tasks
Basic data structure
Most basic data structure unit
Data is stored in main memory unit
Data type must be declared
Problem-oriented data structure
Built by combining basic data structures containing one or more of
the following:
List
Stack
Queue
Tree Structure
Algorithm and the Data Structure
2. Relationships between algorithm and the data
structure
Array processing
Linear search algorithm and the data structure
Most frequently used data search method
Data is searched while subscripts in an array are being
incremented
Maximum number of times the search is repeated is the size of
an array
File processing
Process of reading, editing and printing a file is repeated until
there is no data in a file


Algorithm and the Data Structure
List processing
Data arranged in terms of logic
Data does not need to be shifted during insertion or
deletion
Data can be inserted or deleted by simply
manipulating a pointer
Various Algorithms
Describes the representative algorithms that
have until now been developed in relation to
each type of problem
Search Algorithm
1. Linear (or sequential) search method
Exhaustive search method
A method of combining a table from the top to the end
Data retrieved is collated with each data in a table
Search is successful only if data matches data to retrieve
Search ends when first successful match occurs
Inappropriate for numerous data retrieval
Comparisons are made (N x 2) times
15 3 2 9 6 1 7 8
1 2 3 4 5 6 7 8
TBL
6
(Match)
Searching from the top to the end one by one
Search Algorithm
Sentinel search method
Same data (sentinel) as the data to retrieve is placed at the end
of a table
Used when authenticity of data and data matches data to
retrieve
Comparisons are made (N + 1) times
Loop does not need to carry out the end-of-array test,
making search more efficient
H K A I S D E A
1 2 3 4 5 6 7 8
TBL
N pieces
The same data as data to retrieve is stored
N + 1 (sentinel) pieces
Search Algorithm
2. Binary search method
A method that narrows down target data while successively
dividing a search area into two parts
Requires elements to be arranged in ascending or
descending order
Data smaller than reference data does not need to be
searched if data being searched is larger than reference data
Average number of times = [log
2
N]
Maximum number of times = -[log
2
N] + 1
** Decimal numbers of the value in [] are truncated
Binary Search Method
Step1: A total of the value of a subscript representing the top of a table and
that of a subscript representing the end of a table is a divided by 2.
Step 2: The elements having the value obtained in step 1 as a subscript are
compared with a target element.
Step 3: If there is an element that matches as target element the search is
successful.
Step 4: If the value of a target element is smaller than that of an element in
a table,1 is subtracted from the current subscript and the the end of table.If
the value of a target element is larger than that of an element in a table,1 is
added to the current subscript and the value is used as a subscript for
representing the top of a table.
Step 5: Step 1 through step 4 is repeated. If an element matching a target
element cannot be found at the point where the value of a subscript
representing the top of a table becomes larger than that of a subscript
representing the end of a table, the search is unsuccessful. This completes
the search.
Binary Search Method
1002 1005 1010 1024 1028 1052 1211 1322 1866 2132
TBL(1) TBL(2) TBL(3) TBL(4) TBL(5) TBL(6) TBL(7) TBL(8) TBL(9) TBL(10)
1052 1211 1322 1866 2132
1866 2132
1866
1866
1866
TBL
The first comparison
The second comparison
The third comparison
x
x
x
X>TBL(i)
X>TBL(i)
X=TBL(i)
Sort Algorithm
Reorganizing data in a specific order
Internal sorting
Sorting data in a main memory unit
External sorting
Sorting data stored on a magnetic disk and other auxiliary
storage unit
Selection of sorting method
Speed of sort
How data is sorted
Computational complexity
Sort Algorithm
Sorting
Internal sorting
External sorting
Shaker sort method
Basic selection method
Shell sort method
Basic exchange method
Merge sort method
Quick sort method
Basic insertion method
Exchange Sort
1. Basic Exchange method (bubble sort)
One of the simplest sort methods
Used to compare a pair of data sequentially from the
head of an array
Efficiency is low as data items are unconditionally
compared even if they have been sorted correctly
Process is time-consuming when data volume is large
Maximum computational complexity: 0 (n
2
)
Average computational complexity: 0 (n
2
)
Exchange Sort
Steps of Basic Exchange (Bubble) method
1st & 2nd elements in a table are compared
If 1st element is larger than 2nd, 1st is exchanged with 2nd
If 2nd element is larger than 1st, no exchange occurs
2nd and 3rd elements are compared and steps 2 & 3 are repeated
This routine operation is repeated to the last element in the table.
Maximum value is stored in the last element in that table as it reaches
the last element
Steps 1 through 4 and 5 are executed until the operation reaches the
last but one
Steps 1 through 6 are repeated until only the 1st & 2nd elements in a
table remain
Selection Sort
2. Basic selection method
Like bubble sort, it is one of the simplest sort method
Smallest/largest value is first selected from all data items and it is
exchanged with the data item at the head of an array, then same
routine repeats its execution on all remaining data items
Efficiency is low as data items are unconditionally compared even
if they have been sorted correctly
Process is time-consuming when data volume is large
Maximum computational complexity: 0 (n
2
)
Average computational complexity: 0 (n
2
)
Selection Sort
Steps of basic selection method
Data item with the smallest value is detected in data items
stored in a table
Data item with the smallest detected is exchanged with the first
data item in a table (space into which the data item with the
smallest value can be temporarily in a table)
Data item with the smallest value is detected in the 2nd item in
a table
Data item with the smallest value is detected is exchanged with
the 2nd
The same operation is repeatedly performed until the last data
item but one is reached. When the last data item but one is
reached, data sorting is completed
Selection Sort
Insertion Sort
3. Basic insertion method
Like basic exchange method, this is one of the simplest
sort method
While data items are being sorted, an unsorted data item
is inserted into a proper position in the sequence of
sorted data items
Comparison and insertion speeds are fast as preceding
data items are sorted
Process is time-consuming when data volume is large
Insertion Sort
Steps of the basic insertion method
1st & 2nd elements in a table are compared
If 1st element < 2nd element, nothing is done
If 1st element > 2nd element, 1st element is exchanged with
2nd, i.e. 1st & 2nd elements are in correct order
2nd & 3rd elements are compared
If 2nd element < 3rd element, nothing is done
If 2nd element > 3rd element, 2nd element is exchanged with
3rd element. Then this element is compared with the
preceding element according to steps 2 and 3. These steps are
repeated until it is placed in the right position
Steps 4, 5 and 6 are repeated until the last element in the table
is inserted into the correct position
Insertion Sort
Shaker Sort
Shaker sort method
Algorithm is the same as that of basic exchange method
(bubble sort)
Data items are first compared from left to right, such that
maximum (minimum) value is set at rightmost position
Data items are then compared from right to left, such that
minimum (maximum) value is set at leftmost position
Operation is repeated to perform data sorting
Process is time-consuming when data volume is large
Shell Sort
Shell sort method
Extended version of basic insertion method
Execution speed is faster than basic insertion method
Two items located away from each other a certain interval are
picked out of data sequence and sorting is executed while picked
data items are compared with each other
Interval is called gap, such that gap = number of data (n) 2, which
gradually made smaller and finally set to 1
Pieces of data that are away from each other and located in different
positions are quickly exchanged
Shell Sort
Data items that are sorted in wrong positions
can be resorted to correct positions in earliest
stages of the operation
If part of the data is already sorted, sorting can
be completed very quickly
Shell Sort
Quick Sort
Quick sort method
Fastest sort method using the recursive call method
A method of dividing a large problem into small problems
and solving each small problem individually
Basic algorithm of quick sort method:
Choose a pivot (a key to partition the set of numbers into equal
halves)
Divide all numbers into two partitions
- All numbers in first partition are less than the pivot
- All numbers in second partition are greater than or equal to the pivot
Recursively sort numbers in the two partitions.
The sorted list of numbers is obtained by putting the pivot at the
beginning of the second partition and then appending the numbers in
the second partition after the first partition

Quick Sort
Quick Sort
Example : Sort the list of numbers [15, 17, 8, 13, 9, 24, 18,
2].
Let us choose 15 as the pivot(First element of the set)
The first partition is the list [8, 13, 9, 2] - all smaller than 15.
The second partition is the list [17, 24, 18] - all larger than 15.
The sorted first partition is [2,8,9,13]
The sorted second partition is [17, 18, 24]
The sorted list for the input numbers is obtained as follows :
put the pivot 15 at the beginning of the sorted second partition -
giving the list [15, 17, 18, 24]
append this list at the end of the first partition giving [2,8,9,13, 15, 17,
18, 24]
http://cs.oregonstate.edu/~minoura/cs162/javaProgs/sort
/QuickSort.html
Sort Algorithm
Merge sort method
The recursive call and divide-and-conquer methods are used, as in the
case of the quick sort method
All data items are divided into sections and sorting is executed in each
divided section. This is repeated until there is only one remaining
element in a data sequence
After a data sequence is divided, divided sections are sequentially
merged
The process overall is thus:
Split the original list into two halves
sort each half (using merge sort)
merge the two sorted halves together into a single sorted list W
Merge Sort
Example
34 56 78 12 45 3 99 23
34 56 78 12 45 3 99 23
34 56 78 12 45 3 99 23
34 56 78 12 45 3 99 23
34 56 12 78 3 45 23 99
12 34 56 78 3 23 45 99
3 12 23 34 45 56 78 99
sorting 100 elements:
bubble sort : 13.639008 sec/ 30.08%
insertion sort: 13.368042 sec/ 29.48% (fast for sorted
lists)
shell sort : 13.478973 sec/ 29.73% (poor
implementation)
merge sort : 2.232971 sec/ 4.92%
quick sort : 2.433960 sec/ 5.37%

Source: http://www.faqs.org/faqs/CAD/autolisp-faq/part1/section-
9.html
Sort Algorithm
Recursive Algorithm
A recursive algorithm is one which calls
itself to solve smaller versions of an input
problem
Recursive computer programs require more
memory and computation compared with
iterative algorithms, but they are simpler
and for many cases a natural way of
thinking about the problem.
Example: Algorithm for finding the k-th even

Algorithm -> Even(positive integer k)
Input: k , a positive integer
Output: k-th even natural number (the first even being 0)

Algorithm:
if k = 1, then return 0;
else return Even(k-1) + 2 .


Recursive Algorithm
Now, the same problem can be solved by an
iterative algorithm.

Algorithm -> Even(positive integer k)
Input: k, a positive integer
Output: k-th even natural number (the first even being 0)

Algorithm:
int i, even;
i := 1;
even := 0;
while( i <k ) {
even := even + 2;
i := i + 1;
}
return even .
Recursive Algorithm
Example: Algorithm for testing whether or not a
number x is a natural number

Algorithm -> Natural(a number x)
Input: A number x
Output: "Yes" if x is a natural number, else "No"

Algorithm:
if x < 0, then return "No"
else
if x = 0, then return "Yes"
else return Natural( x - 1 )

http://www.cs.queensu.ca/home/cisc121/2002f/lecturenotes/m
alamb/CISC121A2002fset11-4up.pdf
Recursive Algorithm
Character String Processing
Character string search
Simple string search
Text is compared character by character sequentially
from the head
Boyer-Moore method (BM method)
The basis of the fastest known ways to find one string
of characters in another
Data is collated while characters in the text are skipped
Simple collation
Text is compared character by character sequentially from the
head.
When a character string on which a search should be performed
is detected, the position where it is located is set as a return
value.
Principle:
comparison is repeatedly executed until the first character of a character
string to search matches a character in character strings in text.
If there is a match, each remaining character of the matched character
string is compared with each of a character string to search.
BM Method
Data is collated while characters in the text are
skipped.
Case 1: If there are no character string to search
a character at the tail, and all other characters in the first text portion do
not match any character of a character string to be searched. Therefore,
a search point is moved by the length of a character string to be
searched to allow the next search to start from that point.
BM Method
Case 2: If there is a match with a character at the tail of a
character string to be searched
Because a character at the tail of a character string matches a character in the
text, characters that precede the matched character must be compared. If all
characters match, the value of a subscript of the first character of that
matched text pattern is returned.
BM Method
Case 3: If there is a match with one character
somewhere in a character string but unmatched
with a character at the tail of a character string
Because a character at the tail of a character string matches a
character in the text, characters that precede the matched character
must be compared. If all characters match, the value of a subscript of
the first character of that matched text pattern is returned.
two
BM Method
Distance of movement
Character String Processing
2. Character string compression
Making a character string short by replacing
consecutive characters or blanks with the
number of characters
I M A T A K
# 4 I M # 3 A T A # 5 K
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 K
Special character
TBL A
TBL B
The number of blank characters
If the number of blank characters is two or one, no
compression takes place