DEFINITION OF TERMS

A data structure is viewed as an organization or structuring for a collection of data

items so that computational operations can be performed effectively. The way of
organizing all data items that consider not only the elements stored but also the
relationship between the elements; examples o data structures are stacks, queues,
linked lists, trees and graphs.

An algorithm is simply a set of logical instructions for carrying out a predefined task.

Problems, Algorithms, and Programs

Problems, Algorithms, and Programs

Problems

Programmers commonly deal with problems, algorithms, and computer

programs. These are three distinct concepts.

A problem is a task to be performed. It is best thought of in terms of inputs and

matching outputs. A problem definition should not include any constraints on
how the problem is to be solved. The solution method should be developed only
after the problem is precisely defined and thoroughly understood. However, a
problem definition should include constraints on the resources that may be
consumed by any acceptable solution. For any problem to be solved by a
computer, there are always such constraints, whether stated or implied. For
example, any computer program may use only the main memory and disk space
available, and it must run in a "reasonable" amount of time.

Problems can be viewed as functions in the mathematical sense. A function is a

matching between inputs (the domain) and outputs (the range). An input to a
function might be a single value or a collection of information. The values making
up an input are called the parameters of the function. A specific selection of
values for the parameters is called an instance of the problem. For example, the
input parameter to a sorting function might be an array of integers. A particular
array of integers, with a given size and specific values for each position in the
array, would be an instance of the sorting problem. Different instances might
generate the same output. However, any problem instance must always result
in the same output every time the function is computed using that particular input.

This concept of all problems behaving like mathematical functions might not
match your intuition for the behavior of computer programs. You might know of
programs to which you can give the same input value on two separate
occasions, and two different outputs will result. For example, if you type date to
a typical Linux command line prompt, you will get the current date. Naturally the
date will be different on different days, even though the same command is given.
However, there is obviously more to the input for the date program than the
command that you type to run the program. The date program computes a
function. In other words, on any particular day there can only be a single answer
returned by a properly running date program on a completely specified input. For
all computer programs, the output is completely determined by the program's
full set of inputs. Even a "random number generator" is completely determined
by its inputs (although some random number generating systems appear to
get around this by accepting a random input from a physical process beyond t h e
user’s control). The limits to what functions can be implemented by programs
is part of the domain of Computability.

Examples of problems types:

Sorting – a set of items

Searching – among a set of items

String processing – text, bit strings, gene sequences

Graphs – model objects and their relationships

Numerical – continuous math: solving equations, evaluating functions

Combinatorial – find desired permutation, combination or subset.

THE SOLUTION OF A PROBLEM HAS TWO PARTS: (DATA STRUCTURE AND

ALGORITHM)

Data structure: -a systematic way of organizing and accessing data

How to solve the problem?

Algorithm: - a step-by-step procedure for performing some task in finite time (is a step
list that solves a problem)

An effective method for solving a problem expressed in a finite sequence of steps

 There are common algorithms solving common problems

 The more you think algorithmically, the easier you solve real world issues
  Complex algorithms constructed on simple ones

Algorithms

An algorithm is a method or a process followed to solve a problem. If the

problem is viewed as a function, then an algorithm is an implementation for the
function that transforms an input to the corresponding output. A problem can be
solved by many different algorithms. A given algorithm solves only one problem
(i.e., computes a particular function). The advantage of knowing several
solutions to a problem is that solution A might be more efficient than solution
B for a specific variation of the problem, or for a specific class of inputs to the
problem, while solution B might be more efficient than A for another variation
or class of inputs. For example, one sorting algorithm might be the best for
sorting a small collection of integers (which is important if you need to do this
many times). Another might be the best for sorting a large collection of integers.
A third might be the best for sorting a collection of variable-length strings.

By definition, something can only be called an algorithm if it has all of the following
properties.

1. It must be correct. In other words, it must compute the desired function,

converting each input to the correct output. Note that every algorithm implements
some function, because every algorithm maps every input to some output (even
if that output is a program crash). At issue here is whether a given algorithm
implements the intended function.

2. It is composed of a series of concrete steps. Concrete means that the

action described by that step is completely understood --- and doable --- by the
person or machine that must perform the algorithm. Each step must also be
doable in a finite amount of time. Thus, the algorithm gives us a "recipe" for
solving the problem by performing a series of steps, where each such step is
within our capacity to perform. The ability to perform a step can depend on who or
what is intended to execute the recipe. For example, the steps of a cookie recipe
in a cookbook might be considered sufficiently concrete for instructing a human
cook, but not for programming an automated cookie-making factory.

3. There can be no ambiguity as to which step will be performed next. Often it

is the next step of the algorithm description. Selection (e.g., the if statement) is
normally a part of any language for describing algorithms. Selection allows a
choice for which step will be performed next, but the selection process is
unambiguous at the time when the choice is made.
4. It must be composed of a finite number of steps. If the description for the
algorithm were made up of an infinite number of steps, we could never hope to
write it down, nor implement it as a computer program. Most languages for
describing algorithms (including English and "pseudo code") provide some way to
perform repeated actions, known as iteration. Examples of iteration in
programming languages include the while and for loop constructs. Iteration
allows for short descriptions, with the number of steps actually performed
controlled by the input.

5. It must terminate. In other words, it may not go into an infinite loop.

Programs

We often think of a computer program as an instance, or concrete

representation, of an algorithm in some programming language. Algorithms are
usually presented in terms of programs, or parts of programs. Naturally, there are
many programs that are instances of the same algorithm, because any modern
computer programming language can be used to implement the same
collection of algorithms (although some programming languages can make life
easier for the programmer). To simplify presentation, people often use the terms
"algorithm" and "program" interchangeably, despite the fact that they are really
separate concepts. By definition, an algorithm must provide sufficient detail that
it can be converted into a program when needed.

The requirement that an algorithm must terminate means that not all computer
programs meet the technical definition of an algorithm. Your operating system is
one such program. However, you can think of the various tasks for an operating
system (each with associated inputs and outputs) as individual problems, each
solved by specific algorithms implemented by a part of the operating system
program, and each one of which terminates once its output is produced.

To summarize: A problem is a function or a mapping of inputs to outputs. An

algorithm is a recipe for solving a problem whose steps are concrete and
unambiguous. Algorithms must be correct, of finite length, and must terminate
for all inputs. A program is an instantiation of an algorithm in a programming
language.
COMPLEXITY OF ALGORITHMS

We define the complexity of an algorithm as the amount of computer resources that it

uses (memory and time). We have several algorithms for solving the same problem.
Which one is better?

Criteria that we can use to compare algorithms:

• Conceptual simplicity

• Difficulty to implement

• Difficulty to modify

• Running time

• Space (memory) usage

FUNCTIONS TO DEFINE COMPLEXITY OF AN ALGORITHM

We are particularly interested in two computer resources: memory and time.

Consequently, we define two types of complexity functions:

• Space complexity: amount of memory/ number of memory cells that the algorithm
needs.

• Time complexity: amount of time needed by the algorithm to complete/ number of

(machine) instructions which a program executes during its running time. (The
mathematical notation used to express time complexities is the asymptotic notation).

For both kinds of complexity functions, we can define 3 cases:

• Best case: Least amount of time needed by the algorithm to solve an instance of
the problem of size n.

• Worst case: Largest amount of time needed by the algorithm to solve an instance of
the problem of size n.

• Average case: time to solve instance 1 of size n + time to solve instance 2 of size n
+ time to solvelast instance of size n/ number of instances of size n

• The asymptotic time complexity can be denoted with a big “O” letter. OR
Landau’s symbols O and ѳ (theta) instead of using c a constant for notation
Big-O Notation

This notation gives the tight upper bound of the given function. Generally, it is
represented as f(n) = O(g(n)). That means, at larger values of n, the upper
bound of f(n) is g(n).

Omega-Ω Notation

Similar to the O discussion, this notation gives the tighter

lower bound of the given algorithm and we represent it as f(n)
= Ω(g(n)). That means, at larger values of n, the
tighter lower bound of f(n) is g(n).

Theta-Θ Notation

This notation decides whether the upper and lower bounds of a given function
(algorithm) are the same. The average running time of an algorithm is always between
the lower bound and the upper bound. If the upper bound (O)
and lower bound (Ω) give the same result, then the Θ
notation will also have the same rate of growth.

Examples of complexity

• O(1) – constant time

• O(n) – linear time

2
• O(n ) – square time

 O(nn) – cubic and other polynomials

• O(log n) – logarithmic time

• O(nlog n) – linear logarithmic time

n
• (2 ) – exponential time
Examples of Data Structures and the Associated time complexity

Data Insert Find Remove Finding

structure/Operation (Max/Min)

Array O(1) O(n) O(n) O(n)

-take element to -look at the -sort the array

back of the entire array to make the
array and find it whole filled

Sorted Array O(n) O(log n) O(n) O(1)

-since we have -find it and -we know it’s

to maintain the adjust the at the end of
sorted array array and the array
maintain
sorted
elements

Sorted Linked List O(n) O(n) O(n) O(n)

-to maintain the

sorted Linked
List

Hash set O(1) O(1) O(1) O(n)

-very efficient -very efficient -very efficient -look at all

way to insert way to find way to remove elements in a
table e.g. look
at all chains

Stack Push – O(1) O(n) – pop all Pop – O(1) O(n)-look at

elements from the entire
the stack and stack by
put the stack popping @
to its original elements,
shape pushing them
to original
format

LinkedList O(1) O(n) O(n) O(n)

-traverse the -traverse the -look at all

 list list and elements in
remove it the list

Queue Enq- O(1) O(n)-dequeue, Deq- O(n) -traverse

look at the O(1),Dequeue through the
elements and elements entire queue
find and put
queue to
original shape

ALGORITHM DESIGN TECHNIQUES

An algorithm design technique is a general approach to solve problems algorithmically.

It can be applied to a variety of problems from different areas of computing.

These techniques provide us guidance in designing algorithms for new problems

They represent a collection of tools useful for applications.

Greedy Algorithms

Divide and Conquer

Brute Force

Dynamic programming

Transform and conquer

Backtracking

Genetic Algorithms

GREEDY ALGORITHM

-is an algorithm that follows the problem solving [method/process] heuristic of making
the locally optimal choice at each stage with the hope of finding a global optimum.

"Take what you can get now" strategy. Work in phases. In each phase the currently best
decision is made

The greedy method is a general algorithm design paradigm, built on the following
elements:

Configurations: different choices, collections, or values to find.

Objective function: a score assigned to configurations, which we want to either
maximize or minimize.

Examples of greedy algorithms

Research Travelling Salesman problem

Dijkstra's algorithm

(shortest path is weighted graphs)

Prim's algorithm,

Kruskal's algorithm

(minimal spanning tree in weighted graphs)

Money exchange problem

public class GreedyChange {

public static void main(String[] args) throws Exception {

int amount =132;

int denomination [] = {100,50,20,10,5,2,1}; //US dollar notes

int num;

for (int i =0; i< denomination.length; i++){

if(denomination[i]<=amount){

num =amount/denomination[i];

System.out.println(num+ "" + "$"+denomination[i]);

amount=amount -(num*denomination[i]);

}
Huffman Trees

Other areas of applications:

Job scheduling problem on a computer: you have 9 jobs and 3 processors so you might
start with longest-running jobs first on whatever processor available OR alternatively
start with the shortest job first.

DIVIDE AND CONQUER ALGORITHM

Reduce the problem to smaller problems (by a factor of at least 2) solved recursively
and then combine the solutions

Is a general algorithm design paradigm with steps:?

Divide: divide the input data S into two or more disjoint subsets S 1 and S2

Recur: solve the sub problems recursively

Conquer: Combine the solutions for S1, S2… Sn for S. e.g pow(x,y) and log (n, base)

- This approach can lead us to an algorithm which is more efficient than the brute-
force algorithm
E.g find the closest pair of points:
(a) Divide/split into left half and a right half of approximately n/2 points each
[points by X, the x-axis]
(b) Recursively find the closest pair on each of these two sub-problems
(c) Combine sub-problems solutions to find closest pair in given point set.

Points to be permutations of each other. Find the closest pair (never duplicate
points, just reference them)
Always return the smaller of two values for x
Distance(d) = min (dL, dR)
*No point in the right half closest to the left half vice-versa so not to worry about
the sub-division not in the sub-division XL or XR

Combine steps:
Construct Y-strip to reference all points in the order of points by Y coordinate (sorted by
Y coordinate).
Step1:
With X-coordinate >(greater)XR- d && <(less) XL + d

Step2:
Find any pair within Ystrip with distance < d (one from each side). Consider each point
in Ystrip looking upwards.
*As you’re processing the points recursively, SORT them based on the Y-coordinate
*Compare/selected points with Y-coordinate
Examples of Divide and Conquer Algorithms:

Binary Search

Merge sort

Quick sort

Tree traversal

In general, problems that can be defined recursively

Decrease and Conquer

Reduce the problem to smaller problems solved recursively and then combine
the solutions

Examples of decrease-and-conquer algorithms:

BRUTE FORCE

Enumerates all possible problem solving techniques and tests for validity in problem
solving until the correct one is found. Algorithms using Brute force are not always
efficient.

e.g the asymptotic time complexity for brute force analysis of comparing all pairs of
points. Compute the distance and compare with the best so far.

E.g points 1-------------> compare with n- 1 (points which follows), look at others not
point 1

2-------------> compare with n- 2

3-------------> compare with n- 3

4-------------> compare with n- 4

. 1
 n…………… 0, nothing to compare with because all points are exhausted

*we’re comparing 1+2+3+…(n-2)+(n-1) i.e n(n-1)/2 pairs

*take the ends and get a sum of 1+2+…(n-2)+(n-1) which gives us an iteration equal to
nC2

nC2 = n(n-1)/2 =n2/2-n/2 = (Constant) C.n2

Based on the problem’s statement and definitions of the concepts involved.

Examples:

Sequential search

Exhaustive search: TSP, knapsack

Simple sorts: selection sort, bubble sort

Computing n! for n>=1

Factorial (n)

F1

FOR i 1, n Do

F  f*i

End for

Return f