Chapter One

Introduction to Data Structures and

Algorithm Analysis
 Introduction
• A program is a set of instruction written in
order to solve a problem
• A solution to a problem actually consists of
two things:
• A way to organize the data
• Sequence of steps to solve the problem
• The way data are organized in a computers
memory is said to be Data Structure
• the sequence of computational steps to solve
a problem is said to be an algorithm
• So a program is nothing but data structures
plus algorithms
• the first step to solve a problem is obtaining
ones own abstract view, or model, of the
problem
• This process of modeling is called abstraction
• The model defines an abstract view to the
problem
• the model focuses only on problem related
stuff and that a programmer tries to define
the properties of the problem
• These properties are
• The data which are affected and
• The operations that are involved in the problem
• The process of abstraction will create a well-
defined entity that can be properly handled
• These entities define the data structure of the
program
• An entity with the properties just described is
called an abstract data type (ADT)
 Abstract Data Types (ADT)
• An ADT consists of an abstract data structure
and operations
• Or it can be viewed as an abstraction of a data
structure
• The ADT specifies:
• What can be stored in the Abstract Data Type
• What operations can be done on/by the Abstract Data
Type.
• For example, if we are going to model
employees of an organization:
– This ADT stores employees with their relevant
attributes and discarding irrelevant attributes.
– This ADT supports hiring, firing, retiring, …
operations
• There are lots of formalized and standard
Abstract data types such as Stacks, Queues,
Trees, etc.
Q . Do all characteristics need to be modeled?
 Abstraction
• Abstraction is a process of classifying
characteristics as relevant and irrelevant for
the particular purpose at hand and ignoring
the irrelevant ones

• Applying abstraction correctly is the essence

of successful programming
• How do data structures model the world or some
part of the world?
– The value held by a data structure represents some
specific characteristic of the world
– The characteristic being modeled restricts the possible
values held by a data structure
– The characteristic being modeled restricts the possible
operations to be performed on the data structure.
• Note: Notice the relation between characteristic,
value, and data structures
 Algorithms
• An algorithm is a well-defined computational
procedure that takes some value or a set of values as
input and produces some value or a set of values as
output
• Data structures model the static part of the world
– They are unchanging while the world is changing
• In order to model the dynamic part of the world we
need to work with algorithms
– Algorithms are the dynamic part of a program’s world
model
• An algorithm transforms data structures from one
state to another state in two ways:
– An algorithm may change the value held by a data
structure
– An algorithm may change the data structure itself
• The quality of a data structure is related to its ability
to successfully model the characteristics of the world
• the quality of an algorithm is related to its ability to
successfully simulate the changes in the world
• However, independent of any particular world
model, the quality of data structure and
algorithms is determined by their ability to
work together well
• Generally speaking, correct data structures
lead to simple and efficient algorithms and
correct algorithms lead to accurate and
efficient data structures
 Properties of an algorithm
• Finiteness: Algorithm must complete after a finite number
of steps.
• Definiteness: Each step must be clearly defined, having one
and only one interpretation. At each point in computation,
one should be able to tell exactly what happens next.
• Sequence: Each step must have a unique defined preceding
and succeeding step. The first step (start step) and last step
(halt step) must be clearly noted.
• Feasibility: It must be possible to perform each instruction.
• Correctness: It must compute correct answer for all possible
legal inputs.
• Language Independence: It must not depend on any
one programming language.
• Completeness: It must solve the problem completely.
• Effectiveness: It must be possible to perform each
step exactly and in a finite amount of time.
• Efficiency: It must solve with the least amount of
computational resources such as time and space.
• Generality: Algorithm should be valid on all possible
inputs.
• Input/Output: There must be a specified number of
input values, and one or more result values.
 Algorithm Analysis Concepts
• Algorithm analysis refers to the process of
determining how much computing time and
storage that algorithms will require
– it’s a process of predicting the resource requirement of
algorithms in a given environment
• there are many possible algorithms for a given
problem
– choose the best algorithm for the problem
– how ?
• The main resources are:
– Running Time
– Memory Usage
– Communication Bandwidth
• Running time is usually treated as the most
important since computational time is the
most precious resource in most problem
domains
• There are two approaches to measure the
efficiency of algorithms:
– Empirical: Programming competing algorithms
and trying them on different instances.
– Theoretical: Determining the quantity of
resources required mathematically (Execution
time, memory space, etc.) needed by each
algorithm.
• However, it is difficult to use actual clock-time as a
consistent measure of an algorithm’s efficiency ,
because clock-time can vary based on many things
like
– Specific processor speed
– Current processor load
– Specific data for a particular run of the program
• Input Size
• Input Properties
– Operating Environment
• so, we can analyze an algorithm according to
the number of operations required, rather
than according to an absolute amount of time
involved
• It shows how an algorithm’s efficiency changes
according to the size of the input
 Complexity Analysis
• Complexity Analysis is the systematic study of
the cost of computation, measured either in
time units or in operations performed, or in
the amount of storage space required
• The goal is to have a meaningful measure that
permits comparison of algorithms
independent of operating platform
• There are two things to consider:
– Time Complexity: Determine the approximate
number of operations required to solve a problem
of size n.
– Space Complexity: Determine the approximate
memory required to solve a problem of size n.
• Complexity analysis involves two distinct
phases:
– Algorithm Analysis: Analysis of the algorithm or
data structure to produce a function T(n) that
describes the algorithm in terms of the operations
performed in order to measure the complexity of
the algorithm.
– Order of Magnitude Analysis: Analysis of the
function T(n) to determine the general complexity
category to which it belongs.
• Note : There is no generally accepted set of
rules for algorithm analysis. However, an
exact count of operations is commonly used
Analysis Rules:
Examples:
 Formal Approach to Analysis
• The above process can be simplified by using
some formal approach in which case we can
ignore initializations, loop control, and book
keeping
 For Loops: Formally
• In general, a for loop translates to a summation. The
index and bounds of the summation are the same as
the index and bounds of the for loop.

N
for (int i = 1; i <= N; i++) {

}
sum = sum+i; 1  N
i 1
• Suppose we count the number of additions that are
done. There is 1 addition per iteration of the loop,
hence N additions in total.
 Nested Loops: Formally
• Nested for loops translate into multiple
summations, one for each for loop.
for (int i = 1; i <= N; i++) {
for (int j = 1; j <= M; j++) { N M N

}
sum = sum+i+j;  2   2M  2MN
i 1 j 1 i 1
}

• Again, count the number of additions. The

outer summation is for the outer for loop.
Consecutive Statements: Formally
• Add the running times of the separate blocks
of your code
for (int i = 1; i <= N; i++) {
sum = sum+i;
} N  N N 
   
2
for (int i = 1; i <= N; i++) { 1  2  N  2 N
for (int j = 1; j <= N; j++) {  i 1   i 1 j 1 
sum = sum+i+j;
}
}
 Conditionals: Formally
• If (test) s1 else s2: Compute the maximum of
the running time for s1 and s2.
if (test == 1) {
for (int i = 1; i <= N; i++) {  N N N 
sum = sum+i; max 1,   2  
}}  i 1 i 1 j 1 
else for (int i = 1; i <= N; i++) {
for (int j = 1; j <= N; j++) {  
max N , 2 N 2  2 N 2
sum = sum+i+j;
}}
• Example:
• Suppose we have hardware capable of
executing 106 instructions per second. How
long would it take to execute an algorithm
whose complexity function was: T (n) = 2n2 on
an input size of n=108?
 solution
• The total number of operations to be
performed would be T (108):
• T(108) = 2*(108)2 =2*1016
• The required number of seconds would be
given by T(108)/106 so
• Running time =2*1016/106 = 2*1010
• The number of seconds per day is 86,400 so
this is about 231,480 days (634 years)
 Measures of Times
• In order to determine the running time of an algorithm it
is possible to define three functions Tbest(n), Tavg(n) and
Tworst(n) as the best, the average and the worst case
running time of the algorithm respectively
• Average Case (Tavg): The amount of time the algorithm
takes on an "average" set of inputs.
• Worst Case (Tworst): The amount of time the algorithm
takes on the worst possible set of inputs.
• Best Case (Tbest): The amount of time the algorithm takes
on the smallest possible set of inputs.
• We are interested in the worst-case time,
since it provides a bound for all input
• this is called the “Big-Oh” estimate
 Asymptotic Analysis
• It is concerned with how the running time of
an algorithm increases with the size of the
input in the limit, as the size of the input
increases without bound.
• There are five notations used to describe a
running time function. These are:
– Big-Oh Notation (O)
– Big-Omega Notation ()
– Theta Notation ()
– Little-o Notation (o)
– Little-Omega Notation ()
 The Big-Oh Notation
• Big-Oh notation is a way of comparing
algorithms and is used for computing the
complexity of algorithms
• It’s only concerned with what happens for a
very large value of n
• Therefore only the largest term in the
expression (function) is needed
• Formal Definition: f (n)= O (g (n)) if there exist
c, k ∊ ℛ+ such that for all n≥ k, f (n) ≤ c.g(n).
• The following points are facts that you can use
for Big-Oh problems:
– 1<=n for all n>=1
– n<=n2 for all n>=1
– 2n <=n! for all n>=4
– log2n<=n for all n>=2
– n<=nlog2n for all n>=2
 Examples:
1. f(n)=10n+5 and g(n)=n. Show that f(n) is O(g(n)).
– To show that f(n) is O(g(n)) we must show that
constants c and k such that
– f(n) <=c.g(n) for all n>=k
Or 10n+5<=c.n for all n>=k
– Try c=15. Then we need to show that 10n+5<=15n
– Solving for n we get: 5<5n or 1<=n.
– So f(n) =10n+5 <=15.g(n) for all n>=1.
(c=15,k=1).
2. f(n) = 3n2 +4n+1. Show that f(n)=O(n2).
– 4n <=4n2 for all n>=1 and 1<=n2 for all n>=1
– 3n2 +4n+1<=3n2+4n2+n2 for all n>=1
<=8n2 for all n>=1
• So we have shown that f(n)<=8n2 for all n>=1
Therefore, f (n) is O(n2) (c=8,k=1)