Data Structures & Algorithms

Introduction

Instructor
Dr. Arabi Keshk
E-mail: [email protected]
As you start this course, you may
have some questions:

 What are data structures and algorithms?

 What good will it do me to know about
them?
 Why can't I just use arrays and for loops
to handle my data?
 When does it make sense to apply what I
learn here?
What is an algorithm?
 An algorithm is a step-by-step procedure
for solving a stated problem.
 The stated problem must be solvable, i.e.,
capable of solution by a step-by-step
procedure.
 The algorithm will be performed by a
processor (which may be human,
mechanical, or electronic).
 The algorithm must be expressed in steps
that the processor is capable of
performing.
 The algorithm must eventually terminate.
Algorithms vs. programs (1)
 Algorithms:
– can be performed by humans or
machines
– can be expressed in any suitable
language
– may be as abstract as we like.
 Programs:
– must be performed by machines
– must be expressed in a programming
language
– must be detailed and specific.
Data structures
 A data structure is a systematic way of
organizing a collection of data.
 A static data structure is one whose capacity
is fixed at creation.
E.g.: array.
 A dynamic data structure is one whose
capacity is variable, so it can expand or
contract at any time.
E.g.: linked list, binary tree.
 For each data structure we need algorithms
for insertion, deletion, searching, etc.
Overview of Algorithms
 Inserta new data item.
 Search for a specified item.
 Delete a specified item.
 Object-Oriented Programming
 Problems with Procedural Languages
1- Crude Organizational Units
2- Poor Modeling of the Real World

An object contains both functions and

variables.
A class is a specification—a blueprint—for
one or more objects.
A Runnable Object-Oriented Program
 // demonstrates basic OOP syntax
 // to run this program: C>java BankApp
 import java.io.*; // for I/O
 ////////////////////////////////////////////////////////////////
 class BankAccount
 {
 private double balance; // account balance
 public BankAccount(double openingBalance) // constructor
 {
 balance = openingBalance;
 }
 public void deposit(double amount) // makes deposit
 {
 balance = balance + amount;
 }
 public void withdraw(double amount) // makes withdrawal
 {
 balance = balance - amount;
 }
 public void display() // displays balance
 {
 System.out.println("balance=" + balance);
 }
 } // end class
BankAccount
 class BankApp
 {
 public static void main(String[] args)
 {
 BankAccount ba1 = new BankAccount(100.00); // create acct
 System.out.print("Before transactions, ");
 ba1.display(); // display balance
 ba1.deposit(74.35); // make deposit
 ba1.withdraw(20.00); // make withdrawal
 System.out.print("After transactions, ");
 ba1.display(); // display balance
 } // end main()
 } // end class
BankApp
The output from this program:

 Before transactions, balance=100

 After transactions, balance=154.35

Inheritance and Polymorphism

Inheritance is the creation of one class, called the
extended or derived class, from another class called
the base class.
Polymorphism involves treating objects of different
classes in the same way.
Summary

 • A data structure is the organization of data in a

computer's memory or in a disk file.
 • The correct choice of data structure allows
major improvements in program efficiency.
 • Examples of data structures are arrays, stacks,
and linked lists.
 • An algorithm is a procedure for carrying out a
particular task.
 • In Java, an algorithm is usually implemented by
a class method.
 • Many of the data structures and algorithms
described in this book are most often used to build
databases.
Properties of arrays in general
 An array is a sequence of indexed components.
 Each array component has a fixed and unique index. The
indices range from a lower bound to an upper bound:

low low+1 low+2 high–2 high–1 high indices

a
components

 The length of the array (its number of components) is

fixed when the array is constructed.
 Any array component can be efficiently accessed
(inspected or updated) using its index, in O(1) time.
Properties of arrays in Java
 A Java array’s components are either values
of some stated primitive type, or objects of
some stated class.
 A Java array of length n has lower bound 0
and upper bound n–1.
 A Java array is itself an object. It is allocated
dynamically by “new T[n]”.
class tag length 0 1 n–2 n–1
a T[] n
components
Creating an Array in Java

 int[] intArray; // defines a reference to an array

 intArray = new int[100]; // creates the array, and
// sets intArray to refer to
it
or the equivalent single-statement approach:

int[] intArray = new int[100];

int intArray[] = new int[100]; // alternative syntax

Example: Java primitive array
 Code to create, initialize, and inspect an
array of integers:
int[] primes = {2, 3, 5, 7, 11, 13};
for (int i = 0; i < primes.length; i++)
System.out.println(primes[i]);

primes
class tag length 0 1 2 3 4 5
int[] 6 2 3 5 7 11 13
Example: Java object array (1)
 Suppose that a Date object has fields y,
m, d.
 Code to create an array of Date objects:
Date[] hols = new Date[3];

class tag length 0 1 2

hols Date[] 3
Example: Java object array (2)
 Code to update the array of Date objects:
hols[0] = new Date(2002, 1, 1);
hols[1] = new Date(2001, 5, 1);
hols[2] = new Date(2001, 12, 25);
class tag length 0 1 2
hols Date[] 3
class tag y m d
Date 2001 12 25
class tag y m d
Date 2001 5 1
class tag y m d
Date 2002 1 1
The Array Workshop Applet
There are several actions you would like to be able to
perform:
 Insert a player into the data structure when
the player arrives at the field.
 Check to see if a particular player is present
by searching for his or her number in the
structure.
 Delete a player from the data structure when
the player goes home.
This applet demonstrates the three fundamental
procedures mentioned above:

 The Ins button inserts a new data item.

 The Find button searches for specified data item.
 The Del button deletes a specified data item.
Table 2.1: Duplicates OK Versus No Duplicates
The HighArray class is now wrapped around the
array. In main(), we create an array of this class and
carry out almost the same operations as in the
lowArray.java program: we insert 10 items, search
for an item—one that isn't there—and display the
array contents. Because it's so easy, we delete
three items (0, 55, and 99) instead of one, and
finally display the contents again. Here's the output:
 77 99 44 55 22 88 11 0 66 33
 Can't find 35
 77 44 22 88 11 66 33
 Abstraction

 The process of separating the how from the

what—how an operation is performed
inside a class, as opposed to what's visible
to the class user—is called abstraction.
Abstraction is an important aspect of
software engineering. By abstracting class
functionality we make it easier to design a
program, because we don't need to think
about implementation details at too early a
stage in the design process.
Ex1:
 int s(int[] arr) {
 int s= 0;
 for(int i=0; i<arr.length; i++) {
 s = s + arr[i];
 }
 return s;
}
Ex2:
 int m(int[] arr) {
 int m = arr[0];
 for (int i=1; i<arr.length; i++) {
 if (arr[i] > m) {
 m = arr[i];
 }
 return m;
Quize
1- What is an algorithm?
2- What is a data structure?
3-What is an array?
4- Write an algorithm for calculating a sum of the array
elements.
5- Write Java method that for a given array calculates
and returns the product of all even array elements?
6- Write Java method that for a given array finds
and returns the smallest array element.
7- Write Java method that for a given array finds
and returns the largest array element.
The Ordered Workshop Applet
Linear Search & Binary Search
Binary Search with the find() Method
 The OrdArray Class

 In general, the orderedArray.java program is similar to

highArray.java. The main difference is that find() uses a binary
search, as we've seen.

 Advantages & disadvantage of Ordered Arrays

The major advantage is that search times are much faster than in
an unordered array.
The disadvantage is that insertion takes longer, because all the
data items with a higher key value must be moved up to make
room.
Deletions are slow in both ordered and unordered arrays, because
items must be moved down to fill the hole left by the deleted item
Logarithms
 Linear Search: Proportional to N
T=K*N/2
T=K*N
 O(N) time
 Binary Search: Proportional to log(N)
 T = K * log2(N)
 T = K * log(N)
 O(log N) time.
 What is the running time of the linear search
algorithm? 
 What is the running time of the binary search? 
 Can we apply binary search algorithm when
searching for value 5 within the following array 44,
13, 1, 100, 234, 121, 55, 16? Explain why?
 When searching for value 350 within the following
array 4, 13, 21, 100, 234, 321, 455, 514
It is enough to check only array elements with which
indices?
  Using the big-O notation write the asymptotic upper
bounds for the following expressions:
 a)        n3 +10n2 +100n
 b)       40n5 + 3n
 c)       log n + 1000
 d)       n log n + 200 log n
History
 Euclid (~ 300 BCE), Eratosthenes (~ 200
BCE)
– arithmetic, geometric algorithms.
 al Khwarizmi (~ 800)
– arithmetic, algebraic, geometric
algorithms.
 Napier (~ 1600)
– arithmetic using logarithms.
 Newton (~ 1700)
– differentiation, integration.
 Turing (~ 1940)
– code breaking, solvability.
Introduction
 Algorithms
– History
– Examples
– Algorithms vs. programs
– Performance measure
 Data structures
 Abstract data types
Example: finding a midpoint (1)
 Midpoint algorithm:
To find the midpoint of a given straight-line
segment AB:
1. Draw intersecting circles of equal radius,
centered at A and B
respectively.
2. Let C and D be the points where the circles
intersect.
3. Draw a straight line between C and D.
4. Let E be the point where CD intersects AB.
5. Terminate with answer E.
 Could be performed by a human equipped
with drawing instruments.
Example: finding a midpoint (2)

 Animation:
1. Draw intersecting circles

of equal radius, centered

A
at A and B respectively. D
2. Let C and D be the
points where the circles E
intersect.
3. Draw a straight line C
between C and D.
4. Let E be the point B
where CD intersects AB.
5. Terminate with answer
E.
Example: GCDs (1)
 The greatest common divisor (GCD) of two
positive integers is the largest integer that
exactly divides both. E.g., the GCD of 77
and 21 is 7.
 Euclid’s GCD algorithm:
To compute the GCD of positive integers m and n:
1. Set p to m, and set q to n.
2. Until q exactly divides p, repeat:
2.1. Set p to q, and set q to (p modulo q).
3. Terminate with answer q.
 Could be performed by a human, perhaps
equipped with an abacus or calculator.
Example: GCDs (2)

 Animation:
To compute the GCD of positive integers m and n:
1. Set p to m, and set q to n.
2. Until q exactly divides p, repeat:
2.1. Set p to q, and set q to (p modulo q).
3. Terminate with answer q.

m 77 n 21

p 14
77
21 q 14
21
7
Example: GCDs (3)

 Implementation in Java:
static int gcd (int m, int n) {
// Return the greatest common divisor of
positive integers m and n.
int p = m, q = n;
while (p % q != 0) {
int r = p % q;
p = q; q = r;
}
return q;
}
Introduction
 Algorithms
– History
– Examples
– Algorithms vs. programs
– Performance measure
 Data structures
 Abstract data types
Algorithms vs. programs (2)
 Here we express our algorithms in (precise)
English.
 Steps may be numbered consecutively:
1. Do this. Do this and then do that.
2. Do that.
 The extent of a conditional is indicated by
indentation:
7. If …: Do this and then do that, but
7.1. Do this. only if condition … is true.
7.2. Do that.
 The extent of a loop is indicated by indentation:
8. While …, repeat: Do this and then do that, as
8.1. Do this.
long as condition … is true.
8.2. Do that.
 Algorithms vs. programs (3)
 If we wish to use the algorithm on a computer,
we must first code it in a programming
language.
 There may be many ways of coding the
algorithm, and there is a wide choice of
programming languages. But all the resulting
programs are implementations of the same
underlying algorithm.
 Here we express our implementations in Java.
(Alternatives would be C, Pascal, Ada, etc.)
Introduction
 Algorithms
– History
– Examples
– Algorithms vs. programs
– Performance measure
 Data structures
 Abstract data types
Efficiency
 Given several algorithms to solve the
same problem, which algorithm is “best”?
 Given an algorithm, is it feasible to use it at
all? In other words, is it efficient enough
to be usable in practice?
 How much time does the algorithm
require?
 How much space (memory) does the
algorithm require?
 In general, both time and space
requirements depend on the algorithm’s
input (typically the “size” of the input).
Example: efficiency
 Hypothetical profile of two sorting algorithms:

4
Key:
3 Algorithm A
Algorithm B
time
2
(ms)

0
10 20 30 40 items to be sorted, n
• Algorithm B’s time grows more slowly than A’s.
Efficiency: measuring time
 Measure time in seconds?
+ is useful in practice
– depends on language, compiler, and
processor.
 Count algorithm steps?
+ does not depend on compiler or processor
– depends on granularity of steps.
 Count characteristic operations? (e.g.,
arithmetic ops in math algorithms,
comparisons in searching algorithms)
+ depends only on the algorithm itself
+ measures the algorithm’s intrinsic efficiency.
Remedial Mathematics: Powers

 Consider a number b and a non-negative

integer n.
Then b to the power of n (written bn) is the
multiplication
of n copies of b:
bn = b  …  b
E.g.: b3 = b  b  b
b2 = b  b
b1 = b
b0 = 1
Remedial Mathematics: Logarithms

 Consider a positive number y. Then the

logarithm of y to the base 2 (written log2 y) is the
number of copies of 2 that must be multiplied
together to equal y.
 If y is a power of 2, log2 y is an integer:
since 2 = 10

E.g.: log2 1= 0 since 2 = 21

log2 2 = 1
since 2 = 42

log2 4 = 2
log2 8 = 3 since 2 = 83

• If y is not a power of 2, log2 y is fractional:

E.g.: log2 5  2.32
log2 7  2.81
Remedial Mathematics: Logarithm laws

 log2 (2n) = n
 log2 (xy) = log2 x + log2 y
 log2 (x/y)= log2 x – log2 y

since the no. of 2s multiplied to

make xy is the sum of the no. of 2s
multiplied to make x and the no. of
2s multiplied to make y
Remedial Mathematics: Logarithms example

 How many times must we halve the value of n

(discarding any remainders) to reach 1?
 Suppose that n is a power of 2:
E.g.: 8421 (8 must be halved 3 times)
16  8  4  2  1 (16 must be halved 4 times)
If n = 2m, n must be halved m times.
 Suppose that n is not a power of 2:
E.g.: 9421 (9 must be halved 3 times)
15  7  3  1 (15 must be halved 3 times)
If 2m < n < 2m+1, n must be halved m times.
Remedial Mathematics: Logarithms example

 In general, n must be halved m times if:

2m  n < 2m+1
i.e., log2 (2m)  log2 n < log2 (2m+1)
i.e., m  log2 n < m+1
i.e., m = floor(log2 n).
 The floor of x (written floor(x) or x) is the
largest integer not greater than x.
 Conclusion: n must be halved floor(log n)
2
times to reach 1.
 Also: n must be halved floor(log n)+1 times to
2
reach 0.
Example: power algorithms (1)
 Simple power algorithm:
To compute bn:
1. Set p to 1.
2. For i = 1, …, n, repeat:
2.1. Multiply p by b.
3. Terminate with answer p.
Example: power algorithms (2)
 Analysis (counting multiplications):
Step 2.1 performs a multiplication.
This step is repeated n times.
No. of multiplications = n
Example: power algorithms (3)
 Implementation in Java:
static int power (int b, int n) {
// Return bn (where n is non-negative).
int p = 1;
for (int i = 1; i <= n; i++)
p *= b;
return p;
}
Example: power algorithms (4)
 Idea: b1000 = b500  b500. If we know b500,
we can compute b1000 with only 1 more
multiplication!
 Smart power algorithm:
To compute bn:
1. Set p to 1, set q to b, and set m to n.
2. While m > 0, repeat:
2.1. If m is odd, multiply p by q.
2.2. Halve m (discarding any
remainder).
2.3. Multiply q by itself.
3. Terminate with answer p.
Example: power algorithms (5)
 Analysis (counting multiplications):
Steps 2.1–3 together perform at most 2
multiplications.
They are repeated as often as we must
halve the value of n (discarding any
remainder) until it reaches 0, i.e.,
floor(log2 n) + 1 times.
Max. no. of multiplications =
= 2(floor(log2 n) + 1)
= 2 floor(log2 n) + 2
Example: power algorithms (6)
 Implementation in Java:
static int power (int b, int n) {
// Return bn (where n is non-negative).
int p = 1, q = b, m = n;
while (m > 0) {
if (m%2 != 0) p *= q;
m /= 2; q *= q;
}
return p;
}
Example: power algorithms (7)
 Comparison: simple power
50
algorithm

40

30
multiplications
smart power
20 algorithm

10

0
0 10 20 30 40 50
n
Complexity
 For many interesting algorithms, the exact
number of operations is too difficult to
analyse mathematically.
 To simplify the analysis:
– identify the fastest-growing term
– neglect slower-growing terms
– neglect the constant factor in the fastest-
growing term.
 The resulting formula is the algorithm’s
time complexity. It focuses on the growth
rate of the algorithm’s time requirement.
 Similarly for space complexity.
Example : analysis of power algorithms (1)

 Analysis of simple power algorithm

(counting multiplications):
No. of multiplications = n
Time taken is approximately
proportional to n.
Time complexity is of order n. This is
written O(n).
Example : analysis of power algorithms (2)

 Analysis of smart power algorithm (counting

multiplic’ns):
Neglect slow-growing term, +2.
Max. no. of multiplications =
2 floor(log2 n) + 2
Simplify to 2 floor(log2 n)
Neglect constant factor, 2.
then to floor(log2 n)
then to log2 n Neglect floor(), which on average
subtracts 0.5, a constant term.

Time complexity is of order log n.

This is written O(log n).
Example : analysis of power algorithms (3)

 Comparison: 50
n

40

30

20

10
log n

0
0 10 20 30 40 50
n
O-notation (1)
 We have seen that an O(log n) algorithm is
inherently better than an O(n) algorithm
for large values of n.
O(log n) signifies a slower growth rate
than O(n).
 Complexity O(X) means “of order X”,
i.e., growing proportionally to X.
Here X signifies the growth rate,
neglecting slower-growing terms and
constant factors.
O-notation (2)
 Common time complexities:
O(1) constant time (feasible)
O(log n) logarithmic time (feasible)
O(n) linear time (feasible)
O(n log n) log linear time (feasible)
O(n2) quadratic time (sometimes
feasible)
O(n3) cubic time (sometimes
feasible)
O(2n) exponential time (rarely
feasible)
Growth rates (1)
 Comparison of growth rates:
1 1 1 1 1
log n 3.3 4.3 4.9 5.3
n 10 20 30 40
n log 33 86 147 213
n
n2 100 400 900 1,600
n3 1,000 8,000 27,000 64,000
2n 1,024 1.0 1.1 1.1
million billion trillion
Growth rates (2)
 Graphically: 100
2n n2 n log n

80

60
n
40

20
log n

0
0 10 20 30 40 50
n
Example: growth rates (1)
 Consider a problem that requires n data items to
be processed.
 Consider several competing algorithms to solve
this problem. Suppose that their time
requirements on a particular processor are as
follows:
Algorithm Log: 0.3 log2 nseconds
Algorithm Lin: 0.1 n seconds
Algorithm LogLin: 0.03 n log2 nseconds
Algorithm Quad: 0.01 n2 seconds
Algorithm Cub: 0.001 n3seconds
Algorithm Exp: 0.0001 2nseconds
 Example: growth rates (2)
 Compare how many data items (n)
each algorithm can process in 1, 2,
Log …, 10 seconds:
Lin
LogLi
n
Quad
Cub
0:06
0:09
0:08
0:07
0:05
0:04
0:03
0:02
0:10
0:00
0:01
Exp
0 10 20 30 40 50 60 70 80 90 100
n
Introduction
 Algorithms
– History
– Examples
– Algorithms vs. programs
– Performance measure
 Data structures
 Abstract data types
Data structures
 A data structure is a systematic way of
organizing a collection of data.
 A static data structure is one whose capacity
is fixed at creation.
E.g.: array.
 A dynamic data structure is one whose
capacity is variable, so it can expand or
contract at any time.
E.g.: linked list, binary tree.
 For each data structure we need algorithms
for insertion, deletion, searching, etc.
Example: representing strings

 Possible data structures to represent

the string “Java”:
0 1 2 3
Array: ‘J’ ‘a’ ‘v’ ‘a’

Linked list: ‘J’ ‘a’ ‘v’ ‘a’

 Example: representing sets
 Possible data structures to represent the
set of words
{bat, cat, mat, rat, sat}:
0 1 2 3 4 5 6 7
(Sorted) bat cat mat rat sat
array:
(Sorted)
bat cat mat rat sat
linked list:
Binary
cat
search
tree:
bat rat

mat sat
Introduction
 Algorithms
– History
– Examples
– Algorithms vs. programs
– Performance measure
 Data structures
 Abstract data types
Abstract Data Types
 An abstract data type (ADT) is an organized
collection of information and a set of
operations used to manage that information
 The set of operations defines the interface to
the ADT
 In one sense, as long as the ADT fulfills the
promises of the interface, it doesn't matter
how the ADT is implemented
 Objects are a perfect programming
mechanism to create ADTs because their
internal details are encapsulated
Information Hiding
 ADT hides the implementation details of the
operations and the data from the users of the ADT

 Users can use the operations of an ADT without

knowing how the operation is implemented

 Examples

– A deck of playing cards

– A set of index cards containing contact information
– Telephone numbers stored in your cellular phone
Structure Properties
1. Each one is a collection of elements
2. There is a first element
3. There is a second element, third element, and so on
4. There is a last element
5. Given an element other than the last element, there
is a “next” element
6. Given an element other than the first element, there
is a “previous” element
7. An element can be removed from the collection
8. An element may be added to the collection
9. A specified element can be located in the collection
by systematically going through the collection

 This structure commonly appears in various

applications.
Abstract Data Types
An abstract data type (ADT)
 is a type for encapsulating related data
 is abstract in the sense that it hides distracting
implementation details
Two Parts of Every ADT

Associated
Operations ADT behavior

Data ADT state