ِ‫ﺑِﺳْ مِ ٱ ﱠٰ ِ ٱﻟرﱠ ﺣْ َٰﻣ ِن ٱﻟرﱠ ﺣِﯾم‬

In the name of Allah, Most Gracious, Most Merciful

CSE 4303
Data Structure

Topic: Introduction to data structures, Complexity Time-Space Tradeoff

Asaduzzaman Herok
Lecturer | CSE | IUT
[email protected]
 What is Data Structure?

Data: Simply values or sets of values, raw facts or figure without any specific
meaning.
Data Structure: The logical or mathematical model of a particular organization of
data.

➢ Can store data

Example: Integers, Strings, Floats, … …
➢ Can answer some questions about the stored data
Example: What is the smallest value not greater than x?
➢ Can add or remove data
Example: add the element x after y, remove values less than x.

Asaduzzaman Herok, Lecturer

21 August, 2023 2
Department of Computer Science and Engineering, IUT
 Why Study Data Structure?
Applications of Data Structure:

➢ Computer file system ( Data structure maps file names onto hard drive sectors)
➢ Google and other search engines (Data structure maps keywords on web pages containing those
keywords)
➢ What is the longest common subsequence of two DNA can be found?
➢ Geographic systems (Data structure find data relevant to the current view/location)
➢ Finding large Prime Numbers
➢ Block chain (Linked list)
➢ Google Map (Finding shortest distances in terms of distance and time)
➢ Data Compression (Huffman’s encoding)
➢ Natural Language Processing (Strings)
➢ ……………

Many problems are solved efficiently just using the right data structure …

Asaduzzaman Herok, Lecturer

21 August, 2023 3
Department of Computer Science and Engineering, IUT
 How do We Study Data Structures?
➢ What does the data structure represents?
Computer file system (data structure maps file names onto hard drive track and sectors)

➢ What are the operations does it supports?

○ Reading: looking something up at a particular spot within the data structure.
○ Searching: looking for a particular value within a data structure.
○ Inserting: adding a new value to the data structure.
○ Deleting: removing a value from the data structure.
○ Sorting: rearranging element in some logical order.
○ Merging: Combining records of two different sorted files into one sorted files.

➢ What kind of performance does it have?

○ How long does each operation take? (Time complexity)
○ How much space does it use? (Memory complexity)

Asaduzzaman Herok, Lecturer

21 August, 2023 4
Department of Computer Science and Engineering, IUT
 Classification of Data Structure
Integer
Primitive Data Structure Real
Arrays
Character
Boolean Linked List
Data Structures

Linear Stacks

Queues

Trees
Non Primitive Data Structure
Non Linear
Graphs

Asaduzzaman Herok, Lecturer

21 August, 2023 5
Department of Computer Science and Engineering, IUT
 Memory Allocation
Memory allocation can be classified into followings:

➢ Contiguous
Example: arrays

➢ Linked
Example: linked lists

➢ Indexed
Example: array of pointers.

Asaduzzaman Herok, Lecturer

21 August, 2023 6
Department of Computer Science and Engineering, IUT
 Contiguous Memory Allocation
An array stores n objects in a single contiguous
space of memory.

➔ Can directly access any point randomly. Random access is

possible.
➔ Unfortunately, if more memory is required, a request for new
memory usually requires copying all information into the new
memory.
➔ In general, you cannot request for the operating system to
allocate to you the next n memory locations

Asaduzzaman Herok, Lecturer

21 August, 2023 7
Department of Computer Science and Engineering, IUT
 Linked Memory Allocation
Linked storage such as a linked list associates two pieces of data
with each item being stored:

➔ The object itself, and

➔ A reference to the next item

➔ Random access to any data apart from the beginning is not possible since the address of a
particular data is only stored to its previous data.

The actual linked list class must store two pointers

➔ A head and tail:
◆ Node *head;
◆ Node *tail;

Asaduzzaman Herok, Lecturer

21 August, 2023 8
Department of Computer Science and Engineering, IUT
 Indexed Memory Allocation
With indexed allocation, an array of pointers (possibly NULL) link to a sequence of allocated
memory locations.

Matrices can be implemented using indexed allocation:

Asaduzzaman Herok, Lecturer

21 August, 2023 9
Department of Computer Science and Engineering, IUT
 Other Memory Allocations
We will look at some varieties or hybrids of these
memory allocations including: Arbitrary
➔ Trees relations among
➔ Graphs the objects in a
container

Graph
A rooted tree is similar to a linked
list but with multiple next pointers

Tree
adjacency matrix adjacency list
Asaduzzaman Herok, Lecturer
21 August, 2023 10
Department of Computer Science and Engineering, IUT
 Complexity, Time-space tradeoff
➢ A function that estimates the running time/space with respect to the input size.
➢ Less time and space requirement is a blessing!
➢ Deals with large input size.
➢ Tradeoff: Increased amount of space to store data can sometimes reduce time requirement
(or vice-versa).

Why Do We Care?

Asaduzzaman Herok, Lecturer

21 August, 2023 11
Department of Computer Science and Engineering, IUT
 Complexity, Time-space tradeoff
Assuming 1 ms to perform a division
Solution #1 Solution#2
n=11 9 ms ~2 ms
n=101 99 ms ~9 ms
n=1000003 ~10^6 ms =1000 sec ~10^3 ms
=10^6+3 =16.66min = 1sec
n=10^10 10^10 ms =10^7sec ~10^5 ms = 100sec
=115 days = 1.66 mins

Asaduzzaman Herok, Lecturer

21 August, 2023 12
Department of Computer Science and Engineering, IUT
 Complexity, Time-space tradeoff

Asaduzzaman Herok, Lecturer

21 August, 2023 13
Department of Computer Science and Engineering, IUT
 Time Complexity Analysis
Measures how fast the time requirement of a program grows when the input size increases.

Running time of program may depend on:

➔ Single vs multi processor
➔ Read/write speed of memory
➔ 32-bit or 64-bit
➔ Size of input

For time complexity analysis, we are only interested in (size of input)

● Takes same amount of time Sum(a,b) { Time requirement: ~ 2 time-units

regardless of input size return a+b (1 unit for addition, 1 unit for
● Constant time algorithm } return statement)
● Time Complexity O(1)
Let’s think about
this function

Asaduzzaman Herok, Lecturer

21 August, 2023 14
Department of Computer Science and Engineering, IUT
 Time Complexity Analysis
# times Cost
unit
Comments
1. sumOfList (A, n) { In line 3:
2. total=0 1 1 (c1) - Executes n+1 times. One extra checking for
3. for i=0 to n-1 n+1 2 (c2) breaking condition.
4. total = total + A[i] n 2 (c3) - c2: 1 unit for increment, 1 unit for
5. return total 1 1 (c4) assignment.
6. } In line 4:
- 1 unit for addition, 1 for assignment.

Asaduzzaman Herok, Lecturer

21 August, 2023 15
Department of Computer Science and Engineering, IUT
 Some Growth Functions

Asaduzzaman Herok, Lecturer

21 August, 2023 16
Department of Computer Science and Engineering, IUT
 Asymptotic Analysis

● Asymptotic Analysis is the big idea that helps to analyze algorithms.

● In Asymptotic Analysis, we evaluate the performance of an algorithm in terms of input size

(we don’t measure the actual running time).

● Define mathematical bound of how the time (or space) taken by an algorithm increases with
the input size.

● An algorithm that is asymptotically more efficient will be the best choice for all but very small
inputs.

[Generally, the term ‘asymptotic’ means approaching but never connecting with a line or curve.]

Asaduzzaman Herok, Lecturer

21 August, 2023 17
Department of Computer Science and Engineering, IUT
 Asymptotic Analysis

Asaduzzaman Herok, Lecturer

21 August, 2023 18
Department of Computer Science and Engineering, IUT
 The Big ‘O’

Asaduzzaman Herok, Lecturer

21 August, 2023 19
Department of Computer Science and Engineering, IUT
 The Big ‘O’

Asaduzzaman Herok, Lecturer

21 August, 2023 20
Department of Computer Science and Engineering, IUT
 The Big ‘O’

Asaduzzaman Herok, Lecturer

21 August, 2023 21
Department of Computer Science and Engineering, IUT
 The Big ‘O’

Asaduzzaman Herok, Lecturer

21 August, 2023 22
Department of Computer Science and Engineering, IUT
 The Big ‘Ω’

Asaduzzaman Herok, Lecturer

21 August, 2023 23
Department of Computer Science and Engineering, IUT
 The Big ‘θ’

Asaduzzaman Herok, Lecturer

21 August, 2023 24
Department of Computer Science and Engineering, IUT
 Most Common Growth Functions

Asaduzzaman Herok, Lecturer

21 August, 2023 25
Department of Computer Science and Engineering, IUT
 Growth Rate of Functions

• If an algorithm takes 1 second to run with problem size = 8, what

is the time requirement (approximately) for with the problem
size = 16 ?

Asaduzzaman Herok, Lecturer

21 August, 2023 26
Department of Computer Science and Engineering, IUT
 Running Time on Operators

Operation type Symbols

Retrieving/ storing variables from
memory
Variable assignment =
Integer operations + - * / % ++ --
Bitwise operations & | ^ ~
Relational operations == != < <= > >=
Logical operations && || !
Memory allocation and deallocation new delete

Asaduzzaman Herok, Lecturer

21 August, 2023 27
Department of Computer Science and Engineering, IUT
 Running Time on Block of operations

// swap variables a and b

int temp = a;
a = b;
b = temp;

// update a sequence of values

++index;
prev_modulus = modulus;
modulus = next_modulus;
next_modulus = modulus_table[index];

Asaduzzaman Herok, Lecturer

21 August, 2023 28
Department of Computer Science and Engineering, IUT
 Running Time on Block of Sequence

Asaduzzaman Herok, Lecturer

21 August, 2023 29
Department of Computer Science and Engineering, IUT
 Quadratic Growth
f(n) = n2 g(n) = n2 - 3n +2

• Close to n = 0, they look very different

• Yet on the range n = [0, 1000], they are (relatively) indistinguishable.
Asaduzzaman Herok, Lecturer
21 August, 2023 30
Department of Computer Science and Engineering, IUT
 Polynomial Growth
f(n) = n6 g(n) = n6 - 23n5 +19n4 - 729n3 + 126n2 - 648n

Asaduzzaman Herok, Lecturer

21 August, 2023 31
Department of Computer Science and Engineering, IUT
 Running Time on Control Statement
Next we will look at the following control statements
• These are statements which potentially alter the execution of instructions
• Conditional statements
• if, switch
• Condition-controlled loops
• for, while, do-while

if (condition) {
// true body
} Runtime of a conditional statement =
else { The runtime of the condition (the test) +
// false body The runtime of the body
}

Asaduzzaman Herok, Lecturer

21 August, 2023 32
Department of Computer Science and Engineering, IUT
 Condition Controlled Loops
for (int i=0; i<N; ++i) { int i=0; // initialization
// … while ( i<N ) { // condition
} // …
++i; // increment
}

Asaduzzaman Herok, Lecturer

21 August, 2023 33
Department of Computer Science and Engineering, IUT
 Condition Controlled Loops

Most Inner Loop

Middle Loop

Outer Loop

Asaduzzaman Herok, Lecturer

21 August, 2023 34
Department of Computer Science and Engineering, IUT
 Controlled Statements
switch (i) { if (i==1){ /* do stuff */ }
case 1: /* do stuff */ break; else if (i==2){ /* do other stuff */ }
case 2: /* do other stuff */ break; else if (i==3){ /* do even more stuff*/ }
case 3: /* do even more stuff*/ break; else if (i==4){ /* tired yet? */ }
case 4: /* tired yet? */ break; else if (i==5){ /* do stuff */ }
case 5: /* do stuff */ break; else { /* do default stuff */}
default: /* do default stuff */
}

Asaduzzaman Herok, Lecturer

21 August, 2023 35
Department of Computer Science and Engineering, IUT
 Cases
• As well as determining the runtime of an algorithm, if the data may not be
deterministic, we may be interested in:
• Best-case runtime
• Average-case runtime
• Worst-case runtime
• In many cases, they will be significantly different
• Example:
• Searching a list linearly is simple enough

Asaduzzaman Herok, Lecturer

21 August, 2023 36
Department of Computer Science and Engineering, IUT
 Cases
Searching a list linearly is simple enough.
We will count the number of comparisons
• Best case:
• The first element is the one we’re looking for: O(1)
• Worst case:
• The last element is the one we’re looking for, or it is not in the list: O(n)
• Average case?
• We need some information about the list...

Assume the case we are looking for is in the list and equally likely distributed.
If the list is of size n, then there is a 1/n chance of it being in the ith location :

If summing the probabilities

So its O(n)
Asaduzzaman Herok, Lecturer
21 August, 2023 37
Department of Computer Science and Engineering, IUT
 Acknowledgement

Rafsanjany Kushol
PhD Student, Dept. of Computing Science,
University of Alberta

Sabbir Ahmed
Assistant Professor
Department of CSE, IUT

Asaduzzaman Herok, Lecturer

21 August, 2023 38
Department of Computer Science and Engineering, IUT