Introduction to Data Structures

Topic 1 – Basic Concepts

Outline

▪ What is Data Structure (Chap. 1.1)

▪ What is Algorithm (Chap. 1.4)
▪ Computer Architecture Review
▪ Time & Space Complexity (Chap. 3)
▪ Asymptotic Analysis (Chap. 3)
▪ Analyze a Program
▪ Abstract Data Types (Chap. 1.2)

12
Xingcan
ITEC2620 O
Apr 2021
What is Data Structure

Book Arrangement

A D
• Easy to Put
• Hard to Look Up (search
one by one)
A D B

• Hard to Put (find the location

A B D G
and make a room)
• Easy to Look Up

13
Xingcan
ITEC2620 O
Apr 2021
What is Data Structure

In computer science, a data structure is a data organization,

management, and storage format that enables efficient access and
modification. More precisely, a data structure is a collection of data
values, the relationships among them, and the functions or operations
that can be applied to the data.
-- Wikipedia

14
Xingcan
ITEC2620 O
Apr 2021
What is Algorithm

Book Arrangement
swap swap

lexical order
B A G D A B G D A B D G

B A G D

A A B A B D A B D G

Step 1 Step 2 Step 3 Step 4

15
Xingcan
ITEC2620 O
Apr 2021
What is Algorithm

In mathematics and computer science, an algorithm is a finite

sequence of well-defined, computer-implementable instructions,
typically to solve a class of problems or to perform a computation.
-- Wikipedia

16
Xingcan
ITEC2620 O
Apr 2021
Computer Architecture Review

Data Bus RAM Data Bus CPU

Disk Slow Fast

17
Xingcan
ITEC2620 O
Apr 2021
Computer Architecture Review

Random Access Memory Central Processing Unit

Operations
Program Control Unit MOV <X>, <Y>

Arithmetic / ADD <X>, <Y>

Run
Logic Unit CMP <X>, <Y> Time
Space Data
IMUL <X>, <Y>
Registers
…

18
Xingcan
ITEC2620 O
Apr 2021
Time & Space Complexity

Given a problem with 𝐼𝑛𝑝𝑢𝑡 𝑆𝑖𝑧𝑒 = 𝑛 and an algorithm to solve it:

▪ How much time it takes to run the algorithm?
▫ 𝑇 𝑛 = 𝐶(𝑂𝑝𝑒𝑟𝑎𝑡𝑖𝑜𝑛) × #(𝑂𝑝𝑒𝑟𝑎𝑡𝑖𝑜𝑛𝑠)
▪ How much space do we need to run the algorithm?
▫ 𝑆 𝑛 = 𝐶𝐷𝑎𝑡𝑎𝐼𝑡𝑒𝑚 × #(𝐷𝑎𝑡𝑎𝐼𝑡𝑒𝑚𝑠)

19
Xingcan
ITEC2620 O
Apr 2021
Time & Space Complexity

Check if 8 exists in an unsorted array of size 𝑛:

▪ Best Case: [8, 7, 4…]
The real running time depends on the input
▫ 𝑇𝑏𝑒𝑠𝑡 𝑛 = 1
size and data distribution. In most cases, we
▪ Worst Case: […3, 9, 8] care about the average time/space for
▫ 𝑇𝑤𝑜𝑟𝑠𝑡 𝑛 = 𝑛 random inputs.
▪ Average (Random) Case: […8…]
𝑛
▫ 𝑇𝑎𝑣𝑔 𝑛 =
2

20
Xingcan
ITEC2620 O
Apr 2021
Asymptotic Analysis

In the average case, 𝑇1 𝑛 = 10𝑛 and 𝑇2 𝑛 = 2𝑛2 , which one is better?

𝑇
Input Size 𝟏𝟎𝒏 𝟐
𝟐𝒏
monotonically increasing
𝑛=1 10 2
𝑛=3 30 18
𝑛=5 50 50 2𝑛2
𝑛 = 10 100 200
𝑛 = 20 200 800
10𝑛
How will 𝑇(𝑛) scale with large 𝑛?

21
𝑛 Xingcan
ITEC2620 O
Apr 2021
Asymptotic Analysis

▪ Upper Bound (𝑂)

▪ Lower Bound (Ω)
▪ The Upper Bound = The Lower Bound (Θ)

How fast does a function grow?

22
Xingcan
ITEC2620 O
Apr 2021
Asymptotic Analysis

▪ The Big-Oh notation

▫ Formal: For 𝑇(𝑛) a non-negatively valued function, 𝑇(𝑛) is in
set 𝑂 𝑓 𝑛 if there exist two positive constants 𝑐 and 𝑛0
such that 𝑇(𝑛) ≤ 𝑐𝑓(𝑛) for all 𝑛 > 𝑛0 .
▫ Informal: 𝑇(𝑛) is in 𝑂 𝑓 𝑛 if 𝑇(𝑛) grows no faster than
𝑐𝑓 𝑛 when 𝑛 is large enough.

23
Xingcan
ITEC2620 O
Apr 2021
Asymptotic Analysis

▪ The Big-Omega notation

▫ Formal: For 𝑇(𝑛) a non-negatively valued function, 𝑇(𝑛) is in
set Ω 𝑓 𝑛 if there exist two positive constants 𝑐 and 𝑛0
such that 𝑇(𝑛) ≥ 𝑐𝑓(𝑛) for all 𝑛 > 𝑛0 .
▫ Informal: 𝑇(𝑛) is in Ω 𝑓 𝑛 if 𝑇(𝑛) grows no slower than
𝑐𝑓 𝑛 when 𝑛 is large enough.

24
Xingcan
ITEC2620 O
Apr 2021
Asymptotic Analysis

▪ The Big-Theta notation

▫ Formal: For 𝑇(𝑛) a non-negatively valued function, 𝑇(𝑛) is
Θ 𝑓 𝑛 iff 𝑇(𝑛) is in 𝑂 𝑓 𝑛 and 𝑇(𝑛) is in Ω 𝑓 𝑛 .
▫ Informal: 𝑇(𝑛) is Θ 𝑓 𝑛 if 𝑇(𝑛) grows as fast as 𝑐𝑓 𝑛
when 𝑛 is large enough.

25
Xingcan
ITEC2620 O
Apr 2021
Asymptotic Analysis

It’s safe to suppress constant factors and lower-order terms.

3𝑛2 + 10𝑛

3𝑛2 + 10𝑛 is in 𝑂(𝑛2 )

5𝑛2 3𝑛2

26
Xingcan
ITEC2620 O
Apr 2021
Asymptotic Analysis

Simplifying rules:
1. If 𝑓(𝑛) is in 𝑂(𝑐𝑔 𝑛 ) for any constant 𝑐 > 0, then 𝑓(𝑛) is in 𝑂(𝑔 𝑛 ) ;
2. If 𝑓1 (𝑛) is in 𝑂(𝑔1 (𝑛)) and 𝑓2 (𝑛) is in 𝑂(𝑔2 (𝑛)), then 𝑓1 𝑛 + 𝑓2 (𝑛) is in
𝑂(max(𝑔1 𝑛 , 𝑔2 (𝑛))) and 𝑓1 𝑛 𝑓2 (𝑛) is in 𝑂(𝑔1 𝑛 𝑔2 𝑛 );
3. If 𝑓(𝑛) is in 𝑂(𝑔(𝑛)) and 𝑔(𝑛) is in 𝑂(ℎ 𝑛 ), then 𝑓(𝑛) is in 𝑂(ℎ 𝑛 ).

27
Xingcan
ITEC2620 O
Apr 2021
Asymptotic Analysis
2𝑛 𝑛 3 𝑛2
𝑛!
▪ Classification
▫ log 2 𝑛
▫ 𝑛
▫ 𝑛log 2 𝑛
▫ 𝑛2
▫ 𝑛3
▫ 2𝑛
▫ 𝑛!
𝑛log 2 𝑛
𝑛
log 2 𝑛 28
Xingcan
ITEC2620 O
Apr 2021
Analyze a Program

▪ A Single Simple Statement (Basic Operation): 𝐶

▫ An assignment
▫ A comparison between two variables
▫ An arithmetic operation between two variables
▪ Single Loop (𝑛 times): 𝐶𝑛
▪ Nested Loop (𝑛 × 𝑚 times): 𝐶𝑛𝑚
▪ Branch: max(𝑥, 𝑦, 𝑧 … )
▪ Recursion: 𝐶 × 𝑑𝑒𝑝𝑡ℎ

29
Xingcan
ITEC2620 O
Apr 2021
Abstract Data Type

▪ Abstract Data Type (ADT)

▫ A model defines some data characteristics and operations.
▫ Independent of programming languages.
▫ Interface
▪ Data Structure
▫ The concrete implementation of an ADT.
▫ Can be used in algorithms.

30
Xingcan
ITEC2620 O
Apr 2021
Abstract Data Type

▪ List
▪ Stack/Queue ▪ General Operations
▫ Build (Initialize)
▪ Set
▫ Traverse
▪ Map (Table/Dict) ▫ Add Values
▪ Tree ▫ Remove Values
▪ Priority Queue ▫ Update Values
▫ Merge
▪ Graph

31
Xingcan
ITEC2620 O
Apr 2021
The Learning Path

The ADT The Data Structure The Application

• The Definition • Implementation • Problem/Algorithm
• The Operations • Complexity • In Java

32
Xingcan
ITEC2620 O
Apr 2021