Trees

Definition (recursive): finite set of

elements (nodes) which is either empty or
it is partitioned into m + 1 disjoint subsets
T0,T1,Tm where T0 contains one node
(root) and T1, T2Tm are trees
T:

T1
E.G.M. Petrakis

T0

T2

Trees

Tm
1

Definitions

Depth of tree node: length of the path
from the root to the node

the root has depth 0

Height of tree: depth of the deepest node

Leaf: all leafs are at level d
Full tree: all levels are full
Complete tree: all levels except perhaps
level d are full

E.G.M. Petrakis

Trees

A
B

root

leaf

sub-tree

root (tree) = A
descendents (B) = {F,G} sons (B)
ancestors (H) = {E} parent (H)
degree (tree) = 4: max number of sons
degree (E) = 3
level (H) = 2 level (A) = 0
depth (tree) = 3 max level
E.G.M. Petrakis

Trees

Binary Trees
0

left

right

Finite set of nodes which is either empty

or it is partitioned into sets T0, Tleft, Tright
where T0 is the root and Tleft, Tright are
binary trees

A binary tree with n nodes at level l has at
most2n nodes at level l+1
E.G.M. Petrakis

Trees

Full Binary Tree

A

d: depth

A binary tree has
B
C
l
exactly 2 nodes at
level l <= d
E
F
G
D

Total number of
nodes: N = 2d+1 1
all leaf nodes
0
1
d

N = 2 + 2 + ... + 2 =
at the same level
d

d +1
2
=
2
1

j =0

E.G.M. Petrakis

Trees

Binary Tree Traversal

Traversal: list all nodes in order
a) preorder (depth first order):
root
traverse Tleft preorder
traverse Tright preorder
b) inorder (symmetric order):
traverse Tleft inorder
root
traverse Tright inorder
c) postorder:
traverse Tleft
traverse Tright
root

E.G.M. Petrakis

Trees

preorder: ABDGCEHIF
inorder: DGBAHEICF
postorder: GDBHIEFCA

A
C

E
D
G

E.G.M. Petrakis

Trees

preorder: ABCEIFJDGHKL
inorder: EICFJBGDKHLA
postorder: IEJFCGKLHDBA
B

C
E

F
I

E.G.M. Petrakis

D
G
J

Trees

H
K

Traversal of Trees of Degree >= 2

T0

T1
preorder
T0 (root)
T1 preorder
T2 preorder

TM preorder
E.G.M. Petrakis

T2

inorder
T1 inorder
T0 (root)
T2 inorder

TM inorder
Trees

T
postorder
T1 postorder
T2 postorder

TM postorder
T0 (root)
9

A
B
C

F
E

H
preorder: ABCDHEFG
inorder:
CBHDEAGF
postorder: CHDEBGFA

E.G.M. Petrakis

Trees

10

Binary Tree Node ADT

interface BinNode {
public Object element();
// Return and set the element value
public Object setElement(Object v);
public BinNode left();
// Return and set the left child
public BinNode setLeft(BinNode p);
public BinNode right();
// Return and set the right child
public BinNode setRight(BinNode p);
public boolean isLeaf();

// Return true if this is a leaf node

} // interface BinNode
E.G.M. Petrakis

Trees

11

Binary Tree Node Class

class BinNodePtr implements BinNode {
private Object element;
private BinNode left;
private BinNode right;

// Object for this node

// Pointer to left child
// Pointer to right child

public BinNodePtr() {left = right = null; } // Constructor 1

public BinNodePtr(Object val) {
// Constructor 2
left = right = null;
element = val;
}
public BinNodePtr(Object val, BinNode l, BinNode r) // Construct 3
{ left = l; right = r; element = val; }
E.G.M. Petrakis

Trees

12

Binary Tree Node Class (cont.)

// Return and set the element value
public Object element() { return element; }
public Object setElement(Object v) { return element = v; }
// Return and set the left child
public BinNode left() { return left; }
public BinNode setLeft(BinNode p) { return left = p; }
// Return and set the right child
public BinNode right() { return right; }
public BinNode setRight(BinNode p) { return right = p; }
public boolean isLeaf() // Return true if this is a leaf node
{ return (left == null) && (right == null); }
} // class BinNodePtr
E.G.M. Petrakis

Trees

13

Binary Tree Traversal

void preorder (BinNode tree) {
if (tree == NULL) return;
else {
System.out.print(tree.element() + " ");
preorder ( tree.left( ) );
preorder ( tree.right( ) );
}
}

E.G.M. Petrakis

Trees

14

Binary Search Tree (BST)

10
3

20

15

30

25

Each node stores key K

The nodes of Tleft have keys < K

The nodes of Tright have keys >= K
E.G.M. Petrakis

Trees

15

Search in BST
50
60

30
20
1.
2.
3.
4.

35

65

Compare with root

If x == root(key) => key found !!
If x < key(root) search the Tleft recursively
If x >= key(root) search Tright recursively

E.G.M. Petrakis

Trees

16

Find Min/Max Key

50
60

30
min 20

65 max

35

Find the maximum key in a BST: follow

nodes on Tright until key is found or NULL

Find the minimum key in a BST: follow
nodes on Tleft
E.G.M. Petrakis

Trees

17

Insert Key in BST

50
60

30
20

65

35

40
37

Search until a leaf is found

insert it as left child of leaf if key(leaf) < x

insert it as right child of leaf if key(leaf) >= x
E.G.M. Petrakis

Trees

18

Shape of BST
10

Depends on insertion
order

For random insertion
sequence the BST is
more or less balanced

5
1

e.g., 10 20 5 7 1

For order insertion

sequence the BST
becomes list

7
10

e.g., 1 5 7 10 20
E.G.M. Petrakis

20

20
Trees

19

Delete key x from BST

Three cases:
a) X is a leaf: simply delete it
b) X is not a leaf & it has exactly one sub-tree

the father node points to the node next to x

delete node(x)

c) X is not a leaf & it has two sub-trees

find p: minimum of Tright or maximum of Tleft

this has at most one sub-tree!!
delete it as in (a) or (b)
substitute x with p

E.G.M. Petrakis

Trees

20

a) The key is a Leaf

E.G.M. Petrakis

Trees

21

b) The key has one Sub-Tree

E.G.M. Petrakis

Trees

22

c) The key has Two Sub-Trees

E.G.M. Petrakis

Trees

23

BST Sorting

Insert keys in BST

Insertion sequence:
50 30 60 20 35 40
37 65

Output keys inorder

Average case:
O(nlogn)

Worst case: O(n2)
E.G.M. Petrakis

Trees

50
60

30
20

65

35
40
37

24

Implementation of BSTs

Each node stores the key and pointers to
roots of Tleft, Tright

Array-based implementation:

known maximum number of nodes

fast

good for heaps

Dynamic memory allocation:

no restriction on the number of nodes

slower but general
E.G.M. Petrakis

Trees

25

Dynamic Memory Allocation

Two C++ classes

BinNode: node class (page 12)

node operations: left, right, setValue, isleaf,
father

BST: tree class

composite operations: find, insert, remove,
traverse

Use of help operations:

easier interface to class

encapsulation: implementation becomes private
E.G.M. Petrakis

Trees

26

Elem Interface
Definition of an Object with support for a key field:
// Interface for generic
// element type
public abstract int key(); // Key used for search
// and ordering
} // interface Elem
interface Elem {

E.G.M. Petrakis

Trees

27

BST Class
class BST { // Binary Search Tree implementation
private BinNode root;

// The root of the tree

public BST() { root = null; } // Initialize root to null

public void clear() { root = null; }
public void insert(Elem val)
{ root = inserthelp(root, val); }
public void remove(int key)
{ root = removehelp(root, key); }
public Elem find(int key)
{ return findhelp(root, key); }
public boolean isEmpty() { return root == null; }
E.G.M. Petrakis

Trees

28

BST Class (cont.)

public void print() { // Print out the BST
if (root == null)
System.out.println("The BST is empty.");
else {
printhelp(root, 0);
System.out.println();
}
}

E.G.M. Petrakis

Trees

29

BST Class (cont.)

private Elem findhelp(BinNode rt, int key) {
if (rt == null)
return null;
Elem it = (Elem)rt.element();
if (it.key() > key)
return findhelp(rt.left(), key);
else if (it.key() == key)
return it;
else
return findhelp(rt.right(), key);
}
E.G.M. Petrakis

Trees

30

BST Class (cont.)

private BinNode inserthelp(BinNode rt, Elem val) {
if (rt == null)
return new BinNodePtr(val);
Elem it = (Elem)rt.element();
if (it.key() > val.key())
rt.setLeft(inserthelp(rt.left(), val));
else
rt.setRight(inserthelp(rt.right(), val));
return rt;
}
E.G.M. Petrakis

Trees

31

BST Class (cont.)

private BinNode deletemin(BinNode rt) {
if (rt.left() == null)
return rt.right();
else {
rt.setLeft(deletemin(rt.left()));
return rt;
}
}

} // Binary Search Tree implementation

E.G.M. Petrakis

Trees

32

Array Implementation

General for trees with degree d >= 2

Many alternatives:
1.
2.
3.
4.

array with pointers to father nodes

array with pointers to children nodes
array with lists of children nodes
binary tree array (good for heaps)

E.G.M. Petrakis

Trees

33

a) Pointers to Father Nodes

1
2
3
4
5
6
7

A
B
C
D
E
F
G

0
1
1
2
2
2
3

C
F

Easy to find ancestors

Difficult to find children

Minimum space
E.G.M. Petrakis

Trees

34

b) Pointers to Children Nodes

1
2
3
4
5
6
7

A
B
C
D
E
F
G

2 3
4 5 6
7

C
F

Easy to find children nodes

Difficult to find ancestor nodes
E.G.M. Petrakis

Trees

35

c) Lists of Children Nodes

1
2
3
4
5
6
7

B
C
D
E
F
G

2
4
7

3
5

C
F

Easy to find children nodes

Difficult to find ancestor nodes
E.G.M. Petrakis

Trees

36

d) Binary Tree Array

7

r:
1

parent ( r ) = (r-1)/2
leftchild ( r ) = 2r+1
rightchild ( r ) = 2r+2
leftsibling ( r ) = r-1
rightsibling ( r ) = r+1

E.G.M. Petrakis

4
6
1 2 3 5
0
7

1
4

if
if
if
if
if
Trees

2
6

3
1

4
2

5
3

6
5

0< r <n
2r + 1 < n
2r + 2 < n
r even & 0 < r < n
r odd & 0 < r+1 < n
37

Heap
max element

Complete binary
tree (not BST)

Max heap: every
node has a value >=
children

Min heap: every
node has a value <=
children
E.G.M. Petrakis

7
6

4
min element

3
6

9
Trees

38

Heap Operations
insert new element in heap
build heap from n elements
delete max (min) element from heap
shift-down: move an element from the
root to a leaf

Array implementation of page 36

In the following assume max heap

E.G.M. Petrakis

Trees

39

Insertion
57
25

65

57
48

=>

65

48

65
25
new element

=> 57

48

25

1. Insert last in the array (as leaf) and

2. shift it to its proper position: swap it with parent
elements as long as it has greater value

Complexity O(logn): at most d shift operations in
array (d: depth of tree)
E.G.M. Petrakis

Trees

40

Build Head

Insert n elements in an empty heap

complexity: O(nlogn)

Better bottom-up method:

H1

H2

Assume H1, H2 heaps and shift R down to

its proper position (depending on which
root from H1, H2 has greater value)

The same (recursively) within H1, H2
E.G.M. Petrakis

Trees

41

Example
1 R

H1
5
4

H2
7
3

7
R
5

=>
4

1
2

6
H1

=>
3

6
2

H2

Starts at d-1 level: the leafs need not be

accessed

Shift down elements
E.G.M. Petrakis

Trees

42

Complexity

Up to n/21 elements at d level => 0 moves/elem

Up to n/22 elements at d-1 level => 1 moves/elem

Up to n/2i elements at level i => i-1 moves/elem

d =logn

total number of moves

i
(i

1)n/2
(n)

i =1

E.G.M. Petrakis

Trees

43

Deletion
65
57
25

57

25
48

=>

57

=>
48

25

48

65

The max element is removed

swap with last element in array

delete last element

shift-down root to its proper position

Complexity: O(logn)
E.G.M. Petrakis

Trees

44

Class Heap
public class Heap { //
private Elem[] Heap;
private int size;
private int n;

Heap class
// Pointer to the heap array
// Maximum size of the heap
// Number of elements now in the heap

public Heap(Elem[] h, int num, int max) // Constructor

{ Heap = h; n = num; size = max; buildheap(); }
public int heapsize() // Return current size of the heap
{ return n; }
public boolean isLeaf(int pos) // TRUE if pos is a leaf position
{ return (pos >= n/2) && (pos < n); }
E.G.M. Petrakis

Trees

45

Class Heap (cont.)

// Return position for left child of pos
public int leftchild(int pos) {
Assert.notFalse(pos < n/2, "Position has no left child");
return 2*pos + 1;
}
// Return position for right child of pos
public int rightchild(int pos) {
Assert.notFalse(pos < (n-1)/2, "Position has no right child");
return 2*pos + 2;
}

E.G.M. Petrakis

Trees

46

Class Heap (cont.)

public int parent(int pos) { // Return position for parent
Assert.notFalse(pos > 0, "Position has no parent");
return (pos-1)/2;
}
public void buildheap()
// Heapify contents of Heap
{ for (int i=n/2-1; i>=0; i--) siftdown(i); }

E.G.M. Petrakis

Trees

47

Class Heap (cont.)

// Put element in its correct place
private void siftdown(int pos) {
Assert.notFalse((pos >= 0) && (pos < n), "Illegal heap position");
while (!isLeaf(pos)) {
int j = leftchild(pos);
if ((j<(n-1)) && (Heap[j].key() > Heap[j+1].key()))
j++;
// j is now index of child with greater value
if (Heap[pos].key() > Heap[j].key()) return; // Done
DSutil.swap(Heap, pos, j);
pos = j;
// Move down
}
}

E.G.M. Petrakis

Trees

48

Class Heap (cont.)

public void insert(Elem val) { // Insert value into heap
Assert.notFalse(n < size, "Heap is full");
int curr = n++;
Heap[curr] = val;
// Start at end of heap
// Now sift up until curr's parent's > curr's key
while ((curr!=0) && (Heap[curr].key()<Heap[parent(curr)].key()))
{
DSutil.swap(Heap, curr, parent(curr));
curr = parent(curr);
}
}

E.G.M. Petrakis

Trees

49

Class Heap (cont.)

public Elem removemin() {
// Remove minimum value
Assert.notFalse(n > 0, "Removing from empty heap");
DSutil.swap(Heap, 0, --n); // Swap minimum with last value
if (n != 0)
// Not on last element
siftdown(0); // Put new heap root val in correct place
return Heap[n];
}

E.G.M. Petrakis

Trees

50

Class Heap (cont.)

// Remove value at specified position
public Elem remove(int pos) {
Assert.notFalse((pos > 0) && (pos < n), "Illegal heap position");
DSutil.swap(Heap, pos, --n); // Swap with last value
if (n != 0)
// Not on last element
siftdown(pos); // Put new heap root val in correct place
return Heap[n];
}
} // Heap class

E.G.M. Petrakis

Trees

51

Heap Sort

Insert all elements in new heap array

At step i delete max element

delete: put max element last in array

Store this element at position i+1

After n steps the array is sorted!!

Complexity: O(nlogn) why??
E.G.M. Petrakis

Trees

52

x[0]

92

x[1]
37

x[2]

x[4] x[5]

x[3]
33

86

86

12

48

57

37

x[6]

33

48

25

92

25 x[7]
57

(a) Initial heap

(b) x[7] =
delete(92)

37
33
E.G.M. Petrakis

12

57

92

12

48
25
Trees

86
(c) x[6] =
delete(86)

53

37

48
37
33

12

33

25
57

12

86

48

25
57

86

92

92

(d) x[5] = delete(57)

(e) x[4] = delete(48)

E.G.M. Petrakis

Trees

54

33
12
37

48

() x[3] = delete(37)

25
57

86

92
12 x[1]

25

x[2]
12
37
92

48

33
57

86

(f) x[2] = delete(33)

E.G.M. Petrakis

x[4]
x[8]
Trees

37

25
48

33 x[3]
57

x[5] x[6]
92

86

x[7]

(g) x[1] = delete(25)

55

Huffman Coding

Goal: improvement in space
requirements in exchange of a penalty
in running time

Assigns codes to symbols

code: binary number

Main idea: the length of a code

depends on its frequency

shorter codes to more frequent symbols
E.G.M. Petrakis

Trees

56

Huffman Coding (cont.)

Input: sequence of symbols (letters)

typically 8 bits/symbol

Output: a binary representation

the codes of all symbols are concatenated in
the same order they appear in the input

a code cannot be prefix of another
N


n i p i less than 8 bits/symbol on the
average
n = i =1N

n i
i =1

E.G.M. Petrakis

Trees

57

Huffman Coding (cont.)

The code of each symbol is derived from a
binary tree

each leaf corresponds to a letter

goal: build a tree with the minimum external
path weight (epw)

epw: sum. of weighted path lengths

min epw <=> min n

a letter with high weight (frequency) should
have low depth => short code

a letter with low weight may be pushed deeper
in the Huffman tree => longer code
E.G.M. Petrakis

Trees

58

Building the Huffman Tree

The tree is build bottom-up

Order the letters by ascending frequency
The first two letters become leaves of
the Huffman tree
Substitute the two letters with one with
weight equal to the sum of their weights
Put this new element back on the ordered
list (in its correct place)
Repeat until only one element (root of
Huffman tree) remains in the list

E.G.M. Petrakis

Trees

59

Assigning Codes to Letters

Beginning at the root assign 0 to left
branches and 1 to right branches

The Huffman code of a letter is the binary
number determined by the path from the
root to the leaf of that letter

Longer paths correspond to less frequent
letters (and the reverse)

Replace each letter in the input with its
code
E.G.M. Petrakis

Trees

60

Huffman Example
3

0
Weights: .12 .40 .15 .08 .25
Symbols: a b c
d
e
1
Weights: .20 .40
Symbols:
b

.15
c

Weights: .60
Symbols:

.25
e

Weights: .35
Symbols:

.40
b

4
Weights: 1.0
Symbols:

.25
e

uffman
Tree

e
d

E.G.M. Petrakis

.40
b

a
Trees

61

Huffman Encoding
0

b 0 1 uffman
0 1 Tree
e
1
c 0
d
a

=>

Huffman codes
a: 1111
b: 0
c:110
d:1110
e:10

input: acde => Huffman code: 1111 110 1110 10

E.G.M. Petrakis

Trees

62

Huffman Decoding
uffman
Tree

b
e

11111110100110
a
d
e b c

Repeat until end of code

beginning at the root, take right branch for
each 1 and left branch for each 0 until we reach
letter

output the letter
E.G.M. Petrakis

Trees

63

Critique

Advantages:

compression (40-80%)
cheaper transmission
some security (encryption)
locally optimal code

Disadvantages:

CPU cost
difficult error correction
we need the Huffman tree for the decoding
code not globally optimal

E.G.M. Petrakis

Trees

64