Chapter 6 Heapsort

1
• Problem:
• Maintain a dynamically changing
set S so that every element in S
has a priority (key) k.
• Allow efficiently reporting the
element with maximal priority in S.
• Allow the priority of an element in
S to be increased.
• A (max-)priority queue
supports the following
operations:

• Insert(S, x, k): Insert

element x into S and give it
priority k.

• Delete(S, x): Delete element

x from S.

• Find-Max(S): Report the

element with maximal priority
in S.

• Delete-Max(S): Report the

element with maximal priority
in S and remove it from S.

• Increase-Key(S, x, k):
Change the priority of x to k,
but only if k is greater than x’s
current priority.
•
B
i
n
a
r
y

h
e
a
p
s

a
r
e

p
e
r
f
e
c
t

b
i
n
a
r
y

t
r
e
e
s

t
h
a
t

s
a
t
i
s
f
y

t
h
e

f
o
l
l
o
w
i

75
n
g

h
e
a
p

p
r
o
p
e
r
t
y
:

• F
o
r

e
v
e
r
y

n
o
d
e

w
i
t

54 65
h

p
a
r
e
n
t

u
,

l
e
t

k
v

a
n
d

k
u

b
e

t
h
e

p
r
i
o

41 13
r

27 14
i
t
i
e
s

o
f

t
h
e

e
l
e
m
e
n
t
s

s
t
o
r
e
d

a
t

a
n
d

3 23 4
u
.

T
h
e
n

k
v

k
u
.
• F
o
r

a
n
y

t
w
o

a
n
d

s
o

t
h
a
t

i
s

a
n

a
n
c
e
s
t
o
r

o
f

v
,

k
u

k
v
.
• H
e
a
p

P
r
o
p
e
r
t
y
:
• F
o
r

e
v
e

75
r
y

n
o
d
e

w
i
t
h

p
a
r
e
n
t

u
,

l
e
t

k
v

54 65
a
n
d

k
u

b
e

t
h
e

p
r
i
o
r
i
t
i
e
s

o
f

t
h
e

27 14 41 13
e
l
e
m
e
n
t
s

s
t
o
r
e
d

a
t

a
n
d

u
.

T
h
e

3 23 4
n

k
v

k
u
.
• A
[
i
]

A
[
p
a
r
e
n
t
(
i
)
]
,

∀
i

>

75 54 65 27 14 41 13 3 23 4
 Maximum 75

54 65

27 14 41 13

3 23 4

75 54 65 27 14 41 13 3 23 4
 Maximum 75

54 65

27 14 41 13

3 23 4

75 54 65 27 14 41 13 3 23 4
 Maximum

54 65

27 14 41 13

3 23 4

54 65 27 14 41 13 3 23 4
 No longer 4
the maximum
54 65

27 14 41 13

3 23

4 54 65 27 14 41 13 3 23
• Heapify-Down(A, i)
• 1 l ← left(i)
• 2 r ← right(i)
• 3 largest ← i
• 4 if l ≤ heap-size(A) and A[l] > A[largest]
• 5 then largest ← l
• 6 if r ≤ heap-size(A) and A[r] > A[largest]
• 7 then largest ← r
• 8 if largest > i
• 9 then swap A[i] ↔ A[largest]
• 10 Heapify-Down(A, largest)

54 65

27 14 41 13

3 23
• Heapify-Down(A, i)
• 1 l ← left(i)
• 2 r ← right(i)
• 3 largest ← i
• 4 if l ≤ heap-size(A) and A[l] > A[largest]
• 5 then largest ← l
• 6 if r ≤ heap-size(A) and A[r] > A[largest]
• 7 then largest ← r
• 8 if largest > i
• 9 then swap A[i] ↔ A[largest]
• 10 Heapify-Down(A, largest)

65

54 4

27 14 41 13

3 23
• Heapify-Down(A, i)
• 1 l ← left(i)
• 2 r ← right(i)
• 3 largest ← i
• 4 if l ≤ heap-size(A) and A[l] > A[largest]
• 5 then largest ← l
• 6 if r ≤ heap-size(A) and A[r] > A[largest]
• 7 then largest ← r
• 8 if largest > i
• 9 then swap A[i] ↔ A[largest]
• 10 Heapify-Down(A, largest)

65

54 41

27 14 4 13

3 23
• I
n
s
e
r
t
(
A
,

x
)
• 1

h
e
a
p
-

65
s
i
z
e
(
A
)

h
e
a
p
-
s
i
z
e
(
A
)

1
• 2

A
[
h
e
a
p

54 41
-
s
i
z
e
(
A
)
]

x
• 3

H
e
a
p
i
f
y
-
U
p
(
A
,

h
e
a
p

4 13
-

27 14
s
i
z
e
(
A
)
)

3 23

65 54 41 27 14 4 13 3 23
• I
n
s
e
r
t
(
A
,

x
)
• 1

h
e
a
p
-

65
s
i
z
e
(
A
)

h
e
a
p
-
s
i
z
e
(
A
)

1
• 2

A
[
h
e
a
p

54 41
-
s
i
z
e
(
A
)
]

x
• 3

H
e
a
p
i
f
y
-
U
p
(
A
,

h
e
a
p

4 13
-

27 14
s
i
z
e
(
A
)
)

3 23 59

65 54 41 27 14 4 13 3 23 59
• H
e
a
p
i
f
y
-
U
p
(
A
,

i
)
• 1

w
h
i
l
e

>

a
n
d

A
[
i
]

>

A
[
p
a
r
e
n
t
(
i
)
]
• 2

d
o

s
w
a
p

A
[
i
]

A
[
p
a
r
e
n
t
(
i
)
]
• 3

p
a
r

65
e
n
t
(
i
)

54 41

27 14 4 13

3 23 59
• H
e
a
p
i
f
y
-
U
p
(
A
,

i
)
• 1

w
h
i
l
e

>

a
n
d

A
[
i
]

>

A
[
p
a
r
e
n
t
(
i
)
]
• 2

d
o

s
w
a
p

A
[
i
]

A
[
p
a
r
e
n
t
(
i
)
]
• 3

p
a
r

65
e
n
t
(
i
)

54 41

27 59 4 13

3 23 14
• H
e
a
p
i
f
y
-
U
p
(
A
,

i
)
• 1

w
h
i
l
e

>

a
n
d

A
[
i
]

>

A
[
p
a
r
e
n
t
(
i
)
]
• 2

d
o

s
w
a
p

A
[
i
]

A
[
p
a
r
e
n
t
(
i
)
]
• 3

p
a
r

65
e
n
t
(
i
)

59 41

27 54 4 13

3 23 14
 65
• D
e
l
e
t
e
(
A
,

i
)
• 1

A
[
i
]

A
[
h
e
a
p
-
s
i
z
e
(
A
)
]
• 2

h
e
a
p
-
s

59 41
i
z
e
(
A
)

h
e
a
p
-
s
i
z
e
(
A
)

1
• 3

i
f

>

a
n
d

A
[
i
]

>

24 54 4 13
A
[
p
a
r
e
n
t
(
i
)
]
• 4

t
h
e
n

H
e
a
p
i
f
y
-
U
p
(
A
,

i
)
• 5

e
l
s
e

H
e
a

3 23 14
p
i
f
y
-
D
o
w
n
(
A
,

i
)

65 59 41 24 54 4 13 3 23 14
 65
• D
e
l
e
t
e
(
A
,

i
)
• 1

A
[
i
]

A
[
h
e
a
p
-
s
i
z
e
(
A
)
]
• 2

h
e
a
p
-
s

41
i
z
e
(
A
)

h
e
a
p
-
s
i
z
e
(
A
)

1
• 3

i
f

>

a
n
d

A
[
i
]

>

24 54 4 13
A
[
p
a
r
e
n
t
(
i
)
]
• 4

t
h
e
n

H
e
a
p
i
f
y
-
U
p
(
A
,

i
)
• 5

e
l
s
e

H
e
a

3 23 14
p
i
f
y
-
D
o
w
n
(
A
,

i
)

65 41 24 54 4 13 3 23 14
 65
• D
e
l
e
t
e
(
A
,

i
)
• 1

A
[
i
]

A
[
h
e
a
p
-
s
i
z
e
(
A
)
]
• 2

h
e
a
p
-
s

14 41
i
z
e
(
A
)

h
e
a
p
-
s
i
z
e
(
A
)

1
• 3

i
f

>

a
n
d

A
[
i
]

>

24 54 4 13
A
[
p
a
r
e
n
t
(
i
)
]
• 4

t
h
e
n

H
e
a
p
i
f
y
-
U
p
(
A
,

i
)
• 5

e
l
s
e

H
e
a

3 23
p
i
f
y
-
D
o
w
n
(
A
,

i
)

65 14 41 24 54 4 13 3 23
 65
• D
e
l
e
t
e
(
A
,

i
)
• 1

A
[
i
]

A
[
h
e
a
p
-
s
i
z
e
(
A
)
]
• 2

h
e
a
p
-
s

54 41
i
z
e
(
A
)

h
e
a
p
-
s
i
z
e
(
A
)

1
• 3

i
f

>

a
n
d

A
[
i
]

>

24 14 4 13
A
[
p
a
r
e
n
t
(
i
)
]
• 4

t
h
e
n

H
e
a
p
i
f
y
-
U
p
(
A
,

i
)
• 5

e
l
s
e

H
e
a

3 23
p
i
f
y
-
D
o
w
n
(
A
,

i
)

65 54 41 24 14 4 13 3 23
 65
• I
n
c
r
e
a
s
e
-
K
e
y
(
A
,

i
,

k
e
y
)
• 1

i
f

59 41
k
e
y

>

A
[
i
]
• 2

t
h
e
n

A
[
i
]

k
e
y
• 3

24 54 4 13
e
a
p
i
f
y
-
U
p
(
A
,

i
)

3 23 14

65 59 41 24 54 4 13 3 23 14
 65
• I
n
c
r
e
a
s
e
-
K
e
y
(
A
,

i
,

k
e
y
)
• 1

i
f

59 81
k
e
y

>

A
[
i
]
• 2

t
h
e
n

A
[
i
]

k
e
y
• 3

24 54 4 13
e
a
p
i
f
y
-
U
p
(
A
,

i
)

3 23 14

65 59 81 24 54 4 13 3 23 14
 81
• I
n
c
r
e
a
s
e
-
K
e
y
(
A
,

i
,

k
e
y
)
• 1

i
f

59 65
k
e
y

>

A
[
i
]
• 2

t
h
e
n

A
[
i
]

k
e
y
• 3

24 54 4 13
e
a
p
i
f
y
-
U
p
(
A
,

i
)

3 23 14

81 59 65 24 54 4 13 3 23 14
• Observation: All heap operations, excluding calls to Heapify-Up or Heapify-Down,
take constant time.
• Lemma: Heapify-Up and Heapify-Down take O(lg n) time.
• Corollary: Operations Insert, Delete, Delete-Max, and Increase-Key take O(lg n)
time on a binary heap. Operation Find-Max takes O(1) time.
• Lemma: If the heap property holds,
operations Find-Max and Delete-Max find
and delete the element with maximal
priority.
• Lemma: Given a perfect binary tree and
a pointer to a node v such that the only
possible violation of the heap property is
k v > k parent(v), procedure Heapify-Up
restores the heap property.
• Lemma: Given a perfect binary tree and
a pointer to a node v such
that the only possible violation of the
heap property is k v < k left-child(v)
or kv < k right-child(v), procedure Heapify-
Down restores the heap property.
• Corollary: Operations Insert, Delete,
Increase-Key, and Delete-Max maintain
the heap property.
•How fast can we turn an array A into a heap?
•Build-Heap(A)
1 for i = ⎣n / 2⎦..1
2 do Heapify-Down(A, i)
•Lemma: When procedure Build-Heap terminates, array A is
a binary heap.
•Lemma: Procedure Build-Heap takes linear time.
• Plac
e all
elem
ents
in A
into
a
priori
ty
queu
e
• Retri
eve
the
elem
ents
one
by
one
and
plac
e
them
into
A in
rever
se
order
.

81 59 65 24 54 4 13 3 23 14
• Plac
e all
elem
ents
in A
into
a
priori
ty
queu
e
• Retri
eve
the
elem
ents
one
by
one
and
plac
e
them
into
A in
rever
se
order
.

59 4
81 24
13
65 3
54 23
14
• Plac
e all
elem
ents
in A
into
a
priori
ty
queu
e
• Retri
eve
the
elem
ents
one
by
one
and
plac
e
them
into
A in
rever
se
order
.

81

59 4
24
13
65 3
54 23
14
• Plac
e all
elem
ents
in A
into
a
priori
ty
queu
e
• Retri
eve
the
elem
ents
one
by
one
and
plac
e
them
into
A in
rever
se
order
.

65 81

59 4
24
13
3
54 23
14
• Plac
e all
elem
ents
in A
into
a
priori
ty
queu
e
• Retri
eve
the
elem
ents
one
by
one
and
plac
e
them
into
A in
rever
se
order
.

59 65 81

4
24
13
3
54 23
14
• Plac
e all
elem
ents
in A
into
a
priori
ty
queu
e
• Retri
eve
the
elem
ents
one
by
one
and
plac
e
them
into
A in
rever
se
order
.

3 4 13 14 23 24 54 59 65 81
 4
•

•
59 14
•

3 54 65 24
•

81 23 13

4 59 14 3 54 65 24 81 23 13
 81
•

•
59 65
•

23 54 14 24
•

3 4 13

81 59 65 23 54 14 24 3 4 13
• Priority queues support operations:
– Insert, Delete, and Increase-Key
– Find-Max and Delete-Max
• Binary heaps are priority queues that support the above operations in O(lg
n) time.
• We can sort using a priority queue.
• Heapsort:
– Sorts using the priority-queue idea
– Takes O(n lg n) time (as Mergesort)
– Sorts in place (as Insertion Sort)
• H
e
a
p
i
f
y
-
U
p
(
A
,

i
)
• 1

w
h
i
l
e

>

a
n
d

A
[
i
]

>

A
[
p
a
r
e
n
t
(
i
)
]
• 2

d
o

s
w
a
p

A
[
i
]

A
[
p
a
r
e
n
t
(
i
)
]
• 3

p
a
r

65
e
n
t
(
i
)

54 41

27 14 4 13

3 23 59
• H
e
a
p
i
f
y
-
U
p
(
A
,

i
)
• 1

w
h
i
l
e

>

a
n
d

A
[
i
]

>

A
[
p
a
r
e
n
t
(
i
)
]
• 2

d
o

s
w
a
p

A
[
i
]

A
[
p
a
r
e
n
t
(
i
)
]
• 3

p
a
r

65
e
n
t
(
i
)

54 41

27 59 4 13

3 23 14
• H
e
a
p
i
f
y
-
U
p
(
A
,

i
)
• 1

w
h
i
l
e

>

a
n
d

A
[
i
]

>

A
[
p
a
r
e
n
t
(
i
)
]
• 2

d
o

s
w
a
p

A
[
i
]

A
[
p
a
r
e
n
t
(
i
)
]
• 3

p
a
r

65
e
n
t
(
i
)

59 41

27 54 4 13

3 23 14
 65
• D
e
l
e
t
e
(
A
,

i
)
• 1

A
[
i
]

A
[
h
e
a
p
-
s
i
z
e
(
A
)
]
• 2

h
e
a
p
-
s

59 41
i
z
e
(
A
)

h
e
a
p
-
s
i
z
e
(
A
)

1
• 3

i
f

>

a
n
d

A
[
i
]

>

24 54 4 13
A
[
p
a
r
e
n
t
(
i
)
]
• 4

t
h
e
n

H
e
a
p
i
f
y
-
U
p
(
A
,

i
)
• 5

e
l
s
e

H
e
a

3 23 14
p
i
f
y
-
D
o
w
n
(
A
,

i
)

65 59 41 24 54 4 13 3 23 14
 65
• D
e
l
e
t
e
(
A
,

i
)
• 1

A
[
i
]

A
[
h
e
a
p
-
s
i
z
e
(
A
)
]
• 2

h
e
a
p
-
s

41
i
z
e
(
A
)

h
e
a
p
-
s
i
z
e
(
A
)

1
• 3

i
f

>

a
n
d

A
[
i
]

>

24 54 4 13
A
[
p
a
r
e
n
t
(
i
)
]
• 4

t
h
e
n

H
e
a
p
i
f
y
-
U
p
(
A
,

i
)
• 5

e
l
s
e

H
e
a

3 23 14
p
i
f
y
-
D
o
w
n
(
A
,

i
)

65 41 24 54 4 13 3 23 14
 65
• D
e
l
e
t
e
(
A
,

i
)
• 1

A
[
i
]

A
[
h
e
a
p
-
s
i
z
e
(
A
)
]
• 2

h
e
a
p
-
s

14 41
i
z
e
(
A
)

h
e
a
p
-
s
i
z
e
(
A
)

1
• 3

i
f

>

a
n
d

A
[
i
]

>

24 54 4 13
A
[
p
a
r
e
n
t
(
i
)
]
• 4

t
h
e
n

H
e
a
p
i
f
y
-
U
p
(
A
,

i
)
• 5

e
l
s
e

H
e
a

3 23
p
i
f
y
-
D
o
w
n
(
A
,

i
)

65 14 41 24 54 4 13 3 23
 65
• D
e
l
e
t
e
(
A
,

i
)
• 1

A
[
i
]

A
[
h
e
a
p
-
s
i
z
e
(
A
)
]
• 2

h
e
a
p
-
s

54 41
i
z
e
(
A
)

h
e
a
p
-
s
i
z
e
(
A
)

1
• 3

i
f

>

a
n
d

A
[
i
]

>

24 14 4 13
A
[
p
a
r
e
n
t
(
i
)
]
• 4

t
h
e
n

H
e
a
p
i
f
y
-
U
p
(
A
,

i
)
• 5

e
l
s
e

H
e
a

3 23
p
i
f
y
-
D
o
w
n
(
A
,

i
)

65 54 41 24 14 4 13 3 23
 65
• I
n
c
r
e
a
s
e
-
K
e
y
(
A
,

i
,

k
e
y
)
• 1

i
f

59 41
k
e
y

>

A
[
i
]
• 2

t
h
e
n

A
[
i
]

k
e
y
• 3

24 54 4 13
e
a
p
i
f
y
-
U
p
(
A
,

i
)

3 23 14

65 59 41 24 54 4 13 3 23 14
 65
• I
n
c
r
e
a
s
e
-
K
e
y
(
A
,

i
,

k
e
y
)
• 1

i
f

59 81
k
e
y

>

A
[
i
]
• 2

t
h
e
n

A
[
i
]

k
e
y
• 3

24 54 4 13
e
a
p
i
f
y
-
U
p
(
A
,

i
)

3 23 14

65 59 81 24 54 4 13 3 23 14
 81
• I
n
c
r
e
a
s
e
-
K
e
y
(
A
,

i
,

k
e
y
)
• 1

i
f

59 65
k
e
y

>

A
[
i
]
• 2

t
h
e
n

A
[
i
]

k
e
y
• 3

24 54 4 13
e
a
p
i
f
y
-
U
p
(
A
,

i
)

3 23 14

81 59 65 24 54 4 13 3 23 14
• Observation: All heap operations, excluding calls to Heapify-Up or Heapify-Down,
take constant time.
• Lemma: Heapify-Up and Heapify-Down take O(lg n) time.
• Corollary: Operations Insert, Delete, Delete-Max, and Increase-Key take O(lg n)
time on a binary heap. Operation Find-Max takes O(1) time.
• Lemma: If the heap property holds,
operations Find-Max and Delete-Max find
and delete the element with maximal
priority.
• Lemma: Given a perfect binary tree and
a pointer to a node v such that the only
possible violation of the heap property is
k v > k parent(v), procedure Heapify-Up
restores the heap property.
• Lemma: Given a perfect binary tree and
a pointer to a node v such
that the only possible violation of the
heap property is k v < k left-child(v)
or kv < k right-child(v), procedure Heapify-
Down restores the heap property.
• Corollary: Operations Insert, Delete,
Increase-Key, and Delete-Max maintain
the heap property.
•How fast can we turn an array A into a heap?
•Build-Heap(A)
1 for i = ⎣n / 2⎦..1
2 do Heapify-Down(A, i)
•Lemma: When procedure Build-Heap terminates, array A is
a binary heap.
•Lemma: Procedure Build-Heap takes linear time.
• Plac
e all
elem
ents
in A
into
a
priori
ty
queu
e
• Retri
eve
the
elem
ents
one
by
one
and
plac
e
them
into
A in
rever
se
order
.

81 59 65 24 54 4 13 3 23 14
?

55