Felix Xiaozhu Lin

Lecture 05: Shell Sort

Slides Courtesy: Prof. Vijay Raghunathan

Locality: definition
 Temporal locality
– If at one point a particular memory location is referenced,
then it is likely that the same location will be referenced
again in the near future

 Spatial locality
– If a particular storage location is referenced at a particular
time, then it is likely that nearby memory locations will be
referenced in the near future.

ECE368: Data Structures Lecture 05: 2

Locality: why matters to programmers
 Modern hardware likes programs with locality
– deeper reasons can be learnt in the computer architecture
course

 Example 1: traversing a linked list vs. an array

ECE368: Data Structures Lecture 05: 3

Locality: why matters to programmers
 Example 2: matrix sum
Slow
For j in 0..m
For i in 0.. n
sum += A[i][j]

Fast
For i in 0..n
For j in 0.. m
sum += A[i][j]

ECE368: Data Structures Lecture 05: 4

Locality: why matters to programmers
 Example 3: matrix multiplication
Slow

Fast

ECE368: Data Structures Lecture 05: 5

Locality
 It’s an interplay between algorithms and hardware

 Will not affect O(N) analysis

– Can cause a couple of times performance difference
– But still much less than O(N) gaps, especially when N
grows

 Locality will explain some common structures of

algorithms

ECE368: Data Structures Lecture 05: 6

Question: Why Stable Sorting Desirable?

End goal:
1) All cards are sorted by suit
2) Within each suit, cards are sorted by number
–"Sorting stability playing cards" by User:Dcoetzee, User:WDGraham - Own work, based on File:Cards-2-Heart.svg, File:Cards-7-Spade.svg, File:Cards-5-Spade.svg, File:Cards-5-Heart.svg.
–Licensed under Creative Commons Zero, Public Domain Dedication via Wikimedia Commons –
– http://commons.wikimedia.org/wiki/File:Sorting_stability_playing_cards.svg#mediaviewer/File:Sorting_stability_playing_cards.svg

ECE368: Data Structures Lecture 05: 7

Question: Why Stable Sorting Desirable?

–"Sorting stability playing cards" by User:Dcoetzee, User:WDGraham - Own work, based on File:Cards-2-Heart.svg, File:Cards-7-Spade.svg, File:Cards-5-Spade.svg, File:Cards-5-Heart.svg.
–Licensed under Creative Commons Zero, Public Domain Dedication via Wikimedia Commons –
– http://commons.wikimedia.org/wiki/File:Sorting_stability_playing_cards.svg#mediaviewer/File:Sorting_stability_playing_cards.svg

ECE368: Data Structures Lecture 05: 8

Recap: Insertion sort
 The algorithm builds up a sorted sub-array at the start and
successively inserts items into it
– For each item, we move through the array to find the proper position to
insert the item
 Memory complexity
– optimal, O(1)
 Time complexity:
– Perfectly sorted: O(n)
– Worst case or average input: O(n2)
– On average, insertion sort is better than bubble sort by a constant factor
(n2/4 vs. n2/2)
 Advantages:
– Easy to implement; good for sorted input and/or small input sizes

ECE368: Data Structures Lecture 05: 9

Insertion sort: time
 Worst case: array sorted in decreasing order
 For the j-th item (j from 1 to n-1)
– Dominated by j comparisons and j move operations
 One operation: (1 comparison + 1 move)
 Total = (n-1) + (n-2) + … + 1

= (n-1)*((n-1)+1)/2
= n2/2 - n/2
 Result: O(n2) (both copy and compare)

ECE368: Data Structures Lecture 05: 10

Insertion sort: time – average case
 Average case: array in random order
 For the j-th item (j from 1 to n-1)
– On average j/2 positions from the correct one
– 1+j/2 comparisons; j/2 moves
– Def: 1 operation = (1 comparison + 1 move)
• Assuming equal weight
– ½(1+j/2) + ½(j/2) = (1/2 + j/2) = (1+j)/2
– On average (1+j)/2 comparisons and move operations
 Total = ((n) + (n-1) + … + 3 + 2) / 2
= (n+1)*(n)/4 – 1/2
= n2/4 + n/4 - 1/2 comparisons/moves
 Result: O(n2) (both copy and compare)
ECE368: Data Structures Lecture 05: 11
Insertion sort weakness
 Performs comparisons between adjacent neighbors
– O(n2) comparisons
 Performs swaps/moves between adjacent items
– Take several swaps/moves to correct one inversion (pair of
integers out of order)
– O(n2) swaps/moves

ECE368: Data Structures Lecture 05: 12

Shell sort
 It’s an optimization!
 Allow comparisons and swaps between non-adjacent
elements
 Perform insertion sort on k sub-arrays of the original
array
– r[0], r[k], r[2k], r[3k], …
– r[1], r[1+k], r[1+2k], r[1+3k], …
– …
– r[k-1], r[2k-1], r[3k-1], r[4k-1], …
 Iterate the process, use a smaller k in each pass
 In the last pass, use k = 1 (guarantee correctness)

ECE368: Data Structures Lecture 05: 13

Example of Shell Sort
 Given array
– 3 7 9 0 5 1 6 8 4 2 0 6 1 5 7 3 4 9 8 2
 Arrange into a 2-D array with k = 7 columns and sort
each column
– 3 7 9 0 5 1 6 3 3 2 0 5 1 5
– 8 4 2 0 6 1 5 → 7 4 4 0 6 1 6
– 7 3 4 9 8 2 8 7 9 9 8 2
 String the result back together
– 3 3 2 0 5 1 5 7 4 4 0 6 1 6 8 7 9 9 8 2

ECE368: Data Structures Lecture 05: 14

Example (contd.)
33205157440616879982
 To repeat for k = 3, arrange into a 2-D array with 3
columns and sort each column
– 3 3 2 0 0 1
– 0 5 1 1 2 2
– 5 7 4 3 3 4
– 4 0 6 → 4 5 6
– 1 6 8 5 6 8
– 7 9 9 7 7 9
– 8 2 8 9
 String the result together
– 0 0 1 1 2 2 3 3 4 4 5 6 5 6 8 7 7 9 8 9
 Repeat with k = 1 in the last step (single column)
ECE368: Data Structures Lecture 05: 15
Shell sort: example

Pass 1, k = 5
3 5 6 3 4 5 1 2 j=5 no

3 5 6 3 4 5 1 2 j=6 yes

3 1 6 3 4 5 5 2 j=7 yes

3 1 2 3 4 5 5 6

ECE368: Data Structures Lecture 05: 16

Shell sort: example

Pass 2, k = 1
3 1 2 3 4 5 5 6 j = 1, yes

1 3 2 3 4 5 5 6 j = 2, yes

1 2 3 3 4 5 5 6 no

1 2 3 3 4 5 5 6 j = 3, no

No more swaps in the remainder of insertion sort

ECE368: Data Structures Lecture 05: 17
Recap: Insertion sort

for j ← 1 to n-1
temp_r ← r[j]
i ← j
while i > 0 and r[i-1] > temp_r
r[i] ← r[i-1]
i ← i-1
r[i] ← temp_r

ECE368: Data Structures Lecture 05: 18

Shell sort: algorithm

for each k (in descending order)

1 for j ← k to n-1
2 temp_r ← r[j]
3 i ← j
4 while i ≥ k and r[i-k] > temp_r
5 r[i] ← r[i-k]
6 i ← i-k
7 r[i] ← temp_r

ECE368: Data Structures Lecture 05: 19

Shell sort: choosing k
 Define the sequence recursively
– h(1) = 1
– h(i+1) = 3 * h(i) + 1
– 1, 4, 13, 40, 121, 364, …
 Let x be the largest integer such that h(x) < n
 Set the total number of passes to be x

 For pass j, use k = h(x-j+1)

– h(x), h(x-1), …, h(1)
 Lots of such sequences possible

ECE368: Data Structures Lecture 05: 20

Shell sort: time complexity
 Analysis is quite sophisticated, will skip that for this
course
– Sequence {1, 2, 3, 4, 6, 8, 9, …, 2p3q}, where
2p3q is the largest number smaller than n: O(n(lg n)2)
– Sequence {1, 3, 7, 15, …, 2k-1, …}: O(n1.5)
– Sequence {1, 4, 13, …, 3h(k-1)+1, …}: O(n1.5)
– Sequence {1, 8, 23, …, 4(j+1)+3*2j+1, …}: O(n4/3)
 Other sequences that do well
– Fibonacci sequence raised to (1+√5)
– Ciura’s sequence: 1, 4, 10, 23, 57, 132, 301,
701, 1750, 1750*(2.2), 1750*(2.2)2, …

Which one is larger? O(n(lg n)2) vs O(n1.5)

ECE368: Data Structures Lecture 05: 21
Shell sort: time complexity

ECE368: Data Structures Lecture 05: 22

Shell Sort: Summary
 Ideas:
– Allow comparisons and swaps between non-adjacent
elements
– Perform (insertion) sort on k sub-arrays of the original array
 Space complexity?
– O(1)
 Time complexity?
– O(n1.5), etc. Depends on the gap sequence.
 Stable?
– No

ECE368: Data Structures Lecture 05: 23

Real-world Code
/* kernel/cgroup.c */
/* a simple Shell sort */
static void groups_sort(int *g, int gidsetsize)
{
int base, max, stride;

for (stride = 1; stride < gidsetsize; stride = 3 * stride + 1)

; /* nothing */
stride /= 3;

while (stride) {
max = gidsetsize - stride;
for (base = 0; base < max; base++) {
int left = base;
int right = left + stride;
int tmp = g[right];

while (left >= 0 && g[left] > tmp) {

g[right] = g[left];
right = left;
left -= stride;
}
g[right] = tmp;
}
stride /= 3;
}
}
ECE368: Data Structures Lecture 05: 24