Chapter 5

Sequences

In this chapter we introduce the sequence abstract data type. Sequences will be used throughout
the book. In addition to defining the interface we specify two cost specifications for the functions.
We also spend some time describing three functions scan, reduce and collect. We give
several examples of how to use these functions. We finish by describing a special kind of
sequence called single threaded sequences that allow certain functions to be faster when used in
a specific way.

5.1 Data Types, Cost Specifications, and Data Structures

So far in class we have defined several “problems” and discussed algorithms for solving them.
The idea is that the problem is an abstract definition of what we want in terms of a function
specification, and the algorithms are particular ways to solve/implement the problem. There
can be many algorithms for solving the same problems, and these algorithms can have different
costs.
In addition to abstract functions we also often need to define abstractions over data. In such
an abstraction we define a set of functions (abstractly) over a common data type. We will refer
to the abstractions as abstract data types (ADTs) and their implementations as data structures.
The ADT just defines the functionality of the data type but says nothing about the cost. The
data structure defines a specific way to implement the ADT, which gives a specific cost. Often
it is useful, however, to have a cost specification for an ADT without having to know how it is
implemented. In fact there might be many different implementations that match the same cost
specification. There can also be different implementations that have different cost specifications.
These specifications might pose tradeoffs where some functions are more expensive while others
are cheaper.
In this chapter we will see that an array-based implementations of sequences allow for
constant work to access the nth element of a sequence but linear work (in the sizes of the inputs)
to append two sequences. On the other hand a tree-based implementation takes logarithmic work

73
74 CHAPTER 5. SEQUENCES

for both accessing the nth element and appending. This leads to two different cost specifications,
where neither is strictly better than the other. We can describe these cost specifications without
having to know how the implementation works.
As such, data types can be described at three levels of abstraction:

1. The abstract data type: The definition of the interface for the purpose of describing
functionality and correctness criteria.

2. The cost specification: The definition of the costs for each of the functions in the interface.
There can be multiple different cost specifications for an ADT depending on the type of
implementation.

3. The implementation as a data structure: This is the particular data structure used to
implement the ADT. There might be multiple data structures that match the same cost
specification. If you are a user of the ADT you don’t need to know what this is, although it
is good to be curious.

In this book we try to clearly separate these three parts. Here we give a toy example based on
a counter that just counts up an down and returns a special value when decremented more than
incremented.

Abstract Data Type 5.1 (Counter). A Counter is a type α supporting the following
values and functions:

empty : α = 0
inc(c) : α → α = c+ 1
⊥ c=0
dec(c) : α → (α ∪ ⊥) =
c−1 otherwise

For each value function in the interface the definitions will specify the name and arguments on
the left, the type in the middle after a colon, and the definition of the value or what the function
returns on the right after an equal. In the example the counter returns ⊥, called bottom, when the
counter tries to go below 0. We will use bottom throughout the book to indicated an undefined or
special value of a result. Separately we give the cost specifications.

Cost Specification 5.2 (Simple Counter).

Work Span
inc(c) O(1) O(1)
dec(c) O(1) O(1)
5.2. THE SEQUENCE ADT 75

Here we only have one cost specification due to lack of creativity. We include both the work
and the span for each function, and report costs in terms of asymptotic bounds. Finally we can
describe one or more implementations. In this case we only have one.

Data Structure 5.3 (Integer Counter). We represent the counter with an integer and
use addition with 1 to increment it, and subtraction by 1 to decrement it. This data
structure supports the Counter ADT and matches the Simple Counter cost specification.

For some of the abstract data types we define we also introduce special syntax for the data type
that is used in the description of algorithms. For our example this might be:

Syntax 5.4 (Counter). Special syntax for counters, because we love C.

a++ ⇒ inc(a)
a-- ⇒ dec(a)

5.2 The Sequence ADT

The first (real) ADT we consider in some detail is one for sequences. A sequence is a mathe-
matical notion representing an ordered list of elements, such as h a, b, c, d i. In mathematics a
sequence can be finite or (countably) infinite. In this book, however, we will only consider finite
sequences. It is important you do not associate a sequence with a particular implementation—e.g.
with an array of contiguous locations in the memory of the machine. Instead you should be
thinking of it abstractly in terms of the mathematical definition and what functions on the data
type it supports. Indeed we will consider two different implementations of sequences, one based
on arrays and the other on trees.
We first do a quick review of set theory to more formally define sequences. A relation is a
set of ordered pairs. In particular for two sets α and β, ρ is a relation from α to β if ρ ⊆ α × β.
Here α × β indicates the set of all ordered pairs made from taking the first element from α and
the second from β, also called the Cartesian product of α and β. A mapping (or function) is a
relation ρ such that for every a in the domain of ρ there is only one b such that (a, b) ∈ ρ. A
sequence is a mapping whose domain is {0, . . . , n − 1} for some n ∈ N (we use N to indicate
the natural numbers, including zero).1 For example the sequence h a, b, c, d i corresponds to
the mapping {(0, a), (1, b), (2, c), (3, d)}, or equivalently {(1, b), (3, d), (2, c), (0, a)} since the
ordering of a set does not matter.
The sequence ADT is defined using sets in Abstract Data Type 5.5. In the pseudocode in this
book we use a notation for sequences using angle brackets h i. The translation of this notation to
the functions in the sequence definition is given in figure 5.6. This notation is often more concise
and more clear than writing out the functions.
1
Traditionally sequences are indexed from 1 not 0, but being computer scientists, we violate the tradition here.
76 CHAPTER 5. SEQUENCES

Abstract Data Type 5.5 (Sequences). A Sequence is a type Sα representing a mapping

from N (the natural numbers) to α with domain {0, . . . , n − 1} for some n ∈ N, and
supporting the following values and functions:

empty : Sα = {}
length(A) : Sα → N = |A|
singleton(v) : α → Sα = {(0,
 v)}
v (i, v) ∈ A
nth(A, i) : Sα × N → (α ∪ {⊥}) =
⊥ otherwise
map(f, A) : (α → β) × Sα → Sβ = {(i, f (v)) : (i, v) ∈ A}
tabulate(f, n) : (N → α) × N → Sα = {(i, f (i)) : 0 ≤ i < n}
subseq(A, s, l) : Sα × N × N → Sα = {(i − s, v)
: (i, v) ∈ A | s ≤ i < s + l}
append(A, B) : Sα × Sα → Sα = A ∪ {(i + |A|, v) : (i, v) ∈ B}
filter(f, A) : (α → B) × Sα → Sα = {(| {(j, x) ∈ A | j < i ∧ f (x)} |, v)
v) ∈ A | f (v)}
: (i, P
flatten(A) : SSα → Sα = {(i + (k,X)∈A,k<j |X|, v)
: (i, v) ∈ Y, (j, Y ) ∈ A}
inject(p, A) : SN×α × Sα → Sα = {(i, x): (i, v) ∈ A},
y (j, i, y) ∈ p
x=
v otherwise

where B = {true, f alse}. The additional functions iter, iterh, reduce, and scan
are defined later.

The empty, length and singleton functions should be self explanatory. The nth
function extracts the ith element from a sequence. If the i is out of range it returns ⊥ (bottom).

The map(f, S) function is a higher-order function that applies the function f to each element
of the sequence S returning a new equal length sequence. As we will see in the cost specification,
shortly, the map can be done in parallel across all elements of the sequence. Also, as we will see
in a later chapter, map generalizes to arbitrary mappings. In our pseudocode we use the notation

h f (s) : s ∈ S i

to indicate a map. The tabulate function is similar to map but maps over the sequence of
indices h 0, . . . , n − 1 i.
5.2. THE SEQUENCE ADT 77

Syntax 5.6 (Sequences). Translation from sequence notation to the sequence functions.

Si nth S i
|S| length(S)
hi empty()
hvi singleton(v)
h i, . . . , j i tabulate (fn k ⇒ i + k) (j − i + 1)
S h i, . . . , j i subseq S (i, j − i + 1)
he : p ∈ S i map (fn p ⇒ e) S
he : 0 ≤ i < ni tabulate (fn i ⇒ e) n
hp ∈ S | ei filter (fn p ⇒ e) S
h e1 : p ∈ S | e2 i map (fn p ⇒ e1 ) (filter (fn p ⇒ e2 ) S)
h e : p1 ∈ S1 , p2 ∈ S2 i flatten(map (fn p1 ⇒ map (fn p2 ⇒ e) S2 ) S1 )
h e1 : p1 ∈ S1 , p2 ∈ S2 | e2 i flatten(map (fn p1 ⇒ h e1 : p2 ∈ S2 | e2 i) S1 )
X
e reduce add 0 (map (fn p ⇒ e) S)
p∈S
n
X
e reduce add 0 (map (fn i ⇒ e) h k, . . . , n i)
i=k
P
The can be replaced with min, max, ∪ and ∩ with the presumed meanings, and by
replacing add and 0 with the appropriate values.

Example 5.7. The expression:

2 
x : x ∈ h 9, −1, 4, 11, 13, 2 i

is equivalent to:
map (fn x ⇒ x2 ) h 9, −1, 4, 11, 13, 2 i
and when evaluated returns the sequence:

h 81, 1, 16, 121, 169, 4 i .

We note that map can be implemented using tabulated as follows:

function map f S = tabulate (fn i ⇒ f (nth S i)) (length S)
or equivalently in our sequence notation as
function map f S = h f (Si ) : 0 ≤ i < |S| i

The filter function takes as an argument a predicate (i.e. a function that returns true or
false). It applies this predicate to each element of the input sequence and returns a new sequence
78 CHAPTER 5. SEQUENCES

only containing elements which returned true. The order is maintained. As with map, filter
can be performed in parallel. The definition of filter in 5.5 is a bit messy since each element
that passes the predicate needs to determine its new index, which involves counting how many
other elements before it also pass the predicate. This is not necessarily how its implemented, but
it is how it is defined. In pseudocode we use the notation

h s ∈ S | p(s) i

to indicate filtering the elements in the sequence S based on the predicate p.

Example 5.8. The expression:

h x ∈ h 9, −1, 4, 11, 13, 2 i | x > 5 i

is equivalent to:

filter (fn x ⇒ x > 5) h 9, −1, 4, 11, 13, 2 i

and when evaluated returns the sequence:

h 9, 11, 13 i .

The subseq(A, s, i) function extracts a contiguous subsequence starting at s and with

length l. If the subsequence is out of bounds of A, only the part within A is returned. The
append function should be self explanatory.
The flatten function takes a sequence of sequences and flattens them—i.e. if the input is
a sequence h S1 , S2 , . . . , Sn i it appends all the Si together one after the other. It is often used
when we map over multiple sequences or sets of indices.

Example 5.9. Let’s say we want to generate all contiguous subsequences of a sequence
A. Each sequence can start at any position 0 ≤ i < |A|, and end at any position
i ≤ j < |A|. We can do this with the following pseudocode:

h A h i, . . . , j i : 0 ≤ i < |A|, i ≤ j < |A| i ,

which is equivalent to:

flatten(tabulate
(fn i ⇒ tabulate
(fn l ⇒ subseq(A, i, l + 1))
(length(A) − i))
(length A))

Here we see that the sequence notation can be quite convenient.

5.2. THE SEQUENCE ADT 79

Cost Specification 5.10 (Sequences).

ArraySequence TreeSequence
Work Span Work Span
length(T ) 1 1 1 1
nth(T ) 1 1 log n log n
n n
X n X n
tabulate f n W (f (i)) max S(f (i)) W (f (i)) log n + max S(f (i))
i=0 i=0
i=0
X i=0
X
map f S W (f (s)) max S(f (s)) W (f (s)) log |S| + max S(f (s))
s∈S s∈S
s∈S
X s∈S
X
filter p S W (p(s)) log |S| + max S(p(s)) W (p(s)) log |S| + max S(p(s))
s∈S s∈S
s∈S s∈S
subseq(S, s, l) 1 1 log(|S|) log(|S|)
append(S1 , S2 ) |S1 | + |S2 | 1 log(|S1 | + |S2 |) log(|S1 | + |S2 |)
flatten(S) ||S|| + |S| 1 log(||S||) log(|S|) log(||S|| + |S|)

cost specifications for the Array and Tree based implementations of Sequences.
SampleP
||S|| = X∈S |S|. All are big-O.

From a theoretical point of view, the exact list of functions in the interface does not matter
that much. This is because many of the functions can be implemented with others, as we
saw with map being implemented with tabulate. However, we have to be careful that the
implementation preserves asymptotic costs in the cost specification.
We are now ready to consider the cost specifications for the sequence ADT. We consider
two cost specifications, one which we refer to as ArraySequence, and the other as TreeSequence.
These are given in figure 5.2 for a few of the functions. A full list of costs can be found in
the Cost Specifications part of http://www.cs.cmu.edu/~15210/docs/cost/. The
names of the cost specifications roughly indicate the class of implementation that can achieve
these cost bounds, but there might be many specific implementations that match the bounds. For
examples for the TreeSequence there are many types of trees that might be used. To use the cost
bounds, you don’t need to know the specifics of how these implementations work.

Example 5.11. Consider the code from Example 5.9. We have for the ArraySequence
cost specification for n = |A|:

W (n) = O(n2 ) .

This is because there are O(n2 ) total calls to subseq (A h i, . . . , j i), each of which
take O(1) work. Furthermore the flatten takes O(n2 ) work since the total length is
O(n2 ). We also have
S(n) = O(1)
This is because the subseq has O(1) span, and we take a maximum of the spans across
each of the tabulates. At the end we do a flatten, which also has O(1) span.
80 CHAPTER 5. SEQUENCES

Exercise 5.12. What is the work and span for the code from Example 5.9 if we use the
TreeSequence cost specification?

There are a couple things about the cost specifications we should note. Firstly, note that the
tabulate, map, and filter are all parallel. In particular the span takes the maximum of the
span of the subcalls at each position. The append for the ArraySequence cost specification is
also parallel since the span is only constant, while the work is linear. This is so since the elements
can be copied in parallel. It might not be obvious that filter, for example, can be parallelized. We
will get to this later.
Secondly, note that neither specification dominates the other, in the sense that for some
functions ArraySequence has better bounds (e.g. nth) but for others TreeSequence has better
bounds (e.g. append). Intuitively, for an array implementation we can access the nth element
using a direct access into the array costing constant work, but to append two arrays we need
to put one after the others, which requires moving them, requiring linear work in the sizes. On
the other hand, for a (balanced binary) tree implementation we need to traverse O(lg n) levels
of the tree to get to the nth element, but an append does not require moving the whole tree,
so it can also be done in O(log n) work. More details on these implementations will be given
later in this book. This sort of tradeoff is common in data types. The user can decide to use an
implementation that matches either specification, and this decisions should be based on which
specification leads to better asymptotic performance for their algorithm. For example if making
many calls to nth but no calls to append, then the user might want to use the ArraySequence
specification, and when running the code use an implementation of Sequences that matches that
specification.

Exercise 5.13. Earlier we showed how to implement map using tabulate. Decide
whether this implementation preserves asymptotic costs for an ArraySequence, and then
for TreeSequence.

We will now cover some of the sequence functions in more detail, including reduce, iter,
scan and iterh, tokens, fields, and collect.

5.3 The Scan Function

A function closely related to reduce is scan. It has the interface:

scan f I S : (α × α → α) → α → α seq → (α seq × α)

As with reduce, when the function f is associative, the scan function returns the sum
with respect to f of each prefix of the input sequence S, as well as the total sum of S. Hence
5.3. THE SCAN FUNCTION 81

the function is often called the prefix sums function (or problem). For a function f which is
associative it can be defined as follows:

1 function scan f I S =
2 (h reduce f I (S h 0, . . . , l − 1 i : 0 ≤ l < n) i ,
3 reduce f I S)

This uses our pseudocode notation and the h reduce f I (take(S, i)) : i ∈ h 0, . . . , n − 1 i i
indicates that for each i in the range from 0 to n − 1 apply reduce to the first i elements of S.
For example,
scan + 0 h 2, 1, 3 i = (h reduce + 0 h i , reduce + 0 h 2 i , reduce + 0 h 2, 1 i i
reduce + 0 h 2, 1, 3 i)
= (h 0, 2, 3 i , 6)
Using a bunch of reduces, however, is not an efficient way to calculate the partial sums.

Exercise 5.14. What is the work and span for the scan code shown above, assuming f
takes constant work.

We will soon see how to implement a scan with the following bounds:
W (scan f I S) = O(|S|)
S(scan f I S) = O(log |S|)
assuming that the function f takes constant work. For now we will consider some useful
applications of scans.
Note that the scan function takes the “sum” of the elements before the position i. Sometimes
it is useful to include the value at position i. We therefore also will use a version of such an
inclusive scan.

scanI + 0 h 2, 1, 3 i = h 2, 3, 6 i
This version does not return a second result since the total sum is already included in the last
position.

5.3.1 The MCSS Problem Algorithm 5: Using Scan

Let’s consider how we might use the scan function to solve the Maximum contiguous subsequence
(MCSS) problem. Recall, this problem is given a sequence S to find:
j−1
X
max ( Sk ) .
0≤i≤j≤n
k=i
82 CHAPTER 5. SEQUENCES

As a running example for this section consider the sequence

S = h 1, −2, 3, −1, 2, −3 i .

What if we do an inclusive scan on our input S using addition? i.e.:

X = scanI + 0 S = h 1, −1, 2, 1, 3, 0 i

Now for any j th position consider all positions i < j. To calculate the sum from immediately
after i to j all we have to do is return Xj − Xi . This difference represents the total sum of the
subsequence from i + 1 to j since we are taking the sum up to j and then subtracting off the
sum up to i. For example to calculate the sum between the −2 (location i + 1 = 1) and the 2
(location i = 4) we take X4 − X0 = 3 − 1 = 2, which is indeed the sum of the subsequence
h −2, 3, −1, 2 i.
Now consider how for each j we might calculate the maximum sum that starts at any i ≤ j
and ends at j. Call it Rj . This can be calculated as follows:
j
j X
Rj = max Sk
i=0
k=i
j
= max(Xj − Xi−1 )
i=0
j
= Xj + max(−Xi−1 )
i=0
j−1
= Xj + max(−Xi )
i=0
j−1
= Xj − min Xi
i=0

The last equality is because the maximum of a negative is the minimum of the positive. This
indicates that all we need to know is Xj and the minimum previous Xi , i < j. This can be
calculated with a scan using minimum as the binary combining function. Furthermore the result
of this scan is the same for everyone, so we need to calculate it just once. The result of the scan
is:
(M, _) = scan min 0 X = (h 0, 0, −1, −1, −1, −1 i , −1) ,
and now we can calculate R:

R = h Xj − Mj : 0 ≤ j < |S| i = h 1, −1, 3, 2, 4, 1 i .

You can verify that each of these represents the maximum contiguous subsequence sum ending
at position j.
Finally, we want the maximum string ending at any position, which we can do with a reduce
using max. This gives 4 in our example.
Putting this all together we get the following very simple algorithm:
5.3. THE SCAN FUNCTION 83

Algorithm 5.15 (Scan-based MCSS).

1 function MCSS(S) =
2 let
3 val X = scanI + 0 S
4 val (M ,_) = can min 0 X
5 val Y = h Xj − Mj : 0 ≤ j < |S| i
6 in
7 max(Y )
8 end

Given the costs for scan and the fact that addition and minimum take constant work, this
algorithm has O(n) work and O(log n) span.

5.3.2 Copy Scan

Previously, we used scan to compute partial sums to solve the maximum contiguous sub-
sequence sum problem and to match parentheses. Scan is also useful when you want pass
information along the sequence. For example, suppose you have some “marked” elements that
you would like to copy across to their right until they reach another marked element. One way to
mark the elements is to use options.
That is, suppose you are given a sequence of type α option seq. For example

h NONE, SOME(7), NONE, NONE, SOME(3), NONE i

and your goal is to return a sequence of the same length where each element receives the previous
SOME value. For the example:

h NONE, NONE, SOME(7), SOME(7), SOME(7), SOME(3) i

Using a sequential loop or iter would be easy. How would you do this with scan?
If we are going to use a scan directly, the combining function f must have type

α option × α option → α option

How about

1 function copy(a, b) =
2 case b of
3 SOME( ) ⇒ b
4 | NONE ⇒ a
84 CHAPTER 5. SEQUENCES

What this function does is basically pass on its right argument if it is SOME and otherwise it
passes on the left argument. To be used in a scan it needs to be associative. In particular we need
to show that copy(x, copy(y, z)) = copy(copy(x, y), z) for all x, y and z. There are eight
possibilities corresponding to each of x, y and z being either SOME or NONE. For the cases that
z = SOME(c) it is easy to verify that that either ordering returns z. For the cases that z = NONE
and y = SOME(b) one can verify that both orderings give y, for the cases that y = z = NONE
and x = SOME(a) they both return x, and for all being NONE either ordering returns NONE.
There are many other applications of scan in which more involved functions are used. One
important case is to simulate a finite state automaton.

5.3.3 Contraction and Implementing Scan

Now let’s consider how to implement scan efficiently and at the same time apply one of the
algorithmic techniques from our toolbox of techniques: contraction. Throughout the following
discussion we assume the work of the binary operator is O(1). As described earlier a brute force
method for calculating scans is to apply a reduce to all prefixes. This requires O(n2 ) work and is
therefore not work-efficient since we can do it in O(n) work sequentially.
Beyond the wonders of what it can do, a surprising fact about scan is that it can be
accomplished efficiently in parallel, although on the surface, the computation it carries out
appears to be sequential in nature. At first glance, we might be inclined to believe that any
efficient algorithms will have to keep a cumulative “sum,” computing each output value by
relying on the “sum” of the all values before it. It is this apparent dependency that makes scan
so powerful. We often use scan when it seems we need a function that depends on the results
of other elements in the sequence, for example, the copy scan above.
Suppose we are to run plus_scan (i.e. scan (op +)) on the sequence h 2, 1, 3, 2, 2, 5, 4, 1 i.
What we should get back is
(h 0, 2, 3, 6, 8, 10, 15, 19 i , 20)
We will use this as a running example.

Divide and Conquer: We first consider a divide-and-conquer solution. We can do this by

splitting the sequence in half, solving each half and then trying to put the results together. The
question is how do we put the results together. In particular lets say the scans on the two
halves return (Sl , tl ) and (Sr , tr ), which would be (h 0, 2, 3, 6 i , 8) and (h 0, 2, 7, 11 i , 12) in our
example. Note that Sl already gives us the first half of the solution.

Question 5.16. How do we get the second half?

To get the second half, note that in calculating Sr in the second half we started with the identity
instead of the sum of the first half, tl . Therefore if we add the sum of the first half, tl , to each
element of Sr , we get the desired result. This leads to the following algorithm:
5.3. THE SCAN FUNCTION 85

Algorithm 5.17 (Scan using divide and conquer).

1 function scan f I S =
2 case showt(S) of
3 EMPTY ⇒ (h i , I)
4 | ELT(v) ⇒ (h I i , v)
5 | NODE(L, R) ⇒ let
6 val ((Sl , tl ), (Sr , tr )) = (scan f I L k scan f I R)
7 val Xr = h f (tl , y) : y ∈ Sr i
8 in
9 (append(Sl , Xr ), tl + tr )
10 end

One caveat about this algorithm is that it only works if I is really the “identity” for f , i.e.
f (I, x) = x, although it can be fixed to work in general.
We now consider the work and span for the algorithm. Note that the joining step requires a
map to add tl to each element of Sr , and then an append. Both these take O(n) work and O(1)
span, where n = |S|. This leads to the following recurrences for the whole algorithm:

W (n) = 2W (n/2) + O(n) ∈ O(n log n)

S(n) = S(n/2) + O(1) ∈ O(log n)

Although this is much better than O(n2 ) work, we can do better.

Contraction: To compute scan in O(n) work in parallel, we introduce a new inductive

technique common in algorithms design: contraction. It is inductive in that such an algorithm
involves solving a smaller instance of the same problem, much in the same spirit as a divide-
and-conquer algorithm. But with contraction, there is only one subproblem. In particular, the
contraction technique involves the following steps:
1. Contract the instance of the problem to a smaller instance (of the same sort).
2. Solve the smaller instance recursively.
3. Use the solution to help solve the original instance.
The contraction approach is a useful technique in algorithm design in general but for various
reasons it is more common in parallel algorithms than in sequential algorithms. This is usually
because both the contraction and expansion steps can be done in parallel and the recursion only
goes logarithmically deep because the problem size is shrunk by a constant fraction each time.
We’ll demonstrate this technique first by applying it to a slightly simpler problem, reduce.
To begin, we have to answer the following question: How do we make the input instance smaller
in a way that the solution on this smaller instance will benefit us in constructing the final
solution?
86 CHAPTER 5. SEQUENCES

The idea is simple: We apply the combining function pairwise to adjacent elements of the
input sequence and recursively run reduce on it. In this case, the third step is a “no-op”; it does
nothing. For example on input sequence h 2, 1, 3, 2, 2, 5, 4, 1 i with addition, we would contract
the sequence to h 3, 5, 7, 5 i. Then we would continue to contract recursively to get the final
result. There is no expansion step.

Thought Experiment II: How can we use the same idea to implement scan? What would be
the result after the recursive call? In the example above it would be

(h 0, 3, 8, 15 i , 20).

But notice, this sequence is every other element of the final scan sequence, together with the
final sum—and this is enough information to produce the desired final output. This time, the
third expansion step is needed to fill in the missing elements in the final scan sequence: Apply
the combining function element-wise to the even elements of the input sequence and the results
of the recursive call to scan.
To illustrate, the diagram below shows how to produce the final output sequence from the
original sequence and the result of the recursive call:

Input = h2, 1, 3, 2, 2, 5, 4, 1i
Partial Output = (h0, 3, 8, 15i, 20)
+ + + +

Desired Output = (h0, 2, 3, 6, 8, 10, 15, 19i, 20)

This leads to the following code. The algorithm we present works for when n is a power of
two.

Algorithm 5.18 (Scan Using Contraction, for powers of 2).

1 function scanPow2 f i s =
2 case |s| of
3 0 ⇒ (h i , i)
4 | 1 ⇒ (h i i , s[0])
5 | n ⇒ let
6 val s0 = h f (s[2i], s[2i + 1]) : 0 ≤ i < n/2 i
7 val (r, t) = scanPow2 f i s0
8 in
(
r[i/2] if even(i)
9 (h pi : 0 ≤ i < n i , t), where pi =
f (r[i/2], s[i − 1]) otherwise.
10 end
5.4. REDUCE FUNCTION 87

5.4 Reduce Function

Recall that reduce function has the interface

reduce f I S : (α × α → α) → α → α seq → α

When the combining function f is associative—i.e., f (f (x, y), z) = f (x, f (y, z)) for all
x, y and z of type α—reduce returns the sum with respect to f of the input sequence S. It is
the same result returned by iter f I S. The reason we include reduce is that it is parallel,
whereas iter is strictly sequential. Note, though, iter can use a more general combining
function with type: β × α → β.
The results of reduce and iter, however, may differ if the combining function is non-
associative. In this case, the order in which the reduction is performed determines the result;
because the function is non-associative, different orderings will lead to different answers. While
we might try to apply reduce to only associative operations, unfortunately even some functions
that seem to be associative are actually not. For instance, floating point addition and multiplication
are not associative. In some languages integer addition is also not associative because of the
possibility of overflow, which might raise an exception.
To properly deal with combining functions that are non-associative, it is therefore important to
specify the order that the combining function is applied to the elements of a sequence. This order
is part of the specification of the ADT Sequence. In this way, every (correct) implementation
returns the same result when applying reduce; the results are deterministic regardless of what
data structure and algorithm are used.
For this reason, we define a specific combining tree. In particular we assume reduce is
equivalent to the following code:

Algorithm 5.19 (Reduce definition).

1 function reduce f I S =
2 let
3 function reduce’(S) =
4 case showt(S) of
5 ELT(v) ⇒ v
6 | NODE(L, R) ⇒ f (reduce(L),reduce(R))
7 in
8 case showt(S) of
9 NONE ⇒ I
10 | _ ⇒ f (I ,reduce’(S))
11 end

Recall that showt splits S in half at the middle. If the length of S is odd, then the left “half”
is one large than the right one.
88 CHAPTER 5. SEQUENCES

5.4.1 Divide and Conquer with Reduce

Now, let’s look back at divide-and-conquer algorithms you have encountered so far. Many of
these algorithms have a “divide” step that simply splits the input sequence in half, proceed
to solve the subproblems recursively, and continue with a “combine” step. This leads to the
following structure where everything except what is in boxes is generic, and what is in boxes is
specific to the particular algorithm.

1 function myDandC(S) =
2 case showt(S) of
3 EMPTY ⇒ emptyVal
4 | ELT(v) ⇒ base (v)
5 | NODE(L, R) ⇒ let
6 val (L0 , R0 ) = ( myDandC(L) k myDandC(R) )
7 in
8 someMessyCombine (L0 , R0 )
9 end

Algorithms that fit this pattern can be implemented in one line using the sequence reduce
function. Turning a divide-and-conquer algorithm into a reduce-based solution is as simple as
invoking reduce with the following parameters:

reduce someMessyCombine emptyVal (map base S)

We will take a look two examples where reduce can be used to implement a relatively
sophisticated divide-and-conquer algorithm. Both problems should be familiar to you.

Algorithm 4: MCSS Using Reduce.

The first example is the Maximum Contiguous Subsequence Sum problem from last lecture.
Given a sequence S of numbers, find the contiguous subsequence that has the largest sum—more
formally:
( j )
X
mcss(s) = max sk : 1 ≤ i ≤ n, i ≤ j ≤ n .
k=i

Recall that the divide-and-conquer solution involved strengthening the problem so it returns
four values from each recursive call on a sequence S: the desired result mcss(S), the maximum
prefix sum of S, the maximum suffix sum of S, and the total sum of S. We will denote these as
M , P , S, T , respectively. We refer to this strengthened problem as mcss0 . To solve mcss0 we
can then use the following implementations for combine, base, and emptyVal:
5.4. REDUCE FUNCTION 89

function combine((ML , PL , SL , TL ), (MR , PR , SR , TL )) =

(max(SL + PR , ML , MR ), max(PL , TL + PR ), max(SR , SL + TR ), TL + TR )
function base(v) = (v, v, v, v)
val emptyVal = (−∞, −∞, −∞, 0)

and then solve the problem with:

function mcss’(S) =
reduce combine emptyVal (map base S)

Question 5.20. Is the MCSS combine function described above associative?

It turns out that the combine function for MCSS is associative, as we would expect for the
binary function passed to reduce. Indeed this is true for the all the combine functions we
have used in divide-and-conquer so far. To prove associativity of the MCSS combine function
we could go through all the cases. However a more intuitive way to see it is to consider
what combine(A, combine(B, C)) and combine(combine(A, B), C) should return. In
particular for A, B and C appearing in that order, both ways of associating the combines should
return the overall maximum contiguous sum, the overall maximum prefix sum, the overall
maximum suffix sum, and the overall sum. The divide-and-conquer algorithm would not be
correct if this were not the case.

Stylistic Notes. We have just seen that we could spell out the divide-and-conquer steps in
detail or condense our code into just a few lines that take advantage of the almighty reduce.
So which is preferable, using the divide-and-conquer code or using reduce? We believe this is a
matter of taste. Clearly, your reduce code will be (a bit) shorter, and for simple cases easy to
write. But when the code is more complicated, the divide-and-conquer code is easier to read, and
it exposes more clearly the inductive structure of the code and so is easier to prove correct.

Restriction. You should realize, however, that this pattern does not work in general for divide-
and-conquer algorithms. In particular, it does not work for algorithms that do more than a simple
split that partitions their input in two parts in the middle. For example, it cannot be used for
implementing quick sort as the divide step partitions the data with respect to a pivot. This step
requires picking a pivot, and then filtering the data into elements less than, equal, and greater
than the pivot. It also does not work for divide-and-conquer algorithms that split more than two
ways, or make more than two recursive calls.
90 CHAPTER 5. SEQUENCES

5.5 Analyzing the Costs of Higher Order Functions

In Section 1 we looked at using reduce to solve divide-and-conquer problems. In the MCSS

problem the combining function f had O(1) cost (i.e., both its work and span are constant). In
that case the cost specifications of reduce on a sequence of length n is simply O(n) (linear)
work and O(log n) (logarithmic) span.

Question 5.21. Does reduce have linear work and logarithmic span when the binary
function passed to it does not have constant cost?

Unfortunately when the function passed to reduce does not take constant work, then the work of
the reduce is not necessarily linear. More generally when using a higher-order function that is
passed a function f (or possibly multiple functions) one needs to consider the cost of f .
For map it is easy to find its costs based on the cost of the function applied:
X
W (map f S) = 1 + W (f (s))
s∈S
S(map f S) = 1 + max S(f (s))
s∈S

Tabulate is similar. But can we do the same for reduce?

Merge Sort. As an example, let’s consider merge sort. As you have likely seen from previous
courses you have taken, merge sort is a popular divide-and-conquer sorting algorithm with
optimal work. It is based on a function merge that takes two already sorted sequences and
returns a sorted sequence containing all elements from both sequences. We can use our reduction
technique for implementing divide-and-conquer algorithms to implement merge sort with a
reduce. In particular, we can write a version of merge sort, which we refer to as reduceSort, as
follows:

val combine = merge<

val base = singleton
val emptyVal = empty()
function reduceSort(S) = reduce combine emptyVal (map base S)

where merge< is a merge function that uses an (abstract) comparison operator <. Note that
merging is an associative function.
Assuming a constant work comparison function, two sequences S1 and S2 with lengths n1
and n2 can be merged with the following costs:
W (merge< (S1 , S2 )) = O(n1 + n2 )
S(merge< (S1 , S2 )) = O(log(n1 + n2 ))
5.5. ANALYZING THE COSTS OF HIGHER ORDER FUNCTIONS 91

Question 5.22. What do you think the cost of reduceSort is?

5.5.1 Reduce: Cost Specifications

We want to analyze the cost of reduceSort. Does the reduction order matter? As mentioned
before, if the combining function is associative, which it is in this case, all reduction orders give
the same answer so it seems like it should not matter.
To answer this question, let’s consider the sequential reduction order that is used by iter,
as given by the following code.

function iterSort(S) =
iter merge< (empty()) (map singleton S)

Since the merge is associative this is functionally the same as reduceSort, but will sequentially
add the elements in one after the other. On input x = h x1 , x2 , . . . , xn i, the algorithm will first
merge h i and h x1 i, then merge in h x2 i, then h x3 i, etc.
With this order merge< is called when its left argument is a sequence of varying size between
1 and n − 1, while its right argument is always a singleton sequence. The final merge combines
(n − 1)-element with 1-element sequences, the second to last merge combines (n − 2)-element
with 1-element sequences, so on so forth. Therefore, the total work for an input sequence S of
length n is
n−1
X
W (iterSort S) ≤ c · (1 + i) ∈ O(n2 )
i=1

since merge on sequences of lengths n1 and n2 has O(n1 + n2 ) work.

Question 5.23. Can you see what algorithm iterSort implements?

Using this reduction order the algorithm is effectively working from the front to the rear,
“inserting” each element into a sorted prefix where it is placed at the correct location to maintain
the sorted order. This corresponds to the well-known insertion sort.
We can also analyze the span of iterSort. Since we iterate adding in each element after
the previous, there is no parallelism between merges, but there is parallelism within a merge. We
can calculate the span as
n−1
X
S(iterSort x) ≤ c · log(1 + i) ∈ O(n log n)
i=1

since merge on sequences of lengths n1 and n2 has O(log(n1 + n2 )) span. This means our
algorithm does have a reasonable amount of parallelism, W (n)/S(n) = O(n/ log(n)), but the
real problem is that it does much too much work.
92 CHAPTER 5. SEQUENCES

Question 5.24. Can you think of a way to improve our bound by using a different
reduction order with the merge function?

In iterSort, the reduction tree is unbalanced. We can improve the cost by using a balanced
tree instead.
For ease of exposition, let’s suppose that the length of our sequence is a power of 2, i.e.,
|x| = 2k . Now we lay on top the input sequence a perfect binary tree2 with 2k leaves and merge
according to the tree structure.

Example 5.25. As an example, the merge sequence for |x| = 23 is shown below.

x1 x2 x3 x4 x5 x6 x7 x8

= merge

What would the cost be if we use a perfect tree?

At the bottom level where the leaves are, there are n = |x| nodes with constant cost each.
Stepping up one level, there are n/2 nodes, each corresponding to a merge call, each costing
c(1 + 1). In general, at level i (with i = 0 at the root), we have 2i nodes where each node is a
merge with input two sequences of length n/2i+1 .
Therefore, the work of such a balanced tree of merge< ’s is the familiar sum

log n  n
X
i n 
≤ 2 · c i+1 + i+1
i=0
2 2
log n n
X
= 2i · c
i=0
2i

This sum, as you have seen before, evaluates to O(n log n).
2
This is simply a binary tree in which every node either has exactly 2 children or is a leaf, and all leaves are at
the same depth.
5.5. ANALYZING THE COSTS OF HIGHER ORDER FUNCTIONS 93

Merge Sort. In fact, this algorithm is essentially the merge sort algorithm. We can use our
reduction technique for implementing divide-and-conquer algorithms to implement merge sort
with a reduce.
In particular, we can write a version of merge sort, which we refer to as reduceSort, as
follows:

val combine = merge<

val base = singleton
val emptyVal = empty()
function reduceSort(S) = reduce combine emptyVal (map base S)

where merge< is a merge function that uses an (abstract) comparison operator <.

Summary 5.26. A brief summary of a few points.

• When applying a binary function in reduce, if the function is associative, the

order of applications does not matter for the final result.

• When applying a binary function in reduce, the order of applications does matter
when calculating the cost (work and span), regardless of whether the function is
associative or not.

• Implementing a “reduce” with merge with a sequential order leads to insertion

sort, while implementing with a balanced tree (parallel order) leads to merge sort.

The cost of reduce in general. In general, how would we go about defining the cost of
reduce with higher order functions. Given a reduction tree, we’ll first define R(reduce f I S)
as
n o
R(reduce f I S) = all function applications f (a, b) in the reduction tree .

Following this definition, we can state the cost of reduce as follows:

 
X
W (reduce f I S) = O n + W (f (a, b))
f (a,b)∈R(f I S)
 
S(reduce f I S) = O log n max S(f (a, b))
f (a,b)∈R(f I S)

The work bound is simply the total work performed, which we obtain by summing across all
combine functions. The span bound is more interesting. The log n term expresses the fact that the
94 CHAPTER 5. SEQUENCES

tree is at most O(log n) deep. Since each node in the tree has span at most maxf (a,b) S(f (a, b),
any root-to-leaf path, including the “critical path,” has at most O(log n maxf (a,b) S(f (a, b)) span.
This can be used, for example, to prove the following lemma:

Lemma 5.27. For any combine function f : α × α → α and size function s : α → R+ , if

for any x, y,
1. s(f (x, y)) ≤ s(x) + s(y) and
2. W (f (x, y)) ≤ c (s(x) + s(y)) for some constant c,
then !
X
W (reduce f I S) = O log |S| (1 + s(x)) .
x∈S

Applying this lemma to the merge sort example, we have

W (reduce merge< h i h h a i : a ∈ A i) = O(|A| log |A|)

5.6 Collect

Thus far we considered two very important functions on sequences, scan and reduce.
We now look a third function: collect.

Specification of Collect. Let’s start with something that you may have heard of.

Question 5.28. Do you know of key-value stores?

The term key-value store often refers to a storage systems (which may in on disk or in-
memory) that stores pairs of the form “key x value.”

Question 5.29. Can you think of a way of representing a key-value store using a data
type that we know?

We can use a sequence to represent such a store.

5.6. COLLECT 95

Example 5.30. For example, we may have a sequence of key-value pairs consisting of
our students from last semester and the classes they take.
val Data = h(“jack sprat”, “15-210”),
(“jack sprat”, “15-213”),
(“mary contrary”, “15-210”),
(“mary contrary”, “15-213”),
(“mary contrary”, “15-251”),
(“peter piper”, “15-150”),
(“peter piper”, “15-251”),
. . .i

Note that key-value pairs are intentionally asymmetric: they map a key to a value. This is
fine because that is how we often like them to be.
But sometimes, we often want to put together all the values for a given key.
We refer to this function as a collect.

Example 5.31. We can determine the classes taken by each student.

val classes = h(“jack sprat, h “15-210”, “15-213”, . . . i)
(“mary contrary, h “15-210”, “15-213”,“15-251”, . . . i)
(“peter piper, h “15-210”,”15-251”, . . . i)
. . .i

Collecting values together based on a key is very common in processing databases. In

relational database languages such as SQL it is referred to as “Group by”. More generally it has
many applications and furthermore it is naturally parallel.
We will use the function collect for this purpose, and it is part of the sequence library. Its
interface is:

collect : (α × α → order) → (α × β) seq → (α × β seq) seq

The first argument is a function for comparing keys of type α, and must define a total order
over the keys.
The second argument is a sequence of key-value pairs.
The collect function collects all values that share the same key together into a sequence,
ordering the values in the same order as their appearance in the original sequence.
96 CHAPTER 5. SEQUENCES

Example 5.32. Given sequence of pairs each consisting of a student’s name and a course
they are taking, we want to collect all entries by course number so we have a list of
everyone taking each course. This would give us the roster for each class, which can be
viewed as another sequence of key-value pairs. For example,

val rosters = h(“15-150”, h “peter piper”, . . . i)

(“15-210”, h “jack sprat”, “mary contrary”, . . . i)
(“15-213”, h “jack sprat”, . . . i)
(“15-251”, h “mary contrary”, “peter piper” i)
. . .i

Question 5.33. Can you see how to create a roster?

Example 5.34. If we wanted to collect the entries of Data given above by course
number to create a roster, we could do the following:

val collectStrings = collect String.compare

val rosters = collectStrings(h (c, n) : (n, c) ∈ Data i)

This would give something like:

val rosters = h(“15-150”, h “peter piper”, . . . i)
(“15-210”, h “jack sprat”, “mary contrary”, . . . i)
(“15-213”, h “jack sprat”, . . . i)
(“15-251”, h “mary contrary”, “peter piper” i)
. . .i

We use a map (h (c, n) : (n, c) ∈ Data i) to put the course number in the first position in
the tuple since that is the position used to collect on.

Cost of Collect. Collect can be implemented by sorting the keys based on the given comparison
function, and then partitioning the resulting sequence. In particular, the sort will move the pairs
so that all equal keys are adjacent.
A partition function can then identify the positions where the keys change values, and pull out
all pairs between each change. Doing this partitioning can be done relatively easily by filtering
out the indices where the value changes.
The dominant cost of collect is therefore the cost of the sort.
Assuming the comparison has complexity bounded above by Wc work and Sc span, then the
costs of collect are O(Wc n log n) work and O(Sc log2 n) span.
5.7. USING COLLECT IN MAP REDUCE 97

Exercise 5.35. Complete the details of this implementation. Better yet, implement collect
on your own.

Question 5.36. Can you think of another way to implement collect?

It is also possible to implement a version of collect that runs in linear work using hashing.
But hashing would require that a hash function is also supplied and would not return the keys in
sorted order. Later we discuss tables which also have a collect function. However tables are
specialized to the key type and therefore neither a comparison nor a hash function need to be
passed as arguments.

5.7 Using Collect in Map Reduce

Some of you have probably heard of the map-reduce paradigm first developed by Google for
programming certain data intensive parallel tasks. It is now widely used within Google as well as
by many others to process large data sets on large clusters of machines—sometimes up to tens of
thousands of machines in large data centers. The map-reduce paradigm is often used to analyze
various statistical data over very large collections of documents, or over large log files that track
the activity at web sites. Outside Google the most widely used implementation is the Apache
Hadoop implementation, which has a free license (you can install it at home). The paradigm
is different from the mapReduce function you might have seen in 15-150 which just involved
a map then a reduce. The map-reduce paradigm actually involves a map followed by a collect
followed by a bunch of reduces, and therefore might be better called the map-collect-reduces.
The map-reduce paradigm processes a collection of documents based on a map function
fm and a reduce function fr supplied by the user. The fm function must take a document as
input and generate a sequence of key-value pairs as output. This function is mapped over all the
documents. All key-value pairs across all documents are then collected based on the key. Finally
the fr function is applied to each of the keys along with its sequence of associated values to
reduce to a single value.
In ML the types for map function fm and reduce function fr are the following:
fm : (document → (key × α) seq)
fr : (key × (α seq) → β)
In most implementations of map-reduce the document is a string (just the contents of a file) and
the key is also a string. Typically the α and β types are limited to certain types. Also, in most
implementations both the fm and fr functions are sequential functions. Parallelism comes about
since the fm function is mapped over the documents in parallel, and the fr function is mapped
over the keys with associated values in parallel.
In ML map reduce can be implemented as follows
98 CHAPTER 5. SEQUENCES

1 function mapCollectReduce fm fr docs =

2 let
3 val pairs = flatten h fm (s) : s ∈ docs i
4 val groups = collect String.compare pairs
5 in
6 h fr (g) : g ∈ groups i
7 end

The function flatten simply flattens a nested sequence into a flat sequence, e.g.:

flatten h h a, b, c i , h d, e i i
⇒ h a, b, c, d, e i

As an example application of the paradigm, suppose we have a collection of documents, and

we want to know how often every word appears across all documents. This can be done with the
following fm and fr functions.

function fm (doc) = h (w, 1) : tokens doc i

function fr (w, s) = (w, reduce + 0 s)

Here tokens is a function that takes a string and breaks it into tokens by removing whitespace
and returning a sequence of strings between whitespace.
Now we can apply mapCollectReduce to generate a countWords function, and apply
this to an example case.

val countWords = mapCollectReduce fm fr

countWords h“this is a document”,

“this is is another document”,
“a last document”i
⇒ h (“a”, 2), (“another”, 1), (“document”, 3), (“is”, 3), (“last”, 1), (“this”, 2) i

5.8 Single-Threaded Array Sequences

In this course we will be using purely functional code because it is safe for parallelism and
enables higher-order design of algorithms by use of higher-order functions. It is also easier to
reason about formally, and is just cool. For many algorithms using the purely functional version
makes no difference in the asymptotic work bounds—for example quickSort and mergeSort use
Θ(n log n) work (expected case for quickSort) whether purely functional or imperative. However,
in some cases purely functional implementations lead to up to a O(log n) factor of additional
5.8. SINGLE-THREADED ARRAY SEQUENCES 99

work. To avoid this we will slightly cheat in this class and allow for benign “effect” under the
hood in exactly one ADT, described in this section. These effects do not affect the observable
values (you can’t observe them by looking at results), but they do affect cost analysis—and if
you sneak a peak at our implementation, you will see some side effects.

The issue has to do with updating positions in a sequence. In an imperative language updating
a single position can be done in “constant time”. In the functional setting we are not allowed
to change the existing sequence, everything is persistent. This means that for a sequence of
length n an update can either be done in Θ(n) work with an arraySequence (the whole sequence
has to be copied before the update) or Θ(log n) work with a treeSequence (an update involves
traversing the path of a tree to a leaf). In fact you might have noticed that our sequence interface
does not even supply a function for updating a single position. The reason is both to discourage
sequential computation, but also because it would be expensive.

Consider a function update (i, v) S that updates sequence S at location i with value v
returning the new sequence. This function would have cost Θ(|S|) in the arraySequence cost
specification. Someone might be tempted to write a sequential loop using this function. For
example for a function f : α− > α, a map function can be implemented as follows:

function map f S =
iter (fn ((i, S 0 ), v) ⇒ (i + 1, update (i, f (v)) S 0 ))
(0, S)
S

This code iterates over S with i going from 0 to n − 1 and at each position i updates the
value Si with f (Si ). The problem with this code is that even if f has constant work, with an
arraySequence this will do Θ(|S|2 ) total work since every update will do Θ(|S|) work. By
using a treeSequence implementation we can reduce the work to Θ(|S| log |S|) but that is
still a factor of Θ(log |S|) off of what we would like.

In the class we sometimes do need to update either a single element or a small number
of elements of a sequence. We therefore introduce an ADT we refer to as a Single Threaded
Sequence (stseq). Although the interface for this ADT is quite straightforward, the cost
specification is somewhat tricky. To define the cost specification we need to distinguish between
the latest “copy” of an instance of an stseq, and earlier copies. Basically whenever we update
a sequence we create a new “copy”, and the old “copy” is still around due to the persistence
in functional languages. The cost specification is going to give different costs for updating the
latest copy and old copies. Here we will only define the cost for updating the latest copy, since
this is the only way we will be using an stseq. The interface and costs is as follows:
100 CHAPTER 5. SEQUENCES

Work Span
fromSeq(S) : α seq → α stseq O(|S|) O(1)
Converts from a regular sequence to a stseq.
toSeq(ST) : α stseq → α seq O(|S|) O(1)
Converts from a stseq to a regular sequence.
nth ST i : α stseq → int → α O(1) O(1)
th
Returns the i element of ST. Same as for seq.
update (i, v) ST : (int × α) → α stseq → α stseq O(1) O(1)
th
Replaces the i element of ST with v.
inject I ST : (int × α) seq → α stseq → α stseq O(|I|) O(1)
th
For each (i, v) ∈ I replaces the i element of ST with v.

An stseq is basically a sequence but with very little functionality. Other than converting to and
from sequences, the only functions are to read from a position of the sequence (nth), update a
position of the sequence (update) or update multiple positions in the sequence (inject). To
use other functions from the sequence library, one needs to covert an stseq back to a sequence
(using toSeq).
In the cost specification the work for both nth and update is O(1), which is about as good
as we can get. Again, however, this is only when S is the latest version of a sequence (i.e. noone
else has updated it). The work for inject is proportional to the number of updates. It can be
viewed as a parallel version of update.
Now with an stseq we can implement our map as follows:

1 function map f S = let

2 val S 0 = StSeq.fromSeq(S)
3 val R = iter (fn ((i, S 00 ), v) ⇒ (i + 1, StSeq.update (i, f (v)) S 00 ))
4 (0, S 0 )
5 S
6 in
7 StSeq.toSeq(R)
8 end

This implementation first converts the input sequence to an stseq, then updates each element of
the stseq, and finally converts back to a sequence. Since each update takes constant work, and
assuming the function f takes constant work, the overall work is O(n). The span is also O(n)
since iter is completely sequential. This is therefore not a good way to implement map but it
does illustrate that the work of multiple updates can be reduced from Θ(n2 ) on array sequences
or O(n log n) on tree sequences to O(n) using an stseq.
5.8. SINGLE-THREADED ARRAY SEQUENCES 101

Implementing Single Threaded Sequences. You might be curious about how single threaded
sequences can be implemented so they act purely functional but match the cost specification.
Here we will just briefly outline the idea.
The trick is to keep two copies of the sequence (the original and the current copy) and
additionally to keep a “change log”. The change log is a linked list storing all the updates made
to the original sequence. When converting from a sequence to an stseq the sequence is copied
to make a second identical copy (the current copy), and an empty change log is created. A
different representation is now used for the latest version and old versions of an stseq. In the
latest version we keep both copies (original and current) as well as the change log. In the old
versions we only keep the original copy and the change log. Lets consider what is needed to
update either the current or an old version. To update the current version we modify the current
copy in place with a side effect (non functionally), and add the change to the change log. We
also take the previous version and mark it as an old version removing its current copy. When
updating an old version we just add the update to its change log. Updating the current version
requires side effects since it needs to update the current copy in place, and also has to modify the
old version to mark it as old and remove its current copy.
Either updating the current version or an old version takes constant work. The problem is the
cost of nth. When operating on the current version we can just look up the value in the current
copy, which is up to date. When operating on an old version, however, we have to go back to
the original copy and then check all the changes in the change log to see if any have modified
the location we are asking about. This can be expensive. This is why updating and reading the
current version is cheap (O(1) work) while working with an old version is expensive.
In this course we will use stseqs for some graph algorithms, including breadth-first search
(BFS) and depth-first search (DFS), and for hash tables.
102 CHAPTER 5. SEQUENCES