Multicore programming in
Haskell
Simon Marlow
Microsoft Research
A concurrent web server
server :: Socket -> IO ()
server sock =
forever
(do
acc <- Network.accept sock
forkIO (http acc)
)
create a new thread the client/server
for each new client protocol is implemented
in a single-threaded way
Concurrency = abstraction
• Threads let us implement individual
interactions separately, but have them happen
“at the same time”
• writing this with a single event loop is complex
and error-prone
• Concurrency is for making your program
cleaner.
More uses for threads
• for hiding latency
– e.g. downloading multiple web pages
• for encapsulating state
– talk to your state via a channel
• for making a responsive GUI
• fault tolerance, distribution Parallelism
• ... for making your program faster?
– are threads a good abstraction for multicore?
Why is concurrent programming hard?
• non-determinism
– threads interact in different ways depending on the
scheduler
– programmer has to deal with this somehow: locks,
messages, transactions
– hard to think about
– impossible to test exhaustively
• can we get parallelism without non-
determinism?
What Haskell has to offer
• Purely functional by default
– computing pure functions in parallel is deterministic
• Type system guarantees absence of side-effects
• Great facilities for abstraction
– Higher-order functions, polymorphism, lazy evaluation
• Wide range of concurrency paradigms
• Great tools
The rest of the talk
• Parallel programming in Haskell
• Concurrent data structures in Haskell
Parallel programming in Haskell
par :: a -> b -> b
Evaluate the first return the second
argument in parallel argument
Parallel programming in Haskell
par :: a -> b -> b
pseq :: a -> b -> b
Evaluate the first Return the second
argument argument
Using par and pseq
This does not
import Control.Parallel calculate the value
par
of p.indicates that ap
It allocates
main = could be evaluated
suspension, or
let pseq evaluates
in parallel with q
p = primes !! 3500 result:first,thunk,
then
for
returns
(pseq q (print
(primes !! (p,q))
3500)
q = nqueens 12 • p is sparked
(printby par
(p,q))
in • q is evaluated by pseq
par pp (pseq
par $ pseqq q(print
$ print (p,q)
(p,q)) • p is demanded by print
primes = ... • (p,q) is printed
nqueens = ...
write it like this if you
want (a $ b = a b)
ThreadScope
Zooming in...
The spark is picked up
here
How does par actually work?
Thread 1 Thread 3 Thread 2
CPU 0 CPU 1 CPU 2
Correctness-preserving optimisation
par a b == b
• Replacing “par a b” with “b” does not change
the meaning of the program
– only its speed and memory usage
– par cannot make the program go wrong
– no race conditions or deadlocks, guaranteed!
• par looks like a function, but behaves like an
annotation
How to use par
• par is very cheap: a write into a circular buffer
• The idea is to create a lot of sparks
– surplus parallelism doesn’t hurt
– enables scaling to larger core counts without
changing the program
• par allows very fine-grained parallelism
– but using bigger grains is still better
The N-queens problem
Place n queens on an n x n
board such that no queen
attacks any other, horizontally,
vertically, or diagonally
N queens
[1,3,1] [1,1]
[2,3,1]
[2,1] [1]
[3,3,1]
[4,3,1] [3,1] []
[5,3,1] [4,1]
[2]
[6,3,1]
...
...
...
N-queens in Haskell
nqueens :: Int -> [[Int]] A board is represented as a
nqueens n = subtree n [] list of queen rows
where
children :: [Int] -> [[Int]]
children b = [ (q:b) | q <- [1..n], children calculates the
safe q b ] valid boards that can
be made by adding
subtree :: Int -> [Int] -> [[Int]] another queen
subtree 0 b = [b]
subtree c b =
subtree calculates all
concat $
the valid boards
map (subtree (c-1)) $
starting from the given
children b
board by adding c
more columns
safe :: Int -> [Int] -> Bool
...
Parallel N-queens
• How can we parallelise this?
• Divide and conquer
[1]
– aka map/reduce
– calculate subtrees in parallel, []
join the results
[2]
...
Parallel N-queens
nqueens :: Int -> [[Int]]
nqueens n = subtree n []
where
children :: [Int] -> [[Int]]
children b = [ (q:b) | q <- [1..n],
safe q b ]
subtree :: Int -> [Int] -> [[Int]]
subtree 0 b = [b]
subtree c b =
parList :: [a] -> b -> b
concat $
parList $
map (subtree (c-1)) $
children b
parList is not built-in magic...
• It is defined using par:
parList :: [a] -> b -> b
parList [] b = b
parList (x:xs) b = par x $ parList xs b
• (full disclosure: in N-queens we need a slightly
different version in order to fully evaluate the
nested lists)
Results
• Speedup: 3.5 on 6 cores
• We can do better...
How many sparks?
SPARKS: 5151164 (5716 converted, 4846805 pruned)
• The cost of creating a spark for every tree node is
high
• sparks near the leaves are cheap
• Parallelism works better when the work units are
large (coarse-grained parallelism)
• But we don’t want to be too coarse, or there
won’t be enough grains
• Solution: parallelise down to a certain depth
Bounding the parallel depth
subtree :: Int -> [Int] -> [[Int]]
subtree 0 b = [b] change parList into
subtree c b = maybeParLIst
concat $
maybeParList c $
below the threshold,
map (subtree (c-1)) $
maybeParList is “id” (do
children b
nothing)
maybeParList c
| c < threshold = id
| otherwise = parList
Results...
• Speedup: 4.7 on 6 cores
– depth 3
– ~1000 sparks
Can this be improved?
• There is more we could do here, to optimise
both sequential and parallel performance
• but we got good results with only a little effort
Original sequential version
• However, we did have to change the original
program... trees good, lists bad:
nqueens :: Int -> [[Int]]
nqueens n = gen n
where
gen :: Int -> [[Int]]
gen 0 = [[]]
gen c = [ (q:b) | b <- gen (c-1),
q <- [1..n],
safe q b]
• c.f. Guy Steele “Organising Functional Code for
Parallel Execution”
Raising the level of abstraction
• Lowest level: par/pseq
• Next level: parList
• A general abstraction: Strategies1
A value of type Strategy a is a policy
for evaluating things of type a
parPair :: Strategy a -> Strategy b -> Strategy (a,b)
• a strategy for evaluating components of a pair in
parallel, given a Strategy for each component
1Algorithm + strategy = parallelism, Trinder et. al., JFP 8(1),1998
Define your own Strategies
• Strategies are just an abstraction, defined in
Haskell, on top of par/pseq
type Strategy a = a -> Eval a
using :: a -> Strategy a -> a
data Tree a = Leaf a | Node [Tree a]
parTree :: Int -> Strategy (Tree [Int]) A strategy that
parTree 0 tree = rdeepseq tree evaluates a tree in
parTree n (Leaf a) = return (Leaf a) parallel up to the given
parTree n (Node ts) = do depth
us <- parList (parTree (n-1)) ts
return (Node us)
Refactoring N-queens
data Tree a = Leaf a | Node [Tree a]
leaves :: Tree a -> [a]
nqueens n = leaves (subtree n [])
where
subtree :: Int -> [Int] -> Tree [Int]
subtree 0 b = Leaf b
subtree c b = Node (map (subtree (c-1)) (children b))
Refactoring N-queens
• Now we can move the parallelism to the outer
level:
nqueens n = leaves (subtree n [] `using` parTree 3)
Modular parallelism
• The description of the parallelism can be
separate from the algorithm itself
– thanks to lazy evaluation: we can build a
structured computation without evaluating it, the
strategy says how to evaluate it
– don’t clutter your code with parallelism
– (but be careful about space leaks)
Parallel Haskell, summary
• par, pseq, and Strategies let you annotate purely
functional code for parallelism
• Adding annotations does not change what the program
means
– no race conditions or deadlocks
– easy to experiment with
• ThreadScope gives visual feedback
• The overhead is minimal, but parallel programs scale
• You still have to understand how to parallelise the
algorithm!
• Complements concurrency
Take a deep breath...
• ... we’re leaving the purely functional world
and going back to threads and state
Concurrent data structures
• Concurrent programs often need shared data
structures, e.g. a database, or work queue, or
other program state
• Implementing these structures well is
extremely difficult
• So what do we do?
– let Someone Else do it (e.g. Intel TBB)
• but we might not get exactly what we want
– In Haskell: do it yourself...
Case study: Concurrent Linked Lists
newList :: IO (List a)
Creates a new (empty) list
addToTail :: List a -> a -> IO ()
Adds an element to the tail of the list
find :: Eq a => List a -> a -> IO Bool
Returns True if the list contains the given element
delete :: Eq a => List a -> a -> IO Bool
Deletes the given element from the list;
returns True if the list contained the element
Choose your weapon
CAS: atomic compare-and-swap,
accurate but difficult to use
MVar: a locked mutable variable.
Easier to use than CAS.
STM: Software Transactional
Memory. Almost impossible to
go wrong.
STM implementation
• Nodes are linked with transactional variables
data List a = Null
| Node { val :: a,
next :: TVar (List a) }
• Operations perform a transaction on the
whole list: simple and straightforward to
implement
• What about without STM, or if we want to
avoid large transactions?
What can go wrong?
1 2 3 4
thread 1: “delete 2” thread 2: “delete 3”
Fixing the race condition
Swinging the pointer will
not physically delete the
element now, it has to be
removed later
1 2d
2 3d
3 4
thread 1: “delete 2” thread 2: “delete 3”
Adding “lazy delete”
• Now we have a deleted node:
data List a = Null
| Node { val :: a,
next :: TVar (List a) }
| DelNode { next :: TVar (LIst a) }
• Traversals should drop deleted nodes that
they find.
• Transactions no longer take place on the
whole list, only pairs of nodes at a time.
We built a few implementations...
• Full STM
• Various “lazy delete” implementations:
– STM
– MVar, hand-over-hand locking
– CAS
– CAS (using STM)
– MVar (using STM)
Results
1000
100
CAS
CASusingSTM
Time(s) 10
LAZY
MLC
MLCusingSTM
1 STM
0.1
1 2 3 4 5 6 7 8
Processors
Results (scaling)
6
4
CAS
CASusingSTM
Speedup 3
LAZY
MLC
2
MLCusingSTM
STM
1
0
1 2 3 4 5 6 7 8
Procoessors
So what?
• Large STM transactions don’t scale
• The fastest implementations use CAS
• but then we found a faster implementation...
A latecomer wins the race...
1000
100
CAS
CASusingSTM
Time(s) 10 LAZY
MLC
MLCusingSTM
1 STM
???
0.1
1 2 3 4 5 6 7 8
Processors
And the winner is...
type List a = Var [a]
• Ordinary immutable lists stored in a single
mutable variable
• trivial to define the operations
• reads are fast and automatically concurrent:
– immutable data is copy-on-write
– a read grabs a snapshot
• but what about writes? Var = ???
Choose your weapon
IORef (unsynchronised mutable
variable)
MVar (locked mutable variable)
TVar (STM)
Built-in lock-free updates
• IORef provides this clever operation:
atomicModifyIORef Takes a mutable
:: IORef a variable
-> (a -> (a,b))
and a function to
-> IO b compute the new value
(a) and a result (b)
Returns the result
atomicModifyIORef r f = do
a <- readIORef r
let (new, b) = f a Lazily!
writeIORef r new
return b
Updating the list...
An unevaluated computation
• delete 2 representing the value of
IORef applying delete 2,
NB. a pure operation.
delete 2 The reason this works is
lazy evaluation
(:) (:)
1 2
Lazy immutable = parallel
• reads can happen in parallel with other
operations, automatically
• tree-shaped structures work well: operations
in branches can be computed in parallel
• lock-free: impossible to prevent other threads
from making progress
• The STM variant is composable
Ok, so why didn’t we see scaling?
• this is a shared data structure, a single point of
contention
• memory bottlenecks, cache bouncing
• possibly: interactions with generational GC
• but note that we didn’t see a slowdown either
A recipe for concurrent data structures
• Haskell has lots of libraries providing high-
performance pure data structures
• trivial to make them concurrent:
type ConcSeq a = IORef (Seq a)
type ConcTree a = IORef (Tree a)
type ConcMap k v = IORef (Map k v)
type ConcSet a = IORef (Set a)
Conclusions...
• Thinking concurrent (and parallel):
– Immutable data and pure functions
• eliminate unnecessary interactions
– Declarative programming models say less about
“how”, giving the implementation more freedom
• SQL/LINQ/PLINQ
• map/reduce
• .NET TPL: declarative parallelism in .NET
• F# async programming
• Coming soon: Data Parallel Haskell
Try it out...
• Haskell: http://www.haskell.org/
• GHC: http://www.haskell.org/ghc
• Libraries: http://hackage.haskell.org/
• News: http://www.reddit.com/r/haskell
