Advanced Scala

Advanced Scala
with Cats
Noel Welsh and Dave Gurnell

Early Access, May 2017
underscore
Copyright 2014-16 Noel Welsh and Dave Gurnell.

2
Advanced Scala with Cats

Early Access, May 2017
Copyright 2014-16 Noel Welsh and Dave Gurnell.
Published by Underscore Consul ng LLP, Brighton, UK.
Copies of this, and related topics, can be found at

h p://underscore.io/training. Team discounts, when available, may
also be found at that address. Contact the authors at
[email protected].
Underscore is a team of developers specialising in Scala and func onal

programming. You can nd us on the web at h p://underscore.io and
on Twi er at @underscoreio.
In addi on to wri ng so ware, we provide other training courses,

workshops, books, and mentoring to help you and your team create
be er so ware and have more fun. For more informa on please visit
h p://underscore.io/training.
Contents
Foreword 15
Notes on the Pre-Release Edi on . . . . . . . . . . . . . . . 16
Changelog . . . . . . . . . . . . . . . . . . . . . . . . 16
Omissions . . . . . . . . . . . . . . . . . . . . . . . . 16
Conven ons Used in This Book . . . . . . . . . . . . . . . . 17
Typographical Conven ons . . . . . . . . . . . . . . . 17
Source Code . . . . . . . . . . . . . . . . . . . . . . . 17
Callout Boxes . . . . . . . . . . . . . . . . . . . . . . . 18
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 19
I Theory 21
1 Introduc on 23
1.1 Anatomy of a Type Class . . . . . . . . . . . . . . . . . 23
1.1.1 The Type Class . . . . . . . . . . . . . . . . . . 24
1.1.2 Type Class Instances . . . . . . . . . . . . . . . 24
1.1.3 Interfaces . . . . . . . . . . . . . . . . . . . . . 25
3
4 CONTENTS
1.1.4 Exercise: Printable Library . . . . . . . . . . . . 26

1.1.5 Take Home Points . . . . . . . . . . . . . . . . 28
1.2 Meet Cats . . . . . . . . . . . . . . . . . . . . . . . . 29
1.2.1 Impor ng Type Classes . . . . . . . . . . . . . . 29
1.2.2 Impor ng Default Instances . . . . . . . . . . . 30
1.2.3 Impor ng Interface Syntax . . . . . . . . . . . . 31
1.2.4 Dening Custom Instances . . . . . . . . . . . . 32
1.2.5 Exercise: Cat Show . . . . . . . . . . . . . . . . 32
1.2.6 Take Home Points . . . . . . . . . . . . . . . . 33
1.3 Example: Eq . . . . . . . . . . . . . . . . . . . . . . . 33
1.3.1 Equality, Liberty, and Fraternity . . . . . . . . . 33
1.3.2 Comparing Ints . . . . . . . . . . . . . . . . . . 34
1.3.3 Comparing Op ons . . . . . . . . . . . . . . . . 35
1.3.4 Comparing Custom Types . . . . . . . . . . . . 36
1.3.5 Exercise: Equality, Liberty, and Felinity . . . . . . 36
1.3.6 Take Home Points . . . . . . . . . . . . . . . . 37
1.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 37
2 Monoids and Semigroups 39
2.1 Deni on of a Monoid . . . . . . . . . . . . . . . . . . 41

2.2 Deni on of a Semigroup . . . . . . . . . . . . . . . . 42
2.3 Exercise: The Truth About Monoids . . . . . . . . . . . 43
2.4 Exercise: All Set for Monoids . . . . . . . . . . . . . . . 44
2.5 Monoids in Cats . . . . . . . . . . . . . . . . . . . . . 44
2.5.1 The Monoid Type Class . . . . . . . . . . . . . . 44
CONTENTS 5
2.5.2 Obtaining Instances . . . . . . . . . . . . . . . 45

2.5.3 Default Instances . . . . . . . . . . . . . . . . . 46
2.5.4 Monoid Syntax . . . . . . . . . . . . . . . . . . 47
2.5.5 Exercise: Adding All The Things . . . . . . . . . 47
2.6 Controlling Instance Selec on . . . . . . . . . . . . . . 48
2.6.1 Type Class Variance . . . . . . . . . . . . . . . . 49
2.6.2 Iden cally Typed Instances . . . . . . . . . . . . 51
2.7 Applica ons of Monoids . . . . . . . . . . . . . . . . . 51
2.7.1 Big Data . . . . . . . . . . . . . . . . . . . . . 52
2.7.2 Distributed Systems . . . . . . . . . . . . . . . 52
2.7.3 Monoids in the Small . . . . . . . . . . . . . . . 53
2.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 53
3 Functors 55
3.1 Examples of Functors . . . . . . . . . . . . . . . . . . 55

3.2 More Examples of Functors . . . . . . . . . . . . . . . 57
3.3 Deni on of a Functor . . . . . . . . . . . . . . . . . . 60
3.4 Aside: Higher Kinds and Type Constructors . . . . . . . 61
3.5 Functors in Cats . . . . . . . . . . . . . . . . . . . . . 63
3.5.1 The Functor Type Class . . . . . . . . . . . . . . 63
3.5.2 Functor Syntax . . . . . . . . . . . . . . . . . . 64
3.5.3 Instances for Custom Types . . . . . . . . . . . 65
3.5.4 Exercise: Branching out with Functors . . . . . . 66
3.6 Contravariant and Invariant Functors . . . . . . . . . . . 66
3.6.1 Contravariant functors and the contramap method 67
6 CONTENTS
3.6.2 Invariant functors and the imap method . . . . . 69

3.6.3 Whats With the Name? . . . . . . . . . . . . . 71
3.7 Contravariant and Invariant in Cats . . . . . . . . . . . . 72
3.7.1 Contravariant in Cats . . . . . . . . . . . . . . . 73
3.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 74
4 Monads 77
4.1 What is a Monad? . . . . . . . . . . . . . . . . . . . . 78

4.1.1 Monad Deni on and Laws . . . . . . . . . . . 83
4.1.2 Exercise: Ge ng Func-y . . . . . . . . . . . . . 84
4.2 Monads in Cats . . . . . . . . . . . . . . . . . . . . . . 85
4.2.1 The Monad Type Class . . . . . . . . . . . . . . 85
4.2.3 Monad Syntax . . . . . . . . . . . . . . . . . . . 87
4.3 The Iden ty Monad . . . . . . . . . . . . . . . . . . . . 89
4.3.1 Exercise: Monadic Secret Iden es . . . . . . . 92
4.4 Either . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
4.4.1 Le and Right Bias . . . . . . . . . . . . . . . . 93
4.4.2 Crea ng Instances . . . . . . . . . . . . . . . . 94
4.4.3 Transforming Eithers . . . . . . . . . . . . . . . 97
4.4.4 Fail-Fast Error Handling . . . . . . . . . . . . . . 98
4.4.5 Represen ng Errors . . . . . . . . . . . . . . . 99
4.4.6 Exercise: What is Best? . . . . . . . . . . . . . . 100
4.5 The Eval Monad . . . . . . . . . . . . . . . . . . . . . 100
4.5.1 Eager, lazy, memoized, oh my! . . . . . . . . . . 101
CONTENTS 7
4.5.2 Evals models of evalua on . . . . . . . . . . . . 102

4.5.3 Eval as a Monad . . . . . . . . . . . . . . . . . 105
4.5.4 Trampolining and Eval.defer . . . . . . . . . . . . 107
4.5.5 Exercise: Safer Folding using Eval . . . . . . . . 108
4.6 The Writer Monad . . . . . . . . . . . . . . . . . . . . 109
4.6.1 Crea ng and Unpacking Writers . . . . . . . . . 109
4.6.2 Composing and Transforming Writers . . . . . . 111
4.6.3 Exercise: Show Your Working . . . . . . . . . . 114
4.7 The Reader Monad . . . . . . . . . . . . . . . . . . . . 115
4.7.1 Crea ng and Unpacking Readers . . . . . . . . . 116
4.7.2 Composing Readers . . . . . . . . . . . . . . . 116
4.7.3 Exercise: Hacking on Readers . . . . . . . . . . 117
4.7.4 When to Use Readers? . . . . . . . . . . . . . . 119
4.8 The State Monad . . . . . . . . . . . . . . . . . . . . . 120
4.8.1 Crea ng and Unpacking State . . . . . . . . . . 121
4.8.2 Composing and Transforming State . . . . . . . 122
4.8.3 Exercise: Post-Order Calculator . . . . . . . . . 124
4.9 Dening Custom Monads . . . . . . . . . . . . . . . . 128
4.9.1 Exercise: Branching out Further with Monads . . 129
4.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 129
5 Monad Transformers 131
5.1 A Transforma ve Example . . . . . . . . . . . . . . . . 132

5.2 Monad Transformers in Cats . . . . . . . . . . . . . . . 135
5.2.1 The Monad Transformer Classes . . . . . . . . . 135
8 CONTENTS
5.2.2 Building Monad Stacks . . . . . . . . . . . . . . 136

5.2.3 Construc ng and Unpacking Instances . . . . . . 139
5.2.4 Usage Pa erns . . . . . . . . . . . . . . . . . . 140
5.3 Exercise: Monads: Transform and Roll Out . . . . . . . 142
5.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 144
6 Cartesians and Applica ves 147
6.1 Cartesian . . . . . . . . . . . . . . . . . . . . . . . . . 149

6.1.1 Joining Two Contexts . . . . . . . . . . . . . . . 150
6.1.2 Joining Three or More Contexts . . . . . . . . . 150
6.2 Cartesian Builder Syntax . . . . . . . . . . . . . . . . . 151
6.2.1 Fancy Functors and Cartesian Builder Syntax . . 153
6.3 Cartesian Applied to Dierent Types . . . . . . . . . . . 155
6.3.1 Cartesian Applied to Future . . . . . . . . . . . . 155
6.3.2 Cartesian Applied to List . . . . . . . . . . . . . 156
6.3.3 Cartesian Applied to Either . . . . . . . . . . . . 157
6.3.4 Cartesian Applied to Monads . . . . . . . . . . . 157
6.4 Validated . . . . . . . . . . . . . . . . . . . . . . . . . 158
6.4.1 Crea ng Instances of Validated . . . . . . . . . . 159
6.4.2 Combining Instances of Validated . . . . . . . . 161
6.4.3 Methods of Validated . . . . . . . . . . . . . . . 162
6.4.4 Exercise: Form Valida on . . . . . . . . . . . . . 164
6.5 Apply and Applica ve . . . . . . . . . . . . . . . . . . . 165
6.5.1 The Hierarchy of Sequencing Type Classes . . . . 167
CONTENTS 9
6.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 169
7 Foldable and Traverse 171
7.1 Foldable . . . . . . . . . . . . . . . . . . . . . . . . . . 171
7.1.1 Folds and Folding . . . . . . . . . . . . . . . . . 172
7.1.2 Exercise: Reec ng on Folds . . . . . . . . . . . 173
7.1.3 Exercise: Scaf-fold-ing other methods . . . . . . 174
7.1.4 Foldable in Cats . . . . . . . . . . . . . . . . . . 174
7.2 Traverse . . . . . . . . . . . . . . . . . . . . . . . . . . 179
7.2.1 Traversing with Futures . . . . . . . . . . . . . . 179
7.2.2 Traversing with Applica ves . . . . . . . . . . . 182
7.2.3 Traverse in Cats . . . . . . . . . . . . . . . . . . 186
7.2.4 Unapply, traverseU, and sequenceU . . . . . . . . 187
7.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 189
II Case Studies 191
8 Case Study: Tes ng Asynchronous Code 193
8.1 Abstrac ng over Type Constructors . . . . . . . . . . . 195
8.2 Abstrac ng over Monads . . . . . . . . . . . . . . . . 196
8.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . . 198
9 Case Study: Pygmy Hadoop 199
9.1 Parallelizing map and fold . . . . . . . . . . . . . . . . . 199
9.2 Implemen ng foldMap . . . . . . . . . . . . . . . . . . 201

10 CONTENTS
9.3 Parallelising foldMap . . . . . . . . . . . . . . . . . . . 203
9.3.1 Futures, Thread Pools, and Execu onContexts . . 205
9.3.2 Dividing Work . . . . . . . . . . . . . . . . . . 207
9.3.3 Implemen ng parallelFoldMap . . . . . . . . . . 207
9.3.4 parallelFoldMap with more Cats . . . . . . . . . . 208
9.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 208
9.4.1 Batching Strategies in the Real World . . . . . . 209
9.4.2 Reduc on using Monoids . . . . . . . . . . . . . 209
10 Case Study: Data Valida on 211
10.1 Sketching the Library Structure . . . . . . . . . . . . . 213
10.2 The Check Datatype . . . . . . . . . . . . . . . . . . . 215
10.3 Basic Combinators . . . . . . . . . . . . . . . . . . . . 216
10.4 Transforming Data . . . . . . . . . . . . . . . . . . . . 218
10.4.1 Predicates . . . . . . . . . . . . . . . . . . . . 219
10.4.2 Checks . . . . . . . . . . . . . . . . . . . . . . 221
10.4.3 Recap . . . . . . . . . . . . . . . . . . . . . . . 223
10.5 Kleislis . . . . . . . . . . . . . . . . . . . . . . . . . . 225
10.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . 228
11 Case Study: Commuta ve Replicated Data Types 231
11.1 Eventual Consistency . . . . . . . . . . . . . . . . . . . 231
11.2 The GCounter . . . . . . . . . . . . . . . . . . . . . . 232
11.2.1 Simple Counters . . . . . . . . . . . . . . . . . 233
11.2.2 GCounters . . . . . . . . . . . . . . . . . . . . 234

CONTENTS 11
11.2.3 Exercise: GCounter Implementa on . . . . . . . 235
11.3 Generalisa on . . . . . . . . . . . . . . . . . . . . . . 236
11.3.1 Implementa on . . . . . . . . . . . . . . . . . . 238
11.3.2 Exercises . . . . . . . . . . . . . . . . . . . . . 238
11.4 Abstrac ng GCounter to a Type Class . . . . . . . . . . 239
11.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 248
III Solu ons to Exercises 249
A Solu ons for: Introduc on 251
A.1 Printable Library . . . . . . . . . . . . . . . . . . . . . 251
A.2 Printable Library Part 2 . . . . . . . . . . . . . . . . . . 252
A.3 Printable Library Part 3 . . . . . . . . . . . . . . . . . . 253
A.4 Cat Show . . . . . . . . . . . . . . . . . . . . . . . . 254
A.5 Equality, Liberty, and Felinity . . . . . . . . . . . . . . 255
B Solu ons for: Monoids and Semigroups 257
B.1 The Truth About Monoids . . . . . . . . . . . . . . . . 257
B.2 All Set for Monoids . . . . . . . . . . . . . . . . . . . 258
B.3 Adding All The Things . . . . . . . . . . . . . . . . . . 259
B.4 Adding All The Things Part 2 . . . . . . . . . . . . . . . 260
B.5 Adding All The Things Part 3 . . . . . . . . . . . . . . . 261
C Solu ons for: Functors 263
C.1 Branching out with Functors . . . . . . . . . . . . . . 263

12 CONTENTS
C.2 Showing o with Contramap . . . . . . . . . . . . . . 264
C.3 Showing o with Contramap Part 2 . . . . . . . . . . . 265
C.4 Transforma ve Thinking with Imap . . . . . . . . . . . 265
C.5 Transforma ve Thinking with Imap Part 2 . . . . . . . . 266
C.6 Transforma ve Thinking with Imap Part 3 . . . . . . . . 266
D Solu ons for: Monads 267
D.1 Ge ng Func-y . . . . . . . . . . . . . . . . . . . . . . 267
D.2 Monadic Secret Iden es . . . . . . . . . . . . . . . . 268
D.3 What is Best? . . . . . . . . . . . . . . . . . . . . . . 270
D.4 Safer Folding using Eval . . . . . . . . . . . . . . . . . 270
D.5 Show Your Working . . . . . . . . . . . . . . . . . . . 271
D.6 Hacking on Readers . . . . . . . . . . . . . . . . . . . 273
D.7 Hacking on Readers Part 2 . . . . . . . . . . . . . . . . 273
D.8 Hacking on Readers Part 3 . . . . . . . . . . . . . . . . 274
D.9 Post-Order Calculator . . . . . . . . . . . . . . . . . . 275
D.10 Post-Order Calculator Part 2 . . . . . . . . . . . . . . . 276
D.11 Branching out Further with Monads . . . . . . . . . . . 276
E Solu ons for: Monad Transformers 279
E.1 Monads: Transform and Roll Out . . . . . . . . . . . . 279
E.2 Monads: Transform and Roll Out Part 2 . . . . . . . . . 279

CONTENTS 13
F Solu ons for: Cartesians and Applica ves 283
F.1 Cartesian Applied to Monads . . . . . . . . . . . . . . 283

F.2 Form Valida on . . . . . . . . . . . . . . . . . . . . . 284
F.3 Form Valida on Part 2 . . . . . . . . . . . . . . . . . . 285
G Solu ons for: Foldable and Traverse 291
G.1 Reec ng on Folds . . . . . . . . . . . . . . . . . . . 291

G.2 Scaf-fold-ing other methods . . . . . . . . . . . . . . . 292
G.3 Traversing with Vectors . . . . . . . . . . . . . . . . . 293
G.4 Traversing with Vectors Part 2 . . . . . . . . . . . . . . 294
G.5 Traversing with Op ons . . . . . . . . . . . . . . . . . 294
G.6 Traversing with Validated . . . . . . . . . . . . . . . . 295
H Solu ons for: Case Study: Tes ng Asynchronous Code 297
H.1 Abstrac ng over Type Constructors . . . . . . . . . . . 297

H.2 Abstrac ng over Type Constructors Part 2 . . . . . . . 298
H.3 Abstrac ng over Monads . . . . . . . . . . . . . . . . 298
H.4 Abstrac ng over Monads Part 2 . . . . . . . . . . . . . 299
I Solu ons for: Case Study: Pygmy Hadoop 301
I.1 Implemen ng foldMap . . . . . . . . . . . . . . . . . . 301

I.2 Implemen ng foldMap Part 2 . . . . . . . . . . . . . . 301
I.3 Implemen ng parallelFoldMap . . . . . . . . . . . . . 302
14 CONTENTS
I.4 parallelFoldMap with more Cats . . . . . . . . . . . . . 304
J Solu ons for: Case Study: Data Valida on 307
J.1 Basic Combinators . . . . . . . . . . . . . . . . . . . . 307
J.2 Basic Combinators Part 2 . . . . . . . . . . . . . . . . 308
J.6 Checks . . . . . . . . . . . . . . . . . . . . . . . . . . 314
J.7 Checks Part 2 . . . . . . . . . . . . . . . . . . . . . . . 315
J.8 Checks Part 3 . . . . . . . . . . . . . . . . . . . . . . . 316
J.9 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . 317
J.10 Recap Part 2 . . . . . . . . . . . . . . . . . . . . . . . 320
J.11 Kleislis . . . . . . . . . . . . . . . . . . . . . . . . . . 323
J.12 Kleislis Part 2 . . . . . . . . . . . . . . . . . . . . . . . 323
K Solu ons for: Case Study: Commuta ve Replicated Data Types327
K.1 GCounter Implementa on . . . . . . . . . . . . . . . . 327
K.2 BoundedSemiLa ce Instances . . . . . . . . . . . . . 328
K.3 Generic GCounter . . . . . . . . . . . . . . . . . . . . 329

Foreword
The aims of this book are two-fold: to introduce monads, functors, and
other func onal programming pa erns as a way to structure program
design, and to explain how these concepts are implemented in Cats.
Monads, and related concepts, are the func onal programming equiva-
lent of object-oriented design pa ernsarchitectural building blocks
that turn up over and over again in code. They dier from object-
oriented pa erns in two main ways:
they are formally, and thus precisely, dened; and

they are extremely (extremely) general.
This generality means they can be dicult to understand; everyone nd

abstrac on dicult. However, it is generality that allows concepts like
monads to be applied in such a wide variety of situa ons.
In this book we aim to show the concepts in a number of dierent ways,

to help you build a mental model of how they work and where they
are appropriate. We have extended case studies, a simple graphical
nota on, many smaller examples, and of course the mathema cal def-
ini ons. Between them we hope youll nd something that works for
you.
Ok, lets get started!
15
16 CONTENTS
Notes on the Pre-Release Edi on
This book is in early access status. This means there are unnished
aspects as detailed below. There may be typos and errata in the
text and examples.
As an early access customer you will receive a free copy of the

nal text when it is released, plus free life me updates. If you
spot any mistakes or would like to provide feedback on the book,
please let us know!
Dave Gurnell ( [email protected] ) and Noel Welsh ( noel@

underscore.io ).
Changelog
Star ng from the March 2016 release, here are the major changes to
the book:
Moved the theore cal chapters from Scalaz to Cats (currently

version 0.7.2).
Added sec ons on the Reader, Writer, State, and Eval mon-
ads.
Added a chapter on monad transformers.
Added sec ons on Cartesian to the applica ves chapter.
Added sec ons on Foldable and Traverse.
Added type chart diagrams for Functor and Monad.
Added a new case study on asynchonous tes ng.
New diagrams for the map reduce and valida on case studies.
Omissions
Here are the major things missing from the book:

CONTENTS 17
1. Proof reading, nal tweaks.
2. Upgrade to Cats 1.0!
Conven ons Used in This Book
This book contains a lot of technical informa on and program code. We

use the following typographical conven ons to reduce ambiguity and
highlight important concepts:
Typographical Conven ons
New terms and phrases are introduced in italics. A er their ini al intro-
duc on they are wri en in normal roman font.
Terms from program code, lenames, and le contents, are wri en in

monospace font. Note that we do not dis nguish between singular
and plural forms. For example, might write String or Strings to refer
to java.util.String.
References to external resources are wri en as [hyperlinks][link-

underscore]. References to API documenta on are wri en using
a combina on of hyperlinks and monospace font, for example:
scala.Option.
Source Code
Source code blocks are wri en as follows. Syntax is highlighted appro-

priately where applicable:
18 CONTENTS
object MyApp extends App {

println("Hello world!") // Print a fine message to the user!
}
Most code passes through tut to ensure it compiles. tut uses the Scala
console behind the scenes, so we some mes wrap code in an object to
account for dierences between the console and regular code:
object example {
sealed trait Foo[A]
final case class Bar[A](a: A) extends Foo[A]
println(Bar("wrapping this code in an object makes sure tut

interprets it correctly"))
}
Callout Boxes
We use three types of callout box to highlight par cular content:
Tip callouts indicate handy summaries, recipes, or best prac ces.
Advanced callouts provide addi onal informa on on corner cases

or underlying mechanisms. Feel free to skip these on your rst
read-throughcome back to them later for extra informa on.
Warning callouts indicate common pi alls and gotchas. Make

sure you read these to avoid problems, and come back to them
if youre having trouble ge ng your code to run.
CONTENTS 19
Acknowledgements
Thanks to everyone who helped shape this book: Richard Dallaway,

Jonathon Ferguson, Cody Koeninger, Jason Sco , Toby Weston, Gregor
Ihmor, Narayan Iyer, all our students who have a ended a course online
or in person, and everyone else who gave feedback or encouraged us
to hurry up and nish!
20 CONTENTS
Part I
Theory
21
Chapter 1
Introduc on
Cats contains a wide variety of func onal programming tools and allows
developers to pick and choose the ones we want to use. The majority
of these tools are delivered in the form of type classes that we can apply
to exis ng Scala types.
Type classes are a programming pa ern origina ng in Haskell. They al-
low us to extend exis ng libraries with new func onality, without using
tradi onal inheritance, and without altering the original library source
code.
In this chapter we will refresh our memory of type classes from Under-
scores Essen al Scala book, and take a rst look at the Cats codebase.
We will look at two example type classesShow and Equsing them to
iden fy pa erns that lay the founda ons for the rest of the book.
1.1 Anatomy of a Type Class
There are three important components to the type class pa ern: the
type class itself, instances for par cular types, and the interface methods
that we expose to users.
23
24 CHAPTER 1. INTRODUCTION
1.1.1 The Type Class
A type class is an interface or API that represents some func onality we

want to implement. In Cats a type class is represented by a trait with
at least one type parameter. For example, we can represent generic
serialize to JSON behaviour as follows:
// Define a very simple JSON AST
sealed trait Json
final case class JsObject(get: Map[String, Json]) extends Json
final case class JsString(get: String) extends Json
final case class JsNumber(get: Double) extends Json
// The "serialize to JSON" behavior is encoded in this trait

trait JsonWriter[A] {
def write(value: A): Json
}
1.1.2 Type Class Instances
The instances of a type class provide implementa ons for the types we
care about, including types from the Scala standard library and types
from our domain model.
In Scala we dene instances by crea ng concrete implementa ons of
the type class and tagging them with the implicit keyword:
final case class Person(name: String, email: String)
object JsonWriterInstances {
implicit val stringJsonWriter = new JsonWriter[String] {
def write(value: String): Json =
JsString(value)
}
implicit val personJsonWriter = new JsonWriter[Person] {
def write(value: Person): Json =
JsObject(Map(
1.1. ANATOMY OF A TYPE CLASS 25
"name" -> JsString(value.name),

"email" -> JsString(value.email)
))
}
// etc...
}
1.1.3 Interfaces
An interface is any func onality we expose to users. Interfaces to type

classes are generic methods that accept instances of the type class as
implicit parameters.
There are two common ways of specifying an interface: Interface Ob-

jects and Interface Syntax.
Interface Objects
The simplest way of crea ng an interface is to place methods in a sin-

gleton object:
object Json {
def toJson[A](value: A)(implicit w: JsonWriter[A]): Json =
w.write(value)
}
To use this object, we import any type class instances we care about
and call the relevant method:
import JsonWriterInstances._
Json.toJson(Person("Dave", "[email protected]"))
// res4: Json = JsObject(Map(name -> JsString(Dave), email ->
JsString([email protected])))
Interface Syntax
We can alterna vely use extension methods to extend exis ng types

with interface methods. Cats refers to this as syntax for the type
class:
object JsonSyntax {
implicit class JsonWriterOps[A](value: A) {
def toJson(implicit w: JsonWriter[A]): Json =
w.write(value)
}
}
We use interface syntax by impor ng it along-side the instances for the

types we need:
import JsonWriterInstances._
import JsonSyntax._
Person("Dave", "[email protected]").toJson
// res5: Json = JsObject(Map(name -> JsString(Dave), email ->
JsString([email protected])))
1.1.4 Exercise: Printable Library
Scala provides a toString method to let us convert any value to a

String. However, this method comes with a few disadvantages. It
is implemented for every type in the language, many implementa ons
are of limited use, and we cant opt-in to specic implementa ons for
specic types.
Lets dene a Printable type class to work around these problems:
1. Dene a type class Printable[A] containing a single method

format. format should accept a value of type A and returns a
String.
You may occasionally see extension methods referred to as type enrichment or

pimping. These are older terms that we dont use anymore.
1.1. ANATOMY OF A TYPE CLASS 27
2. Create an object PrintableInstances containing instances of

Printable for String and Int.
3. Dene an object Printable with two generic interface meth-

ods:
format accepts a value of type A and a Printable of the

corresponding type. It uses the relevant Printable to con-
vert the A to a String.
print accepts the same parameters as format and returns
Unit. It prints the A value to the console using println.
See the solu on

Using the Library
The code above forms a general purpose prin ng library that we can
use in mul ple applica ons. Lets dene an applica on now that uses
the library:
1. Dene a data type Cat:
final case class Cat(

name: String,
age: Int,
color: String
)
2. Create an implementa on of Printable for Cat that returns

content in the following format:
NAME is a AGE year-old COLOR cat.
3. Finally, use the type class on the console or in a short demo app:
create a Cat and print it to the console:
// Define a cat:
val cat = Cat(/* ... */)
// Print the cat!
See the solu on

Be er Syntax
Lets make our prin ng library easier to use by dening some extension
methods to provide be er syntax:
1. Create an object called PrintableSyntax.
2. Inside PrintableSyntax dene an implicit class PrintOps[A]

to wrap up a value of type A.
3. In PrintOps dene the following methods:
format accepts an implicit Printable[A] and returns a

String representa on of the wrapped A;
print accepts an implicit Printable[A] and returns Unit.

It prints the wrapped A to the console.
4. Use the extension methods to print the example Cat you created
in the previous exercise.
See the solu on
1.1.5 Take Home Points
In this sec on we revisited the concept of a type class, which allows us

to add new func onality to exis ng types.
The Scala implementa on of a type class has three parts:
1.2. MEET CATS 29
the type class itself, a generic trait;

instances for each type we care about; and
one or more generic interface methods.
Interface methods can be dened in interface objects or interface syntax.

Implicit classes are the most common way of implemen ng syntax.
In the next sec on we will take a rst look at Cats. We will examine
the standard code layout Cats uses to organize its type classes, and see
how to select type classes, instances, and syntax for use in our code.
1.2 Meet Cats
In the previous sec on we saw how to implement type classes in Scala.

In this sec on we will look at how type classes are implemented in Cats.
Cats is wri en using a modular structure that allows us to choose which

type classes, instances, and interface methods we want to use. Lets
take a rst look using cats.Show as an example.
Show is Cats equivalent of the Printable type class we dened in the

last sec on. It provides a mechanism for producing developer-friendly
console output without using toString.
Show denes one method of interest:
def show[A](value: A): String = ???

// show: [A](value: A)String
1.2.1 Impor ng Type Classes
The type classes in Cats are dened in the cats package. We can import
Show directly from this package:
import cats.Show
The companion object of every Cats type class has an apply method
that locates an instance for any type we specify:
val showInt = Show.apply[Int]

// <console>:13: error: could not find implicit value for
parameter instance: cats.Show[Int]
// val showInt = Show.apply[Int]
// ^
Oopsthat didnt work! The apply method uses implicits to look up

individual instances, so well have to bring some instances into scope.
1.2.2 Impor ng Default Instances
The cats.instances package provides default instances for a wide

variety of types. We can import these as shown in the table below.
Each import provides instances of all Cats type classes for a specic
target type:
Import Parameter types
cats.instances.int Int
cats.instances.string String
cats.instances.list List
cats.instances.option Option
cats.instances.map Map and subtypes
cats.instances.all All instances
and so on See the
cats.instances
package for more
1.2. MEET CATS 31
Most people use import cats.implicits.all._ to bring all in-

stances into scope at the same me. In this book we will use
specic imports to show you exactly which instances we need in
each case. Dont feel you have to do this in your code.
Lets import the instances of Show for Int and String:

import cats.instances.int._
import cats.instances.string._
val showInt: Show[Int] = Show.apply[Int]

val showString: Show[String] = Show.apply[String]
Thats be er! We now have access to two instances of Show, and can
use them to print Ints and Strings:
val intAsString: String =
showInt.show(123)
// intAsString: String = 123
val stringAsString: String =

showString.show("abc")
// stringAsString: String = abc
1.2.3 Impor ng Interface Syntax
We can make Show easier to use by impor ng the interface syntax from
cats.syntax.show. This adds a show method to any type for which
we have an instance of Show in scope:
import cats.syntax.show._
val shownInt = 123.show

// shownInt: String = 123
val shownString = "abc".show

// shownString: String = abc
Cats provides separate syntax imports for each type class. We will in-
troduce these as we encounter them in later sec ons and chapters.
1.2.4 Dening Custom Instances
There are two constructor methods on the companion object of Show

that we can use to dene instances for our own types:
// Convert a function to a `Show` instance:

def show[A](f: A => String): Show[A] = ???
// Create a `Show` instance from a `toString` method:

def fromToString[A]: Show[A] = ???
These allows us to quickly construct instances of Show.
import java.util.Date
implicit val dateShow: Show[Date] =

Show.show(date => s"${date.getTime}ms since the epoch.")
These constructors exist for Show but dont make sense for all Cats type
classes. We will introduce constructors for other type classes as we
come to then.
1.2.5 Exercise: Cat Show
Re-implement the Cat applica on from the previous sec on using Show
instead of Printable.
See the solu on

1.3. EXAMPLE: EQ 33
Cats type classes are dened in the cats package. For example, the
Show type class is dened as cats.Show.
Default instances are dened in the cats.instances package. Imports

are organized by parameter type (as opposed to by type class).
Interface syntax is dened in the cats.syntax package. There are sep-
arate syntax imports for each type class. For example, the syntax for
Show is dened in cats.syntax.show.
1.3 Example: Eq
We will nish o this chapter by looking at another useful type class:

cats.Eq.
1.3.1 Equality, Liberty, and Fraternity
We can use Eq to dene type-safe equality between instances of any

given type:
package cats
trait Eq[A] {
def eqv(a: A, b: A): Boolean
// other concrete methods based on eqv...
}
The interface syntax, dened in [cats.syntax.equal][cats.syntax.equal],

provides two methods for performing type-safe equality checks pro-
vided there is an instance Eq[A] in scope:
=== compares two objects for equality;

=!= compares two objects for inequality.
1.3.2 Comparing Ints
Lets look at a few examples. First we import the type class:

import cats.Eq
Now lets grab an instance for Int:

val eqInt = Eq[Int]
We can use eqInt directly to test for equality:

eqInt.eqv(123, 123)
// res1: Boolean = true
eqInt.eqv(123, 234)
// res2: Boolean = false
Unlike Scalas == method, if we try to compare objects of dierent types

using eqv we get a compile error:
eqInt.eqv(123, "234")
// <console>:18: error: type mismatch;
// found : String("234")
// required: Int
// eqInt.eqv(123, "234")
// ^
We can also import the interface syntax in cats.syntax.eq to use the

=== and =!= methods:
import cats.syntax.eq._
123 === 123

123 =!= 234

1.3. EXAMPLE: EQ 35
1.3.3 Comparing Op ons
Now for a more interes ng exampleOption[Int]. To compare values

of type Option[Int] we need to import instances of Eq for Option as
well as Int:
import cats.instances.option._
Now we can try some comparisons:
Some(1) === None

// <console>:26: error: value === is not a member of Some[Int]
// Some(1) === None
// ^
We have received a compile error here because the Eq type class is

invariant. The instances we have in scope are for Int and Option[Int],
not Some[Int]. To x the issue we have to re-type the arguments as
Option[Int]:
(Some(1) : Option[Int]) === (None : Option[Int])

We can do this in a friendlier fashion using the Option.apply and

Option.empty methods from the standard library:
Option(1) === Option.empty[Int]

or using special syntax from cats.syntax.option:

import cats.syntax.option._
1.some === None

1.some =!= None

1.3.4 Comparing Custom Types
We can dene our own instances of Eq using the Eq.instance method,

which accepts a func on of type (A, A) => Boolean and returns an
Eq[A]:
import cats.instances.long._
implicit val dateEqual = Eq.instance[Date] { (date1, date2) =>

date1.getTime === date2.getTime
}
val x = new Date() // now

val y = new Date() // a bit later than now
x === x
x === y
1.3.5 Exercise: Equality, Liberty, and Felinity
Implement an instance of Eq for our running Cat example:

1.4. SUMMARY 37
final case class Cat(name: String, age: Int, color: String)
Use this to compare the following pairs of objects for equality and in-
equality:
val cat1 = Cat("Garfield", 35, "orange and black")
val cat2 = Cat("Heathcliff", 30, "orange and black")
val optionCat1 = Option(cat1)

val optionCat2 = Option.empty[Cat]
See the solu on
In this sec on we introduced a new type classcats.Eqthat lets us

perform type-safe equality checks:
we create an instance Eq[A] to implement equality-tes ng func-

onality for A.
cats.syntax.eq provides two methods of interest: === for test-

ing equality and =!= for tes ng inequality.
Because Eq[A] is invariant in A, we have to be precise about the types

of the values we use as arguments. We some mes need to manually
type expressions in our code to help the compiler locate the correct
type class instances.
1.4 Summary
In this chapter we took a rst look at type classes. We implemented

our own Printable type class using plain Scala before looking at two
examples from CatsShow and Eq.
We have now seen the general pa erns in Cats type classes:
The type classes themselves are generic traits in the cats pack-
age.
Each type class has a companion object with:
an apply method for materializing instances;

typically, one or more addi onal methods for crea ng in-
stances.
Default instances are provided via the cats.instances pack-

age, and are organized by parameter type rather than by type
class.
Many type classes have syntax provided via the cats.syntax

package.
In the remaining chapters of this book we will look at four broad and
powerful type classesMonoids, Functors, Monads, Applicatives,
and more. In each case we will learn what func onality the type class
provides, the formal rules it follows, and how it is implemented in Cats.
Many of these type classes are more abstract than Show or Eq. While
this makes them harder to learn, it makes them far more useful for solv-
ing general problems in our code.
Chapter 2
Monoids and Semigroups
In this sec on we explore our rst type classes, monoid and semigroup.
These allow us to add or combine values. There are instances for Ints,
Strings, Lists, Options, and many, many more. Lets start by looking
at a few simple types and opera ons to see what common principles
we can extract.
Integer addi on
Addi on of Ints is a binary opera on that is closed, meaning that

adding two Ints always produces another Int:
2 + 1
// res0: Int = 3
There is also the iden ty element 0 with the property that a + 0 == 0

+ a == a for any Int a:
2 + 0
// res1: Int = 2
39
40 CHAPTER 2. MONOIDS AND SEMIGROUPS
0 + 2
// res2: Int = 2
There are also other proper es of addi on. For instance, it doesnt mat-
ter in what order we add elements because we always get the same
result. This is a property known as associa vity:
(1 + 2) + 3
// res3: Int = 6
1 + (2 + 3)
// res4: Int = 6
Integer mul plica on
The same proper es for addi on also apply for mul plica on, provided
wee use 1 as the iden ty instead of 0:
1 * 3
// res5: Int = 3
3 * 1
// res6: Int = 3
Mul plica on, like addi on, is associa ve:
(1 * 2) * 3
// res7: Int = 6
1 * (2 * 3)
// res8: Int = 6
String and sequence concatena on
We can also add Strings, using string concatena on as our binary op-
erator:
2.1. DEFINITION OF A MONOID 41
"One" ++ "two"
// res9: String = Onetwo
and the empty string as the iden ty:
"" ++ "Hello"
// res10: String = Hello
"Hello" ++ ""
// res11: String = Hello
Once again, concatena on is associa ve:
("One" ++ "Two") ++ "Three"

// res12: String = OneTwoThree
"One" ++ ("Two" ++ "Three")

// res13: String = OneTwoThree
Note that we used ++ above instead of the more usual + to suggest a par-
allel with sequences. We can do exactly the same with other types of
sequence, using concatena on as as the binary operator and the empty
sequence as our iden ty.
2.1 Deni on of a Monoid
Weve seen a number of addi on scenarios above each with an asso-

cia ve binary addi on and an iden ty element. It will be no surprise to
learn that this is a monoid. Formally, a monoid for a type A is:
an opera on combine with type (A, A) => A

an element empty of type A
This deni on translates nicely into Scala code. Here is a simplied

version of the deni on from Cats:
trait Monoid[A] {
def combine(x: A, y: A): A
def empty: A
}
In addi on to providing these opera ons, monoids must formally obey

several laws. For all values x, y, and z, in A, combine must be associa ve
and empty must be an iden ty element:
def associativeLaw[A](x: A, y: A, z: A)
(implicit m: Monoid[A]): Boolean =
m.combine(x, m.combine(y, z)) == m.combine(m.combine(x, y), z)
def identityLaw[A](x: A)
(implicit m: Monoid[A]): Boolean = {
(m.combine(x, m.empty) == x) &&
(m.combine(m.empty, x) == x)
}
Integer subtrac on, for example, is not a monoid because subtrac on

is not associa ve:
(1 - 2) - 3
// res15: Int = -4
1 - (2 - 3)
// res16: Int = 2
In prac ce we only need to think about laws when we are wri ng our
own Monoid instances for custom data types. Most of the me we can
rely on the instances provided by Cats and assume the library authors
know what theyre doing.
2.2 Deni on of a Semigroup
A semigroup is simply the combine part of a monoid. While many semi-

groups are also monoids, there are some data types for which we can-
2.3. EXERCISE: THE TRUTH ABOUT MONOIDS 43
not dene an empty element. For example, we have just seen that se-
quence concatena on and integer addi on are monoids. However, if
we restrict ourselves to non-empty sequences and posi ve integers, we
lose access to an empty element. Cats has a NonEmptyList data type
that has an implementa on of Semigroup but no implementa on of
Monoid.
A more accurate (though s ll simplied) deni on of Cats Monoid is:
trait Semigroup[A] {
}
trait Monoid[A] extends Semigroup[A] {

def empty: A
}
Well see this kind of inheritance o en when discussing type classes. It

provides modularity and allows us to re-use behaviour. If we dene a
Monoid for a type A, we get a Semigroup for free. Similarly, if a method
requires a parameter of type Semigroup[B], we can pass a Monoid[B]
instead.
2.3 Exercise: The Truth About Monoids
Weve seen a few examples of monoids but there are plenty more to
be found. Consider Boolean. How many monoids can you dene for
this type? For each monoid, dene the combine and empty opera ons
and convince yourself that the monoid laws hold. Use the following
deni ons as a star ng point:
trait Semigroup[A] {
}
trait Monoid[A] extends Semigroup[A] {

def empty: A
}
object Monoid {
def apply[A](implicit monoid: Monoid[A]) =
monoid
}
See the solu on
2.4 Exercise: All Set for Monoids
What monoids and semigroups are there for sets?
See the solu on
2.5 Monoids in Cats
Now weve seen what a monoid is, lets look at their implementa on in
Cats. Once again well look at the three main aspects of the implemen-
ta on: the type class, the instances, and the interface.
2.5.1 The Monoid Type Class
The monoid type class is cats.kernel.Monoid, which is aliased as

cats.Monoid. Monoid extends cats.kernel.Semigroup, which is
aliased as cats.Semigroup. When using Cats we normally import
type classes from the cats package:
2.5. MONOIDS IN CATS 45
import cats.Monoid
import cats.Semigroup
Cats Kernel?
Cats Kernel is a subproject of Cats providing a small set of

typeclasses for libraries that dont require the full Cats toolbox.
While these core type classes are technically dened in the
cats.kernel package, they are all aliased to the cats package
so we rarely need to be aware of the dis nc on.
The Cats Kernel type classes covered in this book are Eq,
Semigroup, and Monoid. All the other type classes we cover are
part of the main Cats project and are dened directly in the cats
package.
2.5.2 Obtaining Instances
Monoid follows the standard Cats pa ern for the user interface: the
companion object has an apply method that returns the type class in-
stance. So if we wanted the monoid instance for String, and we have
the correct implicits in scope, we can write the following:
import cats.Monoid
Monoid[String].combine("Hi ", "there")

// res0: String = Hi there
Monoid[String].empty
// res1: String = ""
which is equivalent to:

Monoid.apply[String].combine("Hi ", "there")

Monoid.apply[String].empty
// res3: String = ""
As we know, Monoid extends Semigroup. If we dont need empty we

can equivalently write:
Semigroup[String].combine("Hi ", "there")

2.5.3 Default Instances
The type class instances for Monoid are organised under cats.instances
in the standard way described in Chapter 1. For example, if we want
to pull in instances for Int we import from cats.instances.int:
import cats.Monoid
Monoid[Int].combine(32, 10)
// res5: Int = 42
Similarly, we can assemble a Monoid[Option[Int]] using instances

from cats.instances.int and cats.instances.option:
import cats.Monoid
val a = Option(22)
// a: Option[Int] = Some(22)
val b = Option(20)
2.5. MONOIDS IN CATS 47
// b: Option[Int] = Some(20)
Monoid[Option[Int]].combine(a, b)
// res6: Option[Int] = Some(42)
Refer back to Chapter 1 for a more comprehensive list of imports.
2.5.4 Monoid Syntax
Cats provides syntax for the combine method in the form of the |+|
operator. Because combine technically comes from Semigroup, we ac-
cess the syntax by impor ng from cats.syntax.semigroup:
import cats.syntax.semigroup._
val stringResult = "Hi " |+| "there" |+| Monoid[String].empty

// stringResult: String = Hi there
val intResult = 1 |+| 2 |+| Monoid[Int].empty

// intResult: Int = 3
2.5.5 Exercise: Adding All The Things
The cu ng edge SuperAdder v3.5a-32 is the worlds rst choice for

adding together numbers. The main func on in the program has sig-
nature def add(items: List[Int]): Int. In a tragic accident this
code is deleted! Rewrite the method and save the day!
See the solu on
Well done! SuperAdders market share con nues to grow, and now
there is demand for addi onal func onality. People now want to add
List[Option[Int]]. Change add so this is possible. The SuperAd-

der code base is of the highest quality, so make sure there is no code
duplica on!
See the solu on
SuperAdder is entering the POS (point-of-sale, not the other POS) mar-
ket. Now we want to add up Orders:
case class Order(totalCost: Double, quantity: Double)
We need to release this code really soon so we cant make any modi-
ca ons to add. Make it so!
See the solu on
2.6 Controlling Instance Selec on
When working with type classes we must consider two issues that con-
trol instance selec on:
What is the rela onship between an instance dened on a type

and its subtypes?
For example, if we dene a Monoid[Option[Int]], will the ex-
pression Some(1) |+| Some(2) select this instance? (Remem-
ber that Some is a subtype of Option).
How do we choose between type class instances when there are

many available?
Weve seen two monoids for Int: addi on and zero, and mul -
plica on and one. Similarly there are at least four monoids for
Boolean (and, or, equal, and not equal). When we write true
|+| false, which instance is selected?
In this sec on we explore how Cats answers these ques ons.

2.6. CONTROLLING INSTANCE SELECTION 49
2.6.1 Type Class Variance
When we dene type classes we can add variance annota ons to the
type parameter like we can for any other generic type. To quickly recap,
there are three cases:
A type with an unannotated parameter Foo[A] is invariant in A.

This means there is no rela onship between Foo[B] and Foo[C]
no ma er what the sub- or super-type rela onship is between B
and C.
A type with a parameter Foo[+A] is covariant in A.

If C is a subtype of B, Foo[C] is a subtype of Foo[B].
A type with a parameter Foo[-A] is contravariant in A.

If C is a supertype of B, Foo[C] is a subtype of Foo[B].
When the compiler searches for an implicit it looks for one matching
the type or subtype. Thus we can use variance annota ons to control
type class instance selec on to some extent.
There are two issues that tend to arise. Lets imagine we have an alge-
braic data type like:
sealed trait A
final case object B extends A
final case object C extends A
The issues are:
1. Will an instance dened on a supertype be selected if one is avail-

able? For example, can we dene an instance for A and have it
work for values of type B and C?
2. Will an instance for a subtype be selected in preference to that

of a supertype. For instance, if we dene an instance for A and B,
and we have a value of type B, will the instance for B be selected
in preference to A?
It turns out we cant have both at once. The three choices give us be-
haviour as follows:
Contravariant
Type Class Variance Invariant Covariant
Supertype instance used? No No Yes

More specic type Yes Yes No
preferred?
Its clear there is no perfect system. Cats generally prefers to use in-
variant type classes. This allows us to specify more specic instances
for subtypes if we want. It does mean that if we have, for example, a
value of type Some[Int], our monoid instance for Option will not be
used. We can solve this problem with a type annota on like Some(1)
: Option[Int] or by using smart constructors that construct values
with the type of the base trait in an algebraic data type. For example,
Cats provides some and none constructors for Option:
import cats.syntax.option._
Some(1)
// res0: Some[Int] = Some(1)
1.some
None
// res2: None.type = None
2.7. APPLICATIONS OF MONOIDS 51
none[Int]
// res3: Option[Int] = None
2.6.2 Iden cally Typed Instances
The other issue is choosing between type class instances when several
are available for a specic type. For example, how do we select the
monoid for integer mul plica on instead of the monoid for integer ad-
di on?
Cats currently has no mechanism for selec ng alterna ve instances,
though this may change in the future.
We can always dene or import a type class instance into the local
scope. This will take precedence over other type class instances in the
implicit scope:
import cats.Monoid
implicit val multiplicationMonoid =

new Monoid[Int] {
def empty: Int = 1
override def combine(x: Int, y: Int): Int = x * y
}
3 |+| 2
// res5: Int = 6
2.7 Applica ons of Monoids
We now know what a moniod isan abstrac on of the concept of

adding or combiningbut where is it useful? Here are a few big ideas
where monoids play a major role. These are explored in more detail in
case studies later in the book.
2.7.1 Big Data
In big data applica ons like Spark and Hadoop we distribute data anal-
ysis over many machines, giving fault tolerance and scalability. This
means each machine will return results over a por on of the data, and
we must then combine these results to get our nal result. In the vast
majority of cases this can be viewed as a monoid.
If we want to calculate how many total visitors a web site has received,
that means calcula ng an Int on each por on of the data. We know the
monoid instance of Int is addi on, which is the right way to combine
par al results.
If we want to nd out how many unique visitors a website has received,

thats equivalent to building a Set[User] on each por on of the data.
We know the monoid instance for Set is the set union, which is the
right way to combine par al results.
If we want to calculate 99% and 95% response mes from our server
logs, we can use a data structure called a QTree for which there is a
monoid.
Hopefully you get the idea. Almost every analysis that we might want
to do over a large data set is a monoid, and therefore we can build
an expressive and powerful analy cs system around this idea. This is
exactly what Twi ers Algebird and Summingbird projects have done.
We explore this idea further in the Map-Reduce case study.
2.7.2 Distributed Systems
In a distributed system, dierent machines may end up with dierent

views of data. For example, one machine may receive an update that
other machines did not receive. We would like to reconcile these dif-
ferent views, so every machine has the same data if no more updates
arrive. This is called eventual consistency.
2.8. SUMMARY 53
A par cular class of data types support this reconcilia on. These data
types are called commuta ve replicated data types (CRDTs). The key
opera on is the ability to merge two data instances, with a result that
captures all the informa on in both instances. This opera on relies on
having a monoid instance. We explore this idea further in the CRDT
case study.
2.7.3 Monoids in the Small
The two examples above are cases where monoids inform the en re
system architecture. There are also many cases where having a monoid
around makes it easier to write a small code fragment. Well see lots of
examples in the case studies in this book.
2.8 Summary
We hit a big milestone in this chapterwe covered our rst type classes
with fancy func onal programming names:
a Semigroup represents an addi on or combina on opera on;

a Monoid extends a Semigroup by adding an iden ty or zero
element.
We can use Semigroups and Monoids by impor ng three things: the

type classes themselves, the instances for the types we care about, and
the semigroup syntax to give us the |+| operator:
import cats.Monoid
import cats.instances.all._
With these three things in scope, we can set about adding anything we
want:
Option(1) |+| Option(2)

val map1 = Map("a" -> 1, "b" -> 2)

val map2 = Map("b" -> 3, "d" -> 4)
map1 |+| map2

// res1: Map[String,Int] = Map(b -> 5, d -> 4, a -> 1)
val tuple1 = ("hello", 123)

val tuple2 = ("world", 321)
tuple1 |+| tuple2

// res2: (String, Int) = (helloworld,444)
Monoids are a great gateway to Cats. Theyre easy to understand and

simple to use. However, theyre just the p of the iceberg in terms of
the abstrac ons Cats enables us to make. In the next chapter well look
at functors, the type class personica on of the beloved map method.
Thats where the fun really begins!
Chapter 3
Functors
In this chapter we will inves gate functors. Functors on their own

arent so useful, but special cases of functors such as monads and ap-
plica ve functors are some of the most commonly used abstrac ons in
Cats.
3.1 Examples of Functors
Informally, a functor is anything with a map method. You probably know

lots of types that have this: Option, List, Either, and Future, to
name a few.
Lets start as we did with monoids by looking at a few types and opera-
ons and seeing what general principles we can abstract.
Sequences
The map method is perhaps the most commonly used method on List.
If we have a List[A] and a func on A => B, map will create a List[B].
55
56 CHAPTER 3. FUNCTORS
List(1, 2, 3).map(x => (x % 2) == 0)

// res0: List[Boolean] = List(false, true, false)
There are some proper es of map that we rely on without even think-
ing about them. For example, we expect the two snippets below to
produce the same output:
List(1, 2, 3).map(_ * 2).map(_ + 4)

// res1: List[Int] = List(6, 8, 10)
List(1, 2, 3).map(x => (x * 2) + 4)

In general, the map method for a List works like this: We start with a
List[A] of length n, we supply a func on from A to B, and we end up
with a List[B] of length n. The elements are changed but the ordering
and length of the list are preserved. This is illustrated in Figure 3.1.
map
List[A] A => B List[B]
Figure 3.1: Type chart: mapping over a List
Op ons
We can do the same thing with an Option. If we have a Option[A] and

a func on A => B, map will create a Option[B]:
Option(1).map(_.toString)
// res3: Option[String] = Some(1)
We expect map on Option to behave in the same way as List:

3.2. MORE EXAMPLES OF FUNCTORS 57
Option(123).map(_ * 4).map(_ + 4)
Option(123).map(x => (x * 2) + 4)
In general, the map method for an Option works similarly to that for a
List. We start with an Option[A] that is either a Some[A] or a None,
we supply a func on from A to B, and the result is either a Some[B] or
a None. Again, the structure is preserved: if we start with a Some we
end up with a Some, and a None always maps to a None. This is shown
in Figure 3.2.
map
Option[A] A => B Option[B]
Figure 3.2: Type chart: mapping over an Op on
3.2 More Examples of Functors
Lets expand how we think about map by taking some other examples
into account:
Futures
Future is also a functor with a map method. If we start with a

Future[A] and call map supplying a func on A => B, we end up with
a Future[B]:
Some func onal purists disagree with this because the excep on handling in
Scala futures breaks the functor laws. Were going to ignore this detail because real
func onal programs dont do excep ons.
import scala.concurrent.{Future, Await}

import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration._
val future1 = Future("Hello world!")

// future1: scala.concurrent.Future[String] = Future(<not
completed>)
val future2 = future1.map(_.length)

// future2: scala.concurrent.Future[Int] = Future(<not completed
>)
Await.result(future1, 1.second)
// res6: String = Hello world!
// res7: Int = 12
The general pa ern looks like Figure 3.3. Seem familiar?
map
Future[A] A => B Future[B]
Figure 3.3: Type chart: mapping over a Future
Func ons (?!)
Can we map over func ons of a single argument? What would this
mean?
All our examples above have had the following general shape:
start with F[A];

supply a func on A => B;
get back F[B].
3.2. MORE EXAMPLES OF FUNCTORS 59
A func on with a single argument has two types: the parameter type
and the result type. To get them to the same shape we can x the
parameter type and let the result type vary:
start with X => A;

supply a func on A => B;
get back X => B.
We can see this with our trusty type chart in Figure 3.4.
map
X => A A => B X => B
Figure 3.4: Type chart: mapping over a Func on1
In other words, mapping over a Function1 is just func on composi-

on:
import cats.instances.function._
import cats.syntax.functor._
val func1 = (x: Int) => x.toDouble

// func1: Int => Double = <function1>
val func2 = (y: Double) => y * 2

// func2: Double => Double = <function1>
val func3 = func1.map(func2)

// func3: Int => Double = scala.runtime.
AbstractFunction1$$Lambda$9405/272413009@2deb99ee
func3(1) // function composition by calling map

// res8: Double = 2.0
func2(func1(1)) // function composition written out by hand

3.3 Deni on of a Functor
Formally, a functor is a type F[A] with an opera on map with type (A

=> B) => F[B]. The general type chart is shown in Figure 3.5.
map
F[A] A => B F[B]
Figure 3.5: Type chart: generalised functor map
Intui vely, a functor F[A] represents some data (the A type) in a context
(the F type). The map opera on modies the data within but retains the
structure of the surrounding context. To ensure this is the case, the
following laws must hold:
Iden ty: calling map with the iden ty func on is the same as doing noth-
ing:
fa.map(a => a) == fa
Composi on: mapping with two func ons f and g is the same as
mapping with f and then mapping with g:
fa.map(g(f(_))) == fa.map(f).map(g)
If we consider the laws in the context of the functors weve discussed

above, we can see they make sense and are true. Weve seen some
examples of the second law already.
A simplied version of the deni on from Cats is:
3.4. ASIDE: HIGHER KINDS AND TYPE CONSTRUCTORS 61
import scala.language.higherKinds
trait Functor[F[_]] {
def map[A, B](fa: F[A])(f: A => B): F[B]
}
If you havent seen syntax like F[_] before, its me to take a brief de-
tour to discuss type constructors and higher kinded types. Well explain
that scala.language import as well.
3.4 Aside: Higher Kinds and Type Constructors
Kinds are like types for types. They describe the number of holes in
a type. We dis nguish between regular types that have no holes, and
type constructors that have holes that we can ll to produce types.
For example, List is a type constructor with one hole. We ll that hole
by specifying a parameter to produce a regular type like List[Int]
or List[A]. The trick is not to confuse type constructors with generic
types. List is a type constructor, List[A] is a type:
List // type constructor, takes one parameter

List[A] // type, produced using a type parameter
Theres a close analogy here with func ons and values. Func ons are
value constructorsthey produce values when we supply parameters:
math.abs // function, takes one parameter

math.abs(x) // value, produced using a value parameter
Kind nota on
We some mes use kind nota on to describe the shape of types

and their constructors. Regular types have a kind *. List has
kind * => * to indicate that it produces a type given a single pa-
rameter. Either has kind * => * => * because it accepts two
parameters, and so on.
In Scala we declare type constructors using underscores but refer to

them without:
// Declare F using underscores:

def myMethod[F[_]] = {
// Refer to F without underscores:

val functor = Functor.apply[F]
// ...
}
This is analogous to specifying a func ons parameters in its deni on

and omi ng them when referring to it:
// Declare f specifying parameters:

val f = (x: Int) => x * 2
// Refer to f without parameters:

val f2 = f andThen f
Armed with this knowledge of type constructors, we can see that the
Cats deni on of Functor allows us to create instances for any single-
parameter type constructor, such as List, Option, or Future.
3.5. FUNCTORS IN CATS 63
Language feature imports
Higher kinded types are considered an advanced language feature

in Scala. Wherever we declare a type constructor with A[_] syn-
tax, we need to import the feature to suppress compiler warn-
ings:
3.5 Functors in Cats
Lets look at the implementa on of functors in Cats. Well follow the

usual pa ern of looking at the three main aspects of the implementa-
on: the type class, the instances, and the interface.
3.5.1 The Functor Type Class
The functor type class is cats.Functor. We obtain instances using

the standard Functor.apply method on the companion object. As
usual, default instances are arranged by type in the cats.instances
package:
import cats.Functor
import cats.instances.list._
val list1 = List(1, 2, 3)

// list1: List[Int] = List(1, 2, 3)
val list2 = Functor[List].map(list1)(_ * 2)

// list2: List[Int] = List(2, 4, 6)
val option1 = Option(123)

// option1: Option[Int] = Some(123)
val option2 = Functor[Option].map(option1)(_.toString)

// option2: Option[String] = Some(123)
Functor also provides the lift method, which converts a func on of

type A => B to one that operates over a functor and has type F[A] =>
F[B]:
val func = (x: Int) => x + 1

// func: Int => Int = <function1>
val lifted = Functor[Option].lift(func)

// lifted: Option[Int] => Option[Int] = cats.
Functor$$Lambda$28362/1686307543@514a39b6
lifted(Option(1))
3.5.2 Functor Syntax
The main method provided by the syntax for Functor is map. Its dif-
cult to demonstrate this with Options and Lists as they have their
own built-in map opera ons. If there is a built-in method it will always
be called in preference to an extension method. Instead we will use
func ons as our example:
val func1 = (a: Int) => a + 1

// func1: Int => Int = <function1>
val func2 = (a: Int) => a * 2

// func2: Int => Int = <function1>
val func3 = func1.map(func2)

3.5. FUNCTORS IN CATS 65
// func3: Int => Int = scala.runtime.

AbstractFunction1$$Lambda$9405/272413009@5f430fe0
func3(123)
// res1: Int = 248
Other methods are available but we wont discuss them here.

Functors are more important to us as building blocks for later
abstrac ons than they are as a tool for direct use.
3.5.3 Instances for Custom Types
We can dene a functor simply by dening its map method. Heres an

example of a Functor for Option, even though such a thing already
exists in cats.instances:
implicit val optionFunctor = new Functor[Option] {

def map[A, B](value: Option[A])(func: A => B): Option[B] =
value.map(func)
}
The implementa on is trivialwe simply call Option's map method.
Some mes we need to inject dependencies into our instances. For

example, if we had to dene a custom Functor for Future, we would
need to account for the implicit ExecutionContext parameter on
future.map. We cant add extra parameters to functor.map so we
have to account for the dependency when we create the instance:
import scala.concurrent.{Future, ExecutionContext}
implicit def futureFunctor(implicit ec: ExecutionContext) =

new Functor[Future] {
def map[A, B](value: Future[A])(func: A => B): Future[B] =
value.map(func)
}
3.5.4 Exercise: Branching out with Functors
Write a Functor for the following binary tree data type. Verify that the
code works as expected on instances of Branch and Leaf:
sealed trait Tree[+A]

final case class Branch[A](left: Tree[A], right: Tree[A])
extends Tree[A]
final case class Leaf[A](value: A) extends Tree[A]
See the solu on
3.6 Contravariant and Invariant Functors
We can think of map as appending a transforma on to a chain. We

start with an F[A], run it through a func on A => B, and end up with
an F[B]. We can extend the chain further by mapping again: run the
F[B] through a func on B => C and end up with an F[C]:
Option(1).map(_ + 2).map(_ * 3).map(_ + 100)

Were now going to look at two other type classes, one that represents
prepending opera ons to a chain, and one that represents building a
bidirec onal chain of opera ons.
One great use case for these new type classes is building libraries that
transform, read, and write values. The content es in ghtly to the
JSON codec case study later in the book.
3.6. CONTRAVARIANT AND INVARIANT FUNCTORS 67
3.6.1 Contravariant functors and the contramap method
The rst of our type classes, the contravariant functor, provides an oper-
a on called contramap that represents prepending a transforma on
to a chain. This is illustrated in Figure 3.6.
contramap
F[B] A => B F[A]
Figure 3.6: Type chart: the contramap method
Well talk about contramap itself directly for now, bringing in the type
class in a moment.
The contramap method only makes sense for certain data types. For
example, we cant dene contramap for an Option because there is no
way of feeding a value in an Option[B] backwards through a func on
A => B.
contramap starts to make sense when we have a data types that repre-
sent tranforma ons. For example, consider the Printable type class
we discussed in Chapter 2:
trait Printable[A] {
def format(value: A): String
}
A Printable[A] represents a transforma on from A to String. We

can dene a contramap method that prepends a transforma on from
another type B:
def contramap[B](func: B => A): Printable[B] =

???
}
def format[A](value: A)(implicit p: Printable[A]): String =

p.format(value)
This says that if A is Printable, and we can transform B into A, then B

is also Printable.
3.6.1.1 Exercise: Showing o with Contramap
Implement the contramap method for Printable above.

See the solu on
Lets dene some basic instances of Printable for String and
Boolean:
implicit val stringPrintable =

new Printable[String] {
def format(value: String): String =
"\"" + value + "\""
}
implicit val booleanPrintable =

new Printable[Boolean] {
def format(value: Boolean): String =
if(value) "yes" else "no"
}
format("hello")
// res4: String = "hello"
format(true)
// res5: String = yes
Dene an instance of Printable that prints the value from this case
class:
final case class Box[A](value: A)
Rather than wri ng out the complete deni on from scratch (new
Printable[Box] etc), create your instance using the contramap
method of one of the instances above.
See the solu on
Your instance should work as follows:
format(Box("hello world"))
// res6: String = "hello world"
format(Box(true))
// res7: String = yes
If we dont have a Printable for the contents of the Box, calls to

format should fail to compile:
format(Box(123))
parameter p: Printable[Box[Int]]
// format(Box(123))
// ^
3.6.2 Invariant functors and the imap method
The second of our type classes, the invariant functor, provides a method
called imap that is informally equivalent to a combina on of map and
contramap. We can demonstrate this by extending Printable to pro-
duce a typeclass for encoding and decoding to a String:
trait Codec[A] {
def encode(value: A): String
def decode(value: String): Option[A]
def imap[B](dec: A => B, enc: B => A): Codec[B] =

???
}
def encode[A](value: A)(implicit c: Codec[A]): String =

c.encode(value)
def decode[A](value: String)(implicit c: Codec[A]): Option[A] =

c.decode(value)
The type chart for imap is showin in Figure 3.7.
imap
,
F[A] A => B B => A F[B]
Figure 3.7: Type chart: the imap method
3.6.2.1 Transforma ve Thinking with Imap
Implement the imap method for Codec above.
See the solu on
Heres an example Codec represen ng parsing and serializing Ints:
See the solu on
Demonstrate your imap method works by crea ng a Codec for conver-

sions between Strings and Boxes:
case class Box[A](value: A)
See the solu on

Your instance should work as follows:
encode(Box(123))
// res13: String = 123
decode[Box[Int]]("123")
// res14: Option[Box[Int]] = Some(Box(123))
3.6.3 Whats With the Name?
Whats the rela onship between contravariance, invariance, and covari-

ance as we usually understand them in Scala, and the names for the
functors above?
The usual meaning of these terms in Scala relates to subtypes. We say
that B is a subtype of A if we can use B anywhere we want an A. Put an-
other way, we can convert A into B and our program keeps on working.
Co- and contravariance usually arises in Scala when working with type
constructors like List and Option. If we declare a type constructor F,
and we want F[B] to be a subtype of F[A] when B is a subtype of A,
we declare the type parameter to be covariant.
trait F[+A] // A is covariant
If B is a subtype of A, and we want F[A] to be a subtype of F[B], then

we declare F to have a contravariant type parameter.
trait F[-A] // A is contravariant
Co- and contravariant functors capture the same principle without the
limita ons of subtyping. As we said above subtyping can be viewed as
a conversion. B is a subtype of A if we can convert A to B. In other words

there exists a func on A => B. A covariant functor, which is what the
standard Functor is, captures exactly this. If F is a (covariant) functor,
whenever we have a F[A] and a conversion A => B we have a F[B]. A
contravariant functor captures the case in the opposite direc on. If F
is a contravariant functor, whenever we have a F[A] and a conversion
B => A we have a F[B].
3.7 Contravariant and Invariant in Cats
Catss Contravariant and Invariant type classes are slightly dier-

ent to its other type classes: they live under cats.functor instead of
cats. Heres a simplied version of the code:
trait Invariant[F[_]] {
def imap[A, B](fa: F[A])(f: A => B)(g: B => A): F[B]
}
trait Contravariant[F[_]] extends Invariant[F] {

def contramap[A, B](fa: F[A])(f: B => A): F[B]
def imap[A, B](fa: F[A])(f: A => B)(fi: B => A): F[B] =

contramap(fa)(fi)
}
trait Functor[F[_]] extends Invariant[F] {

def map[A, B](fa: F[A])(f: A => B): F[B]
def imap[A, B](fa: F[A])(f: A => B)(fi: B => A): F[B] =

map(fa)(f)
}
Cats treats Functor and Contravariant as specialisa ons of

Invariant where one side of the bidirec onal transforma on is
3.7. CONTRAVARIANT AND INVARIANT IN CATS 73
ignored. Cats uses this to provide opera ons that work with any of
the three types of functor.
3.7.1 Contravariant in Cats
We can summon instances of Contravariant using the Contravariant.apply

method. Cats provides instances for data types that consume parame-
ters, including Eq, Show, Writer, WriterT, and Function1:
import cats.Show
import cats.functor.Contravariant
val showString = Show[String]
val showSymbol = Contravariant[Show].

contramap(showString)((sym: Symbol) => s"'${sym.name}")
showSymbol.show('dave)
// res2: String = 'dave
More conveniently, we can use cats.syntax.contravariant, which

provides a contramap extension method:
import cats.syntax.contravariant._
val div2: Int => Double = _ / 2.0

val add1: Int => Int = _ + 1
div2.contramap(add1)(2)
One example is the tupled method provided by the cartesian builder syntax dis-
cussed in Chapter 6.
3.7.1.1 Invariant in Cats
Cats provides instances of Invariant for Semigroup and Monoid. It

also provides an imap extension method via the cats.syntax.invariant
import. Imagine we have a semigroup for a well known type, for exam-
ple Semigroup[String], and we want to convert it to another type
like Semigroup[Symbol]. To do this we need two func ons: one to
convert the Symbol parameters to Strings, and one to convert the
result of the String append back to a Symbol:
import cats.instances.string._ // semigroup for String
import cats.syntax.invariant._ // imap extension method
implicit val symbolSemigroup: Semigroup[Symbol] =

Semigroup[String].imap(Symbol.apply)(_.name)
'a |+| 'few |+| 'words

// res7: Symbol = 'afewwords
3.8 Summary
We covered three types of functor in this chapter: regular covariant

Functors with their map method, as well as Contravariant functors
with their contramap methods, and Invariant functors with their
imap methods.
Regular Functors are by far the most common of these type classes,
but even then is rare to use them on their own. They form the build-
ing block of several more interes ng abstrac ons that we use all the
me. In the following chapters we will look at two of these abstrac-
ons: Monads and Applicatives.
3.8. SUMMARY 75
The map method for collec on types is important because each element
in a collec on can be transformed independently of the rest. This al-
lows us to parallelise or distribute transforma ons on large collec ons,
a technique leveraged heavily in map reduce frameworks like Hadoop.
We will inves gate this approach in more detail in the Pygmy Hadoop
case study later in the book.
The Contravariant and Invariant type classes are more situa onal.
We wont be doing much more work with them, although we will revisit
them to discuss Cartesians, and for the JSON Codec case study later
in the book.
Chapter 4
Monads
Monads are one of the most common abstrac ons in Scala. Many Scala
programmers quickly become intui vely familiar with monads, even if
we dont know them by name.
Informally, a monad is anything with a flatMap method. All of the func-

tors we saw in the last chapter are also monads, including Option, List,
Either, and Future. We even have special syntax to support monads:
for comprehensions. However, despite the ubiquity of the concept, the
Scala standard library lacks a concrete type to encompass things that
can be atMapped. This is one of the benets that Cats brings us.
In this chapter we will take a deep dive into monads. We will start by
mo va ng them with a few examples. Well proceed to their formal
deni on and their implementa on in Cats. Finally, well tour some
interes ng monads that you may not have heard of, providing introduc-
ons and examples of their use.
77
78 CHAPTER 4. MONADS
4.1 What is a Monad?
This is the ques on that has been posed in a thousand blog posts, with
explana ons and analogies involving concepts as diverse as cats, mexi-
can food, and monoids in the category of endofunctors (whatever they
are). Were going to solve the problem of explaining monads once and
for all by sta ng very simply:
A monad is a control mechanism for sequencing computa-

ons.
That was easy! Problem solved, right? Ok, maybe we need some more
discussion
Informally, the most important feature of a monad is its flatMap
method, which allows us to specify what happens next. This is what
we mean by sequencing computa ons. A monad allows us to specify
a sequence of opera ons that happen one a er another. We specify
the applica on-specic part of the computa on as a func on param-
eter, and flatMap runs our func on and takes care of some kind of
complica on (conven onally referred to as an eect). Lets ground
things by looking at some examples.
Op ons
Option is a monad that allows us to sequence computa ons that may

or may not return values. Here are some examples:
def parseInt(str: String): Option[Int] =

scala.util.Try(str.toInt).toOption
def divide(a: Int, b: Int): Option[Int] =

if(b == 0) None else Some(a / b)
Each of these computa ons may fail, as indicated by their Option re-
turn types. The flatMap method on Option allows us to sequence
4.1. WHAT IS A MONAD? 79
these opera ons without having to constantly check whether they re-
turn Some or None:
def stringDivideBy(aStr: String, bStr: String): Option[Int] =

parseInt(aStr).flatMap { aNum =>
parseInt(bStr).flatMap { bNum =>
divide(aNum, bNum)
}
}
We know the seman cs well:
the rst call to parseInt returns a None or a Some;

if it returns a Some, the flatMap method calls our func on and
passes us aNum;
the second call to parseInt returns a None or a Some;
if it returns a Some, the flatMap method calls our func on and
passes us bNum;
the call to divide returns a None or a Some, which is our result.
At each step, flatMap chooses whether to call our func on, and our
func on generates the next computa on in the sequence. This is
shown in Figure 4.1.
flatMap
Option[A] A => Option[B] Option[B]
Figure 4.1: Type chart: atMap for Op on
The result of the computa on is an Option, allowing us to call flatMap

again and so the process con nues. This results in the fail-fast error
handling behaviour that we know and love, where a None at any step
results in a None overall:
stringDivideBy("6", "2")
stringDivideBy("6", "0")
stringDivideBy("6", "foo")
stringDivideBy("bar", "2")
Every monad is also a functor (see below for proof), so we can rely on
both flatMap and map to sequence computa ons that do and and dont
introduce a new monad. Plus, if we have both flatMap and map we can
use for comprehensions to clarify the sequencing behaviour:
def stringDivideBy(aStr: String, bStr: String): Option[Int] =

for {
aNum <- parseInt(aStr)
bNum <- parseInt(bStr)
ans <- divide(aNum, bNum)
} yield ans
Lists
When we rst encounter flatMap as budding Scala developers, we
tend to think of it as a pa ern for itera ng over Lists. This is rein-
forced by the syntax of for comprehensions, which look very much like
impera ve for loops:
def numbersBetween(min: Int, max: Int): List[Int] =

(min to max).toList
for {
x <- numbersBetween(1, 3)
y <- numbersBetween(4, 5)
} yield (x, y)
// res5: List[(Int, Int)] = List((1,4), (1,5), (2,4), (2,5),

(3,4), (3,5))
However, there is another mental model we can apply that highlights

the monadic behaviour of List. If we think of func ons that return
Lists as func ons with mul ple return values, flatMap becomes a
construct that calculates results from permuta ons and combina ons
of intermediate values.
For example, in the for comprehension above, there are three possi-
ble values of x and two possible values of y. This means there are six
possible values of the overall expression. flatMap is genera ng these
combina ons from our code, which simply says get x from here and y
from over there.
The type chart in Figure 4.2 illustrates this behaviour: although the re-
sult of flatMap (List[B]) is the same type as the result of the user-
supplied func on, the end result is actually a larger list created from
combina ons of intermediate As and Bs:
flatMap
List[A] A => List[B] List[B]
Figure 4.2: Type chart: atMap for List
Futures
Future is a monad that allows us to sequence computa ons without
worrying that they are asynchronous:
import scala.concurrent.Future
def getTrafficFromHost(hostname: String): Future[Int] =

??? // grab traffic information using a network client
def getTrafficFromAllHosts: Future[Int] =

for {
traffic1 <- getTrafficFromHost("host1")
} yield traffic1 + traffic2 + traffic3
Again, we specify the code to run at each step, and flatMap takes care
of all the horrifying underlying complexi es of thread pools and sched-
ulers.
If youve made extensive use of Scalas Futures, youll know that the
code above is fetching trac from each server in sequence. This be-
comes clearer if we expand out the for comprehension to show the
nested calls to flatMap:
def getTrafficFromAllHosts: Future[Int] =

getTrafficFromHost("host1").flatMap { traffic1 =>
getTrafficFromHost("host2").flatMap { traffic2 =>
getTrafficFromHost("host3").map { traffic3 =>
traffic1 + traffic2 + traffic3
}
}
}
Each Future in our sequence is created by a func on that receives the

result from a previous Future. In other words, each step in our compu-
ta on can only start once the previous step is nished. This is born out
by the type chart for flatMap in Figure 4.3, which shows the func on
parameter of type A => Future[B]:
In other words, the monadic behaviour of Future allows us to sequence

asynchronous computa ons one a er the other. We can run Futures
in parallel, but that is another story and shall be told another me. Mon-
ads are truly all about sequencing.
flatMap
Future[A] A => Future[B] Future[B]
Figure 4.3: Type chart: atMap for Future
4.1.1 Monad Deni on and Laws
While we have only talked about the flatMap method above, the
monad behaviour is formally captured in two opera ons:
an opera on pure with type A => F[A];

an opera on flatMap with type (F[A], A => F[B]) =>
F[B].
The pure opera on abstracts over constructors, providing a way to cre-

ate a new monadic context from a plain value. flatMap provides the
sequencing step we have already discussed, extrac ng the value from
a context and using the supplied func on to generate the next context
in the sequence. Here is a simplied version of the Monad type class in
Cats:
trait Monad[F[_]] {
def pure[A](value: A): F[A]
def flatMap[A, B](value: F[A])(func: A => F[B]): F[B]

}
In some languages and libraries, notably Haskell and Scalaz, flatMap is referred
to as bind. This is purely a dierence in terminology. Well use the term flatMap for
compa bility with Cats and the Scala standard library.
Importantly, the pure and flatMap methods must obey three laws:
Le iden ty: calling pure then transforming the result with a func on
f is the same as simply calling f:
pure(a).flatMap(f) == f(a)
Right iden ty: passing pure to flatMap is the same as doing nothing:
m.flatMap(pure) == m
Associa vity: flatMapping over two func ons f and g is the same as
flatMapping over f and then flatMapping over g:
m.flatMap(f).flatMap(g) == m.flatMap(x => f(x).flatMap(g))
4.1.2 Exercise: Ge ng Func-y
Every monad is also a functor. If flatMap represents sequencing a

computa on that introduces a new monadic context, map represents
sequencing a computa on that does not. We can dene map in the
same way for every monad using the exis ng methods, flatMap and
pure:
trait Monad[F[_]] {
def pure[A](a: A): F[A]

}
Try dening map yourself now.
See the solu on

4.2. MONADS IN CATS 85
4.2 Monads in Cats
Its me to give monads our standard Cats treatment. As usual well

look at the type class, instances, and syntax.
4.2.1 The Monad Type Class
The monad type class is cats.Monad. Monad extends two other

type classes: FlatMap, which provides the flatMap method, and
Applicative, which provides pure. Applicative also extends
Functor so every Monad also has a map method. Well discuss
Applicatives in a later chapter.
Here are some examples using pure and flatMap, and map directly:
import cats.Monad
val opt1 = Monad[Option].pure(3)

// opt1: Option[Int] = Some(3)
val opt2 = Monad[Option].flatMap(opt1)(a => Some(a + 2))

val opt3 = Monad[Option].map(opt2)(a => 100 * a)

val list1 = Monad[List].pure(3)

// list1: List[Int] = List(3)
val list2 = Monad[List].

flatMap(List(1, 2, 3))(x => List(x, x*10))
// list2: List[Int] = List(1, 10, 2, 20, 3, 30)
val list3 = Monad[List].map(list2)(_ + 123)

// list3: List[Int] = List(124, 133, 125, 143, 126, 153)
Monad provides many other methods as well, including all of the meth-
ods from Functor. See the scaladoc for more informa on.
Cats provides instances for all the monads in the standard library
(Option, List, Vector and so on) via cats.instances:
Monad[Option].flatMap(Option(1))(x => Option(x*2))

Monad[List].flatMap(List(1, 2, 3))(x => List(x, x*10))

// res1: List[Int] = List(1, 10, 2, 20, 3, 30)
import cats.instances.vector._
Monad[Vector].flatMap(Vector(1, 2, 3))(x => Vector(x, x*10))

// res2: Vector[Int] = Vector(1, 10, 2, 20, 3, 30)
The Monad for Future doesnt accept implicit ExecutionContext pa-

rameters to pure and flatMap like Future itself does (it cant because
the parameters arent in the deni ons in the Monad trait). To work
around this, Cats requires us to have an ExecutionContext in scope
when we summon the Monad for Future:
import cats.instances.future._
import scala.concurrent._
4.2. MONADS IN CATS 87
val fm = Monad[Future]
parameter instance: cats.Monad[scala.concurrent.Future]
// val fm = Monad[Future]
// ^
val fm = Monad[Future]
// fm: cats.Monad[scala.concurrent.Future] = cats.instances.
FutureInstances$$anon$1@1c54d4ad
The Monad instances uses the captured ExecutionContext for subse-

quent calls to pure and flatMap:
Await.result(
fm.flatMap(fm.pure(1)) { x =>
fm.pure(x + 2)
},
1.second
)
// res3: Int = 3
In addi on to the above, Cats provides a host of new monads that we

dont have in the standard library. Well familiarise ourselves with the
most important of these in a moment.
4.2.3 Monad Syntax
The syntax for monads comes from three places:
cats.syntax.flatMap provides syntax for flatMap;

cats.syntax.functor provides syntax for map;
cats.syntax.applicative provides syntax for pure.
In prac ce its o en easier to import everything in one go from

cats.implicits. However, well use the individual imports here for
clarity.
We can use pure to construct instances of a monad. Well o en need
to specify the type parameter to disambiguate the par clar instance we
want.
import cats.syntax.applicative._
1.pure[Option]
1.pure[List]
// res5: List[Int] = List(1)
Its dicult to demonstrate the flatMap and map methods directly on

Scala monads like Option and List, because they dene their own ex-
plicit versions of those methods. Instead well write a generic func-
on that performs a calcula on on parameters that come wrapped in a
monad of the users choice:
import cats.Monad
import cats.syntax.flatMap._
def sumSquare[M[_] : Monad](a: M[Int], b: M[Int]): M[Int] =

a.flatMap(x => b.map(y => x*x + y*y))
sumSquare(Option(3), Option(4))
4.3. THE IDENTITY MONAD 89
sumSquare(List(1, 2, 3), List(4, 5))

// res9: List[Int] = List(17, 26, 20, 29, 25, 34)
We can rewrite this code using for comprehensions. The Scala compiler
will do the right thing by rewri ng our comprehension in terms of
flatMap and map and inser ng the correct implicit conversions to use
our Monad:
for {
x <- a
y <- b
} yield x*x + y*y
sumSquare(Option(3), Option(4))
sumSquare(List(1, 2, 3), List(4, 5))

// res11: List[Int] = List(17, 26, 20, 29, 25, 34)
Thats more or less everything we need to know about the generali-

es of monads in Cats. Now lets take a look at some useful monad
instances.
4.3 The Iden ty Monad
In the previous sec on we demonstrated Cats flatMap and map syntax

by wri ng a method that abstracted over dierent monads:
import cats.Monad

for {
x <- a
y <- b
} yield x*x + y*y
This method works well on Options and Lists but we cant call it pass-
ing in plain values:
sumSquare(3, 4)
// <console>:22: error: no type parameters for method sumSquare:
(a: M[Int], b: M[Int])(implicit evidence$1: cats.Monad[M])M
[Int] exist so that it can be applied to arguments (Int, Int
)
// --- because ---
// argument expression's type is not compatible with formal
parameter type;
// found : Int
// required: ?M[Int]
// sumSquare(3, 4)
// ^
// found : Int(3)
// required: M[Int]
// sumSquare(3, 4)
// ^
// found : Int(4)
// required: M[Int]
// sumSquare(3, 4)
// ^
It would be incredibly useful if we could use sumSquare with a com-

bina on of monadic and non-monadic parameters. This would allow
us to abstract over monadic and non-monadic code. Fortunately, Cats
provides the Id type to bridge the gap:
import cats.Id
4.3. THE IDENTITY MONAD 91
sumSquare(3 : Id[Int], 4 : Id[Int])

// res2: cats.Id[Int] = 25
Now we can call our monadic method using plain values. However, the
exact seman cs are dicult to understand. We cast the parameters to
sumSquare as Id[Int] and received an Int back as a result!
Whats going on? Here is the deni on of Id to explain:

package cats
type Id[A] = A
Id is actually a type alias that turns an atomic type into a single-

parameter type constructor. We can cast any value of any type to a
corresponding Id:
"Dave" : Id[String]
// res3: cats.Id[String] = Dave
123 : Id[Int]
List(1, 2, 3) : Id[List[Int]]
// res5: cats.Id[List[Int]] = List(1, 2, 3)
Cats provides instances of various type classes for Id, including

Functor and Monad. These let us call map, flatMap and so on on plain
values:
val a = Monad[Id].pure(3)
// a: cats.Id[Int] = 3
val b = Monad[Id].flatMap(a)(_ + 1)
// b: cats.Id[Int] = 4
for {
x <- a
y <- b
} yield x + y
The main use for Id is to write generic methods like sumSquare

that operate on monadic and non-monadic data types. For example,
we can run code asynchronously in produc on using Future and
synchronously in test using Id:
// In production:
Await.result(sumSquare(Future(3), Future(4)), 1.second)
// res8: Int = 25
// In test:
sumSquare(a, b)
4.3.1 Exercise: Monadic Secret Iden es
Implement pure, map, and flatMap for Id! What interes ng discover-
ies do you uncover about the implementa on?
See the solu on
4.4 Either
Lets look at another useful monadic data type. The Scala standard li-
brary has a type Either. In Scala 2.11 and earlier, Either wasnt tech-
4.4. EITHER 93
nically a monad because it didnt have map and flatMap methods. In

Scala 2.12, however, Either became right biased.
4.4.1 Le and Right Bias
In Scala 2.11, Either was unbiased. It had no map or flatMap method:
// Scala 2.11 example
Right(123).flatMap(x => Right(x * 2))

// <console>:12: error: value flatMap is not a member
// of scala.util.Right[Nothing, Int]
// Right(123).flatMap(x => Right(x * 2))
// ^
Instead of calling map or flatMap directly, we had to decide which side

we wanted to be the correct side by taking a le - or right-projec on:
// Valid in Scala 2.11 and Scala 2.12
val either1: Either[String, Int] = Right(123)

// either1: Either[String,Int] = Right(123)
val either2: Either[String, Int] = Right(321)

// either2: Either[String,Int] = Right(321)
either1.right.flatMap(x => Right(x * 2))

// res2: scala.util.Either[String,Int] = Right(246)
either2.left.flatMap(x => Left(x + "!!!"))

This made the Scala 2.11 version of Either incovenient to use as a

monad. If we wanted to use for comprehensions, for example, we had
to insert calls to .right in every generator clause:
for {
a <- either1.right
b <- either2.right
} yield a + b
In Scala 2.12, Either was redesigned. The modern Either makes the
decision that the right side is always the success case and thus supports
map and flatMap directly. This turns Either into a monad and makes
working with it much more pleasant:
for {
a <- either1
b <- either2
} yield a + b
4.4.2 Crea ng Instances
In addi on to crea ng instances of Left and Right directly, we

can also import the asLeft and asRight extension methods from
cats.syntax.either:
import cats.syntax.either._
val a = 3.asRight[String]
// a: Either[String,Int] = Right(3)
val b = 4.asRight[String]
// b: Either[String,Int] = Right(4)
for {
x <- a
y <- b
4.4. EITHER 95
} yield x*x + y*y

Smart Constructors and Avoiding Over-Narrowing
The asLeft and asRight methods have advantages over

Left.apply and Right.apply in terms of type inference. The
following code provides an example:
def countPositive(nums: List[Int]) =

nums.foldLeft(Right(0)) { (accumulator, num) =>
if(num > 0) {
accumulator.map(_ + 1)
} else {
Left("Negative. Stopping!")
}
}
// found : scala.util.Either[Nothing,Int]
// required: scala.util.Right[Nothing,Int]
// accumulator.map(_ + 1)
// ^
// found : scala.util.Left[String,Nothing]
// required: scala.util.Right[Nothing,Int]
// Left("Negative. Stopping!")
// ^
There are two problems here, both arising because the compiler
chooses the type of accumulator based on the rst parameter
list to foldRight:
1. the type of the accumulator ends up being Right instead

of Either;
2. we didnt specify type parameters for Right.apply so the

compiler infers the le parameter as Nothing.
Switching to asRight avoids both of these problems. It as a re-

turn type of Either, and allows us to completely specify the type
with only one type parameter:
def countPositive(nums: List[Int]) =

nums.foldLeft(0.asRight[String]) { (accumulator, num) =>
if(num > 0) {
accumulator.map(_ + 1)
} else {
Left("Negative. Stopping!")
}
}
countPositive(List(1, 2, 3))
// res7: Either[String,Int] = Right(3)
countPositive(List(1, -2, 3))

// res8: Either[String,Int] = Left(Negative. Stopping!)
In addi on to asLeft and asRight, cats.syntax.either also adds

some useful extension methods to the Either companion object.
The catchOnly and catchNonFatal methods are for capturing
Exceptions in instances of Either:
Either.catchOnly[NumberFormatException]("foo".toInt)
Either.catchNonFatal(sys.error("Badness"))
There are also methods for crea ng an Either from other data types:
Either.fromTry(scala.util.Try("foo".toInt))
Either.fromOption[String, Int](None, "Badness")
4.4. EITHER 97
4.4.3 Transforming Eithers
cats.syntax.either adds a number of useful methods to Either.

We can use orElse and getOrElse to extract values from the right
side: the right value or return a default:
"Error".asLeft[Int].getOrElse(0)
// res9: Int = 0
"Error".asLeft[Int].orElse(2.asRight[String])
The ensure method allows us to check whether a wrapped value sa s-

es a predicate:
-1.asRight[String].ensure("Must be non-negative!")(_ > 0)

// res11: Either[String,Int] = Left(Must be non-negative!)
The recover and recoverWith methods provide similar error handling

to their namesakes on Future:
"error".asLeft[String] recover {
case str: String =>
"Recovered from " + str
}
// res12: Either[String,String] = Right(Recovered from error)
"error".asLeft[String] recoverWith {
case str: String =>
Right("Recovered from " + str)
}
// res13: Either[String,String] = Right(Recovered from error)
There are leftMap and bimap methods to complement map:

"foo".asLeft[Int].leftMap(_.reverse)
// res14: Either[String,Int] = Left(oof)
6.asRight[String].bimap(_.reverse, _ * 7)
"bar".asLeft[Int].bimap(_.reverse, _ * 7)
// res16: Either[String,Int] = Left(rab)
The swap method lets us exchange le for right:
123.asRight[String]
123.asRight[String].swap
// res18: scala.util.Either[Int,String] = Left(123)
Finally, Cats adds a host of conversion methods: toOption, toList,

toTry, toValidated, and so on.
4.4.4 Fail-Fast Error Handling
Either is typically used to implement fail-fast error handling. We se-

quence a number of computa ons using flatMap, and if one computa-
on fails the remaining computa ons are not run:
for {
a <- 1.asRight[String]
b <- 0.asRight[String]
c <- if(b == 0) "DIV0".asLeft[Int] else (a / b).asRight[String
]
} yield c * 100
// res19: scala.util.Either[String,Int] = Left(DIV0)
4.4. EITHER 99
4.4.5 Represen ng Errors
When using Either for error handling, we need to determine what type
we want to use to represent errors. We could use Throwable for this
as follows:
type Result[A] = Either[Throwable, A]
This gives us similar seman cs to scala.util.Try. The problem, how-

ever, is that Throwable is an extremely broad supertype. We have (al-
most) no idea about what type of error occurred.
Another approach is to dene an algebraic data type to represent the

errors that can occur:
sealed trait LoginError extends Product with Serializable
final case class UserNotFound(

username: String
) extends LoginError
final case class PasswordIncorrect(

username: String
) extends LoginError
case object UnexpectedError extends LoginError
case class User(username: String, password: String)
type LoginResult = Either[LoginError, User]
This approach solves the problems we saw with Throwable. It gives us

a xed set of expected error types and a catch-all for anything else that
we didnt expect. We also get the safety of exhaus vity checking on
any pa ern matching we do:
// Choose error-handling behaviour based on type:

def handleError(error: LoginError): Unit =
error match {
case UserNotFound(u) =>
println(s"User not found: $u")
case PasswordIncorrect(u) =>

println(s"Password incorrect: $u")
case UnexpectedError =>

println(s"Unexpected error")
}
val result1: LoginResult = User("dave", "passw0rd").asRight

// result1: LoginResult = Right(User(dave,passw0rd))
val result2: LoginResult = UserNotFound("dave").asLeft

// result2: LoginResult = Left(UserNotFound(dave))
result1.fold(handleError, println)
// User(dave,passw0rd)
result2.fold(handleError, println)
// User not found: dave
4.4.6 Exercise: What is Best?
Is the error handling strategy in the previous exercises well suited for
all purposes? What other features might we want from error handling?
See the solu on
4.5 The Eval Monad
cats.Eval is a monad that allows us to abstract over dierent models

of evalua on. We typically hear of two such models: eager and lazy.
4.5. THE EVAL MONAD 101
Eval throws in a further dis nc on of memoized and unmemoized to

create three models of evalua on:
nowevaluated once immediately (equivalent to val);

laterevaluated once when the value is rst needed (equivalent
to lazy val);
alwaysevaluated every me the value is needed (equivalent to
def).
4.5.1 Eager, lazy, memoized, oh my!
What do these terms mean?

Eager computa ons happen immediately, whereas lazy computa ons
happen on access.
For example, Scala vals are eager deni ons. We can see this using
a computa on with a visible side-eect. In the following example, the
code to compute the value of x happens eagerly at the deni on site.
Accessing x simply recalls the stored value without re-running the code.
val x = {
println("Computing X")
1 + 1
}
// Computing X
// x: Int = 2
x // first access
// res0: Int = 2
x // second access
// res1: Int = 2
By contrast, defs are lazy and not memoized. The code to compute y
below is not run un l we access it (lazy), and is re-run on every access
(not memoized):
def y = {
println("Computing Y")
1 + 1
}
// y: Int
y // first access
// Computing Y
// res2: Int = 2
y // second access
// Computing Y
// res3: Int = 2
Last but not least, lazy vals are lazy and memoized. The code to
compute z below is not run un l we access it for the rst me (lazy). The
result is then cached and re-used on subsequent accesses (memoized):
lazy val z = {
println("Computing Z")
1 + 1
}
// z: Int = <lazy>
z // first access
// Computing Z
// res4: Int = 2
z // second access
// res5: Int = 2
4.5.2 Evals models of evalua on
Eval has three subtypes: Eval.Now, Eval.Later, and Eval.Always.

We construct these with three constructor methods, which create in-
stances of the three classes and return them typed as Eval:
import cats.Eval
// import cats.Eval
val now = Eval.now(1 + 2)

// now: cats.Eval[Int] = Now(3)
val later = Eval.later(3 + 4)

// later: cats.Eval[Int] = cats.Later@49373f5e
val always = Eval.always(5 + 6)

// always: cats.Eval[Int] = cats.Always@53dd8e4b
We can extract the result of an Eval using its value method:
now.value
// res6: Int = 3
later.value
// res7: Int = 7
always.value
// res8: Int = 11
Each type of Eval calculates its result using one of the evalua on mod-
els dened above. Eval.now captures a value right now. Its seman cs
are similar to a valeager and memoized:
val x = Eval.now {
println("Computing X")
1 + 1
}
// Computing X
// x: cats.Eval[Int] = Now(2)
x.value // first access

// res9: Int = 2
x.value // second access

// res10: Int = 2
Eval.always captures a lazy computa on, similar to a def:
val y = Eval.always {
println("Computing Y")
1 + 1
}
// y: cats.Eval[Int] = cats.Always@253eef84
y.value // first access

// Computing Y
// res11: Int = 2
y.value // second access

// Computing Y
// res12: Int = 2
Finally, Eval.later captures a lazy computa on and memoizes the re-

sult, similar to a lazy val:
val z = Eval.later {
println("Computing Z")
1 + 1
}
// z: cats.Eval[Int] = cats.Later@a81d4d5
z.value // first access

// Computing Z
// res13: Int = 2
z.value // second access

// res14: Int = 2
The three behaviours are summarized below:

Eager Lazy
Memoized val, Eval.now lazy val,

Eval.later
Not memoized N/A def, Eval.always
4.5.3 Eval as a Monad
Eval's map and flatMap methods add computa ons to a chain. This is
similar to the map and flatMap methods on scala.concurrent.Future,
except that the computa ons arent run un l we call value to obtain
a result:
val greeting = Eval.always {
println("Step 1")
"Hello"
}.map { str =>
println("Step 2")
str + " world"
}
// greeting: cats.Eval[String] = cats.Eval$$anon$8@3c88d726
greeting.value
// Step 1
// Step 2
// res15: String = Hello world
Note that, while the seman cs of the origina ng Eval instances are
maintained, mapping func ons are always called lazily on demand (def
seman cs):
val ans = for {

a <- Eval.now { println("Calculating A") ; 40 }
b <- Eval.always { println("Calculating B") ; 2 }
} yield {
println("Adding A and B")
a + b
}
// Calculating A
// ans: cats.Eval[Int] = cats.Eval$$anon$8@9284e40
ans.value // first access

// Calculating B
// Adding A and B
// res16: Int = 42
ans.value // second access

// Calculating B
// Adding A and B
// res17: Int = 42
We can use Eval's memoize method to memoize a chain of compu-

ta ons. Calcula ons before the call to memoize are cached, whereas
calcula ons a er the call retain their original seman cs:
val saying = Eval.always {

println("Step 1")
"The cat"
}.map { str =>
println("Step 2")
s"$str sat on"
}.memoize.map { str =>
println("Step 3")
s"$str the mat"
}
// saying: cats.Eval[String] = cats.Eval$$anon$8@397aee40
saying.value // first access

// Step 1
// Step 2
// Step 3
// res18: String = The cat sat on the mat
saying.value // second access

// Step 3
// res19: String = The cat sat on the mat
4.5.4 Trampolining and Eval.defer
One useful property of Eval is that its map and flatMap methods are
trampolined. This means we can nest calls to map and flatMap arbitrar-
ily without consuming stack frames. We call this property stack safety.
For example, consider this func on for calcula ng factorials:
def factorial(n: BigInt): BigInt =

if(n == 1) n else n * factorial(n - 1)
It is rela vely easy to make this method stack overow:
factorial(50000)
// java.lang.StackOverflowError
// ...
We can rewrite the method using Eval to make it stack safe:
def factorial(n: BigInt): Eval[BigInt] =

if(n == 1) Eval.now(n) else factorial(n - 1).map(_ * n)
factorial(50000).value
// java.lang.StackOverflowError
// ...
Oops! That didnt workour stack s ll blew up! This is because were
s ll making all the recursive calls to factorial before we start working
with Eval's map method. We can work around this using Eval.defer,
which takes an exis ng instance of Eval and defers its evalua on un l
later. defer is trampolined like Eval's map and flatMap methods, so
we can use it as a way to quickly make an exis ng opera on stack safe:
def factorial(n: BigInt): Eval[BigInt] =

if(n == 1) {
Eval.now(n)
} else {
Eval.defer(factorial(n - 1).map(_ * n))
}
factorial(50000).value
// res20: BigInt =
33473205095971448369154760940714864779127732238104548077301003219901680221
Eval is a useful tool to enforce stack when working on very large com-
puta ons and data structures. However, we must bear in mind that
trampolining is not free. It avoids consuming stack by crea ng a chain
of func on calls on the heap. There are s ll limits on how deeply we
can nest computa ons, but they are bounded by the size of the heap
rather than the stack.
4.5.5 Exercise: Safer Folding using Eval
The naive implementa on of foldRight below is not stack safe. Make

it so using Eval:
def foldRight[A, B](as: List[A], acc: B)(fn: (A, B) => B): B =

as match {
case head :: tail =>
fn(head, foldRight(tail, acc)(fn))
case Nil =>
acc
}
See the solu on

4.6. THE WRITER MONAD 109
4.6 The Writer Monad
cats.data.Writer is a monad that lets us carry a log along with a

computa on. We can use it to record messages, errors, or addi onal
data about a computa on, and extract the log with the nal result.
One common use for Writers is recording sequences of steps in
mul -threaded computa ons, where standard impera ve logging
techniques can result in interleaved messages from dierent contexts.
With Writer the log for the computa on is ed to the result, so we
can run concurrent computa ons without mixing logs.
Cats data types
Writer is the rst data type weve seen from the [cats.data]
package. This package provides numerous data types: instances
of various type classes that produce useful seman cs. Other ex-
amples from cats.data include the monad transformers that we
will see in the next chapter, and the Validated type that we will
see in Chapter 6.
4.6.1 Crea ng and Unpacking Writers
A Writer[W, A] carries two values: a log of type W and a result of type

A. We can create a Writer from values of each type as follows:
import cats.data.Writer
Writer(Vector(
"It was the best of times",
"It was the worst of times"
), 123)
// res0: cats.data.WriterT[cats.Id,scala.collection.immutable.
Vector[String],Int] = WriterT((Vector(It was the best of
times, It was the worst of times),123))
Weve used a Vector as the log in this example as it is a sequence struc-

ture with an ecient append opera on.
No ce that the type of the writer reported on the console
is actually WriterT[Id, Vector[String], Int] instead of
Writer[Vector[String], Int] as we might expect. In the spirit
of code reuse, Cats implements Writer in terms of another type,
WriterT. WriterT is an example of a new concept called a monad
tranformer, which we will cover in the next chapter.
Lets try to ignore this detail for now. Writer is a type alias for WriterT,
so we can read types like WriterT[Id, W, A] as Writer[W, A]:
type Writer[W, A] = WriterT[Id, W, A]
For convenience, Cats provides a way of crea ng Writers specifying

only the log or the result. If we only have a result we can use the stan-
dard pure syntax. To do this we must have a Monoid[W] in scope so
Cats knows how to produce an empty log:
import cats.syntax.applicative._ // `pure` method
type Logged[A] = Writer[Vector[String], A]
123.pure[Logged]
// res2: Logged[Int] = WriterT((Vector(),123))
If we have a log and no result, we can create a Writer[Unit] using the

tell syntax from cats.syntax.writer:
import cats.syntax.writer._
Vector("msg1", "msg2", "msg3").tell

// res3: cats.data.Writer[scala.collection.immutable.Vector[
String],Unit] = WriterT((Vector(msg1, msg2, msg3),()))
If we have both a result and a log, in addi on to using Writer.apply as

we did above we can use the writer syntax from cats.syntax.writer:
val a = Writer(Vector("msg1", "msg2", "msg3"), 123)

// a: cats.data.WriterT[cats.Id,scala.collection.immutable.
Vector[String],Int] = WriterT((Vector(msg1, msg2, msg3),123)
)
val b = 123.writer(Vector("msg1", "msg2", "msg3"))

// b: cats.data.Writer[scala.collection.immutable.Vector[String
],Int] = WriterT((Vector(msg1, msg2, msg3),123))
We can extract the result and log from a Writer using the value and
written methods respec vely:
a.value
a.written
// res5: cats.Id[scala.collection.immutable.Vector[String]] =
Vector(msg1, msg2, msg3)
or we can extract log and result at the same me using the run method:
val (log, result) = b.run
// log: scala.collection.immutable.Vector[String] = Vector(msg1,
msg2, msg3)
// result: Int = 123
4.6.2 Composing and Transforming Writers
The log in a Writer is preserved when we map or flatMap over it.

flatMap actually appends the logs from the source Writer and the re-
sult of the users sequencing func on. For this reason its good prac ce
to use a log type that has an ecient append and concatenate opera-
ons, such as a Vector:
val writer1 = for {
a <- 10.pure[Logged]
_ <- Vector("a", "b", "c").tell
b <- 32.writer(Vector("x", "y", "z"))
} yield a + b
// writer1: cats.data.WriterT[cats.Id,Vector[String],Int] =
WriterT((Vector(a, b, c, x, y, z),42))
writer1.run
// res6: cats.Id[(Vector[String], Int)] = (Vector(a, b, c, x, y,
z),42)
In addi on to transforming the result with map and flatMap, we can

transform the log in a Writer with the mapWritten method:
val writer2 = writer1.mapWritten(_.map(_.toUpperCase))
// writer2: cats.data.WriterT[cats.Id,scala.collection.immutable
.Vector[String],Int] = WriterT((Vector(A, B, C, X, Y, Z),42)
)
writer2.run
// res7: cats.Id[(scala.collection.immutable.Vector[String], Int
)] = (Vector(A, B, C, X, Y, Z),42)
We can tranform both log and result simultaneously using bimap or

mapBoth. bimap takes two func on parameters, one for the log and
one for the result. mapBoth takes a single func on that accepts two
parameters:
val writer3 = writer1.bimap(
log => log.map(_.toUpperCase),
result => result * 100
)
.Vector[String],Int] = WriterT((Vector(A, B, C, X, Y, Z)
,4200))
writer3.run
)] = (Vector(A, B, C, X, Y, Z),4200)
val writer4 = writer1.mapBoth { (log, result) =>

val log2 = log.map(_ + "!")
val result2 = result * 1000
(log2, result2)
}
.Vector[String],Int] = WriterT((Vector(a!, b!, c!, x!, y!, z
!),42000))
writer4.run
)] = (Vector(a!, b!, c!, x!, y!, z!),42000)
Finally, we can clear the log with the reset method and swap log and
result with the swap method:
val writer5 = writer1.reset

// writer5: cats.data.WriterT[cats.Id,Vector[String],Int] =
WriterT((Vector(),42))
writer5.run
// res10: cats.Id[(Vector[String], Int)] = (Vector(),42)
val writer6 = writer1.swap

// writer6: cats.data.WriterT[cats.Id,Int,Vector[String]] =
WriterT((42,Vector(a, b, c, x, y, z)))
writer6.run
// res11: cats.Id[(Int, Vector[String])] = (42,Vector(a, b, c, x
, y, z))
4.6.3 Exercise: Show Your Working
Writers are useful for logging opera ons in mul -threaded environ-
ments. Lets conrm this by compu ng (and logging) some factorials.
The factorial func on below computes a factorial, prin ng out the
intermediate steps in the calcula on as it runs. The slowly helper func-
on ensures this takes a while to run, even on the very small examples
we need in this book to t the output on the page:
def slowly[A](body: => A) =

try body finally Thread.sleep(100)
def factorial(n: Int): Int = {

val ans = slowly(if(n == 0) 1 else n * factorial(n - 1))
println(s"fact $n $ans")
ans
}
Heres the outputa sequence of monotonically increasing values:
factorial(5)
// fact 0 1
// fact 1 1
// fact 2 2
// fact 3 6
// fact 4 24
// fact 5 120
// res13: Int = 120
If we start several factorials in parallel, the log messages can become

interleaved on standard out. This makes it dicult to see which mes-
sages come from which computa on.
4.7. THE READER MONAD 115
Await.result(Future.sequence(Vector(
Future(factorial(3)),
Future(factorial(3))
)), 5.seconds)
// fact 0 1
// fact 0 1
// fact 1 1
// fact 1 1
// fact 2 2
// fact 2 2
// fact 3 6
// fact 3 6
// res14: scala.collection.immutable.Vector[Int] =
// Vector(120, 120)
Rewrite factorial so it captures the log messages in a Writer.

Demonstrate that this allows us to reliably separate the logs for
concurrent computa ons.
See the solu on
4.7 The Reader Monad
cats.data.Reader is a monad that allows us to compose opera ons

that depend on some input. Instances of Reader wrap up func ons of
one argument, providing us with useful methods for composing them.
One common use for Readers is injec ng congura on. If we have a

number of opera ons that all depend on some external congura on,
we can chain them together using a Reader. The Reader produces one
large opera on that accepts the congura on as a parameter and runs
our program in the order we specied it.
4.7.1 Crea ng and Unpacking Readers
We can create a Reader[A, B] from a func on A => B using the

Reader.apply constructor:
import cats.data.Reader
case class Cat(name: String, favoriteFood: String)

// defined class Cat
val catName: Reader[Cat, String] =

Reader(cat => cat.name)
// catName: cats.data.Reader[Cat,String] = Kleisli(<function1>)
We can extract the func on again using the Reader's run method and
call it using apply as usual:
catName.run(Cat("Garfield", "lasagne"))
// res0: cats.Id[String] = Garfield
We can create Readers from func ons and extract the func ons again.
So far so simple, but what advantage do Readers give us over the raw
func ons?
4.7.2 Composing Readers
The power of Readers comes from their map and flatMap methods,
which represent dierent kinds of func on composi on. The general
pa ern of usage is to create a set of Readers that accept the same
type of congura on, combine them with map and flatMap, and then
call run to inject the cong at the end.
The map method simply extends the computa on in the Reader by pass-
ing its result through a func on:
val greetKitty: Reader[Cat, String] =

catName.map(name => s"Hello ${name}")
greetKitty.run(Cat("Heathcliff", "junk food"))

// res1: cats.Id[String] = Hello Heathcliff
The flatMap method is more interes ng. It allows us to combine mul -

ple readers that depend on the same input type. To illustrate this, lets
extend our gree ng example to produce a login system that checks a
users password and displays dierent messages depending on whether
it was valid:
val feedKitty: Reader[Cat, String] =

Reader(cat => s"Have a nice bowl of ${cat.favoriteFood}")
val greetAndFeed: Reader[Cat, String] =

for {
msg1 <- greetKitty
msg2 <- feedKitty
} yield s"${msg1} ${msg2}"
greetAndFeed(Cat("Garfield", "lasagne"))
// res3: cats.Id[String] = Hello Garfield Have a nice bowl of
lasagne
greetAndFeed(Cat("Heathcliff", "junk food"))

// res4: cats.Id[String] = Hello Heathcliff Have a nice bowl of
junk food
4.7.3 Exercise: Hacking on Readers
The classic use of Readers is to build programs that accept a congura-

on at the end. Lets ground this with a complete example of a simple
login system. Our congura on will consist of two databases: a list of
valid users, and a list of their passwords:
case class Db(

usernames: Map[Int, String],
passwords: Map[String, String]
)
Start by crea ng a type alias DbReader for a Reader that consumes a

Db as input. This will make the rest of our code shorter:
See the solu on
Now create methods that generate DbReaders to look up the username

for an Int user ID, and look up the password for a String username.
The type signatures should be as follows:
def findUsername(userId: Int): DbReader[Option[String]] =

???
def checkPassword(
username: String,
password: String
): DbReader[Boolean] = ???
See the solu on
Finally create a checkLogin method to check the password for a given

user ID. The type signature should be as follows:
def checkLogin(
userId: Int,
password: String
): DbReader[Boolean] = ???
See the solu on
You should be able to use checkLogin as follows:

val db = Db(
Map(
1 -> "dade",
2 -> "kate",
3 -> "margo"
),
Map(
"dade" -> "zerocool",
"kate" -> "acidburn",
"margo" -> "secret"
)
)
// db: Db = Db(Map(1 -> dade, 2 -> kate, 3 -> margo),Map(dade ->
zerocool, kate -> acidburn, margo -> secret))
checkLogin(1, "zerocool").run(db)
// res8: cats.Id[Boolean] = true
checkLogin(4, "davinci").run(db)
// res9: cats.Id[Boolean] = false
4.7.4 When to Use Readers?
As you can hopefully see from the exercise, Readers eec vely pro-
vide a simple tool for doing dependency injec on. We write steps of
our program as instances of Reader, chain them together with map and
flatMap, and build a func on that accepts the dependency as input.
There are many, many ways of implemen ng dependency injec on in

Scala, from simple techniques like methods with mul ple parameter
lists, through implicit parameters and type classes, to complex tech-
niques like the cake pa ern and DI frameworks.
Readers are most useful in situa ons where:
we are construc ng a batch program that can easily be repre-

sented by a func on;
we need to defer injec on of a known parameter or set of pa-

rameters;
we want to be able to test parts of the program in isola on.
By represen ng the steps of our program as Readers we can test that

them as easily as pure func ons, plus we gain access to the map and
flatMap combinators.
For more advanced problems where we have lots of dependencies, or

a our program isnt easily represented as a pure func on, other depen-
dency injec on techniques tend to be more appropriate.
Kleisli arrows
You may have no ced from console output in this sec on that
Reader is implemented in terms of another type called Kleisli.
Kleisli arrows provide a more general form of Reader that gener-
alise over the type constructor of the result type.
Kleislis are beyond the scope of this book, but will be easy to pick
up based on your newfound knowledge of Reader and the con-
tent well cover in the next chapter on monad transformers.
4.8 The State Monad
cats.data.State allows us to pass addi onal state around as part of

a computa on. We dene State instances represen ng atomic opera-
ons on the state, and thread them together using map and flatMap. In
this way we can model mutable state in a purely func onal way, with-
out using muta on.
4.8. THE STATE MONAD 121
4.8.1 Crea ng and Unpacking State
Boiled down to its simplest form, instances of State[S, A] represent

func ons of type S => (S, A). S is the type of the state and and A is
the type of the result.
import cats.data.State
val a = State[Int, String] { state =>

(state, s"The state is $state")
}
// a: cats.data.State[Int,String] = cats.data.StateT@ad944e3
In other words, an instance of State is a combina on of two things:
a transforma on from an input state to an output state;

a computa on of a result.
We can run our monad by supplying an ini al state. State provides

three methodsrun, runS, and runAthat return dierent combina-
ons of state and result. Each method actually returns an instance of
Eval, which State uses to maintain stack safety. We call the value
method as usual to extract the actual result:
// Get the state and the result:

val (state, result) = a.run(10).value
// state: Int = 10
// result: String = The state is 10
// Get the state, ignore the result:

val state = a.runS(10).value
// state: Int = 10
// Get the result, ignore the state:

val result = a.runA(10).value
// result: String = The state is 10
4.8.2 Composing and Transforming State
As weve seen with Reader and Writer, the power of the State
monad comes from combining instances. The map and flatMap
methods thread the State from one instance to another. Because
each primi ve instance represents a transforma on on the state, the
combined instance represents a more complex transforma on.
val step1 = State[Int, String] { num =>
val ans = num + 1
(ans, s"Result of step1: $ans")
}
// step1: cats.data.State[Int,String] = cats.data.StateT@e680f25
val step2 = State[Int, String] { num =>

val ans = num * 2
(ans, s"Result of step2: $ans")
}
// step2: cats.data.State[Int,String] = cats.data.StateT@1119
acde
val both = for {

a <- step1
b <- step2
} yield (a, b)
// both: cats.data.StateT[cats.Eval,Int,(String, String)] = cats
.data.StateT@2d4d4ff3
val (state, result) = both.run(20).value

// state: Int = 42
// result: (String, String) = (Result of step1: 21,Result of
step2: 42)
As you can see, in this example the nal state is the result of applying
both transforma ons in sequence. The state is threaded from step to
step even though we dont interact with it in the for comprehension.
The general model for using the State monad, then, is to represent
each step of a computa on as an instance of State, and compose the
steps using the standard monad operators. Cats provides several con-
venience constructors for crea ng primi ve steps:
get extracts the state as the result;

set updates the state and returns unit as the result;
pure ignores the state and returns a supplied result;
inspect extracts the state via a transforma on func on;
modify updates the state using an update func on.
val getDemo = State.get[Int]

// getDemo: cats.data.State[Int,Int] = cats.data.StateT@26c929b1
getDemo.run(10).value
// res3: (Int, Int) = (10,10)
val setDemo = State.set[Int](30)

// setDemo: cats.data.State[Int,Unit] = cats.data.StateT@1748341
a
setDemo.run(10).value
// res4: (Int, Unit) = (30,())
val pureDemo = State.pure[Int, String]("Result")

// pureDemo: cats.data.State[Int,String] = cats.data.StateT@
1826901
pureDemo.run(10).value
// res5: (Int, String) = (10,Result)
val inspectDemo = State.inspect[Int, String](_ + "!")

// inspectDemo: cats.data.State[Int,String] = cats.data.StateT@
77e7bb7e
inspectDemo.run(10).value
// res6: (Int, String) = (10,10!)
val modifyDemo = State.modify[Int](_ + 1)

// modifyDemo: cats.data.State[Int,Unit] = cats.data.StateT@56
cf32b8
modifyDemo.run(10).value
// res7: (Int, Unit) = (11,())
We can assemble these building blocks into useful computa ons. We

o en end up ignoring the results of intermediate stages when they only
represent transforma ons on the state:
import State._
val program: State[Int, (Int, Int, Int)] = for {

a <- get[Int]
_ <- set[Int](a + 1)
b <- get[Int]
_ <- modify[Int](_ + 1)
c <- inspect[Int, Int](_ * 1000)
} yield (a, b, c)
// program: cats.data.State[Int,(Int, Int, Int)] = cats.data.
StateT@4996bc50
val (state, result) = program.run(1).value

// state: Int = 3
// result: (Int, Int, Int) = (1,2,3000)
4.8.3 Exercise: Post-Order Calculator
The State monad allows us to implement simple evaluators for com-

plex expressions, passing the values of mutable registers along in the
state component. We model the atomic opera ons as instances of
State, and combine them to evaluate whole sequences of inputs. We
can see a simple example of this by implemen ng a calculator for post-
order integer arithme c expressions.
In case you havent heard of post-order expressions before (dont worry

if you havent), they are a mathema cal nota on where we write the
operator a er its operands. So, for example, instead of wri ng 1 + 2
we would write:
1 2 +
Although post-order expressions are dicult for humans to read, they

are easy to evaluate using a computer program.
All we need to do is traverse the symbols from le to right, carrying a

stack of operands with us as we go:
when we see a number, we push it onto the stack;
when we see an operator, we pop two operands o the stack,

operate on them, and push the result in their place.
This allows us to evaluate complex expressions without using parenthe-

ses. For example, we can evaluate (1 + 2) * 3) as follows:
1 2 + 3 * // see 1, push onto stack

2 + 3 * // see 2, push onto stack
+ 3 * // see +, pop 1 and 2 off of stack,
// push (1 + 2) = 3 in their place
3 3 * // see 3, push onto stack
3 * // see 3, push onto stack
* // see *, pop 3 and 3 off of stack,
// push (3 * 3) = 9 in their place
We can write a simple interpreter for these expressions using the State
monad. We can parse each symbol into a State instance represen ng
a context-free stack transform and intermediate result. The State in-
stances can be threaded together using flatMap to produce an inter-
preter for any sequence of symbols.
Lets do this now. Start by wri ng a func on evalOne that parses a sin-
gle symbol into an instance of State. Use the code below as a template.
Dont worry about error handling for nowif the stack is in the wrong
congura on, its ok to throw an excep on and fail.
import cats.data.State
type CalcState[A] = State[List[Int], A]
def evalOne(sym: String): CalcState[Int] = ???
If this seems dicult, think about the basic form of the State instances
youre returning. Each instance represents a func onal transforma on
from a stack to a pair of a stack and a result. You can ignore any wider
context and focus on just that one step:
State[List[Int], Int] { oldStack =>

val newStack = someTransformation(oldStack)
val result = someCalculation
(newStack, result)
}
Feel free to write your Stack instances in this form or as sequences of

the convenience constructors we saw above.
See the solu on
evalOne allows us to evaluate single-symbol expressions as follows.
We call runA supplying Nil as an ini al stack, and call value to unpack
the resul ng Eval instance:
evalOne("42").runA(Nil).value
// res3: Int = 42
We can represent more complex programs using evalOne, map, and

flatMap. Note that most of the work is happening on the stack, so
we ignore the results of the intermediate steps for evalOne("1") and
evalOne("2"):
val program = for {

_ <- evalOne("1")
_ <- evalOne("2")
ans <- evalOne("+")
} yield ans
// program: cats.data.StateT[cats.Eval,List[Int],Int] = cats.
data.StateT@5f0e1a6c
program.runA(Nil).value
// res4: Int = 3
Generalise this example by wri ng an evalAll method that computes

the result of a List[String]. Use evalOne to process each symbol,
and thread the resul ng State monads together using flatMap. Your
func on should have the following signature:
def evalAll(input: List[String]): CalcState[Int] = ???

// evalAll: (input: List[String])CalcState[Int]
See the solu on

We can use evalAll to conveniently evaluate mul -stage expressions:
val program = evalAll(List("1", "2", "+", "3", "*"))
Because evalOne and evalAll both return instances of State, we can

even thread these results together using flatMap. evalOne produces a
simple stack transforma on and evalAll produces a complex one, but
theyre both pure func ons and we can use them in any order as many
mes as we like:
val program = for {
_ <- evalAll(List("1", "2", "+"))
_ <- evalAll(List("3", "4", "+"))
ans <- evalOne("*")
} yield ans
4.9 Dening Custom Monads
We can dene a Monad for a custom type by providing implementa ons

of thee methods: flatMap, pure, and a new method called tailRecM.
Here is an implementa on of Monad for Option as an example:
import cats.Monad
import scala.annotation.tailrec
val optionMonad =
new Monad[Option] {
def flatMap[A, B](opt: Option[A])
(fn: A => Option[B]): Option[B] =
opt flatMap fn
def pure[A](opt: A): Option[A] =

Some(opt)
@tailrec
def tailRecM[A, B](a: A)
(fn: A => Option[Either[A, B]]): Option[B] =
fn(a) match {
case None => None
case Some(Left(a1)) => tailRecM(a1)(fn)
case Some(Right(b)) => Some(b)
}
}
tailRecM is an op misa on in Cats that limits the amount of stack

space used by nested calls to flatMap. The technique comes from a
2015 paper by PureScript creator Phil Freeman. The method should re-
cursively call itself as long as the result of f returns a Right. If we can
make tailRecM tail recursive, we should do so to allow Cats to perform
addi onal internal op misa ons.
4.10. SUMMARY 129
4.9.1 Exercise: Branching out Further with Monads
Lets write a Monad for our Tree data type from last chapter. Heres the
type again, together with the smart constructors we used to simplify
type class instance selec on:
sealed trait Tree[+A]

final case class Branch[A](left: Tree[A], right: Tree[A])
extends Tree[A]
final case class Leaf[A](value: A) extends Tree[A]
def branch[A](left: Tree[A], right: Tree[A]): Tree[A] =

Branch(left, right)
def leaf[A](value: A): Tree[A] =

Leaf(value)
Verify that the code works on instances of Branch and Leaf, and that
the Monad provides Functor-like behaviour for free.
Verify that having a Monad in scope allows us to use for comprehensions,

despite the fact that we havent directly implemented flatMap or map
on Tree.
See the solu on
4.10 Summary
In this chapter weve seen monads up-close. We saw that flatMap

can be viewed as sequencing computa ons, giving the order in which
opera ons must happen. In this view, Option represents a computa-
on that can fail without an error message; Either represents compu-
ta ons that can fail with a message; List represents mul ple possible
results; and Future represents a computa on that may produce a value
at some point in the future.
In this chapter weve also seen some of the custom types and data
structures that Cats provides, including Id, Reader, Writer, and State.
These cover a wide range of uses and many problems can be solved by
using one of these constructs.
Finally, if we do have to implement our own monad instance, weve

have learned about tailRecM. This is an odd wrinklea concession to
building a func onal programming library that is stack-safe by default.
We dont need to understand tailRecM to understand monads, but
having it around gives us mechanical benets that we can be grateful
for when wri ng monadic code.
Chapter 5
Monad Transformers
Monads are like burritos, which means that once you acquire a taste,
youll nd yourself returning to them again and again. This is not with-
out issues. As burritos can bloat the waist, monads can bloat the code
base through nested for-comprehensions.
Imagine we are interac ng with a database. We want to look up a
user record. The user may or may not be present, so we return an
Option[User]. Our communica on with the database could fail for
many reasons (network issues, authen ca on problems, database prob-
lems, and so on), so this result is wrapped up in an Either, giving us a
nal result of Either[Error, Option[User]].
To use this value we must nest flatMap calls (or equivalently,
for-comprehensions):
def lookupUserName(id: Long): Either[Error, Option[String]] =

for {
optUser <- lookupUser(id)
} yield {
for {
user <- optUser
} yield user.name
131
132 CHAPTER 5. MONAD TRANSFORMERS
This quickly becomes very tedious.

A ques on arises. Given two monads, can we make one monad out of
them in a generic way? That is, do monads compose? We can try to
write the code but well soon nd it impossible to implement flatMap:
// This code won't actually compile.

// It's just illustrating a point:
def compose[M1[_] : Monad, M2[_] : Monad] = {
type Composed[A] = M1[M2[A]]
new Monad[Composed] {
def pure[A](a: A): Composed[A] =
a.pure[M2].pure[M1]
def flatMap[A, B](fa: Composed[A])

(f: A => Composed[B]): Composed[B] =
// This is impossible to implement in general
// without knowing something about M1 or M2:
???
}
}
We cant compose monads in general. However, some monad instances

can be made to compose with instance-specic glue code. For these
special cases we can use monad transformers to compose them.
Monad transformers allow us to squash together monads, crea ng one
monad where we previously had two or more. With this transformed
monad we can avoid nested calls to flatMap.
5.1 A Transforma ve Example
Cats provides a library of such transformers: EitherT for composing

Either with other monads, OptionT for composing Option, and so on.
5.1. A TRANSFORMATIVE EXAMPLE 133
Heres an example that uses OptionT to squash List and Option into
a single monad. Where we might use List[Option[A]] we can use
ListOption[A] to avoid nested flatMap calls.
import cats.data.OptionT
type ListOption[A] = OptionT[List, A]
ListOption is a monad that combines the proper es of List and

Option. Note how we build it from the inside out: we pass List, the
type of the outer monad, as a parameter to OptionT, the transformer
for the inner monad.
We can create instances with pure as usual:
import cats.Monad
val result: ListOption[Int] = 42.pure[ListOption]

// result: ListOption[Int] = OptionT(List(Some(42)))
The map and flatMap methods of ListOption combine the corre-

sponding methods of List and Option into single opera ons:
val a = 10.pure[ListOption]
// a: ListOption[Int] = OptionT(List(Some(10)))
val b = 32.pure[ListOption]
// b: ListOption[Int] = OptionT(List(Some(32)))
a flatMap { (x: Int) =>

b map { (y: Int) =>
x + y
}
}
// res1: cats.data.OptionT[List,Int] = OptionT(List(Some(42)))
This is the basics of using monad transformers. The combined map and
flatMap methods allow us to use both component monads without
having to recursively unpack and repack values at each stage in the
computa on. Now lets look at the API in more depth.
Complexity of imports
Note the imports in the code samples abovethey hint at how

everything bolts together.
We import cats.syntax.applicative to get the pure syntax.

pure requires an implicit parameter of type Applicative[ListOption].
We havent met Applicatives yet, but all Monads are also
Applicatives so we can ignore that dierence for now.
We need an Applicative[ListOption] to call pure. We

have cats.data.OptionT in scope, which provides the
implicits for OptionT. However, in order to generate our
Applicative[ListOption], the implicits for OptionT also
require an Applicative for List. Hence the addi onal import
from cats.instances.list.
No ce were not impor ng cats.syntax.functor or

cats.syntax.flatMap. This is because OptionT is a con-
crete data type with its own explicit map and flatMap methods.
It wouldnt hurt to import the syntaxthe compiler will simply
ignore it in favour of the explicit methods.
Remember that were subjec ng ourselves to this shenanigans

because were stubbornly refusing to import our instances from
cats.instances.all. If we did that, everything would just
work.
5.2. MONAD TRANSFORMERS IN CATS 135
5.2 Monad Transformers in Cats
Monad transformers are a li le dierent to the other abstrac ons

weve seenthey dont have their own type class. We use monad
transformers to build monads, which we then use via the Monad type
class. Thus the main points of interest when using monad transformers
are:
the available transformer classes;

building stacks of monads using transformers;
construc ng instances of a monad stack; and
pulling apart a stack to access the wrapped monads.
5.2.1 The Monad Transformer Classes
By conven on, in Cats a monad Foo will have a transformer class called
FooT. In fact, many monads in Cats are dened by combining a monad
transformer with the Id monad. Concretely, some of the available in-
stances are:
cats.data.OptionT for Option;

cats.data.EitherT for Either;
cats.data.ReaderT, cats.data.WriterT, and cats.data.StateT;
cats.data.IdT for the Id monad.
All of these monad transformers follow the same conven on: the rst
type parameter species the monad that is wrapped around the monad
implied by the transformer. The remaining type parameters are the
types weve used to form the corresponding monads.
Kleisli Arrows
Last chapter, in the sec on on the Reader monad, we men oned

that Reader was a specialisa on of a more general concept called
a kleisli arrow (aka cats.data.Kleisli).
We can now reveal that Kleisli and ReaderT are, in fact, the
same thing! ReaderT is actually a type alias for Kleisli. Hence
why we were crea ng Readers last chapter and seeing Kleislis
on the console.
5.2.2 Building Monad Stacks
Building monad stacks is a li le confusing un l you know the pa erns.

The rst type parameter to a monad transformer is the outer monad in
the stackthe transformer itself provides the inner monad. For exam-
ple, our ListOption type above was built using OptionT[List, A]
but the result was eec vely a List[Option[A]]. In other words, we
build monad stacks from the inside out.
Many monads and all transformers have at least two type parameters,
so we o en have to dene type aliases for intermediate stages. For
example, suppose we want to wrap Either around Option. Option is
the innermost type so we want to use the OptionT monad transformer.
We need to use Either as the rst type parameter. However, Either
itself has two type parameters and monads only have one. We need a
type alias to make everything the correct shape:
import cats.instances.either._
type Error = String
// Create a type alias, ErrorOr, to convert Either to

// a type constructor with a single parameter:
type ErrorOr[A] = Either[Error, A]
// Use ErrorOr as a type parameter to OptionT:

type ErrorOptionOr[A] = OptionT[ErrorOr, A]
ErrorOptionOr is a monad. We can use pure and flatMap as usual to

create and transform instances:
val result1 = 41.pure[ErrorOptionOr]

// result1: ErrorOptionOr[Int] = OptionT(Right(Some(41)))
val result2 = result1.flatMap(x => (x + 1).pure[ErrorOptionOr])

// result2: cats.data.OptionT[ErrorOr,Int] = OptionT(Right(Some
(42)))
Now lets add another monad into our stack. Lets create a Future of
an Either of Option. Once again we build this from the inside out with
an OptionT of an EitherT of Future. However, we cant dene this in
one line because EitherT has three type parameters:
import cats.data.EitherT
type FutureEitherOption[A] = OptionT[EitherT, A]

// <console>:25: error: cats.data.EitherT takes three type
parameters, expected: one
// type FutureEitherOption[A] = OptionT[EitherT, A]
// ^
As before, we solve the problem by crea ng a type alias with a single

parameter. This me we create an alias for EitherT that xes Future
and Error and allows A to vary:
import cats.data.{EitherT, OptionT}
type Error = String

type FutureEither[A] = EitherT[Future, String, A]

type FutureEitherOption[A] = OptionT[FutureEither, A]
Our mammoth stack composes not two but three monads. Our map and
flatMap methods cut through three layers of abstrac on:
val answer: FutureEitherOption[Int] =

for {
a <- 10.pure[FutureEitherOption]
b <- 32.pure[FutureEitherOption]
} yield a + b
// answer: FutureEitherOption[Int] = OptionT(EitherT(Future(<not
completed>)))
Kind Projector
If you frequently nd yourself dening mul ple type aliases when

building monad stacks, you may want to try the Kind Projector
compiler plugin. Kind Projector enhances Scalas type syntax to
make it easier to dene par al types. For example:
// import cats.instances.option._
123.pure[EitherT[Option, String, ?]]

// res9: cats.data.EitherT[Option,String,Int] = EitherT(
Some(Right(123)))
Kind Projector cant simplify all type declara ons down to a single
line, but it can reduce the number of intermediate type deni ons
we need to keep our code readable.
5.2.3 Construc ng and Unpacking Instances
As we saw above, we can use pure to directly inject raw values into
transformed monad stacks. We can also create instances from untrans-
formed stacks using the monad transformers apply method:
import cats.syntax.either._ // for foo.asRight

import cats.syntax.option._ // for foo.some
type ErrorOr[A] = Either[String, A]

type ErrorOrOption[A] = OptionT[ErrorOr, A]
// Create using pure:

val stack1 = 123.pure[ErrorOrOption]
// stack1: ErrorOrOption[Int] = OptionT(Right(Some(123)))
// Create using apply:

val stack2 = OptionT[ErrorOr, Int](
123.some.asRight[String]
)
// stack2: cats.data.OptionT[ErrorOr,Int] = OptionT(Right(Some
(123)))
Once weve nished with a monad transformer stack, we can unpack it

using its value method. This returns the untransformed stack. We can
then manipulate the individual monads in the usual way:
stack1.value
// res13: ErrorOr[Option[Int]] = Right(Some(123))
stack2.value
// res14: ErrorOr[Option[Int]] = Right(Some(123))
Each call to value unpacks a single monad transformer, so we may need

more than one call to completely unpack a large stack:
import cats.data.{Writer, EitherT, OptionT}

type LoggedFallable[A] = EitherT[Logged, String, A]
type LoggedFallableOption[A] = OptionT[LoggedFallable, A]
val packed = 123.pure[LoggedFallableOption]

// packed: LoggedFallableOption[Int] = OptionT(EitherT(WriterT((
Vector(),Right(Some(123))))))
val partiallyPacked = packed.value

// partiallyPacked: LoggedFallable[Option[Int]] = EitherT(
WriterT((Vector(),Right(Some(123)))))
val completelyUnpacked = partiallyPacked.value

// completelyUnpacked: Logged[Either[String,Option[Int]]] =
WriterT((Vector(),Right(Some(123))))
5.2.4 Usage Pa erns
Widespread use of monad tranformers is some mes dicult because

they fuse monads together in predened ways. Without careful
thought, we can end up having to unpack and repack monads in
dierent congura ons to operate on them in dierent contexts.
One way of avoiding this is to use monad transformers as local glue
code. Expose untransformed stacks at module boundaries, transform
them to operate on them locally, and untransform them before passing
them on. This allows each module of code to make its own decisions
about which transformers to use. Heres an example:
type Logged[A] = Writer[List[String], A]
// Example method that returns nested monads:

def parseNumber(str: String): Logged[Option[Int]] =
util.Try(str.toInt).toOption match {
case Some(num) => Writer(List(s"Read $str"), Some(num))

case None => Writer(List(s"Failed on $str"), None)
}
// Example combining multiple calls to parseNumber:

def addNumbers(
a: String,
b: String,
c: String
): Logged[Option[Int]] = {
import cats.data.OptionT
// Transform the incoming stacks to work on them:

val result = for {
a <- OptionT(parseNumber(a))
b <- OptionT(parseNumber(b))
c <- OptionT(parseNumber(c))
} yield a + b + c
// Return the untransformed monad stack:

result.value
}
// This approach doesn't force OptionT on other users' code:

val result1 = addNumbers("1", "2", "3")
// result1: Logged[Option[Int]] = WriterT((List(Read 1, Read 2,
Read 3),Some(6)))
val result2 = addNumbers("1", "a", "3")

// result2: Logged[Option[Int]] = WriterT((List(Read 1, Failed
on a),None))
Many monads in Cats are dened using the corresponding transformer

and the Id monad. This is reassuring as it conrms that the APIs for
these monads and transformers are iden cal. Reader, Writer, and
State are all dened in this way:
type Reader[E, A] = ReaderT[Id, E, A] // = Kleisli[Id, E, A]

type Writer[W, A] = WriterT[Id, W, A]
type State[S, A] = StateT[Id, S, A]
In other cases monad transformers have separate deni ons to their

corresponding monads. In these cases, the methods of the transformer
tend to mirror the methods on the monad. For example, OptionT de-
nes getOrElse, and EitherT denes fold, bimap, swap, and other
useful methods.
5.3 Exercise: Monads: Transform and Roll Out
The Autobots, well known robots in disguise, frequently send messages

during ba le reques ng the power levels of their team mates. This
helps them coordinate strategies and launch devasta ng a acks. The
message sending method looks like this:
def getPowerLevel(autobot: String): Response[Int] =

???
Transmissions take me in Earths viscous atmosphere, and messages

are occasionally lost due to malfunc oning satellites or Decep con in-
tercep on. Responses are therefore represented as a stack of monads:
type Response[A] = Future[Either[String, A]]

// defined type alias Response
Op mus Prime is ge ng red of the nested for comprehensions in his

neural matrix. Help him by rewri ng Response using a monad trans-
former.
See the solu on
Now test the code by implemen ng getPowerLevel to retrieve data
from a set of imaginary allies. Heres the data well use:
5.3. EXERCISE: MONADS: TRANSFORM AND ROLL OUT 143
val powerLevels = Map(

"Jazz" -> 6,
"Bumblebee" -> 8,
"Hot Rod" -> 10
)
If an Autobot isnt in the powerLevels map, return an error message

repor ng that they were unreachable. Include the name in the message
for good eect.
See the solu on
Two autobots can perform a special move if their combined power level
is greater than 15. Write a second method, canSpecialMove, that ac-
cepts the names of two allies and checks whether a special move is
possible. If either ally is unavailable, fail with an appropriate error mes-
sage:
def canSpecialMove(
ally1: String,
ally2: String
): Response[Boolean] = ???
See the solu on
Finally, write a method tacticalReport that takes two ally names and
prints a message saying whether they can perform a special move:
def tacticalReport(
ally1: String,
ally2: String
): String = ???
See the solu on
You should be able to use report as follows:

tacticalReport("Jazz", "Bumblebee")
// res25: String = Jazz and Bumblebee need a recharge.
tacticalReport("Bumblebee", "Hot Rod")

// res26: String = Bumblebee and Hot Rod are ready to roll out!
tacticalReport("Jazz", "Ironhide")
// res27: String = Comms error: Ironhide unreachable
5.4 Summary
In this chapter we introduced monad transformers, which eliminate the

need for nested for comprehensions and pa ern matching when work-
ing with stacks of nested monads such as below:
// import cats.syntax.either._
val a = Option(1.asRight[String])
// a: Option[Either[String,Int]] = Some(Right(1))
val b = Option(1.asRight[String])
// b: Option[Either[String,Int]] = Some(Right(1))
val result = for {

x <- a
y <- b
} yield {
for {
u <- x
v <- y
} yield u + v
}
// result: Option[scala.util.Either[String,Int]] = Some(Right(2)
)
Each monad transformer, such as FutureT, OptionT or EitherT, pro-

5.4. SUMMARY 145
vides the code needed to merge its related monad with other monads.
The transformer is a data structure that wraps a monad stack, equipping
it with map and flatMap methods that unpack and repack the whole
stack:
val wrappedA = EitherT(a)

// wrappedA: cats.data.EitherT[Option,String,Int] = EitherT(Some
(Right(1)))
val wrappedB = EitherT(b)

// wrappedB: cats.data.EitherT[Option,String,Int] = EitherT(Some
(Right(1)))
val wrappedResult = for {

x <- wrappedA
y <- wrappedB
} yield x + y
// wrappedResult: cats.data.EitherT[Option,String,Int] = EitherT
(Some(Right(2)))
val result = wrappedResult.value

// result: Option[Either[String,Int]] = Some(Right(2))
The type signatures of monad transformers are wri en from the in-
side out, so an EitherT[Option, String, A] is a wrapper for an
Option[Either[String, A]]. It is o en useful to use type aliases
when wri ng transformer types for deeply nested monads.
With this look at monad transformers, we have now covered everything

we need to know about monads and the sequencing of computa ons
using flatMap. In the next chapter we will switch tack and discuss
two new type classes, Cartesian and Applicative, that support new
kinds of opera on such as zipping independent values within a con-
text.
Chapter 6
Cartesians and Applica ves
In previous chapters we saw how functors and monads let us transform

values using map and flatMap. While functors and monads are both
immensely useful abstrac ons, there are types of transforma on that
are inconvenient to represent with these methods.
One such example is form valida on. When we validate a form we
want to return all the errors to the user, not simply stop on the rst
error we encounter. If we model this with a monad like Either, we fail
fast and lose errors. For example, the code below fails on the rst call
to parseInt and doesnt go any further:
def parseInt(str: String): Either[String, Int] =

Either.catchOnly[NumberFormatException](str.toInt).
leftMap(_ => s"Couldn't read $str")
for {
a <- parseInt("a")
b <- parseInt("b")
c <- parseInt("c")
} yield (a + b + c)
147
148 CHAPTER 6. CARTESIANS AND APPLICATIVES
// res1: scala.util.Either[String,Int] = Left(Couldn't read a)
Another example is the concurrent evalua on of Futures. If we have

several long-running independent tasks, it makes sense to execute
them concurrently. However, monadic comprehension only allows us
to run them in sequence. Even on a mul core CPU, the code below
runs in sequence as you can see from the mestamps:
lazy val timestamp0 = System.currentTimeMillis
def getTimestamp: Long = {

val timestamp = System.currentTimeMillis - timestamp0
Thread.sleep(100)
timestamp
}
val timestamps = for {

a <- Future(getTimestamp)
b <- Future(getTimestamp)
c <- Future(getTimestamp)
} yield (a, b, c)
Await.result(timestamps, 1.second)
// res5: (Long, Long, Long) = (0,106,210)
map and flatMap arent quite capable of capturing what we want here
because they make the assump on that each computa on is dependent
on the previous one:
// context2 is dependent on value1:

context1.flatMap(value1 => context2)
The calls to parseInt and Future.apply above are independent of one

another, but map and flatMap cant exploit this. We need a weaker
6.1. CARTESIAN 149
constructone that doesnt guarantee sequencingto achieve the re-

sult we want. In this chapter we will look at two type classes that sup-
port this pa ern:
Cartesians encompass the no on of zipping pairs of contexts.

Cats provides a CartesianBuilder syntax that combines
Cartesians and Functors to allow users to join values within
a context using arbitrary func ons.
Applica ve functors, also known as Applicatives, extend

Cartesian and Functor and provide a way of applying func-
ons to parameters within a context. Applicative is the
source of the pure method we introduced in Chapter 4.
Applica ves are o en formulated in terms of func on applica on, in-

stead of the cartesian formula on that is emphasised in Cats. This al-
terna ve formula on provides a link to other libraries and languages
such as Scalaz and Haskell. Well take a look a dierent formula ons of
Applica ve, as well as the rela onships between Cartesian, Functor,
Applicative, and Monad, towards the end of the chapter.
6.1 Cartesian
Cartesian is a type class that allows us to zip values within a context.

If we have two objects of type F[A] and F[B], a Cartesian[F] allows
us to combine them to form an F[(A, B)]. Its deni on in Cats is:
trait Cartesian[F[_]] {
def product[A, B](fa: F[A], fb: F[B]): F[(A, B)]
}
As we discussed above, the parameters fa and fb are independent of

one another. This gives us a lot more exibility when dening instances
of Cartesian than we do when dening Monads.
6.1.1 Joining Two Contexts
Whereas Semigroups allow us to join values, Cartesians allow us to

join contexts. Lets join some Options as an example:
import cats.Cartesian
import cats.instances.option._ // Cartesian for Option
Cartesian[Option].product(Some(123), Some("abc"))
// res0: Option[(Int, String)] = Some((123,abc))
If both parameters are instances of Some, we end up with a tuple of the

values within. If either parameter evaluates to None, the en re result is
None:
Cartesian[Option].product(None, Some("abc"))
// res1: Option[(Nothing, String)] = None
Cartesian[Option].product(Some(123), None)
// res2: Option[(Int, Nothing)] = None
6.1.2 Joining Three or More Contexts
The companion object for Cartesian denes a set of methods on top

of product. For example, the methods tuple2 through tuple22 gen-
eralise product to dierent ari es:
import cats.instances.option._ // Cartesian for Option
Cartesian.tuple3(Option(1), Option(2), Option(3))

// res3: Option[(Int, Int, Int)] = Some((1,2,3))
Cartesian.tuple3(Option(1), Option(2), Option.empty[Int])

// res4: Option[(Int, Int, Int)] = None
The methods map2 through map22 apply a user-specied func on to

the values inside 2 to 22 contexts:
6.2. CARTESIAN BUILDER SYNTAX 151
Cartesian.map3(
Option(1),
Option(2),
Option(3)
)(_ + _ + _)
Cartesian.map3(
Option(1),
Option(2),
Option.empty[Int]
)(_ + _ + _)
There are also methods contramap2 through contramap22 and

imap2 through imap22, that require instances of Contravariant and
Invariant respec vely.
6.2 Cartesian Builder Syntax
Cats provides a convenient syntax called cartesian builder syntax, that

provides shorthand for methods like tupleN and mapN. We import the
syntax from cats.syntax.cartesian. Heres an example:
import cats.syntax.cartesian._
(Option(123) |@| Option("abc")).tupled

The |@| operator, be er known as a e ghter, creates a temporary

builder object that provides several methods for combining the param-
eters to create useful data types. For example, the tupled method zips
the values into a tuple:
val builder2 = Option(123) |@| Option("abc")
builder2.tupled
We can use |@| repeatedly to create builders for up to 22 values. Each

arity of builder, from 2 to 22, denes a tupled method to combine the
values to form a tuple of the correct size:
val builder3 = Option(123) |@| Option("abc") |@| Option(true)
builder3.tupled
// res9: Option[(Int, String, Boolean)] = Some((123,abc,true))
val builder5 = builder3 |@| Option(0.5) |@| Option('x')
builder5.tupled
// res10: Option[(Int, String, Boolean, Double, Char)] = Some
((123,abc,true,0.5,x))
The idioma c way of wri ng builder syntax is to combine |@| and

tupled in a single expression, going from single values to a tuple in
one step:
(
Option(1) |@|
Option(2) |@|
Option(3)
).tupled
// res11: Option[(Int, Int, Int)] = Some((1,2,3))
In addi on to tupled, every builder has a map method that accepts

an implicit Functor and a func on of the correct arity to combine the
values:
6.2. CARTESIAN BUILDER SYNTAX 153
case class Cat(name: String, born: Int, color: String)
(
Option("Garfield") |@|
Option(1978) |@|
Option("Orange and black")
).map(Cat.apply)
// res12: Option[Cat] = Some(Cat(Garfield,1978,Orange and black)
)
If we supply a func on that accepts the wrong number or types of pa-

rameters, we get a compile error:
val add: (Int, Int) => Int = (a, b) => a + b

// add: (Int, Int) => Int = <function2>
(Option(1) |@| Option(2) |@| Option(3)).map(add)

// found : (Int, Int) => Int
// required: (Int, Int, Int) => ?
// (Option(1) |@| Option(2) |@| Option(3)).map(add)
// ^
(Option("cats") |@| Option(true)).map(add)

// found : (Int, Int) => Int
// required: (String, Boolean) => ?
// (Option("cats") |@| Option(true)).map(add)
// ^
6.2.1 Fancy Functors and Cartesian Builder Syntax
Cartesian builders also have a contramap and imap methods that ac-
cept Contravariant and Invariant functors. For example, we can com-
bine Monoids and Semigroups using Invariant. Heres an example:
import cats.Monoid
import cats.instances.boolean._
case class Cat(

name: String,
yearOfBirth: Int,
favoriteFoods: List[String]
)
def catToTuple(cat: Cat) =

(cat.name, cat.yearOfBirth, cat.favoriteFoods)
implicit val catMonoid = (

Monoid[String] |@|
Monoid[Int] |@|
Monoid[List[String]]
).imap(Cat.apply)(catToTuple)
Our Monoid allows us to create empty Cats and add Cats together
using the syntax from Chapter 2:
import cats.syntax.monoid._
Monoid[Cat].empty
// res18: Cat = Cat(,0,List())
val garfield = Cat("Garfield", 1978, List("Lasagne"))

val heathcliff = Cat("Heathcliff", 1988, List("Junk Food"))
garfield |+| heathcliff

// res19: Cat = Cat(GarfieldHeathcliff,3966,List(Lasagne, Junk
Food))
6.3. CARTESIAN APPLIED TO DIFFERENT TYPES 155
6.3 Cartesian Applied to Dierent Types
Cartesians dont always provide the behaviour we expect, par cu-

larly for types that also have instances of Monad. We have seen the
behaviour of the Cartesian for Option. Lets look at some examples
for other types.
6.3.1 Cartesian Applied to Future
The seman cs for Future are pre y much what wed expect, providing
parallel as opposed to sequen al execu on:
val futurePair = Cartesian[Future].

product(Future("Hello"), Future(123))
Await.result(futurePair, 1.second)
// res2: (String, Int) = (Hello,123)
The two Futures start execu ng the moment we create them, so they
are already calcula ng results by the me we call product. Cartesian
builder syntax provides a concise syntax for zipping xed numbers of
Futures:
case class Cat(

name: String,
yearOfBirth: Int,
favoriteFoods: List[String]
val futureCat = (
Future("Garfield") |@|
Future(1978) |@|
Future(List("Lasagne"))
).map(Cat.apply)
Await.result(futureCat, 1.second)
// res5: Cat = Cat(Garfield,1978,List(Lasagne))
6.3.2 Cartesian Applied to List
There is a Cartesian instance for List. What value do you think the
following expression will produce?
Cartesian[List].product(List(1, 2), List(3, 4))
There are at least two reasonable answers:
1. product could zip the lists, returning List((1, 3), (2, 4));
2. product could compute the cartesian product, taking every

element from the rst list and combining it with every element
from the second returning List((1, 3), (1, 4), (2, 3),
(2, 4)).
The name Cartesian is a hint as to which answer well get, but lets
run the code to be sure:
6.3. CARTESIAN APPLIED TO DIFFERENT TYPES 157
Cartesian[List].product(List(1, 2), List(3, 4))

// res8: List[(Int, Int)] = List((1,3), (1,4), (2,3), (2,4))
We get the cartesian product! This is perhaps surprising: zipping lists

tends to be a more common opera on.
6.3.3 Cartesian Applied to Either
What about Either? We opened this chapter with a discussion of

fail-fast versus accumula ng error-handling. Which behaviour will
product produce?
type ErrorOr[A] = Either[Vector[String], A]
Cartesian[ErrorOr].product(
Left(Vector("Error 1")),
Left(Vector("Error 2"))
)
// res10: ErrorOr[(Nothing, Nothing)] = Left(Vector(Error 1))
Surprisingly, we s ll get fail-fast seman cs. The product method sees

the rst failure and stops, despite knowing that the second parameter
is also a failure.
6.3.4 Cartesian Applied to Monads
The reason for these surprising results is that, like Option, List and
Either are both monads. To ensure consistent seman cs, Cats Monad
(which extends Cartesian) provides a standard deni on of product
in terms of map and flatMap.
Try wri ng this implementa on now:

import cats.Monad
def product[M[_]: Monad, A, B](

fa: M[A],
fb: M[B]
): M[(A, B)] = ???
See the solu on
We can implement product in terms of the monad opera ons, and Cats
enforces this implementa on for all monads. This gives what we might
think of as unexpected and less useful behaviour for a number of data
types. The consistency of seman cs is actually useful for higher level
abstrac ons, but we dont know about those yet.
So why bother with Cartesian at all? The answer is that we can create
useful data types that have instances of Cartesian (and Applicative)
but not Monad. This frees us to implement product in dierent ways.
Lets examing this further by looking at a new data type for error han-
dling.
6.4 Validated
By now we are familiar with the fail-fast error handling behaviour of

Either. Furthermore, because Either is a monad, we know that the
seman cs of product are the same as those for flatMap. In fact, it
is impossible for us to design a monadic data type that implements er-
ror accumula ng seman cs without breaking the consistency rules be-
tween these two methods.
Fortunately, Cats provides a data type called Validated that has an

instance of Cartesian but no instace of Monad. The implementa on
of product is therefore free to accumulate errors:
6.4. VALIDATED 159
import cats.data.Validated
import cats.instances.list._ // Semigroup for List
type AllErrorsOr[A] = Validated[List[String], A]
Cartesian[AllErrorsOr].product(
Validated.invalid(List("Error 1")),
Validated.invalid(List("Error 2"))
)
// res1: AllErrorsOr[(Nothing, Nothing)] = Invalid(List(Error 1,
Error 2))
Validated complements Either nicely. Between the two we have

support for both of the common types of error handling: fail-fast and
accumula ng.
6.4.1 Crea ng Instances of Validated
Validated has two subtypes, Validated.Valid and Validated.Invalid,

that correspond loosely to Right and Left. We can create instances
directly using their apply methods:
val v = Validated.Valid(123)
// v: cats.data.Validated.Valid[Int] = Valid(123)
val i = Validated.Invalid("Badness")
// i: cats.data.Validated.Invalid[String] = Invalid(Badness)
However, it is o en easier to use the valid and invalid smart con-

structors, which widen the return type to Validated:
val v = Validated.valid[String, Int](123)

// v: cats.data.Validated[String,Int] = Valid(123)
val i = Validated.invalid[String, Int]("Badness")

// i: cats.data.Validated[String,Int] = Invalid(Badness)
As a third op on we can import the valid and invalid extension meth-

ods from cats.syntax.validated:
import cats.syntax.validated._
123.valid[String]
// res2: cats.data.Validated[String,Int] = Valid(123)
"Badness".invalid[Int]
// res3: cats.data.Validated[String,Int] = Invalid(Badness)
Finally, there are a variety of methods on Validated to create instances

from dierent sources. We can create them from Exceptions, as well
as instances of Try, Either, and Option:
Validated.catchOnly[NumberFormatException]("foo".toInt)
// res4: cats.data.Validated[NumberFormatException,Int] =
Invalid(java.lang.NumberFormatException: For input string: "
foo")
Validated.catchNonFatal(sys.error("Badness"))
// res5: cats.data.Validated[Throwable,Nothing] = Invalid(java.
lang.RuntimeException: Badness)
Validated.fromTry(scala.util.Try("foo".toInt))
// res6: cats.data.Validated[Throwable,Int] = Invalid(java.lang.
NumberFormatException: For input string: "foo")
Validated.fromEither[String, Int](Left("Badness"))
Validated.fromOption[String, Int](None, "Badness")

6.4. VALIDATED 161
6.4.2 Combining Instances of Validated
We can combine instances of Validated using any of the methods de-

scribed above: product, map2..22, cartesian builder syntax, and so
on.
All of these techniques require an appropriate Cartesian to be in

scope. As with Either, we need to x the error type to create a type
constructor with the correct number of parameters for Cartesian:
type AllErrorsOr[A] = Validated[String, A]
Validated accumulates errors using a Semigroup, so we need one of

those in scope to summon the Cartesian. If we dont have one we get
an annoyingly unhelpful compila on error:
Cartesian[AllErrorsOr]
parameter instance: cats.Cartesian[AllErrorsOr]
// Cartesian[AllErrorsOr]
// ^
Once we import a Semigroup[String], everything works as expected:
Cartesian[AllErrorsOr]
// res10: cats.Cartesian[AllErrorsOr] = cats.data.
ValidatedInstances$$anon$1@5e0fe145
As long as the compiler has all the implicits in scope to summon a

Cartesian of the correct type, we can use cartesian builder syntax or
any of the other Cartesian methods to accumulate errors as we like:
(
"Error 1".invalid[Int] |@|
"Error 2".invalid[Int]
).tupled
// res11: cats.data.Validated[String,(Int, Int)] = Invalid(Error
1Error 2)
As you can see, String isnt an ideal type for accumula ng errors. We
commonly use Lists or Vectors instead:
(
Vector(404).invalid[Int] |@|
Vector(500).invalid[Int]
).tupled
// res12: cats.data.Validated[scala.collection.immutable.Vector[
Int],(Int, Int)] = Invalid(Vector(404, 500))
The cats.data package also provides the NonEmptyList and

NonEmptyVector types that prevent us failing without at least one
error:
import cats.data.NonEmptyVector
(
NonEmptyVector.of("Error 1").invalid[Int] |@|
NonEmptyVector.of("Error 2").invalid[Int]
).tupled
// res13: cats.data.Validated[cats.data.NonEmptyVector[String],(
Int, Int)] = Invalid(NonEmptyVector(Error 1, Error 2))
6.4.3 Methods of Validated
Validated comes with a suite of methods that closely resem-

ble those available for Either, including the methods from
6.4. VALIDATED 163
[cats.syntax.either]. We can use map, leftMap, and bimap

to transform the values inside the valid and invalid sides:
123.valid.map(_ * 100)
// res14: cats.data.Validated[Nothing,Int] = Valid(12300)
"?".invalid.leftMap(_.toString)
// res15: cats.data.Validated[String,Nothing] = Invalid(?)
123.valid[String].bimap(_ + "!", _ * 100)

// res16: cats.data.Validated[String,Int] = Valid(12300)
"?".invalid[Int].bimap(_ + "!", _ * 100)

// res17: cats.data.Validated[String,Int] = Invalid(?!)
We cant flatMap because Validated isnt a monad. However, we

can convert back and forth between Validated and Either using the
toEither and toValidated methods. This allows us to switch error-
handling seman cs on the y. Note that toValidated comes from
[cats.syntax.either]:
import cats.syntax.either._ // toValidated method

// import cats.syntax.either._
"Badness".invalid[Int]
"Badness".invalid[Int].toEither
// res19: Either[String,Int] = Left(Badness)
"Badness".invalid[Int].toEither.toValidated
As with Either, we can use the ensure method to fail with a specied
error if a predicate does not hold:
// 123.valid[String].ensure("Negative!")(_ > 0)
Finally, we can call getOrElse or fold to extract values from the Valid
and Invalid cases:
"fail".invalid[Int].getOrElse(0)
// res22: Int = 0
"fail".invalid[Int].fold(_ + "!!!", _.toString)

// res23: String = fail!!!
6.4.4 Exercise: Form Valida on
Lets get used to Validated by implemen ng a simple HTML registra-

on form. We receive request data from the client in a Map[String,
String] and we want to parse it to create a User object:
case class User(name: String, age: Int)
Our goal is to implement code that parses the incoming data enforcing
the following rules:
the name and age must be specied;

the name must not be blank;
the the age must be a valid non-nega ve integer.
If all the rules pass, our parser we should return a User. If any rules fail,
we should return a List of the error messages.
To implement this complete example well need to combine rules in se-
quence and in parallel Well use Either to combine computa ons in
sequence using fail-fast seman cs, and Validated to combine them in
parallel using accumula ng seman cs.
Lets start with some sequen al combina on. Well dene two methods
to read the "name" and "age" elds:
6.5. APPLY AND APPLICATIVE 165
readName will take a Map[String, String] parameter, extract

the "name" eld, check the relevant valida on rules, and return
an Either[List[String, String]];
readAge will take a Map[String, String] parameter, extract

the "age" eld, check the relevant valida on rules, and return
an Either[List[String, Int].
Well build these methods up from smaller building blocks. Start by

dening a method getValue that reads a String from the Map given a
eld name.
See the solu on
Next dene a method parseInt that consumes an Int and parses it as
a String.
See the solu on
Next implement the valida on checks: nonBlank to check Strings,
and nonNegative to check Ints.
See the solu on
Now combine getValue, parseInt, nonBlank and nonNegative to
create readName and readAge:
See the solu on
Finally, use a Cartesian to combine the results of readName and
readAge to produce a User. Make sure you switch from Either to
Validated to accumulate errors.
See the solu on
6.5 Apply and Applica ve
Cartesians arent men oned frequently in the wider func onal pro-
gramming literature. They provide a subset of the func onality of a
related type class called an applica ve functor (applica ve for short).

Cartesians and applica ves eec vely provide alterna ve encodings
of the no on of zipping values. Both encodings are introduced in the
same 2008 paper by Conor McBride and Ross Paterson.
Cats models Applicatives using two type classes. The rst, Apply,
extends Cartesian and Functor and adds an ap method that applies
a parameter to a func on within a context. The second, Applicative
extends Apply, adds the pure method introduced in Chapter 4. Heres
a simplied deni on in code:
trait Apply[F[_]] extends Cartesian[F] with Functor[F] {

def ap[A, B](ff: F[A => B])(fa: F[A]): F[B]
def product[A, B](fa: F[A], fb: F[B]): F[(A, B)] =

ap(map(fa)(a => (b: B) => (a, b)))(fb)
}
trait Applicative[F[_]] extends Apply[F] {

def pure[A](a: A): F[A]
}
Breaking this down, the ap method applies a parameter fa to a func on

ff within a context F[_]. The product method from Cartesian is
dened in terms of ap and map.
Dont worry too much about the implementa on of productits di-

cult to read and the details arent par cuarly important. The main point
is that there is a ght rela onship between product, ap, and map that
allows any one of them to be dened in terms of the other two.
Applicative also introduces the pure method. This is the same pure
we saw in Monad. It constructs a new applica ve instance from an
unwrapped value. In this sense, Applicative is related to Apply as
Monoid is related to Semigroup.
6.5. APPLY AND APPLICATIVE 167
6.5.1 The Hierarchy of Sequencing Type Classes
With the introduc on of Apply and Applicative, we can zoom out

and see a a whole family of type classes that concern themselves with
sequencing computa ons in dierent ways. Figure 6.1 shows the big
picture.
Figure 6.1: Monad type class hierarchy
Each type class in the hierarchy represents a par cular set of sequenc-
ing seman cs. It introduces its characteris c methods, and denes all
of the func onality from its supertypes in terms of them. Every monad
is an applica ve, every applica ve a cartesian, and so on.
Because of the lawful nature of the rela onships between the type
classes, the inheritance rela onships are constant across all instances
of a type class. Apply denes product in terms of ap and map; Monad
denes product, ap, and map, in terms of pure and flatMap.
To illustrate this lets consider two hypothe cal data types:
Foo is a monad. It has an instance of the Monad type class that

implements pure and flatMap and inherits standard deni ons
of product, map, and ap;
Bar is an applica ve functor. It has an instance of Applicative

that implements pure and ap and inherits standard deni ons
of product and map.
What can we say about these two data types without knowing more
about their implementa on?
We know strictly more about Foo than Bar, Monad is a subtype

of Applicative, so we can guarantee proper es of Foo (namely
flatMap) that we cannot guarantee with Bar. Conversely, we know
that Bar may have a wider range of behaviours than Foo. It has fewer
laws to obey (no flatMap), so it can implement behaviours that Foo
cannot.
This demonstrates the classic trade-o of power (in the mathema cal
sense) versus constraint. The more constraints we place on a data type,
the more guarantees we have about its behaviour, but the fewer be-
haviours we can model.
Monads happen to be a sweet spot in this trade-o. They are exible

enough to model a wide range of behaviours and restric ve enough
to give strong guarantees about those behaviours. However, there are
situa ons where monads arent the right tool for the job. Some mes
we want thai food, and burritos just wont sa sfy.
Whereas monads impose a strict sequencing on the computa ons they

model, applica ves and cartesians impose no such restric on. This puts
them in another sweet spot in the hierarchy. We can use them to rep-
resent classes of parallel / independent computa ons that monads can-
not.
We choose our seman cs by choosing our data structures. If we choose

a monad, we get strict sequencing. If we choose an applica ve, we lose
the ability to flatMap. This is the trade-o enforced by the consistency
laws. So choose your types carefully, friend!
6.6. SUMMARY 169
6.6 Summary
While monads and functors are the most widely used sequencing data
types weve covered in this book, cartesians and applica ves are the
most general. These type classes provide a generic mechanism to com-
bine values and apply func ons within a context, from which we can
fashion monads and a variety of other combinators.
Cartesians and applica ves are most commonly used as a means of com-
bining independent values such as the results of valida on rules. Cats
provides the Validated type for this specic purpose, along with carte-
sian builder syntax as a convenient way to express the combina on of
rules.
We have almost covered all of the func onal programming concepts

on our agenda for this book. The next chapter covers Traverse and
Foldable, two powerful type classes for conver ng between data
types. A er that well look at several case studies that bring together
all of the concepts covered.
Chapter 7
Foldable and Traverse
In this chapter well look at two type classes that capture itera on over
collec ons:
Foldable abstracts the familiar foldLeft and foldRight oper-

a ons;
Traverse is a higher-level abstrac on that uses Applicatives
to iterate with less pain than with folds.
Well start by looking at Foldable, and then examine cases where fold-
ing becomes complex and Traverse becomes convenient.
7.1 Foldable
The Foldable type class captures the foldLeft and foldRight meth-
ods were used to in sequences like Lists, Vectors, and Streams. Us-
ing Foldable, we can write generic folds that work with a variety of
sequence types. We can also invent new sequences and plug them
into our code. Foldable gives us great use cases for Monoids and the
Eval monad.
171
172 CHAPTER 7. FOLDABLE AND TRAVERSE
7.1.1 Folds and Folding
Lets start with a quick recap on the general concept of folding.

Foldable is a type class for folding over sequences. We supply an
accumulator value and a binary func on to combine it with an item in
the sequence:
def show[A](list: List[A]): String =

list.foldLeft("nil")((accum, item) => s"$item then $accum")
show(Nil)
// res0: String = nil
show(List(1, 2, 3))
// res1: String = 3 then 2 then 1 then nil
The view provided by Foldable is recursive. Our binary func on is

called repeatedly for each item in the sequence, result from each call
becoming the accumulator for the next. When we reach the end of the
sequence, the nal accumulator becomes our result.
Depending on the opera on were performing, the order in which we

fold may be important. Because of this there are two standard variants
of fold:
foldLeft traverses from le to right (start to nish);

foldRight traverses from right to le (nish to start).
Figure 7.1 illustrates each direc on.
foldLeft and foldRight are equivalent if our binary opera on is com-

muta ve. For example, we can sum a List[Int] by folding in either
direc on, using 0 as our accumulator and + as our opera on:
7.1. FOLDABLE 173
1 1
2 2
3
3
0 + 1
3 + 0
1 + 2 2 + 3
3 + 3 1 + 5
6 6
Figure 7.1: Illustra on of foldLe and foldRight
List(1, 2, 3).foldLeft(0)(_ + _)
// res2: Int = 6
List(1, 2, 3).foldRight(0)(_ + _)
// res3: Int = 6
If provide a non-commuta ve operator the order of evalua on makes

a dierence. For example, if we fold using -, we get dierent results in
each direc on:
List(1, 2, 3).foldLeft(0)(_ - _)
// res4: Int = -6
List(1, 2, 3).foldRight(0)(_ - _)
// res5: Int = 2
7.1.2 Exercise: Reec ng on Folds
Try using foldLeft and foldRight with an empty list as the accumu-
lator and :: as the binary operator. What results do you get in each
case?
See the solu on

7.1.3 Exercise: Scaf-fold-ing other methods
foldLeft and foldRight are very general methods. We can use them
to implement many of the other high-level sequence opera ons we
know. Prove this to yourself by implemen ng subs tutes for List's
map, flatMap, filter, and sum methods in terms of foldRight.
See the solu on
7.1.4 Foldable in Cats
Cats Foldable abstracts foldLeft and foldRight into a type class.

Instances of Foldable dene these two methods and inherit a host
of derived methods for free. Cats provides out-of-the-box instances
of Foldable for a handful of Scala data types: List, Vector, Stream,
Option, and Map.
We can summon instances as usual using Foldable.apply and call

their implementa ons of foldLeft directly. Here is an example using
List:
import cats.Foldable
val ints = List(1, 2, 3)
Foldable[List].foldLeft(ints, 0)(_ + _)
// res1: Int = 6
Other sequences like Vector and Stream work in the same way. Here
is an example using Option, which is treated like a sequence of 0 or 1
elements:
val maybeInt = Option(123)

7.1. FOLDABLE 175
Foldable[Option].foldLeft(maybeInt, 10)(_ * _)
// res3: Int = 1230
Finally, here is an example for Map. The Foldable instance folds over
the values in the map (as opposed to its keys). Map has two type param-
eters so we have to x the key type to summon the Foldable:
import cats.instances.map._
type StringMap[A] = Map[String, A]
val stringMap = Map("a" -> "b", "c" -> "d")
Foldable[StringMap].foldLeft(stringMap, "nil")(_ + "," + _)

// res6: String = nil,b,d
7.1.4.1 Folding Right
Foldable denes foldRight dierently to foldLeft, in terms of the

Eval monad:
def foldRight[A, B](fa: F[A], lb: Eval[B])(f: (A, Eval[B]) =>

Eval[B]): Eval[B]
Using Eval means folding is always stack safe, even when the collec-
ons default deni on of foldRight is not. For example, the default
implementa on of foldRight for Stream is not stack safe. The longer
the stream, the larger the stack requirements for the fold. A suciently
large stream will trigger a StackOverflowException:
import cats.Eval
def bigData = (1 to 100000).toStream
bigData.foldRight(0)(_ + _)
// java.lang.StackOverflowError ...
Using Foldable forces us to use stack safe opera ons, which xes the
overow excep on:
import cats.instances.stream._
val eval = Foldable[Stream].

foldRight(bigData, Eval.now(0)) { (num, eval) =>
eval.map(_ + num)
}
// eval: cats.Eval[Int] = cats.Eval$$anon$8@5fa48dad
eval.value
// res10: Int = 705082704
Stack Safety in the Standard Library
Stack safety isnt typically an issue when using the standard li-
brary. The most commonly used collec on types, such as List
and Vector, provide stack safe implementa ons of foldRight:
(1 to 100000).toList.foldRight(0)(_ + _)
// res11: Int = 705082704
(1 to 100000).toVector.foldRight(0)(_ + _)
// res12: Int = 705082704
Weve called out Stream because it is an excep on to this rule.

Whatever data type were using, though, its useful to know that
Eval has our backs.
7.1.4.2 Folding with Monoids
Foldable provides us with a host of useful methods dened on top of

foldLeft. Many of these are facimiles of familiar methods from the
7.1. FOLDABLE 177
standard library: find, exists, forall, toList, isEmpty, nonEmpty,

and so on:
Foldable[Option].nonEmpty(Option(42))
Foldable[List].find(List(1, 2, 3))(_ % 2 == 0)
In addi on to these familiar methods, Cats provides two methods that

make use of Monoids:
combineAll (and its alias fold) combines all elements in the se-
quence using their Monoid;
foldMap maps a user-supplied func on over the sequence and

combines the results using a Monoid.
For example, we can use combineAll to sum over a List[Int]:
import cats.instances.int._ // Monoid for Int
Foldable[List].combineAll(List(1, 2, 3))
// res15: Int = 6
Alterna vely, we can use foldMap to convert each Int to a String and
concatenate them:
import cats.instances.string._ // Monoid for String
Foldable[List].foldMap(List(1, 2, 3))(_.toString)
Finally, we can compose Foldables to support deep traversal of nested

sequences:
import cats.instances.vector._ // Monoid of Vector
val ints = List(Vector(1, 2, 3), Vector(4, 5, 6))
(Foldable[List] compose Foldable[Vector]).combineAll(ints)

// res18: Int = 21
7.1.4.3 Syntax for Foldable
Every method in Foldable is available in syntax form via cats.syntax.foldable.

In each case, the rst argument to the method on Foldable becomes
the receiver of the method call:
import cats.syntax.foldable._
List(1, 2, 3).combineAll
// res19: Int = 6
List(1, 2, 3).foldMap(_.toString)
Explicits over Implicits
Remember that Scala will only use an instance of Foldable if the

method isnt explicitly available on the receiver. For example, the
following code will use the version of foldLeft dened on List:
List(1, 2, 3).foldLeft(0)(_ + _)
// res21: Int = 6
whereas the following generic code will use Foldable:

7.2. TRAVERSE 179
def sum[F[_]: Foldable](values: F[Int]): Int =

values.foldLeft(0)(_ + _)
// sum: [F[_]](values: F[Int])(implicit evidence$1: cats.
Foldable[F])Int
We typically dont need to worry about this dis nc on. Its a

feature! We call the method we want and the compiler uses a
Foldable when needed to ensure our code works as expected.
If we need a stack-safe implementa on of foldRight, simply us-
ing Eval as the accumulator is enough to force the compiler to
select the method from Cats.
7.2 Traverse
foldLeft and foldRight are exible itera on methods but they re-
quire us to do a lot of work to dene accumulators and combinator
func ons. The Traverse type class is a higher level tool that leverages
Applicatives to provide a more convenient, more lawful, pa ern for
itera on.
7.2.1 Traversing with Futures
We can demonstrate Traverse using the Future.traverse and

Future.sequence methods in the Scala standard library. These
methods provide Future-specic implementa ons of the traverse
pa ern. As an example, suppose we have a list of server hostnames
and a method to poll a host for its up me:
val hostnames = List(

"alpha.example.com",
"beta.example.com",
"gamma.demo.com"
)
def getUptime(hostname: String): Future[Int] =

Future(hostname.length * 60) // just for demonstration
Now, suppose we want to poll all of the hosts and collect all of their
up mes. We cant simply map over hostnames because the resulta
List[Future[Int]]would contain more than one Future. We need
to reduce the results to a single Future to get something we can block
on. Lets start by doing this manually using a fold:
val allUptimes: Future[List[Int]] =

hostnames.foldLeft(Future(List.empty[Int])) {
(accum, host) =>
val uptime = getUptime(host)
for {
accum <- accum
uptime <- uptime
} yield accum :+ uptime
}
Await.result(allUptimes, 1.second)
// res2: List[Int] = List(1020, 960, 840)
Intui vely, we iterate over hostnames, call func for each item, and com-
bine the results into a list. This sounds simple, but the code is fairly
unwieldy because of the need to create and combine Futures at every
itera on. We can improve on things greatly using Future.traverse,
which is tailor made for this pa ern:
7.2. TRAVERSE 181
val allUptimes: Future[List[Int]] =

Future.traverse(hostnames)(getUptime)
Await.result(allUptimes, 1.second)
// res3: List[Int] = List(1020, 960, 840)
This is much clearer and more conciselets see how it works. If we

ignore distrac ons like CanBuildFrom and ExecutionContext, the im-
plementa on of Future.traverse in the standard library looks like
this:
object Future {
def traverse[A, B](values: List[A])
(func: A => Future[B]): Future[List[B]] =
values.foldLeft(Future(List.empty[A])) { (accum, host) =>
val item = func(host)
for {
accum <- accum
item <- item
} yield accum :+ item
}
}
This is essen ally the same as our example code above. Future.traverse
is abstrac ng away the pain of folding and dening accumulators and
combina on func ons. It gives us a clean high-level interface to do
what we want:
start with a List[A];

provide a func on A => Future[B];
end up with a Future[List[B]].
The standard library also provides another method, Future.sequence,

that assumes were star ng with a List[Future[B]] and dont need
to provide an iden ty func on:
object Future {
def sequence[B](futures: List[Future[B]]): Future[List[B]] =
traverse(futures)(identity)
// etc...
}
In this case the intui ve understanding is even simpler:
start with a List[Future[A]];

end up with a Future[List[A]].
Future.traverse and Future.sequence solve a very specic prob-

lem: they allow us to iterate over a sequence of Futures and accu-
mulate a result. The simplied examples above only work with Lists,
but the real Future.traverse and Future.sequence work with any
standard Scala collec on.
Cats Traverse type class generalises these pa erns to work with any
type of eect: Future, Option, Validated, and so on. Well ap-
proach Traverse in the next sec ons in two steps: rst well generalise
over the eect type, then well generalise over the sequence type. Well
end up with an extremely valuable tool that trivialises many opera ons
involving sequences and other data types.
7.2.2 Traversing with Applica ves
If we squint, well see that we can rewrite traverse in terms of an

Applicative. Our accumulator from the example above:
Future(List.empty[Int])
is equivalent to Applicative.pure:
7.2. TRAVERSE 183
import cats.Applicative
List.empty[Int].pure[Future]
Our combinator, which used to be this:

def oldCombine(
accum : Future[List[Int]],
host : String
): Future[List[Int]] = {
val uptime = getUptime(host)
for {
accum <- accum
uptime <- uptime
} yield accum :+ uptime
}
is now equivalent to Cartesian.combine:

// Combining an accumulator and a hostname using an Applicative:

def newCombine(
accum: Future[List[Int]],
host: String
): Future[List[Int]] =
(accum |@| getUptime(host)).map(_ :+ _)
By subs tu ng these snippets back into the deni on of traverse we

can generalise it to to work with any Applicative:
def listTraverse[F[_] : Applicative, A, B]

(list: List[A])(func: A => F[B]): F[List[B]] =
list.foldLeft(List.empty[B].pure[F]) { (accum, item) =>
(accum |@| func(item)).map(_ :+ _)
def listSequence[F[_] : Applicative, B]

(list: List[F[B]]): F[List[B]] =
listTraverse(list)(identity)
We can use this new listTraverse to re-implement our up me exam-

ple:
Await.result(
listTraverse(hostnames)(getUptime),
1.second
)
// res11: List[Int] = List(1020, 960, 840)
or we can use it with with other Applicative data types as shown in

the following exercises.
7.2.2.1 Exercise: Traversing with Vectors
What is the result of the following?
listSequence(List(Vector(1, 2), Vector(3, 4)))
See the solu on
What about a list of three parameters?
listSequence(List(Vector(1, 2), Vector(3, 4), Vector(5, 6)))
See the solu on

7.2. TRAVERSE 185
7.2.2.2 Exercise: Traversing with Op ons
Heres an example that uses Options:
def process(inputs: List[Int]) =

listTraverse(inputs)(n => if(n % 2 == 0) Some(n) else None)
What is the return type of this method? What does it produce for the
following inputs?
process(List(2, 4, 6))
See the solu on
7.2.2.3 Exercise: Traversing with Validated
Finally, geres an example that uses Validated:
import cats.instances.list._ // Applicative[ErrorsOr] needs a
Monoid[List]
type ErrorsOr[A] = Validated[List[String], A]
def process(inputs: List[Int]): ErrorsOr[List[Int]] =

listTraverse(inputs) { n =>
if(n % 2 == 0) {
Validated.valid(n)
} else {
Validated.invalid(List(s"$n is not even"))
}
}
What does this method produce for the following inputs?

See the solu on
7.2.3 Traverse in Cats
Our listTraverse and listSequence methods work with any type

of Applicative eect, but they only work with one type of sequence:
List. We can generalise over dierent sequence types using a type
class, which brings us to Cats Traverse. Heres the abbreviated de-
ni on:
package cats
trait Traverse[F[_]] {
def traverse[G[_] : Applicative, A, B](inputs: F[A])(func: A
=> G[B]): G[F[B]]
def sequence[G[_] : Applicative, B](inputs: F[G[B]]): G[F[B]]

=
traverse(inputs)(func)
}
Cats provides instances OF Traverse for List, Vector, Stream,

Option, Either, and a variety of other types. We can summon in-
stances as usual using Traverse.apply as usual and use the traverse
and sequence methods as described in the previous sec on:
import cats.Traverse
Await.result(
Traverse[List].traverse(hostnames)(getUptime),
1.second
7.2. TRAVERSE 187
)
// res0: List[Int] = List(1020, 960, 840)
val numbers = List(Future(1), Future(2), Future(3))
Await.result(
Traverse[List].sequence(numbers),
1.second
)
There are also syntax versions of the methods, imported via

cats.syntax.traverse:
import cats.syntax.traverse._
Await.result(hostnames.traverse(getUptime), 1.second)
// res2: List[Int] = List(1020, 960, 840)
Await.result(numbers.sequence, 1.second)
As you can see, this is much more compact and readable than the
foldLeft code we started with earlier this chapter!
7.2.4 Unapply, traverseU, and sequenceU
One frequent problem people encounter when using Traverse is that

it doesnt play well with eects with two or more type parameters. For
example, suppose we have a List of Eithers:
val eithers: List[Either[String, String]] = List(

Right("Wow!"),
Right("Such cool!")
When we call sequence we get a compile error:
eithers.sequence
// <console>:20: error: Cannot prove that Either[String,String]
<:< G[A].
// eithers.sequence
// ^
The reason for this failure is that the compiler cant nd an implicit
Applicative. This isnt a problem in our codewe have the correct
syntax and instances in scopeits simply a weakness of Scalas type in-
ference that has only recently been xed (more on the x in a moment).
To understand whats going on, lets look again at the deni on of
sequence:
trait Traverse[F[_]]
def sequence[G[_]: Applicative, B]: G[F[B]] =
// etc...
}
To compile a call like eithers.sequence, the compiler has to nd val-

ues for the type parameters G and B. The types it is a emp ng to unify
them with are Either[String, Int] and Int, so it has to make a de-
cision about which parameter on Either to x to create a type con-
structor of the correct shape.
There are two possible solu ons as you can see below:
type G[A] = Either[A, Int]

type G[A] = Either[String, A]
Its obvious to us which unica on method to choose. However, prior

to Scala 2.12, an infamous compiler limita on called SI-2712 prevented
this inferrence.
7.3. SUMMARY 189
To work around this issue Cats provides a u li y type class called

Unapply, whose purpose is to tell the compiler which parameters to
x to create a unary type constructor for a given type. Cats provides
instances of Unapply for the common binary types: Either, Either,
Validated, and Function1, and so on. Traverse provides variants of
traverse and sequence called traverseU and sequenceU that use
Unapply to guide the compiler to the correct solu on:
eithers.sequenceU
// res2: scala.util.Either[String,List[String]] = Right(List(Wow
!, Such cool!))
The inner workings of Unapply arent par cularly important all we

need to know is that this tool is available to x these kinds of problems.
Fixes to SI-2712
SI-2712 is xed in Lightbend Scala 2.12.1 and Typelevel Scala

2.11.8. The x allows calls to traverse and sequence to com-
pile in a much wider set of cases, although tools like Unapply are
s ll necessary in certain situa ons.
The SI-2712 x can be backported to Scala 2.11 and 2.10 using

this compiler plugin.
7.3 Summary
In this chapter we were introduced to Foldable and Traverse, two

type classes for itera ng over sequences.
Foldable abstracts the foldLeft and foldRight methods we know
from collec ons in the standaed library. It adds stack-safe implemen-
ta ons of these methods to a handful of extra data types, and denes
a host of situa onally useful addi ons. That said, Foldable doesnt
introduce much that we didnt already know.
The real power comes from Traverse, which abstracts and generalises
the traverse and sequence methods we know from Future. Using
these methods we can turn an F[G[A]] into a G[F[A]] for any F with
an instance of Traverse and any G with an instance of Applicative.
In terms of the reduc on we get in lines of code, Traverse is one of
the most powerful pa erns in this book. We can reduce folds of many
lines down to a single foo.traverse.
Finally we looked at the Unapply type class, which works around re-
stric ons in the compiler and allows us to use methods like traverse
with types that have mul ple type parameters. Fixes in recent releases
of Scala make Unapply less important than it once was, but will s ll be
a necessity in many Scala versions to come.
and with that, weve nished all of the theory in this book. Theres
plenty more to come, though, as we put everything weve learned into
prac ce in a series of in-depth case studies in part 2!
Part II
Case Studies
191
Chapter 8
Case Study: Tes ng

Asynchronous Code
Well start with a simple case study: how to simplify unit tests for asyn-
chronous code by making them synchronous.
Lets return to the example from Chapter 7 where were measuring

the up me on a set of servers. Well esh out the code into a more
complete structure. There will be two components. The rst is an
UptimeClient that polls remote servers for their up me:
trait UptimeClient {
def getUptime(hostname: String): Future[Int]
}
Well also have an UptimeService that maintains a list of servers and

allows the user to poll them for their total up me:
193
194 CHAPTER 8. CASE STUDY: TESTING ASYNCHRONOUS CODE
class UptimeService(client: UptimeClient) {

def getTotalUptime(hostnames: List[String]): Future[Int] =
hostnames.traverse(client.getUptime).map(_.sum)
}
Weve modelled UptimeClient as a trait because were going to want

to stub it out in unit tests. For example, we can write a test client that al-
lows us to provide dummy data rather than calling out to actual servers:
class TestUptimeClient(hosts: Map[String, Int]) extends

UptimeClient {
def getUptime(hostname: String): Future[Int] =
Future.successful(hosts.getOrElse(hostname, 0))
}
Now, suppose were wri ng unit tests for UptimeService. We want

to test its ability to sum values, regardless of where it is ge ng them
from. Heres an example:
def testTotalUptime() = {
val hosts = Map("host1" -> 10, "host2" -> 6)
val client = new TestUptimeClient(hosts)
val service = new UptimeService(client)
val actual = service.getTotalUptime(hosts.keys.toList)
val expected = hosts.values.sum
assert(actual == expected)
}
// <console>:31: warning: scala.concurrent.Future[Int] and Int
are unrelated: they will most likely never compare equal
// assert(actual == expected)
// ^
// error: No warnings can be incurred under -Xfatal-warnings.
8.1. ABSTRACTING OVER TYPE CONSTRUCTORS 195
The code doesnt compile because weve made a classic error. We

forgot that our applica on code is asynchronous. Our actual result is
of type Future[Int] and out expected result is of type Int. We cant
compare them directly!
There are a couple of ways to solve this problem. We could alter our
test code to accommodate the asynchronousness. However, there is
another alterna ve. Lets make our service code synchronous so our
test works without modica on!
8.1 Abstrac ng over Type Constructors
We need to implement two versions of UptimeClient: an asyn-

chronous one for use in produc on and a synchronous one for use in
our unit tests:
trait RealUptimeClient extends UptimeClient {
}
trait TestUptimeClient extends UptimeClient {

def getUptime(hostname: String): Int
}
The ques on is: what result type should we give to the abstract method
in UptimeClient? We need to abstract over Future[Int] and Int:
trait UptimeClient {
def getUptime(hostname: String): ???
}
At rst this may seem dicult. We want to retain the Int part from
each type but throw away the Future part in the test code. Fortu-
nately, Cats provides a solu on in terms of the iden ty type, Id, that
Technically this is a warning not an error. It has been promoted to an error in our
case because were using the -Xfatal-warnings ag on scalac.
we discussed way back in Sec on 4.3. Id allows us to wrap types in

a type constructor without changing their meaning:
package cats
type Id[A] = A
Id allows us to abstract over them in UptimeClient. Implement this

now:
write a trait deni on for UptimeClient that accepts a type con-

structor F[_] as a parameter;
extend it with two traits, RealUptimeClient and TestUptimeClient,

that bind F to Future and Id respec vely;
write out the method header for getUptime in each case to ver-
ify that it compiles.
See the solu on
You should now be able to esh your deni on of TestUptimeClient

out into a full class based on a Map[String, Int] as before.
See the solu on
8.2 Abstrac ng over Monads
Lets turn our a en on to UptimeService. We need to rewrite it to

abstract over the two types of UptimeClient. Well do this in two
stages: rst well get the class and method headers compiling, then
well turn our a en on to the method bodies. Star ng with the method
headers:
8.2. ABSTRACTING OVER MONADS 197
comment out the body of getTotalUptime (replace it with ???

to make everything compile);
add a type parameter F[_] to UptimeService and pass it on to

UptimeClient.
See the solu on

Now uncomment the body of getTotalUptime. You should get a com-
pila on error similar to the following:

// evidence parameter of type cats.Applicative[F]
// hostnames.traverse(client.getUptime).map(_.sum)
// ^
The problem here is that traverse only works on sequences of values

that have an Applicative. In our original code we were traversing a
List[Future[Int]]. There is an applica ve for Future so that was
ne. In this version we are traversing a List[F[Int]]. We need to
prove to the compiler that F has an Applicative. Do this by adding an
implicit constructor parameter to UptimeService.
See the solu on
Finally, lets turn our a en on to our unit tests. Our test code now
works as intended without any modica on. We create an instance of
TestUptimeClient and wrap it in an UptimeService. This eec vely
binds F to Id, allowing the rest of the code to operate synchronously
without worrying about monads or applica ves:
def testTotalUptime() = {
val hosts = Map("host1" -> 10, "host2" -> 6)
val client = new TestUptimeClient(hosts)
val service = new UptimeService(client)
val actual = service.getTotalUptime(hosts.keys.toList)
val expected = hosts.values.sum
assert(actual == expected)
testTotalUptime()
8.3 Conclusions
This case study provides a nice introduc on to how Cats can help
us abstract over dierent computa onal scenarios. We used the
Applicative type class to abstract over asynchronous and syn-
chronous code. Leaning on a func onal abstrac on allows us to
specify the sequence of computa ons we want to perform without
worrying about the details of the implementa on.
Back in Figure 6.1, we showed a stack of computa onal type classes

that are meant for exactly this kind of abstrac on. Type classes like
Functor, Applicative, Monad, and Traverse provide abstract imple-
menta ons of pa erns such as mapping, zipping, sequencing, and it-
era on. The mathema cal laws on those types ensure that they work
together with a consistent set of seman cs.
We used Applicative in this case study because it was the least

powerful type class that did what we needed. If we had required
flatMap, we could have swapped out Applicative for Monad. If we
had needed to abstract over dierent sequence types, we could have
used Traverse. There are also type classes like ApplicativeError
and MonadError that help model failures as well as successful
computa ons.
Lets move on now to a more complex case study where type classes
will help us produce something more interes ng: a map-reduce-style
framework for parallel processing.
Chapter 9
Case Study: Pygmy Hadoop
In this case study were going to implement a simple-but-powerful par-

allel processing framework using Monoids, Functors, and a host of
other goodies.
If you have used Hadoop or otherwise worked in big data you will
have heard of MapReduce, which is a programming model for doing
parallel data processing across clusters tens or hundreds of machines
(aka nodes). As the name suggests, the model is built around a map
phase, which is the same map func on we know from Scala and the
Functor type class, and a reduce phase, which we usually call fold in
Scala.
9.1 Parallelizing map and fold
Recall the general signature for map is to apply a func on A => B to a

F[A], returning a F[B]:
In Hadoop there is also a shue phase that we will ignore here.
199
200 CHAPTER 9. CASE STUDY: PYGMY HADOOP
map
F[A] A => B F[B]
Figure 9.1: Type chart: functor map
map transforms each individual element in a sequence independently.

We can easily parallelize map because there are no dependencies be-
tween the transforma ons applied to dierent elements (the type sig-
nature of the func on A => B shows us this, assuming we dont use
side-eects not reected in the types).
What about fold? We can implement this step with an instance of
Foldable. Not every functor also has an instance of foldable, but we
can implement a map reduce system on top of any data type that has
both of these type classes. Our reduc on step becomes a foldLeft
over the results of the distributed map.
foldLeft ,
F[A] B (B, A) => B B
Figure 9.2: Type chart: fold
If you remember from our discussion of Foldable, then depending on

the reduc on opera on we use, the order of combina on can have
eect on the nal result. To remain correct we need to ensure our re-
duc on opera on is associa ve:
reduce(a1, reduce(a2, a3)) == reduce(reduce(a1, a2), a3)
If we have associa vity, we can arbitrarily distribute work between our

nodes provided we preserve the ordering on the sequence of elements
were processing.
9.2. IMPLEMENTING FOLDMAP 201
Our fold opera on requires us to seed the computa on with an element

of type B. Since our fold may be split into an arbitrary number of parallel
steps, the seed should not eect the result of the computa on. This
naturally requires the seed to be an iden ty element:
reduce(seed, a1) == reduce(a1, seed) == a1
In summary, our parallel fold will yield the correct results if:
we require the reducer func on to be associa ve;

we seed the computa on with the iden ty of this func on.
What does this pa ern sound like? Thats right, weve come full cir-
cle back to Monoid, the rst type class we discussed in this book. We
are not the rst to recognise the importance of monoids. The monoid
design pa ern for map-reduce jobs is at the core of recent big data
systems such as Twi ers Summingbird.
In this project were going to implement a very simple single-machine

map-reduce. Well start by implemen ng a method called foldMap to
model the data-ow we need.
9.2 Implemen ng foldMap
We saw foldMap briey back when we covered Foldable. It is one

of the derived opera ons that sits on top of foldLeft and foldRight.
However, rather than use Foldable, we will re-implement foldMap
here ourselves as it will provide useful insight into the structure of map
reduce.
Start by wri ng out the signature of foldMap. It should accept the fol-
lowing parameters:
a sequence of type Vector[A];

a func on of type A => B, where there is a Monoid for B;
You will have to add implicit parameters or context bounds to complete

the type signature.
See the solu on
Now implement the body of foldMap. Use the ow chart in Figure 9.3
as a guide to the steps required:
1. start with a sequence of items of type A;

2. map over the list to produce a sequence of items of type B;
3. use the Monoid to reduce the items to a single B.
1. Initial data sequence
2. Map step
3. Fold/reduce step
4. Final result
Figure 9.3: foldMap algorithm
Heres some sample output for reference:

9.3. PARALLELISING FOLDMAP 203
scala foldMap(Vector(1, 2, 3))(identity) // res1: Int =

6scala import cats.instances.string._
// Mapping to a String uses the concatenation monoid:

foldMap(Vector(1, 2, 3))(_.toString + "! ")
// res3: String = "1! 2! 3! "
// Mapping over a String to produce a String:

foldMap("Hello world!".toVector)(_.toString.toUpperCase)
// res5: String = HELLO WORLD!
See the solu on
9.3 Parallelising foldMap
Now we have a working single-threaded implementa on of foldMap,

lets look at distribu ng work to run in parallel. Well use our single-
threaded version of foldMap as a building block.
Well write a mul -CPU implementa on that simulates the way we
would distribute work in a map-reduce cluster as shown in Figure 9.4:
1. we start with an ini al list of all the data we need to process;

2. we divide the data into batches, sending one batch to each CPU;
3. the CPUs run a batch-level map phase in parallel;
4. the CPUs run a batch-level reduce phase in parallel, producing a
local result for each batch;
5. we reduce the results for each batch to a single nal result.
Scala provides some simple tools to distribute work amongst threads.

We could simply use the parallel collec ons library to implement a solu-
on, but lets challenge ourselves by diving a bit deeper and implement-
ing the algorithm ourselves using Futures.
1. Initial data sequence
2. Divide into batches for each CPU
3. Map over the batches in parallel
4. Reduce each batch in parallel
5. Reduce the batches
6. Final result
Figure 9.4: parallelFoldMap algorithm

9.3.1 Futures, Thread Pools, and Execu onContexts
We already know a fair amount about the monadic nature of Futures.

Lets take a moment for a quick recap, and to describe how Scala futures
are scheduled behind the scenes.
Futures run on a thread pool, determined by an implicit ExecutionContext
parameter. Whenever we create a Future, whether through a call to
Future.apply or some other combinator, we must have an implicit
ExecutionContext in scope:
val future1 = Future {

(1 to 100).toList.foldLeft(0)(_ + _)
}
>)
val future2 = Future {

(100 to 200).toList.foldLeft(0)(_ + _)
}
>)
In this example weve imported a ExecutionContext.Implicits.global.

This default context allocates a thread pool with one thread per CPU
in our machine. When we create a Future the ExecutionContext
schedules it for execu on. If there is a free thread in the pool, the
Future starts execu ng immediately. Most modern machines have
at least two CPUs, so in our example it is likely that future1 and
future2 will execute in parellel.
Some combinators create new Futures that schedule work based on

the results of other Futures. The map and flatMap methods, for exam-
ple, schedule computa ons that run as soon as their input values are
computed and a CPU is available:
val future3 = future1.map(_.toString)

// future3: scala.concurrent.Future[String] = Future(<not
completed>)
val future4 = for {

a <- future1
b <- future2
} yield a + b
>)
As we saw in Sec on 7.2, we can convert a List[Future[A]] to a

Future[List[A]] using Future.sequence:
Future.sequence(List(Future(1), Future(2), Future(3)))

// res7: scala.concurrent.Future[List[Int]] = Future(Success(
List(1, 2, 3)))
or an instance of Traverse:
import cats.instances.future._ // Applicative for Future
import cats.instances.list._ // Traverse for List
import cats.syntax.traverse._ // foo.sequence syntax
List(Future(1), Future(2), Future(3)).sequence

// res8: scala.concurrent.Future[List[Int]] = Future(Success(
List(1, 2, 3)))
An ExecutionContext is required in either case. Finally, we can use

Await.result to block on a Future un l a result is available:
Await.result(Future(1), 1.second) // wait forever until a result

arrives
// res9: Int = 1
There are also Monad and Monoid implementa ons for Future available
from cats.instances.future:
import cats.Monad
Monad[Future].pure(42)
import cats.Monoid
Monoid[Future[Int]].combine(Future(1), Future(2))
9.3.2 Dividing Work
Now weve refreshed our memory of Futures, lets look at how we can
divide work into batches. We can query the number of available CPUs
on our machine using an API call from the Java standard library:
Runtime.getRuntime.availableProcessors
// res15: Int = 8
We can par on a sequence (actually anything that implements

Vector) using the grouped method. Well use this to split o batches
of work for each CPU:
(1 to 10).toList.grouped(3).toList
// res16: List[List[Int]] = List(List(1, 2, 3), List(4, 5, 6),
List(7, 8, 9), List(10))
9.3.3 Implemen ng parallelFoldMap
Implement a parallel version of foldMap called parallelFoldMap.

Here is the type signature:
def parallelFoldMap[A, B : Monoid]

(values: Vector[A])
(func: A => B): Future[B] = ???
Use the techniques described above to split the work into batches, one
batch per CPU. Process each batch in a parallel thread. Refer back to
Figure 9.4 if you need to review the overall algorithm.
For bonus points, process the batches for each CPU using your imple-
menta on of foldMap from above.
See the solu on
9.3.4 parallelFoldMap with more Cats
Although we implemented foldMap ourselves above, the method is

also available as part of the Foldable type class we discussed in Sec-
on 7.1.
Reimplement parallelFoldMap using Cats Foldable and Traverseable

type classes.
See the solu on
9.4 Summary
In this case study we implemented a system that imitates map-reduce

as performed on a cluster. Our algorithm followed three steps:
1. batch the data and send one batch to each node;

2. perform a local map-reduce on each batch;
3. combine the results using monoidal addi on.
9.4. SUMMARY 209
9.4.1 Batching Strategies in the Real World
The main bo leneck in real map-reduce is network communica on be-

tween the nodes. To counter this, systems like Hadoop provide mech-
anisms for pre-batching data to limit the communica on required to
distribute work.
Our toy system is designed to emulate this real-world batching be-
haviour. However, in reality we are running all of our work on a single
machine where communca on between nodes is negligable. We dont
actually need to pre-batch data to gain ecient parallel processing of
a list. We can simply map:
val future1: Future[Vector[Int]] =

(1 to 1000).toVector.
traverse(item => Future(item + 1))
and reduce using a Monoid:
val future2: Future[Int] =

future1.map(_.combineAll)
// res4: Int = 501500
9.4.2 Reduc on using Monoids
Regardless of the batching strategy, mapping and reducing with

Monoids is a powerful and general framework. The core idea of
monoidal addi on underlies Summingbird, Twi ers framework that
powers all their internal data processing jobs.
Monoids are not restricted to simple tasks like addi on and string con-
catena on. Most of the tasks data scien sts perform in their day-to-
day analyses can be cast as monoids. There are monoids for all the
following:
approximate sets such as the Bloom lter;

set cardinality es mators, such as the HyperLogLog algorithm;
vectors and hence vector opera ons like stochas c gradient de-
scent;
quan le es mators such as the t-digest
to name but a few.

Chapter 10
Case Study: Data Valida on
In this case study we will build a library for valida on. What do we mean
by valida on? Almost all programs must check their input meets certain
criteria. Usernames must not be blank, email addresses must be valid,
and so on. This type of valida on o en occurs in web forms, but it could
be performed on congura on les, on web service responses, and any
other case where we have to deal with data that we cant guarantee
is correct. Authen ca on, for example, is just a specialised form of
valida on.
We want to build a library that performs these checks. What design

goals should we have? For inspira on, lets look at some examples of
the types of checks we want to perform:
A user must be over 18 years old or must have parental consent.
A String ID must be parsable as a Int and the Int must corre-

spond to a valid record ID.
A bid in an auc on must apply to one or more items and have a

posi ve value.
211
212 CHAPTER 10. CASE STUDY: DATA VALIDATION
A username must contain at least four characters and all charac-

ters must be alphanumeric.
An email address must contain a single @ sign. Split the string at

the @. The string to the le must not be empty. The string to the
right must be at least three characters long and contain a dot.
With these examples in mind we can state some goals:
We should be able associate meaningful messages with each val-

ida on failure, so the user knows why their data is not valid.
We should be able to combine small checks into larger ones. Tak-

ing the username example above, we should be able to express
this by combining a check of length and a check for alphanumeric
values.
We should be able to transform data while we are checking it.

There is an example above requiring we parse data, changing its
type from String to Int.
Finally, we should be able to accumulate all the failures in one

go, so the user can correct all the issues before resubmi ng.
These goals assume were checking a single piece of data. We will also
need to combine checks across mul ple pieces of data. For a login form,
for example, well need to combine the check results for the username
and the password. This will turn out to be quite a small component of
the library, so the majority of our me will focus on checking a single
data item.
10.1. SKETCHING THE LIBRARY STRUCTURE 213
10.1 Sketching the Library Structure
Lets start at the bo om, checking individual pieces of data. Before we

start coding lets try to develop a feel for what well be building. We
can use a graphical nota on to help us. Well go through our goals one
by one.
Providing error messages
Our rst goal requires us to associate useful error messages with a

check failure. The output of a check could be either the value being
checked, if it passed the check, or some kind of error message. We can
abstactly represent this as a value in a context, where the context is the
possibility of an error message as shown in Figure 10.1.
F[A]
Figure 10.1: A valida on result
A check itself is therefore a func on that transforms a value into a value

in a context as shown in Figure 10.2.
A => F[A]
Figure 10.2: A valida on check
Combine checks
How do we combine smaller checks into larger ones? Is this an applica-

ve or cartesian as shown in Figure 10.3?
|@|
A => F[A] A => F[A] A => F[(A, A)]
Figure 10.3: Applica ve combina on of checks
Not really. With a cartesian, both checks are applied to the same value
and result in a tuple with the value repeated. What we want feels
more like a monoid as shown in Figure 10.4. We can dene a sensi-
ble iden tya check that always passesand two binary combina on
operatorsand and or:
|+|
A => F[A] A => F[A] A => F[A]
Figure 10.4: Monoidal combina on of checks
Well probably be using and and or about equally o en with our valida-
on library and it will be annoying to con nuously switch between two
monoids for combining rules. We consequently wont actually use the
monoid API: well use two separate methods, and and or, instead.
Accumula ng errors as we check
Monoids also feel like a good mechanism for accumula ng error mes-
sages. If we store messages as a List or NonEmptyList, we can even
use a pre-exis ng monoid from inside Cats.
Transforming data as we check it
In addi on to checking data, we also have the goal of transforming it.

This seems like it should be a map or a flatMap depending on whether
the transform can fail or not, so it seems we also want checks to be a
monad as shown in Figure 10.5.
10.2. THE CHECK DATATYPE 215
map
A => F[B] B => C A => F[C]
flatMap
A => F[B] B => (A => F[C]) A => F[C]
Figure 10.5: Monadic combina on of checks
Weve now broken down our library into familiar abstrac ons and are
in a good posi on to begin development.
10.2 The Check Datatype
Our design revolves around a Check, which we said was a func on from
a value to a value in a context. As soon as you see this descrip on you
should think of something like
type Check[A] = A => Either[String, A]
Here weve represented the error message as a String. This is probably

not the best representa on. We may want to accumulate messages in
a List, for example, or even use a dierent representa on that allows
for interna onaliza on or standard error codes.
We could a empt to build some kind of ErrorMessage type that holds

all the informa on we can think of. However, we cant predict the users
requirements. Instead lets let the user specify what they want. We can
do this by adding a second type parameter to Check:
type Check[E, A] = A => Either[E, A]
We will probably want to add custom methods to Check so lets declare

it as a trait instead of a type alias:
trait Check[E, A] {
def apply(value: A): Either[E, A]
// other methods...
}
If you think back to Essen al Scala, there are two func onal program-
ming pa erns that we should consider when dening a trait:
we can make it a typeclass, or;

we can make it an algebraic data type (and hence seal it).
Type classes allow us to unify disparate data types with a common inter-
face. This doesnt seem like what were trying to do here. That leaves
us with an algebraic data type. Lets keep that thought in mind as we
explore the design a bit further.
10.3 Basic Combinators
Lets add some combinator methods to Check, star ng with and. This
method combines two checks into one, succeeding only if both checks
succeed. Think about implemen ng this method now. You should hit
some problems. Read on when you do!
trait Check[E, A] {
def and(that: Check[E, A]): Check[E, A] =
???
10.3. BASIC COMBINATORS 217
// other methods...
}
You should very quickly run into a problem: what do you do when both
checks fail? The correct thing to do is to return both errors, but we
dont currently have any way to combine Es. We need a type class that
abstracts over the concept of accumula ng errors as shown in Figure
10.6
E E => E
List[String] List[String] => List[String]
Figure 10.6: Combining error messages
What type class do we know that looks like this? What method or op-
erator should we use to implement the opera on?
See the solu on
There is another seman c issue that will come up quite quickly: should
and short-circuit if the rst check fails. What do you think the most
useful behavior is?
See the solu on
Use this knowledge to implement and. Make sure you end up with the
behavior you expect!
See the solu on
Strictly speaking, Either[E, A] is the wrong abstrac on for the out-
put of our check. Why is this the case? What other data type could we
use instead? Switch your implementa on over to this new data type.
See the solu on

Our implementa on is looking pre y good now. Implement an or com-
binator to compliment and.
See the solu on
With and and or we can implement many of checks well want in prac-
ce. However, we s ll have a few more methods to add. Well turn to
map and related methods next.
10.4 Transforming Data
One of our requirements is the ability to transform data. This allows us

to support addi onal scenarios like parsing input. In this sec on well
extend our check library with this addi onal func onality.
The obvious star ng point is map. When we try to implement this, we
immediately run into a wall. Our current deni on of Check requires
the input and output types to be the same:
type Check[E, A] = A => Either[E, A]
When we map over a check, what type do we assign to the result? It

cant be A and it cant be B. We are at an impasse:
def map(check: Check[E, A])(func: A => B): Check[E, ???]
To implement map we need to change the deni on of Check. Speci-

cally, we need to a new type variable to separate the input type from
the output:
type Check[E, A, B] = A => Either[E, B]
Checks can now represent opera ons like parsing a String as an Int:
10.4. TRANSFORMING DATA 219
val parseInt: Check[List[String], String, Int] =

// etc...
However, spli ng our input and output types raises another issue. Up
un l now we have operated under the assump on that a Check always
returns its input when succesful. We used this in and and or to ignore
the output of the le and right rules and simply return the original input
on success:
(this(a), that(a)) match {

case And(left, right) =>
(left(a) |@| right(a))
.map((result1, result2) => Right(a))
// etc...
}
In our new formula on we cant return Right(a) because its type is

Either[E, A] not Either[E, B]. Were forced to make an arbitrary
choice between returning Right(result1) and Right(result2).
The same is true of the or method. From this we can derive two things:
we should strive to make the laws we adhere to explicit; and

the code is telling us we have the wrong abstrac on in Check.
10.4.1 Predicates
We can make progress by pulling apart the concept of a predicate, which

can be combined using logical opera ons such as and and or, and the
concept of a check, which can transform data.
What we have called Check so far we will call Predicate. For

Predicate we can state the following iden ty law encoding the no on
that a predicate always returns its input if it succeeds:
For a predicate p of type Predicate[E, A] and elements

a1 and a2 of type A, if p(a1) == Success(a2) then a1
== a2.
Making this change gives us the following code:
import cats.syntax.semigroup._ // |+| syntax
import cats.syntax.cartesian._ // |@| syntax
import cats.data.Validated._ // Valid and Invalid
sealed trait Predicate[E, A] {

def and(that: Predicate[E, A]): Predicate[E, A] =
And(this, that)
def or(that: Predicate[E, A]): Predicate[E, A] =

Or(this, that)
def apply(a: A)(implicit s: Semigroup[E]): Validated[E, A] =

this match {
case Pure(func) =>
func(a)

(left(a) |@| right(a)).map((_, _) => a)
case Or(left, right) =>

left(a) match {
case Valid(a1) => Valid(a)
case Invalid(e1) =>
right(a) match {
case Invalid(e2) => Invalid(e1 |+| e2)
}
}
}
}
final case class And[E, A](

left: Predicate[E, A],
right: Predicate[E, A]) extends Predicate[E, A]
final case class Or[E, A](

final case class Pure[E, A](

func: A => Validated[E, A]) extends Predicate[E, A]
10.4.2 Checks
Well use Check to represent a structure we build from a Predicate

that also allows transforma on of its input. Implement Check with
the following interface. using the same ADT strategy we used for
Predicate:
sealed trait Check[E, A, B] {

def apply(a: A): Validated[E, B] =
???
def map[C](func: B => C): Check[E, A, C] =

???
}
See the solu on

What about flatMap? The seman cs are a bit unclear here. The
method is simple enough to declare but its not so obvious what it
means or how we should implement apply. The general shape of
flatMap is shown in Figure 10.7:
How do we relate F in the igure to Check in our code? Check has three
type variables while F only has one.
To unify the types we need to x two of the type parameters. The
idioma c choices are the error type E and the input type A. This gives
flatMap
F[A] A => F[B] F[B]
Figure 10.7: Type chart for atMap
us the rela onships shown in Figure 10.8:
flatMap
A => F[B] B => (A => F[C]) A => F[C]
Figure 10.8: Type chart for atMap applied to Check
In words, the seman cs of applying a FlatMap are:
given an input of type A, convert to F[B];
use the output of type B to choose a Check[E, A, C];
return to the original input of type A and apply it to the chosen

check to generate the nal result of type F[C].
This is quite an odd method. We can implement it, but it is hard to nd

a use for it. Go ahead and implement flatMap for Check, and then well
see a more generally useful method.
See the solu on
We can write a more useful combinator that chains together two

Checks. The output of the rst check is connected to the input of the
second. This is analogous to func on composi on using andThen:
val f: A => B = ???

val g: B => C = ???
val h: A => C = f andThen g
A Check is basically a func on A => Validated[E, B] so we can de-

ne an analagous andThen method:
trait Check[E, A, B] {
def andThen[C](that: Check[E, B, C]): Check[E, A, C]
}
Implement andThen now!
See the solu on
10.4.3 Recap
We now have two algebraic data types, Predicate and Check, and a
host of combinators with their associated case class implementa ons.
Check the following solu on for a complete deni on of each ADT.
See the solu on
We have a complete implementa on of Check and Predicate that do

most of what we originally set out to do. However, we are not nished
yet. You have probably recognised structure in Predicate and Check
that we can abstract over: Predicate has a monoid and Check has a
monad. Furthermore, in implemen ng Check you might have felt the
implementa on doesnt do muchall we do is call through to underly-
ing methods on Predicate and Validated.
There are a lot of ways this library could be cleaned up. However, lets
implement some examples to prove to ourselves that our library really
does work, and then well turn to improving it.
Implement checks for some of the examples given in the introduc on:
A username must contain at least four characters and consist

en rely of alphanumeric characters
An email address must contain an @ sign. Split the string at the @.

The string to the le must not be empty. The string to the right
must be at least three characters long and contain a dot.
You might nd the following predicates useful:

import cats.data.{NonEmptyList, OneAnd, Validated}
type Errors = NonEmptyList[String]
def error(s: String): NonEmptyList[String] =

NonEmptyList(s, Nil)
def longerThan(n: Int): Predicate[Errors, String] =

Predicate.lift(
error(s"Must be longer than $n characters"),
str => str.size > n)
val alphanumeric: Predicate[Errors, String] =

Predicate.lift(
error(s"Must be all alphanumeric characters"),
str => str.forall(_.isLetterOrDigit))
def contains(char: Char): Predicate[Errors, String] =

Predicate.lift(
error(s"Must contain the character $char"),
str => str.contains(char))
def containsOnce(char: Char): Predicate[Errors, String] =

Predicate.lift(
error(s"Must contain the character $char only once"),
str => str.filter(c => c == char).size == 1)
See the solu on

10.5. KLEISLIS 225
10.5 Kleislis
Well nish o this case study by cleaning up the implementa on of

Check. A jus able cri cism of our approach is that weve wri en a
lot of code to do very li le. A Predicate is essen ally a func on A
=> Validated[E, A], and a Check is basically a wrapper that lets us
compose these func ons.
We can abstract A => Validated[E, A] to A => F[B], which youll
recognise as the type of func on you pass to the flatMap method on
a monad. Imagine we have the following sequence of opera ons:
We li some value into a monad (by using pure, for example).

This is a func on with type A => F[A].
We then sequence some transforma ons on the monad using

flatMap.
We can illustrate this as shown in Figure 10.9:
flatMap flatMap
A => F[A] A => F[B] B => F[C]
Figure 10.9: Sequencing monadic transforms
We can also write out this example using the monad API as follows:
val aToB: A => F[B] = ???

val bToC: B => F[C] = ???
def example[A, C](a: A): F[C] =

aToB(a).flatMap(bToC)
Recall that Check is, in the abstract, allowing us to compose func ons
of type A => F[B]. We can write the above in terms of andThen as:
val aToC = aToB andThen bToC
The result is a (wrapped) func on aToC of type A => F[C] that we can
subsequently apply to a value of type A.
We have achieved the same thing as the example method without hav-
ing to reference an argument of type A. The andThen method on Check
is analogous to func on composi on, but is composing func on A =>
F[B] instead of A => B.
The abstract concept of composing func ons of type A => F[B] has a
name: a Kleisli.
Cats contains a data type cats.data.Kleisli that wraps a func on

just Check does. Kleisli has all the methods of Check plus some ad-
di onal ones. If Kleisli seems familiar to you, then congratula ons.
Youve seen through its disguise and recognised it as another concept
from earlier in the book. Kleisli is just another name for the Reader
monad!
Here is a simple example using Kleisli to transform an integer into a

list of integers through three steps:
import cats.data.Kleisli
These steps each transform an input Int into an output of type

List[Int]:
val step1: Kleisli[List, Int, Int] =

Kleisli(x => List(x + 1, x - 1))

Kleisli(x => List(x, -x))

10.5. KLEISLIS 227
Kleisli(x => List(x * 2, x / 2))
We can combine the steps into a single pipeline that combines the un-
derlying Lists using flatMap:
val pipeline = step1 andThen step2 andThen step3
The result is a func on that consumes a single Int and returns eight
outputs, each produced by a dierent combina on of transforma ons
from step1, step2, and step3:
pipeline.run(20)
// res2: List[Int] = List(42, 10, -42, -10, 38, 9, -38, -9)
The only notable dierence between Kleisli and Check in terms of

API is that Kleisli renames our apply method to run.
Lets replace Check with Kleisli in our valida on examples. To do

so we need to make a few changes to Predicate. We must be able
to convert a Predicate to a func on, as Kleisli only works with
func ons. Somewhat more subtly, when we convert a Predicate to
a func on, it should have type A => Either[E, A] rather than A =>
Validated[E, A] because Kleisli relies on the wrapped func on
returning a monad.
Add a method to Predicate called run that returns a func on of the

correct type. Leave the rest of the code in Predicate the same.
See the solu on
Now rewrite our username and rmail valida on example in terms of

Kleisli and Predicate. Here are few ps in case you get stuck:
First, remember that the run method on Predicate takes an implicit

parameter. If you call aPredicate.run(a) it will try to pass the implicit
parameter explicitly. If you want to create a func on from a Predicate
and immediately apply that func on, use aPredicate.run.apply(a)
Second, type inference can be tricky in this exercise. We found that

the following deni ons helped us to write code with fewer type dec-
lara ons.
type Result[A] = Either[Errors, A]
type Check[A, B] = Kleisli[Result, A, B]
// Create a check from a function:

def check[A, B](func: A => Result[B]): Check[A, B] =
Kleisli(func)
// Create a check from a Predicate:

def checkPred[A](pred: Predicate[Errors, A]): Check[A, A] =
Kleisli[Result, A, A](pred.run)
See the solu on
We have now wri en our code en rely in terms of Kleisli and

Predicate, completely removing Check. This is a good rst step to
simplifying our library. Theres s ll plenty more to do, but we have
a sophis cated building block from Cats to work with. Well leave
further improvements up to the reader.
10.6 Conclusions
This case study has been an exercise in removing rather than building
abstrac ons. We started with a fairly complex Check type. Once we re-
alised we were cona ng two concepts, we separated out Predicate
leaving us with something that could be implemented with Kleisli.
Predicate is very much like a stripped down version of the match-

ers found in tes ng libraries like ScalaTest and Specs2. One next step
would be to develop a more elaborate predicate library along these lines.
There are a few other direc ons to consider.
10.6. CONCLUSIONS 229
With the current representa on of Predicate there is no way to im-

plement logical nega on. To implement nega on we need to know the
error message that a successful predicate would have returned if it had
failed (so that the nega on can return that message). One way to imple-
ment this is to have a predicate return a Boolean ag indica ng success
or failure and the associated message.
We could also do be er in how error messages are represented. At the
moment there is no indica on with an error message of the structure of
the predicates that failed. For example, if we represent error messsages
as a List[String] and we get back the message:
List("Must be longer than 4 characters",

"Must not contain a number")
does this message indicate a failing conjunc on (two ands) or a failing

disjunc on (two ors)? We can probably guess in this case but in general
we dont have sucient informa on to work this out. We can solve this
problem by wrapping all messages in a type as follows:
sealed trait Structure[E]
final case class Or[E](messages: List[Structure[E]])

extends Structure[E]
final case class And[E](messages: List[Structure[E]])

final case class Not[E](messages: List[Structure[E]])

final case class Pure[E](message: E)

We can simplify this structure by conver ng all predicates into a nor-

mal form. For example, if we use disjunc ve normal form the structure
of the predicate will always be a disjunc on (logical or) of conjunc ons
(logical and). By doing so we could errors as a List[List[Either[E,

E]]], with the outer list represen ng disjunc on, the inner list repre-
sen ng conjunc on, and the Either represen ng nega on.
Finally, we made several design choices that reasonable people could

disagree with. Should the method that converts a Predicate to a
func on really be called run instead of, say, toFunction? Should
Predicate be a subtype of Function to begin with? The name run
makes sense if you have experienced monad transformers and other
similar abstrac ons, but is not clear if you dont have this experience.
Many func onal programmers come to prefer avoiding subtyping, as
it plays poorly with implicit resolu on and type inference, but there
could be an argument to use it here. As always the best decisions
depend on the context in which the library will be used.
Chapter 11
Case Study: Commuta ve

Replicated Data Types
In this case study we explore commuta ve replicated data types

(CRDTs), a data structure that can be used to reconcile eventually
consistent data.
We start by describing the u lity and diculty of eventually consistent
systems, then show how we can use monoids and their extensions to
solve the issues the issues that arise, and nally model the solu ons in
Scala.
Our goal here is to focus on the implementa on in Scala of a par cular
type of CRDT. Were specically not aiming at a comprehensive survey
of all CRDTs. CRDTs are a fast moving eld, and we advise you to read
the literature to learn about more.
11.1 Eventual Consistency
As soon as a system scales beyond a single machine we have to make

a fundamental choice about how we manage data. We can build a sys-
231
232CHAPTER 11. CASE STUDY: COMMUTATIVE REPLICATED DATA TYPES
tem that is consistent, meaning that all machines have the same view of
data. For example, if a user changes their password then all machines
that store a copy of that password must accept the change before we
consider the opera on to have completed successfully.
Consistent systems are simple to work with but they have their disad-
vantages. They tend to have high latency, as every change can result is
many messages being sent between machines. They can also can have
rela vely low up me. A network problem can cause some machines to
be unable to communicate with others. This is called a network par -
on. When there is a network par on a consistent system may refuse
further updates as allowing them could result in data becoming incon-
sistent between the par oned systems.
An alterna ve approach is an eventually consistent system. This means
that if all machines can communicate and there are no further updates
they will evenutally all have the same view of data. However, at any
par cular point in me machines are allowed to have diering views of
data.
Latency can be lower because eventually consistent systems require
less communica on between machines. A par oned machine can s ll
accept updates and reconcile its changes when the network is xed, so
systems can also can have be er up me. How exactly are we to do this
reconcilia on, though? CRDTs are one approach to the problem.
11.2 The GCounter
Lets look at one par cular CRDT implementa on. Then well a empt
to generalise proper es to see if we can nd a general pa ern.
The data structure we will look at is called a GCounter. It is a distributed
increment-only counter. It can be used, for example, for coun ng the
number of visitors to a site where requests are served by many web
servers.
11.2. THE GCOUNTER 233
11.2.1 Simple Counters
To see why a straigh orward counter wont work, imagine we have two
servers storing a count of visitors. Lets call the machines A and B. Each
machine is storing just an integer counter and the counters all start at
zero.
A: 0
B: 0
A serves three visitors, and B two.
A: 3
B: 2
Now the machines want to merge their counters so they each have an
up-to-date view of the total number of visitors. At this point we know
the machines should add together their counters, because we know the
history of their interac ons. However, there is nothing in the data the
machines store that records this. Nonetheless, lets use addi on as our
strategy for merging counters and see what happens.
A: 5
B: 5
Now A serves a single visitor.
A: 6
B: 4
The machines a empt to merge counters again. If they use addi on as

the merging algorithm they will end up with
A: 10
B: 10
This is clearly wrong! There have only been six visitors in total. Do we
need to store the complete history of interac ons to be able to compute
the correct value? It turns out we do not, so lets look at the GCounter
now to see how it solves this problem in an elegant way.
11.2.2 GCounters
The rst clever idea in the GCounter is to have each machine storing a
separate counter for every machine (including itself) that it knows about.
In the previous example we had two machines, A and B. In this situa on
both machines would store a counter for A and a counter for B.
Machine A Machine B
A: 0 A: 0
B: 0 B: 0
The rule with these counters is that a given machine is only allowed
to increment its own counter. If A serves 3 visitors and B serves two
visitors the counters will look like
Machine A Machine B
A: 3 A: 0
B: 0 B: 2
Now when two machines merge their counters the rule is to take the
largest value stored for a given machine. Given the state above, when
A and B merge counters the result will be
11.2. THE GCOUNTER 235
Machine A Machine B
A: 3 A: 3
B: 2 B: 2
as 3 is the largest value stored for the A counter, and 2 is the largest
value stored for the B counter. The combina on of only allowing ma-
chines to increment their counter and choosing the maximum value on
merging means we get the correct answer without storing the complete
history of interac ons.
If a machine wants to calculate the current value of the counter (given

its current knowledge of other machines state) it simply sums up all the
per-machine counter. Given the state
Machine A Machine B
A: 3 A: 3
B: 2 B: 2
each machine would report the current values as 3 + 2 = 5.
11.2.3 Exercise: GCounter Implementa on
We can implement a GCounter with the interface
final case class GCounter(counters: Map[String, Int]) {

def increment(machine: String, amount: Int) =
???
def get: Int =

???
def merge(that: GCounter): GCounter =

???
}
where we represent machine IDs as Strings.

Finish the implementa on.
See the solu on
11.3 Generalisa on
Weve now created a distributed eventually consistent increment only

counter. This is a nice achievement, but dont want to stop here. In this
sec on we will a empt to abstract the opera ons used in the GCounter
so it will work with more data types than just natural numbers.
The GCounter uses the following opera ons on natural numbers: - ad-
di on (in increment and get); - maximum (in merge); - and the iden ty
element 0 (in increment and merge).
This should already make you feel there is a monoid somewhere in here,
but lets look in more detail on the proper es we rely on.
As a refresher, here are the proper es weve of monoids weve seen
earlier:
the binary opera on + is associa ve, meaning (a + b) + c =

a + (b + c);
the iden ty 0 is commuta ve, meaning a + 0 = 0 + a; and
the iden ty is an iden ty, meaning a + 0 = a.
In increment, we need an iden ty to ini alise the counter. We also

rely on associa vity to ensure the specic sequence of addi ons we
perform gives the correct value.
In get we implicitly rely on associa vity and commu vity to ensure we
get the correct value no ma er what arbitrary order we choose to sum
the per-machine counters. We also implicitly assume an iden ty, which
allows us to skip machines for which we do not store a counter.
11.3. GENERALISATION 237
The proper es merge relies on are a bit more interes ng. We rely on
commu vity to ensure that machine A merging with machine B yields
the same result as machine B merging with machine A. We need as-
socia vity to ensure we obtain the correct result when three or more
machines are merging data. We need an iden ty element to ini alise
empty counters. Finally, we need an addi onal property, called idem-
potency, to ensure that if two machines hold the same data in a per-
machine counter, merging data will not lead to an incorrect result. For-
mally, a binary opera on max is idempotent if a max a = a.
Wri en more compactly, we have:
Commuta ve Idempotent
Method Iden ty Associa ve
increment Y N Y N
get Y Y Y N
merge Y Y Y Y
From this we can see that
increment requires a monoid;

get requires a commuta ve monoid; and
merge required an idempotent commuta ve monoid, also called
a bounded semila ce.
Since increment and get both use the same binary opera on (addi on)
its usual to require the same commuta ve monoid for both.
This inves ga on demonstrates the powers of thinking about proper-

es or laws of abstrac ons. Now we have iden ed these proper es
we can subs tute the natural numbers used in our GCounter with any
data type with opera ons mee ng these proper es. A simple example
is a set, with union being the binary opera on and the iden ty element
the empty set. Set union is idempotent, commuta ve, and associa ve
and therefore ts all our requirements to work with a GCounter. With
this simple subs tu on of Int for Set[A] we can create a GSet type.
11.3.1 Implementa on
Let us now implement this generalisa on in code. Remember

increment and get require a commuta ve monoid and merge requires
a bounded semila ce (or idempotent commuta ve monoid).
Cats provides a Monoid, but no commuta ve monoid or bounded semi-
la ce type class. For simplicity of implementa on well use Monoid
when we really mean a commuta ve monoid, and require the program-
mer to ensure the implementa on is commuta ve. Well implement
our own BoundedSemiLattice type class.
import cats.Monoid
trait BoundedSemiLattice[A] extends Monoid[A] {

def combine(a1: A, a2: A): A
def empty: A
}
In the implementa on above, BoundedSemiLattice[A] extends

Monoid[A] because a bounded semila ce is a monoid (a commuta ve
idempotent one, to be exact).
11.3.2 Exercises
11.3.2.1 BoundedSemiLa ce Instances
Implement some BoundedSemiLattice type class instances (e.g. for

Int and Set).
A closely related library called Spire provides both these abstrac ons.
11.4. ABSTRACTING GCOUNTER TO A TYPE CLASS 239
See the solu on
11.3.2.2 Generic GCounter
Using Monoid and BoundedSemiLattice, generalise GCounter.
When you implement this, look for opportuni es to use methods and
syntax on monoid to simplify your implementa on. This is a good ex-
ample of how type class abstrac ons work at mul ple levels of code.
Were using monoids to design a large componentour CRDTsbut
they are also useful in the small, making our code simpler.
See the solu on
11.4 Abstrac ng GCounter to a Type Class
Weve created a generic GCounter that works with any value that has
(commuta ve) Monoid and BoundedSemiLattice type class instances.
However were s ll ed to a par cular representa on of the map from
machine IDs to values. There is no need to have this restric on, and
indeed it can be useful to abstract away from it. There are many key-
value stores that might like to work with our GCounter, from a simple
Map to a rela onal database.
If we dene a GCounter type class we can abstract over dierent con-

crete implementa ons. This allows us to, for example, seamlessly sub-
s tute an in-memory store for a persistent store when we want to
change performance and durability tradeos.
There are a number of ways we can implement this. Try your own im-
plementa on before reading on.
A simple way to achieve this is by dening a GCounter type class with

dependencies on Monoid and BoundedSemiLattice. I dened this
type class as taking a higher-kinded type with two type parameters, in-
tended to represent the key and value types of the map abstrac on.
import cats.Monoid

def empty: A
}
object BoundedSemiLattice {
implicit object intBoundedSemiLatticeInstance extends
BoundedSemiLattice[Int] {
def combine(a1: Int, a2: Int): Int =
a1 max a2
val empty: Int = 0

}
implicit def setBoundedSemiLatticeInstance[A]:

BoundedSemiLattice[Set[A]] =
new BoundedSemiLattice[Set[A]]{
def combine(a1: Set[A], a2: Set[A]): Set[A] =
a1 union a2
val empty: Set[A] =

Set.empty[A]
}
}
trait GCounter[F[_,_],K, V] {
def increment(f: F[K, V])(k: K, v: V)(implicit m: Monoid[V]):
F[K, V]
def total(f: F[K, V])(implicit m: Monoid[V]): V
def merge(f1: F[K, V], f2: F[K, V])(implicit b:
BoundedSemiLattice[V]): F[K, V]
}
We can easily dene some instances of this type class. Heres a com-
plete example, containing a type class instance for Map and a simple
test.
object GCounterExample {
def empty: A
}
a1 max a2
val empty: Int = 0

}

a1 union a2
val empty: Set[A] =

Set.empty[A]
}
}
def increment(f: F[K, V])(k: K, v: V)(implicit m: Monoid[V])
: F[K, V]
}
object GCounter {
implicit def mapGCounterInstance[K, V]: GCounter[Map, K, V]
=
new GCounter[Map, K, V] {
def increment(f: Map[K, V])(k: K, v: V)(implicit m:

Monoid[V]): Map[K, V] =
f + (k -> (f.getOrElse(k, m.empty) |+| v))
def total(f: Map[K, V])(implicit m: Monoid[V]): V =

f.foldMap(identity)
def merge(f1: Map[K, V], f2: Map[K, V])(implicit b:

BoundedSemiLattice[V]): Map[K, V] =
f1 |+| f2
}
def apply[F[_,_],K, V](implicit g: GCounter[F, K, V]) = g

}
val g1 = Map("a" -> 7, "b" -> 3)

val g2 = Map("a" -> 2, "b" -> 5)
println(s"Merged: ${GCounter[Map, String, Int].merge(g1,g2)}")

println(s"Total: ${GCounter[Map, String, Int].total(g1)}")
}
This implementa on strategy is a bit unsa sfying. Although the struc-

ture of the implementa on will be the same for most of the type class
instances we wont get any code reuse.
One solu on is to capture the idea of a key-value store within a type
class, and then generate GCounter instances for any type that has a
KeyValueStore instance. Heres the code for KeyValueStore, includ-
ing syntax and an example instance for Map.
trait KeyValueStore[F[_,_]] {
def +[K, V](f: F[K, V])(key: K, value: V): F[K, V]
def get[K, V](f: F[K, V])(key: K): Option[V]
def getOrElse[K, V](f: F[K, V])(key: K, default: V): V =

get(f)(key).getOrElse(default)
}
object KeyValueStore {
implicit class KeyValueStoreOps[F[_,_],K, V](f: F[K, V]) {
def +(key: K, value: V)(implicit kv: KeyValueStore[F]): F[K,
V] =
kv.+(f)(key, value)
def get(key: K)(implicit kv: KeyValueStore[F]): Option[V] =

kv.get(f)(key)
def getOrElse(key: K, default: V)(implicit kv: KeyValueStore

[F]): V =
kv.getOrElse(f)(key, default)
}
implicit object mapKeyValueStoreInstance extends KeyValueStore

[Map] {
def +[K, V](f: Map[K, V])(key: K, value: V): Map[K, V] =
f + (key, value)
def get[K, V](f: Map[K, V])(key: K): Option[V] =

f.get(key)
override def getOrElse[K, V](f: Map[K, V])(key: K, default:

V): V =
f.getOrElse(key, default)
}
}
Now we can generate GCounter instances with an implicit def. This

implementa on is moderately advanced: it has a number of type class
dependencies, including one on Foldable that uses a type lambda.
implicit def keyValueInstance[F[_,_],K, V](

implicit
k: KeyValueStore[F],
km: Monoid[F[K, V]],
kf: Foldable[({type l[A]=F[K, A]})#l]
): GCounter[F, K, V] =
new GCounter[F, K, V] {
import KeyValueStore._ // For KeyValueStore syntax
def increment(f: F[K, V])(key: K, value: V)(implicit m:

Monoid[V]): F[K, V] =
f + (key, (f.getOrElse(key, m.empty) |+| value))
def total(f: F[K, V])(implicit m: Monoid[V]): V =

f.foldMap(identity _)

BoundedSemiLattice[V]): F[K, V] =
f1 |+| f2
}
Heres the complete code, including an example. This code is quite long
but the majority of it is boilerplate. We could cut down on the boiler-
plate by using compiler plugins such as Simulacrum and Kind Projector.
object GCounterExample {
import cats.{Monoid, Foldable}

def empty: A
}

a1 max a2
val empty: Int = 0

}

a1 union a2
val empty: Set[A] =

Set.empty[A]
}
}
Monoid[V]): F[K, V]
}
object GCounter {
def apply[F[_,_],K, V](implicit g: GCounter[F, K, V]) = g
implicit class GCounterOps[F[_,_],K, V](f: F[K, V]) {

def increment(key: K, value: V)(implicit g: GCounter[F, K,
V], m: Monoid[V]): F[K, V] =
g.increment(f)(key, value)
def total(implicit g: GCounter[F, K, V], m: Monoid[V]): V

=
g.total(f)
def merge(that: F[K, V])(implicit g: GCounter[F, K, V], b:

g.merge(f, that)
}
implicit def keyValueInstance[F[_,_],K, V](implicit k:

KeyValueStore[F], km: Monoid[F[K, V]], kf: Foldable[({type l
[A]=F[K, A]})#l]): GCounter[F, K, V] =
new GCounter[F, K, V] {
import KeyValueStore._ // For KeyValueStore syntax

Monoid[V]): F[K, V] =
f + (key, (f.getOrElse(key, m.empty) |+| value))
def total(f: F[K, V])(implicit m: Monoid[V]): V =

f.foldMap(identity _)

f1 |+| f2
}
}
trait KeyValueStore[F[_,_]] {
def +[K, V](f: F[K, V])(key: K, value: V): F[K, V]
def get[K, V](f: F[K, V])(key: K): Option[V]
def getOrElse[K, V](f: F[K, V])(key: K, default: V): V =

get(f)(key).getOrElse(default)
}
object KeyValueStore {
implicit class KeyValueStoreOps[F[_,_],K, V](f: F[K, V]) {
def +(key: K, value: V)(implicit kv: KeyValueStore[F]): F[
K, V] =
kv.+(f)(key, value)
def get(key: K)(implicit kv: KeyValueStore[F]): Option[V]

=
kv.get(f)(key)
def getOrElse(key: K, default: V)(implicit kv:

KeyValueStore[F]): V =
kv.getOrElse(f)(key, default)
}
implicit object mapKeyValueStoreInstance extends

KeyValueStore[Map] {
def +[K, V](f: Map[K, V])(key: K, value: V): Map[K, V] =
f + (key, value)
def get[K, V](f: Map[K, V])(key: K): Option[V] =

f.get(key)
override def getOrElse[K, V](f: Map[K, V])(key: K, default

: V): V =
f.getOrElse(key, default)
}
}
object Example {
import KeyValueStore._
import GCounter._
val crdt1 = Map("a" -> 1, "b" -> 3, "c" -> 5)

val crdt2 = Map("a" -> 2, "b" -> 4, "c" -> 6)
crdt1.increment("a", 20).merge(crdt2).total
}
}
11.5 Summary
In this case study weve seen how we can use type classes to model a
simple CRDT, the GCounter, in Scala. Our implementa on gives us a
lot of exibility and code reuse. We are not ed to the data type we
count, nor to the data type that maps machine IDs to counters.
The focus in this case study has been on using the tools that Scala pro-
vides, and not on exploring CRDTs. There are many other CRDTs, some
of which operate in a similar manner to the GCounter, and some of
which have very dierent implementa ons. A fairly recent survey gives
a good overview of many of the basic CRDTs. However this is an ac ve
area of research and we encourage you to read the recent publica ons
in the eld if CRDTs and eventually consistency interest you.
Part III
Solu ons to Exercises
249
Appendix A
Solu ons for: Introduc on
A.1 Printable Library
These steps dene the three main components of our type class. First
we dene Printablethe type class itself:
}
Then we dene some default instances of Printable and package then

in PrintableInstances:
object PrintableInstances {
implicit val stringPrintable = new Printable[String] {
def format(input: String) = input
}
implicit val intPrintable = new Printable[Int] {

def format(input: Int) = input.toString
251
252 APPENDIX A. SOLUTIONS FOR: INTRODUCTION
}
}
Finally we dene an interface object, Printable:
object Printable {
def format[A](input: A)(implicit p: Printable[A]): String =
p.format(input)
def print[A](input: A)(implicit p: Printable[A]): Unit =

println(format(input))
}
Return to the exercise
A.2 Printable Library Part 2
This is a standard use of the type class pa ern. First we dene a set of
custom data types for our applica on:
Then we dene type class instances for the types we care about. These
either go into the companion object of Cat or a separate object to act
as a namespace:
import PrintableInstances._
implicit val catPrintable = new Printable[Cat] {

def format(cat: Cat) = {
val name = Printable.format(cat.name)
val age = Printable.format(cat.age)
val color = Printable.format(cat.color)
s"$name is a $age year-old $color cat."
A.3. PRINTABLE LIBRARY PART 3 253
}
}
Finally, we use the type class by bringing the relevant instances into
scope and using interface object/syntax. If we dened the instances in
companion objects Scala brings them into scope for us automa cally.
Otherwise we use an import to access them:
val cat = Cat("Garfield", 35, "ginger and black")

// cat: Cat = Cat(Garfield,35,ginger and black)
Printable.print(cat)
// Garfield is a 35 year-old ginger and black cat.
A.3 Printable Library Part 3
First we dene an implicit class containing our extension methods:
object PrintableSyntax {
implicit class PrintOps[A](value: A) {
def format(implicit p: Printable[A]): String =
p.format(value)
def print(implicit p: Printable[A]): Unit =

println(p.format(value))
}
}
With PrintOps in scope, we can call the imaginary print and format
methods on any value for which Scala can locate an implicit instance of
Printable:
import PrintableSyntax._
Cat("Garfield", 35, "ginger and black").print

We get a compile error if we havent dened an instance of Printable

for the relevant type:
new Date().print
parameter p: Printable[java.util.Date]
// new Date().print
// ^
A.4 Cat Show
First lets import everything we need from Cats: the Show type class,
the instances for Int and String, and the interface syntax:
import cats.Show
import cats.syntax.show._
Our deni on of Cat remains the same:
In the companion object we replace our Printable with an instance of

Show using one of the deni on helpers discussed above:
A.5. EQUALITY, LIBERTY, AND FELINITY 255
implicit val catShow = Show.show[Cat] { cat =>

val name = cat.name.show
val age = cat.age.show
val color = cat.color.show
s"$name is a $age year-old $color cat."
}
Finally, we use the Show interface syntax to print our instance of Cat:
println(Cat("Garfield", 35, "ginger and black").show)
A.5 Equality, Liberty, and Felinity
First we need our Cats imports. In this exercise well be using the Eq
type class and the Eq interface syntax. Well bring instances of Eq into
scope as we need them below:
import cats.Eq
import cats.syntax.eq._
Our Cat class is the same as ever:

We bring the Eq instances for Int and String into scope for the imple-
menta on of Eq[Cat]:
implicit val catEqual = Eq.instance[Cat] { (cat1, cat2) =>

(cat1.name === cat2.name ) &&

(cat1.age === cat2.age ) &&
(cat1.color === cat2.color)

}
Finally, we test things out in a sample applica on:
val cat1 = Cat("Garfield", 35, "orange and black")

// cat1: Cat = Cat(Garfield,35,orange and black)
val cat2 = Cat("Heathcliff", 30, "orange and black")

// cat2: Cat = Cat(Heathcliff,30,orange and black)
cat1 === cat2

cat1 =!= cat2

val optionCat1 = Option(cat1)

// optionCat1: Option[Cat] = Some(Cat(Garfield,35,orange and
black))
val optionCat2 = Option.empty[Cat]

// optionCat2: Option[Cat] = None
optionCat1 === optionCat2

optionCat1 =!= optionCat2


Appendix B
Solu ons for: Monoids and

Semigroups
B.1 The Truth About Monoids
There are four monoids for Boolean! First, we have and with operator
&& and iden ty true:
implicit val booleanAndMonoid: Monoid[Boolean] =

new Monoid[Boolean] {
def combine(a: Boolean, b: Boolean) = a && b
def empty = true
}
Second, we have or with operator || and iden ty false:
implicit val booleanOrMonoid: Monoid[Boolean] =

def combine(a: Boolean, b: Boolean) = a || b
def empty = false
}
257
258 APPENDIX B. SOLUTIONS FOR: MONOIDS AND SEMIGROUPS
Third, we have exclusive or with iden ty false:
implicit val booleanEitherMonoid: Monoid[Boolean] =

def combine(a: Boolean, b: Boolean) =
(a && !b) || (!a && b)
def empty = false

}
Finally, we have exclusive nor (the nega on of exclusive or) with iden ty
true:
implicit val booleanXnorMonoid: Monoid[Boolean] =

def combine(a: Boolean, b: Boolean) =
(!a || b) && (a || !b)
def empty = true

}
Showing that the iden ty law holds in each case is straigh orward. Sim-
ilarly associa vity of the combine opera on can be shown by enumer-
a ng the cases.
B.2 All Set for Monoids
Set union forms a monoid along with the empty set:
implicit def setUnionMonoid[A]: Monoid[Set[A]] =

new Monoid[Set[A]] {
def combine(a: Set[A], b: Set[A]) = a union b
def empty = Set.empty[A]
B.3. ADDING ALL THE THINGS 259
We need to dene setUnionMonoid as a method rather than a value so

we can accept the type parameter A. Scalas implicit resolu on is ne
with thisit is capable of determining the correct type parameter to
create a Monoid of the desired type:
implicit val intMonoid: Monoid[Int] = new Monoid[Int] {

def combine(a: Int, b: Int) = a + b
def empty = 0
}
val intSetMonoid = Monoid[Set[Int]]

// intSetMonoid: Monoid[Set[Int]] = $anon$1@5ec1a336
intSetMonoid.combine(Set(1, 2), Set(2, 3))

// res2: Set[Int] = Set(1, 2, 3)
Set intersec on forms a semigroup, but doesnt form a monoid because

it has no iden ty element:
implicit def setIntersectionSemigroup[A]: Semigroup[Set[A]] =

new Semigroup[Set[A]] {
def combine(a: Set[A], b: Set[A]) =
a intersect b
}
B.3 Adding All The Things
We can write the addi on as a simple foldLeft using 0 and the + op-
erator:
def add(items: List[Int]): Int =

items.foldLeft(0)(_ + _)
We can alterna vely write the fold using Monoids, although theres not
a compelling use case for this yet:
import cats.Monoid
def add(items: List[Int]): Int =

items.foldLeft(Monoid[Int].empty)(_ |+| _)
B.4 Adding All The Things Part 2
Now there is a use case for Monoids. We need a single method that
adds Ints and instances of Option[Int]. We can write this as a
generic method that accepts an implicit Monoid as a parameter:
import cats.Monoid
def add[A](items: List[A])(implicit monoid: Monoid[A]): A =

items.foldLeft(monoid.empty)(_ |+| _)
We can op onally use Scalas context bound syntax to write the same
code in a friendlier way:
def add[A: Monoid](items: List[A]): A =

items.foldLeft(Monoid[A].empty)(_ |+| _)
We can use this code to add values of type Int and Option[Int] as
requested:
B.5. ADDING ALL THE THINGS PART 3 261
add(List(1, 2, 3))
// res9: Int = 6
add(List(Some(1), None, Some(2), None, Some(3)))

Note that if we try to add a list consis ng en rely of Some values, we

get a compile error:
add(List(Some(1), Some(2), Some(3)))
evidence parameter of type cats.Monoid[Some[Int]]
// add(List(Some(1), Some(2), Some(3)))
// ^
This happens because the inferred type of the list is List[Some[Int]],

while Cats will only generate a Monoid for Option[Int]. Well see how
to get around this in a moment.
B.5 Adding All The Things Part 3
Easywe simply dene a monoid instance for Order!

implicit val monoid: Monoid[Order] = new Monoid[Order] {
def combine(o1: Order, o2: Order) =
Order(
o1.totalCost + o2.totalCost,
o1.quantity + o2.quantity
)
def empty = Order(0, 0)


Appendix C
Solu ons for: Functors
C.1 Branching out with Functors
The seman cs are similar to wri ng a Functor for List. We recurse

over the data structure, applying the func on to every Leaf we nd.
The functor laws intui vely require us to retain the same structure with
the same pa ern of Branch and Leaf nodes:
import cats.Functor
implicit val treeFunctor = new Functor[Tree] {

def map[A, B](tree: Tree[A])(func: A => B): Tree[B] =
tree match {
case Branch(left, right) =>
Branch(map(left)(func), map(right)(func))
case Leaf(value) =>
Leaf(func(value))
}
}
Lets use our Functor to transform some Trees:
263
264 APPENDIX C. SOLUTIONS FOR: FUNCTORS
Branch(Leaf(10), Leaf(20)).map(_ * 2)
// <console>:38: error: value map is not a member of Branch[Int]
// Branch(Leaf(10), Leaf(20)).map(_ * 2)
// ^
Oops! This is the same invariance problem we saw with Monoids. The
compiler cant nd a Functor instance for Leaf. Lets add some smart
constructors to compensate:
def branch[A](left: Tree[A], right: Tree[A]): Tree[A] =

Branch(left, right)
def leaf[A](value: A): Tree[A] =

Leaf(value)
Now we can use our Functor properly:
leaf(100).map(_ * 2)
// res6: Tree[Int] = Leaf(200)
branch(leaf(10), leaf(20)).map(_ * 2)
// res7: Tree[Int] = Branch(Leaf(20),Leaf(40))
C.2 Showing o with Contramap
Heres a working implementa on:
def contramap[B](func: B => A): Printable[B] = {

val self = this
new Printable[B] {
def format(value: B): String =
C.3. SHOWING OFF WITH CONTRAMAP PART 2 265
self.format(func(value))
}
}
}
def format[A](value: A)(implicit p: Printable[A]): String =

p.format(value)
C.3 Showing o with Contramap Part 2
To make the instance generic across all types of Box, we base it on the
Printable for the type inside the Box:
implicit def boxPrintable[A](implicit p: Printable[A]) =

p.contramap[Box[A]](_.value)
C.4 Transforma ve Thinking with Imap
Heres a working implementa on:

trait Codec[A] {
def encode(value: A): String
def decode(value: String): Option[A]
def imap[B](dec: A => B, enc: B => A): Codec[B] = {

val self = this
new Codec[B] {
def encode(value: B): String =
self.encode(enc(value))
def decode(value: String): Option[B] =

266 APPENDIX C. SOLUTIONS FOR: FUNCTORS
self.decode(value).map(dec)
}
}
}
def encode[A](value: A)(implicit c: Codec[A]): String =

c.encode(value)
def decode[A](value: String)(implicit c: Codec[A]): Option[A] =

c.decode(value)
C.5 Transforma ve Thinking with Imap Part 2

implicit val intCodec =
new Codec[Int] {
def encode(value: Int): String =
value.toString
def decode(value: String): Option[Int] =

scala.util.Try(value.toInt).toOption
}
C.6 Transforma ve Thinking with Imap Part 3

implicit def boxCodec[A](implicit c: Codec[A]): Codec[Box[A]] =
c.imap[Box[A]](Box(_), _.value)

Appendix D
Solu ons for: Monads
D.1 Ge ng Func-y
At rst glance this seems tricky, but if we follow the types well see
theres only one solu on. Lets start by wri ng the method header:
trait Monad[F[_]] {
def map[A, B](value: F[A])(func: A => B): F[B] =

???
}
Now we look at the types. Weve been given a value of type F[A].
Given the tools available theres only one thing we can do: call flatMap:
trait Monad[F[_]] {
267
268 APPENDIX D. SOLUTIONS FOR: MONADS

flatMap(value)(a => ???)
}
We need a func on of type A => F[B] as the second parameter. We

have two func on building blocks available: the func parameter of type
A => B and the pure func on of type A => F[A]. Combining these
gives us our result:
trait Monad[F[_]] {

flatMap(value)(a => pure(func(a)))
}
D.2 Monadic Secret Iden es
Lets start by dening the method headers:
import cats.Id
def pure[A](value: A): Id[A] =

???
def map[A, B](initial: Id[A])(func: A => B): Id[B] =

???
def flatMap[A, B](initial: Id[A])(func: A => Id[B]): Id[B] =

???
D.2. MONADIC SECRET IDENTITIES 269
Now lets look at each method in turn. The pure opera on is a

constructorit creates an Id[A] from an ini al value of type A. But A
and Id[A] are the same type! All we have to do is return the ini al
value:
def pure[A](value: A): Id[A] =
value
pure(123)
The map method applies a func on of type A => B to an Id[A], crea ng

an Id[B]. But Id[A] is simply A and Id[B] is simply B! All we have to
do is call the func onno packing or unpacking required:
def map[A, B](initial: Id[A])(func: A => B): Id[B] =
func(initial)
map(123)(_ * 2)
The nal punch line is that, once we strip away the Id type constructors,
flatMap and map are actually iden cal:
def flatMap[A, B](initial: Id[A])(func: A => Id[B]): Id[B] =

func(initial)
// flatMap: [A, B](initial: cats.Id[A])(func: A => cats.Id[B])
cats.Id[B]
flatMap(123)(_ * 2)
No ce that we havent had to add any casts to any of the examples in

this solu on. Scala is able to interpret values of type A as Id[A] and
vice versa, simply by the context in which they are used.
The only restric on to this is that Scala cannot unify dierent shapes of
type constructor when searching for implicits. Hence our need to cast
to Id[A] in the call to sumSquare at the opening of this sec on:
sumSquare(3 : Id[Int], 4 : Id[Int])
D.3 What is Best?
This is an open ques on. Its also kind of a trick ques onthe answer
depends on the seman cs were looking for. Some points to ponder:
Error recovery is important when processing large jobs. We dont

want to run a job for a day and then nd it failed on the last
element.
Error repor ng is equally important. We need to know what

went wrong, not just that something went wrong.
In a number of cases we want to collect all the errors, not just

the rst one we encountered. A typical example is valida ng a
web form. Its a far be er experience to report all errors to the
user when they submit a form than to report them one at a me.
D.4 Safer Folding using Eval
The easiest way to x this is to introduce a helper method called

foldRightEval. This is essen ally our original method with every
occurrence of B replaced with Eval[B], and a call to Eval.defer to
protect the recursive call:
D.5. SHOW YOUR WORKING 271
import cats.Eval
def foldRightEval[A, B](as: List[A], acc: Eval[B])

(fn: (A, Eval[B]) => Eval[B]): Eval[B] =
as match {
case head :: tail =>
Eval.defer(fn(head, foldRightEval(tail, acc)(fn)))
case Nil =>
acc
}
We can redene foldRight simply in terms of foldRightEval and the

resul ng method is stack safe:
def foldRight[A, B](as: List[A], acc: B)

(fn: (A, B) => B): B =
foldRightEval(as, Eval.now(acc)) { (a, b) =>
b.map(fn(a, _))
}.value
foldRight((1 to 100000).toList, 0)(_ + _)

// res22: Int = 705082704
D.5 Show Your Working
Well start by dening a type alias for Writer so we can use it with pure
syntax:
import cats.data.Writer
42.pure[Logged]
// res15: Logged[Int] = WriterT((Vector(),42))
Well import the tell syntax as well:
Vector("Message").tell
// res16: cats.data.Writer[scala.collection.immutable.Vector[
String],Unit] = WriterT((Vector(Message),()))
Finally, well import the Semigroup instance for Vector. We need this
to map and flatMap over Logged:
41.pure[Logged].map(_ + 1)
// res17: cats.data.WriterT[cats.Id,Vector[String],Int] =
WriterT((Vector(),42))
With these in scope, the deni on of factorial becomes:
def factorial(n: Int): Logged[Int] =

for {
ans <- if(n == 0) {
1.pure[Logged]
} else {
slowly(factorial(n - 1).map(_ * n))
}
_ <- Vector(s"fact $n $ans").tell
} yield ans
Now, when we call factorial, we have to run the result to extract the
log and our factorial:
D.6. HACKING ON READERS 273
val (log, result) = factorial(5).run

// log: Vector[String] = Vector(fact 0 1, fact 1 1, fact 2 2,
fact 3 6, fact 4 24, fact 5 120)
// result: Int = 120
We can run several factorials in parallel as follows, capturing their

logs independently without fear of interleaving:
val Vector((logA, ansA), (logB, ansB)) =
Await.result(Future.sequence(Vector(
Future(factorial(5).run),
Future(factorial(5).run)
)), 5.seconds)
// logA: Vector[String] = Vector(fact 0 1, fact 1 1, fact 2 2,
fact 3 6, fact 4 24, fact 5 120)
// ansA: Int = 120
// logB: Vector[String] = Vector(fact 0 1, fact 1 1, fact 2 2,
fact 3 6, fact 4 24, fact 5 120)
// ansB: Int = 120
D.6 Hacking on Readers
Our type alias xes the Db type but leaves the result type exible:
type DbReader[A] = Reader[Db, A]
D.7 Hacking on Readers Part 2
Remember: the idea is to leave injec ng the congura on un l last.

This means se ng up func ons that accept the cong as a parameter
and check it against the concrete user info we have been given:
def findUsername(userId: Int): DbReader[Option[String]] =

Reader(db => db.usernames.get(userId))
def checkPassword(
username: String,
password: String
): DbReader[Boolean] =
Reader(db => db.passwords.get(username).contains(password))
D.8 Hacking on Readers Part 3
As you might expect, here we use flatMap to chain findUsername and

checkPassword. We use pure to li a Boolean to a DbReader[Boolean]
when the username is not found:
import cats.syntax.applicative._ // for `pure`
def checkLogin(
userId: Int,
password: String
): DbReader[Boolean] =
for {
username <- findUsername(userId)
passwordOk <- username.map { username =>
checkPassword(username, password)
}.getOrElse {
false.pure[DbReader]
}
} yield passwordOk

D.9. POST-ORDER CALCULATOR 275
D.9 Post-Order Calculator
The stack opera on required is dierent for operators and operands.

For clarity well implement evalOne in terms of two helper func ons,
one for each case:
def evalOne(sym: String): CalcState[Int] =

sym match {
case "+" => operator(_ + _)
case "-" => operator(_ - _)
case "*" => operator(_ * _)
case "/" => operator(_ / _)
case num => operand(num.toInt)
}
Lets look at operand rst. All we have to do is push a number onto the
stack. We also return the operand as an intermediate result:
def operand(num: Int): CalcState[Int] =

State[List[Int], Int] { stack =>
(num :: stack, num)
}
The operator func on is a li le more complex. We have to pop two

operands o the stack and push the result in their place. The code can
fail if the stack doesnt have enough operands on it, but the exercise
descrip on allows us to throw an excep on in this case:
def operator(func: (Int, Int) => Int): CalcState[Int] =

State[List[Int], Int] {
case a :: b :: tail =>
val ans = func(a, b)
(ans :: tail, ans)
case _ =>
sys.error("Fail!")
D.10 Post-Order Calculator Part 2
We implement evalAll by folding over the input. We start with a pure

CalcState that returns 0 if the list is empty. We flatMap at each stage,
ignoring the intermediate results as we saw in the example:
// import cats.syntax.applicative._
def evalAll(input: List[String]): CalcState[Int] = {

input.foldLeft(0.pure[CalcState]) { (a, b) =>
a flatMap (_ => evalOne(b))
}
D.11 Branching out Further with Monads
The code for flatMap is simple. Its similar to the code for map. Again,
we recurse down the structure and use the results from func to build
a new Tree.
The code for tailRecM is less simple. In fact, its fairly complex! How-
ever, if we follow the types the solu on falls out. Note that we cant
make tailRecM tail recursive in this case because we have to recurse
twice when processing a Branch. We implement the tailRecM method,
and we dont use the tailrec annota on:
D.11. BRANCHING OUT FURTHER WITH MONADS 277
import cats.Monad
implicit val treeMonad = new Monad[Tree] {

def pure[A](value: A): Tree[A] =
Leaf(value)
def flatMap[A, B](tree: Tree[A])

(func: A => Tree[B]): Tree[B] =
tree match {
case Branch(l, r) =>
Branch(flatMap(l)(func), flatMap(r)(func))
case Leaf(value) =>
func(value)
}
def tailRecM[A, B](arg: A)

(func: A => Tree[Either[A, B]]): Tree[B] =
func(arg) match {
case Branch(l, r) =>
Branch(
flatMap(l) {
case Left(l) => tailRecM(l)(func)
case Right(l) => pure(l)
},
flatMap(r) {
case Left(r) => tailRecM(r)(func)
case Right(r) => pure(r)
}
)
case Leaf(Left(value)) =>

tailRecM(value)(func)
case Leaf(Right(value)) =>

Leaf(value)
}
}
Now we can use our Monad to flatMap and map:

branch(leaf(100), leaf(200)).
flatMap(x => branch(leaf(x - 1), leaf(x + 1)))
// res4: Tree[Int] = Branch(Branch(Leaf(99),Leaf(101)),Branch(
Leaf(199),Leaf(201)))
We can also transform Trees using for comprehensions:
for {
a <- branch(leaf(100), leaf(200))
b <- branch(leaf(a - 10), leaf(a + 10))
c <- branch(leaf(b - 1), leaf(b + 1))
} yield c
// res5: Tree[Int] = Branch(Branch(Branch(Leaf(89),Leaf(91)),
Branch(Leaf(109),Leaf(111))),Branch(Branch(Leaf(189),Leaf
(191)),Branch(Leaf(209),Leaf(211))))
The monad for Option provides fail-fast seman cs. The monad for
List provides concatena on seman cs. What are the seman cs of
flatMap for a binary tree? Every node in the tree has the poten al to be
replaced with a whole subtree, producing a kind of growing or feath-
ering behaviour, reminiscent of list concatena on along two axes.

Appendix E
Solu ons for: Monad

Transformers
E.1 Monads: Transform and Roll Out
This is a rela vely simple combina on. We want Future on the outside
and Either on the inside, so we build from the inside out using an
EitherT of Future:
type Response[A] = EitherT[Future, String, A]
E.2 Monads: Transform and Roll Out Part 2
279
280 APPENDIX E. SOLUTIONS FOR: MONAD TRANSFORMERS
type Response[A] = EitherT[Future, String, A]
def getPowerLevel(ally: String): Response[Int] = {

powerLevels.get(ally) match {
case Some(avg) => EitherT.right(Future(avg))
case None => EitherT.left(Future(s"$ally unreachable"))
}
}
We request the power level from each ally and use map and flatMap to
combine the results:
def canSpecialMove(
ally1: String,
ally2: String
): Response[Boolean] =
for {
power1 <- getPowerLevel(ally1)
power2 <- getPowerLevel(ally2)
} yield (power1 + power2) > 15
We use the value method to unpack the monad stack and Await and
fold to unpack the Future and Either:
E.4. MONADS: TRANSFORM AND ROLL OUT PART 4 281
import scala.concurrent.Await
def tacticalReport(
ally1: String,
ally2: String
): String =
Await.result(
canSpecialMove(ally1, ally2).value,
1.second
) match {
case Left(msg) =>
s"Comms error: $msg"
case Right(true) =>
s"$ally1 and $ally2 are ready to roll out!"
case Right(false) =>
s"$ally1 and $ally2 need a recharge."
}

282 APPENDIX E. SOLUTIONS FOR: MONAD TRANSFORMERS
Appendix F
Solu ons for: Cartesians and

Applica ves
F.1 Cartesian Applied to Monads
We can implement product in terms of map and flatMap like so:
def product[M[_] : Monad, A, B](fa: M[A], fb: M[B]): M[(A, B)] =

fa.flatMap(a => fb.map(b => (a, b)))
Unsurprisingly, this code is equivalent to a for comprehension:
def product[M[_] : Monad, A, B](

fa: M[A],
fb: M[B]
): M[(A, B)] =
for {
a <- fa
b <- fb
283
284 APPENDIX F. SOLUTIONS FOR: CARTESIANS AND APPLICATIVES
} yield (a, b)
The seman cs of flatMap are what give rise to the behaviour for List
and Either:
product(List(1, 2), List(3, 4))

// res13: List[(Int, Int)] = List((1,3), (1,4), (2,3), (2,4))
type ErrorOr[A] = Either[Vector[String], A]
product[ErrorOr, Int, Int](

Left(Vector("Error 1")),
Left(Vector("Error 2"))
)
// res14: ErrorOr[(Int, Int)] = Left(Vector(Error 1))
Even our results for Future are a trick of the light. flatMap provides
sequen al ordering, so product provides the same. The only reason we
get parallel execu on is because our cons tuent Futures start running
before we call product. This is equivalent to the classic create-then-
atmap pa ern:
val a = Future("Future 1")
val b = Future("Future 2")
for {
x <- a
y <- b
} yield (x, y)
F.2 Form Valida on
Well be using Either and Validated so well start with some imports:
F.3. FORM VALIDATION PART 2 285
type FormData = Map[String, String]

type ErrorsOr[A] = Either[List[String], A]
type AllErrorsOr[A] = Validated[List[String], A]
The getValue rule extracts a String from the form data. Well be using
it in sequence with rules for parsing Ints and checking values, so well
dene it to return an Either:
def getValue(name: String)(data: FormData): ErrorsOr[String] =
data.get(name).
toRight(List(s"$name field not specified"))
We can create and use an instance of getValue as follows:
val getName = getValue("name") _

// getName: FormData => ErrorsOr[String] = <function1>
getName(Map("name" -> "Dade Murphy"))

// res25: ErrorsOr[String] = Right(Dade Murphy)
In the event of a missing eld, our instance returns an error message

containing an appropriate eld name:
getName(Map())
// res26: ErrorsOr[String] = Left(List(name field not specified)
)
F.3 Form Valida on Part 2
Well use Either again here. We use Either.catchOnly to consume

the NumberFormatException from toInt, and we use leftMap to turn
it into an error message:
type NumFmtExn = NumberFormatException
def parseInt(name: String)(data: String): ErrorsOr[Int] =

Right(data).
flatMap(s => Either.catchOnly[NumFmtExn](s.toInt)).
leftMap(_ => List(s"$name must be an integer"))
Note that our solu on accepts an extra parameter to name the eld
were parsing. This is useful for crea ng be er error messages, but its
ne if you leave it out in your code.
If we provide valid input, parseInt converts it to an Int:
parseInt("age")("11")
// res28: ErrorsOr[Int] = Right(11)
If we provide erroneous input, we get a useful error message:
parseInt("age")("foo")
// res29: ErrorsOr[Int] = Left(List(age must be an integer))
These deni ons use the same pa erns as above:
def nonBlank(name: String)(data: String): ErrorsOr[String] =

Right(data).
ensure(List(s"$name cannot be blank"))(_.nonEmpty)
def nonNegative(name: String)(data: Int): ErrorsOr[Int] =

Right(data).
ensure(List(s"$name must be non-negative"))(_ >= 0)
Here are some examples of use:

nonBlank("name")("Dade Murphy")
nonBlank("name")("")
// res32: ErrorsOr[String] = Left(List(name cannot be blank))
nonNegative("age")(11)
nonNegative("age")(-1)
// res34: ErrorsOr[Int] = Left(List(age must be non-negative))
We use flatMap to combine the rules sequen ally:

def readName(data: FormData): ErrorsOr[String] =
getValue("name")(data).
flatMap(nonBlank("name"))
def readAge(data: FormData): ErrorsOr[Int] =

getValue("age")(data).
flatMap(nonBlank("age")).
flatMap(parseInt("age")).
flatMap(nonNegative("age"))
The rules pick up all the error cases weve seen so far:
readName(Map("name" -> "Dade Murphy"))
readName(Map("name" -> ""))

// res37: ErrorsOr[String] = Left(List(name cannot be blank))
readName(Map())
// res38: ErrorsOr[String] = Left(List(name field not specified)

)
readAge(Map("age" -> "11"))

readAge(Map("age" -> "-1"))

// res40: ErrorsOr[Int] = Left(List(age must be non-negative))
readAge(Map())
// res41: ErrorsOr[Int] = Left(List(age field not specified))
There are a couple of ways to do this, each involving switching from

Either to Validated. One op on is to use product and map:
def readUser(data: FormData): AllErrorsOr[User] =

Cartesian[AllErrorsOr].product(
readName(data).toValidated,
readAge(data).toValidated
).map(User.tupled)
A more idioma c solu on is to use cartesian builder syntax:
def readUser(data: FormData): AllErrorsOr[User] =

(
readName(data).toValidated |@|
readAge(data).toValidated
).map(User.apply)
Both solu ons yield the same results:

readUser(Map("name" -> "Dave", "age" -> "37"))

// res43: AllErrorsOr[User] = Valid(User(Dave,37))
readUser(Map("age" -> "-1"))

// res44: AllErrorsOr[User] = Invalid(List(name field not
specified, age must be non-negative))
The need to switch back and forth between Either and Validated
is annoying. The choice of whether to use Either or Validated as a
default is determined by context. In applica on code, we typically nd
areas that favour accumula ng seman cs and areas that favour fail-fast
seman cs. We pick the data type that best suits our need and switch
to the other as necessary in specic situa ons.

Appendix G
Solu ons for: Foldable and

Traverse
G.1 Reec ng on Folds
Folding from le to right reverses the list:
List(1, 2, 3).foldLeft(List.empty[Int])((a, i) => i :: a)

Folding right to le copies the list, leaving the order intact:
List(1, 2, 3).foldRight(List.empty[Int])((i, a) => i :: a)

Note that we have to carefully specify the type of the accumulator to

avoid a type error. We use List.empty[Int] to avoid inferring the
accumulator type as Nil.type or List[Nothing]:
291
292 APPENDIX G. SOLUTIONS FOR: FOLDABLE AND TRAVERSE
List(1, 2, 3).foldRight(Nil)(_ :: _)
// found : List[Int]
// required: scala.collection.immutable.Nil.type
// List(1, 2, 3).foldRight(Nil)(_ :: _)
// ^
G.2 Scaf-fold-ing other methods
Here are the solu ons:
def map[A, B](list: List[A])(func: A => B): List[B] =

list.foldRight(List.empty[B]) { (item, accum) =>
func(item) :: accum
}
map(List(1, 2, 3))(_ * 2)
def flatMap[A, B](list: List[A])(func: A => List[B]): List[B] =

list.foldRight(List.empty[B]) { (item, accum) =>
func(item) ::: accum
}
flatMap(List(1, 2, 3))(a => List(a, a * 10, a * 100))

// res10: List[Int] = List(1, 10, 100, 2, 20, 200, 3, 30, 300)
def filter[A](list: List[A])(func: A => Boolean): List[A] =

list.foldRight(List.empty[A]) { (item, accum) =>
if(func(item)) item :: accum else accum
}
filter(List(1, 2, 3))(_ % 2 == 1)
// res11: List[Int] = List(1, 3)
G.3. TRAVERSING WITH VECTORS 293
Weve provided two deni ons of sum, one using scala.math.Numeric

(which recreates the built-in func onality accurately)
import scala.math.Numeric
def sumWithNumeric[A](list: List[A])

(implicit numeric: Numeric[A]): A =
list.foldRight(numeric.zero)(numeric.plus)
sumWithNumeric(List(1, 2, 3))
// res13: Int = 6
and one using cats.Monoid (which is more appropriate to the content

of this book):
import cats.Monoid
def sumWithMonoid[A](list: List[A])

(implicit monoid: Monoid[A]): A =
list.foldRight(monoid.empty)(monoid.combine)
sumWithMonoid(List(1, 2, 3))
// res16: Int = 6
G.3 Traversing with Vectors
The argument is of type List[Vector[Int]], so were using

the Applicative for Vector and the return type is going to be
Vector[List[Int]].
Vector is a monad, so its cartesian combine func on is based on

flatMap. Well end up with a Vector of Lists of all the possible
combina ons of List(1, 2) and List(3, 4):
listSequence(List(Vector(1, 2), Vector(3, 4)))

// res14: scala.collection.immutable.Vector[List[Int]] = Vector(
List(1, 3), List(1, 4), List(2, 3), List(2, 4))
G.4 Traversing with Vectors Part 2
With three items in the input list, we end up with combina ons of three
Ints: one from the rst item, one from the second, and one from the
third:
listSequence(List(Vector(1, 2), Vector(3, 4), Vector(5, 6)))
// res16: scala.collection.immutable.Vector[List[Int]] = Vector(
List(1, 3, 5), List(1, 3, 6), List(1, 4, 5), List(1, 4, 6),
List(2, 3, 5), List(2, 3, 6), List(2, 4, 5), List(2, 4, 6))
G.5 Traversing with Op ons
The arguments to listTraverse are of types List[Int] and Int

=> Option[Int], so the return type is Option[List[Int]]. Again,
Option is a monad, so the cartesian combine func on follows from
flatMap. The seman cs are therefore fail fast error handling: if all
inputs are even, we get a list of outputs. Otherwise we get None:
// res20: Option[List[Int]] = Some(List(2, 4, 6))
// res21: Option[List[Int]] = None

G.6. TRAVERSING WITH VALIDATED 295
G.6 Traversing with Validated
The return type here is ErrorsOr[List[Int]], which expands to

Validated[List[String], List[Int]]. The seman cs for carte-
sian combine on validated are accumula ng error handling, so the
result is either a list of even Ints, or a list of errors detailing which
Ints failed the test:
// res26: ErrorsOr[List[Int]] = Valid(List(2, 4, 6))
// res27: ErrorsOr[List[Int]] = Invalid(List(1 is not even, 3 is
not even))

Appendix H
Solu ons for: Case Study: Tes ng

Asynchronous Code
H.1 Abstrac ng over Type Constructors
Heres the implementa on:
import cats.Id
trait UptimeClient[F[_]] {
def getUptime(hostname: String): F[Int]
}
trait RealUptimeClient extends UptimeClient[Future] {

}
trait TestUptimeClient extends UptimeClient[Id] {

def getUptime(hostname: String): Id[Int]
}
Note that, because Id[A] is just a simple alias for A, we dont need
297
298APPENDIX H. SOLUTIONS FOR: CASE STUDY: TESTING ASYNCHRONOUS CODE
to refer to the type in TestUptimeClient as Id[Int]we can simply

write Int instead:
trait TestUptimeClient extends UptimeClient[Id] {

def getUptime(hostname: String): Int
}
Of course, technically speaking we dont need to redeclare getUptime

in RealUptimeClient or TestUptimeClient. However, wri ng every-
thing out helps illustrate the technique.
H.2 Abstrac ng over Type Constructors Part 2
The nal code is similar to our original implementa on of TestUptimeClient,

except we no longer need the call to Future.successful:
class TestUptimeClient(hosts: Map[String, Int])

extends UptimeClient[Id] {
def getUptime(hostname: String): Int =
hosts.getOrElse(hostname, 0)
}
H.3 Abstrac ng over Monads
The code should look like this:
class UptimeService[F[_]](client: UptimeClient[F]) {

def getTotalUptime(hostnames: List[String]): F[Int] =
H.4. ABSTRACTING OVER MONADS PART 2 299
???
// hostnames.traverse(client.getUptime).map(_.sum)
}
H.4 Abstrac ng over Monads Part 2
We can write this as an implicit parameter:
import cats.Applicative
class UptimeService[F[_]](client: UptimeClient[F])

(implicit a: Applicative[F]) {

}
or more tersely as a context bound:
class UptimeService[F[_]: Applicative]

(client: UptimeClient[F]) {

}
Note that we need to import cats.syntax.functor as well as

cats.Applicative. This is because were switching from using
future.map to the Cats generic extension method that requires an
implicit Functor parameter.

300APPENDIX H. SOLUTIONS FOR: CASE STUDY: TESTING ASYNCHRONOUS CODE
Appendix I
Solu ons for: Case Study: Pygmy

Hadoop
I.1 Implemen ng foldMap

/** Single-threaded map reduce function.
* Maps `func` over `values`
* and reduces using a `Monoid[B]`.
*/
def foldMap[A, B](values: Vector[A])(func: A => B): B =
???
I.2 Implemen ng foldMap Part 2
We have to modify the type signature to accept a Monoid for B. With

that change we can use the Monoid empty and |+| syntax as described
in Sec on 2.5.4:
301
302 APPENDIX I. SOLUTIONS FOR: CASE STUDY: PYGMY HADOOP
import cats.Monoid
def foldMap[A, B : Monoid](values: Vector[A])(func: A => B): B =

values.map(func).foldLeft(Monoid[B].empty)(_ |+| _)
We can make a slight altera on to this code to do everything in one

step:
def foldMap[A, B : Monoid](values: Vector[A])(func: A => B): B =

values.foldLeft(Monoid[B].empty)(_ |+| func(_))
I.3 Implemen ng parallelFoldMap
Here is an annotated solu on that splits out each map and fold into a
separate line of code:
import scala.concurrent.duration.Duration
def parallelFoldMap[A, B: Monoid]

(values: Vector[A])
(func: A => B): Future[B] = {
// Calculate the number of items to pass to each CPU:
val numCores = Runtime.getRuntime.availableProcessors
val groupSize = (1.0 * values.size / numCores).ceil.toInt
// Create one group for each CPU:

val groups: Iterator[Vector[A]] =
values.grouped(groupSize)
// Create a future to foldMap each group:

val futures: Iterator[Future[B]] =
I.3. IMPLEMENTING PARALLELFOLDMAP 303
groups map { group =>

Future {
group.foldLeft(Monoid[B].empty)(_ |+| func(_))
}
}
// foldMap over the groups to calculate a final result:

Future.sequence(futures) map { iterable =>
iterable.foldLeft(Monoid[B].empty)(_ |+| _)
}
}
Await.result(parallelFoldMap((1 to 1000000).toVector)(identity),
1.second)
// res18: Int = 1784293664
We can re-use our deni on of foldMap for a more concise solu on.
Note that the local maps and reduces in steps 3 and 4 of Figure 9.4 are
actually equivalent to a single call to foldMap, shortening the en re
algorithm as follows:

(values: Vector[A])
val groups: Iterator[Vector[A]] =

values.grouped(groupSize)
val futures: Iterator[Future[B]] =

groups.map(group => Future(foldMap(group)(func)))
Future.sequence(futures) map { iterable =>

iterable.foldLeft(Monoid[B].empty)(_ |+| _)
}
}
Await.result(parallelFoldMap((1 to 1000000).toVector)(identity),
1.second)
// res19: Int = 1784293664
I.4 parallelFoldMap with more Cats
Well restate all of the necessary imports for completeness:

import cats.Monoid
import cats.Traverse
import cats.instances.int._ // for Monoid

import cats.instances.future._ // for Applicative and Monad
import cats.instances.vector._ // for Foldable and Traverse
import cats.syntax.monoid._ // for |+|

import cats.syntax.foldable._ // for combineAll and foldMap
import cats.syntax.traverse._ // for traverse
Heres the implementa on of parallelFoldMap delega ng as much of

the method body to Cats as possible:
(values: Vector[A])
values
.grouped(groupSize)
.toVector
I.4. PARALLELFOLDMAP WITH MORE CATS 305
.traverse(group => Future(group.toVector.foldMap(func)))

.map(_.combineAll)
}
val future: Future[Int] =

parallelFoldMap((1 to 1000).toVector)(_ * 1000)
Await.result(future, 1.second)
// res3: Int = 500500000
The call to vector.grouped returns an Iterable[Iterator[Int]].

We sprinkle calls to toVector through the code to convert the data
back to a form that Cats can understand. The call to traverse cre-
ates a Future[Vector[Int]] containing one Int per batch. The call
to map then combines the match using the combineAll method from
Foldable.

Appendix J
Solu ons for: Case Study: Data

Valida on
J.1 Basic Combinators
We need a Semigroup for E. Then we can combine values of E using

the combine method or its associated |+| syntax:
import cats.syntax.monoid._
val semigroup = Semigroup[List[String]]
// Combination using methods on Semigroup

semigroup.combine(List("Badness"), List("More badness"))
// res2: List[String] = List(Badness, More badness)
// Combination using Semigroup syntax

List("Oh noes") |+| List("Fail happened")
// res4: List[String] = List(Oh noes, Fail happened)
Note we dont need a full Monoid because we dont need the iden ty
307
308 APPENDIX J. SOLUTIONS FOR: CASE STUDY: DATA VALIDATION
element. We should always try to keep our constraints as small as pos-

sible!
J.2 Basic Combinators Part 2
We want to report all the errors we can, so we should prefer not short-
circui ng whenever possible.
In the case of the and method, the two checks were combining are in-
dependent of one another. We can always run both rules and combine
any errors we see.
There are at least two implementa on strategies.

In the rst we represent checks as func ons. The Check data type
becomes a simple wrapper for a func on that provides our library of
combinator methods. For the sake of disambigua on, well call this im-
plementa on CheckF:
import cats.syntax.either._ // asLeft and asRight syntax
final case class CheckF[E, A](func: A => Either[E, A]) {

def apply(a: A): Either[E, A] =
func(a)
def and(that: CheckF[E, A])

(implicit s: Semigroup[E]): CheckF[E, A] =
J.3. BASIC COMBINATORS PART 3 309
CheckF { a =>
(this(a), that(a)) match {
case (Left(e1), Left(e2)) => (e1 |+| e2).asLeft
case (Left(e), Right(a)) => e.asLeft
case (Right(a), Left(e)) => e.asLeft
case (Right(a1), Right(a2)) => a.asRight
}
}
}
Lets test the behavior we get. First well setup some checks:
import cats.instances.list._ // Semigroup for List
val a: CheckF[List[String], Int] =

CheckF { v =>
if(v > 2) v.asRight
else List("Must be > 2").asLeft
}
// a: CheckF[List[String],Int] = CheckF(<function1>)
val b: CheckF[List[String], Int] =

CheckF { v =>
if(v < -2) v.asRight
else List("Must be < -2").asLeft
}
// b: CheckF[List[String],Int] = CheckF(<function1>)
val check = a and b

// check: CheckF[List[String],Int] = CheckF(<function1>)
Now run the check with some data:
check(5)
// res5: Either[List[String],Int] = Left(List(Must be < -2))
check(0)
// res6: Either[List[String],Int] = Left(List(Must be > 2, Must

be < -2))
Excellent! Everything works as expected! Were running both checks

and accumula ng errors as required.
What happens if we try to create checks that fail with a type that we
cant accumulate? For example, there is no Semigroup instance for
Nothing. What happens if we create instances of CheckF[Nothing,
A]?
val a: CheckF[Nothing, Int] =

CheckF(v => v.asRight)
val b: CheckF[Nothing, Int] =

CheckF(v => v.asRight)
We can create checks just ne but when we come to combine them we

get an error we we might expect:
val check = a and b

parameter s: cats.Semigroup[Nothing]
// val check = a and b
// ^
Now lets see another implementa on strategy. In this approach we

model checks as an algebraic data type, with an explicit data type for
each combinator. Well call this implementa on Check:
sealed trait Check[E, A] {

And(this, that)
def apply(a: A)(implicit s: Semigroup[E]): Either[E, A] =

this match {
case Pure(func) =>
func(a)

(left(a), right(a)) match {
case (Left(e1), Left(e2)) => (e1 |+| e2).asLeft
case (Left(e), Right(a)) => e.asLeft
case (Right(a), Left(e)) => e.asLeft
case (Right(a1), Right(a2)) => a.asRight
}
}
}

left: Check[E, A],
right: Check[E, A]) extends Check[E, A]

func: A => Either[E, A]) extends Check[E, A]
Lets see an example:
val a: Check[List[String], Int] =

Pure { v =>
if(v > 2) v.asRight
else List("Must be > 2").asLeft
}
// a: wrapper.Check[List[String],Int] = Pure(<function1>)
val b: Check[List[String], Int] =

Pure { v =>
if(v < -2) v.asRight
else List("Must be < -2").asLeft
}
// b: wrapper.Check[List[String],Int] = Pure(<function1>)
val check = a and b

// check: wrapper.Check[List[String],Int] = And(Pure(<function1
>),Pure(<function1>))
While the ADT implementa on is more verbose than the func on

wrapper implementa on, it has the advantage of cleanly separa ng
the structure of the computa on (the ADT instance we create) from

the process that gives it meaning (the apply method). From here we
have a number of op ons:
inspect and refactor checks a er they are created;

move the apply interpreter out into its own module;
implement alterna ve interpreters providing other func onality
(for example visualizing checks).
Because of its exibility, we will use the ADT implementa on for the
rest of this case study.
The implementa on of apply for And is using the pa ern for applica ve
functors. Either has an Applicative instance, but it doesnt have the
seman cs we want/ It fails fast instead of accumula ng errors.
If we want to accumulate errors Validated is a more appropriate ab-
strac on. As a bonus, we get more code reuse because we can lean on
the applica ve instance of Validated in the implementa on of apply.
Heres the complete implementa on:

And(this, that)

this match {
case Pure(func) =>
func(a)

}
}

left: Check[E, A],

func: A => Validated[E, A]) extends Check[E, A]
This reuses the same technique for and. We have to do a bit more
work in the apply method. Note that its ok to short-circuit in this case
because the choice of rules is implicit in the seman cs of or.
import cats.data.Validated._ // Valid and Invalid

And(this, that)
def or(that: Check[E, A]): Check[E, A] =

Or(this, that)

this match {
case Pure(func) =>
func(a)


left(a) match {
case Valid(a) => Valid(a)
case Invalid(e1) =>
right(a) match {
case Valid(a) => Valid(a)
}
}
}
}

left: Check[E, A],

left: Check[E, A],

func: A => Validated[E, A]) extends Check[E, A]
J.6 Checks
If you follow the same strategy as Predicate you should be able to

create code similar to the below:
J.7. CHECKS PART 2 315

def apply(in: A)(implicit s: Semigroup[E]): Validated[E, B]
def map[C](f: B => C): Check[E, A, C] =

Map[E, A, B, C](this, f)
}
object Check {
def apply[E, A](pred: Predicate[E, A]): Check[E, A, A] =
Pure(pred)
}
final case class Map[E, A, B, C](

check: Check[E, A, B],
func: B => C) extends Check[E, A, C] {
def apply(in: A)(implicit s: Semigroup[E]): Validated[E, C] =

check(in).map(func)
}

pred: Predicate[E, A]) extends Check[E, A, A] {
def apply(in: A)(implicit s: Semigroup[E]): Validated[E, A] =

pred(in)
}
J.7 Checks Part 2
Its the same implementa on strategy as before with one wrinkle:

Validated doesnt have a flatMap method. To implement flatMap
we must momentarily switch to Either and then switch back to
Validated. The withEither method on Validated does exactly this.

From here we can just follow the types to implement apply.

def flatMap[C](f: B => Check[E, A, C]) =

FlatMap[E, A, B, C](this, f)
// other methods...
}
final case class FlatMap[E, A, B, C](

func: B => Check[E, A, C]) extends Check[E, A, C] {
def apply(a: A)(implicit s: Semigroup[E]): Validated[E, C] =

check(a).withEither(_.flatMap(b => func(b)(a).toEither))
}
// other data types...
J.8 Checks Part 3
Heres a minimal deni on of andThen and its corrsponding AndThen

class:
import Check._

J.9. RECAP 317
def andThen[C](that: Check[E, B, C]): Check[E, A, C] =

AndThen[E, A, B, C](this, that)
}
final case class AndThen[E, A, B, C](

check1: Check[E, A, B],
check2: Check[E, B, C]) extends Check[E, A, C] {

check1(a).withEither(_.flatMap(b => check2(b).toEither))
}
J.9 Recap
Heres our nal implementaton, including some dying and repackaging

of the code:
import cats.syntax.validated._ // .valid and .invalid syntax
import cats.data.Validated._ // Valid and Invalid patterns
Here is our complete implementa on of Predicate, including the

and and or combinators and a Predicate.apply method to create a
Predicate from a func on:
import Predicate._
def and(that: Predicate[E, A]): Predicate[E, A] =

And(this, that)
def or(that: Predicate[E, A]): Predicate[E, A] =

Or(this, that)

this match {
case Pure(func) =>
func(a)


left(a) match {
case Invalid(e1) =>
right(a) match {
}
}
}
}
object Predicate {


func: A => Validated[E, A]) extends Predicate[E, A]
def apply[E, A](f: A => Validated[E, A]): Predicate[E, A] =

Pure(f)
def lift[E, A](error: E, func: A => Boolean): Predicate[E, A]

J.9. RECAP 319
=
Pure(a => if(func(a)) a.valid else error.invalid)
}
Here is a complete implementa on of Check. Due to a type inference

bug in Scalas pa ern matching, weve switched to implemen ng apply
using inheritance:

import Check._
def map[C](f: B => C): Check[E, A, C] =

Map[E, A, B, C](this, f)
def flatMap[C](f: B => Check[E, A, C]) =

FlatMap[E, A, B, C](this, f)
def andThen[C](next: Check[E, B, C]): Check[E, A, C] =

AndThen[E, A, B, C](this, next)
}
object Check {
final case class Map[E, A, B, C](
func: B => C) extends Check[E, A, C] {

check(a) map func
}
final case class FlatMap[E, A, B, C](

func: B => Check[E, A, C]) extends Check[E, A, C] {

check(a).withEither(_.flatMap(b => func(b)(a).toEither))
}
final case class AndThen[E, A, B, C](

next: Check[E, B, C]) extends Check[E, A, C] {

check(a).withEither { _.flatMap (b => next(b).toEither) }
}
final case class Pure[E, A, B](

func: A => Validated[E, B]) extends Check[E, A, B] {
def apply(a: A)(implicit s: Semigroup[E]): Validated[E, B] =

func(a)
}
final case class PurePredicate[E, A](

pred: Predicate[E, A]) extends Check[E, A, A] {

pred(a)
}
def apply[E, A](pred: Predicate[E, A]): Check[E, A, A] =

PurePredicate(pred)
def apply[E, A, B](func: A => Validated[E, B]): Check[E, A, B]

=
Pure(func)
}
J.10 Recap Part 2
Heres our reference solu on. Implemen ng this required more

thought than we expected. Switching between Check and Predicate
J.10. RECAP PART 2 321
at appropriate places felt a bit like guesswork ll we got the rule into
our heads that Predicate doesnt transform its input. With this rule
in mind things went fairly smoothly. In later sec ons well make some
changes that make the library easier to use.
import cats.data.{NonEmptyList, OneAnd, Validated}
Heres the implementa on of checkUsername:

// A username must contain at least four characters
// and consist entirely of alphanumeric characters
val checkUsername: Check[Errors, String, String] =

Check(longerThan(3) and alphanumeric)
And heres the implementa on of checkEmail, built up from a number

of smaller components:
// An email address must contain a single `@` sign.
// Split the string at the `@`.
// The string to the left must not be empty.
// The string to the right must be
// at least three characters long and contain a dot.
val splitEmail: Check[Errors, String, (String, String)] =

Check(_.split('@') match {
case Array(name, domain) =>
(name, domain).validNel[String]
case other =>

"Must contain a single @ character".
invalidNel[(String, String)]
})
val checkLeft: Check[Errors, String, String] =

Check(longerThan(0))
val checkRight: Check[Errors, String, String] =

Check(longerThan(3) and contains('.'))
val joinEmail: Check[Errors, (String, String), String] =

Check { case (l, r) =>
(checkLeft(l) |@| checkRight(r)).map(_+"@"+_) }
val checkEmail: Check[Errors, String, String] =

splitEmail andThen joinEmail
Finally, heres a check for a User that depends on checkUsername and

checkEmail:
final case class User(username: String, email: String)
def createUser(
username: String,
email: String): Validated[Errors, User] =
(checkUsername(username) |@| checkEmail(email)).map(User)
We can check our work by crea ng a couple of example users:

createUser("Noel", "[email protected]")
// res14: cats.data.Validated[wrapper.Errors,User] = Valid(User(
Noel,[email protected]))
createUser("", "dave@underscore@io")
// res15: cats.data.Validated[wrapper.Errors,User] = Invalid(
NonEmptyList(Must be longer than 3 characters, Must contain
a single @ character))
One dis nct disadvantage of our example is that it doesnt tell us where
the errors came from. We can either achieve that through judicious
manipula on of error messages, or we can modify our library to track
error loca ons as well as messages. Tracking error loca ons is outside
the scope of this case study, so well leave this as an exercise to the
reader.
J.11. KLEISLIS 323
J.11 Kleislis
Heres an abbreviated deni on of run. Like apply, the method must

accept an implicit Semigroup:

def run(implicit s: Semigroup[E]): A => Either[E, A] =
(a: A) => this(a).toEither
def apply(a: A): Validated[E, A] =

??? // etc...
// other methods...
}
J.12 Kleislis Part 2
Working around limita ons of type inference can be quite frustra ng

when wri ng this code, Working out when to convert between
Predicates, func ons, and Validated, and Either simplies things,
but the process is s ll complex:
import cats.data.{Kleisli, NonEmptyList, Validated}

Here is the preamble we suggested in the main text of the case study:
type Errors = NonEmptyList[String]
def error(s: String): NonEmptyList[String] =

NonEmptyList(s, Nil)
type Result[A] = Either[Errors, A]
type Check[A, B] = Kleisli[Result, A, B]
def check[A, B](func: A => Result[B]): Check[A, B] =

Kleisli(func)
def checkPred[A](pred: Predicate[Errors, A]): Check[A, A] =

Kleisli[Result, A, A](pred.run)
Our base predicate deni ons are essenitally unchanged:
def longerThan(n: Int): Predicate[Errors, String] =

Predicate.lift(
error(s"Must be longer than $n characters"),
str => str.size > n)
val alphanumeric: Predicate[Errors, String] =

Predicate.lift(
error(s"Must be all alphanumeric characters"),
str => str.forall(_.isLetterOrDigit))
def contains(char: Char): Predicate[Errors, String] =

Predicate.lift(
error(s"Must contain the character $char"),
str => str.contains(char))
def containsOnce(char: Char): Predicate[Errors, String] =

Predicate.lift(
error(s"Must contain the character $char only once"),
str => str.filter(c => c == char).size == 1)
Our username and email examples are slightly dierent in that we make
use of check() and checkPred() in dierent situa ons:
J.12. KLEISLIS PART 2 325
val checkUsername: Check[String, String] =

checkPred(longerThan(3) and alphanumeric)
val splitEmail: Check[String, (String, String)] =

check(_.split('@') match {
case Array(name, domain) =>
Right((name, domain))
case other =>

Left(error("Must contain a single @ character"))
})
val checkLeft: Check[String, String] =

checkPred(longerThan(0))
val checkRight: Check[String, String] =

checkPred(longerThan(3) and contains('.'))
val joinEmail: Check[(String, String), String] =

check { case (l, r) =>
(checkLeft(l) |@| checkRight(r)).map(_+"@"+_) }
val checkEmail: Check[String, String] =

splitEmail andThen joinEmail
Finally, we can see that our createUser example works as expected

using Kleisli:
final case class User(username: String, email: String)
def createUser(username: String, email: String): Either[Errors,

User] = (
checkUsername.run(username) |@|
checkEmail.run(email)
).map(User)
createUser("Noel", "[email protected]")
// res16: Either[Errors,User] = Right(User(Noel,noel@underscore.
io))
createUser("", "dave@underscore@io")
// res17: Either[Errors,User] = Left(NonEmptyList(Must be longer
than 3 characters))

Appendix K
Solu ons for: Case Study:

Commuta ve Replicated Data
Types
K.1 GCounter Implementa on
Hopefully the descrip on above was clear enough that you can get to
an implementa on like the below.
final case class GCounter(counters: Map[String, Int]) {

def increment(machine: String, amount: Int) =
GCounter(counters + (machine -> (amount + counters.getOrElse
(machine, 0))))
def get: Int =

counters.values.sum
def merge(that: GCounter): GCounter =

GCounter(that.counters ++ {
for((k, v) <- counters) yield {
327
328APPENDIX K. SOLUTIONS FOR: CASE STUDY: COMMUTATIVE REPLICATED DATA
k -> (v max that.counters.getOrElse(k,0))

}
})
}
K.2 BoundedSemiLa ce Instances
Its natural to place the instance in the companion object of

BoundedSemiLattice so they are in the implicit scope without
impor ng them.
Implemen ng the instance for Set is good prac ce with using implicit
methods.
def empty: A
}
a1 max a2
val empty: Int = 0

}

a1 union a2
val empty: Set[A] =

K.3. GENERIC GCOUNTER 329
Set.empty[A]
}
}
K.3 Generic GCounter

final case class GCounter[A](counters: Map[String,A]) {

def increment(machine: String, amount: A)(implicit m: Monoid[A
]) =
GCounter(counters + (machine -> (amount |+| counters.
getOrElse(machine, m.empty))))
def get(implicit m: Monoid[A]): A =

this.counters.foldMap(identity)
def merge(that: GCounter[A])(implicit b: BoundedSemiLattice[A

]): GCounter[A] =
GCounter(this.counters |+| that.counters)
}

Advanced Scala

Uploaded by

Copyright:

Available Formats

Advanced Scala

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Advanced Scala

Uploaded by

Copyright:

Available Formats

What is the purpose of the book?

What is the purpose of the book?

What are type classes and how are they used?

What are type classes and how are they used?

Advanced Scala

Noel Welsh and Dave Gurnell

Copyright 2014-16 Noel Welsh and Dave Gurnell.

Advanced Scala with Cats

Copyright 2014-16 Noel Welsh and Dave Gurnell.

Published by Underscore Consul ng LLP, Brighton, UK.

Copies of this, and related topics, can be found at

Underscore is a team of developers specialising in Scala and func onal

In addi on to wri ng so ware, we provide other training courses,

Notes on the Pre-Release Edi on . . . . . . . . . . . . . . . 16

Conven ons Used in This Book . . . . . . . . . . . . . . . . 17

Typographical Conven ons . . . . . . . . . . . . . . . 17

1.1 Anatomy of a Type Class . . . . . . . . . . . . . . . . . 23

1.1.1 The Type Class . . . . . . . . . . . . . . . . . . 24

1.1.2 Type Class Instances . . . . . . . . . . . . . . . 24

1.1.4 Exercise: Printable Library . . . . . . . . . . . . 26

2 Monoids and Semigroups 39

2.1 Deni on of a Monoid . . . . . . . . . . . . . . . . . . 41

2.5.2 Obtaining Instances . . . . . . . . . . . . . . . 45

3.1 Examples of Functors . . . . . . . . . . . . . . . . . . 55

3.6.2 Invariant functors and the imap method . . . . . 69

4.1 What is a Monad? . . . . . . . . . . . . . . . . . . . . 78

4.5.2 Evals models of evalua on . . . . . . . . . . . . 102

5 Monad Transformers 131

5.1 A Transforma ve Example . . . . . . . . . . . . . . . . 132

5.2.2 Building Monad Stacks . . . . . . . . . . . . . . 136

6 Cartesians and Applica ves 147

6.1 Cartesian . . . . . . . . . . . . . . . . . . . . . . . . . 149

6.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 169

7 Foldable and Traverse 171

7.1 Foldable . . . . . . . . . . . . . . . . . . . . . . . . . . 171

7.1.1 Folds and Folding . . . . . . . . . . . . . . . . . 172

7.1.2 Exercise: Reec ng on Folds . . . . . . . . . . . 173

7.1.3 Exercise: Scaf-fold-ing other methods . . . . . . 174

7.1.4 Foldable in Cats . . . . . . . . . . . . . . . . . . 174

7.2 Traverse . . . . . . . . . . . . . . . . . . . . . . . . . . 179

7.2.1 Traversing with Futures . . . . . . . . . . . . . . 179

7.2.2 Traversing with Applica ves . . . . . . . . . . . 182

7.2.3 Traverse in Cats . . . . . . . . . . . . . . . . . . 186

7.2.4 Unapply, traverseU, and sequenceU . . . . . . . . 187

7.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 189

II Case Studies 191

8 Case Study: Tes ng Asynchronous Code 193

8.1 Abstrac ng over Type Constructors . . . . . . . . . . . 195

8.2 Abstrac ng over Monads . . . . . . . . . . . . . . . . 196

8.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . . 198

9 Case Study: Pygmy Hadoop 199

9.1 Parallelizing map and fold . . . . . . . . . . . . . . . . . 199

9.2 Implemen ng foldMap . . . . . . . . . . . . . . . . . . 201

9.3 Parallelising foldMap . . . . . . . . . . . . . . . . . . . 203

9.3.1 Futures, Thread Pools, and Execu onContexts . . 205

9.3.2 Dividing Work . . . . . . . . . . . . . . . . . . 207

9.3.3 Implemen ng parallelFoldMap . . . . . . . . . . 207

9.3.4 parallelFoldMap with more Cats . . . . . . . . . . 208

9.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 208

9.4.1 Batching Strategies in the Real World . . . . . . 209