dean@deanwampler.

com The Haystack, Oregon

@deanwampler
polyglotprogramming.com

Become a Better Developer

with Functional Programming
OSCON, July 26, 2011 1
Friday, April 12, 13
All photos 2010 Dean Wampler, unless other noted. Most of my photos are here: http://www.flickr.com/photos/
deanwampler/. Most are from the Oregon coast, taken before last years OSCON. Some are from the San Francisco area,
including the Bay. A few are from other places Ive visited over the years.

(The Haystack, Cannon Beach, Oregon)

 Functional
Programming
for Java Developers

Dean Wampler

polyglotprogramming.com/
programmingscala.com
fpjava 2
Friday, April 12, 13
I got interested in FP about 5 years ago when everyone was talking about it. I decided it was time to learn myself and I expected
to pick up some good ideas, but otherwise remain primarily an object-oriented developer. Actually, it caused me to rethink my
views and now I tend to use FP more than OOP. This tutorial explains why.
 The problems of our time.
What is Functional
Programming?
Better data structures.
Better concurrency.
Better objects.
3

Friday, April 12, 13

Outline.

(Nehalem State Park, Oregon)

 Nehalem State Park, Oregon

The problems
of our time.
4

Friday, April 12, 13

What problems motivate the need for change, for which Functional Programming is well suited?

(Nehalem State Park, Oregon)

 Concurrency

San Francisco Bay

Friday, April 12, 13
Concurrency is the reason people started discussing FP, which had been primarily an academic area of interest. FP has useful
principles that make concurrency more robust and easier to write.

(San Francisco Bay)

 Horizontal scaling
is
unavoidable.

Friday, April 12, 13

The reason everyone is talking about concurrency is because weve hit the limit of vertical scalability of Moores Law. Now were
scaling horizontally, so we need to know how to exploit multiple CPUs and cores.

(At dusk flying over the Midwest - lightened)

 Multithreaded
programming
is the
assembly language
of concurrency.
7

Friday, April 12, 13

Multithreaded programming is hard and too low-level for most needs.
 Were
Drowning
in Data.

...
Friday, April 12, 13
Not just these big companies, but many organizations have lots of data they want to analyze and exploit.

(San Francisco)
 Mud, Death Hollow Trail, Utah

We need better
modularity.

Friday, April 12, 13

I will argue that objects havent been the modularity success story we expected 20 years ago, especially in terms of reuse. Im
referring to having standards that actually enable widespread interoperability, like electronics, for example. Ill argue that object
abstractions are too high-level and too open-ended to work well.

(Mud near Death Hollow in Utah.)

 Half Dome, Yosemite NP

We need
better
agility.

Friday, April 12, 13

Schedules keep getting shorter. The Internet weeded out a lot of process waste, like Big Documents Up Front, UML design, etc.
From that emerged XP and other forms of Agile. But schedules and turnaround times continue to get shorter.

(Ascending the steel cable ladder up the back side of Half Dome, Yosemite National Park)
 Maligne Lake, Jasper Nat. Park

We need a return
to simplicity.
Friday, April 12, 13
Every now and then, we need to stop, look at what were doing, and remove the cruft weve accumulated. If youre a Java
programmer, recall how efforts like the Spring Framework forced a rethinking of J2EE. I claim that a lot of the code we write,
specifically lots of object middleware, is cruft. Functional programming isnt *simple*, but in my view it reflects a refocusing on
core principles and minimally-sufficient design.

(Maligne Lake, Near Jasper National Park, Jasper, Alberta)

 What is Functional
Programming?

Nehalem State Park, Oregon 12

Friday, April 12, 13

This is rich field, so I cant cover everything. Ill mention the things that I believe are most useful to know for beginners and
those curious about FP.

(Nehalem State Park, Oregon)

 Functional
Programming
is inspired by
Mathematics.
13

Friday, April 12, 13

FP follows the rules for the behavior of functions, variables, and values in mathematics. Everything else falls out from there...
 What is Functional
Programming?

Immutable
Values
14

Friday, April 12, 13

First, values in FP are immutable, but variables that point to different values, arent.
 Immutable Values

y = sin(x)
1 = sin(/2)
x and y are variables.
Once you assign a value to x,
you fix the value assigned to y.
15

Friday, April 12, 13

First, values in FP are immutable, but variables that point to different values, arent.
 Immutable Values

y = sin(x)

You can start over with new values

assigned to the same variables.
But you never modify the values, themselves.
16

Friday, April 12, 13

 Immutable Values

+= 1

What would that mean?

17

Friday, April 12, 13

This would make no sense.
 Immutable Values

If a value is immutable,
synchronizing access is no longer necessary!

Concurrency becomes far easier.

18

Friday, April 12, 13

Of course, you dont need functional programming to make values immutable.
 Java
class List<T> {
final T _head;
final List<T> _tail;
T head() {return _head;}
List<T> tail() {return _tail;}

List (T head, List<T> tail) {

_head = head; _tail = tail;
}

} 19

Friday, April 12, 13

Ill provide some Java examples, but mostly Ruby examples, since its syntax is compact and relatively easy to learn - both good
for presentations like this!
Heres a linked list that well use a lot. It is defined by the head of the list (the left-most element) and the tail or rest of the list,
itself a list! Make the fields final in Java and dont provide setters. (Im dropping public, private, etc. for clarity.) List objects will
be immutable, although we cant control the mutability of T objects!
If you dont like static typing, at least appreciate the fact that you know immediately that tail is also a List<T>.
Im not using JavaBeans conventions here to reduce unnecessary clutter. In fact, is there any reason to NOT make the fields
public?
 Java

List<? extends Object> list =

new List<Integer>(1,
new List<Integer>(2,
new List<Integer>(3, )));

20

Friday, April 12, 13

Creating a list (well see less verbose syntax later).
Im showing *covariant typing*, a poorly understood feature in Java (and it could be implemented better by the language).
Read this as, I declared list to be of List<T> for any subtype T of Object, so List<String> is a subtype of List<Object>, and a
valid object to assign to list. NOTE: this is *different* than assigning Integers (and Strings and Floats and) to a List<Object>.
How should we terminate this list?? What should the final tail be?? Well come back to that.
 Ruby
class List
attr_reader :head, :tail
def initialize(head, tail)
@head = head
@tail = tail
end

end

21

Friday, April 12, 13

So, dont use attr_accessor or attr_writer in Ruby.
If you dont like dynamic typing, at least appreciate the compact, clean syntax.
 Ruby

list = List.new(1,
List.new(2,
List.new(3, )))

22

Friday, April 12, 13

Creating a list (well see less verbose syntax later)
How should we terminate this list?? What should the final tail be?? Well come back to that.
 What is Functional
Programming?

Side-effect
free
functions 23

Friday, April 12, 13

Math functions dont have side effects. They dont change object or global state. All work is returned and assigned to y.
 Functions

y = sin(x)

sin(x) does not change state anywhere!

24

Friday, April 12, 13

Math functions dont have side effects. They dont change object or global state. All work is returned and assigned to y.
 Referential
Transparency
1 = sin(/2)

We can replace sin(/2) with 1.

We can replace 1 with sin(/2)!
Functions and values are interchangeable
25

Friday, April 12, 13

A crucial implication of functions without side effects us that functions and values are interchangeable. A mundane benefit is
that its easy to for an implementation to cache previous work for a given input value, for efficiency. But there are more profound
benefits.
 Functions

y = sin(x)

sin(x) can be used anywhere.

I dont have to worry about the
context where its used
26

Friday, April 12, 13

This makes testing, reuse, and concurrency much easier if I dont have to worry about external state modifications.
 Side-effect free methods
and immutable objects
class List

def add(item)
List.new(item, self)
end

end
Make your methods side-effect free.
Create new instances. 27

Friday, April 12, 13

Dont modify the existing list, make a new one.
(We wont have time discuss how you optimize making copies to minimize that overhead)
 What is Functional
Programming?

First-class
functions
28

Friday, April 12, 13

 First Class Functions
i = 1
l = List.new(i, )
f = lambda { |x|
puts "Hello, #{x}!"
}
First Class: values that can be assigned to
variables, pass to and from functions.
Lambda is a common name for functions.
29

Friday, April 12, 13

A thing is first class in a language if you can use it as a value, which means you can assign it to variables, pass it as an
argument to a function and return it from a function. In Ruby, objects, even classes are first class. Methods are not. Lambdas are
rubys way of defining anonymous functions (A second mechanism, Procs, is similar).
The term lambda comes from Lambda Calculus, a mathematical formalism developed in the 30s that explored how functions
should work. The lambda symbol was used to represent anonymous functions.
 First Class Functions
f = lambda { |x|
puts "Hello, #{x}!"
}
def usearg(arg, &func)
func.call(arg)
end

usearg("Dean", &f)
# "Hello, Dean!"
30

Friday, April 12, 13

There are other syntaxes for defining and calling Ruby procs/lambda.
 First Class Functions

Well see how first-class functions let us build

modular, reusable and composable tools.
31

Friday, April 12, 13

This is one of the most powerful concepts in programming.
 Java?
public interface
Function1Void<A> {
void call(A arg); // arbitrary
}

public static void usearg(

String arg,
Function1Void<String> func) {
func.call(arg);
}}
32

Friday, April 12, 13

Java doesnt have first-class functions. The closest we can come are function objects. Often these interfaces are instantiated as
anonymous inner classes.
I picked an arbitrary name for the function.
 Java?
public static void main() {
usearg("Dean",
new Function1Void<String>(){
public void call(String s){
System.out.printf(
"Hello, %s!\n", s);
}
});
}
33

Friday, April 12, 13

Verbose, ugly and hard to follow.
The ability to communicate ideas in concise terms really matters!! Your brain expends a lot of effort parsing all this code!
 Java?
public interface
Function1Void<A> {
void call(A arg);
}
Another

example
public interface
Function2<A1,A2,R> { function.
R call(A1 arg1, A2 arg2);
} How many one-off interfaces
could you replace with uniform
abstractions like these? 34

Friday, April 12, 13

Java APIs must have hundreds of *structurally* identical interfaces, each with its own ad-hoc interface and method name.
Imagine how much memorization reduction would be facilitated if they were all replaced with uniform abstractions like these?

Side note: Java 8 will *hopefully*, *finally* add a lambda syntax to eliminate lots of this boilerplate.
 Higher-order
Functions
def usearg(arg, &func)
func.call(arg)
end

Functions that take other functions as

arguments or return them as results
are called higher-order functions.
35

Friday, April 12, 13

Theres a technical reason for the name higher-order that we wont have time to discuss, but since youll hear this term used, I
wanted to define it in the way people typically use the term.
 What is Functional
Programming?

Recursion
36

Friday, April 12, 13

 Recursion
class List

def empty?
false # always??
end
def to_s
empty? ?
"()" :
"(#{head.to_s},#{tail.to_s})"
end
tail.to_s is a recursive call.
end 37

Friday, April 12, 13

Recursion is a natural tool for working with recursive data structures, like List. Its also a way to traverse data structures without
mutable loop counters!
Note that we havent shown how to represent an empty list! We will.
If the list is empty, we terminate the recursion, returning the string (). Otherwise, we form a string by calling head.to_s and
tail.to_s. The latter is a recursive call. (We could have left off the to_s here, but to make things explicit...
 Recursion
puts List.new(1,
List.new(2,
List.new(3, EMPTY) # ??

=> "(1,(2,(3,())))"

Well define EMPTY shortly

38

Friday, April 12, 13

Well define EMPTY shortly, which will have an empty? method that returns false.
If we run this code, we get the string shown. Note the nesting of parentheses, reflecting the nesting of structure!
Nehalem State Park, Oregon

Better data
structures
39

Friday, April 12, 13

(Nehalem State Park, Oregon)
 No Nulls?

Better data
structures
40

Friday, April 12, 13

(Nehalem State Park, Oregon)
 Nulls are a serious
source of bugs.

41

Friday, April 12, 13

We know that nulls are a pain...
 If values are immutable,
can we avoid using
nulls?

42

Friday, April 12, 13

Functional programming emphasizes rigor and always having valid values assigned to variables. Can we eliminate the use of
nulls?
 What should happen?
Map<String,String> capitals = ;

String cap =
capitals.get("Camelstan");
String cap2 = cap.toLowerCase();

NullPointerException!!

cap is of type String or Null?

or is Null a subtype of String?
43

Friday, April 12, 13

Lets return to Java, because this section makes more sense for statically-typed languages.
We have a map of the capital cities for the worlds countries. We ask for the capital of Camelstan, then try to use the value.
We forgot to check for null.
If null were of type Null, then could tail be thought of as a variable of type String OR Null?
Actually, Java *effectively* has the notion of a Null type that is a subtype of all other (reference) types, but not explicitly.
 What should happen?
String cap =
capitals.get("Camelstan");

Map<K,V>.get signature:
V get(Object key);

Its lying slightly, because

a V or a null is returned.
44

Friday, April 12, 13

The signature for Map.get doesnt tell the full story of what might happen. Were used to the fact that null might be returned, but
naively, reading this signature, we have every right to believe a valid string will *always* be returned.
 What should happen?
What if we changed the signature?
Option<V> get(Object key);

Option<String> cap =
capitals.get("Camelstan");

Explicitly indicate that a value

might exist or not; it is optional.
45

Friday, April 12, 13

The signature for Map.get doesnt tell the full story of what might happen. Were used to the fact that null might be returned, but
naively, reading this signature, we have every right to believe a valid string will *always* be returned.
Type safety will prevent us from forgetting that the value is optional, in the same way that we can forget that null is returned.
We have to handle the option explicitly.
 Option
interface Option<T> {
boolean hasValue();
T get();
}
final class Some<T> extends
Option<T> {
boolean hasValue(){return true;}
T get() {return t;}
private T t;
// constructor...
} 46

Friday, April 12, 13

Here is the Option interface and the first of TWO implementing classes, which is why its declared final. Some is the object
instantiated when there IS a value.
Some people hate final because its seen as bad for testing (you cant replace the object with a test-oriented subclass). Theres
no need to EVER do that here, and maintaining type safety (at least as much as we can) is more important.
 Option
interface Option<T> {
boolean hasValue();
T get();
}
final class None<T> extends
Option<T> {
boolean hasValue(){return false;}
T get() {throw new Exception();}
}

47

Friday, April 12, 13

Here is the Option interface and the second of the TWO implementing classes. None is the object instantiated when there ISNT a
value.
 An optional value
Map<String,String> capitals = ;
Option<String> cap =
capitals.get("Camelstan");
if (cap.hasValue()) {
String cap2 =
cap.get().toLowerCase();

} else {
logError("Camelstan ");
}
48

Friday, April 12, 13

This code may be a little verbose, but its not much different than the normal null checks youre supposed to do. Also, there are
mechanisms that can be used, like providing iteration over this collection, that can eliminate the explicit hasValue check in
many cases. For example, if you dont care that there is no value; youre just processing a bunch of things, some with values,
some without, then you can easily ignore the without cases...
 Replace Nulls
with Options.

49

Friday, April 12, 13

Change your APIs to use Options whenever Nulls are possible, either as return values or as optional argument values!
 Lists

Better data
structures
50

Friday, April 12, 13

Lets look at one of the functional data structures, List, which weve already looked at a bit, but we need to explore further.
 Lets finish List
class List Previously...

def empty?; false; end
def to_s
empty? ?
"()" :
"(#{head},#{tail})"
end

end
51

Friday, April 12, 13

Lets finish the implementation of List. In particular, lets figure out how to terminate the list, which means representing an
empty list for the tail.
I changed empty? to be all on one line, compared to the previously shown implementation.
I removed the explicit calls to to_s on head and tail in self.to_s; they will be called implicitly.
 A separate object to
class List represent empty.

EMPTY = List.new(nil,nil)
def EMPTY.head
raise "EMPTY list has no head!!"
end
def EMPTY.tail
raise "EMPTY list has no tail!!"
end
def EMPTY.empty?; true; end
def EMPTY.to_s; "()"; end
end
52

Friday, April 12, 13

We declare a *constant* named EMPTY, of type List. We use nil for the head and tail, but they will never be referenced, because
we redefine the head and tail methods for this object (so called singleton methods) to raise exceptions. We also define empty?
to return true and to_s to return ().
By overriding the methods on the instance, weve effectively given it a unique type.
(Theres a more short-hand syntax for redefining these methods, but for simplicity, Ill just use the syntax shown. )
NOTE: It would be reasonable for EMPTY.tail to return itself!
 class List Rewrite to_s.

def to_s
"(#{head},#{tail})"
end

def EMPTY.to_s; "()"; end

end
List.to_s is recursive, but
EMPTY.to_s will terminate the
recursion with no conditional test! 53

Friday, April 12, 13

The check for empty is gone in to_s! Its not an infinite recursion, though, because all lists end with EMPTY, which will terminate
the recursion.
Weve replaced a condition test with structure, which is actually a classic OO thing to do.
 Recall...
puts List.new(1,
List.new(2,
List.new(3, EMPTY)

=> "(1,(2,(3,())))"

54

Friday, April 12, 13

 Lists are represented
by two types:
List and EMPTY.

55

Friday, April 12, 13

For functional linked lists, only two types are used to represent all of them, List and EMPTY. That let us use the structural
difference to manage recursion without conditional tests, among other benefits. We used nil to declare EMPTY, but never used
those values.
 List is an
Algebraic Data Type.

56

Friday, April 12, 13

The name comes from Category Theory, which we wont get into. The key thing to note is that this is a constrained type
hierarchy. There are only two allowed subtypes that can implement the abstraction. (Since this is Ruby, we didnt define an
interface with the key methods.)
 filter, map, fold

Better data
structures
57

Friday, April 12, 13

Lets look at the 3 fundamental operations on data structures and understand their power.
 Filter, map, fold
Return a new collection with
filter some elements removed.

Return a new collection with

map each element transformed.

Compute a new result by

fold accumulating each element.

All take a function argument. 58

Friday, April 12, 13

The function argument tells each method what to do.
 Filter, map, fold
Ruby

filter find_all

map map

fold inject
59

Friday, April 12, 13

These names are not always used in different languages. Java doesnt even have these concepts in its collections! However, some
3rd-party libraries provide them.
 Add map to List
f takes one arg, each item,
and returns a new value for
the new list.
def map(&f)
t = tail.map(&f)
List.new(f.call(head), t)
end
def EMPTY.map(&f); self; end
f.call(head) converts
head into something new.
60

Friday, April 12, 13

Add map first, because its the easiest. Note that we will show the implementations for both List and EMPTY together, to compare
and contrast and to make the behavior of the recursion clear.
 Example of map
list = # 1,2,3,4
lm = list.map {|x| x*x}
puts "list: #{list}"
puts "lm: #{lm}"
# => list: (1,(2,(3,(4,()))))
# => lm: (1,(4,(9,(16,()))))

61

Friday, April 12, 13

Demonstrate mapping a list of 4 integers to their squares. Note that we didnt modify the original list.
 Add filter to List
f takes one arg, each item,
and returns true or false.
def filter(&f)
t = tail.filter(&f)
f.call(head) ?
List.new(head, t) : t
end
def EMPTY.filter(&f); self; end
f.call(head) returns
true or false (keep or discard)
62

Friday, April 12, 13

f.call(head) returns true if we keep the element or false if we discard it. If true, we return a new list with head and whatever t is.
Otherwise, we just return t.
 Example of filter
list = # 1,2,3,4
lf = list.filter {|x| x%2==1}
puts "list: #{list}"
puts "lf: #{lf}"
# => list: (1,(2,(3,(4,()))))
# => lf: (1,(3,()))

63

Friday, April 12, 13

Demonstrate filtering a list of 4 integers to create a new list with just the odd values. Note that we didnt modify the original list.
 There are two folds:
foldl (left) and
foldr (right).

64

Friday, April 12, 13

There are two folds because of the way they group the elements as they parse them, either grouping from the left or the right,
as well see.
 Add foldl to List
accum is the f takes two args, accum
accumulator. and each item, and
returns a new accum.
def foldl(accum, &f)
tail.foldl(
f.call(accum, head), &f)
end
def EMPTY.foldl(accum,&f)
accum
end tail.foldl() is called after
calling f.call()
65

Friday, April 12, 13

Foldl calls tail.foldl after calling f.call(accum, head). Note that it groups the accum with the first element, then works down the
list.
 Add foldr to List
f takes two args, each
item and accum, and
returns a new accum.
def foldr(accum, &f)
f.call(head,
tail.foldr(accum, &f))
end
def EMPTY.foldr(accum,&f)
accum
end tail.foldr() is called
before calling f.call(head,)
66

Friday, April 12, 13

Foldr calls tail.foldr before calling f.call(head,accum). Note that it groups the accum with the last element (because head isnt
handled until the whole recursion finishes!), so it works down to the end of the list first, then builds the accumulator on the way
back up.
Note that the arguments to f are reversed compared to foldl. Well see why this is useful in a moment.
 Example of foldl
ll = list.foldl(0) {|s,x| s+x}
lls= list.foldl("0") {|s,x|
"(#{s}"+#{x})"
}
puts "ll: #{ll}"
puts "lls: #{lls}"
# => ll: 10
# => lls: (((0+1)+2)+3)+4)

67

Friday, April 12, 13

Sum the list using foldl and also build a string that shows us who it proceeded!
 Example of foldr
lr = list.foldr(0) {|x,s| x+s}
lrs= list.foldr("0") {|x,s|
"(#{x}"+#{s})"
}
puts "lr: #{lr}"
puts "lrs: #{lrs}"
# => lr: 10
# => lrs: 1+(2+(3+(4+0))))

68

Friday, April 12, 13

Sum the list using foldr and also build a string that shows us who it proceeded! Note that the block has the x and s args reversed
compared to foldl! This is conventional so the accumulator shows up in the last position, as shown in the string.
 Compare foldl, foldr
foldl: (((0+1)+2)+3)+4) == 10
foldr: 1+(2+(3+(4+0)))) == 10

The sums are the same,

but the strings are not!
Addition is commutative and associative.

69

Friday, April 12, 13

Compare the left recursion with the right recursion. Note that reversing the block args for foldr resulted in this clearly formatted
string showing the right recursion. This is why people like to use that convention.
The additions were the same because + is commutative, but the string formation isnt, as the two strings are different!
 Try subtraction
foldl: (((0-1)-2)-3)-4) == -10
foldr: 1-(2-(3-(4-0)))) == -2

Substitute - for +.
Subtraction is neither
commutative nor associative.

70

Friday, April 12, 13

If you substitute - for +, youll also get different results.
 foldl and foldr
yield different results
for non-commutative
and non-associative
operations.
71

Friday, April 12, 13

There are other differences that we wont have time to discuss, but you might explore. foldl is tail-recursive, so it can be
optimized into a loop (if your language or VM supports that - many dont). In contrast, foldr isnt tail-recursive, but it can be
used to fold over infinite data structures when only a finite subset of it is used.
 Tools of
modularity

Better data
structures
72

Friday, April 12, 13

Lets look at one of the functional data structures, List, which weve already looked at a bit, but we need to explore further.
 filter, map and fold
as modules...

73

Friday, April 12, 13

So, we looked at these. Whats the big deal?? They are excellent examples of why functional programming is the right approach
for building truly modular systems...
 A Good Module:
interface Single responsibility, clear
abstraction, hides internals

composable Easily combines with other

modules to build up
behavior

reusable Can be reused in many

contexts

74

Friday, April 12, 13

Here are some of the qualities you expect of a good module. It exposes an interface that focuses on one task. The use of the
abstraction is clear, with well defined states and transitions, and its easy to understand how to use it. The implementation is
encapsulated.
You can compose this module with others to create more complex behaviors.
The composition implies reusability! Recall that its hard to reuse anything with side effects. Mutable state is also problematic if
the module is shared.
 Group email addresses
Exercise: implement
List.make
addrs = List.make(
"[email protected]",
"[email protected]",
"[email protected]",
"[email protected]",
"[email protected]")
Lets convert to lower case, filter out
spammers, and group the users by address
75

Friday, April 12, 13

Lets group a huge list of email address.
 Group email addresses
grouped = addrs.map {|x|
x.downcase
}.filter {|x|
x !~ /spammer.com$/
}.foldl({}) {|grps,x|
name, addr = x.split('@')
l = grps[addr] || List::EMPTY
grps[addr] = List.new(name,l)
grps
}
76

Friday, April 12, 13

We first map each string to lower case, then remove the strings that end with spammer.com, using a regular expression, and
finally fold over the remaining items. The fold takes an empty hash map {} as the initial value. We split each string on @, then
initialize the list of names for that address, if not already initialized. Now we create a new list, adding the name, and reassign to
the hash map. Finally, the block has to return the hash map for the next pass (or the end of the foldl).
Note: there is mutation of the hash map going on, but it is local to this thread!
 Group email addresses

grouped.each {|key,value|
puts "#{key}: #{value}"
}
=> yahoo.com: (pete,(bob,()))
=> gmail.com: (bill,(dean,()))

We calculated this grouping

in 10 lines of code!!
77

Friday, April 12, 13

For nice output, iterate over the hash map with each and print each key-value pair on its own line.
 If we had
GroupedEmailAddresses
objects,
how much more code
would be required?
78

Friday, April 12, 13

10 lines of code, reusing filter, map, and fold vs. how much custom, one-off code?
 How much more
development time
would be required?

79

Friday, April 12, 13

How much time would you spend implementing the custom solution?
 filter, map, and fold
are ideal modules.
Each has a clear abstraction,
composes with others,
and is reusable.
80

Friday, April 12, 13

What makes them so modularity is their stability, clear abstraction, near infinite composability to build higher-order
abstractions, which implies reusability!
 filter, map, and fold
are combinators.

81

Friday, April 12, 13

The term combinator is a technical term in FP. For our purposes, these functions take other functions as arguments, which is
how they are adapted to different purposes, and they combine with each other to build up more sophisticated calculators.
 Aside:
Did we just break
the Law of Demeter?
addrs.map{}
.filter{}
.foldl(){}
82

Friday, April 12, 13

LoD says its bad to chain method calls together, because each link introduces new object dependencies into the code and
every time the signature for one of these methods changes, it breaks this code. Its a small that indicates the calculation should
be moved to a more appropriate place.
Thats not an issue here. First, we keep returning a List (except at the end), so we arent adding dependencies. Second, map,
filter and fold are so stable, they are unlikely to ever change.
 Persistent data
structures

Better data
structures
83

Friday, April 12, 13

Lets look at one of the functional data structures, List, which weve already looked at a bit, but we need to explore further.
 Isnt copying
immutable values
inefficient.

84

Friday, April 12, 13

We said earlier that values should be immutable, but what if they are huge? Isnt it too expensive to make copies?
 Structure Sharing
class List Recall...
def prepend(head2)
List.new(head2, self)
end

end
Note: were sharing the
original list with the new list:
Structure Sharing

85

Friday, April 12, 13

If we prepend an element to an existing list, we create a new list, but share the old list as the tail. Recall that this would not be
safe if lists or their elements are mutable.
 Structure Sharing lets us
copy values efficiently.
But it only works if the
objects are immutable!!
86

Friday, April 12, 13

 What about Maps,
Sets,Vectors, ... ?

Separate the abstraction

from the implementation...
87

Friday, April 12, 13

List easily supports structural sharing, what about other data structures? If we separate the external interface from the internal
implementation, we can implement these types with data structures that provide efficient copies, also using structure sharing.
 Trees enable
structure sharing
and provide
O(log(n))
access patterns.
88

Friday, April 12, 13

The choice of tree data structure is important, but we wont get into that.
 For simplicity, well
just use unbalanced
binary trees:
average O(ln(n)).
89

Friday, April 12, 13

Binary trees average order log base 2(n) performance. Unbalanced is generally bad, but ignoring balancing lets us focus on the
key concept, structure sharing. Real implementations might use 32-way trees, giving log base 32(n) performance, and use one
or another balanced tree types. The choice of implementation is made to optimize search, insertion, cache locality in the CPU, or
other performance goals.
 Time 0
If value1 is a Map, each
value1
node might contain a
key-value pair.

a1 b1

a2 a3 b2 b3

a4 b4

90

Friday, April 12, 13

Consider value1, which implements a Map, where the
 Time 1

At time 1, value1
value1 value2 still exists. The new
value2 reuses a3,
a4, and the b1 tree

a1 b1

a2 a3 b2 b3

a4 b4

91

Friday, April 12, 13

Consider at some later time, Time1, a new value2 is created that mutates value1, but in fact, it just introduces a new root node
and shares much of value1, which still exists!
 Time 1 Persistent because
previous values
value1 value2 still exist.

a1 b1

a2 a3 b2 b3

a4 b4

92

Friday, April 12, 13

So, a history of versions is maintained, as long as there are references to the old versions. Its not persistent in the database
sense (although you could store these to disk).
 Better Concurrency

End of Cape Lookout, Oregon

93

Friday, April 12, 13

 Better Concurrency
Actors

94

Friday, April 12, 13

 The Actor Model of
Concurrency
is not specifically
functional, but it
follows the theme of
principled mutation. 95

Friday, April 12, 13

This is not a model that came out of the functional research community, but it fits the principle of finding principled ways to
handle and control mutation.
 Actors coordinate
work by sending
1 messages.

Actor1 Actor2

2 3

Alan Kay thought

Actor3 of objects as
message-passing
entities.
96

Friday, April 12, 13

A schematic view. Each actor sends a message, which resides in the receivers mailbox to be processed one at a time. When
finished, the receiver can send a reply, send a message to a different actor, or do nothing.
Alan Kay, the inventor of Smalltalk, had this model in mind (although not in name) as his vision for objects; message passing
entities that coordinate state mutation this way!
Erlang recently made the actor model famous (It was invented in the 70s by Hewitt and others).
 Actor1 might Actor2 might
be handling update the
UI events. 1 database.

Actor1 Actor2

2 3

Actor3 Actor3 might

update the in-
memory objects.
97

Friday, April 12, 13

Each actor has specific responsibilities and owns specific mutable values. This is one model of using them, but be careful
about bottlenecks!
 Actor Libraries
Java Akka, FunctionalJava, Kilim

Revactor, Omnibus, Akka

Ruby through JRuby!

Your language probably has

... an Actor library, too.

98

Friday, April 12, 13

Google for the actor libraries for your language.
 Akka Example
import akka.actor.*;
import static akka.actor.Actors.*;
import java.util.*;

public class MemoryActor

extends UntypedActor {
final Map<String,Date> seen =
new HashMap<String,Date>();

public void onReceive() {}

} 99

Friday, April 12, 13

Lets see a Java example using Akkas Actors. Note that you could do this with JRuby, too!
We declare an actor that will remember the messages (treated as strings for simplicity) that it receives, along with the times
they were received. Well store this information in a HashMap. The parent class is named UntypedActor because well treat all
messages as Objects.
 Akka Example
public void onReceive(
Object message){
String s = message.toString();
String reply = "OK" ;
if (s == "DUMP") {
reply = seen.toString());
} else {
seen.put(s, new Date());
}
getContext().replySafe(reply);
} 100

Friday, April 12, 13

We have to define the onReceive message that is declared abstract in UntypedActor. For simplicity, well just convert the
message to a string. If it equals DUMP, thats our signal to return a dump of the current state of the hash map, as a string.
Otherwise, we add the message string to the hashmap as the key with the current time as the value. Then we send a reply to the
caller, either the dump of the hash map or OK.
 Akka Example
public ActorExample {
public static void main( args) {
ActorRef ar = actorOf(
MemoryActor.class).start();
for (String s: args) {
Object r = ar.sendRequestReply(s);
System.out.println(s+": "+r));
}
Object r=ar.sendRequestReply("DUMP");
System.out.println("DUMP: "+r));
}}
101

Friday, April 12, 13

Finally, a main class to run it. It calls Akkas actorOf method to create an instance of MemoryActor and return a
reference (a.k.a. handle) to it. This handle-body pattern is used so Akka can restart an actor if necessary, then update the
reference to point to the new actor so the reference doesnt become stale!
We loop through the input arguments and send each one to MemoryActor, await the reply, then print it out.
 Akka Example
$ java -cp ActorExample \
I am a Master Thespian!
I: OK
am: OK
a: OK
Master: OK
Thespian!: OK
DUMP: {
am=Wed Jul 25 20:14:44 CDT 2011,
a=Wed Jul 25 20:14:44 CDT 2011,
Master=Wed Jul 25 20:14:44 CDT 2011,
Thespian!=Wed Jul 25 20:14:44 CDT 2011,
I=Wed Jul 25 20:14:44 CDT 2011} 102

Friday, April 12, 13

Compile and run it with the arguments I am a Master Thespian. You get five lines with <string>: OK and a final line (which Ive
wrapped for better legibility, DUMP: <hash_map.toString>. Note that the hash map toString doesnt preserve insertion order,
which is the general case for hash maps.
 For simplicity, we used
synchronous messages.
Asynchronous messages
scale better.
103

Friday, April 12, 13

To simplify the example, I just used synchronous messages, but in a real app, you would use async messages, because they
scale better.
 Better Concurrency
Software
Transactional
Memory

104

Friday, April 12, 13

 ACID Transactions

Atomicity
Consistency
Isolation
Durability

105
Friday, April 12, 13
Youre familiar with ACID transactions, a central feature of relational databases.
 ACID transactions
ensure data integrity.

106

Friday, April 12, 13

They have many benefits...
 Manage memory
with Transactions?
Atomicity
Consistency
Isolation
Durability

107
Friday, April 12, 13
Can we get the same semantics for updating values in memory?? Note that memory isnt durable.
 Software Transactional
Memory (STM)
Atomicity
Consistency
Isolation
Durability

108
Friday, April 12, 13
Software: its managed in software (there were some experimental efforts to do this in hardware in the 90s).
Transactional semantics.
Memory: were mutating values in memory.
 Time 0
Persistent Data
value1
Structures
are just what we need.

a1 b1

a2 a3 b2 b3

a4 b4

109

Friday, April 12, 13

Consider value1, which implements a Map, where the
 Time 0

At time 0, two
ref1 ref2
references, ref1 and
ref2 both refer to the
same value.
value1

a1 b1

a2 a3 b2 b3

a4 b4
110

Friday, April 12, 13

 Time 1

ref1 ref2
At time 1, ref2 has
been moved to the
new value.
value1 value2

a1 b1

a2 a3 b2 b3

a4 b4
111

Friday, April 12, 13

A transaction is used to move the reference. Some APIs resemble the use of the synchronized keyword in Java. The transaction
may include both the construction of value2 and the reassignment of ref2 to value2. However, since values are immutable, its
possible in this case to construct value2 first, then use a transaction to move ref2 to it.
 Time 1

ref1 ref2
In Clojure simple
assignment to mutate
a value isnt allowed.
STM is one of several
value1 value2
mechanisms you
must use.

a1 b1

a2 a3 b2 b3

a4 b4
112

Friday, April 12, 13

Clojure is making STM famous.
The Haystack, Oregon

Better Objects 113

Friday, April 12, 13

Immutable
Values

Better Objects 114

Friday, April 12, 13

 Immutable values
are better for
concurrency and they
minimize obscure
bugs because of
side effects. 115

Friday, April 12, 13

If you must do multithreaded programming, its far easier if your values are immutable, because there is nothing that requires
synchronized access. Also, obscure bugs from non-local side effects are avoided.
 Immutability tools
final or constant variables.
No field setter methods.
Methods have no side effects.
Methods return new objects.
Persistent data structures.
116
Friday, April 12, 13
These techniques help you achieve immutability in any language. Persistent data structures let you make copies of big data
structures efficiently.
 TDD

Better Objects 117

Friday, April 12, 13

 Test Driven Development
(including refactoring)
is still useful in FP,
but there are changes.
118

Friday, April 12, 13

If you must do multithreaded programming, its far easier if your values are immutable, because there is nothing that requires
synchronized access. Also, obscure bugs from non-local side effects are avoided.
 First, you tend to use
more experimentation
in your REPL
and less test first.
119

Friday, April 12, 13

Its somewhat like working out a math problem. You experiment in your Read Eval Print Loop (interactive interpreter), working
out how an algorithm should go. They you commit it to code and write tests afterwards to cover all cases and provide the
automated regression suite. The test-driven design process seems to fit less well, but other people may disagree!
 Testing Money
class Money
PRECISION = 0.00001
attr_reader value
def initialize value
@value = round(value)
end

def round value

# return rounded to ? digits
end

end 120

Friday, April 12, 13

Money is a good domain class to implement as a functional type, because it has well-defined semantics and supports several
algebraic operations!
The round method rounds the value to the desired PRECISION. I picked 5 decimal places, even though we normally only show at
most a tenth of a penny...
 Testing Money

def add value
v = value.instance_of?(Money) ?
value.value : value
Money.new(value + v)
end

end

121

Friday, April 12, 13

The add method tests the value to see if its another Money or a (assumption) a float. It returns a new Money (of course!)
 Imaginary RSpec
describe "Money addition" do
money_gen = Generator.new do
Money(-100.0) to Money(100.0)
end

Define a generator that
generates a random sample
of instances between the
ranges shown.

122

Friday, April 12, 13

RSpec is a popular Ruby testing framework in the style of Behavior Driven Development (BDD). I am showing fictitious extensions
to illustrate a particular functional approach - testing properties that should hold for all instances. So its less about testing by
example and (as much as is possible) testing universal properties.

We start by defining a function that can generate N random sample instances within an arbitrary range.
 Imaginary RSpec
describe "Money addition" do
money_gen = Generator.new do
Money(-100.0) to Money(100.0)
end
property "is commutative" do
money_gen.make_pairs do |m1,m2|
m1.add(m2).should_be_close(
m2.add(m1), Money::PRECISION)
end
end verify that addition is
end commutative!
123

Friday, April 12, 13

In our fictitious RSpec extensions, we verify the property that addition is commutative. We ask the money_gen to create some
random set of pairs, passed to the block, and we verify that m1+m2 == m2+m1 within the allowed precision.
 Test Driven Development
becomes
property verification.

124

Friday, April 12, 13

Of course, youll still write a lot of conventional OO-style tests, too.
 Recall
grouped = addrs.map {|x|
x.downcase
}.filter {|x|
x !~ /spammer.com$/
}.foldl({}) {|grps,x|
name, addr = x.split('@')
l = grps[addr] || List::EMPTY
grps[addr] = List.new(name,l)
grps
} How might you
refactor this code?
125

Friday, April 12, 13

Remember how we grouped email addresses? What is refactoring like in such a world?
 Recall
grouped = addrs.map {|x|
x.downcase
}.filter {|x| Extract Function?
x !~ /spammer.com$/
}.foldl({}) {|grps,x|
name, addr = x.split('@')
l = grps[addr] || List::EMPTY
grps[addr] = List.new(name,l)
grps
}
126

Friday, April 12, 13

We could extract some of these blocks into Ruby procs that we pass in to the methods. This would make the code less dense
and provide opportunities for generalization (e.g., pluggable spam address filters).
We can also do traditional refactoring of some of the lines in the foldl block. However, lets avoid premature refactoring! If the
extracted function is never used anywhere else, dont extract it, unless clarity is a problem.
 Recall
class List Replace Conditional
with Structure
def to_s
"(#{head},#{tail})"
end

def EMPTY.to_s; "()"; end

end
List.to_s is recursive, but
EMPTY.to_s will terminate the
recursion with no conditional test! 127

Friday, April 12, 13

We avoided conditionals in many list methods by using subclass polymorphism. That is we used good ol OO-style refactoring.
 Design
Patterns

Better Objects 128

Friday, April 12, 13

 Does FP make
Design Patterns
obsolete?

129

Friday, April 12, 13

Some people have claimed that FP makes design patterns obsolete. This confuses the idea of patterns with specific examples.
There are some OO patterns that simply go away or are built into functional languages. Other OO patterns are still useful and FP
has its own collection of patterns, although the FP community has not traditionally used that terminology.
 Some OO patterns
go away:
Visitor
Good riddance! 130

Friday, April 12, 13

Visitor is confusing, ugly, and invasive
 Others are built into
the FP languages:
Iterator, Composite,
Command, ...
131

Friday, April 12, 13

Some other patterns are already in the language. Does that mean they *arent* actually patterns?? Or, does it mean that we
shouldnt think of patterns as something that *has* to be external to the language itself?
 Others are new to FP:
Fold, Monoid, Monad,
Applicative, Functor...
We saw fold. The others come
from Category Theory... 132

Friday, April 12, 13

Fold we saw. Im just going to mention these Category Theory patterns, but not define them. Theyre part of the intermediate
material...
 Visitor is replaced by
Pattern Matching and
less reliance on joining
functions + state
into objects.
133

Friday, April 12, 13

Visitor is confusing, ugly, and invasive. Its designed to allow visitors to see object internals without simply exposing internals
with getters. Its a way of adding (or simulating adding) new methods to existing classes for closes-type languages like Java.

The word pattern in pattern matching is not meant in the design pattern sense.
 Pattern Matching is one
of the most pervasive
tools in functional
programming.
134

Friday, April 12, 13

 Haskell/Erlang Like...
String toString(emptyList()){
return "()";
}
String toString(list(head,tail)){
return "("+head+","+tail+")";
}

List<X> list = new List<X>();
toString(list);
135

Friday, April 12, 13

Ive used Java syntax here, but this is the sort of code you see in Haskell and Erlang all the time, for example. A ListToString
*module* would have multiple functions with the same name but different argument lists. The runtime picks the function by
*matching* the argument to the first fit. AND it automatically extracts the head and tail for nonempty lists.
How does this work? Depending on the language, there would be a mechanism to *deconstruct* (or *destructure*) objects. Note
that Im showing our factory methods used in this way. So, there would need to be a symmetry defined in the language for this
purpose. Scala uses a mechanism like this.
 Haskell/Erlang Like...
def to_s(List::EMPTY)
"()"
end
def to_s(List(head,tail))
"("+head+","+tail+")";
end

list = List.new()
to_s(list)
136

Friday, April 12, 13

Ive used illegal Ruby syntax here, but this is the sort of code you see in Haskell and Erlang all the time, for example. A
ListToString *module* would have multiple functions with the same name but different argument lists. The runtime picks the
function by *matching* the argument to the first fit. AND it automatically extracts the head and tail for nonempty lists.
How does this work? Depending on the language, there would be a mechanism to *deconstruct* (or *destructure*) objects. Note
that Im showing something that looks like a constructor call in the second example. So, there would need to be a symmetry
defined in the language for this purpose. Scala uses a mechanism like this.
 Wait!
Why am I defining to_s
outside the classes??

137

Friday, April 12, 13

Why IS to_s (or toString in other languages) in all objects? Yea, its nice for debugging, but when is the format what you want?
What if you want XML today and JSON tomorrow?
 package A package B
ParentA1 ParentB1
toJSON toJSON

ChildA1 ChildA2 ChildB1 ChildB2

toJSON toJSON toJSON toJSON

or
package ToJSON

toJSON(ParentA1)
toJSON(ParentB1)
toJSON(ChildB1)
toJSON(ChildB2)
toJSON(...)

138

Friday, April 12, 13

You really dont want to just bloat your classes with these things AND you want the *implementation* of toJSON for all types to
be defined as modularly as possible. *I argue that putting stuff like this in class hierarchies all over you app scatters the logic
and breaks modularity!
But doesnt package ToJSON break other rules? Like what if we add a new child to a hierarchy? We have to balance these
competing design forces. For List, which is an Algebraic Data Type, this alternative works extremely well. For arbitrary
hierarchies, its more challenging.
 OO
Middleware

Better Objects 139

Friday, April 12, 13

 In a highly-concurrent
world, do we really
want a middle?

140

Friday, April 12, 13

 Which Scales Better?
Web Client 1 Web Client 2 Web Client 3 Web Client 1 Web Client 2 Web Client 3

ParentB1
toJSON

ChildB1
toJSON
ChildB2
toJSON
or Process 1 Process 2 Process 3

Service 1 Service 2 Database Service 1 Service 2 Database

141

Friday, April 12, 13

If we funnel everything through a faithfully-reproduced domain object model, our services will be bigger, harder to decompose
into smaller pieces, and less scalable. *Modeling* our domain to understand it is one thing, but implementing it in code needs to
be rethought. The compelling power of combinators and functional data structures are about as efficient and composable as
possible. Its easier to compose focused, stateless services that way and scale horizontally.
 What about ORM?
Other Logic Query Other Logic Query
Objects

4 1 1

Object Model SQL SQL

ParentB1
toJSON Object-

or
Relational
Mapping
3

ChildB1 ChildB2
toJSON toJSON
Result Set Result Set

2 2

Database Database

Question Object-Relational Mapping 142

Friday, April 12, 13

What if your business logic just worked with the collections returned from your database driver? Its true that some of these
collections, like Javas ResultSet, dont have the powerful combinators weve been discussing, but those methods could be
added as static service methods in a helper class.
The question to ask is this: does the development and runtime overhead of converting to and from objects justify the benefits?
 Object middleware,
including ORM, isnt
bad. It just has costs
like everything else...
143

Friday, April 12, 13

Just remember that every design decision has costs, so evaluate those costs with a clear head...
 Nehalem State Park, Oregon

Recap

Friday, April 12, 13

(Nehalem State Park, Oregon)
 Concurrency

San Francisco Bay

Friday, April 12, 13
Concurrency is the reason people started discussing FP, which had been primarily an academic area of interest. FP has useful
principles that make concurrency more robust and easier to write.

(San Francisco Bay)

 Were
Drowning
in Data.

...

Friday, April 12, 13

Not just these big companies, but many organizations have lots of data they want to analyze and exploit.

(San Francisco)
 Mud, Death Hallow Trail, Utah

We need better
modularity.

Friday, April 12, 13

I will argue that objects havent been the modularity success story we expected 20 years ago, especially in terms of reuse.

(Mud near Death Hollow in Utah.)

 Half Dome, Yosemite NP

We need
better
agility.

Friday, April 12, 13

Schedules keep getting shorter. The Internet weeded out a lot of process waste, lot Big Documents Up Front, UML design, etc.
From that emerged XP and other forms of Agile. But schedules and turnaround times continue to get shorter.

(Ascending the steel cable ladder up the back side of Half Dome, Yosemite National Park)
 Maligne Lake, Jasper Nat. Park

We need a return
to simplicity.
Friday, April 12, 13
Every now and then, we need to stop, look at what were doing, and remove the cruft weve accumulated. I claim that a lot of the
code we write, specifically lots of object middleware, is cruft.

(Maligne Lake, Near Jasper National Park, Jasper, Alberta)

 Going from here:
If you like statically-typed
languages, check out:
Scala
Haskell
F#
OCaml
150
Friday, April 12, 13
Learn a real functional language to see how these ideas work in a language that supports them natively, as well as concepts we
didnt cover. Here is a list of the most popular statically-typed functional languages.
 Going from here:
If you like dynamically-typed
languages, check out:
Clojure
Erlang
Other Lisp dialects

151
Friday, April 12, 13
Here is a list of the most popular dynamically-typed functional languages.
 Going from here:
Channel 9 videos
Blogs, books, ...

152
Friday, April 12, 13
There are excellent MSDN Channel 9 videos on functional programming.
Numerous blogs, books, etc...
 Thank You! Functional
Programming
[email protected] for Java Developers

@deanwampler
Dean Wampler

polyglotprogramming.com

Friday, April 12, 13