CS 320: Concepts of Programming Languages

Wayne Snyder
Computer Science Department
Boston University

Lecture 15: Lambda Calculus II and Evaluation Order

o Properties of Beta-Reduction: Non-termination and confluence
o Evaluation Strategies and their meaning in programming
languages
o Haskell Lazy Evaluation:
§ Simultaneous let definitions
§ Infinite Lists
Lambda Calculus: Properties of Beta-Reduction
Recall from last time:

Alpha-Conversion (change of bound variables):

!x. E → ' !x'. ( E [x := x'] )

where x' is a fresh variable (never seen before in this context).

Intuitively: change the bound variable x and every occurrence of an x corresponding to this
binding to a new variable (your choice, but make sure it doesn't conflict with any other
variables, either bound or free).

Examples:

!x. ( !y. x y ) → ' !x'. ( !y. x' y )

!x. ( !y. x (!x . x y) ) → ' !x'. ( !y. x' (!x . x y) )

Lambda Calculus: Properties of Beta-Reduction
Recall from last time:

Beta-Conversion (function application by parameter passing):

( "x. E ) F →* E[x := F]

where the term ( "x. E ) has undergone alpha-conversion as necessary to prevent free variable
capture when making the substitution of F for x in E.

Examples:

( "x. ( "y. x ("x′ . x' ( y x ) ) ) ) z → * ( "y. z ("x′ . x' ( y z ) ) ) )

( "x. ( "y. x ("x . x ( y x ) ) ) ) z → * ( "y. z ("x . x (y x ) ) ) )

( "x. ( "y. x ("x . x ( y x ) ) ) ) y → 3 ( "x. ( "y′. x ("x . x ( y' x ) ) ) ) y → * ( "y'. y ("x . x (y' x ))))

( "x. ( "y. y ("y . y ( y x ) ) ) ) y → 3 ( "x. ( "y′. y' ("y . y ( y x ) ) ) ) y

→ 3 ( "x. ( "y′. y′ ("y'' . y'' ( y'' x ) ) ) ) y
→ * ( "y′. y′ ("y′′ . y′′ ( y′′ y ) ) ) )
Lambda Calculus: Properties of Beta-Reduction
Note that it does not matter in principle where the beta-redex is, and there could be more than
one:

Examples:

("z. ("x. z x) ) y → * ("x. y x ) -- redex at top of expression

z ( z ( ("x. z x) y ) → * z ( z ( ("x. z x) y ) -- redex deep inside expression

("z. ("x. z x) ("y. y) ) → * ("z. z ( "y. y) ) -- redex inside an abstraction

( "x. x y) (( "x. x) ( "x. z x)) → * ?? -- which one to reduce?

Lambda Calculus: Properties of Beta-Reduction
There may be 0, 1, or more than 1 beta-redex. A lambda-expression with no beta-redexes is
said to be in normal form. Such expressions maybe considered to be "values."

true =def ("x. "y. x) two =def ("f. "x. f (f x) )

Evaluating a pure lambda calculus expression means to beta-reduce it to a normal form, if

possible:

("x. x y) (("x. x) ("x. z x)) → + (("x. x) ("x. z x)) y → + ("x. z x) y → + z y

normal form
But this may not be possible! Beta-reductions may not terminate:

("x. x x) ("x. x x) → + ("x. x x) ("x. x x) → + ("x. x x) ("x. x x) → + … . .

("x. (x x) x) ("x. (x x) x) → + (("x. (x x) x) "x. (x x) x) ("x. (x x) x) → + … . .

Lambda Calculus: Properties of Beta-Reduction
When there is more than one redex, there are two important issues:

(1) Which one to reduce first? In general, what is our overall strategy for choosing redexes?

(2) Does it matter which strategy that we use? What are the consequences of choosing a strategy?

Issue (1) : There are two basic reduction strategies.

(A) Normal or Leftmost Order: "The leftmost, outermost redex is always reduced first. That is,
..... the arguments are substituted into the body of an abstraction before the arguments are
reduced." (Wikipedia)

("x. x y) (("x. x) ("x. z x)) → * (("x. x) ("x. z x)) y

Lambda Calculus: Properties of Beta-Reduction

Reduction Strategies

(A) Normal or Leftmost Order: "The leftmost, outermost redex is always reduced first. That is,
..... the arguments are substituted into the body of an abstraction before the arguments are
reduced." (Wikipedia)

("x. x y) (("x. x) ("x. z x)) → * (("x. x) ("x. z x)) y

(B) Applicative or Strict Order: "The rightmost, innermost redex is always reduced first.
Intuitively this means a function's arguments are always reduced before the function itself.
Applicative order always attempts to apply functions to normal forms, even when this is not
possible." (Wikipedia)

("x. x y) (("x. x) ("x. z x)) → * ("x. x y) ("x. z x)

 Lambda Calculus: Properties of Beta-Reduction

Issue (2): What are the consequences of choosing one strategy over the other?

Several important consequences:

(i) If there is any reduction sequence which terminates in a normal form, Normal Order will
find one (which is why it is called "normal" order, since it finds normal forms).

(ii) Applicative Order may not terminate, even when there does exist some terminating
sequence. Example:

("x. y) (("x. x x) ("x. x x)) → ) y -- normal order

("x. y) (("x. x x) ("x. x x)) → ) ("x. y) (("x. x x) ("x. x x)) → ) ..... -- applicative order

We explored this in hw 01! Preorder = normal order Postorder = applicative

Lambda Calculus: Properties of Beta-Reduction
Issue (2): What are the consequences of choosing one strategy over the other?

(i) If there is any reduction sequence which terminates in a normal form, Normal Order will
find one (which is why it is called "normal" order, since it finds normal forms).

(ii) Applicative Order may not terminate, even when there does exist some terminating
sequence. ! ← ... ! ←

(iii) Beta-reduction is confluent, so when normal forms exist, they are unique:

Confluence: If E reduces to two different expressions F ≠ 5 ,

then F and G both reduce to a common term H: E
E
implies

F G

F G

H
Lambda Calculus: Properties of Beta-Reduction

Issue (2): What are the consequences of choosing one strategy over the other?

(i) If there is any reduction sequence which terminates in a normal form, Normal Order will
find one (which is why it is called "normal" order, since it finds normal forms).

(ii) Applicative Order may not terminate, even when there does exist some terminating
sequence. ! ← ... ! ←

(iii) Beta-reduction is confluent, so when normal forms exist, they are unique:

Punchline: Normal Order will find a unique normal form when one exists; Applicative Order
may not terminate, even when a normal form exists, but if it does, then that normal form is
unique.

If F and G are normal

forms, then F = H = G.
 Evaluation Order in Programming Languages
So this means that except for the problem of non-termination, you will always get
the same answer, no matter what strategy you use (even when we add arithmetic
and other computational processes):

(\x -> x * 3) ((\y -> y + 2) 5 )

normal order applicative order

((\y -> y + 2) 5 ) * 3 (\x -> x * 3) (5 + 2)

(5 + 2) * 3 (\x -> x * 3) 7

7 * 3

21
Evaluation Order in Programming Languages
Most languages use applicative/strict evaluation for function calls, so for
example in Python we would have the following sequence of events:

def times3(x): def plus2(y):

return x * 3 return y + 2

times3( plus2 ( 5 ) ) (\x -> x * 3) ((\y -> y + 2) 5 )

evaluate 5
pass parameter to plus2: (\x -> x * 3) (5 + 2)
y = 5
evaluate 5 + 2
return value 7 (\x -> x * 3) 7

pass parameter to times3: 7 * 3

x = 7
evaluate 7 * 3
return 21 21
Evaluation Order in Programming Languages
However, most languages also use some non-strict evaluation strategies, especially
in two cases: Boolean operators and conditionals (if-then-else).

"Short-Circuit" Evaluation of Boolean Expressions:

Evaluation Order in Programming Languages
However, most languages also use some non-strict evaluation strategies, especially in two
cases: Boolean operators and conditionals (if-then-else).

"Lazy" Evaluation of If-Then-Else:

if A then B else C --evaluate A, then evaluate

--only one of B or C

def ohNo(x):
return ohNo(x) def cond(A,B,C):
if A:
def test(x): return B
if x > 0: else:
return "Positive!" return C
else:
return ohNo(x)

test(10) => "Positive!" cond( 10>0,10,ohNo(10)) => .....

test(-1) => ...... cond( -1>0,-1,ohNo(-1) )=> ..

Evaluation Order in Programming Languages
Haskell uses a version of Normal Order, called Lazy Evaluation, in which
evaluation is ONLY done when absolutely necessary.

times3 x = x * 3 plus2 y = y + 2

(times3 (plus2 5)) = (\x -> x * 3) ((\y -> y + 2) 5 )

= (((\y -> y + 2) 5 ) * 3)

= ((5 + 2) * 3)

= (7 * 3)

= 21

This is normal order evaluation.

Evaluation Order in Programming Languages
But there is a serious efficiency problem with normal order: expressions may be
duplicated and have to be evaluated multiple times:

square3 x = x * x * 3 plus2 y = y + 2

(square3 (plus2 5))

= (\x -> x * x * 3) ((\y -> y + 2) 5 )

= (((\y -> y + 2) 5 ) * ((\y -> y + 2) 5 ) * 3)

= ((5 + 2) * ((\y -> y + 2) 5 ) * 3)

= (7 * ((\y -> y + 2) 5 ) * 3)

= (7 * (5 + 2) * 3)

= (7 * 7 * 3)

= (49 * 3) = 147
Evaluation Order in Programming Languages
Lazy evaluation fixes this by creating a temporary variable bound to the
expression (called a "thunk") which is then only evaluated once:

square3 x = x * x * 3 plus2 y = y + 2

(square3 (plus2 5))

= (\x -> x * x * 3) ((\y -> y + 2) 5 )

= (thunk * thunk * 3)
where thunk = ((\y -> y + 2) 5 )

= (thunk * thunk * 3)
where thunk = (5 + 2)
This is the
= (thunk * thunk * 3)
programming
where thunk = 7
language version of
"memoizing."
= (7 * thunk * 3) where thunk = 7

= (7 * 7 * 3) = (49 * 3) = 147
Recall: Let and Where Expressions in Haskell
let and where expressions allow you to create local variables and avoid having to
write lots of helper functions.

The parameters in a lambda expression are local variables which only have meaning
inside the body of the lambda expression:

(\x -> x + 2*x – 1)

Scope of x

This is a familiar concept in programming languages:

def area(r): Scope of r

pi = 3.1415
Scope of pi
return pi * r * r

In Haskell this is done using the let expression:

let <bindings> in <expression>

Scope of bindings
Let and Simultaneous Equations
Lazy evaluation in Haskell means that no expression is evaluated until it absolutely
has to be. So in a let, nothing is evaluated until the variable has to be used; the
net result is that equations in a let are "simultaneous" and order does not matter:

cylinder r h =
let pi = 3.1415
sideArea = 2 * pi * r * h
topArea = pi * r^2
in sideArea + 2 * topArea
All these do exactly
cylinder r h = the same thing!
let sideArea = 2 * pi * r * h
pi = 3.1415
topArea = pi * r^2
in sideArea + 2 * topArea This is another example of how
Haskell follows mathematical
cylinder r h = practice, not imperative
let sideArea = 2 * pi * r * h programming.
topArea = pi * r^2
pi = 3.1415
in sideArea + 2 * topArea
Let and where in detail: how are bindings evaluated?
When we think of bindings as simultaneous equations, we see how Haskell
interprets equations in let and where:

x = 2 * z equivalent to x = 10
y = 4 y = 4
z = y + 1 z = 5

Main> :r
[1 of 1] Compiling Main
test = let x = 2 * z ( Main.hs, interpreted )
y = 4 Ok, one module loaded.
z = y + 1 Main> test
in (x,y,z) (10,4,5)
Main> test2
test2 = (x,y,z) where (10,4,5)
x = 2 * z
y = 4
z = y + 1
Let and where in detail: how are bindings evaluated?
The same thing is true of equations in your code:

Main> :r
[1 of 1] Compiling Main
Main.hs, interpreted )
Ok, one module loaded.
Main> x
10
Main> y
4
Main> z
5
Let and where in detail: how are bindings evaluated?
This leads to the following behavior with sets of bindings that have no solution as a
set of simultaneous equations:

Gets into infinite digression trying

to figure out value of x! Had to hit
control-c to stop it.
Evaluation Order: Strict vs Lazy
Haskell uses lazy evaluation by default, although you can modify this to make
functions strict.

Main> x = x + 1
Main> "NOOOO, DONT DO IT!!!!!"
"NOOOO, DONT DO IT!!!!!"
Main> x

-- Infinite digression, hit Control-c

If strict evaluation were being used, then x + 1 would be evaluated first, and x is
unbound (since binding to x has not yet been made), as if

Main> x = x + 1

<interactive>:47:5: error: Variable not in scope: x

Main>
Evaluation Order: Strict vs Lazy
But Haskell uses lazy evaluation, so

Main> x = x + 1
Main> x

Look up x, substitute the binding:

(x + 1)

Hm... look up x, substitute the binding:

((x + 1) + 1)

Hm... look up x, substitute the binding:

(((x + 1) + 1) + 1)

etc. ad infinitum ....

Evaluation Order: Strict vs Lazy
This explains how simultaneous equations in let are evaluated, instead of storing
values in the state/environment, we store unevaluated expressions; we only evaluate
them when we have to:

test = let x = 2 * z
y = 4
z = y + 1
in x

Main> test Bindings: [(x,(2 * z)), (y,4), (z,(y+1))]

10
eval( test )
eval( x ) -- look up x and substitute
eval( 2 * z )
eval( z )
eval( y + 1 )
eval( y )
=> 4
=> 5
=> 10