Archon
Archon
Abstract
Existing meta-programming languages operate on encodings of pro-
grams as data. This paper presents a new meta-programming language,
based on an untyped lambda calculus, in which structurally reflective pro-
gramming is supported directly, without any encoding. The language fea-
tures call-by-value and call-by-name lambda abstractions, as well as novel
reflective features enabling the intensional manipulation of arbitrary pro-
gram terms. The language is scope safe, in the sense that variables can
neither be captured nor escape their scopes. The expressiveness of the
language is demonstrated by showing how to implement quotation and
evaluation operations, as proposed by Wand. The language’s utility for
meta-programming is further demonstrated through additional represen-
tative examples. A prototype implementation is described and evaluated.
1 Introduction
This paper presents a new meta-programming language called Archon, in
which programs can operate directly on other programs as data. Existing meta-
programming languages suffer from defects such as encoding programs at too
low a level of abstraction, or in a way that bloats the encoded program terms.
Other issues include the possibility for variables either to be captured or escape
their static scopes. Archon rectifies these defects by trading the complexity
of the reflective encoding for additional programming constructs. We adopt
∗ This work is supported by funding from the U.S. National Science Foundation under
1
the identity function as our encoding, and add new reflective constructs to tra-
verse (unencoded) program terms. These results are achieved in a scope-safe
way: variables may neither be captured by substitution nor escape their scopes.
This work is the crucial first part of the Archon project, which aims to unite
reflective programming and reflective theorem proving.
In order to situate the present work with respect to the existing literature,
we first survey work on several kinds of reflection in Computer Science (Sec-
tion 2). We then further motivate the work by briefly discussing the role of meta-
programming in the Archon project (Section 3). In Sections 4 through 6, we
consider in detail related work by Wand [50]. We will see how Wand’s language
achieves desirable properties which other reflective programming languages lack,
but at the same time fails to be suitable for general meta-programming. This
requires a brief review of the Mogensen-Scott encoding used in Wand’s language
(Section 5). The Archon language itself is presented starting in Section 7. An
outline for the sections of the paper presenting Archon may be found in Sec-
tion 3.3.
2 On Reflection
Abstractly speaking, reflection is concerned with enabling entities at one lin-
guistic level (called the object level) to interact with entities at a higher level
(the meta-level). This is typically done via some kind of encoding of the meta-
level entities in the object language. This abstract description is specialized by
stating what the languages are, what kind of entities are involved, and what
sort of interaction is allowed. A crucial point is how the meta-level entities are
encoded in the object language. The rest of this section categorizes related work
on reflection in Computer Science according to these criteria.
Note that in related work on logical frameworks, object languages (such as
logics) are seen as being embedded in meta-languages (logical frameworks) [33].
This determination of what is object-level and what is meta-level differs from
that of our abstract description of reflection. The difference in viewpoint arises
from a difference in emphasis. Logical frameworks are typically concerned with
representing other deductive systems. In reflection, the emphasis is on the
ability of an object-level language to encode entities from its own meta-level.
2
other forms of reflective programming.
Much early interest in reflective programming languages centered around
language extension via computational reflection [18, 41]. Control operators such
as Scheme’s call/cc are also computationally reflective, since they expose an
aspect of the program’s execution environment, namely its continuation, to the
program itself. Object-oriented programming languages like Smalltalk and
Java include computationally reflective features, for example the ability, in
Java, of a running program to replace its class loader [1, 13].
Object-oriented programming languages typically also provide certain struc-
turally reflective features, mediated by abstracted representations of classes.
For example, Java’s reflection mechanism makes essentially just the names and
types of object and class components available to running object-level programs.
Interaction is restricted mostly to inspection, although certain simple updating
operations like setting the values of object fields are allowed. Object-oriented
languages like Smalltalk allow much more liberal forms of interaction. Run-
time type information in C++ may be considered another example [43]. Similar
functionality is provided in functional languages by intensional polymorphism,
where code can dispatch at runtime on a term’s type [6].
2.1.1 Meta-Programming
Meta-programming languages are based on a shallow encoding of program texts.
In general, the kind of interaction allowed in meta-programming is to perform
arbitrary computation of output values, including program texts, from input
program texts. Note that this excludes meta-programs from modifying running
code. Self-modifying code can arguably be viewed as both structurally and
computationally reflective, since the running code can be viewed as forming
part of the program’s execution environment.
With meta-programming, object-level programs manipulate encoded pro-
gram texts as data. Languages in the LISP family, such as modern dialects
Common LISP, Emacs LISP, and Scheme, exemplify this kind of reflective
programming [42, 27, 21, 28]. In these languages, it is possible to view any
program as a piece of data, and any piece of data as a program (albeit pos-
sibly one which cannot execute without error, as in trying to call something,
such as a numeric literal, which is not a function). Quotation (quote) encodes
program expressions as data, specifically nested lists of symbols. As lists, pro-
gram expressions such as β-redexes, which would otherwise evaluate, are frozen.
They may be inspected and decomposed using LISP’s list constructs. For ex-
ample, one may obtain the applicand (lambda (x) x) from the quoted ap-
plication ’((lambda (x) x) 3) by applying car. Similarly, evaluation (eval)
transforms data into programs. Frozen terms are, so to speak, thawed, and
reduction of their redexes takes place again. Note that since LISP languages
typically include functions for obtaining the name of a symbol as a string (e.g.,
symbol->string in Scheme), detailed lexical information about variables is
available to programs in these languages. So the reflective encoding provided
by these languages is at a rather low level of abstraction.
3
Among the research problems related to meta-programming that have been
considered in the literature are static typing of meta-programs, and how meta-
programs deal with the scoping of variables. It is no accident that the LISP
languages, which best exemplify meta-programming, are not statically typed.
The soft type system of Wright and Cartwright for Scheme covers the features
of Scheme version R4RS, which does not include the meta-programming fea-
ture eval [52]. Static typing of general meta-programs appears difficult, and
the existing literature mostly considers static typing for a weaker form of meta-
programming called staged computation, used also for macro systems [22, 3, 32,
10, 44]. In staged computation, programs may generate programs as data, but
not inspect or decompose them. Some work on static typing of more general
meta-programs has been done, but a full account is currently lacking [31]. Ex-
tensible languages support the addition of new syntax to a host language, as well
as accompanying semantic analyses for more informative error reporting [11].
this sort of example could be done using quasiquotation. The example I gave there does not
work in R5RS Scheme, because lambda abstractions evaluate to procedures, from which one
cannot extract components, even when quoted. The current example using macros does work
in R5RS Scheme. Thanks to John Clements for pointing out my earlier error, and for Robby
Findler, Matthew Flatt, and Matthias Felleisen for an engaging email discussion about the
meaning of this example.
4
reflected programs are represented directly using variables in meta-programs (as
opposed to, for instance, de Bruijn indices or other numbering schemes) greatly
eases the burden of operating on reflected program texts, but poses challenges
for static typing [51, 38, 34]. A great variety of other representation techniques
has been considered, for example in the solutions to the POPLmark challenge [2].
The details of these are not relevant to the current work, other than to note:
while HOAS can introduce technical complications, particularly for typed lan-
guages, it is widely acknowledged to be the most concise and convenient variable
representation technique.
5
correct [17]. Non-trivially encoded terms are also used heavily in the Maude
term rewriting system [4].
6
on such structures directly. Safe extension of the prover’s trusted core then
becomes more feasible, since there is no non-trivial encoding, as implemented
by the Ltac code used in Coq, which falls outside the realm of the prover’s
reasoning system. Reasoning about reflective operations is also easier, since it
is not necessary to account for a non-trivial encoding.
(define-syntax complam
(syntax-rules () ((complam arg1 arg2) (eqv? (exv arg1) (exv arg2)))))
7
of his paper). This matter is discussed in detail in the next three sections. The
syntax and operational semantics of Archon are presented next (Section 7), as
well as examples demonstrating Archon’s utility for meta-programming (Sec-
tion 8). Section 8.4 shows how to implement the reflective primitives of Wand’s
language in Archon, thus further establishing the new language’s expressive-
ness. A prototype implementation of Archon is discussed and evaluated in
Section 9.
4 Wand’s System
Wand defines operations fexpr (for quotation) and eval which map program
terms to their HOAS-based Mogensen-Scott encodings (reviewed below) and
back [50]. He proves that α-equivalence of programs coincides with contextual
equivalence of their encodings. This achieves the goal mentioned above for a
higher-level encoding of terms. The motivation for Wand’s work is, in a sense, a
negative one: he wishes to point out the difficulties which reflective capabilities
raise for traditional approaches to applications like compilation. These rely on
coarser notions of contextual equivalence than just α-equivalence, for example to
do source-to-source optimizations. Such optimizations are intended, of course,
to result in terms which are not α-equivalent to the unoptimized terms. It is
not Wand’s aim to give an account of general meta-programming in his pro-
posed system, since for his purposes, it is enough to establish the triviality of
contextual equivalence (namely, that it coincides with α-equivalence). Indeed,
Wand’s system cannot be used for general meta-programming, at least not in
any obvious way. We explain why (in Section 6 below), after first reviewing his
system.
The syntax for Wand’s system is the following (cf. [50, pages 90–91]):
T ::= x | λx.T | T T | fexpr T | eval T
Here and below we use standard abbreviations from lambda calculus including
left associativity of application, abbreviated notation for consecutively nested
lambda abstractions, and lambda abstraction with scope extending as far as
syntactically possible to the right. To define the operational semantics of this
language, we first designate reduction contexts R and values V , as follows:
R ::= [] | (R M ) | ((λx.M ) R) | (fexpr R) | (eval R)
V ::= λx.M | (fexpr V )
We use a standard notion of reduction contexts R. Recall that these contain
exactly one occurrence of the hole [], which may be filled with a term M with the
notation R[M ] . Reduction contexts indicate, in a compact way, the evaluation
order of the language. The operational semantics is now defined by the following
reduction rules:
R[((λx.M ) V )] → R[[V /x]M ]
R[((fexpr V ) M )] → R[(V pM q)]
R[(eval pM q)] → R[M ]
8
The second and third reduction rules rely on an encoding function p·q, defined
just below. We first observe that intuitively, ((fexpr V ) M ) calls V on the
encoding of M , where M need not be a value. Indeed, the whole point is that
arbitrary program terms M , including ones which would otherwise reduce, are
frozen by fexpr and presented to V . Frozen terms may then be thawed using
eval. As Wand explains in an endnote, it is necessary to include eval as a
primitive in the language, due to the way the encoding works. We will return
to this point below. The encoding is defined as follows:
pxq = λa b c d e.a x
pM N q = λa b c d e.b pM q pN q
pλx.M q = λa b c d e.c λx.pM q
p(fexpr M )q = λa b c d e.d pM q
p(eval M )q = λa b c d e.e pM q
9
c1 , . . . , cn , one for each constructor of the datatype. These arguments are it-
erators, which will be applied according to the structure of the data. This M
then applies the i’th iterator to, not literally the input arguments x1 , . . . , xa(i) ,
but rather to what we might think of as the transposition of those inputs to
the iterators c1 , . . . , cn . That is, each Church-encoded piece of data, such as the
arguments x1 , . . . , xa(i) , begins by accepting n iterators to apply according to
the structure of the data. Our term M transposes the arguments to the con-
structor, so that they apply the given iterators c1 , . . . , cn , rather than whatever
other iterators they would have taken in.
A simple example helps demonstrate this encoding. Consider the inductive
datatype of the natural numbers in unary notation. There are two constructors,
S (successor) and Z (zero), where the former has arity 1 and the latter has arity
0. They are Church encoded like this:
S := λx1 .λs z.s (x1 s z)
Z := λs z.z
With this encoding, the first few numerals are defined as follows, and satisfy
the stated β-equivalences:
0 := Z =β λs z.z
1 := SZ =β λs z.s z
2 := S (S Z) =β λs z.s (s z)
3 := S (S (S Z)) =β λs z.s (s (s z))
10
With this encoding, we may obtain the first few numerals by call-by-value re-
duction (denoted here by ⇓cbv ) using S and Z:
0 := Z ⇓cbv λs z.z
1 := SZ ⇓cbv λs z.s 0
2 := S (S Z) ⇓cbv λs z.s 1
3 := S (S (S Z)) ⇓cbv λs z.s 2
λs z.s (s z)
11
Scott-encoded data, in contrast, is not in any obvious way typable in pure
System F. They appear to require both universal and recursive types for ty-
pability. Since strong normalization fails in the presence of recursive types,
we cannot establish termination of programs manipulating Scott-encoded data
just by static typing in a traditional type system. Nevertheless, the Scott en-
coding enjoys two critical advantages over the Church encoding, which should
make them preferable for general programming. First, constructor terms (like
S (S Z)) evaluate to their intended encodings in call-by-value lambda calculus.
This is not the case with the Church encoding, where a constructor term like
(S Z) evaluates to a value in one step as follows in the call-by-value strategy:
(S Z) → λs z.s (Z s z)
In order to obtain the intended encoding λs z.s z, two β-reductions would need
to be performed beneath a lambda abstraction, which is not allowed with the
call-by-value or call-by-name strategy.
The second advantage of the Scott encoding is that constant-time selector
functions are easily definable, as discussed above. With the Church encoding,
in contrast, known implementations of predecessor are rather complicated, and
run in time linear in the size of the input numeral.
12
by Endnote 2 of his paper. Wand there explains that he was led to include eval
as a primitive operation in the language because it is not in any obvious way
definable, given the Mogensen-Scott encoding. This limitation is not unique to
eval. There is no obvious way to define any non-trivial code-generating meta-
programs (like eval) in Wand’s system. The problem is that there is no way to
compute beneath a lambda abstraction. For example, as mentioned in Wand’s
endnote, suppose we try to implement eval. Mogensen gives code for eval that
works, but only if arbitrary β-reduction is used. In the call-by-value setting of
Wand’s language, we could try something like the following:
13
also interested in completeness. We might take completeness in this context
to mean that every scope-safe function implementable in an untyped lambda
calculus with more general, non-scope-safe meta-programming constructs (e.g.,
LISP) can be implemented in the scope-safe language. While independence and
completeness are both desired properties of the language, proving or disproving
these must remain to future work.
Archon is suitable for general meta-programming, as demonstrated by sev-
eral examples in Section 8 below. The shortcoming of Wand’s language, namely
the inability to compute beneath lambda binders, is repaired in Archon. Or-
thogonally, Archon opts to place the burden of reflection on the programming
language, rather than the reflective encoding. Archon adopts the identity func-
tion for the encoding, at the cost of introducing new reflective programming
constructs for operating on program terms. Hence, Archon meta-programs
operate directly on raw program terms, without any encoding. This shift of the
burden of reflection from the encoding to the language is justified as follows.
First, the language must already be extended in some way to allow computation
beneath lambda abstractions. Once we have begun extending the language to
allow for reflection, we can achieve a simpler language by shifting all reflective
burden from the encoding to the language. This simplification has practical
benefits. In reasoning (either formally or informally) about the behavior of
meta-programs, we can consider the manipulated programs directly, in their
natural form as programs of the language, and not via a non-trivial encoding.
In contrast, with Wand’s language, we must contend not only with additional
reflective language constructs, but with the overhead of the Mogensen-Scott
encoding.
One quantitative measure of the simplicity of the language design is its lo-
cality, in the following sense. Wand’s reduction rules for fexpr and eval require
recursive meta-level computation to apply or undo the Mogensen-Scott encod-
ing, for a single reduction step. In contrast, Archon’s reflective constructs
require only constant-time meta-level computation for a single reduction step.
A single step of evaluation, without any recursive meta-level evaluation, suffices
to eliminate any use of Archon’s reflective constructs, in favor of constructs of
untyped lambda calculus.
7.1 Syntax
The syntax of Archon terms T (we also write M , N , R) appears in Figure 1.
We write x (as well as y and z) only for variables, drawn from some countably
infinite set. We then have call-by-value λ-abstractions and call-by-name λ̄-
abstractions, and applications (written using juxtaposition as usual). When a
call-by-value lambda abstraction appears in an application, its argument must
be evaluated before the application can be β-reduced. In contrast, arguments are
passed to call-by-name lambda abstractions unevaluated. This is useful in meta-
programming for passing around raw program terms, which would otherwise
evaluate.
For meta-programming purposes, it seems required in practice to allow com-
14
T ::= x | λx.T | λ̄x.T | T T | open T T | vcomp T T
| swap T | T : T T T T T T T
putation with open terms (i.e., terms which contain free variables). So in Ar-
chon, (free) variables evaluate to themselves. In LISP languages, in contrast,
evaluation of a free variable is not allowed (and typically causes evaluation to
abort with an error). Applications like a b, where a and b are free variables,
also evaluate to themselves. Indeed, the set of values V in Archon is somewhat
more complex than usual in call-by-value or call-by-name lambda calculus:
The remaining four constructs of the language are the reflective constructs,
informally explained next (the formal operational semantics is given in the next
Section). We rely on no parsing conventions for Archon below, except the
(standard) ones mentioned above from lambda calculus.
15
7.1.2 Comparing Variables
For some meta-programs it is necessary to test whether two free variables are
the same or different. This is done in Archon using the vcomp construct. The
term vcomp T1 T2 evaluates to true if T1 and T2 are identical free variables.
Otherwise it evaluates to false. Here and below, the expressions true and false
are abbreviations for Scott-encoded call-by-name booleans λ̄x.λ̄y.x and λ̄x.λ̄y.y.
Note that the Scott and Church encodings coincide on enumerated datatypes
like the boolean datatype. Note also that using call-by-name abstractions here
(and in other Scott-encoded data) means that unused cases are not executed
when a piece of data is used as a case construct.
It may not be obvious to the reader why this operation is practically useful, but
it turns out to be necessary for a class of practical meta-programs, in particular
those which must recursively traverse Archon terms to compute some resulting
term, using the decomposition operator discussed next. An example of such a
meta-program is given later, in Figure 9.
It is not clear if swap can be defined using other language features. One
might consider defining it by opening the lambda abstraction twice, and then
using a recursive function to traverse the body and swap the variables (aided,
for example, by variable comparison) wherever they occur. But such traversals,
as just mentioned, appear to need swap already. So it may not be possible to
define swap in terms of other operations. This question must be left open. Just
as for open, the behavior of swap when its subexpression does not evaluate to
a consecutively nested lambda abstraction is not important, and is chosen to be
a rather arbitrary term.
7.1.4 Decomposition
The final construct of Figure 1 is called decomposition. This is an inten-
sional case-analysis construct, which takes apart raw (i.e., unevaluated) program
terms, and passes their immediate subterms as arguments to the appropriate
branch of the case. In the decomposition
T :T T T T T T T
the first term is the one being decomposed, and the remaining seven terms
are the terms to use for each of the seven kinds of constructs, in the order
in which they appear in Figure 1. Observe that Archon does have exactly
seven constructs, if we lump call-by-value and call-by-name lambda abstractions
16
together, as we do. They are distinguished in decompositions by passing a
Scott-encoded boolean to the case for lambda abstractions, which is true for
call-by-value, and false for call-by-name. For a simple example of the use of
decomposition, the following term evaluates to b a (here writing in the unused
cases for an arbitrary lambda term):
(a b) : (λx y.y x)
7.3 Meta-Theory
The side condition in E-OpenLam prevents confusion of the newly introduced
free variable with other free variables. The only other potential opportunity for
variable capture is in applying substitutions, but these are capture-avoiding by
definition. This fact and the following theorem, easily proved by induction on
the structure of computations, establish scope safety of the language:
The following theorem can also be established. Together with the implementabil-
ity of a test for α-equivalence in the language (Section 8.2), it suffices to prove
that α-equivalence of programs coincides with contextual equivalence.
17
E-Var
x ⇓ x
E-Lam
λ∗ x.T ⇓ λ∗ x.T
E-VcompEqVars
vcomp x x ⇓ true
T1 6≡ T2 or T1 or T2 not a variable
E-Vcomp
vcomp T1 T2 ⇓ false
T ⇓ λ1 x.λ2 y.R
E-SwapLam
swap T ⇓ λ2 y.λ1 x.R
18
T1 z ⇓ R
E-DecompVar
z : T1 T2 T3 T4 T5 T6 T7 ⇓ R
T2 true λz.M ⇓ R
E-DecompCbv
(λz.M ) : T1 T2 T3 T4 T5 T6 T7 ⇓ R
T2 false λ̄z.M ⇓ R
E-DecompCbn
(λ̄z.M ) : T1 T2 T3 T4 T5 T6 T7 ⇓ R
T3 M N ⇓ R
E-DecompApp
(M N ) : T1 T2 T3 T4 T5 T6 T7 ⇓ R
T4 M N ⇓ R
E-DecompOpen
(open M N ) : T1 T2 T3 T4 T5 T6 T7 ⇓ R
T5 M N ⇓ R
E-DecompVcomp
(vcomp M N ) : T1 T2 T3 T4 T5 T6 T7 ⇓ R
T6 M ⇓ R
E-DecompSwap
(swap M ) : T1 T2 T3 T4 T5 T6 T7 ⇓ R
T7 M M1 M2 M3 M4 M5 M6 M7 ⇓ R
E-DecompDecomp
(M : M1 M2 M3 M4 M5 M6 M7 )
: T1 T2 T3 T4 T5 T6 T7 ⇓ R
19
Theorem 2 For all α-equivalent terms T and T 0 , if T ⇓ R, then T 0 ⇓ R0
where R0 is α-equivalent to R.
and also
Γ ` (open (λ∗ x.T1 ) T2 ) =α (open (λ∗ x0 .T10 ) T20 )
From these facts, we get y 6∈ FV(λ∗ x0 .T10 ) ∪ FV(T20 ). Hence, we have
8 Meta-Programming Examples
We now consider meta-programming examples in Archon. We may distin-
guish meta-programs which recursively inspect code from meta-programs which
also recursively produce it. The former class can be implemented in Wand’s
language, since inspection of Mogensen-Scott encoded lambda terms is sup-
ported. As discussed in Section 6, the latter class is not in any obvious way
implementable, except in trivial cases (e.g., constant functions). Archon can
implement both kinds of meta-programs, and has some advantages for imple-
menting code-inspecting meta-programs. So we begin by comparing Archon
and Wand’s language on a meta-program which is canonical for Wand’s pur-
poses, namely testing arbitrary terms for α-equivalence. Note that this is prov-
ably not implementable in untyped lambda calculus, so both Wand’s language
and Archon are more expressive (see, e.g., [23, Section 3.3.4]). Section 8.3
gives another representative meta-program, namely to compute from n and f
the function λx.f n x, where f n denotes n-fold application of f .
20
Figure 1]). It must furthermore operate only on closed lambda terms, since it
has no means to inspect free variables. Indeed, as eq traverses terms, it must
substitute some entities which it can inspect for all the bound variables retained
in the Mogensen-Scott encoding. It may use Scott- or Church-encoded natural
numbers for this purpose. The number to introduce for the next bound variable
it encounters must be taken as an additional input to eq.
So we have
on app p(a b)q (λx y.x) false
evaluating to a, and
21
use for its bound variable. This, of course, causes the number to be substituted
for the bound variable. It can then be inspected in the case for variables (the
first case).
This operator must be used instead of fix, since the Archon implementation
of eq uses call-by-name lambda abstraction to receive the two terms to test for
α-equivalence.
The Archon code for eq also needs a helper function beta reducek for per-
forming a single β-reduction. The basic idea of this function is that given two
terms, where the first term M is either of the form λx.B or of the form λ̄x.B, and
the second is N ; it should return [N/x]B. We cannot return this term directly,
without risk that it might reduce. So we return it instead via a continuation,
provided as a third argument. Here and below, let I abbreviate λz.z. The code
for this helper function is the following, which requires some explanation (note
that is used here just as another variable, with the name chosen to indicate
22
eq := fix λeq n s t.
(s (λx.eqnat x t)
(λs1 s2 .on app t (λt1 t2 .and (eq n s1 t1 ) (eq n s2 t2 )) false)
(λs1 .on lam t (λt1 .eq (S n) (s1 n) (t1 n)) false)
(λs1 .on fexpr t (λt1 .eq n s1 t1 ) false)
(λs1 .on eval t (λt1 .eq n s1 t1 ) false))
eq := nfix λeq.λ̄s t.
let eq2 = λon op.λ̄m n.
(on op t (λ̄m2 n2 .and (eq m m2 ) (eq n n2 ))
false) in
(s : (vcomp s t)
(λ̄cbv1 s.
on lam t (λ̄cbv2 t.
and (beq cbv1 cbv2 )
((open s λ̄x tb.(beta reducek t x (eq tb)))
I))
false)
(eq2 on app)
(eq2 on open)
(eq2 on vcomp)
(λ̄m.on swap t (eq m) false)
(λ̄m0 m1 m2 m3 m4 m5 m6 m7 .
on decomp t
(λ̄n0 n1 n2 n3 n4 n5 n6 n7 .
(and (eq m0 n0 ) (and (eq m1 n1 )
(and (eq m2 n2 ) (and (eq m3 n3 )
(and (eq m4 n4 ) (and (eq m5 n5 )
(and (eq m6 n6 ) (eq m7 n7 )))))))))
false))
23
informally that it is not subsequently used):
The action of this code is rather subtle, but it well illustrates the power of
the open construct. Our idea in computing [N/x]B from λx.B and N is illus-
trated informally by the following transformation sequence (the case for λ̄x.B
is similar):
In more detail, we first wish to insert a dummy binder beneath the λx, so that
applying the lambda abstraction to N will not trigger reductions in B. So we
first want to compute a term M1 defined to be λx.λz.B. Then we want to
convert the call-by-value λx to a call-by-name λ̄x, so that applying the lambda
abstraction to N will not cause N to evaluate. So we next want to compute
a term M2 defined to be λ̄x.λz.B. Finally, we can achieve our β-reduction by
applying this M2 directly to N , and allowing reduction in Archon to perform
the substitution. This results in a term R of the form λz.[N/x]B. It suffices
now just to open this lambda abstraction and apply the continuation k to its
body (which is [N/x]B). Whatever result the continuation produces will then
be closed beneath a binding λz, since uses of open (such as this one opening R)
always rebind the variable of the opened lambda abstraction. To eliminate this
dummy binding, we apply the result to an arbitrary value (here, I).
We leave it to the reader to confirm that the code above computes M1 as
specified, and just consider in detail the computation of M2. The code for
beta reducek computes M2 (“λ̄x.λz.B”) from M1 as follows. It opens a new
lambda abstraction λ̄y.y, using λ̄ y.M1 y. The latter term accepts the bound
variable (y, received via variable ) and the body (also y, received by variable
y) of this λ̄y.y. It then applies M1 to y. This results in λz.[y/x]B. The result
of this computation is then closed beneath a rebinding λ̄y, since this is always
how evaluation of an open expression finishes. So the result is λ̄y.λz.[y/x]B,
which is α-equivalent to the desired term λ̄x.λz.B. The use of variable y here is
purely for readability of the example. Thanks to the scope safety of Archon,
the code works just as well if we use x instead of y.
This example provides some small evidence for completeness. It shows that
an operation, namely changing a lambda abstraction from call-by-value to call-
by-name, which one might otherwise doubt possible in Archon, is in fact im-
plementable. Similar code can convert a call-by-name to a call-by-value abstrac-
tion.
24
8.2.2 The Code for eq
Figure 6 gives the Archon code for the test eq for α-equivalence. Its basic struc-
ture resembles that of the code in Wand’s language (Figure 5). Some differences
have already been noted. The top-level case analysis is performed using Ar-
chon’s decomposition operator, instead of by application of a Mogensen-Scott
encoded term. The first case, for variables, is implemented using vcomp, as ex-
pected. The case for lambda abstractions (the second case) is the most complex.
Naturally, for two lambda abstractions to be α-equivalent, they must either both
be call-by-value or both call-by-name. This is checked by beq cbv1 cbv2 . To con-
tinue the comparison, we wish to compare the bodies. But we must make sure
the bodies are expressed using the same variable. So we open one lambda ab-
straction (“s”), obtain its bound variable (“x”) and do a single β-reduction of
t on x (using beta reducek, defined in the previous Section). This causes uses of
t’s bound variable to be replaced in its body (“tb”) by x. We may then compare
the two bodies. The lambda binder placed around the (boolean) result is then
removed by applying to I (defined, as stated above, to be λz.z).
25
iter h := fix λiter h x n f.
(n (λp.let r = iter h x p f in
((open r λ̄x rb.λd.f rb) I))
λd.x)
open∗ := λx.λ̄y.open x y
This term works just like open except that it evaluates its first argument. It
is used in the code for decode to cause the recursive calls to execute instead
of intensionally analyzing them. (Recall from Section 7.2 that for inessential
reasons, we have designed the operational semantics of open so that it does not
evaluate its first argument.) Also, recall that, as stated above, I is defined to
be λz.z.
The only complication that arises is for the case of lambda abstractions
(the second case in the Archon code of Figure 9). After the body has been
decoded and evaluation of the open expression completes, we have a dummy
lambda abstraction trapped beneath the binding that has been replaced by the
evaluation of open. This is similar to the situation with iter in Section 8.3.
We just swap the two bindings using swap. For example, the Mogensen-Scott
26
pxq = λV L A O C S D.V x
pλz.T q = λV L A O C S D.L true λz.pT q
pλ̄z.T q = λV L A O C S D.L false λ̄z.pT q
pT1 T2 q = λV L A O C S D.A pT1 q pT2 q
popen T1 T2 q = λV L A O C S D.O pT1 q pT2 q
pvcomp T1 T2 q = λV L A O C S D.C pT1 q pT2 q
pswap T q = λV L A O C S D.S pT q
pT : T1 T2 T3 T4 T5 T6 T7 q = λV L A O C S D.
D pT q pT1 q pT2 q pT3 q
pT4 q pT5 q pT6 q pT7 q
27
xλV L A O C S D.V xy = x
xλV L A O C S D.L true λx.T y = λx.xT y
xλV L A O C S D.L false λ̄x.T y = λ̄x.xT y
xλV L A O C S D.A T1 T2 y = xT1 y xT2 y
xλV L A O C S D.O T1 T2 y = open xT1 y xT2 y
xλV L A O C S D.C T1 T2 y = vcomp xT1 y xT2 y
xλV L A O C S D.S T y = swap xT y
xλV L A O C S D.D T T1 T2 T3
T4 T5 T6 T7 y = xT y : xT1 y xT2 y xT3 y
xT4 y xT5 y xT6 y xT7 y
28
encoding of λ̄x.x is
If decode is called on this term, the code in the second case of Figure 9 tells us
first to open the λx and recursively decode the body (decode F 0 ). This gives us
just x, frozen beneath a dummy lambda abstraction. When the open expression
finishes, we thus have λ̄x.λd.x. To finish decoding the lambda abstraction, we
need to pull the λd out from beneath the λ̄x. This is done (by the code in the
second case) using swap, which yields λd.λ̄x.x, as desired.
In all the other cases, a combination of open and call-by-name abstraction is
used to move live code from under dummy abstractions and reassemble that code
under a new dummy abstraction. Since open always replaces the bound variable
(in this case, a now unused dummy), it is necessary to apply the resulting term to
several arbitrary terms (here, I) to eliminate the unused dummies. Correctness
of the implementation can be expressed as follows:
9 Implementation
As pointed out by Wand, in any language where α-equivalence of program terms
coincides with contextual equivalence of their encodings, compilation steps like
source-to-source optimization are rendered unsound. Sound, efficient imple-
mentation of structurally reflective languages like Archon is thus a non-trivial
issue. It is not an insuperable one, however. For example, source-to-source
optimization could be allowed for programs which are not analyzed reflectively,
as determined by a static analysis. Furthermore, while source-to-source opti-
mization is unsound in general, it may still be possible to design more efficient
abstract machines for Archon, incorporating optimizations to the entire oper-
ational semantics. Exploring these ideas must remain to future work.
This section discusses initial efforts at an graph-reducing interpreter. Ap-
proaches based on compilation to an abstract machine result in large perfor-
mance gains over interpretation, and are considered necessary for serious im-
plementation. Nevertheless, interpretation is the appropriate starting point for
a language with novel constructs like Archon’s (cf. [19]). The prototype Ar-
chon interpreter is written in just under 1000 lines of Java, version 1.4.2. It
may be downloaded from http://cl.cse.wustl.edu/archon/, together with
all the examples considered in this paper. Since Java source code may be com-
piled to native code using the gcj compiler, we obtain a reasonably efficient
executable. The use of a garbage-collected language eliminates the burden of
implementing garbage collection in the interpreter. For this reason, functional
languages are also a good choice for implementation of Archon.
To reduce a β-redex, a pointer (which we will call the assigned variable
pointer) is set from the in-memory representation of the bound variable to the
argument value. Most of the computation time in graph reduction is consumed
29
copying lambda abstractions. Lambda abstractions must be copied, in order
to avoid setting the same assigned variable pointer in two different ways for
two different applications. We will call the operation of duplicating lambda
expressions cloning. The Archon interpreter uses three optimizations to reduce
the amount and cost of cloning:
1. Do not clone lambda abstractions the first time they are applied in a
β-reduction. Only clone them for reductions after the first one.
2. Clone lambda abstractions only when following an assigned variable pointer.
This is justified because the only point in which sharing is introduced to
the term graph is when the assigned variable pointer is set from a variable
which is used more than once.
30
Benchmark −1 − 3 −1 + 3 +1 − 3 +1 + 3
eq.a 53.1 (118,586) 6.8 (202,888) 9.1 (32,690) 3.4 (90,113)
fact.a 7.2 (6,563) 0.3 (13,118) 0.2 (1,639) 0.2 (4,102)
cache. The version of gcj used is 3.4.4. A “+1” or “−1” in the heading
indicates whether optimization 1 of the previous Section is enabled or disabled,
respectively (similarly for “+3” and “−3”). Optimization 2 is not easy to disable
in the implementation, and so its effect is not measured here.
The eq.a benchmark takes 295,146 β-reductions for Archon to evaluate,
and fact.a take 12,381. The number of times an expression is cloned is given
in Figure 10 in parentheses. Note that with optimization 3 turned off, there are
fewer clonings, because each cloning does more work (by following all assigned
variable pointers).
31
10 Conclusion
This paper has defined the Archon directly reflective meta-programming lan-
guage. This language satisfies a number of desirable properties, including pu-
rity and scope safety. Programs are encoded using higher-order abstract syntax
(since they are trivially encoded as themselves). Archon is also suitable for
code-generating meta-programs. The language extends untyped lambda calcu-
lus with call-by-value and call-by-name abstractions, as well as novel reflective
features for swapping consecutive nested lambda binders, opening lambda ab-
stractions to compute on their bodies, comparing free variables, and decompos-
ing arbitrary program terms. An optimized interpreter has also been presented.
As we have seen, Archon is substantially closer to completeness than Wand’s
system, in which code-generating meta-programs are not (in any direct way)
generally implementable. Whether or not the proposed language is actually
complete is left open.
Acknowledgments. Many thanks to the anonymous reviewers of previ-
ous versions of this paper for their thorough reading and insightful criticisms.
Their comments on earlier versions helped greatly improve this paper. Thanks
to Walid Taha for helpful conversations on meta-programming and the ideas
of the current paper. Thanks also to Matthew Bensley for contributions to
the Archon project including help implementing several of the examples from
Sections 8 and 9.
References
[1] K. Arnold, J. Gosling, and D. Holmes. The Java Programming Language.
Prentice Hall, 2000.
[2] B. Aydemir, A. Bohannon, M. Fairbairn, J. Foster, B. Pierce, P. Sewell,
D. Vytiniotis, G. Washburn, S. Weirich, and S. Zdancewic. Mechanized
metatheory for the masses: The POPLmark Challenge. In Proceedings
of the Eighteenth International Conference on Theorem Proving in Higher
Order Logics (TPHOLs 2005), 2005.
[3] C. Chen and H. Xi. Meta-Programming through Typeful Code Represen-
tation. Journal of Functional Programming, 15(6):797–835, 2005.
[4] M. Clavel, F. Durán, S. Eker, P. Lincoln, N. Martı́-Oliet, and J. Meseguer.
Metalevel Computation in Maude. In Proc. 2nd Intl. Workshop on Rewrit-
ing Logic and its Applications, Electronic Notes in Theoretical Computer
Science. Elsevier, 1998.
[5] R. Constable. Using reflection to explain and enhance type theory. In Proof
and Computation, NATO ASI Series. Springer-Verlag, 1994.
[6] K. Crary, S. Weirich, and G. Morrisett. Intensional polymorphism in type-
erasure semantics. Journal of Functional Programming, 12(06):567–600,
2002.
32
[7] H. Curry, J. Hindley, and J. Seldin. Combinatory Logic, volume 2. North-
Holland Publishing Company, 1972.
[8] M. Davis and J. Schwartz. Metamathematical Extensibility for Theorem
Verifiers and Proof-Checkers. Computers and Mathematics with Applica-
tions, 5:217–230, 1979.
[9] J. Ferber. Computational reflection in class based object-oriented lan-
guages. In OOPSLA ’89: Conference Proceedings on Object-Oriented Pro-
gramming Systems, Languages and Applications, pages 317–326, New York,
NY, USA, 1989. ACM Press.
[13] A. Goldberg and D. Robson. Smalltalk-80: The Language and Its Imple-
mentation. Addison-Wesley, 1983.
[14] J. Harrison. Metatheory and Reflection in Theorem Proving: A Survey
and Critique. Technical Report CRC-053, SRI Cambridge, Millers Yard,
Cambridge, UK, 1995.
33
[21] R. Kelsey, W. Clinger, J. Rees, et al. Revised5 Report on the Algorithmic
Language Scheme. SIGPLAN Notices, 33(9):26–76, 1998.
[22] I.-S. Kim, K. Yi, and C. Calcagno. A Polymorphic Modal Type System for
Lisp-Like Multi-Staged Languages. In The 33rd ACM SIGPLAN-SIGACT
Symposium on Principles of Programming Languages, pages 257–268, 2006.
[23] J. Klop and R. de Vrijer. Examples of TRSs and Special Rewriting For-
mats. In TERESE, editor, Term Rewriting Systems, chapter 3. Cambridge
University Press, 2003.
34
[36] A. Robinson and A. Voronkov, editors. Handbook of Automated Reasoning.
Elsevier and MIT Press, 2001.
[37] H. Rueß. Computational Reflection in the Calculus of Constructions and
its Application to Theorem Proving. In Proceedings of the Third Interna-
tional Conference on Typed Lambda Calculi and Applications, pages 319–
335. Springer-Verlag, 1997.
[38] C. Schürmann, A. Poswolsky, and J. Sarnat. The ∇-Calculus. Functional
Programming with Higher-Order Encodings. In Proceedings of the 7th In-
ternational Conference on Typed Lambda Calculi and Applications, pages
339–353. Springer-Verlag, 2005.
[39] J. Seldin and J. Hindley, editors. To H.B. Curry: Essays on Combinatory
Logic, Lambda Calculus, and Formalism. Academic Press, 1980.
[40] J. Siskind and B. Pearlmutter. First-Class Nonstandard Interpretations
by Opening Closures. In Proceedings of the Symposium on Principles of
Programming Languages (POPL), 2007.
[41] B. Smith. Reflection and Semantics in LISP. In Proceedings of the 11th
ACM SIGACT-SIGPLAN Symposium on Principles of Programming Lan-
guages, pages 23–35, 1984.
[42] G. Steele. Common LISP: the Language (2nd ed.). Digital Press, 1990.
[43] B. Stroustrup. The C++ Programming Language (3rd Edition). Addison-
Wesley, 1997.
[44] W. Taha. Multi-Stage Programming: Its Theory and Applications. PhD
thesis, Oregon Graduate Institute, November 1999.
[45] The Coq Development Team. The Coq Proof Assistant Reference Manual,
Version V8.0, 2004. http://coq.inria.fr.
[46] The GHC Team. The Glorious Glasgow Haskell Compilation System
User’s Guide, Version 6.6.1, 2007. http://www.haskell.org/ghc/docs/
latest/html/users_guide/.
[47] The PLT Group. PLT DrScheme: Programming Environment Manual,
2007. http://download.plt-scheme.org/doc/drscheme/.
[48] P. Wadler. Theorems for free! In 4th International Conference on Func-
tional Programming and Computer Architecture, 1989.
[49] C. Wadsworth. Some Unusual λ-Calculus Numeral Systems, pages 215–230.
In Seldin and Hindley [39], 1980.
[50] M. Wand. The Theory of Fexprs is Trivial. Lisp and Symbolic Computation,
10(3):189–199, 1998.
35
[51] E. Westbrook. Free variable types. In Seventh Symposium on Trends in
Functional Programming (TFP 06), April 2006.
[52] A. Wright and R. Cartwright. A Practical Soft Type System for Scheme.
ACM Trans. Program. Lang. Syst., 19(1):87–152, 1997.
36
A Haskell Code for the Factorial Benchmark
The type of Scott-encoded unary numbers may be considered to be
µN. (∀b. (N → b) → b → b) → N
module Ns where
37
B Archon Code for the Factorial Benchmark
Note that the prototype Archon implementation uses “ˆ ” for λ̄ and “\” for λ.
Also, to allow simple recursive descent parsing, application is written explicitly,
in prefix notation, with “@”. Finally “$ x T1 T2” means that we define x to be
the value of T1 in T2. Arguments in applications are sometimes printed directly
below the corresponding “@” symbol.
$ zero ^ s ^ z z
$ succ \ n ^ s ^ z @ s n
$ one @ succ zero
$ fix \f @ \x @ f \y @ @ x x y
\x @ f \y @ @ x x y
$ plus @ fix \ plus \ n \ m @ @ n ^ p @ @ plus p
@ succ m
m
$ mult @ fix \ mult \ n \ m @ @ n ^ p @ @ plus @ @ mult p m m
zero
$ fact @ fix \ fact \ n @ @ n ^ p @ @ mult n @ fact p
one
$ show @ fix \ show \ n @ @ n ^ p @ S @ show p
Z
$ test0 @ fact @ succ @ succ @ succ @ succ @ succ zero
38
C Scheme Code for the Factorial Benchmark
(define (nz)
(lambda (s z) z))
(define (ns n)
(lambda (s z) (s n)))
(define (nat-to-string n)
(n (lambda (n) (string-append "S " (nat-to-string n))) "Z"))
(define (plus n m)
(n (lambda (p) (plus p (ns m))) m))
(define (mult n m)
(n (lambda (p) (plus (mult p m) m)) (nz)))
(define (fact n)
(n (lambda (p) (mult (fact p) n)) (ns (nz))))
(define (test0)
(fact (ns (ns (ns (ns (ns (nz))))))))
(define (test)
(let ((x (test0))) (mult x x)))
39