Reflection and Open Implementations
Reflection and Open Implementations
Éric Tanter
This section introduces the concept of reflection and its application to program-
ming languages. What reflection actually means is pretty well embodied in the
following explanation of the word reflect:
This section is structured as follows. Section 1.1 gives a brief historical in-
troduction to the difference between programs and data and concludes with the
appealing idea of conceiving programs as data for other programs 1 . Then, the
notions of metaprogramming and reflection are defined (Section 1.2). Section 1.3
exposes the seminal experiments in reflection in programming languages, based
on the idea of reflective towers. Finally, Section 1.4 discusses characteristics of
reflective languages as well as some of these languages.
1
Credits for this historical introduction to the distinction between programs and data
go to Julien Vayssière [111].
2
Metasystem
A computational system whose domain is another computational
system.
Reflective system
A metasystem causally connected to itself.
(a) (b)
reflective
system (c)
Fig. 1. Computational system (a), metasystem (b) and reflective system (c).
Reflection
An entity’s integral ability to represent, operate on, and other-
wise deal with its self in the same way that it represents, operates
on, and deals with its primary subject matter.
4
..
.
RPP running at level 3
The reflective tower is a special case of processing towers, in the sense that
it is infinite and homogeneous. Finite heterogeneous processing towers are actu-
ally commonplace: consider a Java program at level 0, run by the Java Virtual
Machine which is a machine language program running at level 1, which in turn
is run by the hardware at level 2, thereby stopping the tower. In a reflective
language, user code may not only run at level 0, but at any level above, hence
gaining power to direct the course of its own execution.
5
Dealing with Infinity To deal with infinity, des Rivières and Smith introduced
the notion of the degree of introspection of a program: in any single program p
and input i, only a finite number of levels n are needed to run the program; this
number is the degree of introspection of the considered program. Hence, given
n, the level n + 1 interpreter can be replaced by an implementation processor G,
which is a real, non-reflective processor.
Since n is unlikely to be determined without actually running program p, the
implementation processor G is proposed to be a level-shifting processor (LSP):
such a processor is able, when it is determined (dynamically) that a new level of
processing is required, to create the explicit state of the LSP on the fly as if it
had run since the beginning of the program, and to resume the computation from
this state. Therefore, along the execution of a program, G will shift up levels,
progressively climbing to higher and higher reflective levels. Recall that shifts up
are triggered by calling reflective procedures. In order to be efficient, however,
a LSP should never run at any higher level than necessary, hence requiring the
ability to shift down as soon as possible (Fig. 3).
execution
time
Reification and reflection Wand and Friedman bring some more light on
what shifting up and down actually means, in two major papers that attempted
at giving a more formal, denotational account of reflection. They show in [36]
that the concept of reflection as formulated by Smith can be decomposed in
two processes, called reification and reflection, which respectively correspond to
shifting up and down:
Reification
The process by which the state of the interpreter is passed to the
program itself, suitably packaged (reified) so that the program
can manipulate it.
In the context of a conventional operational semantics model, the state of the
interpreter is defined as interpreter registers holding an expression, an environ-
ment and a continuation. As further mentioned, the process of reification can be
6
thought of as converting program into data. The data representing the piece of
program is also called a reification.
Reflection
The process by which program values are re-installed as the state
of the interpreter.
In this context, the program values are defined as values for an expression, an
environment, and a continuation. The process of reflection can therefore be seen
as converting data into program. It is also sometimes referred to as absorption [28,
91, 24] or deification [126, 31].
Following their quest for a formal understanding of reflection, Wand and
Friedman manage to give a semantic account of Smith’s reflective tower. Using
a meta-continuation semantics, their account of the reflective tower does not
employ reflection to explain reflection. It is presented in an appropriately-named
paper: The Mystery of the Tower Revealed: A Non-Reflective Description of the
Reflective Tower [113].
actions the program is allowed to carry over these reifications can also vary.
Therefore, a first distinction is made between introspection and intercession:
Introspection
The ability of a program to simply reason about reifications of
otherwise implicit aspects of itself or of the programming lan-
guage implementation (processor).
In analogy with file systems, introspection can be seen as a read access to reifi-
cations.
Intercession
The ability of a program to actually act upon reifications of oth-
erwise implicit aspects of itself or of the programming language
implementation (processor).
Following the same analogy, intercession corresponds to a write access to reifica-
tions. The causal connection property ensures that changes made to reifications
are indeed effective.
Another distinction is made between structural and behavioral reflection, de-
pending on the representation reifications give access to:
Structural reflection
The ability of a program to access a representation of its struc-
ture, as it is defined in the programming language.
For instance, in an object-oriented language, structural reflection gives access to
the classes in the program as well as their defined members.
Behavioral reflection
The ability of a program to access a dynamic representation of
itself, that is to say, of the operational execution of the program
as it is defined by the programming language implementation
(processor).
In an object-oriented language, behavioral reflection could for instance give ac-
cess to base-level operations such as method calls, field accesses, as well as the
state of the execution stack of the various threads in the program.
Behavioral reflection was actually pioneered by Smith, as discussed previ-
ously, and is much more difficult to implement than structural reflection since it
is not restricted to the static representation of programs. We will come back on
implementation issues later in this chapter.
The distinction between introspection and intercession and that of behavioral
and structural representations are indeed orthogonal: the former determines the
kind of access given to the representation, whose type is determined by the lat-
ter. Moreover, these distinctions are completely valid in the context of metapro-
grams, not only reflective ones. For instance, a preprocessor is a metaprogram
that uses both structural introspection and intercession. Conversely, an Inte-
grated Development Environment (IDE) only needs structural introspection to
8
in the code. We shall come back on this in Section 4.4 when discussing partial
reflection.
Explicit MOPs are used by base objects to communicate with the metalevel.
This can concretely be done by sending messages to a given metaobject
or by actually changing a metaobject by another one. This usually results
11
Metaclass model In this model, the meta relation is merged with the type
relation: the class of an object is considered as its metaobject, since classes
actually describe the structure and behavior of their instances. Metaclasses are
therefore the metaobjects of classes, because of their ability to describe the
internal structure and behavior of a class. Note that some languages only provide
one metaclass that describes all classes in the system. This was the case of
Smalltalk-76, and is still the case in Java, which furthermore closes the door to
extension since the unique metaclass cannot be subclassed. A unique metaclass
is actually not convenient to allow semantic variations because it propagates
changes to all classes in the system. Conversely, in languages like ObjVLisp and
Smalltalk-80, each class is the unique instance of its own metaclass.
From a behavioral point of view, it is equivalent for an object o to receive a
message m, or for the class of o to receive a message handleMessage (that takes
as parameters both o and m). Thus, the default handling of a method is de-
scribed in the metaclass, since it is where the handleMessage method is defined.
Specializing the default interpretation of a message implies a substitution or an
alteration of the metaclass. This model presents the drawback that all instances
of a class share the same message interpreter: there is no possibility to special-
ize the interpreter for a unique object3 . Furthermore, metaclass substitution is
dangerous and can quickly lead to inconsistencies. Finally, it is not possible for
the metaobject (the class) to keep personal characteristics of objects. These lim-
itations are summarized by Ferber saying that “metaclasses are not meta in the
computational sense, although they are meta in the structural sense” [34].
Metaobject model In this model, the metalink is different from the instance-
of link: classes and metaobjects are distinct objects. Each object has its own
metaobject.
The main distinction between this model and the metaclass model lies in
the separation of structural and computational (behavioral) reflection: in the
metaclass model, classes are used for both structural description (definition of
the instance structure and the set of applicable operations) and computational
description (how a message is interpreted and a method is applied); conversely,
the metaobject model splits them apart: classes handle the structural part, while
metaobjects handle the behavioral part. This model presents many advantages.
First, it is easy to modify the metaobject of a single object; second, an object can
be monitored by its metaobject; finally, defining new ways of handling messages
simply consists in defining new classes of metaobjects (e.g. by subclassing a
default metaobject class).
Message reification Ferber introduces yet another model that consists of reify-
ing the communication itself. In this model, each communication is an object,
instance of a message class, that can react to the send message. It is therefore the
3
Unless a dictionary is kept in the class in order to distinguish between instances, but
then this is really close to the metaobject model, presented afterwards.
13
Where is the reflective tower? The three models presented above are mainly
based on the lookup/apply protocol of object-oriented languages. The CLOS
MOP [53] is rather based around the generic function model, where the appli-
cation of a generic function is reified as a generic function [66]. As discussed
by Malenfant et al., reflective towers appear in both kinds of approaches. Des
Rivières [27] has pointed out the existence of the reflective tower in the CLOS
MOP: the tower appears because, when invoked, the generic function that de-
scribes how generic functions are applied is itself a generic function, and therefore
must invoke itself. This infinite meta-regression is however simply avoided (by
not reifying the application of the MOP generic function). Besides, in [65], the
existence of the reflective tower in the lookup/apply model is shown. The tower
appears because apply methods are themselves methods, which much have their
own apply method.
As argued in [31], since these approaches to reflection do not feed higher-
level interpreters with the code of lower-level interpreters, they have no semantic
foundation, but allow for more efficient implementations. In contrast, MetaJ [31],
3-KRS [62] and Agora [24] are semantics-based, following Smith’s seminal work
on 3-Lisp. According to [66], the most important difference however is that the 3-
Lisp (and alike) tower is potentially infinite, whereas in object-oriented models,
the towers are finite by construction, and the languages always provide mecha-
nisms that stop the tower at some fixed level, possibly differing from methods
to methods.
cation metaobject, etc.. Defining new kind of behaviors can be done by incre-
mentally extending metaobject classes. Furthermore, metaobjects may in some
cases be reusable: indeed, standing above objects from a meta viewpoint makes it
possible for metaobjects to be generic, and hence adequate on different types of
objects. As Stroud and Wu conclude, the key of MOP-based approaches among
others is their flexible approach to reification [94]. Indeed, dedicated preproces-
sors and the like also use a form of reification to operate, but they do so in a very
ad hoc way, that makes them unsuitable for handling other concerns or being
extended.
As we said earlier, the notion of metaobject was first introduced by Pattie Maes
in the context of 3-KRS [63]. In this model, a metaobject is an object which
reflects the structural, and possibly also the computational aspect of a single
object.
Still, in 3-KRS, several metaobjects participate in the representation of a
single object4 : the metaobject of an object has slots that are filled by primi-
tive metaobjects. These primitive metaobjects together represent the complete
3-KRS interpreter. ABCL/R [114], a reflective version of the object-oriented
concurrent system ABCL/1 [127], is another example of such an architecture. In
ABCL/R, the arity of the metalink is also 1-to-1, and the different aspects of a
base object (in this case, variables, scripts, local evaluator and message queue)
are held in the state variables of the metaobject.
Such architectures are qualified as individual-based architectures [70] since a
single object is the unit of computation at the base level (from a metalevel point
of view). The reflective tower (that comes from the fact that a metaobject is also
an object) is called an individual tower. The limitation of such an architecture
4
A distinction is introduced in [70] between metalevel objects and metaobjects. While
metaobjects are metalevel objects, the reverse is not true: some metalevel objects
simply reside at the metalevel, without actually being bound to a base object.
16
lies in its lack of a global view of computation. Since each metaobject is self-
contained (in the sense that it only “sees” its referent), other parts of the base
computation are only accessible through explicit access to their metaobjects.
To address this issue in scenarios dealing with resource management where
a more global view of the computation is needed, the idea of group-wide reflec-
tion was introduced [115]. In group-wide reflective architectures, the collective
behavior of a group of objects is represented as coordinated actions of a group of
metalevel objects, called the metagroup. The reflective tower, that comes from
the fact that a metagroup is itself an object group, is called a group tower. In
this model, there is no intrinsic relation between a particular object and a meta-
object. Rather, the entire object group is the unit of base-level computation
(from a metalevel point of view). The disadvantage of such a model is that the
identity of a given base object is lost at the metalevel, and must therefore be
reconstructed manually.
Naturally, Satoshi et al. proposed the amalgamation of both architectures,
called the hybrid group architecture [70], implemented in ABCL/R2. In this ar-
chitecture, both the individual tower and the group tower are preserved. In their
application context, coordinated resource management, they observe that the
hybrid group architecture does not merely combine the benefits of both archi-
tectures. Rather, it enables advanced coordinated resource management schemes
to be modeled, which would hardly be feasible with previous architectures. This
work therefore brings a first justification to the interest of a more flexible met-
alink. However, in the proposed hybrid architecture, a limitation is that an object
cannot belong to more than one group: although this limitation does make sense
in the precise application context of ABCL/R2, it is, in a more general setting,
questionable. Satoshi et al. actually end up arguing that architectural issues
are fundamental, and that research on more effective architectures should be
pursued.
In their work on the Iguana language (a fully-featured MOP for C++ 5 ),
Gowing et al. expose a fairly flexible approach to the metalink [42]. Metaobject
instances may be shared by several objects, and an object may be controlled
by several metaobjects. In their model, there is one metaobject per reification
category (i.e. features of the object model of the language that can be reified, such
as object creation and deletion, activation frames, state access, etc.). This has
an interesting impact on the modularity of the metalevel, exploited in particular
by McAffer [73], as discussed hereafter.
“[...] the metalevel has been thought of as a place for making small
changes requiring small amounts of code and interaction. We believe that
the metalevel should be viewed as any other potentially large and complex
application – it is in great need of management mechanisms.” [73]
Fine-grained MOPs McAffer also argues for the freedom to design meta-
objects at the appropriate level of granularity. This is indeed just standard
object-oriented programming practice, that leads to better robustness, encap-
sulation, and modularity.
In this direction, a significant improvement in the modularity of a metalevel
architecture is the concept of fine-grained MOPs, introduced in Iguana [42]. This
proposal was motivated by the will to make metaobject protocols practical for
operating systems [41], more precisely, as a mechanism for adaptable system
components. The fine-grained MOPs of Iguana are an enhancement, in terms
of fine granularity and combination possibilities, of the Multi-Model Reflection
Framework developed for AL-1/D [76, 75].
A MOP essentially specifies a reflective object model: the object model spec-
ified by a MOP is implemented by metaobjects. The idea of fine-grained MOPs
is to allow multiple reflective object models to coexist in a given application. For
example, in an application, a distributed object could use a distributed object
model while other objects of the system use the standard (local) object model.
Furthermore, if an object subsequently needs to modify its object model (that
is, its metalevel implementation), it can do so knowing that any changes will not
affect other object models: this is called metalevel locality of change [42]. Iguana
hence provides a very elegant and flexible way of structuring customized met-
alevels from elementary building blocks, with the protocols themselves provid-
ing higher-level building blocks. The Iguana approach to fine-grained metalevel
structuring was later on ported to Java, with Iguana/J [83, 84].
4 Implementing Reflection
4.1 Infinity
system, where the number of metalevels is fixed to four [126]), or more generally,
by relying on laziness, as in all other reflective systems mentioned until now.
Infinite meta-regressions, also called circularities, can also be easily dis-
charged. As explained in [53], there are two kinds of circularity issues: boot-
strapping issues, which are involved with how to get a reflective system up and
running in the first place, and are usually easily tackled in an ad hoc manner; and
metastability issues, which have to do with how a reflective system manages to
run, and to stay running even while fundamental aspects of its implementation
are being changed. Metastability issues require a bit more care and anticipation,
however. By noticing that they are indeed similar to recursion, similar practices
apply: typically, stopping on some special cases, like in well-founded recursion
(think of the factorial function, for instance, which stops recursion for n = 0).
Indeed, like for recursion, the particular way regression is stopped depends on
each particular case (see for instance [53], Appendix C, for the CLOS MOP,
or [15], for the OpenC++ MOP).
generated code, the object model of the language is completely implicit and can
not be accessed7 . Therefore, as discussed in [42], adding reflection to a compiled
language entails maintaining the metalevel information beyond the compilation
process and also transforming the generated code with the appropriate links to
the metalevel information that controls its behavior. Concretely, this widely-
used technique consists in transforming code to introduce so-called hooks to the
metalevel, also known as metalevel interceptions (MLIs) [128].
Typically, hooks are pieces of code in charge of the reification process: inserted
in the code, they trigger a shift to the metalevel when reached by the execution
flow. Therefore, even if the fact that some metalevel behavior should occur is
statically determined (i.e. anticipated), the precise metaobjects implementing
that behavior can still be accessed and changed dynamically, thus supporting
dynamic adaptation of behavior. Obviously, this technique involves a significant
execution overhead if used at each and every place in the code, since the program
must evaluate both the hooks (i.e. code that builds a representation of the inter-
cepted piece of program) and the metalevel code. This is where considerations
related to partial reflection come into play, discussed in depth in Section 4.4.
Note furthermore that we hereby considered the implementation of dynamic
behavioral reflection, provided by so-called runtime MOPs: in fact, the obser-
vation that some metacomputation need not happen at runtime led to the in-
troduction of compile-time MOPs and other related techniques, as discussed in
Section 4.3.
Seen in this light, reflection in programming languages naturally fits in the trend
of ever late binding times, by postponing the binding of almost all elements of
programs and languages to the runtime.
The actual trade-off between functionality and cost in reflective architectures
is further clarified by introducing the notion of binding modes [23]. While the
binding time describes when an association occurs, the binding mode describes
the permanency of the binding. This mode may be either static, if it cannot be
undone, or dynamic, if it may be undone and redone. Systems can therefore be
characterized according to their position in the range going from static binding
at compile time, to dynamic binding at runtime. The hook insertion technique
presented above imposes binding to be done at compile time (or load time) and
may support both static and dynamic binding. Conversely, the interpreter-based
approaches support dynamic binding at runtime (and are hence better suited for
unanticipated software adaptation). A characterization of many systems in light
of this range can be found in [84].
The Java case The Java programming language first started without any re-
flective mechanisms, and has been successively updated, until JDK version 1.3.
The standard Java reflection API [102] mainly supports structural introspec-
tion. Indeed, the ability to obtain metaobjects representing classes, methods,
fields and constructor is restricted to introspection: it is not possible to modify
a given class through such an API. It is, however, possible to instantiate a class
through its representation (an instance of class Class) or to invoke a method
(through an instance of class Method). This limited support for reflection called
for many proposals of reflective extensions to appear. Still, the standard re-
flection API is really useful and widely used, for instance for serialization [98],
remote method invocation [99] and component architectures [103]. Consequently,
its implementation has been aggressively optimized since its first versions. It is
interesting to note that it is also useful for implementing reflective extensions,
since it provides basic (and necessary) features for providing behavioral reflec-
tion. Structural intercession, on the other hand, is not supported in Java, except
in a limited manner when the virtual machine is running in debug mode.
The fact that so much information is present in the Java bytecode motivated
many reflective extensions to be based on bytecode transformation (e.g. Dalang [117],
Kava [119], Jinline [105], Javassist [18, 19]). Transforming bytecode presents sev-
eral advantages over source code transformation, as done by OpenJava [107] and
Reflective Java [123] for instance. First, the source code is not always available,
in particular when considering binary COTS (commodity off-the-shelf) compo-
nents or distributed systems, and second, since Java supports dynamic class
loading, this makes it possible to transform classes lazily as they are loaded. An
exhaustive discussion of Java bytecode transformation approaches can be found
in [105].
Apart from approaches based on source code transformation or bytecode
transformation, some approaches are based on a modified or extended virtual ma-
chine. We mentioned the case of Iguana/J in the previous section. Guaraná [77]
23
and MetaXa [56, 40] are other examples. Some approaches also use the fact that
Java features Just-In-Time (JIT) compilation to operate at this level (e.g. [69,
84]). Finally, it is also possible to use the debugging interface of Java [101],
although this interface is only available when the virtual machine is run in a
(costly) debug mode.
Among code transformation approaches providing runtime behavioral reflec-
tion, it is interesting to note that some approaches are based on the use of
interception objects rather than direct code transformation: this is the case of
Dalang, the MOP of ProActive [12], and the standard dynamic proxies intro-
duced with the JDK 1.3 [100]. The major inconvenient of this approach is to
introduce two objects (the interceptor and the original object) when conceptu-
ally there is only one: this gives rise to the famous “self problem” first discussed
in [61]. Other disadvantages of this approach are discussed in [118], where Welch
and Stroud motivate the evolution of Dalang, based on interceptor objects, to
Kava, based on bytecode rewriting.
first used to build the initial environment, and similarly, the final environment
is transformed to an output result (e.g. by printing). Therefore, we assimilate
data and environments.
The execution equation of a standard program is:
P rog is the set of program texts (programs for short), and Env is the set of
environments (data for short). The result of ξ!P " is thus a function that takes
D (initial environment) as input to produce the result (final environment).
Now, let L be a metaprogram that also includes the program of the inter-
preter, P be a base-level program, and D be the base-level data given to P .
Then the execution of a reflective program is described as follows:
The result of ξ!L" is now a function that takes both P and D as input. In other
words, the metaprogram L is executed to interpret the base-level program P
with the data D.
Currying by itself does not improve performance, and actually involves chang-
ing the MOP. But this technique allows the protocol implementor to cache the
intermediate result ξ!L"(P ) and reuse it later. This way, less metacomputation
is executed at runtime. Note that ξ!L"(P ) may even be computed in advance at
load or compile time.
The currying technique has its disadvantages: it requires transforming a pro-
tocol, hence making it difficult to use, and requires the language to provide
efficient lambda functions (so that applying cached functions actually represents
a gain).
Several attempts have been made to apply this technique to “compile away
the metalevel” as much as possible. For instance, [3] propose an approximation
of the reflective tower of metacircular interpreters that relies on duplication and
sharing of environments among interpreters, which is then optimized through
partial evaluation. In [8], partial evaluation techniques are applied to eliminate
the use of the reflection API of Java as much as possible, resulting in notable
improvements in the execution of the serialization framework, for instance.
The basic idea is to partially evaluate the metalevel program with respect to
the base program. The execution model of the partial evaluator is:
Note that ξ!PE "(L, P ) is equivalent to ξ!L"(P ) in the currying technique (ex-
pression (C)), except that it is a program text (to which ξ must be applied
in order to be executed), not a (directly executable) function. Apart from the
benefit of not requiring lambdas, this approach has the benefit of not requiring
to change the protocol. However, this technique is extremely difficult to imple-
ment, as acknowledged by all researchers in this field. For instance, Asai et al. are
unable to reach fully automatic partial evaluation in their model. Much work
remains to be done for partial evaluation to become widely applicable.
compilers in modern virtual machines, like for Java, actually confirms this fact,
all the more as their techniques are ever improving. Logically, attempts have been
made to build dynamic compile-time MOPs [69], operating as JIT compilers.
Finally, it is interesting to notice that compile-time MOPs can be used to
implement runtime MOPs: hook introduction can indeed be viewed as a static
metaprogramming technique. Chiba has shown how Javassist can be used to
quickly implement a simple runtime MOP [18]. In fact, compile-time MOPs
are advanced macro processing systems, which have the particularity that the
data structures used for processing are metaobjects, rather than abstract syntax
trees. Chiba has highlighted that this very difference makes compile-time MOPs
particularly well-suited to easily implement a large range of transformations of
object-oriented programs [17].
The idea of partial reflection was first motivated in the 1990 OOPSLA/ECOOP
workshop on Reflection and Metalevel Architectures in Object-Oriented Pro-
gramming [45]. First of all, Brian Smith asserted that there exists a continuous
spectrum of causal connection, between a base level and a metalevel. One end of
the spectrum represents traditional, non-reflective, systems, while at the other
end lie systems where the causal connection is full, like in 3-Lisp. In between
are partial connections, and Smith argued that this is where most real world
problems lie, and that research efforts should focus on this middle range. During
this workshop, the inefficiency of reflection was discussed. Reflection was said to
be inefficient because, as opposed to compilation, which consists in embedding a
set of assumptions, reflection retracts some of these assumptions. Having such
retractions everywhere, to achieve full reflection, is the cause for inefficiency.
Therefore the idea that careful consideration must be taken when choosing what
needs to be reflected upon was suggested: this is partial reflection. At that time,
nothing was indeed said about what it means for a system to be partially reflec-
tive, or what means should be provided to specify the partiality of reflection.
In this section, we first propose a formal definition of the execution model ad-
vocated by partial reflection. Then we discuss approaches to selective reification
(Section 4.4). The issue of how selectivity is defined is the subject of Section 4.4,
while Section 4.4 examines the problem of specifying the actual protocol between
base objects and metaobjects, including the shape of reifications.
The left expression of (PR) describes the part of the program that is directly
executed and is therefore the same as (NR). The right expression describes the
reflected part of the program, and is therefore similar to (R). This double ex-
pression illustrates the fact that implementation techniques presented in the
previous section are not incompatible with partial reflection, since they can be
applied to make ξ!L"(Pr , D) more efficient. The difference in perspective is also
highlighted, since partial reflection allows an hybrid execution model, made up
of two simultaneous execution expressions.
Approaches to partial reflection can therefore be discriminated based on the
possibilities they offer to specify ρ, as well as on the permanency of ρ (see
Section 4.2). Dynamic approaches that support dynamic binding at runtime will
typically make it possible to highly control the degree of reflectivity of a partially-
reflective application during execution. Conversely, fully static approaches will
provide means to fix ρ once and for all before execution.
Dalang [117], Reflective Java [123], the ProActive MOP [12], MetaXa [56, 40] and
Guaraná [77] (all in the context of Java), only make it possible to select which
classes are made reflective. However this selection does not mean that a class is
reified as such, but that all method invocations on (instances of) this class will
be reified: the set of execution points is implicit, determined at a coarse-grained
level, the class.
Actual MOP definition With actual MOP definition, we refer to the definition
of the actual interface of metaobjects used by an implicit MOP. For instance,
an Iguana [42] metaobject controlling the invocation of methods has a method
named invoke that takes as parameters an object, a method pointer, and actual
invocation arguments packed in an array. Iguana/J [84] adopts a similar interface
too, except that the method name is execute, the order of parameters is not the
same, and the method pointer is rather a Method object, as provided by Java.
In Kava [119], the metaobject controlling field read accesses must have both a
beforePutField and an afterPutField method that take as parameters the
name of the accessed field and the field value.
29
All reflective systems therefore provide fixed MOPs: as flexible as they may
be, they impose the actual interfaces of metaobjects. Furthermore, since they
do not allow the precise selection of reified information, they all adopt a gen-
eral approach, reifying all the information describing the intercepted operation
(although the shape may change, between plain parameters, arrays, or object
wrappers). This is unfortunate when a metaobject actually does not need some
part of this information (or worse, if it does not need any information at all),
because reification has a cost, since it may imply repetitively constructing arrays
or instantiating some wrapper classes. Furthermore, these MOPs only reify the
information that describes the intercepted operation as implicitly conceived by
their designers.
Consider the case of Reflective Java [123]: it interestingly introduces the
notion of method categories, as a way to distinguish between reified method
invocations. In their script language, users can for instance specify that get
methods are of one category, while put methods are of another category. This
kind of classification is obviously interesting for better metalevel engineering.
However the way that it is done in Reflective Java is quite questionable: the
category information is passed at runtime to metaobjects as an extra parame-
ter, which is typically tested via a switch statement in order to determine the
corresponding metabehavior –a not-so-nice object-oriented programming prac-
tice indeed. Therefore, although Reflective Java already represents a progress
over other approaches that do not support such classification within a bunch of
reified operations, it further highlights the limitation of fixed MOPs. In the case
of categories, one would like to be able to have different metaobject methods
invoked depending on the category of the reified method invocation.
High flexibility in specifying the MOP –in its most essential meaning of
information bridge between base objects and metaobjects– has therefore not
been addressed by runtime reflective systems. A possible reason for this, apart
from the fact that it may not have been identified as a key issue by runtime MOP
designers, may reside in the difficulty of making this decision accessible to users.
If we consider compile-time and load-time MOPs, only Javassist version 2 and
above [19] actually makes it possible to extract whatever piece of information
from a base-level program in a convenient manner.
5 Open Implementations
Since the beginning of this chapter, reflection has always been considered in the
context of programming languages. Even though we have discussed the notion
of reflective applications that exhibit specifically-tailored reflective capabilities,
as motivated by [30], the focus has always been to reify elements of programs
related to their structure (as defined by the language) or their execution seman-
tics. This section discusses a more general notion of reflection, which led to the
introduction of open implementations. Section 5.1 is an introduction to the con-
cept, Section 5.2 explains why open implementations are important, and finally,
30
Section 5.3 surveys techniques and approaches for providing open implementa-
tions.
Two observations can help understand the relation between computational re-
flection and implementational reflection [82]. On the one hand, a language in-
terpreter is itself the implementation of a language: this suggests that compu-
tational reflection is a special case of implementational reflection. On the other
hand, the interface of any system can be seen as a language8 , and the implemen-
tation of the system as an interpreter for that language: this now suggests that
implementational reflection is a special case of computational reflection. Rao
logically suggests that computational reflection and implementational reflection
are just different characterization of the same essential capability.
As with computational reflection, basic access to implementational aspects
of a system does not make it an open implementation as such. The difference is
similar to that made between reflective facilities and fully reflective architectures
(Section 1.4). Recall that a language with a reflective architecture allows much
more open-ended access to the implementation of a language [64], in particu-
lar by allowing users to write code that is called by the language interpreter.
Therefore the concept of a reflective architecture can be reformulated in terms
of the implementation of a system, and leads to the concept of open implemen-
tation [82]:
Open implementation
A system with an open implementation provides (at least) two
linked interfaces to its clients: a base-level interface to the func-
tionality of the system similar to the interface of other such
systems, and a metalevel interface that reveals some aspects of
how the base-level interface is implemented.
8
This statement by Rao, also mentioned elsewhere [45], might be a slight overstate-
ment indeed: the interface of a system may be more precisely viewed as a vocabulary
rather than as a language as such, although there is a language behind the correct
use of this vocabulary.
31
metalevel
interface
One of the roles of the metalevel interface is to specify points at which users
can add code that implements some base-level behavior with different semantics
and/or performance (Fig. 4). Rao notices that the causal connection require-
ment of reflection is straightforwardly met since metalevel code actually directly
implements aspects of the base level. Indeed, reflective systems providing meta-
object protocols are open implementations of interpreters (e.g. [53, 31]), and
systems providing compile-time MOPs are open implementations of compilers
(e.g. [58, 15]).
Patrick Steyaert has proposed a very interesting account of reflection [91],
which is entirely based on open implementations rather than on a tower of
metacircular interpreters. This work is focused on open implementations for pro-
gramming languages, proposing a framework for object-based languages. In [26],
this approach is used to construct the reflective tower based on open implemen-
tations, bringing a cleaner understanding of reflection. In his PhD dissertation,
Steyaert actually uses open implementations as a criteria to differentiate a lan-
guage with reflective facilities from a language with a reflective architecture: the
former only requires implementational access to the metasystem, while the latter
derives from access to the metalevel interface of an open-implemented metasys-
tem. The open implementation of a programming language is implemented in
one language, called the implementation language, and actually implements a
set of languages (depending on the metalevel interface), called the engendered
languages. He further argues that not every open implementation is suitable as
the basis for a reflective architecture, introducing the notion of open implemen-
tations with reflective potential. Such potential consists in that “all first class
values (primitive values, functions, objects, etc.) can freely travel between im-
plementation language and engendered language, and that both languages can
transparently use each others first class values”. This property is known as lin-
guistic symbiosis [46].
The idea of metacircularity, which refers to the fact that an interpreter is
written in the same language that it interprets, does not really fit well in the
realm of open implementations. Rao discusses this issue in the context of Sil-
ica, a window system: although the metalevel of Silica is written in the same
32
language used to implement its base level, it is indeed not written in the base-
level “language” that it provides. Actually, it does not even make sense to write
a window system in the “window system language” that it implements. This
observation seems to apply as well to all systems that are not programming
language interpreters.
it possible to support a “CLOS region”, rather than a single “CLOS point” [53],
hence solving the dilemma they were facing.
Open System
System
Fig. 5. Contrasting traditional system design (left) and open implementation design
(right). (Adapted from [Kiczales et al., 1993].)
A system designer typically considers a range of use cases and features the
system should support elegantly. The system is designed accordingly. This pro-
cess will generally be iterative and ad hoc, but the point is that the designer is
working with two different levels of design at the same time: the level of design-
ing particular use cases in terms of a given system, and the level of designing the
system itself to support the lower-level design processes. This is illustrated on
the left part of Fig. 5. Open implementation design is similar, with the addition
of yet one more level of design process. In this case, the designer is not thinking
about a single system that can be used to handle the use cases, but rather a
whole range of systems, that can support an even wider set of use cases. This is
illustrated on the right part of Fig. 5.
Therefore, the first question that pops up when designing an open implemen-
tation is what range of implementations users should be able to specify [58]. As
35
declarative style: the interface makes it possible for the user to describe, in a
simple declarative language, the expected usage of the module.
strategy style: the user is given the possibility to choose the appropriate strat-
egy in a fixed list of available strategies.
layered style: the user may, in addition to selecting one of the built-in strate-
gies, provide a new strategy.
It has to be noted that all these styles are optional, in the sense that the user
is left with the possibility of not declaring, selecting or providing anything, and
get a default implementation strategy.
[54] then discusses the associated tradeoffs and types of appropriate situations
of these styles. The declarative style has the advantage of not constraining the
implementation, but it makes it difficult for the client to know how the provided
information influences the module strategy. This approach is most appropriate
when it is easy to choose an effective implementation strategy if the client behav-
ior is known. Conversely, with the strategy style, the client precisely selects the
strategy to use. However, he might choose badly. This style is appropriate when
36
– feature locality refers to the fact that an open implementation should provide
access to individual features of the base system;
– textual locality means that convenient means should be provided for users
to indicate what behavior of the base system they would like to be different;
– object locality is the possibility to affect the implementation on a per-object
basis;
– strategy locality refers to the possibility to affect individual parts of an im-
plementation strategy;
– implementation locality points to the fact that a simple customization should
be simple to implement: good default implementation must be provided,
along with means for incremental deviation from that default.
“[...] very often, the concepts that are most natural to use at the meta-
level cross-cut those provided at the base level.”
“[...] We are, in essence, trying to find a way to provide two effective
views of a system through cross-cutting localities.”
“[...] The structure of complex systems is such that it is natural for people
to make this jump from one locality to another, and we have to find a
way to support that.”
[50]
At that time, only the problem was formulated. Apart from recognizing that
the dual-interface framework might therefore very well evolve to a multi-interface
framework, no sketch of a solution was given to this locality issue. A few years
later, and after a bunch of concrete case studies, the first paper presenting
Aspect-Oriented Programming [55] was published at ECOOP.
References
[72] J. McAffer. Meta-level programming with CodA. In Olthoff [78], pages 190–214.
[73] J. McAffer. Engineering the meta-level. In Kiczales [51], pages 39–61.
[74] N. Meyrowitz, editor. Proceedings of the 2nd International Conference on Object-
Oriented Programming Systems, Languages and Applications (OOPSLA 87), Or-
lando, Florida, USA, Oct. 1987. ACM Press. ACM SIGPLAN Notices, 22(12).
[75] H. Okamura and Y. Ishikawa. Object location control using meta-level program-
ming. In M. Tokoro and R. Pareschi, editors, Proceedings of the 8th European
Conference on Object-Oriented Programming (ECOOP 94), volume 821 of Lec-
ture Notes in Computer Science, pages 299–319. Springer-Verlag, July 1994.
[76] H. Okamura, Y. Ishikawa, and M. Tokoro. AL-1/D: A distributed programming
system with multi-model reflection framework. In Proceedings of the Interna-
tional Workshop on Reflection and Meta-level Architectures, pages 36–47, Tokyo,
Japan, Nov. 1992.
[77] A. Oliva and L. E. Buzato. The design and implementation of Guaraná. In
Proceedings of the 5th USENIX Conference on Object-Oriented Technologies &
Systems (COOTS 99), pages 203–216, San Diego, CA, USA, May 1999.
[78] W. G. Olthoff, editor. Proceedings of the 9th European Conference on Object-
Oriented Programming (ECOOP 95), volume 952 of Lecture Notes in Computer
Science, Åarhus, Denmark, Aug. 1995. Springer-Verlag.
[79] Proceedings of the 4th International Conference on Object-Oriented Programming
Systems, Languages and Applications (OOPSLA 89), New Orleans, Louisiana,
USA, Oct. 1989. ACM Press. ACM SIGPLAN Notices, 24(10).
[80] A. Paepcke, editor. Object-Oriented Programming: The CLOS Perspective. MIT
Press, 1993.
[81] D. Parnas. On the criteria for decomposing systems into modules. Communica-
tions of the ACM, 15(12):1053–1058, Dec. 1972.
[82] R. Rao. Implementational reflection in Silica. In America [1], pages 251–266.
[83] B. Redmond and V. Cahill. Iguana/J: Towards a dynamic and efficient reflective
architecture for Java. In ECOOP 2000 Workshop on Reflection and Metalevel
Architectures, June 2000.
[84] B. Redmond and V. Cahill. Supporting unanticipated dynamic adaptation of
application behavior. In B. Magnusson, editor, Proceedings of the 16th European
Conference on Object-Oriented Programming (ECOOP 2002), number 2374 in
Lecture Notes in Computer Science, pages 205–230, Málaga, Spain, June 2002.
Springer-Verlag.
[85] F. Rivard. Smalltalk: a reflective language. In Kiczales [51], pages 21–38.
[86] M. Ségura-Devillechaise, J.-M. Menaud, G. Muller, and J. Lawall. Web cache
prefetching as an aspect: Towards a dynamic-weaving based solution. pages
110–119.
[87] M. Shapiro, P. Gautron, and L. Mosseri. Persistence and migration for C++
objects. In S. Cook, editor, Proceedings of the 3rd European Conference on
Object-Oriented Programming (ECOOP 89), British Computer Society Work-
shop Series, pages 191–204, Nottingham (GB), July 1989. The British Computer
Society, Cambridge University Society.
[88] S. K. Shrivastava, G. N. Nixon, and G. D. Parrington. An overview of the Arjuna
distributed programming system. IEEE Software, 8(1):66–73, Jan. 1991.
[89] B. C. Smith. Reflection and semantics in a procedural language. Technical
Report 272, MIT Laboratory of Computer Science, 1982.
[90] B. C. Smith. Reflection and semantics in Lisp. In Proceedings of the 14th Annual
ACM Symposium on Principles of Programming Languages (POPL), pages 23–
35, Jan. 1984.
43