Squeak Makes a Good Python Debugger

Bringing Other Programming Languages into Smalltalk’s Tools

Fabio Niephaus Tim Felgentreff

Hasso Plattner Institute, University of Potsdam Hasso Plattner Institute, University of Potsdam
Potsdam, Germany Potsdam, Germany
[email protected] [email protected]

Tobias Pape Robert Hirschfeld

Hasso Plattner Institute, University of Potsdam Hasso Plattner Institute, University of Potsdam
Potsdam, Germany Potsdam, Germany
[email protected] [email protected]

ABSTRACT 1 BACKGROUND AND MOTIVATION

Interactive debuggers are indispensable in many software devel- Programming environments play a central role in software develop-
opment scenarios. However, they are often hard to extend and ment. Developers have come to expect language-specific amenities
more importantly, their capabilities are limited to an application from their development environments, ranging from syntax high-
programming interface (api) provided by the runtime executing lighting, refactoring support, context-specific auto-completion, to
the corresponding programming language. interactive debugging and live code reloading. Most environments
We propose an approach that allows to use the live tools of a offer at least some of these features. Furthermore, there is a re-
Smalltalk environment for other programming languages. The ap- newed push in current research to allow developers to use live,
proach is based on interpreter-level composition, ultimately making run-time data that can be explored and manipulated to understand
a full-fledged integrated development environment (ide) part of and extend software systems [6, 27], through edit-and-continue
the language execution process. This allows to directly control in- style debugging and coding against live data. However, efforts to-
terpreters of foreign languages from Smalltalk. It also enables tool wards that goal are often tied to specific domains or programming
reuse and provides the ability to rapidly build new tools. languages (or both).
As an example, we demonstrate how we have combined Squeak/ ide systems such as Eclipse or NetBeans try to provide a frame-
Smalltalk and PyPy’s Python implementation. We then reused work to build development tools for a variety of programming lan-
Squeak’s debugger, so that it enables edit-and-continue style de- guages. Their architecture is set up to minimize the dependency
bugging of Python applications — something that is currently not on any particular language feature, to support a range of different
supported by Python’s PDB or any Python ide. languages. This has the advantage that code can be reused between
tools for different languages, at the cost of not integrating deeply
CCS CONCEPTS with any language.
• Software and its engineering → Integrated and visual devel- Most combinations of ides and runtimes share a common de-
opment environments; Software testing and debugging; Virtual ma- bugging architecture (see Figure 1a). When the program under de-
chines; velopment is running, the tools connect to it through some runtime
api to offer inspection and debugging depending on the underlying
KEYWORDS capabilities of the runtime. For example, the Java Virtual Machine
(jvm) Tools Interface (jvmti) offers read-only 1 inspection capabil-
Smalltalk, Python, debuggers, integrated environments, virtual ma-
ities for a running Java program. It can stop and inspect, force early
chines
returns with a particular return value, or set local variable values
ACM Reference format: in the top frame. The HotSpot virtual machine (vm) also allows
Fabio Niephaus, Tim Felgentreff, Tobias Pape, and Robert Hirschfeld. 2017. restarting a frame that is already running and hot code swapping for
Squeak Makes a Good Python Debugger. In Proceedings of Programming frames that are not active on the stack. Microsoft’s Common Lan-
Experience Workshop, Brussels, Belgium, April 2017 (PX’17), 8 pages. guage Runtime (clr) has a dedicated Interface (ICorDebug) that
DOI: 10.475/123_4 allows edit-and-continue active stack frames,2 and ICorDebugEval
allows the ide to inject and execute arbitrary code.3
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
An ide constructed in this way can only reflect on the execu-
for profit or commercial advantage and that copies bear this notice and the full citation tion state through the runtime api. To inspect and modify objects,
on the first page. Copyrights for components of this work owned by others than the representations for them are reproduced in the context of the ide.
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specific permission Objects and properties that cannot be transferred through the run-
and/or a fee. Request permissions from [email protected]. time api, cannot be inspected or modified.
PX’17, Brussels, Belgium
1 https://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html
© 2017 Copyright held by the owner/author(s). Publication rights licensed to ACM.
2 https://msdn.microsoft.com/en-us/library/ms231220.aspx
123-4567-24-567/08/06. . . $15.00
DOI: 10.475/123_4 3 https://blogs.msdn.microsoft.com/jmstall/2006/01/04/partition-of-icordebug/

1
PX’17, April 2017, Brussels, Belgium Fabio Niephaus, Tim Felgentreff, Tobias Pape, and Robert Hirschfeld

(a) Eclipse is separate from the language process. (b) Squeak/Smalltalk is part of the language process.

Figure 1: Architectural comparison of Eclipse and Squeak/Smalltalk.

On the other hand, live programming systems, such as Squeak/ We propose the idea to use a Smalltalk environment as an ide
Smalltalk, are well-suited to provide full access to the runtime state for other languages and perform the integration on interpreter-
and provide developers with means to adapt their tools at devel- level. This way, our architecture avoids the n-to-m problems of
opment time according to their needs. In such systems, the ide is interfacing multiple languages by mapping all languages into the
entirely contained in the running process, and thus has full reflec- language of the live environment. This can be achieved by com-
tive access to the state of the system and can directly inspect and posing a Smalltalk interpreter with another language’s interpreter.
manipulate all objects therein, as well as provide edit-and-continue Interpreter composition is especially convenient when both lan-
style debugging (see Figure 1b). guages are implemented in the same interpreter framework, such
However, due to the deep integration with the runtime, tool as RPython [1], as demonstrated in Section 3.
implementations in live, always-on programming environments However, this kind of composition states the problem of how
like Squeak are very language-specific, and even the means to write to run two or more interpreters at the same time. Smalltalk has a
tools do not carry over very well. Thus, even those tools that have concept of processes which can be scheduled dynamically [14]. As
been reproduced for other programming systems have been written we demonstrate later, it is possible to make the execution of a non-
from ground up. Smalltalk interpreter part of a Smalltalk-level process. This way,
We propose an architecture for constructing a multi-language the Smalltalk scheduler decides when to continue with the other
runtime that attempts to combine the benefits of both approaches: interpreter loop, which ensures that the developer can interact with
a common code-base for tool developers and live, immediate access the Smalltalk environment at the same time.
to the running application for inspecting and manipulating state. In such a setup, it is now possible to control interpreters of a for-
Our contributions are as follows: eign language with tools written in Smalltalk. For this to work, the
vm needs to be extended in such a way that it exposes a number of
• An architecture to compose multiple languages within the
interpreter-controlling primitives which can then be called accord-
same live programming environment with reflective capa-
ingly from within a Smalltalk environment. In order to be able to
bilities for full execution control from within the runtime.
execute a program written in another language, primitives for ex-
• An implementation of said architecture using PyPy [21]
ample need to exist which provide an entry point as well as an end
and RSqueak/VM [5, 12].
point for the corresponding interpreter loop. Similarly, primitives
• An implementation of a debugger that works for both,
for restarting and stepping in call frames need to be implemented
Python and Squeak/Smalltalk.
to enable debugging support.
In Section 2 we introduce our approach. Then, we demonstrate Squeak already contains various tools originally demonstrated in
how this approach can be implemented with an example in Sec- Smalltalk-80, including an interactive debugger [13]. Since Squeak’s
tion 3. Afterwards, we discuss advantages and disadvantages of tools are designed to work as part of a framework, it is straight-
our approach in Section 4. Related work is then mentioned in Sec- forward to adopt them, so they can be used for other languages as
tion 5. Finally in Section 6, we conclude the paper and describe well. In addition, new tools, such as application-specific debugging
future work. tools, can quickly be built in Smalltalk [25].

2 APPROACH 3 IMPLEMENTATION
Various interpreted programming languages provide only very lim- We have applied our approach in Section 2, so that we can use
ited debugging support which can be a burden for developers when Squeak/Smalltalk as an ide for Python. The resulting architecture
trying to understand a misbehavior that occurs in their application. of our implementation is depicted in Figure 2. On vm-level, there
On the other hand, Smalltalk is not only a programming language, is an interpreter loop for each language as well as a vm plugin
but also an ide. This means, that the ide is an actual part of the with a set of primitives which allow Smalltalk to interact with the
process running Smalltalk. This gives developers full control over Python bytecode loop. In the Smalltalk image, we have introduced
the running applications and therefore allows for very comprehen- different classes in order to bridge between Python and Smalltalk.
sive debugging tools. In the following, we explain details of this implementation.
2
Squeak Makes a Good Python Debugger PX’17, April 2017, Brussels, Belgium

Figure 2: Architecture of the Smalltalk and Python compo- Figure 3: Mixed-stack Smalltalk and Python execution.
sition.
Squeak frame on the top of the stack. When the Python interpreter
loop switches back to Squeak, the top Python frame 3 creates an
3.1 VM-level Implementation artificial suspended Squeak “resume” frame 4 . When the Squeak
First, we need to build a vm that is capable of executing both, scheduler resumes this artificial frame later, it immediately returns
Smalltalk and Python code. This can be achieved by composing control back to the top Python frame 3 .
a Smalltalk interpreter with a Python interpreter. RSqueak/VM [5] Since interpreters maintain some execution state directly on the
is a RPython-based implementation of Squeak [16], a Smalltalk im- stack, we use Stacklets,4 which are provided by the RPython stan-
plementation derived from Smalltalk-80 and a live programming dard library. Stacklets allow us to implement minimal coroutines
environment. PyPy [21] is a Python implementation and the first for regions of the C stack to switch between the two interpreter
RPython vm. loops. Since this is a feature of the RPython framework and not of
By simply combining the two interpreters, we can create a virtual any specific language implementation, the same principle can also
machine with support for both programming languages. However, be applied to any other interpreter written in RPython.
to use Squeak/Smalltalk as a live development environment for Similar to other Smalltalk vms, RSqueak/VM can be extended
Python, we need to be able to run both interpreters concurrently. with plugins. Not only have we added vm primitives that can be
Smalltalk implements co-operative multitasking through pro- called from within the Smalltalk environment to control the PyPy
cesses [14]. We leverage this and integrate the execution of Python interpreter loop. Since RPython is a subset of Python which com-
bytecodes with a Smalltalk-level process, leaving the decision when piles to C, we were able to build the entire composition in a single
to run the next bytecodes up to the Smalltalk scheduler. This al- PythonPlugin. The code of the plugin for example also patches the
lows us to interact with the Squeak/Smalltalk environment as usual PyPy bytecode loop as well as the class which represents Python
while a Python program is running, as well as to interrupt this pro- frames internally before the actual translation begins. After trans-
cess to inspect it from Smalltalk. lating this new virtual machine, we can open a Squeak/Smalltalk
We create a mixed stack of Python and Smalltalk frames that image and make it aware that the vm also supports the Python
is managed like any other Squeak process (cf. Figure 3). The core programming language.
responsibility of the vm in our approach is to maintain the sender-
relationship 2 – 1 and 4 – 3 across interpreter loops. 3.2 Bridging between Squeak and PyPy
We switch from Squeak to Python by executing a primitive 1 We start by providing a class PythonObject. This class is special,
to enter Python code. This creates a Python frame 2 that executes because the vm will automatically expose objects of the Python ob-
in the Python interpreter loop. When this frame returns, the vm ject space as instances of this new class. In addition, all primitives of
will return from the primitive 1 and transfer control back to its the PythonPlugin are able to automatically convert primitive data
sender. types between Python and Smalltalk. Python strings are therefore
While the Python interpreter is running, we maintain a counter for example converted to Smalltalk ByteStrings and vice-versa.
of how many Python bytecodes have been executed to decide when Now that the vm can inject any kind of Python object into an im-
to transfer control back to Squeak in order to give other processes age, we need be able to interact with these objects. For this reason,
a chance to run (e.g., so that the UI process can handle user in- we have added appropriate primitives to the virtual machine. The
put). Since Python frames are not visible to Squeak and thus the most important primitive is the pythonEval primitive which can
Squeak scheduler cannot switch directly back to Python frames at
a later time, we create an entry point for the scheduler by putting a 4A lightweight threading mechanism in the spirit of tasklets.
3
PX’17, April 2017, Brussels, Belgium Fabio Niephaus, Tim Felgentreff, Tobias Pape, and Robert Hirschfeld

evaluate Python expressions and execute Python code. Similar to

Python’s compile builtin, it expects Python source code, a corre-
sponding filename, as well as a string which describes the mode
and can either be “eval” or “exec”. When a Python expression is
evaluated, the primitive returns the result. Otherwise, it will exe-
cute the Python code and return nil, or fail if for example a syntax
error occurred. To clearly separate methods that call primitives
from methods that can be called on PythonObjects, we add a new
class called Python. To its metaclass, we add for example a method
named primEval:filename:mode: which can be used to call the
pythonEval primitive. As an example, we could call the following
to get a new Python object instance:
Python
Figure 4: Proxying Python frames for the Smalltalk Debug-
primEval: 'object()'
ger.
filename: '<string>'
mode: 'eval'
We have also modified the method lookup for these kinds of current execution context in Python. Then, we iterate over all par-
wrapped Python objects. First, the lookup is done on the Python ent frames and generate objects from a new class PythonContext
side. If it fails, the lookup continues in the PythonObject class which is a subclass of MethodContext. These objects are basi-
inside the Smalltalk environment. This allows us to implement cally only used to hold a reference to the corresponding Python
and override methods that are necessary for the tool support in frame. This connection between the two stacks is similar to the
Squeak/Smalltalk while also preserving the original Python behav- implementation of MethodContexts in the Cog vm [18], where
ior. As an example, the following method is used to facilitate that Smalltalk context objects are connected to their C stack frame
PythonObjects can understand the message class: counter part. Each PythonContext is linked to its parent simi-
PythonObject>>class lar to how MethodContexts reference their sender. The sender of
the bottommost PythonContext is then set to thisContext and
^ self __class__
finally the debugger is opened on the topmost PythonContext.
Sending __class__ to self will retrieve the value of the corre- By overriding the method the debugger calls to retrieve Smalltalk
sponding attribute of the Python object. If a selector name resolves code, we can make the debugger display Python code whenever it
to a Python callable instead of resolving to an attribute, it is called encounters a PythonContext. In this case, the method uses the ref-
directly. This allows to call Python methods and to pass arguments erenced Python frame to look up the corresponding Python source
in. For example, sending append: 'some text' to a Python list ob- file. This is possible, because the Python frame contains the cur-
ject will call append('some text') on the actual object. Multiple rent line number (frame.f_lineno), the filename of the Python
arguments can be passed using Squeak/Smalltalk’s keyword syntax. code (frame.f_code.co_filename), as well as the first line in the
For example, index: 'a'startingAt: 2 will call index('a', 2) code (frame.f_code.co_firstlineno). With this information, a
on the Python object — the message name is only considered up to helper method can read the right file and parse out the correspond-
the first colon, so we can make up readable keywords. ing Python code which is then displayed in the code editor of the
With all of the above, we can start adopting and building tools debugger. To further improve usability, we automatically adjust
in Squeak/Smalltalk which are able to control and modify a Python indentation, because in Python, indentation is part of the syntax.
program at runtime. Moreover, we have overridden the method in PythonContext
that normally returns a list of temporary Smalltalk variables, so that
3.3 Adapting Squeak’s Debugger it returns “self pyFrame f_locals keys”, namely all keys that
In order to implement a Smalltalk-style debugger for Python, we are part of the frame’s local namespace. This causes the debugger
first start with a subclass of Squeak’s Debugger. to display the Python locals in the bottom right list.
We want this new PythonDebugger to display Python frames In total, only six method overrides in PythonDebugger were
on top of the Smalltalk contexts that have triggered the execution necessary to provide basic support for Python.
of Python code. The Smalltalk debugger is normally opened on Lastly, we need to instruct the Smalltalk environment to use the
thisContext which is the current method context. The list of new debugger. This can be done by making a new PythonToolSet
all contexts displayed by the debugger is generated by traversing the default, which Squeak/Smalltalk will then use to look up which
MethodContext objects starting with thisContext and following tool to open when an exception occurs.
the reference to its sender. After a user interrupt, the PythonDebugger may look like shown
Since Python code is being executed in a Smalltalk process, we in Figure 5. The Python application running is a HelloWorld “Flask”-
need to override the entry point to debug processes and modify based web server. We can now explore the current control flow
the context on which the debugger is opened. We add Python starting from the DoIt Smalltalk frame which started the applica-
frames on top of thisContext, as can be seen in Figure 4 . We tion, up to the topmost Python frame including the latest line being
have implemented a primitive that returns the top frame of the executed at the time of the user interrupt. The list includes frames
4
Squeak Makes a Good Python Debugger PX’17, April 2017, Brussels, Belgium

User Interrupt
3.4 Instrumenting more Squeak tools
self.finish_request(request, client_address) (line 318 in lib-python/2.7/SocketServer.py)
self.process_request(request, client_address) (line 290 in lib-python/2.7/SocketServer.py)
self._handle_request_noblock() (line 233 in lib-python/2.7/SocketServer.py)
Similar to the PythonDebugger, we have adopted other tools that
HTTPServer.serve_forever(self) (line 511 in werkzeug/serving.py)
srv.serve_forever() (line 673 in werkzeug/serving.py)
come with Squeak/Smalltalk. It only took little effort to adopt
inner() (line 708 in werkzeug/serving.py)
run_simple(host, port, self, **options) (line 841 in flask/app.py)
Squeak’s interactive Inspector and Explorer tools. Only one
app.run() (line 2 in exec_source.py>)
Python class>>resumeFrame
method needed to be overridden to provide a PythonWorkspace
UndefinedObject>>DoIt which supports Smalltalk-style doIts, printIts, inspectIts, and
exploreIts for Python. Moreover, we have basic Python support
Proceed Restart Into Over Through Full Stack Where Tally

def _handle_request_noblock(self):
"""Handle one request, without blocking. in the SystemBrowser which allows us to create Python classes,
I assume that select.select has returned that the socket is and add Python methods. Finally, it now is possible to rapidly build
readable before this function was called, so there should be
no risk of blocking in get_request().
custom tools for Python development, such as a tool that allows
""" to observe the execution of Python bytecodes which would allow
try:
request, client_address = self.get_request() debugging on Python bytecode level. With for example the Vivide
except socket.error:
return framework [25], one could also build data-flow-based applications
if self.verify_request(request, client_address):
try:
which can consist of Python and Smalltalk code.
self.process_request(request, client_address)
except:
self.handle_error(request, client_address)
self.shutdown_request(request) 4 DISCUSSION
else:
self.shutdown_request(request) This work is aimed at providing the best of two previously sepa-
self
all inst vars
<- Select receiver’s field stack top
all temp vars
a <socket._socketobject rate worlds: the framework approach to cross-language ide and
debugger development provided by environments such as Eclipse
object at
superclass self 0x000000010e948cd0>
methodDict request
format client_address or NetBeans, and the live and immediate debugging nature of sys-
instanceVariables
tems such as Squeak/Smalltalk. On top of that, to be considered
Figure 5: Interrupting a Python application. useful, we seek for an approach that impedes performance of the
integrated languages as little as possible.

4.1 Tool Frameworks and Live Development

with the Python expression that started the server, Flask’s main As shown, our Python debugger is implemented as a refinement of
entry point, request handling in Werkzeug (one of Flask’s few de- the default Squeak debugger. Through the inter-language interface
pendencies), as well as the “SocketServer” module which is part (cf. Section 3.2), this debugger can adapt the Python runtime state
of the Python standard library. This immediately gives developers to the expectations of the Squeak tools. The implementation of
a lot of information to understand the running application. For the debugger was straightforward as described in Section 3.3. The
example, Flask is considered to be a lightweight web framework. amount of adaptation necessary on both, the vm and the tools
Developers can now interactively experience what that means, and side, was comparatively low. Hence, we think that other languages
discover that Flask uses a library called “Werkzeug” which in turn can easily provide their own adapters to interface with the Squeak
uses “SocketServer” in order to serve requests and in which files debugger, allowing for a variety similar to that of Eclipse, NetBeans,
and lines the corresponding functions can be found. or other ides.
When Python code is being executed which throws a Python- Due to this easiness, we also wrote initial adapters for Squeak’s
level exception, the virtual machine informs the image accordingly, code browser, the “workspace” (a tool to evaluate short code snip-
so that the image can arrange to open the PythonDebugger. A pets), and provided an adapter method for the PythonObject class
simple example in which a naively-implemented average function to work with Squeak’s object inspector. We are thus able — in a
is called with an empty list is depicted in Figure 6. This imple- rudimentary fashion — to write and evaluate Python code and to
mentation cannot handle the case of an empty list, therefore a interactively explore Python objects in the live system.
ZeroDivision exception is thrown. The PythonDebugger’s title Our experience in writing those tools was every bit as enjoy-
contains the error description and the topmost frame is presented able for us as writing Smalltalk tools is — the interactive environ-
to the developer with the corresponding error line. It is possible ment allowed us to adapt the tools as we went along, implementing
to inspect the iterable parameter in order to realize that its size features in the moment we wanted them. These advantages, in-
is zero. The user can now insert a check to ensure a division by herent to live development environments in the style of Squeak/
zero cannot happen again and then save the code. The method that Smalltalk [25], are thus made available to Python developers, allow-
usually hot-swaps Smalltalk methods after saving is overridden, so ing them to adapt tools at runtime, without having to restart the
that the new version of the Python code is written to disk into its ide or re-execute a running program to iterate over it. Since the
source file first. Then the vm compiles the new code, replaces the tools work on the Python objects directly, there was no need for
Python code object of the selected Python frame with the newly proxy objects besides PythonContext, and these were only neces-
produced one, and resets the frame. The next time the Python inter- sary because Python frames are not exposed in the Python language
preter continues, it will restart the frame and then the misbehavior itself. Notwithstanding, the interaction and library reuse goes be-
is eliminated. This can be done by clicking the “Proceed” button yond tools and also works in the other direction, so Python code
which will let the Smalltalk-level Python process continue. can be used for Smalltalk projects.
5
PX’17, April 2017, Brussels, Belgium Fabio Niephaus, Tim Felgentreff, Tobias Pape, and Robert Hirschfeld

ZeroDivisionError: integer division by zero

def average(iterable): (line 2 in example.py)
average(list()) (line 1 in eval_source.py)
Python class>>openDebuggerWithPythonFrames:
Python class>>resumeFrame
Python class>>eval:filename:mode:
Proceed Restart Into Over Through Full Stack Where Tally

def average(iterable):
return sum(iterable) / len(iterable)

self <- Select receiver’s field thisContext #()

all inst vars stack top
superclass all temp vars
methodDict iterable
format

Figure 6: Debugging a Python exception.

4.2 Performance of Combined Languages 5.1 Debugging and Debuggers

The RPython toolchain makes it simple to combine multiple in- The Smalltalk debugger [13] and debuggers like it have capabili-
terpreters, but there is a performance impact in doing so. There ties such as object- and stack-inspection, edit-and-continue, restart,
are two reasons, one inherent to RPython, the other due to our resume, etc. (see above), and can be extended with new concepts,
approach. for example with test-driven fault navigation [19]. However, more
The first reason is a current limitation of the RPython frame- prevalent debuggers present different sets of capabilities. Gdb [24]
work: a very potent call stack optimization (called virtualizables) and descendant or similar debuggers (llvm, Xcode, etc.) typically
that avoids allocation of stack frame objects can only be applied present a very low-level debugging experience. While providing
to at most one interpreter. In our case, this means that while rich intercession (for example breaking, hardware breakpoints, in-
RSqueak/VM runs at full speed, there is an overhead to each PyPy terrupt handling) or inspection, these are very close to the machine/
method call. Although we have not yet put a focus on performance, processor and not necessarily to the program. Especially debug-
a ballpark measurement of the Richards Benchmark,5 shows that ging languages that are not very C- or assembler-like requires a lot
our implementation performs comparable to CPython. of effort for effective inspection. Edit-and-continue in gdb is only
The second reason is that our approach— for the time being— re- supported in a severely limited fashion under the idea of altering ex-
lies on the cooperative scheduling mechanism of Squeak/Smalltalk ecution [24, chapter 17]. Similarly geared towards the machine, the
processes to concurrently execute code in different interpreter loops. Microsoft Visual Studio Debugger [17], supports a comparable set
Currently, each interpreter maintains counters to decide when to of capabilities, paired with a more user-friendly edit-and-continue
give the other processes a chance to run. At minimum, the user mechanism.
interface (ui) process of Squeak will get a chance to handle user in- On the other hand, debuggers for higher-lever or dynamic lan-
put, but there may also be other processes running concurrently at guages like Python or Ruby provide more direct inspection— often
the same or higher priority that get scheduled instead. This means used in a read-only post-mortem fashion — but also limited stop-
that even when there is no user input and no other concurrent and-resume and seldom edit-and-continue. For JavaScript, certain
computation, there is the overhead of maintaining the counters — tools, such as Chrome DevTools, provide more capable stop-and-
at the moment at each bytecode dispatch in PyPy. This overhead resume and a limited edit-and-continue. However, they are hardly
can be reduced if we switch to counters only for loops and method extendable or scriptable.
calls. As another mitigation strategy, we are considering to allow Ongoing research based on the Truffle framework investigates
convenient deactivation of the ui process and only start it when a how to leverage a language implementation framework to provide
Python error needs to be handled or when the Python process is fast, unified debugging facilities to different language implementa-
interrupted by the user with a signal. This might be desirable for a tions [22, 26].
deployment scenario in any case.
5.2 Interpreter Composition and
5 RELATED WORK Embedded Languages
Related work can be found both, in the domain of debuggers as
The composition of programming languages and— in extension—
well as interpreter composition.
their implementations has been investigated since the late 1960s [8].
5 https://www.cl.cam.ac.uk/~mr10/Bench.html More recent research targets advanced just-in-time (jit) compiler
6
Squeak Makes a Good Python Debugger PX’17, April 2017, Brussels, Belgium

frameworks, such as RPython with Unipycation [2] (composition ACKNOWLEDGMENTS

of Python/PyPy and Prolog/Pyrolog) or Truffle with Sulong (able to We would like to thank Carl Friedrich Bolz and the PyPy team
compose Ruby/JRuby and similar with C/LLVM) [15], among oth- for their help integrating the PyPy interpreter into RSqueak/VM.
ers [3]. A composition of Ruby with Smalltalk that executes mixed We gratefully acknowledge the financial support of HPI’s Research
stacks as Smalltalk processes is also implemented in MagLev, the School 6 and the Hasso Plattner Design Thinking Research Pro-
Ruby implementation on the GemStone/S Smalltalk vm [23]. Our gram.7
composition approach bears most resemblance with Unipycation.
From a different point of view, interpreter composition is similar
to language embedding, however, research that targets the latter REFERENCES
is typically concerned with the language as being written than [1] Davide Ancona, Massimo Ancona, Antonio Cuni, and Nicholas D. Matsakis. 2007.
RPython: A Step Towards Reconciling Dynamically and Statically Typed OO
with the runtime environment as being executed. Examples include Languages. In Proceedings of the 2007 Symposium on Dynamic Languages (DLS
Eco [9], a syntax-directed-style editor for language composition, as ’07). ACM, New York, NY, USA, 53–64. DOI:http://dx.doi.org/10.1145/1297081.
1297091
well as domain-specific language (dsl) tool environments such as [2] Edd Barrett, Carl Friedrich Bolz, and Laurence Tratt. 2013. Unipycation: A Case
Helvetia [20] or language workbenches [10]. Our approach, how- Study in Cross-language Tracing. In Proceedings of the 7th ACM Workshop on
ever, is focused on debugging, both with respect to composition Virtual Machines and Intermediate Languages (VMIL ’13). ACM, New York, NY,
USA, 31–40. DOI:http://dx.doi.org/10.1145/2542142.2542146
and tooling; more extensive tooling and editing capabilities are still [3] Edd Barrett, Carl Friedrich Bolz, and Laurence Tratt. 2015. Approaches to inter-
under investigation. preter composition. Computer Languages, Systems & Structures 44, Part C (2015),
199–217. DOI:http://dx.doi.org/10.1016/j.cl.2015.03.001 arXiv:1409.0757
[4] Spenser Bauman, Carl Friedrich Bolz, Robert Hirschfeld, Vasily Kirilichev, Tobias
Pape, Jeremy G Siek, and Sam Tobin-Hochstadt. 2015. Pycket: A Tracing JIT for
a Functional Language. In Proceedings of the 20th ACM SIGPLAN International
Conference on Functional Programming (ICFP 2015), Vol. 50. ACM, New York, NY,
USA, 22–34. DOI:http://dx.doi.org/10.1145/2784731.2784740
[5] Carl Friedrich Bolz, Adrian Kuhn, Adrian Lienhard, Nicholas D. Matsakis, Oscar
6 CONCLUSIONS AND FUTURE WORK Nierstrasz, Lukas Renggli, Armin Rigo, and Toon Verwaest. 2008. Back to the Fu-
ture in One Week — Implementing a Smalltalk VM in PyPy. In Self-Sustaining Sys-
In this paper, we presented an approach to using a Smalltalk en- tems. Lecture Notes in Computer Science, Vol. 5146. Springer Berlin Heidelberg,
vironment for debugging other programming languages. We ap- Berlin, Heidelberg, 123–139. DOI:http://dx.doi.org/10.1007/978-3-540-89275-5_
plied this approach to the RSqueak/VM and PyPy, so that Squeak/ 7
[6] Gilad Bracha. 2012. Debug mode is the only mode. https://gbracha.blogspot.
Smalltalk can be used for Python debugging and direct interaction com/2012/11/debug-mode-is-only-mode.html. (2012). Talk at the 2012 meeting
between the virtual execution environments. Squeak/Smalltalk can of the IFPI TC2 Working Group on Language Design.
[7] Paul Butterworth, Allen Otis, and Jacob Stein. 1991. The GemStone Object
therefore control the Python interpreter, which enables edit-and- Database Management System. Commun. ACM 34, 10 (Oct. 1991), 64–77. DOI:
continue style debugging of Python programs. We demonstrated http://dx.doi.org/10.1145/125223.125254
that only little effort is required to adapt Squeak’s debugger to al- [8] Thomas E. Cheatham, Jr. 1969. Motivation for Extensible Languages. SIGPLAN
Not. 4, 8 (Aug. 1969), 45–49. DOI:http://dx.doi.org/10.1145/1115858.1115869
ter and control the execution of Python code. Such features are [9] Lukas Diekmann and Laurence Tratt. 2014. Eco: A Language Composition
not available in most other Python implementations, including the Editor. In Software Language Engineering: 7th International Conference, SLE
reference CPython implementation. Moreover, we adapted the in- 2014, Västerås, Sweden, September 15-16, 2014. Proceedings, Benoît Combemale,
David J. Pearce, Olivier Barais, and Jurgen J. Vinju (Eds.). Springer International
spection tools in Squeak/Smalltalk to be able to inspect and explore Publishing, Cham, 82–101. DOI:http://dx.doi.org/10.1007/978-3-319-11245-9_5
Python objects. We can now build new tools for Python in Squeak/ [10] Sebastian Erdweg, Tijs van der Storm, Markus Völter, Meinte Boersma, Remi
Bosman, William R. Cook, Albert Gerritsen, Angelo Hulshout, Steven Kelly, Alex
Smalltalk more rapidly. Loh, Gabriël D. P. Konat, Pedro J. Molina, Martin Palatnik, Risto Pohjonen, Eugen
However, a debugger alone does not make an ide. Our initial Schindler, Klemens Schindler, Riccardo Solmi, Vlad A. Vergu, Eelco Visser, Kevin
results with the debugger are promising and demand expansion to van der Vlist, Guido H. Wachsmuth, and Jimi van der Woning. 2013. The State
of the Art in Language Workbenches. In Software Language Engineering: 6th
other tools as well as languages. We plan to adapt Squeak tools, International Conference, SLE 2013, Indianapolis, IN, USA, October 26-28, 2013.
such as the system browser, to Python in such a way that Squeak Proceedings, Martin Erwig, Richard F. Paige, and Eric Van Wyk (Eds.). Number
can act as an ide for Python applications, but benefiting from the 8225 in Lecture Notes in Computer Science. Springer International Publishing,
Cham, 197–217. DOI:http://dx.doi.org/10.1007/978-3-319-02654-1_11
dynamicity of Squeak’s environment. Moreover, it will be inter- [11] Tim Felgentreff. 2013. Topaz Ruby. http://lanyrd.com/2013/wrocloverb/sccygw/,
esting to see how well much larger Python applications can be https://github.com/topazproject/topaz. (March 2013). Invited Talk at the 2013
edition of Wroclove.rb.
debugged and developed using these tools. It is currently possible [12] Tim Felgentreff, Tobias Pape, Patrick Rein, and Robert Hirschfeld. 2016. How
to affect Python objects from Squeak/Smalltalk but not vice versa, to Build a High-Performance VM for Squeak/Smalltalk in Your Spare Time: An
which would be also useful. For example, a Squeak/Smalltalk image Experience Report of Using the RPython Toolchain. In Proceedings of the 11th
Edition of the International Workshop on Smalltalk Technologies (IWST’16). ACM,
could act as a persistable object space or even database similar to New York, NY, USA, Article 21, 10 pages. DOI:http://dx.doi.org/10.1145/2991041.
GemStone [7]. The changes required on the Python side are how- 2991062
ever still uncertain. To extend our approach to other languages, [13] Adele Goldberg. 1984. Smalltalk-80: The Interactive Programming Environment.
Addison-Wesley Longman, Boston, MA, USA. The Red Book.
we have already started integrating Topaz [11]/Ruby and plan to [14] Adele Goldberg and David Robson. 1983. Smalltalk-80: The Language and Its
integrate other interpreters based on RPython, such as Pyrolog [2]/ Implementation. Addison-Wesley Longman, Boston, MA, USA. The Blue Book.
Prolog or Pycket [4]/Racket.
Lastly, we plan to evaluate our approach more thoroughly, on
the one hand— regarding the programming experience— with user
studies, on the other hand — regarding performance — with mean- 6 https://hpi.de/en/research/research-school.html

ingful benchmarks. 7 https://hpi.de/en/dtrp/

7
PX’17, April 2017, Brussels, Belgium Fabio Niephaus, Tim Felgentreff, Tobias Pape, and Robert Hirschfeld

[15] Matthias Grimmer, Chris Seaton, Thomas Würthinger, and Hanspeter Mössen-
böck. 2015. Dynamically Composing Languages in a Modular Way: Sup-
porting C Extensions for Dynamic Languages. In Proceedings of the 14th In-
ternational Conference on Modularity. ACM, New York, NY, USA, 1–13. DOI:
http://dx.doi.org/10.1145/2724525.2728790
[16] Dan Ingalls, Ted Kaehler, John Maloney, Scott Wallace, and Alan Kay. 1997.
Back to the Future: The Story of Squeak, a Practical Smalltalk Written in Itself.
SIGPLAN Not. 32, 10 (Oct. 1997), 318–326. DOI:http://dx.doi.org/10.1145/263700.
263754
[17] Microsoft. 2017. Debugging in Visual Studio. (Jan. 2017). https://msdn.microsoft.
com/en-us/library/sc65sadd(d=default,l=en-us,v=vs.140).aspx
[18] Eliot Miranda. 2011. The Cog Smalltalk Virtual Machine: Writing a JIT in a High-
level Dynamic Language. In 5th Workshop on Virtual Machines and Intermediate
Languages (VMIL).
[19] Michael Perscheid, Michael Haupt, Robert Hirschfeld, and Hidehiko Masuhara.
2012. Test-driven fault navigation for debugging reproducible failures. Infor-
mation and Media Technologies 7, 4 (2012), 1377–1400. DOI:http://dx.doi.org/10.
11185/imt.7.1377
[20] Lukas Renggli, Tudor Gîrba, and Oscar Nierstrasz. 2010. Embedding Languages
without Breaking Tools. In ECOOP 2010 – Object-Oriented Programming: 24th
European Conference, Maribor, Slovenia, June 21-25, 2010. Proceedings, Theo
D’Hondt (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 380–404. DOI:
http://dx.doi.org/10.1007/978-3-642-14107-2_19
[21] Armin Rigo and Samuele Pedroni. 2006. PyPy’s approach to virtual machine
construction. In Companion to the 21st ACM SIGPLAN symposium on Object-
oriented programming systems, languages, and applications (OOPSLA ’06). ACM,
New York, NY, USA, 944–953. DOI:http://dx.doi.org/10.1145/1176617.1176753
[22] Chris Seaton, Michael L. Van De Vanter, and Michael Haupt. 2014. Debug-
ging at Full Speed. In Proceedings of the Workshop on Dynamic Languages and
Applications (Dyla’14). ACM, New York, NY, USA, Article 2, 13 pages. DOI:
http://dx.doi.org/10.1145/2617548.2617550
[23] Matthias Springer. 2016. Inter-language Collaboration in an Object-oriented
Virtual Machine. arXiv preprint (2016). arXiv:1606.03644
[24] Richard M. Stallman, Roland Pesch, and Stan Shebs. 2011. Debugging with GDB:
The GNU Source-Level Debugger, V 7.3.1 (10th ed.). GNU Press, Boston, MA, USA.
[25] Marcel Taeumel, Bastian Steinert, and Robert Hirschfeld. 2012. The VIVIDE
programming environment: connecting run-time information with program-
mers’ system knowledge. In Proceedings of the ACM International Symposium on
New Ideas, New Paradigms, and Reflections on Programming and Software (On-
ward! 2012). ACM, New York, NY, USA, 117–126. DOI:http://dx.doi.org/10.1145/
2384592.2384604
[26] Michael L. Van De Vanter. 2015. Building Debuggers and Other Tools: We Can
"Have It All". In Proceedings of the 10th Workshop on Implementation, Compilation,
Optimization of Object-Oriented Languages, Programs and Systems (ICOOOLPS
’15). ACM, New York, NY, USA, Article 2, 3 pages. DOI:http://dx.doi.org/10.1145/
2843915.2843917
[27] Bret Victor. 2012. Stop drawing dead fish. (May 2012). http://san-francisco.
siggraph.org/stop-drawing-dead-fish/ Talk to the San Fransicso ACM SIG-
GRAPH.