An environment that works the way nonprogrammers

expect is more inviting and helps users become more

confident and productive.

NATURAL PROGRAMMING
LANGUAGES AND By Brad A. Myers,
John F. Pane and Andy Ko

ENVIRONMENTS
Over the last six years, we have been working to create programming
languages and environments that are more natural, or closer to the way people
think about their tasks. Our goal is to make it possible for people to express
their ideas in the same way they think about them. To achieve this, we have
performed various studies about how people think about programming tasks,
both when trying to create a new program and when trying to find and fix
bugs in existing programs. We then use this knowledge to develop new tools
for programming and debugging. Our user studies have shown the resulting
systems provide significant benefits to users.
It is somewhat surprising that for looping, conditionals, and niques available for 60 years:
in spite of over 30 years of assignments shown to cause breakpoints, print statements,
research in the areas of empirical many errors for both beginning and showing the values of vari-
studies of programmers (ESP) and expert programmers in the ables.
and human-computer interac- C language. Systems such as Our thorough investigation of
tion (HCI), the designs of new MacroMedia’s Director and the ESP and HCI literature
programming languages and Flash, Microsoft’s Visual Basic, revealed many results that can be
debugging tools have generally and general-purpose program- used to guide the design of new
not taken advantage of what has ming environments like programming systems, many of
been discovered. For example, MetroWerks’ CodeWarrior and which have not be utilized in
the C#, JavaScript, and Java lan- Microsoft’s Visual C++, all pro- previous designs. However, there
guages use the same mechanisms vide the same debugging tech- are many significant gaps in our

COMMUNICATIONS OF THE ACM September 2004/Vol. 47, No. 9 47

knowledge about how people reason about programs implies it works in the way people expect. By “nat-
and programming. For example, there has been very ural programming” we are aiming for the language
little study about which fundamental paradigms of and environment to work the way that nonpro-
computing are the most natural or what questions grammers expect.
people ask when debugging. We are performing user Why would this make programming easier? One
studies which investigate these questions. way to define programming is the process of trans-
It is in this context that we developed the Natural forming a mental plan in familiar terms into one
Programming design process, that treats usability as a compatible with the computer [3]. The closer the lan-
first-class objective by following these steps: guage is to the programmer’s original plan, the easier
this refinement process will be. This is closely related
• Identify the target audience and the domain, that to the concept of directness that, as part of direct
is, the group of people who will be using the sys- manipulation, is a key principle in making user inter-
tem and the kinds of problems they will be work- faces (UI) easier to use. UI designers and researchers
ing on. have been promoting directness at least since Ben
• Understand the target audience, by studying the Shneiderman identified the concept in 1983, but it
actual language, techniques, and thinking they has not even been a consideration in most program-
naturally use when trying to solve problems. This ming language designs.
includes an awareness of general HCI principles Conventional programming languages require the
as well as prior work in the psychology of pro- programmer to make tremendous transformations
gramming and empirical studies. When issues or from the intended tasks to the code design. For exam-
questions arise that are not answered by the prior ple, a typical program to add a set of numbers in C
work, conduct new user studies to examine them. uses three kinds of parentheses and three kinds of
• Design the new system based on this informa- assignment operators in five lines of code, whereas a
tion. single “SUM” operator is sufficient in a spreadsheet
• Evaluate the system to measure its success, and to [2]. We argue that if the computer language were to
understand any new problems the users have. enable people to express algorithms and data more
Redesign the system based on this evaluation, and like their natural expressions, the transformation
then reevaluate it, following the standard HCI effort would be reduced.
principle of iterative design. Similarly, debugging activities could benefit from
being more natural. Research describes debugging as
Thus, the Natural Programming approach is an an exploratory activity aimed at investigating a pro-
application of the standard user-centered design gram’s behavior, involving several distinct and inter-
process to the specific domain of programming lan- leaving activities [12]:
guages and environments.
This article provides an overview of some of the • Hypothesizing what runtime actions caused failure;
work of the Natural Programming project.1 We • Observing data about a program’s runtime state;
explore why naturalness might be better for develop- • Restructuring data into different representations;
ers and what might be more natural in programs for • Exploring restructured runtime data;
graphics and data processing based on initial user • Diagnosing what code caused faulty runtime
studies. The results were used in the design of a new actions; and
language and environment called Human-centered • Repairing erroneous code to prevent such actions.
Advances for the Novice Development of Software
(HANDS). We also discuss our survey of program- Current debugging tools support some of these
mers creating small and medium-size programs using activities, while hindering others. For example, break-
a different environment called Alice (see www points and code-stepping support observation of con-
.alice.org). trol flow but hinder exploration and restructuring,
whereas visualization tools help restructure data but
Why Natural Might Be Better hinder diagnosis and observation [5]. Yet none of
The premise of our research project is that program- these tools support hypothesizing activities. The argu-
mers will have an easier job if their programming ment behind our Natural Programming approach to
tasks are made more natural. By “natural,” we mean debugging is that support for such question-related
“faithfully representing nature or life,” which here activities will significantly improve success. If pro-
grammers have a weak hypothesis about the cause of
1More details are available from other papers at www.cs.cmu.edu/~NatProg). a failure, any implicit assumptions about what did or

48 September 2004/Vol. 47, No. 9 COMMUNICATIONS OF THE ACM

 The goal is to make it possible for people to express their ideas in
the same way they think about them.

encouraged because it is a useful

The bee with the most nectar is: Stripes
He has this much nectar: 8 debugging strategy. When
All the bees have collected: 45
novices test their code incremen-
tally, they perform better [2].
The idea that the program-
cardname Bumbles
ming environment can help users
x
y
777
178 construct programs has a long
kind bee
back
nectar
bumble.gif
5
history. “Syntax Directed Editors”
speed 1
direction 251
(also called “Structure Editors”)
have been used to help eliminate
syntax errors at least since the
Cornell Program Synthesizer
[11], and there have been many
variations. Modern tools like
Visual Basic provide context-
dependent pop-up menus that
insert correctly formatted code,
but do not restrict what users can
type. Alice takes an extreme
did not happen at run- Figure 1. The HANDS system stance and only allows syntactically correct statements
portrays the components of a
time will go unchecked. program on a round table. All
to be entered, since all editing is performed by drag-
Not only do these data is stored on cards, and ging-and-dropping statements and using pop-up
unchecked assumptions the programmer inserts code menus to specify parameters. A number of environ-
cause debugging to take atinto Handy’s thought bubble
the upper left corner. When ments have adapted the successful spreadsheet style of
more time [12], but they the play button is pressed, end-user programming to other domains (for exam-
Handy begins responding to
also result in new errors. events ple, [1, 4]).
by manipulating cards
For example, in a study of according to the instructions in Many tools have been created to help with debug-
Alice users we found that the thought bubble. ging, but most focus on visualization of data and con-
50% of all errors were due trol flow. Research by Kehoe, Stasko, and Taylor
to programmers’ false assumptions in the hypotheses suggests, however, that these are not particularly help-
they formed while debugging existing errors [6]. ful for debugging [5]. Communications devoted a spe-
Related work. We build on the research of many cial section to new ideas for debugging in April 1997,
others, who have studied programming and debug- but none of the systems have been user tested or
ging. Prior studies (see [10] for a summary) have widely deployed [8].
shown that features of the programming environment
are a crucial part of making a programming language Language Studies
effective and easy to use. For example, providing We conducted two studies to examine the language
immediate feedback about problems and testing helps and structure that children and adults naturally use
with problem solving. The ability to test partial solu- in solving problems before they have been exposed
tions is an important feature for novices and experts to programming. Participants were presented with
alike, especially during testing and when reusing code. programming tasks and asked to solve them on
This incremental and frequent testing should be paper using whatever text or diagrams they wanted
COMMUNICATIONS OF THE ACM September 2004/Vol. 47, No. 9 49
to use. To avoid biasing the subjects’ answers, the operands. Lists can be generated as needed, by using
study materials were constructed with great care, query operators that search all of the cards for the
using graphical depictions and terse descriptions of ones matching the programmer’s criteria. Queries and
problem scenarios. One study used the PacMan aggregate operations work in tandem to enable the
video game and another used database access scenar- programmer to concisely express actions that would
ios more typical of business programming tasks. require iteration in most languages. For example, the
Some observations from these studies were: following is a typical HANDS statement combining
queries and aggregates:
• An event-based or rule-based structure was often
used, where actions were taken in response to set the nectar of all flowers to 0
events. For example, “When PacMan loses all his
lives, it’s game over.”
• Aggregate operators (acting on a set of
objects all at once) were used much more
often than iterating through the set and act-
ing on the objects individually. For example,
“Move everyone below the 5th place down
by one.”
• Participants rarely used Boolean expressions,
but when they did they were likely to make
errors. That is, their expressions were not
correct if interpreted according to the rules
of Boolean logic in most programming lan-
guages.
• Participants often drew pictures to sketch out
the layout of the program, but resorted to
text to describe actions and behaviors.

Additional details about these studies are reported To examine the effective- Figure 2. The Alice
programming environment
in [9]. ness of HANDS, we con- with a “why didn’t”
ducted a user study. question in progress.
The HANDS Environment and Ten-year-olds were able to
Language learn the HANDS system during a three-hour ses-
The next step was to design and implement sion, and then use it to solve programming problems.
HANDS, a new programming language and envi- Children using the full-featured version of HANDS
ronment. The various components of this system performed significantly better than their peers who
were designed in response to the observations in our used a version modified to be more like typical pro-
studies as well as prior work. gramming systems. Additional details about this
HANDS uses an event-based language that fea- study are reported in [9].
tures a new model for computation, provides queries
and aggregate operators that match the way nonpro- Debugging Studies
grammers express problem solutions, has high visibil- Our studies of the language and structure that peo-
ity of program data, and includes domain-specific ple use helped us to design a more natural program-
features for the creation of interactive animations and ming language. We are also performing studies of
simulations. how programming environments can help program-
In HANDS, the computation is represented as an mers avoid making errors, and help them find and fix
agent named Handy, sitting at a table manipulating a the errors they have already made. An important part
set of cards (see Figure 1). All the data in the system of these studies is determining what causes program-
is stored on these cards, which are global, persistent, mers to make errors in the first place.
and visible on the table. We have integrated many strands of prior research
HANDS is event-based—a programming style on human error, and have found that errors are ulti-
that most closely matches the problem solutions in mately caused by long chains of breakdowns that hap-
our studies. It has full support for aggregate opera- pen for one of two reasons: some breakdowns occur in
tions—all operators can accept lists or singletons as the programmer’s head, such as using an inappropriate

50 September 2004/Vol. 47, No. 9 COMMUNICATIONS OF THE ACM

strategy, or having a misunderstanding about a partic- insertion of new errors that had to be debugged.
ular aspect of a programming language. The other type
of breakdown is caused by things outside of the pro- Design of the WhyLine
grammers’ head, in the programming language and No existing programming environment allows users
environment. For example, when it is difficult to to ask these kinds of “why” questions. By analyzing
inspect the values of variables at runtime, program- the control flow graph of the programs, and annotat-
mers may have breakdowns in debugging. Or, when a ing it with the complete history of all assignments
language supports different meanings for the same text and uses of properties’ values, we are able to answer
(such as the “+” operator in Java), programmers may these questions directly in the Alice UI (see Figure 2)
accidentally introduce errors. through the “Why did” and “Why didn’t” menus.
The submenus contain the objects in the
world that were or could have been
affected.
In addition to highlighting the relevant
piece of code, we present a visualization of
the answer to the question in Whyline—
the Workspace that Helps You Link
Instructions to Numbers and Events (see
Figure 3).
We performed a user study comparing
the Alice environment with and without
the Whyline. Subjects were Master’s stu-
dents with programming experience rang-
ing from beginning Visual Basic to
extensive C++ and Java. Analyzing six sit-
uations that were identical across the two
conditions, the Whyline significantly
decreased debugging time from an aver-
Figure 3. The Thus, to prevent errors, program- age of 155 seconds per bug down to 20 seconds,
Whyline’s
answer shows a ming environments should help pre- which is a factor of 7.8. Furthermore, in the 90 min-
vent these types of breakdowns. We
visualization of the utes allotted, programmers with the Whyline com-
runtime actions have developed methods of studying pleted 40% more tasks than those without. Full
that prevented Pac
from resizing. programmers’ work in order to deter- details are available in [7].
mine how the programming language
and environment might be changed Conclusion
to prevent breakdowns. We have recently focused on We are currently working on extending these ideas
preventing breakdowns in debugging. in many directions. We are investigating new
In order to see what tools might be useful, we per- domains in which to design more natural languages.
formed two studies of both experts’ and novices’ pro- We are extending the programming environment
gramming activity [6] using the Alice programming research to help with other parts of program analysis
environment (see Figure 2). We chose Alice for these and creation.
studies because it simplifies the creation of programs While making programming languages and envi-
by using drag-and-drop to place tiles of code into the ronments more natural may be controversial when
code area and pop-up menus to choose parameters. aimed at professional programmers, we believe it is of
This interaction prevents all type errors and syntax significant importance for end-user development. In
errors (see www.alice.org for more details). addition to supplying new knowledge and tools
We observed that all of the programmers’ questions directly, the human-centered approach followed by
at the time of failure were one of two types: 32% were the Natural Programming project provides a model of
why did questions, which assume the occurrence of an a methodology that can be followed by other devel-
unexpected runtime action, and 68% were why didn’t opers and researchers when designing their own lan-
questions, which assume the absence of an expected run- guages and environments. We believe this will result
time action. Furthermore, 50% of all errors were due to in more usable and effective tools that allow both end-
programmers’ false assumptions in the hypotheses they users and professionals to write more useful and cor-
formed while debugging existing errors, resulting in the rect programs. c

COMMUNICATIONS OF THE ACM September 2004/Vol. 47, No. 9 51

References
1. Burnett, M., Yang, S., and Summet, J. A scalable method for deductive 11. Teitelbaum, T. and Reps, T. The Cornell Program Synthesizer: A syn-
generalization in the spreadsheet paradigm. ACM Trans. Computer- tax-directed programming environment. Commun. ACM 24, 9 (Sept.
Human Interaction 9, 4 (2002), 253–284. 1981), 563–573.
2. Green, T.R.G. and Petre, M. Usability analysis of visual programming 12. von Mayrhauser, A. and Vans, A.M. Program understanding behavior
environments: A cognitive dimensions framework. J. Visual Languages during debugging of large scale software. In Proceedings of 7th Annual
and Computing 7, 2 (1996), 131–174. Workshop for Empirical Studies of Programmers. (Alexandria, VA, 1997).
3. Hoc, J.-M. and Nguyen-Xuan, A. Language semantics, mental models
and analogy. J.-M. Hoc et al., Eds. Psychology of Programming. Acade-
mic Press. London, 1990, 139–156. Brad A. Myers ([email protected]) is a professor in the Human
4. Johnson, J.A., Nardi, B.A., Zarmer, C.L., and Miller, J.R. Ace: Build- Computer Interaction Institute, School of Computer Science at
ing interactive graphical applications. Commun. ACM 36, 4 (Apr.
Carnegie Mellon University, Pittsburgh, PA.
1993). ACM, NY, 41–55.
5. Kehoe, C., Stasko, J., and Taylor, A. Rethinking the evaluation of algo- John F. Pane ([email protected]) is an associate information scientist
rithm animations as learning aids: An observational study. Interna- at RAND, Pittsburgh, PA.
tional J. Human-Computer Studies 54, 2 (2001), 265–284. Andy Ko ([email protected]) is a Ph.D. student in the Human
6. Ko, A.J. and Myers, B.A. Development and evaluation of a model of Computer Interaction Institute at Carnegie Mellon University,
programming errors. IEEE Symposia on Human-Centric Computing Pittsburgh, PA.
Languages. (Auckland, New Zealand, 2003), 7–14; www-
cs.cmu.edu/~ajko/HCC2003.pdf.
7. Ko, A.J. and Myers, B.A. Designing the Whyline, a debugging inter- This research has been funded in part under NSF grants IRI-9900452 and IIS-
face for asking why and why not questions about runtime failures. In 0329090, and by the EUSES Consortium under NSF grant ITR-0325273. Opinions,
Proceedings of 2004 Human Factors in Computing Systems (CHI04). findings, and conclusions or recommendations expressed in this material are those of
Vienna, Austria, Apr. 2004), 151–158. the authors and do not necessarily reflect those of the NSF.
8. Lieberman, H. The debugging scandal and what to do about it. Com-
mun. ACM 40, 4 (Apr. 1997). Special section, 26–78.
9. Pane, J. A programming system for children that is designed for usabil- Permission to make digital or hard copies of all or part of this work for personal or
ity. Ph.D. thesis, 2002. Carnegie Mellon University, Pittsburgh, PA; classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full cita-
www.cs.cmu.edu/~pane/thesis/.
tion on the first page. To copy otherwise, to republish, to post on servers or to redis-
10. Pane, J.F. and Myers, B.A. Usability issues in the design of novice pro- tribute to lists, requires prior specific permission and/or a fee.
gramming systems. School of Computer Science Technical Report,
CMU-CS-96-132 (Aug. 1996), Carnegie Mellon University, Pitts-
burgh, PA; www.cs.cmu.edu/~pane/tr96/. © 2004 ACM 0001-0782/04/0900 $5.00

Notice of Intent to De-Charter

ACM is considering the de-charter of the following chapters due to inactivity. Members
interested in revitalizing their chapter should contact Susan Wood, Local Activities
Coordinator, [email protected]. ACM will terminate the chapters listed here after 90 days unless inter-
ested volunteers express a desire to reactivate their chapter and prepare acceptable revitalization plans.

Belgian ACM Chapter Mexico ACM SIGCAS Chapter

Bulgarian ACM SIGCHI Chapter New York City ACM SIGCSE Chapter
Central Ohio ACM SIGCHI (BuckCHI) Chapter Ottawa ACM SIGART Chapter
Clear Lake Area ACM SIGAda Chapter Rio Grande ACM Chapter
Czech ACM Chapter Rochester ACM Chapter
Dallas/Ft. Worth ACM SIGCHI Chapter Rome/Italy ACM SIGAPL Chapter
Egypt ACM Chapter Southeast Michigan ACM SIGART Chapter
Huachuca ACM SIGAda Chapter Tampa Bay ACM Chapter
Hungarian ACM Chapter Toronto ACM SIGAPL Chapter
Huntsville ACM SIGAda Chapter Washington D.C. ACM SIGAda Chapter
Hyderabad/India ACM Chapter Westchester Fairfield ACM Chapter
Italian ACM Chapter Yemen ACM Chapter
Kaluga Region/Russian ACM SIGAPL Chapter Yogyakarta-Indonesia ACM Chapter
Los Angeles ACM SIGCHI Chapter Yokohama/Japan ACM SIGAda Chapter
LVIV/Ukraine ACM Chapter Oktibehha ACM SIGGRAPH Chapter

52 September 2004/Vol. 47, No. 9 COMMUNICATIONS OF THE ACM