Manhattan: End-User Programming for Music

Chris Nash
Department for Computer Science and Creative Technologies,
University of the West of England,
Frenchay Campus, Coldharbour Lane,
Bristol, BS16 1AH
[email protected]

ABSTRACT generate music using dynamic composition techniques, as used in

This paper explores the concept of end-user programming languages generative, procedural, aleatoric, or process-based music. However,
in music composition, and introduces the Manhattan system, which by raising the abstraction level, composers concede low-level control
integrates formulas with a grid-based style of music sequencer. of musical elements (e.g. individual notes or events); it becomes
Following the paradigm of spreadsheets, an established model of harder to deviate from the formal algorithm [16].
end-user programming, Manhattan is designed to bridge the gap The Manhattan project seeks to develop a unified environment
between traditional music editing methods (such as MIDI sequencing supporting both contemporary and traditional music creativity, by
and typesetting) and generative and algorithmic music – seeking both flexibly combining low-level (concrete) and high-level (abstract)
to reduce the learning threshold of programming and support flexible composition practices, in ‘mixed-mode’ works and workflows,
integration of static and dynamic musical elements in a single work. supporting fusion and crossover music that bridges mainstream and
Interaction draws on rudimentary knowledge of mathematics and avant-garde aesthetics (cf. [8]). Example scenarios include:
spreadsheets to augment the sequencer notation with programming sequenced music using dynamic elements (e.g. random values,
concepts such as expressions, built-in functions, variables, pointers harmonisation, context-sensitive phrases, non-linear musical forms);
and arrays, iteration (for loops), branching (goto), and conditional music using formulas to simplify or automate editing; generative
statements (if-then-else). In contrast to other programming tools, processes outputting music that can be directly edited or manipulated;
formulas emphasise the visibility of musical data (e.g. notes), rather pieces with parallel or synchronised, but otherwise separate parts,
than code, but also allow composers to interact with notated music each entered manually or generated programmatically.
from a more abstract perspective of musical processes. Manhattan1 proposes a synthesis of sequencing and scripting
To illustrate the function and use cases of the system, several elements, based on the concept of musical grids and formulas. This
examples of traditional and generative music are provided, the latter paper draws on research into digital creativity (see [14]), end-user
drawing on minimalism (process-based music) as an accessible programming (see [13]), and the cognitive dimensions of notations
introduction to algorithmic composition. Throughout, the system and [4] to explain how the spreadsheet paradigm successfully integrates
approach are evaluated using the cognitive dimensions of notations end-user editing with programming functionality, and illustrate how
framework, together with early feedback for use by artists. the concept of formulas is adapted and extended for use in computer
music, to create a system combining sequencer interaction and the
core features of an imperative programming language, with support
Keywords for conditional statements, loops, branching, variables, pointers, and
end-user programming, algorithmic composition, generative music,
arrays. The sequencing element draws on tracker notation, a general-
minimalism, sequencers, trackers, digital music notations
purpose, text-based approach for specifying detailed patterns of
1. INTRODUCTION music, with an appearance and interaction style similar to
spreadsheets, and musically similar to an advanced step sequencer or
Several disjunctions exist between mainstream music editing
data list. A working prototype is detailed, using analogies to
packages, based on arranging and transcribing notes or musical
programming languages (such as BASIC) and music notation, with
events (e.g. sequencers, score editors), and music programming tools,
practical examples illustrating applications ranging from traditional
based on defining abstract processes that dynamically generate music
music editing to generative music, revisiting two pieces of process-
(e.g. SuperCollider, Max). The divide spans not only working
based music, Steve Reich’s Piano Phase and Arvo Pärt’s Fratres.
methods and interaction styles (manual vs. generative, direct
manipulation vs. programming, usability vs. virtuosity), but also 2. BACKGROUND
artistic aesthetics, practices and communities (popular vs. avant- Most systems can be categorised as “low threshold and low ceiling”
garde, traditional vs. experimental). (built for usability) or “high threshold and high ceiling” (built for
Traditional low-level music editing (sequencing, transcribing, experts) [9], as seen in the design of music editors and programming
arranging) focuses on a static, concrete and detailed specification of languages [2]. In music, end-user tools such as sequencers are based
music, where individual notes are addressable and manipulated on low-level data structures (notes, MIDI), manipulated through
manually. Based on the recording studio, sequencers focus on preset abstractions, processes or metaphors from traditional practices
capturing a live performance and freezing it on a linear timeline, such as music performance and the studio [3], confining the creative
offering limited opportunities or tools to articulate abstract musical process to established paths and working styles [11]. By contrast,
processes or concepts [2]. By contrast, music programming tools programming enables the formal definition of more abstract musical
enable composers to define abstract processes, and programmatically processes, but at the expense of access to low-level data and the
freedom to deviate from algorithms [16]. The goal of this project is to
Permission to make digital or hard copies of all or part of this work for personal
or classroom use is granted without fee provided that copies are not made or find a flexible structure that permits both direct editing of low-level
distributed for profit or commercial advantage and that copies bear this notice data and the definition of dynamic elements and musical processes.
and the full citation on the first page. To copy otherwise, to republish, to post on
servers or to redistribute to lists, requires prior specific permission and/or a fee.
NIME’14, June 30 – July 03, 2014, Goldsmiths, University of London, UK. 1
Named for the grid-like street layout of New York, used in
Copyright remains with the author(s). computing to describe rectilinear distances in grid-based geometry,
and the original setting for early minimal (process-based) music.
2.1 Spreadsheets and End-User Programming
Spreadsheets are widely recognised as a successful paradigm in end-
user programming [7][13]; a programming language designed for
non-programmers [10] - it is “possible that more people program
with spreadsheets than any other programming environment.” [5]
The spreadsheet UI is designed for usability, based on the simple,
flexible, and familiar structure of the table, extended by formulas that
allow users to define basic relationships between cells, through
mastery of only two concepts: cells as variables, and functions as
relations between cells [13]. Formulas are automatically recalculated
when their dependent values change, providing instant feedback that
enables users to “tinker” and experiment with “what-if” scenarios.
Users can also selectively learn and use formulas, providing a low
threshold for novices, who can plug values into cells manually, and a
scalable approach to more complex functionality that allows them to Figure 2. The reViSiT tracker [11]
appropriate and extend the notation for their own uses [5][10].
Ko et al. [6] characterise the spreadsheet metaphor as a concrete, 3. MANHATTAN
human-centric approach to end-user programming, in contrast to Manhattan system is designed as an integrated composition
more abstract or computer-centric approaches (see Figure 1), but one environment combining direct music editing (i.e. sequencing) and
that does not extend well to general-purpose computation [13]. end-user music programming. Section 3.1 introduces tracker
Using the cognitive dimensions of notations [4],2 Hendry and notation, a textual, grid-based style of sequencer software, similar in
Green [5] discuss the spreadsheet’s concreteness as closeness of form and function to the spreadsheet, which is extended to support
mapping, allowing users to transfer knowledge and formulate formula expressions in Section 3.2. A variety of use cases are then
problems within the task domain (see [10]); using a familiar, visual detailed in subsequent sections.
representation that allows users to feel as though they are working
directly on the task [7]. Additions and changes to the notation can be 3.1 Trackers as grid-based sequencers
made quickly and in any order, facilitating rapid editing (low Trackers ([11], e.g. Figure 2) are a class of sequencer based on a
viscosity), reducing the need to plan (premature commitment), concise text notation, edited using the computer keyboard. Music is
enabling “idea sketching”, and offsetting error-proneness in represented in fixed grids (patterns), visually similar to a spreadsheet,
exploratory design contexts [5]. Formulas similarly support where columns represent tracks and rows represent fixed time slices
progressive evaluation, allowing the testing of partial and incomplete like a step sequencer. Each cell has spaces for: pitch, instrument,
solutions [4], in contrast to tools where programmers must recompile, volume (or panning) and one of an predefined set of musical effects,
re-execute, or re-enter data in order to test a change [7][10]. This for example: C#5 01 64 D01 triggers [C#5], using voice [01],
engenders a high “level of liveness”, facilitating tinkering, volume [64], with a slow [01] diminuendo [D].3
exploratory creativity, and learning by experimentation [12]. As in spreadsheets [5][10], the rapid editing interaction (low
Spreadsheets offer a declarative, constraint-based model of viscosity) and fast feedback cycle (progressive evaluation) supports a
programming that avoids control flow (loops, branching), in favour high degree of liveness, enabling sketching and flow [12]. In both
of data flow – similar to visual programming languages, a popular cases, formulas offer a visual and interactive mode that easily
approach to end-user and novice programming, also seen in music integrated with the existing editing UI, emphasising the visibility of
(e.g. Max). However, cell dependencies and execution order can be data, rather than code. Unlike spreadsheets, however, there is a linear
unclear, making it difficult to define events and processes over time, time axis and sequential execution order, spatially illustrating the
which is needed in music [17]. In the this paper, the spreadsheet event timing, similar to a score or graphical views in sequencers. As
model is extended to support the control and abstraction of time, in detailed in Section 4, this engenders a shift from the declarative,
an imperative programming style, based on familiar concepts of data-flow programming style of spreadsheets, to an imperative
musical time. programming style, with an explicit representation of time, while
retaining the 2D grid layout that allows music relationships to be
shown spatially (see [13]). Moreover, there is a high degree of
parallelism implicit in the pattern, as in music generally; musicians
are familiar with many complex concepts of concurrency, e.g.
synchronisation of parts, counter-point, polyrhythms, polytempi; the
interplay of parallel processes is visible in adjacent tracks of the
pattern, where a sequential (left-to-right) execution order further
simplifies concurrency, a major hurdle for novice coders [13].
3.2 Integrating Formulas
Like spreadsheets, scripting is introduced at the cell-level, where
properties of individual notes or events are manually entered or
defined by formula expressions written using mathematical operators
(e.g. +, -, /, *), built-in functions (e.g. abs, rnd, mod), conditional
statements (e.g. if-then-else), and references to data elsewhere in the
piece. Unlike spreadsheets, each cell defines multiple values and
formulas corresponding to different musical properties (pitch,
Figure 1. Programming Metaphors (from [6])
instrument, volume, panning, effect).

2 3
The framework was developed to describe usability factors in Manhattan is based on the reViSiT tracker (Figure 1), originally
programming languages, but has subsequently been adapted for developed as a VST plugin for sequencers, extending the tracker
wider use in HCI, including computer music [2][11]. paradigm to improve usability and musical expressivity [11].
 Figure 3. Formula Editing in Manhattan, with examples.
Formula evaluation occurs when cells are played, updating existing Since tracker notation is alphanumeric, it can be directly referenced
data in the cell. Resulting notes and values are thus both audible and in formula expressions, preserving closeness of mapping and
visible, and also become editable, just like manually sequenced consistency. For example, pitches can be entered as shown in the
music. Outside of playback, formulas are evaluated upon editing, pattern (e.g. C#5 + 4). Cell properties are accessed like class
offering instant feedback and enabling experimentation. Partial or members in object-oriented programming (e.g. [2].volume,
incomplete processes can thus be incrementally tested, supporting @Foo.effect.type). Most properties are represented as integers or
progressive evaluation and liveness (see Section 4.2.1 for example). enumerated values (pitches, effect types), but processed using
While relationships and constraints are declared as in spreadsheets, floating-point numbers, extended to also support complex types
the serialised execution (playback) order of cells engenders an using NaN-tagging.5 This, for example, enables whole cells to be
imperative style of programming, supporting control flow structures referenced, compared, and copied, e.g. [3:1] = [3:0].
like loops (for, while) and both conditional (if-then-jump) and Formulas are visible only during editing, which can impact code
unconditional (goto) branching that can be used to control playback readability, hide dependencies between elements, and make the flow
(e.g. musical form). While this increases the number of concepts to of complex pieces hard to follow. Hendry and Green [5] suggest
learn and cognitive effort in coding [10][15], music notations include using alternative visual modes and secondary notation to address
analogous concepts – e.g. musicians know iteration (loops) as repeats such issues. Thus, in addition to the label mechanism, an additional
and branching (goto) as jumps such as D.S. (dal segno).4 visual mode can be toggled to visualise dependencies between
Like spreadsheets, all data is contained in the visible cells of the formulas and referenced data (see Figures 3 and 4). During editing,
pattern and referenced using Cartesian-style x/y coordinates ([3:4]), the current cell’s dependencies are shown; during playback, all
but can be given a custom label to provide mnemonic handles in currently playing cells’ dependencies are shown. While the
formulas (@Foo) that act like named variables (or pointers). ‘spaghetti’ associated with non-structured programming (and visual
Syntactic sugar is also provided to abbreviate references, such as programming, e.g. Max) is apparent for more complex processes,
defaulting to current column ([3]; row 3, current channel) or relative channels can be selectively muted and disabled to restrict formula
coordinates ([-1]; previous row). This exploits the locality of evaluation and dependency visualisation, making it easier to inspect,
formulas [10], keeping expressions concise and portable, and learn, and debug expressions iteratively. Other visualisations are also
allowing a formula and its referenced cells to be moved or copied as supported: displaying all formulas in the pattern grid at once, or
a block without breaking their function. The label mechanism also serialising them into an imperative-style code listing – which could
provides a form of secondary notation that can be used to annotate or later be extended to detect control flow structures (for, if-then-else,
comment sections of the music or formulas, which also support end- while) and automatically translate the listing to a BASIC-style
of-line commenting (`). Like spreadsheets, unused cells in the grid pseudocode (e.g. Figure 4; cf. CogMap in [5]).
provide space for informal descriptions of musical elements, As an imperative programming environment, Manhattan is broadly
processes, or other information such as section headers, lyrics, etc. equivalent to non-structured programming languages (e.g. BASIC),
Data is ‘initialised’ by a conventional edit to the pattern, avoiding where code structure is largely inherent in the underlying machine –
the confusion novice coders might have with variable declaration and here, the grid of the pattern. Although non-structured programming is
initialisation, or memory allocation [13]. Labels and coordinates, unsuited to major software projects, musical algorithms are less
however, support pointer arithmetic to provide array functionality complex. The simplicity of the syntax and low-level structure make
(e.g. @Foo[2] adds two addresses to point to the second row below such languages easy for novices to learn, drawing on rudimentary
“Foo”). Users can thus enter a series of values in the pattern, label the maths skills while exposing the basic building blocks of coding [15].
series, and reference it as an array in code. The data is always visible, Similarly, issues of maintainability, readability and correctness are
but can be placed in muted tracks to avoid playback, for use as arguably less critical in artistic expression; so long as artists can
formula constants or parameters. Muted cells are not evaluated, also understand and read their own work, the removal of structural
allowing users to selectively execute and test formulas that are parts constraints grants them greater editing freedom (though become
of a larger system. In unmuted channels, unused cell digits (e.g. significant in collaborative scenarios and knowledge sharing).
volume, effect parameters) may also be repurposed to store The current lack of support for function definition means recursion
intermediate values or formulas without being themselves audible. is not directly supported, though its omission reduces the chance of
stability issues or coding errors arising (e.g. infinite recursion).
4
In trackers, repeats and jumps are supported by effects (Cxx, jump
5
to row xx; SB0/SBx, repeat SB0 to SBx, x times), which can be con- The process of setting a floating-point variable’s Not-a-Number bit,
trolled via formulas and used with an if-then-else clause (x ? y : z) using remaining bits to encode custom binary data, thus supporting
to enable conditional loops and branching, as seen in Section 4.2.2. both fast math operations and extended data types.
However, Manhattan is Turing complete (see Video 1) and, in many In mathematics and programming, conciseness represents an
cases, iteration offers a practical alternative to recursion, while also artistic aesthetic; solutions that are shorter or simpler are often seen as
representing a concept more easily assimilated by novice coders more elegant or beautiful. A similar aesthetic exists in digital music
[13][15] and familiar to musicians (see earlier). communities, such as tracking [11], which explicitly celebrate the
virtuosity involved in creating complex art using limited or minimal
4. FORMULAS IN PRACTICE resources (e.g. explicitly restricting file size or limiting polyphony).
Formulas can be used for a variety of purposes, from isolated These ‘minimal’ trends are not associated with experimental music,
expressions that simplify or automate editing to more generative but more mainstream genres, and practitioners may thus see formulas
examples that use algorithms to define the content or structure of a as a new outlet for demonstrating technical virtuosity.
piece. This section discusses several use cases for the system,
illustrating how formulas can not only support and extend 4.2 Generative Music
conventional music editing, but also enable generative processes; Formulas can be used to explore and develop techniques in
using two examples from process-based, minimalist music: Arvo contemporary art music, such as algorithmic composition, aleatoric
Pärt’s Fratres and Steve Reich’s Piano Phase. music (e.g. using random function, rnd), atonal and serial music
based on alternative pitch systems, or otherwise derived from formal
4.1 Traditional Music Editing or mathematical processes.
Formulas can be used to support, simplify, or automate traditional While the examples here demonstrate how the system can be used
music editing tasks. Such uses target productivity, rather than to define formal processes in generative music; the underlying
creativity, but also represent intermediate and practical applications sequencer architecture enables integration with manually-edited
that introduce end-users to basic functions, syntax, and coding music, supporting crossovers between artistic styles. The minimalist
concepts, acting as a stepping stone to more complex uses (e.g. music of Steve Reich, himself a notable proponent of (re)integrating
generative music). disparate art and popular music cultures, has inspired many popular
Formulas can benefit usability by reducing hard mental operations, artists and reversions of his process-based music (e.g. the Orb,
offering users simpler ways to work with tracker notation, such as ColdCut, Brian Eno, David Bowie), and it is hoped that end-user
entering values as fractions (64/7) or hex values (used in effect music programming tools can be used to bring generative processes
settings) as decimals (#24=36). Defining simple relationships or into mainstream music. Minimalism, specifically, is a school of
constraints ([3].volume/2, [-1].pitch+4) can reduce the knock-on experimental music based on simple generative and process-based
viscosity of editing, so that subsequent changes to referenced data are works often routed in consonant tonality, offering a musical aesthetic
automatically propagated to dependent cells. Basic formulas can thus more accessible to composers and listeners accustomed to popular
be used to define common musical devices and techniques, such as and traditional styles. The style shifts the artistic focus from
arpeggios, echoes, transposition, or automatic harmonisation. In this individual notes and phrases to processes and more abstract concepts
way, composition becomes a constraint-satisfaction problem [1], in a way that can be supported and explored in a formula-based
where composers develop 'solutions by finding and solving editing environment like Manhattan.
constraints that gradually restrict the set of possible solutions;
formulas allow them to fix known constraints and focus 4.2.1 Steve Reich’s “Piano Phase” Revisited
experimentation on unresolved artistic decisions. Piano Phase is a 1967 minimalist composition by Steve Reich, based
Formulas can also be used to simulate elements of musical prosody on a repeated phrase of twelve notes played by two pianists at
in live performances, by adding small, random variations to the different tempi. Over several minutes, cycles of each phrase
notated music (e.g. timing, volume, pitch). While this approach is gradually move out of phase, before returning to synchronisation.
unlikely to rival the emotion or virtuosity of human rendition, it can During the piece, melodic interaction between the two parts creates
make textures sound richer and rhythms more natural, masking the interesting counterpoints that dramatically change the character of
digital precision that can make digital music feel rigid or mechanical. the music; the listener unconsciously merging the parts, chunking the
By abstracting time, control flow statements can reduce the evolving stream of notes into distinct melodies or harmonic textures.
diffuseness of music. Repeats and jumps are common in music In Manhattan, the piece is replicated by encoding the twelve-note
notation, allowing composers to abstract patterns and more concisely phrase as an array, then offsetting playback to simulate the phase
represent music. Scores include simple conditional statements, such change. For the first part, the original phrase (@Notes array) is
as the first and second repeat endings or codas, which can be played back looped. Each cycle, a counter increments to shift the
encapsulated in formulas (e.g. if-then-else). Formulas can similarly phase of the second part. To refine the phase shift, each increment
be used to abstract dynamic forms and progressions in many styles of plays 16 times, each time delaying the second part by a sixteenth of a
folk and popular music, from cumulative songs based on extending row, using the note delay effect (SDx, where x is delay), slowly
and repeating short phrases (e.g. 12 Days of Christmas) to the edging to the next row and array offset. The pattern and formulas are
iterative layering and switching of elements in progressive music shown in Figure 3, and execution in Video 2.
(e.g. house, trance).

Figure 4. Steve Reich’s Piano Phase

in Manhattan, with formula listing.
 Figure 5.
Arvo Pärt’s Fratres in
Manhattan with formulas
(right) and BASIC-style
pseudocode (left).

In an imperative model of programming, code manipulates the method, based on the other parts. This method selects a pitch by
state of the program (e.g. memory). In Manhattan, this state is stepping away from the given note a set number of times, exclusively
preserved in the visible pattern data, and remains after playback ends, using pitches of a chosen chord. In Fratres, Pärt uses the A minor
making it easy to stop, edit, and continue – enabling the user to triad, restricting steps to A, C, E. In the first half of each sequence,
interactively test and tinker with variable and formulas without where the progression begins and steps away (1, 2), Pärt takes two
restarting from the beginning, thus improving the liveness of coding steps up from the lower voice (second position superior). In the
and debugging. For example, in a long and slowly-evolving second half of the sequence, where the melodic progression steps
progression like Piano Phase, the user can edit the pattern data (i.e. toward and ends (3, 4), he takes two steps down from the upper voice
@Phase.volume) to jump to an arbitrary stage of the piece. (second position inferior).
The example also highlights limitations of the grid-based formula In Manhattan, the melodic progression is modelled using a single,
system. In Figure 3, phase is modelled explicitly, in order to simulate looped pattern, divided into two halves that alternate between falling
polytempi, which is hard to visualise in a grid that imposes a unified and rising versions of the sequence. Each half is divided into four
tempo. This limitation is an inherent trade-off imposed by the linear row sections corresponding to the phases of the sequence, repeated
timeline of sequencers, which surrenders some of the abstractive three times. On each iteration, the second and third sections are
power with respect to musical time, in order to preserve a more extended by incrementing their repeats (SBx), allowing their formulas
concrete, traditional paradigm for manual editing. to step further along the scale. Within the these sections, the repeat
iteration is found by querying an internal counter (.repeat) for the
4.2.2 Arvo Pärt’s “Fratres” Revisited repeat effect (SBx). At the third iteration, the instruction to jump back
Fratres is a series of pieces written between 1977 and 1992 by Arvo to the beginnging (Cxx) is cleared, allowing playback to proceed to a
Pärt, based on a formal process that generates a melodic progression, rising version of sequence. The current iteration of the whole
which is used to produce a three-part harmony. The melody is built sequence is tracked using a counter that increments when played
of four phases: (1) beginning on a given pitch, (2) stepping along a (@Phase.volume). This musical structure and melodic progression is
diatonic scale away from the pitch, then (3) stepping in the same encapsulated by the first two tracks of the pattern shown in Figure 4,
direction along the scale towards the pitch, until (4) returning to the used for repeats/jumps and the progression in numeric values,
pitch. The sequence repeats three times, each time increasing the respectively.
steps taken along the scale (see Figure 5). The entire process is
repeated several times in the piece, alternating between stepping
down and up the scale.
The harmony comprises a lower voice using the melodic
progression, a parallel upper voice transposed 9 scale steps up
(diatonically) and a middle voice determined by the tintinnabuli Figure 6. Melodic and harmonic progression in Fratres.
 The next three tracks (03-05) use these values to index arrays 6. ACKNOWLEDGEMENTS
stored in muted channels, containing the notes for playback. The Many thanks to Sam Aaron, Darren Edge, TOPLAP, and the
@DMinor array contains the scale of D Minor, used for lower and [livecode] mailing list for valuable insights and feedback on the
upper voices: the lower directly indexes the array; the upper adds 9 to project. Thanks also to composers Esa Ruoho (aka Lackluster),
the original offset, transposing the part. Lastly, depending on the Maarten van Strien, and Phill Phelps for ongoing user feedback.
phase (2 or 3), the middle voice uses either the lower or upper voice’s
offset to index one of two arrays of pre-calculated tintinnabuli values, 7. SUPPORTING MATERIALS / VIDEOS
for notes in the adjacent scale: second position superior The following videos available from: http://video.revisit.info:
(@2ndPosSup) and inferior (@2ndPosInf) respectively.
Video 3 shows the script in action, and Figure 4 shows the pattern Video 1. Manhattan: Conway’s Games of Life.
and formulas with equivalent BASIC-style pseudocode. The A video of the system running Conway’s Game of Life, a cellular
comparison highlights the close mapping between music (repeats, automaton known to be Turing complete.
jumps) and programming concepts (conditional clauses, iteration, Video 2. Manhattan: Steve Reich’s Piano Phase.
branching), though broader musical processes are notably clearer in A video of the system simulating a piece of process-based,
the pseudocode, and partially-obfuscated by the dispersal and minimalist music based on repeated playback of the same phrase,
selective visibility of formulas in the pattern. While this can again be gradually moving in and out of phase.
seen as a trade-off against the increased visibility of music data Video 3. Manhattan: Arvo Pärt’s Fratres.
(notes), it would be feasible to automatically-translate pattern A video of the system simulating a piece of process-based,
formulas into code-style listings, as in Figure 4, within the editing minimalist music based on a mathematical melodic prog-ression,
environment itself (cf. [5]). harmonised using Pärt’s tintinnabuli method.
5. DISCUSSION & FUTURE WORK 8. REFERENCES
This paper has presented end-user programming as an approach to [1] J. Alty. Navigating though Compositional Space: The
unifying computer-based composition practices based on more Creativity Corridor. Leonardo, 28, 3 (1995), 215-9.
concrete, low-level direct editing interfaces and more abstract and [2] A. Blackwell and N. Collins. The programming language as a
formal programming languages, and described a system integrating musical instrument. Proc. of PPIG 2005, 120-130.
formulas in a grid-based sequencer. As in spreadsheets, formulas [3] M. Duignan. Computer mediated music production: A study of
lower the threshold for, and enable graduated levels of, programming abstraction and activity. PhD thesis, Victoria University of
by extending rather than replacing conventional workflows, Wellington, 2007.
supporting applications in both mainstream and experimental music, [4] T. Green and M. Petre. Usability Analysis of Visual
plus fusions of the two. The visibility of data keeps the results of code Programming Environments: a ‘cognitive dimensions’
processes apparent, while the mapping of code elements to a linear framework. Journ. of Vis. Lang. & Comp. 7:131-74, 1996.
timeline and execution order engenders a flexible imperative-style [5] D.G. Hendry and T. Green. Creating, comprehending and
programming language with control flow structures based on familiar explaining spreadsheets: a cognitive interpretation of what
concepts from music, mathematics, and spreadsheet use. discretionary users think of the spreadsheet model. Int. Journal
Early feedback from artists using the system has been both positive of Human-Comp. Studies, 40, 1994, 1033-1065.
and informative, and the design of the system has already been [6] A.J. Ko, B.A. Myers, and H.H. Aung. Six Learning Barriers in
shaped by responses from users. However, a more in-depth study of End-User Programming Systems. Proceedings of VL/HCC
user experiences is planned, to collect feedback from non-musicians, 2004, 2004, 199-206.
traditional sequencer users, and composers in experimental music. [7] C. Lewis and G.M. Olson. Can Principles of Cognition Lower
Formulas could enable end-user programming in other types of the Barriers to Programming? Empirical Studies of
music software, though it remains to be seen if a suitable reference Programmers: Second Workshop, 1987, 248-263.
system can be developed when the UI is not based around a unified [8] A. McLean. Artist-Programmers and Programming Languages
notation (e.g. sequencer) or grid (e.g. score editor). Sequencers may for the Arts. PhD Thesis, Goldsmiths, 2011.
pose a challenge, where the prosody in live performance already [9] B. Myers, S.E. Hudson, and R. Pausch. Past, Present, and
raises issues for translating data to notation [3][12]. Future of User Interface Software Tools. ACM Trans. on
Research in end-user programming (and spreadsheets) offers Computer-Human Interaction, 7(1): 3–28, 2000.
directions for extending Manhattan, including: graphical debugging, [10] B.A. Nardi. A Small Matter of Programming: Perspectives on
meta-programming (code macros), self-modifying code (formulas as End User Computing. 1993.
cell properties), function calling and recursion [15], modularity [11] C. Nash and A. Blackwell. Tracking Virtuosity and Flow in
(custom abstractions or groupings) [7], and external bindings to other Computer Music. Proceedings of ICMC 2011, 575–82.
tools (e.g. DAWs, Max/MSP/Jitter), protocols (MIDI, SysEx, OSC, [12] C. Nash and A. Blackwell. Liveness and Flow in Notation
IP) and both general-purpose and live coding programming Use. Proceedings of NIME 2012, 28–33.
languages (C/C++, Java, SuperCollider, Max). [13] J.F. Pane and B.A. Myers. Usability Issues in the Design of
As users gain experience, extensions can offer more flexible Novice Programming Systems. Carnegie Mellon University,
coding styles, supporting advanced programming. Ultimately, scripts Technical Report CMU-CS-96-132, 1995.
could be situated in the music (in cells) with capabilities, syntax, and [14] M. Resnick et al. Design Principles for Tools to Support
interaction styles similar to general-purpose programming languages, Creative Thinking. Creativity Support Tools (ed. Shneiderman
with the ability affect data in any part of the pattern, significantly et al.), 2005, 25-36.
increasing the expressivity of the notation and spreadsheet model. [15] J. Rogalski and R. Samurçay. Acquisition of Programming
While any extension adding complexity risks raising the threshold for Knowledge and Skills. Psychology of Programming (ed. J.-M.
novices, so long as the music remains visible and editable, scripting Hoc et al.), 1990, 157-174.
functionality optional, and formulas exist to provide stepping-stones [16] R. Rowe et al. Putting Max in Perspective. Computer Music
for beginners, the provision of multiple end-user programming Journal 17, 2, 1993, 3-11
approaches may illustrate a mechanism for scaling the challenge and [17] G. Wang. A History of Programming and Music. Cambridge
offering a graduated learning experience, raising the creative ceiling Companion to Electronic Music (ed. N. Collins and J.
of the notation as the user develops ability. d’Escriván), 2007