Physicality Feedback PDF
Physicality Feedback PDF
Physicality Feedback PDF
Sonoric landscapes are both heard and seen. They exist 2.4 Musical Gesture
because of human experience and human
consciousness. Music…connects to the visible human
body, not only as the receiver of sound but also as its Physical gesture in computer music performance has
agent or producer. The human embodiment of music is often been seen primarily as a controlling input to an
central to any understanding of music’s socio-cultural interactive system, signifying visual and musical intention.
agency. The semantic content of music–its discursive François Delalande (1988) describes the division of musical
"argument"—is never solely about its sound and the art gesture into three levels: effective gesture—that necessary
of hearing. It is instead about the complex relations to mechanically produce sound; accompanist
between sound and hearing as these are registered and gesture—movements associated with effective gesture
as they mediate the entire experience of being. That engaging the whole body but not directly related to the act
experience is physical; intellectual, in the broad of sound production; and, figurative gesture—wholly
meaning of the word; and spiritual, though hardly symbolic gestures of the performer. Many interesting
restricted to the religious or the mystical. But it is studies of musical gesture as it relates to the issue of
especially to be understood as the result of mediations gestural control in music have focused on effective gesture
between the ear and the eye. The sonoric landscape is and only to a lesser extent accompanist gesture (Cadoz and
peopled and hence interactive. It is external to the Wanderly, 2000). We find that accompanist gesture is an
human subject yet internalized by its sight and sound. equally important aspect of physicality in interactive
(Leppert, 1995: 18) performance; how gesture can result from a physical
(bodily) relationship with a gestural controller embodying
The act of listening/observing in a musical context has sonic feedback properties; in this sense, gesture is in part the
been described as "vicarious performance," (Cone, 1968: trace of a performer/instrument relationship.
2.5 Social Contexts of Chamber Music ...an instrument where the gestural controller is
independent from the sound synthesis model, both
related by intermediate mapping strategies. "Composed
I suggest that they [music and dance] have remained
key factors in human life, and are, in particular, means instruments" typically use two layers of parameter
for people to bridge gaps of communication and mapping on top of a more-or-less arbitrary synthesis
understanding between their lives in societies that engine to match various controller devices played by
prescribe certain ideas, sentiments, and definitions of the performer to the sound synthesis result. (Wanderley,
experience, and their bodily experiences as individual Schnell, and Rovan, 1998:2)
feeling beings. (John Blacking, 1995: 214-2)
This two-stage modular approach is useful in conceiving of
Through empathetic connection, chamber music creates the mapping of a gestural controller to a body and its
a sense of intimacy between the performers of an ensemble, movement/instrumental vocabulary, as separate from
and between the ensemble and the audience. Electro- structuring the abstract connections between performative
acoustic music performance practice has rarely engaged the actions and sound production. In our work, we extend this
intimacy of this musical context. The tradition of “tape and notion of a composed instrument to include: the design of
instrument” composition creates a certain “tyranny of the the gestural controller itself; the choices of sounds,
tape,” leaving a performer to chase the unyielding synthesis and digital signal processing methods for a
progression of the fixed media play-back. Most often, even particular performance and the integration of new sonic
in more “interactive” contexts, the live instrumentalist’s display systems in the performance feedback loop. Our
sound is amplified and drawn into the general stereo or instruments, described in previous papers, include extended
multi-channel field of sound reinforcement creating a sonic and abstracted traditional string instruments, systems for
“plane-of-separation” between the performer’s location and interactive dance/movement performances, and the
their physical gesture. In contexts with multiple live invention of distinctive new musical controllers and multi-
performers, general sound reinforcement schemes dislocate channel spherical sonic display systems: sensor/speaker
the identities of sonic production and their location—this arrays. As the name suggests, we find that creating
separation often subverts the intimacy of musical "composed instruments" is very much an act of
performance and prioritizes presentation over process. "composition," in the traditional sense.
A focus of electro-acoustic performance practice has
been the drive for larger and more impressive “immersive” 3.1 Sensor/Speaker Arrays: physical
multi-channel sound systems—some electronic string conduction of sound and natural
quartets play with sound levels equivalent to rock bands, localization
performance excitement often being created through
“larger-than-life” sonic displays and choreographed
spatialization. Relatively little has been done to create Essential to the development of subtle gestural
electronic instruments and display systems that engage the performance interfaces are equally responsive sonic
tradition, intimacy, and human scale of chamber displays; they are of central importance in the feedback loop
performance. between physical gesture and sonic response. The sonic
Context implies an immersion of body in a culturally display must reinforce the nuance of physical gesture and
constructed environment, a sensually ordered and situated offer localized sonic feedback for the performers on stage.
body. In the following sections, we describe the design and Our focus on sonic display designs in the composition of
musical application of instruments and sonic display interactive electronic instruments has impacted our personal
systems intended to engage and amplify the kinesthetic pleasure and satisfaction in performance, as well as our
empathy and sonic feedback of traditional musical ability to interact comfortably within an ensemble. In recent
performance contexts, without obliterating them. We developments, this approach has extended to actually
maintain that physicality and musical gesture are keys to holding the sensor/speaker instrument in the lap of the
sustaining and extending social traditions within a performer reminiscent of a small cello, or, wearing small
technological context, and suggest these systems as one speakers on the body of a performer. In these cases, there is
approach towards this end. a direct physical conduction of sound into the body of the
performer much as one finds in a traditional acoustic
instrument. This direct physical feedback greatly augments
3 Composing the Instrument the “touch” and intuitive control of the instrument as an
extension of the body.
The concept of a generic “composed instrument,” has When a family of spherical speaker arrays is used in
been described by Wanderley, Schnell, and Rovan as: performance the result is a rich ensemble “mix” taking full
advantage of the natural acoustics of the hall, not unlike a
string quartet or other conventional chamber ensemble. The 4.1 Various Violins and the Violinist “Pose”
effectiveness of our sound system is often judged relative to
the impressiveness and familiarity of large multi-channel Musical instruments have a way of defining how their
surround-sound type sonic display systems from the point of players will look, both in detail and in a general sense. This
view of the audience, the comfort and communication of the
"pose" is reflective of the expressive nature of the
performers, or acoustic blend of the ensemble, being of instrument and the player; the image of Anne Sophie Mutter
secondary importance. However, we have found that the
playing Brahms in itself is revealing about expressive intent,
use of single-point display systems has fundamentally as is the image of Hauk Buen playing Hardanger fiddle, or
changed our approach to live electro-acoustic music
Mark Wood playing electric violin. Subtle differences in
performance and suggest that the inter-performer instrument design contribute to enormous visual and
communication and subtle sonic nuance they enable is of
physical differences in playing style (Trueman, 1999).
primary importance in the development of electronic This remains true with electronic instruments, to an even
chamber music. Our systems invite the listener to lean greater extent. One example is our use of sensor bows. By
forward, listen and look, as with a conventional chamber themselves, our sensor bows suggest a variety of kinds of
ensemble, rather than sit back and soak up an immersive, physical interaction with electronic sound; moving the frog
surround-sound environment.
in various positions, which may require moving the entire
body, and simply pressing the bow in various locations, all
3.2 Individual Instruments for Idiosyncratic are effective ways of physically playing the sensor bow. In
Performance this way, sensor bows transform the string player into a kind
of dancer, and require their players to modify their
Rather than formulating general, universal strategies for traditional technique. This often creates an interesting
interface design, our approach to creating new instruments technical conflict—certain techniques, while effective for
for electro-acoustic music has focused on composing the sensor bow, may be useless for playing the traditional
idiosyncratic, personal structures that reinforce individual string instrument, and vice-versa. Finding points of cross-
approaches to performance. Our backgrounds, while all section, where playing both instruments simultaneously is
including conservatory training on western instruments, physically and musically fulfilling, is one of the fascinating
derive from such disparate fields as Jazz, Norwegian challenges presented by the technology.
Hardanger fiddle music, and traditional/contemporary
Japanese performance. The instruments we make reflect 4.2 Dancing Music / Sonifying Dance
these varied backgrounds and extend our voices and bodies
into a new context of interactive performance possibilities. One of the differences between dance (in a conventional
Of importance are not only the technical devices and sense) and instrumental performance is the way they define
concepts these instruments embody, but also their impact on gesture. In dance, gestures are enacted at least in part for the
personal and social aspects of music making, including the their visual impact, much more so then for musical
musical values they reinforce and the instrumental/dance performers where gestures are (mostly) born out of a
movements they may imply. In the design of composed physical relationship with their instrument in the process of
instruments, we have attempted to create sensing creating sound.
mechanisms and musical mappings that take into account In our pieces for interactive dance the sensual
the enculturated movements and social cues of the parameters of sound and vision become fused. While
performer as well as the performance context. This design historically (Western art) music has accompanied dance, or
focus issues agency to the performer, privileging the social the dancer has been bound to the strictures of music,
process and context of the musical situation over the interactive performance environments enable the dancer to
technology. simultaneously articulate sound and gesture.
We have found that this multi-modal expression of the
4 Composing the Body body challenges the performer to straddle established
boundaries of composer/musician/dancer. The very struggle
in this blurred inter-disciplinary context has created a new
Movements are mapped onto our bodies through our
paradigm, a sensual re-orientation of expressive and
varied instrumental and dance training, becoming an
structural parameters of performance. Is the instrumentalist
essential mode for artistic expressivity. In technological
“dancing” music? Has the dancer become a musical
performance contexts, this embodied cultural knowledge
instrument? Is the music “moving” the dance, or are the
can be amplified and applied to human/computer interaction
movements “playing” the music? In our current work we
through various forms of physical sensing.
have found that the body has indeed become an instrument,
and, through physical synergism, has subsumed both the
dancer and musician.
The nature of our movement and musical expression, as and formal academic concerts to smaller, less formal venues
well as our collaborative creative process, has significantly such as clubs, galleries and chamber music contexts. This
changed as a result of our work in this field. We find development also relates to a shift in the participants of the
ourselves involved in the composition of the sonic genre and the make-up of the audience; numerous “laptop
geography of a stage or of a dancer’s body. This process artists” and followers of more popularly based electronic
raises issues involving the negotiation of control and music are now interested in our music.
correspondence created by conceptual technological A particularly rich musical context has been found in our
linkages between disciplines. ensemble “interface,” where we perform with numerous
gestural interfaces and a complete ensemble of spherical
4.3 The Voice, Breath: affect and affected speaker arrays. In our performances, great attention is given
to the sonic installation, design, and resources of the
“instruments,” and of the overall ensemble. However, the
The voice, so centrally located in the body, would seem
structure of the music is left open. Most pieces result from
immune to the impact of electronic instruments. The voice
previously unvisited sonic combinations of our systems,
is inextricably tied to breath, while breath is tied to
brought about through free interaction between the players.
sentience, existence, and intension. It can be “felt” via
In this sense, our performance systems privilege traditional
kinetic empathy between performers, implying a lived
modes of human musical interaction over human/machine
experience of a mutually created sonic environment. In our interaction. Through the design of an ensemble of extended
work, the voice has served as an instrument and as a model
instruments, we are composing the musical context while
for the construction of composed instruments, both in a leaving details of structure and articulation open and the
literal sense (via sampling) and in a more general musical
result of our musical interactions.
sense, guiding our improvisations and our mappings of
physical input to sonic output. In turn, we have found in the
performing of these composed instruments that the
instruments themselves deeply impact how we speak and
breathe.