Proceedings of ICAD 05-Eleventh Meeting of the International Conference on Auditory Display, Limerick, Ireland, July 6-9,

2005

UNDERSTANDING IMMERSIVE AUDIO: A HISTORICAL AND SOCIO-CULTURAL

EXPLORATION OF AUDITORY DISPLAYS

Milena Droumeva

School
for Interactive Arts and Technologies,
Department of Applied Science,
Simon Fraser University,
Vancouver,
Canada.
[email protected]

ABSTRACT surrounding these changes. Specifically, I will look at the

emergence of background music and media as a ubiquitous
This paper examines the historical and socio-cultural social environment, new architectural acoustics and
underpinnings of immersive audio seen from the paradigms disembodied sound, the emergence of digital technology,
of Acoustic Ecology and Acoustic Communication. The immersive audio in cinema and games, and finally – sound
paper offers the view that in order to understand the many design in complex interactive AD systems. These
implications of immersive sound, both from a design phenomena will be discussed within the framework of
perspective and from a cultural studies perspective, we need acoustic communication as a way of conceptualizing and
to first examine its social and technological histories. The understanding complex auditory display systems, and
paper explores examples and concepts of immersive sound summary tables will be provided after each major section t o
from the natural, electroacoustic, digital and interactive emphasize and define core concepts. By understanding the
domains, and presents a case study of Ec(h)o - a real audio- history of embedded sound and surrogate sound
augmented immersive museum installation. environments, and employing a communication-based
sound approach to conceptualize them, we will be better
1. INTRODUCTION equipped to understand more recent phenomena, such as
immersive audio. Here it must be noted that while
Interactive systems, especially ones that involve sonification constitutes another important research area i n
auditory displays, are becoming more and more integrated i n the filed of designed environments and immersion, for the
today’s public and private environments. Such systems purposes of brevity, this paper does not address sonification
range from simple computing and/or communication directly. Instead, it discusses the more general concept of
devices to complex gaming simulations and other still designed and immersive auditory display systems.
experimental physical interactive environments. In each
case, sound design plays an important role in generating a 1. SOUND-AS-ENVIRONMENT
sense of place, movement, and interaction, providing
aesthetic affect, and giving a tangible cultural context for Before immersive audio, there is immersive sound.
the activity at hand. Before embedded auditory displays in surrogate
Yet such recent developments in auditory display (AD) environments, there are naturally embedded sounds in the
systems and immersive audio have not occurred in isolation acoustic environment. Our preoccupation with technology
from other social, cultural and technological changes over often allows us to forget or ignore the ‘analogue’ processes
the last century. Advancements in sound reproduction, in nature that constitute the foundation of digital events and
electroacoustics and multimedia are only one part of a larger simulations. There are certain modes of listening that
equation. Scientific developments in acoustics, operate in a natural acoustic environment and provide u s
psychoacoustics and sound engineering, together with a with information about our surroundings while we employ a
greater understanding of audition and perception, comprise complex set of skills to interpret these signals. There is n o
another part. The ensuing socio-cultural changes in the reason to think that these processes do not continue t o
patterns of listening, modes of perception, and interaction operate in technologically-mediated AD environments [1].
with our technologically-extended world, all comprise the Immersive audio models the way we hear, listen, perceive
foundation of the development, popularity and ubiquity of and respond to sounds in our natural sound environments.
modern auditory displays. In turn, our ready acceptance of Yet immersive AD displays often rely on a less than
electroacoustic sound as a means of communication and a comprehensive understanding of acoustics, communication
banal necessity of life, contributes to an increased blurring and ecology of sound. The two models presented here
of boundaries between public and private, reality and virtual attempt to build a more solid framework for understanding
simulation. These interlinked processes not only affect the the acoustic environment and our relationship to sound i n
practice of AD design, but need to be considered within its situated contexts. As such, these approaches are valuable
very framework and context, as they embody its implicit tools in forming design practices and concepts associated
history. with immersive audio.
This paper aims to briefly survey major events in the
emergence and acceptance of electroacoustic sound in the
2 0th century, and to comment on the socio-cultural processes

ICAD05-162
Proceedings of ICAD 05-Eleventh Meeting of the International Conference on Auditory Display, Limerick, Ireland, July 6-9,
2005

1.1. Acoustic Ecology Model 1.1. Acoustic Communication Model

A pioneer in the field of Acoustic Ecology, R. Murray Following Schafer’s work, Barry Truax developed a
Schafer first defined the notion of a soundscape to mean a multi-disciplinary framework for understanding sound
holistic system of sound events constituting an acoustic based in notions of acoustic ecology. This framework –
environment and functioning in an ecologically balanced, acoustic communication – moves away from the energy-
sustainable way [2]. Born out of the threat of urban noise transfer view of sound, where the focus is on the mechanical
pollution, Schafer’s ideas centre on conceptualizing an transmission and reception of sound vibrations. Instead, i t
ecological balance in the acoustic frontier. Around his work models sound, listener and environment in a holistic
in the World Soundscape Project, he develops the terms ‘hi- interconnected system, where sound mediates a two-way
fi’ and ‘lo-fi’ to describe different states of aural stasis in the relationship between listener and environment [4]. This new
environment. A hi-fi soundscape, exemplified in Schafer’s understanding of sound allows us to bring considerations of
work by the natural environmental soundscape, is one where culture and political economy into the soundscape paradigm
frequencies occupy their own “spectral niches” [3] and are alongside auditory perception and cognition. Drawing o n
heard distinctly, thus creating a high signal-to-noise ratio cultural and sociological histories of sound, as well as major
[4]. A lo-fi soundscape, on the other hand, often exemplified communication theorists, Truax creates a rich perspective of
by modern urban city noise, is one where [amplified] sound, the role of sound before and after the emergence of
traffic and white noise, mask sound signals and obstruct electroacoustics, and of shifts in listening patterns. Using
clear aural communication, creating a low signal-to-noise the acoustic communication model, the soundscape can be
ratio [4]. seen as a multi-faceted framework of reference representing
Schafer’s answer to noise pollution and the unbalanced the many relationships that sound mediates between
urban soundscape is a combination of aural education, environment, society, listeners, culture, public and private
sound awareness and a new public approach to the sound domains, class, status and politics.
environment through soundscape composition [2]. His The acoustic communication model also extends to a
central thesis is that the acoustic environment could and new understanding of psychoacoustics, listening and
should be heard as a musical composition and we must perception. Traditional models of auditory perception
acknowledge our own responsibility for its composition [3]. conceptualize listening as a process of neural transmission
This responsibility has both positive and negative of incoming vibrations to the brain [7]. In fact, as pointed
embodiments – soundmaking and music, or noise and signal out by Truax [5] and others, listening is a complex activity
masking. As Schafer’s colleague and acoustic involving multi-levelled conscious attention and higher
communication theorist, Barry Truax points out that, “the cognitive functions, including memory, template matching,
necessity of the ecological concept springs from the context foregrounding (attentive listening) and backgrounding
of loss, or at least from the present threat to survival. The (holistic listening based in Gestalt pattern recognition).
question for us now is whether a new balance can be Two major groups of listening are ‘everyday listening’ -
regained. Can we – with consciousness – be part of a new [8], [9], [10], [1] – omni-directional, semi-distracted,
eco-system?” [5]. In light of soundscape design, we have t o adaptive-interactive listening that focuses on immediate
be informed by the past, and maintain an ecological balance information-processing of sound, and ‘analytic listening’
of sound components. To start, we have to understand [4] – attention to detail, ‘expert’ activity focussed on an
design as a system that “comprises the knowledge and the aesthetic or analytical experience. Based on these patterns,
techniques that we understand and can put into practice,” [5] Truax has developed a number of terms exemplifying major
and that it involves everyone as listeners and soundmakers, listening modes and processes. Listening-in-search
not just the designer/composer. involves a determined seeking of a particular sound
Based on his work for the World Soundscape Project, template in an aurally busy environment. Listening-in-
which began in the 1970s, Schafer developed several core readiness involves background listening with an
terms to describe soundscape components. Keynotes are underlying expectation for a particular sound or set of
ambient, background sounds that are present a large portion sound signals (such as a baby cry) [4]. The ‘cocktail party
of the time in a space; sound signals are foreground sound effect,’ conceptualized by many AD designers, is a special
events that transmit information about the environment, and mode of listening-in-search, which involves a ‘zooming in’
soundmarks, similarly to landmarks, are unique sounds on a particular sound source – often semantic-based (speech)
typical for local community soundscapes and often - in an environment of competing sound information with
characterize them [4]. Yet these terms are not set categories, the same spectrum.
but characteristics of sound, as each individual listener’s
perception will determine the class to which the sound Key Concepts from the Acoustic Ecology and Acoustic
belongs. Communication models:
Another important idea that Schafer develops in his
1993 book Voices of Tyranny, Temples of Silence, is the A high-information environment with a
Hi-fi
concept of ‘acoustic space’. His conceptualization has many high signal-to-noise ratio
implications for soundscape design and auditory perception
An environment with a high degree of
studies, because it emphasizes the multi-directionality of Lo-fi
masking and low signal-to-noise ratio
the sound field and the complex, almost unpredictable
nature of sound behaviour. Acoustic space consists of many An ambient sound, present in an acoustic
sounds and is coloured by all physical properties of the Keynote community most of the time and
environment, including the listener, thus creating a unique cognitively backgrounded by listeners
atmosphere of sound and place [6]. Along with acoustic
coloration - the audible properties of sound produced b y Foreground sound events, providing
Sound signal
their interaction with the physical environment - these information
soundscape components characterize an acoustic community
Unique sounds, characterizing a
[3]. Soundmark
community

ICAD05-163
Proceedings of ICAD 05-Eleventh Meeting of the International Conference on Auditory Display, Limerick, Ireland, July 6-9,
2005

The process of recreating a soundscape paying audience, it becomes situated in make-believe

Soundscape surrogate environments [14]. In the era of individualized
using electroacoustic techniques of sound
Composition representation, as Attali argues, “music can no longer affirm
manipulation
that society is possible. It repeats the memory of another
Omni-directional, semi-distracted, society” [13]. This new status of music as an object, and not
Everyday adaptive-interactive listening that we an everyday activity, helps shape a new type of listening – a
Listening engage in on a daily basis with the goal of more attentive, analytical listening, focussed on the
immediate information processing individual elements of sound, rather than on a holistic
perception.
Analytical Attentive, foreground listening, usually to
Listening the finer details of sound
1.1. Listening in the Electroacoustic Era
Cocktail-party An aural ‘zooming in’ in a busy acoustic
effect environment Attali’s theory of mass music, based on Adorno and
The result in perceptual loss due to white, Horkheimer’s texts on mass culture, proclaims the
Masking traffic, or other broadband noise destruction of cultural meaning. “The absence of meaning i s
prominently present in the environment the necessary condition for the legitimacy of a technocracy’s
power,” he announces grimly [13], describing the era of
A sound field of propagation and “repetition,” the stockpiling of music and the emergence of
Acoustic Space interaction between sound and copyright - music as intellectual property. In addition, the
environment era of mass music turns the idea of silence into a “death i n
the heart of life” [13]. Schafer [2] also comments on the
vilification of silence in the contemporary soundscape as a
1. MUSIC-AS-ENVIRONMENT ‘dead space’ and a negative force in society. Repetition
symbolizes uniformity, compliance and ‘programmed
Every historical shift in the sonic environment helps events’, as Ursula Franklin [16] argues in “Silence and the
contextualize and determine the directions in which Notion of the Commons”. She describes the disappearance of
subsequent acoustic environments are heading. In this silence as an “enabling environment” and its replacement b y
section, I discuss the critical transition from sound-as- “the silencing that comes when there is the megaphone, the
environment to music-as-environment, where music, boombox, the PA1 system” [16].
especially background music, becomes the ‘normal’ ambient Mass reproduction of music exists long before
environment, and new listening patterns emerge. electroacoustics with opera and instrumental concerts [14];
however, electroacoustic technology makes possible the
exact replication of any sound and its independent
1.1. Listening Before Electroacoustic Reproduction recollection thereafter. This schizophonic sound [2] i s
disembodied from its source, context and time of occurrence,
Walter Ong’s extensive work on orality and and becomes an abstract ‘aural object’ of representation [17].
communication points to many of the characteristics of Sound comes to symbolize power, control, use and exchange
listening prior to print literacy and the recording and value, and private ownership [13]. Electroacoustics also puts
stockpiling of speech/sound [11]. The ephemeral quality of sound under unprecedented scrutiny as well as aesthetic
sound - the fact that nothing could be repeated or appreciation, fostering analytical listening [4]. Alongside
reproduced in the exact same way again - contributed to an the move towards clean, non-reverberant sound i n
active, feedback-oriented ‘everyday listening’ [4], where architectural acoustics [15], electroacoustic technologies,
sounds mediate the communication between people and the especially digital sound, further remove any perceptual
environment that surrounds them. Albert Bregman, James reference to space by eliminating acoustic coloration. The
Ballas and other theorists of sound, have focussed precisely transition facilitates the easy transfer of sound/music
on this ‘everyday listening’ of environmental sounds, and objects into a variety of different surrogate environments –
their work points to an understanding of auditory restaurants, concert halls, stadiums, malls, schools, etc. This
perception in terms of sound events, emphasizing the fluidity in turn results in the acceptance of music-as-
functional ecology of this interaction [12]. This balance i s environment in both the private and public spheres of life,
reinforced by another important condition of the period and leads to the blurring of the lines between the two.
before electricity and amplification – that sounds can only Background music, defined by Satie as music “like
be as ‘loud’ as it is physically possible for the sound source furniture,” [18] not only becomes part of the environment,
to transmit energy, so problems of masking and obstructed but is the environment. It builds invisible surrogate
communication are minimized [4]. relationships between people sparing them from obligatory
In terms of musical expression and soundmaking prior interaction and “filling up heavy silences between friends”
to electroacoustics, Jacques Attali gives us his account of [18].
the different relationships that music mediates between Similar to the Telharmonium of 1906, which provides
people and society before and after the mass reproduction of ‘atmosphere music’ in restaurants using the telephone line
sound. In the stage of ‘sacrifice,’ music is simply a means of [19], the Muzak Corporation was created in 1922 to first
“channelling society’s violence” [13]. In other words, music provide music over the telephone [13], and later t o
reaffirms order and counteracts chaos, exemplified by noise. ‘program’ music in various public spaces by use of market
In this regard, music, much like sound in the acoustic research patterns. Muzak is perhaps the first materialization
communication model, is a mediator of social relationships of complex soundscape design. Hildegard Westerkamp’s
in the context of the environment and community. In the research in background music solidifies the views put
period of ‘representation,’ starting in the 1 7th century i n forward by Attali and others – that music in the era of
Europe, music begins to embody more than just sounds. It schizophonic mass reproduction becomes a “soundtrack for
becomes a symbol of power, status, cultural expression [13].
It starts exhibiting use/exchange value as an object, and
with the emergence of virtuosity, concerts and opera for a 1
Public Announcement

ICAD05-164
Proceedings of ICAD 05-Eleventh Meeting of the International Conference on Auditory Display, Limerick, Ireland, July 6-9,
2005

consumerism” [20]. Sound “becomes associated in our cultural changes ensuing from its introduction in the private
memories with environments and products. In essence i t and public social worlds. Radio literally ‘educates the
becomes the ‘ambiance’ of the media environment” [5]. consumer’ and creates a new language of media
However, the new fluidity and flexibility of sound freed of consumption, aural sensitivity, listening and cognition. One
its source does not result in endless diversity of spaces and of the major functions that radio fulfils is “bracketing the
sounds, but in the emergence of archetypal surrogate day” through program flow – that is, providing structure t o
environments. These most often commercial spaces come daily activities by setting predictable patterns – news at
with pre-packaged sound quality – compressed, narrowband noon, followed by music, announcer, advertising, then more
dynamic range and dry digital sound, as well as content news, etc [22]. In addition, as with background music i n
standards – slow light rock for the department store aisles malls and restaurants, radio ‘sets the mood’ for the day and
and fast pop at the restaurants. These emerging sound “lubricates” social relations [22] in alienated urban settings.
environments foster “passive listening” [20] and superficial As such, radio functions as a shared aural environment and
disengagement from the social environment. As Westerkamp implied shared physical and mental space. Similar t o
argues, the phenomenon of background music also results i n background music, radio settles as a predictable
the inevitable silencing of spontaneous human ‘accompaniment media’ to daily life, first confined to the
soundmaking, and with it our active interaction with place home, and soon invading the streets and offices [23]. This
and time [20]. transgression of public-private boundaries leads way t o
portable, personalized sound accompaniment in the age of
the ‘Sony Walkman,’ as characterized by cultural theorist
Key concepts from the Music-As-Environment framework: Paul du Gay [24]. Accompaniment media becomes a standard
sonic companion, and listening habits adjust accordingly,
Music has use and exchange as we become the perceived composers of our public and
Music as an object of private electroacoustic experience.
value, and it symbolizes power,
representation Combined with structured programming, radio changes
status, etc.
the listening experience from background listening to media
Ambient music programming listening. An important aspect of media listening, also
Background music such as Muzak, radio, TV flow, characteristic of TV flow as an extension of radio flow, is the
etc. pattern of amplitude flow. That is, the majority of normal
programming is broadcast in narrow bandwidth at relatively
Electroacoustic sound that is constant amplitude, designed to blend with the background
Schizophonic sound separated from its source and of daily activities, while advertising and other special
played at another space/time elements are broadcast with a broader spectrum and greater
dynamic range, demanding foreground attention from the
Inattentive, background
listener [4].
Passive listening listening, lack of active
In effect, media sound does not merely foster a dependent
interaction with sound
kind of listening, but it tells us how to listen. It trains us t o
A re-created environment, into increase or decrease our auditory attention by use of
which a chosen sound or an carefully crafted cues, until they become second nature.
Surrogate environment
auditory display system is These gestalts of auditory perception then seamlessly
inserted integrate in cinema sound, carrying the promise of total
immersion, suspension of disbelief, and realistic experience
design. Ultimately, as our environment changes and we
1. MEDIA-AS-ENVIRONMENT become more saturated in media flow we start to experience
sonic phenomena such as radio, TV, portable audio and
Archetypal sonic spaces such as concert halls, malls, and Muzak as environmental sound. This dramatically changes
restaurants, initially defined as public environments, our relationship with the acoustic environment, as
gradually move into people’s private lives with the electroacoustic and acoustic sounds become intertwined and
emergence of the phonograph, the telephone, and especially blend into each other, rendering the modes of listening that
the radio. The private space becomes another sound- we use for these two sound milieus interchangeable. As a
programmed surrogate environment, usually designed for result, we begin relying less on active, engaged,
passive, background listening. This shift is not unrelated t o information-processing listening, and more on habitual
the trend of development of mass media and mass background and media listening in all of our surroundings.
production/consumption. In a society where urban It is perhaps the emergence of interactive sound design that
alienation and post-industrial capitalism shatter traditional finally shifts the attention back to a more holistic
forms of community and social interaction, media becomes a perception of sound and active engagement with it, relying
surrogate social milieu for the ‘masses,’ and binds people on a more locative, communicational ‘everyday listening’
together in imaginary relationships. Radio is a particularly mode.
important cornerstone in mass media development and
plants the roots of contemporary media language, such as
Key Concepts from the Media-As-Environment framework:
temporal flow, structural density, foregrounding-
backgrounding cues, advertising language, audience as
product and media’s overall relation to consumerism. When Media programming creating a
Structured
wireless radio technology first emerged it was praised as a predictable pattern of sound/information
Flow
web of interconnection, universal communication and for commercial benefit
utopian democracy – “house of our dreams” as named b y
Media A new perceptual language of distracted
Gaston Bachelard [21].
listening listening and media perception
Mendelsohn’s work focuses specifically on the
characteristics of radio programming, conceptualising the

ICAD05-165
Proceedings of ICAD 05-Eleventh Meeting of the International Conference on Auditory Display, Limerick, Ireland, July 6-9,
2005

1. DESIGN-AS-ENVIRONMENT immersed in a compelling, increasingly tangible experience

[21].
As discussed above, background music is the earliest There are two predecessors to virtual audio that shape
fully designed and controlled surrogate sonic environment, and influence its authenticity and perceptual-listening
which, through interaction with other phenomena in sound framework. One is the microphone, which, similar to radio,
technologies and media, continues to shape the provides surrogate intimacy and “a spiritual and
contemporary listening experience. Other modern examples atmospheric nearness of broadcaster and listener” [21]. The
of immersive audio are film sound, video game sound and microphone eradicates implied physical distance in the
interactive auditory display installations. These interactive demolition of metaphorical distance between real and
systems can be either virtual – where sound is entirely representation. Another predecessor of virtual reality is the
engineered and usually delivered via personal headphones headphone technology. The possibility of bringing the
to the exclusion of other acoustic sounds; or physical – outer world into the inner world and creating a personalized
realized through computer programming but embedded in a surrogate sound environment has many cultural critics
shared physical and cultural space. These alternate sound fascinated. In his book Doing Cultural Studies, Paul du Gay
environments are made possible by advances in digital looks specifically at the phenomenon of the Walkman and
audio, virtual reality, the Internet and computer terms the experience of listening to portable audio a
programming. “soundtrack to life” [24]. Headphones literally immerse u s
in a designed soundtrack of imaginary space, and mediate
relationships between the listener and that space,
1.1. Sound in Cinema establishing a new VR phenomenology.
Audio technology creates a “sound field” designed t o
In its basic definition, an immersive sound space is a sound as if it occurred naturally in the environment,
carefully and intuitively designed surrogate environment however, the very existence of digital media has
that creates a more full-bodied experience involving the implications for the role of virtual audio. Sound is divorced
senses to a fuller capacity than traditional media. Sound i n completely from a physical source, and is controlled entirely
cinema is an important cornerstone in soundscape deisgn by code. It is technically abstracted – sound attributes,
because it simultaneously builds and draws on media and distances and behaviours are de-constructed, mapped and
background sound, as well as influences and feeds into more coded, and then reconstructed to fit a virtual space. This way,
sophisticated virtual and multimedia environments. Using algorithms represent sound and sound behaviour, rather
Barry Truax’s acoustic communication model for sound than expressing its physical characteristics, similar t o
design, an immersive environment contains three major Attali’s argument that musical code is a “language without
elements: speech, music and soundscape [4]. In cinema, meaning” [13]. Ultimately, the abstract nature of virtual
these break down to vocal material, composed musical score audio and the limitations and opportunities brought by the
and sound effects grid, including spatialization and/or technology of headphones and program code create a “new
interactive-adaptive programming. Hearing these sounds i n space of perception and embodiment” [21].
a darkened auditorium and in combination with larger-than-
life visuals promotes the experience of immersion. However,
it is important to note that the idea of immersion as virtual 1.1. Immersive, Interactive and Adaptive Audio
or “augmented” reality is not only an extension of earlier
electroacoustic technologies but also capitalizes on media In traditional AD design, concerned with alerts and
listening and decoding. As Truax comments, “once notifications, there are important considerations about the
background listening becomes a habit, it is ready for environment or context in which sounds are heard and
exploitation by the media” [4]. The cinema soundscape i s perceived. Aside from issues of auditory perception and
reduced to “easily recognizable sound objects” [21] and streaming, there are challenges with masking in busy sound
even though realism is increased through sound, “this environments. With physical interaction systems,
realism is not born of the ‘real’ [but is] constructed through specifically responsive environments, where almost every
other media” [21] - radio and television. Sound, is thus parameter of the experience is technology-driven, mediated
hyper-real rather than real. This view is also echoed b y and controlled, the AD system is the environment. Not only
Christian Metz in his Aural Objects essay, where he is the sound, and hence – the experience – designed i n
describes film sound as being based in sound objects – minute detail, but users most often experience the space via
individual representations of real life sounds, and signifiers headphones, further minimizing acoustic sounds and
in a larger conceptual media language [17]. colorations.
With the development of 3-D audio the restrictions of
stereophony are finally transgressed. Binaural recording and
1.1. Listening and Virtual Audio precise ITD (Interaural Time Delay), ILD (Interaural Level
Difference) and HTRF (Head-Related Transfer Function)
Virtual Reality (VR) is a special type of surrogate digital filters take headphone-based virtual audio to a new
environment based in digital audio, 3-D animation, rich perceptual level. By applying multidimensional streaming
graphics and audio coding. In VR spaces, sound is a virtual techniques, sound can effectively be reproduced anywhere i n
‘aural object’ of representation, carefully designed to elicit virtual space simulating a tangible sound source [25]. In
recognition, action and response, or create an emotional- addition, multi-channel speaker systems, commercially
psychological mood. As such, virtual audio builds on the known as ‘surround sound,’ provide a shared immersion
patterns of listening and recognition created by previously experience, where, ideally, each speaker functions as a sound
existing media and aural phenomena, yet it also results i n source on its own. In effect, AD design has finally shifted
new sensitivities and cognitive modes of interaction. back to a more ‘acoustic’ model of aurality, sound
Following from McLuhan’s conceptualization of the distribution and perception. As McLuhan argues, quoted b y
electronic world as oral [6], virtual audio creates a new kind Schafer, “the electric world” is aural, and it moves us back t o
of aurality in technological environments. Using this virtual an acoustic space of preliterate cultures [6]. In this type of
“aural medium,” one can “enter a space of no space” and be approach, sound has a strong connection to the environment

ICAD05-166
Proceedings of ICAD 05-Eleventh Meeting of the International Conference on Auditory Display, Limerick, Ireland, July 6-9,
2005

and mediates a relationship between the listener and space, artifacts are incorporated into the soundscapes. Another
reinforcing acoustic communication. element to the audio experience is a number of recorded
Interactive sound design, thus, attempts to use our narratives, that can be recalled via three aural ‘prefaces’
senses in a more physical, embodied act of listening, appearing in the left, centre and right speakers of the
engagement and interaction. It extends the existing headphones, utilizing sound localization techniques. All of
soundscape within the surrogate sound environment b y these elements: speech, soundmarks and keynotes interact
drawing attention to itself, rather than passively ‘filling up’ with each other creating an ‘acoustic space’ that
a space to create a mood for consumption. It does so b y characterises the present community.
asking us to relate to, orient ourselves through sound, locate
other objects within a space and give and receive feedback
about actions and events. Interactive audio also provides
variety and coloration by creating adaptive algorithms for
sound behaviour and allowing active modulation and
interaction with its listeners.

Key Concepts from the Design-As-Environment

framework:
An environment that simulates real-life
multi-modal interaction with space.
Immersive Space
Audio immersion simulates a natural
sound field
Sounds that symbolize real sources or
events, and have meaning outside of
Aural objects
their actual characteristics and
application
Sound designed to simulate reality
using sampled or digitally produced
Virtual Audio
sound, and presented in stereophonic or
multichannel format
A term used in physical interactive
Augmented
installations, describing added
Reality
technological aspects to the space

1.1. Ec(h)o: A Case Study Figure 1. Ec(h)o floor plan of the Museum of Nature in Ottawa

While this auditory display installation does not take

As an example of ‘ubiquitous computing’ and
ownership of solving all the complex problems i n
interactive-adaptive algorithm coding, Ec(h)o is multi-
soundscape design put forward in this paper, it does
modal sound installation for the Museum of Nature i n
nevertheless, provide an example of a design, which makes
Ottawa, Canada. It is “an audio augmented reality interface
use of both everyday listening modes for communicational,
utilizing spatialized soundscapes and a semantic Web
information-processing purposes, and background listening
approach to information” [26]. In this sound-enhanced
modes for creating a sense of place, cultural context and
exhibit the visitor navigates through several layers of audio
atmosphere of meaning.
– ambient soundscapes, location-based soundmarks,
hierarchical audio icons for selection and audio narratives
relating to the artifacts at hand. This interactive museum 6. CONCLUSIONS
installation is an example of design-as-environment
providing a multi-modal shared surrogate environment in a Immersion creates a new mode of perception and
physical space. Individual motion-tracked headphones embodiment incorporating a history of listening and
deliver the immersive sound experience, and the interaction cultural memory. AD systems have come a long way from
takes place via an ergonomic colour-coded cube, connected pure tone beeps and alarms to complex hierarchical audio
to visual recognition software [26]. The soundscape menus, rich immersive cinema and video game sound, and
elements are modelled upon the acoustic ecology and interactive and adaptive audio in ‘total immersion’
acoustic communication models, where sound, environment installations. Along the way many listening patterns have
and listener create a ‘feedback loop’ of interaction [4] (see influenced sound design, including a variety of ‘everyday
Figure 1 for a floor plan schematic – small circles represent listening’ patterns, as well as ‘background’ or ‘distracted’
soundmarks and big circles represent keynotes). The acoustic listening in the electroacoustic era, and ‘media listening’ i n
functionality of this system is defined by sound behaviour the age of structured flow and mass reproduction. The
that is “adapted to its environment and understands beauty conceptualisation of sound design has also changed
as a value expressed through people’s attitudes [4]. As significantly from seeing sound as an object of
shown in the diagram below, there are several thematic representation to focussing on sound events by grounding
soundscapes, localized in a radius around a relevant exhibit them in a source and generating a more variable, un-
and broadcasted over FM frequencies. These soundscapes are correlated sound field. Ultimately, contemporary interactive-
modulated in real time by the visitor’s movements, fading i n immersive audio systems make a return to more holistic
and out of each other. In addition, soundmarks, chosen notions of sound perception, acoustic community and space.
specifically to reflect important aspects of a given set of In an ecological, communication-based system, sound i s

ICAD05-167
Proceedings of ICAD 05-Eleventh Meeting of the International Conference on Auditory Display, Limerick, Ireland, July 6-9,
2005

well adapted to its environment and not only communicates [19] Weidenaar, R, Magic Music from the Telharmonium,
useful information and a richer multi-modal experience, but Scarecrow Press, 1995.
it mediates the very relationship between listener and that [20] Westerkamp, Hildegard, “Listening and Soundmaking:
interactive environment. In addition, immersive sound A Study of Music-as-Environment,” In D. Lander & M.
design as composition is a way of taking control of the Lexier, Ed., Sound by Artists, Art Metropole & Walter
soundscape and counteracting the distracted listening that Phillips Gallery, 1990.
the media world promotes. Through electroacoustics, we can
deconstruct the elements of a sound environment, re- [21] Dyson, Frances, “When the Ear is Pierced”, In M. A.
construct and re-conceptualize them through sound design, Moser, D. MacLeod & Banff Centre for the Arts, Ed.,
and partake in soundmaking rather than sound Immersed in technology: art and virtual environments
consumption. Cambridge, Mass.: MIT Press, pp. xxv, 339, 1996.
[22] Mendelsohn, H., “Listening to Radio,” In Dexter &
White, Ed., People, Society and Mass Communication,
7. REFERENCES
1964.
[23] New York Commission on Noise Abatement, 1960
[1] Kramer, Gregory, “An Introduction to Auditory [24] Du Gay, Paul, Doing cultural studies: the story of the
Display,” In G. Kramer Ed., Auditory display: Sony Walkman, London; Thousand Oaks Calif.: Sage i n
sonification, audification, and auditory interfaces. association with The Open University, 1997.
Reading, Mass: Addison-Wesley, 1994. [25] Brungart, Douglas S, “Near-Field Virtual Audio
[2] Schafer, R. Murray, The tuning of the world, Toronto: Displays,” Presence, vol.11(1), pp.93-106, 2002.
McClelland and Stewart, 1977. [26] Wakkary, Ron, Hatala, Marek, Newby, Kenneth,
[3] Wrightson, K., “An Introduction to Acoustic Ecology,” Evernden, Dale, & Droumeva, Milena, “Interactive
Soundscape, The Journal of Acoustic Ecology, vol.1(1), Audio Content: An Approach to Audio Content for a
pp.10-13, 2000. Dynamic Museum Experience through Augmented
[4] Truax, Barry, Acoustic communication (2nd ed.). Audio Reality and Adaptive Information Retreival,”
Westport, CT: Ablex Pub, 2001. presented at The 2004 Museums and the Web
[5] Truax, Barry, “Electroacoustic Music: The Inner and Conference, 2004.
Outer World,” In P.Seymour & T.Howell Eds., A
Companion to Contemporary Musical Thought (pp.
363-386). London: Ruthledge, 1992.
[6] Schafer, R. Murray, Voices of tyranny: temples o f
silence, Ontario, Canada: Arcana Editions, 1993.
[7] Cook, Perry, Music, cognition, and computerized
sound: an introduction to psychoacoustics, Perry R.
Cook Ed., Cambridge, Mass.: MIT Press, 1999.
[8] Bregman, Albert, “Foreward,” In G. Kramer Ed., Auditory
display: sonification, audification, and auditory
interfaces. Reading, Mass: Addison-Wesley, 1994.
[9] Ballas, J., “Common Factors in the Identification of an
Assortment of Brief Everyday Sounds,” Journal o f
Experimental Psychology: Human Perception and
Performance, vol.19(2), pp.250-267, 1993.
[10] Gaver, William, “Using and Creating Auditory Icons,”
In G. Kramer Ed., Auditory display: sonification,
audification, and auditory interfaces. Reading, Mass:
Addison-Wesley, 1994.
[11] Ong, Walter J., Orality and literacy: the technologizing
of the word. London; New York: Methuen, 1982.
[12] Bregman, Albert S., Auditory scene analysis: the
perceptual organization of sound. Cambridge, Mass.:
MIT Press, 1990.
[13] Attali, Jacques, Noise: the political economy of music.
Minneapolis: University of Minnesota Press, 1985.
[14] McLeary, Susan, “Afterword.” In Noise: the political
economy of music, Minneapolis: University of
Minnesota Press, 1985.
[15] Thompson, Emily Ann, The soundscape of modernity:
architectural acoustics and the culture of listening i n
America, 1900-1933, Cambridge, Mass.: MIT Press,
2002.
[16] Franklin, Ursula, “Silence and the Notion of the
Commons,” Soundscape: The Journal of Acoustic
Ecology, vol.1(2), pp.14-17, 2000.
[17] Metz, Christian, “Aural Objects,” In E. W. J. Belton Ed.,
Film Sound, Columbia University Press, 1985.
[18] Cage, John, Silence: lectures and writings, Wesleyan
pbk. Ed., Middletown, Conn.: Wesleyan University
Press, 1973.

ICAD05-168