The 21th International Conference

st International Conference on
on Auditory
Auditory Display
Display (ICAD
(ICAD–2015)
2015) July 8–10,
July 8-10, 2015, Graz, Austria

3DJ: A SUPERCOLLIDER FRAMEWORK FOR REAL-TIME SOUND SPATIALIZATION

Andres Perez-Lopez

[email protected]

ABSTRACT Spatial sound is a common element in the audiovisual industry

The field of real time sound spatizalization is recently receiving nowadays. An example of this fact might be the importance of spa-
much attention, as suggested by the large number of proposals ap- tial sound in cinema productions, or the existing commercial sur-
peared in last years - both from software spatialization frameworks rond 5.1, 7.1, 10.2 reproduction systems. Along with them, main
and from hardware spatialization interfaces. However, most of the Digital Audio Workstations allow spatial sound post-processing
proposed works do not take into account the existing knowledge through the use of plugins. In such cases, sound spatialization is
in Human Computer Interaction Design, which causes them to re- performed offline.
main in a simplified approach. We propose a theoretical basis for The usage of sound spatialization in a real-time (online)
real-time spatialization design from a holistic perspective, based environment is though still not fully exploited. Cannon and
on the Digital Musical Instruments theory, and use it to provide Favila point that this fact might be due to the “control complexity
a comparative review of recent proposals. Furthermore, we de- required to perform spatial motion” [3]. Nevertheless, in a study
velop our own state-of-the-art software spatialization system, 3Dj, carried by Peters et at., electroacoustic composers working with
which may help in the task of design and evaluation of new pro- spatial sound rated ”Spatial Rendering in Real-Time” as one of
posals for real-time sound spatialization in the fields of interactive the most desirable features for compositional frameworks [4].
performance, data sonification or virtual environments.
But the possibilities of live 3D sound are not limited to elec-
troacoustic music performace. Indeed, as the Special Theme of
1. INTRODUCTION
ICAD 2015 suggests (”ICAD in Space: interactive spatial sonifi-
cation”), real-time spatialization brings new perspectives and pos-
Spatial position is one of the perceivable sound aspects. However,
sibilities to a variety of fields such as exploratory data sonification,
in opposition to other qualities such as frequency or amplitude,
virtual/augmented reality environments or auditory display.
spatial position did never become a key element of western
musical language and theory, due to different reasons.
Despite that fact, several historical proposals which used space as 2. SPATIALIZATION SYSTEMS
a compositional element can be found. Polychoral Antiphony is
considered one of the oldest practices, which consisted of several 2.1. Abstract Representation of Spatial Sound
choirs singing simultaneously in different church locations;
Alessandro Striggios Missa sopra Ecco si beato giorno for five As mentioned earlier, two-channel stereophony (stereo) might be
choirs might be one of the most relevant pieces [1]. the most common sound spatialization technique nowadays. In
stereo panning, the level and the phase of each channel is adjusted,
Spatial sound has not adopted a main role with the ad- so that the perceived sound is positioned on an imaginary line be-
vent of recording and playback capabilities. The two-channel tween the speakers. The same technique is used in the modern
stereophony, which might be considered as the de facto standard surround systems.
speaker configuration, limits the sonic image to the line between However, such techniques present a great drawback: for
speakers, thus reducing drastically the spatial dimension of a reliable reproduction of existing spatial sound material, the
sound. Therefore, spatial sound was mainly explored from the speaker layout must be exactly the one for which the material
electroacoustic composition point of view - Edgar Vareses Poéme was produced. Additional speakers will not provide any extra
Electronique might be one of the most noticeable contributions. information in the playback stage, and less speakers will lead to an
information loss. Furthermore, a different speaker placement will
Since the 70s, an increasing amount of spatial audio devel- cause a positional distortion. Geier et al. classify such technique
opments have appeared. Both scientific aspects, as for example as channel-based [5], and we propose in addition the related term
the Ambisonics theory [2], and aesthetic ones, as can be the case layout-dependent.
of the Acousmonium, contributed to the growing interest in sound
difussion. Furthermore, the constant increase in computational On the other hand, object-based paradigm present a much
power also contributed to the widespread and adoption of spatial more flexible scenario. In that way, each different sound source
sound in different fields and contexts. is represented by an object, which is located in a virtual room sim-
ulator or Sound Scene [5], and contains both audio information
and position metadata (as in Figure 1). With such an abstraction
layer, it is possible to adapt the corresponding audio output to the
This work is licensed under Creative Commons Attribution Non current speaker layout (since it is a layout-independent paradigm).
Commercial 4.0 International License. The full terms of the License are Modern spatialization software usually presents an object-based
available at http://creativecommons.org/licenses/by-nc/4.0 approach due to its flexibility and conceptual easiness.

ICAD 2015 - 165

The 21th International Conference
st International Conference on
on Auditory
Auditory Display
Display (ICAD
(ICAD–2015)
2015) July 8–10,
July 8-10, 2015, Graz, Austria

azimuth: 90º
Schacher [8] presents a very interesting hierarchical perspec-
speaker positions tive on real-time sound spatialization, which borrows concepts
0º, 30º, 110ª, -110º, -30º
from 3D modelling. In his proposal, sound objects present their
own physical behavior, which may include both trajectories and
DECODER physical modelling. In fact, despite the lack of a canonical set
of dynamic behaviors, many authors have developed a variety of
proposals for predefined motions; Schmele offers a broad review
of these dynamic approaches [9, chapter 2.3].

Schacher further defines two complementary interaction

modes with the sound scene:
Top-down Users have a total direct control of the sound scene
(the most common approach).
Bottom-up Users interact from inside the sound scene; their ac-
tions behave according to the scene’s physical rules, thus
Figure 1: Channel-based, layout-dependent (left) vs. object-based,
losing an absolute control of the scene.
layout-independent (right) metaphors
Since sound object parameters are described by metadata, it
may be possible to think of a common, system-independent scene
2.2. Real-Time Spatialization Systems description format. The SpatDIF protocol [10] is a mature pro-
posal towards the standarization of sound scene description, stor-
In the object-based approach, the sound spatialization task can be age and transmission. Such protocol may overcome the historical
divided into two individual domains: a Scene Simulator, which lack of system interoperability [5, 10].
describes the spatial position of the objects, and a Spatial Render,
which actually synthesizes the corresponding sound according to 2.3. Comparative of Real-Time Spatialization Systems
the used spatialization technique (VBAP, Ambisonics, WFS...).
We thus use the concept of Spatialization System as a system Table 1 presents a review of the most relevant current systems for
consisting of a scene simulator and a spatial render. The schema is real-time sound spatialization, and compares them with respect
represented in Figure 2. to the parameters and features presented in Section 2.2. Spatial-
ization systems are organized according to their implementation
characteristics. Systems located at the left side are standalone
SPATIALIZATION SYSTEM
applications: Zirkonium [11], Spat [12], Sound Scape Render [13]
and Sound Element Spatializer [14]. Systems on the right side are
SOUND SCENE based on existing sound processing environments: BEASTMulch
[15] (SuperCollider), Spatium [16] (Max/MSP) and OMPrisma
audio
OBJECT 1 SPATIAL PLAYBACK
[17] (OpenMusic).
... RENDER spatialized SYSTEM
metadata audio
From the table we can extract some conclusions:
OBJECT N

• There is a general lack of dynamic behavioural features.

When implemented, the systems only provide a small subset
of predefined motions, and physical modelling is unsatisfac-
Figure 2: Spatialization System structure tory in terms of appearance.
• None of the systems present a hierarchical structure of sound
As pointed by Marshall et al., sound spatialization is a multi- objects. Therefore, they only feature the top-down interaction
dimensional system, featuring 2D spatial position and volume pa- mode.
rameters for each sound object in the simplest case [6]. He further • All systems are highly dependent on OSX operating systems.
proposes the following taxonomy of spatialization system control In fact, the only two systems which are also compatible with
parameters: GNU/Linux presented major problems: SSR only allows 2D
spatialization, among other limitations, and SES is not avail-
• Source position: 2D / 3D spatial coordinates able.
• Source characteristics: Size, directivity, presence, brillance • I must finally highlight the importance of Free Software im-
• Environmental parameters: Room size, reverberation, plementations, which is partially covered by the systems. Free
doppler effect, equalization, air absorption, distance decay software licenses ensure the availability of code and the po-
tential for improvement, as well as provide the basis for exper-
McGee and Wright [7] extend the taxonomy by including imental replications; all these characteristics are considered as
some desirable system features. Among them, we can highlight fundamental for a scientific research environment.
configurable speaker setup (layout-independent paradigm), arbi-
trary number of sources, and support for diverse spatialization
techniques.

ICAD 2015 - 166

The 21th International Conference
st International Conference on
on Auditory
Auditory Display
Display (ICAD
(ICAD–2015)
2015) July 8–10,
July 8-10, 2015, Graz, Austria

Table 1: Comparative of Real-Time Spatialization System Software

Standalone systems SPE based systems
Zirkonium Spat SSR SES BEAST Spatium OM Prisma
Mulch
3D position ⇥
Source size ? ⇥ ? ? ?
Source directivity ? ? ⇥ ⇥ ? ? ?
Room parameters ? ⇥ ⇥ ?
Distance cue ⇥ ?
Configurable
speaker setup
Arbitrary number ? ? ? ? ⇥ (16) ?
sound sources
Behavior support ? ? ⇥ ⇥ ?
Hierarchies support ⇥ ⇥ ⇥ ⇥ ⇥ ⇥ ⇥
Render type VBAP HRTF HRTF DBAP ? VBAP VBAP
VBAP VBAP VBAP HOA* HOA
HOA HOA* HOA WFS DBAM
WFS ViMiC
OSC ⇥ ? ?
Description format ? SpatDIF ASDF SpatDIF ? ? SpatDIF
Platform OSX OSX Linux/OSX Linux/OSX OSX OSX OSX
License BSD proprietary GPL proprietary GPL CC LGPL

INSTRUMENT

gestures
GESTURAL mapping SOUND
PERFORMER
CONTROLLER SYNTHESIS
primary
feedback

secondary
feedback

Figure 3: Digital Musical Instrument (by [18])

ICAD 2015 - 167

The 21th International Conference
st International Conference on
on Auditory
Auditory Display
Display (ICAD
(ICAD–2015)
2015) July 8–10,
July 8-10, 2015, Graz, Austria

3. DIGITAL MUSICAL INSTRUMENTS 3.3. DMI for Sound Spatialization

Marshall et al. explore theoretically the problem of interactive
3.1. Concept
sound spatialization [6]. They propose a performer role classifi-
We commonly understand traditional Musical Instruments as de- cation, based on the required cognitive load.
vices capable to produce live music, which transform the per- • Performer controls only sound spatialization.
former gestures into sound events in real-time [19].
• Performer controls both sound synthesis and spatialization.
On the other hand, the Digital Musical Instrument (DMI) con-
cept [18] defines a device in which the gestural interface is com- • Performer directly controls sound synthesis, and spatializa-
pletely decoupled from the sound synthesis1 ; their control parame- tion is performed indirectly or unconsciously.
ters are related through mapping strategies. Furthermore, the pro-
duced sound may be potentially any imaginable one, since it is 3.4. Review of Spatialization DMI
not restricted by the physical and acoustical constraints of conven-
tional instruments [20]. Table 2 performs a comparative analysis of DMI for sound spa-
Figure 3 schematizes the structure of a Digital Musical tialization, according to the parameters commented in this section.
Instrument, as proposed by Wanderley [18]. Selected DMI proposals are taken from recent NIME and ICMC
conferences: Bokowiec [23], Bredies [24], Cannon [3], Carami-
In words of Jordà, “performing music with ‘intelligent de- aux [25], Carlson [26], Fohl [27], Johnson [28], Marentakis [29],
vices’ [Digital Musical Instruments] tends towards an interactive Marshall [30, 6], Nixdorf [31] and Park [32].
dialogue between instrument and instrumentalist” [20]. We can extract some common trends by analysing Table 2.
Interactivity is, according to Winkler [21], a continuous qual- Most of the comments may be explained by the lack of background
ity, determined by both the amount of freedom given to the per- in interaction and DMI design fields.
former, and the computer’s ability to respond in an appropriate • Most of the proposals do not take into account the multi-
manner. That response is represented in Figure 3 via the feedback thread/shared control paradigms; in Bokowiec [23], they are
arrows: described as unusual features. This fact is related with the
• Primary feedback: Visual, tactile and/or haptic feedback pro- overall simple 2D position control, avoiding more complex
vided by the input controller. input modalities and parameter mappings.
• Secondary feedback: Auditive feedback provided by the • There is a general trend on relating individual performances
sound generator. with potential trained users, and multi-user performances with
casual, non-expert performers.
It is also possible to classify feedback according to its physical
• Visual active feedback is preferred for all the implementa-
nature: passive, when it is a consequence of the physical charac-
tions. Only Carlson [26] offers haptic active feedback, in his
teristics of the instrument; or active, as if the response follows a
non-standard proposal.
predefined pattern.
• Most of the proposals did not present or mention any evalua-
tion. This is due to various reasons, among them, the lack of a
3.2. Design Considerations standard evaluation methodology, and the prototype character
of most of proposals may be pointed out.
Several aspects might be taken into account for a proper interaction
design. Table 3 provides a review from the same spatialization DMI
proposals, in this case compared according to the parameters of
Multithread - shared control When many control parameters the spatialization system used.
are available, the performer may not have the capability The following comments can be extracted from Table 3:
of controlling every parameter simultaneously. In this sce-
nario, multithread and shared control paradigms [22] pro- • First of all, there is a general lack of full periphonic (3D) sup-
vide the capacity of taking and leaving the control of pro- port, in contrast with the capabilities of the spatialization sys-
cesses at will, ensuring that their normal behaviour is con- tems. This is probably due to various reasons, which can be
tinued without the user’s direct control. the lack of 3D interfaces [3], the technical limitations (Wave
Field Synthesis), or simply the control complexity.
Intended users The interactive experience may vary depending • The historical trend towards direct position control is fol-
on the user approach: casual users expect only enjoying lowed. Even when most of the proposals also allow trajec-
a positive experience, but trained ones might search some tory control, the use of high-level physical abstractions might
kind of expressivity [19]. lead to innovative proposals, as in the case of the Caramiaux’s
Number of performers Traditionally, digital musical instru- particle system [25].
ments have been designed for a single trained performer. • Only the Sound Flinger by Carlson et al. [26] proposes a
However, in special cases, multiple performers are consid- bottom-up interaction mode, in which the user is located in-
ered, as in the context of interactive installations. side the sound scene. All other proposals follow the tradi-
tional top-down approach.
1 The existence of digital musical instruments whose control and syn-
thesis modules might be coupled is possible. However, in that case, such
instrument could not be considered a Digital Musical Instrument.

ICAD 2015 - 168

The 21th International Conference
st International Conference on
on Auditory
Auditory Display
Display (ICAD
(ICAD–2015)
2015) July 8–10,
July 8-10, 2015, Graz, Austria

Table 2: Comparison of Spatialization Instruments according to DMI dessign

Multithread Intended Number of Rol of Interface Active Evaluation
Shared control User performers performer feedback
Bokowiec trained 1 spatialization & synthesis gesture tracker ⇥ ⇥
Bredies ⇥ casual 1/many spatialization tabletop visual ⇥
Cannon ⇥ trained 1 spatialization & synthesis extended visual ⇥
Caramiaux ⇥ casual 1 spatialization & synthesis gesture tracker visual ⇥
Carlson ⇥ both 1 spatialization slider visual/haptic ⇥
Fohl ? 1 spatialization gloves ⇥ ⇥
Johnson ⇥ casual 1/many spatialization tabletop visual
Marentakis ⇥ trained 1 spatialization & synthesis extended ⇥ ⇥
Marshall1 ⇥ trained 1 spatialization gesture tracker ⇥ ⇥
Marshall2 ⇥ trained 1 spatialization gesture tracker visual ⇥
Nixdorf ⇥ trained 1 spatialization ? visual
Park ⇥ both 1/many spatialization smartphone visual

Table 3: Comparison of Spatialization Instruments according to spatialization system parameters

Periphonic Control Trajectories Hierarchies Interaction Spatialization Spatialization
parameters mode technique system
Bokowiec ⇥ position & trajectories ⇥ top-down ? ⇥
Bredies ⇥ position (groups) top-down (SSR) SSR
Cannon ⇥ position & trajectories ⇥ top-down FOA ⇥
Caramiaux ⇥ physical model ⇥ top-down WFS / VBAP ⇥
Carlson ⇥ physical model ⇥ ⇥ bottom-up VBAP ⇥
Fohl ⇥ position & trajectories ⇥ top-down WFS ⇥
Johnson ⇥ position ⇥ ⇥ top-down VBAP ⇥
Marentakis ⇥ position ⇥ ⇥ top-down VBAP ⇥
Marshall1 position & environmental ⇥ top-down ViMiC ⇥
Marshall2 position & environmental ⇥ ⇥ top-down ? ⇥
Nixdorf ⇥ position ⇥ ⇥ top-down ? ⇥
Park ⇥ position / casual ⇥ ⇥ top-down custom ⇥

SPATIALIZATION INSTRUMENT

SPATIALIZATION SYSTEM

SOUND
PROCESSOR

STORAGE /
SOUND SCENE
TRANSMISSION
REPRESENTATION
USER 1
BEHAVIORS
OBJECTS
gestures TRAJECTORIES
... USER FORMAT RENDER PLAYBACK
MAPPING
INTERFACE SYSTEM
HIERARCHICAL PHYSICAL
RELATIONSHIPS MODEL
USER N
passive
feedback
ACTIVE
FEEDBACK
GENERATOR

active
feedback

auditive
feedback

Figure 4: Proposed Spatialization Instrument schema

ICAD 2015 - 169

The 21th International Conference
st International Conference on
on Auditory
Auditory Display
Display (ICAD
(ICAD–2015)
2015) July 8–10,
July 8-10, 2015, Graz, Austria

• Among the spatial sound techniques used, there is only one

proposal using First Order Ambisonics; none of them uses
Higher Order Ambisonics.
• It is very interesting to observe the general lack of usage of
spatialization system frameworks, with the exception of the
Bredies [24], in which a spatialization system is used (SSR,
which is in fact developed by the same authors). That sit-
uation causes the interfaces not to reuse all the capabilities
provided by the spatialization systems.
• Finally, we must remark the difficulty of replicating those
Spatialization Instruments. In the case of control interfaces, Figure 5: 3Dj internal structure
it is clear that such replicability may be difficult or expensive,
specially when using custom hardware. However, a common
spatialization system might contribute to the Spatialization 5.1. Design Specifications
Instrument reproducibility. That fact might help to ease the Device independence To provide compatibility with user inter-
creation and usage of standard evaluation methodologies for face protocols, such as HID, MIDI and OSC.
Spatialization Instruments.
Flexible mapping To allow arbitrarily complex relations between
input gestures and parameters.
4. SPATIALIZATION INSTRUMENTS
Control parameters To provide a variety of potentially relevant
control parameters, as reviewed in Section 2.2.
As commented in Section 3.4, instrument designs for sound spa-
tialization usually lack the knowledge of the state of the art of in- Feedback To integrate visual feedback, and to allow other kinds
teraction design, adopting a narrow perspective into the problem. of active feedback by the user interface protocols.
In our opinion, a holistic approach is preferred and, consequently, Spatial render To provide the tools for using different sound spa-
we present the concept of Spatialization Instruments: tialization techniques, such as VBAP, HOA, WFS or Bin-
aural.
A Spatialization Instrument is a Digital Musical
Instrument, which has the ability of manipulating Exchange format To be compatible with scene description for-
the spatial dimension of the produced sound, mats, such as SpatDIF.
independently of its capability of producing or Modularity The use of standard formats and protocols provide
manipulating other sound dimensions2,3 . the basis for software modularity, which might be useful
for distance or high computing load scenarios.
Figure 4 depicts a conceptual structure of the parts composing Free Software The free spreading, modification and understand-
a Spatialization Instrument. We can appreciate all separate compo- ing of the software is a compulsory requisite for research
nents described in Sections 2 and 3, and also how are they linked tools. Furthermore, multiplatform compatibility may be
to each other. useful for a wide software adoption.
We must point out that the idea of Spatialization Instruments
is a conceptual approach for the design of DMI for sound spatial- 5.2. Implementation: 3Dj
ization. Therefore, instruments presented in Section 3.4 may be
considered Spatialization Instruments. In order to implement the desired features, we opted for the Su-
perCollider environment [33]. SuperCollider is a real-time audio
processor and an object-oriented programming language. It is Free
5. SOFTWARE IMPLEMENTATION Software and multiplatform, and embraced by a big community of
technicians and artists, both users and developers.
Previous sections highlighted the fact that existing spatialization Therefore, our spatialization system takes the form of a
systems do not provide all desired control and interactivity. Fur- SuperCollider quark (i.e. external library), called 3Dj. The code
thermore, their availability in terms of cost or multiplatform com- is available through the internal quark installation system, and in
patibility is not always ensured. This is the motivation to imple- its code repository [34].
ment our own spatialization system, with the objective of serving
as a tool for Spatialization Instruments development. Moreover, its 3Dj follows a modular structure, as can be seen in Figure 5.
adoption might help in the replication of the design and evaluation. The main building blocks are the Sound Scene and the Spatial
Render, which are connected though a custom implementation of
2 A Spatialization Instrument is a DMI in the sense that it shows the SpatDIF through OSC.
DMI characteristics: interface/sound synthesis decoupling, and potential
to produce any kind of sound. In that case, any kind of sound refers to the The Sound Scene provides the abstraction layer for managing
fact that the sound position might be any desired, not restricted to physical the sound objects. It provides a simple acoustic room model, and a
constraints, and controlled by the user with arbitrary mappings.
3 According to the physical laws, any changes in a sound’s position physical model, with parameters such as gravity, medium viscosity
will imply temporal and spectral changes as well. We refer here only to the or wall friction; it is also possible to interact with the objects by
capability to modify non-spatial sound dimensions, in the sense that these exerting forces to them, thus supporting the bottom-up interaction
other sound dimensions are control parameters of the sound synthesis unit. mode.

ICAD 2015 - 170

The 21th International Conference
st International Conference on
on Auditory
Auditory Display
Display (ICAD
(ICAD–2015)
2015) July 8–10,
July 8-10, 2015, Graz, Austria

Nonetheless, the Auditory Display and Sonification commu-

nities may also greatly benefit from the framework. 3Dj, through
the SuperCollider scripting language, provides the means to eas-
ily perform any arbitrarily complex real-time spatialization task.
This feature might be particularly useful for virtual environments,
augmented reality or live data sonification.
In addition to that, we must take into account that SuperCol-
lider itself is a highly optimized environment for sound processing
and synthesis. Accordingly, any desired sound aspect might be
controlled by 3Dj’s SuperCollider instance itself, reducing the
overall system complexity and interdependency.

It is also true that 3Dj (in fact, the SuperCollider language)

Figure 6: 3Dj environment screenshot may have a longer learning curve than other spatialization frame-
works, specially those ones in which user interaction is based on
Graphical User Interfaces or Graphical Programming Languages.
A manual is conveniently provided for end users [34].
Object grouping and joint management are provided by the
But the initial difficulty is greatly compensated by the arbi-
SuperCollider language features. Furthermore, there is a set
trary complexity of sound spatialization mappings and behaviors
of predefined motions that an object can adopt: for instance,
that might be reached, which surpass features of other spatializa-
linear, brownian, simple harmonic or orbital. A simple graphical
tion frameworks. Furthermore, the SuperCollider characteristics
representation of the sound scene is generated, in order to provide
provide many appealing features: native capabilities of real-time
direct active feedback (Fig. 6). The sound scene state is saved
networked remote sound synthesis, native OSC integration, or (as
at a configurable rate, formatted into SpatDIF format, and sent
already mentioned) integration with state-of-the-art sound process-
through OSC to the destination Spatial Render.
ing, synthesis and classification capabilities.
The Spatial Render receives the SpatDIF information, parses
it and synthesizes the sound accordingly. Three sound spatializa- 6. CONCLUSIONS AND FUTURE WORK
tion techniques are available: VBAP, HRTF Binaural, and HOA.
This study contributes to the state of the art of real-time sound
In the case of the latter which is provided by the AmbEnc
spatialization in the following manners:
module, the generated audio is in intermediate Ambisonics for-
mat, which must be further processed by an Ambisonics De- • Proposing the Spatialization Instrument concept as a way to
coder; in our case, we provided the wrapper for AmbDec [35] for integrate the existing knowledge in interaction design into
GNU/Linux systems. Moreover, HOA provides the use of a vari- real-time spatialization domain, thus analysing the problem
ery of experimental sound source shapes, different from punctual from a holistic approach.
ones, such as parallel, meridian, or custom spherical surfaces. • Creating a critical review of most recent spatialization sys-
Furthermore, the Spatial Render provides SpatDIF logger and tems and Spatialization Instruments.
playback capabilities, in order to record the incoming messages,
• Developing a State-of-Art framework for live spatialization.
as well as the generated ones, and replay them; this feature might
specially be of interest in the case of non-live spatialization. Regarding the future work, a variety of proposals can be car-
ried in the context of our research:
5.3. Discussion • To develop different Spatialization Instruments, which may
be adequate for diverse use scenarios.
In this Section we presented our framework for real-time sound • To research and to develop an evaluation methodology for
spatialization. 3Dj is built upon the desirable features for spatial- spatialization instruments; use our spatialization system to
ization systems, as described in Section 2. Furthermore, there is a perform a case-study evaluation.
special focus on the application of that systems for the interactive
performance, based on the contents developed in Sections 3 and 4.
7. ACKNOWLEDGMENTS
Therefore, the tool is primarily addressed to those develop-
Thanks to Barcelona Media for allowing the usage of facilities,
ers working in Human-Computer Interaction Design and Spatial
which was fundamental for the software development. Thanks also
Sound. In that way, they might benefit from a very flexible, cus-
to Davide Scaini (Department of Information and Communica-
tomizable and open-source framework with unique features, and
tion Technologies, Universitat Pompeu Fabra) and Daniel Arteaga
center their efforts in the design tasks. The same concept might
(Dolby Iberia) for their help in the development process
also apply to composers, performers and artistic designers.
The Free Software distribution model of 3Dj (through the
General Public License) facilitates design and experimental
replicability, as already commented. The usage of the SpatDIF
protocol, together with its storage and playback capabilities,
contributes as well to that goal.

ICAD 2015 - 171

The 21th International Conference
st International Conference on
on Auditory
Auditory Display
Display (ICAD
(ICAD–2015)
2015) July 8–10,
July 8-10, 2015, Graz, Austria

8. REFERENCES [21] T. Winkler, “Making motion musical: Gesture mapping

strategies for interactive computer music,” in ICMC Proceed-
[1] D. Moroney, “Alessandro striggio’s mass in forty and sixty ings, 1995, pp. 261–264.
parts,” Journal of the American Musicological Society, vol.
[22] J. Chadabe, “Interactive composing: An overview,” Com-
110, no. 3, pp. 1598–1608, March 2007.
puter Music Journal, pp. 22–27, 1984.
[2] M. A. Gerzon, “Periphony: With-height sound reproduc-
tion,” Journal of the Audio Engineering Society, vol. 21, [23] M. A. Bokowiec, “Voct (ritual): An interactive vocal work
no. 1, pp. 2–10, 1973. for bodycoder system and 8 channel spatialization,” in Pro-
ceedings of the International Conference on New Interfaces
[3] J. Cannon and S. Favilla, “Expression and spatial motion: for Musical Expression, 2011.
Playable ambisonics,” in Proceedings of the International
Conference on New Interfaces for Musical Expression, 2010, [24] K. Bredies, N. A. Mann, J. Ahrens, M. Geier, S. Spors, and
pp. 120–124. M. Nischt, “The multi-touch soundscape renderer,” in Pro-
ceedings of the working conference on Advanced visual in-
[4] N. Peters, G. Marentakis, and S. McAdams, “Current tech- terfaces. ACM, 2008, pp. 466–469.
nologies and compositional practices for spatialization: A
qualitative and quantitative analysis,” Computer Music Jour- [25] B. Caramiaux, S. F. Alaoui, T. Bouchara, G. Parseihian,
nal, vol. 35, no. 1, pp. 10–27, 2011. M. Rébillat, et al., “Gestural auditory and visual interactive
platform,” in Proceedings of the 14th International Confer-
[5] M. Geier, J. Ahrens, and S. Spors, “Object-based audio re-
ence on Digital Audio Effects (DAFx-11), 2011.
production and the audio scene description format,” Organ-
ised Sound, vol. 15, no. 03, pp. 219–227, 2010. [26] C. Carlson, E. Marschner, and H. McCurry, “The sound
flinger: A haptic spatializer,” in Proceedings of International
[6] M. T. Marshall, J. Malloch, and M. M. Wanderley, “Gesture
Conference on New Interfaces for Musical Expression, 2011,
control of sound spatialization for live musical performance,”
pp. 138–139.
in Gesture-Based Human-Computer Interaction and Simula-
tion. Springer, 2009, pp. 227–238. [27] W. Fohl and M. Nogalski, “A gesture control interface for a
[7] R. McGee and M. Wright, Sound element spatializer. Ann wave field synthesis system,” in International Conference on
Arbor, MI: MPublishing, University of Michigan Library, New Interfaces for Musical Expression, 2013.
2011. [28] B. Johnson and A. Kapur, “Multi-touch interfaces for phan-
[8] J. C. Schacher, “Gesture control of sounds in 3d space,” in tom source positioning in live sound diffusion,” in Proceed-
Proceedings of the 7th international conference on New in- ings of International Conference on New Interfaces for Mu-
terfaces for musical expression. ACM, 2007, pp. 358–362. sical Expression, 2013.
[9] T. Schmele, “Exploring 3d audio as a new musical lan- [29] G. Marentakis, N. Peters, and S. McAdams, “Dj spat: Spa-
guage,” Ph.D. dissertation, Masters thesis, Universitat Pom- tialized interactions for djs,” in Proceedings of the 2007 In-
peu Fabra, 2011. ternational Computer Music Conference (ICMC), 2007, pp.
360–363.
[10] N. Peters, T. Lossius, and J. C. Schacher, “The spatial sound
description interchange format: Principles, specification, and [30] M. Marshall, N. Peters, A. R. Jensenius, J. Boissinot, M. M.
examples,” Computer Music Journal, vol. 37, no. 1, pp. 11– Wanderley, and J. Braasch, “On the development of a system
22, 2013. for gesture control of spatialization,” 2006.
[11] Zirkonium. http://www.zkm.de/zirkonium. [31] J. J. Nixdorf and D. Gerhard, “Real-time sound source spa-
[12] Spat. http://forumnet.ircam.fr/product/spat/. tialization as used in challenging bodies: implementation and
performance,” in Proceedings of the 2006 conference on New
[13] M. Geier and S. Spors, “Spatial audio with the soundscape interfaces for musical expression. IRCAMCentre Pompi-
renderer,” in In VDT Convention, 2012. dou, 2006, pp. 318–321.
[14] R. M. McGee and M. Wright, “Sound element spatializer,” [32] S. Park, S. Ban, D. Hong, and W. Yeo, “Sound surfing net-
in n Proceedings of the International Computer Music Con- work (ssn): Mobile phone-based sound spatialization with
ference 2011, 2011. audience collaboration,” in New Interfaces for Musical Ex-
[15] Beastmulch. http://www.birmingham.ac.uk/facilities/ pression, vol. 2013, 2013, pp. 111–114.
BEAST/research/mulch.aspx.
[33] J. McCartney, “Rethinking the computer music language:
[16] Spatium. http://spatium.ruipenha.pt/. Supercollider,” Computer Music Journal, vol. 26, no. 4, pp.
[17] Omprisma. http://www.idmil.org/software/omprisma. 61–68, 2002.
[18] M. M. Wanderley, “Gestural control of music,” in Interna- [34] 3Dj quark. https://github.com/andresperezlopez/3Dj.
tional Workshop Human Supervision and Control in Engi- [35] F. Adriaensen, “Ambdec 0.4. 2 user manual,” kokkinizita. lin-
neering and Music, 2001, pp. 632–644. uxaudio. org, 2011.
[19] S. Jordà, “Digital instruments and players: Part ii–diversity,
freedom and control,” in Proceedings of the International
Computer Music Conference, 2004, pp. 706–710.
[20] N. Collins and J. d’Escrivan, “Interactivity and live computer
music,” in The Cambridge Companion to Electronic Music.
Cambridge University Press, p. 312.

ICAD 2015 - 172