FEATURE ARTICLE

Audio for cinema

Dialog recording and future trends
Francis Rumsey
Consultant Technical Writer

Dialog recording, editing, and replacement is probably one of the most

important aspects of movie sound production. It is the basis of good
storytelling, as poor dialog is the “best way to ruin a movie.” Panelists during
Audio for Cinema sessions at the 143rd Convention tackle this fascinating
topic. Afterwards, a panel chaired by Nuno Fonseca debated the future
challenges of audio for cinema.

A udio for Cinema

was an important
feature of the 143rd
meant by that was that although you can
turn up with the best equipment possible,
many of the factors that affect how the dia-
rate microphone signals for any given produc-
tion mix, said Peter, from which a mix track
is created, as well as the original microphone
Convention, which log will actually turn out in audio terms are signals being recorded individually for any
took place last fall in affected by other things. Locations are criti- later processing. Often the audio mixers used
New York. Organized cal, for example, and there is much to be said are traditional analog devices, but there is an
by Nuno Fonseca, the for sending a production sound mixer along increasing trend toward using multichannel
convention track on with a location scout to look at venues. If the sound cards feeding digital systems with asso-
that topic covered a Nuno Fonseca location is very noisy, for example, the basic ciated control surfaces. “It’s all about signal
number of different rules of signal to noise still apply and there to noise ratio” said Peter, suggesting that the
themes in cinema audio, including a substan- may be little the audio engineer can do to purpose of having all of those microphones is
tial workshop on dialog in film and television make it right. Wardrobe choices are another to find suitable sources at any given time that
and a panel session on the future of audio for factor. If talent is wearing “difficult to wire” maximize the chance of picking up the words
cinema. wardrobe it presents numerous challenges spoken above background sounds. That said,
Panelists Bobby Johanson, Peter to the recording crew, many of which might the aim is to make a mix track that matches
Schneider, and Alexa Zimmerman went into have been avoided if one had been involved in the camera perspective, he said, but one that
the process of supporting the storytelling some of the decisions. Furthermore, not all keeps the background levels consistent. The
that is done with dialog, looking at produc- actors project their voices well and may talk human ear tends to notice changes in back-
tion and postproduction sound, it being well indistinctly. Having the director on side with ground level very easily, as they are unnatural,
known that poor dialog intelligibility is the the sound crew is of great value, and direc- and this is one of the typical things you notice
“best way to ruin a movie.” tors that understand the implications of their on student films, for example, where they
decisions for sound are very valuable. haven’t taken this factor into account.
PRODUCTION DIALOG Boom microphones will normally be in the Most production sound mixers seem to be
Peter Schneider of Gotham Sound started out hands of a boom operator, wireless mics will big fans of the sound of boom mics, said Peter,
by pointing out that 80% of the production be located on the clothes or body of actors, even though there is a big push for greater use
dialog options and quality are determined and “plant” microphones hidden somewhere of lavalier or lapel mics on the actors them-
before day one of the production. What he on the set. There could easily be 16 or 17 sepa- selves. His preference is partly because the

182 J. Audio Eng. Soc., Vol. 66, No. 3, 2018 March

 FEATURE ARTICLE

microphones on booms mix algorithm, and he gallantly admitted that tricky as one can be constantly making adjust-
are typically better than he thought the automix had probably done a ments in an attempt to follow.
lapel mics, and partly better job, being more seamless in its results.
because the sound from DIALOG REPLACEMENT
them naturally matches DIALOG EDITING AND RE-RERECORDING
the camera perspec- Alexa Zimmerman, a dialog and ADR editor Bobby Johanson explained that his techniques
tive when handled by a on the panel, briefly explained what a sound as a re-recording mixer have evolved not to
skilled operator. That editor does. Once the production sound has pay too much attention to the static environ-
Peter Schneider
said, there are lots of been captured on the set and handed over ment where a re-recording is being made.
situations where lavalier mics simply have to sound postproduction, the sound editor His job is to recreate the sound captured by
to be used, such as on a busy street where begins to work on that content to make it production sound people like Peter Schnei-
the noise levels are very high, or on sets with sound “more alive” and more intelligible. der, who spoke earlier, which means that one
odd reflections or surfaces where you simply Alexa has had to remove the sound of high- may need a large space to offer the flexibility
can’t get a boom mic close enough. Later on tech equipment from dialog that was sup- for creating close and distant sounds. Space is
Peter added that some new lavalier micro- posed to have been shot in the 1800s for also important for an actor to perform prop-
phones coming out at the moment have quite example. She has also woven together var- erly, offering the space to move around and
remarkable improvements in their specifica- ious lines from a documentary to tell the project into, as actors have the challenge of
tions, one brand offering a 14-dB improve- story that the director wants. In some cases getting back into character and redoing what
ment in dynamic range, for example. There’s there are good original takes from produc- they did on set convincingly. These factors
also a growth in real time noise reduction tion sound that the director really likes, but make it not so important to have a typical
algorithms that can be used in the recording that have some audio problem, and in those “perfect” recording studio, because dialog
chain, although it can be difficult to get the cases it may be possible for the sound editor recording is a messier business than music
production budget to pay for it. to use a combination of modern signal pro- recording and imperfections are character-
Another challenge in production sound is cessing and editing to make it work. This has istic of original set recordings too. Often the
that there’s been a push to shoot using multi- become easier, but when it’s really not possi- mic techniques have an ambient recording
ple cameras at the same time on some sets, ble one can go to an ADR stage and re-record. nature rather than being close, particularly
where it’s thought to be more economical to Although the dialog editor’s job is not really when trying to capture groups of actors try-
do so. The problem is that when one camera to “direct” an actor in such cases, there may ing to emulate a previously quiet “crowd of
is on a wide shot where another is close in, be subtle ways in which one can use technical thousands,” and there can be quite a lot of
it really compromises the sound. It can be requirements to get them to speak in the way moving around in a sound stage as people
impossible to use a boom mic, for example, that’s needed, Alexa suggested. There are also come and go. Bobby said that these record-
because it would be in the shot for one of situations where key talent is speaking while ings are basically a multitrack “mono gig,”
the cameras. A way of ameliorating this for walking through a “crowd of thousands,” and but he has used stereo MS pairs and other
the sound guys, if there is an insistence on in most such cases the crowd is kept fairly stereo techniques.
using multiple simultaneous cameras, is to quiet when recording on set. That makes it Looping is a common traditional tech-
try to shoot multiple angles together rather necessary to go back and recreate the sound nique used in automatic dialog replacement
than wide and close shots. Shows like “House of the crowd and generate the sonic atmo- (ADR), said Bobby, whereby a loop of the
of Cards,” though, use computer-generated sphere of the scene, making a busy sidewalk original sound and picture is played repeat-
imagery (CGI) techniques to “paint out” the feel like one, say. edly, enabling a re-recording actor to get in
boom microphone in many shots, which is Alexa said that the pitch of the actor’s voice the groove for a particular attempt to sync
possibly one reason that it sounds as good as it is one of the factors about which she has new dialog. They might do this with sound
does, suggested Peter. Lavalier mics have also something useful to say. Matching the original only in their headphones, or with a picture
been painted out in some cases, so this is an pitch and inflections when replacing dialog too. Another widely used method is to cue
increasingly useful tool in the box and it can is particularly hard, when an actor may be actors with three beeps, with them supposed
be wise to have a word with the visual effects struggling to get back into a part. When an to start on the virtual fourth beep. A one-sided
supervisor to discover whether such resources actor has spent perhaps six months getting headphone is usually used in these cases, as
might be available on a project. into a particular voice sound or dialect, it the actors can hear their own voices natu-
Interestingly, Peter was complimentary can be difficult to pick it up again for a re- rally alongside the original. With replacement
about automatic mixing for the purpose recording session. Sometimes there are audio dialog the performance is probably the most
of creating daily mix tracks for production signal processing tools that may help, such as important
dialog. When he does it manually each acting pitch shifters, to match the re-recorded mate- factor and it’s
character is highlighted in the script in differ- rial a bit more closely to production dialog. remarkable
ent colors and he moves faders up and down Matching the phase of a boom and lavalier how often
in real time, deciding who to bring up in the microphone is also something that she works one can get
mix, or sometimes who not to bring up when to achieve, as it helps to improve the sound away with
everyone is speaking at once. He once did a when mixing the two. When either the actors some words
“head to head” challenge with an automatic or the microphones move, though, this can be Bobby Johanson in the audio

J. Audio Eng. Soc., Vol. 66, No. 3, 2018 March 183

FEATURE ARTICLE

being different to those originally spoken on are taking over from traditional methods metadata and
the picture. As many will know there is a that were used to avoid picking up unwanted file naming
strong “McGurk effect” that leads one to hear sounds in the first place. In the past one might conventions
the word or syllable that is seen rather than have had various tricks to help actors control had already
the one that is heard. Even so, if you look the amount of spitting and clicking sounds been referred
closely, said Bobby, you can often see this made by their mouths, but these days it is to as a key
Kevin Collier
phenomenon in film and TV productions. It’s relatively straightforward to put such problem factor by the
also possible simply to replace a particular tracks through an algorithm of some sort to dialog audio panelists
syllable or sibilance in a word, and quite often fix it. Similar approaches apply to things like previously. Creative
this does the trick. headphone bleed that is breaking through engineers are spending
Bobby pointed out that it can often be quite into an actor’s microphone, which can now some 20% of their time
hard to decide which sound to match when be reduced afterwards without too much diffi- simply on preservation
doing replacement dialog. He’s often listening culty, suggested Alexa. There are also remark- of the assets in this
to about three things at the same time and able possibilities for EQ matching between way, as well as having
trying to decide how to match to the original an original recording and a replacement one, to solve software prob- Douglas Greenfield
mix track. He’ll usually use both a boom and using automatic analysis to work out the spec- lems, and there was a
a lavalier mic for re-recording, so that both tral balance of one recording and apply it to strong wish to have these matters work in a
can be provided for the final mix. Sometimes a another. more seamless fashion so that the time could
decision in later mixing can be just to use, say, be spent on creative work. Standards for the
one of the original microphone tracks such FUTURE OF AUDIO FOR CINEMA interchange of metadata are relatively slow
as the lavalier, and it’s hard to guess how that Chair of the Audio for Cinema track, Nuno in being implemented, but having something
might work out. Every so often it’s possible Fonseca, introduced panelists Kevin Collier, like that, handled the same by different soft-
to get hold of a dialog pre-mix before doing Douglas Greenfield, Avi Laniado, and Brian ware packages, would be a great boon.
re-recording, which makes a world of differ- Vessa to discuss the future of audio for cinema. Brian Vessa noted that there is a wide diver-
ence to the decisions that have to be made. One of the things that has really changed gence in the quality of sound reproduced in
Jobs of re-recordists and dialog editors can in the last ten years, it was said, is the remark- different cinemas. Some basic installations
be anything from recording a simple voice- able number of separate tracks that can be found in small towns in the USA, for exam-
over to knocking out multiple swear words handled at once by computer audio systems. ple, are not really cut out for dealing with
from dialog for airline versions of movies or This, coupled with the growth in the number the high quality and wide dynamic range of
television shows. In fact Bobby said that the of rendering channels needed for immer- sound available on current sound tracks. The
traditional boundaries between the roles are sive audio, has dramatically expanded the result can be unpleasant and distorted, and
breaking down in many cases, such that he channel capacity of systems and the means people can complain about it being too loud at
is very much a “one-man band” these days, of controlling these. Another factor that the same time as having an “underwhelming
rather than there necessarily being a separate has really affected the pace of work, it was experience.” At the other end of the scale are
recordist and editor. When doing sound for suggested, is the rise of nonlinear editing and the premiere quality rooms where a compel-
animated movies one is sometimes recording postproduction systems. The speed with which ling cinematic experience can be had. The
dialog from the actors prior to their char- things can be changed, and moved between danger is that people will just sit at home and
acters being animated, so in those cases it’s operators or departments, has increased watch things on TV, where the sound qual-
not so much dialog replacement as picture the pressure on people working in this area ity of consumer systems can be quite good.
replacement, one might say. The picture is substantially. It’s possible to capture, edit, and What will save cinema, it was suggested, is
often “dubbed” onto the dialog, depending conform material on one platform, resulting being able to deliver an experience that you
on the preferred approach of the animation in a blurring between previously separate simply can’t get anywhere else. (Interestingly,
company. For those sorts of recording the parts of the job, and this has led to an expecta- this is almost exactly what was said in the
actors have to be rather more static as it needs tion that miracles can be worked in very little 1940s by Garrity and Hawkins when discuss-
to be very “clean” dialog for animation. This time. ing Fantasound and the future of cinema in
has the effect of turning the re-recording Wrestling with the vagaries of different the SMPTE Journal, so the challenges for
mixer into something of a production sound manufacturers’ systems and their constant cinema sound don’t seem to have changed
mixer. For these sorts of sessions the setup is upgrades is a major impediment to efficiency fundamentally.)
quite complicated, and there may need to be in today’s workflows. Probably because of the Avi Laniado suggested that it was sad but
multiple cameras on the artists, depending on flexibility and complexity of current systems, true that the average movie-goer really doesn’t
the requirements of the animation company. the time spent on setting them up for individ- seem to have much idea about all the differ-
Sometimes a backup microphone set 10 dB ual projects or engineers now takes a substan- ent cinema sound formats and the quality
lower in gain will be employed in case one of tial chunk of the time available. Particularly of sound possible or experienced. In reality
the actors suddenly gets over-loud in the heat problematic is the preservation of metadata most of the talk about this happens between
of the moment. from one step of the process to another, as people in the business. Cinema owners will
These days, it was suggested, signal process- this can get messed up in numerous differ- tend to respond to the average movie-goer,
ing tools such as de-clickers and de-noisers ent ways. The challenges of ensuring correct making it hard to persuade them that it’s

184 J. Audio Eng. Soc., Vol. 66, No. 3, 2018 March

 FEATURE ARTICLE

worth spending a lot different outputs being created for differ- detracted or distracted from that could be
on good immersive ent venues and devices. That requires good detrimental. An alternative point of view
audio systems. They mixing metadata structures that get passed suggested that VR could be an alternative
often prefer to put in through systems in a way that ensures an storytelling tool that could use more of the
a few more reclining acceptable result for each version. So far the viewer’s personal choice in experiencing the
seats, for example. tools available to do this are not particularly story. It requires rendering at the reproduc-
Surprisingly perhaps, sophisticated or successful. tion end of the chain to be possible, as it has
it seems that it’s even Standards for the calibration of cinema to be dynamically changeable. People taking
Brian Vessa quite hard to persuade rooms are one of the things that SMPTE is part in VR movies would also need the space
film directors to adopt immersive sound working on, said Brian Vessa, in an attempt to be able to move around a bit without
formats for their movies. Avi suggested that to find a common way of characterizing the hurting each other, it was proposed. VR is
he works hard to convince directors that it’s acoustics and reproduction qualities. SMPTE a very personal experience, said one panel
worth spending their budgets on such things, has also been trying to come up with a stan- member, whereas movie-watching is some-
but when he mentions things like Atmos or dard delivery bitstream for immersive audio thing of a group experience. Furthermore,
IMAX he is often met with some skepticism, that is not so dependent on the individual it’s very hard to compare VR and cinema
with a preference shown for good old-fash- manufacturers’ formats. This seems to be because the latter is the product of a direc-
ioned 35mm film and its standard surround almost ready to go. How that gets put into a tor deciding what you’re going to see and
sound options. standard file format or DCP (digital cinema hear, whereas VR presents a number of
It turns out that almost as much time is package) is then the next step. If we can just choices or options for the viewer to decide
spent making the multiple versions of differ- get down to delivering one thing, he said, we about. Movies could perhaps act as an adver-
ent films these days as is spent making the stand a better chance of promoting immer- tisement for VR experiences, but really they
final mix. That’s because there are so many sive audio to a skeptical business. Otherwise should be thought about as different things.
different formats and destinations for the we run the risk of them being too confused
same movie and a slightly different version to risk putting in the investment.
has to be made for each one, usually manu- As far as the potential for VR in movies Editor’s note: recordings of the workshops
discussed here may be downloaded from
ally. It’s not really a sustainable proposition, is concerned, Avi felt that it would only be https://www.mobiltape.com/2017-audio-
the panel members felt, and there really has useful if it added to or supported the story- engineering-society-143rd-convention
to be a way of mixing once and number of telling capacity of movies. Aspects of it that

AES E-Library Gain immediate access to over

15,000 fully searchable PDF files
http://www.aes.org/e-lib/ documenting audio research from
1953 to the present day.
The E-library includes every AES
paper published at a convention,
conference or in the Journal.

All individual AES members receive a FREE subscription to the AES E-Library.
A subscription allows you to download any document in the E-Library at no additional cost.
Cost per paper without subscription: $33
Institutional annual subscription: $1800 per year

J. Audio Eng. Soc., Vol. 66, No. 3, 2018 March 185