Game Audio Development with Unity 5.X

Game Audio Development with Unity 5.X

Design a blockbuster game soundtrack with Unity 5.X

Micheal Lanham

BIRMINGHAM - MUMBAI

Game Audio Development with Unity 5.X

Copyright © 2017 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: June 2017

Production reference: 1280617

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-78728-645-0

www.packtpub.com

Credits

About the Author

Micheal Lanham is a solutions architect with petroWEB and currently resides in Calgary, Alberta, in Canada. In his current role, he develops integrated GIS applications with advanced spatial search capabilities. He has worked as a professional and amateur game developer building desktop and mobile games for over 15 years. In 2007, Micheal was introduced to Unity 3D and has been an avid developer, consultant, and manager of multiple Unity game and graphic projects since.

Micheal has previously written Augmented Reality Game Development, also published by Packt Publishing in 2017.

I would really like to thank the team at Packt Publishing for all their support and assistance in making this book so easy and enjoyable to write. It has been a joy working with the team, which is composed of nothing but dedicated professionals. I would especially like to thank Reshma Raman and Jason Pereira. Reshma, the Senior Acquisition Editor, understood and agreed with the vision I had for this book. Jason, the Content Development Editor, has been a joy to work with and has worked hard assisting me when needed. Of course, no book would ever make it past the first draft if it were not for the dedication and hard work of the reviewers. So I would also like to make a special thanks to all the dedicated and hard work of the reviewers.

Writing a book would not be possible if I didn't have the support of my family and friends, Especially, with the help of my little tester and ideas contributor, Ava, who has been a great asset in developing several of the books exercises. Yet, none of this would be possible without the help and support of Rhonda. Rhonda has been gracious enough to draw some of the initial diagrams, lend her musical ear, and help with those first draft reviews.

I found this book a pleasure to write, and none of it would have been possible without any of the people I mentioned above.

About the Reviewer

Francesco Sapio obtained his Computer Science and Control Engineering degree from Sapienza University of Rome, Italy, with a couple of semesters in advance, graduating summa cum laude. He is currently studying a Master of Science in Engineering in Artificial Intelligence and Robotics at the same university.

He is a Unity 3D and Unreal expert, a skilled game designer, and an experienced user of the major graphics programs. He developed Gea 2: A New Earth (Sapienza University of Rome), an educational game for high school students to learn concepts of physics, and Sticker Book (series) (Dataware Games), a cross-platform series of games for kids. In addition, he worked as consultant for the (successfully funded by Kickstarter) game Prosperity - Italy 1434 (Entertainment Game Apps, Inc), and for the open online collaborative ideation system titled Innovoice (Sapienza University of Rome). Moreover, he has been involved in different research projects, such as Belief-Driven-Pathfinding (Sapienza University of Rome), a new technique of pathfinding in videogames that was presented as a paper at the DiGRA-FDG Conference 2016; and Project Anima (Royal Melbourne Institute of Technology), which included developing a recommendation system for games.

He has authored several books for Packt. He is an active writer on the topic of game development. Recently, he authored the book Getting Started with Unity 5.x 2D Game Development, which takes your hand and guides you through the amazing journey of game development, the successful Unity UI Cookbook, which has been translated into other languages, and teaches readers how to develop exciting and practical user interfaces for games within Unity, and a short e-guide called What do you need to know about Unity. In addition, he co-authored the book Unity 5.x 2D Game Development Blueprints. Furthermore, he has also been a reviewer for the following books: Game Development Patterns and Best Practices, Game Physics Cookbook, Mastering Unity 5.x, Unity 5.x by Example, and Unity Game Development Scripting.

Francesco is also a musician and a composer, especially of soundtracks for short films and video games. For several years, he worked as an actor and dancer, where he was a guest of honor at the theatre Brancaccio in Rome. In addition, he is a very active person, having volunteered as a children's entertainer at the Associazione Culturale Torraccia in Rome.

Finally, Francesco loves math, philosophy, logic, and puzzle solving, but most of all, creating video games--thanks to his passion for game designing and programming.

You can find him at www.francescosapio.com.

Acknowledgments

I’m deeply thankful to my parents for their infinite patience, enthusiasm, and support throughout my life. Moreover, I’m thankful to the rest of my family, in particular to my grandparents, since they have always encouraged me to do better in my life with the Latin expressions Ad maiora and Per aspera ad astra.

Finally, a huge thanks to all the special people around me whom I love, in particular to my girlfriend; I’m grateful for all of your help in everything. I do love you.

www.PacktPub.com

For support files and downloads related to your book, please visit www.PacktPub.com.

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

https://www.packtpub.com/mapt

Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt books and video courses, as well as industry-leading tools to help you plan your personal development and advance your career.

Why subscribe?

Fully searchable across every book published by Packt

Copy and paste, print, and bookmark content

On demand and accessible via a web browser

Customer Feedback

Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial process. To help us improve, please leave us an honest review on this book's Amazon page at https://www.amazon.com/dp/1787286452.

If you'd like to join our team of regular reviewers, you can e-mail us at [email protected]. We award our regular reviewers with free eBooks and videos in exchange for their valuable feedback.

Help us be relentless in improving our products!

To my father Leslie, who taught me how to love and appreciate music of all forms.

Table of Contents

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Downloading the color images of this book

Errata

Piracy

Questions

Introducing Game Audio with Unity

Introduction to game audio

Main game audio components

Diegetic versus non-diegetic sound

Audio layers or groups

Getting started with unity

Downloading unity

An introduction to unity

Downloading and importing project assets

A tour of the village

Importing audio into unity

Inspecting the imported audio files

Audio sources and listeners

Adding an audio source

3D sound and spatial blending

Summary

Scripting Audio

Introduction to scripting in Unity

Building a musical keyboard

Importing and playing notes

Enhancing the virtual keyboard

Weapons and sound

Throwing the ax

Understanding the axController script

Playing sounds on collision

Randomizing audio

Background music

Summary

Introduction to the Audio Mixer

Introducing the Unity Audio mixer

Creating mixer groups

Shaping audio with effects

Visualizing audio equalization

The master mixer and controlling signal flow

Routing audio signals in the mixer to effects

Audio effects breakdown

Equalization effects

Lowpass and Lowpass simple

Highpass and Highpass simple

ParamEQ

Delay effects

Echo

SFX reverb

Flange

Chorus

Other effects

Pitch Shifter

Normalize

Compressor

The duck volume effect

Summary

Advanced Audio Mixing

Recording audio changes with snapshots

Pausing the scene

Audio Mixer scripting with parameters

A dynamic audio wind effect

Creating environmental audio zones

Dynamic music mixing

Summary

Using the Audio Mixer for Adaptive Audio

Introducing adaptive audio

Building adaptive audio cues

Creating mood with adaptive music

Vertical remixing with the Audio Mixer

Footsteps with adaptive sound

Summary

Introduction to FMOD

Getting started with FMOD Studio

Installing FMOD Studio

Back to the basics

Effects and mixing

Mixing in a reverb effect

Parameters, snapshots, and triggers

Summary

FMOD for Dyanmic and Adaptive Audio

Dynamic wind and environmental zones

Scripting with FMOD

Revisiting the footsteps example

Adaptive music with FMOD

Transition timelines and stingers

Summary

Visualizing Audio in Games

Analyzing audio with Fast Fourier Transform (FFT) windows

Audio spectrum

Deconstructing signals using FFT and windowing

Examining an audio visualizer

Uncovering the details

Visualization performance and windowing

Audio-driven lighting

Microphone-driven visualizations

Summary

Character Lip Syncing and Vocals

Real-time lip syncing

Speech and phonemes

Blend shape animation

Real-time character lip sync

Recording vocals

Summary

Composing Music

Getting started with Reaper

MIDI and virtual instruments

Virtual instruments

Basic music theory

Chords and chord progressions

Chord progression

Melody and scale

Enhancing the composition

Recording music and vocals

Recording and optimizing vocals

Summary

Audio Performance and Troubleshooting

Audio performance considerations

Profiling performance with Unity

Audio optimization tips and tricks for Unity

Memory optimization tips and tricks

Effects optimization tips and tricks

FMOD performance and optimization

Audio troubleshooting

Summary

Preface

From those exotic sound effects that improve game fun and playability to that dark musical score that sets the entire atmosphere of a scene or game audio is essential to any great game. Yet it is perhaps one of the most often overlooked assets in the development of a game by new or indie developers. Instead, they dedicate the bulk of their effort to graphics and other visuals, often deferring the audio portion until the later stages of production. They do this not because good audio is not a critical aspect to any good game, but as a single or indie developer they feel AAA game quality audio is out of reach. Of course, that is far from the truth, especially with the great audio system built into Unity and the many freely available commercials plugins and tools.

This book was written to introduce the reader to the extensive audio framework built within Unity and other freely available commercials tools and plugins. Often, audio development tools expect a certain level of knowledge in sound, mixing and music. By following through the books material and exercises a reader should be able to master all the essentials of audio development. From the basics of implementing an audio source; to spatial sound, mixing, effects and dynamic and adaptive audio system. As well there will be time spent learning professionals like FMOD Studio for game audio and Reaper for music composition. While the book assumes no prior knowledge of working with Unity or audio, there will be enough new and interesting content that even the more advanced developer will find useful.

What this book covers

This book starts from the fundamentals of starting game audio development with Unity and progresses to using advanced commercial tools for professional game audio and music composition. It is assumed the reader will follow the chapters in sequential order as many chapters build from previous exercises. Each of the chapters is listed below with a quick summary:

Chapter 1, Introducing Game Audio in Unity, Unity is introduced and the reader is guided through installation of the software and setup of the first project. Which includes importing audio assets, audio source essentials and setting up spatial sounds.

Chapter 2, Scripting Audio, the basics of scripting in Unity is introduced with the writing of a simple musical keyboard. Followed by; scripting weapons, sound effects, physics and implementing ambient sounds and music.

Chapter 3, Introduction to the Unity Audio Mixer, introduces the basics of the Audio Mixer, mixing audio sources and effects, with further coverage on mixer signal routing and using the Duck Volume effect.

Chapter 4, Advanced Audio Mixing, covers using advanced capabilities of the Audio Mixer. From mixer parameter scripting, snapshots, snapshot transitions, and developing audio transition zones.

Chapter 5, Using the Audio Mixer for Adaptive Audio, introduces adaptive audio and covers techniques for mixing adaptive music.The rest of the chapter is spent implementing adaptive audio in sound and music.

Chapter 6, Introduction to FMOD, starts with the installation and setup of FMOD Studio. Then, it goes over the similarities and differences between the Unity Audio Mixer and FMOD.

Chapter 7, FMOD for Dynamic and Adaptive Audio, builds upon previous advanced examples and converts or adapts them to use FMOD. The chapter covers several advanced techniques of working with FMOD Studio.

Chapter 8, Visualizing Audio in Games, starts with a background of how sound can be broken down by frequency with mathematical techniques know as FFT Windowing. Those techniques are then used to create stunning audio-visual dancing lights and graphics.

Chapter 9, Character Lip Syncing and Vocals, builds upon previous work and builds a real-time character lip syncing demo. By covering character bone and vertex animation, lip-sync animation with phonemes, and recording with Unity.

Chapter 10, Composing Music, introduces the basics of music composition with Reaper using some simple music theory. It covers the installation and setup of Reaper, MIDI basics, virtual instruments, audio rendering and recording.

Chapter 11, Audio Performance and Troubleshooting, covers the fundamentals of audio performance with Unity and FMOD. From performance profiling to the tips and tricks of optimizing and troubleshooting performance issues.

What you need for this book

In order to be successful with this book you will need to have an eagerness to learn the basics and master game audio development. No previous game or audio development knowledge is required. Although, it may be helpful to have some knowledge of scripting with a C language like C#, JavaScript or C++, but it is not essential. Covering some of the material may be easier if you know how to play an instrument or understand sound or music theory, but again this is not required. You should have a desktop or laptop computer running Windows XP+, Mac OS, or Linux to complete the book's essential exercises.

Who this book is for

This book is for anyone who wants to learn how to create better sound for their games or just have a better understanding of how audio works in games. While this book is directed at the individual or indie development team, it certainly could be put to good use by sound designers or audio enthusiasts.

Conventions

In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, and user input are shown as follows: As the scene is running, be sure to select the VirtualKeyboard object in the Hierarchy window.

A block of code is set as follows:

New terms and important words are shown in bold. Words that you see on the screen, in menus or dialog boxes for example, appear in the text like this: Click on the checkbox next to the Record setting on the Keyboard component. Click on the mouse in the Game view and then type some keys to play the virtual instrument.

Warnings or important notes appear in a box like this.

Tips and tricks appear like this.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book-what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.

To send us general feedback, simply e-mail [email protected], and mention the book's title in the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

You can download the code files by following these steps:

Log in or register to our website using your e-mail address and password.

Hover the mouse pointer on the SUPPORT tab at the top.

Click on Code Downloads & Errata.

Enter the name of the book in the Search box.

Select the book for which you're looking to download the code files.

Choose from the drop-down menu where you purchased this book from.

Click on Code Download.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

WinRAR / 7-Zip for Windows

Zipeg / iZip / UnRarX for Mac

7-Zip / PeaZip for Linux

The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Game-Audio-Development-with-Unity-5X. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Downloading the color images of this book

We also provide you with a PDF file that has color images of the screenshots/diagrams used in this book. The color images will help you better understand the changes in the output. You can download this file from https://www.packtpub.com/sites/default/files/downloads/GameAudioDevelopmentwithUnity5X_ColorImages.pdf.

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books-maybe a mistake in the text or the code-we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.

To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.

Piracy

Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.

Please contact us at [email protected] with a link to the suspected pirated material.

We appreciate your help in protecting our authors and our ability to bring you valuable content.

Questions

If you have a problem with any aspect of this book, you can contact us at [email protected], and we will do our best to address the problem.

Introducing Game Audio with Unity

Welcome to our journey into game audio and audio development with Unity. It is the leading cross-platform game engine that provides an abundance of features to make developing games easy. For this book, it doesn't matter if you are completely new to game audio, Unity, or game development. We will start with the assumption that you are a complete novice and build from there. With each new chapter, we will be introducing new concepts and additional material. Of course, if you are an experienced Unity developer with basic knowledge of audio development, you may want to skim over the first couple of chapters. Regardless of if you are a novice or master, the later chapters in this book will cover plenty of Unity features and tools to give you the knowledge of developing AAA game quality audio for your games.

In this chapter, we will cover the basics of Unity and add audio to a project and scene in Unity. As this is the first chapter, we will be covering a quick introduction to game audio and then spend some time installing and configuring Unity. From there, we will introduce the core Unity components and how they function. Here is a summary of what we will cover in this chapter:

Introduction to game audio

Getting started with unity

Importing audio into unity

Audio sources and listeners

3D sound and spatial blending audio

As with most of the chapters in this book, it is expected that you follow along and complete each of the exercises. Completing the exercises hands on will reinforce all the concepts and provide you with a good foundation for the more complex material introduced in later chapters. All the material required to complete the exercises will either be provided via the book's downloaded source code or will be freely available online. Therefore, in almost all cases, you will be required to have an internet connection. If you feel that you understand the content in a chapter and just want to review the completed examples, then follow the instructions provided in the book's download to build the final examples.

Introduction to game audio

Chances are, if you have played a few computer, video, or mobile games in your life, you are already quite familiar with what great game audio should sound like. Perhaps; you even cherish the audio from a couple of your favorite games. However, if someone asked you how they composed or mixed that audio, unless you were a professional audio technician, you would likely be at a loss on how the audio in a game mixes together. Fortunately, in this section, we will start to introduce some of the basic concepts of game audio, which we will of course build upon in later chapters in this book.

Main game audio components

Game audio, unlike audio a sound designer would compose for film or television, will typically be broken down into three distinct areas as listed here:

Music: This includes content such as theme or background music, in-game music such as a radio or a band playing, and can even include certain audio effects. Creating or obtaining unique music for your game could be as complex as recording a music track in a sound studio to purchasing rights to a music track of the asset store. In Chapter 10, Composing Music, we will explore how to compose your own unique tunes.

Sound: This generally includes sound effects, such as creaking doors, explosions, footsteps, clicking, beeping, and many others. Most sound designers or developers will often browse through many of the free or paid sound effects sites to find a sound they like. Another option is to record your own sound effects, essentially becoming a Foley artist. In Chapter 10, Composing Music, we will look at some techniques to create some unique sound effects in software.

Foley artists are traditionally people who create sound effects for film, television, and now games. The term Foley was coined from the originator, Jack Foley, who created real-time sound effects for early motion pictures. Today, the practice is no longer real time, but the name Foley is still synonymous with recording and creating sound effects.

Vocals: The addition of vocals to games is a relatively new addition. Vocals were first introduced in cut or intro game scenes but now they are a main element in any AAA title. Furthermore, most AAA titles will enlist a well-known actor to provide vocals for further enhancement of the game. In Chapter 9, Character Lip Syncing and Vocals, we will cover some details about generating vocals. As well in several other chapters of this book, we will highlight the use of vocal cues in scenes.

While game audio design is similar to film and television, in many components, it differs primarily by its implementation. Audio for film follows a linear well-defined pattern, whereas in games audio is non-linear and may even be dynamic or adaptive. We will of course cover many aspects of how game audio is dynamic and adaptive in the Adaptive Sound and Music section.

Diegetic versus non-diegetic sound

If you have never heard of the term diegetic, you may just have gotten a queasy feeling in your stomach wondering what you got yourself