Chapter 4 The Synthesis of Sound by Computer

4.0 Analogue Synthesis

4.1 Introduction to Synthesis
4.2 Additive Synthesis
4.3 Filters
4.4 Formant Synthesis
4.5 Introduction to Modulation
4.6 Waveshaping
4.7 Frequency Modulation
4.8 Granular Synthesis
4.9 Physical Modeling

Analogue Synthesis
Most analogue synthesis employs a method known as “subtractive synthesis” . Here the desired
sound is
achieved by filtering out the undesired parts of the sound from a broad initial sound source. An
analogue
synthesiser can be broken down into three main types of components;
• sound sources
• sound modifiers
• controllers

Sound Sources - Oscillator

Sound sources in analogue synthesisers are called “oscillators” and “noise generators”. Oscillators
are
electronic circuits which generate a waveform with a recognisable harmonic structure. Common
waveforms are:
sine, triangle, sawtooth and square wave. The fundamental frequency of the waveform can be
controlled from
another source such as a keyboard or another oscillator.
sine triangle sawtooth square noise
Filters:
Tipos de Filtro
Lowpass Filter

Highpass filter

Bandpass Filter

Lowpass filter with resonance

Highpass filter with resonance

Bandpass filter with narrow Q

Sound Modifiers - Voltage Controlled Amplifier

As well as filtering the frequency content (or the spectra) of a waveform, synthesisers can also
shape the
amplitude. In analogue synthesisers this is done using a “voltage controlled amplifier” or VCA. The
VCA
controls the overall level of the waveform just like any standard amplifier. Like a standard amplifier,
you can
adjust the level of the signal by turning the volume control.

Controllers - Envelope Generator: ADSR

The Ring Modulator - Another Modifier

The Ring Modulator is a module which takes two input signals and produces 1 output signal. The
output
frequencies are the sum and the difference of the input frequencies. In a simple example, if two sine
waves,
one at 400 Hz and the other at 500 Hz (sine waves only produce one frequency) , are passed
through the Ring
Modulator, the output will be 900 and 100 Hz. If other waveforms are used, the results are much
more complex.
This is because there are a range of harmonic frequencies present in the input sounds. The Ring
Modulator
can be used to generate enharmonic frequencies in order to create metallic sounds

Frequency Modulation
In the previous example you controlled the frequency of the oscillators with the joystick or to put it
another way,
the joystick was modulating the frequency of the oscillators. Another way to control an oscillator’s
frequency is
to use another oscillator as a control source. This type of configuration is commonly referred to as
frequency
modulation or FM (the same principal is used in FM radio where the carrier wave is is modulated by
the signal
wave). If the modulating frequency is very high the effect will be complex, somewhat like the Ring
Modulator.
However at very low frequencies (such as 5 Hz and lower) the effect will sound like a kind of
vibrato. Some
synthesisers have oscillators specifically designed to create very low (sub audible) frequencies in
order to
create low frequency modulation. These oscillators are sometimes called Low Frequency
Oscillators or LFOs.
Low frequency modulation is often referred to as Pitch modulation because of the vibrato like effect.
In
Matrixsynth the three oscillators can each function as an LFO. The LFO mode is selected using the
sub-menu
under the level dial in the oscillator modules. When an oscillator is set to LFO mode the range of
frequencies
that can be produced falls mainly in the sub-audible range.

Pulsewidth Modulation
When you alter the waveform in an oscillator using the shape dial you are varying the symmetrical
proportions of
the waveforms shape. For example a symmetrical triangle can be altered to produce a sawtooth
wave
Changing the pulsewidth of a waveform results in a change in the balance of harmonic frequencies
present in
the sound. This is easy to hear when you vary the shape of a triangle wave. When the wave is
symmetrical (a
triangle wave) the sound is relatively simple (with few of the upper harmonic frequencies present).
However,
when the pulsewidth is altered such that the waveform is closer to a sawtooth wave the sound will
be much
brighter. This is caused by the presence of upper harmonic frequencies.

Sample and Hold

The sample and hold module creates a control signal that can be used to vary the parameter of a
sound
generator or sound modifier. The sample and hold process involves sampling an input signal at
regular intervals
and outputting a signal that is the value of the last sample.

Section 4.2: Additive Synthesis

Additive synthesis refers to a number of related synthesis techniques, all based
on the idea that complex tones can be created by the summation, or addition,
of simpler tones. As we saw in Chapter 3, it is theoretically possible to break
up any complex sound into a number of simpler ones, usually in the form of
sine waves. In additive synthesis, we use this theory in reverse.

Figure 4.1  Two waves joined by a plus sign.

Figure 4.2  This organ has a great many pipes, and together they function exactly like
an additive synthesis algorithm.

Each pipe essentially produces a sine wave (or something like it), and by selecting
different combinations of harmonically related pipes (as partials), we can create
different combinations of sounds, called (on the organ) stops. This is how organs get
all those different sounds: organists are experts on Fourier series and additive
 synthesis (though they may not know that!).

The technique of mixing simple sounds together to get more complex sounds
dates back a very long time. In the Middle Ages, huge pipe organs had a great
many stops that could be "pulled out" to combine and recombine the sounds
from several pipes. In this way, different "patches" could be created for the
organ. More recently, the telharmonium, a giant electrical synthesizer from the
early 1900s, added together the sounds from dozens of electro-mechanical
tone generators to form complex tones. This wasn’t very practical, but it has
an important place in the history of electronic and computer music.

This applet demonstrates how sounds are mixed together.

Applet 4.2
Mixed sounds

The Computer and Additive Synthesis

While instruments like the pipe organ were quite effective for some sounds,
they were limited by the need for a separate pipe or oscillator for each tone
that is being added. Since complex sounds can require anywhere from a couple
dozen to several thousand component tones, each needing its own pipe or
oscillator, the physical size and complexity of a device capable of producing
these sounds would quickly become prohibitive. Enter the computer!

A short excerpt from Kenneth Gaburo’s composition "Lemon Drops," a classic of

electronic music made in the early 1960s.

This piece and another extraordinary Gaburo work, "For Harry," were made at the
Soundfile 4.1 University of Illinois at Urbana-Champaign on an early electronic music instrument called
Excerpt from the harmonic tone generator, which allowed the composer to set the frequencies and
Kenneth amplitudes of a number of sine wave oscillators to make their own timbres. It was
Gaburo’s extremely cumbersome to use, but it was essentially a giant Fourier synthesizer, and,
composition theoretically, any periodic waveform was possible on it!
"Lemon Drops"

It’s a tribute to Gaburo’s genius and that of other early electronic music pioneers that they
were able to produce such interesting music on such primitive instruments. Kind of makes
it seem like we’re almost cheating, with all our fancy software!
If there is one thing
computers are good at,
it’s adding things
together. By
using digital
oscillators instead of
actual physical devices,
a computer can add up
any number of simple
sounds to create
extremely complex
waveforms. Only the
speed and power of the
computer limit the
number and complexity
of the waveforms.
Modern systems can
easily generate and mix
thousands of sine
waves in real time.
This makes additive
synthesis a powerful
and versatile
performance and
synthesis tool. Additive
synthesis is not used so
much anymore (there
are a great many other,
more efficient
techniques for getting
complex sounds), but
it’s definitely a good
thing to know about.

A Simple Additive
Synthesis Sound
Let’s design a simple
sound with additive
synthesis. A nice
example is the
generation of a square
wave.
 You can probably imagine what a square wave would look like. We start with
just one sine wave, called thefundamental. Then we start adding odd
partials to the fundamental, the amplitudes of which are inversely proportional
to their partial number. That means that the third partial is 1/3 as strong as the
first, the fifth partial is 1/5 as strong, and so on. (Remember that the
fundamental is the first partial; we could also call it the first harmonic.) Figure
4.3 shows what we get after adding seven harmonics. Looks pretty square,
doesn’t it?

Now, we should admit that there’s an easier way to synthesize square waves:
just flip from a high sample value to a low sample value every nsamples. The
lower the value of n, the higher the frequency of the square wave that’s being
generated. Although this technique is clearer and easier to understand, it has its
problems too; directly generating waveforms in this way can cause unwanted
frequency aliasing.

Figure 4.4 The Synclavier was an early digital electronic music instrument that used a large oscillator
bank for additive synthesis. You can see this on the front panel of the instrument—many of the LEDs
indicate specific partials! On the Synclavier (as was the case with a number of other analog and digital
instruments), the user can tune the partials, make them louder, even put envelopes on each one.

This applet lets you add sine waves together at various amplitudes, to see how additive
synthesis works.

Applet 4.3
Additive
synthesis

This applet lets you add spectral envelopes to a number of partials. This means that you
can impose a different amplitude trajectory for each partial, independently making each
 Applet 4.4 louder and softer over time.
Spectral
envelopes
[Under This is really more like the way things work in the real world: partial amplitudes evolve
Development over time—sometimes independently, sometimes in conjunction with other partials (in a
Available Oct. phenomenon called common fate). This is called spectral evolution,and it’s what makes
2004] sounds live.

A More Interesting Example

OK, now how about a more interesting example of additive synthesis? The
quality of a synthesized sound can often be improved by varying its
parameters (partial frequencies, amplitudes, and envelope) over time. In fact,
Xtra bit 4.4
time-variant parameters are essential for any kind of "lifelike" sound, since all
Spectral formula naturally occurring sounds vary to some extent.
of a waveform

Soundfile 4.2 Soundfile 4.3 Soundfile 4.4 Soundfile 4.5

Sine wave Regular Sine wave Regular
speech speech speech speech
These soundfiles are examples of sentences reconstructed with sine waves. Soundfile
4.2 is the sine wave version of the sentence spoken in Soundfile 4.3, and Soundfile
4.4 is the sine wave version of the sentence spoken in Soundfile 4.5.

Sine wave speech is an experimental technique that tries to simulate speech with just
a few sine waves, in a kind of primitive additive synthesis. The idea is to pick the sine
waves (frequencies and amplitudes) carefully. It’s an interesting notion, because sine
waves are pretty easy to generate, so if we can get close to "natural" speech with just
a few of them, it follows that we don’t require that much information when we listen
to speech.

Sine wave speech has long been a popular idea for experimentation by psychologists
and researchers. It teaches us a lot about speech—what’s important in it, both
perceptually and acoustically.

These files are used with the permission of Philip Rubin, Robert Remez,
and Haskins Laboratories.

Attacks, Decays, and Time Evolution in Sounds

As we’ve said, additive synthesis is an important tool, and we can do a lot with
it. It does, however, have its drawbacks. One serious problem is that while it’s
good for periodic sounds, it doesn’t do as well with noisy or chaotic ones.

For instance, creating the steady-state part (the sustain) of a flute note is

simple with additive synthesis (just a couple of sine waves), but creating
the attack portion of the note, where there is a lot of breath noise, is nearly
impossible. For that, we have to synthesize a lot of different kinds of
information: noise, attack transients, and so on.

And there’s a worse problem that we’d love to sweep under the old
psychoacoustical rug, too, but we can’t: it’s great that we know so much about
steady-state, periodic, Fourier-analyzable sounds, but from a cognitive and
perceptual point of view, we really couldn’t care less about them! The ear and
brain are much more interested in things like attacks, decays, and changes over
time in a sound (modulation). That’s bad news for all that additive synthesis
software, which doesn’t handle such things very well.

That’s not to say that if we play a triangle wave and a sawtooth wave, we
couldn’t tell them apart; we certainly could. But that really doesn’t do us much
good in most circumstances. If angry lions roared in square waves, and cute
cuddly puppy dogs barked in triangle waves, maybe this would be useful, but
we have evolved—or learned to hear attacks, decays, and other transients as
being more crucial. What we need to be able to synthesize are transients,
spectral evolutions, and modulations. Additive synthesis is not really the best
technique for those.

Another problem is that additive synthesis is very computationally expensive.

It’s a lot of work to add all those sine waves together for each output sample of
sound! Compared to some other synthesis methods, such as frequency
modulation (FM) synthesis, additive synthesis needs lots of computing power
to generate relatively simple sounds.

But despite its drawbacks, additive synthesis is conceptually simple, and it

corresponds very closely to what we know about how sounds are constructed
mathematically. For this reason it’s been historically important in computer
sound synthesis.

Figure 4.5  A typical ADSR (attack, decay, sustain, release) steady-state modulation.

This is a standard amplitude envelope shape used in sound synthesis.
 The ability to change a sound’s amplitude envelope over time plays an important part
in the perceived "naturalness" of the sound.

Shepard Tones

One cool use of additive synthesis is in the generation of a very interesting

phenomenon called Shepard tones.Sometimes called "endless glissandi,"
Shepard tones are created by specially configured sets of oscillators that add
their tones together to create what we might call a constantly rising tone.
Certainly the Shepard tone phenomenon is one of the more interesting topics
in additive synthesis.

In the 1960s, experimental psychologist Roger Shepard, along with composers

James Tenney and Jean-Claude Risset, began working with a phenomenon that
scientifically demonstrates an independent dimension in pitch perception
called chroma, confirming the circularity of relative pitch judgments.

What circularity means is that pitch is perceived in kind of a circular way: it

keeps going up until it hits an octave, and then it sort of starts over again. You
might say pitch wraps around (think of a piano, where the C notes are evenly
spaced all the way up and down). Bychroma, we mean an aspect of pitch
perception in which we group together the same pitches that are related as
frequencies by multiples of 2. These are an octaveapart. In other words, 55 Hz
is the same chroma as 110 Hz as 220 Hz as 440 Hz as 880 Hz. It’s not exactly
clear whether this is "hard-wired" or learned, or ultimately how important it is,
but it’s an extraordinary idea and an interesting aural illusion.

We can construct such a circular series of pitches in a laboratory setting using

synthesized Shepard tones.These complex tones are comprised of partials
separated by octaves. They are complex tones where all the non-power-of-two
numbered partials are omitted.

These tones slide gradually from the bottom of the frequency range to the top.
The amplitudes of the component frequencies follow a bell-shaped spectral
envelope (see Figure 4.6) with a maximum near the middle of the standard
musical range. In other words, they fade in and out as they get into the most
common frequency range. This creates an interesting illusion: a circular
Shepard tone scale can be created that varies only in tone chroma and
collapses the second dimension of tone height by combining all octaves. In
other words, what you hear is a continuous pitch change through one octave,
but not bigger than one octave (that’s a result of the special spectra and the
amplitude curve). It’s kind of like a barber pole: the pitches sound as if they
just go around for a while, and then they’re back to where they started (even
though, actually, they’re continuing to rise!).
 Figure 4.10  Bell-shaped spectral envelope for making Shepard tones.

Shepard wrote a famous paper in 1964 in which he explains, to some extent,

our notion of octave equivalence using this auditory illusion: a sequence of
these Shepard tones that shifts only in chroma as it is played. The apparent
fundamental frequency increases step by step, through repeated cycles.
Listeners hear the pitch steps as climbing continuously upward, even though
the pitches are actually moving only around the chroma circle. Absolute pitch
height (that is, how "high" or "low" it sounds) is removed from our perception
of the sequence.

Soundfile 4.6
Shepard tone

Figure 4.7 Try clicking on Soundfile 4.6. After you listen to the soundfile once, click
again to listen to the frequencies continue on their upward spiral.
 Used with permission from Susan R. Perry, M.A., Dept. of Psychology, University of
Tennessee.

The Shepard tone contains a large amount of octave-related harmonics across the
frequency spectrum, all of which rise (or fall) together. The harmonics toward the low and
high ends of the spectrum are attenuated gradually, while those in the middle have
maximum amplification. This creates a spiraling or barber pole effect. (Information from
Soundfile 4.7 Doepfer Musikelektronik GmbH.)
Shepard tone
Soundfile 4.7 is an example of the spiraling Shepard tone effect.

James Tenney is an important computer music composer and pioneer who worked at Bell
Laboratories with Roger Shepard in the early 1960s.

This piece was composed in 1969. This composition is based on a set of continuously
Soundfile 4.8 rising tones, similar to the effect created by Shepard tones. The compositional process is
"For Ann" simple: each glissando, separated by some fixed time interval, fades in from its lowest note
(rising), by
and fades out as it nears the top of its audible range. It is nearly impossible to follow,
James Tenney.
aurally, the path of any given glissando, so the effect is that the individual tones never
reach their highest pitch.

<-- Back to Previous Page Next Section -->

©Burk/Polansky/Repetto/Roberts/Rockmore. All rights reserved.

  

<-- Back to Previous Page TOC Next Section -->

 Chapter 4: The Synthesis of Sound by Computer

Section 4.3: Filters

The most common way to think about filters is as functions that take in a
signal and give back some sort of transformed signal. Usually, what comes
out is "less" than what goes in. That’s why the use of filters is sometimes
referred to as subtractive synthesis.

It probably won’t surprise you to learn that subtractive synthesis is in many

ways the opposite of additive synthesis. In additive synthesis, we start with
simple sounds and add them together to form more complex ones. In
subtractive synthesis, we start with a complex sound (like noise) and subtract,
or filter out, parts of it. Subtractive synthesis can be thought of as sound
sculpting—you start out with a thick chunk of sound containing many
possibilities (frequencies), and then you carve out (filter) parts of it. Filters are
one of the sound sculptor’s most versatile and valued tools.

Older telephones had around an 8k low-pass filter imposed on their audio signal, mostly
for noise reduction and to keep the equipment a bit cheaper.

Soundfile 4.9
Telephone
simulations

White noise (every frequency below the Nyquist rate at equal level) is filtered so we hear
only frequencies above 5 kHz.

Soundfile 4.10
High-pass
filtered
noise

Here we hear only frequencies up to 500 Hz.

Soundfile 4.11
Low-pass
filtered
noise
Four Basic Types of Filters

Figure 4.8  Four common filter types (clockwise from upper left): low-pass, high-
pass, band-reject, band-pass.

Figure 4.8 illustrates four basic types of filters: low-pass, high-pass, band-
pass, and band-reject. Low-pass and high-pass filters should already be
familiar to you—they are exactly like the "tone" knobs on a car stereo or
boombox. A low-pass (also known as high-stop) filter stops, or attenuates,
high frequencies while letting through low ones, while a high-pass (low-stop)
filter does just the opposite.
 This applet is a good example of how filters, combined with something like noise, can
produce some common and useful musical effects with very few operations.

Applet 4.5
Using filters

Band-Pass and Band-Reject Filters

Band-pass and band-reject filters are basically combinations of low-pass and

high-pass filters. A band-pass filter lets through only frequencies above a
certain point and below another, so there is a band of frequencies that get
through. A band-reject filter is the opposite: it stops a band of frequencies.
Band-reject filters are sometimes called notch filters, because they can notch
out a particular part of a sound.

Comb filters are a very specific type of digital process in which a short delay (where some
number of samples are actually delayed in time) and simple feedback algorithm (where
outputs are sent back to be reprocessed and recombined) are used to create a rather
extraordinary effect. Sounds can be "tuned" to specific harmonics (based on the length of
Applet 4.6 the delay and the sample rate).
Comb filters

Low-Pass and High-Pass Filters

Low-pass and high-pass filters have a value associated with them called
the cutoff frequency, which is the frequency where they begin "doing their
thing." So far we have been talking about ideal, or perfect, filters, which cut
off instantly at their cutoff frequency. However, real filters are not perfect,
and they can’t just stop all frequencies at a certain point. Instead, frequencies
die out according to a sort of curve around the corner of their cutoff
frequency. Thus, the filters in Figure 4.8 don’t have right angles at the cutoff
frequencies—instead they show general, more or less realistic response curves
for low-pass and high-pass filters.

Cutoff Frequency

The cutoff frequency of a filter is defined as the point at which the signal is
attenuated to 0.707 of its maximum value (which is 1.0). No, the number
0.707 was not just picked out of a hat! It turns out that the power of a signal is
determined by squaring the amplitude: 0.7072 = 0.5. So when the amplitude
of a signal is at 0.707 of its maximum value, it is at half-power. The cutoff
frequency of a filter is sometimes called its half-power point.

Transition Band

The area between where a filter "turns the corner" and where it "hits the
bottom" is called the transition band.The steepness of the slope in the
transition band is important in defining the sound of a particular filter. If the
slope is very steep, the filter is said to be "sharp"; conversely, if the slope is
more gradual, the filter is "soft" or "gentle."

Things really get interesting when you start combining low-pass and high-
pass filters to form band-pass and band-reject filters. Band-pass and band-
reject filters also have transition bands and slopes, but they have two of them:
one on each side. The area in the middle, where frequencies are either passed
or stopped, is called thepassband or the stopband. The frequency in the
middle of the band is called the center frequency, and the width of the band is
called the filter’s bandwidth.

You can plainly see that filters can get pretty complicated, even these simple
ones. By varying all these parameters (cutoff frequencies, slopes, bandwidths,
etc.), we can create an enormous variety of subtractive synthetic timbres.

A Little More Technical: IIR and FIR Filters

Filters are often talked about as being one of two types: finite impulse
response (FIR) and infinite impulse response (IIR). This sounds complicated
(and can be!), so we’ll just try to give a simple explanation as to the general
idea of these kinds of filters.

Finite impulse response filters are those in which delays are used along with
some sort of averaging. Delays mean that the sound that comes out at a given
time uses some of the previous samples. They’ve been delayed before they get
used.

We’ve talked about these filters in earlier chapters. What goes into an FIR is
always less than what comes out (in terms of amplitude). Sounds reasonable,
right? FIRs tend to be simpler, easier to use, and easier to design than IIRs,
and they are very handy for a lot of simple situations. An averaging low-pass
filter, in which some number of samples are averaged and output, is a good
example of an FIR.

Infinite impulse response filters are a little more complicated, because they

have an added feature: feedback. You’ve all probably seen how a microphone
and speaker can have feedback: by placing the microphone in front of a
speaker, you amplify what comes out and then stick it back into the system,
which is amplifying what comes in, creating a sort of infinite amplification
loop. Ouch! (If you’re Jimi Hendrix, you can control this and make great
music out of it.)

Well, IIRs are similar. Because the feedback path of these filters consists of
some number of delays and averages, they are not always what are
called unity gaintransforms. They can actually output a higher signal than that
which is fed to them. But at the same time, they can be many times more
complex and subtler than FIRs. Again, think of electric guitar feedback—IIRs
are harder to control but are also very interesting.
 Figure 4.9  FIR and IIR filters.

Filters are usually designed in the time domain, by delaying a signal and then
averaging (in a wide variety of ways) the delayed signal and the nondelayed one.
These are called finite impulse response (FIR) filters, because what comes out uses a
finite number of samples, and a sample only has a finite effect.

If we delay, average, and then feed the output of that process back into the signal, we
create what are called infinite impulse response (IIR) filters. The feedback process
actually allows the output to be much greater than the input. These filters can, as we
like to say, "blow up."

These diagrams are technical lingo for typical filter diagrams for FIR and IIR filters.
Note how in the IIR diagram the output of the filter’s delay is summed back into the
input, causing the infinite response characteristic. That’s the main difference
between the two filters.

Thanks to Fernando Pablo Lopez-Lezcano for these graphics.

Designing filters is a difficult but key activity in the field ofdigital signal
processing, a rich area of study that is well beyond the range of this book. It is
interesting to point out that, surprisingly, even though filters change the
frequency content of a signal, a lot of the mathematical work done in filter
design is done in the time domain, not in the frequency domain. By using
things like sample averaging, delays, and feedback, one can create an
 extraordinarily rich variety of digital filters.

For example, the following is a simple equation for a low-pass filter. This
equation just averages the last two samples of a signal (where x(n) is the
current sample) to produce a new sample. This equation is said to have aone-
sample delay. You can see easily that quickly changing (that is, high-
frequency) time domain values will be "smoothed" (removed) by this
equation.

x(n) = (x(n) + x(n - 1))/2

In fact, although it may look simple, this kind of filter design can be quite
difficult (although extremely important). How do you know which
frequencies you’re removing? It’s not intuitive, unless you’re well schooled in
digital signal processing and filter theory, have some background in
mathematics, and know how to move from the time domain (what you have)
to the frequency domain (what you want) by averaging, delaying, and so on.
<-- Back to Previous Page Next Section -->

©Burk/Polansky/Repetto/Roberts/Rockmore. All rights reserved.

  

<-- Back to Previous Page TOC Next Section -->

Chapter 4: The Synthesis of Sound by Computer

Section 4.4: Formant Synthesis

 Formant synthesis is a special but important case of subtractive synthesis.
Part of what makes the timbre of a voice or instrument consistent over a
wide range of frequencies is the presence of fixed frequency peaks,
called formants.

These peaks stay in the same frequency range, independent of the actual
(fundamental) pitch being produced by the voice or instrument. While there
are many other factors that go into synthesizing a realistic timbre, the use of
formants is one way to get reasonably accurate results.

Figure 4.10  A trumpet plays two different notes, a perfect fourth apart, but the
formants (fixed resonances) stay in the same places.

Resonant Structure

The location of formants is based on the resonant physical structure of the

sound-producing medium. For example, the body of a certain violin exhibits
a particular set of formants, depending upon how it is constructed. Since
Applet 4.7
most violins share a similar shape and internal construction, they share a
Formants similar set of formants and thus sound alike. In the human voice, the vocal
tract and nasal cavity act as the resonating body. By manipulating the shape
and size of that resonant space (i.e., by changing the shape of the mouth and
throat), we change the location of the formants in our voice. We recognize
different vowel sounds mainly by their formant placement. Knowing that,
we can generate some fairly convincing synthetic vowels by manipulating
 formants in a synthesized set of tones. A number of books list actual formant
frequency values for various voices and vowels (including Charles Dodge’s
highly recommended standard text, Computer Music—Dodge is a great
pioneer in computer music voice synthesis).

Composing with Synthetic Speech

Generating really good and convincing synthetic speech and singing voices
is more complex than simply moving around a set of formants—we haven’t
mentioned anything about generating consonants, for example. And no
Xtra bit 4.5
speech synthesis system relies purely on formant synthesis. But, as these
Change of examples illustrate, even very basic formant manipulation can generate
resonance sounds that are undoubtedly "vocal" in nature.

Figure 4.11  A spectral picture of the voice, showing formants. Graphic courtesy
of the alt.usage.english newsgroup.
 Xtra bit 4.6
Formant
manipulations

Figure 4.12  Composer Paul Lansky.

"Notjustmoreidlechatter" was made on a DEC MicroVaxII computer in 1988. All the

"chatter" pieces (there are three in the set) use techniques known as linear predictive
coding, granular synthesis, and a variety of stochastic mixing techniques.

Soundfile 4.12 Paul Lansky is a well-known composer and researcher of computer music who teaches
"Notjustmoreidlech at Princeton University. He has been a leading pioneer in software design, voice
atter" of Paul
synthesis, and compositional techniques.
Lansky

Used with permission from Paul Lansky.

Paul Lansky writes:

"Over ten years ago I wrote three 'chatter' pieces, and then decided to quit while I was
ahead. The urge to strike again recently overtook me, however, and after my lawyer
Soundfile 4.13 assured me that the statute of limitations had run out on this particular offense, I once
"idlechatterjunior" again leapt into the fray. My hope is that the seasoning provided by my labors in the
of Paul Lansky
intervening years results in something new and different. If not, then look out for 'Idle
from 1999
Chatter III'... ."

Used with permission from Paul Lansky.

 Composer Sarah Myers used an interview with her friend Gili Rei as the source material
for her composition. "Trajectory of Her Voice" is a ten-part canon that explores the
musical qualities of speech. As the verbal content becomes progressively less
comprehensible as language, the focus turns instead to the sonorities inherent in her
Soundfile 4.14 voice.
Composition by
composer Sarah
This piece was composed using the Cmix computer music language in 1998 (Cmix was
Myers entitled
"Trajectory of Her written by Paul Lansky).
Voice"

Over the years, computer voice simulations have become better and better. They still
sound a bit robotic, but advances in voice synthesis and acoustic technology make
voices more and more realistic. Bell Telephone Laboratories has been one of the leading
research facilities for this work, which is expected to become extremely important in the
Soundfile 4.15 near future.
Synthetic speech
example, "Fred"
voice from the
Macintosh
computer

In this piece, based on a reading by Australian sound-poet Chris Mann, the composer
tries to separate vowels and consonants, moving them each to a different speaker. This
was inspired by an idea of Mann's, who always wanted to do a "headphone piece" in
which he spoke and the consonants appeared in one ear, the vowels in another.
Soundfile 4.16
Carter Sholz’s 1-
minute piece
"Mannagram"

One of the most interesting examples of formant usage is in playing

the trump,sometimes called the jaw-harp. Here, a metal tine is plucked and the shape of
the vocal cavity is used to create different pitches.

Soundfile 4.17
The trump

<-- Back to Previous Page Next Section -->

©Burk/Polansky/Repetto/Roberts/Rockmore. All rights reserved.

  
<-- Back to Previous Page TOC Next Section -->

Chapter 4: The Synthesis of Sound by Computer

Section 4.5: Amplitude Modulation

Introduction to Modulation

We might be more familiar with the term modulation in relationship to radio.

Radio transmission utilizesamplitude modulation (AM) and frequency
modulation(FM), but we too can create complex waveforms by using these
techniques.

Modulated signals are those that are changed regularly in time, usually by
other signals. They can get pretty complicated. For example, modulated signals
can modulate other signals! To create a modulated signal, we begin with two
or more oscillators (or anything that produces a signal) and combine the output
signals of the oscillators in such a way as to modulate the amplitude,
frequency, and/or phase of one of the oscillators.

In the amplitude modulation equation, the DC offset (a signal that is essentially

a straight line) is added to the signal m(t) and multiplied by a sinusoid with
frequency fc. Ac is the carrier amplitude, and ka is the modulation index.

Applet 4.9 shows what happens, in the case of frequency modulation, if the
modulating signal is low frequency. In that case, we’ll hear something
like vibrato (a regular change in frequency, or perceived pitch). We can also
Applet 4.8 modulate amplitude in this way (tremolo), or even formant frequencies if we
LFO
modulation want. Low-frequency modulations (that is, modulators that themselves are
low-frequency signals) can produce interesting sonic effects.

But for making really complex sounds, we are generally interested in high-
frequency modulation. We take two audio frequency signals and multiply them
together. More precisely, we start with a carrier oscillator and attach
a modulating oscillator to modify and distort the signal that the carrier
oscillator puts out. The output of the carrier oscillator can include its original
signal and the sidebands or added spectra that are generated by the modulation
process.

Amplitude Modulation

Figure 4.13 shows how we might construct a computer music instrument to do

amplitude modulation. The two half-ovals are often called unit generators, and
they refer to some software device like an oscillator, a mixer, a filter, or an
envelope generator that has inputs and outputs and makes and transforms
digital signals.
 Figure 4.13 Amplitude modulation, two operator case.

A low-pass moving filter that uses a sine wave to control a sweep between 0 Hz and 500
Hz.

Soundfile 4.18
Low-pass
moving filter
(modulated by
sine)
 A high-pass moving filter that uses a sine wave to control a sweep between 5,000 Hz and
15,000 Hz.

Soundfile 4.19
High-pass
moving filter
(modulated by
sine)

A low-pass moving filter that uses a sawtooth wave to control a sweep between 0 Hz and
500 Hz.

Soundfile 4.20
Low-pass
moving filter
(modulated by
sawtooth)

A high-pass moving filter that uses a sawtooth wave to control a sweep between 5,000 Hz
and 15,000 Hz.

Soundfile 4.21
High-pass
moving filter
(modulated by
sawtooth)
 Figure 4.14  James Tenney’s "Phases," one of the earliest and still most interesting
pieces of computer-assisted composition. The pictures above are his "notes" for the
piece, which constitute a kind of score.

Tenney made use of some simple modulation trajectories to control timbral

parameters over time (like amplitude modulation rate, spectral upper limit, note-
duration, and so on). By simply coding these functions in the computer and linking
the output to the synthesis engine, Tenney was able to realize a number of highly
original works in which he controlled the overall, large-scale process, but the micro-
structure was largely determined by the computer making use of his curves.

"Phases" was released on an Artifact CD of James Tenney’s computer music.

<-- Back to Previous Page Next Section -->

©Burk/Polansky/Repetto/Roberts/Rockmore. All rights reserved.

  
<-- Back to Previous Page TOC Next Section -->

Chapter 4: The Synthesis of Sound by Computer

Section 4.6: Waveshaping

Waveshaping is a popular synthesis-and-transformation technique that

turns simple sounds into complex sounds. You can take a pure tone, like a
sine wave, and transform it into a harmonically rich sound by changing its
shape. A guitar fuzz box is an example of a waveshaper. The unamplified
electric guitar sound is fairly close to a sine wave. But the fuzz box
amplifies it and gives it sharp corners. We have seen in earlier chapters
that a signal with sharp corners has lots of high harmonics. Sounds that
have passed through a waveshaper generally have a lot more energy in
their higher-frequency harmonics, which gives them a "richer" sound.

Simple Waveshaping Formulae

A waveshaper can be described as a function that takes the original

signal x as input and produces a new output signaly. This function is
called the transfer function.

y = f(x)

This is simple, right? In fact, it’s much simpler than any other function
we’ve seen so far. That’s because waveshaping, in its most general form,
is just any old function. But there’s a lot more to it than that. In order to
change the shape of the function (and not just make it bigger or smaller),
the function must be nonlinear, which means it has exponents greater than
1, or transcendental (like sines, cosines, exponentials, logarithms, etc.).
You can use almost any function you want as a waveshaper. But the most
useful ones output zero when the input is zero (that’s because you usually
don’t want any output when there is no input).
0 = f(0)

Let’s look at a very simple waveshaping function:

y = f(x) = x * x * x = x3

What would it look like to pass a simple sine wave that varied from -
1.0 to 1.0 through this waveshaper? If our input x is sin(wt), then:

y = x3 = sin3(wt)

If we plot both functions (sin(x) and the output signal), we can see that the
original input signal is very round, but the output signal has a narrower
peak. This will give the output a richer sound.

Figure 4.15  Waveshaping by x3.

This example gives some idea of the power of this technique. A simple
function (sine wave) gets immediately transformed, using simple math
and even simpler computation, into something new.

One problem with the y = x3 waveshaper is that forx-values outside the

range –1.0 to +1.0, y can get very large. Because computer music systems
(especially sound cards) generally only output sounds between –1.0 and
+1.0, it is handy to have a waveshaper that takes any input signal and
outputs a signal in the range –1.0 to +1.0. Consider this function:
 y = x / (1 + |x|)

When x is zero, y is zero. Plug in a few  numbers for x, like 0.5, 7.0,
1,000.0, –7.0, and see what you get. As x gets larger (approaches positive
infinity), y approaches +1.0 but never reaches it. As x approaches negative
infinity, yapproaches –1.0 but never reaches it. This kind of curve is
sometimes called soft clipping because it does not have any hard edges. It
can give a nice "tubelike" distortion sound to a guitar. So this function has
some nice properties, but unfortunately it requires a divide, which takes a
lot more CPU power than a multiply. On older or smaller computers, this
can eat up a lot of CPU time (though it’s not much of a problem
nowadays).

Here is another function that is a little easier to calculate. It is designed for

input signals between –1.0 and +1.0.

y = 1.5x - 0.5x3

This applet plays a sine wave through various waveshaping formulae.

Applet 4.9
Changing the shape
of a waveform

Chebyshev Polynomials

A transfer function is often expressed as a polynomial, which looks like:

y = f(x) = d0 + d1x + d2x2 + d3x3 + ... + dNxN

The highest exponent N of this polynomial is called the "order" of the

polynomial. In Applet 4.9, we saw that the x2formula resulted in a
doubling of the pitch. So a polynomial of order 2 produced strong second
harmonics in the output. It turns out that a polynomial of order N will only
generate frequencies up to the Nth harmonic of the input sine wave. This
is a useful property when we want to limit the high harmonics to a value
below the Nyquist rate to avoid aliasing.

Back in the 19th century, Pafnuty Chebyshev discovered a set of

polynomials known as the Chebyshev polynomials. Mathematicians like
them for lots of different reasons, but computer musicians like them
because they can be used to make weird noises, er, we mean music. These
Chebyshev polynomials have the property that if you input a sine wave of
amplitude 1.0, you get out a sine wave whose frequency is N times the
frequency of the input wave. So, they are like frequency multipliers. If the
amplitude of the input sine wave is less than 1.0, then you get a complex
mix of harmonics. Generally, the lower the amplitude of the input, the
lower the harmonic content. This gives musicians a single number,
sometimes called the distortion index, that they can tweak to change the
harmonic content of a sound. If you want a sound with a particular
mixture of harmonics, then you can add together several Chebyshev
polynomials multiplied by the amount of the harmonic that you desire. Is
this cool, or what?

Here are the first few Chebyshev polynomials:

T0(x) = 1
T1(x) = x
T2(x) = 2x2 – 1
T3(x) = 4x3 – 3x
T4(x) = 8x4 – 8x2 + 1

You can generate more Chebyshev polynomials using thisrecursive

formula (a recursive formula is one that takes in its own output as the next
input):

Tk+1(x) = 2xTk(x) – Tk–1(x)

Table-Based Waveshapers

Doing all these calculations in realtime at audio rates can be a lot of work,
even for a computer. So we generally precalculate these polynomials and
put the results in a table. Then when we are synthesizing sound, we just
take the value of the input sine wave and use it to look up the answer in
the table. If you did this during an exam it would be called cheating, but
in the world of computer programming it is called optimization.

One big advantage of using a table is that regardless of how complex the
original equations were, it always takes the same amount of time to look
up the answer. You can even draw a function by hand without using an
equation and use that hand-drawn function as your transfer function.
 This applet plays sine waves through polynomials and hand-drawn waves.

Applet 4.10
Waveshaping

Don Buchla’s Synthesizers

Don Buchla, a pioneering designer of synthesizers, created a series of

instruments based on digital waveshaping. One such instrument, known as
the Touche, was released in 1978. It had 16 digital oscillators that could
be combined into eight voices. The Touche had extensive programming
capabilities and had the ability to morph one sound into another.
Composer David Rosenboom worked with Buchla and developed much of
the software for the Touche. Rosenboom produced an album in 1981
called Future Travelusing primarily the Touche and the Buchla 300 Series
Music System.

Now that you know a lot about waveshaping, Chebyshev polynomials, and transfer
functions, we’ll show you what happens when the information gets into the wrong
hands!

Soundfile 4.22a These soundfiles are two recordings done in the mid-1980s, at the Mills College
Experimental Center for Contemporary Music, by one of our authors (Larry Polansky). They use
waveshaping: "Toyoji
an highly unusual live, interactive computer music waveshaping system.
Patch"

"Toyoji Patch" was a piece/software/installation written originally for trombonist and

composer Toyoji Tomita to play with. It was a real-time feedback system, which fed
live audio into the transfer function of a set of six digital waveshaping oscillators.
The hardware was an old S-100 68000-based computer system running the original
prototype of a computer music language called HMSL, including a GUI and set of
Soundfile 4.22b
Experimental
instrument drivers and utilities for controlling synthesizer designer Don Buchla’s
waveshaping: "Toyoji 400 series digital waveshaping oscillators.
Patch"
The system could be used with or without a live input, since it made use of an
external microphone to feed back its output to itself. The audio time domain signal
used a transfer function to modify itself, but you could also alter Chebyshev
coefficients in realtime, redraw the waveform, and control a number of other
parameters.

Both of these sound excerpts feature the amazing contemporary flutist Anne
LaBerge. In the first version, LaBerge is playing but is not recorded. She is in
another room, and the output of the system is fed back into itself through a
microphone. By playing, she could drastically affect the sound (since her flute went
immediately into the transfer function). However, although she’s causing the
 changes to occur, we don’t actually hear her flute. In the second version, LaBerge is
in front of the same microphone that’s used for feedback and recording.

In both versions, Polansky was controlling the mix and the feedback gain as well as
playing with the computer.
<-- Back to Previous Page Next Section -->

©Burk/Polansky/Repetto/Roberts/Rockmore. All rights reserved.

  

<-- Back to Previous Page TOC Next Section -->

Chapter 4: The Synthesis of Sound by Computer

Section 4.7: FM Synthesis

One goal of synthesis design is to find efficient algorithms, which don’t take a

lot of computation to generate rich sound palettes. Frequency modulation
synthesis (FM) has traditionally been one of the most popular techniques in this
Applet 4.11
regard, and it provides a good way to communicate some basic concepts about
Frequency sound synthesis. You’ve probably heard of frequency modulation—it is the
modulation technique used in FM radio broadcasting. We took a look at amplitude
modulation (AM) in Section 4.5.

History of FM Synthesis

FM techniques have been around since the early 20th century, and by the 1930s
FM theory for radio broadcasting was welldocumented and understood. It was
not until the 1970s, though, that a certain type of FM was thoroughly
researched as a musical synthesis tool. In the early 1970s, John Chowning, a
composer and researcher at Stanford University, developed some important
new techniques for music synthesis using FM.

Chowning’s research paid off. In the early 1980s, the Yamaha Corporation
introduced their extremely popular DX line of FM synthesizers, based on
Chowning’s work. The DX-7 keyboard synthesizer was the top of their line,
and it quickly became thedigital synthesizer for the 1980s, making its mark on
both computer music and synthesizer-based pop and rock. It’s the most popular
synthesizer in history.

FM turned out to be good for creating a wide variety of sounds, although it is

not as flexible as some other types of synthesis. Why has FM been so useful?
Well, it’s simple, easy to understand, and allows users to tweak just a few
"knobs" to get a wide range of sonic variation. Let’s take a look at how FM
works and listen to some examples.

Figure 4.16  The Yamaha DX-7 synthesizer was an extraordinarily popular

instrument in the 1980s and was partly responsible for making electronic and
computer music a major industry. 

Thanks to Joseph Rivers and The Audio Playground Synthesizer Museum for this
photo.

Simple FM

In its simplest form, FM involves two sine waves. One is called the modulating
wave, the other the carrier wave. The modulating wave changes the frequency
of the carrier wave. It can be easiest to visualize, understand, and hear when the
modulator is low frequency.
 Figure 4.17  Frequency modulation, two operator case.

Vibrato

Carrier: 500 Hz; modulator frequency: 1 Hz; modulator index: 100.

Soundfile 4.23
Vibrato sound

FM can create vibrato when the modulating frequency is less than 30 Hz.
Okay, so it’s still not that exciting—that’s just because everything is
moving slowly. We’ve created a very slow, weird vibrato! That’s because we
were doing low-frequency modulation. In Soundfile 4.23, the frequency (fc) of
the carrier wave is 500 Hz and the modulating frequency (fm) is 1 Hz. 1 Hz
means one complete cycle each second, so you should hear the frequency of
the carrier rise, fall, and return to its original pitch once each second.

fc = carrier frequency, m(t) = modulating signal, and Ac= carrier amplitude.

Note that the frequency of the modulating wave is the rate of change in the
carrier’s frequency. Although you can’t tell from the above equation, it also
turns out that the amplitude of the modulator is the degree of change of the
carrier’s frequency, and the waveform of the modulator is the shape of change
of the carrier’s frequency.

In Figure 4.17 showing the unit generator diagram for frequency modulation
(remember, we showed you one of these in Section 4.5), note that each of the
sine wave oscillators has two inputs: one for frequency and one for amplitude.
For our modulating oscillator we are using 1 Hz as the frequency, which
becomes fm to the carrier (that is, the frequency of the carrier is changed 1 time
per second). The modulator’s amplitude is 100, which will determine how
muchthe frequency of the carrier gets changed (at a rate of 1 time per second).

The amplitude of the modulator is often called the modulation depth, since this

value determines how high and low the frequency of the carrier wave will go.
In the sound example, the fc ranges from 400 Hz to 600 Hz (500 Hz – 100 Hz
to 500 Hz + 100 Hz). If we change the depth to 500 Hz, then our fcwould range
from 0 Hz to 1,000 Hz. Humans can only hear sounds down to about 30 Hz, so
there should be a moment of "silence" each time the frequency dips below that
point.
 Carrier: 500 Hz, modulator frequency: 1 Hz, modulator index: 500.

Soundfile 4.24
Vibrato sound

Generating Spectra with FM

If we raise the frequency of the modulating oscillator above 30 Hz, we can start
to hear more complex sounds. We can make an analogy to being able to see the
spokes of a bike wheel if it rotates slowly, but once the wheel starts to rotate
faster a visual blur starts to occur.

So it is with FM: when the modulating frequency starts to speed up, the sound
becomes more complex. The tones you heard in Soundfile 4.24 sliding around
are called sidebandsand are extra frequencies located on either side of the
carrier frequency. Sidebands are the secret to FM synthesis. The frequencies of
the sidebands (called, as a group, the spectra) depend on the ratio of fc to fm.
John Chowning, in a famous article, showed how to predict where those
sidebands would be using a simple mathematical idea called Bessel
functions.By controlling that ratio (called the FM index) and using Bessel
functions to determine the spectra, you can create a wide variety of sounds,
from noisy jet engines to a sweet-sounding Fender Rhodes.
 Figure 4.18  FM sidebands.

Soundfiles 4.25 through 4.28 show some simple two-carrier FM sounds with
modulating frequencies above 30 Hz.

Carrier: 100 Hz; modulator frequency: 280 Hz; FM index: 6.0 -> 0.

Soundfile 4.25
Bell-like sound

Carrier: 250 Hz; modulator frequency: 175 Hz; FM index: 1.5 -> 0.

Soundfile 4.26
Bass clarinet-
type sound
 Carrier: 700 Hz; modulator frequency: 700 Hz; FM index: 5.0 -> 0.

Soundfile 4.27
Trumpet-like
sound 

Carrier: 500 Hz; modulator frequency: 500 -> 5,000 Hz; FM index: 10.

Soundfile 4.28
FM sound

An Honest-to-Goodness Computer Music Example of FM (Using

Csound)

One of the most common computer languages for synthesis and sound
processing is called Csound, developed by Barry Vercoe at MIT. Csound is
popular because it is powerful, easy to use, public domain, and runs on a wide
variety of platforms. It has become a kind of lingua franca for computer music.
Csound divides the world of sound into orchestras, consisting of instruments
that are essentially unit-generator designs for sounds, and scores (or note lists)
that tell how long, loud, and so on a sound should be played from your
orchestra.

The code in Figure 4.19 is an example of a simple FM instrument in Csound.

You don’t need to know what most of it means (though by now a lot of it is
probably self-explanatory, like sr, which is just the sampling rate). Take a close
look at the following line:

asig foscil p4*env, cpspch(p5), 1,3,2,1 ;simple FM

Commas separate the various parts of this command.

Asig is simply a name for this line of code, so we can use it later (out
asig).

foscil is a predefined Csound unit generator, which is just a pair of

oscillators that implement simple frequency modulation (in a compact
way).

p4*env states that the amplitude of the oscillator will be multiplied by

an envelope (which, as seen in Figure 4.19, is defined with the
 Csound linseg function).

cpspch(p5) looks to the fifth value in the score (8.00,which will be a

middle C) to find what the fundamental of the sound will be. The next
three values determine the carrier and modulating frequencies of the
two oscillators, as well as the modulation index (we won’t go into what
they mean, but these values will give us a nice, sweet sound). The final
value (1) points to a sine wave table(look at the first line of the score).

Yes, we know, you might be completely confused, but we thought you’d like to
see a common language that actually uses some of the concepts we’ve been
discussing!

Computer Code for an FM Instrument and a Score to Play the

Instrument in Csound

Figure 4.19  This is a Csound Orchestra and Score blueprint for a simple FM

synthesis instrument. Any number of basic unit generators, such as oscillators, adders,
multipliers, filters, and so on, can be combined to create complex instruments.

Some music languages, like CSound, make extensive use of the unit generator
model.Generally, unit generators are used to create instruments (the orchestra), and
 then a set of instructions (a score) is created that tells the instruments what to do.

Now that you understand the basics of FM synthesis, go back to the beginning
of this section and play with Applet 4.12. FM is kind of interesting
theoretically, but it’s far more fun and educational to just try it out.
<-- Back to Previous Page Next Section -->

©Burk/Polansky/Repetto/Roberts/Rockmore. All rights reserved.

  

<-- Back to Previous Page TOC Next Section -->

Chapter 4: The Synthesis of Sound by Computer

Section 4.8: Granular Synthesis

When we discussed additive synthesis, we learned that complex sounds can be

created by adding together a number of simpler ones, usually sets of sine
waves.Granular synthesis uses a similar idea, except that instead of a set of
sine waves whose frequencies and amplitudes change over time, we use many
thousands of very short(usually less than 100 milliseconds) overlapping
soundbursts or grains. The waveforms of these grains are often sinusoidal,
although any waveform can be used. (One alternative to sinusoidal waveforms
is to use grains of sampled sounds, either pre-recorded or captured live.) By
manipulating the temporal placement of large numbers of grains and their
frequencies, amplitude envelopes, and waveshapes, very complex and time-
variant sounds can be created.
 This applet lets you granulate a signal and alter some of the typical parameters of granular
synthesis, a popular synthesis technique.

Applet 4.12
Granular
synthesis

Figure 4.20  A grain is created by taking a waveform, in this case a sine wave, and
multiplying it by an amplitude envelope.

How would a different amplitude envelope, say a square one, affect the shape of the
grain? What would it do to the sound of the grain?

Clouds of Sound

Granular synthesis is often used to create what can be thought of as "sound

 clouds"—shifting regions of sound energy that seem to move around a sonic
space. A number of composers, like Iannis Xenakis and Barry Truax, thought
of granular synthesis as a way of shaping large masses of sound by using
granulation techniques. These two composers are both considered pioneers of
this technique (Truax wrote some of the first special-purpose software for
granular synthesis). Sometimes, cloud terminology is even used to talk about
ways of arranging grains into different sorts of configurations.

Figure 4.21  Visualization of a granular synthesis "score." Each dot represents a grain

at a particular frequency and moment in time. An image such as this one can give us a
good idea of how this score might sound, even though there is some important
information left out (such as the grain amplitudes, waveforms, amplitude envelopes,
and so on).

What sorts of sounds does this image imply? If you had three vocal performers, one
for each "cloud," how would you go about performing this piece? Try it!

This is an excerpt of a composition by computer music composer and researcher Mara

Helmuth titled "Implements of Actuation." The sounds of an electric mbira and bicycle
wheels are transformed via granular synthesis.

Soundfile 4.29 There are a great many commercial and public domain applications to do granular synthesis
"Implements of because it is relatively easy to implement and the sounds can be very interesting and
Actuation"
attractive.
<-- Back to Previous Page Next Section -->
©Burk/Polansky/Repetto/Roberts/Rockmore. All rights reserved.

  

<-- Back to Previous Page TOC Next Chapter -->

Chapter 4: The Synthesis of Sound by Computer

Section 4.9: Physical Modeling

We’ve already covered a bit of material on physical modeling without even

telling you—the ideas behind formant synthesis are directly derived from our
knowledge of the physical construction and behavior of certain instruments.
Like all of the synthesis methods we’ve covered, physical modeling is not one
specific technique, but rather a variety of related techniques. Behind them all,
however, is the basic idea that by understanding how sound/vibration/air/string
behaves in some physical system (like an instrument), we can model that
system in a computer and thus synthetically generate realistic sounds.

Karplus-Strong Algorithm

Let’s take a look at a really simple but very effective physical model of a
plucked string, called the Karplus-Strong algorithm (so named for its principal
inventors, Kevin Karplus and Alex Strong). One of the first musically useful
physical models (dating from the early 1980s), the Karplus-Strong algorithm
has proven quite effective at generating a variety of plucked-string sounds
(acoustic and electric guitars, banjos, and kotos) and even drumlike timbres.
 Fun with the Karplus-Strong plucked string algorithm.

Applet 4.13
Karplus-Strong
plucked string
algorithm

Here’s a simplified view of what happens when we pluck a string: at first the
string is highly energized and it vibrates like mad, creating a
fairly complex (meaning rich in harmonics) sound wave whose fundamental
frequency is determined by the mass and tension of the string. Gradually,
thanks to friction between the air and the string, the string’s energy is depleted
and the wave becomes less complex, resulting in a "purer" tone with fewer
harmonics. After some amount of time all of the energy from the pluck is gone,
and the string stops vibrating.

If you have access to a stringed instrument, particularly one with some very
low notes, give one of the strings a good pluck and see if you can see and hear
what’s happening per the description above.

How a Computer Models a Plucked String with the Karplus-Strong

Algorithm

Now that we have a physical idea of what’s happening in a plucked string, how
can we model it with a computer? The Karplus-Strong algorithm does it like
this: first we start with a buffer full of random values—noise. (A buffer is just
some computer memory (RAM) where we can store a bunch of numbers.) The
numbers in this buffer represent the initial energy that is transferred to the
string by the pluck. The Karplus-Strong algorithm looks like this:

To generate a waveform, we start reading through the buffer and using the
values in it as sample values. If we were to just keep reading through the buffer
over and over again, what we’d get would be a complex, pitched waveform. It
would be complex because we started out with noise, but pitched because we
would be repeating the same set of random numbers. (Remember that any time
we repeat a set of values, we end up with a pitched (periodic) sound. The pitch
we get is directly related to the size of the buffer (the number of numbers it
contains) we’re using, since each time through the buffer represents one
complete cycle (or period) of the signal.)

Now here’s the trick to the Karplus-Strong algorithm: each time we read a
value from the buffer, we average it with the last value we read. It is this
averaged value that we use as our output sample. We then take that averaged
sample and feed it back into the buffer. That way, over time, the buffer gets
more and more averaged (this is a simple filter, like the averaging filter
described in Section 3.1). Let’s look at the effect of these two actions
separately.

Averaging and Feedback

First, what happens when we average two values? Averaging acts as a low-

pass filter on the signal. Because we’re averaging the signal, the signal changes
less with each sample, and by limiting how quickly it can change we’re
limiting the number of high frequencies it can contain (since high frequencies
have a high rate of change). So, averaging a signal effectively gets rid of high
frequencies, which according to our string description we need to do—once the
string is plucked, it should start losing harmonics over time.

The "over time" part is where feeding the averaged samples back into the
buffer comes in. If we were to just keep averaging the values from the buffer
but never actually changing them (that is, sticking the average back into the
buffer), then we would still be stuck with a static waveform. We would keep
averaging the same set of random numbers, so we would keep getting the same
results.

Instead, each time we generate a new sample, we stick it back into the buffer.
That way our waveform evolves as we move through it. The effect of this low-
pass filtering accumulates over time, so that as the string "rings," more and
more of the high frequencies are filtered out of it. The filtered waveform is
then fed back into the buffer, where it is filtered again the next time through,
and so on. After enough times through the process, the signal has been
averaged so many times that it reaches equilibrium—the waveform is a flat line
the string has died out.
Figure 4.22  Applying the Karplus-Strong algorithm to a random waveform. After 60
passes through the filter/feedback cycle, all that’s left of the wild random noise is a
gently curving wave.

The result is much like what we described in a plucked string: an initially complex,
periodic waveform that gradually becomes less complex over time and ultimately
fades away.
 Figure 4.23  Schematic view of a computer software implementation of the basic
Karplus-Strong algorithm.

For each note, the switch is flipped and the computer memory buffer is filled with
random values (noise). To generate a sample, values are read from the buffer and
averaged. The newly calculated sample is both sent to the output stream and fed back
into the buffer. When the end of the buffer is reached, we simply wrap around and
continue reading at the beginning. This sort of setup is often called a circular
buffer. After many iterations of this process, the buffer’s contents will have been
transformed from noise into a simple waveform.

If you think of the random noise as a lot of energy and the averaging of the buffer as a
way of lessening that energy, this digital explanation is not all that dissimilar from
what happens in the real, physical case.

Thanks to Matti Karjalainen for this graphic.

Physical models generally offer clear, "real world" controls that can be used to
play an instrument in different ways, and the Karplus-Strong algorithm is no
exception: we can relate the buffer size to pitch, the initial random numbers in
the buffer to the energy given to the string by plucking it, and the low-pass
buffer feedback technique to the effect of air friction on the vibrating string.

Many researchers and composers have worked on the plucked string sound as a kind of
basic mode of physical modeling.

One researcher, engineer Charlie Sullivan (who we're proud to say is one of our Dartmouth
Soundfile 4.30 colleagues!) built a "super" guitar in software. Here’s the heavy metal version of "The Star
Super guitar Spangled Banner."

Understand the Building Blocks of Sound

Physical modeling has become one of the most powerful and important current
techniques in computer music sound synthesis. One of its most attractive
features is that it uses a very small number of easy-to-understand building
 blocks—delays, filters, feedback loops, and commonsense notions of how
instruments work—to model sounds. By offering the user just a few intuitive
knobs (with names like "brightness," "breathiness," "pick hardness," and so
on), we can use existing sound-producing mechanisms to create new, often
fantastic, virtual instruments.

Soundfile 4.31
An example of
Perry Cook’s
SPASM

Figure 4.24  Part of the interface from Perry R. Cook’s SPASM singing voice
software. Users of SPASM can make Sheila, a computerized singer, sing. Perry Cook
has been one of the primary investigators of musically useful physical models. He’s
released lots of great physical modeling software and source code.

<-- Back to Previous Page Next Chapter -->

©Burk/Polansky/Repetto/Roberts/Rockmore. All rights reserved.