Introduction to Music Informatics

Donald Byrd
Rev. 4 Nov. 2007

Copyright © 2006-07, Donald Byrd

 Welcome
• This is a very exciting time for music!
• Music informatics is hard to teach
– Music and technology are both changing quickly
– Few students have much background in both
– Solution: compromise
• Spend some time bringing each “up to speed”
• Either limit tech. demands in both areas, or…
• Work in teams
• How much background does everyone have?

30 Aug. 2006 2
 Class Background: Results 1
• Music: everyone has good background; several
have degrees
• Programming: all over the map
– Few know the R language at all
• Experience w/ music recommenders, iTunes,
etc.: variable, but most don’t have much

29 Aug. 2007 3
 Class Background: Results 2
• Have 10 students, 1 auditor
• Music Theory
– Q1: 10; Q2: 10; Q3: 8 + 2 partial
• Best of any class I’ve ever had
• Audio & Computer Technology
– Q4: 6; Q5: 3 + 7 partial; Q6: 6 + 4 partial
– Q7: 5 + 4 partial; Q8: 1 + 4 partial; Q9: 3 + 4 partial
• Q8 is hard, esp. to say briefly: cf. my Vocabulary!

rev. 9 Sep. 07 4
 Class Plans & Procedures 1
• Communication
– Web site (has many helpful resources)
• …including “Miscellaneous Class Procedures”
– E-mail
– Announcements in class
– OnCourse? not for now
• Goals & competencies
• Assignments & grading: cf. class syllabus
– Project & presentations most important
– Preparation & participation are important
– Participation => avoid “death by PowerPoint”
– Importance/frequency of quizzes is TBD

rev. 9 Sep. 07 5
 Class Plans & Procedures 2
• What about programming?
– More “reading knowledge” than writing
– …but will write some, in R
• Midterm presentation
• Final project: presentation & paper
– List of possibilities to appear “soon”
• Some with programming, some without
– Proposal due in a few weeks
– Presentations last weeks of class
– Paper due last week of classes
• Will have specific grading rubrics
rev. 27 Aug. 07 6
 What Is Music Informatics?
• Some definitions
1. Like music information technology, but more
research-oriented
2. Music information retrieval & related areas
• ISMIR = “International Conference (Symposium) on
Music Information Retrieval”, but… Change name?
3. Discipline #1 plus a curriculum (SoI def.)
• Tim Bell’s overview (2006)
– Moving music from one “form” or “place” to another
(my words)

26 Aug. 2007 7
 Visual
display
Digital
image

Human
memory

Digital
semantic

Audio
30 Aug. 2006 8
 Visual
display
Digital
image
Weak links

Human
memory

Digital
semantic

Audio
30 Aug. 2006 9
 We’ll Consider All Kinds of Music
• Existing music covers a huge range
• I follow Schickele/Ellington philosophy
– Peter Schickele: All musics are created equal
– Duke Ellington: If it sounds good, it is good
• Great, but makes many problems much harder for
computer
– If pop only, can rely on existence of lyrics
– If classical only, can rely on dynamic range
– If pop or folk, can assume texture is melody &
accompaniment
– If many genres, can assume 12 notes/octave
rev. 30 Aug. 07 10
We’ll Consider All Representations of
Music (Preview)

Digital Audio
Audio (e.g., CD, MP3):
like speech

Time-stamped Events
Time-stamped Events
(e.g., MIDI file): like
unformatted text

Music Notation:
Musiclike
Notation
text with complex
formatting

27 Jan. 11
 Music & Research 1
• Different approaches to research
– Highly controlled environments
• More objective, easier to quantify results
– Less controlled
• Better for exploration
– Sim. to “naturalists 1st, scientists later” idea
• Music isn’t a science! But…
– What music “scholars” do similar to what “scientists”
(esp. social scientists) do
– In a way, physical scientists have a much easier job!

30 Aug. 2006 12
 Music & Research 2
• Different approaches to research
– What “scholars” do is similar to what “scientists” (esp. social
scientists) do
– Good reason: music is subtle & subjective
– …but it’s not magic!
• David Huron: Explanatory Goals of Music Analysis
– Learned as a music student: “methodology is fetish; rigour is a
form of self-deception”
– …as a music scholar: methodology is “simply a way of
internalizing the lessons learned from past scholarly mistakes”
• The “Scholarly Method”

30 Aug. 2006 13
 Music & Research 3
• Example: Gladwell’s article “The Formula”, on predicting
hit songs or movies by computer
– Not a scholarly article, but is it plausible?
– Physical aspects of audio are easy for computers
– …but what we care about is almost always perceptual
– What are concepts like melody & harmony?
• Core Competency
– “Understand the difference between physical (objective) and
perceptual (subjective) parameters of musical sound, and why
computers can deal much more easily with the former, while
people can deal much more easily with the latter”

31 Aug. 2007 14
 Classification: Logician General’s
Warning
• Classification is dangerous to your understanding
– Almost everything in the real world is messy
– Absolute correlations between characteristics are rare
– Example: some mammals lay eggs; some are “naked”
– Example: is the piano a keyboard, a string, or a percussion instrument?
• People say “an X has characteristics A, B, C…”
• Nearly always mean “an X has A, & usually B, C…”
• Leads to:
– People who know better claiming absolute correlations
– Arguments among experts over which characteristic is most fundamental
– Don changing his mind
• But lack of classification is also dangerous to your understanding!
• Cf. version of this on my Teaching page

rev. 30 Aug. 07 15
 Examples of Music Informatics
Research/Technology in Action
• Listen Game
– Nice example of “games with a purpose”
– …but doesn’t seem to be usable anymore!
• sCrAmBlEd?HaCkZ!
– Audio mosaicing
• http://www.popmodernism.org/scrambledhackz/
• "Together" Listening Experiments
– PhD dissertation research by Matt Wright
– on perception of musical rhythm
• “Perceptual Attack Time”
• http://ccrma.stanford.edu/~matt/together/
rev. 12 Sep. 07 16
 Our Own Music
• “Music We're Interested In, and What's Special
About It?”
– Everyone contributes something
– Will play & briefly discuss (from standpoint of music
informatics) in class
– Audio files will be on Web site
– Will use for something
• Maybe test lossy compression, or play with audio-
recognition programs?
• Definitely, test audio segmentation program!
– Thanks to Nina Fales for the idea

30 Aug. 2007 17
 Materials for Studying Audio 1
• Waveforms & sounds
– Fourier Series Applet (www.falstad.com/fourier/)
– Simple “artificial” waveforms
• Sine, square, triangle, etc.
• Standard on old analog synths (Moog, etc.)
• Sine wave (tuning fork) is the simplest
• Sine function from trigonometry
– Fourier’s theorem
• Any periodic function (repeating waveform) = sum of
harmonically related sine waves
• Add up harmonically related sine ways to make
(approximation to) square wave, etc.

31 Aug. 2007 18
 Materials for Studying Audio 2
• What are interesting sounds really like?
– Static waveforms are simple, boring
• Not just sine, square, etc.: cf. the “random waveform
generator” (R demo)
– Acoustic instrument sounds are never static
– …a big reason they’re interesting
• Musical instrument samples
• Audacity audio editor
– For Windows, Mac OS 9 and X, Linux
– Download from http://audacity.sourceforge.net/

31 Aug. 2007 19
 Programming in R: No Problem!
• R is very interactive: can use as powerful calculator
• Assignments will be simple
• Much help available: from Don & other students
• Why R?
– NOT because it's great for statistics!
– easy to do simple things with it, including graphs and handling audio files
• probably not good for complex programs
– free, and available for all popular operating systems
– very interactive => easy to experiment
– has good documentation
– Prof. Raphael is using it, and he thinks it's good for music informatics
– Prof. Raphael is using it, and standardizing is a good thing

30 Aug. 2006 20
 Rudiments of R
• Originally for statistics; good for far more
• How to get R
– Web site: http://cran.us.r-project.org/
– Versions for Linux, Mac OS X, Windows
– Already on STC computers & in M373
• Tutorial (kind of math-heavy):
• http://xavier.informatics.indiana.edu/~craphael/teach/symbolic_music/
• Can use R interactively as a powerful graphing, musicing,
etc. calculator
• …but it’s not perfect: sometimes very cryptic

9 Sep. 07 21
 Representations of Music & Audio 1

• Three basic forms (representations) of music

– Audio: most important for most people (general public)
– MIDI files: often best/essential for some musicians, especially for
pop, rock, film/TV
– Notation: often best/ essential for musicians (even amateurs) &
music scholars
– Essential difference: how much explicit structure
• Music holdings of Library of Congress: over 10M items
– Includes over 6M pieces of sheet music and 100K’s of scores of
operas, symphonies, etc.: all notation!
• Differences are profound

rev. 8 Sep. 2006 22

 Representations of Music & Audio 2

Digital Audio
Audio (e.g., CD, MP3):
like speech

Time-stamped Events
Time-stamped Events
(e.g., MIDI file): like
unformatted text

Music Notation:
Musiclike
Notation
text with complex
formatting

1 Sep. 2006 23
 Rudiments of Musical Acoustics
• Need some musical acoustics for almost anything
in digital audio
• Need a bit now (use with R), more later
• Acoustics: part of physics
– Concepts like frequency & amplitude
• Psychoacoustics: part of psychology
– Concepts like pitch & loudness (perceptual)

rev. 9 Sep. 07 24
 Rudiments of Digital Audio
• Sampling rate => maximum frequency
– Human hearing goes up to ca. 15-20K Hz
– Need 2 samples per cycle
– CDs: 44,100
• Sample width (in bits per sample) => Signal-to-
Quantization Noise ratio
– About 6 dB per bit
– CDs: 16 bits = ca. 96 dB SQNR

rev. 9 Sep. 07 25
 Research in Music Informatics 1
• Scientific method vs. scholarly “method”
– Wide variation in use from field to field
– Music theory doesn’t use scientific method (EJI says)
– …but lots of music informatics does
• Debugging as an example of scientific method
– Hypothesis: program X has no bugs
– Methodology: look for bugs
– …but no amount of testing can prove the absence of bugs, just
their presence
– Cf. Einstein: “No experiments can prove me right; one experiment
can prove me wrong.”
– …& Dijkstra: “Testing can only show the presence of bugs, never
their absence.”
– A theorem (in math) can be proven; a theory (like a program)
can’t!
rev. 4 Nov. 07 26
 Research in Music Informatics 2
• Evaluating reliability of info sources
– Especially difficult on the Web: cf. www.dhmo.org
• It’s easy to jump to wrong conclusion: why?
– Backus on why musicians’ explanations in acoustics are almost
always wrong
– Cf. Logician General’s Classification Warning
– Almost everything in the world is messy
• Def. of “trombone” may be more clearcut than of “piano”
• …but it’s still not well-defined!

rev. 4 Nov. 07 27
 Research in Music Informatics 3
• How do you study a question in the most objective
possible way?
– If it’s a matter of perception, maybe by asking people what they
think or hear!
– Example: determining “Perceptual Attack Time” of a sound
• Cf. D. Huron on what he learned as a music student vs. as
a music scholar
– Methodology as way to avoid repeating mistakes
• Check out my “Information Sources for Music Informatics
Students”

rev. 19 Sep. 07 28