Outline

• Introduction

• Music Making

• MIDI and OSC

• GUIs

• Image Manipulation
Workshop #102 — SIGCSE 2016
Bill Manaris, College of Charleston • Timer and Scheduling

• Conclusion

1 2

What is JythonMusic? Why Python?

• JythonMusic is an open source environment for music • Traditional languages (C++, Java) evolved for large-scale programming.
making and creative programming activities.
• Emphasis on structure and discipline

• It is intended for musicians and programmers of all levels • Simple problems != simple programs
and backgrounds.
• Scripting languages (Perl, Python, TCL) designed for simplicity and
• JythonMusic is written in Python (gentle for beginners, flexibility.
powerful enough for experts). • Simple problems = simple, elegant solutions

• It provides libraries for music making, image manipulation, • More amenable to experimentation and incremental development
building graphical user interfaces (GUIs), and for
• Python: Ideal first language, useful throughout curriculum
connecting to external MIDI and OSC devices, such as
digital pianos, smartphones, and tablets. (Quoted from John Zelle, “Teaching Computer Science with Python”, SIGCSE 2003 Workshop)

3 4
 Courses CS Principles - 7 Big Ideas
• Creativity - computing is a creative human activity.

• JythonMusic has been used in • Abstraction - helps understand and solve problems.

• CITA 180 / MUSC 146 - “Introduction to Computer • Data and Information - facilitate the creation of knowledge.
Music and Aesthetics: Programming Music, Performing • Algorithms - tools for developing and expressing solutions.
Computers”
• Programming - creative process that produces computational
• HONS 381 - “Computer Music on a Laptop: artifacts.
Composing, Performing, Interacting”
• The Internet - interconnecting devices provide ways to solve
problems.

• Global Impact - computing enables innovation in other fields

including mathematics, science, humanities, and arts.

5 6

Motivation - Laptop Orchestra Image Sonification

bit.ly/CharlestonLaptopOrchestra https://vimeo.com/64110119

7 8
 Art Installation - Time Jitters Hyperinstrument #1 - Monterey Mirror

https://vimeo.com/64110119 bit.ly/montereymirror

9 10

Hyperinstrument #2 - SoundMorpheus Introduction

• JythonMusic libraries

• Music - primitives for creating music notes, phrases,

parts, and scores, for playing live, and for reading and
writing as MIDI or XML files

• also primitives for loading and looping audio files

bit.ly/soundmorpheus1

11 12
 Introduction Introduction
• JythonMusic libraries (cont’d) • JythonMusic libraries (cont’d)

• GUI - primitives for building graphical user interfaces • Image - primitives for supporting image sonification
for music performance and other interactive tasks and other types of image manipulation

• MIDI - primitives for connecting to external MIDI • Timer - primitives for scheduling events, such as
devices (e.g., pianos, guitars, synthesizers, etc.) specifying envelope manipulation of audio, as well as
constructing various animation tasks
• OSC - primitives for connecting to other devices via
Open Sound Control (e.g., smartphones, tablets, • Zipf - primitives for extracting measurements from
computers, synthesizers, etc.) musical data

13 14

Running JythonMusic
Programs • JEM (Jython Environment for Music)

• developed and maintained by Tobias Kohn

• JEM Editor

• Run, run selection, run line, run paragraph

(supports live coding)

• Console (>>>)

• Also, terminal window (low level), and other editors

15 16
 • developed and maintained by Tobias Kohn • terminal window (Mac shown below; Windows similar)

17 18

` Outline
• Introduction
• Hello World! — play a single note
• Music Making

• MIDI and OSC

• GUIs

• Image Manipulation

• Timer and Scheduling

• Conclusion

19 20
 Note
Music Library
• notes contain the simplest possible musical events,
• The music library defines functions for giving you consisting of pitch, duration, dynamic, and
control and expression over musical parameters panning.

• They are organized in three categories: • pitch (0 - 127, also constants, e.g., C4)

• Transcription: Note, Phrase, Part, Score

• duration (1.0 = quarter note, also constants, e.g., QN)

• dynamic (0 - 127, also constants, e.g., FF)

• Composition: Mod, View, Read / Write
• panning (0.0 - 1.0)
• Performance (e.g., live coding): AudioSample,
MidiSequence, LiveSample, Metronome, Play

21 22

Note Note
• how to create notes • note functions

23 24
 Pitch Pitch Constants

25 26

Duration Duration Constants

• note durations are represented using real numbers
(floats).

• 4.0 stands for whole note, 2.0 for a half note, 1.0 for a
quarter note, etc.

27 28
 TED Ed - How to read music Activity #2

• Play a different note

• A0 whole (WN) note, half volume (64), center pan (0.5)
• G5 half (HN) note, full volume (128), extreme right (1.0)
• rest (REST) whole (WN) note
• BF3 double whole (WN+WN) note, full volume (128),
center (0.5)
• durations are numbers - can be added, multiplied, etc.

29 30

Phrase Phrase
• how to create phrases
• phrases contain a sequence of notes
• Notes are played sequentially
• use Rest notes, for gaps

• phrases may also contain chords

• chords are concurrent pitches with same duration
• stay tuned…

• phrases have start times

• float, e.g., 0.0 (in QN time units)
• no start time means continue after previous phrase (if any)

31 32
 Phrase Example - Melody
• phrase functions

33 34

Activity #3 MIDI Instruments

• Write a program that plays the following • MIDI has 128 different instruments

.
.
.

notice the B flat

35 36
Example - Instrument, Tempo Activity #4
• Change the default instrument. Try a few different
options. Any preferences?

37 38

Example - Chords Part

• parts contain phrases to be played by a single
instrument
• phrases played in parallel, may overlap
• part has a MIDI channel (0-15)
• part may have a title (string)

• each part assigns MIDI instrument (0-127) to

MIDI channel (0-15)
• one instrument per channel!

• MIDI channel 9 reserved for percussion

• stay tuned…

39 40
 Part Part
• how to create parts • part functions

41 42

Example - String Quartet Live Coding with String Quartet

43 44
Example - Autumn Leaves Drums and Percussion
• MIDI channel 9 reserved for percussion
• Pitch of notes (in phrases) in a part going to channel 9,
determines which percussive instrument to play
• constants available

45 46

Drums and Percussion Example - Drums

47 48
 Example - Drum Machine Score
• scores contain parts

49 50

Example - Smoke on the Water Mod functions

• Mod class provides functions for modifying
phrases, parts and scores. These functions modify
the data given to them.

• for example, Mod.repeat() creates repetitions of the

given musical material:

• Mod.repeat(phrase, 41)

• Mod functions are organized by pitch, duration,

dynamic, and randomness.

51 52
 Mod functions Steve Reich - Piano Phase
Mod.repeat() - repeats material
Mod.shift() - changes start time
Mod.retrograde() - reverses notes
Mod.transpose() - changes pitch by number of steps
Mod.elongate() - changes note duration by factor
Mod.invert() - mirrors notes along horizontal axis (a pitch)
Mod.palindrome() - appends the retrograde of the notes
Mod.quantize() - rounds start time and duration of notes
Mod.randomize() - randomly changes note aspects
Mod.fadeIn(), Mod.fadeOut() - as name suggests
Mod.normalize() - maximizes volume (maintain differences)
Mod.compress() - maximizes volume (reduce differences)
…etc.

53 54

Create a Canon - Row your Boat Activity #5

• Modify your earlier program to create a canon
• use Mod functions, as per “Row your Boat”
• use different instruments
• asterisks ( ) below indicate start of new voice
(total of 4 voices)

55 56
 Outline
• Introduction
Midi Library
• Music Making • The MIDI library defines functions for
communicating with MIDI instruments and devices
• MIDI and OSC
• These devices need to be physically connected
• GUIs to your computer
• Image Manipulation • There are two classes: MidiIn, and MidiOut
• Timer and Scheduling • To use these classes, you need:
• Conclusion from midi import *

57 58

MidiIn Class Example MidiIn Program #1

• MidiIn objects receive incoming MIDI messages from MIDI devices

(e.g., MIDI keyboards, drumpads, etc.)

• these devices need to be physically connected (via wire) to your

computer

• To create a MidiIn object:

midiIn = MidiIn()

• creates a MidiIn object to receive incoming MIDI messages from

a device

• when called, it presents the user with a GUI to select a MIDI

device

59 60
 Example MidiIn Program #2
• onNoteOn() connects NOTE_ON events with a
callback function

• that function should accept four parameters:

• event type (an integer)

• channel (0 - 15)

• data1 (pitch, 0 - 127)

• data2 (volume, 0 - 127)

• also onNoteOff, onSetInstrument(), onInput()

61 62

MidiOut Class
• onInput() connects a particular event type (an
integer) with a callback function

• that function should accept four parameters: • MidiOut objects send MIDI messages to devices (e.g.,
synthesizer) that listen for incoming MIDI events
• event type (an integer) • these devices need to be physically connected (via wire) to
your computer
• channel (0 - 15)
• To create a MidiOut object:
• data1 (an integer, depends on event type) midiOut = MidiOut()

• data2 (an integer, depends on event type) • creates a MidiOut object to send MIDI messages to a device

• when called, it presents the user with a GUI to select a

• also note showMessages() and hideMessages() MIDI device

63 64
Example MidiOut Program Example - Draw Circles via MIDI

65 66

OSC Library
• The OSC (Open Sound Control) library defines
functions for communicating with OSC enabled devices
(e.g., smartphones, tablets, other computers, etc.)

• These devices may be anywhere on the Internet

(even overseas)

• There are two classes: OscIn, and OscOut

• To use these classes, you need:

from osc import *

67 68
 OscIn Class Example OscIn Program #1

• OscIn objects receive incoming OSC messages from

OSC devices (e.g., an OSC-enabled smartphone, or
tablet)

• To create a OscIn object:

oscIn = OscIn( port )

• creates an OscIn object to receive incoming messages

on port (1024 to 65535, not being used already)

• also make sure firewall permits this port (UPD)

69 70

Example OscIn Program #2

• When you run this program, it will output the
following:

OSC Server started:

Accepting OSC input on IP address
xxx.xxx.xxx.xxx at port 57110
(use this info to configure OSC clients)

• where “xxx.xxx.xxx.xxx” is the IP address of the

receiving computer (e.g., “192.168.1.223”)

• very helpful!

71 72
• Notice the special OSC address “/.*”
OscOut Class
• matches all incoming addresses
• OscOut objects send OSC messages to devices (e.g.,
• onInput() uses regular expressions to specify computers) that listen for incoming OSC events
OSC addresses
• To create a OscOut object:
• usually simple OSC addresses suffice, e.g.,
oscOut = OscOut( IPaddress, port )
• “/gyro”
• creates an OscOut object to send messages to
device at IPaddress (a string, e.g., “192.168.1.223”)
• “/accelerometer”, etc.
and port (1024 to 65535)
• also note showMessages() and hideMessages() • again, make sure firewall permits this port (UPD)

73 74

Example OscOut Program Example - Draw Circles via OSC

• OSC message arguments may be integer, float,

string, and boolean

• Exercise: Let’s try “talking” to each other

75 76
 Outline
• Introduction
GUI Library
• Music Making • The GUI library defines functions for building graphical
user interfaces. They are organized as follows:
• MIDI and OSC
• Display
• GUIs • Widgets: Label, Button, Checkbox, Slider, DropDownList,
TextField, TextArea, Icon, Menu
• Image Manipulation
• Graphics: Line, Circle, Point, Oval, Rectangle, Polygon, Arc
• Timer and Scheduling
• Events: Keyboard Events, Mouse Events, Display Events
• Conclusion • Color

77 78

Display Display
• how to create displays
• to build a GUI, you need at least one display

• displays are application windows

• contain other GUI objects (widgets and graphics)
• a program may have several displays open
• origin (0, 0) is at the top-left corner

79 80
 Display Graphics
• display functions
• Graphics objects are used to draw various
geometric shapes on a display
• Line
• Circle
• Point
• Oval
• Rectangle
• Polygon

etc.

81 82

Example - Draw Random Circles Widgets

• Widgets are used to display information and

receive user input
• Label
• Button
• Slider
• Checkbox
• TextField
• Icon
• Menu

etc.

83 84
 Example - Button Instrument Play functions

85 86

AudioSample Example - Audio Manipulation

• AudioSample is used for playing audio files
(samples or loops) in real-time
• may be played once, looped, paused, resumed, and
change pitch/frequency, volume, and panning
• supported audio file formats are WAV and AIF
(16, 24 and 32 bit PCM, and 32-bit float)

87 88
 AudioSample AudioSample functions
• how to create AudioSamples

89 90

Activity #6 Example - Circle Instrument

• Download a few audio files, and modify the last
program to loop them, and change their frequency
and volume. Perform your “composition”.

• Get sounds from

• https://www.freesound.org/

• http://soundbible.com/

• many other sites

91 92
 Example - Virtual Piano Outline
• Introduction

• Music Making

• MIDI and OSC

• GUIs

• Image Manipulation

• Timer and Scheduling

• Conclusion

93 94

Image Class
Image Library

• The image library defines functions for displaying

and manipulating JPEG and PNG graphic files

95 96
 Example - Sonify Image Example - Sonify Image

97 98

Outline
• Introduction
Timer Library
• Music Making

• MIDI and OSC

• GUIs
• The timer library defines functions for scheduling
• Image Manipulation tasks (e.g., playing sounds, animation)

• Timer and Scheduling

• Conclusion

99 100
 Timer Class Timer Class

• creates a Timer t, which will call function Play.noteOn(A4) repeatedly every 500
milliseconds (i.e., half second)

101 102

Outline Conclusion
• Introduction • for more info, see
• Music Making
http://jythonmusic.org
• MIDI and OSC
• also see
• GUIs
“Making Music with
• Image Manipulation
Computers: Creative
Programming in Python”,
• Timer and Scheduling
Chapman & Hall/CRC
Textbooks in Computing
• Conclusion

103 104