Creative

Coding

Beat Rossmy and

Albrecht Schmidt

Edition 2020,
Class at LMU Munich
 Creative Coding

Implementing an ∞-Exhibition

B EAT ROSSMY AND A LBRECHT S CHMIDT

E DITION 2020

T HIS BOOK IS WORK IN PROGRESS AND BASED ON A CLASS AT LMU M UNICH

Cover Design by Maya Czupor Copyright c 2019

Copyright c 2020 what license? should discuss thi...
 Table of Contents

1 Programming and Creative Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.1 Concept of this Book 7
1.2 The Goal: an ∞-Exhibition 7
1.3 Target Group and Expected Skills 8

2 Aesthetics and Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.1 Programming a Random Mondrian 9
2.2 Random Creation and Aesthetics 11
2.3 Encoding Information in Art 11
2.4 Tasks - 1st lecture 12

3 Pixel based Art . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3.1 Programming a "Pixel Converter" 14
3.2 Convert to: Pointillism 16
3.3 Convert to other styles 16
3.4 Tasks - 2nd lecture 16

4 Virtual Brushes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.1 Programming a "virtual brush" 18
4.2 Implement: Finite Color 20
6

4.3 Image Brushes 20

4.4 Tasks - 3rd lecture 20

5 Lines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
5.1 Frieder Nake 21
5.2 Implement: Specify Angles 23
5.3 What else to implement? 23
5.4 Tasks - 3rd lecture 23

6 Fractals and Recursive Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

6.1 Nature 24
6.2 How can we change the code? 26
6.3 Tasks - 4th lecture 26

7 Input and Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

7.1 Intro 27
7.2 Video 28
7.3 Audio 28
7.4 Tasks - 7th lecture 29
Concept of this Book
The Goal: an ∞-Exhibition
Target Group and Expected Skills

1. Programming and Creative Coding

1.1 Concept of this Book

The concept of the book is to get people excited for programming artistic pieces. In this book we
teach basic concepts that make up building blocks for interactive and potentially artistic programs.
The focus is to help the reader to create creative software and to learn how to program such software.
The artistic part is up to the reader, but we hope to give good starting points, where readers can bring
in their creativity.
In parts we suggest critical reflection on creative coding and programming art pieces. In the
lecture at LMU this is done as discussions, that are not captured in the book. We recommend readers
to show their ideas and programs to other and discuss and defend it. This aids reflection and learning,
similar to what we do in the course.
Besides the programming concepts and the implementation techniques, we discuss different topics,
including:
• How aesthetics can be programmed?
• Creativity vs. determined by algorithms?
• Creative information visualization vs. art?
• Evolutionary systems and creativity?
• ...

1.2 The Goal: an ∞-Exhibition

In the course we create different interactive software programs. With the selection of the topics we
try to cover many different programming concepts and implementation techniques.
All the programs are designed to create changing new outputs, either after some time or based on
user interaction. All programs together can then be set-up as an exhibition that is ever changing,
which we call an ∞-Exhibition. If the screens with the software are set-up in a circle, where screens
face outwards, a visitor of the exhibition can walk around an will always see new exhibits.
 8 Chapter 1. Programming and Creative Coding

1.3 Target Group and Expected Skills

The course and book is designed for readers studying at the intersection of art, media, and technology.
We expect that readers have an interest in interactive art and that they want to improve their abilities
to create interactive installations.
We expect that readers have basic programming skills in processing. If readers have no program-
ming skills at all, we suggest to do an introductory tutorial (e.g. https://processing.org/
tutorials/ ). For readers with programming experience in another languages, we expect that
they can follow the course and read up on processing details as they go along.
Programming a Random Mondrian
Random Creation and Aesthetics
Encoding Information in Art
Tasks - 1st lecture

2. Aesthetics and Information

The oeuvre of many famous modern painters includes very abstract paintings reduced to simple
geometric forms, primary colours and clear pictorial structures. When you look at such paintings,
many of you will think that they are easy to copy. At first sight you can grasp the colour palette, the
organisation of the forms or simply the whole picture pattern. This allows us to visually separate a
Mondrian from a Malevich. But can our understanding of these images be easily formalized and
captured as an algorithm?

2.1 Programming a Random Mondrian

Following this idea, the first task of the book is to use Processing to recreate a mondrianesque image.
Implement a program that creates a drawing that is inspired by Mondrian.

Figure 2.1: Example of a Mondrian Painting.

The program should create a new picture every 120 seconds or when the user presses a button on
the keyboard.
10 Chapter 2. Aesthetics and Information

Useful Commands
void keyPressed();
random();
randomSeed();
second();
minute();
hour();

The following listing is a starting point for your project.

size(400,400); // draving canvas, coordinate systems
background(255); // background is white
fill(255, 0, 0); // color is red
rect(50, 100, 250, 100);
fill(0, 255, 0); // color is green
rect(250, 250, 100, 100);
fill(0, 0, 255); // color is blue
rect(50, 320, 150, 10);
fill(255, 255, 0); // color is yellow
rect(50, 340, 150, 6);
fill(255, 255, 255); // color is white
rect(50, 360, 150, 4);
fill(0, 0, 0); // color is black
rect(50, 380, 150, 3);

Figure 2.2: Resulting Image from the example code.

See https://processing.org/reference/random_.html for examples how to use

the random function.
 2.2 Random Creation and Aesthetics 11

2.2 Random Creation and Aesthetics

Experiment with the parameters in the random creation. Explore the impact of parameters such as
the size, dimension, line width, colors, and number of element on the aesthetics of the resulting
picture.
Why are some pictures are aesthetically more pleasing than others? How can you change your
algorithm to always create aesthetic drawings, while still having a great variety?

2.3 Encoding Information in Art

People have discussed the idea of informative art, see [Fer07; HS03; RSH00]. Can this utilitarian
approach to creating visual displays based on know artworks be considered art?
Create a display that shows the time. Take a well know artwork as inspiration and encode the time
in your visualization (a 15 minutes resolution is sufficient). To a casual observer it should not be
apparent that the visualization shows the time.
What are the difficulties in designing such a visual display?
The following listing shows a basic clock. You can use this as a starting point for your own
project.
// Variabeln definieren - ausserhalb von setup() und draw()
// da sie in beiden Funktionen verwendet werden
int stunde, minute, sekunde;
void setup() {
// Fenstergroesse festlegen
size(260, 100);
// Variabeln mit 0 initialisieren
stunde = 0;
minute = 0;
sekunde = 0;
// die Funktion draw() wird 2x pro Sekunde ausgefuehrt
frameRate(2);
}
void draw() {
// loesche den Hintergrund: setze ihn schwarz20background(0);
// setze Zeichenfarbe fuer Text und Recheck: weiss
fill(255, 255, 255);
// speichere in den Variablen die aktuelle Zeit
stunde = hour();
minute = minute();
sekunde = second();
// Kontrollausgabe der Zeit in der Console - kann spaeter
geloeschtwerden
// in processing kann man mit "+" einen String aus Buchstaben und
Zahlenbauen
println(stunde + ":" + minute + ":" + sekunde);
// zeige die Zeit im Fenster an
textSize(32);
text(stunde + ":" + minute + ":" + sekunde, 10, 30);
// zeichne einen Sekundenbalken
 12 Chapter 2. Aesthetics and Information

// Sekunde hat einen wert von 0 bis 59

// Sekunde multipliziert mit 4 hat einen Wert zwischen 0 und 236
rect(10, 40, sekunde*4, 20);
}

Ther is another example of a analog clock that may be helpful: https://processing.org/

examples/clock.html

2.4 Tasks - 1st lecture

In the 1st session we try to mimic paintings of the famous dutch painter Piet Mondrian. He is famous
for his abstract paintings that are reduced to a minimal palette of colors and geometric structures.
His systematic approach invites to be recreated with algorithms.
Preparation
If Processing isn’t already installed on your computer, go to https://processing.org/
download/ download the program and install it on your machine. Open some example sketches
and familiarize yourself with the Processing IDE. If you feel comfortable, proceed with the following
task.
Breakout Session
During the following task we try to approach the generation of random Mondrianesque images
in a systematic way. Step by step we go from the analysis of real and fake Mondrians, over
theoretical concepts for the description of Mondrianesque structures, to working implementations of
the algorithm.
• Search on the internet for pictures of the painter Mondrian (see Figure 2.3). What are the
connecting commonalities? Can you spot artworks not painted by Mondrian? Try to figure out
the "recipe" that makes a distinguishable Mondrian.
• Let us discuss your approach to generate Mondrianesque images with the help of an algorithm.
Try to sketch such structures and formalize them first.
• Try to implement one of the approaches we discussed in the plenum. Does it lead to the expected
result? Does it work as simply as expected or have there been unexpected complications?
• If your algorithm is working, try to extend it to add your own style to the generated "Mondrians".
Tweak parameters, change the color palette or even introduce more randomness.
Discussion
We just created algorithms that mimic the style of a famous artist. Let us discuss your impressions
and thoughts along the way. As a starting point think about the following questions:
• Can you call the product of these algorithms "art"?
• Can art in general be scripted? (Pollock, Warhol, ...)
• Does the used technology influence such a differentiation? (Style transfer performed with
neural networks vs classic "conditional" algorithms)
Homework
1. Reading: Read the following papers [Fer07; HS03; RSH00] and write a short (300-500 words)
essay on Informative Art. Your essay should contain an explanation what it is, why people
would want to do it, pros and cons.
2.4 Tasks - 1st lecture 13

Figure 2.3: Artworks painted by Mondrian and Mondrian-inspired artworks.

2. Implementation: Create a display that shows the time. Take Mondrian algorithm and encode
the time in your visualization (a 15 minutes resolution is sufficient).
3. Collect feedback on your implementation. Record 3 videos from people seeing your
implementation (15 seconds each).
Programming a "Pixel Converter"
Convert to: Pointillism
Convert to other styles
Tasks - 2nd lecture

3. Pixel based Art

3.1 Programming a "Pixel Converter"

Implement a program that takes an image as an input, analyzes the content and transfers it to a new
output design. As an example look at pictures from Pointilism, ASCII art, etc..

Figure 3.1: The shown examples demonstrate the translation of a color information to a interpreted output.

The program should take a predefined image from your file system or a snapshot over the integrated
webcam of your computer.

Useful Commands
PVector vec;
loadImage("testfile.jpg");
IntList list;
color c;
red(c);
green(c);
blue(c;)
3.1 Programming a "Pixel Converter" 15

The following code is a starting point for your project.

PImage source;
int area = 25;
void setup () {
size(900,600);
source = loadImage("test.jpg");
noStroke();
for (int y=0; y<source.height; y+=area) {
for (int x=0; x<source.width; x+=area){
fill(getAverageColor(x,y,area,area,source));
rect(x,y,area,area);
}
}
}
color getAverageColor ( int x, int y, int w, int h, PImage img) {
float r = 0;
float g = 0;
float b = 0;
for (int i=0; i<w*h; i++) {
int index = x+(i%w)+y*img.width+(img.width*(i/w));
r += red(img.pixels[index]);
g += green(img.pixels[index]);
b += blue(img.pixels[index]);
}
return color(r/(w*h),g/(w*h),b/(w*h));
}

Figure 3.2: Resulting Image from the example code.

 16 Chapter 3. Pixel based Art

3.2 Convert to: Pointillism

Try to extend the given code to approximate the style of a pointillistic artwork:
• Draw many circles instead of one big square.
• Randomize the position of these circles.
• Randomize color values.
What effects can you observe that interfere with the pointillisitc style? Try to make your algorithm
generate more vivid outcomes. If you look closely at the pointillistic image shown in the beginning
of the chapter, one can observe that the painter uses small colors dots of primary colors that are
mixed in our brain to multiple shades of the color palette. Can we imitate this effect?

3.3 Convert to other styles

In the first example code a function was given that translates a given area of an image to the average
color of this area. What other informations are contained in such areas?
• color spread
• color palette
• ...
Try to define functions that take an image, coordinates an dimensions of the subarea to be observed
and extract these informations.
color getAverageColor ( int x, int y, int w, int h, PImage img) {
float r = 0;
float g = 0;
float b = 0;
for (int i=0; i<w*h; i++) {
int index = x+(i%w)+y*img.width+(img.width*(i/w));
r += red(img.pixels[index]);
g += green(img.pixels[index]);
b += blue(img.pixels[index]);
}
return color(r/(w*h),g/(w*h),b/(w*h));
}
ArrayList<Integer> getColors ( int x, int y, int w, int h, PImage img) {
ArrayList<Integer> colors = new ArrayList<Integer>();
for (int i=0; i<w*h; i++) {
int index = x+(i%w)+y*img.width+(img.width*(i/w));
color c = img.pixels[index];
if (!colors.contains(c)) colors.add(c);
}
return colors;
}
...

3.4 Tasks - 2nd lecture

In the 2nd session we try to process image information and translate these into new "styles". As
a source of inspiration we discuss the genre of pointillism as well as pictures from the ASCII art
3.4 Tasks - 2nd lecture 17

scene. Further, we try to go beyond these two styles an approach new ideas and concepts.
Breakout Session
During the following task we try to think of our algorithms as transfer functions. With an image
given we take the contained information (colors of pixels) and transfer it into: colored shapes, ASCII
characters, ...
In this model the image is the input, which is then analyzed on a subarea level. The contained
information is transformed by a small instruction set and thereby applied to the whole image.
• Search on the internet for pointillisitc images and for examples of ASCII art.
• Discuss the different concepts how colors, shapes and more are abstracted.
• Start to implement your analyzed concept based on the previously given example code.
• Where are the weak points of the algorithm? Can you spot its algorithmic nature? How can
theses problems be addressed and solved?
Discussion
What are the most interesting aspects of this approach? Let us discuss your impressions and thoughts
along the way. As a starting point think about the following questions:
• Can you call the product of these algorithms "art"?
• For now, we have reduced information (subareas), can we also extend to a higher richness of
detail?
• Could you imagine to use such a technique in your own work?
Homework
1. Reading: Read the following papers http://www.cse.cuhk.edu.hk/~ttwong/
papers/asciiart/asciiart.pdf to get an idea about the challanges and approaches
to ASCII art.
2. Implementation: Look into the following webpage http://paulbourke.net/
dataformats/asciiart/ and adopt these rules for ASCII art to our algorithm. Im-
plement a "Convert to: ASCII" sketch.
3. Think of the following: For the moment our ASCII sketch produces random character sequences
based on the color value of the source pixels. Can we use words instead of random sequences?
What do you have to do to take as an additional input something like Shakespeare’s collected
work and search in there for matching "color/character" sequences? Try an implementation.
Frieder Nake
Implement: Specify Angles
What else to implement?
Tasks - 3rd lecture

5. Lines

5.1 Frieder Nake

In this section we will look into the algorithmic art of Frieder Nake. Nake is famous for his first
explorations of the "Graphomat" (today we call such machines plotters). He decided to experiment
with this new type of machines in an artistic way during his task to write software for this new
hardware which was targeted at architects and engineers to speed up and ease the drawing process.
Instead of typical calibration tests he wrote algorithms that "created" artistic and randomly generated
drawings.

Figure 5.1: Rule sets can specify angles minimum and maximum distances, ... .

We will try to implement an algorithm that generates similar aesthetics. Further, it would be
interesting to "watch" the algorithm draw. How can we follow along with the "plotter" during the
"creation" process?
22 Chapter 5. Lines

Useful Commands
PVector vec;
PVector.random2D();
PVector.add(vec1,vec2);
PVector.sub(vec1,vec2);
PVector.mult(vec1,f);
random();
constrain();

Figure 5.2: Nake’s work is manifold and nevertheless beautiful in its simplicity.

The following code is a starting point for your project.

PVector plotterHead,last;
PVector direction;
void setup () {
size(800,800);
smooth();
background(255,250,240);
stroke(0,100);
strokeWeight(4);
plotterHead = new PVector(random(800),random(800));
last = plotterHead.copy();
direction = PVector.random2D();
}
void draw () {
plotterHead = PVector.add(plotterHead,direction);
line(plotterHead.x,plotterHead.y,last.x,last.y);
last = plotterHead.copy();
if (random(100)<=1 || (plotterHead.x>=width || plotterHead.x<=0 ||
plotterHead.y>=height || plotterHead.y<=0)) {
direction = PVector.random2D();
}
}
 5.2 Implement: Specify Angles 23

5.2 Implement: Specify Angles

Try to give the algorithm a "rule set" that specifies the angles to be used. Should there be just 90
degree angles or random angles or ...?
• Just pick specific angles instead of random ones.
• Think of using an array that contains the valid angles?
How can we apply more complex rules such as: 90 degree lines are longer then 60 degree ones.

5.3 What else to implement?

If we compare our outcome to the found drawings of Nake’s algorithms where are the differences?
• Try to find and point out as many differences as occurre to you.
• Do you think every drawing of the algorithm is an artwork?
• Did the artist picked the best ones?
Discuss the given tasks and questions in your group.

5.4 Tasks - 3rd lecture

In the 3rd session we want to discuss how we can implement simple abstract and line-based
algorithmic art.
Breakout Session
During the following task we take a closer look at different kinds of brushes and strokes.
• Search on the internet for examples of Nake’s work.
• Discuss the different parameters he used over the course of his different cycles.
• what role do randomness and order play in his work?
• Is the beauty just in the aesthetic of the drawings or as well in the simplicity of the logic behind?
• Expand the example code with some of your ideas. Try to find the most elegant solution for
your specific problem.
Homework
1. Reading: Read through the following web document http://dreher.netzliteratur.
net/4_Medienkunst_Kybernetik.html and this paper written by Frieder Nake
[Nak05] http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.
470.2597&rep=rep1&type=pdf
2. Implementation: Implement two of Nake’s other algorithms (as shown in Figure 5.2 or found
online).
Nature
How can we change the code?
Tasks - 4th lecture

6. Fractals and Recursive Structures

6.1 Nature
In this section we will look into common structures known from nature called "Fractals". Fractals
are structures that reoccur in them self. Common examples would be: mountains, snow flakes,
trees. All these structures show the same features from a distant point of view as well as under
close examination. A tree is just a single branch from which other branches emanate, from which
smaller branches emanate from which ... In nature these structures come to an logical end. The
last branch emanates into leaves which mark the end of the structure, but on a mathematical level
these structures can go into infinity. The most prominent example for mathematical fractals is the
Mandelbrot Fractal.

Figure 6.1: Even every day objects as broccoli reveal fractal qualities.

We will try to implement a fractal in a generic manner and from there explore the impact of
different rule sets, randomness or geometrical elements (lines, circles, ...) on the visual outcome.

Useful Commands
translate();
6.1 Nature 25

rotate();
push();
pop();

The following code is a starting point for your project.

// adapted from @urbanoxygen_ on instagram
float t = 0;
float diameter = 1000;
int levels = 2;
void setup() {
size(1080, 1080);
}
void draw() {
background(118,233,255);
t = frameCount/1500.0;
translate(width/2, height/2);
rotate(PI*t*4);
yinYang(diameter, levels);
}
void yinYang(float diameter, int n) {
noStroke();
fill(0);
arc(0, 0, diameter, diameter, 0, PI);
fill(255);
arc(0, 0, diameter, diameter, PI, 2*PI);
if(n==1) {
fill(0);
ellipse(-diameter/4, 0, diameter/2, diameter/2);
fill(255);
ellipse(diameter/4, 0, diameter/2, diameter/2);
ellipse(-diameter/4, 0, diameter/4, diameter/4);
fill(0);
ellipse(diameter/4, 0, diameter/4, diameter/4);
} else {
push();
translate(diameter/4, 0);
rotate(PI*t*4);
yinYang(diameter/2, n-1);
pop();
push();
translate(-diameter/4, 0);
rotate(PI*t*4);
yinYang(diameter/2, n-1);
pop();
}
}
 26 Chapter 6. Fractals and Recursive Structures

6.2 How can we change the code?

Try to change parameters of the code.
• Make the fractal grow outward.
• Introduce randomness.
• Add different angles.
• ...

6.3 Tasks - 4th lecture

In the 4th session we want to discuss how we can implement recursive fractal structures
Breakout Session
During the following task we will take a look into star systems.
• Compare these systems to the fractals we have seen so far. Do the same/similar rules apply?
• What features do we have to add to our code to improve the visual effect.
• What role do randomness and order play here?
Homework
1. Implementation: Implement an animated solar system with several layers of planets and
moons.
2. Final Project: Prepare some sketches or other materials to communicate your idea for your
final project
 Bibliographie

[Fer07] Alois Ferscha. “Informative art display metaphors”. In:

International Conference on Universal Access in Human-Computer Interaction. Springer.
2007, pages 82–92 (cited on pages 11, 12).
[HS03] Lars Erik Holmquist and Tobias Skog. “Informative art:
information visualization in everyday environments”. In:
Proceedings of the 1st international conference on Computer graphics and interactive techniques in Australasia a
ACM. 2003, pages 229–235 (cited on pages 11, 12).
[Nak05] Frieder Nake. “Computer art: a personal recollection”. In:
Proceedings of the 5th conference on Creativity & cognition. ACM. 2005, pages 54–62
(cited on page 23).
[RSH00] Johan Redström, Tobias Skog, and Lars Hallnäs. “Informative art: using amplified artworks as infor-
mation displays”. In: Proceedings of DARE 2000 on Designing augmented reality environments.
ACM. 2000, pages 103–114 (cited on pages 11, 12).