Music–color associations are mediated by emotion

Stephen E. Palmera,1, Karen B. Schlossa, Zoe Xua, and Lilia R. Prado-Leónb

a
Psychology Department, University of California, Berkeley, CA 94720; and bErgonomics Research Center, University Center of Art, Architecture, and Design,
University of Guadalajara, Guadalajara, Jalisco, 44250, Mexico

Edited by Dale Purves, Duke-National University of Singapore Graduate Medical School, Singapore, Singapore, and approved April 1, 2013 (received for
review July 23, 2012)

Experimental evidence demonstrates robust cross-modal matches How might music-to-color associations occur in nonsynesthetes?
between music and colors that are mediated by emotional The two most plausible hypotheses are (i) the direct connection
associations. US and Mexican participants chose colors that were hypothesis that there are direct, unmediated associations between
most/least consistent with 18 selections of classical orchestral music colors and musical sounds (3, 9, 16) and (ii) the emotional me-
by Bach, Mozart, and Brahms. In both cultures, faster music in the diation hypothesis that color and music are linked through shared
major mode produced color choices that were more saturated, emotional associations (3, 8, 13–15, 17–19).* Although some em-
lighter, and yellower whereas slower, minor music produced the pirical support has been claimed for hypothesis ii, the small sets of
opposite pattern (choices that were desaturated, darker, and colors and/or descriptors (15, 17, 18), the small set of musical
bluer). There were strong correlations (0.89 < r < 0.99) between selections (13–15, 18), and the potential relevance of cultural
the emotional associations of the music and those of the colors comparisons preclude firm conclusions.
chosen to go with the music, supporting an emotional mediation The present results demonstrate clear, robust connections
hypothesis in both cultures. Additional experiments showed simi- between music and color that are widely shared across both
larly robust cross-modal matches from emotionally expressive faces individuals and cultures. They differ from previous findings in
to colors and from music to emotionally expressive faces. These that they (i) associate specific dimensions of color (saturation,
results provide further support that music-to-color associations lightness, and yellowness-blueness) with specific high-level mu-
are mediated by common emotional associations. sical dimensions (tempo and mode), (ii) show clear evidence of
mediation by emotional dimensions (happy–sad and angry–
|
color cognition cross-modal associations | music cognition | calm), and (iii) demonstrate a strong cultural invariance across
emotion mediation hypothesis
US and Mexican participants. Specifically, people’s experiences
of 18 brief, classical orchestral selections that varied in tempo

R esearchers have attempted to identify systematic links be-

tween music and color. Perhaps the most direct connection
comes from the fascinating phenomenon of music–color synes-
(slow/medium/fast), mode (major/minor), and composer (Bach/
Mozart/Brahms) are closely associated with the saturation,
lightness, and yellowness of the colors they chose to “go best”
thesia (1–4). A small minority of individuals, including some dis- with the music.
tinguished artists (e.g., Kandinsky and Klee) and musicians (e.g.,
Scriabin and Rimsky-Korsokov) report diverse cross-modal expe- Experiment 1 Color, Music, and Emotion
riences of color while hearing musical sounds (1). Scientific studies We studied music–color associations in nonsynesthetes in the
initially failed to establish general correspondences because syn- United States and Mexico for 18 classical orchestral selections
esthetic sound-to-color mappings appeared idiosyncratic (3). using the 37 colors of the Berkeley Color Project (20, 21). The
Nonsynesthetic people also have music-to-color associations colors (Fig. 1) were chosen to vary systematically in hue, satu-
but do not actually experience colors while hearing music. Rela- ration, and lightness. The eight hues included four unique hues
tively low-level sound-to-color associations—e.g., higher pitches (red/green/blue/yellow), plus four intermediate hues with ap-
being associated with lighter colors (2, 5–7)—appear to hold for proximately equal amounts of adjacent unique hues (orange/
both synesthetes and nonsynesthetes (1). Reliable pitch–hue chartreuse/cyan/purple). These hues were sampled at four cuts
associations have been reported in children (8) although these (saturation/lightness levels), with the saturated (S) colors being
effects were probably due to lightness, where spectral yellow and the maximally saturated colors that our monitor could produce,
green (lightest) were associated with higher pitches, red and or- muted (M) colors being approximately halfway between each
ange (midlightness) with midlevel pitches, and blue and violet
S-color and neutral-gray, light (L) colors being approximately
(darkest) with lower pitches. There is evidence for other low-level
halfway between each S-color and white, and dark (D) colors
auditory–visual associations such as timbre–saturation (9), loud-
being approximately halfway between each S-color and black in
ness–brightness (7), and pitch–size (10, 11) [Spence (12)].
Munsell space (Table S1). We also included white, black, and the
Other studies have investigated music–color correspondences
at a higher level. Bresin found that music in the major mode was
associated with lighter colors than music in the minor mode (13),
Author contributions: S.E.P. and K.B.S. designed research; K.B.S., Z.X., and L.R.P.-L. per-
but only two melodies were studied. Sebba reported that stu- formed research; S.E.P. and K.B.S. contributed new reagents/analytic tools; S.E.P., K.B.S.,
dents used warmer, more saturated, lighter, and more highly and Z.X. analyzed data; and S.E.P. and K.B.S. wrote the paper.
contrasting colors in creating images while listening to a major The authors declare no conflict of interest.
Mozart selection than did students listening to a minor Albinoni This article is a PNAS Direct Submission.
selection (14). Again, only two musical selections were used, 1
To whom correspondence should be addressed. E-mail: [email protected].
and students chose the musical selections rather than being
This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.
randomly assigned, so students who are more inclined to choose 1073/pnas.1212562110/-/DCSupplemental.
major music may merely be more inclined to create images with *The concept of emotion is notoriously difficult to define. Typical definitions identify
warmer, more saturated, lighter, and more highly contrasting emotions as “conscious feelings” that have both physiological and cognitive compo-
colors. Barbiere et al. found that “gray” was associated with nents, followed by a list of prototypical examples (e.g., fear, anger, joy, and sorrow)
sadder music whereas “red,” “yellow,” “green,” and “blue” were with no boundary conditions to clarify less obvious possibilities. Emotional associations
are systematic connections between emotions and other mental states caused by expe-
associated with happier music (15), but only four musical selec- riences that are not intrinsically emotional, such as hearing music or seeing colors. As we
Downloaded by guest on June 19, 2020

tions were studied and, more importantly, no actual colors were use the term, emotional associations may include conscious experiences of feelings and/
presented in the selection task, but only words. or cognitive content that accompanies such feelings.

8836–8841 | PNAS | May 28, 2013 | vol. 110 | no. 22 www.pnas.org/cgi/doi/10.1073/pnas.1212562110

 minus an analogous weighted average of the saturation ratings of
the five colors chosen to be least consistent with that music (Id,m):


Cd;m = 5c1;d;m + 4c2;d;m + 3c3;d;m + 2c4;d;m + 1c5;d;m 15 [1]


Id;m = 5i1;d;m + 4i2;d;m + 3i3;d;m + 2i4;d;m + 1i5;d;m 15; [2]

where cj,d,m represents the value along dimension d of the jth

color picked as most consistent with musical selection m, where j
ranges from 1 to 5, and ij,d,m represents the corresponding value
of the jth color picked as most inconsistent with musical selection
m. The MCA for a given selection on dimension d is then,

MCAd;m = Cd;m Id;m : [3]

These values were computed for each subject in the United

States and Mexico for each of the 18 musical selections and each
of the four color appearance dimensions. Fig. 2 shows average
MCA values for US participants as a function of tempo and
mode, averaged over composers, for each of the four color
dimensions. Separate ANOVAs for each color dimension
showed that, averaged over composers, faster tempi were gen-
erally associated with more saturated, lighter, and yellower
(warmer) colors [F(2,94) = 96.13, 32.54, 62.82, P < 0.001, re-
spectively]. The same was true for major (vs. minor) mode [F
(1,47) = 96.13, 51.31, 29.71, P < 0.001, respectively]. By the same
Fig. 1. The display of 37 colors that was presented during the music–color token, slower tempi and music in the minor mode were associ-
association task: red, orange, yellow, chartreuse, green, cyan, blue, and ated with less saturated, darker/cooler colors. For major (but not
purple at four different lightness-saturation levels (saturated, light, muted,
and dark), plus three grays, white, and black. (See text and Table S1 for
minor) music, slow tempi were associated with greener colors
details.) The gray corresponding to the saturated (Top Left) and muted than medium or faster tempi [F(1,47) = 69.61, 62.87 P < 0.001,
(bottom right) cuts were the same because the saturated and muted cuts respectively]. Main effects of composer were also present, with
had similar mean lightnesses. Brahms’s music being associated with less saturated, darker,
bluer colors than Bach’s and Mozart’s music [F(2,94) = 11.55,
49.58, 16.97, P < 0.001, respectively], with Bach’s and Mozart’s
three grays whose lightnesses were approximately the average music not differing from each other.
lightness of the L-, M- (and S-), and D-colors (Fig. 1 has 38 col- Reliable differences between composers were also present for
ored squares because the gray for the S and M cut is the same). many of the two- and three-way interactions (Fig. S1). The trends
Participants listened to 18 50-s samples of orchestral music that that are mentioned above varied somewhat for different com-
varied in tempo (slow/medium/fast) and mode (major/minor) posers, especially in the case of Brahms. No doubt many of these
(Table S2) while viewing the 37-color array (Fig. 1) (tempi for
different composers were not the same, however: e.g., the fast
selection for Brahms was significantly slower than the fast selec-
Sat.

Light

80 Sat.-Unsat. 80 Light-Dark
tions for both Bach and Mozart). They were asked to choose the
Major
five colors, in order, that were most consistent with the music and 40 40 Major
MCA Score

then the five colors, in order, that were least consistent with the Minor Minor
music (subsequent analyses have shown that essentially the same 0 0
results are obtained if only the three most/least consistent colors
-40 -40
Unsat.

are chosen).
Dark

We also measured participants’ emotional associations sepa- -80 -80

rately for each color and each musical selection by asking them Slow Med. Fast Slow Med. Fast
to rate how strongly associated it was with each of eight emotional Tempo Tempo
descriptors that were relevant for both music and color—happy,
Yellow

Red

80 Yellow-Blue 80 Red-Green
sad, angry, calm, strong, weak, lively, and dreary—using a line-
mark ratings scale that ranged from −100 to +100. They also 40 40
MCA Score

performed four color–appearance ratings of each of the 37 colors Major Major

for the degree to which they were red-green (R/G), yellow-blue 0 Minor 0 Minor
(Y/B), light-dark (L/D), and saturated-unsaturated (S/U) (20).
We examined the relation between the dimensional variations -40 -40
Green
Blue

in our music samples (i.e., tempo and mode) and the dimensional
-80 -80
structure of color by computing a music–color association (MCA) Slow Med. Fast Slow Med. Fast
PSYCHOLOGICAL AND
COGNITIVE SCIENCES

score for each of the 18 musical selections along each of four Tempo Tempo
color appearance dimensions (R/G, Y/B, L/D, and S/U) as rated
Fig. 2. Dimensional color associations for music at slow/medium/fast tempi
by the same participants. Conceptually, the MCA score for and in major/minor mode for the saturation, lightness, yellowness-blueness,
a given musical selection m on a given dimension d (say, satu- and redness-greenness of colors chosen as most/least consistent with the
Downloaded by guest on June 19, 2020

ration) is a linearly weighted average of the saturation ratings of music, as computed using the Music–Color Association index (MCA) defined
the five colors chosen as most consistent with that music (Cd,m) by Eqs. 1–3. (Error bars represent SEMs.)

Palmer et al. PNAS | May 28, 2013 | vol. 110 | no. 22 | 8837
 differences are due to the particular musical selections we chose, studies of color–emotion associations (22–24), the two emotional
as they varied widely in musical features other than tempo and dimensions were interpretable as positive/negative valence (happy/
major/minor mode, such as loudness, orchestration and timbre, sad) and high/low potency (strong/weak). Average happy/sad ratings
pitch height, melodic structure, harmonic structure, note rate (as were correlated +0.93 with the coordinate values of the colors along
opposed to tempo), dynamic range (loudness contrast), plus the dimension 1 in Fig. 3, and strong/weak ratings were correlated +
enormous stylistic differences among music from the Baroque 0.93 with the coordinate values of the colors along dimension
(Bach), Classical (Mozart), and Romantic (Brahms) eras. 2. Similar dimensions are frequently obtained in MDS solutions
Cultural dependency was assessed by comparing the results of of many other emotional stimuli (22, 23, 25, 26). We obtained
exactly the same experiment for 49 Mexican participants at the similar results when the corresponding Mexican ratings were
University of Guadalajara. The pattern of results for tempo, scaled in the same way.
mode, and composer were remarkably similar (compare Fig. 2 The analogous emotional MDS of the US data on the 18
with Fig. S2), as indexed by very high correlations between cul- musical selections yielded a similarly good 2D solution (Fig. 3B),
tures for each color appearance dimension: +0.88 for R/G, +0.96 accounting for 99% of the variance (stress = 0.02). The same two
for Y/B, +0.97 for L/D, and +0.95 for S/U. ANOVAs including dimensions characterize the solution: positive/negative valence
both cultural datasets were performed for each of these dimen- (happy/sad) and high/low potency (strong/weak). Happy/sad rat-
sions, with culture (United States/Mexico) as a between-subject ings correlated +0.96 with the coordinate values of the colors
factor and composers, tempi, and modes as within-subject fac- along dimension 1, and strong/weak ratings correlated +0.96 with
tors, with a Bonferroni-corrected critical alpha of 0.01 to adjust the coordinate values of the colors along dimension 2. Again,
for multiple comparisons for the four color-appearance dimen- similar results were obtained when the corresponding Mexican
sions. The results revealed no interactions between culture and ratings were scaled in the same way.
tempo for R/G, Y/B, L/D, and S/U [F(2,190) = 0.20, 0.95, 1.89, The fact that the same two emotional dimensions emerged in
2.33, P > 0.01, respectively], mode [F(1,95) = 1.36, 0.00, 5.60, separate MDS solutions for colors and musical selections sup-
0.00, P > 0.01, respectively], or tempo × mode (Fs < 1.02). There ports the emotional mediation hypothesis. A more direct test can
were minor main effects of culture for the different color be formulated in terms of correlational analyses. In particular,
dimensions, for which Mexican participants chose somewhat emotional mediation implies that there should be a high corre-
lighter, yellower, and greener colors than US participants did [F lation between the emotional ratings of the musical selections
(1,95) = 5.44, 8.07, 7.78, P < 0.05, 0.01, 0.01, respectively]. There and the emotional ratings of the colors people chose as consis-
were also minor interactions involving culture and composer for tent/inconsistent with those selections. We computed the music–
the R/G and S/U dimensions, but the patterns and ordering of color association (MCAe,m) for each emotional dimension e and
the conditions were the same in both countries. musical selection m that is entirely analogous to the music–color
The emotional mediation hypothesis suggests that, as people association (MCAd,m) defined by Eqs. 1–3, except that the
listen to the music, they have emotional responses while listening emotional dimension e replaces color dimension d. This measure
to music and then pick colors with similar emotional content. For provided a combined, weighted index of, say, the happiness/sadness
example, Bach’s fast-paced dance in F-major might convey of the 10 colors chosen as being most/least consistent with each
a happy, energetic emotion, consistent with happy, energetic colors musical selection.
(e.g., saturated-yellow and light-yellow) whereas Brahms’ slow- We then computed the correlation between the average
paced Adagio in C-minor might convey sad, depressive feelings, rating for each emotional dimension for each of the 18 musical
consistent with sad, depressive colors, (e.g., black, dark-gray, and selections versus the MCAe,m values of the colors chosen as
dark-cyan). This hypothesis was assessed by analyzing the emo- positively/negatively associated with each musical selection.
tional ratings provided separately for the colors and musical Fig. 4 shows the four scatter plots of these correlations for each
selections. (approximately) polar pair of emotional terms: happy/sad,
We computed the average ratings of each color for each of the
strong/weak, lively/dreary, and angry/calm. The strength of
eight emotional adjectives (Fig. S3). The ratings of three concep-
these correlations is equally remarkable in the US and Mexican
tually opposite pairs of emotional terms were nearly perfect
data: happy/sad (+0.97, +0.97), lively/dreary (+0.99, +0.96),
inverses of each other in both the US and Mexican data, as indexed
strong/weak (+0.96, +0.96), and angry/calm (+0.89, +0.93), re-
by high negative correlations between the ratings of the 37 colors
spectively. These results are clearly consistent with the emotional
for each of these pairs: happy/sad (r = −0.94, −0.94), strong/weak
mediation hypothesis (3, 8, 13–15, 17–19). Because the evidence
(r = −0.97, −0.87), and lively/dreary (r = −0.99, −0.95). Angry and
calm, however, were less than fully opposite (r = −0.69, −0.13, P <
0.001, P > 0.05, respectively). Corresponding analyses of the av-
erage emotional ratings for the 18 musical selections (Fig. S4) A B
revealed the same pattern: happy/sad (r = −0.98, −0.96), strong/ 1.5 Color-Emotion MDS 1.5 Music-Emotion MDS
Happy

weak (r = −0.99, −0.97), and lively/dreary (r = −0.95, −0.98) are Slow

effectively single, bipolar dimensions, but angry/calm is not (r = Med.
Fast
Dimension 1

−0.51, −0.31, P < 0.05, P > 0.05, respectively).

Major
We then performed separate emotional multidimensional 0 0 Minor
scalings (MDSs) of the colors and the music. First, we computed
Mozart
the correlation of average emotional association ratings for each Brahms
pair of colors over the eight emotional terms for the US data to Bach
Sad

use as input to mdscale (www.mathworks.com/help/stats/mdscale. -1.5 -1.5

-1.5 0 1.5 -1.5 0 1.5
html), an MDS program. The emotional ratings of S-yellow and Weak Strong Weak Strong
S-orange, for example, are very similar (r = +0.98) because both Dimension 2 Dimension 2
are rated quite high on happiness, strength, and liveliness, and
Fig. 3. Best-fitting solutions for emotional multidimensional scaling of (A)
low on sadness, weakness, dreariness, and calmness. The colors and (B) music based on emotion similarity as determined by the cor-
resulting 37 × 37 correlation matrix served as an inverse ordinal relation between all possible pairs of colors (A) and musical selections (B) for
distance measure in the scaled space. ratings on eight emotional terms: happy, sad, angry, calm, strong, weak,
Downloaded by guest on June 19, 2020

We obtained a good 2D solution (Fig. 3A), accounting for lively, and dreary. Dimensions are labeled by the emotional terms whose
95% of the variance (stress = 0.08). Consistent with previous ratings best fit the projections of the colors (A) and musical selections (B).

8838 | www.pnas.org/cgi/doi/10.1073/pnas.1212562110 Palmer et al.

 Happy
100 Happy-Sad 100 Lively-Dreary somewhat saturated reddish colors (e.g., dark reds) (see SI Text

Lively
r = .97 r = .99 for statistical analyses of these data).
50 50
MCA Score

To examine the emotional mediation hypothesis, we computed

0 0 the correlations between the ratings of the three emotional
dimensions for the 14 faces and for the colors selected as most/
-50 -50
least consistent with the corresponding faces using the color–

Dreary
Sad

-100 -100 Slow emotion association for faces (FCAe,f) for emotional dimension
-100 0 100 -100 0 100 Med. e and face f (Eqs. 1–3). These correlations were again strikingly
Fast
Sad Happy
Music Emotion Rating
Dreary Lively
Music Emotion Rating high: +0.97 for happy/sad, +0.94 for angry/calm, and +0.85 for
Major weak/strong. Although there is clearly a direct association be-
Minor
100 Strong-Weak 100 Angry-Calm tween certain colors and certain facial expressions of emotions
Strong

Angry
r = .96 r = .89 Mozart due to correlated changes in skin color (e.g., faces typically be-
50 50
MCA Score

Brahms come redder when expressing extreme anger), other associations

Bach
0 0 are not of this type. Faces do not typically get yellower or lighter
when people become happier nor darker and bluer when they
-50 -50
become sadder. The fact that the same overall pattern of cor-
Weak

Calm

-100 -100 relations emerges further supports the emotional mediation hy-
-100 0 100 -100 0 100 pothesis of the coupling between dimensions of color and music
Weak Strong Calm Angry
Music Emotion Rating Music Emotion Rating
in experiment 1.

Fig. 4. Scatterplots and correlations between the emotional ratings of the Experiment 3 Music, Faces, and Emotion
18 musical selections (x axis) and the emotional associations of the colors If the emotional mediation hypothesis is correct, as the results of
chosen as most/least consistent with them (y axis) for the four emotional experiments 1 and 2 suggest, then any two sets of stimuli that
dimensions studied: happy/sad, angry/calm, strong/weak, and lively/dreary.
have strong associations with common emotional content should
be systematically relatable. Because experiment 1 showed that
is purely correlational, however, it does not establish that the re- music and color can be related through emotional associations
lation is causally mediated by emotion or in which direction it and experiment 2 showed that color and faces can be related
through the same emotional associations, it follows that music
operates. It is also possible that there are redundant, direct asso-
and faces should be analogously relatable. We tested this pre-
ciations between color and music. These issues are addressed in
diction in experiment 3 by playing the 18 musical selections from
part by experiments 2 and 3. experiment 1 and asking participants to choose the two most/
Experiment 2 Color, Faces, and Emotion least consistent faces among a set of 13 faces that varied in the
degree to which they expressed the emotions of happiness, sad-
If the strong links between colors and musical selections mea- ness, anger, and calmness. We used the seven female faces from
sured in experiment 1 are mediated by common emotional experiment 2 augmented by 25%- and 75%-morphed versions of
associations, then analogous results should emerge if we asked the same happy, sad, and angry faces.
participants to pick the colors that are most/least consistent with We calculated the music–face association (MFAe,m) for each
any other set of stimuli that are strongly associated with the same emotion, e, and each musical selection, m, analogous to the
emotional dimensions of happy/sad, angry/calm, and strong/ MCAd,m measure for music–color associations (Eqs. 1–3), except
weak. Perhaps the clearest and most transparently emotional that the 13 faces were substituted for the 37 colors, the emo-
visual stimuli that meet this criterion are images of individual tional dimensions of happy, sad, and angry were substituted for
human faces expressing different emotions to different degrees. the four color appearance dimensions, and only two faces were
The emotional mediation hypothesis implies that, when asked to selected as being most/least consistent with each musical selec-
match colors to facial expressions, the correlations between the tion. Analogous to results from experiments 1 and 2, there were
emotional ratings of the faces and the emotional ratings of the strong correlations between the emotional content of the music
colors chosen to go well/poorly with the faces will be quite high, and the emotional content of the faces that were chosen as
analogous to the results of experiment 1 for color and music. consistent/inconsistent with the music for happy/sad (r = +0.97),
Experiment 2 tests this prediction by repeating experiment 1’s
cross-modal mapping task with emotional faces simply replacing
the musical selections. The 14 face images we used came from
Marian and Shimamura’s (27) gray-scale, morphed versions of
the NimStim set (28), including male and female expressions of
calm (neutral), 50% happy, 100% happy, 50% angry, 100%
angry, 50% sad, and 100% sad (Fig. 5).
We calculated the face–color association measure (FCAd,f) for
each color dimension, d, and each face, f, analogously to the
MCAd,m measure for music, except that the 14 emotional faces
were substituted for the 18 musical selections (Eqs. 1–3). The
results are plotted in Fig. 6 for the four color appearance
dimensions of S/U, L/D, Y/B, and R/G.
The results (Fig. 6) indicate that neutral/calm faces were
PSYCHOLOGICAL AND
COGNITIVE SCIENCES

paired with desaturated, moderately light colors that were

slightly cool (e.g., low saturation blues and greens), sad faces Fig. 5. The 14 happy, sad, and angry emotionally expressive faces of a male
and female in experiment 2. Participants chose the five colors that were
were paired with darker, desaturated, cool colors (e.g., dark- most/least consistent with each face from the 37 colors shown in Fig 1. The
bluish or dark-greenish grays), happy faces were paired with 50% emotional faces were morphed halfway between the 0% (neutral) and
Downloaded by guest on June 19, 2020

light, highly saturated, warm colors (e.g., vivid and pastel yellows, corresponding 100% emotional face (27). Face images are from the NimStim
oranges, and reds), and angry faces were paired with dark, database (28).

Palmer et al. PNAS | May 28, 2013 | vol. 110 | no. 22 | 8839
 Sat. Sat.-Unsat. Light-Dark from experiment 2. For example, faster, major music was system-

Light
80 80
Happy atically associated with happier faces, and slower, minor music with
40 Happy 40 sadder faces. Again, correlations between the emotional ratings of
FCA Score

the musical selections and the weighted average of emotional rat-

0 Angry 0 ings for the faces chosen as most/least consistent with the music
Sad were remarkably high: 0.97 (happy/sad), 0.94 (lively/dreary), and
-40 -40
Unsat.

0.88 (strong/weak).

Dark
Sad Angry
-80 -80
The fact that the pattern of cross-domain matching results is
0% 50% 100% 0% 50% 100% so clearly and consistently related to emotion in all three studies
Emotionality Emotionality provides strong support for emotional mediation as a mechanism
of at least some cross-modal associations. There is also evidence
Yellow

Yellow-Blue

Red
80 80 Red-Green
for emotional mediation in fragrance-to-color associations (31). It
40 40 does not rule out the possibility that there might also be direct or
Happy Angry
FCA Score

other sorts of associations. Spence, for example, argues that three

0 0 Happy types of cross-modal matching mechanisms have been established
Angry Sad empirically: structural correspondences based on common neural
-40 Sad -40 coding, statistical correspondences based on natural covariation
Green
Blue

of cross-modal attributes, and semantically mediated correspon-

-80 -80 dences based on common descriptive terminology (12). Although
0% 50% 100% 0% 50% 100%
Emotionality Emotionality he mentions affectively mediated correspondences as an addi-
tional possibility, he does not endorse them as having a strong
Fig. 6. Average color appearance values of the colors chosen as going best/ empirical basis. We believe that the foregoing results provide
worst with emotional faces (FCA scores) are plotted as a function of the clear evidence of cross-modal correspondences based on emotion.
degree of emotionality (0%, 50%, and 100%) for saturation, lightness, yel- Even though the present evidence does not rule out cross-
lowness-blueness, and redness-greenness. (Error bars represent SEMs.)
modal sensory associations based on covariation, theoretical
parsimony suggests that they are unnecessary. Moreover, it is
dreary/lively (r = +0.94), and weak/strong (r = +0.88), with generally unclear what the basis for such direct associations might
weaker but reliable correlations for angry/calm (r = +0.51). be, except for a very few cases, and those are actually related to
These findings provide additional support for the emotional emotional factors. It is realistic for angry faces to be associated
mediation hypothesis. with redder colors (experiment 2) because angry faces do tend
to be redder than happy or sad faces due to increased blood flow,
General Discussion but happy faces are not yellower nor sad faces bluer. Neither are
The results of experiment 1 showed remarkably strong associa- there any obvious reasons why fast, major music should be sys-
tions between high-level dimensions of classical orchestral music tematically associated with saturated, light, yellowish colors, or
by Bach, Mozart, and Brahms and appearance dimensions of slow, minor music with desaturated, dark, bluish colors.
color in nonsynesthetic participants. In particular, faster music in A great deal remains unknown about the nature and cause of
the major mode was generally associated with more saturated, these cross-modal associations. Might they be accounted for by
lighter, yellower colors, whereas slower music in the minor mode some more abstract nonemotional features such as the three
was associated with more desaturated (grayer), darker, bluer dimensions of the semantic differential (25): evaluation (good/
colors. Three further results showed that these music-to-color bad), potency (strong/weak), and activity (active/passive)? Do
associations were highly consistent with the emotional mediation people actually experience the emotions they associate with music
hypothesis. First, separate 2D emotional MDSs of colors and and/or colors or are they merely cognitive associations? Will the
musical selections were well explained by the same two emotional correlational evidence of emotional mediation diminish when
dimensions: happy/sad (positive/negative valence) and strong/ people’s affective experiences are compromised, either due to
weak (high/low potency). Second, the correlations between emo- neurological deficits or pharmacological interventions? Do the
tional ratings for the musical selections and a linearly weighted results generalize to forms other than classical orchestral music and
average of corresponding emotional ratings for the colors chosen to cultures in which nonwestern music is the norm? What factors
to go best/worst with the music were remarkably high: happy/sad produce the emotional associations to music and what factors
(+0.97), lively/dreary (+0.99), strong/weak (+0.96), and angry/ produce those to colors? And do music–color synesthetes have the
calm (+0.89). Third, all of the previously mentioned results were same associations between music and colors as nonsynesthetes?
essentially the same for participant samples in the United States Materials and Methods
and Mexico, as expected given prior results that color–emotion
Participants. There were 48 US and 49 Mexican participants in experiment
(24) and music–emotion associations may well be universal (29, 1, 24 US participants in experiment 2, and 16 US participants in experiment 3.
30). It is as yet unclear whether similar results will be evident in All factors of each experiment were fully orthogonal, within-subject designs
cultures that use nonwestern musical scales and structures. except culture (US/Mexican), which was between-subjects. All participants
The results of experiments 2 and 3 further support emotional had normal color vision (screened using the Dvorine Pseudo-Isochromatic
mediation. In experiment 2, people produced analogous effects Plates) and gave informed consent. We did not explicitly screen for synethesia
when choosing colors that are most/least consistent with emotion- but interviewed participants about their experiences of colors in their daily
ally expressive faces. First, there were highly reliable associations lives, and none reported experiencing color while listening to music. We note
between emotionally expressive faces and particular dimensions of that music–color synesthesia is very rare among the general population (0.2%)
(32). The institutional review boards at the University of California, Berkeley
color appearance. Second, face–color associations produced simi-
(Committee for the Protection of Human Subjects) and at the University of
larly high correlations between the emotional content of the faces Guadalajara (Academic Secretary Dr. Francisco Javier González Madariaga)
and the emotional associations of the colors chosen to be most/least approved the experimental protocol.
consistent with them: 0.97 (happy/sad), 0.94 (angry/calm), and 0.85
(weak/strong). Experiment 3 showed analogous patterns of results Experiment 1 Design, Displays, and Procedure. The three tasks were completed
Downloaded by guest on June 19, 2020

when people made associations between the musical selections by all 97 participants on different days, embedded in a battery of 32 tasks, so
from experiment 1 and an augmented set of the emotional faces it was not obvious that they were related. Unless otherwise specified, all

8840 | www.pnas.org/cgi/doi/10.1073/pnas.1212562110 Palmer et al.

 designs, displays, and procedures were the same for US and Mexican par- sad, and angry) with two levels of intensity (100% and 50% morphed) plus
ticipants. The 37 colors (Fig. 1) are specified in Table S1 and the 18 musical the single calm/neutral expression (27, 28). The colors were the 37 colors
selections in Table S2. Participants viewed the calibrated computer monitor described above. The colors and faces were rated along three bipolar
(an 18-inch diagonal Dell M990, 1024 × 768 pixel resolution, in the United emotional dimensions: happy/sad, angry/calm, and strong/weak. Displays
States and an 18-inch iMac monitor (Apple), 1680 × 1050 pixels, in Mexico) were programmed and presented using Presentation (www.neurobs.com)
from ∼70 cm. The background was always neutral gray [International and displayed on a 19-inch Dell E196FPf monitor (resolution 1040 × 768
Commission on Illumination (CIE) x = 0.312, y = 0.318, Y = 19.26]. Participants pixels). All three tasks were completed by the same participants during one
listened to the music through speakers connected to the same Dell and iMac experimental session in the following order.
computers in the United States and Mexico, respectively. Face–color associations. Each display contained one face (6.7 inches × 8.3 inches)
Music–color associations. Participants saw all 37 colors (always displayed as on the left of the screen with the display of 37 colors (Fig. 1) on the right of the
shown in Fig. 1) while listening to each of the 18 musical selections. [Data screen. Participants were asked to click (in order) on the five colors that were
were subsequently collected from 30 different participants when the color most/least consistent with the face, as specified in experiment 1.
array was rotated by 180°, thus reversing both up-down and left/right color Color–emotion associations. This task was the same as in experiment 1, except
positions. The results were nearly identical to the data reported above (r = that participants rated each color along the following 3 bipolar dimensions:
0.94, 0.94, 0.96, and 0.83 for saturation, light/dark, yellow/blue, and red/ happy/sad, angry/calm, strong/weak.
green dimensions, respectively). Between-subjects ANOVAs comparing them Face–emotion associations. Participants were presented with each face, one at
with the music-to-color associations reported above showed no significant a time, centered on the screen, which they rated along the same three
main effects or interactions after Bonferroni corrections for multiple com- dimensions as for colors. The color appearance rating data were those
parisons.] They initially heard each selection for 50 s, while they considered reported in experiment 1.
which colors were most consistent (went best) and most inconsistent (went
worst) with the music. They were then prompted to click on the five best Experiment 3 Design, Displays, and Procedure. In all respects not mentioned
colors, with each color disappearing when clicked. All colored squares then below, the methods of experiment 3 are the same as those of experiments 1 and
reappeared, and participants clicked on the five worst colors. Each musical 2. The 13 faces included the 7 female faces from experiment 2, plus morphed
selection looped continually during the music–color association task, with faces with 25% and 75% emotionality for each emotion (happy, sad, and
fade-in/fade-out transitions at the start and end.
angry). The music was the same as in experiment 1. Displays were presented
Color–emotion associations. Participants rated each of the 37 colored squares
on an 18-inch iMac (Apple) monitor (1680 × 1050 pixels) using Presentation
(100 × 100 pixels) singly in random order for consistency with each of the
software (www.neurobs.com). Participants completed all three tasks within
eight emotional terms (happy, sad, angry, calm, lively, dreary, active, passive,
the same experimental session in the following order: music–face associa-
strong, and weak) by sliding a cursor along a continuous scale and clicking to
tions, face–emotion associations, and music–emotion associations.
record their final position. The to-be-rated emotional term was centered
Music–face associations. This task was the same as the music–color association
above the square, and the 400-pixel response scale was centered 250 pixels
task in experiment 1, except that participants were presented with the array
below the square, with the left, right, and center points labeled “in-
of 13 faces (each was 0.7 inches × 0.9 inches), and their task was to click first
consistent,” “consistent,” and “neutral,” respectively. Responses were scaled
on the two faces that were most consistent with the music, followed by the
from −100 to +100. Trials were blocked by emotional term, with block order
two faces that were most inconsistent with the music.
randomized. Before each block, participants anchored the end points for the
Face–emotion associations. This task was the same as the color–emotion task in
particular emotional term by viewing all colors simultaneously and consid-
experiment 1 except that participants rated how consistent each face was
ering which color was most/least consistent with that term. Trials were
with the emotional terms of experiment 1.
separated by 500-ms intertrial intervals.
Music–emotion associations. This task was the same as the music–emotion task
Music–emotion associations. Participants heard each musical selection in ran-
in experiment 1.
dom order and rated its consistency with each of the same eight emotional
terms using the same procedure as for color–emotion associations.
ACKNOWLEDGMENTS. We thank two anonymous reviewers, Daniel Levitin,
Joe Austerweil, Chris Lau, Patrick Lawler, Diane Marian, Jonathan Sammar-
Experiment 2 Design, Displays, and Procedure. In all respects not mentioned tino, Allison Yamanashi Leib, Amy Liu, and Saki Wang for their help with this
below, the methods of experiment 2 were the same as experiment 1. The 14 research. The project was supported by a Google Gift, and the material is
faces were the orthogonal combination of 2 genders (male/female) × 7 facial based on work supported by National Science Foundation Grants 1059088
expressions created from the orthogonal combination of 3 emotions (happy, and 0745820 (to S.E.P.).

1. Cytowic RE, Eagleman D (2009) Wednesday Is Indigo Blue (MIT Press, Cambridge, MA). 16. Pridmore RW (1992) Music and color: Relations in the psychophysical perspective.
2. Ward J, Huckstep B, Tsakanikos E (2006) Sound-colour synaesthesia: To what extent Color Res Appl 17(1):57–61.
does it use cross-modal mechanisms common to us all? Cortex 42(2):264–280. 17. Odbert HS, Karwoski TF, Eckerson AB (1942) Studies in synesthetic thinking. I. Musical
3. Marks LE (1975) On colored-hearing synesthesia: Cross-modal translations of sensory and verbal associations of color and mood. J Gen Psychol 26(1):153–173.
dimensions. Psychol Bull 82(3):303–331. 18. Cutietta RA, Haggerty KJ (1987) A comparative study of color association with music
4. Karwoski TF, Odbert HS (1938) Color-music. Psychol Monogr 50(2):1–60. at various age levels. J Res Music Educ 35(2):78–91.
5. Marks LE (1974) On associations of light and sound: The mediation of brightness, 19. Arnheim R (1986) New Essays on the Psychology of Art (Univ of California Press, Berkeley).
pitch, and loudness. Am J Psychol 87(1-2):173–188. 20. Palmer SE, Schloss KB (2010) An ecological valence theory of human color preference.
6. Hubbard TL (1996) Synesthesia-like mappings of lightness, pitch, and melodic interval. Proc Natl Acad Sci USA 107(19):8877–8882.
Am J Psychol 109(2):219–238. 21. Schloss KB, Poggesi RM, Palmer SE (2011) Effects of university affiliation and “school
7. Marks LE (1987) On cross-modal similarity: Auditory-visual interactions in speeded spirit” on color preferences: Berkeley versus Stanford. Psychon Bull Rev 18(3):498–504.
discrimination. J Exp Psychol Hum Percept Perform 13(3):384–394. 22. Wright B, Rainwater L (1962) The meanings of color. J Gen Psychol 67(1):89–99.
8. Simpson RH, Quinn M, Ausubel DP (1956) Synesthesia in children: Association of colors 23. Valdez P, Mehrabian A (1994) Effects of color on emotions. J Exp Psychol Gen 123(4):394–409.
with pure tone frequencies. J Genet Psychol 89(1):95–103. 24. D’Andrade R, Egan M (1974) The colors of emotion. Am Ethnologist 1(1):49–63.
9. Caivano JL (1994) Color and sound: Physical and psychophysical relations. Color Res 25. Osgood CE, Suci GJ, Tannenbaum PH (1957) The Measurement of Meaning (Univ of
Appl 19(2):126–133. Illinois Press, Urbana).
10. Evans KK, Treisman A (2010) Natural cross-modal mappings between visual and au- 26. Collier GL (1996) Affective synesthesia: Extracting emotion space from simple per-
ditory features. J Vis 10(1):6, 1–12. ceptual stimuli. Motiv Emot 20(1):1–32.
11. Parise CV, Spence C (2012) Audiovisual crossmodal correspondences and sound sym- 27. Marian DE, Shimamura AP (2013) Contextual influences on dynamic facial ex-
bolism: A study using the implicit association test. Exp Brain Res 220(3–4):1–15. pressions. Am J Psychol 126(1):53–65.
12. Spence C (2011) Crossmodal corrospondences: A tutorial review. Atten Percept Psy- 28. Tottenham N, et al. (2009) The NimStim set of facial expressions: Judgments from
chophys 73(4):971–995. untrained research participants. Psychiatry Res 168(3):242–249.
13. Bresin R (2005) What is the color of that music performance? Proceedings of the In- 29. Bowling DL, Sundararajan J, Han S, Purves D (2012) Expression of emotion in Eastern
PSYCHOLOGICAL AND
COGNITIVE SCIENCES

ternational Computer Music Conference–ICMC 2005 (International Computer Music and Western music mirrors vocalization. PLoS ONE 7(3):e31942.
Association, San Francisco, CA), pp 367–370. Available at http://urn.kb.se/resolve?urn=urn: 30. Krumhansl CL (2002) Music: A Link Between Cognition and Emotion. Curr Dir Psychol
nbn:se:kth:diva-51920. Accessed October 10, 2009. Sci 11(2):45–50.
14. Sebba R (1991) Structural correspondence between music and color. Color Res Appl 31. Schifferstein HNJ, Tanudjaja I (2004) Visualising fragrances through colours: The
16(2):81–88. mediating role of emotions. Perception 33(10):1249–1266.
15. Barbiere JM, Vidal A, Zellner DA (2007) The color of music: Correspondence through 32. Simner J, et al. (2006) Synaesthesia: The prevalence of atypical cross-modal experiences.
Downloaded by guest on June 19, 2020

emotion. Empir Stud Arts 25(2):193–208. Perception 35(8):1024–1033.

Palmer et al. PNAS | May 28, 2013 | vol. 110 | no. 22 | 8841