Coding of Colour in Human Visual Cortex

Erin Goddard

School of Psychology

The University of Sydney

2010

Thesis submitted for the degree of Doctor of Philosophy

Acknowledgments

Over the course of completing this work I have been greatly assisted by the advice, expla-

nations and criticisms of my supervisor, Prof. Colin Clifford and my associate supervisor,

Dr. Samuel Solomon. I thank them for their patience and encouragement, and for sharing

their research expertise. I have also benefitted from and enjoyed the research environment

created by the other members of the Colour, Form and Motion Lab: Dr. Scott McDonald, Dr.

Kiley Seymour, Damien Mannion, and Karen Whittingham.

I was financially supported by (a) an Australian Postgraduate Award, and (b) a top-up

scholarship from the Australian Research Council Centre of Excellence in Vision Science.

My research was funded by grants awarded to Prof. Colin Clifford from the Australian Re-

search Council, the National Health and Medical Research Council, and the University of

Sydney.

I have been consistently supported in every other way by my family and friends, to whom

I owe much thanks. In particular, I thank my husband Andrew, my parents Peta and Paul,

my sisters Meghan and Madilen, and my parents-in-law, John and Glenys.

 Abstract

Colour is a rich and important component of our visual experience, providing information

that helps us determine surface properties and recognise objects and illuminants, as well

as affecting our mood and aesthetic enjoyment of the world around us. Over the last cen-

tury, significant advances have been made in our understanding of retinal and subcortical

processing of chromatic visual information, but cortical mechanisms of colour vision re-

main less well understood. In this thesis I present the results of three lines of investigation

into how cortical areas process chromatic information. The first two approaches use func-

tional Magnetic Resonance Imaging (fMRI), which allows for noninvasive investigation of

the functional neural response of human subjects, via the Blood Oxygenation Level Depen-

dent (BOLD) response. Firstly I use fMRI to define the early cortical visual areas retinotopi-

cally, and compare the response of these areas to colour and achromatic versions of a movie

clip. I use these functional measurements, along with maps of visual field coverage, to ad-

dress the question of how area V4, an area suggested to show a strong response to colour,

is organised in humans. I argue against the split dorsal / ventral model of human area V4,

which is homologous to macaque, finding that the data instead support a ventral hemifield

model. Secondly, I use fMRI to test for areas in human visual cortex whose responses can be

used to classify stimuli that would be indistinguishable to the postulated chromatic chan-

nels of subcortical areas. I present evidence that, as early as V1, the stimuli can be classified

based on the BOLD response. This result implies that information about the colour of the

stimulus is represented in cortex in a different way from that thought to occur in subcortical

areas. Finally, I use psychophysical adaptation to show that the colour opponent aftereffect

is of lesser magnitude when the adapting stimuli are perceived to be changing surfaces un-

der a constant illuminant, compared with when they are perceived instead to be constant

surfaces under a changing illumination. The two classes of adapting stimuli were the same

in their time averaged cone contrast values, and so should elicit the same degree of adapta-

tion in receptoral mechanisms. I argue that the difference in the magnitude of the aftereffect

is most parsimoniously explained by the existence of adaptable neural representations of

the reflectance of surfaces in a scene, offering insight into how the visual system achieves

colour constant responses. I relate these three experiments to the existing literature, and

also present future directions of research for better understanding the cortical mechanisms

underlying colour perception.

Contents

Declaration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv

List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

1 Introduction 1

1.1 Cortical mechanisms of colour vision . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Aim and scope of thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 Overview of thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Subcortical Mechanisms underlying Colour Vision 5

2.1 Physics of Colour . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.2 Historical development of colour vision theories . . . . . . . . . . . . . . . . . 6

2.3 Retinal processing of colour . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.4 Colour processing in the Lateral Geniculate Nucleus (LGN) . . . . . . . . . . 17

3 Higher-Order Colour Processing and Cortical Mechanisms 21

3.1 Psychophysical evidence for ‘higher-order’ mechanisms . . . . . . . . . . . . 21

3.2 Colour constancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.3 Are there specialised colour centres within cortex? . . . . . . . . . . . . . . . . 35

4 Using Functional Neuroimaging to Understand Cortical Colour Processing 37

4.1 Introduction to MRI physics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

4.2 The Blood Oxygenation Level Dependent (BOLD) signal . . . . . . . . . . . . 40

4.3 Experimental design in neuroimaging experiments . . . . . . . . . . . . . . . 44

i
 Contents

5 Cortical response to colour vs achromatic stimuli and the V4d debate 50

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

5.2 Materials and Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

5.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

5.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

6 Decoding non-cardinal colours from activity in human visual cortex 79

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

6.2 Materials and Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

6.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

6.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

7 Selective adaptation to surface colour 110

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

7.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

7.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

7.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

7.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

8 General Discussion 133

8.1 Novel Contributions of Presented Work . . . . . . . . . . . . . . . . . . . . . . 133

8.2 Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

References 139

Appendices 168

A Monitor Calibration 169

A.1 Estimating the cone fundamental primaries (LMS) of each RGB coordinate . . 169

A.2 Modelling the interaction between RGB channels (DLP only) . . . . . . . . . . 175

B Published Paper Resulting from Thesis 179

C Author Contributions to Experimental Work 210

ii
Declaration

This thesis is submitted to the University of Sydney in fulfilment of the requirement for

the degree of Doctor of Philosophy. The work presented in this thesis, to the best of my

knowledge, original except as acknowledged in the text. I hereby declare that I have not

submitted this material, either in full or in part, for a degree at this or any other institution.

Erin Goddard

iii
List of Figures

2.1 Trichromatic colour space from Helmholtz (1866) . . . . . . . . . . . . . . . . . 9

2.2 Retinal ganglion cell types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.3 Retinal circuitry underlying colour pathways . . . . . . . . . . . . . . . . . . . 15

2.4 DKL colour space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.1 Colour discrimination ellipses . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

3.2 Composition of the light signal . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

4.1 Retinotopic and attentionotopic mapping . . . . . . . . . . . . . . . . . . . . . 47

5.1 Two alternate models of human V4 . . . . . . . . . . . . . . . . . . . . . . . . . 54

5.2 Retinotopic mapping and colour responsiveness stimuli . . . . . . . . . . . . . 61

5.3 Definition of retinotopic areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

5.4 Colour response across visual cortex . . . . . . . . . . . . . . . . . . . . . . . . 69

5.5 BOLD response to chromatic and achromatic stimuli . . . . . . . . . . . . . . . 70

5.6 Average colour responsiveness of early visual areas . . . . . . . . . . . . . . . 72

5.7 Visual field coverage of areas V1, V2, V3 and V4 . . . . . . . . . . . . . . . . . 74

6.1 Non-cardinal colour stimuli . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

6.2 Projection onto cardinal mechanisms by non-cardinal colour stimuli . . . . . . 84

6.3 Fixation task performance for each subject . . . . . . . . . . . . . . . . . . . . . 89

6.4 Example map of functionally defined retinotopic areas . . . . . . . . . . . . . 93

6.5 Univariate bias for lime-magenta over orange-cyan . . . . . . . . . . . . . . . 98

6.6 Univariate and multivariate classifier performance . . . . . . . . . . . . . . . . 100

iv
 List of Figures

6.7 Multivariate classifier performance as a function of the number of voxels . . . 101

7.1 Example red-green adapting stimuli . . . . . . . . . . . . . . . . . . . . . . . . 116

7.2 Example blue-yellow adapting stimuli . . . . . . . . . . . . . . . . . . . . . . . 117

7.3 Example psychometric curves . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

7.4 Individual results: adapt to red, green, blue and yellow . . . . . . . . . . . . . 122

7.5 Averaged group results: adapt to surface colour . . . . . . . . . . . . . . . . . 123

7.6 Mean and variance of L, M and S cone contrast values . . . . . . . . . . . . . . 124

7.7 Individual results: low contrast control condition . . . . . . . . . . . . . . . . 125

7.8 Group results: low contrast control condition . . . . . . . . . . . . . . . . . . . 126

7.9 Individual results: adapting stimuli matched for onset and offset phase . . . . 128

7.10 Group results: adapting stimuli matched for onset and offset phase . . . . . . 129

A.1 Measured xyY coordinates for CRT monitor . . . . . . . . . . . . . . . . . . . . 170

A.2 Measured xyY coordinates for LCD monitor . . . . . . . . . . . . . . . . . . . 171

A.3 Measured xyY coordinates for DLP projector . . . . . . . . . . . . . . . . . . . 177

A.4 Schematic illustrating the masking model . . . . . . . . . . . . . . . . . . . . . 178

v
List of Tables

6.1 Cone contrast values of non-cardinal colour stimuli . . . . . . . . . . . . . . . 85

7.1 Adapting stimuli chromaticities . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

vi
Chapter 1

Introduction

1.1 Cortical mechanisms of colour vision

Colour vision provides useful information about objects and surfaces. Colour changes can

help us to separate the boundaries of an object from its background, as in detecting ripe fruit

amongst foliage. Colour often provides information about an object, such as whether meat

is raw or cooked, and whether a person is tanned or sunburnt, healthy or unwell. Colour

also adds considerably to our aesthetic appreciation of the world around us.

At the receptoral level, colour vision is based upon the three types of photoreceptor with

different but overlapping sensitivities to wavelength of light. These three signals are com-

pared by ganglion cells in the retina, and by cells in the lateral geniculate nucleus (LGN)

which then relays these signals to occipital cortex. In occipital cortex, there are areas which

have been proposed to be particularly specialised for the coding of colour in the visual scene,

although the nature and degree of this specialisation remains controversial. For higher level

processing of colour information, occipital cortical areas must interact with other cortical

areas, including areas involved in language, memory and object knowledge.

In vision research, colour vision has been a productive area of research for over 200 years.

Before they could be verified physiologically, the trichromatic input to our visual system, as

well as the opponent coding of colour (where, for example, red is the perceptual opposite of

1
 Chapter 1. Introduction

green), were inferred purely from observations of our perceptual experience of colour. With

the advent of physiological techniques, including electrophysiology, the response properties

of individual cells in the visual system could be characterised. From electrophysiology, we

now know a great deal about the mechanisms underlying colour vision in the retina and

LGN.

In cortex, the mechanisms underlying colour vision are not as clearly understood, largely

because response properties of cortical cells are more complex and hence more difficult to

characterise. Psychophysical studies on higher level colour processes, such as colour con-

stancy, often use a computational approach, starting by asking how the visual system could

potentially solve the problems of colour vision, rather than devising and testing models

based on neural processes. This is primarily due to the fact that the neural mechanisms are

not known in sufficient detail to inform modelling processes. Additionally, almost all elec-

trophysiological research has been carried out using non-human primates, mostly macaque

monkeys, as a model of human visual processing. There is a large degree of homology

between macaque and human, particularly for early cortical visual areas, but there is dis-

agreement about whether this is true for higher visual areas, including cortical areas thought

to be specialised for colour processing in the macaque.

Recent advances in non-invasive imaging techniques, including functional magnetic res-

onance imaging (fMRI) have enabled us to ask questions of the organisation and func-

tional specialisation in human visual cortex. Imaging complements electrophysiological

techniques by offering a different scale at which the system can be studied, and by allowing

easier comparison between macaque and human visual processing.

1.2 Aim and scope of thesis

The aim of the work presented here is to further elucidate the neural mechanisms of colour

vision, with a focus on processing of colour in human visual cortex. I will use fMRI and

psychophysical methods to examine the organisation of visual areas implicated in colour

vision, and to infer how chromatic information that is relayed from the LGN is transformed

2
 Chapter 1. Introduction

in visual cortex. In this thesis I do not consider colour vision in non-primate animals, or

anomalous colour vision in humans or non-human primates. I assume that colour vision,

particularly surface colour perception, is useful and of evolutionary benefit, but I do not

review the literature relating to the evolution of colour vision and the functional advantages

that may have contributed to evolutionary pressures. I consider some of the higher-order

functions of colour vision but do not present a detailed review of how colour perception

interacts with memory, language and other cognitive functions. My focus is primarily on the

neural mechanisms underlying colour perception in human visual cortex, and I only review

retinal and other sub-cortical mechanisms insofar as they are necessary to understand the

input to visual cortex.

1.3 Overview of thesis

In Chapter 2 I present a brief overview of some early advances in our understanding of

colour vision, before the relevant physiology was described. I then outline the processing

that takes place in the retina and lateral geniculate nucleus (LGN) that is relevant to the

processing of chromatic information.

In Chapter 3 I move to what is known of colour processing that takes place beyond the

retina and LGN. I review evidence for ‘higher order’ colour mechanisms that combine and

transform the information from parallel subcortical channels. I also present evidence from

electrophysiology, neuroimaging and lesion studies which implicate different cortical visual

areas to have a particular role in colour vision. Finally, I present a range of areas where

colour vision interacts with higher level vision, such as in colour constancy, and with cogni-

tive functions, such as memory and language.

In Chapter 4 I introduce the methodology of fMRI, its advantages and limitations, and its

application to vision research. I also give an overview of retinotopic mapping of human

visual cortex, one of the productive areas of vision research using fMRI.

In Chapter 5 I describe an fMRI experiment in which two models of the organisation of

human area V4 were tested. I tested the alternative arrangements of V4 using retinotopic

3
 Chapter 1. Introduction

mapping combined with functional responsiveness, specifically the response across visual

cortex to colour versus achromatic stimuli. Based on the results of this experiment I argue

that V4 is an area where the homology between macaque and human breaks down.

In Chapter 6 I used fMRI combined with multivariate pattern classification analysis to test

for the existence of higher order colour mechanisms, which combine information from the

two chromatically opponent channels thought to be relayed independently to cortex from

subcortical areas. The results of this experiment demonstrate that as early as V1, information

from the two channels is combined.

In Chapter 7 I present the results of a psychophysical adaptation experiment, in which I

tested for the presence of adaptable mechanisms tuned to the reflectance of objects in a

scene, rather than the raw colour signal that reaches the eye. The aftereffect I describe cannot

be explained by low level adaptation alone, and is most parsimoniously explained by the

existence of adaptable surface reflectance sensitive mechanisms.

Finally, in Chapter 8 I present a general discussion of the experimental work presented in

this thesis and how it contributes to our understanding of the cortical mechanisms of colour

vision.

4
Chapter 2

Subcortical Mechanisms underlying

Colour Vision

Our rich experience of colour includes the ability to discriminate and identify a diverse

range of combinations of hue, saturation and luminance, yet our perceptual experience is

based on the activity of just three categories of cone photoreceptor and the transformation

of these signals by sub-cortical and cortical areas.

2.1 Physics of Colour

Visible light comprises wavelengths of approximately 380 to 740 nm. The perceived colour

of a light depends on the power at each wavelength in the light, described as the spectral

power distribution of the light. When we view a scene, the light entering our eye from each

point depends both on the spectral power distribution of the illuminant and the reflectance

properties of the surface from which the light is reflected, as outlined in greater detail in Sec-

tion 3.2. Our visual system is trichromatic: there are three cone types in the normal human

retina, each with different but overlapping spectral sensitivity functions, and our percep-

tion of colour is based on the relative signals of these three classes of photoreceptor. This

means that we cannot discriminate every possible variation in the distribution of power

5
 Chapter 2. Subcortical Mechanisms underlying Colour Vision

across wavelength in visible light. We cannot use a photoreceptor’s response to determine

whether it is responding to a weak preferred stimulus or to a stronger non-preferred stim-

ulus, a principle referred to as ‘univariance’. We see any distribution of light which results

in the same activity of the three classes of cones as the same; lights which are physically

different but perceptually identical are termed ‘metameric’. It is important to note that this

principle only holds for isolated lights, referred to as ‘aperture colours’ (derived from the

notion of viewing a light through an aperture, in the absence of any surrounding context).

Perceptually, lights which are metameric can appear very different in the presence of a sur-

round, which can change the interpretation of a scene even for simple uniform surrounds

(Faul et al., 2008; Ekroll et al., 2004).

The trichromatic processing of colour by the visual system also means that any visible light

can be represented as the sum of three independent lights, as in colour matching experi-

ments (Brainard & Stockman, 2010), and the entire perceptual colour space (for aperture

colours) can be described as a three dimensional space. Several colour spaces are useful

for describing colour. The three dimensions of colour space can be based on physiological

mechanisms: for example, the extent to which a colour would activate each class of cone

photoreceptor; or on perceptual dimensions, where, for example, distances between differ-

ent colours are based on our ability to discriminate these colours. Below I consider the neural

substrates of colour vision, referring to different colour spaces based on the representation

of colour at each stage of the visual processing hierarchy.

2.2 Historical development of colour vision theories

Historically, theories of colour processing in the human visual system developed from two

viewpoints. The first, the Young-Helmholtz approach, explained perceptual phenomena

in terms of the trichromatic nature of early visual processing. Prior to the physiological

knowledge of different cone classes, their existence was hypothesised based on perceptual

observations and experiments (Palmer, 1777; Young, 1802). Palmer (1777) hypothesised the

existence of three particles in the retina, each of which is ‘moved’ by light of different wave-

6
 Chapter 2. Subcortical Mechanisms underlying Colour Vision

lengths. Young (1802) was the first to grasp both that the underlying physical variable, light

wavelength, was continuous, and to infer from trichromatic colour matching that our vi-

sual experience must be based on three kinds of retinal receptors, each broadly tuned for

wavelength and with different peak sensitivities (Mollon, 2003).

Helmholtz used perceptual experience of mixtures of lights and of pigments to explore and

describe the relationship between perception, the physical world, and the physiology of the

eye (von Helmholtz, 1866), recognising that our experience of colour must be understood in

terms of both the physical properties of light and objects and of the trichromacy of our eye.

He noted points at which our perceptual intuitions can be misleading about the physical

world: for example the sensation that violet and red are adjacent colours, with a smooth

perception gradation from red, through purple, to violet, when physically these lights are

at opposite extremes of the visible spectrum. Helmholtz described the additive nature of

light colours and the subtractive nature of pigments, and noted that while we may feel that

we can ‘see’ yellow and blue in green, this is not based on sensation but on inference learnt

from mixing pigments. We do not have direct perceptual access to the wavelengths which

comprise a given light.

Since our experience of colour is based on the activity of just three classes of photorecep-

tor, the range of all perceptually distinguishable colours can be described using just three

dimensions; Helmholtz (1866) gives the example of hue, saturation and luminosity. Alterna-

tively, colours can be defined as a mixture of three ‘fundamental colours’. Helmholtz noted

that using red, green, and violet gives a better coverage of colour space than red, yellow,

and blue, (which had also been proposed as fundamental colours), as shown in Figure 2.1,

A. Helmholtz used this and other observations to infer the spectral sensitivity of the long

(L), medium (M) and short (S) wavelength sensitive cones before they were known physio-

logically, as shown in Figure 2.1 (von Helmholtz, 1866).

Knowledge of the spectral sensitivities of our cones continues to inform understanding of

colour vision, particularly in describing its limits. Later research following a similar ap-

proach was based on the rationale that human perceptual limits on discrimination of differ-

ent colours can be derived from the shapes of the spectral sensitivity functions of the cones.

7
 Chapter 2. Subcortical Mechanisms underlying Colour Vision

Such an approach has proven most valuable for explaining phenomena caused by limita-

tions at this early stage of receptoral processing (Buchsbaum & Gottschalk, 1983; Kaiser &

Boynton, 1996). Cone spectral sensitivity functions can be used to derive colour matching

functions, which are used to calculate which lights are metameric to a human observer with

normal colour vision. Colour matching functions can be used to convert a colour signal into

a three dimensional representation of colour, for example using cone fundamental primaries

(LMS) or the CIE tristimulus coordinates (XYZ) as the three dimensions. Within such three

dimensional spaces, metameric colour signals map onto identical points.

Another approach to studying colour vision is that of Hering, which developed in the late

19th century as a result of observations relating to the opponent nature of many colour ex-

periences (Krauskopf, 1999). For example, Hering noted that while one may experience

a yellowish green, or a reddish blue, one does not experience a reddish green or a bluish

yellow (Hering, 1964). Hering also noted simultaneous contrast effects, for example that

objects on a green background appear redder, and that people with colour vision deficien-

cies cannot distinguish pairs of hues (red and green, or yellow and blue) (Fairchild, 2005).

The opponent coding suggested by Hering is found in the cone-opponent nature of post-

photoreceptor processing of colour. After the early trichromatic representation of colour by

the cone receptors, the signal is converted into two opponent channels, in which signals

from L and M cones are opposed (an L-M channel) and signals from S cones are opposed

with the combination of L and M cones (an S-(L+M) channel). Von Kries (1902), in dis-

cussing the phenomenon of chromatic adaptation and aftereffects, noted that colour could

meaningfully be represented in a colour space where opponent colours were given opposite

signs, for example where there is a red-green axis in which reds are given positive values

and greens negative values, going through a white origin. A similar organisation is used to-

day in the DKL colour space (Derrington et al., 1984; Krauskopf, 1999), illustrated below in

Figure 2.4 and the Macleod-Boynton equiluminant plane (MacLeod & Boynton, 1979).

Von Kries and others recognised that both trichromacy and the opponent nature of colour

could be accounted for by a single two-stage model (von Kries, 1902, 1905), but this was not

initially accepted as the mainstream view, which saw the strong evidence for trichromacy

8
 Chapter 2. Subcortical Mechanisms underlying Colour Vision

A B

1.

2.

3.
R O Y G B V

Wavelength (colour)

C D
1

0
Relative Absorptance

700 600 500 400

1

0
700 600 500 400
1

0
700 600 500 400
Wavelength (nm)

Figure 2.1: Trichromatic colour space, figures from von Helmholtz (1866). In A and C, Figures 4 and
5 (respectively) from Helmholtz (1866) are redrawn. In A, Helmholtz’s perceptual colour
space in redrawn as in his figure; the coloured triangle was not in the original, and the
colour space was rotated from his original to match the CIE 1931 xy space, shown in B,
which is a similar colour space currently used as a standard space. Helmholtz used the
figure shown in A to illustrate the fact that using red (R), green (G) and violet (V) as fun-
damental lights gives better coverage of this perceptual colour space than red, yellow
(Y) and cyan blue (C). Coloured lights that can be generated by some combination of
each set of fundamental colours are those within each triangle. Using this colour space,
and other observations from mixing coloured lights, Helmholtz predicted the approxi-
mate spectral sensitivity of the three classes of cones hypothesised to exist by Young; his
predictions of absorption as a function of wavelength for these cones (which he labelled
1, 2 and 3) are shown in C. For reference, D, redrawn from Solomon & Lennie (2007),
Figure 1a, shows the known spectral sensitivity of the long (L), medium (M) and short
(S) wavelength sensitive cones, as recorded physiologically.

9
 Chapter 2. Subcortical Mechanisms underlying Colour Vision

as discrediting Hering’s notion of opponent coding. Physiological evidence for trichro-

macy preceded that for the opponent-coding of colour, and it was not until these opponent

processes were described quantitatively (Hurvich & Jameson, 1955, 1956; Jameson & Hur-

vich, 1955, 1956), that a two-stage model of colour processing, including an opponent stage,

gained widespread acceptance (de Valois & de Valois, 1993). The actual opponent mecha-

nisms differ slightly from those predicted by Hering on the basis of perceptual experience,

as discussed in greater detail in Section 3.1.4, but the general acceptance of opponent coding

in colour vision was an important advance.

Since the mid 20th century, the dominant model of colour processing has been one of trichro-

macy in the retina, with three classes of cones, followed by an opponent stage. The signals

from the three classes of cones are transformed into red-green and blue-yellow opponent

channels and an achromatic channel within the retina, which is carried in parallel through

the lateral geniculate nucleus (LGN), and then further processing by visual cortex. At the

sub-cortical level, the chromatically opponent channels (L-M and S-(L+M)) carry informa-

tion in parallel to visual cortex via the parvocellular and koniocellular layers of the LGN

(Krauskopf et al., 1982; Derrington et al., 1984; Chatterjee & Callaway, 2003).

Cortical mechanisms of colour vision are generally less well understood. Psychophysical

adaptation experiments indicate the existence of higher-order colour mechanisms in the hu-

man visual system, which combine signals from both the opponent channels of sub-cortical

areas (Krauskopf et al., 1986; Webster & Mollon, 1991; Zaidi & Shapiro, 1993). Within cortex,

various visual areas, particularly in ventral visual cortex, have been proposed as ‘colour cen-

tres’ in macaque (Zeki, 1973) and human (Lueck et al., 1989) visual cortex. These processing

stages are considered in greater detail in Chapter 3.

10
 Chapter 2. Subcortical Mechanisms underlying Colour Vision

2.3 Retinal processing of colour

2.3.1 Cones

At the receptoral level, colour vision is mediated by the cone photoreceptors, whose coni-

cal shape morphologically distinguishes them from the rod photoreceptors. In the normal

human retina there are three classes of cones, with overlapping but different spectral sensi-

tivity functions. These three cone types have a peak absorbance of light at  430nm (short

wavelength, or ‘blue’ cone),  530nm (medium wavelength, ‘green’ cone) and  560nm

(long wavelength, ‘red’ cone) (Solomon & Lennie, 2007). Spectral sensitivity functions give

the probability that the cone will absorb a photon of light, as a function of wavelength.

The ratio of signals from the three classes of cones is used to derive information about the

chromatic properties of the light.

Cone spectral sensitivity functions have been measured behaviorally in humans using test

stimuli whose detection depends upon the sensitivity of one cone type. Originally this

method was used by Smith & Pokorny (1975) to estimate the spectral sensitivity functions of

each cone type, and these have since been modified with retesting for L and M cones (Stock-

man & Sharpe, 2000). Stockman & Sharpe (2000) tested X-chromosome linked dichromats,

missing either L (protanopes) or M (deuteranopes) cone pigments, to isolate the sensitivity

of the largely overlapping L and M cones. Spectral sensitivity functions have also be esti-

mated using microspectrophotometry (Dartnall et al., 1983) and using electrophysiological

measurements (Schnapf et al., 1987). As mentioned above, the spectral sensitivity functions

of cones can be used to predict which colours are metameric to normal observers.

Amongst observers with normal (trichromatic) colour vision, there is considerable variabil-

ity in the proportion of each type of cone. S cones make up around 5-10% of the cones in

the retina (Curcio et al., 1991), and typically avoid the central fovea of old world monkeys

(Williams et al., 1981; Curcio et al., 1991; Martin & Grünert, 1999), but not new world mon-

keys (Martin & Grünert, 1999). M and L cones, which are genetically similar and most likely

evolved from a common cone type, (Nathans et al., 1986a,b; Dacey, 2000) make up the re-

mainder of cones, but the L/M cone ratio is reported to vary between 0.37 and 9.71 (Roorda

11
 Chapter 2. Subcortical Mechanisms underlying Colour Vision

& Williams, 1999; Hagstrom et al., 1998; Brainard et al., 2000; Hofer et al., 2005). These varia-

tions seem to result in little change to colour perception, for example in observers’ identifica-

tion of unique yellow (the wavelength of light perceived to be neither reddish nor greenish)

(Brainard et al., 2000). This lack of variation in colour perception with different L/M ratios

suggests that neural circuitry underlying colour perception compensates for variation in the

L/M ratio. The organisation of L and M cones in human retina is not significantly different

from random (Roorda et al., 2001; Williams & Roorda, 1999), while the S cones are regularly

spaced in the peripheral retina (Curcio et al., 1991; Williams & Roorda, 1999).

2.3.2 Spectral processing within the retina

The retina contains a basic, three-neuron excitatory pathway, in which the photoreceptors

transduce the light stimulus and bipolar interneurons relay the photoreceptor signals to

ganglion cells, the output neurons of the retina. A summary of retinal ganglion cell types

and their targets in the LGN is shown in Figure 2.2. Responses in the basic excitatory path-

way of the retina are modulated by two classes of inhibitory interneuron, the horizontal

and amacrine cells, which function at the photoreceptor-bipolar and the bipolar-ganglion

synapses respectively (Dacey, 2000). Horizontal cells preserve the sign of the signal from

the cones and pass this signal on to the cone bipolar cells (Haverkamp et al., 2000).The bipo-

lar cells that contact the cones can be separated into two classes: the OFF cone bipolar cells

which are hyperpolarised by light, and the ON bipolar cells which are depolarised (Wässle,

2004). OFF cone bipolar cells have ionotropic receptors that invert the sign of the glutamate

mediated signal from the cones, whereas ON cone bipolar cells have metapatropic receptors

that preserve the sign. Similarly, there are two classes of ganglion cells; OFF and ON gan-

glion cells that receive excitatory input from the OFF and ON bipolar cells respectively. This

means that OFF ganglion cells are excited by stimuli that are darker than the background,

while ON ganglion cells are excited by light increments (Wässle, 2004).

Of particular relevance to colour vision, there exist retinal ganglion cells whose receptive

fields are spectrally opponent (de Monasterio & Gouras, 1975; Lee, 1999). That is, ganglion

cells that receive input from more than one class of cone photoreceptor, and with opposite

12
 Chapter 2. Subcortical Mechanisms underlying Colour Vision

Retina LGN
Midget ganglion cells
Central
red ON
green ON
red OFF
green OFF
blue OFF?
Parvocellular
Peripheral
luminance ON
luminance OFF

Small bistratified ganglion cells

blue ON

Koniocellular

Parasol ganglion cells

luminance ON
luminance OFF

Magnocellular
Other ganglion cell types

?
?
?

Figure 2.2: Summary of the retinal ganglion cell types and their targets in the LGN. Adapted from
Dacey (2000), Figure 1.

13
 Chapter 2. Subcortical Mechanisms underlying Colour Vision

sign, carrying the difference signal between these two cone types. Reviews of spectral op-

ponency, and the retinal circuitry likely to underlie red-green and blue-yellow opponency

are given by Lee (1999), Dacey (2000) and Dacey & Packer (2003). A summary of the likely

circuitry in each case is given below, and is illustrated in Figure 2.3.

Red-green opponency

The red-green opponent signal is carried by the midget ganglion cells, which project to the

parvocellular layers of the LGN (Reid & Shapley, 1992). The midget ganglion cells receive

input from the midget bipolar cells, which in turn receive input from M and L cones, but

not S cones (Lee & Grünert, 2007). Both bipolar and ganglion cells have a ‘centre-surround’

organisation, in which their receptive field is made up of a small excitatory centre and a

larger inhibitory surround. The majority of these cells respond to a luminance difference,

and can be separated into two classes: ON cells respond maximally to a luminance incre-

ment in the centre of their receptive field and a luminance decrement in the surround, while

OFF cells respond maximally to a luminance decrement in the centre and an increment in

the surround. Where there is a difference in the ratio of M and L cone inputs to the centre

and surround, the cell will also have a centre-surround chromatic preference, with the sign

depending on whether the receptive field centre is dominated by input from L or M cones.

This leads to four classes of ‘centre-surround’ receptive fields with a chromatic component;

they can be either L-ON, L-OFF, M-ON or M-OFF. The excitatory centre of a midget bipo-

lar cell is given by input from one or more L and/or M cones, and the inhibitory surround

results from input to the bipolar cell from H1 horizontal cells, which receive input from a

range of L and M cones.

The circuitry for red-green opponency likely developed from existing circuitry of the central

midget pathway, which evolved to enable high spatial acuity vision (Dacey, 2000; Wässle,

2004; Lennie & Movshon, 2005). It is unclear whether red-green opponency in the midget

ganglion and diffuse bipolars arises from selective targeting of different cone types, or from

random sampling of the L/M cone mosaic. In the central retina, the excitatory input to the

midget pathway approximates the response of a single cone: midget bipolar cells receive

14
 Chapter 2. Subcortical Mechanisms underlying Colour Vision

input from just one cone, and output predominantly to just one ganglion cell (Calkins et al.,

1994). Particularly in the central retina it is possible that opponency arises from random

connections because the receptive field centre often receives input predominantly from a

single cone, while the larger surround is likely to receive input from more equal numbers

of L and M cones than the centre (Dacey, 2000; Lennie, 2000). According to this random

wiring hypothesis, colour opponency should decrease in the more peripheral retina where

both centre and surround are larger and receive input from multiple cones, but this was not

found in macaque retina, where colour opponent midget ganglion cells were found in the

periphery (20-50 degrees eccentricity) (Martin et al., 2001). Also in opposition to the random

wiring hypothesis are reports that cone inputs to both the centre and surround of parvocel-

lular receptive fields in the LGN are specific for either L or M cones (Reid & Shapley, 1992,

2002), implying mechanisms that selectively target different cone types (Dacey & Packer,

2003).

A B C
L M L M S L L M L M S L L M L M S L
cones cones cones

midget diffuse blue cone diffuse

bipolar cell bipolar cell bipolar cell bipolar cell

OFF OFF
ON-OFF ON
midget parasol
ganglion cell small bistratified ganglion cells
ON ganglion cell

Figure 2.3: Major elements of likely retinal circuitry underlying the red-green chromatically oppo-
nent pathway (A), the blue-yellow chromatically opponent pathway (B) and the lumi-
nance pathway (C). Redrawn from Martin (1998), Figure 1.

Blue-yellow opponency

The retinal circuitry conveying signals from the S cones differs qualitatively from that con-

veying M and L cone signals (Calkins, 1999). Unlike the red-green opponent cells, which

occur as L-ON and L-OFF cells in approximately equal numbers, blue-yellow ganglion cells

are biased towards blue-ON cells.

The small bistratified retinal ganglion cell (Dacey, 1993) carries the S-(L+M) or blue-ON op-

15
 Chapter 2. Subcortical Mechanisms underlying Colour Vision

ponent signal (Dacey & Lee, 1994; Calkins et al., 1998). Signals from S cones are opposed to

the combination of L+M cone signals by the S-ON bipolar cells, which receive excitatory in-

put exclusively from S-cones (Mariani, 1984), and output to the blue-ON ganglion cell. The

inhibitory surround of the S-ON bipolar cell is derived from input from the H2 horizontal

cells; the S-ON bipolar is the first site of opponency between the S and (L+M) cone signals

(Dacey, 2000). Another contribution to the S-(L+M) spectral opponency is made at the level

of the ganglion cell, where the (L+M)-OFF bipolar pathway modifies the signal from the

S-ON bipolar cell to enhance spectral antagonism and change the centre-surround arrange-

ment of the receptive field inherited from the S-ON bipolar cell. Unlike the L-M opponent

cells, the S-(L+M) ganglion cells have spatially coextensive ON and OFF subregions in their

receptive fields (Dacey, 2000; Crook et al., 2009). While both the S-ON and (L+M)-OFF bipo-

lar inputs contribute to the spectral opponency of the blue-ON ganglion cell, a recent study

found an earlier origin of the opponent signal. Packer et al. (2010) report that blue-yellow

spectral opponency is present in the response of S-cones, an opponent signal that likely

originates in L+M horizontal cell feedback to S cones.

Initially it was unclear whether an S-OFF pathway existed in the retina, but the potential

circuitry has now been identified (Klug et al., 2003; Dacey et al., 2003; Dacey & Packer, 2003).

While the S-ON pathway has its own S-ON bipolar and ganglion cell, the S-OFF pathway

uses the midget bipolar and ganglion cells also used in the red-green opponent pathway.

Klug et al. (2003), in a sample of 118 cones, found 5 S-cones, all of which contacted an S-

ON bipolar cell and an S-cone OFF midget bipolar cell, but never an ON midget bipolar

cell. These S-cone OFF midget bipolar cells were morphologically indistinguishable from

OFF midget bipolar cells contacted instead by L or M cones. Each S-cone OFF bipolar cell

contacts an OFF midget ganglion cell, as in the L and M cone midget pathway.

Luminance mechanisms

Luminance is the phenomenological correlate of radiance, the percept of lightness or dark-

ness. It is defined by a spectral luminous efficiency function, or luminosity function, V pλq,

which specifies how effective each wavelength is in contributing to the percept of lightness.

16
 Chapter 2. Subcortical Mechanisms underlying Colour Vision

Estimates of V pλq differ slightly with the technique used to define it, but standard methods,

such as heterochromatic flicker photometry, yield closely matching estimates (Sharpe et al.,

2005; Stockman & Sharpe, 2000; Lennie et al., 1993). Estimates of V pλq indicate that the sen-

sitivity of normal human luminance perception corresponds to a channel combining signals

of the same sign from L and M cones in a ratio of 2:1, approximating the signal carried by

the magnocellular pathway (Lee et al., 1988). This suggests that luminance perception relies

predominantly on the combined signal from L and M cones carried by the magnocellu-

lar pathway, although other mechanisms are likely to contribute, particularly under certain

conditions of chromatic adaptation (Lennie et al., 1993; Stockman et al., 2005; Stockman &

Plummer, 2005). S-cones make no substantial contribution to luminance perception at low

luminance (Buchsbaum & Gottschalk, 1983; Lennie & D’Zmura, 1988), and add an inverted

(S-OFF), delayed signal at higher luminance levels (Ripamonti et al., 2009). For a review of

luminance mechanisms, see Lennie et al. (1993).

2.4 Colour processing in the Lateral Geniculate Nucleus (LGN)

Retinal signals are relayed through the LGN in three channels, two chromatically opponent

channels and a spectrally broadband mechanism approximating the luminosity function

V pλq (Lennie & Movshon, 2005). Axons of the retinal ganglion cells leave the eye via the

optic nerve and terminate in the lateral geniculate nucleus (LGN). The LGN comprises four

dorsal parvocellular layers and two ventral magnocellular layers, interleaved with the much

thinner koniocellular layers or interlaminar zones (Solomon & Lennie, 2007). The red-green

and blue-yellow opponent signals are carried by the parvocellular and koniocellular layers

of the LGN, and are relayed in parallel to the cortex. Magnocellular responses resemble a

luminance mechanism (V pλq) but so do parvocellular cells for high spatial frequency stimuli.

Stimuli used to isolate the three channels carried through the LGN are used to define the

DKL colour space (Derrington et al., 1984), illustrated in Figure 2.4.

The chromatic opponency of the midget retinal ganglion cells and the small bistratified gan-

glion cell is inherited by their targets in the parvocellular and koniocellular layers of the

17
 Chapter 2. Subcortical Mechanisms underlying Colour Vision

L+M+S
(achromatic)

-L+M
(green)
-S
(yellow)

+S
(violet)
+L-M
(red)

Figure 2.4: The Derrington et al. (1984) (DKL) space is based on modulations that isolate the two
chromatically opponent channels of the LGN, along with the luminance channel. The
axes are defined in terms of combinations of the long (L), medium (M), and short (S)
wavelength sensitive cones, as indicated beside each axis. The ‘constant S’ plane is
red-green opponent, the ‘constant L-M’ plane is violet-yellow opponent, and the third,
achromatic plane varies with luminance; all three intersect at an achromatic, average
luminance point (the origin). The hue, saturation and brightness of a coloured light
are given the polar coordinates of the light in the space: by the azimuth, elevation and
distance from the origin (respectively).

18
 Chapter 2. Subcortical Mechanisms underlying Colour Vision

lateral geniculate nucleus (Solomon & Lennie, 2007; Dacey & Lee, 1994; Calkins et al., 1998).

Midget ganglion cells that project to the parvocellular layers comprise approximately 80%

of the input to the LGN from the retina (Callaway, 2005). The parvocellular pathway does

not exclusively carry chromatic information, but is also the pathway associated with high

spatial acuity form vision. The parvocellular pathway likely evolved to be specialised to

carry high spatial resolution signals at high contrast levels in the photopic range, and fol-

lowing the divergence of L and M cone types the same circuitry could be used to carry a

red-green opponent signal (Martin, 1998).

The red-green opponent signal, carried by the parvocellular layers of the LGN, is carried

by cells with a centre-surround organisation like that of the midget retinal ganglion cells.

Wiesel & Hubel (1966) initially described two classes of receptive field organisation in the

parvocellular layers of the LGN, Type I cells which had a central excitatory region and a

larger suppressive surround, and Type II cells which had spatially coextensive excitatory

and inhibitory regions. It is now thought that the Type II cells they encountered were not

parvocellular cells, but blue-on cells belonging to the koniocellular layers (Martin, 1998).

Parvocellular cells typically receive little or no input from S-cones, although a minority of

parvocellular cells receive weak inhibition from S-cones (Tailby et al., 2008c). In addition

to their red-green opponency, most parvocellular cells have a luminance on- or off- type

response. This gives rise to four classes of cell, with the excitatory centre driven either by

L or M cones, and the L/M modulation in phase with a luminance on- or off- response

(Martin, 1998).

The blue-yellow channel compares the activity of the S cones with the combined activity

of L and M cones. Unlike the red-green opponent channel, blue-ON and blue-OFF neu-

rons are not functionally symmetric: blue-OFF cells are less sensitive to S-cone contrast than

blue-ON cells, and have larger receptive fields (Tailby et al., 2008c). Blue-ON cells in the

LGN are generally thought to have the same coextensive receptive field structure of the

small bistratified ganglion cell, which opposes the S and L+M signals in the retina (Dacey,

2000; Derrington et al., 1984), corresponding to the Type II organisation of Wiesel & Hubel

(1966), although Tailby et al. (2008c) showed that blue-ON cells show band-pass spatial fre-

19
 Chapter 2. Subcortical Mechanisms underlying Colour Vision

quency tuning, consistent with some spatial antagonism. Tailby et al. (2008c) also report that

blue-OFF cells have low-pass spatial frequency tuning, consistent with these cells having a

spatially coextensive centre and surround.

Overall, the LGN relays information from the retina to cortex with little transformation of

the signals. The chromatically opponent signals, carried in parallel through the LGN, also

enter the cortex in parallel (Chatterjee & Callaway, 2003). The koniocellar layers of the LGN,

carrying the S-ON responses, project to the superficial layers 3 and 4A of primary visual

cortex, while the parvocellular and magnocellular cells project predominantly to layers 4Cβ

and 4Cα, respectively (Solomon & Lennie, 2007). Transformations of the signal relayed from

the LGN to cortex are discussed in Chapter 3.

20
Chapter 3

Higher-Order Colour Processing and

Cortical Mechanisms

Compared with pre-cortical processing, less is known of cortical processing of chromatic

information; there is less agreement on both the response properties of cortical neurons

with chromatic selectivity and the function of these neurons in colour vision. Below is a

brief overview of some general areas of research.

3.1 Psychophysical evidence for ‘higher-order’ mechanisms

Colour processing in the retina and LGN is well described by a three channel model, but psy-

chophysical evidence points to the existence of more than three channels of chromatic infor-

mation. The psychophysical results summarised here address the question of whether there

are ‘higher-order’ colour mechanisms in the human visual system that receive input from

both of the chromatically opponent channels of sub-cortical areas. Overall, the strongest

psychophysical evidence for higher-order mechanisms comes from the effects of habitua-

tion: decreases in sensitivity and shifts in perceived hue induced by prolonged viewing of

a particular colour. Other methods have been used to test for higher-order colour mecha-

nisms (see reviews by Krauskopf, 1999 and Eskew, 2009). Different approaches have yielded

21
 Chapter 3. Higher-Order Colour Processing and Cortical Mechanisms

conflicting results, or are not universally accepted as requiring a model beyond one based

on cardinal mechanisms (Eskew, 2009). Below I describe evidence for non-cardinal mecha-

nisms from habituation experiments, and also outline variations in chromatic discrimination

across wavelength, an area where cardinal mechanisms may be sufficient to account for the

results. In both these areas of investigation, and others not covered here, cardinal mecha-

nisms are most commonly referred to as comprising two opponent channels, based on the

known physiology, but the rationale of these experiments cannot be used to discriminate

between two bipolar mechanisms or four unipolar mechanisms (Eskew, 2009).

3.1.1 Sensitivity decreases induced by habituation

Krauskopf et al. (1982) used habituation to identify the ‘cardinal’ directions of colour space

(red-green and blue-yellow, corresponding to L-M and S/(L+M) mechanism isolating stim-

uli). They showed that habituation to red-green did not reduce sensitivity to blue-yellow

stimuli, and similarly that habituation to blue-yellow did not reduce sensitivity to red-green

stimuli. This is consistent with the red-green and blue-yellow stimuli being processed with

independently adaptable mechanisms. They also found that habituation to either cardi-

nal affected sensitivity to all non-cardinal (intermediate) colours, and habituation to a non-

cardinal modulation reduced sensitivity to cardinal colours. This is consistent with all non-

cardinal colours evoking a response in both cardinal channels; habituation to either cardinal

affects the perception of all colours except the opposite cardinal, and habituation to a non-

cardinal colour habituates both cardinal channels to some extent.

In 1986, Krauskopf et al. reanalysed the data from the earlier paper and argued that desensi-

tisation induced by adaptation to chromatically modulating fields could not be explained by

opponent mechanisms tuned to cardinal colour directions alone, that is, that there must also

be mechanisms selective for non-cardinal colour directions. In favour of this conclusion,

Krauskopf et al. (1986) presented three lines of evidence. First they performed a Fourier

analysis on the post-habituation decreases in sensitivity, as a function of test colour. The

Fourier analysis revealed a pattern of second harmonics which were inconsistent with only

cardinal mechanism habituation, implying the existence of adaptable non-cardinal mecha-

22
 Chapter 3. Higher-Order Colour Processing and Cortical Mechanisms

nisms in addition to the cardinal mechanisms. Second they repeated the original experiment

with conditioning and test stimuli confined to the isoluminant plane, and found similar re-

sults, again supporting the notion of higher-order colour mechanisms. Third, they tested

observers’ ability to discriminate stimulus colour at detection threshold, as discussed in the

section below.

3.1.2 Colour appearance changes induced by habituation

Habituation to coloured or achromatic stimuli can also induce changes in perceived hue, a

phenomenon that has been used to probe for evidence of non-cardinal colour mechanisms.

Guth & Moxley (1982) used prolonged viewing of achromatic flicker to demonstrate an in-

teraction between the achromatic channel and the red-green opponent channel. They mea-

sured changes in perceived hue following monocular habituation to a flickering black/white

field, which alternated at 1 Hz (flicker condition) or 50 Hz (fused condition, perceived as a

steady grey field). They found greater brightness adaptation in the 1 Hz condition: the

perceived brightness of a white test decreased to a greater extent following habituation to

the 1 Hz flicker. Guth & Moxley attributed this result to greater adaptation at the level of

the post-receptoral achromatic channel, since the receptors should have the same quanta

catches in both conditions. They also tested the impact of achromatic flicker habituation

on the perceived hue of chromatic tests. Red and green (long and medium) tests were per-

ceived as more yellow (an intermediate wavelength) following prolonged viewing of 1 Hz

flicker stimulus, compared with the 50 Hz (fused) stimulus. Guth & Moxley (1982) argue

that this result is consistent with the achromatic channel having an inhibitory influence on

the red-green channel, implying an interaction between these cardinal channels.

In a study following a similar rationale to Krauskopf et al. (1986), Webster & Mollon (1991,

1994) showed post-adaptation shifts in colour appearance which imply non-cardinal mech-

anisms that are adapted independently of cardinal ones. Using a model of three indepen-

dent post-receptoral channels that linearly combine the cone signals (L-M, S, and achro-

matic), they made testable predictions about how contrast adaptation should alter colour

appearance. Specifically, they predicted changes in both perceived contrast and in appar-

23
 Chapter 3. Higher-Order Colour Processing and Cortical Mechanisms

ent hue that would be invoked by contrast adaptation according to the three channel model.

Changes in perceived hue are predicted to occur whenever the adaptation results in a greater

decrease in sensitivity of one of the chromatically opponent channels than the other. Fur-

thermore, these changes in hue would only occur for test stimuli that lie intermediate to

the L-M or S axes of equiluminant colour space; stimuli that lie along the cardinal axes can

reduce in contrast but should not shift in hue if the percept of these colours is based on only

one channel.

Webster & Mollon (1994) found results contrary to those predicted by the three channel

model in both changes in perceived contrast and shifts in perceived hue. Contrast adapta-

tion to each of the eight colour directions they tested resulted in reduced perceived contrast

for all test stimuli, but the greatest reduction was for those stimuli that lay along the adapt-

ing axis, and the least reduction for stimuli along the orthogonal colour direction. This

was true for both cardinal and non-cardinal adapting stimuli, implying the adaptation of

non-cardinal chromatic channels in addition to the adaptation of the cardinal mechanisms.

In colour appearance they found that adaptation biased the perceived hue of test stimuli

away from the adapting axis and toward the orthogonal axis, again for all adapting stimuli.

Webster & Mollon (1994) argue that the three channel model of colour processing cannot

account for the reported changes in perceived hue of test stimuli along the L-M and S axes,

again providing evidence of independently adaptable mechanisms tuned to non-cardinal

directions in colour space.

An alternative interpretation of the effect of habituation on colour appearance is presented

by Zaidi & Shapiro (1993). Zaidi & Shapiro (1993) propose a model that includes just two

chromatically opponent mechanisms (red-green and blue-yellow), but allows for interac-

tion between these channels. According to this model, habituation of one channel induces

changes not only in the response properties within that channel, but also on the magnitude

of interaction between the channels. This model is based on the notion of adaptive orthog-

onalisation, that the most efficient means of conveying visual information is to modify the

responsiveness of each channel such that their responses are mutually orthogonal. Zaidi

& Shapiro (1993) fit the free parameters of their model to measurements of post-adaptation

24
 Chapter 3. Higher-Order Colour Processing and Cortical Mechanisms

changes in colour discrimination thresholds, and show that the model can account for those

data. They also use these parameters to predict the effect of habituation on colour appear-

ance, and qualitatively simulate the results of Webster & Mollon (1991). This model does

not provide evidence against higher-order chromatic mechanisms: the interaction between

channels is a higher-order mechanism since the two chromatically opponent channels are

not independent. However, the model demonstrates that the existence of mechanisms tuned

to non-cardinal directions in colour space is not necessary to explain these higher-order ef-

fects.

3.1.3 Colour Discrimination

As mentioned above, in addition to their habitation experiments, Krauskopf et al. (1986)

tested detection threshold discrimination ability as a further test for higher-order mecha-

nisms. Their rationale was that if cardinal mechanisms were the only ones employed at

detection threshold, then non-cardinal colours should vary in appearance depending on

whether they were detected by one or the other or both of the cardinal mechanisms. They

did not find a dissociation between detection and discrimination performance, even for non-

cardinal colours. They presented this result as further evidence against the notion that only

cardinal mechanisms are used at detection threshold. However, this final argument is only

sound if the non-cardinal stimulus is equally detectable by the two cardinal mechanisms

(Eskew, 2009), and so this result may not reflect the action of non-cardinal mechanisms at

detection threshold.

Further to the work of Krauskopf et al. (1986), Mullen & Kulikowski (1990) tested dis-

crimination ability at detection threshold for monochromatic flashed spots against a white

background. They found evidence of four distinguishable bands of wavelength at detec-

tion threshold, and inferred that detection at threshold is subserved by at least four mech-

anisms, each associated with a different colour sensation. At detection threshold, wave-

lengths within each band were not discriminable, while wavelengths from different bands

were discriminable. Mullen & Kulikowski (1990) note that the chromatic tuning of each

band is consistent with detection relying on the L-M opponent mechanism, and either the S-

25
 Chapter 3. Higher-Order Colour Processing and Cortical Mechanisms

(L+M) opponent mechanism and/or the luminance mechanism. These results imply the ex-

istence of at most four chromatic mechanisms, and so are consistent with two chromatically

opponent, cardinally tuned mechanisms. However, this does not preclude the possibility

that there are higher-order mechanisms that respond to suprathreshold stimuli.

In a similar study, Calkins et al. (1992) extended the work of Mullen & Kulikowski (1990)

to test discrimination thresholds for suprathreshold stimuli. They used a 578 nm adapting

field, and monochromatic stimuli above 530 nm. Within this limited range, they found two

bands of wavelength where stimuli can be rendered indiscriminable by appropriate scaling

of intensity. This result held for stimuli up to 0.7 log units above detection threshold, sug-

gesting that at least for this bandwidth of wavelengths, the results of Mullen & Kulikowski

(1990) at threshold are also true of suprathreshold stimuli.

In 1992, Krauskopf & Gegenfurtner tested colour discrimination thresholds for suprathresh-

old colours in an equiluminant plane. When subjects were adapted to a modulation along

one of the cardinal axes, they had an increased discrimination threshold for stimuli that

varied along the same cardinal axis. However, for stimuli that varied along the orthogonal

cardinal axis to the one that was adapted, subjects did not show an increase in discrimi-

nation threshold, consistent with only cardinal sensitive mechanisms being affected by the

adaptation.

In an experiment where subjects adapted to white but were tested on their discrimination

performance for a range of locations in colour space, Krauskopf & Gegenfurtner (1992)

found evidence of non-cardinally tuned mechanisms. Shown in Figure 3.1 are the results of

this experiment. Their rationale was that if discrimination performance were based only on

the responses of two cardinal mechanisms, then the major and minor axes of the discrimina-

tion ellipses would always be parallel to the cardinal axes. At test directions intermediate to

the cardinal axes (450 , 1350 , 2250 and 3150 ), the ellipses would be circular. Instead, as Figure

3.1 shows, along the orange-cyan non-cardinal axis the discrimination ellipses were oriented

along this non-cardinal axis, inconsistent with performance based on cardinal mechanisms

alone. It may be that this result depends upon suprathreshold stimuli; in an earlier study

that tested discrimination ellipses at low luminance levels (25 Td) the ellipses did not vary

26
 Chapter 3. Higher-Order Colour Processing and Cortical Mechanisms

in shape for different reference chromaticities (Boynton et al., 1986).

10

5
S - (L + M)

-5

-10
-10 -5 0 5 10

L-M
Figure 3.1: Colour discrimination ellipses of Krauskopf & Gegenfurtner (1992), adapted from their
Figure 14. Subjects were tested on their discrimination ability around 16 different lo-
cations, equally spaced around the white point, while adapted to white. Ellipses show
the discrimination thresholds for a range of colour directions from the central reference
point in each ellipse. The coloured circles (not in original figure) illustrate the approxi-
mate colour of the reference stimulus in that region of the colour space.

In a variation on a conventional chromatic discrimination task, Zaidi & Halevy (1993) used

stimuli that varied smoothly in colour over time, rotating in the equiluminant plane about

the white point. They tested subjects’ ability to discriminate the interval containing a probe

that consisted of an abrupt change (in the otherwise smooth rotation) along one of sixteen

directions in the colour space. In addition to varying the probe direction, Zaidi & Halevy

(1993) varied the phase of the rotation at which the probe was added. Subjects’ discrimina-

tion thresholds were approximately proportional to the cosine of the colour angle between

the two stimuli, regardless of the phase at which the probe was added and the colour direc-

tion of the probe. From a quantitative model of their data, Zaidi & Halevy (1993) argue that

their results cannot be accounted for by a system with mechanisms tuned to just four differ-

ent directions in colour space, providing further evidence for the existence of higher-order

27
 Chapter 3. Higher-Order Colour Processing and Cortical Mechanisms

colour mechanisms that are engaged in chromatic discrimination.

Overall, results from colour discrimination studies can be incorporated into a model of dis-

crimination based on a limited number of chromatic mechanisms. While the number and

tuning of any higher-order mechanisms remains unclear, there is sufficient evidence to reject

a model based on cardinal mechanisms alone. Although not reviewed here, other authors

have investigated discrimination thresholds for stimuli in the L-M isochromatic plane (the

S cone isolating plane), and also found evidence of a limited number of labelled line mech-

anisms (Eskew et al., 2001; Newton & R. T., 2003).

3.1.4 Appearance of Cardinal Colours

The evidence I have presented so far for the existence of higher-order mechanisms is based

on experiments that are aimed at finding ‘lower level’ quantitative evidence from simple

perceptual tasks. However, there are also qualitative observations that point to the exis-

tence of higher-order mechanisms. I do not review these in detail here, but as an example

mention observations relating the opponent mechanisms of the LGN to the phenomenolog-

ically opponent red-green and blue-yellow directions of Hurvich & Jameson (1955).

The physiological opponent processes described above (L-M and S-(L+M)) do not corre-

spond in their spectral sensitivities to the red-green and blue-yellow that are phenomeno-

logically opponent (Hurvich & Jameson, 1957; Krauskopf et al., 1982; Webster & Mollon,

1994). The red-green axis, or blue-yellow equilibrium line of Hurvich & Jameson (1955)

is relatively close to the L-M cardinal axis, but the blue-yellow axis, or red-green equilib-

rium line, does not correspond to the S-(L+M) cardinal axis. The S-(L+M) axis is closer

to ‘violet-lime yellow’ than ‘blue-yellow’, and the blue-yellow axis of Hurvich & Jameson

(1955) is tilted towards the orange-cyan non-cardinal axis. This observation suggests that

the physiological correlates of colour perception are located in visual cortex, and involve

the transformation of the opponent channels that are input from the LGN.

28
 Chapter 3. Higher-Order Colour Processing and Cortical Mechanisms

3.2 Colour constancy

‘Colour constancy’ refers to our ability to recognise the colour of a surface despite changes

in illumination. The distribution of wavelengths that reach our eye is a function of both the

chromatic properties of the illuminant, and the reflectance properties of the surface. In the

simplest situation, for matte, flat surfaces in a clear atmosphere, the reflected light, referred

to as the ‘colour signal’ (Buchsbaum & Gottschalk, 1983), is calculated by a multiplication

of the spectral power distribution (SPD) and the surface reflectance function (SRF) at each

wavelength, as illustrated in Figure 3.2.

Phenomenologically, the separation of the colour signal into surface and illuminant prop-

erties is immediate: Helmholtz (1866) notes that we have what he describes as an illusory

sensation of being able to separate a scene perceptually into layers, saying ‘we often imagine

we see the two colors simultaneously at the same place, one through the other, as it were.

The effect is very much as if objects were seen through a colored veil or mirrored in a colored

surface.’ 1 . Helmholtz describes this as an illusion because he recognised that the eye cannot

have direct access to the constituent parts which we perceive as separable. In recent years,

this phenomenon has been tested in its application to simultaneous colour contrast, where

subjects have difficulty matching the colour of a test embedded in a uniform surround due

to the automatic segmentation of the scene into a surface and an illuminant or transparency

(Ekroll et al., 2004; Faul et al., 2008).

The raw colour signal that reaches the eye is coded by the visual system from the differential

response of the three cone types, giving a degraded form of the total information in the raw

colour signal as a function of wavelengths in the visible range, as illustrated in Figure 3.2. By

what means is the output of the cone photoreceptors transformed into an automatic percept

of surface and illuminant properties? It is not theoretically possible for the visual system to

be perfectly colour constant when it does not have access to the precise raw colour signal,

but is more accurate to think of the visual system making an educated guess regarding

surface and illuminant properties. Below I introduce various theories as to how the visual
1
pg 89, von Helmholtz (1866)

29
 Chapter 3. Higher-Order Colour Processing and Cortical Mechanisms

system accomplishes this task.

Figure 3.2: Adapted from Smithson (2005), Fig 1. If the spectral power distribution of the illuminant
as a function of wavelength is denoted I pλq, and the surface reflectance function of a
particular surface is denoted S pλq, then the wavelength composition of the reflected
light, or colour signal Lpλq is given by a multiplication of these two functions at each
wavelength: Lpλq  I pλq  S pλq

3.2.1 How good is our colour constancy ability?

Many studies have demonstrated that observers are able to perform good, although not

perfect, judgements of surface reflectance under different illuminants (Helson, 1938; Judd,

1940; McCann et al., 1976; Hansen et al., 2007b; Granzier et al., 2009). However, this abil-

ity to achieve colour constancy varies considerably with the structure of the scene in which

the surface is located (Kraft & Brainard, 1999). In seeking to understand the neural pro-

cesses underlying colour constancy, it is not necessary that a model can accurately extract

the true illuminant and surface properties. As Brainard & Freeman (1997) note, from a com-

putational perspective, colour constancy can be achieved even when the estimates of the

30
 Chapter 3. Higher-Order Colour Processing and Cortical Mechanisms

physical parameter (surface and illuminant properties) are incorrect, as long as estimates of

surface properties do not vary with the illuminant. Models of how the visual system sepa-

rates surface reflectance from illuminant properties are better tested on human performance

than on how accurately the model predicts the physical parameters. Any reasonable model

must predict both the successes and failures of human colour constancy (Gilchrist et al.,

1999; Brainard et al., 2006).

3.2.2 Low-level models of colour constancy

One of the simplest models of colour constancy is to assume that, across space, the average

reflectance of all surfaces is achromatic: a ‘grey world’. If this assumption is true, then the

chromaticity of the illuminant corresponds to the mean cone coordinates of the scene, as

proposed in Land’s retinex model (Land & McCann, 1971; Land, 1986). Buchsbaum (1980)

was the first to propose explicitly that a grey world assumption might be used by the visual

system, a proposal included in many models since (see Hurlbert, 1986 and Ebner, 2007 for

reviews of such models).

Kraft & Brainard (1999) tested three classic hypotheses regarding colour constancy; that it

is mediated by local adaptation, by adaptation to the spatial mean, or by adaptation to the

most intense image region, and find that all three of these simple mechanisms are ruled

out by their results. While retinal mechanisms, such as multiplicative scaling (Ives, 1912;

Land & McCann, 1971; Shapley & Enroth-Cugell, 1984; Brainard & Wandell, 1992; Foster

& Nascimento, 1994), can account for a large proportion of our colour constancy ability,

they cannot account for all of it (Brill & West, 1986; Worthey & Brill, 1986), particularly

when the average chromaticity of the surfaces in the scene varies from grey. While retinex

theory predicts colour constancy well for flat, matte scenes, it fails for more complex scenes

(Smithson, 2005).

31
 Chapter 3. Higher-Order Colour Processing and Cortical Mechanisms

3.2.3 Higher-level cues to illuminant chromaticity

Simple models of colour constancy break down particularly for complex scenes, with mul-

tiple illuminants, three dimensional structure and glossy surfaces, where these extra cues

provide additional information that appears to be utilised by the visual system. For a re-

view of models of colour constancy incorporating the cues discussed below, and others, see

Maloney (1999).

Specular highlights

For all surfaces, the reflected light can be separated into two physically different categories:

firstly the diffuse component, light that is reflected following scattering and absorption of

light by molecules beneath the surface, secondly the specular component, light which is

reflected at the interface between the air and the surface (D’Zmura & Lennie, 1986; Lee,

1986; Lu et al., 2000).

The relative contribution of these two components depends on the gloss of the surface, and

on geometric arrangement. In particular, specular highlights, where the reflected light is

dominated by the specular component, will occur where the surface normal bisects the di-

rections of incidence and reflectance (Phong, 1975; Lu et al., 2000). A model of colour con-

stancy that does not incorporate the diffuse and specular components of reflected light will

fail when considering three dimensional scenes, in which the ratio of these two components

varies spatially.

Specular highlights could also be used to improve an estimation of the illuminant, since the

specular component of any reflected light is a scaled version of the spectral power distribu-

tion of the illuminant. As the specular component increases relative to the diffuse compo-

nent, the chromaticity of the reflected light approaches that of the illuminant. If a scene, illu-

minated by a single illuminant, contains two or more objects of different surface reflectance,

then it is theoretically possible to use specular highlights to extract the illuminant chromatic-

ity (D’Zmura & Lennie, 1986; Lee, 1986; Lehmann & Palm, 2001).

32
 Chapter 3. Higher-Order Colour Processing and Cortical Mechanisms

Problems with using only specular highlights to estimate the chromaticity of the illuminant

are raised by D’Zmura & Lennie (1986). Firstly, to estimate the illuminant, such models re-

quire that the visual system has already segmented the scene into different surfaces, group-

ing those of the same reflectance. Yet this task is one of the proposed aims of estimating

the illuminant. Secondly, any mechanism relying on specular highlights cannot explain any

colour constancy found for observers viewing flat or matte scenes.

Scene geometry and diffuse reflections

Light may be reflected from more than one surface in a scene before reaching the eye, which

further complicates the problem which the visual system must solve, but could also provide

additional information regarding the illuminant. There is evidence that inter-reflections are

accounted for when discounting the chromaticity of the illuminant, as demonstrated by a

chromatic Mach Card illusion (Bloj et al., 1999).

Investigations of perceived lightness (the achromatic analogue of colour constancy) also

show the impact of geometric arrangement on the perceived illuminant (Gilchrist et al.,

1999; Adelson, 2000).

Correlations between colour and lightness

Golz & MacLeod (2002) proposed that information regarding the illuminant can also be

derived from correlations between brightness and chromaticity in a scene. They use the

example of disambiguating a white room under a reddish illuminant from a red room under

a white illuminant. While the two rooms could have the same mean chromaticity, in the

white room the luminance and redness of surfaces will be correlated to a greater extent than

in the red room. They argue that using correlations between luminance and chromaticity

could theoretically allow the red and white rooms in this example to be disambiguated,

even where they have the same mean redness.

33
 Chapter 3. Higher-Order Colour Processing and Cortical Mechanisms

Object and illuminant memory

Prior knowledge about typical illuminant properties may be used to help the visual sys-

tem arrive at a ‘best guess’ of illuminant properties. In describing how prior knowledge

might be used, models have been proposed that use a small number of linear basis spectra

to approximate the spectral power distribution of the illuminant and the reflectance func-

tions of surfaces in a scene (D’Zmura & Lennie, 1986; Maloney & Wandell, 1986; Brainard &

Freeman, 1997; Brainard et al., 2006). These models are motivated by the fact that for most

illuminants and surfaces, the majority of variance in their spectral power distributions and

reflectance functions can be accounted for with a small number of basis functions (Vrhel

et al., 1994; Maloney, 1999).

Brainard & Freeman (1997) combine this simplified representation of the illuminant and sur-

face reflectance with a prior distribution of expected illuminants and surfaces to generate a

Bayesian model of colour constancy. When tested against a range of psychophysical results,

this model was shown to predict both successes and failures of colour constancy (Brainard

et al., 2006). However, as the authors indicate, the model does not prescribe a neural imple-

mentation of these calculations.

Knowledge regarding the typical colour of certain objects may also be used to inform our

judgements (Bruner et al., 1951). Hansen et al. (2006) demonstrated that an achromatic ba-

nana was perceived as slightly yellowish, suggesting an automatic assignment of colour to

familiar objects. Object colour may be of particular relevance to the visual system in re-

gard to object identity (Hurlbert, 1996), despite the fact that colour varies with illumination.

Colour does not vary with viewpoint to the same degree as shape (Hurlbert, 1996), making

it a relatively reliable cue to object identity.

3.2.4 Neural Implementations

Models of colour constancy are frequently motivated by considering what function colour

contexts serve and how a system designed for this function would optimally be arranged,

as opposed to the approach of starting with the response properties of the neural networks

34
 Chapter 3. Higher-Order Colour Processing and Cortical Mechanisms

(Brainard et al., 2006). As in the example above, such models can be evaluated by compar-

ing model predictions with measurements of perception, while remaining uninformative

regarding the neural mechanism that could underlie it.

Lennie (1999) argues that most computational models of colour constancy imply mecha-

nisms existing the in the cortex, since they propose computations that depend on capabili-

ties that could only arise in cortex. In macaque, V4 has been suggested to be of particular

importance for colour constancy (Wild et al., 1985; Walsh et al., 1993), along with area IT

(Komatsu, 1998). Furthermore, in cynomolgus and rhesus monkeys, Kusunoki et al. (2006)

show that changing the simulated illuminant of a scene induced changes in the spectral tun-

ing curves of V4 cells, which they argue shows the cells responding in a colour constant

manner.

3.3 Are there specialised colour centres within cortex?

Evidence for higher-order colour mechanisms, that combine information from the subcor-

tical chromatically opponent channels, along with evidence for high-level colour process-

ing such as colour constancy, points to the existence of cortical mechanisms that perform

these transformations of chromatic information. It is unclear whether this information is

processed in a spatially distributed manner, by many cortical areas, or whether there exist

specific modules of colour processing. There is debate regarding the relative contribution of

different cortical areas to the processing of chromatic information, and whether it is appro-

priate that these areas are labelled specialised ‘colour centres’.

Zeki and colleagues argue that cortical area V4 in monkey (Zeki, 1973, 1977), and human

cortex (Lueck et al., 1989; McKeefry et al., 1997) is specialised for colour processing. Others

have suggested that, compared with other cortical areas, V4 contains no greater proportion

of chromatically selective cells (Schein et al., 1982) and that the tuning curves of cells in

V4 are no more sharply tuned as a function of wavelength (de Monasterio & Schein, 1982;

Schein & Desimone, 1990). Gegenfurtner (2003) reviews evidence for colour centres in cor-

tex, focussing on electrophysiological studies and evidence from cerebral achromatopsia,

35
 Chapter 3. Higher-Order Colour Processing and Cortical Mechanisms

and argues against the notion that specialised colour centres exist.

Other areas that have been suggested to be particularly responsive to colour include a region

anterior to V4 termed V4α (Lueck et al., 1989; Zeki & Bartels, 1999; Murphey et al., 2008),

and ventral areas surrounding V4, including V8 (Hadjikhani et al., 1998) and VO (Wade

et al., 2002; Liu & Wandell, 2005; Brewer et al., 2005; Mullen et al., 2007; Jiang et al., 2007).

The evidence for each of these areas being specialised for colour processing is covered in

greater detail in the introduction to Chapter 5, with a particular emphasis on evidence for

the functional role of area V4, the most extensively studied of the areas proposed to be

specialised for the processing of colour.

36
Chapter 4

Using Functional Neuroimaging to

Understand Cortical Colour
Processing

Historically, colour vision research has a strong foundation in both psychophysical and

physiological research, as introduced in Section 2.2. More recently, neuroimaging techniques

have been used to supplement these methodologies and further advance our understand-

ing of corticial mechanisms underlying colour perception. One of the newest and most

promising imaging techniques is Magnetic Resonance Imaging (MRI), suggested to be the

most important advance in biomedical imaging since the introduction of X-rays (Logothetis,

2008). As an introduction to this imaging technique, which is used in two of the three ex-

periments documented in this thesis, in this chapter I present a brief overview of MRI and

functional MRI (fMRI), and compare it with other techniques.

4.1 Introduction to MRI physics

Magnetic Resonance Imaging (MRI) uses a strong static magnetic field combined with tran-

sient magnetic fields to induce alignment in the electromagnetic spin of protons, and to

37
 Chapter 4. Using Functional Neuroimaging to Understand Cortical Colour Processing

measure the rate of relaxation from this aligned state. The physics underlying the transfor-

mation of magnetic field variations into high resolution images is not covered in detail here,

but can be found in McRobbie et al. (2007c) and Lipton (2008). Below is an outline of the

basic scan parameters, a description of T 1 and T 2 , two different processes that can both be

measured using MRI, and of spin-echo and gradient-echo signal formation.

4.1.1 Sources of MR signals: T 1 and T 2

The static magnetic field (B0 ) aligns a fraction of the protons in the body to the z-axis (head-

feet axis). In addition to the static magnetic field, gradient coils are used to generate spa-

tially linear variations in the static field strength which are used to localise the MR signal,

described in detail by McRobbie et al. (2007d). Finally, radiofrequency coils are used to gen-

erate alternating magnetic fields. Radiofrequency coils can also be used as receiver coils

and detect the MR signal that is used to generate the MR images; alternatively the receiver

coil can be separate from the coil used to generate the pulse sequence (McRobbie et al.,

2007b).

During a pulse sequence, the magnetic field induced by the radiofrequency coil selectively

resonates with some protons to shift their alignment from B0 and induce phase coherence

in the spin of these protons. The angle between the initial and the new alignment is termed

the flip angle (FA). Once the radiofrequency pulse ceases, the protons begin to relax. This

relaxation has two main features, a dephasing of the spin of protons, and a realignment with

B0 , as they lose the energy from the pulse (McRobbie et al., 2007a). T 1 , the longitudinal or

spin-lattice relaxation time, is a measure of the time taken for these protons to realign with

B0 , and is a relatively slow decay, taking in the order of seconds, and increasing with the

strength of the static magnetic field (McRobbie et al., 2007a). T 2 is the transverse relaxation

time, or dephasing of the spin of protons. This dephasing is a random, exponential decay,

partly caused by energy exchange between nearby protons and partly by protons changing

one-anothers’ local magnetic field strength, which in turn induces a change in their preces-

sional frequency (McRobbie et al., 2007a). T 2 is a faster decay process than T 1 , taking only a

few hundred milliseconds, and it does not vary with static magnetic field strength. Both T 1

38
 Chapter 4. Using Functional Neuroimaging to Understand Cortical Colour Processing

and T 2 vary in time with different tissues, forming the basis of the signal used to construct

MR images.

4.1.2 Spin-echo sequences, gradient-echo sequences and T 2

MRI can be used to generate high resolution anatomical images, which take a minimum of a

few minutes to acquire. The time taken to acquire each image increases with the resolution

of the image being acquired and the size of the area being scanned. In functional MRI, a

lower spatial resolution image is acquired more frequently (for example, every 3 seconds),

and is used to measure changes over time. The ‘time to repetition’ (TR) specifies the time

between one image acquisition commencing and the next commencing. Scan parameters are

used to select a signal of interest which changes over time. The BOLD (Blood Oxygenation

Level Dependent) response is one such signal of interest, and is considered in greater detail

below.

Spin-echo and gradient-echo are two approaches to generating the MR signal, differing in

the pulse sequences used. Spin-echo sequences are suited to the generation of anatomical

images and are used to produce T 1 and T 2 weighted images (Pinus & Mohamed, 2006).

Both sequences start with a radiofrequency excitation pulse, typically of 900 for spin-echo

sequences and generally smaller angles in gradient-echo (McRobbie et al., 2007a). In a spin-

echo sequence, the spins begin to dephase after the initial pulse for a specified time, then a

1800 pulse is applied, reversing the direction of the spins and causing them to start to move

back into phase alignment. After the same time has elapsed, the phases of all spins will come

back into phase, creating an ‘echo’ signal which is measured by the receiver coil. The ‘time

to echo’ (TE) is the time from the initial pulse until the echo signal. The echo magnitude

depends on T 2 and diffusion of protons out of the scanned area of tissue, but the signal is

dominated by T 2 (McRobbie et al., 2007a).

Gradient-echo sequences use radiofrequency pulses in conjunction with positive and nega-

tive gradients of magnetization that are added to the main static field (B0 ). Inhomogeneities

in the static magnetic field induced by the gradient alter the rate of transverse relaxation,

39
 Chapter 4. Using Functional Neuroimaging to Understand Cortical Colour Processing

but not longitudinal relaxation. The gradient-echo sequence uses a similar principle to the

spin-echo sequence, but the gradient changes the proton’s rate of precession and hence the

rate at which protons dephase and rephase. The initial radiofrequency pulse is followed

immediately by a negative gradient, which is followed by a positive gradient of equal time.

The positive gradient cancels out the dephasing caused by the negative gradient, and the re-

maining dephasing is due to changes in T 2 (McRobbie et al., 2007a). T 2 is used to denote the

apparent transverse relaxation time, caused by the combined effects of T 2 and the magnetic

field inhomogeneities (McRobbie et al., 2007a). Gradient-echo sequences are more likely to

be used to create functional image sequences, creating T 2 and T 2 weighted images (Pinus &

Mohamed, 2006).

4.2 The Blood Oxygenation Level Dependent (BOLD) signal

The Blood Oxygenation Level Dependent (BOLD) signal is used extensively to investigate

neural processing, but it is an indirect measure of neural activity. The usefulness of the

BOLD signal for measuring neural activity relies on the tight coupling between neuronal

activity, blood flow and blood oxygenation, and the spatial specificity of these effects.

4.2.1 Spatial and temporal resolution: theoretical and practical limits

Temporally, the haemodynamic response measured in fMRI is slow relative to the spiking

activity of neurons. Following a 100ms flash of light, the blood flow response in V1 may

start to increase after two seconds, peak between six and eight seconds, and return to base-

line sixteen to twenty seconds after the flash (Constable, 2006). The haemodynamic delay

also varies slightly with different layers of cortex (Jin & Kim, 2008), which would be of rele-

vance to high spatial resolution studies. Despite being slow, the BOLD signal is potentially

precise: Ogawa et al. (2000) combined fMRI with electroencephalographic (EEG) measure-

ments and demonstrated that the fMRI signal had a temporal precision in the order of tens

of milliseconds.

40
 Chapter 4. Using Functional Neuroimaging to Understand Cortical Colour Processing

The spatial resolution of an MRI scan increases with magnetic field strength, and can be

optimised with selection of appropriate pulse sequences (Logothetis, 2008). When all other

imaging parameters are held constant, signal increases in direct proportion to voxel volume,

while noise remains approximately constant, meaning that higher spatial resolution (with

smaller voxel size) reduces the signal-to-noise ratio for each voxel (Constable, 2006). At

ultra-high resolution, with in-plane resolution of just tens of micrometers, the voxel size

is such that blurring caused by diffusion of molecules during the T 2 decay time becomes

noticeable (Constable, 2006).

Physiologically, the spatial resolution of fMRI is limited by the scale of the capillary net-

works; the BOLD signal cannot be more finely tuned than the spatial specificity of the

haemodynamic response. By finding the distance between capillary flow control valves and

the start of venous drainage in chinchilla auditory cortex, Harrison et al. (2002) estimated

the theoretical limit of voxel size to be 100-150µm, although they note that primary visual

cortex may have finer vascular control, of less than 100µm, based on the width of orientation

columns. Practically, the maximum spatial resolution of fMRI is also limited by physiologi-

cal noise, primarily from pulsatile flow of blood through the brain and from signal changes

with the respiratory cycle (Constable, 2006).

In fMRI studies in vision research the typical spatial resolution is much coarser than these

theoretical limits. Using the 300 top-cited papers, Logothetis (2008) calculated the average

voxel size to be 55mm3 , which when unfiltered would contain approximately 5.5 million

neurons. Even with advances in MRI technology and methodology, the spatial resolution of

fMRI cannot reach the same scale as electrophysiological recordings. The relatively coarse

spatial scale of fMRI makes it suited to addressing questions of large populations of neurons

and spatially diffuse neuromodulatory effects such as attention (Logothetis, 2008).

4.2.2 What does the BOLD signal correlate with?

Roy & Sherrington (1890) were the first to suggest that regional cerebral blood flow could

be used to infer that neuronal activity was taking place in that location. It is now known

41
 Chapter 4. Using Functional Neuroimaging to Understand Cortical Colour Processing

that neuronal electrical and synaptic activity triggers increases in cerebral blood flow, cere-

bral blood volume, and the cerebral metabolic rates of oxygen and glucose consumption

(Kim & Bandettini, 2006). PET (positron emission tomography) measurements revealed that

cerebral blood flow increases at a greater rate than the cerebral metabolic rate of oxygen

consumption, leading to increased capillary and venous oxygenation with neural activity

(Fox & Raichle, 1986; Fox et al., 1988). This mismatch showed that increased blood oxy-

genation could potentially be used as a measure of increased neural activity. The BOLD

signal measures changes in the amount of deoxygenated haemoglobin (dHb) in the blood,

and is based on the fact that deoxygenated haemoglobin is paramagnetic, while oxygenated

haemoglobin is diamagnetic. When oxygenated haemoglobin loses its oxygen, this induces

a difference in the magnetic susceptibility between the blood vessel and its surrounding tis-

sue (Ogawa et al., 1990a). In 1990, it was shown in rat brain in a 7T scanner that functional

imaging using BOLD contrast was feasible (Ogawa & Lee, 1990; Ogawa et al., 1990a,b), and

human studies followed soon after (see review by Kim & Bandettini, 2006).

Since these first studies which demonstrated that it was feasible to measure the BOLD re-

sponse using fMRI, this has become a mainstream technique in research on perception, cog-

nition and other areas. Typically the BOLD signal is used to derive and test hypotheses

relating to neuronal firing, but it is not yet clear what the haemodynamic signal correlates

with in terms of neuronal activity. In an early modelling approach, Boynton et al. (1996)

tested the hypothesis that fMRI BOLD responses are proportional to local average neu-

ral activity averaged across a period of time. They measured the BOLD signal in human

V1 while subjects viewed high contrast flickering checkerboards, and tested whether the

BOLD signal changed with stimulus contrast and duration in a manner consistent with the

known pattern of firing of V1 neurons. Boynton et al. (1996) concluded that their results

were well described by a model of the BOLD response with a linear combination of the neu-

ral response, the haemodynamic transformation, and a constant, additive noise component.

However, the fact that the BOLD response correlates with expected neural activity in this

case does not show that the BOLD signal is directly related to neuronal spiking.

In 2008, Logothetis reviewed what is known of the origins of the haemodynamic signal, and

42
 Chapter 4. Using Functional Neuroimaging to Understand Cortical Colour Processing

the implications of this for the interpretation of the BOLD signal in functional neuroimaging.

Logothetis (2008) concluded that the indirect nature of the BOLD signal as a measure of

neural activity was a greater limit on the technique than current MRI technology, and that

the increasing usefulness of fMRI depends on a better understanding of the BOLD signal to

a greater extent than on potential advances in the physics and engineering underlying MRI

methodology.

Since single- and multiple-unit electrophysiology have been used extensively to investigate

neural functioning, it is of particular interest to understand how the BOLD signal relates

to the spiking patterns of neurons. In studies of anaesthetised and awake monkeys, Logo-

thetis et al. (2001) and Goense & Logothetis (2008) measured the BOLD signal, local field

potentials, and the spiking of neurons simultaneously. They found that while the BOLD sig-

nal correlates with the spiking rate of single and multiple neurons, local field potentials are

better quantitative predictors of the BOLD response. In awake behaving monkeys, Goense

& Logothetis (2008) found that while multiple unit responses adapted to the visual stim-

ulus, both local field potentials and the BOLD signal remained unchanged. Similarly, in

anaesthetised cat primary visual cortex, Viswanathan & Freeman (2007) found that changes

in tissue oxygen concentration correlated with changes in local field potential to a greater

extent than with spiking rate.

Local field potentials are a component of the extracellular field potential measured in electro-

physiology, and can be differentiated from neuronal spiking activity by separating the sig-

nal into frequency bands, with local field potentials taken as those below 300Hz (Logothetis,

2008). Local field potentials reflect the perisynaptic activity of the neural population, includ-

ing post-synaptic activity, slow changes in the overall activity of the population, and spike

afterpotentials (Logothetis, 2008). This means that local field potentials are biased towards

the synchronised dendrosomatic components of the input signals to neurons, rather than

output (neuronal spiking) (Goense & Logothetis, 2008; Logothetis, 2003), although presy-

naptic input and spiking are typically correlated with one another (Goense & Logothetis,

2008).

Relative to spiking rates, local field potentials are affected by neuromodulatory processes to

43
 Chapter 4. Using Functional Neuroimaging to Understand Cortical Colour Processing

a greater extent (Goense & Logothetis, 2008; Logothetis, 2008). Goense & Logothetis (2008)

found that local field potentials in the nMod range (20-60 Hz) were the most predictive of

the BOLD signal. Neuromodulatory input to an area, including the influences of attention,

motivation, arousal and memory, have been shown to correlate well with local field poten-

tials in the nMod range (Belitski et al., 2008), leading Goense & Logothetis (2008) to conclude

that the BOLD signal is particularly driven by these neuromodulatory effects.

In addition to the component of the BOLD signal that correlates with local field potentials,

Sirotin & Das (2009) recently proposed another component of the BOLD signal that does not

correlate with neuronal spiking or local field potentials, but with task-related expectation

of a visual stimulus. They used optical imaging to measure blood oxygenation and volume

in V1 of awake, behaving macaque, while the animal performed a task in which there were

trials with and without visual stimuli. While local field potentials and multiple unit activity

correlated well with blood oxygenation and volume in the stimulus-present trials, there was

an ‘anticipatory’ increase in blood oxygenation and volume in the stimulus-absent trials

that was not correlated with local field potentials or multiple unit activity. This finding

suggests that non-visually evoked BOLD responses in visual cortex are less closely related

to neuronal activity, and highlights the fact that the nature of the BOLD signal is yet to be

comprehensively described.

In summary, of the neuronal signals measured electrophysiologically, the BOLD signal is

most directly related to local field potentials. Since local field potentials are typically corre-

lated with neuronal firing, the BOLD response can be used as an indirect measure of neu-

ronal output. In addition to being an indirect measure of neuronal spiking, the BOLD signal

is most likely modulated to a greater extent than the spike rate by attention and other neu-

romodulatory effects.

4.3 Experimental design in neuroimaging experiments

Utilisation of fMRI as a research tool requires the spatial and temporal resolution of fMRI,

and the physiological basis of the BOLD signal, to be taken into account in experimental

44
 Chapter 4. Using Functional Neuroimaging to Understand Cortical Colour Processing

design and analysis. Two broad categories of experimental design are event-related and

block designs.

In a block design, ‘blocks’ of the stimulus that are much longer than the haemodynamic lag

(for example, 15 or 30 seconds in duration) are presented, typically in a counterbalanced

order. Averaging the BOLD response across multiple blocks enhances signal strength, and

the counterbalanced order of presentation ensures that if the correction for haemodynamic

lag is not accurate, the other block types are equally represented in the stimuli preceding

and following each of the blocks included in the average.

In some cases, a blocked design is not possible, for example where the stimuli cannot be pre-

sented for an extended period, or where there is reason to believe the response to different

blocks may vary over time, as in cognitive experiments where task strategy may change over

time. In such cases, an event-related design can be used, where the underlying response at

a precise point in time is separated from the haemodynamic lag. As such, event-related

designs require a more precise model of the haemodynamic response than block designs,

or they require a ‘no stimulus’ case so that the response to the stimulus can be isolated by

comparing the response with and without the stimulus of interest.

4.3.1 Retinotopic Mapping

An area of productive research in visual fMRI is that of characterising the cortical retinotopic

maps of the visual field that are used to define early visual areas. The high spatial resolu-

tion of fMRI (compared with other noninvasive neuroimaging methods), coupled with the

ability to image the entire visual cortex simultaneously, make fMRI particularly suited to

this task. In the last couple of decades, fMRI has been used extensively to investigate the

retinotopic organisation of visual areas in occipital cortex and beyond, in both humans and

non-human primates. Retinotopic mapping of non-human primate cortex using fMRI can

be compared with results from retrograde and anterograde tracing experiments, and also

allows for comparison between non-human primates and humans. One area of debate over

the homology between humans and non-human primates is in the organisation of area V4,

45
 Chapter 4. Using Functional Neuroimaging to Understand Cortical Colour Processing

which is addressed in greater detail in Chapter 5.

In early visual cortex, retinotopic maps of different areas meet at the occipital pole with their

representations of the foveal part of the visual field. Further from this foveal confluence,

neurons have receptive fields that respond to more peripheral eccentricities. Engel et al.

(1994) were the first to map this gradual change in preferred eccentricity in humans using

fMRI. They used an expanding ring stimulus, an annulus containing a flickering checker-

board which gradually expanded in diameter, to a maximum outer diameter of 10 degrees

visual angle. Subjects fixated at the centre of the display and viewed four cycles of the stim-

ulus, each lasting 48 seconds, while fMRI measurements of the BOLD signal were acquired.

By finding the phase of the stimulus cycle to which each voxel responded most, Engel et al.

(1994) mapped the preferred eccentricity of each voxel, and showed that preferred eccen-

tricity increased with distance from the occipital pole.

Sereno et al. (1995) and DeYoe et al. (1996) extended on this work by mapping not only

the preferred eccentricity of each voxel, but also the preferred polar angle. These results

were acquired using an expanding ring stimulus, along with a rotating wedge stimulus in a

separate scan, and finding the preferred phase of each stimulus for each voxel. Combining

these maps of eccentricity and polar angle preference, they defined areas V1, V2, V3 and V4

in human visual cortex. These studies demonstrated the feasibility of using fMRI to derive

retinotopic maps.

The organisation of areas V1, V2, V3 and V3A are generally accepted, and are homologous

to the macaque (Sereno et al., 1995; Tootell et al., 1997). V1 has a continuous representation of

the contralateral hemifield, V2 is split into ventral and dorsal components, which represent

the upper and lower contralateral quarterfields respectively, and both share a border with

V1. V3, like V2, is split into two quarterfield representations, which abut the ventral and

dorsal parts of V2. V1, V2 and V3 converge at the main foveal confluence at the occipital

pole. V3A is dorsal to these visual areas, and has a separate foveal representation that is

shared with V3B (Tootell et al., 1997). There is controversy over the organisation of some

other visual areas, but the dominant view, as presented in a review by Wandell et al. (2007),

is illustrated in Figure 4.1, A.

46
 Chapter 4. Using Functional Neuroimaging to Understand Cortical Colour Processing

Figure 4.1: Example retinotopic maps from fMRI of human visual cortex. In A, Figures 1 and 2 from
Larsson & Heeger (2006) are redrawn, showing the retinotopic organisation of occipital
cortex in two example subjects (above) and for the average across subejcts (n  15). Re-
sults from the right and left hemisphere are shown on the left and right of the figure
respectively. In B, Figure 7A from Arcaro et al. (2009) is redrawn, showing the retino-
topic and attentionotopic organisation of the right ventral occipital cortex of one subject.

47
 Chapter 4. Using Functional Neuroimaging to Understand Cortical Colour Processing

According to this dominant viewpoint, lateral to the dorsal part of V3 are the lateral occip-

ital areas, LO1 and LO2, both of which have a complete representation of the contralateral

hemifield (Larsson & Heeger, 2006). After some initial confusion over the definition of V3B

(see discussion in Wandell et al., 2007), it is now conventionally defined as a hemifield rep-

resentation dorsal to V3A, and sharing the foveal representation of V3A, as shown in the

retinotopic maps of (Larsson & Heeger, 2006), included in Figure 4.1. Dorsal of V3B is area

V7 (Press et al., 2001). MT, a motion responsive area lateral to LO2, is typically defined on

the basis of its response to motion vs static stimuli as well as its retinotopic organisation

(Smith et al., 1998; Goebel et al., 1998; Huk et al., 2002).

Area V4 is most commonly thought to consist of a ventral hemifield representation (also

referred to as human V4 or ‘hV4’) bordering the ventral part of V3 (Brewer et al., 2005; Lars-

son & Heeger, 2006; Wandell et al., 2007; Arcaro et al., 2009), but an alternative arrangement

(Hansen et al., 2007a) is detailed in Chapter 5. Beyond hV4, areas VO1 and VO2 each repre-

sent the contralateral hemifield and share a common fovea (Brewer et al., 2005).

Retinotopic mapping of early visual areas is now standard practice in most fMRI studies of

visual perception. Particularly for higher visual areas, these retinotopic maps can be sup-

plemented by ‘attentionotopic’ mapping of visual field coverage, which utilises the strong

attentional modulation of the BOLD signal. Attentionotopic maps are generated using a

similar principle to retinotopic maps. Subjects are asked to make covert shifts of attention

to different places in the visual field, without moving their eyes. This allows modulation

of the BOLD response in visual cortex to be plotted as a function of the area of the visual

field to which attention was directed. Attentionotopic maps can be clearer than retinotopic

maps, particularly for higher areas, presumably since the effects of attention are greater in

these areas.

Silver et al. (2005) used this method to define areas IPS1 and IPS2, located beyond area V7

in parietal cortex. Swisher et al. (2007) later identified these areas using standard retinotopic

techniques, and also found two further maps, IPS3 and IPS4. In ventral occipital cortex,

Arcaro et al. (2009) combined retinotopic and attentionotopic mapping to define two new

areas in the parahippocampal cortex, PHC1 and PHC2. These maps, anterior to VO2, each

48
 Chapter 4. Using Functional Neuroimaging to Understand Cortical Colour Processing

contain a hemifield representation of the contralateral visual field. Arcaro et al. (2009) tested

whether the maps obtained from attentionotopic and retinotopic mapping were similar, and

found good qualitative and quantitative agreement between the two mapping procedures,

as illustrated in Figure 4.1, B.

Retinotopic maps in the lateral geniculate nucleus (LGN) have also been revealed (Schneider

et al., 2004), although its small size and location deep in the brain make it difficult to derive

maps in this area. Other retinotopic maps in parietal and frontal cortex have been described

(Wandell et al., 2007; Hagler & Sereno, 2006), but are not reviewed here.

49
Chapter 5

Cortical response to colour vs

achromatic stimuli and the V4d
debate

5.1 Introduction

Much of our current understanding of the human visual system has its foundation in animal

models, but more recently functional MRI has facilitated comparison of visual processing

in humans and non-human primates. One area of comparison is the retinotopic organisa-

tion of early visual cortex. On the whole there is close homology between humans and

non-human primates in the organisation of their visual areas (Sereno et al., 1995), but one

point of suggested difference is in area V4. Unlike macaque, where V4 is split into ventral

and dorsal components that together represent a hemifield, in humans it is most commonly

reported that V4 has an unbroken ventral hemifield representation, without a dorsal com-

ponent (Brewer et al., 2005; Larsson & Heeger, 2006). An alternative view is presented by

Hansen et al. (2007a), who argue that current evidence for the ventral hemifield model of

human V4 is not sufficiently strong to reject a model homologous to the organisation found

in macaque.

50
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

5.1.1 Organisation of V4 in non-human primates

The retinotopic organisation of visual areas in non-human primates has been investigated

primarily by mapping visual field coverage using functional and anatomical connections

between visual areas (Felleman & van Essen, 1991), although more recently fMRI methods

have also been applied (Brewer et al., 2002; Fize et al., 2003). In macaque, the location and

visual field representation of areas V1, V2 and MT are the most clearly defined (van Essen

& Zeki, 1978; Felleman & van Essen, 1991; Lyon & Kaas, 2002; Rosa & Tweedale, 2005).

Adjoining V2, area V3 is most likely another hemifield representation, mirror reversed to

V2 (Lyon & Kaas, 2002; Brewer et al., 2002; Fize et al., 2003), although it has previously been

proposed that the ventral and dorsal parts may be separate areas (referred to as VP and V3,

respectively), and that the dorsal part may represent an entire visual hemifield (area DM,

see Lyon & Kaas, 2002, for summary).

Partly due to uncertainty of its borders with V3, and also due to the larger receptive field

size of neurons there (for example, van Essen & Zeki, 1978), the definition of area V4 is dis-

puted. Currently, area V4 in macaque is usually defined as a split hemifield representation,

made up of dorsal and ventral quarterfields, in a similar manner to areas V2 and V3. The

main uncertainty concerns the anterior borders of V4 (Fize et al., 2003). The quarterfields of

V4v and V4d share a border with V3 at a vertical meridian representation, but it is unclear

whether the anterior border of V4d reaches a horizontal meridian, or whether the dorsal

component of V4 represents just under a quarterfield. According to the first view (Brewer

et al., 2002; Pigarev et al., 2002), both V4v and V4d represent a quarterfield, while according

to the second view (Gattass et al., 1988; Fize et al., 2003), V4d does not represent the hori-

zontal meridian and V4v represents the remainder of the visual field, including the upper

quarterfield and slightly below the horizontal meridian.

In an fMRI study Brewer et al. (2002) used travelling wave stimuli, standard for mapping

retinotopy in humans, to investigate the organisation of visual areas in macaque. In agree-

ment with the electrophysiological work, they report that V4 is a split hemifield with ventral

and dorsal quarterfields. They also quantitatively analysed the visual field coverage and

51
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

found an asymmetry in the eccentricity maps of V4: the central visual field falls more in the

ventral than the dorsal region.

Soon after its identification, area V4 was reported to have a strong preference for colour

(Zeki, 1973; van Essen & Zeki, 1978), which led to the suggestion that area V4 may be a

‘colour centre’ in early visual cortex. Others have argued against this idea (Lennie, 1999),

citing evidence that the spectral sensitivity of V4 cells is no narrower than that of earlier

areas (de Monasterio & Schein, 1982; Desimone et al., 1985). However, it may be that V4 is

not specialised for discriminating different wavelengths of light (Heywood et al., 1995), but

is involved in higher order processing of chromatic information (Schiller & Logothetis, 1990;

Walsh et al., 1992, 1993) or colour constancy (Wild et al., 1985; Kusunoki et al., 2006).

5.1.2 Controversy over the organisation of human V4

The functional role and location of human area V4 remains controversial. Following reports

of strong colour responses in rhesus monkey visual cortex (Zeki, 1973), earlier reports of

specific deficits in colour vision following cortical lesions were reinterpreted as potential

evidence for a colour processing area in human visual cortex (for a review, see Zeki, 1990).

In normal human cortex, a possible homologue of macaque V4 was first reported by Lueck

et al. (1989) and later by Zeki et al. (1991), who used PET to show a colour responsive area

in the region of the lingual and fusiform gyri.

With the advent of high resolution fMRI, our understanding of retinotopic map organisation

in human visual cortex has advanced considerably (for a review, see Wandell et al., 2007).

Early studies (Sereno et al., 1995; DeYoe et al., 1996; Tootell et al., 1997; Hadjikhani et al.,

1998) found visual field maps, each with full hemifield representation, that were consistent

with human V1, V2 and V3 of the same organisation as found in macaque. However, when

looking for the human homologue of macaque V4, these early studies found only a ventral

quadrant representation, without a clear dorsal counterpart (see earlier review by Wandell,

1999). The ambiguity of the area corresponding to V4d in macaques was due in part to

disagreement over the definition of human V3A; an area that in macaque adjoins V3d along

52
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

with V4d.

McKeefry & Zeki (1997) reported a colour sensitive region in ventral human visual cortex,

without a dorsal component, which they labelled V4. Although they did not use retinotopic

mapping, they showed that this region responds to both the superior and inferior parts of

the visual field, and suggested that an entire visual hemifield may be represented ventrally

in human V4. Later studies using retinotopic mapping supported this claim: Wade et al.

(2002) found a hemifield representation in ventral V4, which they labelled ‘hV4’ (human

V4) to disambiguate it from V4 as defined in macaque. In recent years, many studies have

used retinotopic mapping to define hV4 as a ventral hemifield representation (Wandell et al.,

2005; Brewer et al., 2005; Arcaro et al., 2009; Larsson & Heeger, 2006), although coverage of

the lower vertical meridian in hV4 is typically less clear than in V1-V3 (Larsson & Heeger,

2006; Tyler et al., 2005).

An alternative scheme for the organisation of V4 in humans is that the lower vertical merid-

ian and nearby angles are represented by a dorsal component of V4. Most recently Hansen

et al. (2007a) suggested the V4d model as providing quantitatively better descriptions of vi-

sual field coverage than the hV4 model (but see Winawer et al., 2010, for an account of how

fMRI studies tend to underestimate the visual field coverage of ventral V4). Their model

has a ventral V4, which represents slightly more than a quadrant of the visual field, and a

dorsal V4, abutting V3d, which represents the lower vertical meridian and nearby angles.

Unlike Larsson & Heeger (2006), who argued for the existence of two lateral occipital areas,

each with a hemifield representation, Hansen et al. (2007a) reported the existence of a single

lateral occipital region beyond V4d with no clear retinotopy. The two alternate models of

human V4 and LO are illustrated schematically in Figure 5.1.

5.1.3 Colour responsivity of human V4 and ventral occipital cortex

As with non-human primates, in humans there have been varying reports of whether V4

shows a preference for coloured stimuli, and whether there is a ‘colour centre’ in human

visual cortex. Lesion studies suggest the particular involvement of ventral areas in process-

53
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

Single ventral hemifield hV4 map Ventral & dorsal components of V4

V2d
V3A/B
Eccentricity Maps

V3d V4d
LO1
V1
LO
V2v LO2

V3v hV4 V4v

V2d
V3A/B
Polar Maps

V3d V4d
LO1
V1
LO
V2v LO2

V3v hV4 V4v

Internal Topography
Fractured

Figure 5.1: Two alternate models of human V4. In each of the four panels, the schematic shows the
predicted visual field preference across a flattened map of the right hemisphere of visual
cortex. The upper maps are coloured according to eccentricity preference and the lower
maps show polar preference, with the semicircular key to the lower right indicating the
part of the visual field corresponding to each colour. The model on the left, with a single
ventral hemifield hV4 map, is derived from the definitions of Brewer et al. (2005) and
Larsson & Heeger (2006). The model on the right, where V4 is split into ventral and
dorsal components, is based on the definitions in Hansen et al. (2007a). Those areas
which are common to both models have been whited out and are not labelled on the
right hand side to highlight the points of difference between the models.

54
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

ing colour. Lesions to ventral parts of occipital cortex can result in cortical colour blindness

(achromatopsia) or various colour vision deficits (dyschromatopsias) (Zeki, 1990; Meadows,

1974; Damasio et al., 1980; Gallant et al., 2000; Beauchamp et al., 2000). In a meta-analysis of

92 reported cases of cerebral achromatopsia, Bouvier & Engel (2006) found a ventral region

of lesion overlap, likely including but not limited to ventral V4. Bouvier & Engel (2006)

conclude that these lesion studies provide good evidence of a common region damaged

in achromatopsia that is important for colour vision, but also note that damage to this re-

gion does not always impair colour vision, nor does it exclusively impair colour perception,

tending to cause a range of visual deficits.

Early fMRI studies (without retinotopic mapping of visual areas), while searching for a hu-

man colour centre, reported a strong colour preference in ventral areas that most likely in-

clude ventral V4 and a more anterior area labelled V4α (Beauchamp et al., 1999; Bartels &

Zeki, 2000). Hadjikhani et al. (1998) and Tootell & Hadjikhani (2001) report that V8 but not

V4 responds to colour afterimages, although the area they label V8 would be included in V4

(at least in part) according to the ventral hemifield model of V4. In a classifier study on the

binding of motion and colour, Seymour et al. (2009) report that while classifiers trained on

data from V1, V2, V3, V3A and MT+ were biased towards learning the motion rather than

the colour of the stimulus, when trained on data from V4 the classifiers showed a weaker

bias for learning colour rather than motion.

Another area suggested to be a ‘colour centre’ in humans is area VO, which is located adja-

cent to V3v and V4v with a separate foveal representation, and shows a strong response to

coloured stimuli (Wade et al., 2002; Liu & Wandell, 2005; Brewer et al., 2005; Mullen et al.,

2007). Mullen et al. (2007) report that VO but not V4 shows an overall preference for isolu-

minant colours over luminance defined stimuli with the same cone contrast. Liu & Wandell

(2005) compared V1 and VO in their response to colour and other stimulus features, and

showed that while V1 and VO both respond to colour, VO, but not V1, has a temporal fre-

quency tuning similar to colour perception. Jiang et al. (2007) also reported that VO but

not earlier areas showed responsiveness which correlated with perceptual experience when

subjects viewed chromatic flicker.

55
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

Anterior to V4 and VO, in a region that possibly corresponds to V4α, Murphey et al. (2008)

made single-unit recordings in a human and found an ‘anterior colour centre’ whose re-

sponses corresponded to colour perception.

5.1.4 Functional role and the definition of early visual areas

In defining visual areas, it is reasonable to assume that each visual area with retinotopic

organisation should have a complete map of the visual field (Zeki, 2003) and homogenous

functional response properties across the visual field representation. The colour and form

selectivity of V4 neurons, shown in both macaque and in human, suggests that V4 is spe-

cialised for mid-level form vision, including but not limited to the processing of colour (for

example, see Gallant et al., 1993, 1996; David et al., 2006; Kumano et al., 2008; Arcizet et al.,

2009; Mysore et al., 2008; Pasupathy, 2006; Dumoulin & Hess, 2007; Wilkinson et al., 2000;

Vinberg & Grill-Spector, 2008; Thielscher et al., 2008). V4 is also reported to be more greatly

influenced by attention than earlier areas, in both macaque and human (Moran & Desimone,

1985; Haenny & Schiller, 1988; McAdams & Maunsell, 1999; Mehta et al., 2000; Maunsell &

Cook, 2002; Maunsell, 1995; Kastner et al., 1998; Tootell et al., 1998; Schwartz et al., 2005;

Hansen et al., 2007a). Some functional properties of human ventral V4 and its putative dor-

sal component have previously been measured in an attempt to differentiate between the

alternative models of human V4, as outlined below.

Using a model similar to that presented by Hansen et al. (2007a), Tootell & Hadjikhani (2001)

report a human V4v which represents a quadrant of the contralateral hemifield, and define

V4d as the corresponding dorsal area which represents the lower quadrant. However, they

also argue that while these areas are complementary in their visual field coverage, they do

not share common functional selectivity. Specifically, they report that while V4d responds

to kinetically defined contours, V4v does not. They also tested colour selectivity, but found

that neither V4v nor V4d preferred chromatic to luminance defined stimuli; instead, they

found strong chromatic selectivity only in the part of ventral occipital cortex that they term

V8.

56
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

Hansen et al. (2007a) measured the degree of attentional modulation across human visual

cortex to compare the functional properties of human V4v and putative V4d with surround-

ing areas. Subjects were scanned while covertly attending to a series of wedges, and spatial

tuning in some areas tended to shift toward the location of the attended stimulus. Hansen

et al. (2007a) found that these attention-induced shifts in spatial tuning were greater in LO

than in V4v and V4d, and greater in V4v and V4d than in V3. They argue that the interme-

diate magnitude of attentional modulation in V4v and V4d is consistent with these regions

constituting a single visual area.

Wade et al. (2008) compared the response of human and macaque V4 to coloured Mon-

drian stimuli, using fMRI. In macaque, they found similar responses to colour in ventral

and dorsal V4. In humans, they found a strong response to colour in ventral V4, consistent

with earlier reports, and also found some evidence of a response to colour in the area corre-

sponding to V4d as defined by Hansen et al. (2007a). The dorsal colour response was weak,

but there were ‘some hints of response’ in the location corresponding to putative V4d. Wade

et al. (2008) also note it is conceivable that any preference for colour in V4d may have been

overlooked by previous studies because of its smaller size, compared with V4v.

5.1.5 Scope of the current study

Since V4 has been implicated in processing colour, here I tested the colour response of

human occipital cortex using fMRI. I tested for voxels that showed a larger response to

coloured stimuli than to greyscale stimuli of the same luminance. I used complex stimuli

(movie clips), which, unlike many previous investigations of colour selectivity (Lueck et al.,

1989; McKeefry & Zeki, 1997; Beauchamp et al., 1999; Wade et al., 2008), have a wide range

of spatial and temporal frequencies and contain visual objects, including people, while typi-

cally containing a greater variation of image content than standard natural image databases

(Bex et al., 2009). In using these stimuli I hoped to maximise the chance of eliciting any

colour-sensitive response from ventral V4, in order that I might compare this functional

response with its suggested dorsal counterpart. I combined these measures of colour re-

sponsiveness with functionally defined retinotopic maps, acquired separately in high res-

57
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

olution scans. Although previous studies have used naturalistic stimuli to measure colour

response (Bartels & Zeki, 2000, 2004), these earlier studies did not use retinotopic mapping.

By defining the early visual areas retinotopically I can compare the colour responsiveness of

V4 according to the V4v/V4d and ventral hemifield models.

In summary, Hansen et al. (2007a) argue for the dorsal component model of human V4 on

the basis of visual field coverage, homology with macaques, and similarity in attentional

modulation. Here I consider both the visual field coverage of V4 according to the two mod-

els, as well as the functional response properties of the suggested dorsal and ventral parts

of V4 to coloured versus achromatic stimuli.

5.2 Materials and Methods

5.2.1 Subjects

Data were collected on six subjects (three male), aged between 25 and 40 years, with nor-

mal or corrected to normal visual acuity and normal colour vision, as tested using Ishihara

plates (Ishihara, 1990). All subjects provided informed consent, and the entire study was car-

ried out in accordance with guidelines of the University of Sydney Human Research Ethics

Committee.

5.2.2 Chromatic, spatial and temporal stimulus properties

I used retinotopic mapping to define early visual areas, and then tested the responsiveness

of these areas to chromatic over achromatic stimuli. I used standard stimuli to derive maps

of polar and eccentricity preference, and then chromatic and achromatic versions of the

movie Strictly Ballroom to test the responsiveness of these areas to chromatic and achromatic

stimuli.

58
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

Rotating wedge and expanding ring stimuli

For retinotopic mapping I used standard rotating wedge and expanding ring stimuli to find

the preferred polar angle and eccentricity preference of each voxel (Engel et al., 1994; Wan-

dell et al., 2007; Schira et al., 2009). Example frames of the rotating wedge and expanding

ring stimuli are shown in Figure 5.2, (A and B respectively). The rotating wedge stimulus

consisted of a black and white checkerboard in a 450 wedge centred on a fixation marker at

the centre of the screen and with an overall radius of half the height of the screen. The checks

evenly divided the wedge into three smaller wedges, and each of these 150 wedges was fur-

ther divided into 20 ring segments. The wedge rotated in increments of 150 , once every 1.5

seconds. As it rotated, the checks of each 150 segment of the wedge moved smoothly toward

or away from the central fixation marker, with the direction of movement alternating with

each segment. The width of each ring segment increased logarithmically with distance from

the centre, as illustrated in Figure 5.2, A. Subjects each completed four runs of 6.45 min-

utes duration: two where the wedge rotated clockwise, and two where the wedge rotated

counter-clockwise.

The ring stimulus was made up of the same checkerboard pattern as the wedge stimulus,

except that instead of black and white checks the checks were randomly chosen colours, and

the checks did not move, but updated with a new colour at a rate of 8Hz. The ring always

had an annulus of 3 ring segments, and expanded by one ring segment every 1.8 seconds,

cycling through the entire range of ring sizes every 36 seconds. Each subject completed at

least two runs of 6.45 minutes duration.

Both the wedge and ring stimuli were drawn on a background with a fixation cross of thin

grey lines, as shown in Figure 5.2. Subjects were instructed to fixate at the central point

of the stimulus, and in the case of the rotating wedge stimulus were required to perform a

dimming task at fixation. Differences between the wedge and ring stimuli (such as the use

of colour in the ring stimuli) were not theoretically motivated, but were due to the fact that

the eccentricity maps were acquired using an earlier protocol.

59
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

Chromatic vs achromatic stimulus

To compare the response of early visual areas to chromatic vs achromatic stimuli, I generated

chromatic and achromatic versions of a commercial DVD movie. I used Baz Luhrmann’s

Strictly Ballroom (1992, M&A Film Corporation, Australian Film Finance Corporation), and

selected a section of the movie which contained a range of depths, objects, people and

colours. I chose to use a movie instead of a simpler stimulus in order to improve the like-

lihood of evoking a colour sensitive response across neurons which prefer a range of (for

example) spatial and temporal frequencies, and orientations. Example frames from the se-

lected clip are shown in Figure 5.2, C.

The second chapter of Strictly Ballroom was extracted as an uncompressed avi file using

Handbrake software (http://handbrake.fr/). Frames were extracted from the avi file

using Matlab (version 7), and the RGB values of each frame were stored separately. Next,

to create an achromatic version of each frame, I determined the photopic luminance of each

RGB value in the frame, and substituted this RGB with a grey value (R=G=B) of the same

photopic luminance (determined using the Stockman & Sharpe, 2000, 2-degree luminosity

function and calibrated for our display apparatus). Once achromatic versions of each frame

were generated, the frames were recombined into avi files of 2 minutes each. Two versions of

the movie clip were created: both had the same 2 minute continuous segment of the movie,

and alternated between chromatic and achromatic every 15 seconds. For the first version,

the movie started in colour (Colour/BW order), for the second the movie started in black

and white (BW/Colour order).

Experimental scans using the chromatic vs achromatic stimuli were completed during a sin-

gle session for each subject. The session included four functional scans, each lasting 4.5

minutes. During each scan the subject viewed 18 blocks of the experimental stimulus, con-

sisting of 2.25 loops of either the Colour/BW (on odd numbered scans) or the BW/Colour

clip (on even numbered scans). Throughout all experimental scans, subjects were instructed

to maintain fixation on a central point, and performed a dimming task at this location.

60
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

A B

Order 1
(Colour/BW)

Order 2
(BW/Colour)

Figure 5.2: Stimuli used to map visual field coverage (A and B) and colour responsiveness (C).

61
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

5.2.3 Fixation Task

Throughout the wedge and the chromatic vs achromatic scans, subjects performed a dim-

ming task at fixation in order to ensure they maintained fixation. In the centre of the screen,

a small cross (height: 0.2 degrees visual angle) was drawn on a circular grey background.

Throughout each scan, the cross switched between white and black at an average rate of

1Hz, with random jitter of up to 300ms added to the switch times. Subjects responded by

pressing a button at the onset of a black cross and releasing it when the cross switched to

white.

5.2.4 Colour calibration procedures and display system

Stimuli were generated and displayed using Matlab (version 7) software in conjunction with

routines from PsychToolbox 3.0.8 (Brainard, 1997; Pelli, 1997) on a Dell Latitude laptop com-

puter driving an nVidia Quadro NVS 110M graphics card to draw stimuli to a Dell 5100 MP

projector, which projected the images through a Faraday shielded window onto a translu-

cent perspex screen. Subjects, while lying in the scanner, viewed the perspex screen through

a mirror mounted above the head cage which reflected the image from the screen located

behind the scanner. The screen was viewed from a distance of 167cm, and the image sub-

tended an area of 18.7 by 13.3 degrees visual angle. Stimuli were calibrated in situ for the

DLP projector and screen arrangement, using measurements of the screen obtained from

the doorway of the scanning room with a PR-655 SpectraScan spectrophotometer (by Photo

Research Inc.). Scanning took place in a darkened room.

Changes in both chromaticity and luminance of the screen with increasing R, G and B values,

along with interactions between the G+B (C), R+B (M), R+G (Y) and R+G+B (K) channels

were taken into account when generating the experimental stimuli. Modelling the interac-

tion between channels was of particular importance for our DLP projection system which

rendered each RGB input using four channels (R, G, and B channels, along with a white

channel to boost luminance). The CIE (xyY) coordinates were calculated using the Stock-

man & Sharpe (2000) 2-degree cone spectral sensitivities for each of the seven sets of 16

62
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

points measured during calibration, and each of these was interpolated to 255 values us-

ing the best fitting spline. The spline fits were then used to calculate the luminance and

chromaticity for each combination of R, G and B intensity values using the masking model

described in Tamura et al. (2003). These measurements of luminance were used in the pro-

cess of creating an achromatic version of each movie frame, as described above. For a more

detailed description of the calibration process, refer to Appendix A.

5.2.5 fMRI methods

fMRI data were collected using a 3T Philips scanner (St Vincent’s Public Hospital, Sydney,

Australia), with a birdcage head coil.

Anatomical measurements and definition of grey matter

The anatomical image for each subject was generated from the average of three scans. Two

of these were high resolution (1 x 1 x 1mm) structural MR images of each subject’s whole

brain, acquired using a Turbo Field Echo (TFE) protocol for enhanced grey-white contrast.

A third, higher resolution (0.75 x 0.75 x 0.75mm) scan of the caudal half of the head was also

acquired in order to recover more anatomical detail of the occipital lobes.

Using the Statistical Parametric Mapping (SPM) software package SPM5 (described in de-

tail by Frackowiak et al., 1997), anatomical images were each reoriented to approximately

the same space using anterior and posterior commissures as anatomical landmarks. Fine

alignment of these anatomical images was carried out using normalised mutual information

based coregistration, and all of the anatomical images were resampled so that they were in

the same voxel space with a resolution of 0.75 x 0.75 x 0.75mm. From each image I removed

intensity inhomogeneities using a nonparametric inhomogeneity correction method (Man-

jón et al., 2007), and normalised the images such that the white matter had approximately

equal intensity. The coregistered, inhomogeneity corrected, normalised images were then

averaged together to produce a mean anatomical image for each subject.

ITKGray software (Yushkevich et al., 2006) was used to specify the white matter of each sub-

63
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

ject, initially using automatic segmentation tools, then using manual editing. The segmen-

tation image was imported into mrGray, part of the mrVista software package developed

by the Stanford Vision and Imaging Lab (http://white.stanford.edu/software/).

In mrGray, grey matter was ‘grown’ out from the white matter in a sheet with a maximum

thickness of 4 voxels.

Functional measurements

fMRI data were acquired using a T2*-sensitive, FEEPI pulse sequence, with echo time (TE)

of 32ms; time to repetition (TR) of 3000ms; flip angle 900 ; and field of view 192mm x 69mm x

192mm. The standard retinotopy measurements (wedge and ring stimuli) were acquired in

46 slices of thickness 1.5mm, with effective in-plane resolution 1.5mm x 1.5mm. The colour

vs black and white stimuli were acquired in 33 slices of thickness 2mm, with effective in-

plane resolution 2mm x 2mm. Slices were collected in an interleaved, ascending order, in

a coronal plane tilted such that the scan covered the whole of the occipital lobe and the

posterior part of the parietal and temporal lobes. Using SPM5, all functional data were

preprocessed to correct for slice time and head motion before alignment to the structural

data. Data from functional scans were aligned to a whole head anatomical scan acquired in

the same session, using normalised mutual information based coregistration. The functional

data from chromatic vs achromatic scans were then aligned to the data from the wedge and

ring scans (which were acquired in a separate session) by first aligning the within session

anatomical from one to the within session anatomical of the other, then applying the same

transformation to the functional data. Data from the first and last 15 second blocks in the

chromatic vs achromatic scans were excluded from this analysis, and data were labelled

with the stimulus (colour or black and white) occurring 6 seconds previously in order to

approximately compensate for haemodynamic lag.

64
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

Definition of Retinotopic Areas

Visual areas were defined retinotopically using data from the wedge and ring stimuli (de-

scribed above). Data from clockwise-rotating wedge runs were temporally reversed before

averaging with data from the counterclockwise-rotating wedge runs. Average data from

both wedge and ring runs were smoothed with a Gaussian kernel of half width 1.5 mm,

then projected onto a computationally flattened representation of the cortex for each hemi-

sphere of each subject, using mrVista. The resultant polar angle and eccentricity preference

maps for each subject are shown in Figure 5.3.

Retinotopic maps for each subject are shown in Figure 5.3. Areas V1, V2, V3 and V3A/B

were manually defined on the phase and eccentricity maps derived from the wedge and ring

stimuli, using the conventions common to Brewer et al. (2005), Larsson & Heeger (2006) and

Hansen et al. (2007a). According to these definitions the foveal representation at the occipital

pole is shared by V1, V2, and V3, while V3A and V3B, which border the dorsal part of V3,

share a dorsal fovea. For this analysis I did not attempt to separate V3A and V3B.

I defined areas hV4, LO1 and LO2 according to the conventions of Larsson & Heeger (2006),

where hV4 is a hemifield representation of the contralateral visual field sharing a border

with the ventral part of V3 and sharing the foveal representation of V1, V2 and V3. In

Figure 5.3, hV4 is given by the union of V4v and V4v (extra). Areas LO1 and LO2 both

have a hemifield representation, and extend laterally from V3d. In Figure 5.3, LO1 and LO2

comprise the regions in red and light blue, and the boundary between them is drawn as

a black line in the light blue section. LO1 includes both the red region and the light blue

section bordering V4d; LO2 is the more lateral of the light blue sections.

I used the definitions of Hansen et al. (2007a) to define ventral and dorsal V4 (V4v and V4d),

and area LO. According to these conventions, V4v extends laterally from the ventral border

of V3v, where the upper vertical meridian is represented, and shares the foveal representa-

tion of V1, V2 and V3. V4v includes a representation of the contralateral upper visual field,

and the contralateral horizontal meridian, extending slightly into the lower visual field. V4d

represents the remaining part of the contralateral visual field, with a lower vertical meridian

65
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

CA
V1 V4v (extra)
V2 V4d
V3 LO
V3A/B VO
V4v

Preferred
CC Polar Angle

Preferred
Eccentricity

DM

EG

KS

SM

Figure 5.3: See next page for caption.

66
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

Figure 5.3: (See previous page for Figure). Polar angle (left) and eccentricity (right) maps for the
right hemisphere of each of the six subjects. Key to the top left of the polar angle maps
indicates the location of the early visual areas (for details of definition, see text). Cortical
regions outside the retinotopically defined areas are shown in lower contrast.

representation along its border with V3d. V4d also borders V3A/B. According to Hansen

et al. (2007a), LO shares a border with V4d and extends laterally, but is not clearly defined

on the basis of its retinotopic organisation. Here I defined LO as the union of LO1 and LO2

(as defined above), minus the area defined as V4d; that is, the entire light blue section in the

maps in Figure 5.3.

For areas VO1 and VO2 (not shown in Figure 5.1, but drawn on maps in Figure 5.3) I fol-

lowed the conventions of Brewer et al. (2005), defining VO1 as a hemifield representation

(where it existed), extending ventrally from V4v, and VO2 as a hemifield representation ex-

tending ventrally from VO1. VO1 and VO2 share a foveal representation that is separate

from the main foveal confluence at the occipital pole.

Assessing the chromatic vs achromatic responsiveness of each voxel

I used the response of each voxel during scans where subjects were viewing clips from the

film Strictly Ballroom to assess colour responsiveness. Averaged data from the colour and

black and white blocks were smoothed with a Gaussian kernel of half width 1.5 mm, and

the mrVista corAnal tool was used to find, for each voxel, the coherence and phase of the

harmonic of its response corresponding to an alternation with stimulus colour (with a period

of 30 seconds). Voxels were thresholded according to the coherence of the harmonic for each

subject in relation to the average coherence across all visual areas; only voxels for which the

coherence exceeded one standard deviation above the mean were included. Those voxels

which met this criterion were then partitioned into ‘chromatic preferring’ and ‘achromatic

preferring’ according to the phase of the harmonic, which corresponded to whether their

response was greater in the colour or achromatic blocks of the stimulus.

67
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

5.3 Results

I defined the early visual areas of each subject on the basis of polar angle and eccentricity

preference, then found the responsiveness across the early visual areas to chromatic vs

achromatic stimuli. The maps of polar angle and eccentricity preference that were used to

retinotopically define visual areas for each subject are shown in Figure 5.3. The criteria used

to define each area retinotopically are described in the Methods section above.

5.3.1 Generally increased responsiveness to colour over achromatic stimulus

blocks

Figure 5.4 shows all voxels for which there was a response modulation with stimulus colour

changes; that is, which showed a coherent response modulation with a period of 30 seconds,

corresponding to two blocks from the stimulus (see Methods for more details).

For most subjects, there were very few voxels that showed an increased responsiveness

to greyscale over colour stimuli (shown in blue in Figure 5.3). Since the colour stimuli in-

cluded the same luminance contrast as the achromatic stimuli, the lack of voxels showing

an increased responsiveness to achromatic over colour stimuli is unsurprising; voxels which

responded only to luminance variation should show the same response to both types of

stimulus rather than an increased response to the achromatic blocks.

5.3.2 Colour response of area V4 and its putative dorsal component

The response of the ventral part of V4 (V4v) showed a modulation with the colour/achromatic

cycle of the stimulus: the BOLD response in V4v was greater during chromatic than achro-

matic blocks. The average modulation across V4v can be seen in the leftmost plots in Figure

5.5 for each subject (A) and averaged across subjects (B). Voxels lateral to V4v (V4v extra), in-

cluded in the ventral hemifield model of V4 but not the split dorsal/ventral model, showed

a similar modulation to that seen in V4v, as seen in the middle plots of Figure 5.5. In con-

trast, V4d did not show a modulation with the colour/achromatic cycle of the stimulus, as

68
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

CA Visual Areas
V1 V4v (extra)
V2 V4d
V3 LO
V3A/B VO
V4v

CC
Colour Preference

Colour
preferring

Achromatic
preferring

DM

EG

KS

SM

Figure 5.4: See next page for caption.

69
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

Figure 5.4: (See previous page for Figure). Maps of colour response across all visual areas for each
subject. Data from both the left and right hemispheres of each subject are shown on the
flattened cortical maps to the left and right respectively. Only those voxels where there
was a coherent response alternation at the rate of the stimulus alternation (period 30
seconds) are shown in colour. The threshold coherence level was set for each subject as
one standard deviation above the mean coherence across visual areas. Voxels which re-
sponded preferentially to colour blocks are in red, ones that preferred achromatic blocks
are in blue. As in Figure 5.3 the key in the top left of each hemisphere indicates the iden-
tity of each visual area, and regions outside the retinotopically defined areas are shown
in lower contrast.

seen in the relatively flat response across time in the rightmost plots of Figure 5.5. The dif-

ference between V4v and V4d in their response to these colour and achromatic stimuli is

inconsistent with their belonging to a single visual area.

A
1 1 1
V4v V4v (extra)
Percent Signal Change

Percent Signal Change

Percent Signal Change

V4d CA
CC
DM
0 0 0 EG
KS
SM

-1 -1 -1
0 15 30 0 15 30 0 15 30
Time (secs) Time (secs) Time (secs)

B
1 1 1
V4v V4v (extra) V4d
Percent Signal Change

Percent Signal Change

Percent Signal Change

0 0 0

-1 -1 -1
0 15 30 0 15 30 0 15 30
Time (secs) Time (secs) Time (secs)

Figure 5.5: Average response across all voxels in areas V4v, V4v (extra) and V4d to colour and achro-
matic stimuli, for individual subjects (A) and averaged across 6 observers (B). BOLD
response (% signal change) was averaged across each 30s cycle from all runs. Each cycle
is aligned such that the first 15s (shaded in light grey) are the response to the chromatic
stimuli and the second 15s (shaded in darker grey) are the response to the achromatic
stimuli. The mean response across observers is shown as a red line in B, the grey areas
indicate the mean {
2 between-subjects standard error (95% confidence intervals) for
the average across observers.

To compare the colour responsiveness of each area, I plotted the percentage of voxels that

showed a coherent preference for colour over achromatic stimulus blocks, as shown in Fig-

ure 5.6. There was a general trend for colour responsiveness in areas V1, V2 and V3 to be

lower than that in the ventral areas, and higher than that in the dorsal areas. If human V4

70
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

is split into ventral and dorsal components (Figure 5.6, A), then the single area comprises a

ventral part of high colour responsiveness, and a dorsal part of lower colour responsive-

ness, and area LO is a single area of low colour responsiveness.

If instead human V4 is a single ventral hemifield representation (Figure 5.6, B), then hV4

is an area of high colour responsiveness and LO comprises two subparts (LO1 and LO2)

which both show lower colour responsiveness than areas V1, V2 and V3. Considering only

the assumption that visual areas should have common functionality, these data argue in

favour of the single hemifield model of V4.

5.3.3 Visual field coverage according to the hemifield and split dorsal/ventral
models of V4

To further compare the two models of human V4, I derived for each area plots of visual field

coverage according to the definitions of the two models, shown in Figure 5.7. These maps

include a dot for each voxel in the region of interest, with placement determined by the

preferred phase of the response to the wedge (polar angle) and ring (eccentricity) stimuli.

For this analysis I used the same area definitions shown in Figure 5.3, but I used the averaged

data prior to spatial blurring.

Visual field coverage was generally good in areas V1, V2 and V3, although there was inter-

subject variation in the patchiness of coverage. Area V4v, common to both models of V4,

provided good coverage of the visual field above the horizontal meridian, and extending

slightly below the meridian in both left and right quadrants. This coverage is consistent

with the parameters used to define area V4v (see Methods).

The critical area of difference between the models is how well they account for coverage

in the remaining part of the visual field, assuming that area V4 should contain a complete

representation. In the split model, the dorsal part of V4 should represent the lower vertical

meridian and its surrounds. In the single hemifield model, this part of the visual field should

be covered by extending the ventral border of V4v, tested in this study by the inclusion of

the additional area labelled V4v (extra) in Figure 5.7.

71
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

A 5

Normalised Power of Response Modulation

V2d
V3A/B
V3d V4d

V1
LO
V2v

V3v V4v
with Stimulus Colour

0
VO2 VO1 V4v V3v V2v V1 V2d V3d V3A V4d LO
5
B
Normalised Power of Response Modulation

V2d
V3A/B
V3d
LO1
V1

V2v LO2

V3v hV4
with Stimulus Colour

0
VO2 VO1 hV4 V3v V2v V1 V2d V3d V3A LO1 LO2

Figure 5.6: Colour responsiveness of each area, according to the split dorsal / ventral model of V4
(A) and the ventral hemifield model of V4 (B). Reported responsiveness to chromatic
stimuli is the percentage of voxels that show a reliable modulation (coherence > 1 S.D.
above the individual’s mean coherence) with stimulus colour and a greater response to
chromatic than achromatic blocks. Error bars show the mean {
1 standard error of
the between-subjects mean.

72
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

For clarity, visual field coverage within the common area (V4v) and the areas of difference

(V4d and V4v (extra)) are plotted separately above the maps of coverage by areas V4 and

hV4. Both V4d and V4v (extra) have voxels which respond to the lower part of the visual

field which is not covered by V4v. Unlike Hansen et al. (2007a), I found that visual field cov-

erage according to the ventral hemifield model of V4 is not clearly deficient, weakening the

claim that V4 must include V4d for this area to have coverage of the entire visual field.

5.4 Discussion

I used fMRI to investigate the organisation of colour responsiveness across early visual cor-

tex, with the particular aim of testing whether the response of ventral V4 matches that of

its putative dorsal component. I compared the BOLD response of human occipital cortex to

coloured movie segments vs greyscale versions of the same segments. Across all early vi-

sual areas there were voxels that preferred colour stimuli, and very few voxels that preferred

achromatic stimuli, consistent with stimuli that were closely matched in luminance content.

There were generally more voxels that preferred colour stimuli in ventral than dorsal areas.

Specifically, I found a strong colour-sensitive response in ventral V4, but not in its putative

dorsal component.

5.4.1 Ventral V4 shows an increased response to coloured stimuli, dorsal V4

does not

I found that the dorsal region proposed to be part of V4 by Hansen et al. (2007a) showed

one of the lowest colour responses, while ventral V4 showed the highest of the areas con-

sidered. This functional difference argues against these two parts of cortex comprising the

same visual area.

The colour responsivity of V4v that I found here is contrary to the findings of Tootell &

Hadjikhani (2001) and Mullen et al. (2007), who reported colour response in areas around

V4v but no colour response in V4v itself. I believe that the difference is one of sensitivity,

73
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

CA CC DM EG KS SM

V1

V2

V3

V4v V4d

V4

V4v V4v
(extra)

hV4

Figure 5.7: Visual field coverage of V1, V2, V3, and V4 according to the split dorsal / ventral model
of V4 (V4v + V4d = V4) and the ventral hemifield model of V4 (V4v + V4v (extra) =
hV4). Each dot corresponds to the visual field preference of one voxel, either from the
left hemisphere (red dots) or right hemisphere (blue dots). The intensity of each dot was
scaled according to the coherence of its visual field preference, from white (incoherent)
to red/blue (0.7 coherence value or higher, where 1 is the maximum). Areas where the
red and blue dots overlap sum to black.

74
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

that the stimuli I used were particularly suited to revealing the strong colour response in

these ventral areas since they have the advantage of being complex, naturalistic stimuli,

which should put the human visual system into a more natural operating range, due to

their broadband spatial and temporal frequency content, and the presence of a variety of

movement, objects and people. However, I did find slightly higher colour responsiveness

in the area ventral to V4v, as seen in Figure 5.6, in that the addition of the extra part of V4v

slightly increased the colour responsiveness of hV4 relative to V4v alone.

The low colour responsiveness of V4d and LO (or LO1 and LO2) is consistent with these

areas being functionally similar, although the response of the dorsal areas when taken alone

does not provide strong evidence for one model over the other.

There was a general bias for ventral areas to show a greater colour response than dorsal

areas, but the difference between V4v and V4d was greater than this general bias. V4v shows

a higher responsiveness to colour than the adjoining V3v, and a similar responsiveness to

VO1. V4d shows lower colour responsiveness than V3d, and similar colour responsiveness

to LO. Areas V2 and V3 showed a smaller bias in the opposite direction: for V2 and V3 the

dorsal component shows slightly higher colour responsiveness than the ventral component,

and the magnitude of this difference was less than the difference between V4v and V4d. I

believe this result clearly demonstrates a functional difference between V4v and V4d, which

strongly argues against their belonging to a single visual area.

5.4.2 Small bias for dorsal V2 and V3 to show greater colour responsiveness
than their ventral parts

As noted above, in V2 and V3 the dorsal component showed slightly higher colour re-

sponsiveness than the ventral component, and overall these areas showed a weaker colour

response than ventral V4 and VO. In macaque V2 there are colour-selective cells (for exam-

ple, Hubel & Livingstone, 1987; Kiper et al., 1997) and maps of hue preference (Xiao et al.,

2003). Using fMRI, Wade et al. (2008) found that the dorsal and ventral parts of macaque

V2 showed similar colour responsiveness, consistent with the general agreement that the

75
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

ventral and dorsal parts of V2 form a single visual area in both macaque and human.

Colour responsiveness in V3 has been less extensively investigated than colour responsive-

ness in V4, but V3 shows a similar proportion of colour sensitive cells to V2 (Gegenfurtner

et al., 1997). In macaque it has previously been reported that the ventral part of V3 (also

called VP) responds to colour more than the dorsal part (Burkhalter et al., 1986; Felleman &

van Essen, 1987); the bias I found goes in the opposite direction.

5.4.3 Visual field coverage according to the two models of V4

One of the central arguments Hansen et al. (2007a) proposed in favour of the V4d/V4v

model of human V4 was that visual field coverage in ventral V4 excluded the lower vertical

meridian and surrounding angles. I found that the ventral hemifield model of V4 was not as

lacking in coverage of the visual field as Hansen et al. (2007a) reported it to be, but for most

subjects a more complete visual field coverage was provided by the split V4d/V4v model

(see Figure 5.7).

The fact that I found any coverage of the lower vertical meridian in ventral V4 weakens

the claim of Hansen et al. (2007a), who report that this coverage is not found in the region

surrounding V4v. V4d does tend to provide better visual field coverage than the extra region

near V4v, but when I have already demonstrated a functional difference between these areas,

the question becomes whether the coverage in area V4v is clearly deficient without recourse

to inclusion of a dorsal part. I believe it is not, and that the coverage of this extra part of

V4v, while not as complete as V4d, is not clearly deficient. This view is supported by the

findings of Winawer et al. (2010), who recently described BOLD signal artifacts in the region

of the transverse sinus. Ventral human V4 is often situated around the transverse sinus

(Winawer et al., 2010), which alone may account for a weaker, less coherent response in the

region corresponding to the lower vertical meridian representation according to the ventral

hemifield model.

76
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

5.4.4 Implications for homology between humans and non-human primates

If human V4 is a single ventral hemifield, this is a significant departure from the accepted

model of V4 in macaque, where V4 is split into dorsal and ventral hemifield representations.

To what extent should we expect homology between humans and non-human primates? Be-

tween primate species and within individuals of the same species, there is least variation for

areas V1 and MT, which are specified early in development (Rosa & Tweedale, 2005). Pro-

gressing from V1 to V2, V3 and other areas, maps of the visual field become less clearly

defined, and variability between individuals increases (Rosa & Tweedale, 2005). The no-

tion that these areas are less clearly defined is consistent with the history of controversy

regarding definitions of V3 and V4 in macaques, as described above. Functional differences

between species are also more likely for later visual areas; for example, Wandell et al. (2007)

note that while V3A is not especially responsive to motion in macaque, it is strongly respon-

sive in humans.

Since V4 is a later visual area, it might be less surprising if there were a breakdown of the

homology between macaque and humans in this area than in say, V1 or V2. It is unclear how

differences in retinotopic organisation might be linked to differences in functional organisa-

tion. The difference in retinotopic organisation between macaques and humans may imply

that there is also a difference in the functional role of V4, although the colour responsiveness

I found here is consistent with reports of V4 being of particular importance in colour percep-

tion for macaques. Further work may reveal a functional difference between macaque V4

and human ventral hemifield V4, but current evidence is consistent with both areas being

involved in form vision, and particularly concerned with colour.

5.4.5 Conclusions

Hansen et al. (2007a) argued that human V4 includes a dorsal component on the basis that

dorsal V4 complements the otherwise deficient visual field coverage of ventral V4, and that

dorsal and ventral V4 show similar attentional modulation. They also make the point that

a model which is homologous with that accepted for non-human primates should be main-

77
 Chapter 5. Cortical response to colour vs achromatic stimuli and the V4d debate

tained until there is strong evidence to reject it. I have shown that while V4v shows a strong

colour responsiveness, V4d shows a weaker colour responsiveness even than some sur-

rounding areas. I believe this functional difference in the responsiveness of V4v and V4d

argues against their belonging to a single visual area, and that visual field coverage in ven-

tral V4 is not deficient enough to warrant maintenance of the split V4d/V4v model. Instead,

I agree with previous suggestions that in humans V4 is composed of a single ventral hemi-

field representation.

78
Chapter 6

Decoding non-cardinal colours from

activity in human visual cortex

The work described in this chapter was published in the Journal of Vision (Goddard et al.,

2010), and the paper is included in Appendix B.

6.1 Introduction

Mechanisms of colour vision in cortex have not been as well characterised as those in sub-

cortical areas, particularly in humans. I used fMRI in conjunction with univariate and mul-

tivariate (pattern) analysis to test for the initial transformation of sub-cortical inputs by hu-

man visual cortex. In macaque it has been demonstrated that there are cells as early as V1

which prefer chromatic directions away from the cardinal directions that isolate the L-M and

S-(L+M) mechanisms that carry information in parallel to visual cortex via the parvocellu-

lar and koniocellular layers of the LGN (de Valois et al., 2000; Conway, 2001), implying a

combination of information from the sub-cortical channels early in visual cortex. For corti-

cal cells to have chromatic preference intermediate to the cardinal axes, there must be some

combination of the L-M and S-(L+M) channels.

In a recent fMRI study, Brouwer & Heeger (2009) found that the principal components of the

79
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

response in V1 are consistent with a response dominated by an opponent coding of colour,

as found in sub-cortical areas, but that by hV4 and VO it more closely resembles our percep-

tual colour space. That is, V1 shows a differential response to variations in colour but not

a continuous representation of hue, but in higher areas colours of similar hue evoke similar

responses. This finding does not rule out the possibility that the signals of the fundamen-

tal pathways (L-M and the S-(L+M)) are combined in the early visual areas, such as V1, a

possibility I address specifically in this study.

I obtained high resolution functional images of the BOLD (blood-oxygen-level-dependent)

response from subjects’ occipital and parietal lobes while they viewed coloured stimuli. Pre-

vious studies of cortical chromatic mechanisms in humans have used perceptually relevant

hues (Brouwer & Heeger, 2009; Parkes et al., 2009). The stimuli used here were not cho-

sen for their perceptual relevance, but were designed to be metameric to the hypothesized

sub-cortical chromatic mechanisms. Specifically, the stimuli were designed so as to fulfil

the following conditions: (1) to induce the same magnitude of activity in the L-M opponent

channel; and (2) to induce the same magnitude of activity in the S-(L+M) opponent chan-

nel. This was achieved by combining a given L-M modulation with a given S-cone isolating

modulation in each of two different phases. When -S was in phase with M then the stimulus

appeared lime-magenta; when +S was in phase with M then it appeared orange-cyan. The

lime-magenta and orange-cyan stimuli can only be distinguished by BOLD if there are cells

which receive a combination of inputs from the L-M and the S-(L+M) pathways.

Univariate and multivariate analyses tested whether the BOLD response within each corti-

cal visual area depended upon the colour of the stimulus. Univariate analyses show what

information about the stimulus is detectable in the average activity across a region, while

multivariate classifiers are capable of also learning differences in the pattern of activation

between stimuli.

Both in the univariate and multivariate analyses employed here, an algorithm is trained to

classify the stimulus from activity across a region, and tested on novel data. Above chance

performance indicates that the area contains information about the stimulus dimension that

was varied. Here, the premise is that if the activity across a visual area can be used to

80
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

discriminate between the stimuli then that area contains a representation of colour that could

only be generated through a transformation of the signals from the sub-cortical L-M and S-

(L+M) pathways.

6.2 Materials and Methods

6.2.1 Colour calibration procedures and display system

Stimuli were generated and displayed using Matlab (version 7) software on a Dell Latitude

laptop computer driving an nVidia Quadro NVS 110M graphics card to draw stimuli to a

35x26cm Philips LCD monitor, with 60 Hz refresh rate, viewed from a distance of approxi-

mately 1.58m. Scanning took place in a darkened room. Subjects, while lying in the scanner,

viewed the monitor through a mirror mounted above the head cage which reflected the im-

age from the LCD monitor located behind the scanner. Stimuli were calibrated in situ for the

LCD monitor and mirror arrangement, using measurements obtained with a PR-655 Spec-

traScan spectrophotometer (by Photo Research Inc.). For a more detailed description of the

calibration process, refer to Appendix A.

Changes in both chromaticity and luminance of the screen with increasing R, G and B values

were taken into account when generating the experimental stimuli. The CIE (xyY) coordi-

nates measured for 16 values during calibration were interpolated to 255 values using the

best-fitting spline, and these were used to calculate the luminance and chromaticity for each

combination of R, G and B intensity values.

Data were collected on five subjects (three male), aged between 24 and 40 years, with nor-

mal or corrected to normal visual acuity and normal colour vision, as tested using Ishihara

plates (Ishihara, 1990). All subjects provided informed consent, and the entire study was car-

ried out in accordance with guidelines of the University of Sydney Human Research Ethics

Committee.

81
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

6.2.2 Chromatic, spatial and temporal stimulus properties

Example stimuli are shown in Figure 6.1. The stimulus was an annulus, centered on the fix-

ation point, with an inner diameter of 0.8 degrees visual angle, and an outer diameter of 7.8

degrees. The remainder of the screen was held at the mean luminance, which was 24.7 cd{m2

[CIE (1931) x, y  0.300, 0.337], and all stimuli were produced by spatiotemporal modulation

around this point. The annulus contained a spatial pattern that counterphased sinusoidally

at a temporal frequency of 1 Hz. The spatial pattern was the multiplication of a radial and

a concentric sinusoidal modulation, (the resultant plaid pattern is shown in Figure 6.1). All

these modulations can be represented in a three-dimensional colour space described previ-

ously (Derrington et al. (1984); DKL space). Along the L-M axis only the signals from L and

M cones vary, in opposition, without variation in luminance. Along the orthogonal S-cone

isolating axis there is no modulation of either the L or M cones. The L-M and S axes de-

fine a plane in which only chromaticity varies. Normal to this plane is the luminance axis

along which the signals from the L and M cones vary in proportion. The axes were derived

from the Stockman & Sharpe (2000) 2-degree cone spectral sensitivities and adjusted indi-

vidually for each observer (see below). The scaling of these axes is largely arbitrary; here I

used modulations along the isoluminant axes that were 90% of the maximum modulation

achievable within the gamut of the monitor. Modulation along the L-M axis produced max-

imum cone contrasts of 15.4% in the L-cone and 17.8% in the M-cone; along the S-cone axis

the maximum S-cone contrast was 79.6%. Cone contrast values for all stimuli are listed in

Table 6.1. Each frame of the stimulus was generated prior to the experiment as a bitmapped

image, and then these images were drawn to the screen for each stimulus presentation using

routines from PsychToolbox 3.0.8 (Brainard, 1997; Pelli, 1997).

There were four stimuli, chosen such that over time all four would: (1) equally stimulate

the L-M opponent channel; and (2) equally stimulate the S-(L+M) opponent channel. The

angle of each stimulus within the isoluminant plane was intermediate to that of the L-M

and S-(L+M) axes, and was defined as a vector addition of modulations along those two

axes. When the L-M modulation was in phase with the S-(L+M) modulation, the stimulus

modulated between magenta and lime; when these pairings were reversed the modulation

82
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

Figure 6.1: Stimuli used in the fMRI experiment; A: the colour of the stimulus modulated sinu-
soidally between the upper plaid and the lower plaid at 1 Hz. The stimuli on the left are
orange and cyan, on the right, the stimuli are lime and magenta. For both colour pairs,
minimum motion was used to determine each subjects’ perceived equiluminance point,
and a 25% luminance modulation was added. In the first and third pair of stimuli the
light/dark modulation is paired with cyan/orange and lime/magenta, respectively. In
the second and fourth stimuli these pairings are reversed. B: Example stimulus with fix-
ation task. At fixation, there was a light grey cross surrounded by a high contrast ring, as
illustrated above. The high contrast ring provided feedback to subjects when they made
small eye movements, since an afterimage would become visible. While subjects fixated
on the central cross (partially obscured by the digit), they were required to respond with
a button press whenever either of two target items was presented in the digit stream.
Digits updated at 3 Hz were presented in random order, and could be 0 to 9 inclusive,
each in either black or white. The target items were a conjunction of a digit and a par-
ticular colour, for example either a black 3 (shown here) or a white 7. The fixation task
was unrelated to the experimental stimulus, which was presented in the annular region
surrounding fixation.

83
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

Figure 6.2: Modulation of cardinal sensitive mechanisms over time in the experimental stimulus, for
an example 18 seconds including a transition from a light magenta-dark lime block to a
light cyan-dark orange block. At each transition, the non-cardinal modulation switched
from a lime-magenta block to an orange-cyan block or vice versa. At each transition
there was a phase reversal in either the L-M or the S-(L+M) cardinal modulation; here
there is a phase reversal in the L-M modulation at the time of transition. The luminance
(light-dark) modulation had no reversals at any time. The amplitude of response of any
cardinal sensitive mechanisms should be constant across the lime-magenta and orange-
cyan blocks. Phase reversals are the only cue that could be used to discriminate the
stimuli where the cardinal sensitive mechanisms are kept independent, but the block
order was balanced such that this cue could not be used to predict the stimulus.I used
one of two block orders for each run, where the second was a reversal of the first order.

84
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

appeared orange-cyan. The point of subjective isoluminance (the angle of the isoluminant

plane from the luminance axis) was estimated separately for each observer using the mini-

mum motion technique described by Anstis & Cavanagh (1983), for the magenta-lime and

for the orange-cyan modulations. A 25% luminance modulation was then added to the

subjectively-defined isoluminant modulation in one of two phases. The four stimuli there-

fore appeared: light magenta - dark lime, dark magenta - light lime, light orange - dark cyan,

and dark orange - light cyan (example stimuli in Figure 6.1 A, and example cone contrast

values in Table 6.1).

Stimulus Colour L-cone Contrast M-cone Contrast S-cone Contrast

Dark Cyan -0.154 0.015 0.688
Light Cyan 0.031 0.178 0.796
Dark Orange -0.031 -0.178 -0.796
Light Orange 0.154 -0.015 -0.688
Dark Magenta -0.031 -0.178 0.688
Light Magenta 0.154 -0.015 0.796
Dark Lime -0.154 0.015 -0.796
Light Lime 0.031 0.178 -0.688

Table 6.1: Cone contrast values for stimuli calibrated for the subjective equiluminance point of ob-
server EG. The background of the stimuli had CIE xy coordinates of 0.30, 0.34, and a
luminance (Y) of 6.78cd{m2 .

The luminance modulation was added so that if there were any residual differences in lumi-

nance between the lime-magenta and orange-cyan blocks, this difference should be masked

by the much larger luminance modulation. The contrast response of early visual areas to lu-

minance defined stimuli is steeper at low than high contrast (for example, see Liu & Wandell

(2005)). Thus a luminance artifact in the stimuli would result in a much smaller difference

in the response to the two stimuli than if there were no luminance modulation (and hence

lower luminance contrast). The same rationale underlies the use of random luminance noise

to mask potential luminance artifacts (Birch et al., 1992; Kingdom et al., 1992; Sumner et al.,

2008).

85
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

6.2.3 Experimental design

All experimental scans were completed during a single session for each subject. The session

included ten functional scans, each lasting 4.5 minutes. During each scan the subject viewed

18 blocks of the experimental stimulus, alternating between orange-cyan and lime-magenta

blocks. Each block was 15 seconds long, and data from the first and last block were excluded

from the analysis. In order to change the colour of the stimuli, either the L-M or the S-(L+M)

modulation must change phase, as illustrated in Figure 6.2. This phase reversal is likely to

induce an increased response of a neural population which responds to the relevant cardinal

colour, as in the characteristic response rebound used in fMRI adaptation (for example, see

Kourtzi et al. (2003); Engel & Furmanski (2001)). It is also possible that the response to the

phase reversal may be evident in the BOLD signal, even when averaging across activity

within a block, and could potentially be used to discriminate the two types of stimuli (for

example, if an orange-cyan block always commenced with a L-M phase reversal). In order

to eliminate this potential source of information about the colour of the stimuli, I balanced

the stimuli so that the pattern of phase reversals could not be used to predict the stimulus.

I used one of two block orders for each run, where the second was a reversal of the first

order.

Localiser

To select those voxels in each visual area most responsive to the experimental stimuli, two

additional localiser scans were conducted during the same session as the experimental scans

for each subject. Using a localiser scan that is separate from the experimental data avoids

circularity that could otherwise be present (Kriegeskorte et al., 2009). Localiser scans in-

cluded a total of 17 blocks of 15 seconds each, consisting of stimulus blocks interleaved with

blocks of fixation only. The stimulus blocks included 2 lime-magenta blocks and 2 orange-

cyan blocks, in addition to 4 black-white blocks where the stimulus had the same spatial

arrangement but was modulated between black and white.

86
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

Fixation Task

Throughout experimental and localiser scans, subjects performed a task at fixation that was

unrelated to the annular experimental stimulus. This task was designed to be attention-

ally demanding in order to direct attention away from the experimental stimulus as much

as possible. While this would have reduced the BOLD response, and so likely decreased

the ratio of signal to noise in the data which were input to the classifier, it greatly reduces

the chance that subjects could have systematically directed more attention to one type of

stimulus block.

Subjects were required to detect a conjunction of contrast polarity and number in a digit

stream of the digits 0 to 9 inclusive, presented at fixation, updated at 3 Hz. Digits were

either black or white, against the mean grey of the background, as seen in Figure 6.1, B, and

the order was randomly generated for each run. Subjects responded with a button press

to the onset of either of two target digits, one only when black and the other only when

white (for example a black 3 or a white 7). Responding to a conjunction of digit and contrast

polarity made this a difficult task. Target digits were updated at the beginning of each run

to increase task difficulty and minimise practice effects.

All subjects performed the task significantly above chance (p   0.01, permutation test),

demonstrating that they were engaged in the task, but each subject also made errors, imply-

ing that the task was not trivial and required attention. For no subject was there a significant

difference in performance between lime-magenta and orange-cyan blocks, consistent with

equal attentional resources being devoted to the task in each case.

Subject performance on the fixation task is shown in Figure 6.3, which plots the mean prob-

ability of a button press during the first 3 seconds following a stimulus onset. The perfor-

mance of subjects on this task confirmed that it was attentionally demanding: for no subject

was every target followed by a button press (misses), or was every response preceded by a

target (false alarms). I considered a button press onset during the first three seconds follow-

ing a stimulus onset to be a hit (stimulus duration was always 333 ms); each button press

was counted as a response for at most one stimulus. The miss rate across subjects ranged

87
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

from 0.23 to 0.40, and the false alarm rate ranged from 0.001 to 0.004, showing that the

task was sufficiently demanding that subjects’ performance did not reach ceiling. To assess

whether subjects’ performance was above chance, I compared the stimulus onsets with the

response onsets for each subject, then performed the same analysis on 1000 bootstrapped

versions of the data where the stimulus onsets were the same but the response onsets were

shuffled in time. I used two methods to compare the relationship between stimulus and

response onsets. The first was d1 , a measure of sensitivity based on signal detection theory,

which is given by

d1  zpHq
zpF q (6.1)

where the hit rate H = number of hits / total number of target stimuli, false alarm rate F

= number of false alarms / total number of non-target stimuli, and z is the inverse of the

normal distribution function. The d1 between the stimulus onsets and response onsets for

each subject ranged from 2.79 to 3.78, which was significantly greater p p   0.01q than the
mean d1 of the bootstrapped data in each case. I also used a more sensitive test to measure

performance, by calculating the mutual information between the pattern of stimulus onsets

and subject responses. Calculating mutual information does not require assumptions of

normally distributed signal and noise with equal variance, which underlie the calculation

of d1 . The mutual information (MI) between the timing of the target onsets and response

onsets was found using

¸ Pp s|rq
MI  PprqPp s|rqlog2
P p sq
(6.2)
r,s

where Pprq is the probability of a response onset during the entire duration of the run, Pp sq

is the probability of a stimulus onset during the run, and Pp s|rq is the conditional probability

of a response given a stimulus, and a response is defined as following a stimulus when it

occurs in the first three seconds following the onset of the stimulus (as above). The mutual

information between target and response onsets was between 0.51 and 0.97 bits of informa-

tion across subjects. This was also significantly greater p p   0.01q than the mean MI of the
bootstrapped data, for each subject. This finding that performance was significantly above

chance for all subjects indicates they were engaging in the task while not performing at

ceiling, making it unlikely that subjects directed their attention towards the stimulus.

88
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

Figure 6.3: Average probability of a response in the 3 seconds following the onset of a target stim-
ulus. The red line plots the probability of a response over time, relative to the onset of
the target; the blue area shows the same probability for the bootstrapped data, which
indicates the average probability that would be expected if subjects were responding
at random. The upper row shows each subject’s performance during the lime-magenta
blocks; the lower rows shows each subject’s performance during the orange-cyan blocks.

Since I was especially concerned that there not be a difference in attention directed towards

the stimulus between the two types of block, I also tested for any difference in performance

between lime-magenta and orange-cyan blocks. In Figure 6.3, the results from these two

conditions are plotted separately. By calculating the MI for each block type, for each run, I

had 10 estimates of MI for each type of block. A two sample t-test did not find a significant

difference between the means of these two populations for any of the subjects. While little

inference can be drawn from this null result, it means I have no evidence that there was a

difference between the block types in the attention directed towards the stimulus.

6.2.4 fMRI methods

fMRI data were collected using a 3T Philips scanner (Symbion Imaging Centre, Prince of

Wales Medical Research Institute, Sydney, Australia), with a birdcage head coil.

89
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

Anatomical measurements and definition of grey matter

The anatomical image for each subject was generated from the average of three scans. Two

of these were high resolution (1 x 1 x 1mm) structural MR images of each subject’s whole

brain, acquired using a Turbo Field Echo (TFE) protocol for enhanced grey-white contrast.

A third, higher resolution (0.75 x 0.75 x 0.75mm) scan of the caudal half of the head was also

acquired in order to recover more anatomical detail of the occipital lobes.

Using the Statistical Parametric Mapping (SPM) software package SPM5 (Frackowiak et al.,

1997), anatomical images were each reoriented to approximately the same space using ante-

rior and posterior commissures as anatomical landmarks. Fine alignment of these anatomi-

cal images was carried out using normalised mutual information based coregistration, and

all of the anatomical images were resampled so that they were in the same voxel space

with a resolution of 0.75 x 0.75 x 0.75mm. From each image I removed intensity inhomo-

geneities using a nonparametric inhomogeneity correction method (Manjón et al., 2007), and

normalised the images such that the white matter had an approximate intensity of 1. The

coregistered, inhomogeneity corrected, normalised images were then averaged together to

produce a mean anatomical image for each subject.

ITKGray software (Yushkevich et al., 2006) was used to define the white matter of each sub-

ject, initially using automatic segmentation tools, then using manual editing. The segmen-

tation image was imported into mrGray, part of the mrVista software package developed

by the Stanford Vision and Imaging Lab (http://white.stanford.edu/software/).

In mrGray, grey matter was ‘grown’ out from the white matter in a sheet with a maximum

thickness of 4 voxels.

Functional measurements

fMRI data were acquired using a T2*-sensitive, FEEPI pulse sequence, with echo time (TE)

of 32ms; time to repetition (TR) of 3000ms; flip angle 900 ; field of view 192mm x 70.5mm x

192mm; effective in-plane resolution 1.5mm x 1.5mm, and slice thickness 1.5mm. 47 slices

were collected in an interleaved, ascending order, in a coronal plane tilted such that the scan

90
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

covered the whole of the occipital lobe and the posterior part of the parietal and temporal

lobes. Using SPM5, all functional data were preprocessed to correct for slice time and head

motion before alignment to the structural data. Data from functional scans were aligned to

a whole head anatomical scan acquired in the same session, using normalised mutual infor-

mation based coregistration. The functional data were then aligned to the subject’s average

anatomical by first aligning the within session anatomical with the average anatomical scan,

then applying the same transformation to the functional data.

Definition of Retinotopic Areas

For each subject, the precise anatomical locations of the early areas of visual cortex (V1,

V2, V3, V3A/B, hV4, and VO) were found functionally using standard retinotopic mapping

procedures (Engel et al. (1997); Larsson & Heeger (2006), see Wandell et al. (2007) for a

summary). Subjects were scanned while viewing first a rotating wedge then an expanding

ring stimulus, overlaid on a fixation cross of light grey lines, as shown on the key above the

maps in Figure 6.4 (Schira et al., 2009).

Averaged data from the wedge and ring stimuli were smoothed with a Gaussian kernel of

half width 1.5 mm, then projected onto a computationally flattened representation of the

cortex for each hemisphere of each subject, using mrVista. Areas V1, V2, V3, V3A/B and

hV4 were manually defined on the phase and eccentricity maps derived from the wedge

and ring stimuli (shown for an example subject in Figure 6.4, A and B, respectively), using

the conventions described by Larsson & Heeger (2006). According to these definitions the

foveal representation at the occipital pole is shared by V1, V2, V3, and hV4, while V3A and

V3B, which border the dorsal part of V3, share a dorsal fovea. For this analysis I did not

attempt to separate V3A and V3B. I defined hV4 as a ventral hemifield representation that

borders the ventral part of V3.

For area VO, the phase and eccentricity maps were considered in conjunction with a flat-

tened map of those voxels that responded more to chromatic than achromatic stimuli in the

localiser scan (as shown for an example subject in Figure 6.4, C). Where it existed, I used

91
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

the hemifield representation from the phase and eccentricity maps to define VO. Where the

retinotopic map from the wedge and ring stimuli was unclear, I tended towards a liberal def-

inition of VO, in order to avoid excluding any voxels in the region which showed a prefer-

ence for chromatic stimuli in the localiser analysis. Each retinotopic area was defined on the

flattened map of a subject’s cortex then transformed into the space of the subject’s anatomi-

cal, smoothed (FWHM=1.5mm), and resliced to the resolution of the functional images using

4th degree B-spline interpolation. Voxels assigned to each visual area were allocated a value

reflecting the cumulative influence of such transformations. To prevent overlapping vox-

els between adjacent visual areas, each voxel was assigned to the visual area for which it

possessed the greatest value.

Area MT+ was defined on the basis of a separate localiser scan in which blocks of low con-

trast static and moving dots were interleaved with fixation only blocks. In SPM5, I specified

a general linear model of this data, and defined MT+ by finding lateral clusters of voxels

that responded more to moving than to static dots. The definition of MT+ was projected

onto the flattened map for the purposes of illustration, as in Figure 6.4, but the original 3D

definition of MT+ was used in the analysis of the functional data.

Functional Preprocessing

Data from each of the two localiser scans were processed using the methods described

above, then analysed using a General Linear Model (GLM) in SPM5. I pooled responses

to the luminance and chromatically defined stimuli and contrasted these with the response

to the fixation only blocks. The subsequent analyses included only those voxels that re-

sponded significantly more (p   0.05, uncorrected) to the stimulus than fixation only. These

voxels are shown on an example flattened map for one subject in Figure 6.4, D.

For all functional scans the BOLD signal was labelled with the stimulus presented 2 images

(6 seconds) previously, in order to compensate for the delayed haemodynamic response,

then was highpass filtered (cutoff 128 s, using filtering methods from SPM5) in order to

remove low frequency confounds in the data, and finally converted into z-scores for each

92
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

Figure 6.4: Example maps of functionally defined retinotopic areas for the left and right hemi-
spheres of subject DM. In each of A, B, C and D the underlying greyscale image shows
the flattened map of visual cortex, centred on the occipital pole; the darker the grey the
deeper the sulcus. In D the greyscale anatomical map was darkened to increase the visi-
bility of the overlaid image . A & B: Flattened maps of visual cortex overlaid with phase
maps of the response to the wedge and ring stimuli, respectively. Above these maps is a
schematic of the stimulus (top left) and a colour map showing the area of the visual field
which each colour corresponds to in the phase maps (top right). In C the same flattened
map of visual cortex is overlaid with a heat map showing those voxels which responded
more to the chromatic than the achromatic stimuli; the significance of this result for each
voxel is indicated by the T-statistic colour map above. Areas V1, V2, V3, V3A/B and hV4
were defined on the basis of the wedge and ring phase maps in A and B, while area VO
was defined according to a combination of the wedge and ring phase maps in A and B,
and the contrast in C. MT+ was defined according to a motion versus static dots localiser
(not shown). The borders of each of these areas are drawn on each of the maps in A - D;
the key on the right indicates which of the outlined regions corresponds to each visual
area. In D, the heat map indicates those voxels that were included in the analysis: those
which responded significantly more to chromatic or achromatic versions of the stimuli
than to fixation; the significance of this result for each voxel is indicated by the T-statistic
colour map above.

93
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

of the ten runs in order to reduce variability from inter-run differences. Data from each

voxel were z-scored separately. Within each 15 second block the BOLD response, normalised

according to the procedures described above, was averaged across the 5 measurements to

give a single score for each block in each run.

6.2.5 Classifier Analysis

Classifiers were restricted to each of several functionally defined visual areas for each subject

and trained to discriminate the two patterns. I compared the performance of classifiers

trained on two types of data for each area: in the univariate case, the classifier was trained

and tested on the average activation across voxels within an area (that is, 1 value per block),

while in the multivariate case the classifier was trained and tested on the pattern of activity

across voxels within an area (n values per block).

I used a linear support vector machine (SVM) classification technique in the analysis. Sup-

port vector machines are powerful in their ability to learn a decision rule for multivariate

data (Bennett & Campbell, 2000): in this case, for n voxels with 144 data points each (72 from

lime-magenta and 72 from orange-cyan blocks), they learn the hyperplane which best sep-

arates the data points in an n dimensional space, where each dimension corresponds to the

normalised response of one voxel (using linear SVMs requires that the hyperplane’s projec-

tion onto any two dimensions is linear). In learning a decision rule, support vector machines

offer a way of balancing the aim of maximally separating the data from different sets, with

the risk of over-fitting the data and learning a decision rule that is based on random fluctu-

ations rather than replicable signal (Burges, 1998; Norman et al., 2006). To ensure that the

classifier performance reflected signal present in the data, I evaluated classifier performance

on its ability to generalise, i.e. to correctly discriminate data that were not included in the

training set. For the univariate classifier the technique was the same, but there was only

one dimension along which the 144 data points varied, so the power of the support vector

machine was not utilised.

94
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

Cross-validation: leave-one-out train and test

Classification analysis was performed using a Matlab (version 7.5) interface to SVM-light

6.01 (Joachims, 1999). The classifier was trained on the scores from 9 runs and tested on a

tenth; this procedure was repeated 10 times. This leave-one-out train and test procedure

resulted in the data from each run being used as test data once, giving an average classifier

performance (reported in the Results section as a percentage correct) based on the accuracy

across 160 classifications, while ensuring that the test data never included data that were

used in training.

I repeated the classification analysis with increasing numbers of voxels (n) within each vi-

sual area, from n  3 voxels, to the n  maxN, where maxN is the total number of voxels that

reached significance in the localiser analysis. Voxels in each area were ranked according to

their t statistic from the localiser analysis, based on the separate localiser scans, in order to

select voxels that responded to the area of visual field occupied by the experimental stimuli

and exclude those which represented areas of the visual field which were more foveal or pe-

ripheral than the annular stimuli. The top n most significant voxels were used in each case.

Classifier performance generally increased as more voxels were included in the analysis, but

there was some variability around this trend. To summarise classification performance (as

reported below) I fitted the classifier performance (P) as number of voxels (n) increased with

an exponential growth function which reaches a limit (L), given by

P  0.5 pL
0.5qp1
e
n{cq (6.3)

where 0.5 is chance performance (at n  0), and c is a curvature parameter, specifying how
many voxels the curve takes to reach the limit, L. When the curve fitted the data, the classi-

fier performance reported below corresponds to the limit (L) of the growth function. When

the curve could not be fitted to the data within 100 iterations of the Matlab function nlinfit

(usually when classifier performance was low), the average classifier performance, rather

than the limit of the curve, is reported as the summary statistic of classifier performance.

Classifier performance as a function of number of voxels, along with the best-fitting curve,

95
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

is plotted in Appendix 6.7.

Permutation Analysis

To test the statistical significance of classifier performance, I ran a permutation analysis

to estimate classification accuracy expected by chance alone. Permutation tests are non-

parametric and so do not include an assumption of normality, and such tests have previ-

ously been employed to evaluate classification analysis (Mourao-Miranda et al., 2005). For

each area in each subject I performed the same analysis as that described above, except that

before training the classifier I randomly permuted the stimulus labels associated with each

block in the training data set. Using 1000 repetitions of this permutation analysis, I gener-

ated a population of 1000 estimates of the classifier accuracy that could be expected in cases

where the data did not contain any stimulus-related information. For each iteration of the

permutation analysis I averaged these estimates across subjects for each area and compared

these 1000 values with the observed between-subject mean accuracy. In the statistics re-

ported below for classifier performance, p-values were calculated by finding the proportion

of these 1000 estimates which was greater than the observed classifier accuracy.

6.3 Results

I tested for the presence of cortical representations of colour capable of discriminating stim-

uli that cannot be distinguished by the L-M and S-(L+M) sub-cortical opponent channels.

The lime-magenta and orange-cyan stimuli contain the same L-M and S-(L+M) modula-

tions, varying only in the phase that these modulations are added together. For the lime-

magenta stimuli, the L-M and S-(L+M) modulations are the same phase, whereas for the

orange-cyan stimuli the L-M and S-(L+M) modulations were in opposite phase. I used fMRI

to measure changes in BOLD signal as an indirect measure of neural activity, then asked the

extent to which the visual stimulus could be discriminated from patterns of brain activity in

a predefined visual area.

96
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

Below, I report stimulus related differences in both the mean activity and pattern of activ-

ity across a range of regions. There was a small but reliable bias across subjects for lime-

magenta over orange-cyan stimuli in the mean activity across each region, and I found that

the difference in the mean activity was sufficient for a univariate classifier to learn to cor-

rectly discriminate the stimuli. I also found evidence of additional stimulus-related infor-

mation in the pattern of activity across V1 and V2, using multivariate classifiers.

6.3.1 Consistent bias for lime-magenta over orange-cyan stimuli in average ac-
tivity

There was a significant difference in the response to lime-magenta vs orange-cyan stimuli,

impossible without combination of signals from the fundamental cone-opponent channels.

For the univariate analysis I averaged across those voxels within each area for which there

was a significant difference in their response to the localiser stimulus versus fixation. The

average z-scored activity across an area for each block was treated as a separate measure

of the area’s response to lime-magenta or orange-cyan stimuli, giving 80 measurements for

each; the distributions of these measurements for each subject are shown in Figure 6.5.

In all subjects each area that showed a significant bias for one colour modulation showed the

same bias; signal was greater for lime-magenta than for orange-cyan. As shown in Figure

6.5, the mean of the 80 lime-magenta blocks was significantly greater than that of the 80

orange cyan-blocks (p   0.05, two-tailed t-test) in 25 areas across 5 subjects. The only area for

which there was not a significant difference between the two types of stimuli for any subject

was area MT+. I conclude that stimuli which equally modulate the cardinal axes of colour

space are not equally represented in visual cortical areas, and this biased representation is

seen as early as V1.

Consistent with the differences in the average response across each region, univariate classi-

fiers performed significantly better than chance as early as V1, as shown in Figure 6.6 (light

grey bars). The average univariate classification performance across subjects in all areas was

significantly above chance (p   0.05, one-tailed permutation test). MT+ showed the lowest

97
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

Figure 6.5: Mean activity in response to lime-magenta vs orange-cyan blocks, across different cor-
tical visual areas, for each subject. For each block, the z-scored BOLD response was
averaged across all voxels for which there was a significant difference in their response
to the localiser stimulus versus fixation. The histograms plot the distribution of these
averages. Behind each histogram, a normal distribution of the same mean and stan-
dard deviation is plotted for reference. For all subjects, where there was a significant
difference between the response to lime-magenta and orange-cyan blocks (tested with a
two-tailed t test), the response to the lime-magenta blocks was greater than the response
to orange-cyan blocks. The functional data did not include a fixation block; the average
% signal change in the colour blocks in the localiser data, relative to fixation ( 1 stan-
dard error of the between subject mean) was V1: 0.65 ( 0.16); V2: 0.78 ( 0.19); V3: 0.56
( 0.17); V3A/B: 0.18 ( 0.16); hV4: 0.67 ( 0.27); VO: 0.35 ( 0.19); MT+: -0.09 ( 0.17).

98
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

performance for 4 of 5 subjects.

For 4 of 5 subjects, the area with the best classifier performance was V2. The earlier cortical

visual areas (V1, V2 and V3) generally outperformed the dorsal area V3A/B, as well as the

ventral areas hV4 and VO. V1, V2 and V3 generally had a greater number of voxels, which

may account for their high performance. In order to test this, I repeated the classifier analysis

on 100 voxels that were randomly chosen from those voxels in the area that responded to

the localiser stimulus. For this subset of 100 voxels, the classifier accuracy averaged across

subjects in V1, V2 and V3 (61, 64 and 61%) was still better than in V3A/B, hV4 and VO (45,

56 and 52%). This suggests that differences in classifier performance cannot be accounted

for by the generally greater number of voxels included in the analysis for areas V1, V2 and

V3. Area MT+ had fewer than 100 voxels that responded to the localiser in all subjects, and

so was excluded from this reanalysis.

6.3.2 Additional information in the pattern of activity for areas V1 and V2

Multivariate pattern classifiers were also trained to discriminate the two types of stimu-

lus, allowing us to test for additional stimulus related information in the pattern of activity

across each area. The univariate classifier was trained on a subset (9 of 10 runs) of the aver-

age data (as plotted in Figure 6.5), and tested on the remainder. The multivariate classifier

was trained and tested in the same way on data which was not averaged across voxels, so

that in addition to the average it could learn differences in the pattern of activity across an

area between blocks. This analysis allows a comparison of the results from the univariate

and multivariate classification techniques, which gives an indication of how the informa-

tion in the pattern of activity across an area differs from information given by the mean

response.

For each subject, the classifier analysis was repeated for increasing numbers of voxels from

each area. Figure 6.7 shows multivariate classifier performance as an increasing number

of voxels were included in the classifier, on a semilogarithmic scale. It also shows the best

fitting curve (see Equation 6.3), from which the limit was reported as the classifier perfor-

99
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

Figure 6.6: Univariate and Multivariate Classifier Performance across subjects. Mean univariate
and multivariate classifier accuracies across subjects for each visual area are plotted in
light grey and dark blue bars, respectively. Error bars indicate  1 standard deviation
of the population of classification accuracy estimates generated using the permutation
analysis. Chance performance (50%) is shown with the dashed line. In all visual areas ex-
cept MT+, the multivariate classification accuracy was significantly (p   0.01, one-tailed
permutation test) above the chance performance predicted by the permutation analysis,
while the univariate classification accuracy was significantly above chance (p   0.05,
one-tailed permutation test) in all areas. The multivariate pattern classifier generally
outperformed the univariate classifier, and this difference was significant (p   0.01,
two-tailed permutation test) for areas V1 and V2 when compared with the differences
predicted by chance according to the permutation analysis. In areas VO and MT+, the
univariate classifier outperformed the multivariate classifier, but this difference was not
significant.

100
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

mance in Figure 6.6, or the mean classifier performance in cases where the curve did not fit

the data.

CA CC DM EG SM
100 24 100 538 100 155 100 98 100 232

V1
50 50 50 50 50
3 835 3 582 3 838 3 909 3 987
100 244 100 13 100 100 102 100 56

V2 177
50 50 50 50 50
3 998 3 758 3 860 3 1213 3 861
100 33 100 100 13 100 35 100 30
V3
50 50 50 50 50
3 682 3 814 3 785 3 557 3 620
100 127 100 6 100 40 100 100 37
V3A
50 50 50 50 50
3 385 3 740 3 297 3 192 3 216
100 100 100 100 100 96
hV4
50 50 50 50 50
3 117 3 542 3 426 3 245 3 267
100 100 30 100 100 100
VO
50 50 50 50 50
3 232 3 447 3 297 3 161 3 284
100 100 100 100 100
MT+
50 50 50 50 50
3 8 3 31 3 7 3 32 3 23

Number of voxels included in classifier (log scale)

Figure 6.7: Multivariate classifier performance as a function of the number of voxels included in
the classifier, for each area and each subject. In each plot the filled blue line plots the
classifier performance, and the filled red line plots the best-fitting exponential growth
function, given by Equation 6.3. Where the exponential growth function fitted the data,
the limit of this function was taken as the classifier performance in that area for that sub-
ject; the number of voxels taken to reach this limit is given by the red number on these
plots. Where the exponential growth function did not fit the data (usually when perfor-
mance was low), the mean was taken as the classifier performance, which is plotted as a
dashed red line.

101
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

Multivariate classification performance across areas (see Figure 6.6) followed a similar trend

to that found for the univariate classifiers; earlier visual areas (V1, V2 and V3) tended to

outperform V3A/B, hV4 and VO. Classification accuracy was poorest in MT+, where per-

formance was not significantly different from chance. This trend was also found when the

classifier was trained and tested on one hundred voxels, randomly chosen from those voxels

in the area that responded to the localiser stimulus: the average between-subjects classifier

accuracy in V1, V2 and V3 (63, 67 and 59%) was still better than in V3A/B, hV4 and VO (57,

56 and 49%). Classification performance was generally higher for multivariate classifiers,

and this difference was significant (p   0.01, two-tailed permutation test) for areas V1 and
V2. In VO and MT+, the univariate classifier outperformed the multivariate classifier, but

this difference was not significant.

Since there was no requirement on the classifiers to predict an equal number of orange-cyan

and lime-magenta blocks, it was possible for classification performance to be better for one

type of test stimuli: for example, if classification of orange-cyan test stimuli was at chance

but classification performance on lime-magenta blocks was perfect, giving an overall per-

formance of 75%. I found that this was not the case; for both univariate and multivariate

classifiers, classification performance for lime-magenta test stimuli and classification perfor-

mance for orange-cyan stimuli showed a positive linear correlation (univariate: slope  0.33,

R2  0.13, p   0.05, multivariate: slope  0.85, R2  0.61, p   0.01).

Increased performance of the multivariate classifier compared with the univariate classifier

in V1 and V2 indicates that there are reliable, stimulus-related patterns of activity in these ar-

eas. If the pattern of activity across voxels was uninformative about the non-cardinal colour

of the stimulus, the multivariate classifier performance would be expected to be at best the

same as the univariate case (since if the classifier learnt a pattern that was not stimulus-

related, performance could decrease).

102
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

6.4 Discussion

I found evidence in human visual cortex for representations of colour as early as V1 that

combine information from the L-M and S-(L+M) opponent pathways hypothesised to carry

information in parallel from sub-cortical areas to cortex. The ability to use BOLD activity

to discriminate stimuli matched for the postulated sub-cortical mechanisms demonstrates

that the neural population must include neurons modulated by signals from both the chro-

matically opponent pathways. Below I discuss the implications of these results for how

colour is represented in human visual cortex, in particular the bias for lime-magenta over

orange-cyan stimuli, and differences between visual areas in classifier performance.

6.4.1 Origin of asymmetry in the representation of two non-cardinal colour mod-

ulations

I found a common bias across cortical visual areas for lime-magenta over orange-cyan blocks,

even though the stimuli were matched for cone contrast, and for the response of sub-cortical

pathways. The consistency of this bias across subjects suggests that it reflects a typical asym-

metry in cortical representations of colour. Specifically, this finding implies that there is a

more numerous or more active population of neurons which respond to lime and/or ma-

genta than to orange and/or cyan stimuli.

There is some evidence for a bias in the opposite direction in single-unit recordings in

macaque V1, and from human psychophysics. Both Conway (2001) and Solomon & Lennie

(2005) found a bias when testing the responses of macaque V1 cells to L, M and S-cone

isolating stimuli. Of the 45 (Conway, 2001) and 19 (Solomon & Lennie, 2005) L-M colour

opponent cells that also responded to S-cone isolating stimuli, for 93% and 89% of cells

(respectively) the response to the S-cone isolating stimulus had the same sign as the M-

cone isolating stimulus; that is, the cells preferred a colour direction closer to orange-cyan

than lime-magenta. Krauskopf & Gegenfurtner (1992) report subtle psychophysical asym-

metries for human observers between the non-cardinal axes in the effects of adaptation on

discrimination threshold. Their data are consistent with a greater prevalence of adaptable

103
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

mechanisms tuned to orange-cyan than to lime-magenta. The data imply a bias of the op-

posite direction in human visual cortex, suggesting further work is necessary to reconcile

these findings. In a recent study on human discrimination thresholds, Danilova & Mollon

(2010) found a pattern of discrimination thresholds which may point to the presence of a

colour channel in which M-cone signals are opposed by a combination of L- and S-cone sig-

nals. This hypothetical channel would lie closer to the lime-magenta modulation than the

orange-cyan modulation, and could be a psychophysical correlate of the bias I observed in

the fMRI data.

6.4.2 Organisation of colour processing in early human cortical areas

While significant classifier performance indicates representations of non-cardinal colours,

differences in classifier performance between areas are difficult to interpret. Brouwer &

Heeger (2009) found highest classifier performance in V1, yet their principal components

analysis suggested that the representation of colour in hV4 and VO more closely matches

our perceptual experience. Classifier performance depends not only on the presence of rel-

evant information (here, non-cardinal representations of colour) within an area, but also on

the accessibility of this information at the coarse spatial scale of the functional measure-

ments.

For areas V1 and V2, multivariate classifiers significantly outperformed univariate classi-

fiers, showing that there was stimulus related information in the pattern of activity. In

macaque V1 and V2 there are orderly maps of hue selectivity (including both cardinal and

non-cardinal colours) across the surface of the cortex (Xiao et al., 2003, 2007). If similar

maps exist early in human visual cortex, their existence may increase the chance of biased

sampling of chromatic preferences across voxels. The size of these hue maps, which each

represent a large spectrum of hues, is only around 200 µm across the surface of the cortex

in macaque V1, with individual maps separated by around 400 µm (Xiao et al., 2007). If

maps of approximately the same size exist in human V1, a single voxel would contain ap-

proximately 6 hue maps. It is unlikely that a single voxel would sample neurons whose

preference included only a narrow range of hues, but it could be that this map arrangement

104
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

would make it more likely for biases between voxels to arise from anisotropic sampling of

these maps. Furthermore, in macaque, the hue maps in V2 are on average 2 to 2.5 times

longer than the hue maps in V1 (Xiao et al., 2007). Larger maps with the same voxel res-

olution should increase the likelihood of biased sampling of hue maps, and increase the

magnitude of the biases, which could underlie the tendency in the results for classifiers in

V2 to outperform classifiers in V1.

Alternatively, it is possible that the stimulus related information in the pattern of activity

is not due to qualitatively different patterns of response for lime-magenta and orange-cyan

but instead to a single pattern of visually responsive voxels that respond more strongly

in the lime-magenta case. Since there is a univariate bias, the multivariate classifier could

potentially be learning the difference between a strong signal in noise and a weaker version

of the same signal (also in noise). The increased performance of the multivariate versus

univariate classifiers might then be based purely on the ability of the multivariate classifiers

to ascribe more weight to individual voxels on the basis of their signal-to-noise ratio. Further

empirical and theoretical work will be required before it is possible to discriminate with

certainty between these alternatives.

Response of dorsal visual areas

Poor classifier performance in MT+ is consistent with the classifier results of Brouwer &

Heeger (2009), as well as evidence from MT of rhesus monkey (Dubner & Zeki, 1971; Britten

et al., 1992) and human MT+ (Zeki et al., 1991; Tootell et al., 1995; Huk et al., 2002) that this

area is not generally selective for the colour of surfaces and is less responsive to chromatic

than achromatic stimuli (Gegenfurtner et al., 1994; Wandell et al., 1999; Liu & Wandell, 2005),

although sensitivity to chromatic motion (Wandell et al., 1999; Barberini et al., 2005) has

been reported. Likewise, the reduced classifier performance in V3A/B with respect to V1,

V2 and V3 may reflect the general preference of this dorsal area for motion (Tootell et al.,

1997), and its reduced responsivity to chromatically defined stimuli (Liu & Wandell, 2005).

Nevertheless, for each subject, MT+ had fewer voxels than any other area I defined, which

alone may account for decreased classifier performance.

105
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

Response of ventral visual areas

Areas hV4 and VO are often thought to be specialised for the processing of colour. In

macaque V4 evidence from both single unit recordings (Zeki, 1983) and from neuroimaging

(Conway & Tsao, 2006) implicates this area as a ‘colour centre’. In human, there is converg-

ing evidence from patients with cerebral achromatopsia (Zeki, 1990) and from PET (Lueck

et al., 1989) and fMRI (Liu & Wandell, 2005; McKeefry & Zeki, 1997; Hadjikhani et al., 1998;

Bartels & Zeki, 2000; Wade et al., 2002; Mullen et al., 2007) neuroimaging studies that both

hV4 and VO are involved in colour vision. Additionally, there is evidence that the response

properties of VO match colour perception in showing weaker responses to high than to low

temporal frequencies (Liu & Wandell, 2005; Jiang et al., 2007), while V1 does not. It therefore

might be expected that classifier performance would be greatest in these areas, but this was

not the case for the stimuli used here, or for more perceptually relevant hues (Brouwer &

Heeger, 2009). I consider five possible reasons for this.

First, the definitions of hV4 and VO used here may not include the region of ventral visual

cortex that is specialised for colour processing; Hadjikhani et al. (1998) reported colour se-

lectivity in area V8, but not hV4. I think this account of the results is unlikely because the

definition of hV4 used here, corresponding to that of Wandell et al. (2007), would include

part of the V8 described by Hadjikhani et al. (1998), which showed colour selectivity, with

the remainder of their V8 corresponding to VO in this study.

Second, areas hV4 and VO may be more susceptible to task specific demands than other

areas. I both asked subjects not to attend to the stimuli and required them to engage with

a task at fixation. By diverting attention from the experimental stimulus I aimed to avoid

artifactual classifier performance that was based not on differences in the stimulus-driven

response but on differences in attention between the two conditions. However, when atten-

tion is directed to a task unrelated to the stimulus, the stimulus-driven BOLD response is

suppressed, and this suppression increases with the attentional load of the task (Rees et al.,

1997; Schwartz et al., 2005). Single-unit recordings in macaque and fMRI in humans show

that V4 and hV4 show greater attentional modulation than earlier visual areas (Reynolds &

106
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

Chelazzi, 2004; Schwartz et al., 2005; Hansen et al., 2007a).

Third, reduced performance of multivariate classifiers in hV4 and VO may result if the voxel

size (1.5 mm each side) resulted in biases when sampling neural representations of colour

in V1 and V2, but not hV4 or VO. The spatial arrangement of chromatic preferences may be

either less ordered, ordered in a different way, or ordered on a smaller spatial scale than in

the earlier visual areas.

Fourth, any nonlinearity in the signal which differs between areas may enhance or reduce

the stimulus discriminability in the BOLD response. For example, the contrast response

of VO to L-M and S-(L+M) modulating stimuli is more nonlinear than V1 (Liu & Wandell,

2005). It is not clear how such nonlinearities should affect classifier performance, but it is

possible that the reduced classifier performance in ventral areas was due to such nonlinear-

ities.

Finally, there may be increased noise in the imaging results for these areas, which are typ-

ically located further from the surface of the head than the earlier visual areas and dorsal

areas, and near the transverse sinus, which can cause imaging artifacts (Winawer et al.,

2010). I think it likely that the reduced performance of the classifiers in hV4 and VO reflects

some combination of the impact of attention, nonlinearities, imaging noise and (for multi-

variate classifiers) the spatial arrangement of colour processing within these areas, rather

than implying their diminished selectivity for colour.

6.4.3 Limitations of this study

All these conclusions are based on the assumption that the stimuli induced responses in

sub-cortical pathways that are indistinguishable when considering the responses of each

pathway independently. I consider a number of reasons why this assumption may be in-

valid.

Macular pigmentation selectively attenuates shorter wavelengths in the central two degrees

of the visual field (Wyszecki & Stiles, 1967; Hammond et al., 1997). When defining the stim-

uli I used the Stockman & Sharpe (2000) 2 degree cone spectral sensitivities, which take into

107
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

account the impact of macular pigmentation. Since macular pigmentation does not extend

beyond the central 2 degrees of visual field (Hammond et al., 1997; Stringham et al., 2006),

it is possible that, for the region peripheral to this, the stimuli were no longer balanced for

responses they induce in the sub-cortical pathways. To rule out this potential artifact, I re-

peated the classifier analysis, limiting the voxels included in the classifier to those within

2 degrees visual angle from fixation and thus excluding any voxels which respond to an

area of visual field for which the stimuli may not have been balanced. With this analysis,

I found classifier performance was reduced, but remained significantly above chance in all

areas except MT+ (data not shown). This rules out the possibility that classifier performance

in the original analysis was based on artifacts in the stimuli caused by macular pigmenta-

tion.

It is important also to emphasise the robustness of these conclusions to any inaccuracies in

the determination of subjective equiluminance for each subject. For example, let us con-

sider the situation if there were a 1% artifactual modulation in the lime-magenta blocks

and no artifact in orange-cyan blocks. The 25% luminance modulation was added in op-

posite phases for different lime-magenta blocks, meaning that the effect of the luminance

artifact would be to increase effective luminance contrast to 26% in half of the lime-magenta

blocks and decrease it to 24% in the other half. For such a bidirectional effect to introduce

a bias in the univariate response between lime-magenta and orange-cyan blocks, the con-

trast response function in the vicinity of 25% luminance contrast would have to be highly

non-linear. Furthermore, any such bias would be unlikely to show the consistency between

subjects observed here. In terms of the mulitvariate analysis, a classifier would have to learn

a disjunctive discrimination between (24 or 26%) vs (25%) modulation in luminance in order

to be able to classify the stimuli based on luminance alone. The use of linear classifiers min-

imises the possibility that luminance artifacts were used as a cue to discrimination.

More fundamentally, it could be that the classic model of two chromatically-opponent sub-

cortical channels is insufficient to capture the selectivity of all neurons in the lateral genic-

ulate nucleus (LGN). The majority of observed LGN responses can be accounted for by the

classic model, but some evidence suggests that the chromatic responses of the LGN may

108
 Chapter 6. Decoding non-cardinal colours from activity in human visual cortex

not be fully described by the two channels. It is difficult to investigate physiological cor-

relates of the higher-order colour mechanisms in the LGN, since these mechanisms have

primarily been revealed by psychophysical habituation (Krauskopf et al., 1986), and there

is little or no habituation of cells in the LGN, yet these cells may contribute to the colour

tuning of the adaptable cortical cells (Tailby et al., 2008a). Signals tuned to colour directions

away from the two opponent mechanisms have also been proposed (Zaidi & Shapiro, 1993;

Webster & Mollon, 1991, 1994) that could theoretically be inputs to cortical areas. Finally

Tailby et al. (2008b) recently reported that in macaque LGN, a subset of neurons responded

to modulations along both the S-(L+M) and L-M axes. The present study does not address

the question of whether the model of two chromatically opponent sub-cortical channels is

sufficient to describe the coding of colour by the LGN. I plan to investigate this possibility

further in future investigations using human fMRI.

109
Chapter 7

Selective adaptation to surface

colour

The work described in this chapter is due to be published in the Journal of Vision (Goddard

et al., In Press), and the paper is included in Appendix B.

7.1 Introduction

7.1.1 Colour constancy

To identify the colour of surfaces in a scene correctly, we must separate out the reflectance

properties of surfaces from the spectral properties of the light illuminating the scene, an

ability called ‘colour constancy’. Humans generally perform well, though imperfectly, when

asked to identify surface properties under different illumination conditions (Helson, 1938;

Judd, 1940; Worthey, 1985; Craven & Foster, 1992; Brainard et al., 1997).

Many higher order scene features may be used as cues when accomplishing colour con-

stancy, including specular highlights (Lee, 1986), interreflections (Bloj et al., 1999), lumi-

nance colour correlations (Golz & MacLeod, 2002), colour memory (Hansen et al., 2006) and

Gestalt laws (Gilchrist et al., 1999). Such cues are likely to require interactions with higher

110
 Chapter 7. Selective adaptation to surface colour

order object representation, abstract colour knowledge and lexical representation of colour

names. For example, neuropsychological literature points to a dissociation between the cod-

ing of colour object knowledge and lexical coding of colour names, and shows that both

these abilities can be impaired in patients with normal colour vision (Beauvois & Saillant,

1985; Luzzatti & Davidoff, 1994, for review, see Tanaka et al., 2001).

7.1.2 Neural mechanisms for colour constancy

That colour constancy is accomplished requires that surface colour is neurally encoded, but

the stage at which that is done remains unclear. Receptoral mechanisms, particularly multi-

plicative scaling (Ives, 1912; Land & McCann, 1971; Shapley & Enroth-Cugell, 1984; Brainard

& Wandell, 1992; Foster & Nascimento, 1994), can account for a large proportion of colour

constancy (Smithson, 2005). But retinal mechanisms alone are insufficient to account for all

of our colour constancy (Brill & West, 1986; Worthey & Brill, 1986).

Beyond the retina it is less clear what neural mechanisms might underlie colour constancy.

In occipital cortex, a potential candidate is area V4. In non-human primates, area V4 has

been argued to contain a neural representation of surface reflectances (Zeki, 1983; Wild et al.,

1985; Kusunoki et al., 2006), but the question of whether V4 is unique in these properties re-

mains controversial (de Monasterio & Schein, 1982; Conway & Tsao, 2006; Gegenfurtner,

2003). In humans there is less evidence that hV4 (the suggested homologue of V4) is a spe-

cialised ‘colour centre’ (Gegenfurtner, 2003), and no work specifically shows colour constant

response properties in hV4 or other areas.

Finally, the higher order nature of many cues thought to contribute to colour constancy and

the neuropsychological literature suggest the involvement of cognitive areas beyond occip-

ital cortex, but how these influences are implemented neurally remains ill-defined.

Models of colour constancy typically imply a processing stage in visual cortex that trans-

forms photoreceptor activations into a representation of surface colour (Brainard & Free-

man, 1997; Lennie, 1999; D’Zmura & Lennie, 1986; Logvinenko & Maloney, 2006; Robilotto

& Zaidi, 2004). Yet the large proportion of colour constancy accomplished at the level of

111
 Chapter 7. Selective adaptation to surface colour

the retina, combined with the cognitive nature of many cues thought to contribute to colour

constancy, leaves open the question whether a neural representation of surface colour in-

termediate to the retinal and cognitive stages exists. Here I sought to address this question

using a colour opponent aftereffect.

7.1.3 Colour opponent aftereffects

Prolonged viewing of a field of one colour induces a colour opponent aftereffect. For ex-

ample, after viewing a red field, a neutral grey appears tinged with green. This effect is

mediated predominantly by mechanisms at a low level: photoreceptors reduce their respon-

siveness over a period of prolonged stimulation, changing the response of post-receptoral

mechanisms to subsequent stimuli, and shifting the appearance of those stimuli away from

the colour of the adapting stimulus. When adapted to red, the photoreceptor response to a

neutral grey is the same as the response to green under neutral adaptation.

Here I used prolonged viewing of two types of stimuli to investigate the aftereffects of adap-

tation to surface colour. I used stimuli that induce the same amount of receptoral adaptation,

but which evoke different percepts of surface colour. I will show an aftereffect that cannot

be wholly attributed to adaptation of receptors and neural mechanisms responsive to raw

quantal catch. I attribute this effect to the adaptation of neural mechanisms that encode

surface colour in a manner more robust to illuminant changes than the encoding by retinal

mechanisms, located beyond the retina and most likely beyond other subcortical areas, but

before cognitive areas.

7.2 Methods

7.2.1 Colour calibration procedures and display system

Stimuli were generated and displayed using Matlab (version 7) software, with routines from

PsychToolbox (Brainard, 1997; Pelli, 1997), on a Dell OpliPlex 755 computer driving an ATI

Radeon HD 2400 Pro graphics card (8 bits per channel) to draw stimuli to a 33 x 24 cm Sony

112
 Chapter 7. Selective adaptation to surface colour

Trinitron E220 cathode ray tube monitor, refreshed at 85 Hz. Experiments took place in a

darkened room with black walls, and the monitor was viewed from a distance of 0.57m.

The monitor was calibrated using a ColorCAL colorimeter (Cambridge Research Systems

Ltd.). Changes in both chromaticity and luminance of the screen with increasing R, G and

B values were taken into account when generating the experimental stimuli. The CIE (xyY)

coordinates measured for 16 values during calibration were interpolated to 255 values using

the best-fitting spline, and these were used to calculate the xyY coordinates of each combina-

tion of R, G and B intensity values, resulting in a 256 x 256 x 256 matrix of xyY coordinates.

Each xyY coordinate was transformed into an LMS cone excitation coordinate according to

the method described in Brainard (1996), using the Stockman & Sharpe (2000) 2-degree cone

spectral sensitivity functions. Stimuli were specified in terms of their LMS cone excitation

coordinates and the closest RGB value was found by finding the lowest root mean square er-

ror between the target cone excitation and the values achievable on the monitor. Where the

lowest root mean square error was greater than 0.01, or the best fitting RGB value included a

channel at maximum intensity, the stimulus was considered to be out of range and excluded.

For a more detailed description of the calibration process, refer to Appendix A.

7.2.2 Observers

Ten observers (five male), aged 20 to 35 years old, took part; eight were naïve to the purposes

of the investigation (all observers except the authors EG and SS). All had normal or corrected

to normal visual acuity, and normal colour vision as assessed using Isihara (Ishihara, 1990)

and the Hardy-Rand-Rittler (HRR, 4th edition, published by Richmond Products) pseu-

doisochromatic plates. Observers provided informed consent, and the entire study was

carried out in accordance with guidelines of the Human Research Ethics Committee of the

University of Sydney.

113
 Chapter 7. Selective adaptation to surface colour

7.2.3 Adapting stimuli

Adapting stimuli simulated either a constant scene under changing illumination, or a chang-

ing scene under constant illumination, as illustrated schematically in Figures 7.1 and 7.2. In

the constant scene condition the stimulus simulated an array of flat, matte surfaces under

diffuse illumination that varied sinusoidally between two different illuminants at 1 Hz. In

the changing scene condition, the same surfaces, illuminants and rate of illuminant change

were used as for the constant scene condition, but the phase of the illuminant modulation

was randomised across the surfaces. The two conditions were identical when averaged

over time; across each cycle of illumination change (1 second), they had the same surfaces

under the same local illumination. The adapting conditions differed only in whether the

illuminant modulation was the same for all surfaces (the ‘correlated’ condition, Figures 7.1

and 7.2, A) or had a different randomly chosen phase for each surface (the ‘decorrelated’

condition, Figures 7.1 and 7.2, B).

Rendered surfaces were randomly drawn from 184 surface reflectance functions supplied

by Hannah Smithson (Smithson & Zaidi, 2004). These were derived from a series of mea-

surements of natural and man-made objects (Chittka et al., 1994; Vrhel et al., 1994; Hiltunen,

1996; Marshall, 2000), including a broad range of colours. The two illuminants were an in-

candescent bulb, and daylight (CIE Standard Illuminants A and D65, respectively), and were

scaled such that they had approximately equal photopic luminance, using the Stockman &

Sharpe (2000) 2-degree luminosity function.

Three-thousand ellipses were drawn at random locations within the stimulus field and each

was rendered with one of the surfaces. The heights of the ellipses ranged from 0.03 to 1.3

degrees visual angle, with the probability of each height h inversely proportional to h3 . The

width of each ellipse varied randomly between 1{3h and 3h. Overlaid on this array of sur-

faces were the circular adapting stimuli, located to the left and right of fixation, with di-

ameters subtending 8 degrees. The total arrangement was a rectangle subtending 30 x 16

degrees, and the remainder of the screen was black.

For each observer, the adapting stimuli were either green to the left and red to the right

114
 Chapter 7. Selective adaptation to surface colour

Cone Fundamental Macleod-Boynton

Primaries Chromaticity Coordinates
Adapting
Illuminant Surface L M S (L-M)/(L+M) S/(L+M)

Red 0.027 0.015 0.008 0.742 0.361

Green 0.022 0.023 0.008 0.601 0.362
CIED65
Blue 0.023 0.022 0.019 0.624 0.801
Yellow 0.023 0.022 0.015 0.624 0.647

Red 0.026 0.011 0.002 0.782 0.111

Green 0.021 0.018 0.002 0.652 0.115
CIE A
Blue 0.022 0.017 0.005 0.674 0.252
Yellow 0.022 0.017 0.004 0.674 0.204

Table 7.1: Adapting Stimuli Chromaticities. The cone fundamental primaries (LMS) and Macleod-
Boynton chromaticity coordinates ((L-M)/(L+M) and S/(L+M)) are shown for each of the
adapting surfaces, under both illuminants. Over time, the illumination of the adapting
stimuli modulated sinusoidally between CIE Standard Illuminant D65 (daylight) and CIE
Standard Illuminant A (an incandescent bulb).

of fixation, or yellow-green to the left and blue to the right of fixation; chromaticities of all

adapting stimuli under both illuminants are shown in Table 7.1. Adapting surfaces were

rendered under the same illumination as the background surfaces; in the ‘correlated’ condi-

tion the modulation was in the same phase for each of the two adapting surfaces and for the

background, whereas for the ‘decorrelated’ condition the modulations of the two adapting

surfaces were perfectly out of phase with each other. In the correlated condition, the illu-

minant modulation always commenced and finished with the daylight (CIE Standard Illu-

minant D65) illumination, while the onset phases of the illuminant modulation for surfaces

in the decorrelated case were randomly drawn from all points in the cycle. In the decorre-

lated condition, the adapting surface on the left commenced and finished with CIE Standard

Illuminant A illumination and the adapting surface on the right commenced and finished

with CIE Standard Illuminant D65. Each adaptation run began with a minute of adaptation

before the first trial, and there were 6 seconds (6 stimulus cycles) top-up adaptation before

each subsequent trial.

115
 Chapter 7. Selective adaptation to surface colour

Figure 7.1: Example stimuli for the ‘Correlated’ and ‘Decorrelated’ conditions, with fixation cross.
Adapting stimuli consisted of up to 3000 ellipses with colours that simulated flat matte
surfaces under diffuse illumination. Over time, the illumination sinusoidally modulated
from an incandescent bulb (CIE Illuminant A) to daylight (CIE Illuminant D65). Across
one cycle of this illumination change, the two adapting stimuli had the same surfaces
under the same illumination. The difference between the two conditions was that the
changes of different surfaces were either correlated with each other (A) or decorrelated,
with random onset phases (B). In C, example frames of the test stimuli are shown. On
the left is an example frame with the red reference to the right of fixation ((L-M)/(L+M)
chromaticity coordinate of 0.67). On the right is an example frame with the green refer-
ence to the left of fixation ((L-M)/(L+M) chromaticity coordinate of 0.59).

116
 Chapter 7. Selective adaptation to surface colour

Figure 7.2: Example stimuli for the ‘Correlated’ and ‘Decorrelated’ conditions, with blue-yellow
adapting stimuli. Conventions as in Figure 7.1. In C, example frames of the test stimuli
are shown. On the left is an example frame with the blue reference to the right of fixation
(S/(L+M) chromaticity coordinate of 0.45). On the right is an example frame with the
yellow reference to the left of fixation (S/(L+M) chromaticity coordinate of 0.10).

117
 Chapter 7. Selective adaptation to surface colour

7.2.4 Test stimuli and observer’s task

I measured the perceived chromaticity of test surfaces in each of three conditions: prior to

adaptation, and during adaptation to correlated and decorrelated adapting stimuli. On each

trial, the period of top-up adaptation was followed by a test stimulus, where the two adapt-

ing stimuli were replaced by a test and a reference surface. If the observer failed to respond

within 3 seconds of the test stimuli being presented, the display returned to another 6 sec-

onds of top-up adaptation before presenting the same trial again. The spatial arrangement

of background surfaces remained the same as they were during the adapting stimulus, and

the illumination of the entire scene was CIE Standard Illuminant D65. For the condition

without adaptation, all stimulus and timing parameters were the same as for the adapting

stimuli, with the exception that the initial adaptation period and the top-up adaptation peri-

ods were omitted; one test stimulus was replaced immediately by the next, meaning that the

total duration of this condition was approximately a quarter the duration of the adaptation

conditions.

Response times did not vary greatly between conditions; the median reaction times in each

condition, averaged across subjects (+/- 1 standard deviation), were 1.18 s (+/- 0.32 s) in the

correlated adaptor condition, 1.15 s (+/- 0.32 s) in the decorrelated adaptor condition, and

1.25 s (+/- 0.47 s) in the no adapt condition.

In conditions in which observers were adapted to red and green, they were asked to respond

with a button press to indicate whether the surface to the left or right of fixation appeared

‘more red’ (observers were instructed to treat ‘more red’ and ‘less green’ as equivalent).

I estimated the chromaticity of the test which had perceptual equality with the reference

(the test that was equally likely to be reported more red or more green than a reference)

during two sessions, each of which was made up of eight interleaved adaptive psychophys-

ical staircases (Kontsevich & Tyler, 1999) consisting of 30 trials each. For each staircase the

test surface had a fixed S/(L+M) chromaticity coordinate and position (either S/(L+M) =

0.24 or 0.71, located to the left of fixation, or S/(L+M) = 0 or 0.47, located to right), while

the (L-M)/(L+M) chromaticity coordinate was set by the adaptive staircase. On each trial,

118
 Chapter 7. Selective adaptation to surface colour

the reference surface located on the opposite side to the test had the same S/(L+M) chro-

maticity coordinate as the test surface. Its (L-M)/(L+M) chromaticity coordinate was 0.59

(green, when the reference was on the left and the test was on the right) or 0.67 (red, when

the reference was on the right). I shifted the chromaticity of the reference stimuli towards

the adapting chromaticity at that location in order to ensure that observers’ point of equal

redness/greenness was within the gamut of the monitor. I introduced this shift after a pilot

experiment with the same reference stimuli on the left and right where the adaptive staircase

reached the end of the range of the monitor before finding a chromaticity that observers re-

ported to be ‘more red’ 50% of the time. Ten randomly generated spatial arrangements of the

background surfaces were interleaved throughout the run; the adaptation and test phases

of each trial were formed from the same configuration.

The data from each staircase (30 trials) were used to make one estimate of the chromaticity of

the test surface that the observer was equally likely to report as more red or more green than

the reference. The proportion of ‘more green’ responses (Ŷ) as a function of test chromaticity

(x) was fitted using the logistic equation:

1
c
Ŷ  c{2 (7.1)
1 e
p x
aq.{b

where a is the test chromaticity of perceptual equality with the reference (the value of x

where Ŷ=0.5), b is a curvature parameter, and c is the miss rate. Example data from 2 stair-

cases for one observer, along with fitted curves, are shown in Figure 7.3.

Four observers adapted to red and green surfaces, and two of these observers along with

an additional four observers adapted to blue and yellow surfaces. The design for blue and

yellow surfaces was equivalent: observers reported whether the left or right surface was

‘more blue’ (equivalent to ‘less yellow’). Test surfaces had a fixed (L-M)/(L+M) chromatic-

ity coordinate and position (either (L-M)/(L+M) = 0.62 or 0.72, located to the left of fixation,

or (L-M)/(L+M) = 0.57 or 0.67 located to right), while the S/(L+M) chromaticity coordinate

was set by the adaptive staircase. The reference surface had the same (L-M)/(L+M) chro-

maticity coordinate as the test surface, and had an S/(L+M) chromaticity coordinate of 0.10

119
 Chapter 7. Selective adaptation to surface colour

(yellow, when reference was on left and test on right) or 0.45 (blue, when reference was on

right).

7.3 Results

7.3.1 Opponent aftereffect for surface colour

I measured the shift in perceived chromaticity of a test surface while subjects were adapted

to red and green, or blue and yellow surfaces, using a two alternative forced choice paradigm

(see Methods for details).

Figure 7.3: An example pair of psychometric curves show the adaptation-induced shifts in chro-
maticity of the test surface during adaptation to green, for one naïve observer, JC. Pos-
itive values along the abscissa are test surfaces that are more green than the reference.
For both conditions the curves are shifted to the right, indicating a colour-opponent af-
ter effect. Any extra shift in the ‘correlated’ condition cannot be attributed to receptoral
adaptation, since when averaged over time the two conditions contain the same LMS
cone contrast.

Figure 7.3 shows for one observer how the appearance of a single test surface changed with

adaptation to a green surface. The magnitude of this change in appearance depended on

the context of the green adapting surface, which varied with adaptation condition. Without

adaptation, the chromaticity of subjective equality with the reference was nearly veridical

(the point of subjective equality was 0.0012, and for clarity that curve is not shown). During

adaptation to the decorrelated stimulus the curve shifted substantially to the right - the

observer required the test surface to be greener than the reference to see it as the same.

120
 Chapter 7. Selective adaptation to surface colour

This shift is expected because the adaptor changes the state of the receptoral and neural

mechanisms that lie over the test surface. If there were only adaptation of mechanisms

responsive to raw quantal catch, then the aftereffect induced during the correlated condition

should be of the same magnitude. It was not: instead the psychometric function shifted

further to the right.

The magnitude of the aftereffect in the two adaptation conditions for individual observers

is shown in Figure 7.4. In Figure 7.5 the results are summarised; the average difference be-

tween the conditions for each adapting colour is expressed as a percentage of the shift in the

decorrelated case. For each adapting colour, the perceived chromaticity of the test surface

shifted away from the colour of the adapting surface, consistent with a colour opponent

aftereffect. As in the example in Figure 7.3, the colour opponent aftereffect was usually of

greater magnitude in the correlated condition than in the decorrelated condition.

Statistical analysis

For the statistical analysis I excluded the data collected without adaptation and for each

adapting colour (red, green, blue and yellow) performed a 2-way analysis of variance. In

each 2-way analysis of variance, I included observer as a random effect and tested for a main

effect of adaptation condition (correlated versus decorrelated). For adaptation to red, green,

and blue the correlated adaptor induced an aftereffect of significantly greater magnitude

(red: p   0.01, F1,3  30.19, green: p   0.05, F1,3  27.30, blue: p   0.05, F1,5  10.20). For
adaptation to yellow, the correlated adaptor induced an aftereffect of greater magnitude, but

this difference was not significant (p  0.20, F1,5  2.03).

7.3.2 Effect cannot be explained by frame-by-frame differences in spatial cone

contrast

The correlated and decorrelated adaptors above have identical distributions of colours over

space and time, but within any frame of the stimulus the spatial variance in chromaticities is

lower in the correlated condition. Brown & MacLeod (1997) demonstrated that colours em-

121
 Chapter 7. Selective adaptation to surface colour

A 0.106
B 1.21
C: Correlated
Adaptation
* Condition
Magnitude of Difference Required for Perceptual Match ( (L-M)/(L+M))

D: Decorrelated

( ( S/(L+M))
Adaptation
Condition

AG

Magnitude of Difference Required for Perceptual Match

** * n.s.
CA

EG
0.071
JC

JK

SM

SS
0.61
TF

** p<0.01
0.035
* p<0.05

0 0
C D C D C D C D
Adapt Red Adapt Green Adapt Blue Adapt Yellow

Figure 7.4: Adaptation induced shifts in perceived chromaticity of a reference surface, after adapta-
tion to red and green (A) or blue and yellow (B). Mean shifts in perceived chromaticity
along the (L-M)/(L+M) (A) and S/(L+M) (B) dimensions are shown for each observer
(points, +/- 1 standard error of the mean) and averaged across observers (grey bars).
The magnitude of shifts in perceived chromaticity of test surfaces are plotted relative to
the physical chromaticity of the reference surface. The direction of these shifts depended
on adaptor colour: when observers adapted to red or blue, the shifts were positive along
the (L-M)/(L+M) and S/(L+M) dimensions respectively; for green and yellow the shifts
were negative. In each case the direction of the shift was consistent with the subject re-
quiring a test surface whose chromaticity was shifted towards the adapting colour for
a perceptual match with the reference. When the adapting surfaces were red, green or
blue, the correlated condition induced an aftereffect of significantly greater magnitude
than the decorrelated condition. When the adapting surface was yellow, the effect went
in the same direction, but the difference between conditions was not significant (see text
for details).

122
 Chapter 7. Selective adaptation to surface colour

40

in Decorrelated Condition
% of Adaptation Effect
20

0
Red Green Blue Yellow

Colour of Adapting Surface

Figure 7.5: Adaptation induced shift in chromaticity: average difference between the correlated and
decorrelated adaptation conditions, as a percentage of the shift in the decorrelated con-
dition. Here data from Figure 7.4 are replotted, averaged across all observers. Error bars
indicate +/- 1 standard error of the between subjects mean.

bedded in a background of greater variance appear less saturated; for the stimuli used here,

this would predict that the adapting stimuli would appear less saturated in the decorrelated

condition, which might diminish the opponent aftereffect. To test this possibility I matched

the correlated and decorrelated conditions for variance in LMS cone excitation coordinates

across individual frames of the adapting stimulus.

To generate adapting stimuli with the same variance, the L, M and S cone excitation coor-

dinates of decorrelated stimulus surrounds were scaled by 1.00, 0.97 and 0.66, respectively,

as illustrated in Figure 7.6. All other stimulus properties were unchanged. Using these

stimuli, I repeated the decorrelated condition for two observers (one naïve) for both the red-

green and blue-yellow adapting stimuli. Individual results are plotted in Figure 7.7, and

summarised in Figure 7.8.

As in the main experiment, the correlated adaptor induced an aftereffect of greater magni-

tude than the low contrast decorrelated adaptor. This difference was not significant when

analysed in the same manner as above, with a 2-way analysis of variance for each adapting

colour (red: p  0.22, F1,1  7.77, green: p  0.26,F1,1  5.31, blue: p  0.13, F1,1  22.41,
yellow: p  0.29, F1,1  4.33), but unlike the main experiment, where each condition was

completed by at least four subjects, only two subjects completed each condition here. Sub-

123
 Chapter 7. Selective adaptation to surface colour

A 0.0224 (+/-0.0086)
B 0.0224 (+/-0.0086)
C 0.0224 (+/-0.0086)
0.06 0.06 0.06

0.04 0.04 0.04

L-cone
0.02 0.02 0.02

0 0 0
0 0.5 1 0 0.5 1 0 0.5 1
0.06 0.0162 (+/-0.0065) 0.06 0.0162 (+/-0.0067) 0.06 0.0162 (+/-0.0065)
0.04 0.04 0.04
M-cone
0.02 0.02 0.02

0 0 0
0 0.5 1 0 0.5 1 0 0.5 1
0.06 0.06 0.06

0.04
0.0098 (+/-0.0036) 0.04
0.0098 (+/-0.0055) 0.04
0.0098 (+/-0.0036)
S-cone
0.02 0.02 0.02

0 0 0
0 0.5 1 0 0.5 1 0 0.5 1
Time (sec) Time (sec) Time (sec)

Figure 7.6: Stimulus statistics for the background surfaces in the correlated illuminant condition
(A), decorrelated illuminant condition (B), and the reduced LMS variance decorrelated
illuminant condition (C). The mean L, M and S values are shown in the top, middle
and bottom plots, respectively, for one cycle of the adapting stimulus. The grey shaded
region is the mean +/- 1 standard deviation, and the black lines plot the maximum and
minimum L, M and S values on each frame. The mean (+/- 1 standard deviation) is also
written in the top right of each plot. The reduced LMS variance decorrelated illuminant
condition (C) was generated by scaling the decorrelated illuminant condition B to have
the same variance of L, M and S values as the correlated illuminant condition A.

jects who completed both experiments generally showed comparable magnitude of illusion

in the decorrelated condition and the low contrast decorrelated condition and in all cases

adaptation to the correlated scene induced an effect of greater magnitude than the low con-

trast decorrelated scene. The difference in magnitude of the average effect here (Figure 7.8)

and in the main experiment (Figure 7.5) therefore can mostly be attributed to inter-subject

variability: some of the subjects with high illusion magnitudes in the main experiment were

not available for this control experiment.

Overall, these results show that the difference in aftereffects seen during adaptation to cor-

related and decorrelated surrounds cannot be accounted for by differences in spatial cone

contrast.

7.3.3 Effect remains when adapting stimuli are matched for onset and offset
phase

In the main experiment, the onset and offset phases of the adapting stimuli varied between

124
 Chapter 7. Selective adaptation to surface colour

A 0.106
B 1.21
C: Correlated
Adaptation
Condition
Magnitude of Difference Required for Perceptual Match ( (L-M)/(L+M))

D: Decorrelated

( ( S/(L+M))
Adaptation
Condition
L: Low Contrast
Decorrelated
Adaptation
Magnitude of Difference Required for Perceptual Match
Condition

0.071 AG

EG

TF

0.61

0.035

0 0
C D L C D L C D L C D L
Adapt Red Adapt Green Adapt Blue Adapt Yellow

Figure 7.7: Adaptation-induced shift of perceived chromaticity during adaptation to red, green,
blue and yellow. Data for observers AG, EG and TF from Figure 7.4 are replotted beside
the adaptation-induced shift found using a low contrast version of decorrelated adapt-
ing stimulus (L). The low contrast decorrelated stimulus was matched to have the same
frame-by-frame L, M and S variance as the correlated adapting stimulus. Conventions
as in Figure 7.4.

125
 Chapter 7. Selective adaptation to surface colour

40

in Decorrelated Condition
% of Adaptation Effect
20

0
Red Green Blue Yellow

Colour of Adapting Surface

Figure 7.8: Adaptation induced shift in chromaticity: average difference between the correlated and
low contrast decorrelated adaptation conditions, as a percentage of the shift in the low
contrast decorrelated condition. Here data from the red-green and blue-yellow condi-
tions (shown in Figure 7.7) are averaged across 2 observers, and across different refer-
ence chromaticities. Error bars indicate +/- 1 SEM.

the correlated and decorrelated adaptation conditions. In the correlated condition, all adapt-

ing surfaces were rendered under the same illuminant as the test surfaces (CIE Standard

D65) at the beginning and end of each adaptation period. In the decorrelated condition,

only the red and blue adapting surfaces were rendered under CIE Standard D65 at the be-

ginning and end of each adaptation period, the green and yellow adaptors were rendered

under CIE Standard A. To test the possibility that this difference between the conditions at

the transition between adaptation and test may have contributed to the differing aftereffects,

I performed an additional control experiment where the adapting stimuli were all rendered

under an equal mixture of the two illuminants at their onset and offset.

For this experiment I repeated a subset of the original experiment for two observers who

completed the original version (one naïve), and for two new naïve observers. I tested the

red-green adaptation condition, using one of the four S/(L+M) coordinates for the test stim-

uli (which was the same as for the adapting stimuli). If the modulation of the illuminant

over time is CIE Standard D65 at 00 phase, CIE Standard A at 1800 phase, and an equal

mixture of the two occurs at 900 and 2700 , then in the original experiment the adaptation in

the correlated condition always commenced at 00 , and in the deccorelated condition the red

and blue adapting surfaces commenced at 00 while the green and yellow adapting surfaces

126
 Chapter 7. Selective adaptation to surface colour

commenced at 900 . In this control experiment, adapting stimuli in the correlated condition

commenced at either 900 or 2700 , and each staircase included an equal number of both types,

randomly interleaved in order. In the decorrelated condition, the two adapting surfaces re-

mained 1800 out of phase with one another, but on each trial one surface commenced at 900

and the other at 2700 . Each staircase in the decorrelated condition included an equal number

of trials where the surfaces commencing at 900 and 2700 were on the left and right, or on the

right and left, respectively. Since each adaptation period included an integer number of cy-

cles, the onsets and offsets of the two adapting surfaces were always an equal combination

of the two illuminants. All other stimulus properties were unchanged. Individual results

are plotted in Figure 7.9, and summarised in Figure 7.10.

In this control experiment, the correlated adaptor induced an aftereffect of greater mag-

nitude than the decorrelated adaptor for both the red and green adapting surfaces. As

for the previous experiments, I performed a 2-way analysis of variance for each adapting

colour, and found that this difference was significant for the red adapting surface (p   0.01,

F1,3  51.33) but not the green (p  0.12, F1,3  4.08). Subjects who completed the original
experiment showed a similar pattern of results in this control condition.

In the original experiment, where the adapting stimuli differed in their onset and offset

phase, it is possible that they induced different degrees of adaptation in a transient, low-

level mechanism. In this control experiment, the onset and offset phases of the adapting

stimuli were always at a point in the cycle where the illuminant was an equal mixture of

the two illuminants. Additionally, for both adapting surfaces and for both adaptation con-

ditions there was an equal number of 900 and 2700 onset/offset phases. Any transient, low-

level mechanism should be adapted to the same extent in the correlated and decorrelated

adaption condition, but the magnitude of the aftereffect induced in the correlated condition

remains greater than that induced in the decorrelated condition. Overall, these results sug-

gest that the finding in the original experiment cannot be accounted for by differences in the

onset and offset phases.

127
 Chapter 7. Selective adaptation to surface colour

C: Correlated
0.106 Adaptation
Condition
Magnitude of Difference Required for Perceptual Match ( (L-M)/(L+M))

D: Decorrelated
Adaptation
Condition
**
AG
n.s. DA

EG
0.071
IM

** p<0.01

0.035

0
C D C D
Adapt Red Adapt Green

Figure 7.9: Adaptation-induced shift of perceived chromaticity during adaptation to red and green,
for correlated and decorrelated adapting surfaces that were matched in their onset and
offset phases (see text for details). Conventions as in Figure 7.4.

128
 Chapter 7. Selective adaptation to surface colour

40

in Decorrelated Condition
% of Adaptation Effect
20

0
Red Green

Colour of Adapting Surface

Figure 7.10: Adaptation induced shift in chromaticity for adapting stimuli that were matched in
their onset and offset phases (see text for details). Bars show the average difference
between the correlated and decorrelated adaptation conditions, as a percentage of the
shift in the low contrast decorrelated condition. Data from Figure 7.9 are averaged
across 4 observers. Error bars indicate +/- 1 SEM.

7.4 Discussion

7.4.1 Implications for representations of surface colour

I have shown that adaptation to scenes consistent with changing illumination induces a

colour opponent aftereffect of greater magnitude than adaptation to stimuli consistent with

a changing scene. These scenes recruit the same adaptation of receptors, and of neural mech-

anisms responsive to raw quantal catch. I interpret this increased aftereffect as evidence of

adaptable neural mechanisms sensitive to the colour of surfaces, independent of illumina-

tion, which are adapted to some extent by the display which simulates a changing scene

(the ‘decorrelated’ condition), and to a greater extent when the display simulates a constant

scene (the ‘correlated’ condition). While this representation of surface colour may not in-

clude all cognitive factors thought to contribute to colour constancy, it must incorporate the

result of mechanisms beyond retinal adaptation, since the stimuli in the two conditions were

matched for the extent to which they would adapt retinal mechanisms.

At a minimum, the neural mechanisms giving rise to colour constancy must include three

broadly defined stages: firstly the receptoral processes which compensate for gross shifts

of light intensity through multiplicative scaling; secondly, an adaptable stage by which the

129
 Chapter 7. Selective adaptation to surface colour

processing of some global stimulus statistics have led to a further discounting of the illu-

minant, providing a more robust representation of surface reflectance; thirdly the cogni-

tive processes, including the influences of object knowledge, language and scene interpreta-

tion.

7.4.2 Location of adaptable surface reflectance detectors

What mechanisms might provide the adaptable representations of surface colour? It is

unlikely that these representations could arise before visual cortex, both because they are

adaptable, and because there must be a separation of surface and illuminant properties be-

yond that accomplished by retinal mechanisms. Neural mechanisms of the LGN important

for colour perception do not appear to habituate (Tailby et al., 2008b; Solomon et al., 2004;

de Valois et al., 1966), implying that adaptation of these mechanisms cannot be responsible

for the increased aftereffect. Furthermore, there is no evidence of neurons in the LGN with

chromatic properties thought to be essential for colour constancy beyond that achieved by

the retina. Specifically, double opponent cells are argued to be necessary for separating sur-

face and illuminant (Lennie & D’Zmura, 1988; Hurlbert, 1996). Double opponent cells have

receptive fields that include two L-M opponent components of opposite sign, which leads to

a preferential response to chromatic edges (Solomon & Lennie, 2007). While they may exist,

no cells with this property have been reported in retina or LGN. Within visual cortex, it is

possible that adaptable mechanisms occur as early as V1, where double opponent cells have

been reported in macaque (Conway, 2001; Conway & Tsao, 2006; Johnson et al., 2001).

7.4.3 Relation to other scission perception

Colour constancy, and the task of separating illuminant and surface colour properties, may

helpfully be considered to lie within the broader category of any perceptual scission of lay-

ers within a scene. For example, I think it likely that mechanisms which underlie colour

constancy are also involved in the perception of transparent layers in a scene (Gerbino et al.,

1990; Anderson & Winawer, 2005; Westland & Ripamonti, 2000). In relation to transparency,

130
 Chapter 7. Selective adaptation to surface colour

the result reported here would imply there is an adaptable representation of at least the low-

ermost layer in the scene. A possibility not explored by this study is that there are separate

representations of different layers in each scene, each of which is independently adaptable.

This would predict separate adaptation to illuminant colour, or transparent surface colour,

along with surface reflectance properties. Such a mechanism would be a neural correlate

of the separate layer representations in intrinsic image models, among others, which pro-

pose that scene properties such as reflectance, illumination and transparency are decom-

posed and represented as separate layers or ‘images’ (Adelson, 2000; Anderson & Winawer,

2008). Future work could examine whether representations of other layers are indepen-

dently adaptable, and if so, whether there is a limit to the number of layers which the visual

system encodes in this way.

7.4.4 Potential limitations of this study

In interpreting the results of this study I have assumed that prolonged viewing of the ‘cor-

related’ and ‘decorrelated’ stimuli should induce the same degree of adaptation in receptors

and neural mechanisms whose responses scale with the output of the receptors. This as-

sumption may be invalid if the stimuli are not matched in the responses they evoke in such

mechanisms. In my results I describe a control experiment where we tested whether the

lower spatial variance in chromaticities in the correlated condition could account for the

increased magnitude of the aftereffect in this case; I found that it could not.

My design also assumes that I am engaging mechanisms that adapt on a timescale of one

second or longer, since the stimuli in the two conditions are only matched for cone con-

trast when averaged over the entire stimulus cycle. If more transient low-level adaptation

were contributing to the effect, the time-dependent differences between the two types of

stimuli will no longer be matched for these mechanisms. In the second control experiment

described in our results, I balanced the onset and offset phases of our adapting stimuli in

order to match the stimuli for any transient low-level mechanisms, and found an effect of

comparable magnitude to that in the original experiment. However, it remains possible that

my results were influenced by other adaptable mechanisms integrating information over

131
 Chapter 7. Selective adaptation to surface colour

a longer period than a transient mechanism, but less than a second, or that give unequal

weighting to stimuli presented in the preceding second. Chromatic adaptation likely in-

cludes many processes with varying time constants (Webster, 1996; Fairchild, 2005). For ex-

ample, Fairchild & Reniff (1995) report psychophysical evidence suggesting that chromatic

adaptation includes mechanisms that adapt over the course of a few seconds, or over many

tens of seconds.

7.5 Conclusions

One problem with approaching the neural mechanisms of colour constancy experimentally

is that it is difficult to separate out responses of neurons to the raw wavelengths, or colour

contrast, from responses that show an additional degree of colour constancy. I have demon-

strated psychophysically that such neural representations exist, and that they are adaptable.

The procedure here reveals intermediate mechanisms in the representation of surface re-

flectance and may therefore be useful for future investigations of colour constancy and ani-

mal models. I hope to use this aftereffect in conjunction with functional imaging to further

elucidate the neural substrates of human colour constancy ability.

132
Chapter 8

General Discussion

In my introductory chapters I broadly outlined what is currently known of the neural mech-

anisms underlying colour vision. I showed that while mechanisms in the retina and lateral

geniculate nucleus have been investigated extensively, relatively less is known of the cor-

tical processing of chromatic information. We know from psychophysical phenomena that

higher-order colour representations must exist, and that there must be neural substrates for

complex colour judgements such as those involved in colour constancy. The aim of my

thesis was to understand further the cortical processes of colour vision.

8.1 Novel Contributions of Presented Work

In the work presented in this thesis I used three lines of research enquiry, each aimed at

further revealing the coding of colour in human visual cortex.

8.1.1 Functional Role and Retinotopic Organisation of Human Area V4

Functional MRI is well suited to testing the retinotopic organisation of visual areas in human

and non-human primate cortex (Wandell et al., 2007). In Chapter 5 I outlined evidence

supporting the ventral hemifield model of human V4 (Larsson & Heeger, 2006) over a model

of V4 homologous with macaque (Hansen et al., 2007a).

133
 Chapter 8. General Discussion

In this experiment I found that human area V4 (defined according to the ventral hemifield

model) responded more strongly to chromatic than achromatic stimuli, a result that was con-

sistent across all observers. Previous studies have found mixed results regarding whether

human area V4 shows a strong response to colour stimuli. One study that did not find any

evidence of human V4 showing a particular responsiveness to colour is that of Mullen et al.

(2007), who report that VO but not V4 prefers isoluminance colours over luminance defined

stimuli of the same cone contrast. However Wade et al. (2008), when using coloured and

achromatic Mondrian stimuli, find that human V4 responds more strongly than V1 to the

addition of chromatic, but not luminance contrast.

The results presented here add to the evidence in favour of human V4 particularly being

responsive to chromatic stimuli. Unlike previous studies, which have predominantly used

simple, often isoluminant stimuli to test for a response to chromatic stimuli, here I used

naturalistic stimuli (movie clips) which were identical except that in the achromatic case,

the attribute of interest (colour) was removed. This approach has the advantage of putting

the human visual system into its normal operating range in terms of, for example, the range

of spatial and temporal frequencies present in the stimuli.

8.1.2 Localisation of Representations of ‘Higher-Order’ Colour

In Chapter 6 I reported the results of an fMRI experiment where I used multivariate pattern

classification analysis to demonstrate that as early as V1, information from the postulated

subcortical L-M and S-(L+M) opponent channels is combined. The results of this experiment

do not distinguish whether this transformation of chromatic information is mediated by

feedforward or feedback mechanisms, but they show that at least an initial transformation

of chromatic information is present in the neural representation in early visual cortex.

This experiment also revealed a consistent bias across subjects and visual areas: the BOLD

response to lime-magenta stimuli was consistently higher than that to orange-cyan stimuli.

As discussed in Chapter 6, there is some evidence for discrimination thresholds (Danilova

& Mollon, 2010) that may be a psychophysical correlate of the bias observed here. There

134
 Chapter 8. General Discussion

are also other results from electrophysiology (Conway, 2001; Solomon & Lennie, 2005) and

psychophysics (Krauskopf & Gegenfurtner, 1992) that are consistent with a bias in the op-

posite direction, suggesting further research is necessary to reconcile these findings. The

reliability of the bias present in the data presented here, when stimuli were closely matched

for cone contrast and the responses they would elicit in subcortical channels, suggests that

fMRI may be suitable for investigating other biases in the cortical colour space relative to

the subcortical representation of colour. This could potentially be of use in testing models of

cortical colour transformations that include predictions about the distribution of selectivities

in cortical representations.

8.1.3 Evidence for Adaptable Neural Representations of Surface Colour

In Chapter 7 I described and measured an aftereffect that cannot be explained by low-level

adaptation alone, and is most parsimoniously explained by the existence of adaptable mech-

anisms that are sensitive to surface colour. This result validates an assumption of many

models of colour constancy, and presents a new aftereffect that may be used to probe sur-

face colour sensitive mechanisms.

In an extension of these psychophysical experiments, I hope to test for adaptable mecha-

nisms sensitive to the colour of the illuminant. If illuminant colour is represented in a sim-

ilar manner to surface colour, there may be a measurable opponent aftereffect of perceived

illuminant colour, on top of the opponent aftereffect to the raw colour signal.

8.2 Future Directions

8.2.1 Future Research Possibilities Arising from Presented Work

The work I have presented in this thesis presents new experimental questions and offers

insight into experimental approaches that may be fruitful in the future. In particular, the

non-cardinal colour experiment demonstrates that the BOLD response can be used to de-

tect stimulus-related information in cortex for stimuli that are closely matched both in the

135
 Chapter 8. General Discussion

responses they would evoke in the cones and in subcortical mechanisms, even with stimuli

containing an unrelated luminance modulation. This sensitivity suggests that fMRI can be

used to answer subtle questions about cortical responsiveness to colour. The approach of

using naturalistic stimuli with and without one critical feature may be useful to other re-

search questions; for example, subjects could be presented with a naturalistic movie with

and without stereoscopic cues to test which regions of the cortex are particularly involved

in depth perception mediated by stereoscopic cues.

The surface reflectance aftereffect documented in Chapter 7 offers another phenomenon that

can be used to investigate and test models of colour constancy. As described above, I plan to

extend this psychophysical work to probe for adaptable mechanisms sensitive to illuminant

colour. Using fMRI, I plan to test for cortical regions whose response profile corresponds

either to surface colour, illuminant colour, or raw colour signal. While these three factors

are often correlated with one another, I hope to use stimuli similar to those in Chapter 7,

possibly combined with an fMR adaptation protocol, to attempt to separate out these re-

sponses.

Finally, as mentioned above, the bias across visual cortex in all subjects for lime-magenta

stimuli to evoke a greater BOLD response than orange-cyan stimuli is another phenomenon

to investigate further. I plan to test for any psychophysical correlates of this effect, including

chromatic discrimination thresholds.

8.2.2 Remaining Questions of Coding of Colour in Human Visual Cortex

When embarking on this thesis I aimed to understand further the neural mechanisms under-

lying colour vision, with a focus on cortical processes as those that are less well understood

than retinal and subcortical mechanisms. The work I have presented here furthers our un-

derstanding in a few specific areas of the general field, but the process has also changed the

way in which I consider the problem of understanding the coding of colour in human visual

cortex. Below I describe a few ways in which my thinking has changed while completing

this thesis.

136
 Chapter 8. General Discussion

The influence of attention on BOLD responses has led me not only to realise the importance

of attentional controls in fMRI experiments, but has also emphasised the error of assuming

the visual system represents all parts and all aspects of our visual field. While behavioral

results demonstrate the inaccuracy of our intuition that we ‘see’ the entire visual scene, neu-

roimaging results demonstrated to me that response characteristics of even early visual areas

need to be considered as a function not only of the visual stimulus, but of the observer’s task

and its attentional demands. In the neuroimaging experiments presented here I sought to

control attentional artifacts by directing subjects’ attention away from the stimuli, but in fu-

ture I would also like to use attentional and task manipulations to test how different visual

areas are recruited in different ways depending on the task.

I have also become increasingly aware of the substantial overlap in response profile of early

visual areas, revealed both by electrophysiological and neuroimaging techniques. There

do appear to be cortical areas that respond relatively more to particular features, with the

archetypal example of motion processing in MT, and to a lesser extent the preference for

colour in ventral cortical areas. However, as described in Chapters 5 and 6, I found that

almost all of the cortical areas I looked at showed a greater response to chromatic than to

achromatic stimuli, and that their activity could be used to distinguish between non-cardinal

colours. These results suggest that increased responsiveness to colour, or increased preve-

lance of chromatically selective neurons, may be an insufficient reason to term an area a

‘colour processing centre’. Rather than seeking out areas that are particularly responsive to

colour, it may be more fruitful to assume that chromatic information is coded along with

other visual features by most areas in visual cortex, and aim to understand how the repre-

sentation of chromatic information by the visual system is transformed from one area to the

next, along with the interactions between these areas.

Approaching the problem of colour constancy has also made me see how this aspect of

vision relates to other areas of perception. The question of how we achieve colour constancy

is typically approached by considering our perception of object or surface colour, but in our

perceptual experience the illuminant is not only ‘discounted’ to yield the surface colour, but

is perceived as a separate layer of the scene. As discussed in Chapter 3, this is not a novel

137
 Chapter 8. General Discussion

idea, but for me it highlighted the need to incorporate the notion of multiple layers into

our understanding how the perceptual system reconstructs the visual world. Within this

framework, there may be substantial overlap between mechanisms underlying transparency

and illuminant perception.

Finally, while I have generally sought to relate cortical processing of colour to the mecha-

nisms that provide the input to human visual cortex, I am aware that higher level influences,

such as memory and language, also must be taken into account. The impact of these factors

on perception and behavior suggests that their influence might feedback to areas in early

visual cortex. Functional neuroimaging of human subjects offers the potential to test such

hypotheses.

Neuroimaging offers the opportunity to test homology between humans and non-human

primates, as well as testing hypotheses generated from electrophysiological experiments.

Similarly, psychophysics and neuroimaging experiments can be used to inform and gener-

ate hypotheses for each other. Overall, the combination of psychophysics, neuroimaging,

electrophysiology and lesion studies in future work promises to reveal more of the cortical

mechanisms underlying colour perception.

138
References

Adelson, E. H. (2000). Lightness perception and lightness illusions. In Gazzaniga, M. S. (Ed.)

The cognitive neurosciences. Chap. 24, (pp. 339–351). Cambridge, MA: MIT Press, 2nd ed.

Anderson, B. L. & Winawer, J. (2005). Image segmentation and lightness perception. Nature,

434(7029), 79–83.

Anderson, B. L. & Winawer, J. (2008). Layered image representations and the computation

of surface lightness. Journal of Vision, 8(7), 18, 1–22.

Anstis, S. & Cavanagh, P. (1983). A minimum motion technique for judging equiluminance.

In Mollon, J. D. & Sharpe, R. T. (Eds.) Colour vision: Physiology and psychophysics. (pp.

156–166). New York, USA: Academic Press.

Arcaro, M. J., McMains, S. A., Singer, B. D., & Kastner, S. (2009). Retinotopic organization of

human ventral visual cortex. Journal of Neuroscience, 29(34), 10638–10652.

Arcizet, F., Jouffrais, C., & Girard, P. (2009). Coding of shape from shading in area V4 of the

macaque monkey. BioMed Central Neuroscience, 10, 140.

Barberini, C. L., Cohen, M. R., Wandell, B. A., & Newsome, W. T. (2005). Cone signal inter-

actions in direction-selective neurons in the middle temporal visual area (MT). Journal of

Vision, 5(7), 1, 603–621.

Bartels, A. & Zeki, S. (2000). The architecture of the colour centre in the human visual brain:

new results and a review. European Journal of Neuroscience, 12(1), 172–193.

Bartels, A. & Zeki, S. (2004). Functional brain mapping during free viewing of natural scenes.

Human Brain Mapping, 21(2), 75–85.

139
 References

Beauchamp, M. S., Haxby, J. V., Jennings, J. E., & DeYoe, E. A. (1999). An fMRI version of the

Farnsworth-Munsell 100-Hue test reveals multiple color-selective areas in human ventral

occipitotemporal cortex. Cerebral Cortex, 9(3), 257–263.

Beauchamp, M. S., Haxby, J. V., Rosen, A. C., & DeYoe, E. A. (2000). A functional MRI case

study of acquired cerebral dyschromatopsia. Neuropsychologia, 38(8), 1170–1179.

Beauvois, M. & Saillant, B. (1985). Optic aphasia for colours and colour agnosia: a distinc-

tion between visual and visuo-verbal impairments in the processing of colours. Cognitive

Neuropsychology, 2, 1–48.

Belitski, A., Gretton, A., Magri, C., Murayama, Y., Montemurro, M. A., Logothetis, N. K.,

& Panzeri, S. (2008). Low-frequency local field potentials and spikes in primary visual

cortex convey independent visual information. Journal of Neuroscience, 28(22), 5696–5709.

Bennett, K. P. & Campbell, C. (2000). Support vector machines: Hype or hallelujah? SIGKDD

Explorations, 2(2), 1–13.

Bex, P. J., Solomon, S. G., & Dakin, S. C. (2009). Contrast sensitivity in natural scenes depends

on edge as well as spatial frequency structure. Journal of Vision, 9, 10, 1–19.

Birch, J., Barbur, J. L., & Harlow, A. J. (1992). New method based on random lumi-

nance masking for measuring isochromatic zones using high resolution colour displays.

Opthalmic and Physiological Optics, 12(2), 133–136.

Bloj, M. G., Kersten, D., & Hurlbert, A. C. (1999). Perception of three-dimensional shape

influences colour perception through mutual illumination. Nature, 402(6764), 877–879.

Bouvier, S. E. & Engel, S. A. (2006). Behavioral deficits and cortical damage loci in cerebral

achromatopsia. Cerebral Cortex, 16(2), 183–191.

Boynton, G., Engel, S., Glover, G., & Heeger, D. (1996). Linear systems analysis of functional

magnetic resonance imaging in human V1. Journal of Neuroscience, 16(13), 4207–4221.

Boynton, R. M., Nagy, A. L., & Eskew, R. T. (1986). Similarity of normalized discrimination

ellipses in the constant-luminance chromaticity plane. Perception, 15(6), 755–763.

140
 References

Brainard, D. H. (1996). Cone contrast and opponent modulation color spaces. In Kaiser,

P. K. & Boynton, R. M. (Eds.) Human Color Vision. (pp. 563–579). Washington, DC: Optical

Society of America, 2nd ed.

Brainard, D. H. (1997). The Psychophysics toolbox. Spatial Vision, 10(4), 433–436.

Brainard, D. H., Brunt, W. A., & Speigle, J. M. (1997). Color constancy in the nearly natural

image. I. Asymmetric matches. Journal of the Optical Society of America A-Optics Image

Science and Vision, 14(9), 2091–2110.

Brainard, D. H. & Freeman, W. T. (1997). Bayesian color constancy. Journal of the Optical

Society of America A-Optics Image Science and Vision, 14(7), 1393–1411.

Brainard, D. H., Longere, P., Delahunt, P. B., Freeman, W. T., Kraft, J. M., & Xiao, B. (2006).

Bayesian model of human color constancy. Journal of Vision, 6(11), 10, 1267–1281.

Brainard, D. H., Roorda, A., Yamauchi, Y., Calderone, J. B., Metha, A., Neitz, M., Neitz, J.,

Williams, D. R., & Jacobs, G. H. (2000). Functional consequences of the relative numbers

of L and M cones. Journal of the Optical Society of America A-Optics Image Science and Vision,

17(3), 607–614.

Brainard, D. H. & Stockman, A. (2010). Colorimetry. In Bass, M., Enoch, J. M., & Lakshmi-

narayanan, V. (Eds.) Handbook of Optics: Sponsored by the Optical Society of America., Vol. 3.

Chap. 10, (pp. 10.1–10.56). New York: McGraw-Hill, 3rd ed.

Brainard, D. H. & Wandell, B. A. (1992). Asymmetric color matching: How color appearance

depends on the illuminant. Journal of the Optical Society of America A-Optics Image Science

and Vision, 9(9), 1433–1448.

Brewer, A. A., Liu, J., Wade, A. R., & Wandell, B. A. (2005). Visual field maps and stimulus

selectivity in human ventral occipital cortex. Nature Neuroscience, 8(8), 1102–1109.

Brewer, A. A., Press, W. A., Logothetis, N. K., & Wandell, B. A. (2002). Visual areas in

macaque cortex measured using functional magnetic resonance imaging. Journal of Neu-

roscience, 22(23), 10416–10426.

141
 References

Brill, M. H. & West, G. (1986). Chromatic adaptation and color constancy: A possible di-

chotomy. COLOR research and application, 11, 196–204.

Britten, K., Shadlen, M., Newsome, W., & Movshon, J. (1992). The analysis of visual-motion -

a comparison of neuronal and psychophysical performance. Journal of Neuroscience, 12(12),

4745–4765.

Brouwer, G. J. & Heeger, D. J. (2009). Decoding and reconstructing color from responses in

human visual cortex. Journal of Neuroscience, 29(44), 13992–14003.

Brown, R. O. & MacLeod, D. I. (1997). Color appearance depends on the variance of sur-

round colors. Current Biology, 7(11), 844–89.

Bruner, J. S., Postman, L., & Rodrigues, J. (1951). Expectation and the perception of color.

American Journal of Psychology, 64(2), 216–227.

Buchsbaum, G. (1980). A spatial processor model for object color-perception. Journal of the

Franklin Institute-Engineering and Applied Mathematics, 310(1), 1–26.

Buchsbaum, G. & Gottschalk, A. (1983). Trichromacy, opponent colours coding and op-

timum colour information transmission in the retina. Proceedings of the Royal Society of

London, Series B-Biological Sciences, 220(1218), 89–113.

Burges, C. J. C. (1998). A tutorial on support vector machines for pattern recognition. Data

Mining and Knowledge Discovery, 2, 121–167.

Burkhalter, A., Felleman, D. J., Newsome, W. T., & van Essen, D. C. (1986). Anatomical

and physiological asymmetries related to visual areas V3 and VP in macaque extrastriate

cortex. Vision Research, 26(1), 63–80.

Calkins, D. (1999). Synaptic organization of cone pathways in the primate retina. In Gegen-

furtner, K. & Sharpe, L. (Eds.) Color Vision: From Genes to Perception. (pp. 163–180). Cam-

bridge, UK: Cambridge University Press.

Calkins, D. J., Schein, S. J., Tsukamoto, Y., & Sterling, P. (1994). M and L cones in macaque

fovea connect to midget ganglion cells by different numbers of excitatory synapses. Na-

ture, 371(6492), 70–72.

142
 References

Calkins, D. J., Thornton, J. E., & Pugh, E. N. (1992). Monochromatism determined at a long-

wavelength/middle-wavelength cone-antagonistic locus. Vision Research, 32(12), 2349–

2367.

Calkins, D. J., Tsukamoto, Y., & Sterling, P. (1998). Microcircuitry and mosaic of a blue-

yellow ganglion cell in the primate retina. Journal of Neuroscience, 18(9), 3373–3385.

Callaway, E. M. (2005). Structure and function of parallel pathways in the primate early

visual system. Journal of Physiology, 566(Pt 1), 13–19.

Chatterjee, S. & Callaway, E. M. (2003). Parallel colour-opponent pathways to primary visual

cortex. Nature, 426(6967), 668–671.

Chittka, L., Shmida, A., Troje, N., & Menzel, R. (1994). Ultraviolet as a component of flower

reflections, and the colour perception of hymenoptera. Vision Research, 34(11), 1489–1508.

Constable, R. T. (2006). Challenges in fMRI and its limitations. In Faro, S. H. & Mohamed,

F. B. (Eds.) Functional MRI: Basic Principles and Clinical Applications. (pp. 75–98). New York,

USA: Springer.

Conway, B. R. (2001). Spatial structure of cone inputs to color cells in alert macaque primary

visual cortex (V-1). Journal of Neuroscience, 21(8), 2768–283.

Conway, B. R. & Tsao, D. Y. (2006). Color architecture in alert macaque cortex revealed by

fMRI. Cerebral Cortex, 16(11), 1604–1613.

Craven, B. J. & Foster, D. H. (1992). An operational approach to colour constancy. Vision

Research, 32(7), 1359–1366.

Crook, J. D., Davenport, C. M., Peterson, B. B., Packer, O. S., Detwiler, P. B., & Dacey, D. M.

(2009). Parallel ON and OFF cone bipolar inputs establish spatially coextensive recep-

tive field structure of blue-yellow ganglion cells in primate retina. Journal of Neuroscience,

29(26), 8372–8387.

Curcio, C. A., Allen, K. A., Sloan, K. R., Lerea, C. L., Hurley, J. B., Klock, I. B., & Milam,

A. H. (1991). Distribution and morphology of human cone photoreceptors stained with

anti-blue opsin. Journal of Comparative Neurology, 312(4), 610–624.

143
 References

Dacey, D. M. (1993). Morphology of a small-field bistratified ganglion cell type in the

macaque and human retina. Visual Neuroscience, 10(6), 1081–1098.

Dacey, D. M. (2000). Parallel pathways for spectral coding in primate retina. Annual Review

of Neuroscience, 23, 743–775.

Dacey, D. M. & Lee, B. B. (1994). The ’blue-on’ opponent pathway in primate retina origi-

nates from a distinct bistratified ganglion cell type. Nature, 367(6465), 731–735.

Dacey, D. M. & Packer, O. S. (2003). Colour coding in the primate retina: Diverse cell types

and cone-specific circuitry. Current Opinion in Neurobiology, 13(4), 421–427.

Dacey, D. M., Peterson, B. B., Robinson, F. R., & Gamlin, P. D. (2003). Fireworks in the pri-

mate retina: In vitro photodynamics reveals diverse LGN-projecting ganglion cell types.

Neuron, 37(1), 15–27.

Damasio, A., Yamada, T., Damasio, H., Corbett, J., & McKee, J. (1980). Central achromatop-

sia: behavioral, anatomic, and physiologic aspects. Neurology, 30(10), 1064–1071.

Danilova, M. & Mollon, J. (2010). Parafoveal color discrimination: A chromaticity locus of

enhanced discrimination. Journal of Vision, 10(1), 4, 1–9.

Dartnall, H. J., Bowmaker, J. K., & Mollon, J. D. (1983). Human visual pigments: Microspec-

trophotometric results from the eyes of seven persons. Proceedings of the Royal Society of

London, Series B-Biological Sciences, 220(1218), 115–130.

David, S. V., Hayden, B. Y., & Gallant, J. L. (2006). Spectral receptive field properties explain

shape selectivity in area V4. Journal of Neurophysiology, 96(6), 3492–3505.

de Monasterio, F. & Gouras, P. (1975). Functional properties of ganglion cells of the rhesus

monkey retina. Journal of Physiology, 251(1), 167–195.

de Monasterio, F. M. & Schein, S. J. (1982). Spectral bandwidths of color-opponent cells of

geniculocortical pathway of macaque monkeys. Journal of Neurophysiology, 47(2), 214–224.

de Valois, R., Cottaris, N., Elfar, S., Mahon, L., & Wilson, J. (2000). Some transformations

144
 References

of color information from lateral geniculate nucleus to striate cortex. Proceedings of the

National Academy of Sciences of the USA, 97(9), 4997–5002.

de Valois, R. L., Abramov, I., & Jacobs, G. H. (1966). Analysis of response patterns of LGN

cells. Journal of the Optical Society of America, 56(7), 966–977.

de Valois, R. L. & de Valois, K. K. (1993). A multi-stage color model. Vision Research, 33(8),

1053–1065.

Derrington, A. M., Krauskopf, J., & Lennie, P. (1984). Chromatic mechanisms in lateral

geniculate nucleus of macaque. Journal of Physiology, 357, 241–265.

Desimone, R., Schein, S. J., Moran, J., & Ungerleider, L. G. (1985). Contour, color and shape

analysis beyond the striate cortex. Vision Research, 25(3), 441–452.

DeYoe, E. A., Carman, G. J., Bandettini, P., Glickman, S., Wieser, J., Cox, R., Miller, D., &

Neitz, J. (1996). Mapping striate and extrastriate visual areas in human cerebral cortex.

Proceedings of the National Academy of Sciences of the USA, 93(6), 2382–2386.

Dubner, R. & Zeki, S. M. (1971). Response properties and receptive fields of cells in an

anatomically defined region of the superior temporal sulcus in the monkey. Brain Research,

35(2), 528–532.

Dumoulin, S. O. & Hess, R. F. (2007). Cortical specialization for concentric shape processing.

Vision Research, 47(12), 1608–1613.

D’Zmura, M. & Lennie, P. (1986). Mechanisms of color constancy. Journal of the Optical Society

of America A-Optics Image Science and Vision, 3(10), 1662–1672.

Ebner, M. (2007). How does the brain arrive at a color constant descriptor? Proceedings of the

2nd international conference on Advances in brain, vision and artificial intelligence, (pp. 84–93).

Ekroll, V., Faul, F., & Niederee, R. (2004). The peculiar nature of simultaneous colour contrast

in uniform surrounds. Vision Research, 44(15), 1765–1786.

Engel, S. A. & Furmanski, C. S. (2001). Selective adaptation to color contrast in human

primary visual cortex. Journal of Neuroscience, 21(11), 3949–3954.

145
 References

Engel, S. A., Glover, G. H., & Wandell, B. A. (1997). Retinotopic organization in human

visual cortex and the spatial precision of functional MRI. Cerebral Cortex, 7(2), 181–192.

Engel, S. A., Rumelhart, D. E., Wandell, B. A., Lee, A. T., Glover, G. H., Chichilnisky, E. J., &

Shadlen, M. N. (1994). fMRI of human visual cortex. Nature, 369(6481), 525.

Eskew, R. T. (2009). Higher-order color mechanisms: A critical review. Vision Research, 49(22),

2686–2704.

Eskew, R. T., Newton, J. R., & Giulianini, F. (2001). Chromatic detection and discrimination

analyzed by a Bayesian classifier. Vision Research, 41(7), 893–909.

Fairchild, M. D. (2005). Color Appearance Models. Chichester, UK: John Wiley & Sons, Inc.,

2nd ed.

Fairchild, M. D. & Reniff, L. (1995). Time course of chromatic adaptation for color-

appearance judgments. Journal of the Optical Society of America A-Optics Image Science and

Vision, 12(5), 824–833.

Faul, F., Ekroll, V., & Wendt, G. (2008). Color appearance: The limited role of chromatic

surround variance in the "gamut expansion effect". Journal of Vision, 8(3), 30, 1–20.

Felleman, D. J. & van Essen, D. C. (1987). Receptive field properties of neurons in area V3 of

macaque monkey extrastriate cortex. Journal of Neurophysiology, 57(4), 889–920.

Felleman, D. J. & van Essen, D. C. (1991). Distributed hierarchical processing in the primate

cerebral cortex. Cerebral Cortex, 1(1), 1–47.

Fize, D., Vanduffel, W., Nelissen, K., Denys, K., chef d’Hotel, C., Faugeras, O., & Orban,

G. A. (2003). The retinotopic organization of primate dorsal V4 and surrounding areas: A

functional magnetic resonance imaging study in awake monkeys. Journal of Neuroscience,

23(19), 7395–7406.

Foster, D. H. & Nascimento, S. M. (1994). Relational colour constancy from invariant cone-

excitation ratios. Proceedings Biological Sciences / The Royal Society, 257(1349), 115–121.

Fox, P. T. & Raichle, M. E. (1986). Focal physiological uncoupling of cerebral blood flow and

146
 References

oxidative metabolism during somatosensory stimulation in human subjects. Proceedings

of the National Academy of Sciences of the USA, 83(4), 1140–1144.

Fox, P. T., Raichle, M. E., Mintun, M. A., & Dence, C. (1988). Nonoxidative glucose consump-

tion during focal physiologic neural activity. Science, 241(4864), 462–464.

Frackowiak, R. S., Friston, K. J., Frith, C. D., Dolan, R. J., & Mazziotta, J. C. (1997). Human

Brain Function. San-Diego, CA: Academic Press.

Gallant, J. L., Braun, J., & van Essen, D. C. (1993). Selectivity for polar, hyperbolic, and

cartesian gratings in macaque visual cortex. Science, 259(5091), 100–103.

Gallant, J. L., Connor, C. E., Rakshit, S., Lewis, J. W., & van Essen, D. C. (1996). Neural

responses to polar, hyperbolic, and cartesian gratings in area V4 of the macaque monkey.

Journal of Neurophysiology, 76(4), 2718–2739.

Gallant, J. L., Shoup, R. E., & Mazer, J. A. (2000). A human extrastriate area functionally

homologous to macaque V4. Neuron, 27(2), 227–235.

Gattass, R., Sousa, A. P., & Gross, C. G. (1988). Visuotopic organization and extent of V3 and

V4 of the macaque. Journal of Neuroscience, 8(6), 1831–1845.

Gegenfurtner, K. R. (2003). Cortical mechanisms of colour vision. Nature Reviews Neuro-

science, 4(7), 563–572.

Gegenfurtner, K. R., Kiper, D. C., Beusmans, J. M., Carandini, M., Zaidi, Q., & Movshon,

J. A. (1994). Chromatic properties of neurons in macaque MT. Visual Neuroscience, 11(3),

455–466.

Gegenfurtner, K. R., Kiper, D. C., & Levitt, J. B. (1997). Functional properties of neurons in

macaque area V3. Journal of Neurophysiology, 77(4), 1906–1923.

Gerbino, W., Stultiens, C. I., Troost, J. M., & de Weert, C. M. (1990). Transparent layer con-

stancy. Journal of Experimental Psychology-Human Perception and Performance, 16(1), 3–20.

Gilchrist, A., Kossyfidis, C., Bonato, F., Agostini, T., Cataliotti, J., Li, X. J., Spehar, B., Annan,

147
 References

V., & Economou, E. (1999). An anchoring theory of lightness perception. Psychological

Review, 106(4), 795–834.

Goddard, E., Mannion, D., McDonald, J., Solomon, S., & Clifford, C. (2010). Combination of

subcortical color channels in human visual cortex. Journal of Vision, 10(5), 25, 1–17.

Goddard, E., Solomon, S., & Clifford, C. (In Press). Adaptable mechanisms sensitive to

surface color in human vision. Journal of Vision.

Goebel, R., Khorram-Sefat, D., Muckli, L., Hacker, H., & Singer, W. (1998). The constructive

nature of vision: Direct evidence from functional magnetic resonance imaging studies of

apparent motion and motion imagery. European Journal of Neuroscience, 10(5), 1563–1573.

Goense, J. B. M. & Logothetis, N. K. (2008). Neurophysiology of the BOLD fMRI signal in

awake monkeys. Current Biology, 18(9), 631–640.

Golz, J. & MacLeod, D. I. (2002). Influence of scene statistics on colour constancy. Nature,

415(6872), 637–640.

Granzier, J. J. M., Brenner, E., & Smeets, J. B. J. (2009). Reliable identification by color under

natural conditions. Journal of Vision, 9(1), 39, 1–8.

Guth, S. & Moxley, J. (1982). Hue shifts following differential postreceptor achromatic adap-

tation. Journal of the Optical Society of America, 72(2), 301–303.

Hadjikhani, N., Liu, A. K., Dale, A. M., Cavanagh, P., & Tootell, R. B. (1998). Retinotopy and

color sensitivity in human visual cortical area V8. Nature Neuroscience, 1(3), 235–241.

Haenny, P. E. & Schiller, P. H. (1988). State dependent activity in monkey visual cortex. I.

Single cell activity in V1 and V4 on visual tasks. Experimental Brain Research, 69(2), 225–

244.

Hagler, D. J. & Sereno, M. I. (2006). Spatial maps in frontal and prefrontal cortex. Neuroimage,

29(2), 567–577.

Hagstrom, S. A., Neitz, J., & Neitz, M. (1998). Variations in cone populations for red-green

color vision examined by analysis of mRNA. Neuroreport, 9(9), 1963–1967.

148
 References

Hammond, B. R., Wooten, B. R., & Snodderly, D. M. (1997). Individual variations in the

spatial profile of human macular pigment. Journal of the Optical Society of America A-Optics

Image Science and Vision, 14(6), 1187–1196.

Hansen, K. A., Kay, K. N., & Gallant, J. L. (2007a). Topographic organization in and near

human visual area V4. Journal of Neuroscience, 27(44), 11896–11911.

Hansen, T., Olkkonen, M., Walter, S., & Gegenfurtner, K. R. (2006). Memory modulates color

appearance. Nature Neuroscience, 9(11), 1367–1368.

Hansen, T., Walter, S., & Gegenfurtner, K. R. (2007b). Effects of spatial and temporal context

on color categories and color constancy. Journal of Vision, 7(4), 2, 1–15.

Harrison, R. V., Harel, N., Panesar, J., & Mount, R. J. (2002). Blood capillary distribution

correlates with hemodynamic-based functional imaging in cerebral cortex. Cerebral Cortex,

12(3), 225–233.

Haverkamp, S., Grünert, U., & Wässle, H. (2000). The cone pedicle, a complex synapse in

the retina. Neuron, 27(1), 85–95.

Helson, H. (1938). Fundamental problems in color vision I. The principle governing changes

in hue, saturation, and lightness of non-selective samples in chromatic illumination. Jour-

nal of Experimental Psychology, 23, 439–476.

Hering, E. (1964). Outline of a theory of the light sense. Cambridge, Massachusetts: Harvard

University Press. Translated by Leo M. Hurvich and Dorothea Jameson from original 1920

version.

Heywood, C. A., Gaffan, D., & Cowey, A. (1995). Cerebral achromatopsia in monkeys.

European Journal of Neuroscience, 7(5), 1064–1073.

Hiltunen, J. (1996). Munsell Colors Matte (spectrophotometer mea-

surements by Hiltunen). Retrieved September 10, 1999, from

http://www.it.lut.fi/ip/research/color/database/download.html#munsell_spec_matt.

Hofer, H., Carroll, J., Neitz, J., Neitz, M., & Williams, D. R. (2005). Organization of the human

trichromatic cone mosaic. Journal of Neuroscience, 25(42), 9669–9679.

149
 References

Hubel, D. H. & Livingstone, M. S. (1987). Segregation of form, color, and stereopsis in

primate Area 18. Journal of Neuroscience, 7(11), 3378–3415.

Huk, A., Dougherty, R., & Heeger, D. (2002). Retinotopy and functional subdivision of

human areas MT and MST. Journal of Neuroscience, 22(16), 7195–7205.

Hurlbert, A. (1986). Formal connections between lightness algorithms. Journal of the Optical

Society of America A, 3(10), 1684–1693.

Hurlbert, A. (1996). Colour vision: putting it in context. Current Biology, 6(11), 1381–1384.

Hurvich, L. M. & Jameson, D. (1955). Some quantitative aspects of an opponent-colors the-

ory. II. Brightness, saturation, and hue in normal and dichromatic vision. Journal of the

Optical Society of America, 45(8), 602–616.

Hurvich, L. M. & Jameson, D. (1956). Some quantitative aspects of an opponent-colors the-

ory. IV. A psychological color specification system. Journal of the Optical Society of America,

46(6), 416–421.

Hurvich, L. M. & Jameson, D. (1957). An opponent-process theory of color vision. Psycho-

logical Review, 64(1), 6, 384–404.

Ishihara, S. (1990). Ishihara’s tests for color-blindness, 38 plate ed. Tokyo/Kyoto: Kanehara,

Shuppan Co. Ltd.

Ives, H. (1912). The relation between the color of the illuminant and the color of the illu-

minated object. Transactions of the Illumination Engineering Society, 7, 62–72. Reprinted in:

COLOR Research and Application (1995), 20(1), 70-75.

Jameson, D. & Hurvich, L. (1955). Some quantitative aspects of an opponent-colors theory.

I. Chromatic responses and saturation. Journal of the Optical Society of America, 45, 546–552.

Jameson, D. & Hurvich, L. M. (1956). Some quantitative aspects of an opponent-colors the-

ory. III. Changes in brightness, saturation, and hue with chromatic adaptation. Journal of

the Optical Society of America, 46(6), 405–415.

150
 References

Jiang, Y., Zhou, K., & He, S. (2007). Human visual cortex responds to invisible chromatic

flicker. Nature Neuroscience, 10(5), 657–662.

Jin, T. & Kim, S.-G. (2008). Cortical layer-dependent dynamic blood oxygenation, cerebral

blood flow and cerebral blood volume responses during visual stimulation. Neuroimage,

43(1), 1–9.

Joachims, T. (1999). Making large-scale SVM learning practical. In Scholkopf, B., Burges,

C., & Smola, A. (Eds.) Advances in Kernel Methods - Support Vector Learning. Chap. 11, (pp.

41–56). Cambridge, MA: MIT Press.

Johnson, E. N., Hawken, M. J., & Shapley, R. (2001). The spatial transformation of color in

the primary visual cortex of the macaque monkey. Nature Neuroscience, 4(4), 409–416.

Judd, D. B. (1940). Hue saturation and lightness of surface colors with chromatic illumina-

tion. Journal of the Optical Society of America, 30(1), 2–32.

Kaiser, P. K. & Boynton, R. M. (1996). Human color vision. Washington, DC: Optical Society

of America, 2nd ed.

Kastner, S., Weerd, P. D., Desimone, R., & Ungerleider, L. G. (1998). Mechanisms of di-

rected attention in the human extrastriate cortex as revealed by functional MRI. Science,

282(5386), 108–111.

Kim, S.-G. & Bandettini, P. A. (2006). Principles of functional mri. In Faro, S. H. & Mohamed,

F. B. (Eds.) Functional MRI: Basic Principles and Clinical Applications. (pp. 3–23). New York,

USA: Springer.

Kingdom, F., Moulden, B., & Collyer, S. (1992). A comparison between colour and luminance

contrast in a spatial linking task. Vision Research, 32(4), 709–717.

Kiper, D. C., Fenstemaker, S. B., & Gegenfurtner, K. R. (1997). Chromatic properties of neu-

rons in macaque area V2. Visual Neuroscience, 14(6), 1061–1072.

Klug, K., Herr, S., Ngo, I. T., Sterling, P., & Schein, S. (2003). Macaque retina contains an

S-cone OFF midget pathway. Journal of Neuroscience, 23(30), 9881–9887.

151
 References

Komatsu, H. (1998). Mechanisms of central color vision. Current Opinion in Neurobiology,

8(4), 503–508.

Kontsevich, L. L. & Tyler, C. W. (1999). Bayesian adaptive estimation of psychometric slope

and threshold. Vision Research, 39(16), 2729–2737.

Kourtzi, Z., Erb, M., Grodd, W., & Bülthoff, H. H. (2003). Representation of the perceived

3-D object shape in the human lateral occipital complex. Cerebral Cortex, 13(9), 911–920.

Kraft, J. M. & Brainard, D. H. (1999). Mechanisms of color constancy under nearly natural

viewing. Proceedings of the National Academy of Sciences of the USA, 96(1), 307–312.

Krauskopf, J. (1999). Higher order color mechanisms. In Gegenfurtner, K. & Sharpe, L.

(Eds.) Color Vision: From Genes to Perception. (pp. 303–316). Cambridge, UK: Cambridge

University Press.

Krauskopf, J. & Gegenfurtner, K. (1992). Color discrimination and adaptation. Vision Re-

search, 32(11), 2165–2175.

Krauskopf, J., Williams, D. R., & Heeley, D. W. (1982). Cardinal directions of color space.

Vision Research, 22(9), 1123–1131.

Krauskopf, J., Williams, D. R., Mandler, M. B., & Brown, A. M. (1986). Higher order color

mechanisms. Vision Research, 26(1), 23–32.

Kriegeskorte, N., Simmons, W. K., Bellgowan, P. S. F., & Baker, C. I. (2009). Circular analysis

in systems neuroscience: the dangers of double dipping. Nature Neuroscience, 12(5), 535–

540.

Kumano, H., Tanabe, S., & Fujita, I. (2008). Spatial frequency integration for binocular cor-

respondence in macaque area V4. Journal of Neurophysiology, 99(1), 402–408.

Kusunoki, M., Moutoussis, K., & Zeki, S. (2006). Effect of background colors on the tuning

of color-selective cells in monkey area V4. Journal of Neurophysiology, 95(5), 3047–3059.

Land, E. H. (1986). Recent advances in retinex theory. Vision Research, 26(1), 7–21.

152
 References

Land, E. H. & McCann, J. J. (1971). Lightness and retinex theory. Journal of the Optical Society

of America, 61(1), 1–11.

Larsson, J. & Heeger, D. J. (2006). Two retinotopic visual areas in human lateral occipital

cortex. Journal of Neuroscience, 26(51), 13128–13142.

Lee, B. (1999). Receptor inputs to primate ganglion cells. In Gegenfurtner, K. & Sharpe, L.

(Eds.) Color Vision: From Genes to Perception. (pp. 203–218). Cambridge, UK: Cambridge

University Press.

Lee, B. B., Martin, P. R., & Valberg, A. (1988). The physiological basis of heterochromatic

flicker photometry demonstrated in the ganglion cells of the macaque retina. Journal of

Physiology, 404, 323–347.

Lee, H. C. (1986). Method for computing the scene-illuminant chromaticity from specular

highlights. Journal of the Optical Society of America A-Optics Image Science and Vision, 3(10),

1694–1699.

Lee, S. C. S. & Grünert, U. (2007). Connections of diffuse bipolar cells in primate retina are

biased against S-cones. Journal of Comparative Neurology, 502(1), 126–140.

Lehmann, T. M. & Palm, C. (2001). Color line search for illuminant estimation in real-world

scenes. Journal of the Optical Society of America A-Optics Image Science and Vision, 18(11),

2679–2691.

Lennie, P. (1999). Color coding in the cortex. In Gegenfurtner, K. R. & Sharpe, L. T. (Eds.)

Color Vision: From Genes to Perception. (pp. 235–247). Cambridge, UK: Cambridge Univer-

sity Press.

Lennie, P. (2000). Color vision: putting it together. Current Biology, 10(16), R589–R591.

Lennie, P. & D’Zmura, M. (1988). Mechanisms of color vision. Critical Reviews in Neurobiol-

ogy, 3(4), 333–400.

Lennie, P. & Movshon, J. A. (2005). Coding of color and form in the geniculostriate visual

pathway. Journal of the Optical Society of America A-Optics Image Science and Vision, 22(10),

2013–2033.

153
 References

Lennie, P., Pokorny, J., & Smith, V. C. (1993). Luminance. Journal of the Optical Society of

America A-Optics Image Science and Vision, 10(6), 1283–1293.

Lipton, M. (2008). Totally Accessible MRI. New York, USA: Springer.

Liu, J. & Wandell, B. A. (2005). Specializations for chromatic and temporal signals in human

visual cortex. Journal of Neuroscience, 25(13), 3459–3468.

Logothetis, N. K. (2003). The underpinnings of the BOLD functional magnetic resonance

imaging signal. Journal of Neuroscience, 23(10), 3963–3971.

Logothetis, N. K. (2008). What we can do and what we cannot do with fMRI. Nature,

453(7197), 869–878.

Logothetis, N. K., Pauls, J., Augath, M., Trinath, T., & Oeltermann, A. (2001). Neurophysio-

logical investigation of the basis of the fMRI signal. Nature, 412(6843), 150–157.

Logvinenko, A. D. & Maloney, L. T. (2006). The proximity structure of achromatic surface

colors and the impossibility of asymmetric lightness matching. Perception & Psychophysics,

68(1), 76–83.

Lu, R., Koenderink, J. J., & Kappers, A. M. L. (2000). Specularities on surfaces with tangential

hairs or grooves. Computer Vision and Image Understanding, 78(3), 320–335.

Lueck, C. J., Zeki, S., Friston, K. J., Deiber, M. P., Cope, P., Cunningham, V. J., Lammertsma,

A. A., Kennard, C., & Frackowiak, R. S. (1989). The colour centre in the cerebral cortex of

man. Nature, 340(6232), 386–389.

Luzzatti, C. & Davidoff, J. (1994). Impaired retrieval of object-colour knowledge with pre-

served colour naming. Neuropsychologia, 32(8), 933–950.

Lyon, D. C. & Kaas, J. H. (2002). Evidence for a modified V3 with dorsal and ventral halves

in macaque monkeys. Neuron, 33(3), 453–461.

MacLeod, D. I. & Boynton, R. M. (1979). Chromaticity diagram showing cone excitation by

stimuli of equal luminance. Journal of the Optical Society of America, 69(8), 1183–1186.

Maloney, L. T. (1999). Physics-based approaches to modeling surface color perception. In

154
 References

Gegenfurtner, K. R. & Sharpe, L. T. (Eds.) Color Vision: From Genes to Perception. (pp. 387–

416). Cambridge, UK: Cambridge University Press.

Maloney, L. T. & Wandell, B. A. (1986). Color constancy: a method for recovering surface

spectral reflectance. Journal of the Optical Society of America A-Optics Image Science and

Vision, 3(1), 29–33.

Manjón, J. V., Lull, J. J., Carbonell-Caballero, J., García-Martí, G., Martí-Bonmatí, L., & Rob-

les, M. (2007). A nonparametric MRI inhomogeneity correction method. Medical Image

Analysis, 11(4), 336–345.

Mariani, A. P. (1984). Bipolar cells in monkey retina selective for the cones likely to be blue-

sensitive. Nature, 308(5955), 184–186.

Marshall, N. J. (2000). Communication and camouflage with the same ‘bright’ colours in reef

fishes. Philosophical Transactions of the Royal Society of London Series B-Biological Sciences,

355(1401), 1243–1248.

Martin, P. R. (1998). Colour processing in the primate retina: recent progress. Journal of

Physiology, 513(3), 631–638.

Martin, P. R. & Grünert, U. (1999). Analysis of the short wavelength-sensitive ("blue") cone

mosaic in the primate retina: comparison of new world and old world monkeys. Journal

of Comparative Neurology, 406(1), 1–14.

Martin, P. R., Lee, B. B., White, A. J., Solomon, S. G., & Rüttiger, L. (2001). Chromatic sensi-

tivity of ganglion cells in the peripheral primate retina. Nature, 410(6831), 933–936.

Maunsell, J. H. (1995). The brain’s visual world: Representation of visual targets in cerebral

cortex. Science, 270(5237), 764–769.

Maunsell, J. H. R. & Cook, E. P. (2002). The role of attention in visual processing. Philosophical

Transactions of the Royal Society of London Series B-Biological Sciences, 357(1424), 1063–1072.

McAdams, C. J. & Maunsell, J. H. (1999). Effects of attention on orientation-tuning functions

of single neurons in macaque cortical area V4. Journal of Neuroscience, 19(1), 431–41.

155
 References

McCann, J. J., McKee, S. P., & Taylor, T. H. (1976). Quantitative studies in retinex theory.

A comparison between theoretical predictions and observer responses to the "color Mon-

drian" experiments. Vision Research, 16(5), 445–458.

McKeefry, D. J., Watson, J. D. G., Frackowiak, R. S. J., Fong, K., & Zeki, S. (1997). The activity

in human areas V1/V2, V3, and V5 during the perception of coherent and incoherent

motion. Neuroimage, 5(1), 1–12.

McKeefry, D. J. & Zeki, S. (1997). The position and topography of the human colour centre

as revealed by functional magnetic resonance imaging. Brain, 120, 2229–2242.

McRobbie, D., Moore, E., Graves, M., & Prince, M. (2007a). Getting in tune: resonance and

relaxation. In MRI from picture to proton. Chap. 8, (pp. 137–166). Cambridge: Cambridge

University Press.

McRobbie, D., Moore, E., Graves, M., & Prince, M. (2007b). Let’s talk technical: MR equip-

ment. In MRI from picture to proton. Chap. 9, (pp. 167–191). Cambridge: Cambridge Uni-

versity Press.

McRobbie, D., Moore, E., Graves, M., & Prince, M. (2007c). MRI from picture to proton. Cam-

bridge: Cambridge University Press.

McRobbie, D., Moore, E., Graves, M., & Prince, M. (2007d). Spaced out: spatial encoding.

In MRI from picture to proton. Chap. 7, (pp. 108–136). Cambridge: Cambridge University

Press.

Meadows, J. C. (1974). Disturbed perception of colours associated with localized cerebral

lesions. Brain, 97(4), 615–632.

Mehta, A. D., Ulbert, I., & Schroeder, C. E. (2000). Intermodal selective attention in monkeys.

I: Distribution and timing of effects across visual areas. Cerebral Cortex, 10(4), 343–358.

Mollon, J. D. (2003). The origins of modern color science. In The Science of Color. Chap. 1,

(pp. 1–39). Elsevier Ltd.

Moran, J. & Desimone, R. (1985). Selective attention gates visual processing in the extrastri-

ate cortex. Science, 229(4715), 782–784.

156
 References

Mourao-Miranda, J., Bokde, A. L., Born, C., Hampel, H., & Stetter, M. (2005). Classifying

brain states and determining the discriminating activation patterns: Support vector ma-

chine on functional MRI data. Neuroimage, 28(4), 980–995.

Mullen, K. T., Dumoulin, S. O., McMahon, K. L., de Zubicaray, G. I., & Hess, R. F. (2007).

Selectivity of human retinotopic visual cortex to S-cone-opponent, L/M-cone-opponent

and achromatic stimulation. European Journal of Neuroscience, 25(2), 491–502.

Mullen, K. T. & Kulikowski, J. J. (1990). Wavelength discrimination at detection threshold.

Journal of the Optical Society of America A, 7(4), 733–742.

Murphey, D. K., Yoshor, D., & Beauchamp, M. S. (2008). Perception matches selectivity in

the human anterior color center. Current Biology, 18(3), 216–220.

Mysore, S. G., Vogels, R., Raiguel, S. E., & Orban, G. A. (2008). Shape selectivity for

camouflage-breaking dynamic stimuli in dorsal V4 neurons. Cerebral Cortex, 18(6), 1429–

1443.

Nathans, J., Piantanida, T. P., Eddy, R. L., Shows, T. B., & Hogness, D. S. (1986a). Molecular

genetics of inherited variation in human color vision. Science, 232(4747), 203–210.

Nathans, J., Thomas, D., & Hogness, D. S. (1986b). Molecular genetics of human color vision:

the genes encoding blue, green, and red pigments. Science, 232(4747), 193–202.

Newton, J. R. & R. T., J. E. (2003). Chromatic detection and discrimination in the periphery:

A postreceptoral loss of color sensitivity. Visual Neuroscience, 20(5), 511–521.

Norman, K. A., Polyn, S. M., Detre, G. J., & Haxby, J. V. (2006). Beyond mind-reading: multi-

voxel pattern analysis of fMRI data. Trends in Cognitive Sciences, 10(9), 424–430.

Ogawa, S. & Lee, T. M. (1990). Magnetic resonance imaging of blood vessels at high fields:

In vivo and in vitro measurements and image simulation. Magnetic Resonance in Medicine,

16(1), 9–18.

Ogawa, S., Lee, T. M., Kay, A. R., & Tank, D. W. (1990a). Brain magnetic resonance imaging

with contrast dependent on blood oxygenation. Proceedings of the National Academy of

Sciences of the USA, 87(24), 9868–9872.

157
 References

Ogawa, S., Lee, T. M., Nayak, A. S., & Glynn, P. (1990b). Oxygenation-sensitive contrast in

magnetic resonance image of rodent brain at high magnetic fields. Magnetic Resonance in

Medicine, 14(1), 68–78.

Ogawa, S., Lee, T. M., Stepnoski, R., Chen, W., Zhu, X. H., & Ugurbil, K. (2000). An approach

to probe some neural systems interaction by functional mri at neural time scale down to

milliseconds. Proceedings of the National Academy of Sciences of the USA, 97(20), 11026–11031.

Packer, O. S., Verweij, J., Li, P. H., Schnapf, J. L., & Dacey, D. M. (2010). Blue-yellow oppo-

nency in primate S cone photoreceptors. Journal of Neuroscience, 30(2), 568–572.

Palmer, G. (1777). Theories of colors and vision. In MacAdam, D. L. (Ed.) Sources of Color

Science (1970). (pp. 40–47). Cambridge, MA: MIT Press.

Parkes, L., Marsman, J., Oxley, D., Goulermas, J., & Wuerger, S. (2009). Multivoxel fMRI

analysis of color tuning in human primary visual cortex. Journal of Vision, 9(1), 1, 1–13.

Pasupathy, A. (2006). Neural basis of shape representation in the primate brain. Progress in

Brain Research, 154, 293–313.

Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: transforming num-

bers into movies. Spatial Vision, 10(4), 437–442.

Phong, B. T. (1975). Illumination for computer generated pictures. Communications of the

ACM, 18(6), 311–317.

Pigarev, I. N., Nothdurft, H.-C., & Kastner, S. (2002). Neurons with radial receptive fields

in monkey area V4A: evidence of a subdivision of prelunate gyrus based on neuronal

response properties. Experimental Brain Research, 145(2), 199–206.

Pinus, A. B. & Mohamed, F. B. (2006). fmri scanning methodologies. In Faro, S. H. & Mo-

hamed, F. B. (Eds.) Functional MRI: Basic Principles and Clinical Applications. (pp. 24–57).

New York, USA: Springer.

Press, W. A., Brewer, A. A., Dougherty, R. F., Wade, A. R., & Wandell, B. A. (2001). Visual

areas and spatial summation in human visual cortex. Vision Research, 41(10-11), 1321–1332.

158
 References

Rees, G., Frith, C. D., & Lavie, N. (1997). Modulating irrelevant motion perception by vary-

ing attentional load in an unrelated task. Science, 278(5343), 1616–1619.

Reid, R. C. & Shapley, R. M. (1992). Spatial structure of cone inputs to receptive fields in

primate lateral geniculate nucleus. Nature, 356(6371), 716–718.

Reid, R. C. & Shapley, R. M. (2002). Space and time maps of cone photoreceptor signals in

macaque lateral geniculate nucleus. Journal of Neuroscience, 22(14), 6158–6175.

Reynolds, J. H. & Chelazzi, L. (2004). Attentional modulation of visual processing. Annual

Review of Neuroscience, 27, 611–647.

Ripamonti, C., Woo, W. L., Crowther, E., & Stockman, A. (2009). The S-cone contribution to

luminance depends on the M- and L-cone adaptation levels: silent surrounds? Journal of

Vision, 9(3), 10.1–1016.

Robilotto, R. & Zaidi, Q. (2004). Limits of lightness identification for real objects under

natural viewing conditions. Journal of Vision, 4(9), 9, 779–797.

Roorda, A., Metha, A. B., Lennie, P., & Williams, D. R. (2001). Packing arrangement of the

three cone classes in primate retina. Vision Research, 41(10-11), 1291–1306.

Roorda, A. & Williams, D. R. (1999). The arrangement of the three cone classes in the living

human eye. Nature, 397(6719), 520–522.

Rosa, M. G. P. & Tweedale, R. (2005). Brain maps, great and small: Lessons from comparative

studies of primate visual cortical organization. Philosophical Transactions of the Royal Society

of London Series B-Biological Sciences, 360(1456), 665–691.

Roy, C. S. & Sherrington, C. S. (1890). On the regulation of the blood-supply of the brain.

Journal of Physiology, 11(1-2), 85–158.

Schein, S. J. & Desimone, R. (1990). Spectral properties of V4 neurons in the macaque. Journal

of Neuroscience, 10(10), 3369–3389.

Schein, S. J., Marrocco, R. T., & de Monasterio, F. M. (1982). Is there a high concentration of

159
 References

color-selective cells in area V4 of monkey visual cortex? Journal of Neurophysiology, 47(2),

193–213.

Schiller, P. H. & Logothetis, N. K. (1990). The color-opponent and broad-band channels of

the primate visual system. Trends in Neurosciences, 13(10), 392–398.

Schira, M. M., Tyler, C. W., Breakspear, M., & Spehar, B. (2009). The foveal confluence in

human visual cortex. Journal of Neuroscience, 29(28), 9050–9058.

Schnapf, J. L., Kraft, T. W., & Baylor, D. A. (1987). Spectral sensitivity of human cone pho-

toreceptors. Nature, 325(6103), 439–441.

Schneider, K. A., Richter, M. C., & Kastner, S. (2004). Retinotopic organization and func-

tional subdivisions of the human lateral geniculate nucleus: A high-resolution functional

magnetic resonance imaging study. Journal of Neuroscience, 24(41), 8975–8985.

Schwartz, S., Vuilleumier, P., Hutton, C., Maravita, A., Dolan, R. J., & Driver, J. (2005). Atten-

tional load and sensory competition in human vision: modulation of fMRI responses by

load at fixation during task-irrelevant stimulation in the peripheral visual field. Cerebral

Cortex, 15(6), 770–786.

Seime, L. & Hardeberg, J. Y. (2003). Colorimetric characterization of LCD and DLP projection

displays. Journal of the Society for Information Display, 11(2), 349–358.

Sereno, M. I., Dale, A. M., Reppas, J. B., Kwong, K. K., Belliveau, J. W., Brady, T. J., Rosen,

B. R., & Tootell, R. B. (1995). Borders of multiple visual areas in humans revealed by

functional magnetic resonance imaging. Science, 268(5212), 889–893.

Seymour, K., Clifford, C. W. G., Logothetis, N. K., & Bartels, A. (2009). The coding of color,

motion, and their conjunction in the human visual cortex. Current Biology, 19(3), 177–183.

Shapley, R. & Enroth-Cugell, C. (1984). Visual adaptation and retinal gain controls. Progress

in Retinal Research, 3, 263–346.

Sharpe, L. T., Stockman, A., Jagla, W., & Jägle, H. (2005). A luminous efficiency function,

V*(λ), for daylight adaptation. Journal of Vision, 5(11), 3, 948–968.

160
 References

Silver, M., Ress, D., & Heeger, D. (2005). Topographic maps of visual spatial attention in

human parietal cortex. Journal of Neurophysiology, 94(2), 1358–1371.

Sirotin, Y. B. & Das, A. (2009). Anticipatory haemodynamic signals in sensory cortex not

predicted by local neuronal activity. Nature, 457(7228), 475–479.

Smith, A. T., Greenlee, M. W., Singh, K. D., Kraemer, F. M., & Hennig, J. (1998). The pro-

cessing of first- and second-order motion in human visual cortex assessed by functional

magnetic resonance imaging (fMRI). Journal of Neuroscience, 18(10), 3816–3830.

Smith, V. C. & Pokorny, J. (1975). Spectral sensitivity of the foveal cone photopigments

between 400 and 500 nm. Vision Research, 15(2), 161–171.

Smithson, H. & Zaidi, Q. (2004). Colour constancy in context: roles for local adaptation and

levels of reference. Journal of Vision, 4(9), 3, 693–710.

Smithson, H. E. (2005). Sensory, computational and cognitive components of human colour

constancy. Philosophical Transactions of the Royal Society of London Series B-Biological Sciences,

360(1458), 1329–1346.

Solomon, S. G. & Lennie, P. (2005). Chromatic gain controls in visual cortical neurons. Journal

of Neuroscience, 25(19), 4779–4792.

Solomon, S. G. & Lennie, P. (2007). The machinery of colour vision. Nature Reviews Neuro-

science, 8(4), 276–286.

Solomon, S. G., Peirce, J. W., Dhruv, N. T., & Lennie, P. (2004). Profound contrast adaptation

early in the visual pathway. Neuron, 42(1), 155–162.

Stockman, A. & Plummer, D. J. (2005). Spectrally opponent inputs to the human luminance

pathway: slow +L and -M cone inputs revealed by low to moderate long-wavelength

adaptation. Journal of Physiology, 566(1), 77–91.

Stockman, A., Plummer, D. J., & Montag, E. D. (2005). Spectrally opponent inputs to

the human luminance pathway: slow +M and -L cone inputs revealed by intense long-

wavelength adaptation. Journal of Physiology, 566(1), 61–76.

161
 References

Stockman, A. & Sharpe, L. T. (2000). The spectral sensitivities of the middle- and long-

wavelength-sensitive cones derived from measurements in observers of known genotype.

Vision Research, 40(13), 1711–1737.

Stringham, J. M., Hammond, B. R., Wooten, B. R., & Snodderly, D. M. (2006). Compensation

for light loss resulting from filtering by macular pigment: relation to the S-cone pathway.

Optom Vis Sci, 83(12), 887–894.

Sumner, P., Anderson, E. J., Sylvester, R., Haynes, J. D., & Rees, G. (2008). Combined ori-

entation and colour information in human V1 for both L-M and S-cone chromatic axes.

Neuroimage, 39(2), 814–824.

Swisher, J. D., Halko, M. A., Merabet, L. B., McMains, S. A., & Somers, D. C. (2007). Visual

topography of human intraparietal sulcus. Journal of Neuroscience, 27(20), 5326–5337.

Tailby, C., Solomon, S. G., Dhruv, N. T., & Lennie, P. (2008a). Habituation reveals funda-

mental chromatic mechanisms in striate cortex of macaque. Journal of Neuroscience, 28(5),

1131–1139.

Tailby, C., Solomon, S. G., & Lennie, P. (2008b). Functional asymmetries in visual pathways

carrying S-cone signals in macaque. Journal of Neuroscience, 28(15), 4078–4087.

Tailby, C., Szmajda, B. A., Buzás, P., Lee, B. B., & Martin, P. R. (2008c). Transmission of

blue (S) cone signals through the primate lateral geniculate nucleus. Journal of Physiology,

586(24), 5947–5967.

Tamura, N., Tsumura, N., & Miyake, Y. (2003). Masking model for accurate colorimetric

characterization of LCD. Journal of the Society for Information Display, 11(2), 333–339.

Tanaka, J., Weiskopf, D., & Williams, P. (2001). The role of color in high-level vision. Trends

in Cognitive Sciences, 5(5), 211–215.

Thielscher, A., Kölle, M., Neumann, H., Spitzer, M., & Grön, G. (2008). Texture segmentation

in human perception: A combined modeling and fMRI study. Neuroscience, 151(3), 730–

736.

162
 References

Tootell, R. B. & Hadjikhani, N. (2001). Where is ‘dorsal V4’ in human visual cortex? Retino-

topic, topographic and functional evidence. Cerebral Cortex, 11(4), 298–311.

Tootell, R. B., Hadjikhani, N., Hall, E. K., Marrett, S., Vanduffel, W., Vaughan, J. T., & Dale,

A. M. (1998). The retinotopy of visual spatial attention. Neuron, 21(6), 1409–1422.

Tootell, R. B., Mendola, J. D., Hadjikhani, N. K., Ledden, P. J., Liu, A. K., Reppas, J. B., Sereno,

M. I., & Dale, A. M. (1997). Functional analysis of V3A and related areas in human visual

cortex. Journal of Neuroscience, 17(18), 7060–7078.

Tootell, R. B., Reppas, J. B., Kwong, K. K., Malach, R., Born, R. T., Brady, T. J., Rosen, B. R., &

Belliveau, J. W. (1995). Functional analysis of human MT and related visual cortical areas

using magnetic resonance imaging. Journal of Neuroscience, 15(4), 3215–3230.

Tyler, C., Likova, L., Chen, C., Kontsevich, L., Schira, M., & Wade, A. (2005). Extended

concepts of occipital retinotopy. Current Medical Imaging Reviews, 1(3), 319–329.

van Essen, D. C. & Zeki, S. M. (1978). The topographic organization of rhesus monkey

prestriate cortex. Journal of Physiology, 277, 193–226.

Vinberg, J. & Grill-Spector, K. (2008). Representation of shapes, edges, and surfaces across

multiple cues in the human visual cortex. Journal of Neurophysiology, 99(3), 1380–1393.

Viswanathan, A. & Freeman, R. D. (2007). Neurometabolic coupling in cerebral cortex re-

flects synaptic more than spiking activity. Nature Neuroscience, 10(10), 1308–1312.

von Helmholtz, H. (1866). Physiological optics. In MacAdam, D. L. (Ed.) Sources of Color

Science (1970). (pp. 84–100). Cambridge, MA: MIT Press.

von Kries, J. (1902). Chromatic adaptation. In MacAdam, D. (Ed.) Sources of Color Science

(1970). (pp. 109–119). Cambridge, MA: MIT Press.

von Kries, J. (1905). Influence of adaptation on the effects produced by luminous stimuli. In

MacAdam, D. L. (Ed.) Sources of Color Science (1970). (pp. 120–126). Cambridge, MA: MIT

Press.

163
 References

Vrhel, M. J., Gershon, R., & Iwan, L. S. (1994). Measurement and analysis of object reflectance

spectra. COLOR research and application, 19(1), 4–9.

Wade, A., Augath, M., Logothetis, N., & Wandell, B. (2008). fMRI measurements of color in

macaque and human. Journal of Vision, 8(10), 6, 1–19.

Wade, A. R., Brewer, A. A., Rieger, J. W., & Wandell, B. A. (2002). Functional measurements

of human ventral occipital cortex: Retinotopy and colour. Philosophical Transactions of the

Royal Society of London Series B-Biological Sciences, 357(1424), 963–973.

Walsh, V., Carden, D., Butler, S. R., & Kulikowski, J. J. (1993). The effects of V4 lesions on the

visual abilities of macaques: Hue discrimination and colour constancy. Behavioural Brain

Research, 53(1-2), 51–62.

Walsh, V., Kulikowski, J. J., Butler, S. R., & Carden, D. (1992). The effects of lesions of area

V4 on the visual abilities of macaques: Colour categorization. Behavioural Brain Research,

52(1), 81–89.

Wandell, B. A. (1999). Computational neuroimaging of human visual cortex. Annual Review

of Neuroscience, 22, 145–173.

Wandell, B. A., Brewer, A. A., & Dougherty, R. F. (2005). Visual field map clusters in human

cortex. Philosophical Transactions of the Royal Society of London Series B-Biological Sciences,

360(1456), 693–707.

Wandell, B. A., Dumoulin, S. O., & Brewer, A. A. (2007). Visual field maps in human cortex.

Neuron, 56(2), 366–383.

Wandell, B. A., Poirson, A. B., Newsome, W. T., Baseler, H. A., Boynton, G. M., Huk, A.,

Gandhi, S., & Sharpe, L. T. (1999). Color signals in human motion-selective cortex. Neuron,

24(4), 901–909.

Webster, M. A. (1996). Human colour perception and its adaptation. Network: Computation

in Neural Systems, 7, 587–634.

Webster, M. A. & Mollon, J. D. (1991). Changes in colour appearance following post-

receptoral adaptation. Nature, 349(6306), 235–238.

164
 References

Webster, M. A. & Mollon, J. D. (1994). The influence of contrast adaptation on color appear-

ance. Vision Research, 34(15), 1993–2020.

Westland, S. & Ripamonti, C. (2000). Invariant cone-excitation ratios may predict trans-

parency. Journal of the Optical Society of America A-Optics Image Science and Vision, 17(2),

255–264.

Wiesel, T. & Hubel, D. (1966). Spatial and chromatic interactions in the lateral geniculate

body of the rhesus monkey. Journal of Neurophysiology, 29, 1115–1156.

Wild, H. M., Butler, S. R., Carden, D., & Kulikowski, J. J. (1985). Primate cortical area V4

important for color constancy but not wavelength discrimination. Nature, 313(5998), 133–

135.

Wilkinson, F., James, T. W., Wilson, H. R., Gati, J. S., Menon, R. S., & Goodale, M. A. (2000).

An fMRI study of the selective activation of human extrastriate form vision areas by radial

and concentric gratings. Current Biology, 10(22), 1455–1458.

Williams, D. & Roorda, A. (1999). The trichromatic cone mosaic in the human eye. In

Gegenfurtner, K. & Sharpe, L. (Eds.) Color Vision: From Genes to Perception. (pp. 113–122).

Cambridge, UK: Cambridge University Press.

Williams, D. R., MacLeod, D. I., & Hayhoe, M. M. (1981). Foveal tritanopia. Vision Research,

21(9), 1341–1356.

Winawer, J., Horiguchi, H., Sayres, R., Amano, K., & Wandell, B. (2010). Mapping hV4 and

ventral occipital cortex: The venous eclipse. Journal of Vision, 10(5), 1, 1–22.

Worthey, J. A. (1985). Limitations of color constancy. Journal of the Optical Society of America

A, 2(7), 1014–1026.

Worthey, J. A. & Brill, M. H. (1986). Heuristic analysis of von Kries color constancy. Journal

of the Optical Society of America A, 3(10), 1708–1712.

Wyszecki, G. & Stiles, W. S. (1967). Color Science: Concepts and Methods, Quantitative Data and

Formulas. John Wiley & Sons, Inc.

165
 References

Wässle, H. (2004). Parallel processing in the mammalian retina. Nature Reviews Neuroscience,

5(10), 747–757.

Xiao, Y., Casti, A., Xiao, J., & Kaplan, E. (2007). Hue maps in primate striate cortex. Neuroim-

age, 35(2), 771–786.

Xiao, Y., Wang, Y., & Felleman, D. J. (2003). A spatially organized representation of colour

in macaque cortical area V2. Nature, 421(6922), 535–539.

Young, T. (1802). On theory of light and colours. In MacAdam, D. L. (Ed.) Sources of Color

Science (1970). (pp. 51–52). Cambridge, MA: MIT Press.

Yushkevich, P. A., Piven, J., Hazlett, H. C., Smith, R. G., Ho, S., Gee, J. C., & Gerig, G. (2006).

User-guided 3D active contour segmentation of anatomical structures: Significantly im-

proved efficiency and reliability. Neuroimage, 31(3), 1116–1128.

Zaidi, Q. & Halevy, D. (1993). Visual mechanisms that signal the direction of color changes.

Vision Research, 33(8), 1037–1051.

Zaidi, Q. & Shapiro, A. G. (1993). Adaptive orthogonalization of opponent-color signals.

Biological Cybernetics, 69(5-6), 415–428.

Zeki, S. (1983). Colour coding in the cerebral cortex: The responses of wavelength-selective

and colour-coded cells in monkey visual cortex to changes in wavelength composition.

Neuroscience, 9(4), 767–781.

Zeki, S. (1990). A century of cerebral achromatopsia. Brain, 113, 1721–1777.

Zeki, S. (2003). Improbable areas in the visual brain. Trends in Neurosciences, 26(1), 23–26.

Zeki, S. & Bartels, A. (1999). The clinical and functional measurement of cortical (in)activity

in the visual brain, with special reference to the two subdivisions (V4 and V4 alpha) of

the human colour centre. Philosophical Transactions of the Royal Society of London Series B-

Biological Sciences, 354(1387), 1371–1382.

Zeki, S., Watson, J. D. G., Lueck, C. J., Friston, K. J., Kennard, C., & Frackowiak, R. S. J.

166
 References

(1991). A direct demonstration of functional specialization in human visual-cortex. Journal

of Neuroscience, 11(3), 641–649.

Zeki, S. M. (1973). Colour coding in rhesus monkey prestriate cortex. Brain Research, 53(2),

422–427.

Zeki, S. M. (1977). Colour coding in the superior temporal sulcus of rhesus monkey visual

cortex. Proceedings of the Royal Society of London, Series B-Biological Sciences, 197(1127), 195–

223.

167
Appendices

168
Appendix A

Monitor Calibration

In carrying out the experiments described in this thesis, I used a standard Cathode Ray Tube

(CRT) monitor for the psychophysics experiment described in Chapter 7, a Liquid Crystal

Display (LCD) monitor for the fMRI experiment described in Chapter 6, and a Digital Light

Processing (DLP) projector for the fMRI experiment described in Chapter 5.

A.1 Estimating the cone fundamental primaries (LMS) of each RGB

coordinate

In all experiments, I took into account changes in both chromaticity and luminance with

increasing R, G and B values.

A.1.1 Measuring the xyY coordinates for increasing R, G and B values

For the CRT monitor, I measured the xyY values of each gun in 32 evenly spaced intensity

steps using a ColourCal driven by software from Cambridge Research Systems. As seen in

Figure A.1, luminance (Y) values increased with intensity and were fitted using a gamma

function, which was used to interpolate the values. The chromaticity coordinates (xy) of the

guns were relatively constant for high, but not low intensities. To ensure precise rendering

169
 Appendix A. Monitor Calibration

of the stimuli, the chromaticity coordinates of each gun were also interpolated, this time

using a spline fitted to the data.

A
Luminance (cd/m2)

20 1 1

y Coordinate
x Coordinate
0.5 0.5

0 0 0
0 64 128 192 256 0 64 128 192 256 0 64 128 192 256

B
Luminance (cd/m2)

50 1 1

y Coordinate
x Coordinate

0.5 0.5

0 0 0
0 64 128 192 256 0 64 128 192 256 0 64 128 192 256

C
Luminance (cd/m2)

10 1 1
y Coordinate
x Coordinate

0.5 0.5

0 0 0
0 64 128 192 256 0 64 128 192 256 0 64 128 192 256

Figure A.1: For increasing intensities of red (A), green (B) and blue (C), the CIE coordinates of the
monitor was measured using a ColorCAL colorimeter (by Cambridge Research Systems
Ltd.). The black crosses show the readings obtained, and the line plots show the best
fitting gamma function (in the case of luminance values), and spline (in the case of xy
chromaticity coordinates) that were used to interpolate the data.

For the LCD monitor, I used a similar approach, measuring the xyY coordinates of the out-

put for increasing R, G and B values in 16 evenly spaced steps, using a PR-655 SpectraScan

spectrophotometer (by Photo Research Inc.). I carried out this calibration in situ, from the

same approximate viewing distance as it was viewed in the scanner, taking measurements

from the mirror through which subjects viewed the stimuli. Figure A.2 shows the measured

xyY values, along with the best fitting splines. Unlike the CRT monitor, where the increasing

luminance with R, G and B values is well described by a gamma function, the LCD shows

170
 Appendix A. Monitor Calibration

a sigmoidal increase or ‘S’ shaped curve of increasing luminance values, which was fitted

with a spline instead of a gamma function. The splines fitted the measured luminance and

chromaticity values well, except perhaps at high intensities, but the stimuli for this experi-

ment were displayed in a fairly narrow range of pixel intensities, centred on the background

grey, as illustrated by the grey bars in Figure A.2.

A
Luminance (cd/m2)

20 1 1

y Coordinate
x Coordinate
0.5 0.5

0 0 0
0 64 128 192 256 0 64 128 192 256 0 64 128 192 256

B
Luminance (cd/m2)

50 1 1

y Coordinate
x Coordinate

0.5 0.5

0 0 0
0 64 128 192 256 0 64 128 192 256 0 64 128 192 256

C
Luminance (cd/m2)

10 1 1
y Coordinate
x Coordinate

0.5 0.5

0 0 0
0 64 128 192 256 0 64 128 192 256 0 64 128 192 256

Figure A.2: For increasing intensities of red (A), green (B) and blue (C), the CIE coordinates of the
monitor were measured using a PR-655 SpectraScan spectrophotometer (by Photo Re-
search Inc.), from the same approximate viewing distance as it was viewed in the scan-
ner, from the mirror through which subjects viewed the stimuli. The black crosses show
the readings obtained, and the line plots the best fitting spline that was used to interpo-
late the data. The ranges of intensities used in the non-cardinal colour experiment are
indicated by the grey shaded region in each plot.

171
 Appendix A. Monitor Calibration

A.1.2 Estimating the XYZ chromaticity coordinate of each combination of RGB

intensities

In both experiments, I used the measured R, G and B values to estimate the xyY coordinate

of each RGB combination, generating a 256 x 256 x 256 matrix of possible outputs of each

monitor. For each combination of R, G and B intensities, I calculated the xyY of each of the

red, green and blue channels using the interpolated values in Figures A.1 and A.2. I then

calculated the XYZ chromaticity coordinate of that combination of values using the method

described in Brainard (1996), as follows:

If the interpolated x, y and Y coordinates of the red channel at an intensity of i are given by

Rxi , Ryi and RYi , respectively, then the z chromaticity coordinate (Rzi ) is given by:

Rzi  1
Rxi
Ryi (A.1)

Similarly, the z chromaticity coordinates of the green (G) and blue (B) channels, at intensities

j and k are given by:

Gz j  1
Gx j
Gy j (A.2)

and:

Bz j  1
Bx j
By j (A.3)

These values can be used to generate the conversion matrix for these channel intensities

(Ci jk ), given by:

 
Gx j
 
Rx i Bxk



Ry i Gy j

Byk

Ci jk   1 1 1 
 (A.4)
 Rzi Gz j

Bzk
Ryi Gy j Byk

172
 Appendix A. Monitor Calibration

Equation A.4 is adapted from Equation A.3.9 in Brainard (1996). Note that if chromaticity

values (x and y) did not change with increasing luminance, this matrix would be the same

for each combination of R, G and B intensities.

Then the XYZ chromaticity coordinate of the combined output of the three channels at in-

tensities R  i, G  j and B  k is given by:

 


RYi 

XYZi jk  Ci jk  
GY j 
 
(A.5)

BYk

This transformation assumes a negligible effect of interaction between red, green and blue

channels, and no variation in chromaticity of the monitor across space.

A.1.3 Converting the XYZ values to LMS cone fundamental primaries

I then converted the XYZ coordinates of each possible output of the monitor to cone funda-

mental primaries (LMS), to estimate the response the stimulus would elicit in each type of

cone. For this calculation I again followed the procedures outlined in Brainard (1996).

I used the CIE 1931 XYZ colour matching functions and the Stockman-Sharpe 2 degree

cone fundamentals (Stockman & Sharpe, 2000) that are found in the PsychColorimetric-

Data folder of the PsychToolbox routines for Matlab(Brainard, 1997; Pelli, 1997). Both the

XYZ colour matching functions and the cone fundamentals are given as a function of wave-

length; after removing values at wavelengths which are stored for one but not the other,

they range from 390nm to 780nm in increments of 5nm (a total of 79 values). If MXYZ is a 79 x

3 matrix with the colour matching functions for X, Y and Z stored in the columns, and MLMS

is a 79 x 3 matrix with the 2 degree cone fundamentals for L, M and S stored in the columns,

then the 3 x 3 conversion matrix Mconvert is given by:

Mconvert  p683  MXYZ q
1 MLMS (A.6)

173
 Appendix A. Monitor Calibration

where 683 is a scaling factor that ensures that the calculated luminance (Y) will have units

of candelas/m2 . Using this conversion matrix Mconvert , I transformed each of the XYZ chro-

maticity coordinates calculated above into LMS cone fundamental primaries, according to

the formula:

   


L 
 

X


 M  Mconvert  Y  (A.7)
   
S Z

The LMS values for each possible combination of R, G and B intensities were compared to

the target LMS values of the experimental stimuli to find the closest match in each case.

Cone fundamental primaries, rather than any other coordinates, were chosen as a physio-

logically meaningful dimension along which to compare values, in which the coordinates

are expressed in comparable units (quantal sensitivity). The error (E) between the target

stimulus colour (LMS stim )and each combination of RGB intensities for the monitor (LMS mon )

was defined as the distance between each pair of points in the 3 dimensional LMS space,

given by:

b
E  pLstim
Lmonq2 p Mstim
Mmonq2 pS stim
S monq2 (A.8)

In cases where E ¡ 0.001, or where the best fitting RGB coordinate included any of the red,
green or blue channels at maximum intensity, the stimulus colour was excluded as out of

range. The mean L, M and S values for the psychophysics experiment in Chapter 7 were

0.0224, 0.0162 and 0.0098 respectively. The maximum and minimum L, M and S values for

this experiment were plotted earlier in Figure 7.6. For the fMRI experiment described in

Chapter 6 the same exclusion criteria were applied, but no values were excluded since the

stimuli were all within range. This was expected since the magnitude of each chromatic

modulation was defined as 90% of the maximum modulation achievable in the gamut of the

monitor.

174
 Appendix A. Monitor Calibration

A.2 Modelling the interaction between RGB channels (DLP only)

As mentioned above, the method described so far assumes a negligible effect of interaction

between red, green and blue channels. For the DLP projector, I chose to correct for any

interaction between these channels, since these interactions can be significant. For exam-

ple, Seime & Hardeberg (2003) report that DLP projectors with a single digital micromirror

device (DMD) frequently utilise a white channel which is active only at high luminance val-

ues to increase the range of achievable brightness values. When testing the Dell 5100 MP, I

found a luminance increase for white only compared with that predicted by summing the

R, G and B channels (illustrated in Figure A.3), consistent with this model including a white

channel. I used the ‘Masking Model’ of Tamura et al. (2003) to model this and any other

interactions between channels while restricting the number of measurements made during

calibration.

For the calibration of the Dell 5100 MP (DLP) projector used in the experiment described in

Chapter 5, I measured the xyY coordinates of the projection using the PR-655 SpectraScan

spectrophotometer (by Photo Research Inc.). In addition to measuring these values for in-

creasing values of R, G and B, I also measured the coordinates for increasing values of cyan

(G  B,R  0), magenta (R  B, G  0), yellow (R  G, B  0) and grey (R  G  B). The

luminance (Y) and chromaticity (xy) values (16 for each of R, G, B, C, M, Y and K) were each

interpolated to 256 values with a spline; the raw data along with the interpolated values are

shown in Figure A.3.

Using the ‘Masking Model’ (Tamura et al., 2003), I took into account changes in both chro-

maticity and luminance of the screen with increasing R, G and B values, along with inter-

actions between the G B (cyan), R B (magenta), R G (yellow) and R G B (grey)

channels when estimating the xyY coordinate of each RGB combination. The basic principle

of the model is illustrated for an example case in Figure A.4. In accordance with this model,

the XYZ of a given RGB combination is estimated using at most three of the seven channels

measured. Where R , G, R , B and G , B, the channel (R, G or B) of highest value (i) is

designated the primary P colour, the sum of the primary colour and the channel (R, G or B)

175
 Appendix A. Monitor Calibration

of next highest value ( j) is designated the secondary S colour, and a grey (R G B) of the

lowest value (k) is the tertiary colour (T ).

In a modified version of Equation A.4, where R, G and B are replaced with P, S and T , the

conversion matrix for a given set of channel intensities (Ci jk ) is given by:

 
Px j S xj
 
Pxi S xk T xk



Pyi Py j S yj S yk

T yk

Ci jk   1 1 1 1 1 
 (A.9)
 Pzi Pz j S zj S zk

T zk
Pyi Py j S yj S yk T yk

and Equation A.5 is modified to give:

 


PYi




PY j 
 
XYZi jk  Ci jk  
 S Yj   (A.10)

 


S Yk 
 
T Yk

I used this method when preparing the stimuli for the experiment described in Chapter 5, in

which I presented subjects with colour and achromatic versions of movie clips. The colour

version of the movie was presented without any scaling of the RGB values imported from

the commercial DVD (see Materials and Methods in Chapter 5 for more detail). Using the

masking model, for each RGB value in the colour version of the movie, I found its luminance

(Y), and then found the achromatic value (R G B) of the same photopic luminance

using the interpolated values shown in Figure A.3, G. This ensured the stimuli were closely

matched for luminance along with all other scene features except for colour.

176
 Appendix A. Monitor Calibration

A Luminance (cd/m2 ) 100 1 1

y Coordinate
x Coordinate
0 0 0
0 256 0 256 0 256
B 1
400 1
Luminance (cd/m2 )

y Coordinate
x Coordinate
0 0 0
0 256 0 256 0 256
C 1
50 1
Luminance (cd/m2 )

y Coordinate
x Coordinate

0 0 0
0 256 0 256 0 256
D 1
500 1
Luminance (cd/m2 )

y Coordinate
x Coordinate

0 0 0
0 256 0 256 0 256

E 1
200 1
Luminance (cd/m2 )

y Coordinate
x Coordinate

0
0 0 0 256
0 256 0 256
F 1
500 1
Luminance (cd/m2 )

y Coordinate
x Coordinate

0
0 0 0 256
0 256 0 256
G 1
1000 1
Luminance (cd/m2 )

y Coordinate
x Coordinate

0
0 0 0 256
0 256 0 256

Figure A.3: Measured xyY coordinates for the DLP projector at St. Vincent’s hospital. The CIE
coordinates of the monitor for increasing values of R, G, B, C, M, Y and K are shown in
A to G respectively. Measurements shown were recorded using a PR-655 SpectraScan
spectrophotometer (by Photo Research Inc.) through the door of the MRI room. In this
case the measured projection was not reflected through the mirror that subjects used
to view the stimuli, but in separate measurements the mirror was found to have flat
spectral properties. The dark grey crosses show the readings obtained, the coloured
line plots the best fitting spline that was used to interpolate the data, and the dashed
black line shows the CMYK values based on predictions from measured RGB values,
assuming that the three channels are independent.

177
 Appendix A. Monitor Calibration

1
(proportion of maximum)
Channel Intensity

i
P = R i- R j
(red, R)
j
S = Yj - Yk
(yellow, R+G)
k
T = Kk
(grey, R+G+B)
0
R G B

Figure A.4: Schematic illustrating the principle of the Masking Model, adapted from Tamura et al.
(2003), Figure 3. In the example illustrated, where the intensity of R ¡ G ¡ B, the
resultant output is approximated by the sum of the primary (P), secondary (S ) and
tertiary (T ) colours indicated on the right of the figure (red, yellow and grey). For this
case the output is given by pRi
R j q pY j
Yk q Kk .

178
Appendix B

Published Paper Resulting from

Thesis

179
Journal of Vision (2010) 10(5):25, 1–17 http://journalofvision.org/content/10/5/25 1

Combination of subcortical color channels in human

visual cortex
School of Psychology, The University of Sydney, and the
Australian Centre of Excellence in Vision Science,
Erin Goddard Sydney, Australia

School of Psychology, The University of Sydney, and the

Australian Centre of Excellence in Vision Science,
Damien J. Mannion Sydney, Australia

School of Psychology, The University of Sydney,

J. Scott McDonald Sydney, Australia

School of Medical Sciences and Bosch Institute,

The University of Sydney, and the Australian
Centre of Excellence in Vision Science,
Samuel G. Solomon Sydney, Australia

School of Psychology, The University of Sydney,

and the Australian Centre of Excellence
Colin W. G. Clifford in Vision Science, Sydney, Australia

Mechanisms of color vision in cortex have not been as well characterized as those in sub-cortical areas, particularly in
humans. We used fMRI in conjunction with univariate and multivariate (pattern) analysis to test for the initial transformation
of sub-cortical inputs by human visual cortex. Subjects viewed each of two patterns modulating in color between orange-
cyan or lime-magenta. We tested for higher order cortical representations of color capable of discriminating these stimuli,
which were designed so that they could not be distinguished by the postulated L–M and S–(L + M) sub-cortical opponent
channels. We found differences both in the average response and in the pattern of activity evoked by these two types of
stimuli, across a range of early visual areas. This result implies that sub-cortical chromatic channels are recombined early in
cortical processing to form novel representations of color. Our results also suggest a cortical bias for lime-magenta over
orange-cyan stimuli, when they are matched for cone contrast and the response they would elicit in the L–M and S–(L + M)
opponent channels.
Keywords: color perception, fMRI, multivariate pattern classifiers, occipital cortex, vision
Citation: Goddard, E., Mannion, D. J., McDonald, J. S., Solomon, S. G., & Clifford, C. W. G. (2010). Combination of
subcortical color channels in human visual cortex. Journal of Vision, 10(5):25, 1–17, http://journalofvision.org/content/10/5/25,
doi:10.1167/10.5.25.

well understood, although psychophysical adaptation

Introduction experiments indicate the existence of higher-order color
mechanisms in the human visual system, which receive
Our rich experience of color includes the ability to input from both the opponent channels of sub-cortical areas
discriminate and identify a diverse range of combinations (Krauskopf, Williams, Mandler, & Brown, 1986; Webster
of hue, saturation and luminance, yet our perceptual & Mollon, 1991; Zaidi & Shapiro, 1993). In macaque it
experience is based on the activity of just three categories has been demonstrated that there are cells as early as V1
of cone photoreceptor and the transformation of these which prefer chromatic directions away from the cardinal
signals by sub-cortical and cortical areas. At the sub- directions that isolate the L–M and S–(L + M) mecha-
cortical level, there exist chromatically opponent channels nisms (Conway, 2001; de Valois, Cottaris, Elfar, Mahon,
(L–M and S–(L + M)) that carry information in parallel to & Wilson, 2000), implying a combination of information
visual cortex via the parvocellular and koniocellular layers from the sub-cortical channels early in visual cortex. For
of the LGN (Derrington, Krauskopf, & Lennie, 1984). cortical cells to have chromatic preference intermediate to
Cortical mechanisms of color vision are generally less the cardinal axes, there must be some combination of the
doi: 1 0. 11 67 / 1 0 . 5 . 2 5 Received November 16, 2009; published May 21, 2010 ISSN 1534-7362 * ARVO
Journal of Vision (2010) 10(5):25, 1–17 Goddard et al. 2

L–M and S–(L + M) channels. In a recent fMRI study, contains information about the stimulus dimension that was
Brouwer and Heeger (2009) found that the principal varied. Here, our premise is that if we can use the activity
components of the response in V1 are consistent with a across a visual area to discriminate between our stimuli
response dominated by an opponent coding of color, as then that area contains a representation of color that could
found in sub-cortical areas, but that by hV4 and VO it more only be generated through a transformation of the signals
closely resembles our perceptual color space. That is, V1 from the sub-cortical L–M and S–(L + M) pathways.
shows a differential response to variations in color but not a
continuous representation of hue, while in higher areas
colors of similar hue evoke similar responses. This finding
does not rule out the possibility that the signals of the Materials and methods
fundamental pathways (L–M and the S–(L + M)) are
combined in the early visual areas, such as V1, a possibility
we address specifically in this study. Color calibration procedures and display
We obtained high resolution functional images of the system
BOLD (blood-oxygen-level-dependent) response from
subjects’ occipital and parietal lobes while they viewed Stimuli were generated and displayed using Matlab
colored stimuli. Previous studies of cortical chromatic (version 7) software on a Dell Latitude laptop computer
mechanisms in humans have used perceptually relevant driving an nVidia Quadro NVS 110M graphics card to draw
hues (Brouwer & Heeger, 2009; Parkes, Marsman, Oxley, stimuli to a 35 ! 26 cm Philips LCD monitor, with 60 Hz
Goulermas, & Wuerger, 2009). Our stimuli were not refresh rate, viewed from a distance of approximately
chosen for their perceptual relevance, but were designed 1.58 m. Scanning took place in a darkened room. Subjects,
to be metameric to the hypothesized sub-cortical chro- while lying in the scanner, viewed the monitor through a
matic mechanisms. Specifically, the stimuli were designed mirror mounted above the head cage which reflected the
so as to fulfill the following conditions: (1) to induce the image from the LCD monitor located behind the scanner.
same magnitude of activity in the L–M opponent channel; Stimuli were calibrated in situ for the LCD monitor and
and (2) to induce the same magnitude of activity in the mirror arrangement, using measurements obtained with a
S–(L + M) opponent channel. This was achieved by PR-655 SpectraScan spectrophotometer (by Photo
combining a given L–M modulation with a given S-cone Research Inc.).
isolating modulation in each of two different phases. When Changes in both chromaticity and luminance of the
jS was in phase with M then the stimulus appeared lime- screen with increasing R, G and B values were taken into
magenta; when +S was in phase with M then it appeared account when generating the experimental stimuli. The
orange-cyan. The lime-magenta and orange-cyan stimuli CIE (xyY) coordinates measured for 16 values during
can only be distinguished by the BOLD signal if there are calibration were interpolated to 255 values using the best-
cells which receive a combination of inputs from the L–M fitting spline, and these were used to calculate the luminance
and the S–(L + M) pathways. and chromaticity for each combination of R, G and B
Univariate and multivariate analyses tested whether the intensity values.
BOLD response within each cortical visual area depended Data were collected on five subjects (three male), aged
upon the color of the stimulus. Univariate analyses show between 24 and 40 years, with normal or corrected to
what information about the stimulus is detectable in the normal visual acuity and normal color vision, as tested
average activity across a region, while multivariate classi- using Ishihara plates (Ishihara, 1990). All subjects pro-
fiers are capable of also learning differences in the pattern vided informed consent, and the entire study was carried
of activation between stimuli. Multivariate classification out in accordance with guidelines of the University of
analysis (for a review, see Haynes and Rees, 2006) is a Sydney Human Research Ethics Committee.
useful tool to test for differences in the BOLD response of
a visual area even where the mean activity of the area is
not significantly different between stimuli, and has been Chromatic, spatial and temporal stimulus
used to infer the selectivity of different early visual areas properties
for a range of basic visual attributes and their combination
(Haynes & Rees, 2005a, 2005b; Kamitani & Tong, 2005, Example stimuli are shown in Figure 1. The stimulus
2006; Mannion, McDonald, & Clifford, 2009; Parkes et al., was an annulus, centered on the fixation point, with an
2009; Seymour, Clifford, Logothetis, & Bartels, 2009; inner diameter of 0.8 degrees visual angle, and an outer
Sumner, Anderson, Sylvester, Haynes, & Rees, 2008). diameter of 7.8 degrees. The remainder of the screen was
Both in the univariate and multivariate analyses held at the mean luminance, which was 6.78 cd/m2 [CIE
employed here, an algorithm is trained to classify the (1931) x, yÈ0.300, 0.337], and all stimuli were produced
stimulus from activity across a region, and tested on novel by spatiotemporal modulation around this point. The
data. Above chance performance indicates that the area annulus contained a spatial pattern that counterphased
Journal of Vision (2010) 10(5):25, 1–17 Goddard et al. 3

Figure 1. Stimuli used in the fMRI experiment; A: the color of the stimulus modulated sinusoidally between the upper plaid and the lower
plaid at 1 Hz. The stimuli on the left are orange and cyan, on the right, the stimuli are lime and magenta. For both color pairs, minimum
motion was used to determine each subjects’ perceived equiluminance point, and a 25% luminance modulation was added. In the first
and third pair of stimuli the light/dark modulation is paired with cyan/orange and lime/magenta, respectively. In the second and fourth
stimuli these pairings are reversed. B: Example stimulus with fixation task. At fixation, there was a light gray cross surrounded by a high
contrast ring, as illustrated above. The high contrast ring provided feedback to subjects when they made small eye movements, since an
afterimage would become visible. While subjects fixated on the central cross (partially obscured by the digit), they were required to
respond with a button press whenever either of two target items were presented in the digit stream. Digits updated at 3 Hz, were
presented in random order, and could be 0 to 9 inclusive, each in either black or white. The target items were a conjunction of a digit and a
particular color, for example either a black 3 (shown here) or a white 7. The fixation task was unrelated to the experimental stimulus, which
was presented in the annular region surrounding fixation. C: Modulation of cardinal sensitive mechanisms over time in the experimental
stimulus, for an example 18 seconds including a transition from a light magenta-dark lime block to a light cyan-dark orange block. At each
transition, the non-cardinal modulation switched from a lime-magenta block to an orange-cyan block or vice versa. At each transition there
was a phase reversal in either the L–M or the S–(L + M) cardinal modulation; here there is a phase reversal in the L–M modulation at the
time of transition. The luminance (light–dark) modulation had no reversals at any time. The amplitude of response of any cardinal sensitive
mechanisms should be constant across the lime-magenta and orange-cyan blocks. Phase reversals are the only cue that could be used
to discriminate the stimuli where the cardinal sensitive mechanisms are kept independent, but the block order was balanced such that
this cue could not be used to predict the stimulus. We used one of two block orders for each run, where the second was a reversal of the
first order.

sinusoidally at a temporal frequency of 1 Hz. The spatial the L and M cones vary in proportion. The axes were
pattern was the multiplication of a radial and a concentric derived from the Stockman and Sharpe (2000) 2-degree
sinusoidal modulation, (the resultant plaid pattern is shown cone spectral sensitivities and adjusted individually for
in Figures 1A and 1B). All these modulations can be each observer (see below). The scaling of these axes is
represented in a three-dimensional color space described largely arbitrary; here we used modulations along the
previously (Derrington et al., 1984; DKL space). Along isoluminant axes that were 90% of the maximum modu-
the L–M axis only the signals from L and M cones vary, lation achievable within the gamut of the monitor.
in opposition, without variation in luminance. Along the Modulation along the L–M axis produced maximum cone
orthogonal S-cone isolating axis there is no modulation of contrasts of 15.4% in the L-cone and 17.8% in the M-cone;
either the L or M cones. The L–M and S axes define a along the S-cone axis the maximum S-cone contrast was
plane in which only chromaticity varies. Normal to this 79.6%. Cone contrast values for all stimuli are listed in
plane is the luminance axis along which the signals from Table 1. Each frame of the stimulus was generated prior
 Journal of Vision (2010) 10(5):25, 1–17 Goddard et al. 4

L-cone M-cone S-cone analyses shown in Figure 4 the classifier was trained and
t1.1 Stimulus Color Contrast Contrast Contrast tested with two groups of blocks: one group included the
two types of lime-magenta blocks (the two types differed
t1.2 Dark Cyan j0.154 0.015 0.688 in the relative phase of the luminance modulation) and the
t1.3 Light Cyan 0.031 0.178 0.796 other group included the two types of orange-cyan blocks.
t1.4 Dark Orange j0.031 j0.178 j0.796 We also performed a control analysis, where the classifier
t1.5 Light Orange 0.154 j0.015 j0.688 was trained to discriminate lime-magenta vs. orange-cyan
t1.6 Dark Magenta j0.031 j0.178 0.688 on blocks that had only one luminance phase (one of the
t1.7 Light Magenta 0.154 j0.015 0.796
two types of lime-magenta blocks, and one of two types
t1.8 Dark Lime j0.154 0.015 j0.796 of orange-cyan blocks), and then tested on its ability to
t1.9 Light Lime 0.031 0.178 j0.688 discriminate the other two types. The results of this
t1.12
t1.10 Table 1. Cone contrast values for stimuli calibrated for the analysis are shown in Appendix B.
subjective equiluminance point of observer EG. The background
of the stimuli had CIE xy coordinates of 0.30, 0.34, and a
luminance (Y) of 6.78 cd/m2.
Experimental design

to the experiment as a bitmapped image, and then these All experimental scans were completed during a single
images were drawn to the screen for each stimulus session for each subject. The session included ten func-
presentation using routines from PsychToolbox 3.0.8 tional scans, each lasting 4.5 minutes. During each scan
(Brainard, 1997; Pelli, 1997). the subject viewed 18 blocks of the experimental stimulus,
There were four stimuli, chosen such that over time all alternating between orange-cyan and lime-magenta blocks.
four would: (1) equally stimulate the L–M opponent Each block was 15 seconds long, and data from the first
channel; and (2) equally stimulate the S–(L + M) opponent and last block were excluded from our analysis. In order
channel. The angle of each stimulus within the isoluminant to change the color of the stimuli, either the L–M or the
plane was intermediate to that of the L–M and S–(L + M) S–(L + M) modulation must change phase, as illustrated
axes, and was defined as a vector addition of modulations in Figure 1C. This phase reversal is likely to induce an
along those two axes. When the L–M modulation was in increased response of a neural population which responds
phase with the S–(L + M) modulation the stimulus to the relevant cardinal color, as in the characteristic
modulated between magenta and lime; when these pairings response rebound used in fMRI adaptation (for example,
were reversed the modulation appeared orange-cyan. The see Engel & Furmanski, 2001; Kourtzi, Erb, Grodd, &
point of subjective isoluminance (the angle of the isolu- Bülthoff, 2003). It is also possible that the response to the
minant plane from the luminance axis) was estimated phase reversal may be evident in the BOLD signal, even
separately for each observer using the minimum motion when averaging across activity within a block, and could
technique described by Anstis and Cavanagh (1983), for potentially be used to discriminate the two types of stimuli
the magenta-lime and for the orange-cyan modulations. A (for example, if an orange-cyan block always commenced
25% luminance modulation was then added to the sub- with a L–M phase reversal). In order to eliminate this
jectively defined isoluminant modulation in one of two potential source of information about the color of the
phases. The four stimuli therefore appeared: light magenta– stimuli, we balanced the stimuli so that the pattern of
dark lime, dark magenta–light lime, light orange–dark cyan, phase reversals could not be used to predict the stimulus.
and dark orange–light cyan (example stimuli in Figure 1A, We used one of two block orders for each run, where the
and example cone contrast values in Table 1). second was a reversal of the first order.
The luminance modulation was added so that if there
were any residual differences in luminance between the
lime-magenta and orange-cyan blocks, this difference Localizer
should be masked by the much larger luminance modu- To select those voxels in each visual area most
lation. The contrast response of early visual areas to responsive to the experimental stimuli, two additional
luminance defined stimuli is steeper at low than high localizer scans during the same session as the experimental
contrast (for example, see Liu & Wandell, 2005). Thus a scans for each subject. Using a localizer scan that is
luminance artifact in our stimuli would result in a much separate from the experimental data avoids circularity that
smaller difference in the response to the two stimuli than could otherwise be present (Kriegeskorte, Simmons,
if there was no luminance modulation (and hence lower Bellgowan, & Baker, 2009). Localizer scans included a
luminance contrast). The same rationale underlies the use total of 17 blocks of 15 seconds each, comprised of
of random luminance noise to mask potential luminance stimulus blocks interleaved with blocks of fixation only.
artifacts (Birch, Barbur, & Harlow, 1992; Kingdom, The stimulus blocks included 2 lime-magenta blocks and
Moulden, & Collyer, 1992; Sumner et al., 2008). For the 2 orange-cyan blocks, in addition to 4 black-white blocks
Journal of Vision (2010) 10(5):25, 1–17 Goddard et al. 5

where the stimulus had the same spatial arrangement but Mazziotta, 1997), anatomical images were each reoriented
was modulated between black and white. to approximately the same space using anterior and
posterior commissures as anatomical landmarks. Fine
alignment of these anatomical images was carried out
Fixation task
using normalized mutual information based coregistration,
Throughout experimental and localizer scans, subjects and each of the anatomical images were resampled so that
performed a task at fixation that was unrelated to the they were in the same voxel space with a resolution of
annular experimental stimulus. This task was designed to 0.75 ! 0.75 ! 0.75 mm. From each image we removed
be attentionally demanding in order to direct attention intensity inhomogeneities using a nonparametric inhomo-
away from the experimental stimulus as much as possible. geneity correction method (Manjón et al., 2007), and
While this would have reduced the BOLD response, and normalized the images such that the white matter had an
so likely decreased the ratio of signal to noise in the data approximate intensity of 1. The coregistered, inhomogeneity
which were input to the classifier, it greatly reduces the corrected, normalized images were then averaged together
chance that subjects could have systematically directed to produce a mean anatomical image for each subject.
more attention to one type of stimulus block. ITKGray software (Yushkevich et al., 2006) was used to
Subjects were required to detect a conjunction of contrast define the white matter of each subject, initially using
polarity and number in a digit stream of the digits 0 to 9 automatic segmentation tools, then using manual editing.
inclusive, presented at fixation, updated at 3 Hz. Digits The segmentation image was imported into mrGray, part
were either black or white, against the mean gray of the of the mrVista software package developed by the
background, as seen in Figure 1B, and the order was Stanford Vision and Imaging Lab (http://white.stanford.
randomly generated for each run. Subjects responded with edu/software/). In mrGray, gray matter was ‘grown’ out
a button press to the onset of either of two target digits, from the white matter in a sheet with a maximum thickness
one only when black and the other only when white (for of 4 voxels.
example a black 3 or a white 7). Responding to a con-
junction of digit and contrast polarity made this a difficult
task. Target digits were updated at the beginning of each Functional measurements
run to increase task difficulty and minimize practice effects. fMRI data were acquired using a T2*-sensitive, FEEPI
All subjects performed the task significantly above pulse sequence, with echo time (TE) of 32 ms; time to
chance (p G 0.01, permutation test), demonstrating that repetition (TR) of 3000 ms; flip angle 90; field of view
they were engaged in the task, but each subject also made 192 mm ! 70.5 mm ! 192 mm; effective in-plane
errors, implying that the task was not trivial and required resolution 1.5 mm ! 1.5 mm, and slice thickness 1.5 mm.
attention. For no subject was there a significant difference 47 slices were collected in an interleaved, ascending order,
in performance between lime-magenta and orange-cyan in a coronal plane tilted such that the scan covered the
blocks, consistent with equal attentional resources being whole of the occipital lobe and the posterior part of the
devoted to the task in each case. parietal and temporal lobes. Using SPM5, all functional
data were preprocessed to correct for slice time and head
motion before alignment to the structural data. Data from
fMRI methods functional scans were aligned to a whole head anatomical
scan acquired in the same session, using normalized mutual
fMRI data were collected using a 3T Philips scanner information based coregistration. The functional data were
(Symbion Imaging Centre, Prince of Wales Medical then aligned to the subject’s average anatomical by first
Research Institute, Sydney, Australia), with a birdcage aligning the within session anatomical with the average
head coil. anatomical scan, then applying the same transformation to
the functional data.
Anatomical measurements and definition
of gray matter Definition of retinotopic areas
The anatomical image for each subject was generated For each subject, the precise anatomical locations of the
from the average of three scans. Two of these were high early areas of visual cortex (V1, V2, V3, V3A/B, hV4,
resolution (1 ! 1 ! 1 mm) structural MR images of each and VO) were found functionally using standard retino-
subject’s whole brain, acquired using a Turbo Field Echo topic mapping procedures (Engel, Glover, & Wandell,
(TFE) protocol for enhanced gray–white contrast. A third, 1997; Larsson & Heeger, 2006; see Wandell, Dumoulin,
higher resolution (0.75 ! 0.75 ! 0.75 mm) scan of the & Brewer, 2007, for a summary). Subjects were scanned
caudal half of the head was also acquired in order to while viewing first a rotating wedge then an expanding ring
recover more anatomical detail of the occipital lobes. stimulus, overlaid on a fixation cross of light gray lines, as
Using the Statistical Parametric Mapping (SPM) soft- shown on the key above the maps in Figure 2 (Schira,
ware package SPM5 (Frackowiak, Friston, Frith, Dolan, & Tyler, Breakspear, & Spehar, 2009).
Journal of Vision (2010) 10(5):25, 1–17 Goddard et al. 6

Figure 2. Example maps of functionally defined retinotopic areas for the left and right hemispheres of subject DM. In each of A, B, C and D
the underlying grayscale image shows the flattened map of visual cortex, centered on the occipital pole; the darker the gray the deeper
the sulcus. In D the grayscale anatomical map was darkened to increase the visibility of the overlaid image. A & B: Flattened maps of
visual cortex overlaid with phase maps of the response to the wedge and ring stimuli, respectively. Above these maps is a schematic of
the stimulus (top left) and a color map showing the area of the visual field which each color corresponds to in the phase maps (top right).
In C the same flattened map of visual cortex is overlaid with a heat map showing those voxels which responded more to the chromatic
than the achromatic stimuli; the significance of this result for each voxel is indicated by the T-statistic color map above. Areas V1, V2, V3,
V3A/B and hV4 were defined on the basis of the wedge and ring phase maps in A and B, while area VO was defined according to a
combination of the wedge and ring phase maps in A and B, and the contrast in C. MT+ was defined according to a motion versus static
dots localizer (not shown). The borders of each of these areas are drawn on each of the maps in A–D; the key on the right indicates which
of the outlined regions corresponds to each visual area. In D, the heat map indicates those voxels that were included in the analysis:
those which responded significantly more to chromatic or achromatic versions of our stimuli than to fixation; the significance of this result
for each voxel is indicated by the T-statistic color map above.

Averaged data from the wedge and ring stimuli were hemifield representation from the phase and eccentricity
smoothed with a Gaussian kernel of half width 1.5 mm, maps to define VO. Where the retinotopic map from the
then projected onto a computationally flattened represen- wedge and ring stimuli was unclear, we tended towards a
tation of the cortex for each hemisphere of each subject, liberal definition of VO, in order to avoid excluding any
using mrVista. Areas V1, V2, V3, V3A/B and hV4 were voxels in the region which showed a preference for
manually defined on the phase and eccentricity maps derived chromatic stimuli in the localizer analysis. Each retinotopic
from the wedge and ring stimuli (shown for an example area was defined on the flattened map of a subject’s cortex
subject in Figures 2A and 2B, respectively), using the then transformed into the space of the subject’s anatomical,
conventions described by Larsson and Heeger (2006). smoothed (FWHM = 1.5 mm), and resliced to the resolution
According to these definitions the foveal representation at of the functional images using 4th degree B-spline
the occipital pole is shared by V1, V2, V3, and hV4, while interpolation. Voxels assigned to each visual area were
V3A and V3B, which border the dorsal part of V3, share a allocated a value reflecting the cumulative influence of such
dorsal fovea. For our analysis we did not attempt to transformations. To prevent overlapping voxels between
separate V3A and V3B. We defined hV4 as a ventral adjacent visual areas, each voxel was assigned to the visual
hemifield representation that borders the ventral part of V3. area for which it possessed the greatest value.
For area VO, the phase and eccentricity maps were Area MT+ was defined on the basis of a separate localizer
considered in conjunction with a flattened map of those scan in which blocks of low contrast static and moving dots
voxels that responded more to chromatic than achromatic were interleaved with fixation only blocks. In SPM5, we
stimuli in the localizer scan (as shown for an example specified a general linear model of this data, and defined
subject in Figure 2C). Where it existed, we used the MT+ by finding lateral clusters of voxels that responded
Journal of Vision (2010) 10(5):25, 1–17 Goddard et al. 7

more to moving than to static dots. The definition of MT+ varied, so the power of the support vector machine was
was projected onto the flattened map for the purposes of not utilized.
illustration, as in Figure 2, but the original 3D definition of
MT+ was used in the analysis of the functional data.
Cross-validation: Leave-one-out train and test
Classification analysis was performed using a Matlab
Functional preprocessing (version 7.5) interface to SVM-light 6.01 (Joachims, 1999).
Data from each of the two localizer scans were processed The classifier was trained on the scores from 9 runs and
using the methods described above, then analyzed using a tested on a tenth; this procedure was repeated 10 times.
General Linear Model (GLM) in SPM5. We pooled This leave-one-out train and test procedure resulted in the
responses to the luminance and chromatically defined data from each run being used as test data once, giving an
stimuli and contrasted these with the response to the fixation average classifier performance (reported in the Results
only blocks. The subsequent analyses included only those section as a percentage correct) based on the accuracy
voxels that responded significantly more (p G 0.05, across 160 classifications, while ensuring that the test data
uncorrected) to the stimulus than fixation only. These never included data that were used in training.
voxels are shown on an example flattened map for one We repeated the classification analysis with increasing
subject in Figure 2D. numbers of voxels (n) within each visual area, from n = 3
For all functional scans the BOLD signal was labeled with voxels, to the n = maxN, where maxN is the total number
the stimulus presented 2 images (6 seconds) previously, in of voxels that reached significance in the localizer analysis.
order to compensate for the delayed hemodynamic response, Voxels in each area were ranked according to their t statis-
then was highpass filtered (cutoff 128 s, using filtering tic from the localizer analysis, based on the separate local-
methods from SPM5) in order to remove low frequency izer scans, in order to select voxels that responded to the
confounds in the data, and finally converted into z-scores area of visual field occupied by our experimental stimuli
for each of the ten runs in order to reduce variability from and exclude those which represented areas of the visual field
inter-run differences. Data from each voxel were z-scored which were more foveal or peripheral than our annular
separately. Within each 15 second block the BOLD stimuli. The top n most significant voxels were used in each
response, normalized according to the procedures described case. Classifier performance generally increased as more
above, was averaged across the 5 measurements to give a voxels were included in the analysis, but there was some
single score for each block in each run. variability around this trend. To summarize classification
performance (as reported below) we fit the classifier
performance (P) as number of voxels (n) increased with
Classifier analysis an exponential growth function which reaches a limit (L),
given by
Classifiers were restricted to each of several functionally
defined visual areas for each subject and trained to dis- P ¼ 0:5 þ ðLj 0:5Þð1j ejn=c Þ; ð1Þ
criminate the two patterns. We compared the performance
of classifiers trained on two types of data for each area: in where 0.5 is chance performance (at n = 0), and c is a
the univariate case, the classifier was trained and tested on curvature parameter, specifying how many voxels the
the average activation across voxels within an area (that curve takes to reach the limit, L. When the curve fit the
is, 1 value per block), while in the multivariate case the data, the classifier performance reported below corresponds
classifier was trained and tested on the pattern of activity to the limit (L) of the growth function. When the curve
across voxels within an area (n values per block). could not be fit to the data within 100 iterations of the
We used a linear support vector machine (SVM) Matlab function nlinfit (usually when classifier perfor-
classification technique in our analysis. Support vector mance was low), the average classifier performance, rather
machines are powerful in their ability to learn a decision than the limit of the curve, is reported as the summary
rule for multivariate data (Bennett & Campbell, 2000): in statistic of classifier performance. Classifier performance
our case, for n voxels with 144 data points each (72 from as a function of number of voxels, along with the best-
lime-magenta and 72 from orange-cyan blocks) they learn fitting curve, is plotted in Appendix A.
the hyperplane which best separates the data points in an n
dimensional space, where each dimension corresponds to
the normalized response of one voxel (using linear SVMs, Permutation analysis
we require that the hyperplane’s projection onto any two To test the statistical significance of classifier perfor-
dimensions is linear). We evaluated classifier performance mance, we ran a permutation analysis to estimate classi-
on its ability to generalize, i.e. to correctly discriminate fication accuracy expected by chance alone. Permutation
data that were not included in the training set. For the tests are non-parametric and so do not include an assumption
univariate classifier the technique was the same, but there of normality, and such tests have previously been employed
was only one dimension along which the 144 data points to evaluate classification analysis (Mourao-Miranda, Bokde,
Journal of Vision (2010) 10(5):25, 1–17 Goddard et al. 8

Born, Hampel, & Stetter, 2005). For each area in each sub- giving 80 measurements for each; the distributions of these
ject we performed the same analysis as that described above, measurements for each subject are shown in Figure 3.
except that before training the classifier we randomly per- In all subjects each area that showed a significant bias
muted the stimulus labels associated with each block in the for one color modulation showed the same bias; signal was
training data set. Using 1000 repetitions of this permutation greater for lime-magenta than for orange-cyan. As shown
analysis, we generated a population of 1000 estimates of in Figure 3, the mean of the 80 lime-magenta blocks was
the classifier accuracy that could be expected in cases where significantly greater than the 80 orange cyan-blocks (p G
the data did not contain any stimulus-related information. 0.05, two-tailed t-test); in 25 areas across 5 subjects. The
For each iteration of the permutation analysis we averaged only area for which there was not a significant difference
these estimates across subjects for each area and compared between the two types of stimuli for any subject was area
these 1000 values with the observed between-subject mean MT+. We conclude that stimuli which equally modulate
accuracy. In the statistics reported below for classifier per- the cardinal axes of color space are not equally repre-
formance, p-values were calculated by finding the propor- sented in visual cortical areas, and this biased representa-
tion of these 1000 estimates which were greater than the tion is seen as early as V1.
observed classifier accuracy. Consistent with the differences in the average response
across each region, univariate classifiers performed signifi-
cantly better than chance as early as V1, as shown in Figure 4
(light gray bars). The average univariate classification per-
Results formance across subjects in all areas was significantly above
chance (p G 0.05, one-tailed permutation test). MT+ showed
We tested for the presence of cortical representations of the lowest performance for 4 of 5 subjects.
color capable of discriminating stimuli that cannot be dis- For 4 of 5 subjects, the area with the best classifier
tinguished by the L–M and S–(L + M) sub-cortical oppo- performance was V2. The earlier cortical visual areas (V1,
nent channels. Our lime-magenta and orange-cyan stimuli V2 and V3) generally outperformed the dorsal area V3A/B,
contain the same L–M and S–(L + M) modulations, varying as well as the ventral areas hV4 and VO. V1, V2 and V3
only in the phase that these modulations are added together. generally had a greater number of voxels, which may
For the lime-magenta stimuli, the L–M and S–(L + M) account for their high performance. In order to test this, we
modulations are the same phase, whereas for the orange- repeated the classifier analysis on 100 voxels that were
cyan stimuli the L–M and S–(L + M) modulations were in randomly chosen from those voxels in the area that
opposite phase. We used fMRI to measure changes in BOLD responded to the localizer stimulus. For this subset of
signal as an indirect measure of neural activity, then asked 100 voxels, the classifier accuracy averaged across subjects
the extent to which the visual stimulus could be discriminated in V1, V2 and V3 (61, 64 and 61%) was still better than
from patterns of brain activity in a predefined visual area. in V3A/B, hV4 and VO (45, 56 and 52%). This suggests that
Below, we report stimulus related differences in both the differences in classifier performance cannot be accounted for
mean activity and pattern of activity across a range of by the generally greater number of voxels included in the
regions. There was a small but reliable bias across subjects analysis for areas V1, V2 and V3. Area MT+ had fewer
for lime-magenta over orange-cyan stimuli in the mean than 100 voxels that responded to the localizer in all sub-
activity across each region, and we found that the differ- jects, and so was excluded from this reanalysis.
ence in the mean activity was sufficient for a univariate
classifier to learn to correctly discriminate the stimuli. We
also found evidence of additional stimulus-related infor- Additional information in the pattern
mation in the pattern of activity across V1 and V2, using of activity for areas V1 and V2
multivariate classifiers.
Multivariate pattern classifiers were also trained to discrim-
inate the two types of stimulus, allowing us to test for addi-
Consistent bias for lime-magenta over tional stimulus related information in the pattern of activity
orange-cyan stimuli in average activity across each area. The univariate classifier was trained on a
subset (9 of 10 runs) of the average data (as plotted in
There was a significant difference in the response to Figure 3), and tested on the remainder. The multivariate
lime-magenta vs. orange-cyan stimuli, impossible without classifier was trained and tested in the same way on data
combination of signals from the fundamental cone-opponent which was not averaged across voxels, so that in addition to
channels. For the univariate analysis we averaged across the average it can learn differences in the pattern of activity
those voxels within each area for which there was a sig- across an area between blocks. This analysis allows a com-
nificant difference in their response to the localizer stimulus parison of the results from the univariate and multivariate
versus fixation. The average z-scored activity across an classification techniques, which gives an indication of how
area for each block was treated as a separate measure of the information in the pattern of activity across an area differs
the area’s response to lime-magenta or orange-cyan stimuli, from information given by the mean response.
Journal of Vision (2010) 10(5):25, 1–17 Goddard et al. 9

Figure 3. Mean activity in response to lime-magenta vs. orange-cyan blocks, across different cortical visual areas, for each subject. For
each block, the z-scored BOLD response was averaged across all voxels for which there was a significant difference in their response to
the localizer stimulus versus fixation. The histograms plot the distribution of these averages. Behind each histogram, a normal distribution
of the same mean and standard deviation is plotted for reference. For all subjects, where there was a significant difference between the
response to lime-magenta and orange-cyan blocks (tested with a two-tailed t test), the response to the lime-magenta blocks was greater
than the response to orange-cyan blocks. The functional data did not include a fixation block; the average % signal change in the color
blocks in the localizer data, relative to fixation (T1 standard error of the between subject mean) were V1: 0.65 (T0.16); V2: 0.78 (T0.19);
V3: 0.56 (T0.17); V3A/B: 0.18 (T0.16); hV4: 0.67 (T0.27); VO: 0.35 (T0.19); MT+: j0.09 (T0.17).

Multivariate classification performance across areas (see VO (57, 56 and 49%). Classification performance was
Figure 4) followed a similar trend to that found for the generally higher for multivariate classifiers, and this
univariate classifiers; earlier visual areas (V1, V2 and V3) difference was significant (p G 0.01, two-tailed permutation
tended to outperform V3A/B, hV4 and VO. Classification test) for areas V1 and V2. In VO and MT+, the univariate
accuracy was poorest in MT+, where performance was not classifier outperformed the multivariate classifier, but this
significantly different from chance. This trend was also difference was not significant.
found when the classifier was trained and tested on one Since there was no requirement on our classifiers to
hundred voxels, randomly chosen from those voxels in the predict an equal number of orange-cyan and lime-magenta
area that responded to the localizer stimulus: the average blocks, it was possible for classification performance to be
between-subjects classifier accuracy in V1, V2 and V3 better for one type of test stimulus. For example, if
(63, 67 and 59%) was still better than in V3A/B, hV4 and classification of orange-cyan test stimuli were at chance
Journal of Vision (2010) 10(5):25, 1–17 Goddard et al. 10

hypothesized to carry information in parallel from sub-

cortical areas to cortex. The ability to use BOLD activity
to discriminate stimuli matched for the postulated sub-
cortical mechanisms demonstrates that the neural popula-
tion must include neurons modulated by signals from both
the chromatically opponent pathways. Below we discuss
the implications of these results for how color is repre-
sented in human visual cortex, in particular the bias for
lime-magenta over orange-cyan stimuli, and differences
between visual areas in classifier performance.

Origin of asymmetry in the representation

of two non-cardinal color modulations
Figure 4. Univariate and Multivariate Classifier Performance
across subjects. Mean univariate and multivariate classifier accu- We found a common bias across cortical visual areas
racy across subjects for each visual area are plotted in light gray for lime-magenta over orange-cyan blocks, even though
and dark blue bars, respectively. Error bars indicate T1 standard our stimuli were matched for cone contrast, and for the
deviation of the population of classification accuracy estimates response of sub-cortical pathways. The consistency of this
generated using the permutation analysis. Chance performance bias across subjects suggests that it reflects a typical
(50%) is shown with the dashed line. In all visual areas except MT+, asymmetry in cortical representations of color. Specifi-
the multivariate classification accuracy was significantly (p G 0.01, cally, this finding implies that there is a more numerous or
one-tailed permutation test) above the chance performance pre- more active population of neurons which respond to lime
dicted by the permutation analysis, while the univariate classifica- and/or magenta than to orange and/or cyan stimuli.
tion accuracy was significantly above chance (p G 0.05, one-tailed There is some evidence for a bias in the opposite direction
permutation test) in all areas. The multivariate pattern classifier in single-unit recordings in macaque V1, and from human
generally outperformed the univariate classifier, and this difference psychophysics. Both Conway (2001) and Solomon and Lennie
was significant (p G 0.01, two-tailed permutation test) for areas V1 (2005) found a bias when testing the responses of macaque
and V2 when compared with the differences predicted by chance V1 cells to L, M and S-cone isolating stimuli. Of the 45
according to the permutation analysis. In areas VO and MT+, the (Conway, 2001) and 19 (Solomon & Lennie, 2005) L–M
univariate classifier outperformed the multivariate classifier, but color opponent cells that also responded to S-cone isolating
this difference was not significant. stimuli, for 93% and 89% of cells (respectively) the response
to the S-cone isolating stimulus had the same sign as the
but classification performance on lime-magenta blocks M-cone isolating stimulus; that is, the cells preferred a color
were perfect, this would give an overall performance of direction closer to orange-cyan than lime-magenta. Krauskopf
75%. We found that this was not the case; for both univar- and Gegenfurtner (1992) report subtle psychophysical asym-
iate and multivariate classifiers, classification performance metries for human observers between the non-cardinal axes
for lime-magenta test stimuli and classification perfor- in the effects of adaptation on discrimination threshold. Their
mance for orange-cyan stimuli showed a positive linear data are consistent with a greater prevalence of adaptable
correlation (univariate: slope = 0.33, R2 = 0.13, p G 0.05, mechanisms tuned to orange-cyan than to lime-magenta.
multivariate: slope = 0.85, R2 = 0.61, p G 0.01). Our data imply a bias of the opposite direction in human
Increased performance of the multivariate classifier visual cortex, suggesting further work is necessary to recon-
compared with the univariate classifier in V1 and V2 cile these findings. In a recent study on human discrim-
indicates that there are reliable, stimulus-related patterns ination thresholds, Danilova and Mollon (2010) found that
of activity in these areas. If the pattern of activity across discrimination thresholds were lowest along a line in chro-
voxels were uninformative about the non-cardinal color of maticity space connecting unique blue and unique yellow.
the stimulus, we would expect the multivariate classifier The hypothetical channel Danilova and Mollon (2010)
performance to be at best the same as the univariate case propose to account for their results would lie closer to the
(since if the classifier learnt a pattern that was not lime-magenta modulation than the orange-cyan modula-
stimulus-related, performance could decrease). tion, and these results could be a psychophysical correlate
of the bias we observed in our fMRI data.

Discussion Organization of color processing in early

human cortical areas
We found evidence in human visual cortex for repre-
sentations of color as early as V1 that combine informa- While significant classifier performance indicates repre-
tion from the L–M and S–(L + M) opponent pathways sentations of non-cardinal colors, differences in classifier
Journal of Vision (2010) 10(5):25, 1–17 Goddard et al. 11

performance between areas are difficult to interpret. well as evidence from MT of rhesus monkey (Britten,
Brouwer and Heeger (2009) found highest classifier Shadlen, Newsome, & Movshon, 1992; Dubner & Zeki,
performance in V1, yet their principal components analysis 1971) and human MT+ (Huk, Dougherty, & Heeger,
suggested that the representation of color in hV4 and VO 2002; Tootell et al., 1995; Zeki et al., 1991) that this area
more closely matches our perceptual experience. Classifier is not generally selective for the color of surfaces and is
performance depends not only on the presence of relevant less responsive to chromatic than achromatic stimuli
information (here, non-cardinal representations of color) (Gegenfurtner et al., 1994; Liu & Wandell, 2005; Wandell
within an area, but also on the accessibility of this et al., 1999), although sensitivity to chromatic motion
information at the coarse spatial scale of our functional (Barberini, Cohen, Wandell, & Newsome, 2005; Wandell
measurements. et al., 1999) has been reported. Likewise, the reduced
For areas V1 and V2, multivariate classifiers signifi- classifier performance in V3A/B with respect to V1, V2
cantly outperformed univariate classifiers, showing that and V3 may reflect the general preference of this dorsal
there was stimulus related information in the pattern of area for motion (Tootell et al., 1997), and its reduced
activity. In macaque V1 and V2 there are orderly maps of responsivity to chromatically defined stimuli (Liu &
hue selectivity (including both cardinal and non-cardinal Wandell, 2005). Nevertheless, for each subject, MT+ had
colors) across the surface of the cortex (Xiao, Casti, Xiao, fewer voxels than any other area we defined, which alone
& Kaplan, 2007; Xiao, Wang, & Felleman, 2003). If may account for decreased classifier performance.
similar maps exist early in human visual cortex, their
existence may increase the chance of biased sampling of
chromatic preferences across voxels. The size of these hue Response of ventral visual areas
maps, which each represent a large spectrum of hues, is
only around 200 2m across the surface of the cortex in Areas hV4 and VO are often thought to be specialized
macaque V1, with individual maps separated by around for the processing of color. In macaque V4 evidence from
400 2m (Xiao et al., 2007). If maps of approximately the both single unit recordings (Zeki, 1983) and from neuro-
same size exist in human V1, a single voxel would contain imaging (Conway & Tsao, 2006) implicate this area as a
approximately 6 hue maps. It is unlikely that a single ‘color center’. In human, there is converging evidence
voxel would sample neurons whose preference included from patients with cerebral achromatopsia (Zeki, 1990)
only a narrow range of hues, but it could be that this map and from PET (Lueck et al., 1989) and fMRI (Bartels &
arrangement would make it more likely for biases between Zeki, 2000; Hadjikhani, Liu, Dale, Cavanagh, & Tootell,
voxels to arise. Furthermore, in macaque, the hue maps in 1998; Liu & Wandell, 2005; McKeefry & Zeki, 1997;
V2 are on average 2 to 2.5 times longer than the hue maps Mullen, Dumoulin, McMahon, de Zubicaray, & Hess,
in V1 (Xiao et al., 2007). Larger maps with the same 2007; Wade, Brewer, Rieger, & Wandell, 2002) neuro-
voxel resolution should increase the likelihood of biased imaging studies that both hV4 and VO are involved in color
sampling of hue maps, and increase the magnitude of the vision. Additionally, there is evidence that the response
biases, which could underlie the tendency in our results properties of VO match color perception in showing
for classifiers in V2 to outperform classifiers in V1. weaker responses to high than to low temporal frequencies
Alternatively, it is possible that the stimulus related (Jiang, Zhou, & He, 2007; Liu & Wandell, 2005), while
information in the pattern of activity is not due to V1 does not. It therefore might be expected that classifier
qualitatively different patterns of response for lime-magenta performance would be greatest in these areas, but this was
and orange-cyan but instead a single pattern of visually not the case for our stimuli, or for more perceptually
responsive voxels which respond more strongly in the lime- relevant hues (Brouwer & Heeger, 2009). We consider
magenta case. Since there is a univariate bias, the multi- five possible reasons for this.
variate classifier could potentially be learning the difference First, our definitions of hV4 and VO may not include
between a strong signal in noise and a weaker version of the the region of ventral visual cortex that is specialized for
same signal (also in noise). The increased performance of color processing; Hadjikhani et al. (1998) reported color
the multivariate versus univariate classifiers might then be selectivity in area V8, but not hV4. We think this account
based purely on the ability of the multivariate classifiers to of our results is unlikely because our definition of hV4,
ascribe more to weight individual voxels on the basis of their corresponding to that of Wandell et al. (2007), would
signal-to-noise ratio. Further empirical and theoretical work include part of the V8 described by Hadjikhani et al.
will be required before it is possible to discriminate with (1998), which showed color selectivity, with the remainder
certainty between these alternatives. of their V8 corresponding to our VO.
Second, areas hV4 and VO may be more susceptible to
task specific demands than other areas. We both asked
Response of dorsal visual areas subjects not to attend to the stimuli and required them to
engage with a task at fixation. By diverting attention from
Poor classifier performance in MT+ is consistent with the experimental stimulus we aimed to avoid artifactual
the classifier results of Brouwer and Heeger (2009), as classifier performance that was based not on differences in
Journal of Vision (2010) 10(5):25, 1–17 Goddard et al. 12

the stimulus-driven response but on differences in attention peripheral to this, our stimuli were no longer balanced for
between the two conditions. However, when attention is responses they induce in the sub-cortical pathways. To
directed to a task unrelated to the stimulus, the stimulus- rule out this potential artifact, we repeated the classifier
driven BOLD response is suppressed, and this suppression analysis, limiting the voxels included in the classifier to
increases with the attentional load of the task (Rees, Frith, those within 2 degrees visual angle from fixation and thus
& Lavie, 1997; Schwartz et al., 2005). Single-unit record- excluding any voxels which respond to an area of visual
ings in macaque and fMRI in humans show that V4 and field for which the stimuli may not have been balanced.
hV4 show greater attentional modulation than earlier With this analysis, we found classifier performance was
visual areas (Hansen, Kay, & Gallant, 2007; Reynolds & reduced, but remained significantly above chance in all
Chelazzi, 2004; Schwartz et al., 2005). areas except MT+ (data not shown). This rules out the
Third, reduced performance of multivariate classifiers in possibility that classifier performance in the original
hV4 and VO may result if our voxel size (1.5 mm each analysis was based on artifacts in the stimuli caused by
side) resulted in biases when sampling neural representa- macular pigmentation.
tions of color in V1 and V2, but not hV4 or VO. The spatial It is important also to emphasize the robustness of our
arrangement of chromatic preferences may be either less conclusions to any inaccuracies in the determination of
ordered, ordered in a different way, or ordered on a smaller subjective equiluminance for each subject. For example,
spatial scale than in the earlier visual areas. let us consider the situation if there was a 1% artifactual
Fourth, any nonlinearity in the signal which differs modulation in the lime-magenta blocks and no artifact in
between areas may enhance or reduce the stimulus orange-cyan blocks. The 25% luminance modulation was
discriminability in the BOLD response. For example, the added in opposite phases for different lime-magenta
contrast response of VO to L–M and S–(L + M) mod- blocks, meaning that the effect of the luminance artifact
ulating stimuli is more nonlinear than V1 (Liu & Wandell, would be to increase effective luminance contrast to 26%
2005). It is not clear how such nonlinearities should affect in half of the lime-magenta blocks and decrease it to 24% in
classifier performance, but it is possible that the reduced the other half. For such a bidirectional effect to introduce a
classifier performance in ventral areas were due to such bias in the univariate response between lime-magenta and
nonlinearities. orange-cyan blocks, the contrast response function in the
Finally, there may be increased noise in our imaging vicinity of 25% luminance contrast would have to be highly
results for these areas, which are typically located further non-linear. Furthermore, any such bias would be unlikely to
from the surface of the head than the earlier visual areas show the consistency between subjects observed here. In
and dorsal areas, and near the transverse sinus, which can terms of the multivariate analysis, a classifier would have to
cause imaging artifacts (Winawer, Horiguchi, Sayres, learn a disjunctive discrimination between (24 or 26%) vs.
Amano, & Wandell, in press). We think it likely that the (25%) modulation in luminance in order to be able to
reduced performance of the classifiers in hV4 and VO classify the stimuli based on luminance alone. Our use of
reflects some combination of the impact of attention, linear classifiers minimizes the possibility that luminance
nonlinearities, imaging noise and (for multivariate classi- artifacts were used as a cue to discrimination.
fiers) the spatial arrangement of color processing within More fundamentally, it could be that the classic model
these areas, rather than implying their diminished selec- of two chromatically opponent sub-cortical channels is
tivity for color. insufficient to capture the selectivity of all neurons in the
lateral geniculate nucleus (LGN). The majority of observed
LGN responses can be accounted for by the classic model,
Limitations of this study but some evidence suggests that the chromatic responses of
the LGN may not be fully described by the two channels. It
All these conclusions are based on the assumption that is difficult to investigate physiological correlates of the
our stimuli induced responses in sub-cortical pathways higher-order color mechanisms in the LGN, since these
that are indistinguishable when considering the responses mechanisms have primarily been revealed by psycho-
of each pathway independently. We consider a number of physical habituation (Krauskopf et al., 1986), and there is
reasons why this assumption may be invalid. little or no habituation of cells in the LGN, yet these cells
Macular pigmentation selectively attenuates shorter may contribute to the color tuning of the adaptable cortical
wavelengths in the central two degrees of the visual field cells (Tailby, Solomon, Dhruv, & Lennie, 2008a). Signals
(Hammond, Wooten, & Snodderly, 1997; Wyszecki & tuned to color directions away from the two opponent
Stiles, 1967). When defining our stimuli we used the mechanisms have also been proposed (Webster & Mollon,
Stockman and Sharpe (2000) 2 degree cone spectral 1991, 1994; Zaidi & Shapiro, 1993) that could theoret-
sensitivities, which take into account the impact of ically be inputs to cortical areas. Finally Tailby, Solomon,
macular pigmentation. Since macular pigmentation does and Lennie (2008b) recently reported that in macaque
not extend beyond the central 2 degrees of visual field LGN, a subset of neurons responded to modulations along
(Hammond et al., 1997; Stringham, Hammond, Wooten, both the S–(L + M) and L–M axes. Our present study does
& Snodderly, 2006), it is possible that for the region not address the question of whether the model of two
Journal of Vision (2010) 10(5):25, 1–17 Goddard et al. 13

chromatically opponent sub-cortical channels is sufficient semilogarithmic scale. It also shows the best fitting curve
to describe the coding of color by the LGN. We plan to (see Equation 1), from which the limit was reported as the
investigate this possibility further in future investigations classifier performance in Figure 4, or the mean classifier
using human fMRI. performance in cases where the curve did not fit the data.

Appendix A
Appendix B
Multivariate classifier performance as a
function of the number of voxels Generalization of classification across
different luminance–color pairings
For each subject, the classifier analysis was repeated for
increasing numbers of voxels from each area. Figure A1 Our main analyses, grouping lime-magenta and orange-
shows multivariate classifier performance as an increasing cyan blocks of different luminance pairings, show that
number of voxels were included in the classifier, on a the classifier generalizes across luminance levels. This is

Figure A1. Classifier Performance as a function of the number of voxels included in the classifier, for each area and each subject. In each
plot the filled blue line plots the classifier performance, and the filled red line plots the best-fitting exponential growth function, given by
Equation 1. Where the exponential growth function fitted the data, the limit of this function was taken as the classifier performance in that
area for that subject; the number of voxels taken to reach this limit is given by the red number on these plots. Where the exponential
growth function did not fit the data (usually when performance was low), the mean was taken as the classifier performance, which is
plotted as a dashed red line.
Journal of Vision (2010) 10(5):25, 1–17 Goddard et al. 14

The results of this analysis are shown in Figure B1. We

found that in area MT+, classification performance was
significantly below chance (p G 0.01, two-tailed t-test)
across subjects, and for individual CC in areas V1, V2, V3
and V3A/B (p G 0.01, permutation test). Below chance
performance is consistent with the classifier being sig-
nificantly good at learning a decision rule based on the
pairing of luminance and one of the cardinal modulations
(for example, successfully learning to separate the data as
dark green-light red vs. dark red-light green). This below
chance performance neither supports nor excludes the
possibility that the original classifier performance was
based on a luminance artifact.
For subjects SM, DM and EG classification performance
was significantly (p G 0.05, permutation test) above chance
for V1 and V2, with SM and DM also significantly (p G
Figure B1. Multivariate classifier performance, training on one pair
0.05, permutation test) above chance in area V3. The fact
of luminance levels and testing on the other two. Results are
that classifier performance is significantly above chance
averaged across the four different combinations of train/test pairs,
for some areas in some subjects gives evidence that in
error bars indicate the between subjects mean T1 SD. The **
these cases the original classifier performance could not
denote that the classifier was significantly different from chance
have been based on a luminance artifact.
(p G 0.01) in a two-tailed t-test. Additionally, the filled symbols
indicate cases where the individual’s classifier was significantly
(p G 0.05) different from 50%, as assessed using a bootstrapped
estimate of the variance by taking scoring the classifier’s perfor-
Acknowledgments
mance for 1000 iterations of randomly shuffled labels assigned to
the test data.
This work was supported by an Australian Postgraduate
Award to E.G., an Australian Research Fellowship to C.C.
and the Australian Centre of Excellence for Vision Science.
We thank Kirsten Moffat and the Prince of Wales Medical
because both training and test data contain an equal number Research Institute for assistance with fMRI data collection,
of the two phases of luminance–color relations so the Mark Schira for assistance with retinotopic analysis, and
classifier has to generalize across different luminance levels Bill Levick for helpful comments regarding macular
to learn the stimuli. If the classifier can learn this more pigmentation.
general rule, it at first sight seems reasonable to test
whether it can generalize from one level to the other. Commercial relationships: none.
However, there is a subtle but important point to make Corresponding author: Erin Goddard.
here. If we train on one luminance level and test on the Email: [email protected].
other, the classifier may learn a different rule than lime- Address: Griffith Taylor Bldg A19, The University of
magenta vs. orange-cyan. If the classifier is trained on Sydney, Sydney, New South Wales, Australia.
dark lime-light magenta vs. dark orange-light cyan, the
classifier could learn to separate the training data as dark
green-light red vs. dark red-light green. If this different References
rule were learned by the classifier then when tested with
light lime-dark magenta vs. light orange-dark cyan it Anstis, S., & Cavanagh, P. (1983). A minimum motion
would give the opposite result to learning the non-cardinal technique for judging equiluminance. In J. D. Mollon
color modulation. Thus successful classification would & R. T. Sharpe (Eds.), Colour vision: Physiology and
provide evidence in support of the classifier learning the psychophysics (pp. 156–166). London: Academic Press.
non-cardinal color modulation, and argue against the
notion that the original classifier performance could have Barberini, C. L., Cohen, M. R., Wandell, B. A., &
been based on an artifact. But a negative result neither Newsome, W. T. (2005). Cone signal interactions in
supports nor excludes the possibility that classifier per- direction-selective neurons in the middle temporal
formance was due to a luminance artifact. In short, in the visual area (mt). Journal of Vision, 5(7):1, 603–621,
original analysis any small luminance artifact could have http://journalofvision.org/content/5/7/1, doi:10.1167/
worked in favor of classifier performance but in this new 5.7.1. [PubMed] [Article]
analysis there is a 25% contrast luminance modulation Bartels, A., & Zeki, S. (2000). The architecture of the
working against classifier performance. colour centre in the human visual brain: New results
Journal of Vision (2010) 10(5):25, 1–17 Goddard et al. 15

and a review. European Journal of Neuroscience, 12, the spatial precision of functional MRI. Cerebral
172–193. [PubMed] Cortex, 7, 181–192. [PubMed]
Bennett, K. P., & Campbell, C. (2000). Support vector Frackowiak, R. S., Friston, K. J., Frith, C. D., Dolan, R. J.,
machines: Hype or hallelujah? SIGKDD Explorations, & Mazziotta, J. C. (1997). Human brain function.
2, 1–13. San Diego: Academic Press.
Birch, J., Barbur, J. L., & Harlow, A. J. (1992). New Gegenfurtner, K. R., Kiper, D. C., Beusmans, J. M.,
method based on random luminance masking for Carandini, M., Zaidi, Q., & Movshon, J. A. (1994).
measuring isochromatic zones using high resolution Chromatic properties of neurons in macaque mt.
colour displays. Ophthalmic Physiology Optics, 12, Visual Neuroscience, 11, 455–466. [PubMed]
133–136. [PubMed] Hadjikhani, N., Liu, A. K., Dale, A. M., Cavanagh, P., &
Brainard, D. H. (1997). The psychophysics toolbox. Tootell, R. B. (1998). Retinotopy and color sensitivity
Spatial Vision, 10, 433–436. [PubMed] in human visual cortical area V8. Nature Neuro-
science, 1, 235–241. [PubMed]
Britten, K., Shadlen, M., Newsome, W., & Movshon, J.
(1992). The analysis of visual motionVA comparison Hammond, B. R., Wooten, B. R., & Snodderly, D. M.
of neuronal and psychophysical performance. Journal (1997). Individual variations in the spatial profile of
of Neuroscience, 12, 4745–4765. [PubMed] [Article] human macular pigment. Journal of Optical Society
of America A, Optics, Image Science, and Vision, 14,
Brouwer, G. J., & Heeger, D. J. (2009). Decoding and 1187–1196. [PubMed]
reconstructing color from responses in human visual
cortex. Journal of Neuroscience, 29, 13992–14003. Hansen, K. A., Kay, K. N., & Gallant, J. L. (2007).
[PubMed] Topographic organization in and near human visual
area V4. Journal of Neuroscience, 27, 11896–11911.
Conway, B. R. (2001). Spatial structure of cone inputs to [PubMed] [Article]
color cells in alert macaque primary visual cortex
(V-1). Journal of Neuroscience, 21, 2768–2783. Haynes, J., & Rees, G. (2005a). Predicting the orientation
[PubMed] [Article] of invisible stimuli from activity in human primary
visual cortex. Nature Neuroscience, 8, 686–691.
Conway, B. R., & Tsao, D. Y. (2006). Color architecture [PubMed]
in alert macaque cortex revealed by fMRI. Cerebral
Cortex, 16, 1604–1613. [PubMed] Haynes, J.-D., & Rees, G. (2005b). Predicting the stream
of consciousness from activity in human visual
Danilova, M., & Mollon, J. (2010). Parafoveal color cortex. Current Biology, 15, 1301–1307. [PubMed]
discrimination: A chromaticity locus of enhanced dis-
crimination. Journal of Vision, 10(1):4, 1–9, http:// Haynes, J. D., & Rees, G. (2006). Decoding mental states
journalofvision.org/content/10/1/4, doi:10.1167/ from brain activity in humans. Nature Reviews on
10.1.4. [PubMed] [Article] Neuroscience, 7, 523–534. [PubMed]
Derrington, A. M., Krauskopf, J., & Lennie, P. (1984). Huk, A., Dougherty, R., & Heeger, D. (2002). Retinotopy
Chromatic mechanisms in lateral geniculate nucleus and functional subdivision of human areas MT and
of macaque. The Journal of Physiology, 357, 241–265. MST. Journal of Neuroscience, 22, 7195–7205.
[PubMed] [Article] [PubMed] [Article]
de Valois, R., Cottaris, N., Elfar, S., Mahon, L., & Wilson, J. Ishihara, S. (1990). Ishihara’s tests for color-blindness, 38
(2000). Some transformations of color information plate ed. Tokyo: Kanehara Shuppan Co. Ltd.
from lateral geniculate nucleus to striate cortex. Jiang, Y., Zhou, K., & He, S. (2007). Human visual cortex
Proceedings of the National Academy of Sciences of responds to invisible chromatic flicker. Nature Neuro-
the United States of America, 97, 4997–5002. science, 10, 657–662. [PubMed]
[PubMed] [Article] Joachims, T. (1999). Making large-scale SVM learning
Dubner, R., & Zeki, S. M. (1971). Response properties practical. In B. Scholkopf, C. Burges, & A. Smola (Eds.),
and receptive fields of cells in an anatomically Advances in Kernel methodsVSupport vector learning
defined region of the superior temporal sulcus in the (chapter 11, pp. 41–56). Cambridge, MA: MIT Press.
monkey. Brain Research, 35, 528–532. [PubMed] Kamitani, Y., & Tong, F. (2005). Decoding the visual and
Engel, S. A., & Furmanski, C. S. (2001). Selective subjective contents of the human brain. Nature
adaptation to color contrast in human primary visual Neuroscience, 8, 679–685. [PubMed] [Article]
cortex. Journal of Neuroscience, 21, 3949–3954. Kamitani, Y., & Tong, F. (2006). Decoding seen and
[PubMed] [Article] attended motion directions from activity in the human
Engel, S. A., Glover, G. H., & Wandell, B. A. (1997). visual cortex. Current Biology, 16, 1096–1102.
Retinotopic organization in human visual cortex and [PubMed] [Article]
Journal of Vision (2010) 10(5):25, 1–17 Goddard et al. 16

Kingdom, F., Moulden, B., & Collyer, S. (1992). A Parkes, L., Marsman, J., Oxley, D., Goulermas, J., &
comparison between colour and luminance contrast in Wuerger, S. (2009). Multivoxel fMRI analysis of color
a spatial linking task. Vision Research, 32, 709–717. tuning in human primary visual cortex. Journal of
[PubMed] Vision, 9(1):1, 1–13, http://journalofvision.org/content/
Kourtzi, Z., Erb, M., Grodd, W., & Bülthoff, H. H. (2003). 9/1/1, doi:10.1167/9.1.1. [PubMed] [Article]
Representation of the perceived 3-D object shape in Pelli, D. G. (1997). The videotoolbox software for visual
the human lateral occipital complex. Cerebral Cortex, psychophysics: Transforming numbers into movies.
13, 911–920. [PubMed] Spatial Vision, 10, 437–442. [PubMed]
Krauskopf, J., & Gegenfurtner, K. (1992). Color dis- Rees, G., Frith, C. D., & Lavie, N. (1997). Modulating
crimination and adaptation. Vision Research, 32, irrelevant motion perception by varying attentional
2165–2175. [PubMed] load in an unrelated task. Science, 278, 1616–1619.
Krauskopf, J., Williams, D. R., Mandler, M. B., & Brown, [PubMed]
A. M. (1986). Higher order color mechanisms. Vision Reynolds, J. H., & Chelazzi, L. (2004). Attentional
Research, 26, 23–32. [PubMed] modulation of visual processing. Annual Reviews on
Kriegeskorte, N., Simmons, W. K., Bellgowan, P. S. F., & Neuroscience, 27, 611–647. [PubMed]
Baker, C. I. (2009). Circular analysis in systems Schira, M. M., Tyler, C. W., Breakspear, M., & Spehar, B.
neuroscience: The dangers of double dipping. Nature (2009). The foveal confluence in human visual cortex.
Neuroscience, 12, 535–540. [PubMed] [Article] Journal of Neuroscience, 29, 9050–9058. [PubMed]
Larsson, J., & Heeger, D. J. (2006). Two retinotopic visual Schwartz, S., Vuilleumier, P., Hutton, C., Maravita, A.,
areas in human lateral occipital cortex. Journal of Dolan, R. J., & Driver, J. (2005). Attentional load and
Neuroscience, 26, 13128–13142. [PubMed] [Article] sensory competition in human vision: Modulation of
Liu, J., & Wandell, B. A. (2005). Specializations for fMRI responses by load at fixation during task-
chromatic and temporal signals in human visual irrelevant stimulation in the peripheral visual field.
cortex. Journal of Neuroscience, 25, 3459–3468. Cerebral Cortex, 15, 770–786. [PubMed]
[PubMed] [Article] Seymour, K., Clifford, C. W. G., Logothetis, N. K., &
Lueck, C. J., Zeki, S., Friston, K. J., Deiber, M. P., Cope, P., Bartels, A. (2009). The coding of color, motion, and
Cunningham, V. J., et al. (1989). The colour centre in their conjunction in the human visual cortex. Current
the cerebral cortex of man. Nature, 340, 386–389. Biology, 19, 177–183. [PubMed]
[PubMed] Solomon, S. G., & Lennie, P. (2005). Chromatic gain
Manjón, J. V., Lull, J. J., Carbonell-Caballero, J., Garcı́a- controls in visual cortical neurons. Journal of Neuro-
Martı́, G., Martı́-Bonmatı́, L., & Robles, M. (2007). A science, 25, 4779–4792. [PubMed] [Article]
nonparametric MRI inhomogeneity correction Stockman, A., & Sharpe, L. T. (2000). The spectral sen-
method. Medical Image Analysis, 11, 336–345. sitivities of the middle- and long-wavelength-sensitive
[PubMed] cones derived from measurements in observers of
Mannion, D. J., McDonald, J. S., & Clifford, C. W. G. known genotype. Vision Research, 40, 1711–1737.
(2009). Discrimination of the local orientation struc- [PubMed]
ture of spiral glass patterns early in human visual Stringham, J. M., Hammond, B. R., Wooten, B. R., &
cortex. Neuroimage, 46, 511–515. [PubMed] Snodderly, D. M. (2006). Compensation for light loss
McKeefry, D. J., & Zeki, S. (1997). The position and resulting from filtering by macular pigment: Relation
topography of the human colour centre as revealed by to the S-cone pathway. Optometry Visual Science, 83,
functional magnetic resonance imaging. Brain, 120, 887–894. [PubMed]
2229–2242. [PubMed] Sumner, P., Anderson, E. J., Sylvester, R., Haynes, J. D.,
Mourao-Miranda, J., Bokde, A. L., Born, C., Hampel, H., & Rees, G. (2008). Combined orientation and colour
& Stetter, M. (2005). Classifying brain states and information in human V1 for both L–M and S-cone
determining the discriminating activation patterns: chromatic axes. Neuroimage, 39, 814–824. [PubMed]
Support vector machine on functional MRI data. Tailby, C., Solomon, S. G., Dhruv, N. T., & Lennie, P.
Neuroimage, 28, 980–995. [PubMed] (2008a). Habituation reveals fundamental chromatic
Mullen, K. T., Dumoulin, S. O., McMahon, K. L., mechanisms in striate cortex of macaque. Journal of
de Zubicaray, G. I., & Hess, R. F. (2007). Selectivity of Neuroscience, 28, 1131–1139. [PubMed]
human retinotopic visual cortex to S-cone-opponent, Tailby, C., Solomon, S. G., & Lennie, P. (2008b). Func-
L/M-cone-opponent and achromatic stimulation. tional asymmetries in visual pathways carrying S-cone
European Journal of Neuroscience, 25, 491–502. signals in macaque. Journal of Neuroscience, 28,
[PubMed] 4078–4087. [PubMed] [Article]
Journal of Vision (2010) 10(5):25, 1–17 Goddard et al. 17

Tootell, R. B., Mendola, J. D., Hadjikhani, N. K., Ledden, Wyszecki, G., & Stiles, W. S. (1967). Color science:
P. J., Liu, A. K., Reppas, J. B., et al. (1997). Functional Concepts and methods, quantitative data and formulas.
analysis of V3A and related areas in human visual cortex. New York: John Wiley & Sons, Inc.
Journal of Neuroscience, 17, 7060–7078. [PubMed] Xiao, Y., Casti, A., Xiao, J., & Kaplan, E. (2007). Hue
Tootell, R. B., Reppas, J. B., Kwong, K. K., Malach, R., maps in primate striate cortex. Neuroimage, 35,
Born, R. T., Brady, T. J., et al. (1995). Functional 771–786. [PubMed] [Article]
analysis of human MT and related visual cortical Xiao, Y., Wang, Y., & Felleman, D. J. (2003). A spatially
areas using magnetic resonance imaging. Journal of organized representation of colour in macaque cortical
Neuroscience, 15, 3215–3230. [PubMed] area V2. Nature, 421, 535–539. [PubMed]
Wade, A. R., Brewer, A. A., Rieger, J. W., & Wandell, B. A. Yushkevich, P. A., Piven, J., Hazlett, H. C., Smith, R. G.,
(2002). Functional measurements of human ventral Ho, S., Gee, J. C., et al. (2006). User-guided 3D
occipital cortex: Retinotopy and colour. Philosoph- active contour segmentation of anatomical structures:
ical Transactions of the Royal Society of London B: Significantly improved efficiency and reliability.
Biological Sciences, 357, 963–973. [PubMed] [Article] Neuroimage, 31, 1116–1128. [PubMed]
Wandell, B. A., Dumoulin, S. O., & Brewer, A. A. (2007). Zaidi, Q., & Shapiro, A. G. (1993). Adaptive orthogonal-
Visual field maps in human cortex. Neuron, 56, ization of opponent-color signals. Biology Cybernics,
366–383. [PubMed] 69, 415–428. [PubMed]
Wandell, B. A., Poirson, A. B., Newsome, W. T., Baseler, Zeki, S. (1983). Colour coding in the cerebral cortex: The
H. A., Boynton, G. M., Huk, A., et al. (1999). Color responses of wavelength-selective and colour-coded
signals in human motion-selective cortex. Neuron, 24, cells in monkey visual cortex to changes in wavelength
901–909. [PubMed] composition. Neuroscience, 9, 767–781. [PubMed]
Webster, M. A., & Mollon, J. D. (1991). Changes in Zeki, S. (1990). A century of cerebral achromatopsia. Brain,
colour appearance following post-receptoral adapta- 113, 1721–1777. [PubMed]
tion. Nature, 349, 235–238. [PubMed]
Zeki, S., Watson, J. D. G., Lueck, C. J., Friston, K. J.,
Webster, M. A., & Mollon, J. D. (1994). The influence of Kennard, C., & Frackowiak, R. S. J. (1991). A direct
contrast adaptation on color appearance. Vision demonstration of functional specialization in human
Research, 34, 1993–2020. [PubMed] visual-cortex. Journal of Neuroscience, 11, 641–649.
Winawer, J., Horiguchi, H., Sayres, R., Amano, K., & [PubMed]
Wandell, B. (in press). Mapping hV4 and ventral
occipital cortex: The venous eclipse. Journal of Vision.
[Article]
 Journal of Vision (2010) 0(0):1, 1–13 http://www.journalofvision.org/content/0/0/1 1

1 Adaptable mechanisms sensitive to surface color

32 in human vision
4
5 School of Psychology, The University of Sydney,
6 Sydney, NSW, Australia, &
7 ARC Centre of Excellence in Vision Science,
98 Erin Goddard The Australian National University, Canberra, ACT, Australia AQ1
10
11 School of Medical Sciences and Bosch Institute,
12 The University of Sydney, Sydney, NSW, Australia, &
13 ARC Centre of Excellence in Vision Science,
14
15 Samuel Solomon The Australian National University, Canberra, ACT, Australia AQ1
16
17 School of Psychology, The University of Sydney,
18 Sydney, NSW, Australia, &
19 ARC Centre of Excellence in Vision Science,
20
21 Colin Clifford The Australian National University, Canberra, ACT, Australia AQ1

22 “Color constancy” refers to our ability to recognize the color of a surface despite changes in illumination. A range of cues
23 and mechanisms, from receptoral adaptation to higher order cognitive cues, is thought to contribute to our color constancy
24 ability. Here we used psychophysical adaptation to probe for an adaptable representation of surface color. We used stimuli
25 that were matched for cone contrast when averaged over time but were consistent with either a constant scene under
26 changing illumination or a changing scene. The color opponent aftereffect during adaptation to the constant scene was
27 greater than that induced by the changing scene stimulus. Since the stimuli were matched for the responses they would
28 elicit in receptoral mechanisms, the increased aftereffect in the constant scene condition cannot be wholly attributed to
29 adaptation of receptors and neural mechanisms responsive to raw quantal catch. We interpret our result as most
30 parsimoniously explained by the existence of adaptable mechanisms responsive to surface color, most likely located in
31 early visual cortex.
32 Keywords: color appearance/constancy, color vision, visual cortex
33 Citation: Goddard, E., Solomon, S., & Clifford, C. (2010). Adaptable mechanisms sensitive to surface color in human vision.
34 Journal of Vision, 0(0):1, 1–13, http://www.journalofvision.org/content/0/0/1, doi:10.1167/0.0.1.

35
with higher order object representation, abstract color
36
37 Introduction knowledge, and lexical representation of color names. For
54
55
example, neuropsychological literature points to a dissocia- 56
38 Color constancy tion between the coding of color object knowledge and 57
lexical coding of color names and shows that both these 58
39 To identify the color of surfaces in a scene correctly, we abilities can be impaired in patients with normal color vision 59
40 must separate out the reflectance properties of surfaces (Beauvois & Saillant, 1985; Luzzatti & Davidoff, 1994; for 60
41 from the spectral properties of the light illuminating the review, see Tanaka, Weiskopf, & Williams, 2001). 61
42 scene, an ability called “color constancy.” Humans 62
43 generally perform well, though imperfectly, when asked
44 to identify surface properties under different illumination Neural mechanisms for color constancy 63
45 conditions (Brainard, Brunt, & Speigle, 1997; Craven &
46 Foster, 1992; Helson, 1938; Judd, 1940; Worthey, 1985). That color constancy is accomplished requires that 64
47 Many higher order scene features may be used as cues surface color is neurally encoded, but the stage at which 65
48 when accomplishing color constancy, including specular that is done remains unclear. Receptoral mechanisms, 66
49 highlights (Lee, 1986), inter-reflections (Bloj, Kersten, & particularly multiplicative scaling (Brainard & Wandell, 67
50 Hurlbert, 1999), luminance color correlations (Golz & 1992; Foster & Nascimento, 1994; Ives, 1912; Land & 68
51 MacLeod, 2002), color memory (Hansen, Olkkonen, McCann, 1971; Shapley & Enroth-Cugell, 1984), can 69
52 Walter, & Gegenfurtner, 2006), and Gestalt laws (Gilchrist account for a large proportion of color constancy 70
53 et al., 1999). Such cues are likely to require interactions (Smithson, 2005). However, retinal mechanisms alone 71

doi: 1 0. 11 67 / 0 . 0 . 1 Received March 1, 2010; published 0 0, 2010 ISSN 1534-7362 * ARVO

 Journal of Vision (2010) 0(0):1, 1–13 Goddard, Solomon, & Clifford 2

72 are insufficient to account for all of our color constancy mechanisms, located beyond the retina and most likely 127
73 (Brill & West, 1986; Worthey & Brill, 1986). beyond other subcortical areas, but before cognitive 128
74 Beyond the retina, it is less clear what neural mecha- areas. 129
130
75 nisms might underlie color constancy. In occipital cortex, 131
76 a potential candidate is area V4. In non-human primates,
area V4 has been argued to contain a neural representation
77
78 of surface reflectances (Kusunoki, Moutoussis, & Zeki, Methods 132
133
79 2006; Wild, Butler, Carden, & Kulikowski, 1985; Zeki,
80 1983), but the question of whether V4 is unique in these Color calibration procedures 134
81 properties remains controversial (Conway & Tsao, 2006; and display system 135
82 de Monasterio & Schein, 1982; Gegenfurtner, 2003). In
83 humans, there is less evidence that hV4 (the suggested Stimuli were generated and displayed using Matlab 136
84 homologue of V4) is a specialized “color center” (version 7) software, with routines from PsychToolbox 137
85 (Gegenfurtner, 2003), and no work specifically shows (Brainard, 1997; Pelli, 1997), on a Dell OpliPlex 755 138
86 color constant response properties in hV4 or other areas. computer driving an ATI Radeon HD 2400 Pro graphics 139
87 Finally, the higher order nature of many cues thought to card (8 bits per channel) to draw stimuli to a 33 ! 24 cm 140
88 contribute to color constancy and the neuropsychological Sony Trinitron E220 cathode ray tube monitor, refreshed 141
89 literature suggest the involvement of cognitive areas at 85 Hz. Experiments took place in a darkened room with 142
90 beyond occipital cortex, but how these influences are black walls, and the monitor was viewed from a distance 143
91 implemented neurally remains ill-defined. of 0.57 m. The monitor was calibrated using a ColorCAL 144
92 Models of color constancy typically imply a processing colorimeter (Cambridge Research Systems). Changes in 145
93 stage in visual cortex that transforms photoreceptor both chromaticity and luminance of the screen with 146
94 activations into a representation of surface color (Brainard increasing R, G, and B values were taken into account 147
95 & Freeman, 1997; D’Zmura & Lennie, 1986; Lennie, when generating the experimental stimuli. The CIE (xyY) 148
96 1999; Logvinenko & Maloney, 2006; Robilotto & Zaidi, coordinates measured for 16 values during calibration were 149
97 2004). Yet the large proportion of color constancy interpolated to 255 values using the best fitting spline, and 150
98 accomplished at the level of the retina, combined with these were used to calculate the xyY coordinates of each 151
99 the cognitive nature of many cues thought to contribute to combination of R, G, and B intensity values, resulting in a 152
100 color constancy, leaves open the question whether a 256 ! 256 ! 256 matrix of xyY coordinates. 153
101 neural representation of surface color intermediate to the We did not obtain color matching or spectral sensitivity 154
102 retinal and cognitive stages exists. Here we sought to functions for individual observers but used measurements 155
103 address this question using a color opponent aftereffect. from “standard observers” (see Brainard & Stockman, 2010 156
104
for a discussion of errors that could be introduced by this 157
assumption). Each xyY coordinate was transformed into an 158
105 Color opponent aftereffects LMS cone excitation coordinate using a conversion matrix 159
derived from the left matrix division of the CIE (1931) AQ2
160
106 Prolonged viewing of a field of one color induces a color matching functions by the Stockman and Sharpe 161
107 color opponent aftereffect. For example, after viewing a (2000) 2-degree cone spectral sensitivity functions. Both 162
108 red field, a neutral gray appears tinged with green. This the CIE (1931) color matching functions and the Stockman 163
109 effect is mediated predominantly by mechanisms at a low and Sharpe (2000) 2-degree cone spectral sensitivity 164
110 level: photoreceptors reduce their responsiveness over a functions were obtained from the “PsychColorimetricData” 165
111 period of prolonged stimulation, changing the response of folder of PsychToolbox, and the CIE (1931) color matching 166
112 post-receptoral mechanisms to subsequent stimuli, and functions were scaled by 683 in order that the Y (luminance) 167
113 shifting the appearance of those stimuli away from the coordinate had units of candelas/m2. Stimuli were specified 168
114 color of the adapting stimulus. When adapted to red, the in terms of their LMS cone excitation coordinates and the 169
115 photoreceptor response to a neutral gray is the same as closest RGB value was found by finding the lowest root 170
116 the response to green under neutral adaptation. mean square error between the target cone excitation and 171
117 Here we used prolonged viewing of two types of stimuli the values achievable on the monitor. Where the lowest root 172
118 to investigate the aftereffects of adaptation to surface mean square error was greater than 0.01, or the best fitting 173
119 color. We used stimuli that induce the same amount of RGB value included a channel at maximum intensity, the 174
120 receptoral adaptation but which evoke different percepts stimulus was considered to be out of range and excluded. 175
121 of surface color. We will show an aftereffect that cannot 176
122 be wholly attributed to adaptation of receptors and neural
123 mechanisms responsive to the time-averaged input signal. Observers 177
124 We attribute this effect to the adaptation of neural
125 mechanisms that encode surface color in a manner more Ten observers (five males), aged 20 to 35 years old, took 178
126 robust to illuminant changes than the encoding by retinal part; eight were naive to the purposes of the investigation 179
 Journal of Vision (2010) 0(0):1, 1–13 Goddard, Solomon, & Clifford 3

180 (all observers except the authors EG and SS). All had array of flat, matte surfaces under diffuse illumination that 195
181 normal or corrected-to-normal visual acuity and nor- varied sinusoidally between two different illuminants at 196
182 mal color vision as assessed using Isihara (1990) and 1 Hz. In the changing scene condition, the same surfaces, 197
183 the Hardy–Rand–Rittler (HRR, 4th edition, published illuminants, and rate of illuminant change were used as for 198
184 by Richmond Products) psuedoisochromatic plates. the constant scene condition, but the phase of the 199
185 Observers provided informed consent, and the entire illuminant modulation was randomized across the surfa- 200
186 study was carried out in accordance with guidelines of ces. The two conditions were identical when averaged 201
187 the Human Research Ethics Committee of the University over time; across each cycle of illumination change (1 s), 202
188 of Sydney. they had the same surfaces under the same local 203
189 illumination. The adapting conditions differed only in 204
whether the illuminant modulation was the same for all 205
190 Adapting stimuli surfaces (the “correlated” condition, Figure 1A) or had a 206
different randomly chosen phase for each surface (the 207
191 Adapting stimuli simulated either a constant scene under “decorrelated” condition, Figure 1B). 208
192 changing illumination or a changing scene under constant Rendered surfaces were randomly drawn from 184 209
193 illumination, as illustrated schematically in Figure 1. surface reflectance functions supplied by Hannah Smithson 210
194 In the constant scene condition, the stimulus simulated an (Smithson & Zaidi, 2004). These were derived from a 211

Figure 1. Sample stimuli for the “correlated” and “decorrelated” conditions, with fixation cross. Adapting stimuli consisted of up to 3000
ellipses with colors that simulated flat matte surfaces under diffuse illumination. Over time, the illumination sinusoidally modulated from an
incandescent bulb (CIE Illuminant A) to daylight (CIE Illuminant D65). Across one cycle of this illumination change, the two adapting
stimuli had the same surfaces under the same illumination. The difference between the two conditions was that the changes of different
surfaces were either (A) correlated with each other or (B) decorrelated, with random onset phases. (C) Sample frames of the test stimuli.
On the left is a sample frame with the red reference to the right of fixation ((L j M)/(L + M) chromaticity coordinate of 0.67). On the right is
a sample frame with the green reference to the left of fixation ((L j M)/(L + M) chromaticity coordinate of 0.59).
 Journal of Vision (2010) 0(0):1, 1–13 Goddard, Solomon, & Clifford 4

212 series of measurements of natural and man-made objects Illuminant D65. Each adaptation run began with a 252
213 (Chittka, Shmida, Troje, & Menzel, 1994; Hiltunen, minute of adaptation before the first trial, and there was 253
214 1996; Marshall, 2000; Vrhel, Gershon, & Iwan, 1994), 6-s (6 stimulus cycles) top-up adaptation before each 254
215 including a broad range of colors. The two illuminants subsequent trial. 255
216 were an incandescent bulb and daylight (CIE Standard 256
217 Illuminants A and D65, respectively) and were scaled
218 such that they had approximately equal photopic lumi- Test stimuli and observer’s task 257
219 nance, using the Stockman and Sharpe (2000) 2-degree
220 luminosity function. We measured the perceived chromaticity of test 258
221 Three thousand ellipses were drawn at random locations surfaces in each of three conditions: prior to adaptation 259
222 within the stimulus field and each was rendered with one and during adaptation to correlated and decorrelated 260
223 of the surfaces. The heights of the ellipses ranged from adapting stimuli. On each trial, the period of top-up 261
224 0.03 to 1.3 degrees visual angle, with the probability of adaptation was followed by a test stimulus, where the two 262
225 each height h inversely proportional to h3. The width of adapting stimuli were replaced by a test and a reference 263
226 each ellipse varied randomly between 1/3h and 3h. surface. If the observer failed to respond within 3 s of the 264
227 Overlaid on this array of surfaces were the circular test stimuli being presented, the display returned to 265
228 adapting stimuli, located to the left and right of fixation, another 6 s of top-up adaptation before presenting the 266
229 with diameters subtending 8 degrees. The total arrange- same trial again. The spatial arrangement of background 267
230 ment was a rectangle subtending 30 ! 16 degrees, and the surfaces remained the same as they were during the 268
231 remainder of the screen was black. adapting stimulus, and the illumination of the entire scene 269
232 For each observer, the adapting stimuli were either was CIE Standard Illuminant D65. For the condition 270
233 green to the left and red to the right of fixation, or yellow– without adaptation, all stimulus and timing parameters 271
234 green to the left and blue to the right of fixation; were the same as for the adapting stimuli, with the 272
235 chromaticities of all adapting stimuli under both illumi- exception that the initial adaptation period and the top-up 273
236 nants are shown in Table 1. Adapting surfaces were adaptation periods were omitted; one test stimulus was 274
237 rendered under the same illumination as the background replaced immediately by the next, meaning that the total 275
238 surfaces; in the “correlated” condition, the modulation duration of this condition was approximately a quarter the 276
239 was in the same phase for each of the two adapting duration of the adaptation conditions. 277
240 surfaces and for the background; for the “decorrelated” Response times did not vary greatly between condi- 278
241 condition, the modulations of the two adapting surfaces tions; the median reaction times in each condition, 279
AQ3
242 were perfectly out of phase with each other. In the averaged across subjects (T1 standard deviation), were 280
243 correlated condition, the illuminant modulation always 1.18 s (T0.32 s) in the correlated adaptor condition, 1.15 s 281
244 commenced and finished with the daylight (CIE Standard (T0.32 s) in the decorrelated adaptor condition, and 1.25 s 282
245 Illuminant D65) illumination, while the onset phases of (T0.47 s) in the no adapt condition. 283
246 the illuminant modulation for surfaces in the decorre- In conditions in which observers were adapted to red 284
247 lated case were randomly drawn from all points in the and green, they were asked to respond with a button press 285
248 cycle. In the decorrelated condition, the adapting surface to indicate whether the surface to the left or right of 286
249 on the left commenced and finished with CIE Standard fixation appeared “more red” (observers were instructed to 287
250 Illuminant A illumination and the adapting surface on treat “more red” and “less green” as equivalent). We 288
251 the right commenced and finished with CIE Standard estimated the chromaticity of the test that had perceptual 289

t1.1 Cone fundamental primaries Macleod–Boynton chromaticity coordinates

t1.2 Illuminant Adapting surface L M S (L j M)/(L + M) S/(L + M)

t1.3 CIED65 Red 0.027 0.015 0.008 0.742 0.361

t1.4 Green 0.022 0.023 0.008 0.601 0.362
t1.5 Blue 0.023 0.022 0.019 0.624 0.801
t1.6 Yellow 0.023 0.022 0.015 0.624 0.647
t1.7 CIE A Red 0.026 0.011 0.002 0.782 0.111
t1.8 Green 0.021 0.018 0.002 0.652 0.115
t1.9 Blue 0.022 0.017 0.005 0.674 0.252
t1.10 Yellow 0.022 0.017 0.004 0.674 0.204

t1.13
t1.11 Table 1. Adapting stimuli chromaticities. The cone fundamental primaries (LMS) and Macleod–Boynton chromaticity coordinates ((L j M)/
(L + M) and S/(L + M)) are shown for each of the adapting surfaces, under both illuminants. Over time, the illumination of the adapting
stimuli modulated sinusoidally between CIE Standard Illuminant D65 (daylight) and CIE Standard Illuminant A (an incandescent bulb).
 Journal of Vision (2010) 0(0):1, 1–13 Goddard, Solomon, & Clifford 5

290 equality with the reference (the test that was equally likely
291 to be reported more red or more green than a reference)
292 during two sessions, each of which was made up of eight
293 interleaved adaptive psychophysical staircases (Kontsevich
294 & Tyler, 1999) consisting of 30 trials each. For each
295 staircase, the test surface had a fixed S/(L + M) chro-
296 maticity coordinate and position (either S/(L + M) = 0.24
297 or 0.71, located to the left of fixation, or S/(L + M) = 0
298 or 0.47, located to right), while the (L j M)/(L + M)
299 chromaticity coordinate was set by the adaptive staircase.
300 On each trial, the reference surface located on the
301 opposite side to the test had the same S/(L + M)
302 chromaticity coordinate as the test surface. Its (L j M)/ Figure 2. A sample pair of psychometric curves shows the
303 (L + M) chromaticity coordinate was 0.59 (green, when adaptation-induced shifts in chromaticity of the test surface during
304 the reference was on the left and the test was on the right) adaptation to green, for one naive observer, JC. Positive values
305 or 0.67 (red, when the reference was on the right). We along the abscissa are test surfaces that are more green than the
306 shifted the chromaticity of the reference stimuli toward reference. For both conditions, the curves are shifted to the right,
307 the adapting chromaticity at that location in order to indicating a color-opponent aftereffect. Any extra shift in the
308 ensure that observers point of equal redness/greenness “correlated” condition cannot be attributed to receptoral adapta-
309 was within the gamut of our monitor. We introduced this tion, since when average over time the two conditions contain the
310 shift after a pilot experiment with the same reference same LMS cone contrast.
311 stimuli on the left and right where the adaptive staircase
312 reached the end of the range of our monitor before finding
313 a chromaticity that observers reported to be “more red” chromaticity coordinate of 0.10 (yellow, when reference 343
314 50% of the time. Ten randomly generated spatial arrange- was on the left and test on the right) or 0.45 (blue, when 344
315 ments of the background surfaces were interleaved reference was on the right). 345
346
316 throughout the run; the adaptation and test phases of 347
317 each trial were formed from the same configuration.
318 The data from each staircase (30 trials) were used to
319 make one estimate of the chromaticity of the test surface Results 348
349
320 that the observer was equally likely to report as more red
321 or more green than the reference. The proportion of “more Opponent aftereffect for surface color 350
322 green” responses (Y^ ) as a function of test chromaticity (x)
323 were fit using the following logistic equation: We measured the shift in perceived chromaticity of a 351
test surface while subjects were adapted to red and green, 352
1j c or blue and yellow surfaces, using a two-alternative 353
Y^ ¼ c=2 þ ; ð1Þ
1 þ ejðxjaÞ=b forced-choice paradigm (see Methods section for details). 354
Figure 2 shows for one observer how the appearance of 355
324
325 where a is the test chromaticity of perceptual equality a single test surface changed with adaptation to a green 356
326 with the reference (the value of x where Y^ = 0.5), b is a surface. The magnitude of this change in appearance 357
327 curvature parameter, and c is the miss rate. Sample data depended on the context of the green adapting surface, 358
328 from 2 staircases for one observer, along with fitted which varied with adaptation condition. Without adapta- 359
329 curves, are shown in Figure 2. tion, the chromaticity of subjective equality with the 360
330 Four observers adapted to red and green surfaces, and reference was nearly veridical (the point of subjective 361
331 two of these observers along with additional four equality was 0.0012, for clarity that curve is not shown). 362
332 observers adapted to blue and yellow surfaces. The design During adaptation to the decorrelated stimulus, the curve 363
333 for blue and yellow surfaces was equivalent: observers shifted substantially to the rightVthe observer required 364
334 reported whether the left or right surface was “more blue” the test surface to be greener than the reference to see it 365
335 (equivalent to “less yellow”). Test surfaces had a fixed as the same. This shift is expected because the adaptor 366
336 (L j M)/(L + M) chromaticity coordinate and position changes the state of the receptoral and neural mechanisms 367
337 (either (L j M)/(L + M) = 0.62 or 0.72, located to the that lie over the test surface. If there were only adaptation 368
338 left of fixation, or (L j M)/(L + M) = 0.57 or 0.67 of mechanisms responsive to the time-averaged input 369
339 located to right), while the S/(L + M) chromaticity signal, then the aftereffect induced during the correlated 370
340 coordinate was set by the adaptive staircase. The reference condition should be of the same magnitude. It was not: 371
341 surface had the same (L j M)/(L + M) chromaticity instead the psychometric function shifted further to the 372
342 coordinate as the test surface and had an S/(L + M) right. 373
 Journal of Vision (2010) 0(0):1, 1–13 Goddard, Solomon, & Clifford 6

374 The magnitude of the aftereffect in the two adaptation Statistical analysis 386
375 conditions for individual observers are shown in Figure 3.
376 In Figure 4, the results are summarized; the average For our statistical analysis, we excluded the data 387
377 difference between the conditions for each adapting color collected without adaptation and for each adapting color 388
378 is expressed as a percentage of the shift in the decorrelated (red, green, blue, and yellow) performed a 2-way analysis 389
379 case. For each adapting color, the perceived chromaticity of variance. In each 2-way analysis of variance, we 390
380 of the test surface shifted away from the color of the included observer as a random effect and tested for a main 391
381 adapting surface, consistent with a color opponent after- effect of adaptation condition (correlated versus decorre- 392
382 effect. As in the example in Figure 2, the color opponent lated). For adaptation to red, green, and blue, the correlated 393
383 aftereffect was usually of greater magnitude in the adaptor induced an aftereffect of significantly greater 394
384
385 correlated condition than in the decorrelated condition. magnitude (red: p G 0.01, F1,3 = 30.19, green: p G 0.05, 395

Figure 3. Adaptation-induced shifts in perceived chromaticity of a reference surface, after adaptation to (A) red and green or (B) blue and
yellow. Mean shifts in perceived chromaticity along the (A) (L j M)/(L + M) and (B) S/(L + M) dimensions are shown for each observer
(points, T1 standard error of the mean) and averaged across observers (gray bars). The magnitude of shifts in perceived chromaticity of
test surfaces is plotted relative to the physical chromaticity of the reference surface. The direction of these shifts depended on adaptor
color; when observers adapted to red or blue, the shifts were positive along the (L j M)/(L + M) and S/(L + M) dimensions, respectively;
for green and yellow, the shifts were negative. In each case, the direction of the shift was consistent with the subject requiring a test AQ3
surface whose chromaticity was shifted toward the adapting color for a perceptual match with the reference. When the adapting surfaces
were red, green, or blue, the correlated condition (C) induced an aftereffect of significantly greater magnitude than the decorrelated
condition (D). When the adapting surface was yellow, the effect went in the same direction, but the difference between conditions was not
significant (see text for details).
 Journal of Vision (2010) 0(0):1, 1–13 Goddard, Solomon, & Clifford 7

significant when analyzed in the same manner as above, 428

with a 2-way analysis of variance for each adapting color 429
(red: p = 0.22, F1,1 = 7.77, green: p = 0.26, F1,1 = 5.31, 430
blue: p = 0.13, F1,1 = 22.41, yellow: p = 0.29, F1,1 = 4.33), 431
but unlike the main experiment, where each condition was 432
completed by at least four subjects, only two subjects 433
completed each condition here. Subjects who completed 434
both experiments generally showed comparable magni- 435
tude of illusion in the decorrelated condition and the low- 436
contrast decorrelated condition, and in all cases, adaptation 437
to the correlated scene induced an effect of greater mag- 438
nitude than the low-contrast decorrelated scene. The dif- 439
ference in magnitude of the average effect here (Figure 6) 440
and in the main experiment (Figure 4) therefore can mostly 441
Figure 4. Adaptation-induced shift in chromaticity: average differ- be attributed to inter-subject variability: some of the sub- 442
ence between the correlated and decorrelated adaptation con- jects with high illusion magnitudes in the main experiment 443
ditions, as a percentage of the shift in the decorrelated condition. were not available for this control experiment. 444
Here data from Figure 3 are replotted, averaged across all Overall, these results show that the difference in 445
observers. Error bars indicate T1 standard error of the between- aftereffects seen during adaptation to correlated and 446
subjects mean. decorrelated surrounds cannot be accounted for by differ- 447
ences in spatial cone contrast. 448
449
396 F1,3 = 27.30, blue: p G 0.05, F1,5 = 10.20). For adaptation
397 to yellow, the correlated adaptor induced an aftereffect of Effect remains when adapting stimuli 450
398 greater magnitude, but this difference was not significant are matched for onset and offset phases 451
399 (p = 0.20, F1,5 = 2.03).
400 In the main experiment, the onset and offset phases of 452
the adapting stimuli varied between the correlated and 453
401 Effect cannot be explained by frame-by-frame decorrelated adaptation conditions. In the correlated 454
402 differences in spatial cone contrast condition, all adapting surfaces were rendered under the 455
same illuminant as the test surfaces (CIE Standard D65) at 456
403 The correlated and decorrelated adaptors above have the beginning and end of each adaptation period. In the 457
404 identical distributions of colors over space and time, but decorrelated condition, only the red and blue adapting 458
405 within any frame of the stimulus, the spatial variance in surfaces were rendered under CIE Standard D65 at the 459
406 chromaticities is lower in the correlated condition. beginning and end of each adaptation period; the green 460
407 Brown and MacLeod (1997) demonstrated that colors and yellow adaptors were rendered under CIE Standard A. 461
408 embedded in a background of greater variance appear less To test the possibility that this difference between the 462
409 saturated; for our stimuli, this would predict that the conditions at the transition between adaptation and test 463
410 adapting stimuli would appear less saturated in the may have contributed to the differing aftereffects, we 464
411 decorrelated condition, which might diminish the opponent performed an additional control experiment where the 465
412 aftereffect. To test this possibility, we matched the adapting stimuli were all rendered under an equal mixture 466
413 correlated and decorrelated conditions for variance in of the two illuminants at their onset and offset. 467
414 LMS cone excitation coordinates across individual frames For this experiment, we repeated a subset of the original 468
415 of the adapting stimulus. experiment for two observers who completed the original 469
416 To generate adapting stimuli with the same variance, version (one naive) and for two new naive observers. We 470
417 the L, M, and S cone excitation coordinates of decorre- tested the red–green adaptation condition, using one of the 471
418 lated stimulus surrounds were scaled by 1.00, 0.97, and four S/(L + M) coordinates for the test stimuli (which 472
419 0.66, respectively, as detailed in Appendix A. All other was the same as for the adapting stimuli). If the 473
420 stimulus properties were unchanged. Using these stimuli, modulation of the illuminant over time is CIE Standard 474
421 we repeated the decorrelated condition for two observers D65 at 0- phase, CIE Standard A at 180- phase, and an 475
422 (one naive) for both the red–green and blue–yellow equal mixture of the two occurs at 90- and 270-, then in 476
423 adapting stimuli. Individual results are plotted in Figure 5 the original experiment the adaptation in the correlated 477
424 and summarized in Figure 6. condition always commenced at 0-, and in the decorre- 478
425 As in the main experiment, the correlated adaptor lated condition, the red and blue adapting surfaces 479
426 induced an aftereffect of greater magnitude than the low- commenced at 0- while the green and yellow adapting 480
427 contrast decorrelated adaptor. This difference was not surfaces commenced at 90-. In this control experiment, 481
Journal of Vision (2010) 0(0):1, 1–13 Goddard, Solomon, & Clifford 8

Figure 5. Adaptation-induced shift of perceived chromaticity during adaptation to red, green, blue, and yellow. Data for observers AG, EG,
and TF from Figure 3 are replotted beside the adaptation-induced shift found using a low-contrast version of decorrelated adapting
stimulus (L). The low-contrast decorrelated stimulus was matched to have the same frame-by-frame L, M, and S variances as the
correlated adapting stimulus. Conventions as in Figure 3.

adapting stimuli in the correlated condition commenced at 482

either 90- or 270-, and each staircase included an equal 483
number of both types, randomly interleaved in order. In 484
the decorrelated condition, the two adapting surfaces 485
remained 180- out of phase with one another, but on each 486
trial one surface commenced at 90- and the other at 270-. 487
Each staircase in the decorrelated condition included an 488
equal number of trials where the surfaces commencing at 489
90- and 270- were on the left and right, or on the right and 490
left, respectively. Since each adaptation period included 491
an integer number of cycles, the onsets and offsets of the 492
two adapting surfaces were always an equal combination 493
of the two illuminants. All other stimulus properties were 494
unchanged. Individual results are plotted in Figure 7, and 495
Figure 6. Adaptation-induced shift in chromaticity: average differ- summarized in Figure 8. 496
ence between the correlated and low-contrast decorrelated In this control experiment, the correlated adaptor 497
adaptation conditions, as a percentage of the shift in the low- induced an aftereffect of greater magnitude than the 498
contrast decorrelated condition. Here data from the red–green decorrelated adaptor for both the red and green adapting 499
and blue–yellow conditions (shown in Figure 5) are averaged surfaces. As for the previous experiments, we performed a 500
across 2 observers and across different reference chromaticities. 2-way analysis of variance for each adapting color and 501
Error bars indicate T1 SEM. found that this difference was significant for the red 502
 Journal of Vision (2010) 0(0):1, 1–13 Goddard, Solomon, & Clifford 9

Overall, these results suggest that the finding in the original 521
experiment cannot be accounted for by differences in the 522
onset and offset phases. 523
524
525

Discussion 526
527

Implications for representations 528

of surface color 529

We show that a color opponent aftereffect of greater 530

magnitude is induced by adaptation to scenes consistent 531
with changing illumination than by those that are 532
consistent with a changing scene. These scenes recruit 533
the same adaptation of receptors and of neural mecha- 534
nisms responsive to the time-averaged input signal. We 535
interpret this increased aftereffect as evidence of adaptable 536
neural mechanisms sensitive to the color of surfaces, 537
independent of illumination, which are adapted to some 538
extent by the display that simulates a changing scene (the 539
“decorrelated” condition) and to a greater extent when the 540
display simulates a constant scene (the “correlated” 541
condition). While this representation of surface color 542
may not include all cognitive factors thought to contribute 543
to color constancy, it must incorporate the result of 544
mechanisms beyond retinal adaptation, since the stimuli 545
in our two conditions were matched for the extent to 546
which they would adapt retinal mechanisms. 547
At a minimum, the neural mechanisms giving rise to 548
color constancy must include three broadly defined stages: 549
First, the receptoral processes that compensate for gross 550

Figure 7. Adaptation-induced shift of perceived chromaticity

during adaptation to red and green, for correlated and decorre-
lated adapting surfaces that were matched in their onset and
offset phases (see text for details). Conventions as in Figure 3.

503 adapting surface (p G 0.01, F1,3 = 51.33) but not the green
504 (p = 0.12, F1,3 = 4.08). Subjects who completed the
505 original experiment showed a similar pattern of results in
506 this control condition.
507 In the original experiment, where the adapting stimuli
508 differed in their onset and offset phases, it is possible that
509 they induced different degrees of adaptation in a transient,
510 low-level mechanism. In this control experiment, the
511 onset and offset phases of the adapting stimuli were
512 always at a point in the cycle where the illuminant was an
513 equal mixture of the two illuminants. Additionally, for
514 both adapting surfaces and for both adaptation conditions, Figure 8. Adaptation-induced shift in chromaticity for adapting
515 there was an equal number of 90- and 270- onset/offset stimuli that were matched in their onset and offset phases (see
516 phases. Any transient, low-level mechanism should be text for details). Bars show the average difference between the
517 adapted to the same extent in the correlated and decorre- correlated and decorrelated adaptation conditions, as a percent-
518 lated adaptation conditions, but the magnitude of the age of the shift in the low-contrast decorrelated condition. Data
519 aftereffect induced in the correlated condition remains from Figure 7 are averaged across 4 observers. Error bars
520 greater than that induced in the decorrelated condition. indicate T1 SEM.
 Journal of Vision (2010) 0(0):1, 1–13 Goddard, Solomon, & Clifford 10

551 shifts of light intensity through multiplicative scaling; components of opposite sign, which leads to a preferential 579
552 second, an adaptable stage by which the processing of some response to chromatic edges (Solomon & Lennie, 2007). 580
553 global stimulus statistics have led to a further discounting While they may exist, no cells with this property have 581
554 of the illuminant, providing a more robust representation of been reported in retina or LGN. Within visual cortex, it is 582
555 surface reflectance; and third, the cognitive processes, possible that adaptable mechanisms occur as early as V1, 583
556 including the influences of object knowledge, language, where double opponent cells have been reported in 584
AQ3
557 and scene interpretation. macaque (Conway, 2001; Conway & Tsao, 2006; Johnson, 585
558 Hawken, & Shapley, 2001). 586
587
559 Location of adaptable surface reflectance
560 detectors Relation to other scission perception 588

561 What mechanisms might provide the adaptable repre- Color constancy, and the task of separating illuminant 589
562 sentations of surface color? It is unlikely that these and surface color properties, may helpfully be considered 590
563 representations could arise before visual cortex, both to lie within the broader category of any perceptual 591
564 because they are adaptable, and because there must be a scission of layers within a scene. For example, we think it 592
565 separation of surface and illuminant properties beyond likely that mechanisms that underlie color constancy are 593
566 that accomplished by retinal mechanisms. Neural mecha- also involved in the perception of transparent layers in a 594
567 nisms of the LGN important for color perception do not scene (Anderson & Winawer, 2005; Gerbino, Stultiens, 595
568 appear to habituate (de Valois, Abramov, & Jacobs, Troost, & de Weert, 1990; Westland & Ripamonti, 2000). 596
569 1966; Solomon, Peirce, Dhruv, & Lennie, 2004; Tailby, In relation to transparency, our result would imply that 597
570 Solomon, & Lennie, 2008), implying that adaptation of there is an adaptable representation of at least the 598
571 these mechanisms cannot be responsible for the increased lowermost layer in the scene. A possibility not explored 599
572 aftereffect. Furthermore, there is no evidence of neurons by this study is that there are separate representations of 600
573 in the LGN with chromatic properties thought to be different layers in each scene, each of which is independ- 601
574 essential for color constancy beyond that achieved by the ently adaptable. This would predict separate adaptation to 602
575 retina. Specifically, double opponent cells are argued to be illuminant color, or transparent surface color, along with 603
576 necessary for separating surface and illuminant (Hurlbert, surface reflectance properties. Such a mechanism would 604
577 1996; Lennie & D’Zmura, 1988). Double opponent cells be a neural correlate of the separate layer representations 605
578 have receptive fields that include two L–M opponent in intrinsic image models, among others, which propose 606

Figure A1. Stimulus statistics for the background surfaces in the (A) correlated illuminant condition, (B) decorrelated illuminant condition,
and (C) the reduced LMS variance decorrelated illuminant condition. The mean L, M, and S values are shown in the top, middle, and
bottom plots, respectively, for one cycle of the adapting stimulus. The gray shaded region is the mean T1 standard deviation, and the
black lines plot the maximum and minimum L, M, and S values on each frame. The mean (T1 standard deviation) is also written in the top
right of each plot. The reduced LMS variance decorrelated illuminant condition (C) was generated by scaling the decorrelated illuminant
condition B to have the same variance of L, M, and S values as the correlated illuminant condition A.
 Journal of Vision (2010) 0(0):1, 1–13 Goddard, Solomon, & Clifford 11

607 that scene properties such as reflectance, illumination, and psychophysically that such neural representations exist 660
608 transparency are decomposed and represented as separate and that they are adaptable. The procedure here reveals 661
609 layers or “images” (Adelson, 2000; Anderson & Winawer, intermediate mechanisms in the representation of sur- 662
610 2008). Future work could examine whether representa- face reflectance and may therefore be useful for future 663
611 tions of other layers are independently adaptable, and if investigations of color constancy and animal models. We 664
612 so, whether there is a limit to the number of layers that the hope to use this aftereffect in conjunction with functional 665
613 visual system encodes in this way. imaging to further elucidate the neural substrates of human 666
614 color constancy ability. 667

615 Potential limitations of this study

616 In interpreting the results of this study, we have

Appendix A 668
669
617 assumed that prolonged viewing of the “correlated” and
618 “decorrelated” stimuli should induce the same degree of Stimulus LMS statistics 670
619 adaptation in receptors and neural mechanisms whose
620 responses scale with the output of the receptors. This Figure A1. 671
621 assumption may be invalid if the stimuli are not matched in 672
622 the responses they evoke in such mechanisms. In our
623 results, we describe a control experiment where we tested
624 whether the lower spatial variance in chromaticities in the
625 correlated condition could account for the increased Acknowledgments 673
674
626 magnitude of the aftereffect in this case; we found that it
627 could not. We thank Hannah Smithson for providing reflectance 675
628 Our design also assumes that we are engaging mecha- spectra of the simulated surfaces. This work was 676
629 nisms that adapt on a time scale of 1 s or longer, since the supported by an Australian Postgraduate Award to EG 677
630 stimuli in the two conditions are only matched for cone and an Australian Research Fellowship to CC and by the 678
631 contrast when averaged over the entire stimulus cycle. If Australian Centre of Excellence in Vision Sciences. 679
632 more transient low-level adaptation were contributing to
633 the effect, the time-dependent differences between the two Commercial relationships: none. 680
634 types of stimuli will no longer be matched for these Corresponding author: Erin Goddard. 681
635 mechanisms. In the second control experiment described Email: [email protected]. 682
636 in our results, we balanced the onset and offset phases of Address: School of Psychology, The University of 683
637 our adapting stimuli in order to match the stimuli for any Sydney, Sydney, NSW 2006, Australia. 684
638 transient low-level mechanisms and found an effect of
639 comparable magnitude to that in the original experiment. References 685
686
640 However, it remains possible that our results were
641 influenced by other adaptable mechanisms integrating Adelson, E. H. (2000). Lightness perception and lightness 687
642 information over a longer period than a transient mech- illusions. In M. S. Gazzaniga (Ed.), The cognitive 688
643 anism, but less than a second, or that give unequal neurosciences (chap. 24, 2nd ed., pp. 339–351). 689
644 weighting to stimuli presented in the preceding second. Cambridge, MA: MIT Press. 690
645 Chromatic adaptation likely includes many processes with
646 varying time constants (Fairchild, 2005; Webster, 1996). Anderson, B. L., & Winawer, J. (2005). Image segmenta- 691
647 For example, Fairchild and Reniff (1995) report psycho- tion and lightness perception. Nature, 434, 79–83. 692
648 physical evidence suggesting that chromatic adaptation Anderson, B. L., & Winawer, J. (2008). Layered image 693
649 includes mechanisms that adapt over the course of a few representations and the computation of surface light- 694
650 seconds, or over many tens of seconds. ness. Journal of Vision, 8(7):18, 1–22, http://www. 695
651
652
journalofvision.org/content/8/7/18, doi:10.1167/ 696
8.7.18. [PubMed] [Article] 697
Beauvois, M., & Saillant, B. (1985). Optic aphasia for 698
653
654 Conclusions colours and colour agnosia: A distinction between 699
visual and visuo-verbal impairments in the processing 700
655 One problem with approaching the neural mechanisms of colours. Cognitive Neuropsychology, 2, 1–48. 701
656 of color constancy experimentally is that it is difficult to Bloj, M. G., Kersten, D., & Hurlbert, A. C. (1999). 702
657 separate out responses of neurons to the raw wavelengths, Perception of three-dimensional shape influences 703
658 or color contrast, from responses that show an addi- colour perception through mutual illumination. 704
659 tional degree of color constancy. We have demonstrated Nature, 402, 877–879. 705
 Journal of Vision (2010) 0(0):1, 1–13 Goddard, Solomon, & Clifford 12

706 Brainard, D. H. (1997). The psychophysics toolbox. Fairchild, M. D., & Reniff, L. (1995). Time course of 768
708
707 Spatial Vision, 10, 433–436. chromatic adaptation for color-appearance judgments. 769
709 Brainard, D. H., Brunt, W. A., & Speigle, J. M. (1997). Journal of the Optical Society of America A, Optics, 770
710 Color constancy in the nearly natural image. Image Science, and Vision, 12, 824–833. 771
772
711 I. Asymmetric matches. Journal of the Optical Society Foster, D. H., & Nascimento, S. M. (1994). Relational 773
712 of America A, Optics, Image Science, and Vision, 14, colour constancy from invariant cone-excitation 774
714
713 2091–2110. ratios. Proceedings of the Royal Society: Biological 775
715 Brainard, D. H., & Freeman, W. T. (1997). Bayesian Sciences, 257, 115–121. 776
777
716 color constancy. Journal of the Optical Society of Gegenfurtner, K. R. (2003). Cortical mechanisms of 778
717 America A, Optics, Image Science, and Vision, 14, colour vision. Nature Reviews Neuroscience, 4, 779
719
718 1393–1411. 563–572. 780
781
720 Brainard, D. H., & Stockman, A. (2010). Colorimetry.
Gerbino, W., Stultiens, C. I., Troost, J. M., & de Weert, 782
721 In M. Bass, J. M. Enoch, & V. Lakshminarayanan
C. M. (1990). Transparent layer constancy. Journal 783
722 (Eds.), Handbook of optics: Sponsored by the Optical
of Experimental PsychologyVHuman Perception and 784
723 Society of America (vol. 3, chap. 10, 3rd ed.,
Performance, 16, 3–20. 785
725
724 pp. 10.1–10.56). New York: McGraw-Hill.
726 Brainard, D. H., & Wandell, B. A. (1992). Asymmetric Gilchrist, A., Kossyfidis, C., Bonato, F., Agostini, T., 786
727 color matching: How color appearance depends on Cataliotti, J., Li, X. J., et al. (1999). An anchoring 787
728 the illuminant. Journal of the Optical Society of theory of lightness perception. Psychological Review, 788
729 America A, Optics, Image Science, and Vision, 9, 106, 795–834. 789
732
731
730 1433–1448. Golz, J., & MacLeod, D. I. (2002). Influence of scene 790
733 Brill, M. H., & West, G. (1986). Chromatic adaptation and statistics on colour constancy. Nature, 415, 637–640. 791
734 color constancy: A possible dichotomy. Color Hansen, T., Olkkonen, M., Walter, S., & Gegenfurtner, 792
736
735 Research and Application, 11, 196–204. K. R. (2006). Memory modulates color appearance. 793
737 Brown, R. O., & MacLeod, D. I. (1997). Color appearance Nature Neuroscience, 9, 1367–1368. 794
738 depends on the variance of surround colors. Current Helson, H. (1938). Fundamental problems in color vision: 795
740
739 Biology, 7, 844–889. I. The principle governing changes in hue, saturation, 796
741 Chittka, L., Shmida, A., Troje, N., & Menzel, R. (1994). and lightness of non-selective samples in chromatic 797
742 Ultraviolet as a component of flower reflections, and illumination. Journal of Experimental Psychology, 798
743 the colour perception of hymenoptera. Vision 23, 439–476. 799
745
744 Research, 34, 1489–1508. Hiltunen, J. (1996). Munsell colors matte (spectropho- 800
746 Conway, B. R. (2001). Spatial structure of cone inputs to tometer measurements by Hiltunen). Retrieved Sep- 801
747 color cells in alert macaque primary visual cortex tember 10, 1999, from http://www.it.lut.fi/ip/research/ 802
749
748 (V-1). Journal of Neuroscience, 21, 2768–283. color/database/download.html#munsell_spec_matt. 803
750 Conway, B. R., & Tsao, D. Y. (2006). Color architecture Hurlbert, A. (1996). Colour vision: Putting it in context. 804
751 in alert macaque cortex revealed by fMRI. Cerebral Current Biology, 6, 1381–1384. 805
752 Cortex, 16, 1604–1613. Ishihara, S. (1990). Ishihara’s tests for color blindness 806
753 Craven, B. J., & Foster, D. H. (1992). An operational (38 plate ed). Tokyo/Kyoto, Japan: Kanehara, Shuppan. 807
754 approach to colour constancy. Vision Research, 32, Ives, H. (1912). The relation between the color of the 808
755 1359–1366. illuminant and the color of the illuminated object. 809
756 de Monasterio, F. M., & Schein, S. J. (1982). Spectral Transactions of the Illumination Engineering Society, 810
757 bandwidths of color-opponent cells of geniculocort- 7, 62–72. 811
758 ical pathway of macaque monkeys. Journal of Neuro- Johnson, E. N., Hawken, M. J., & Shapley, R. (2001). The 812
759 physiology, 47, 214–224. spatial transformation of color in the primary visual 813
760 de Valois, R. L., Abramov, I., & Jacobs, G. H. (1966). cortex of the macaque monkey. Nature Neuroscience, 814
761 Analysis of response patterns of LGN cells. Journal 4, 409–416. 815
762 of the Optical Society of America, 56, 966–977. Judd, D. B. (1940). Hue saturation and lightness of surface 816
763 D’Zmura, M., & Lennie, P. (1986). Mechanisms of color colors with chromatic illumination. Journal of the 817
764 constancy. Journal of the Optical Society of America Optical Society of America, 30, 2–32. 818
765 A, Optics, Image Science, and Vision, 3, 1662–1672. Kontsevich, L. L., & Tyler, C. W. (1999). Bayesian 819
766 Fairchild, M. D. (2005). Color appearance models adaptive estimation of psychometric slope and 820
767 (2nd ed.). Chichester, UK: John Wiley & Sons. threshold. Vision Research, 39, 2729–2737. 821
 Journal of Vision (2010) 0(0):1, 1–13 Goddard, Solomon, & Clifford 13

822 Kusunoki, M., Moutoussis, K., & Zeki, S. (2006). Effect Smithson, H. E. (2005). Sensory, computational and 867
823 of background colors on the tuning of color-selective cognitive components of human colour constancy. 868
824 cells in monkey area V4. Journal of Neurophysiology, Philosophical Transactions of the Royal Society of 869
825 95, 3047–3059. London B: Biological Sciences, 360, 1329–1346. 871
870
826 Land, E. H., & McCann, J. J. (1971). Lightness and Solomon, S. G., & Lennie, P. (2007). The machinery 872
827 retinex theory. Journal of the Optical Society of of colour vision. Nature Reviews Neuroscience, 8, 873
828 America, 61, 1–11. 276–286. 875
874
829 Lee, H. C. (1986). Method for computing the scene- Solomon, S. G., Peirce, J. W., Dhruv, N. T., & Lennie, P. 876
830 illuminant chromaticity from specular highlights. (2004). Profound contrast adaptation early in the 877
831 Journal of the Optical Society of America A, Optics, visual pathway. Neuron, 42, 155–162. 879
878
832 Image Science, and Vision, 3, 1694–1699. Stockman, A., & Sharpe, L. T. (2000). The spectral sen- 880
833 Lennie, P. (1999). Color coding in the cortex. In K. R. sitivities of the middle- and long-wavelength-sensitive 881
834 Gegenfurtner & L. T. Sharpe (Eds.), Color vision: cones derived from measurements in observers of 882
835 From genes to perception (pp. 235–247). Cambridge, known genotype. Vision Research, 40, 1711–1737. 884
883
836 UK: Cambridge University Press. Tailby, C., Solomon, S. G., & Lennie, P. (2008). Functional 885
837 Lennie, P., & D’Zmura, M. (1988). Mechanisms of asymmetries in visual pathways carrying S-cone 886
838 color vision. Critical Reviews in Neurobiology, 3, signals in macaque. Journal of Neuroscience, 28, 887
839 333–400. 4078–4087. 888

840 Logvinenko, A. D., & Maloney, L. T. (2006). The Tanaka, J., Weiskopf, D., & Williams, P. (2001). The role 889
841 proximity structure of achromatic surface colors and of color in high-level vision. Trends in Cognitive 890
842 the impossibility of asymmetric lightness matching. Sciences, 5, 211–215. 891
843 Perception & Psychophysics, 68, 76–83. Vrhel, M. J., Gershon, R., & Iwan, L. S. (1994). 892
844 Luzzatti, C., & Davidoff, J. (1994). Impaired retrieval of Measurement and analysis of object reflectance 893
845 object-colour knowledge with preserved colour nam- spectra. Color Research and Application, 19, 4–9. 894
846 ing. Neuropsychologia, 32, 933–950. Webster, M. A. (1996). Human colour perception and its 895
847 Marshall, N. J. (2000). Communication and camouflage adaptation. Network: Computation in Neural Systems, 896
848 with the same “bright” colours in reef fishes. Philo- 7, 587–634. 897
849 sophical Transactions of the Royal Society of London Westland, S., & Ripamonti, C. (2000). Invariant cone- 898
850 B: Biological Sciences, 355, 1243–1248. excitation ratios may predict transparency. Journal of 899
851 Pelli, D. G. (1997). The VideoToolbox software for visual the Optical Society of America A, Optics, Image 900
852 psychophysics: Transforming numbers into movies. Science, and Vision, 17, 255–264. 901
853 Spatial Vision, 10, 437–442. Wild, H. M., Butler, S. R., Carden, D., & Kulikowski, J. J. 902
854 Robilotto, R., & Zaidi, Q. (2004). Limits of lightness (1985). Primate cortical area V4 important for color 903
855 identification for real objects under natural viewing constancy but not wavelength discrimination. Nature, 904
856 conditions. Journal of Vision, 4(9):9, 779–797, http:// 313, 133–135. 905
857 www.journalofvision.org/content/4/9/9, doi:10.1167/ Worthey, J. A. (1985). Limitations of color constancy. 906
858 4.9.9. [PubMed] [Article] Journal of the Optical Society of America A, 2, 907
859 Shapley, R., & Enroth-Cugell, C. (1984). Visual adapta- 1014–1026. 908
860 tion and retinal gain controls. Progress in Retinal Worthey, J. A., & Brill, M. H. (1986). Heuristic analysis 909
861 Research, 3, 263–346. of von Kries color constancy. Journal of the Optical 910
862 Smithson, H., & Zaidi, Q. (2004). Colour constancy in Society of America A, 3, 1708–1712. 911
863 context: Roles for local adaptation and levels of Zeki, S. (1983). Colour coding in the cerebral cortex: The 912
864 reference. Journal of Vision, 4(9):3, 693–710, http:// responses of wavelength-selective and colour-coded 913
865 www.journalofvision.org/content/4/9/3, doi:10.1167/ cells in monkey visual cortex to changes in wave- 914
866 4.9.3. [PubMed] [Article] length composition. Neuroscience, 9, 767–781. 915
916
Appendix C

Author Contributions to Experimental

Work

Of the work presented in this thesis, there are three papers either published, submitted or in

preparation. The authorship of this papers is/will be as follows:

Chapter 5:

Goddard, E., Mannion, D., McDonald, J., Solomon, S., & Clifford, C. (Submitted). Colour

preference argues against a dorsal component of human V4.

Chapter 6:

Goddard, E., Mannion, D., McDonald, J., Solomon, S., & Clifford, C. (2010). Combination of

subcortical color channels in human visual cortex. Journal of Vision, 10(5), 25, 1Ð17.

Chapter 7:

Goddard, E., Solomon, S., & Clifford, C. (In Press). Adaptable mechanisms sensitive to sur-

face colour in human vision. Journal of Vision.

210
 Appendix C. Author Contributions to Experimental Work

Erin Goddard (Each experiment)

Study conception
Protocol and stimulus programming
Data collection
Data analysis
Manuscript preparation
Damien J. Mannion (fMRI experiments only)

Protocol programming
Data collection
Manuscript advice
J. Scott McDonald (fMRI experiments only)

Protocol programming
Data collection
Manuscript advice
Samuel G. Solomon (Each experiment)

Study conception
Manuscript advice
Guidance on all aspects of the project
Colin W.G. Clifford (Each experiment)

Study conception
Data collection
Manuscript advice
Guidance on all aspects of the project

211