VISUAL PERCEPTION

TOM N. CORNSWEET
Stanford Research Institute

T E

Academic Press New York and London

 Contents

Preface
XI

I. INTRODUCTION

Information 2

II. THE EXPERIMENT OF HECHT, SCHLAER, AND

PIRENNE

The General Design of the Experiment 7

The State of the Subject —Dark Adaptation 7
Location of the Test Flash in the Visual Field g
Size of the Test Flash—Spatial Summation 13
Duration of the Test Flash —Temporal Summation jg
Color of the Test Flash—The Spectral Sensitivity Curve jg
The Experiment Itself 23
Contents

The Interpretation of Results

Problems

HI. THE PHYSICS OF LIGHT

A Definition of "Seeing" 27
Light Sources 29
Lenses and Refraction 31
The Intensity of an Image 42
Depth of Focus 45
The Stimulus in the Hecht Experiment 46
Collimated Light 53
Sources of imperfection of the Retinal Image 56
Measurements of the Real Retinal Image 60
Problems 67

IV. QUANTAL FLUCTUATIONS

Quantal Fluctuations in the Stimulus 68

The Relationship between Quantal Fluctuation and the
Subject's Variability 72
Sources of Subject Variability 78
Quantal Fluctuations at Suprathreshold Light Levels 81
Problems 88

V. THE ACTION OF LIGHT ON ROD PIGMENTS

Changes in Rhodopsin Molecules in the Light and in Darkness 91

The Characteristics and Perceptual Correlates of State a 96
The Characteristics and Perceptual Correlates of States
b, c, and d 105
Problem 116

VI. THE. EXCITATION OF RODS

The Fundamentals of Neural Activity 118

The Excitation of Retinal Structures as a Consequence of
the Absorption of Quanta 124
Dark Adaptation and Rod Excitation 122
The Early Stage of Dark Adaptation 121

VIL CONES AND CONE PIGMENT

135
Histological Properties of Rods and Cones
Psychophysical Distinctions between Rods and Cones 12
Individual Differences 1
149
The Nature of Cone Pigments
The Kinetics of Cone Pigments 15
Problem 12
vii Contents

VIII. COLOR VISION I —DISCRIMINATIONS AMONG

WAVELENGTH MIXTURES

Color Names 155

Monochromacy 156
Dichromacy 160
Trichromacy 170
Color Blindness 172
Wavelength Mixture Space 175
Color Reproduction for the Dichromat 182
The Color Mixture Space of the Trichromat 187
A Cure for Color-Blindness 194
Problems 198

IX. COLOR VISION II-RETINAL COLOR SYSTEMS

Possible Trichromatic Mechanisms 200

Measurements of the Mechanisms of Human Color Systems 205
Microspectrophotometry of the Human Retina 214
Classes of Cones in the Retina 216
Tetrachromacy 217
Evaluation of the Assumption That All Absorbed Quanta
Produce Identical Effects 219
The Stability of Wavelength Mixture Matches 221
Problem 223

X. COLOR VISION III-THE PERCEPTION OF

COLOR

The Relationship between Perceived Color and the Physical

Stimulus 225
Differences between Hue, Saturation, and Brightness 234
Factors other Than Wavelength That Influence Hue 236
Stimulus Generalization 240
The Physiological Correlates of Perceived Colors 243
Logarithmic Transformations and Approximations
to Them 249
Application of a Nonlinear Transformation to the Perception
of Hue 253
Physiological Measures of Wavelength-Dependent Responses 258

XL THE PSYCHOPHYSIOLOGY OF BRIGHTNESS —I

SPATIAL INTERACTION IN THE VISUAL SYSTEM

Demonstrations that Brightness Is Not a Simple Function of

Intensity 270
Evidence Concerning the Physiological Nature of Spatial
Interaction in the Visual System 284
Lateral Inhibition in the Retinas of Mammals 304
viii Contents

XII. PSYCHOPHYSIOLOGY OF BRIGHTNESS-II

MODULATION TRANSFER FUNCTIONS

Modulation Transfer Functions

Conditions Necessary for Correct Use of the MTF
Human Visual Modulation Transfer Functions
Perceptual Phenomena Related to the Transfer Function 342
Physiological Implications of the Modulation Transfer
Function 354

XIII. BRIGHTNESS AND COLOR CONSTANCY

Is All This Perception? 365

A Physiological Explanation of Brightness Constancy 371
The Limits of Brightness Constancy 374
Hue Contrast and Hue Constancy 380

XIV. TEMPORAL PROPERTIES OF THE VISUAL SYSTEM

Phase 387
The Temporal Modulation Transfer Function 387
Physiological Correlates of Temporal Events 410

XV. STIMULUS GENERALIZATION

The Generalization of Visual Shapes 423

Physiological Evidence for Mammalian Generalization
Mechanisms 428

XVL SPECULATIONS ON "HIGHER PROCESSES"

Why Did Inhibition Evolve? 434

"Higher” Processes 438

Appendix I VISUAL ANGLE 444

Appendix II FILTER TRANSMISSION VERSUS DENSITY 446

Appendix III HOW TO BUILD AN

OPHTHALMOSCOPE
Theory of Operation
Specific Construction Details

Appendix IV DEMONSTRATION OF COLOR

CONTRAST (COLORED SHADOWS) 4
ix Contents

REFERENCES 455

SUGGESTED GENERAL READINGS 462

Author Index
465
Subject Index
469
Preface

This book grew out of courses that I have taught at Yale University
and the University of California, Berkeley. It is aimed at bright under­
graduate and graduate students, regardless of their academic back­
grounds. Although the text leans heavily upon physical and phys­
iological concepts, I have tried to include explanations of the relevant
physics and physiology, so that readers with limited backgrounds in
these areas will not be handicapped.
I have covered what I believe are the fundamental topics under­
lying the entire, broad field of visual perception. This material can
serve both as a factual background for further topics in perception,
and also, I hope, as a set of paradigms for approaches to additional
topics in vision and other sensory modalities. The areas discussed
most extensively are those related to the perception of brightness and
color. These topics are considered in depth, and I have tried to discuss
xii Preface

both aspects of perception that are well understood and problems that
have not yet been solved.
I consider that a perceptual phenomenon has a scientific explana­
tion if it can be shown to be a particular instance of a more general
perceptual property or law, or if its physiological correlates are
understood. In this light, I have restricted the coverage to only two
kinds of topics, those for which there is a widely accepted explanation
at the present time, and those for which I can imagine one or more
plausible explanations that may not yet have been adequately tested.
Thus, I have excluded many topics (for example, the influence of
motivation upon perception) because my own capacity for inventing
coherent and complete explanations is simply not sufficient to handle
them, and others because I do not know enough about them to explain
them plausibly (for example, the perception of movement).
Because each section of this book is built upon the material that
has preceded it, the text should be read in the order presented. I
would also urge the reader not to skip any of the material; if he is
already familiar with a given topic (for example, the physics of light,
Chapter III), he should skim those pages, not skip them, since there
may well be material there that is new to him (for example, the ma­
terial on the optics of the eye in Chapter III).
This is not a reference work; rather, it is aimed at developing an
understanding of visual perception. For this reason, the number of
references is limited. I have tried to select those references on each
topic that seemed most likely to lead the interested reader to a more
complete listing of the relevant literature. (Many of the references
that are included are given in the figure captions.) There is a listing
of some secondary sources at the end of the book that provide more
complete documentation.
There are brief problem sets following some chapters. These prob­
lems are an integral part of the text, in that many of them require
the reader to consider aspects of the topics that are not directly
covered in the text itself. I have furnished answers to only a small
proportion of the problems, because I have found that easy access to
answers often short-circuits even the most earnest student's thinking.
I OOO INTRODUCTION

AT every instant, the amount of information available to us is immea­

surably great. Electromagnetic radiation of all wavelengths and com­
binations of wavelengths —radio waves, light waves, and X rays —
shower us all the time. Our environment is saturated with sounds,
changes in air pressure and temperature, and changes in the chemical
composition of the air. No real system, physical or biological, could
possibly register and make use of all of this vast array of combinations
of the physical conditions in the environment. The information that
is actually registered or acted upon by a real organism is always a
very small fraction of that which could be used by an imaginary per­
fect system. All organisms select part of the information in their en­
vironment to register or act upon, and the rest of that information is
lost to them. The factors that determine the way in which this selec­
 2 Introduction

tion is performed are properties of the organism itself. They are the
ways in which the organism interacts with the physical properties
of its environment.
In this book, the nature of our perceptual system will be discussed in
terms of the kinds of information that we are able to assimilate from the
world. This selection of information is governed by the structure of our
receptors and the neural circuits that are connected to them. The topic
defined by these considerations consists of the relationships between
the physical variables in the environment and the physiological prop­
erties of the sensory systems of an organism. Our knowledge of these
relationships is based in part on physiological studies, but it is also
based in large part on a certain class of perceptual studies in which a
subject is presented with stimuli, and in certain carefully prescribed
ways, is asked what he sees. This book will discuss data gathered both
in physiological and perceptual experiments.
The study of the relationship between information assimilation and
the physiology of the visual system is only a part of the topic called
visual perception. We use language and other symbols to refer to our
experiences. We say that a light is bright or red. The relationships be­
tween these symbols and their corresponding physical and physiologi­
cal variables are also in the domain of perception, but, while they are
closely related to information processing, they must be treated sepa­
rately, and will be so treated here.

INFORMATION The term information will be used in its common-sense meaning in

this book, and it is important to clarify just what the common-sense
meaning of that term is. If you acquire some information, that means
that you know something that you did not know before. For example,
suppose that a red bulb is on. (It could be on or off.) If your eyes are
built in such a way that they can interact with the energy sent out by
the bulb, and if your eyes are open and pointed at the bulb, and if there
is nothing opaque between you and it, you will be able to acquire the
bit of information that the light is on (instead of the alternative, off).
This particular bit of information thus would allow you to make the
correct choice between two alternatives, light on and light off. You
will also acquire many other bits of information related to the color,
the shape, etc. of the bulb.
In general, visual information is transmitted to us by light energy, or
more accurately, by differences in light energy (in this example, the
difference between the energy when the bulb is on versus that when it
3 Information

is off), but it is useful to distinguish between the energy and the infor­
mation itself. For example, you might acquire the information that the
bulb is on even if you are blind. Someone else might tell you. In that
case, the information was carried to you by sound energy, but it is ex­
actly the same information. A more relevant example is this: When the
light from the bulb is absorbed by your visual receptors, the energy
that carries the bit of information that the bulb is on (instead of off) is
changed from light to chemical energy, but the bit of information itself
is unchanged.
If the red light is on but the subject's eyes are closed, we may say
that the information is present in the environment, but it is not assimi­
lated by his system. Now suppose his eyes are open. How can we find
out whether or not his system loses the information that the bulb is on?
We must have some means of measuring the transmission of informa­
tion through the subject's system.
We have been assuming that we know whether or not the bulb is on.
Say that we know it is on because we can see it. (You may wish to call
this the definition of the fact that the bulb is on.) If we want to know
whether or not the subject's system is capable of retaining or acting
upon that information (i.e., whether or not he sees this particular kind
of light), we must engage in a somewhat complicated procedure. Sup­
pose we just ask him if he sees the light, and he says "yes." That re­
sponse tells us nothing positive about his response to the light. It only
tells us about his response to the question. He might be lying, or he
might not really understand the question. (For instance, if he had been
blind from birth, he might well use the word "see" to refer to some­
thing when he is able to imagine it.) To determine whether or not his
system really retains the information about the light, we must give him
a series of trials such that, in some of them the bulb is on and in the
remainder the bulb is off, recording his response to each trial. We then
look at the correlation between the presence or absence of the light
and his responses, and if that correlation is great enough, we can con­
clude that his system does not lose the information carried by the rays
from the bulb. (There are obviously several restrictions we must im­
pose on the procedure, making sure that no information other than the
light itself is available to the subject. For instance, the bulb should
not make a noise audible to the subject.)
The measurement just described tells us whether or not the subject's
system in its entirety, including his eyes, brain, vocal cords, toes, etc.,
is capable of retaining and acting upon information about the presence
or absence of the light. If we want to be more specific about which
parts of his system are involved in the process, we must do additional
4 Introduction

experiments. One class of such experiments involves cutting off-things

like toes. If he can demonstrate that he still retains the information after
the removal of his toes, we know that toes are not necessary. If the
removed toes themselves respond in a way correlated with the pres­
ence or absence of the light, we can conclude that they are sufficient
for the process.
A second class of experiments involves the measurement of re­
sponses within the system. For example, we might place electrodes on
the nerves leading from the eyes to the brain, and determine whether
or not the electrical responses recorded there correlate with the pres­
ence or absence of the light. If there is a correlation, then we may say
that the information is present at that point in the nervous system.
The discussion above has been quite general, and, in some ways,
loose. It is intended primarily as an introduction to the use of the word
information in the context of perceptual studies, because this concept
will be used extensively throughout the remainder of the book. The
collection of phenomena that we call visual perception will be dis­
cussed in terms of information processing, and the physiological corre­
lates of this processing will be analyzed where it is possible to do so.
The likelihood that an animal will survive depends upon his ability
to use information from his environment. That is, he must be able to
sense the occurrence of various conditions (the lion in the grass) and
also to respond appropriately (faint). The ideal animal would be able
to sense every aspect of the infinite array of information present in the
environment and respond appropriately. However, there is obviously
no advantage in being able to sense information for which the appro­
priate response is impossible, or for which there is no useful response.
Furthermore, there is no survival value in having the capability for
sensing some form of information that is never present in the environ­
ment. If we assume that the processes of evolution result in the selec­
tion of mechanisms advantageous for survival and the elimination of
useless mechanisms, then it follows that the range of stimuli to which
our sensory systems respond will be limited by the nature of the inter­
action between the environment and our response capabilities. In
other words, our sensory systems select and transmit, from the infinite
array of information impinging upon us, those aspects of the informa­
tion that are useful, and lose the remaining information.
Our sensory capacities are not only limited by the factors just dis­
cussed, however. Each of us is of a finite size, and evolution still has a
long way to go. Thus, there are many kinds of information that would
be useful, but we are simply not big enough to contain the systems
needed for detecting them or responding to them. (While we wait for
4 Introduction

experiments. One class of such experiments involves cutting off-things

like toes. If he can demonstrate that he still retains the information after
the removal of his toes, we know that toes are not necessary. If the
removed toes themselves respond in a way correlated with the pres­
ence or absence of the light, we can conclude that they are sufficient
for the process.
A second class of experiments involves the measurement of re­
sponses within the system. For example, we might place electrodes on
the nerves leading from the eyes to the brain, and determine whether
or not the electrical responses recorded there correlate with the pres­
ence or absence of the light. If there is a correlation, then we may say
that the information is present at that point in the nervous system.
The discussion above has been quite general, and, in some ways,
loose. It is intended primarily as an introduction to the use of the word
information in the context of perceptual studies, because this concept
will be used extensively throughout the remainder of the book. The
collection of phenomena that we call visual perception will be dis­
cussed in terms of information processing, and the physiological corre­
lates of this processing will be analyzed where it is possible to do so.
The likelihood that an animal will survive depends upon his ability
to use information from his environment. That is, he must be able to
sense the occurrence of various conditions (the lion in the grass) and
also to respond appropriately (faint). The ideal animal would be able
to sense every aspect of the infinite array of information present in the
environment and respond appropriately. However, there is obviously
no advantage in being able to sense information for which the appro­
priate response is impossible, or for which there is no useful response.
Furthermore, there is no survival value in having the capability for
sensing some form of information that is never present in the environ­
ment. If we assume that the processes of evolution result in the selec­
tion of mechanisms advantageous for survival and the elimination of
useless mechanisms, then it follows that the range of stimuli to which
our sensory systems respond will be limited by the nature of the inter­
action between the environment and our response capabilities. In
other words, our sensory systems select and transmit, from the infinite
array of information impinging upon us, those aspects of the informa­
tion that are useful, and lose the remaining information.
Our sensory capacities are not only limited by the factors just dis­
cussed, however. Each of us is of a finite size, and evolution still has a
long way to go. Thus, there are many kinds of information that would
be useful, but we are simply not big enough to contain the systems
needed for detecting them or responding to them. (While we wait for
5 Information

evolution to provide us with ionizing radiation receptors, we build

devices like Geiger counters. The results of this kind of technology
may be considered either to hinder evolution or to be a part of it.)
There appears to be a very real order to the kinds of selections of infor­
mation that have been forced upon our sensory systems by this size
limitation, and we are making rapid progress in understanding the na­
ture of these selections and the mechanisms that make them. That is
the heart of the content of this book.