Ethovision: A Versatile Video Tracking System For Automation of Behavioral Experiments

Behavior Research Methods, Instruments, & Computers
2001, 33 (3), 398-414
EthoVision: A versatile video tracking system

for automation of behavioral experiments
LUCAS P. J. J. NOLDUS, ANDREW J. SPINK, and RUUD A. J. TEGELENBOSCH
Noldus Information Technology, Wageningen, The Netherlands
The need for automating behavioral observations and the evolution of systems developed for that
purpose is outlined. Video tracking systems enable researchers to study behavior in a reliable and con-
sistent way and over longer time periods than if they were using manual recording. To overcome limi-
tations of currently available systems, we have designed EthoVision, an integrated system for auto-
matic recording of activity, movement, and interactions of animals. The EthoVision software is
presented, highlighting some key features that separate EthoVision from other systems: easy file man-
agement, independent variable definition, flexible arena and zone design, several methods of data ac-
quisition allowing identification and tracking of multiple animals in multiple arenas, and tools for vi-
sualization of the tracks and calculation of a range of analysis parameters. A review of studies using
EthoVision is presented, demonstrating the system’s use in a wide variety of applications. Possible fu-
ture directions for development are discussed.
Manual Versus Automated tative measurements of the animals’ behavior (for more
Behavioral Observation details about how this works, see below). Automated ob-
The behavior of animals is commonly recorded in ei- servation using video tracking is particularly suitable for
ther a manual or a semiautomated way. Traditionally, a measuring locomotor behavior, expressed as spatial mea-
researcher observes the animal; if the researcher consid- surements (distance, speed, turning, etc.) that the human
ers that a certain behavior pattern is displayed, he or she observer is unable to accurately estimate (Bure sÏová, Bol-
notes the behavior, either by writing it down or by entering huis, & BureÏs, 1986; Spruijt, Buma, van Lochem, & Rous-
the data into an event-recording program (Noldus, 1991; seau, 1998; Spruijt, Pitsikas, Algeri, & Gispen, 1990).
Noldus, Trienes, Hendriksen, Jansen, & Jansen, 2000). Automated observation systems also allow the study of
Manual recording of behavior can be implemented with behaviors that occur briefly and are then interspersed
a relatively low investment, and, for some behaviors, it with long periods of inaction (Martin, Prescott, & Zhu,
may be the only way to detect and record their occur- 1992) and behaviors that occur over many hours, such as
rence; however, automated observation can provide sig- diurnal variation in behavior (Olivo & Thompson, 1988;
nificant advantages. Behaviors are recorded more reli- Spruijt & Gispen, 1983).
ably because the computer algorithm always works in the
same way, and the system does not suffer from observer Historical Development of
fatigue or drift. For instance, in contrast to manual ob- Automated Observation
servation, video tracking carries out pattern analysis on Technology for automated recording of animal behav-
a video image of the observed animals to extract quanti- ior and movement has evolved dramatically in the past
decade. Early systems, using hard-wired electronics, were
able to track only a single animal in highly artificial en-
vironments (i.e., test arenas devoid of any substrate, bed-
Many colleagues at Noldus Information Technology have contrib-
uted toward the development of EthoVision over the years; their hard
ding, or objects besides the animal). For example, an open
work and dedication are acknowledged here. Special thanks to Berry field can be sampled with a grid of infrared beams either
Spruijt (Utrecht University), who has been an ongoing source of inspi- as the sole detectors (Clarke, Smith, & Justesen, 1985;
ration. We are also grateful to those users who provided us with their Kirkpatrick, Schneider, & Pavloski, 1991; Robles, 1990)
research design and data for use in the review section of this paper. or in combination with other methods, such as placing a
We also thank Niels Cadee, Martine Janzen, Coen van Kaam, Tiffany
Mayton-TheCrow, and Rudie Trienes for reviewing an earlier version series of strain gauge transducers under the arena to es-
of the manuscript and providing useful comments. Correspondence timate the animal’s position (Gapenne, Simon, & Lannou,
should be addressed to L. P. J. J. Noldus, Noldus Information Technol- 1990). Various ways have been used to measure the mag-
ogy b.v., P.O. Box 268, 6700 AG Wageningen, The Netherlands (e-mail: nitude of an animal’s motion with types of touch-sensitive
[email protected]; URL: http://www.noldus.com).
sensors. A crude estimate of movement can be gained by
Note: The authors have a commercial interest in the product described placing the animal on a bass loudspeaker and monitoring
in this paper. the loudspeaker’s electrical output when its cone is moved
Copyright 2001 Psychonomic Society, Inc. 398

ETHOVISION VIDEO TRACKING SYSTEM 399
by the rat (Silverman, Chang, & Russell, 1988). Sensors converter to enable real-time conversion of the entire
can also measure the position of the animal (and, hence, video image to a high-resolution grid of pixels. It is also
locomotion), for instance, by changes in the capacitance possible to acquire digitized video images by first con-
of a plate when the animal is in proximity to it (Clarke, verting the video input to a digital video format, such as
Smith, & Justesen, 1992) or changes in body resistance AVI, and then using the AVI file as the input for object
(Tarpy & Murcek, 1984). Other comparable detection detection (Derry & Elliot, 1997). However, this method
methods have included use of ultrasound (Akaka & has two disadvantages: (1) The AVI file quickly gets
Houck, 1980) and microwave Doppler radar (Martin & very large (and so only trials of a limited duration can be
Unwin, 1980). A modern radar-based “actometer” is carried out), and (2) the method does not allow for real-
able to detect very small movements (which is particu- time live data acquisition. Alternatively, a digital overlay
larly important in studies on insect behavior) and has the board can also be used to obtain positional data of
advantage of working in complete darkness (Knoppien, tracked animals without the need for a frame grabber
van der Pers, & van Delden, 2000). The position of a rat (Hartmann, Assad, Rasnow, & Bower, 2000; Rasnow,
in an open field has been recorded by attaching the rat to Assad, Hartmann, & Bower, 1997).
a computer joystick via a series of rods attached by a col- A number of modern video tracking systems use frame
lar to the rat’s neck (Brodkin & Nash, 1995). Animal be- grabbers to digitize analog video signals. This enables
havior can also be measured using a computerized ver- high-speed data acquisition and, therefore, tracking of
sion of a Skinner box (Skinner, 1938), in which the animals that are moving relatively fast. However, most
subject has to tap a touch-sensitive monitor (Morrison & of these systems have severe limitations. They can track
Brown, 1990; Sahgal & Steckler, 1994). The motion of only one animal in one arena. If they can track multiple
individual limbs can be monitored automatically using objects, they cannot identify them individually. They tend
actigraph sensors, which detect movement by means of to require simple backgrounds (in terms of their gray
a piezoelectric accelerometer (May et al., 1996). An- scale values) and can deal with only a limited range of
other technique for automatic classification of behav- experimental setups. These systems also cannot handle
ioral patterns uses sensors to detect the mechanical vi- color video.
bration of a platform on which a cage with a mouse is
placed (Schlingmann, van de Weerd, Baumans, Remie, THE E THOV ISION SYSTEM
& van Zutphen, 1998).
Development History
Video Tracking The EthoVision video tracking system, which we pre-
Video tracking systems were introduced in the early sent here, was developed to overcome the limitations of
1990s, offering clear advantages of flexibility, spatial the techniques and systems mentioned above. EthoVision
precision, and accuracy over the various hardware devices has been designed as a general-purpose video tracking,
listed above. However, with early systems the actual path movement analysis, and behavior recognition system.
of the animal still had to be entered manually by the ex- On the basis of a high-resolution color video frame grab-
perimenter, by following the track of the animal with a ber and flexible software, it is a versatile image process-
computer mouse (Pereira & Oliveira, 1994), a joystick ing system designed to automate behavioral observation
(Morrel-Samuels & Krauss, 1990), or a digitizing tablet and movement tracking on multiple animals simultane-
or similar device (Santucci, 1995). Another early method, ously against a variety of complex backgrounds.
still used in some commercially available systems, is to Our group has been involved with the development,
feed the analog video signal to a dedicated video track- deployment, and support of EthoVision for almost a dec-
ing unit, which detects peaks in the voltage of the video ade. A predecessor of the current EthoVision video track-
signal (indicating a region of high contrast between the ing system was first developed in the 1980s at the Rudolf
tracked animal and the background), and use this to pro- Magnus Institute for Neurosciences (RMI) of Utrecht
duce the x,y coordinates of the tracked animals. This out- University (Spruijt, Hol, & Rousseau, 1992). In 1992,
put is then fed to the serial port of a computer (Klapdor, Noldus Information Technology and the RMI joined
Dulfer, & van der Staay, 1996; Vorhees, Acuff-Smith, forces in order to develop this prototype into a mature
Minck, & Butcher, 1992). These analog systems have the video tracking system. Over the years, development pro-
disadvantage of being relatively inflexible (dedicated to ceeded in close partnership with two universities (Utrecht
particular experimental setups) and can normally track University and Wageningen University) and three phar-
only one animal in rather restricted lighting and back- maceutical companies (Bayer AG, Germany; H. Lund-
ground conditions. beck A/S, Denmark; Solvay Pharmaceuticals b.v., The
Greater flexibility is achieved by the use of a video Netherlands). The first DOS-based implementation was
digitizer. The first video digitizers were of the column released in 1993. The system has undergone numerous
scan type. These had no internal memory of their own updates over the years, on the basis of feedback from
and were able to sample the video signal only at rather users around the world, which has resulted in a compre-
low rates (Olivo & Thompson, 1988). In contrast, a mod- hensive package for studies of movement and behavior.
ern frame grabber uses a high-speed analog-to-digital The software has recently been redesigned from the
400 NOLDUS, SPINK, AND TEGELENBOSCH
ground up for 32-bit Windows platforms, with a greater animals are (the scene). The analog video signal is digi-
range of features and a highly interactive graphical user tized by a frame grabber and passed on to the computer’s
interface for display of experimental design, experimen- memory. The frame grabber can either digitize the cam-
tal arena, and tracks of movement. The first Windows era’s signal directly or take input from a video cassette
version of EthoVision was released at the end of 1999. recorder. The software then analyzes each frame in order
This paper constitutes the first comprehensive publi- to distinguish the tracked objects from the background,
cation about EthoVision (version 2.2, the latest version on the basis of either their gray scale (brightness) or their
at the time of writing) in the scientific literature. The hue and saturation (color) values. Having detected the
specifications and functionality of the DOS program objects, the software extracts the required features—in
have never been published, which is why many of the the case of EthoVision 2.2, this includes the position of
features already present in that version are given in some the mathematical center of each object (center of grav-
detail here. In the sections below, we will outline the ity) and its surface area. These values are written to a
functionality of the EthoVision software and present an track file on disk. Calculations are carried out on the fea-
overview of studies using EthoVision, to demonstrate tures to produce quantified measurements of the ani-
the system’s use in a wide variety of applications. mals’ behavior. For instance, if the position of an animal
is known for each video frame, and the whole series of
How EthoVision Works frames is analyzed, the average speed of locomotion of
EthoVision is an integrated system, comprising vari- an animal during an experiment or the distance between
ous software and hardware components (Figure 1). A several individually identified animals can be calculated.
CCD video camera records the area in which the subject In addition, if certain regions are identified as being of
Figure 1. Diagram of an EthoVision setup. The analog signal from a CCD camera is fed into a frame grabber inside the computer,
which digitizes the image. EthoVision then analyzes the signal to produce quantitative descriptions of the tracked animal’s behavior.
A video cassette recorder and monitor are optional components.
interest (e.g., the center and edges of an open field), the

proportion of time spent by the animals in those regions
can also be calculated.
E THOV ISION SOFTWARE SPECIFICATIONS
EthoVision may be thought of as consisting of a series

of modules that function as an integrated whole. The
functions of each of the modules are outlined below. The
software has been written in Microsoft Visual C++
(using Visual Studio 6.0 with STL and MFC). The 32-bit
code has been optimized for Windows 98, NT, and 2000.
File Management
Information in EthoVision is organized at several lev-
els. The highest level is called a workspace (i.e., a con-
tainer for one or more experiments). A workspace can be
used to keep together experiments that have a specific
relation (a similar setup, performed by the same experi-
menter, etc.). An experiment embodies a series of trials
carried out with a particular experimental setup. The
data from one animal collected during a trial (the x,y co-
ordinates and body surface) is referred to as a track. In
addition to the data files generated by EthoVision, the
user can create profiles, which contain all the settings
made for a particular function. The EthoVision Workspace
Explorer enables the user to manage all these files through
a simple interface (similar to the Windows Explorer), so
that, for instance, settings defining the acquisition method
(the tracking profile) can be dragged and dropped from
one experiment to another (see Figure 2). An entire ex-
periment or workspace can be backed up to a single file,
to facilitate both safe keeping of the data and transfer be-
tween different computers. Tracks (and their associated
independentvariable values) can be imported into the cur-
rent experiment from another experiment, enabling analy-
sis of data across multiple experiments.
Experiment Design
Independent variables. As well as tracking objects,
EthoVision also allows the researcher to define a com-
plete experimental protocol, in terms of the independent
variables of an experiment and their values. Up to 99 in-
dependent variables (e.g., treatment, room temperature, Figure 2. The EthoVision Workspace Explorer. The workspace
genetic strain, etc.) can be defined. EthoVision also au- “BRMIC” contains three experiments. Experiment “F15, fe-
tomatically records a series of system variables, such as male, cohort 1” is active (shown in bold), and it contains a series
the profile (settings file) used, time and date of the trial, of different profiles, illustrating the different uses to which these
trial duration, and so on. These independent variables can stored settings may be put.
be used to select, sort, and group data, both when plot-
ting tracks and when analyzing the data. For instance,
one can select to plot all the tracks from mice of a par- Scheduling trials. Prior to data acquisition, one can
ticular genotype or calculate the mean time taken to reach predefine the values of the independent variables for a
the target of a water maze by control rats relative to treated series of individual trials one plans to run in an experi-
rats. When the trial is carried out, the user can, for each ment. In this way, the design of the experiment and the
user-defined independent variable, select whether to ac- testing can be done before the actual trials are performed.
cept the values predefined in the experimental protocol EthoVision can thus be used to schedule the trials (i.e.,
or enter new values. In addition, the possible range or a assist the experimenter in applying the correct treatment
series of exact possible values can be defined for each and select the correct animals). During data acquisition,
independent variable (see Figure 3). the values of the independent variables are displayed on
Figure 3. Defining independent variables with EthoVision. The system-independent variables are shaded, and the user-defined vari-
ables have a white background. The independent variable properties are also shown for the selected variable (treatment).
the computer monitor, providing immediate feedback on cumulative zone (e.g., if a maze has more than one tar-
the characteristics of the trial (which animal is currently get). Zones can also be defined as hidden zones, for use
being tested, what the treatment is, etc.). Once the ex- with nest boxes, burrows, and so on. The system assumes
periment design has been entered, it can be used in mul- that when an animal disappears from view and it was last
tiple experiments, thus reducing preparation time for re- seen adjacent to a hidden zone, it is inside the hidden
peated experiments. zone. A hidden zone can also double as a normal zone:
If the animal is visible within the boundaries of the zone,
Arena and Zone Definition EthoVision assumes that it is on top of, for example, the
Arenas. Up to 16 enclosures can be placed under one nest box. All zones can be defined and altered either be-
camera, and each enclosure can be treated as a separate fore or after data acquisition, allowing iterative explor-
independent replicate (called an arena; see Figure 4). atory data analysis of the effects of changing zone posi-
The animals in these arenas can be tracked simultane- tions and shapes. Points of interest can also be defined
ously. The arena can be of any shape, which is easily de- (e.g., the center of an open field or a novel object that is
fined using special drawing tools. Multiple arena pro- placed in a stable).
f iles and zone definitions can be defined for each Calibration. The arena can be calibrated so that pa-
experiment. This is particularly handy when, for in- rameters, such as velocity, are calculated in meaningful
stance, a water maze experiment is prepared with multi- units, such as millimeters or inches, instead of pixels.
ple platform positions. When setting up the experiment, There are two calibration methods. In standard calibra-
the user can assign the proper arena definition to each tion, the user measures a series of lines (e.g., a meter
track file before the experiment is started. If one defines ruler placed on the floor of a cage). If the camera image
an arena in the video image in which object tracking is distorted, the standard calibration will not be accurate
takes place, the system ignores parts that do not belong (the standard deviation of the calibration factors is given
to the defined arena during data acquisition. This re- to enable the user to determine this), and the user can opt
duces the chance that events outside of the experimental for advanced calibration. In advanced calibration, a grid
arena will interfere with the actual measurement. By of points is mapped onto the arena, and a series of coor-
using different drawing options (rectangle, circle, poly- dinate points are entered, giving a calibration that is ac-
gon, curve, line, and freehand), one can quickly draw any curate for the entire image, even when it is distorted. This
shape of experimental setup and enter it into the com- enables the distance-based parameters to be accurately
puter. This means that one can use the system for a water calculated when the arena is in a room with a low ceil-
maze experiment, an open field test, or any other stan- ing, when an exceptionally large arena necessitates the
dard test. Because the signal from the video camera is use of a very wide angle or fish-eye lens, or when the
directly displayed on the computer screen, the arena out- lens is not directly overhead in relation to the arena.
lines can be traced with high accuracy.
Zones and points of interest. The user can define up Data Acquisition
to 99 regions of interest (called zones). These can be Running a trial. After the user has defined the sam-
used in the analysis (e.g., to compute the time spent in ple rate (up to 30 per second) and trial duration, Etho-
different parts of a cage) and in automatic start and stop Vision is ready for data acquisition. During tracking, the
conditions. For instance, the trial can be stopped auto- computer shows a combination of the recorded path, the
matically when a mouse has visited all the arms of an detected object shape, the current arena /zone definition,
eight-arm maze. Zones can be added together to make a and the original video image (see Figure 5). Elapsed time,
Figure 4. An EthoVision arena definition for a water maze experiment. The circular arena (region from which data are acquired;
here labeled “Watermaze”) is divided into four zones (labeled “North,” “East,” “South,” and “West”). The small zone “Platform” de-
fines the location of the platform. The area outside the arena is ignored, leaving data acquisition undisturbed by any movement that
takes place in that part of the scene (e.g., an operator walking by).
number of samples, and various other statistics are also to objects larger than those defined as noise and that
provided in real time. This allows immediate action if a have a subtracted value other than zero constitute the dif-
tracking error occurs. An optional remote control can be ference between the reference and the live images, due to
used to start and stop trials at a distance from the com- the presence of the animal. The user can define whether
puter (e.g., as you stand next to the experimental setup). the animal has to be lighter or darker than the back-
Object detection. To ensure optimal object detection ground or just different. This method tolerates more dif-
in any experimental setup, EthoVision offers three dif- ferences in light intensity across the scene. It thus becomes
ferent object detection methods. Gray scaling is the fast- possible to separate an animal with light skin on a gray-
est. This method defines the animal as all connecting ish background from its (black) shadow. The method is
pixels that are both brighter than a low gray scale thresh- also suitable for animals with a heterogeneous fur pat-
old and darker than a high threshold, and it defines the tern, such as hooded rats, spotted mice, or cows.
background as all other pixels. The thresholds can either The third detection method, color tracking, uses the
be set manually by the user or be calculated automatically color of the animal (or a marker painted on it) to identify
by the program. Gray scaling is a fast detection method and track it. EthoVision uses the hue and saturation com-
(allowing the highest sample rate), but it cannot be used ponents of the hue–saturation-intensity (HSI) color space
if the same gray scale values are present in both the ani- model to track objects (Spink, Buma, & Tegelenbosch,
mal’s image and the background. 2000; see Figure 6). By using both hue and saturation,
The second method, subtraction, first (before the start EthoVision can better distinguish objects that are more
of the trial) makes a reference image, with no animals similar in color to each other than if only using hue (e.g.,
present, and then (during the trial) subtracts the gray objects with the same hue but differing saturation val-
scale value of each pixel of the reference image from the ues; see Figure 7), which is why the system can track as
equivalent pixel of the live image. Any pixels that belong many as 16 different colors in each arena at once (under
Figure 5. Data acquisition in progress during a test of a parasitic wasp searching for hosts in a petri dish. In addition to the live video
image with overlaid zone borders and movement track, EthoVision shows tracking statistics (elapsed time, number of samples, pro-
cessor load, etc.), measurement values of the object (x, y coordinates, surface area), and characteristics of the trial (independent vari-
ables). Image courtesy of J. P. Monge (University of Tours, Tours, France).
suitable conditions). In addition, the use of these two sion can track at low, medium, and high resolution. At
complementary detection descriptors makes the object high resolution (5 768 3 576 pixels), EthoVision can
identification more robust, so that, for instance, objects track an animal in an arena with a variable shape and up
can be tracked more reliably if the light intensity (bright- to 200 times the animal’s size. By comparison, an in-
ness) is uneven across the arena. frared detector system typically operates in an arena of
With all three methods, objects that are either smaller 0.5 3 0.5 m with 12–32 beams (though systems have
than the animal (e.g., droppings) or larger than the ani- been developed with a grid of 24 3 24 beams, in an arena
mal (e.g., reflections) can be excluded on the basis of 2 m in diameter; e.g., Robles, 1990). Other options in-
their size. clude the possibility to manually adjust contrast, bright-
Object identification. EthoVision distinguishes be- ness, hue, and saturation, in order to optimize digitiza-
tween two animals in the same arena on the basis of size. tion of the video signal. This is useful when analyzing
To distinguish two animals that are the same size, the trials that were prerecorded on videotape. Furthermore,
user can partly color one animal to the same grayness as one can scan the complete arena or use a moving scan
the background, thus reducing its apparent size (see Fig- window. With the latter option, only a limited area around
ure 8). Color tracking can be used to distinguish more the object’s last position is searched. As a result, track-
than two animals (up to 16 per arena). If the animals being ing is not disturbed by occasional moving objects else-
tracked are the same color, markers can be used to paint where in the arena (e.g., reflections). When an animal is
blobs on the animals, which the system will be able to temporarily out of view, EthoVision can be set to resume
track. For more details on color tracking, see the “Object tracking by automatically repositioning the scan window.
Detection” section above. Image filtering. EthoVision calculates the mathemat-
Image resolution. The user can further fine-tune data ical point at the center of the digitized picture of the an-
acquisition by modifying the image resolution. EthoVi- imal’s body, including the tail (e.g., in a rat or mouse).
Selection and Visualization of Track Data

The user can specify data sets at several hierarchical
levels for track analysis and visualization. The data can
be selected, sorted, and grouped by independent vari-
ables. In addition, the track files can be split up (nested)
by time. Thus, a selection of the tracks in an experiment
can be plotted on a single screen—for instance, in a ma-
trix of treatment level 3 genetic strain. Other variables,
such as gender or day of the week, can be shown in dif-
ferent colors and line styles. Furthermore, time windows
can be applied (e.g., just the first 60 sec of the track).
The track plots can be displayed with or without the arena
and zones or the background image of the experimental
setup. The plots can be exported as graphic files in a va-
riety of standard formats, ready for inclusion in presen-
tations, reports, or research articles. Figure 10 shows a
typical EthoVision visualization of tracks.
Figure 6. The HSI color space model. The horizontal distance

from the center of a horizontal plane to its outside represents a
color’s saturation (fullness), the angle on that plane represents its
hue (color), and the vertical distance (z-axis) represents its inten-
sity (brightness).
This means that this point is farther toward the animal’s

posterior than is correct. Furthermore, the system will
still detect movement of the animal when only the tail is
moving. In order to prevent this, the tail can be removed
from the image using image filtering (erosion). The bars
of a cage can also be filtered out using a similar tech-
nique (dilation). These filters also help to avoid acciden-
tal mix-up of the animal with background noise and to im-
prove the computation of body-shape-related parameters.
Recording specific behaviors. In addition to auto-
matically tracking the movement of the animal, EthoVi-
sion can automatically detect certain behaviors, such as
rearing and moving (see the Appendix for details). To
allow detection of rearing behavior of rodents using a
single overhead camera, EthoVision stores the surface
area of the animal’s body for each sample taken. Etho-
Vision also allows the researcher to manually record (by
keystroke or mouse click) behaviors that cannot be de-
tected automatically (e.g., sniffing).
Experiment control. EthoVision also has a facility
for automatic experiment control (see Figure 9). This
can be used to start or stop data acquisition depending on
the location of the animal. For instance, the system can
be set to start a trial when a rat enters a water maze and
stop it when the rat reaches the platform or to stop the Figure 7. Object identification based on color. Each shape on
trial after the animal has been in a user-defined zone for the disk (which is displayed in color on screen) represents a vol-
more than a user-defined length of time. One can also let ume in HSI color space, and the user can drag the shape so that
the system perform series of trials with user-defined num- it includes the range of hues and saturations present in the ani-
mal or marker being tracked. When the color defined here matches
ber of trials and intertrial interval (e.g., 72 consecutive the color of the animal or marker, the software displays those pix-
trials, each with 5-min duration and 55-min intervals in be- els in the color selected in the “Show color” column, thus giving
tween), which is useful for studies of behavioral rhythms. visual feedback as to when the object is correctly identified.
Figure 8. To distinguish two rats that are the same size, one has been partly colored black. After ap-
plying a gray level threshold, only the bright parts of the body remain, thus creating an apparent size
difference.
EthoVision can replay the recorded tracks on the party programs, such as Microsoft Excel, SAS, SPSS,
screen, allowing a detailed and interactive visual analy- The Observer (Noldus et al., 2000), WinTrack (Lipp &
sis of the data. One can choose for playback to be either Wolfer, 2000; Wolfer & Lipp, 1992), or SEE (Drai, Ben-
at the original speed or at a user-defined speed and set jamini, & Golani, 2000). More detailed information
the number of data points that are displayed simultane- about these parameters is given in the Appendix.
ously. One can also move through the tracks, showing
each time only a selection of data points. REVIEW OF STUDIES USING E THOV ISION
Data Analysis In contrast to some other video analysis systems (e.g.,

Data selection for analysis is carried out in the same Mukhina, Bachurin, Lermontova, & Zefirov, 2001; Pan,
way as for data visualization, and, in fact, the same data Lee, & Lim, 1996; Spooner, Thomson, Hall, Morris, &
profile is used so that the data selection made for visu- Salter, 1994), EthoVision is a generic tool that can be
alization is automatically available for analysis (and vice used in a wide variety of setups and applications. There
versa). The user can group tracks by the values of inde- are, of course, practical limitations; however, in princi-
pendent variables (e.g., the mean velocity can be calcu- ple, EthoVision can be used to track animals in any sit-
lated for all animals that received a particular treatment) uation where the animals are within sight of a camera
and further fine-tune the data using filter settings (e.g., and in a delimited and constant area that is no more than
minimal distance moved) to eliminate slight “apparent” 200 times the size of the animal. The majority of studies
movements and using down-sampling steps to eliminate using EthoVision are carried out using small rodents, but
redundant data if a lower sample rate describes the path the system is also used in research on insects, fish, birds,
better. A wide range of quantitative measures of behav- primates, farm animals, and other mammals. The review
ior is available in EthoVision, including parameters for below gives an overview of the main types of research
location and time, path shape, individualbehavioral states, using EthoVision; however, it is by no means an exhaus-
and social interactions, and these can be described with tive list of all current uses of EthoVision.
a full range of descriptive statistics. The parameters, sta-
tistics, and raw data (i.e., series of x,y coordinates and Neuroscience and Pharmacology
other extracted image features for each individual as a The most common application of EthoVision is as a
function of time) can be exported in a variety of formats tool for measuring the behavior of rats and mice that
for further analysis and production of graphs in third- have had their brain chemistry altered by application of
Figure 9. The Experiment Control dialog, with three conditions and two actions. The prop-
erties of the third condition are shown in detail.
drugs or neurochemicals (e.g., Ploeger, Spruijt, & Cools, cessing and spatial memory. EthoVision can be used to
1994; Sams-Dodd, 1995), surgery (e.g., Eijkenboom & define a zone where the platform is and measure the time
van der Staay, 1998; van Rijzingen, Gispen, & Spruijt, taken for the rat or mouse to reach the zone. It can also
1995), or the use of a particular genotype (e.g., Mini- be used to assess the reason an animal might “fail” the
chiello et al., 1999; Spink, Tegelenbosch, Buma, & Nol- test. For instance, some strains of mice hug the walls when
dus, 2001). Research of this type often combines the stressed (thigmotaxis) and, therefore, never find the plat-
above techniques (e.g., testing whether injecting a min- form. This can be an appropriate behavioral response in
eralocorticoid receptor antagonist can counter the effects the animals’ natural environment, and it is important not
of a lesion; Oitzl, Josephy, & Spruijt, 1993). The brain to confuse this with failing to learn where the platform
functioning and behavior of rats and mice are often used is because of an error in its learning mechanism (Gass
as models of human brain functioning and behavior in et al., 1998; Lipp & Wolfer, 2000). Figure 10 shows the
the study, for instance, of the effects of addiction (Chow, tracks from a Morris water maze experiment.
Tomkins, & Sellers, 1996; Miczek, Nikulina, Kream, With an open field test, the researcher simply places
Carter, & Espejo, 1999) or sleep deprivation (Meerlo, one or more animals in an open space (usually a square
Overkamp, Benning, Koolhaas, & van den Hoofdakker, enclosure with the side about 5–10 times as long as the
1996). The majority of neuropharmacological studies length of the animal’s body). As an animal becomes less
use either the Morris water maze to test spatial learning anxious, it spends less time by the walls and more time
(Morris, 1984; see, e.g., Dalm, Grootendorst, de Kloet, in the open (e.g., Meerlo et al., 1996). Novel objects can
& Oitzl, 2000; Lipp & Wolfer, 2000; see Figures 4 and be introduced into the field, and the animal will investi-
10) or the open field test to study anxiety and/or social gate the object, spending a variable amount of time by
interaction (e.g., Spruijt et al., 1992; van den Berg, Spruijt, the object, depending on the animal’s state (Whishaw,
& van Ree, 1996). These tests are ideally suited to auto- Haun, & Kolb, 1999). One of the strengths of EthoVi-
matic data acquisition and analysis with EthoVision; they sion is that the arena can be divided up into a large num-
take place in controlled conditions in a simple defined ber of zones, of any size and shape, so that the positions
area, and the resulting behavior can be readily quantified. of objects in the field and areas around the object can be
With the Morris water maze, the researcher tests spa- accurately defined and the positions and movements of
tial memory by measuring how long a rat or mouse takes the animal can be measured in relation to those zones. In
to swim to a hidden platform. When it reaches the plat- addition, the zones can be defined after data acquisition
form, it can stop swimming. The animal is first trained, has occurred, so that if areas of interest become apparent
and it normally relies on visual clues to find the platform only after an initial analysis, these can be investigated
again. The test thus usually assesses both visual pro- further. Moving novel objects can also be placed in the
Figure 10. Visualization of an EthoVision water maze experiment (data from Minichiello et al., 1999). The Workspace Explorer can
be seen on the left. In the main window, eight tracks are plotted, sorted by trial number (1–4) and genetic strain. The “Play Console”
(with VCR-type controls) and “Play Options” dialog box are also shown.
open field, and these can be tracked in the same way that 2000). Healing et al. (1997) concluded that EthoVision
the animals are tracked (Spruijt et al., 1992). The way can be used to reliably assess changes in motor activity
that the animals move in relation to the other objects can to a standard acceptable to regulatory authorities.
then be quantified by EthoVision, and this can be inter-
preted in terms of approach, avoidance, and so on. Entomology
Although EthoVision is most commonly used to track
Toxicology rats and mice, it can also be used successfully to track in-
A standard assessment for the effects (or lack of ef- sects and arachnids. For example, Bouman, Simek,
fects) of toxins and suspected toxins is the measurement Zemek, Dusbabek, and Zahradnickova (1999) looked at
of any changes in behavior under standard conditions the effect of female status on behavioral interactions be-
(Moser et al., 1997). EthoVision is used by a number of tween male and female Ixodes ricinus ticks, whose walk-
groups for these measurements. For example, Hany et al. ing pattern was tracked and analyzed by EthoVision. The
(1999) looked at the behavioral effects on rat pups of ex- latency to contact between male and female ticks and
posing their mothers to PCBs during the gestation pe- speed and duration of movement of males toward fe-
riod. The behavior of the rats in an open field was mea- males were interpreted as indicators of female attrac-
sured using EthoVision. The open field was divided into tiveness over short distances, and the mean distance be-
inner and outer zones, and it was shown that rats exposed tween the sexes was interpreted as an index of the overall
to PCBs avoided the outer zone. Because there was no attractiveness of the female. Bouman et al. found that the
difference in the total distance moved between treat- males did not walk randomly but were attracted to the
ments, this was interpreted as a change in the emotional females and that the males were more attracted to the fe-
state of the rats, rather than an impairment of motor males that were engorged on guinea-pig blood.
function. Weinand-Härer, Lilienthal, Winterhoff, and Kröber and Guerin (1999) also studied ticks with
Winneke (1996) also used EthoVision to analyze the ef- EthoVision. They analyzed the path moved by Boophilus
fects of PCBs on rat pups’ performance in a Morris water microplus and Ixodes ricinus in relation to a patch of
maze. Another typical toxicological application is re- water. In general, as soon as one of the tick’s legs touched
search on the behavioral effects of nicotine (Kim et al., the water, the tick rotated and walked away from the
water. However, if the ticks were dehydrated, they did one). Finally, the angle of the two bodies and point of
not avoid the water. It was hypothesized that ticks have contact were calculated.
to strike a balance between their need for water vapor to
maintain water balance and the danger of freezing when Fish Behavior
coming into contact with ice crystals in the winter. One of the first uses of an early predecessor of the cur-
Parasitic wasps such as Encarsia formosa are of great rent EthoVision program was to study locomotor activity
commercial value in biological control of greenhouse in Arctic char (Winberg, Nilsson, Spruijt, & Höglund,
pests. A video tracking system such as EthoVision can 1993). This species is still being studied with EthoVision.
be a useful tool in selecting species suitable under given The char are placed in a fluvarium, the upstream end of
conditions for controlling certain pests. For example, which is divided into two halves. Each half receives water
Drost, Qiu, Posthuma-Doodeman, and van Lenteren from a separate source. Every 90 min, the tanks are
(2000) measured the velocity and turning rate of parisi- switched, and the fish are tracked for the latter half of
toids of Bemisia argentifolia whitefly, in order to char- each 90-min period. The tanks are backlit with light with
acterize the searching pattern and to identify the best a wavelength of less than 750 nm, which is detectable by
natural enemy. The wasps were tracked in a petri dish lit a monochrome camera but not by the fish. The analysis
from below, which gave a good high-contrast image. The is based on the frequency that the fish are in each half of
edge of the leaf disk was defined as a zone and was ex- the fluvarium (which is defined as an EthoVision zone),
cluded from analysis by nesting (because the boundary which is in effect a Y-maze. Bjerselius, Olsén, and Zheng
causes unnatural 180º turns). The data will be used to (1995a, 1995b) found that male fish avoided the half of
create a simulation model of host-searching behavior for the fluvarium that had elevated levels of the female hor-
each of the species studied. Krips, Kleijn, Wilems, Gols, mone indicating that the female was not spawning. If, in-
and Dicke (1999) carried out a similar study to deter- stead of a semiochemical being added, the two halves of
mine the effects of leaf hair density on the searching ef- a fluvarium were attached to tanks containing other char,
ficiency of Phytoseiulus persimilis (a predatory mite used the char in the fluvarium had a different frequency of be-
for biological control of spider mites); although a species ing in the two halves of the fluvarium, depending on
may have been successfully used to control the pest on a whether the water came from fish that were siblings (Ol-
plant with smooth leaves, it may be quite unable to cope sén & Winberg, 1996) and whether the siblings had an
with leaves with dense hairs. Movement parameters have identical major histocompatability complex (Olsén,
the potential of becoming a standard measurement in the Grahn, Lohm, & Langefors, 1998).
quality control of mass-reared insects (van Schelt, Buma, Ylieff, Sanchez-Colero, Poncin, Voss, and Ruwet (2000)
Moskal, Smit, & van Lenteren, 1995). EthoVision has also were able to use EthoVision to track damsel fish (Chromis
been used in studies of behavioral aging in Drosophila chromis) marked with colored pearls to quantify vari-
(Le Bourg & Minois, 1999). ables that cannot be measured accurately by direct ob-
servation methods. Having determined that the colored
Animal Welfare marker did not affect velocity, distance moved, or time in
EthoVision has been used to study stress effects on a various zones, they investigated the effects of water tem-
variety of species, including captive animals such as farm perature and fish density on the behavior of the fish. They
Ï
animals (Bokkers & Koene, 2000; Sustr, SÏ pinka, & New- found that, at high temperatures, the f ish swam faster
berry, 2000), laboratory animals (Ruis et al., 1999), and when they were near the surface, which was hypothesized
zoo animals (Fuchs, Kramer, Hermes, Netter, & Hiemke, to be an adaptation to escape from predatory birds.
1996; van Kampen, Schmitt, Hiemke, & Fuchs, 2000). Van Ginneken et al. (1997) used EthoVision to com-
For example, battery hens were tracked with EthoVision bine calorimetry of tilapia with analysis of activity. In
as they walked toward food. The latency to pecking the this way, the effects of various movements on the in-
food and the speed of walking toward the food were mea- creased metabolic rate of the fish under various light and
sured and used to assess the animals’ motivation and oxygen conditions could be analyzed.
ability to walk (Bokkers & Koene, 2000).
Ï
Sustr et al. (2000) marked the front and rear of pigs DISCUSSIO N
with different colors and tracked these with EthoVision.
They were then able to use the EthoVision data to ana- As can be seen from the “Review” section of this paper,
lyze pig play and pig fights in terms of mutual spatial one of the strengths of EthoVision is that it is flexible
position and orientation. First, the data were corrected enough to be used in a wide variety of experimental set-
for samples missed when, for example, the head was ups and applicationsand with a wide-ranging assortment
dipped down and out of sight of the camera. Then, the of species. Some other systems have been specifically
actual positions of the heads and snouts were calculated, designed for a particular experiment, such as the Morris
using the positions of the markers on the pigs and the water maze (e.g., Mukhina et al., 2001; Spooner et al.,
known distances from the markers to the heads and 1994), whereas EthoVision can work with any shape of
snouts. These positions were then used to calculate how arena, with a large variety of backgrounds and lighting
close the pigs were to each other and which pig was “ac- conditions. The ability of EthoVision to track multiple
tive” (i.e., its snout was nearer to the body of the other animals in an arena means not only that social interac-
tions can be studied (Rousseau, Spruijt, & Gispen, 1996; tical analysis of variables derived from both EthoVision
Sams-Dodd, 1995; Sgoifo, Kole, Buwalka, de Boer, & track files and various other data, it is necessary for the
Koolhaas, 1998; Spruijt et al., 1992) but that separate various data streams to be synchronized. At the time of
parts of the animals can be marked and tracked indepen- this writing, we are carrying out a research project, to-
Ï
dently (Sustr et al., 2000). The sophisticated range of data gether with several partners, to resolve this issue (Hooge-
selection options and the wide array of variables that boom, 2000).
EthoVision can calculate to quantify the animals’ be- Automatic detection of complex behaviors. In its
havior (parameters) mean that the user is able to use the current implementation, EthoVision is able to detect
system to produce accurate descriptors of complex be- whether an animal is moving or whether a rodent is rear-
haviors in diverse situations and is able to gain an over- ing, but (in common with other commercial systems) it
view of the behavior from the visualization of the ani- cannot automatically detect other body postures or be-
mals’ tracks. havioral patterns (unless, of course, it is possible to de-
fine them in terms of the existing parameters). Research
Future Developments on automatic classification of body postures and behav-
Of course, despite the power and flexibility of a sys- ioral acts in digitized video sequences was already in pro-
tem such as EthoVision, there is always room for further gress more than two decades ago (Kernan et al., 1980)
development and improvement. We are currently in- but has not, to our knowledge, found its way into a com-
volved in, and in the past have carried out, a number of mercial product so far. Recently, however, considerable
special projects developing customized versions of Etho- progress has been made with the use of model-based pat-
Vision. Which of these customizations are integrated tern recognition, statistical classification, and neural
into the standard software depends on both the technical networks to automatically detect rodent behaviors, such
feasibility and interest in the application. The following as sitting, grooming, and stretched attend (Rousseau,
aspects are some indications of the way in which the van Lochem, Gispen, & Spruijt, 2000; van Lochem,
EthoVision system might be developed in the future. Buma, Rousseau, & Noldus, 1998). Further refinement of
Support of digital video. At the moment, the Etho- these and other algorithms (e.g., Heeren & Cools, 2000;
Vision software takes its input from an analog video Twining, Taylor, & Courtney, 2001) is necessary before
camera, the signal of which is digitized by a frame grabber. they can be incorporated into the EthoVision system.
A logical development in the future is that EthoVision This development is likely to benefit from recent work
would be able to take direct input from either digital cam- done on automated coding of human facial expressions,
eras or digital video files. using digital image processing, active shape modeling,
Analysis of thermographic images. The EthoVision and neural network techniques for pattern classification
system is designed to measure animals’ behavior and not (Ekman & Friesen, 1978; Lanitis, Taylor, & Cootes, 1997;
the physiological changes that may have influenced that Tian, Kanade, & Cohn, 2001).
behavior. Of course, it is used in many studies with a Tracking large and indeterminate numbers of ani-
physiological aspect (as in many of the studies cited in mals. Tracking large and continually varying numbers
this “Review” section), and physiological data gathered of animals, especially if crowded close together, calls for
at the same time are commonly analyzed together with techniques quite different from the object identification
the behavioral data. A specialized application of Etho- methods used by EthoVision. Some progress has been
Vision can use the system to directly measure a physio- made using techniques such as track segmentation and
logical parameter. It is well known that stress can cause reconstruction (Buma, Moskal, & Liang, 1998) and ac-
changes in body temperature of various homeothermic tive modeling and prediction of the animals’ shapes (Bul-
species, such as rats and mice (Clement, Mills, & Brock- pitt, Boyle, & Forbes, 2000; Sergeant, Boyle, & Forbes,
way, 1989). However, the very act of inserting a rectal 1988).
probe or implanting a biotelemetry device (both are com-
monly used techniques to measure an animal’s body tem- Availability
perature) stresses the animal and raises its temperature EthoVision is commercially available from Noldus In-
(van der Heyden, Zethof, & Olivier, 1997). As an alter- formation Technology and various international distrib-
native, Houx, Buma, and Spruijt (2000) have developed utors. Readers can contact the first author for more in-
a noninvasive method to measure body temperature. An formation or visit the EthoVision homepage on the Web
infrared thermographic video camera is connected to a (http://www.noldus.com/products/ethovision/).
customized version of EthoVision, and the gray scale
value (i.e., brightness) of each pixel in the image is pro- REFERENCES
portional to the animal’s temperature at that location.
Thus, the animal’s behavior and temperatures of differ- Akaka, W. H., & Houck, B. A. (1980). The use of an ultrasonic mon-
ent regions of the body can all be measured synchronously. itor for recording locomotor activity. Behavior Research Methods &
Further research is necessary before this technique can Instrumentation, 12, 514-516.
Bjerselius, R., Olsén, K. H., & Zheng, W. B. (1995a). Behavioural
become a standard tool. and endocrinological responses of mature male goldfish to the sex
Synchronization of behavioral data (video) and pheromone 17a,20b-dihydroxy-4-pregnen-3-one in the water. Jour-
physiological signals. In order to carry out a full statis- nal of Experimental Biology, 198, 747-754.
Bjerselius, R., Olsén, K. H., & Zheng, W. B. (1995b). Endocrine, go- Gapenne, O., Simon, P., & Lannou, J. (1990). A simple method for
nadal and behavioral responses of male crucian carp to the hormonal recording the path of a rat in an open field. Behavior Research Meth-
pheromone 17a,20b-dihydroxy-4-pregnen-3-one. Chemical Senses, ods, Instruments, & Computers, 22, 443-448.
20, 221-230. Gass, P., Wolfer, D. P., Balschun, D., Rudolph, D., Frey, U., Lipp,
Bokkers, E., & Koene, P. (2000). Motivation and ability to walk in H. P., & Schütz, G. (1998). Deficits in memory tasks of mice with
broilers and layer chicks on two diets. In Proceedings of the 34th In- CREB mutations depend on gene dosage. Learning & Memory, 5,
ternational Congress of the International Society of Applied Ethology 274-288.
(Florianopolis, 17–20 October 2000), p. 115. Florianopolis: UFSC. Hany, J., Lilienthal, H., Roth-Härer, A., Ostendorop, G., Hein-
Bouman, E. A. P., Simek, P., Zemek, R., Dusbabek, F., & Zahrad- zow, B., & Winneke, G. (1999). Behavioral effects following single
nickova, H. (1999). Methods in the study of sexual behaviour of the and combined maternal exposure to PCB 77 (3,4,3¢-tetrabiphenyl)
tick Ixodes ricinus. In Abstracts of the 13th European Meeting of the and PCB 47 (2,4,2¢,4¢-tetrabiphenyl) in rats. Neurotoxicology & Ter-
Society for Vector Ecology (6-11 September 1999), p.84. Corona, atology, 21, 147-156.
CA: Society for Vector Ecology. Hartmann, M. J., Assad, C., Rasnow, B., & Bower, J. M. (2000). Ap-
Brodkin, J., & Nash, J. F (1995). A novel apparatus for measuring rat plications of video mixing and digital overlay to neuroethology.Meth-
locomotor behavior. Journal of Neuroscience Methods, 57, 171-176. ods: A Companion to Methods in Enzymology, 21, 385-391.
Bulpitt, A. J., Boyle, R. D., & Forbes, J. M. (2000). Monitoring be- Healing, G., Harvey, P. W., McFarlane, M., Buss, N. A. P. S., Mal-
havior of individuals in crowded scenes. In Proceedings of Measur- lyon, B. A., & Cockburn, A. (1997). Assessment of motor activity
ing Behavior 2000, 3rd International Conference on Methods and in regulatory neurotoxicitystudies: Validation of the EthoVision video
Techniques in Behavioral Research (Nijmegen, 15–18 August 2000), tracking system in rats treated with amphetamine and chlorproma-
pp. 28-30. Wageningen, the Netherlands: Noldus Information Tech- zine. Toxicology Methods, 7, 279-287.
nology. Heeren, D. J., & Cools, A. R. (2000). Classifying postures of freely
Buma, M. O. S., Moskal, J. R., & Liang, D. (1998). EthoVision Multi- moving rodents with the help of Fourier descriptors and a neural net-
Pro: Improved animal identification during automatic multi-object work. Behavior Research Methods, Instruments, & Computers, 32,
tracking. In Proceedings of Measuring Behavior ’98, 2nd Interna- 56-62.
tional Conference on Methods and Techniques in Behavioral Re- Hoogeboom,P. J. (2000). Data synchronisationthrough post-processing.
search (Groningen, 18–21 August 1998), pp. 103-104. Wageningen, In Proceedings of Measuring Behavior 2000, 3rd International Con-
the Netherlands: Noldus Information Technology. ference on Methods and Techniques in Behavioral Research (Nij-
BureÏsová, O., Bolhuis, J. J., & BureÏs, J. (1986). Differential effects megen, 15–18 August 2000), pp. 147-150. Wageningen, the Nether-
of cholinergic blockade on performance of rats in the water tank nav- lands: Noldus Information Technology.
igation task and in a radial water maze. Behavioral Neuroscience, Houx, B. B., Buma, M. O. S., & Spruijt, B. M. (2000). Non-invasive
100, 476-482. tracking with automated infrared thermography: Measuring inside
Chow, B., Tomkins, D. M., & Sellers,E. M. (1996). Developing an ani- out. In Proceedings of Measuring Behavior 2000, 3rd International
mal model for drug craving. Behavioural Pharmacology, 7(Suppl. 1), Conference on Methods and Techniques in Behavioral Research (Nij-
16. megen, 15–18 August 2000), pp. 151-152. Wageningen, the Nether-
Clarke, R. L., Smith, R. F., & Justesen, D. R. (1985). An infrared de- lands: Noldus Information Technology.
vice for detecting locomotor activity. Behavior Research Methods, Kernan, W. J., Jr., Higby, W. J., Hopper, D. L., Cunningham, W.,
Instruments, & Computers, 17, 519-525. Lloyd, W. E., & Reiter, L. (1980). Pattern recognition of behavioral
Clarke, R. L., Smith, R. F., & Justesen, D. R. (1992). A programmable events in the nonhuman primate. Behavior Research Methods & In-
proximity-contact sensor to detect location or locomotion of animals. strumentation, 12, 524-534.
Behavior Research Methods, Instruments, & Computers, 24, 515-518. Kim, H.-C., Jhoo, W.-K., Kim, W.-K., Suh, J.-H., Shin, E.-J., Kato, K.,
Clement, J. C., Mills, P., & Brockway, B. P. (1989). Use of teleme- & Ko, K.-H. (2000). An immunocytochemical study of mitochondrial
try to record body temperature and activity in mice. Journal of Phar- manganese-superoxide dismutase in the rat hippocampus after kainate
macological Methods, 21, 129-140. administration. Neuroscience Letters, 281, 65-68.
Dalm, S., Grootendorst, S., de Kloet, E. R., & Oitzl, M. S. (2000). Kirkpatrick, T., Schneider, C. W., & Pavloski, R. (1991). A com-
Quantification of swim patterns in the Morris water maze. Behavior puterized infrared monitor for following movement in aquatic animals.
Research Methods, Instruments, & Computers, 32, 134-139. Behavior Research Methods, Instruments, & Computers, 23, 16-22.
Derry, J. F., & Elliott, C. J. H. (1997). Automated 3-D tracking of a Klapdor, K., Dulfer, B. G., & van der Staay, F. J. (1996). A
video-captured movement using the example of an aquatic mollusk. computer-aided method to analyse foot print patterns of rats, mice
Behavior Research Methods, Instruments, & Computers, 29, 353-357. and humans. In Proceedings of Measuring Behavior ‘96, Interna-
Drai, D., Benjamini, Y., & Golani, I. (2000). SEE: Software for the tional Workshop on Methods and Techniques in Behavioral Research
exploration of exploration. In Proceedings of Measuring Behavior (Utrecht, 16–18 October 1996), p. 60. Wageningen, the Netherlands:
2000, 3rd International Conference on Methods and Techniques in Be- Noldus Information Technology.
havioral Research (Nijmegen, 15–18 August 2000), pp. 89-90. Wa- Knoppien, P., van der Pers, J. N. C., & van Delden, W. (2000). Quan-
geningen, the Netherlands: Noldus Information Technology. tification of locomotion and the effect of food deprivation on loco-
Drost, Y. C., Qiu, Y. T., Posthuma-Doodeman, C. J. A. M, & van motor activity in Drosophila. Journal of Insect Behavior, 13, 27-43.
Lenteren, J. C. (2000). Comparison of searching patterns of five Krips, O. E., Kleijn, P. W., Wilems, P. E. L., Gols, G. J. Z., &
parasitoid species of Bemisia argentifolii Bellows and Perring (Hom., Dicke, M. (1999). Leaf hairs influence searching efficiency and pre-
Aleyrodidae). Journal of Applied Entomology, 124, 105-112. dation rate of the predatory mite Phytoseiulus persimilis (Acari: Phy-
Eijkenboom, M., & van der Staay, F. J. (1998). Effects of brain lesions toseiidae). Experimental & Applied Aracology, 23, 119-131.
on Morris water maze performance in rats: Testing the reliability of Kröber, T., & Guerin, P. M. (1999). Ioxid ticks avoid contact with liq-
an automatic video tracking system. In Proceedings of Measuring uid water. Journal of Experimental Biology, 202, 1877-1883.
Behavior ‘98, 2nd International Conference on Methods and Tech- Lanitis, A., Taylor, C. J., & Cootes, T. F. (1997). Automatic inter-
niques in Behavioral Research (Groningen, 18–21 August 1998), pretation and coding of face images using flexible models. IEEE Trans-
pp. 137-138. Wageningen, the Netherlands: Noldus Information actions on Pattern Analysis & Machine Intelligence, 19, 743-756.
Technology. Le Bourg, E., & Minois, N. (1999). A mild stress, hypergravity expo-
Ekman, P., & Friesen, W. V. (1978). The facial action coding system: sure postpones behavioral aging in Drosophila melanogaster. Ex-
A technique for the measurement of facial movement. San Francisco: perimental Gerontology, 34, 157-172.
Consulting Psychologists Press. Lipp, H. P., & Wolfer, D. P. (2000). Assessing behavior, memory and
Fuchs, E., Kramer, M., Hermes, B., Netter, P., & Hiemke, C. (1996). learning of genetically modified mice by factor analysis based on
Psychological stress in tree shrews: Clomipramine counteracts behav- track analysis using EthoVision and WinTrack. In Proceedings of
ioral and endocrine changes. Pharmacology, Biochemistry & Behav- Measuring Behavior 2000, 3rd International Conference on Meth-
ior, 54, 219-228. ods and Techniques in Behavioral Research (Nijmegen, 15–18 Au-
gust 2000), pp. 199-200. Wageningen, the Netherlands: Noldus In- Rasnow, B., Assad, C., Hartmann, M. J., & Bower, J. M. (1997). Ap-
formation Technology. plications of multimedia computers and video mixing to neuroethol-
Martin, B. R., Prescott, W. R., & Zhu, M. (1992). Quantification of ogy. Journal of Neuroscience Methods, 76, 83-91.
rodent catalepsy by a computer-imaging technique. Pharmacology, Robles, E. (1990). A method to analyze the spatial distribution of be-
Biochemistry & Behavior, 43, 381-386. havior. Behavior Research Methods, Instruments, & Computers, 22,
Martin, P. H., & Unwin, D. M. (1980). A microwave Doppler radar ac- 540-549.
tivity monitor. Behavior Research Methods & Instrumentation, 12, Rousseau, J. B. I., Spruijt, B. M., & Gispen, W. H. (1996). Automated
517-520. observation of multiple individually identified rats. In Proceedings of
May, C. H., Sing, H. C., Cephus, R., Vogel, S., Shaya, E. K., & Wag- Measuring Behavior ’96, International Workshop on Methods and
ner, H. N., Jr. (1996). A new method of monitoring motor activity Techniques in Behavioral Research (Utrecht, 16–18 October 1996),
in baboons. Behavior Research Methods, Instruments, & Computers, pp. 86-87. Wageningen, the Netherlands: Noldus Information Tech-
28, 23-26. nology.
Meerlo, P., Overkamp, G. J. F., Benning, M. A., Koolhaas, J. M., & Rousseau, J. B. I., van Lochem, P. B. A., Gispen, W. H., & Spruijt,
van den Hoofdakker, R. H. (1996). Long-term changes in open B. M. (2000). Classification of rat behavior with an image-processing
field behavior following a single social defeat in rats can be reversed method and a neural network. Behavior Research Methods, Instru-
by sleep deprivation. Physiology & Behavior, 60, 115-119. ments, & Computers, 32, 63-71.
Miczek, K. A., Nikulina, E., Kream, R. M., Carter, G., & Espejo, Ruis, M. A. W., te Brake, J. H. A., Buwalda, B., de Boer, S. F.,
E. F. (1999). Behavioral sensitization to cocaine after a brief social Meerlo,P., Korte, S. M., Blokhuis, H. J., & Koolhaas, J. H. (1999).
defeat stress: c-fos expression in the PAG. Psychopharmacology, Housing familiar male wildtype rats together reduces the long-term
141, 225-234. adverse behavioural and physiological effects of social defeat. Psycho-
Minichiello, L., Korte, M., Wolfer, D. P., Kühn, R., Unsicker, K., neuroendocrinology, 24, 285-300.
Cestari, V., Rossi-Arnaud, C., Lipp, H. P., Bonhoeffer, T., & Sahgal, A., & Steckler, T. (1994). TouchWindows and operant be-
Klein, R. (1999). Essential role for TrkB receptors in hippocampus- haviour in rats. Journal of Neuroscience Methods, 55, 59-64.
mediated learning. Neuron, 24, 401-414. Sams-Dodd, F. (1995). Automation of the social interaction test by a
Morrel-Samuels, P., & Krauss, R. M. (1990). Cartesian analysis: A video tracking system: Behavioral effects of repeated phenocyclidine
computer–video interface for measuring motion without physical con- treatment. Journal of Neuroscience Methods, 59, 157-167.
tact. Behavior Research Methods, Instruments, & Computers, 22, Santucci, A. C. (1995). An affordable computer-aided method for con-
466-470. ducting Morris water maze testing. Behavior Research Methods, In-
Morris, R. (1984). Development of a water-maze procedure for studying struments, & Computers, 27, 60-64.
spatial learning in the rat. Journal of Neuroscience Methods, 11, 47-60. Schlingmann, F., van de Weerd, H. A., Baumans, V., Remie, R., &
Morrison, S. K., & Brown, M. F. (1990). The touch screen system in van Zutphen, L. F. M. (1998). A balance device for the analysis of
the pigeon laboratory: An initial evaluation of its utility. Behavior Re- behavioural patterns of the mouse. Animal Welfare, 7, 177-188.
search Methods, Instruments, & Computers, 22, 123-126. Sergeant, D. M., Boyle, R. D., & Forbes, J. M. (1988). Computer vi-
Moser, V. C., Becking, G. C., Cuomo, V., Frantik, E., Kulig, B. M., sual tracking of poultry. Computers & Electronics in Agriculture, 21,
MacPhail, R. C., & Tilson, H. A. (1997). The IPCS collaborative 1-18.
study on neurobehavioral screening methods: IV. Control data. Neuro- Sgoifo, A., Kole, M. H., Buwalka, B., de Boer, S. F., & Koolhaas,
toxicology, 18, 947-968. J. M. (1998). Video tracking of social behaviors and telemetered ECG
Mukhina, T. V., Bachurin, S. O., Lermontova, N. N., & Zefirov, measurements in colony housed rats. In Proceedings of Measuring
N. S. (2001). Versatile computerized system for tracking and analy- Behavior ‘98, 2nd International Conference on Methods and Tech-
sis of water maze tests. Behavior Research Methods, Instruments, & niques in Behavioral Research (Groningen, 18–21 August 1998),
Computers, 33, 371-380. pp. 258-259. Wageningen, the Netherlands: Noldus Information Tech-
Noldus, L. P. J. J. (1991). The Observer: A software system for col- nology.
lection and analysis of observational data. Behavior Research Methods, Silverman, R. W., Chang, A. S., & Russell, R. W. (1988). A
Instruments, & Computers, 23, 415-429. microcomputer-controlled system for measuring reactivity in small
Noldus, L. P. J. J., Trienes, R. J. H., Hendriksen, A. H. M., Jansen, animals. Behavior Research Methods, Instruments, & Computers, 20,
H., & Jansen, R. G. (2000). The Observer Video-Pro: New software 495-498.
for the collection, management, and presentation of time-structured Skinner, B. F. (1938). The behavior of organisms. New York: Appleton-
data from videotapes and digital media files. Behavior Research Meth- Century-Crofts.
ods, Instruments, & Computers, 32, 197-206. Spink, A. J., Buma, M. O. S., & Tegelenbosch, R. A. J. (2000). Etho-
Oitzl, M. S., Josephy, M., & Spruijt, B. M. (1993). An ACTH/MSH(4-9) Vision color identification: A new method for color tracking using
analog counteracts the behavioral effects of a mineralocorticoid recep- both hue and saturation. In Proceedings of Measuring Behavior
tor antagonist. Pharmacology, Biochemistry & Behavior, 44, 447-450. 2000, 3rd International Conference on Methods and Techniques in
Olivo, R. F., & Thompson, M. C. (1988). Monitoring animals’ move- Behavioral Research (Nijmegen, 15–18 August 2000), pp. 295-297.
ments using digitized video images. Behavior Research Methods, In- Wageningen, the Netherlands: Noldus Information Technology.
struments, & Computers, 20, 485-490. Spink, A. J., Tegelenbosch, R. A. J., Buma, M. O. S., & Noldus,
Olsén, K. H., Grahn, M., Lohm, J., & Langefors, Å. (1998). MHC L. P. J. J. (2001). The EthoVision video tracking system—A tool for
and kin discrimination in juvenile Arctic char, Salvelinus alpinus (L.). behavioral phenotyping of transgenic mice. Physiology & Behavior,
Animal Behaviour, 56, 319-327. 73, 719-730.
Olsén, K. H., & Winberg, S. (1996). Learning and sibling odor prefer- Spooner, R. I. W., Thomson, A., Hall, J., Morris, R. G. M., &
ence in juvenile Artic char, Salvelinus alpinus (L.). Journal of Chem- Salter, S. H. (1994). The Atlantis Platform: A new design and fur-
ical Ecology, 22, 773-786. ther developments of BureÏsová’s on-demand platform for the water
Pan, W. H. T., Lee, C.-R., & Lim, L.-H. (1996). A new video path ana- maze. Learning & Memory, 1, 203-211.
lyzer to monitor travel distance, rearing, and stereotypic movement Spruijt, B. M., Buma, M. O. S., van Lochem, P. B. A., & Rousseau,
of rats. Journal of Neuroscience Methods, 70, 39-43. J. B. I. (1998). Automatic behavior recognition: What do we want to
Pereira, P., & Oliveira, R. F. (1994). A simple method using a single recognize and how do we measure it? In Proceedings of Measuring Be-
video camera to determine the three-dimensional position of a fish. Be- havior ‘98, 2nd International Conference on Methods and Techniques
havior Research Methods, Instruments, & Computers, 26, 443-446. in Behavioral Research (Groningen, 18–21 August 1998), pp. 264-
Ploeger, G. E., Spruijt, B. M., & Cools, A. R. (1994). Spatial local- 266. Wageningen, the Netherlands: Noldus Information Technology.
ization in the Morris water maze in rats: Acquisition is affected by Spruijt, B. M., & Gispen, W. H. (1983). Prolonged animal observations
intra-accumbens injections of the dopaminergic antagonist haloperidol. by use of digitized video displays. Pharmacology, Biochemistry &
Behavioral Neuroscience, 108, 927-934. Behavior, 19, 765-769.
Spruijt, B. M., Hol, T., & Rousseau, J. B. I. (1992). Approach, avoid- and Techniques in Behavioral Research (Groningen, 18–21 August
ance, and contact behavior of individually recognized animals auto- 1998), pp. 203-204. Wageningen, the Netherlands: Noldus Informa-
matically quantified with an imaging technique. Physiology & Behav- tion Technology.
ior, 51, 747-752. van Rijzingen, I. M. S., Gispen, W. H., & Spruijt, B. M. (1995). Ol-
Spruijt, B. M., Pitsikas, N., Algeri, S., & Gispen, W. H. (1990). factory bulbectomy temporarily impairs Morris maze performance:
Org2766 improves performance of rats with unilateral lesions in the An ACTH(4-9) analog accelerates return of function. Physiology &
fimbria fornix in a spatial learning task. Brain Research, 527, 192- Behavior, 58, 147-152.
197. van Schelt, J., Buma, M. O. S., Moskal, J., Smit, J., & van Len-
Ï
Sustr, Ï
P., Spinka, M., & Newberry, R. C. (2000). Automatic computer teren, J. C. (1995). EntoTrack: Using video imaging to automate the
analysis of pig play. In Proceedings of Measuring Behavior 2000, 3rd quality control of mass-reared arthropods [Abstract]. In Proceedings
International Conference on Methods and Techniques in Behavioral of IOBC Workshop Quality Control (Santa Barbara, 9–12 October
Research (Nijmegen, 15–18 August 2000), pp. 307-308. Wagenin- 1995).
gen, the Netherlands: Noldus Information Technology. Vorhees, C. V., Acuff-Smith, K. D., Minck, D. R., & Butcher, R. E.
Tarpy, R. M., & Murcek, R. J. (1984). An electronic device for detecting (1992). A method for measuring locomotor behavior in rodents:
activity in caged rodents. Behavior Research Methods, Instruments, Contrast-sensitive computer-controlled video tracking activity as-
& Computers, 16, 383-387. sessment in rats. Neurotoxicology & Teratology, 14, 43-49.
Tian, Y. L., Kanade, T., & Cohn, F. F. (2001). Recognizing action units Weinand-Härer, A., Lilienthal, H., Winterhoff, H., & Win-
for facial expression analysis. IEEE Transactions on Pattern Analy- neke, G. (1996). Exposure to a coplanar PCB congener or PTU and
sis & Machine Intelligence, 23, 97-116. behavioral measurement in rat offspring. In Proceedings of Measur-
Twining, C. J., Taylor, C. J., & Courtney, P. (2001). Robust tracking ing Behavior ‘96, International Workshop on Methods and Tech-
and posture description for laboratory rodents using active shape mod- niques in Behavioral Research (Utrecht, 16–18 October 1996),
els. Behavior Research Methods, Instruments, & Computers, 33. p. 108. Wageningen, the Netherlands: Noldus Information Technol-
van den Berg, C. L., Spruijt, B. M., & van Ree, J. M. (1996). En- ogy.
dogenous opioids and opiate antagonists in the development of social Whishaw, I. Q., Haun, F., & Kolb, B. (1999). Analysis of behavior in
behaviour. Behavioural Pharmacology, 7(Suppl. 1), 115. laboratory rodents. In U. Windhorst & H. Johansson (Eds.), Modern
van der Heyden, J. A. M., Zethof, T. J. J., & Olivier, B. (1997). techniques in neuroscience (pp. 1243-1268).Berlin: Springer-Verlag.
Stress-induced hyperthermia in singly housed mice. Physiology & Winberg, S., Nilsson, G., Spruijt, B. M., & Höglund, U. (1993).
Behavior, 62, 463-470. Spontaneous locomotor activity in Arctic char measured by a com-
van Ginneken, V. J. T., Addink, A. D. F., van den Thillart, puterized imaging technique: Role of brain serotonergic activity.
G. E. E. J. M., Korner, F., Noldus, L., & Buma, M. (1997). Meta- Journal of Experimental Biolology, 179, 213-232.
bolic rate and level of activity determined in tilapia (Oreochromis Wolfer, D. P., & Lipp, H. P. (1992). A new computer program for de-
mossambicus Peters) by direct and indirect calorimetry and video tailed off-line analysis of swimming navigation in the Morris water
monitoring. Thermochimica Acta, 291, 1-13. maze. Journal of Neuroscience Methods, 41, 65-74.
van Kampen, M., Schmitt, U., Hiemke, C., & Fuchs, E. (2000). Diaze- Ylieff, M. Y., Sanchez-Colero, C., Poncin, P., Voss, J., & Ruwet,
pam has no beneficial effects on stress-induced behavioral and en- J. C. (2000). Measuring effects of different temperatures on swim-
docrine changes in male tree shrews. Pharmacology, Biochemistry & ming activity and social behavior in groups of Mediterranean marine
Behavior, 65, 539-546. fish with the EthoVision Color-Pro video tracking system. In Pro-
van Lochem, P. B. A., Buma, M. O. S., Rousseau, J. B. I., & Noldus, ceedings of Measuring Behavior 2000, 3rd International Conference
L. P. J. J. (1998). Automatic recognition of behavioral patterns of rats on Methods and Techniques in Behavioral Research (Nijmegen,
using video imaging and statistical classification. In Proceedings of 15–18 August 2000), pp. 350-351. Wageningen, the Netherlands:
Measuring Behavior ‘98, 2nd International Conference on Methods Noldus Information Technology.
APPENDIX
EthoVision’s Analysis Parameters
Each selected parameter (e.g., velocity) is calculated for each Distance to zone center. The shortest distance between the
individual sample. The user selects which statistic is required center of gravity of the tracked animal and the center of a user-
(e.g., mean), and the combination of the two is displayed (e.g., defined zone.
mean velocity) per track or per group (e.g., the mean of the total Distance to zone border. The shortest distance between the
distance moved for all animals that received a given treatment). center of gravity of the tracked animal and the border of a user-
Input filters can be applied to the data to down-sample the points defined zone.
and/or exclude samples where there is just an apparentmovement In zone. Whether or not the animal is in a particular zone.
due to, for example, breathing. Most of the parameters (e.g., dis- Zones can be redrawn at any time. They can also be added to-
tance moved and velocity)are continuousvariables,but some are gether to form cumulative zones and can be hidden zones for
state variables because they are either true or false (in zone, rela- use with burrows or nest boxes.
tive movement,moving,rearing,and manuallyrecordedbehaviors). These basic parameters are used to calculate a series of de-
rived parameters (see below).
Location and Time
Distance moved. The length of the vector connecting two Path Shape
sample points (i.e., the distance of the center of gravity of the This category describes the geometrical shape of the path
tracked animal between one sample and the next). This is cal- traveled by an animal. Some of these parameters can be based
culated using Phythagoras’ theorem (a 2 + b 2 5 c 2). on unsigned degrees (absolute) and signed degrees (relative).
Velocity. Distance moved per time unit (i.e., speed). Heading. The direction of movement in relation to a user-
Distance to point. The distance between the center of grav- defined reference line.
ity of the tracked animal and a location inside or outside the arena Turn angle. Angle between the movement vectors of two
defined by the user. consecutive sample intervals (absolute or relative).
APPENDIX (Continued)
Angular velocity. Speed of change in direction of move- Proximity. Whether or not the tracked animal is closer than
ment (i.e., amount of turning per unit of time [absolute or rela- a defined distance from another animal. The user can define
tive]). The angular velocity of each sample is the turn angle for threshold values for both “in proximity” and “not in proximity”
that sample, divided by the sample interval. to ensure that the frequency of transitions is biologically mean-
Meander. Change in direction of movement relative to the ingful.
distance moved (i.e., amount of turning per unit distance [ab- Relative movement. Whether or not an animal shows a rel-
solute or relative]). The mean of each sample is the turn angle ative displacement toward (“moving to”) or away from (“mov-
for that sample, divided the distance moved from the last sample. ing from”) another tracked animal. EthoVision does not just
measure whether two objects get closer to or farther away from
Individual Behavioral States each other, but the speed and direction of movement is taken
Activities of the animal are assigned to behavioral states. into account so that it is possible to correctly assess situations
Movement. Whether or not the tracked animal’s velocity ex- where Object A is moving toward Object B, but B is moving
ceeds a user-defined level. The user can set thresholds for both away from A (Spruijt et al., 1992).
“start velocity” and “stop velocity,” and the parameter is calcu- Speed of moving to and from. The distance-weightedspeed
lated over a running average of a user-defined number of sam- at which an animal moves toward or away from another animal.
ples. The parameter has two states: “moving” and “not moving.” Closer objects are given a heavier weighting, and movements
Rearing. The state in which a rodent’s body is vertically between objects a long way from each other have a slower speed
erected. It is a useful parameter for assessing exploratory be- of movement. “Speed of moving to” is calculated only if the rel-
havior. It is detected by measuring the decrease in surface area ative movement is “moving to.”
when the rat or mouse stands up. The user can define a running Net relative movement. The signed, distance-weighted
average and percentage decrease in area to customize the param- change in distance between two animals. Net relative movement
eter for different species and conditions. combines the two parameters “Speed of moving from” and
Manually recorded behaviors. While EthoVision is track- “Speed of moving to” and can be used to quantify avoidance and
ing the animals, the user can register any predefined behaviors aggression.
by pressing a key (or with the mouse), and these can be ana- The parameters of social interactions, based on the relative
lyzed in exactly the same way as any of the state variables auto- movement between pairs of objects, can also be calculated for
matically measured by EthoVision. the relative movement between animal(s) and zones/points of
interest (i.e., a static position in the arena). This allows the user,
Social Interactions for example, to calculate the speed with which an animal moves
On the basis of relative movement between pairs of simulta- toward a novel object or when an animal is in proximity with
neously tracked animals, distances and movements are classi- the center of the open field.
fied and assigned to particular parameters of social behavior.
Distance between objects. The distance between the center (Manuscript received February 15, 2001;
of gravity of two animals. accepted for publication May 22, 2001.)

Ethovision: A Versatile Video Tracking System For Automation of Behavioral Experiments

Uploaded by

Copyright:

Available Formats

Ethovision: A Versatile Video Tracking System For Automation of Behavioral Experiments

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Ethovision: A Versatile Video Tracking System For Automation of Behavioral Experiments

Uploaded by

Copyright:

Available Formats

Behavior Research Methods, Instruments, & Computers

2001, 33 (3), 398-414

EthoVision: A versatile video tracking system

Copyright 2001 Psychonomic Society, Inc. 398

interest (e.g., the center and edges of an open field), the

E THOV ISION SOFTWARE SPECIFICATIONS

EthoVision may be thought of as consisting of a series

Selection and Visualization of Track Data

Figure 6. The HSI color space model. The horizontal distance

This means that this point is farther toward the animal’s

Data Analysis In contrast to some other video analysis systems (e.g.,

You might also like