Eye Tracking Application on Emotion Analysis for

Marketing Strategy
H. Zamani, A. Abas, M. K. M.Amin
Bio Cognition Laboratory, Bio-Inspired System and Technology iKohza (Research Group),
Malaysia – Japan International Institute of Technology (MJIIT),
Universiti Teknologi Malaysia (UTM) 54100 Kuala Lumpur, Malaysia.
[email protected]

Abstract—This paper present on understanding the human (Ophthalmoception). Through sense of vision, information is
behavior with eye tracking studies related to consumer cognition provided by the eye that functioning almost like a camera.
in marketing product. The study of human behavior using eye From the eyes, several visual pathways travel to many parts in
tracking is a growing multidisciplinary field that links the brain such as visual cortex, a part required for conscious
electronics, psychology and cognitive science to study the human
vision. Though, the human eye is not a one-way
behavior on problem solving and decision making. In this paper,
we particularly investigate the human behavior in consumer communication channel. Emotion similarly is expresses
cognition. We conducted experiments to track the human eyes by through our eyes. Latest technologies have made it possible to
using the Tobii TX300 eye tracker. The eye–mind relationship utilize recorded data associated with both functions of the eyes
can help to use eye motions activity measurements expressing to [7, 8].
some degree about human behavior. The result shows that the
visual attention of human is very much related to the cognition of II. BACKGROUND OF EYE TRACKING
the products.
In 1879, Louis Emile Javal [9], a French ophthalmologist
Index Terms—Emotion Analysis; Eye Tracking Application;
was amongst pioneers, defining eye tracking measurement
Neuromarketing.
through text reading. She observed that people do not read
I. INTRODUCTION smoothly and continuously across a page, but pause (fixations)
on some words while moving quickly (saccades) through
According to Holbrook & Hirschman (1982), for the past two others. Edmund Huey [8] later built a device that was used to
decades, marketing researchers tended to neglect the track eye movement using small contact lens provided with a
emotional part of consumer behavior while much focusing on hole for pupil. The lens was connected to an aluminum pointer
decision oriented models of information processing [1], while to observe the gaze direction during reading and he could
all marketers hope to understand customers and their purchase study which words a reader would pause on.
decisions. However, when explicitly asked, most people Dodge and Cline [8] developed photochronograph, the first
cannot fully and sometimes, do not even want to explain their accurate and non-invasive eye tracker based on corneal
choices. Somehow, they are not aware about the inner reflection. It was the result from their study regarding the
processes that drive their reactions and decisions. This is velocity of eye movements. However, the photochronograph
because 95% of the decision making process occurred plate can be used only to record horizontal eye movements.
subconsciously [2]. How does the brain decide and initiate Nonetheless, after four years, in 1905, a photo device that can
preferences probably one of the most crucial questions in records motions of eyes on film in both direction, horizontally
neuroscience, psychology and economics today? and vertically is developed by an American psychologist and
Consumer neuroscience is a growing multidisciplinary area education reformer, Charles H. Judd [8].
of study that include psychology, neuroscience, and Eye tracking was used for educational research as well as by
economics to understand how the brain is physiologically medical researchers and physicians for a long time, more than
influenced by advertising and marketing [3]. The term elsewhere because the device is considerably very expensive.
“Neuromarketing” suggests a combination of marketing Only recently that it became more accessible to a new niche:
research and brain research [4, 5]. Approximately, 25% of the marketers [8, 9].
brain regions related to vision. Based on Neuromarketing
III. EYE TRACKING APPLICATION IN NEUROMARKETING
studies advertising images can elicit different degree of local
brain activity and these changes can be measured in term of
In general, eye tracking is used to determine the direction of
emotional arousing picture [6].
human gaze based on the position of the eye of a person at a
“The eyes are the windows to our soul”. This well-known
duration time and its moving sequence. This technology
proverb about the sense of vision might be the spark to the
covers for example human computer interaction, brain
emergence of eye tracking technology. Humans have five
computer interaction, assistive technology, eLearning,
traditionally recognized senses including sight or vision
psychology investigation, and virtual and augmented reality.

ISSN: 2180-1843 e-ISSN: 2289-8131 Vol. 8 No. 11 87

Journal of Telecommunication, Electronic and Computer Engineering

Some of the vendors that developed and brought eye were familiar to the respondents based on the post experiment
tracking technology to the market are SensoMotoric interview conducted.
Instruments, Tobii, iMotions, MyGaze. These technologies are After the participant’s eye movements are recorded, areas of
dedicated on camera-based eye-tracking systems. The light interest (AOI) are drawn using the Tobii Studio 3.4.1
source and camera are perpetually attached to a monitor. Most software. This is to extract fixations for specific area in the
eye trackers are contained of three main mechanisms. These stimulus given. It helps researchers analyze the various
are a set of illuminators or projectors, cameras, and systems components of a visual scene. Each AOI is a polygon
for vision and mathematical analysis and calculations. The encompassing an attribute of interest within the image. After
computer that is connected to the eye tracker is use to run the the AOIs were defined, the data were exported to the statistical
algorithms using a specified processing system. software of the Tobii Studio 3.4.1 for every AOI. The eye
Most modern eye tracker practices a method called corneal tracker collecting several metrics such fixation count (number
reflection to detect and track the location of the eye as it of fixations), fixation duration (length of fixations in
moves. This method uses a light source to illuminate the eye. milliseconds), total fixation duration (fixation count times
Then a reflection is produced that is detected by a fixation duration), percent fixated (percent of sample who
high-resolution camera. The reflection of the light source on fixated in the AOI), time to first fixation (in milliseconds), and
the cornea and in the pupil, is identified by the image captured scan path (chronological fixations and saccades) for each AOI.
by the camera. Advanced image processing algorithms are
then used to establish the point of gaze related to the eye and V. RESULTS AND DISCUSSION
the stimuli [10]. For this research, Tobii TX300 eye tracker
will be used for the purpose of data acquisition and analysis. The eye movement data in this research is illustrated using
gaze plots and heat map. While eye tracking camera records
Eye Tracking Metrics several metrics, the results shown below only for total fixation
The usability of eye tracking systems is evaluated by count. This is because fixation counts are often used to
metrics that are relevant to the tasks and their inherent investigate consumer behavior because information
cognitive activities. The most important specific metrics are acquisition occurs during fixations. The total fixation counts
[8, 11]: Fixation: the duration to process the image by fovea for nine respondents were depicted using bar chart. Each AOI
(80ms to 27ms for visual search). Saccade: the interlude represent by different color of polygon corresponds to the
between two fixations (3 degrees for visual search). Gaze color of bar on the bar chart. From the multiple fixation points
Duration: accumulative duration and average spatial setting and saccades, we can tell the participant looked back and forth
of consecutive fixations series within an area of interest. Area between elements several times. These repeated movements
of interest (AOI): area of visual display and environment that suggest they weighed each option, unsure which product they
is of interest to the research or design team and thus defined want. Highest number of fixation count shows the main focus
by them (not by the participant). Scan Path: spatial of attention.
organization of fixations sequence. In the following each different parameter explained briefly
The raw data graphical depictions of an eye tracking with figures of corresponding result of response attention.
experiment can be shown in various ways, but two most
frequently used are gaze plot and heat map. Heat maps show A. Size of products
fixation concentration, represent by three different colors; red, Figure 1 shows comparison between a bottled Coca Cola
yellow, and green. Red specifies ranges of high fixation and a canned Coca Cola to identify which size is more
attention, followed by yellow, and then green. Gaze plots preferable by the respondent. The AOI for bottled
show the fixations, saccades, and scan path. Each participant Coca Cola is pink and yellow for canned coca Cola Based on
is represented by a different color of circle of gaze plot. Each Figure 1, the total fixation count for bottled Coca Cola is
fixation is denoted by a chronologically circle numbered. The higher compared to the canned Coca Cola. Form post
circle diameter corresponds to the length of the fixation in experiment interview, the respondents who chose the bottled
milliseconds [12]. Coca Cola stated that they preferred the bigger size.

IV. EXPERIMENTAL DESIGN B. Design of Packaging

The original image of Nutella spread in Figure 2 compares
The experiment was designed to happen under the most the three jar of Nutella in term of packaging’s design. The bar
realistic conditions as if they were realistically shopping in the chart in Figure 2 shown that fixation count sum is highest for
retail environment. Nine respondents participated in this study. the first jar (yellow AOI), followed by second jar (green AOI),
The experiment was conducted inside the closed room at Bio and third jar (blue AOI). The product and its packaging
cognition laboratory. A sequence of five images was displayed influence attention, evaluation, and ultimately, impact the
on the Tobii TX300 screen for four minutes per image. The consumer purchasing decision on the product [13]. The buying
images represent different parameters in order to find human choice and the time spent detecting the packaging shows that
visual attention on the main parameters. The parameters are participants preferred the one with generally recognized
position of product on shelf (shelf number), price, brand, packaging. Based on the post experiment interview, the
advertising, design of packaging, and size. These parameters respondents who chose the first jar commented on their
were defined as the AOI. All the images displayed represent answer, saying that their choice is due to the first jar is the
the actual product available at local Malaysia’s market and design of Nutella spread that is used in local market. The

88 ISSN: 2180-1843 e-ISSN: 2289-8131 Vol. 8 No. 11

 Eye Tracking Application on Emotion Analysis for Marketing Strategy

cognitive information had prevented them from choosing the (yellow), and the ‘description’ (turquoise). The number of
second or third jar because the design is not familiar to them fixation on the food setup recorded as shown in Figure 4 was
[14]. the highest at 202 counts followed by 26 count at the price and
13 counts at the description.

Figure 2: Comparison of three jar of Nutella in term of packaging’s design

E. Brand and Price

Brand and Price: In this section, two products that is almost
Figure 1: Comparison between a bottled Coca Cola and a canned Coca Cola identical to each other is compared to find out the element that
attract respondent attention when choosing between two
similar products. In Figure 5 compared two cars with similar
C. Position on Shelf physical appearance but from different brand, one is local
The AOI for position of product on the shelf is defined by (Perodua Nautica) and another is from international brand
the row number of the shelf. Starting from the upper row as (Toyota Rush). The AOI is narrowed down to only at the car
‘1st’ denoted with dark orange coloration, followed by second emblem. This is done to figure out if respondent prefer normal
row as ‘2nd’ with mint color, ‘3rd’ with pink, ‘4th’ with light brand or strong brand. From Figure 5, it is clearly that
orange, and yellowish­green for ‘5th’ row. From the bar chart majority of the respondents’ attention were on the turquoise
in Figure 3, the third row caught the most attention since it got polygon (Toyota Rush) compared to green polygon (Perodua
the highest fixation counts. Meanwhile, no fixation count was Nautica).
recorded at the first row. When shopping, consumers observe
VI. CONCLUSION
and evaluate only some division of the hundreds of choices
available on supermarket shelves. The actions of consumers at
From the research conducted and post-experiment interview,
the point of purchase is therefore subjective by visual attention
we can conclude that the product that a person is likely to buy
factors: the product arrangement on the shelf, colors, and other
is the one that has the highest number of fixation. “Look More
factors [13].
Like More”. The experiment also proves that the visual
attention of human being reflects their consuming and
D. Advertisement
purchasing behavior. This can be seen from the result of the
Three AOIs were defined for the KFC advertisement in
heat maps. The effects of purchaser behavior, promotion,
Figure 4. The AOIs defined were the ‘food’ (blue), the ‘price’

ISSN: 2180-1843 e-ISSN: 2289-8131 Vol. 8 No. 11 89

Journal of Telecommunication, Electronic and Computer Engineering

marketing, pricing, products circulation and decision making

can be analyze on a much more scientific basis. A further
research on consumer cognition and its visual attention will
allow researchers to have greater understanding of human
behavior for many applications e.g. human emotional behavior
in marketing, health care, personal traits, wellness, and many
more.

Figure 4: Result of advertisement on visual attention

Figure 3: Comparison of position of product on shelf

90 ISSN: 2180-1843 e-ISSN: 2289-8131 Vol. 8 No. 11

 Eye Tracking Application on Emotion Analysis for Marketing Strategy

REFERENCES

[1] M. Holbrook and J. O’Shaughnessy, “The role of emotion in

advertising,” Psychol. Mark., 1984.
[2] G. Calvert and M. Brammer, “Predicting consumer behavior: using
novel mind-reading approaches,” IEEE Pulse, 2012.
[3] R. Khushaba, C. Wise, S. Kodagoda, and J. Louviere, “Consumer
neuroscience: Assessing the brain response to marketing stimuli using
electroencephalogram (EEG) and eye tracking,” Expert Syst. with, 2013.
[4] W. Dimpfel, “Neuromarketing: Neurocode-Tracking in Combination
with Eye-Tracking for Quantitative Objective Assessment of TV
Commercials,” J. Behav. Brain Sci., vol. 5, pp. 137-147, 2015.
[5] A. Javor, M. Koller, and N. Lee, “Neuromarketing and consumer
neuroscience: contributions to neurology,” BMC, 2013.
[6] T. Bergerud, “How Neuromarketing can Help Strengthen the Non- - ‐
profit Sector : An Eye-tracking Study of Emotional Appeals in
Fundraising Communications,” Copenhagen Business School, 2013.
[7] M. Koller, T. Salzberger, G. Brenner, and P. Walla, “Broadening the
range of applications of eye-tracking in business research,” Anal. Porto
Alegre, 2012.
[8] R. Lupu and F. Ungureanu, “A survey of eye tracking methods and
applications,” Bul Inst Polit Iasi, 2013.
[9] David Leggett, “A Brief History of Eye-Tracking,” 2010. [Online].
Available: http://www.uxbooth.com/articles/a-brief-history-of-eye-
tracking/. [Accessed: 19-Jan-2010].
[10] J. K. Haneveld, Eye Tracking in User Experience Design, vol. 139, no.
4. 2014.
[11] K. Rayner and M. Castelhano, “Eye movements,” Scholarpedia, vol. 2,
no. 10, p. 3649, 2007.
[12] H. Khachatryan and A. L. Rihn, “Eye-Tracking Methodology and
Applications in Methodology : Experimental Design , Sampling , and
Data,” pp. 1–5, 2014.
[13] R. dos Santos and J. de Oliveira, “Eye Tracking in Neuromarketing: A
Research Agenda for Marketing Studies,” International, 2015.
[14] R. Hurley, D. Hutcherson, and C. Tonkin, “Measuring physiological
arousal towards packaging: tracking electrodermal activity within the
consumer shopping environment,” J. Appl.Packaging Research, Vol.7,
no. 3, pp.76-90, 2015.

Figure 5: Compared two cars with similar physical appearance but from
different brand

ISSN: 2180-1843 e-ISSN: 2289-8131 Vol. 8 No. 11 91