10

Graphical Means of Directing Users' Attention

in the Visual Interface

Shumin Zhai Julie Wright Ted Selker Sabra-Anne Kelin

IBM Almaden Research Center

650 Harry Road, MS NWE/B2
San Jose, CA 95120 USA
{zhai, amethyst, selker}@almaden.ibm.com, [email protected]

ABSTRACT This paper presents two experiments in using "masks" to direct users' attention within a graphical
user interface. Bleaching, darkening, blurring, and a solid-color pattern overlay (screening) are used to de-emphasize
background material, causing the target to visually "pop-out" at the user. The tradeoff between effectively directing
the user's attention and ensuring the readability of the background material is explored. Experimental results indicate
that there is a wide range of darkening and screening levels that can create a pop-out effect without degrading the
readability of the masked area.

KEYWORDS attention, pop-out effect, help systems, visual masking, user interface techniques

1. INTRODUCTION COACH provides help and navigational control

Computers can present users with a seemingly information in an external bubble.
endless variety of options and data. This complexity, Directing a user's attention by masking
which can easily overload users, has lead to the use of (de-emphasizing) parts of the visual display also has
techniques such as 'wizards', simplified, step-by-step applications beyond help or tutor systems. Many
interfaces that break up tasks into small, easy-to-digest examples exist in mainstream applications, such as
parts. This simplicity, however, is often gained at the Lotus Word Pro and Microsoft Windows95. When a
expense of the flexibility and efficiency more user chooses to quit Microsoft Windows95, the desktop
traditional interfaces can provide. For instance, a is covered with a pattern of black dots, making it
wizard often locks the user into a particular sequence appear darker than the confmnation dialog. In Word
of choices that may not be suitable for a particular user. Pro, the brightness of colors on inactive icons or
The tradeoff between simplicity and flexibility/ controls is reduced, causing them to look washed-out.
efficiency suggests a third interface approach that Masked objects may retain their readability so as to
combines both the simplicity of wizards and the provide orientation, access to obscure or advanced
flexibility and efficiency of a more traditional interface. options, or recall of previous settings. Yet by creating
One such an approach is illustrated by the Cognitive perceptual distance between the highlighted
Adaptive Help (COACH) agent system, which (unmasked) and de-emphasized (masked) areas, it is
annotates traditional interfaces for inexperienced users possible to direct the user's visual attention to the
(Seiker, 1994). In COACH, dialogs are partially emphasized area.
covered with a bitmap (mask). Only one step in the The process of highlighting information is a
task remains uncovered at a time, in order to focus the significant part of traditional graphic design.
user's attentions on this single step. At each step, Underlining, changing font, changing color, and
changing the background are commonly used in printed

Human-Computer Interaction: INTERACT'97 S. Howard. J. Hammond & G. Lindgaard (editors)

Published by Chapman & Hall ©IFIP 1997
60 Part Two Technical Sessions

media. Many such visual dimensions that can produce mimic a person's vision system: only the area being
a "pop out" effect. One candidate dimension is color. focused on is received in high acuity, but the objects in
However, a color's pop-out effect is complicated. For the periphery are in low resolution. We hoped the
example, an orange dot will stand out amidst a mass of user's attention would be automatically drawn to the
yellow dots. An orange dot will also stand out amidst unblurred object.
many red dots, but the same orange dot will not stand An experiment was designed to test if these
out among a collection of both yellow and red dots techniques were indeed effective. Adobe Photoshop
(D'Zmura, 1991). Consider also a 460-nm was used to produce the masked image. To darken the
(wavelength) spot. Such a spot will usually appear masked areas, the brightness was reduced (-100), while
blue. But, if that same spot is made very small, it will the contrast was increased (+35). Bleached masking
look white. Brightness is also a factor. The perceived was created by reducing the contrast (-60). A Gaussian
brightness of a region depends heavily on the filtering of 1.5 pixels was applied to create the blurred
brightness of the surrounding area. A lighter condition.
background tends to darken, while a darker background In the experimental task, a dialog was created
tends to lighten. Color, luminance, contrast, area, containing over 40 common interface items (buttons,
spatial frequency, and temporal considerations are radio buttons, check-boxes, entry fields, and so forth,
among the factors that interact to determine visual see Figure 1 and 2). For each mask condition, subjects
perception (Rogowitz, 1983). Most importantly, color
is also a dimension that application designers rely
heavily upon. Using color to produce a pop out effect
in a computer interface is likely to interfere with the
application's use of color.
Groupings of many small elements can form
textures that also have a pop-out effect. For example, a
vertical bar will stand out in a field of horizontal bars
(D'Zmura, 1991). Elements of different size or
complexity, such as a group of plus signs C+')
randomly oriented in a field of randomly oriented

-
minus signs C-') will also pop-out, but the same
phenomenon will not work using a '1" and an 'L' t:_~ ~"-t- . )

(Julesz, et al., 1983). Using texture to produce a

'pop-out' effect may also invade or restrict an
application designer's visual design space.
Instead of using color or texture that invade an
Figure I: Dialog for Experiment 1 before masking
application designer's design space, the techniques
explored in the following experiments manipulate
brightness, contrast, and resolution to produce a
perceptual difference between what is masked and what
is unmasked. We were interested in testing if these
approaches would produce effective pop-out effect
without damaging the legibility of the masked objects.

2. EXPERIMENT 1

2.1 Design
Three masking techniques, namely darkening,
bleaching, and blurring, were used in this experiment.
Each has a metaphor grounded in everyday experience.
Darkening creates the effect of a spotlight on a
darkened stage. Bleaching makes the masked area look
faded, possibly causing the bright, vivid colors in the Figure 2: The same dialog after masking (by
unmasked area to stand out. Blurring was an attempt to darkening) all objects but the button 'mouth'
 Graphical means of directing users attention in the visual interface 61

Selection Ti mes
With Slmdald Erro r error ~ rs.
4.5,---;-...l...------:-...l..--------1--_ 9 o Target detection
4
~ Background reading
3.5
3
2.5
2

.5
o
Test I Test 2 Test 3 Blurred Dark Bleached
Test Figure 4: Subjective rating in Experiment 1
(10 = very easy, 0 = very difficult)
Figure 3: Response times (seconds) for experiment 1
Based upon the performance measurements and the
subjective ratings, the blurring technique appeared to
were shown the dialog with the mask applied
be inferior to the other two techniques, both for target
everywhere but on the target item (Figure 2). Subjects
selection and background readability. The difference
were instructed to fmd the target that stood out and to
between bleaching and darkening was less clear.
click it as quickly as possible. The dialog would
Darkening appeared to be better for target pop-out
disappear, then reappear with a different target item
effect; bleaching better for maintaining background
unmasked. Three tests were conducted for each
readability (Figure 4).
masking technique. Each test consisted of 20 target
This experiment indicated that the design of a mask
selections. The testing order of the three masking
could affect both performance and preference, and that
techniques was balanced in a Latin square pattern. A
an interesting design (such as the blurring technique)
total of 18 subjects participated in the experiment. A
does not necessarily produce good results. It is also
questionnaire, filled out at the end of the experiment,
informative to note that the target type (either icon,
asked subjects to rate the three masking techniques
words or others) did not change the masking effect.
according to the ease of fmding the target and the
There are a number of issues that were not resolved
readability of the masked portion of the dialog. by this experiment. First, while the fact that the
subjects were able to fmd the unmasked target (they
2.2 Results were asked to click objects that stand out, no additional
Repeated measure analysis of variance (ANOYA) description of the targets was given) showed that all of
on the selection time data showed that the blurring the masking techniques effectively directed the user to
technique was significantly inferior to the darkening the targets and two of the techniques were more
technique (Fl, 15 = 6.5. P < 0.05). Other pair effective than the other, it was unclear how strong such
comparisons were insignificant. Subjects' performance an effect was in comparison to searching for a target in
improved through practice (from early tests to later the absence of a mask. Second, the experiment's
tests, see Figure 3) significantly (F 2, 130 = 7.3. P < 0.01), assessment of the readability of the masked area was
supported only by subjective data. Third, we suspected
but such a factor did not interact with the techniques.
that the gain in making a target stand out might
Neither experimental order, nor target type (pictures,
sacrifice the readability of the masked area. This would
words and others) nor any interaction was significant.
certainly be true if masking was taken to the extreme .
As measured by subjective rating data, all pairs of
Consider, for example, the most extreme level of the
techniques were significantly different from each other,
darkening technique. The background area would be
both for target selection (p < 0.05) and for readability
darkened until it turned solid black. This would
of background information (p < 0.001) (see Figure 4).
62 Part Two Technical Sessions

COAL Darkened
COAL Bleached

COAL
No Mask Level I Level 2 Level 3 Level 4 Level 5
Figure 5: Sample buttons for each mask and density in Experiment 2
certainly cause the target item to stand out, but items in testing. The lowest levels of masking appeared
the masked area would be completely hidden. The suffIciently close to no-mask condition and the highest
tradeoff between the pop-out effect and the readability levels of masking made the masked words nearly
of the masked area might be a function of the masking indistinguishable.
density: the heavier the masking is, the stronger the A target matching task was designed for this
pop-out effect might be, but the poorer the readability experiment. Subjects were instructed to select, as
of the masked area could become (See Figure 5). Is this quickly as possible, the button that matches the target
a linear tradeoff, or is there, as in the design of partially word at the top of the dialog as shown in Figure 6. The
transparent menus (Harrison, et aI., 1996), a range of advantage of this experimental scenario, in comparison
masking density that provides both a pop-out effect and with Experiment 1 was that all three aspects of masking
readability of the masked objects? A second could be evaluated. First, subjects' performance of
experiment was designed to address these issues. searching the target word when no mask was applied
served as the normal, control condition. Such a normal
condition can also be considered as Level 0 masking
3. EXPERIMENT 2 with any of the three technique. Second, when the
target word was highlighted by various levels of masks
3.1 Design applied to other words, the pop-out effect could be
measured in comparison to the control condition.
Four masking conditions were tested: bleaching,
Third, when the target word was masked together with
darkening, screening, and no-masking (see Figure 5 for
all other buttons, the readability of the masked area
the samples used). The bleaching and darkening masks
were similar to those used in the fIrst experiment. The could be evaluated.
A total of 13 subjects performed the task under the
addition of the screening technique, which overlaid a
normal control condition as well as all fIve levels of all
uniform, periodic, yellow pattern onto the masked
three masking techniques. For each level in each
areas, was motivated by its low cost of implementation:
masking technique, two sets of trials were run. In one
only a percentage of pixels have to be replaced with
set of trials, the target word was left unmasked
yellow color. The blurred mask was not tested because
(therefore highlighted) while the rest of the buttons
it did not perform well in the fIrst experiment. The "no
mask" condition was used as a control condition.
Experiment 1 only tested one level of masking for
each technique. This experiment tested fIve levels
(from 1 to 5) of masking with for each technique were
tested. The fIve levels of bleaching were created by
reducing the contrast (-30, -45, -60, -75, -95). The fIve
levels of darkening were created by increasing the
contrast (+50) and decreasing brightness (-20, -40, -60,
-80, -100). The levels of screening were created by
replacing a pattern of pixels with yellow pixels (RGB
255 255 85) such that a percentage of the pixels were
blocked out (12%, 25%, 37%, 62%, 75%). Note that
the intervals within each technique were not necessarily
equal. The values were chosen and adjusted to cover as
wide as possible a range of appearance (see Figure 5)
Figure 6: Test dialog for Experiment 2 (in darkening
according the authors' subjective experience and pilot
condition)
 Graphical means of directing users attention in the visual interface 63

16,--'------'------'-------'------l.----'L-,-
6 Extremely
14 Easy
4
o Restrained
• Free 2

Or-~~----------------------+

-2

o Restrained
-4
2 • Free
Extremely
O~-r____r-=U~nm~as~k~e~d_.----._--_.~ -6 Difficult
No Mask-20 -40 -60 -80 -100 No Mask -20 -40 -60 -80 -100
Figure 7: Mean task completion times and standard Figure 8: Subjective ratings with the darkening
errors when the target was (top curves) masked or technique. +5 is extremely easy and -5 is extremely
(bottom curves) highlighted / unmasked using the difficult.
darkening technique.
Figure 7 illustrates the performance results of the
were masked. In the second set, the entire dialog was darkening technique. First, let us examine the pop-out
masked, including the target word. effect when the target was the highlighted/unmasked
Each set of trials consisted of 5 practice trials word. The bottom curves showed that, in comparison to
followed by 20 test trials. Each of the test trials in a set Level 0 (no masking), selection time was significantly
corresponded to one of the 20 words (buttons) in the reduced when the rest of window was masked
dialog (Figure 6), so all sets of trials encompassed the (darkened) to each of the masked levels: Fs. ss = 110.9,
equal total travel distances. The order of the buttons p < .0001, once again showing that darkening is an
was changed randomly between sets to avoid effective means of creating a pop-out effect.
memorization of a button's placement. Interestingly, even the slightest level of darkening
The order of the masking conditions was tested appeared to make the unmasked object "pop
determined by a Latin-square sequence of the out": mean selection time changed from 3.9 seconds at
darkening, bleaching, and screening techniques. The no darkening to 1.4 second at Levell (-20) darkening.
normal control condition was randomly inserted into It appeared that even the lowest level of masking
the Latin-square sequence. For a given masking enabled the subjects to be "cued" to, instead of
condition, the order in which the densities were tested searching for, the target. Further darkening did not
was random. make subjects perform much faster: mean selection
During pilot testing, subjects tended to move closer time changed from 1.41 seconds at Level 1 to 1.36
to the computer screen under the more difficult second at Level 5. Such a difference was not
conditions, presumably to compensate for increased statistically significant. Subjective rating data,
difficulty. It seemed possible that subjects could however, appeared to be more sensitive to the
maintain the same performance by exerting extra effort darkening level increase (Figure 8). The higher levels
in more difficult conditions. A between-subject factor of darkening appeared to make the unmasked target
was introduced to explore this possibility. Six of the more distinguishable and were rated significantly easier
subjects were allowed to move their head position than the lower levels of darkening. For example, a
freely and seven subjects were prevented from moving T-test showed p < .0001 significance between Levell
closer than 1.5 feet by a string across the chest. (-20) and 5 (-100).
Second, as shown by the top curves in Figure 7,
3.2 Results subjects' performance in reading the masked area was
not significantly affected by darkening level except at
the very extreme (Level 5). T-tests showed that
Masking by Darkening selection in the masked area at the highest level (-100)
64 Part Two Technical Sessions

was significantly slower than the other levels (p <

.0001), but the differences among other levels tested
14
were relatively small (from 3.9 seconds at Level 0 to "." 0 Restrained

•
~

4.4 seconds at Level 4, statistically insignificant). It is 50 12 Free

remarkable that subjects maintained similar •
~

performance for such a wide range of apparent ';' 1

difficulty (See Figure S). Subjective rating scores on .s
....
selecting masked targets moved towards "extremely •c: s
~

difficult" as darkening level increased (Figure 8). Not 0

Q.
6
only was highest level (-100) rated significantly more •
~

II!
difficult than other levels (p < 0.0001), but levels 3 and 4
4 were also rated more difficult than Level 0 (no
darkening): p < O.OS. 2
Viewing condition (Restrained vs. Free moving)
0
was weakly significant to selection time in masked No Mask-3~ -45 -60 -75 -95
area: F t •ll = 4.4, P = 0.06. By moving closer to the
screen as one strategy to compensate the increasing Figure 9: Task completion times when the target was
difficulty, the free moving group had faster selection (top curves) masked or (bottom curves) highlighted /
time only at the very dark levels. There was a strong unmasked using the bleaching technique.
interaction between viewing condition and darkening
level (F s. ss = 8.6, P < 0.0001) in selecting masked Extremely
targets. This was the only case in all performance and Easy
subjective data with all masking techniques where 4
viewing condition made a statistically significant
difference in the experiment. 2
To summarize, the results showed that except at the
very extreme, a wide range of darkening levels can 0+-~~~~------------------4
create a pop-out effect without significantly damaging
the readability of the masked objects, although as the
-2
darkening level increases users may subjectively
experience a slight increase in pop-out effect and a
slight decrease in readability (compare Figure 7 and 8).
Subjects tended to maintain the same level of
-4
Extremely •
-6 Difficult
performance by exerting additional effort, as indicated
by the smaller time increase in selecting masked targets No Mask-3D -45 -60 -75 -95
by the free moving group (Figure 7). Figure 10: Subjective ratings with the bleaching
technique. +S is "extremely easy" and -S is "extremely
Masking by Bleaching difficult" .
As shown in Figure 9 (bottom curves), the lowest
level (-30) of bleaching tested did not appear to be selecting the unmasked target became significantly
effective enough to direct the subjects' attention. easier (see Figure 10).
Although it caused significantly shorter selection time The readability of the bleached area was similar to
than without masking (mean time 3.89 vs. 3.33, p < that of the area masked by darkening: selection time at
.OOS), stronger bleaching produced still shorter the very extreme (-9S) was significantly different from
selection time: p < .0001. This suggests that Levell the rest (p < 0.0001). Differences among Level 0 (no
bleaching used in the experiment did not produce the mask) to Level 4 (-7S) were not significant, although
pop-out effect needed to fully utilize viewers' subjective rating indicated increased difficulty.
preattentive perception. For the rest of the bleaching Overall, in comparison to the darkening technique,
levels, mean selection times were (from Level 2 to a much narrower range of bleaching levels can
Level S) 1.6S, 1.33, 1.38, 1.39 seconds, no significant accommodate both the pop-out effect and readability
difference was found among these means. Subjective (compare Figure 9 against 7, Figure 10 against 8),
rating indicated that as bleaching level increased, although the best performances achieved by the
 Graphical means of directing users attention in the visual interface 65

16~~----~----~----L---~----~~

bleaching technique were similar to those achieved by

14
the darkening technique (about 4 seconds for selection
o Restrained
in masked area and 1.4 seconds for selecting unmasked • Free
target).
Masking by screening
As shown in Figure 11, even the slightest level of
screening tested (Levell, 12% pixels screened out)
appeared to provide a strong cue: mean selection time
of unmasked targets changed from 3.9 seconds at Level
o screening to 1.5 second at Level 1 screening (p <
0.0001). Further screening did not appear to be much
more effective; mean selection time changed from 1.5 No Mask 12% 25% 37% 62% 75%
seconds at Level 1 to 1.38 seconds at Level 5 (not Figure 11: Task completion times when the target was
statistically significant). Subjectively, even the lowest (top curves) masked or (bottom curves) highlighted /
level of screening (level 1) made the task significantly unmasked using the screening technique.
easier (p < 0.0001). Further increase in screening level
made the task seem even easier (p < 0.05) (Figure 12).
As the percentage of pixels masked out increased,
the readability of the masked area was expected to
decrease. Indeed, T-tests showed significantly
different ratings from lower levels to higher levels (e.g.
p < 0.05 from Level 2 to Level 4) (see Figure 12).
However, the time of selecting the target word from the
masked area remained flat except at the extreme Level
5 (Figure 11). From Level 0 to Level 4, mean selection
time changed from 3.9 to 4.1 seconds (not statistically
significant). Only Level 5 was significantly different
from the rest (p < 0.0001). This once again showed
users' ability to overcome increased difficulty without
significantly reducing time performance.
Overall, a wide range of screening, from 12% of
pixels to 62% pixels being masked out, produced the
optimal performance (again 4 seconds for selection in
masked area and 1.4 seconds for selecting unmasked Figure 12: Subjective ratings with the screening
target). technique. Horizontal axis is the degree of screening.
+5 is "extremely easy" and -5 is "extremely difficult".
4. DISCUSSION
All three techniques used in the second experiment The range of such adequate masking levels that can
were effective when adequate levels of masking were provide strong cues to direct users' attention without
selected. The highlighted target selection time at such readability loss in the masked area differed among the
levels was around 1.4 seconds for all three techniques, three techniques. The flat area of the performance
much faster than with no masking (around 4 seconds). curve where performance did not differ significantly in
Bear in mind that motor control (moving cursor to this experiment was -20 to -80 with the darkening
target) was involved in target selection. The visual technique, -45 to -75 with the bleaching technique, and
difference between fmding a pop-out target and 12% to 62 % for the screening condition. Bleaching
searching for the same target without masking cues was provided a narrower optimal range than the other two
much greater than 4.0: 1.4, the ratio of response times techniques. Darkening and screening both worked well,
for the two conditions. Furthermore, for several levels although they offer very different masking appearances
of each mask, the time to fmd an item in the masked and require different implementation mechanisms.
area was similar to Level 0 (no mask), suggesting that Although subjective data tend to vary from
the readability of the masked areas had been well individual to individual, the rating data collected in the
preserved. experiments did offer information not revealed by the
66 Part Two Technical Sessions

perfonnance data. The rated difficulty changed faster important, while allowing the user act according to his
than the perfonnance measurement as masking level own will.
moved to the two extrema. It is interesting that this
increase in subjective difficulty did not correspond to 6. ACKNOWLEDGMENTS
immediate perfonnance loss. Subjects tended to We would like to thank Ron Barber, Paul Maglio
compensate for the increased difficulty, perhaps by and other members of the USER (User System
exerting extra effort. See (Wickens, 1992) for related Ergonomics Research) group at the IBM Almaden
theories of perfonnance and resources. Research Center for their contributions to the
Note also that the experimental scenario is simpler development of this research.
than a typical application where both the visual features
and the user's cognitive activities are much more REFERENCES
complex. A narrowed range of effective masking level
is expected in more complex applications.
D'Zmura, M. Color in Visual Search. Vision Research,
31,6(1991), 951-966.

5. CONCLUSION Harrison, B., and Vicente, K. An Experimental

Two experiments were conducted to evaluate Evaluation of Transparent Menu Usage.
means of drawing user's attention to various parts of a Proceedings of CHI 96: Human Factors in
computer dialog. Four masking techniques were Computing Systems (1996), 391-398.
studied. One of them, blurring, did not appear to be as
effective as the rest. Two techniques, darkening and Julesz, B., and Bergen, J. R. Textons, the Fundamental
screening, can provide a wide range of masking levels Elements in Preattentive Vision and Perception of
that could produce strong pre-attentive visual cues to Textures. The Bell System Technical Journal, 62, 6
pop-out the targets and still allow the user to read the (July-August, 1983), 1619-1645.
masked area. Masks with as little as 12 % pixels being
screened out could produce a strong pop-out effect. Rogowitz, B. E. The Human Visual System: A Guide
The studies presented here indicated that important for the Display Technologist. Proceedings of the
information can be made to pop-out using darkening or SID, 24, 3 (1983), 235-252.
screening masks. Darkening or screening the less
relevant parts of the user interface provides strong Selker, T. COACH: A teaching agent that learns.
attention cues to the important information. Yet the Communications of the ACM 37, 7 (1994),92-99.
non-important information can still be accessed. Such a
way of directing a user's attention by masking is a Wickens, C. D. Engineering Psychology and Human
graceful human-machine communication method: Performance, Second Edition (1992).
telling the user what the user interface agent "thinks" is HarperCollins Publishers, Inc.