Enhancing the Design of Web Navigation Systems: The Influence of User Disorientation on

Engagement and Performance

Author(s): Jane Webster and Jaspreet S. Ahuja
Source: MIS Quarterly, Vol. 30, No. 3 (Sep., 2006), pp. 661-678
Published by: Management Information Systems Research Center, University of Minnesota
Stable URL: http://www.jstor.org/stable/25148744 .
Accessed: 28/06/2014 07:19

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .
http://www.jstor.org/page/info/about/policies/terms.jsp

.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact [email protected].

Management Information Systems Research Center, University of Minnesota is collaborating with JSTOR to
digitize, preserve and extend access to MIS Quarterly.

http://www.jstor.org

This content downloaded from 193.142.30.124 on Sat, 28 Jun 2014 07:19:06 AM

All use subject to JSTOR Terms and Conditions
 Webster & Ahuja/EnhancingtheDesign ofWeb NavigationSystems

tMEfm I I CTI Research Article

^^ l\f

Enhancing the Design of Web Navigation Systems:

The Influence of User Disorientation on
Engagement and Performance1

By: Jane Webster Keywords: Disorientation, lost, engagement, flow,World

Queen's School of Business Wide Web, Internet,design,Web-based learning, electronic
143 Union Street commerce, electronic shopping,Web navigation, commercial
Kingston, ON K7L3N6 Web site, performance, efficiency, intentions
CANADA
[email protected]

Jaspreet S. Ahuja
a sense
132 Alfred Patterson Drive Readers need of context, of their place with

in an In paper docu
Markham, ON L6E1K9 organization of information.
"
CANADA ments this sense of "where you are is a mixture of

[email protected] graphic and editorial organizational cues supplied

by thegraphic design of the book, the organization
of the text,and thephysical sensation of the book as
an object. Electronic documents provide none of the
Abstract physical cues we take for granted in assessing
information. When we see a Web hypertext linkon
This paper draws on research from a wide literature base to thepage we have few cues towhere we will be led,
develop
a model
relating Web navigation systems, disorienta how much information is at theother end of the link,
tion, engagement, user
performance, and intentions. The and exactly how the linked informationrelates to the
model is tested inan experimental study examining theeffects current page. Even the view of individual Web
of one simple and twoglobal navigation systems. Although pages is restrictedfor many users. (Lynch and
well-accepted design guidelines were followed for thefirst Horton 2002)
global navigation system, itwas not superior to the simple
system. However, the second global navigation system On IBM's website, themost popular feature was the
resulted in lower disorientation than the simple system. search function, because the site was difficult to
Based on thestudy's results, twodesign guidelines togovern navigate. The second most popular feature was the
"
thedevelopment offuture Web-based systems are suggested. "help button, because the search technologywas
so ineffective. IBM's solution was a 10-week effort
to redesign the site....In thefirst week after the
redesign, use of the "help" button decreased 84
Ron Weber was the accepting senior editor for this paper. JohnGallaugher
percent, while sales increased 400 percent.
and Anthony R. Hendrickson served as reviewers. The thirdreviewer and the
associate editor chose to remain anonymous. 2003)
(UsabilityNet

MIS QuarterlyVol. 30 No. 3 pp. 661-678/September2006 661

This content downloaded from 193.142.30.124 on Sat, 28 Jun 2014 07:19:06 AM

All use subject to JSTOR Terms and Conditions
Webster & Ahuja/EnhancingtheDesign ofWeb NavigationSystems

IntroductionWKKKBm^^^^^^^^^m tion,getting lost,and determining the correctbuttons fornavi

gation (Benbunan-Fich 2001). In this study,we investigate
A common problem users face on theWeb is disorientation navigation support by examining the effects of simple and
(Danielson 2003; Head et al. 2000; Wen 2003; Yu and Roh global navigation systems on disorientation and performance.
2002) or the tendency to lose one's sense of location inaWeb
site. This can result in frustration,loss of interest,and amea Reducing disorientation not only has performance benefits but
surable decline inuser efficiency (McDonald and Stevenson also allows users to be more engaged in theirWeb inter
1998). Despite itspromise to produce complex, richly inter actions. Engagement, or the feeling thata system has caught,
connected, and cross-referenced bodies ofmultimedia infor captured, and captivated user interest(Jacques et al. 1995), is
mation, theWeb can also produce disorganized tangles of an importantgoal for system design (Mayes 1992) and may
haphazardly connected documents thatmake locating infor even encourage users to revisitWeb sites (Kim and Moon
mation difficult (Utting and Yankelovich 1989). In addition, 1998).
the organization of a site is an important influence onWeb
site effectiveness (Palmer 2002) and user attitudes (Chen and Although these represent important issues faced by Web
Wells 1999). Thus, researchers suggest thatwell-designed designers, littleempirical research has addressed user percep
navigation systems can help to reduce disorientation (Head et tions of disorientation (Olnick Kutzschan andWebster 2006),
al. 2000). improved navigation support (Park and Kim 2000), or
increased user engagement (Mallon andWebb 2000). Thus,
Navigation systems, or systems that support users' deter our study proceeds as follows. We begin by describing per
mination of paths throughWeb sites,may include features ceived disorientation, navigation, and engagement, distin
such as links to other pages, search facilities, directories, or guishing between related but distinct constructs. We then
sitemaps. According toNielsen, users have difficultyfinding present a model linking these constructs to performance and
informationand need support in the formof a strong sense of intentions by drawing on research from a variety of areas,
structure and place; these can be provided by a navigation including human factors, cognitive psychology, marketing,
system.Nevertheless, lack of navigation supportwas cited as and informationsystems.We present an experiment that tests
one of the 10most-common mistakes inWeb design inboth thismodel. We conclude by presenting implications for
1996 and 1999 (Nielsen 1996a, 1999b). Surveys indicate that practice and research.

over 60 percent of Web users are either "searching" or

"browsing" (Pitkow et al. 1998), reinforcing the importance
of presenting information in an easily searchable and locat
able manner. Consequently, designers have proposed thatany Theoretical Model I^HBBHIHBHHH
useable hypertext system should include an effectivemech
anism for navigation (Lightfoot 1997). Our model has threemain constructs: disorientation, naviga
tion systems, and We con
engagement. develop hypotheses

Staying oriented is frequently a problem in virtual environ

these constructs to user performance and future
necting
ments (Darken and Sibert 1993). Thus, thebasic rationale for intentions to use a system. For this study, we conceptualize

navigation support is to remind users where and atwhat level

as both search correctness and search time and
performance

they currentlyare inWeb sites (Spiller and Lohse 1998). In conceptualize future intentions similar to the marketing
1994, about 90 percent of commercial Web sites had poor construct of "intention to revisit" (Luna et al. 2002).
usability: they either did not have the information users
needed, or the informationwas not easily locatable (Nielsen
and Sano 1994). By 2002, Nielsen reported thatonly 49 per Perceived Disorientation
cent of sites followed usability guidelines (such as providing
search engines). Even thehighest-scoring site only followed Disorientation represents "perhaps the oldest (and arguably
66 percent of the guidelines (Netsavvy Communications themost devastating) problem ofWeb navigation" (Danielson
2002). According to a Zona Research Inc. (1998) survey, 28 2003, p. 154). Disorientation has been defined as the ten
a non
percent of Internet-savvy users reported that it
was either dency to lose one's sense of location and direction in
somewhat or extremely difficult to locate specific products, linear document (Ahuja and Webster 2001; Conklin 1987;
and 62 percent gave up looking forproducts on-line. Finally, Head et al. 2000). It has been called the "getting lost"
a protocol analysis concerning the usability of a popular phenomenon that occurs when "the user does not have a clear

greeting cardWeb site demonstratedmany navigation issues, conception of relationships within the system, does not know
including problems with finding desired links and informa his present location in the system relative to thedisplay struc

662 MIS QuarterlyVol. 30 No. 3/September2006

This content downloaded from 193.142.30.124 on Sat, 28 Jun 2014 07:19:06 AM

All use subject to JSTOR Terms and Conditions
 Webster & Ahuja/Enhancing the Design of Web Navigation Systems

ture,and finds itdifficult to decide where to look nextwithin (Smith 1996). In this regard, some research has foundweak
the system" (Woods 1984, pp. 229-230), and the "lost in relations between perceived disorientation and users' actions
hyperspace" phenomenon that occurs when users experience (Ahuj a andWebster 2001). Other research has demonstrated
"cognitive problems of finding theirway in the information that perceived disorientation relates to accuracy of mental
space" (Smith 1996, p. 365). models and task time,while actions do not (Otter and Johnson
2000). Thus, perceived disorientation represents a more
Disorientation has ledWeb users tomake such remarks as "I appropriate indicator. This assertion is consistentwith those
had specific places I wanted to go and couldn't understand made by researchers in the behavioral sciences, who recom

how to find them" (Jarvenpaa and Todd 1997, p. 74). mend that questionnaire measures represent more suitable
Disoriented users might reopen the same page (a single hyper methods of measurement for understanding users' internal
text document in theWeb site) repeatedly. Disorientation feelings (Sandelands and Buckner 1989).
might also takedifferentforms, such as those demonstrated in
Web user surveys describing difficulties in finding pages
known to exist, finding a page already visited, and visualizing
Navigation Systems
paths taken and paths thatcould be taken (Pitkow and Kehoe
1996).
Navigation has been defined as "a process of tracking one's
Disorientation is related to, but conceptually distinct from, position in a physical environment to arrive at a desired des
tination" (Cutmore et al. 2000, p. 224). Navigation, although
ease of use, or the "effort one experiences in the process of
originally denoting a physical process, has been extended to
carrying out tasks using a given system" (Davis et al. 1992, include the process of determining a path to be traveled by
pp. 1114-1115). Disorientation represents a feeling or state
of the user (e.g., "I didn't know how to get tomy desired any object through any environment (Darken and Sibert
location" [Ahuja andWebster 2001, p. 18]),while ease of use 1993). It "has emerged as a useful metaphor for information
access" (Isakowitz and Bieber 1995, p. 6).
refers to the user's perception of a system (e.g., "Learning to

operate [softwarename] would be easy forme" [Davis 1989,

Computer-based navigation has been defined as "the deci
p. 340]). For instance, users may perceive that they learned
sions and actions that contribute to a person's ability to find
touse aWeb site easily, but can still feel disoriented. That is,
and examine data organized in the computermedium" (Watts
theymay find iteasy tobecome skillful at using theWeb site, Perotti andWoods 1999, p. 270). More generally, ithas been
but they stillmay become lost and not know how to get to
conceived as "the creation and interpretationof an internal
theirdesired locations in the site. Empirical research further
mental model" (Spence 1999, p. 921). A mental model, in
suggests that disorientation and ease of use are related but
turn,has been described as the organization of knowledge
distinct constructs. In one study, differingmenu layouts
about how a system works or operates (Norman 1988).
related to disorientation but not to ease of use (Bernard and
Mental models may be more or less correct and may have
Hamblin 2003). In another study,disorientation and ease of
been formed spontaneously via experience or systematically
use loaded on separate factors in an exploratory factor
via training (Santhanam and Sein 1994). Correct mental
analysis; in a second study reported in the same article, two models users tasks
help complete effectively. Incorrect
a were
competing priori models compared using confirmatory
mental models can lead to breakdowns.
factor analysis,2 and a x2 difference testdemonstrated that the performance
Marchionini and Shneiderman (1988) propose that effective
"distinct yet correlated" model fit the data better (Ahuja and
Webster 2001). hypertextusage depends on the quality of themental models
users for systems.
develop

Our model incorporates perceived disorientation, rather than

measures of users' actions. Disoriented actions are Drawing on these definitions of navigation, we describe a
objective
Web navigation system as a system that is designed to aid
typicallymeasured through thenumber of pages opened and users in the creation and interpretationof an internalmental
thenumber opened more thanonce (McDonald and Stevenson
model thathelps themfind and examine data on aWeb site.3
1998). However, many pages may be opened for other
Next, we compare simple with global navigation systems.
reasons, such as exploring aWeb site; thus, employing users'
actions as a proxy for disorientation can be problematic

A Web navigation systemmay be as straightforward as

providing navigation
bars (that is, specific areas on each page with links to other pages). Web
The firstmodel tested the hypothesis that disorientation was distinct yet
navigation systems are browser-independent; that is, they exist in addition to
negatively correlated with ease of use; the second thatdisorientation and ease any navigation buttons, such as "Back" or "Home," on common browsers
of use were one construct. such as Netscape or Explorer .

MIS QuarterlyVol. 30 No. 3/September2006 663

This content downloaded from 193.142.30.124 on Sat, 28 Jun 2014 07:19:06 AM

All use subject to JSTOR Terms and Conditions
Webster & Ahuja/Enhancing the Design of Web Navigation Systems

Simple navigation systems present local links to other pages because many had navigation links inmore thanone part
in the site (such as "previous" and "next"). Global navigation of the screen)
systems provide overviews of the entire site (such as site
maps) on each page (Park and Kim 2000). Consider a site none had stationary navigation links when scrolling
with threemain sections: About Us, Catalog, and Contact Us. through a screen
If the site is small, a simple navigation system could be
implemented by putting a linkon every page back to the front 77 percent had navigation links on all pages
page. On the main page for each of those sections, or
"second-level pages," a navigation bar could link to the site's 70 percent had consistent displays
frontpage (first level) and theother two sections' frontpages
(second levels). In contrast, a global navigation systemmight These popular sitesmay have been created by professional
include navigation bars with links toAbout Us, Catalog, and
design firms,but those created in-housemay have even lower
Contact Us on all pages, a detailed sitemap, a table of con statistics. For example, a study of 184 nonprofit organiza
tents, and a local search engine (i.e., one that searches only tions revealed that only 30 percent had a sitemap, only 33
the contents of thatparticularWeb site).4
percent had good or excellent search facilities, 32 percent did
not provide informationconcerning theuser's location in the
Global navigation systems are not yet the norm. Liu et al.
site, and 40 percent did not include clear navigation links
(1997) found thatonly 40 percent of company sites surveyed
(Cukier andMiddleton 2003). Thus, many Web sites do not
provided search facilities. Moreover, only 30 percent include all of the functionalityof global navigation systems.
provided an index or directory. Further, Spiller and Lohse
(1998) reported that only 6 percent offered product search,
and only 4 percent of sites had a site index. We further Although not yet a general practice, designers have proposed
that a visual representation of the entire systemwill aid in
investigated these statisticsby examining popularWeb sites.5
We found that navigation (Nielsen 1999a), and researchers are beginning to
explore this area. Researchers have examined the effects of
34 percent had a sitemap hyperlink annotations that represent Internet connection
speeds ("traffic lights" [Campbell and Maglio 1999]), the
81 percent had a search facility impacts of a historymechanism (Head et al. 2000), the effects
of "add-on" links (such as horizontal links to neighboring
93 percent had some type of navigation (70 percent had pages [Kim and Yoo 2000]), and the consequences of add-on
a "bar" and 23 percent had a "tree" type of navigation) links in conjunction with a historymechanism (Park and Kim
2000). However, these studies do not examine the effects of
most had navigation links at the top of the screen (89 other global navigation system characteristics, such as
percent had navigation at the top of the screen, 51 detailed sitemaps or search functions.
percent had partial navigation links at the bottom of the
screen, 34 percent had some navigation at the leftof the
screen - these numbers do not add up to 100 percent
Engagement

Users are engaged in a systemwhen it "holds theirattention

melsen (2000) points out that users often examine Web content while
and they are attracted to itfor intrinsic rewards" (Jacques et
ignoring global navigation elements. In spite of this, he cautions that
al. 1995, p. 58). Users describe theirexperiences when inter
navigation elements should not be removed fromWeb pages, proposing
instead that users will notice these elements when they need them. Rather, acting with engaging systems as feelings that the system has
he suggests that global navigation should be limited to five or six useful
caught, captured, and captivated their interest (Jacques et al.
elements, including features such as search functions, overviews of the
current region, and links to levels above the current location.
1995).6 Engagement is similar to flow, a state representing
the extent of pleasure and involvement in an activity
To conduct the coding, we hired an HCI graduate student to code themost (Csikszentmihalyi 1975). More specifically, flow has been
popular U.S. Web sites from home and work listed in the JupiterResearch described as a multidimensional construct encompassing
report, "Top 50 U.S. Web and Digital Media Properties of September 2002" perceptions of user control, attention focus, arousal of curio
(http://www.jupiterresearch.conVbin/item.pl/home). Of the 50 sites listed,
three requested personal information such as SIN number; thus, 47 of the 50
sites were coded. This student had not been involved with our study, nor was
she aware of the purpose of the coding. We asked her to code a variety of Like engagement, involvement may also exhibit focused attention; however,

design issues, ranging from "site map" (present or absent) to "location of involvement may occur because of task demands or deadlines and thusmay
as examples for not be enjoyable (Sandelands and Buckner 1989). In contrast, engagement
navigation" to "consistency of displays"; we coded two sites
her, and she coded the remainder independently. includes intrinsic interest.

664 MIS Quarterly Vol. 30 No. 3/September 2006

This content downloaded from 193.142.30.124 on Sat, 28 Jun 2014 07:19:06 AM

All use subject to JSTOR Terms and Conditions
 Webster & Ahuja/Enhancing the Design of Web Navigation Systems

I User \
^-^ A Performance )
^\H2 J
f _-\
I Navigation \?-' N.
I SystemsJ WAX
/
\H6

\( \ H5 / c
Future \\
_ Engagement /
^~-^^ V J
1-._
J Intentions
]
toUse
/N,
^-"-^""^ \^ ^y.-_ V /
/ Perceived \-" X^_
_^x
I Disorientation i-"
\v J H7

Figure 1. Model of Navigation Systems' Effects on Disorientation and Outcomes of

Searching Web Sites

sity,and intrinsic interest ina computer interaction (Webster than simple navigation systems. As described earlier, a
etal. 1993). common cause of disorientation is determiningwhere a link
leads or being able to return to a document already visited
Engagement has been characterized as flow without user (Utting and Yankelovich 1989; Wen 2003; Woods et al.
control, or
perceptions of attention focus, curiosity, and intrin 1990). This has led designers to propose thatWeb browsing
sic interest(Chapman et al. 1999). Engagement is a subset of can be improved by maintaining the context around the
flow and represents a more
passive state. For instance, one hypertext being viewed through the use of well-designed
may be highly engaged inobserving (but not controllingwhat navigation systems (Newfield et al. 1998).
is happening in) a movie. Similarly, exploration-based
training incorporatesmore user control than instruction-based Global navigation systems provide a visual representation of
trainingbecause the user sets thepace of the training. Thus, the entire system. We suggest that these visual representa
the formerwill more likely result in flow than the latter tions should reduce users' cognitive overload resulting from
(Chapman et al. 1999). Analogously, one would expect that the additional effort and concentration needed tomaintain
users exploring a Web site would experience more control several tasks or trails at one time (Conklin 1987). We pro
(than those performingdirectedWeb searches) because these pose thatusers of global navigation systemswill relymore on
users would be setting thepace and direction of theirexplora
recognition than recall by identifyinga piece of information
tion. Thus, engagement appears to represent a more appro and referencing it to informationalready stored in long-term
priate construct than flow when the user experiences less memory (Wickens 1992). For example, as we move through
control, such as might occur in directed information searches.7 aWeb sitewith a global navigation system,we may recognize
the props that are present, such as buttons, icons, and maps,

and use themwith relative ease. This process is passive, and

Development of Hypotheses we rely on recognition memory to tell us what we are
experiencing. The use of anchors, or invariantfeatures of the
Figure 1 shows themodel to be tested.We firstpropose that Web site, helps reduce the load on users' cognitive resources
global navigation systems will result in less disorientation (Tognazzine 1998;Woods 1984). Therefore, global naviga
tion systems should help users recognize their positions,
therebydecreasing cognitive overload and disorientation.
'We incorporated engagement, rather than flow, in thismodel because the
users inour study were directed to find specific information and were given
In contrast,when users travel to deeper levels of aWeb site
a short period of time to do so. The users were not expected to have much

opportunity to experience the control aspect of flow. using a simple navigation system, theprevious linksdisappear

MIS QuarterlyVol. 30 No. 3/September2006 665

This content downloaded from 193.142.30.124 on Sat, 28 Jun 2014 07:19:06 AM

All use subject to JSTOR Terms and Conditions
Webster & Ahuja/Enhancing the Design of Web Navigation Systems

and are replaced by new links. To travelback, theuser must help orient users and increase their efficiency. Users can
click on a "Back" or "Home" link (Yu and Roh 2002). This travel to and find desired informationmore quickly (Darken
results in "volatile" interactionwith theWeb site, because and Sibert 1993; Yu and Roh 2002). Users become impatient
users encounter differing navigation links over a short time with inefficientWeb sites and are inclined to go elsewhere
period (Danielson 2003). Navigational volatility, or "the (Nielsen andWagner 1996). Research has also shown that
number of hyperlinks appearing on the destination page that users browse more quickly with a global than a simple
did not appear in the same screen location as on the source navigation system (Yu and Roh 2002). Moreover, they are
page" correlates with perceived disorientation (Danielson less likely to abandon information-seeking taskswhen a site
2003, p. 141). Further, because simple navigation systems map is present (Danielson 2002). Thus, we propose
provide fewer site representations, they result in higher
reliance on recall than recognition. Recall involves active, Hypothesis 2: Global Web navigation systems result
deliberate retrieval of information from long-term to short in higher user
performance than simple navigation
termmemory (Trumbo 1998). Short-termmemory, however, systems.
has limited capacity (Head et al. 2000) Cognitive resources
used for planning where to go and keeping track of position Disorientation should relate to users' perceptions of engage
can become overloaded (Morris and Hinrichs 1996). As a ment and ultimately theirperformance and intentions to use
aWeb site. Although littleempirical research has addressed
result, the cognitive resources available for comprehension
and understandingWeb sitematerial will be reduced (Thuring engagement in computer interactions,we can draw on related
et al. 1995; Yu and Roh 2002). This will result in higher research to propose how engagement relates to other con
structs in our model. As described above, engagement is
cognitive loads for simple navigation systems.
made up of three dimensions: attention focus, arousal of
We propose thatglobal navigation systems help createmore curiosity, and intrinsic interest. Research examining these
accurate mental models by giving users a feel for the structure dimensions should help to explain how engagement might
relate to other constructs in themodel.
ofWeb sites as theymove through them. This is analogous
to becoming increasingly familiar with a geographical
Users who feel lostwill experience less attention focus while
environment, such as a citywhere we newly reside (Thorn
searching theWeb. Practitioners propose that engagement
dyke 1981). Through accurate mental models, users should with aWeb sitewill be linked to how successfully the user
be able todetermine their locations and what alternatives they
can move across the pages and through the space (Fleming
have to progress to desired pages. In short,a well-designed
1998). Further, they suggest thatdisoriented users who are
conceptual model should reduce users' memory loads (Otter unable to find the information theywant cannot become
and Johnson 2000) because it provides a blueprint for the
engrossed inwhat they are doing (e.g., Spool et al. 1999).
Web site (Yu and Roh 2002).
Thus, higher disorientation should lead to lower perceptions
of engagement.
A global navigation system could also add to the user's
cognitive load because it contributes to the amount of infor
Hypothesis 3: Higher perceived disorientation
mation presented to users (Yu and Roh 2002). On balance, results in lower engagement with a Web site.
however, we propose that cognitive load and disorientation
decrease with well-designed global navigation systems. We propose thatengagement will relate positively to perfor
we
Therefore, propose mance because the increased focus of attention should lead to
more effectiveWeb searches. Research on flow and related
Hypothesis 1: Global Web navigation systems result constructs supports this assertion. For example, researchers
in lowerperceived disorientation than simple navi have proposed that Web flow should be a central construct in
gation systems. explaining users' Web shopping experiences (Oinas
Kukkonen 2000). Research has shown that higher flow
Compared to global navigation systems, simple navigation relates to outcomes like extent of use and expected use of the
systems can result in reduced performance because they re Web, quantity of communication, and trainingoutcomes (e.g.,
duce the speed of completing a given task or increase the time Novak et al. 2000; Trevino andWebster 1992;Webster and
to find information (Conklin 1987; Fleming 1998; Nielsen Martocchio 1995). Research has also found positive relation
use
ships between (1)Web enjoyment and frequency ofWeb
1996a, These outcomes occur because simple naviga
1999a).
tion systems do not have a flexible path mechanism: users for entertainment (Atkinson and Kydd 1997) and (2) intrinsic
must pass through intermediate pages to locate information motivation and quality of output (Davis et al. 1992). Thus,
(Yu and Roh 2002). In contrast, global navigation systems we propose

666 MIS QuarterlyVol. 30 No. 3/September2006

This content downloaded from 193.142.30.124 on Sat, 28 Jun 2014 07:19:06 AM

All use subject to JSTOR Terms and Conditions
 Webster & Ahuja/EnhancingtheDesign ofWeb NavigationSystems

Hypothesis 4: Higher engagement in a Web site Methodology i

results in higherperformance.
The hypotheses were tested in an experiment that compared
Engagement should also positively affect intentions to use a three versions of the same on-line learningWeb site that
Web site in future because of users' intrinsic interest. differed in thenavigation systemused butwere consistent in
Engaged users enjoy theactivity or product,which may make content. Participants were assigned randomly to one of three
themwant to prolong the activity (Sandelands 1988) or use experimental conditions using a pretest-posttestcontrol group
the product again (Jordan 1998). As with flow, the positive design. This design allowed us to control forother unwanted
subjective experience becomes an important reason for influences. Before conducting the main experiment, the
performing theactivity (Csikszentmihalyi 1975) and intending experimental manipulations and measures were refined after
to revisit a site (Koufaris 2002; Luna et al. 2002). Thus, two pilot experiments. In themain experiment, participants
higher engagement should result in a more positive view of searched for specific informationon theWeb sites. The three
the computer interaction and higher motivation to interact
systemswere placed on a local server, reducing variance in
with the software in the future (Webster et al. 1993). Web page retrieval times. Each session lasted almost 1hour.
Two graduate research assistants conducted all experimental
Related empirical research helps to explain the expected sessions. A scriptwas used by the research assistants to give
positive relationship between engagement and intentions. instructions to the participants regarding the purpose of the
Users visitWeb sites thatfacilitate higher flowmore regularly
study and the details of the task. Data were collected by
and for longer periods of time (Nel et al. 1999) and report
(1) interaction logs recorded by server software, (2) perfor
higher intentions to revisit the sites (Luna et al. 2002). mance on a formal search, and (3) preexperimental and post
Further,higher flow relates to higher voluntary use (Webster
experimental questionnaires. (Appendix B outlines the
et al. 1993). Trainees who experience higher flow report
measures and their correlations.)
higher transferof training to theworkplace (Webster and
Martocchio 1995). Similarly, users who enjoy a computer
activity reporthigher intentions to use it in the future (Davis
et al. 1992), thosewith more positive attitudes toward aWeb Experimental Manipulations
site aremore likely to use it (Lederer et al. 1998), and those
with higher intrinsic enjoyment exhibit higher intentions to The Web site used for the experiment was adapted from an
return to theWeb site (Koufaris 2002). Thus, we propose ongoing Technical Writing course offered on theWeb by a
North American university. This course, designed to teach
Hypothesis 5: Higher engagement in a Web site principles of technical writing to nontechnical students,
leads to higher intentions to use theWeb site in the included complete books on various aspects of technical
future. writing. For the experiment, the amount of content was
reduced, resulting in three hierarchical levels; nevertheless,
Users who perform betterwhile searching aWeb site should this had no impact on the look and feel of the site.8 In the
be more likely towant to revisit theWeb site in the future. course, users were presented with an opening graphical view
Thus, we propose of theirvirtual office and various options such as Communi
cation, Assignments, Bookshelf, Information, and Calendar.
Hypothesis 6: Higher performance in using a Web
site leads tohigher intentions to use thatWeb site in We compared a simple navigation system and two global
thefuture.
navigation systems (see Figure 2, which diagrams the
navigation systems found at a second level for "Communi
Finally, users will lose interest inWeb sites when they cation," and Table 1,which compares the features in these
experience disorientation (McDonald and Stevenson 1998)
navigation systems). All systems used the course's original
because theybecome frustrated and cannot accomplish their
mixed topology structureand some typeof navigation system.
goals (Bessiere et al. 2003). Given that frustrationresults in The firstnavigation system consisted of the existing navi
outcomes such as withdrawal, we suggest thatmore dis
gation system of theWeb course. It contained simple links;
oriented users will be less likely towant to use theWeb site
that is, links to the following page and to lower-level sections.
in the future. Thus, we propose
Both global navigation systems allowed participants tomove

Hypothesis 7: Higher perceived disorientation

results in lower intentions to use theWeb site in the
A study of on-line stores revealed thatmost had three or fewer levels (Spiller
future.
andLohse 1998).

MIS QuarterlyVol. 30 No. 3/September2006 667

This content downloaded from 193.142.30.124 on Sat, 28 Jun 2014 07:19:06 AM

All use subject to JSTOR Terms and Conditions
Webster & Ahuja/Enhancing the Design ofWeb Navigation Systems

Simple Navigation System (Control):

( . ^
Communication

Newsgroup Mail List Chat Suggestions Instructor's Comments

V_J
Basic Global Navigation System (Condition#1):
Home Site Map ^\
f
Communication Calendar Assignments Instructor'sComments Bookshelf Course Info
f |
Newsgroup
Mail List
Chat
Suggestions
Instructor's Comments

Site Map

Search thisSite

V Home Communication Calendar Assignments Instructor'sComments Bookshelf Course Info

J

Enhanced Global Navigation System (Condition#2):

/ Virtual
office
\
/ +Communication
\
Newsgroup
Mail List
Chat

Suggestions
Instructor's Comments
+ Calendar
+
Assignments
+ Instructor's Comments
+ Bookshelf
+ Course Info.
Site Map

this
V Search Site /

Figure 2. Second-Level Screen Layouts of Navigation

Systems_I

668 MIS QuarterlyVol. 30 No. 3/September2006

This content downloaded from 193.142.30.124 on Sat, 28 Jun 2014 07:19:06 AM

All use subject to JSTOR Terms and Conditions
 Webster & Ahuja/Enhancing the Design of Web Navigation Systems

I Table 1. Comparison of Navigation Conditions

Simple Navigation System Basic Global Navigation System Enhanced Global Navigation System
(Control) (Condition #1) (Condition #2)
=
Hyperlinks only Navigation bars (top currentand Navigation tree (Explorer view in left
=
parent level, left child level,and frame)
bottom = top level)
Site-map (in top& left) Site-map (in leftframe)
Search form (on left) Search form (in leftframe)
Navigation disappears when scrolling Navigation can (partially)disappear Navigation always viewable when
(i.e., hyperlinks forprevious page when scrolling (site-map, search form scrolling (site-map, search formand
disappear) and navigation bars on leftand top; navigation tree always in leftframe)
bottom bars appear when scrolling to
bottom)

throughdocuments by clicking on links and navigating in a (7) making transitions to pages "graceful"; and (8) using a
hierarchy. Both were text-based and were available on all consistent display representation. Any changes recom

pages.9 Additionally, both systems included backtrack mended by the CHI Group were implemented before
facilities on all pages, cross-linking in certain locations, and conducting thepilot experiments described below.
search forms and site maps.10

The two global navigation systems,which we label basic and

Both global navigation systems were evaluated by the enhanced, differedmainly with respect to the position and
university's local Association for Computing Machinery design of the global navigation system. The basic global
CHI (Computer-Human Interaction) Group for consistency navigation system (condition #1) included navigation bars.
with usability guidelines. More specifically, theCHI Group It provided navigation features on the top, left,and bottom
compared the sites with commonly accepted guidelines of all pages. Thus, the navigation systemwas present on
developed by the HCI community (e.g., Fleming 1998; every page. Nonetheless, the systemmight not always be
Furnas 1999; Nielsen 1999b; Tognazzine 1998; Woods totally visible to a user (e.g., when scrolling down to the
1984). These guidelines included (1) providing a visual middle of a long page). For the enhanced global system
representation of the system; (2) having a broad rather than (condition #2), thenavigation systemwas available on every
a deep network ofmenus; (3) providing invariant features; page (on the left,using frames) in the form of a collapsible
(4) having facilities to backtrack and return to the parent tree structure (similar to the Explorer view in
level; (5) providing self-explanatory labels on thenavigation Windows ), rather than navigation bars. Because frames

system; (6) indicating the user's position in theWeb site; were used, the enhanced global navigation system was

always visible when scrolling down a long page. As

described below, the design of these global systems was
Graphics were incorporated in the systems, but not for the navigation based on the results of two pilot experiments,which found
systems. Text was selected over graphics for the navigation systems, that thebasic global navigation system could have negative
because text is recommended for persons with disabilities and provides
effects compared with a simple navigation system.
faster loading times (Lazar et al. 2003). A graphical navigation systemmay
look attractive, but an appropriate one can only be created by a competent
graphic designer. As noted by Nielsen (1999a), bloated graphic design is
theworst offender in terms of response times, and graphics should not be
used ifnot needed. Pilot Experiments

We chose to use sitemaps because some research comparing map displays Two pilot experiments were conducted. The first incor
with site maps (also called route lists) shows that site maps result in less
confusion. Display maps generally are graphical representations of a space porated two conditions: theWeb sitewith a simple navi
(like a map of a university campus), while site maps generally are gation system (the control group) and the same Web site
hierarchical representations of a space (like an organization chart). with a basic global navigation system (condition #1).
Although Thorndyke and Hayes-Roth (1982) expected that themap display
would be superior to a sitemap, they found the opposite: following a route
Participants were 21 graduate students fromone department.
helped reorient users when lost,whereas using a map display caused more
Although we (and our local CHI Group) believed thatwe
confusion. had designed the navigation system to be consistent with

MIS QuarterlyVol. 30 No. 3/September2006 669

This content downloaded from 193.142.30.124 on Sat, 28 Jun 2014 07:19:06 AM

All use subject to JSTOR Terms and Conditions
Webster & Ahuja/EnhancingtheDesign ofWeb NavigationSystems

usability guidelines, we found a drop inperformance and an None of the students had seen theWeb site prior to the
increase indisorientation with the global navigation system experiment.
(condition #l).n
Approximately 56 percent of theparticipantswere in the age
During interviews, some condition #1 pilot participants group of 17 to 20, and approximately 90 percent were 24 or
observed that theglobal navigation systemwas complicated younger. Most were female (60 percent), undergraduate stu
and difficult to use. For example, they felt that having to dents (80 percent), from theArts school (36 percent), with
scroll to the top or the bottom of a page to navigate was others from Applied Health Sciences, Engineering, Math,
undesirable. They suggested that this resulted in forgetting and Sciences. Most (88 percent) had spent at least 1 year
current positions and the various options available. Our using the Internet,which is normal for their age group
global navigation system apparently introducedmore navi (Pitkow et al. 1998). Over 80 percent had more than 9 years
gation volatility into the users' interactions (Danielson of education in the English language. Approximately 57
2003). To address these problems, another condition was percent ranked themselves as novices with respect to their
designed with a navigation tree (rather thana navigation bar) knowledge ofWeb design.
thatwas always stationaryon the leftof the screen (condition
#2).

In the second pilot experiment, six graduate students from Results

the same department pretested the experiment again with
condition #2. Results for this experiment indicated that Hypotheses were testedwith structural equation modeling
participants using the enhanced global navigation system (SEM) using AMOS (Arbuckle and Wothke 1999). The
found the site less disorienting. They achieved higher per navigation dummy variables, CI and C2 (described in
formance than those using either the simple or the basic Appendix A), were allowed to relate because, by design,
global navigation systems. dummy variables are partly redundant (Cohen and Cohen
1983, pp. 189-190).
Based on these pilot results,we decided to include all three
The AMOS analysis for thehypothesized model resulted in
navigation systems in themain experiment. This allowed us
to explore whether some global navigation systems (such as a good model fit (Hair et al. 1995): a x2 to degrees of free
condition #2)might positively affectoutcomes, while others dom ratio of 1.94, a goodness of fit index of .97, an adjusted
(such as condition #1)might undermine performance more goodness of fit index of .92, and a rootmean square residual
than even of .07). Figure 3 presents thepath coefficients for themodel.
simple navigation systems.
The hypotheses were testedby examining the significance of
the standardized path coefficients shown inFigure 3.
Participants
HI, which predicted thatusers of a global navigation system
Graduate and students from various would report lower disorientation than those interactingwith
undergraduate depart
ments at a North American university acted as participants. a system, was supported. As shown in
simple navigation
The only prerequisite forparticipation was that the students Table 2, those interactingwith the enhanced global naviga
be knowledgeable about using computers and familiarwith tion system reported the lowest disorientation. Interestingly,
theWorld Wide Web. Participants were recruitedbymaking thepath coefficients inFigure 3 show thatparticipants inter
brief presentations in courses with more than 100 students. actingwith thebasic global navigation system did not report
Of the approximately 1,200 students approached in class lower disorientation than those interactingwith the simple
=
rooms, 275 signed up, and 207 participated in the experi system. CI did not relate to disorientation (P -.06) while
=
C2 did (P -.29).
ment. Participants were paid $10 each. We chose to pay
participants rather than offer incentives based on perfor
mance. We felt thatofferingperformance incentiveswould H2 predicted that global navigation systems would out
be inconsistentwith students learning from a Web-based perform simple navigation systems. The enhanced global
system or customers looking formerchandise on aWeb site. navigation system demonstrated the highest performance
(thehighest score on thequiz and the lowest time),while the
basic global navigation system demonstrated the lowest
performance (Table 2). Path coefficients presented in
This finding is not uncommon. Woods (1984) also noted previous
research that has found an unexpected increase in disorientation with Figure 3 support this conclusion: CI related negatively to
= =
systems designed to follow usability guidelines. performance (P -.33), while C2 did not (P .16).

670 MIS QuarterlyVol. 30 No. 3/September2006

This content downloaded from 193.142.30.124 on Sat, 28 Jun 2014 07:19:06 AM

All use subject to JSTOR Terms and Conditions
 Webster & Ahuja/Enhancing the Design of Web Navigation Systems

Internet
^+>
[ f Experience?-?._

C1 33 ~?"-?
"17 "* User
Navigation Systems: C2. 16
-*^*??-^ Performance

/ \
C1: -.06
.35/^ \-.02
C2: -.29 / \

\ c */ -65
\ . SO^Engagement '-^ Future
\ Intentions
* ^^^^ _^_
Perceived to Use
-7"27-"
Disorientation

Notes: Bolded path coefficients significant at p < .05.

C1 and C2 are two dummy variables representing the three navigation systems

Figure 3. Navigation Systems' Effects on Disorientation and Outcomes of Web Site

Searching_I

Table 2. Comparison of Means across Navigation Conditions I

Means Simple Navigation Basic Global Navigation Enhanced Global
(Controlling for Internet System Navigation
System System
Experience) (Control) (Condition #1) (Condition #2)
Perceived Disorientation 3.393.24 2.65
User Performance:
Number of Correct Answers 6.49 6.14 6.89
User Performance:
Time in Seconds 1199.38 1095.33
1359.53

Four of the remaining five hypotheses were supported. H3, tation (Otter and Johnson 2000; Park and Kim 2000). Our
which predicted that lower disorientation would relate to results indicate that the design of navigation systems can
=
higher user engagement, was supported (P -.50). H4, which affect user performance and disorientation. Moreover, they
predicted that engagement would relate positively to indicate thatstrongrelationships exist between disorientation,
was =
performance, supported (P .35). H5, which predicted engagement, performance, and intentions to use aWeb site.
that engagement would relate positively to intentions to use
theWeb site in the future,was supported (P = .65). H6,
which predicted thatperformance would relate positively to
Implications forPractice
intentions to use theWeb in the future,was not supported (p
=
-.02). Finally, H7, which predicted thathigher disorienta Our study results demonstrate that simple navigation systems
tionwould result in lower intentionswas supported (P = -.27). can be better than some global navigations systems.We were
initially surprised to find in our pilot experiments that
adhering to usability guidelines for a global Web navigation
Discussion and Conclusions system does not necessarily result in less disorientation and
higher performance than a simple navigation system.
This study responds to calls for more research aimed at Similarly, inourmain experiment,we found no difference in
understanding users' Web behaviors (e.g., Pereira 2000) and disorientation between the simple navigation system and the
specifically the effects ofWeb system designs on disorien basic global navigation system. Moreover, we found better

MIS QuarterlyVol. 30 No. 3/September2006 671

This content downloaded from 193.142.30.124 on Sat, 28 Jun 2014 07:19:06 AM

All use subject to JSTOR Terms and Conditions
Webster & Ahuja/Enhancing the Design ofWeb Navigation Systems

performance (task time) for the simple navigation system. plore pages, especially thosewith informationabout multiple
Nonetheless, our enhanced global navigation system resulted related products. Further,we did not examine the effects of
in lower disorientation than the simple navigation system and site content or graphics on facilitating user attraction. For
thebasic global system.12Recall, our basic and global navi sites designed to encourage users' exploratory behaviors, we
gation systems differed in twoways: the type of navigation suggest that designers measure users' perceptions of flow
feature (bars for the basic and trees for enhanced), and the (which includes user control) rather than theirengagement.
placement of thenavigation features. Additionally, thenavi
gation features could disappear when scrolling througha long
page in our basic global navigation system,while theywere Limitations and Suggestions for
always visible in our enhanced global navigation system. It Future Research
seems thatnavigational volatilitywas reduced in theenhanced
system. Thus, it appears thatusing a navigation tree rather We studied university students interactingwith an on-line
thannavigation bars and keeping thenavigation features con
learningWeb site. Although we were able to control for
tinuously in view are importantways of reducing disorien Internet in our analyses, students may be younger
experience
tation and increasing performance. and more skilled with theWeb than employees interacting
with intranets, job applicants submitting their resumes to
To keep the navigation features continuously in view, our
organizations' Web sites, or customers ordering products or
enhanced global navigation system used frames. Neverthe services via theWeb. Nevertheless, we believe that our
less, there are some with frames, such as book
problems research results have implications not only for the design of
marking, printing, and the accuracy of URLs13 (Nielsen educational and training sites but for job-applicant and
1996b). In the short run, designers will need toweigh the customer sites.
reduced disorientation benefits of frames against problems
that can arise with the use of frames. In addition, our research
We compared one simplewith two global navigation systems.
suggests that designers should be cautious when designing Because both of our global navigation systems consisted of
navigation systems: they should not implementmore com search engines, site maps, and navigation bars or trees, we
plex global navigation systemswithout firstcomparing users' could not determine the relative importance of each feature.
disorientation and performance when using simpler systems. Future research should examine the relative importance of
Doing so may inadvertently result in negative outcomes for
these navigation features to reduced disorientation and
users.
increased performance. Further,we did not study theorgani
zation of the navigation content. Researchers are
system's
We recommend that designers measure perceived disorien
beginning to examine users' personal ontologies forWeb
tation before and aftermodifying Web designs. Perceived
navigation (e.g., Chaffee and Gauch 2000) and this represents
disorientation instruments likeAhuja andWebster's (2001) another direction for future research
promising (Gruninger
can be completed in a fewminutes and are simpler to collect and Lee 2002).
and than users' actions, verbal or videos.
analyze protocols,
In spite of this, designers will still want to collect users' We were intriguedby the fact thata global navigation system
actions for certain time-critical tasks (event-driven, high-con thatadheres tousability guidelines might not be superior to a
such as nuclear
simple one. As indicated earlier, other research supports this
sequence domains, power plant operations,
and mission
finding (Woods 1984). This points to the need for further
patient monitoring systems, space operations

[Woods andWatts 1997]). development of theory-based usability guidelines. Based on

our study results,we suggest two guidelines for investigation.
This study emphasizes the design of navigation systems to
First,we propose that the entire hierarchy of a site be present
enhance user On the other mar
searching efficiency. hand, on every page with expandable and collapsible sections or
ketersmay have goals other thanuser efficiency?they may branches of thehierarchy (aswith navigation trees). This will
want to create that encourage users to stay and ex
designs act as a conceptual model to allow users to travel forward or
backward to any level and thereby help to increase the
accuracy of theirmental models. Second, we propose that
1
familiaritywith an enhanced global navigation system through navigation features be available in fixed positions on every
Participants'
theiruniversityWeb site isnot an alternative explanation for the findings; the
page (as with frames). Doing so should reduce users' cogni
university did not have an enhanced system when the study was conducted. tive loads. These two guidelines are exploratory and emerge
from our study results. Future research needs to study their
Our participants did not need to bookmark, print, or reference URLs, and
therefore these were not issues in the current experiment. efficacy forWeb design.

672 MIS QuarterlyVol. 30 No. 3/September2006

This content downloaded from 193.142.30.124 on Sat, 28 Jun 2014 07:19:06 AM

All use subject to JSTOR Terms and Conditions
 Webster & Ahuja/EnhancingtheDesign ofWeb NavigationSystems

A strengthof this study's design was the consistent content, References

task,and structureof theWeb sites. Nevertheless, we focused
only on formal searching, a relatively intensive form of Ahuja, J. S., and Webster, J. "Perceived Disorientation: An

extracting informationon a topic, because thisbest represents Examination of a New Measure toAssess Web Design Effective
users' actions during focused searching on electronic com ness," Interacting with Computers (14:1), 2001, pp. 15-29.
merce sites (Park and Kim 2000). However, informal Arbuckle, J. L., and Wothke, W. AMOS 4.0 User's Guide, SPSS

searching is themost frequent search mode for employees Inc., Chicago, IL, 1999.

using theWeb in theirjobs (Choo et al. 2000). For informal Atkinson, M., and Kydd, C. "Individual Characteristics Associated
WithWorld Wide Web Use: An Empirical StudyofPlayfulness
searching,we expect that the relation between users' actions
and perceived disorientation will be even weaker than for and Motivation," The DATA BASE for Advances in Information

formal searching. In support of this claim, Head et al. (2000) Systems (28:2), 1997,pp. 53-62.
R. Protocol to Evaluate
compared users' in general information retrieval
actions Benbunan-Fich, "Using Analysis the
of a Commercial Web & Manage
(exploratory searches) with specific information retrieval. Usability Site," Information
ment 151-163.
They found thatusers delved more deeply intoWeb sites and (39), 2001, pp.
and Hamblin, C. versus Indexed Menu
revisited pages more often in general information retrieval. Bernard, M., "Cascading

Thus, perceived disorientation should represent a useful tool Design," Usability News (5:1), 2003 (available online at
forassessing Web sites forboth formal and informal searches. http://psychology.wichita.edu/surl/usabilitynews/51/menu.htm).
Bessiere, K., Ceaparu, I., Lazar, J.,Robinson, J., and Shneiderman,
Further, because the task used in this study gave users little
B. "Social and Psychological Influences on Computer User
chance to exercise control, we assessed rather
engagement
Frustration," inMedia Access: Social and Psychological Dimen
than flow. On theother hand, we would expect users to exhi
sions of New Use, E. Bucy and J.Newhagen
bitmany more exploratory behaviors and to have many more Technology (eds.),
Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 2003, pp.
chances to exercise control during informal searching. In
169-192.
such cases, flow would represent a more con
appropriate
Campbell, C, and Maglio, P. P. "Facilitating Navigation in Infor
struct. Thus, future studies should examine the relation
mation Spaces: Road-Signs on the World Wide Web," Inter
between disorientation and flow during informal searching.
national Journal of Human-Computer Studies (50), 1999, pp.
309-327.
In summary, disorientation represents a common
problem
Chaffee, J., and Gauch, S. "Personal Ontologies forWeb Naviga
faced by Web users, and Web designs can have powerful
tion," in Proceedings of theNinth International on
Conference
effects on disorientation. With increasing user sophistication
Information and Knowledge Management, ACM Press, New
and advancing technologies, some argue thatdisorientation
York, November 6-11, 2000, pp. 227-234.
will lessen over time. Conversely, others propose thatdis
Chapman, P., Selvarajah, S., and Webster, J. "Engagement in
orientationwill remain a critical issue: itwill become trans
Multimedia Training Systems," in Proceedings the 32nd
of
formed from how to get to a location to how to return to a Hawaii International Conference on System Sciences, R. H.
location (Wen 2003). To reduce disorientation and enhance Jr. (ed.), IEEE
Sprague, Computer Society Press, Los Alamitos,
performance, we have proposed two theoretically driven CA, 1999 (CD-ROM).
design guidelines, but many opportunities exist for extending and Wells, W. D.
Chen, Q., "Attitude Toward the Site," Journal of
this research. Future research should continue todevelop and
Advertising Research, September-October 1999, pp. 27-37.
testdesign guidelines based on theory. C. W., and Turnbull, D. theWeb: An
Choo, Detlor, B., "Working
Empirical Model ofWeb Use," inProceedings of the 33rdHawaii
International Conference on System Sciences, R. Sprague (ed.),
IEEE Computer Society Press, Los Alamitos, CA, January 4-7,
Acknowledgments 2000.

Cohen, J., and Cohen, P. Applied Multiple Regression/Correlation

EvNet, theNetwork for the Evaluationof Education and Analysis for the Behavioural Sciences, Erlbaum, Hillsdale, NJ,
Training Technologies, provided financial support for this 1983.

research. We would like to thankPaul Beam for providing Conklin, J. "Hypertext: An Introduction and Survey," IEEE
access to his courseWeb site; Saibal 17-41.
Ray, Tracy Chen, and Computer (20:9), 1987, pp.
M. Boredom and Anxiety,
Adrienne Olnick Kutzschan for theirresearch assistance; and Csikszentmihalyi, Beyond Jossey-Bass,
San Francisco, CA, 1975.
Derek Chapman, Ji-YeMao, David Zweig, and threeanony
Cukier, W., and Middleton, C. A. theWeb Presence of
mous reviewers for theirhelpful comments on earlier versions "Evaluating
Voluntary Sector Organizations: An Assessment of Canadian
of thispaper.
Web Sites," IT & Society (1:3),Winter 2003, pp. 102-130.

MIS QuarterlyVol. 30 No. 3/September2006 673

This content downloaded from 193.142.30.124 on Sat, 28 Jun 2014 07:19:06 AM

All use subject to JSTOR Terms and Conditions
Webster & Ahuja/EnhancingtheDesign ofWeb NavigationSystems

Cutmore, T. R. H., Hine, T. J.,Maberly, K. J., Langford, N. M., and Kim, J., and Yoo, B. "Toward the Optimal Link Structure of the

Hawgood, G. "Cognitive and Gender Factors Influencing Cyber Shopping Mall," International Journal of Human
Navigation in a Virtual Environment," International Journal of ComputerStudies (52), 2000, pp. 531-551.
Human-Computer Studies (53), 2000, pp. 223-249. Koufaris, M.
"Applying the Technology Acceptance Model and

Danielson, D. R. "Transitional Volatility inWeb Navigation," IT & Flow to Online Consumer Behavior," Information
Theory
Society (1:3),Winter 2003, pp. 131-158. Systems Research (13:2), June 2002, pp. 205-223.
Danielson, D. R. "Web Navigation and the Behavioral Effects of Lazar, J., Bessiere, K., Ceaparu, I., Robinson, J., and Shneiderman,
Constantly Visible Site Maps," Interacting with Computers B. I'm Lost: User Frustration inWeb Navigation," IT &
"Help!
(14:5), October 2002, pp. 601-618. Society (1:3),Winter 2003, pp. 18-26.
Darken, R. P., and Sibert, J. L. "A Toolset forNavigation inVirtual Lederer, A. L., Maupin, D. J., Sena, M. P., and Zhuang, Y. "The

Environments," in Proceedings of the Sixth Annual ACM Role ofEase ofUse, Usefulness andAttitude in thePredictionof
Symposium on User Interface Software and Technology, S. E. World Wide Web Usage," inProceedings of the 1998 ACM
Hudson, R. Pausch, B. V. Zanden, and J. D. Foley (eds.), ACM SIGCPR Conference,RituAgarwal (ed.),ACM Press,New York,
Press, New York, 1993, pp. 157-165. 1998, pp. 195-204.
Davis, F. D. "Perceived Usefulness, Perceived Ease of Use, and J.M. "A New forWebsite an
Lightfoot, Technique Design Using
User of Information MS
Acceptance Technology," Quarterly Interactive Visualization Hypertext Tool," Proceedings of the
(13:3), 1989,pp. 319-339. 1997ACM SIGCPR Conference,ACM Press,New York, 1997,
Davis, F. D.,
Bagozzi, R. P., and Warshaw, P. R. "Extrinsic and 14-19.
pp.
Intrinsic Motivation to Use Computers in the Workplace," C. L., Arnett, K. L. M., and Beatty, R. M. "Web
Liu, P., Capella,
Journal ofAppliedSocial Psychology (22), 1992,pp. 1111-1132. Sites of theFortune500 Companies: Facing CustomersThrough
Fleming, J. Web
Navigation: Designing the User Experience, Home and Management
Pages," Information (31), 1997, pp.
O'Reilly & Associates, Sebastopol, CA, 1998. 335-345.
Furnas, G. W. "Effective View Navigation," in Readings in L. A., and de Juan, M. D. "Cross-Cultural and
Luna, D., Peracchio,
Information Visualization: Using Vision to Think, S. K. Card, J. Cognitive Aspects of Web Site Navigation," Journal of the
D. Mackinlay, and B. Shneiderman Kaufmann
(eds), Morgan Academy ofMarketing Science (30:4), 2002, pp. 397-410.
Publishers, Inc., San Francisco, 1999, pp. 589-596. Basic Design Prin
Lynch, P. J., and Horton, S. Web Style Guide:
M., and Lee. J. "Ontology: and Design,"
Gruninger, Applications ciples for Creating Web Sites (2nd ed.), Yale University Press,
Communications of the ACM (45:2), February 2002, pp. 39-41. New Haven, CT, 2002 (available online at http://www.
Hair, J. F., Anderson, R. E., Tatham, R. L., and Black, W. C. Multi
webstyleguide.com/index.html).
variate Data Analysis with Readings (4th ed.), Prentice-Hall, R. C, and Austin, J. T. of Structural
MacCullum, "Applications
Englewood Cliffs,NJ, 1995. Equation Modeling in Psychological Research," in Annual
Head, M., Archer, N., and Yuan, Y. "World Wide Web Navigation Review of Psychology (51), S. T. Fiske, D. L. Schacter, C. Zahn
Aid," International Journal of Human-Computer Studies (53), Waxler (eds.), Annual Reviews, Palo Alto, CA, 2000, pp.
2000, pp. 301-330. 201-226.
Isakowitz, T., and Bieber, M. "Special Section: Navigation in
Mallon, B., and Webb, B. "Structure, Causality, Visibility and
Information-Intensive Environments," Journal ofManagement inNarra
Interaction: Propositions for Evaluating Engagement
Information Systems (11:4), 1995, pp. 5-8. tive Multimedia," International Journal of Human-Computer
C. M., S., and Leitch, R. A. "Towards an
Jackson, Chow, Studies (53), 2000, pp. 269-287.
of the Behavioral Intention toUse an Information
Understanding Marchionini, G., and Shneiderman, B. Facts vs. Browsing
"Finding
System," Decision 1997, pp. 357-389.
Sciences
(28:2), in Hypertext
Knowledge Systems," Computer (21), 1988, pp.
J., and Carey, J. T. "Engagement as a Design
Jacques, R., Preece, 70-79.
for Canadian Journal of Educational
Concept Hypermedia, J. T. "The 'M-Word': Multimedia Interfaces and Their
Mayes,
Communications, Spring, 1995, pp. 49-59. Role in Interactive inMultimedia
Learning Systems," Interface
S. L., and Todd, P. A. "Consumer Reactions to Elec
Jarvenpaa, in Education, A. D. N. Edwards and S. Holland (eds.),
Design
tronic Shopping on theWorld Wide Web," International Journal
Springer-Verlag, Berlin, 1992, pp. 1-22.
of Electronic Commerce (1:2), 1997, pp. 59-88. R. J. "Effects of Text Structure and
McDonald, S., and Stevenson,
Jordan, P. W. "Human Factors for Pleasure in Product Use,"
Prior Knowledge of the Learner on Navigation in Hypertext,"
Applied Ergonomics (29:1), 1998,pp. 25-33. Human Factors (40:1), 1998, pp. 18-27.
K. G., and Sorbom, D. LISREL 7: A Guide to the
Joreskog, and Henrichs, R. Web A Different
Morris, M., Page Design:
and Applications IL, 1989.
Program (2nd ed.), SPSS, Chicago, Sunsoft Press/Sun Mountain
Multimedia, Microsystems, View,
H. H. "The Process of Causal Attribution," American
Kelly, 1996.
CA,
Psychologist (28), 1973,pp. 107-128. J.-P., and Davies, T. "Going with
Nel, D., van Niekerk, R, Berthon,
Kim, J., and Moon, J.Y. "Designing Towards Emotional Usability
the Flow: Web Sites and Customer Involvement," Internet
in Customer Interfaces: Trustworthiness of Cyber-Banking
Research: Electronic Networking Applications and Policy (9:2),
Interfaces," Interacting with Computers (10), 1998, pp.
System
1999, pp. 109-116.
1-29.

674 MIS QuarterlyVol. 30 No. 3/September2006

This content downloaded from 193.142.30.124 on Sat, 28 Jun 2014 07:19:06 AM

All use subject to JSTOR Terms and Conditions
 Webster & Ahuja/Enhancing the Design of Web Navigation Systems

Netsavvy Communications. "E-Commerce Web Sites are Getting Pitkow, J., and Kehoe, C. Trends in theWWW User
"Emerging
(Slowly) Better...," June 10, 2002 (available online at
Population," Communications of theACM (39:6), 1996, pp.
http://sellitontheweb.com/ezine/news0576.shtml). 106-108.

Newfield, D., Sethi, B. S., and Ryall, K. "Scatchpad: Mechanisms Pitkow, J.,Kehoe, C, and Rogers, J. "GVU's WWW User Survey,"
for Better Navigation in Directed Web Searching," in Pro Georgia Tech Research, Atlanta, GA, 1998 (available online at

ceedings of the 11thAnnual ACM on User http://www.gvu.gatech.edu/user_surveys/survey-1998-04/).

Symposium Interface
and Technologyx ACM Press, New 1 -8. Russell, D. W., Kahn, J. H., Spoth, R., and Altmaier, E. M.
Software York, 1998, pp.
J. "Is Navigation Jakob Nielsen's "Analyzing Data from Experimental Studies: A Latent Variable
Nielsen, Useful?," Alertbox,
Structural Equation Modeling Approach," Journal ofGounseling
January 9,2000 (http://www.useit.com/alertbox/20000109.html).
Nielsen, J. "Top Ten Mistakes inWeb Design," Jakob Nielsen's Psychology (45), 1998,pp. 18-29.
Sandelands, L. E. "Effects of Work and Play Signals on Task
Alertbox, 1996a (http://www.useit.com/ alertbox/9605.html).
J. "The Ten New Mistakes of Web Jakob
Evaluation,"Journal ofAppliedSocialPsychology (18:12), 1988,
Nielsen, Top Design,"
pp. 1032-1048.
Nielsen's Alertbox, May 30, 1999a (http://www.useit.com/
Sandelands, L. E., and Buckner, G. C. "Of Art and Work:
alertbox/990530.html). Aesthetic Experience and thePsychology ofWork Feelings," in
Nielsen, J. '"Top Ten Mistakes' Revisited Three Years Later,"
Research in Organizational Behaviour, L. L. Cummings and
Jakob Nielsen's Alertbox, May 2,1999b (http://www.useit.com/ B.M. Staw (eds.), JAI Press, Greenwich, CT, 1989, pp.105-131.
alertbox/990502.html). and Sein, M. K.
Santhanam, R., "Improving End-User Proficiency:
Nielsen, J. "Why Frames Suck (Most of the Time)," Jakob
Effects of Conceptual Training and Nature of Interaction,"
Nielsen's Alertbox, December 1996b (http://www.useit.com/
Information Systems Research (5), 1994, pp. 378-399.
alertbox/9612.html). a Practical Measure
Smith, P. A. "Towards of Hypertext Usability,"
Nielsen, J., and Sano, D. "SunWeb: User Interface Design for Sun
Interacting with Computers (4), 1996, pp. 365-381.
Microsystem's Internal Web," in Proceedings of the Second Spence, R. "A Framework for Navigation," International Journal
World Wide Web Conference: Mosaic and theWeb, Chicago, IL, ofHuman-Computer Studies (51), 1999, pp. 919-945.
October 17-20, 1994, pp. 547-557 (available at http:// P., and Lohse, G. L. "A Classification of Internet Retail
Spiller,
www.useit.com/papers/sunweb/). Stores," International Journal of Electronic Commerce (2:2),
Nielsen, J., and Wagner, A. "User Interface Design for theWorld
1998, pp. 29-56.
Wide Web," Proceedings of theCHI 96,ACM Press,New York,
Spool, J.M., Scanlon, T., Schroeder, W., Snyder, C, andDeAngelo,
1996, p. 330. T. Web Site Usability: A Designer's Guide, Morgan Kaufmann
Norman, D. The Psychology & Row,
of Everyday Things, Harper Publishers, Inc., San Francisco, CA, 1999.
New York, 1988. P. W. "Distance Estimation from Cognitive
Thomdyke, Maps,"
Novak, T. P., Hoffman, D. L., and Yung, Y.-F. "Measuring the CognitivePsychology (13), 1981,pp. 526-550.
Customer Experience in Online Environments: A Structural Thomdyke, P. W., and Hayes-Roth, D. "Differences in Spatial

Modeling Approach," Marketing Science (19:1), 2000, pp. 22-42. Knowledge Acquired from Maps and Navigation," Cognitive
Oinas-Kukkonen, H. "Balancing the Vendor and Consumer Psychology (14), 1982,pp. 560-589.
Requirements for electronic Shopping Systems," Information Thuring, M., Hannemann, J., and Haake, J.M. "Hypermedia and

Technology and Management (1), 2000, pp. 73-84. Cognition: Designing for Comprehension," Communications of
Olnick Kutzschan, A., and Webster, J. "HO as MIS," inHuman theACM(3S:8), 1995,pp. 57-66.
Interaction in Management Tognazzine, B. "First Principles of Interaction Design," 1998
Computer Information Systems:
Foundations. Advances inManagement (available online at
Information Systems, P. http://www.asktog.com/basics/
Zhang and D. Galletta (eds.), M. E. Sharpe, Inc., Armonk, New firstPrinciples.html).
Trevino, L. K, and Webster, J. "Flow in Computer-Mediated
York, 2006, pp. 32-47.
and H. "Lost in Hyperspace: Metrics and Communication," Communication Research (19), 1992, pp.
Otter, M., Johnson,
Mental with Computers 539-573.
Models," Interacting (13), 2000, pp.
1-40. Trumbo, J. "Spatial Design and Memory: A Conceptual Approach
to the Creation of Navigable in Multimedia
Palmer, J. W. "Web Site Usability, and Performance Space Design,"
Design,
Interactions (5), July-August 1998, pp. 26-34.
Metrics," Information Systems Research (13:2), June 2002, pp.
151-167. UsabilityNet. "Cost Justifying
Usability," 2003 (available online at
http ://www.usabilitynet. org/management/c_cost.htm).
Park, J., and Kim, J. "Contextual Navigation Aids for Two World
Utting, K, and Yankelovich, N. "Content and Orientation in
Wide Web Systems," International Journal ofHuman-Computer
Hypermedia Networks," A CM Transactions on
Interaction (12), 2000, pp. 193-217. Information

R. Systems (7:1), 1989, pp. 58-84.

Pereira, E. "Optimizing Interaction for the
Human-Computer
Watts-Perotti, J., and Woods, D. D. "How Experienced Users Avoid
Electronic Commerce Journal
Environment," of Electronic
Getting Lost in Large Display Networksm," International
Commerce Research (1:1), 2000, pp. 23-44 (available at http://
Journal of Human-Computer Interaction 1999,
(11:4), pp.
www.csulb.edu/joumals/jecr/p_i.htm). 269-299.

MIS QuarterlyVol. 30 No. 3/September2006 675

This content downloaded from 193.142.30.124 on Sat, 28 Jun 2014 07:19:06 AM

All use subject to JSTOR Terms and Conditions
Webster & Ahuja/EnhancingtheDesign ofWeb NavigationSystems

Webster, J., and Ho, H. "Audience Engagement in Multimedia Yu, B-M., and Roh, S-Z. "The Effects of Menu Design on

Presentations," The Data Base for Advances in Information Information-Seeking Performance and User's Attitude on the

Systems (28:2), 1997,pp. 63-77. World Wide Web," Journal of the American Society for Infor
Webster, J., and Martocchio, J. J. "The Differential Effects of mation Science and Technology (53:11), 2002, pp. 923-933.
Software Training Previews on Training Outcomes," Journal of Zona Research Inc. "1998 Holiday Shopping: Is Web-Based

Management (21:4), 1995, pp. 757-787. Buying Reaching Critical Mass?," 1998 (http://www.
Webster, J., and Martocchio, J. J. "Microcomputer Playfulness: binteractive.com/press/archive/release68.asp).
Development of a Measure with Workplace Implications," MIS
Quarterly (16:2), 1992,pp. 201-226.
Webster, J., Trevino, L. K, and Ryan, L. "The Dimensionality and
Correlates of Flow inHuman-Computer Interactions," Computers About theAuthors
inHuman Behavior (9), 1993, pp. 411-426.
Wen, J. "Post-Valued Recall Web Pages: User Disorientation Hits Jane Webster received her Ph.D. from New York University and is
theBig Time," IT & Society (1:3), 2003, pp. 184-194. the E. Marie Shantz Professor of MIS in the School of Business at

Wickens, CD.Engineering Psychology and Human Performance Queen's University in Canada. She has served as a senior editor for

(2nd ed.), Harper Collins Publishers, New York, 1992. MIS Quarterly and guest associate editor of Information Systems
Woods, D. D. "Visual Momentum: A Concept to Improve the Research. She currently serves on the advisory board for AIS

Cognitive Coupling of Person and Computer," International SIGCHI and is thevice presidentof Publications forAIS. She has
Journal ofMan-Machine Studies (21), 1984, pp. 229-244. published in a variety of journals including the Academy ofMan
Woods, D. D., Roth, E. M., Stubler, W. F., and Mumaw, R. J. agement Journal, Communication Research, Journal of Organiza
"Navigating Through Large Display Networks in Dynamic tional Behavior, MIS Quarterly, and Organization Science. Her

Control Proceedings of the 34th Annual Meeting research investigates the impacts of technologies in the support of
Applications,"
Factors Santa Monica, distributed work, recruit
of the Human Society, CA, 1990, pp. organizational communication, employee
396-399. ment and selection, employee monitoring, training and learning, and
Woods, D. D., and Watts, J. C. "How Not to Have to Navigate human-computer interaction issues.

Through Too Many Displays," inHandbook ofHuman-Computer

Interaction, M. Helander, T. K. Landauer and P. Prabhu is the director of Research with Electrolinks, a
(eds.), Jaspreet Ahuja
Elsevier, Amsterdam, 1997, pp. 617-650. pioneer in Broadband Over Powerlines (BPL). He also is a senior

Yoo, Y., and Alavi, M. "Media and Group Cohesion: Relative consultant and manages telecom projects for companies like Bell

Influences on Social Presence, Task Participation, and Group Canada. The present research is based on his Master's thesis,

Consensus," MIS (25:3), 2001, pp. 371-390. completed at the University ofWaterloo in Canada.
Quarterly

676 MIS QuarterlyVol. 30 No. 3/September2006

This content downloaded from 193.142.30.124 on Sat, 28 Jun 2014 07:19:06 AM

All use subject to JSTOR Terms and Conditions
 Webster & Ahuja/Enhancing the Design of Web Navigation Systems

Appendix A
H_______H__B________^^
Measures

Navigation Systems, the control condition (the simple navigation system), condition #1 (the basic global navigation system), and condition
#2 (the enhanced global navigation system), were compared. They were coded into two dummy variables (CI and C2) for analyses. That is,
the control condition (the simple navigation system) was represented as 0 on both CI and C2, condition #1 (the basic global navigation system)
as 1 on CI and 0 on C2, and condition#2 (the enhanced global navigation system)as 0 on CI and 1 on C2. The use of dummy coding of
experimental conditions for SEM analyses is recommended when investigating direct rather than moderating effects of experimental conditions

(Joreskog and Sorbom 1989; MacCallum and Austin 2000; Russell et al. 1998) and is consistent with others' use of dummy coding to represent

experimental groups in SEM analyses (e.g., Webster and Martocchio 1995; Yoo and Alavi 2001).

Perceived Disorientation was captured with Ahuja and Webster's (2001) seven-item perceived disorientation measure on seven-point scales

ranging from "never" to "always" (evidence of the measure's construct validity is reported inAhuja and Webster). The items were
While Iwas browsing:
I felt lost
I felt like Iwas going around in circles
Itwas difficultto finda page thatI had previouslyviewed
Navigating between pages was a problem
I didn't know how to get tomy desired location
I feltdisoriented
After browsing for a while I had no idea where to go next

was captured with Webster and Ho's seven-item measure on seven-point scales
Engagement (1997) engagement ranging from "strongly
disagree" to "strongly agree." The items were
TheWeb site:
Kept me totally absorbed in the browsing
Held my attention
Excited my curiosity
Aroused my imagination
Was fun
Was intrinsically interesting
Was engaging

was
User Performance determined by two measures: the time taken to answer the questions (with a lower time representing higher
performance) and the number of questions answered correctly during the formal search. The time taken to answer the questions was determined
by the back-end program and was reverse-coded for analyses.

To determine the number of correct answers, participants were presented with a blue sheet during asked them to find the
browsing which
answers to eight questions. These questions were pretested in the pilot experiments, and answers to these questions were located in different
sections and levels of theWeb site. There were three types of questions.

Simple fact questions:

When isAssignment #4due?
Who is the technical contact person?
Which is the name of the first assignment that is due?

Judgment questions:
From the software available for download, which one do you think is the best to capture screenshots?
Which editorwould be best touse for thiscourse (to edit SGML documents)?
What is the best way to print modules of the course?

Comparison offacts questions:

If you wanted to know what the instructor has asked you to do, which is the best place to look: the Newsgroup or Instructor's
comments?
Which is the lowestBrowser thatis supportedby thiscourse:Netscape 3 orNetscape 4?

MIS QuarterlyVol. 30 No. 3/September2006 677

This content downloaded from 193.142.30.124 on Sat, 28 Jun 2014 07:19:06 AM

All use subject to JSTOR Terms and Conditions
Webster & Ahuja/Enhancing the Design of Web Navigation Systems

Future Intentions to Use. Six items from Jackson et al. (1997) captured intentions on seven-point scales ranging from "strongly disagree"
to "strongly agree." The items were
For learningTechnicalWriting
Using thisweb sitewould be a good idea
Iwould intend to use this web site

Using thisweb sitewould be a foolish idea

Iwould like the idea of using thisweb site
Using this web site would be unpleasant
I would intend to use this web site very frequently

Control Variables. In addition to the variables included in themodel, we incorporated 10 control variables that past research suggests could

relate to the outcomes of interest. More specifically, we examined theWeb skill variables (Koufaris 2002) of Internet experience and Web

attributions for performance 1973), computer (Webster and Martocchio 1992), ease of use (Davis 1989),
design experience, (Kelly playfulness
and demographic variables (Koufaris 2002) of age, gender, years of education in the English language, level of education, and subject of
education. We compared these variables across the three conditions and only Internet experience (a one-item measure of "How long have you
been the Internet?" with three response choices, 1-6 months, 7-12 months, and more than 12 months) differed; hence itwas the only
using
control variable included in the analysis.

Appendix B

Correlations Between Measures3 IIMHHBBHBHHBHBHHHHBBMM

I I 1I 2 l 3 I 4l 5 l 6
1.Navigation
Systems: I I II I I I
01 (N/A)
-.50***
C2_ (N/A)_j
Disorientation .09 -.26*** (.90) I
|TPerceived
.03 .04 -.45*** (.92) I I II I
[TEngagement
4. UserPerformance: I I III I II -.19** I .07
Time -.27*** .24*** (N/A)
(reverse-coded)
Correct -.20** | .21** -.27*** .21** .26***
j Answers_| (N/A)_
Intentions to Use -.05 .03 -.54*** .67*** .24*** .20** I
(.90)
pTFuture
Experience
|6rinternet I -.00 | .17* | -.01 | -.07 | .17* | | -.05 | (N/A)|
.31***

alnternal-consistency reliabilities on the diagonal.

<.05
*p
< .01
**p
< .001
***p

678 MIS QuarterlyVol. 30 No. 3/September2006

This content downloaded from 193.142.30.124 on Sat, 28 Jun 2014 07:19:06 AM

All use subject to JSTOR Terms and Conditions