See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/279894167

Measuring e-Commerce Success: Applying the DeLone & McLean Information Systems Success Model

Article  in  International Journal of Electronic Commerce · September 2004

DOI: 10.1080/10864415.2004.11044317

CITATIONS READS
1,149 9,308

2 authors:

William Delone Ephraim McLean

American University Washington D.C. Georgia State University
42 PUBLICATIONS   24,427 CITATIONS    139 PUBLICATIONS   26,896 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Society of Information Management (SIM) IT Trend Study View project

All content following this page was uploaded by Ephraim McLean on 28 July 2015.

The user has requested enhancement of the downloaded file.

This article was downloaded by: [Georgia State University]
On: 28 July 2015, At: 13:12
Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: 5 Howick Place, London, SW1P 1WG

International Journal of Electronic Commerce

Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/mjec20

Measuring e-Commerce Success: Applying the DeLone & McLean Information Systems Success Model
a b
William H. DeLone & Ephraim R. McLean
a
Kogod School of Business at American University in Washington, DC.
b
Robinson College of Business of Georgia State University in Atlanta
Published online: 08 Dec 2014.

To cite this article: William H. DeLone & Ephraim R. McLean (2004) Measuring e-Commerce Success: Applying the DeLone & McLean Information Systems Success Model, International Journal of Electronic Commerce, 9:1, 31-47

To link to this article: http://dx.doi.org/10.1080/10864415.2004.11044317

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or
warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or
endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims,
proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly
forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
 Measuring e-Commerce Success: Applying the
DeLone & McLean Information Systems
Success Model
William H. DeLone and Ephraim R. McLean

ABSTRACT: Information technology and the Internet have had a dramatic effect on busi-
ness operations. Companies are making large investments in e-commerce applications
but are hard pressed to evaluate the success of their e-commerce systems. The DeLone &
Downloaded by [Georgia State University] at 13:12 28 July 2015

McLean Information Systems Success Model can be adapted to the measurement chal-
lenges of the new e-commerce world. The six dimensions of the updated model are a
parsimonious framework for organizing the e-commerce success metrics identified in the
literature. Two case examples demonstrate how the model can be used to guide the iden-
tification and specification of e-commerce success metrics.

KEY WORDS AND PHRASES: e-commerce, evaluation of information systems, informa-

tion systems success, use of information systems, user satisfaction, value of information
technology.

The Internet has dramatically affected the conduct of business. Markets, in-
dustries, and businesses are being transformed. The new economy demands
the exploitation of new models and paradigms. Information technology (IT)
now drives businesses and markets. In the new economy, the Internet has
become a powerful and ubiquitous communication mechanism to facilitate
the consummation and processing of business transactions. This has led to
substantial changes in traditional industries and companies. Firms are attempt-
ing to understand and measure the impact of IT so that they can make intelli-
gent decisions regarding crucial IT investments.
All this notwithstanding, basic business principles still hold. The laws of
economics have not been rewritten. The long-term success or failure of com-
panies is determined by their ability to generate positive net revenues. Simi-
larly, there has been no change in the fundamental role of IT in facilitating
business transactions and communicating relevant information to decision-
makers. However, the decision makers now include customers, both internal
and external. Time compression and the magnitude of change may be dra-
matic, but IT still has the same goals and objectives.
This paper proposes that even though new business models are emerging,
the fundamental role of IT has not changed, and thus the methodology for
measuring the success of information systems (IS) should not change. Although
there are many new technological developments, the dependent variable—IS
success—and its underlying dimensionalities are still the same. The DeLone
& McLean IS Success Model is an existing success-measurement framework
that has found wide application since its publication in 1992 [7]. With the
addition of new metrics, an updated version of the model can be applied to e-
commerce success measurement [8, 9].
For the purposes of this paper, e-commerce is defined as the use of the
Internet to facilitate, execute, and process business transactions. Business
International Journal of Electronic Commerce / Fall 2004, Vol. 9, No. 1, pp. 31–47.
Copyright © 2004 M.E. Sharpe, Inc. All rights reserved.
1086-4415/2004 $9.50 + 0.00.
 32 WILLIAM H. DELONE AND EPHRAIM R. MCLEAN

transactions involve a buyer and seller and the exchange of goods or ser-
vices for money.

Updated DeLone & McLean Success Model

The original DeLone & McLean Success Model provided a comprehensive

framework for measuring the performance of information systems [7]. The
new and updated model is based on the empirical and theoretical contribu-
tions of researchers who have tested or discussed the original model [8, 9].
The updated model, presented in Figure 1, consists of six interrelated dimen-
Downloaded by [Georgia State University] at 13:12 28 July 2015

sions of information systems success:

• System quality
• Information quality
• Service quality
• Use
• User satisfaction
• Net benefits
The primary improvements to the original model include (a) the addition
of service quality to reflect the importance of service and support in success-
ful IS systems, and (b) the collapsing of individual impacts and organizational
impacts into a more parsimonious net benefits construct.

Service Quality

In the original formulation of the DeLone & McLean model [7], the dual di-
mensions of system and information quality seemed sufficient to capture the
essential characteristics of information systems being delivered to users. In
the intervening decade, however, it became apparent that a third dimension
was needed, service quality [8, 9]. As Pitt, Watson, and Kavan observed, “Com-
monly used measures of IS effectiveness focus on the products rather than the
services of the IS function. Thus, there is a danger that IS researchers will
mismeasure IS effectiveness if they do not include in their assessment pack-
age a measure of IS service quality” [32, p. 173].
This need has become even more apparent with the advent of e-commerce
and the demand of customers for support from their Web providers. Thus,
service quality is added to Figure 1.

Net Benefits

The new net benefits construct immediately raises three issues that must be
addressed: What qualifies as a “benefit”? for whom? and at what level of
analysis? The original formulation of the DeLone & McLean model used the
term “impact.” Seddon used “consequences” and “net benefits” in his charac-
terization of outcomes [38]. We have come to prefer “net benefits” because the
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 33

INFORMATION
QUALITY
INTEN- USE
TION TO
USE
SYSTEM
QUALITY NET
BENEFITS

USER
SATISFACTION
Downloaded by [Georgia State University] at 13:12 28 July 2015

SERVICE
QUALITY

Figure 1. Updated DeLone & McLean IS Success Model.

original term, “impacts,” may be positive or negative, thus possibly leading

to confusion as to whether the results are good or bad. In addition, the inclu-
sion of “net” in “net benefits” is important, because no outcome is wholly
positive and without any negative consequences. Thus, “net benefits” is prob-
ably the most accurate descriptor of the final success variable.
The second issue of concern is benefits for whom—the designer, the spon-
sor, the user, or others? Different actors or players may have different views of
what constitutes a benefit. Thus, it is impossible to define net benefits without
first defining the context or frame of reference. The fact that the DeLone &
McLean model does not define the context is a matter of detail, not of over-
sight. The focus of any proposed study must be defined. The model may be
useful to both Microsoft and the user community, but each may have a very
different definition of what constitutes net benefits—and thus IS success—
from its own perspective.
Finally, the level of analysis must be addressed. Are the benefits to be
measured from the individual’s perspective, the employer’s, or that of the
industry or of the nation? The challenge for the researcher is to clearly and
carefully define the stakeholders and the context in which net benefits are to
be measured.

The DeLone & McLean Success Model for

e-Commerce Measurement

Since its publication in 1992, nearly 300 articles in refereed journals have re-
ferred to, and made use of, the DeLone & McLean IS Success Model as the
basis for measuring the dependent variable in IS research [7]. The model is
based on Shannon and Weaver’s classic communication theory, as adapted by
Mason, to measure IS impacts [23, 40]. As a powerful communications and
 34 WILLIAM H. DELONE AND EPHRAIM R. MCLEAN

commerce medium, the Internet is a communication and IS phenomenon that

lends itself to a measurement framework (i.e., the DeLone & McLean model)
built on communication theory. In the e-commerce context, the primary sys-
tem users are customers or suppliers rather than internal users. Customers
and suppliers will use the system to make buying or selling decisions and
execute business transactions. These electronic decisions and transactions may
affect individual users, organizations, industries, and even national economies.
Molla and Licker first proposed that the original DeLone and McLean model
could be extended to measure e-commerce success [25]. This paper is based
on the updated DeLone and McLean IS Success Model as revised in 2002 and
extended in 2003 [8, 9]. Molla and Licker used the original DeLone and McLean
Downloaded by [Georgia State University] at 13:12 28 July 2015

model published in 1992 as a basis of their proposal [7, 25]. This paper adds
the new construct of service quality and updates the important net benefits
construct (i.e., individual and organizational impact constructs in the original
model), and it provides an extensive list of success metrics based on a com-
prehensive review of e-commerce articles in the IS and marketing literature. It
concludes with two case examples that demonstrate the utility of the pro-
posed e-commerce success framework. Molla and Licker did not attempt to
demonstrate the application of the model.
The six success dimensions of the DeLone & McLean IS Success Model can
be applied to the e-commerce environment as follows:
1. System quality, in the Internet environment, measures the desired
characteristics of an e-commerce system. Usability, availability,
reliability, adaptability, and response time (e.g., download time) are
examples of qualities that are valued by users of an e-commerce
system.
2. Information quality captures the e-commerce content issue. Web
content should be personalized, complete, relevant, easy to under-
stand, and secure if prospective buyers or suppliers are to initiate
transactions via the Internet and return to a site on a regular basis.
3. Service quality, the overall support delivered by the service provider,
applies regardless of whether the support is delivered by the IS
department or a new organizational unit or is outsourced to an
Internet service provider. This dimension is more important in an e-
commerce environment than ever before, because the users are now
customers rather than employees, and therefore, poor user support
will translate into lost customers and lost sales.
4. Usage measures everything from a visit to a Web site and navigation
within the site to information retrieval and execution of a transaction.
5. User satisfaction is an important means of measuring customers’
opinions of an e-commerce system and should cover the entire
customer experience cycle from information retrieval through
purchase, payment, receipt, and service.
6. Net benefits are the most important success measures, because they
capture the balance of the positive and negative impacts of
e-commerce on customers, suppliers, employees, organizations,
markets, industries, economies, and even society as a whole. Have
Internet purchases saved individual consumers time and money?
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 35

Have the benefits, such as larger markets, supply-chain efficiencies,

and customer responsiveness, yielded positive net benefits for an
organization? Have country investments in e-commerce infrastruc-
ture yielded a net positive growth in GNP? Have societal invest-
ments in e-commerce infrastructure and education reduced poverty?
Net benefits measures are determined by the context and objectives of the
specific e-commerce investment. There will be a variety of e-commerce net
benefits measures, but many will be the same ones that have been developed
and tested for information systems investments in general.
Net benefits success measures are clearly important, but they cannot be
Downloaded by [Georgia State University] at 13:12 28 July 2015

analyzed and understood without system, information, and service-quality

measurements. For example, in the e-commerce environment, the impact of a
Web site design on customer purchases cannot be fully understood without
an evaluation of the usability of the Web site and the relevance for purchasing
decisions of the information provided to the prospective purchaser.

E-Commerce Metrics: Old and New

A review of articles on e-commerce and electronic data interchange (EDI) in

recent academic and trade journals (1996–2002) yielded many suggested mea-
sures of e-commerce success. IS and marketing journals were included in the
search for e-commerce success metrics. Most of the articles were conceptual
in nature, but some were empirical and, therefore, attempted to operationalize
e-commerce success metrics. Some of the proposed measures are IS measure-
ment staples, whereas others have been newly developed for the e-commerce
environment. Most important, all of the proposed measures can be classified
under the six dimensions of the DeLone & McLean model.
The collections of e-commerce measures identified in the journals are dis-
cussed below under the relevant DeLone & McLean success dimensions. These
e-commerce success measures are listed in Tables 1 through 7. The tables differ-
entiate traditional management information systems (MIS) success measures
that have been applied to e-commerce and new measures that have surfaced
recently in the e-business environment. The e-commerce metrics identified in
the tables are meant to be illustrative and not necessarily comprehensive.

System Quality

The system quality metrics for e-commerce shown in Table 1 are primarily the
metrics that have been used in IS research for the last two decades. The key
measures of system quality are still usefulness, usability, responsiveness, reli-
ability, and flexibility. Some of the functionality item measures, such as
versionablity, are likely to differ in the e-commerce environment. What is dif-
ferent is the relative importance of the system-quality measures. When the us-
ers are customers as opposed to employees, their use is typically volitional, and
this means that poor usability, usefulness, or responsiveness can discourage
customer usage of an e-commerce system. Expected benefits are unlikely to be
 36 WILLIAM H. DELONE AND EPHRAIM R. MCLEAN

Traditional MIS success measures e-commerce source(s)

Usability, ease of use Spiller & Lohse 1998;

• Help features Molla & Licker 2001
• Intuitiveness
• Attractiveness Liu & Arnett 2000
Download time Spiller & Lohse 1998;
Palmer 2002
System responsiveness, response time Tiwana 1998; Molla & Licker 2001
Dependability, reliability, availability Liu & Arnett 2000; Tiwana 1998;
Ünal 2000; Molla & Licker 2001
Adaptability, flexibility
Usefulness, functionality
Downloaded by [Georgia State University] at 13:12 28 July 2015

• “Versionability” Reisenwitz & Cutler 1998; Varian 1997

• Transaction capabilities Parsons et al. 1998
• Environmental scanning Achrol & Kotler 1999
• Customer feedback capability Peppers & Rogers 1997; Palmer 2002
Security, secure transactions Gupta et al. 1998; Ünal 2000
Scalability
Interactivity Palmer 2002

New e-commerce success measures e-commerce source(s)

Customization Palmer 2002

Ease of navigation Palmer 2002; Molla & Licker 2001
Privacy Molla & Licker 2001
Security Molla & Licker 2001

Table 1. E-Commerce System Quality Measures.

Notes: Traditional MIS success measures are listed in tables 1 to 7 in DeLone & McLean [7] or were cited in
MIS literature in the early to mid-1990s. Sources listed in italics represent empirical studies with
operationalized e-commerce metrics. E-commerce measures with no source citation in the right-hand column
were found in the trade literature.

realized when system quality is unsatisfactory. In addition, system security be-

comes a more significant system-quality issue, because e-commerce is typically
conducted over the Internet rather than a private, proprietary network.

Information Quality

Research articles on e-commerce have not devoted much attention to infor-

mation-quality or content-quality success measures (see Table 2). Many of the
traditional information-quality measures apply, especially relevance. When
customers are the users, and customer-purchase decisions are the objective,
new, dynamic personalization measures are important because of the mass
customization developments occurring in sales and marketing.

Service Quality

Liu and Arnett identified service quality as an important measure of Web site
success [21]. In their empirical study, service quality was measured as quick
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 37

Traditional MIS success measures e-commerce source(s)

Accuracy Molla & Licker 2001

Relevance
• Customer preference information Peppers & Rogers 1997;
Molla & Licker 2001
Understandability Molla & Licker 2001
Completeness Zwass 1996; Palmer 2002;
• Customer information integration across Molla & Licker 2001
multiple channels
Currency D’Ambra & Rice 2001;
Molla & Licker 2001
Competitive intelligence Teo & Choo 2001
Downloaded by [Georgia State University] at 13:12 28 July 2015

New e-commerce success measures e-commerce source(s)

Dynamic content Parsons et al. 1998

Content personalization Barua et al. 2000; Molla & Licker 2001
Variety of information Palmer 2002

Table 2. E-Commerce Information Quality Measures.

Notes: Traditional MIS success measures are listed in tables 1 to 7 in DeLone & McLean [7] or were cited in
MIS literature in the early to mid-1990s. Sources listed in italics represent empirical studies with
operationalized e-commerce metrics.

responsiveness, assurance, empathy, and following-up service. Service quality

can also be measured by the effectiveness of on-line support capabilities, such
as answers to frequently asked questions, customized site intelligence, and
order tracking [25]. None of the other e-commerce articles reviewed for this
paper addressed service quality (i.e., user support). There is no debating the
importance of supporting customers as they attempt to execute transactions
via e-commerce systems, whether the support is offered through help desks,
hotlines, service centers, or the like. It is necessary to look to the service-quality
research stream for success measures that make sense in the e-commerce envi-
ronment—measures such as responsiveness and technical competence.

User Satisfaction

The literature review did not reveal any e-commerce-specific instruments for
measuring user satisfaction. Molla and Licker emphasize the importance of
“customer e-commerce satisfaction” and define it as “the reaction or feeling
of a customer in relation to his/her experience with all aspects of an e-com-
merce system” [25, p. 7]. Reichheld and Schefter’s “e-loyalty” represents a
good surrogate measure of customer satisfaction in the e-commerce environ-
ment [36]. Mehta and Sivadas proposed that customer attitudes are impor-
tant measures of e-commerce success [24]. It is recommended here that
researchers adopt and adapt user information satisfaction and end-user sup-
port satisfaction instruments as appropriate for specific e-commerce research.
Some items will need to be reworded, and new items will have to be added to
the traditional measurement instruments.
 38 WILLIAM H. DELONE AND EPHRAIM R. MCLEAN

Nature of use e-commerce source(s)

Information search Young & Benamati 2000

Receiving customer orders Young & Benamati 2000
Accepting customer payments
Customer service requests Young & Benamati 2000
Purchase orders
Payments to vendors

Amount of use e-commerce source(s)

Number of e-commerce site visits D’Ambra & Rice 2001;

Molla & Licker 2001
Downloaded by [Georgia State University] at 13:12 28 July 2015

Length of stay
Number of purchases completed

Table 3. E-Commerce System Use Measures.

System Use

As with traditional information systems [18], customer or supplier use is an

important measure of success for e-commerce systems, especially because
customer use is more often voluntary. The nature and amount of the usage are
both important indicators of success. The usage metrics in Table 3 were found
almost exclusively in the trade literature.

Net Benefits

Measures of net benefits success address the ultimate impact of the e-com-
merce system and therefore represent the most important category of success
measurement. An e-commerce or e-business system can benefit a single user
(usually a customer), a group of users, an organization, or an entire industry.
Hence, the net benefits success measures found in the literature are organized
by level (individual, group, organization, and industry e-commerce measures)
in Tables 4 through 7. Two factors accounted for the many new e-commerce
success measures identified in this section: (a) the new context—e-commerce,
and (b) the new research domain—marketing research literature in addition
to IS literature.
How do individual users benefit from the use of e-commerce systems? The
individual benefit measures identified in the MIS and marketing literature
can be found in Table 4.
The Internet and e-commerce systems enable people to work together to
achieve specific objectives. How do groups of users benefit from the use of e-
commerce systems? The group benefit measures identified in the MIS and
marketing literature are shown in Table 5.
Some of the most important benefits from an e-commerce system accrue to
the organization that invested in the system. How do organizations benefit
from the use of e-commerce systems? The organization benefit measures iden-
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 39

Traditional MIS success measures e-commerce source(s)

Enhanced customer support and services Raghunathan & Madey 1999;

Rapert & Brent 1998;
Griffith & Krampf 1998
Improved customer knowledge Loftus 1997
Reduced information search time Hoque & Lohse 1999

New e-commerce success measures e-commerce source(s)

Improved customer experience Hoffman & Novak 1996

Entertainment D’Ambra & Rice 2001
Reduced shopping cost D’Ambra & Rice 2001
Downloaded by [Georgia State University] at 13:12 28 July 2015

Real-time marketing offers

Table 4. Individual Benefits from e-Commerce System.

Notes: Traditional MIS success measures are listed in tables 1 to 7 in DeLone & McLean [7] or were cited in
MIS literature in the early to mid-1990s. Sources listed in italics represent empirical studies with
operationalized e-commerce metrics. E-commerce measures with no citation in the right-hand column were
found in the trade literature.

Traditional MIS success measures e-commerce source(s)

Communication effectiveness Sengupta & Zhao 1998

Improved knowledge sharing O’Callaghan 1999

New e-commerce success measures e-commerce source(s)

Selling team coordination Loftus 1997

Table 5. Group Benefits from e-Commerce System.

Notes: Traditional MIS success measures are listed in tables 1 to 7 in DeLone & McLean [7] or were cited in
MIS literature in the early to mid-1990s.

tified in the MIS and marketing literature can be found in Table 6. Not sur-
prisingly, traditional organizational success measures, such as cost efficien-
cies, increased sales, profits, and return on investment, are suggested as key
e-commerce success measures. Some measures not typically found in MIS re-
search are proposed as well, such as global reach and click-to-buy ratio.
The Internet has facilitated interorganizational communications that have
resulted in industry-level efficiencies. How do industries benefit from the use
of e-commerce systems? The industry benefit measures identified in the MIS
and marketing literature can be found in Table 7.
One article located in the literature search proposed economy-level mea-
sures of e-commerce success. Colecchia proposed three dimensions of success
at the country level: readiness for e-commerce as measured by access and
technology infrastructure; intensity as measured by e-commerce transaction
volume; and impact as measured by economic-efficiency gains, employment
gains, and new products and services [5].
 40 WILLIAM H. DELONE AND EPHRAIM R. MCLEAN

Traditional MIS success measures e-commerce source(s)

Growth in customer base Peppers & Rogers 1997

Increased sales Griffith & Krampf 1998
Market share
Profit Teo & Too 2000
Return on investment Barua et al. 2001
Customer lock-in Shapiro & Varian 1999
Competitive advantage Takacs & Freiden 1998
Economies of scale Teo & Too 2000
Organizational efficiency Teo & Too 2000;
Barua, Whinston, & Yin 2000
Sales process efficiency Hoffman & Novak 1997
Downloaded by [Georgia State University] at 13:12 28 July 2015

Productivity Barua et al. 2001

Operational excellence Morash & Clinton 1998; Quinn 1999
Reduced cycle time Hoogeweegen & Wagenaar 1996;
O’Callaghan 1999; Barua et al. 2001

New E-commerce success measures e-commerce source(s)

Global reach Demers & Lev 2000

Customer loyalty Demers & Lev 2000; Molla & Licker 2001
Stickiness Demers & Lev 2000
Brand awareness
Customer responsiveness Teo & Too 2000;
Hoogeweegen & Wagenaar 1996
Market responsiveness Teo & Too 2000
Customer acquisition Gonsalves et al. 1999; Barua et al. 2001
Customer retention Parthasarathay & Bhattacherjee 1998
Click-to-buy ratio

Table 6. Organizational Benefits from e-Commerce System.

Notes: Traditional MIS success measures are listed in tables 1 to 7 in DeLone & McLean [7] or were cited in
MIS literature in the early to mid-1990s. Sources listed in italics represent empirical studies with
operationalized e-commerce metrics. E-commerce measures with no citation in the right-hand column were
found in the trade literature.

New e-commerce success measures e-commerce source(s)

Interorganizational transaction efficiency Baron, Shaw, & Bailey 2000

• Supply-chain efficiency; cost reductions throughout
the supply chain
Supply-chain integration, synchronization Morash & Clinton 1998; Quinn 1999;
Ünal 2000
Improved trading partner relationships Hoogeweegen & Wagenaar 1996
• Virtual partnerships Leidner 1999
Interorganizational coordination and synergy Achrol & Kotler 1999

Table 7. Industry Benefits from e-Commerce System.

Most of the success measures found in e-commerce articles are measures

already used in IS research. The major differences are context and focus. In
the e-commerce environment, the system users are customers and suppliers,
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 41

and the purpose of the system is primarily the execution of business transac-
tions. Thus, the e-commerce context does not require a new set of success
metrics. The updated DeLone & McLean model can serve as an appropriate
framework for organizing e-commerce success metrics.

Applying e-Commerce Success Measures:

Two Case Examples

The utility of the updated DeLone & McLean IS Success Model and related e-
commerce metrics can be demonstrated with two case examples. One involves
Downloaded by [Georgia State University] at 13:12 28 July 2015

a large, high-profile “bricks-and-clicks” business, and the other a small, tradi-

tional, regional retailer. The proposed e-commerce metrics are meant to be
demonstrative rather than exhaustive. In other words, the two examples show
how the DeLone & McLean model can be used to guide both practical and
empirical success studies. Although the examples are compelling logically,
the next step is to test the metrics empirically.

Case 1: Barnes & Noble

The entry of Barnes & Noble into on-line book sales in 1997 is well documented
in two Harvard Business School cases entitled “Leadership Online: Barnes &
Noble vs. Amazon.com” [19]. Amazon.com’s growth in sales and market value
demonstrated the value of the on-line model. Customers could order from
their homes. More titles were available than at brick-and-mortar bookstores,
and at lower prices. Detailed customer information enabled on-line bookstores
to personalize their customers’ electronic purchasing experience. Barnes &
Noble decided to compete opposite Amazon.com in cyberspace. The purpose
here is not to discuss this well-known case in detail but to consider the mea-
sures by which Barnes & Noble might measure the success of its e-commerce
business. A proposed e-commerce success measurement model is displayed in
Figure 2 using the updated DeLone and McLean framework [8, 9].
The ultimate measures of success for Barnes & Noble’s on-line book sales
business are incremental sales revenue and investor reactions to the company’s
“success” as reflected in its market valuation. To understand these net benefit
results, the e-commerce researcher must also measure the quality of the user’s
experience and the customer’s usage of, and satisfaction with, the system.
The Barnes & Noble Web site must be easy to use and available whenever the
customer wishes to access it. The information displayed must be relevant to
the customer’s interests and must be complete. A quick e-mail response to
purchase transactions is an issue of on-line service quality. If the customer’s
question cannot be resolved by e-mail, then a call center should be available.
Customers’ usage can be measured by the frequency of their site visits, and
their satisfaction can be measured by repeat purchases. All these measures, as
displayed in Figure 2, represent a comprehensive (but again, not exhaustive)
success measurement model for Barnes & Noble on-line, based on the up-
dated DeLone & McLean model.
 42 WILLIAM H. DELONE AND EPHRAIM R. MCLEAN
Downloaded by [Georgia State University] at 13:12 28 July 2015

Figure 2. Barnes & Noble.com IS Success Model

Case 2: ME Electronics

ME Electronics (the company name is changed for confidentiality) is a quality

consumer electronics retail chain targeting wealthy customers by offering high-
end audio and video brands through nine retail stores in a large metropolitan
area. ME Electronics has maintained a Web site for the past four years. The
Web site supports a broad-based communications strategy and serves as a
one-way communication channel to the customer, used to disseminate com-
pany information, such as new product introductions. At present, customers
cannot place orders through ME’s Web site. ME faces a competitive challenge
from large-scale electronics retailers with fully functional e-commerce sites.
ME’s e-business strategy for the near future is based on electronic customer-
relationship management (ECRM). The strategy involves an interactive Web
site and a comprehensive customer database that tracks all customer interac-
tions. The value proposition involves service excellence through greater con-
venience, lower costs, and personalized customer service.
The purpose here is not to discuss the wisdom of this strategy but to dem-
onstrate the utility of the updated DeLone & McLean model in an entirely
different e-commerce context. How should ME Electronics measure the suc-
cess of its new e-business efforts? An e-business success model for the firm is
displayed in Figure 3.
The strategic objective of ME Electronics’ new e-business strategy is CRM.
The net benefit of this new strategy should be increased sales per customer.
This benefit will be realized only if customers are satisfied with their Web site
experience. They should be encouraged to periodically complete e-mail-based
customer-satisfaction surveys that measure system quality (download speed
and ease of use), relevance of the information found on the site, ME’s respon-
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 43

SYSTEM
QUALITY
Download Time
Ease of Use
Downloaded by [Georgia State University] at 13:12 28 July 2015

Figure 3. ME Electronics EC Success Model

siveness to customer e-mail inquiries, and overall satisfaction with the on-line
experience. ME should also collect data on the number of electronic interac-
tions with each customer and correlate interactions with sales activity. All these
measures, as displayed in Figure 3, represent a comprehensive success-mea-
surement model for ME Electronics’ new e-business strategy.
These two examples demonstrate the flexibility and relevance of the up-
dated DeLone & McLean model as a framework for measuring e-commerce
success.

Conclusions and Recommendations

This paper adapts the well-established DeLone & McLean IS Success Model
to the metrical challenges of the new e-commerce world [7, 8, 9]. Drawing
from the IS and marketing literature published over recent years, the six di-
mensions of the updated DeLone & McLean model comprise a parsimonious
framework for organizing the various e-commerce success metrics identified
in the literature. This exercise leads to the following recommendations.
Researchers and practitioners should not let themselves be carried away
by the hype of the new economy and led to believe that this new and rapidly
changing environment requires entirely new measures of IS success. One
should look first at the cumulative tradition, and determine which existing
and validated success measures can be used in the e-commerce environment.
As much as possible, tried and true measures should be enhanced and ex-
panded with modifications or, where necessary, new measures should be con-
sidered. Selection of e-commerce success dimensions and measures should be
contingent on the objectives and the context of the empirical investigation,
 44 WILLIAM H. DELONE AND EPHRAIM R. MCLEAN

but tested and proven measures should be used whenever possible. Com-
pletely new and untested metrics should be adopted only as a last resort.
The multidimensional and interdependent nature of e-commerce success,
as reflected in the DeLone & McLean IS Success Model, requires careful atten-
tion to the definition and measurement of every aspect of this dependent vari-
able. It is important to measure the possible interactions among the success
dimensions in order to isolate the effects of independent variables on one or
more of them. Cause can too easily be confused with effect. Viewing the DeLone
& McLean IS Success Model from both a process perspective and a variance
perspective, as suggested by Seddon [38], can be useful in identifying and
understanding these interactions.
Downloaded by [Georgia State University] at 13:12 28 July 2015

Furthermore, despite the multidimensional and contingent nature of e-com-

merce success, an attempt should be made to significantly reduce the number
of different measures used to measure success, so that research results can be
compared and findings validated.
Finally, e-commerce studies should include net benefits measures and not
be content to collect only surrogate measures, such as Web site hits (i.e., use).
Such benefits can be measured on at least four levels: individual, group, orga-
nizational, and industry. Tables 4 through 7 give examples of measures for
each level. Taken together, the metrics presented in this paper, and especially
in the two case examples, show that a blending of well-established and new
measures is needed. These measures become most useful, however, when fit-
ted into an overall structure or framework—a framework like the one pro-
vided by the DeLone & McLean Information Systems Success Model.

REFERENCES

1. Achrol, R., and Kotler, P. Marketing in the network economy. Journal of

Marketing, 63 (1999), 146–163.
2. Baron, J.; Shaw, M.; and Bailey, A. Web-based e-catalog systems in B2B
procurement. Communications of the ACM, 43, 5 (2000), 93–100.
3. Barua, A.; Konana, P.; Whinston, A.; and Yin, F. Measures for e-business
value assessment. IT Professional, 3, 1 (2001), 47–51.
4. Barua, A.; Whinston, A.; and Yin, F. Value and productivity in the
Internet economy. Computer, 33, 5 (2000), 102–105.
5. Colecchia, A. Defining and measuring electronic commerce: Towards the
development of an OECD methodology. Paper presented at the Interna-
tional Statistical Institute Cutting Edge Conference on the Measurement of
E-Commerce, Singapore, December 1999. Available at www.singstat.gov.sg/
conferences/ec/d8.pdf.
6. D’Ambra, J., and Rice, R.E. Emerging factors in user evaluation of the
World Wide Web. Information and Management, 38, 6 (2001), 373–384.
7. DeLone, W., and McLean, E. Information systems success: The quest for
the dependent variable. Information Systems Research, 3, 1 (1992), 60–95.
8. DeLone, W., and McLean, E. Information systems success revisited. In
R.H. Sprague, Jr. (ed.), Proceedings of the Thirty-fifth Hawaii International
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 45

Conference on System Science (CD-ROM). Los Alamitos, CA: IEEE Computer

Society Press, 2002.
9. DeLone, W., and McLean, E. The DeLone and McLean model of infor-
mation systems success: A ten-year update. Journal of Management Informa-
tion Systems, 19, 4 (2003), 9–30.
10. Demers, E., and Lev, B. A rude awakening: Internet shakeout in 2000.
Working Paper No. FR 00–13. University of Rochester, Simon Business
School, September 2000. Available at http://papers.ssrn.com/sol3/
papers.cfm?abstract_id=244231/.
11. Gonsalves, G.; Lederer, A.; Mahaney, R.; and Newkirk, H. A customer
resource life-cycle interpretation of the World Wide Web on competitiveness
Downloaded by [Georgia State University] at 13:12 28 July 2015

in expectations and achievements. International Journal of Electronic Com-

merce, 4, 1 (fall 1999), 103–120.
12. Griffith, D., and Krampf, R. An examination of the Web-based strategies
of the top 100 U.S. retailers. Journal of Marketing Theory and Practice, 6, 3
(1998), 12–23.
13. Gupta, A.; Stahl, D.O.; and Whinston, A.B. Managing computing re-
sources in intranets: An electronic commerce perspective. Decision Support
Systems, 24, 1 (1998), 55–69.
14. Hoffman, D., and Novak, T. Marketing in hypermedia computer-
mediated environments: Conceptual foundations. Journal of Marketing, 60, 3
(1996), 50–68.
15. Hoffman, D., and Novak, T. A new marketing paradigm for electronic
commerce. Information Society, 13 (1997), 43–54.
16. Hoogeweegen, M., and Wagenaar, R. A method to assess expected net
benefits of EDI investments. International Journal of Electronic Commerce, 1, 1
(fall 1996), 73–94.
17. Hoque, A., and Lohse, G. An information search cost perspective for
designing interfaces for electronic commerce. Journal of Marketing Research,
36, 8 (1999), 387–394.
18. Lassila, K., and Brancheau, J. Adoption and utilization of commercial
software packages: Exploring utilization equilibria, transitions, triggers, and
tracks. Journal of Management Information Systems, 16, 2 (1999), 63–90.
19. Leadership online: Barnes & Noble vs. Amazon.com. Cases 798–063 and
799–138. Harvard Business School, 1998 and 2000.
20. Leidner, D.E. Virtual partnerships in support of electronic commerce:
The case of TCIS. Journal of Strategic Information Systems, 8, 1 (1999), 105–117.
21. Liu, C., and Arnett, K. Exploring the factors associated with Web site
success in the context of electronic commerce. Information & Management, 38,
1 (2000), 23–33.
22. Loftus, B. The impact of an emerging technology on the early buyer–
seller relationship. Journal of Marketing Theory and Practice, 5, 2 (1997), 20–29.
23. Mason, R.O. Measuring information output: A communication systems
approach. Information and Management, 1, 5 (1978), 219–234.
24. Mehta, R., and Sivadas, E. Direct marketing on the Internet: An empiri-
cal assessment of consumer attitudes. Journal of Direct Marketing, 9 (1995),
21–32.
 46 WILLIAM H. DELONE AND EPHRAIM R. MCLEAN

25. Molla, A., and Licker, P.S. E-commerce systems success: An attempt to
extend and respecify the DeLone and McLean model of IS success. Journal
of Electronic Commerce Research, 2, 4 (2001), 1–11.
26. Morash, E., and Clinton, S. Supply chain integration: Customer value
through collaboration: Closeness versus operational excellence. Journal of
Marketing Theory and Practice, 6, 4 (1998), 104–119.
27. O’Callaghan, R. From reengineering to electronic commerce: Old ques-
tions, new challenges. Journal of Strategic Information Systems, 8, 1 (1999),
61–62.
28. Palmer, J.W. Web site usability, design, and performance metrics. Infor-
mation Systems Research, 13, 2 (2002), 151–167.
Downloaded by [Georgia State University] at 13:12 28 July 2015

29. Parsons, A.; Zeisser, M.; and Waitman, R. Organizing today for the
digital marketing of tomorrow. Journal of Interactive Marketing, 12, 1 (1998),
31–46.
30. Parthasarathy, M., and Bhattacherjee, A. Understanding post-adoption
behavior in the context of online services. Information Systems Research, 9, 4
(1998), 362–379.
31. Peppers, D., and Rogers, M. Enterprise One to One: Tools for Competing in
the Interactive Age. New York: Currency Doubleday, 1997.
32. Pitt, L.F.; Watson, R.T.; and Kavan, C.B. Service quality: A measure of
information systems effectiveness. MIS Quarterly, 19, 2 (1995), 173–188.
33. Quinn, C. How leading edge companies are marketing, selling, and
fulfilling over the Internet. Journal of Interactive Marketing, 13, 4 (1999),
39–50.
34. Raghunathan, M., and Madey, G. A firm-level framework for planning
electronic commerce information systems infrastructure. International Journal
of Electronic Commerce, 4, 1 (fall 1999), 121–145.
35. Rapert, M., and Brent, W. Service quality as a competitive opportunity.
Journal of Services Marketing, 12, 3 (1998), 223–235.
36. Reichheld, F., and Schefter, P. E-loyalty. Harvard Business Review, 78, 4
(2000), 195–213.
37. Reisenwitz, T., and Cutler, B. Dogmatism and Internet usage by univer-
sity students: Are dogmatics late adopters? Journal of Marketing Theory and
Practice, 6, 2 (1998), 43–50.
38. Seddon, P.B. A respecification and extension of the DeLone and McLean
model of IS success. Information Systems Research, 8, 3 (1997), 240–253.
39. Sengupta, K., and Zhao, L. Improving the communicational effectiveness
of virtual organizations through workflow automation. International Journal
of Electronic Commerce, 3, 1 (fall 1998), 49–69.
40. Shannon, C.E., and Weaver, W. The Mathematical Theory of Communica-
tion. Urbana: University of Illinois Press, 1949.
41. Shapiro, C., and Varian, H. Information Rules: A Strategic Guide to the
Network Economy. Boston: Harvard Business School Press, 1999.
42. Spiller, P., and Lohse, G. A classification of Internet retail stores. Interna-
tional Journal of Electronic Commerce, 2, 2 (winter 1998–99), 29–56.
43. Takacs, S., and Freiden, J. Changes on the electronic frontier: Growth and
opportunity of the World Wide Web. Journal of Marketing Theory and Practice,
6, 2 (1998), 24–37.
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 47

44. Teo, T., and Too, B. Information systems orientation and business use of
the Internet: An empirical study. International Journal of Electronic Commerce,
4, 4 (summer 2000), 105–130.
45. Teo, T.S.H., and Choo, W.Y. Assessing the impact of using the Internet
for competitive intelligence. Information and Management, 39, 1 (2001), 67–83.
46. Tiwana, A. Interdependency factors influencing the World Wide Web as
a channel of interactive marketing. Journal of Retailing and Consumer Services,
5, 4 (1998), 245–253.
47. Ünal, A. Electronic commerce and multi-enterprise supply/value/
business chains. Information Sciences, 127, 1/2 (2000), 63–68.
48. Varian, H. Versioning information goods. Proceedings of the Digital
Downloaded by [Georgia State University] at 13:12 28 July 2015

Information and Intellectual Property. Cambridge: Harvard University Press,

1997, pp. 1–13, www.sims.berkeley.edu/~hal/Papers/version.pdf.
49. Young, D., and Benamati, J. Differences in public Web sites: The current
state of large U.S. firms. Journal of Electronic Commerce Research, 1, 3 (2000),
94–105.
50. Zwass, V. Electronic commerce: Structures and issues. International
Journal of Electronic Commerce, 1, 1 (fall 1996), 3–23.

WILLIAM H. DELONE ([email protected]) is an associate professor of infor-

mation systems and chair of the Information Technology Department at the Kogod
School of Business at American University in Washington, DC. He has a B.S. in math-
ematics from Villanova University, an M.S. in industrial administration from Carnegie-
Mellon University, and a Ph.D. in computers and IS from the University of California,
Los Angeles. His primary areas of research include the assessment of information
systems’ effectiveness and value, the implementation and use of information technol-
ogy in small and medium-sized businesses, and the global management of IT. He has
been published in Information Systems Research, Journal of Management Information Sys-
tems, MIS Quarterly, DATABASE, Journal of Global Information Management, and Journal
of Information Technology Management.

EPHRAIM R. MCLEAN ([email protected]) is a regents’ professor and occupant of

the George E. Smith Eminent Scholar’s Chair in Information Systems at the Robinson
College of Business of Georgia State University in Atlanta. Before moving to Georgia
State University in 1987, he was on the faculty of the University of California, Los
Angeles, for 18 years. He has a B.M.E. from Cornell University and an S.M. and Ph.D.
from the Sloan School of Management at the Massachusetts Institute of Technology.
Dr. McLean’s research focuses on the management of information services, the value
of IS investments, and career issues for IS professionals. He has published more than
125 papers in Information Systems Research, Journal of Management Information Systems,
MIS Quarterly, Management Science, Communications of the ACM, DATABASE, Harvard
Business Review, Sloan Management Review, and other journals. His coauthored book,
Information Technology for Management, now in its third edition, is the second-largest-
selling IS textbook in the world. He is also the Executive Director of the Association
for Information Systems (AIS) and in 1999 was made a Fellow of the AIS.

View publication stats