01jwSTOTTLEMYER_37659 6/5/01 11:29 AM Page 1

PA R T

One
Managing the Web
Testing Process
01jwSTOTTLEMYER_37659 6/5/01 11:29 AM Page 2
01jwSTOTTLEMYER_37659 6/5/01 11:29 AM Page 3

CHAPTER

The Web Testing Process

Testing a Web site is a relatively new concept in the information technology (IT)
field. Many businesses will test one part of a Web site, failing to see the impor-
tance of testing all the major components. Many businesses have not been as
successful as others have because of this lack of testing; therefore, the need to
test different aspects of the Web site has increased.

The presence of online businesses on the World Wide Web (WWW) has
become almost overwhelming. Because of this, you must test your site if you
want to succeed, and to do so you need to identify the testing processes and
methodologies that are most applicable to your business. Individuals can pur-
chase just about anything on the Internet such as books, medicine, flowers, and
paper supplies. To compete in this market, a Web business must be able to han-
dle the volume, secure purchases, and deliver goods to customers. For this to
happen, businesses should take Web testing seriously.

3
01jwSTOTTLEMYER_37659 6/5/01 11:29 AM Page 4

4 Chapter 1

Web Testing Challenges

Understanding the Web test process is essential for deciding how to proceed
with the selection of a Web test process, automated test tools, and methodolo-
gies. Following are several challenges that need to be considered when deciding
on the Web process that is most applicable for your business:

The Web is in a state of constant change. The developer and tester need to
understand how changes will affect their development and the Web site
test process. As technology changes, testers will need to understand how
this will affect them and how they will handle their testing responsibilities.

When setting up the test scenarios, the tester needs to understand how
to implement different scenarios that will meet different types of
business requirements. For example, is a tester testing a site with graphic
user interface (GUI) buttons and text boxes or testing HyperText Markup
Language (HTML) code? Simulating response time by pressing buttons
and inputting different values will verify if correct calculations are valid.
(See the section Business Requirements for more details.)

The test environment can be a difficult part of the setup for the tester.
You need to be aware of all of the different components that make up the
environment; the networking piece can be especially difficult to simulate.
The following several considerations need to be addressed (Chapter 6,
“Preparing the Web Environment for Testing,” will further address these):
• Multiple server tiers
• Firewalls
• Databases
• Database servers
In the test environment, it is important to know how the different compo-
nents will interact with each other. This is illustrated in Figure 1.1.

When setting up the Web testing environment, special consideration

should be given to how credit card transactions are handled, carried
out, and verified. Because testers are responsible for setting up the test
scenarios, they will need to be able to simulate the quantity of transactions
that are going to be processed on the Web site.

Security is a constant concern for business on the Internet as well as for

developers and testers. There are hackers who enjoy breaking the secu-
01jwSTOTTLEMYER_37659 6/5/01 11:29 AM Page 5

The Web Testing Process 5

Web Browser

Internet

Web Server

Figure 1.1 Interaction between the Web browser, Internet, and Web server.

rity on a Web site. (We will talk about security as a methodology of testing
in Chapter 2, “Testing Methodology.”)

This book is a complete Web testing toolkit for testing today’s fast paced,
highly functional Web sites. It will walk you through the process topic by topic
and help you set up the Web test process that best fits your business.

Test Plan Development

The objective of a test plan is to provide a roadmap so that the Web site can be
evaluated through requirements or design statements. A test plan is a document
that describes objectives and the scope of a Web site project. When you prepare
a test plan, you should think through the process of the Web site test. The plan
should be written so that it can successfully give the reader a complete picture
of the Web site project and should be thorough enough to be useful. Following
are some of the items that might be included in a test plan. Keep in mind that
the items may vary depending on the Web site project.
01jwSTOTTLEMYER_37659 6/5/01 11:29 AM Page 6

6 Chapter 1

PROJECT

Title of the project:

Date:

Prepared by:

PURPOSE OF DOCUMENT

Objective of testing: Why are you testing the application? Who, what, when,
where, why, and how should be some of the questions you ask in this sec-
tion of the test plan.

Overview of the application: What is the purpose of the application? What are
the specifications of the project?

TEST TEAM

Responsible parties: Who is responsible and in charge of the testing?

List of test team: What are the names and titles of the people on the test team?

RISK ASSUMPTIONS

Anticipated risks: What types of risks are involved that could cause the test to
fail?

Similar risks from previous releases: Have there been documented risks from
previous tests that may be helpful in setting up the current test?

SCOPE OF TESTING

Possible limitations of testing: Are there any factors that may inhibit the test,
such as resources and budget?

Impossible testing: What are the considerations involved that could prevent
the tests that are planned?

Anticipated output: What are the anticipated outcomes of the test and have
they been documented for comparison?

Anticipated input: What are the anticipated outcomes that need to be com-
pared to the test documentation?
01jwSTOTTLEMYER_37659 6/5/01 11:29 AM Page 7

The Web Testing Process 7

TEST ENVIRONMENT

Hardware:

What are the operating systems that will be used?

What is the compatibility of all the hardware being used?

Software:

What data configurations are needed to run the software?

Have all the considerations of the required interfaces to other systems been
used?
Are the software and hardware compatible?

TEST DATA

Database setup requirements: Does test data need to be generated or will a

specific data from production be captured and used for testing?

Setup requirements: Who will be responsible for setting up the environment

and maintaining it throughout the testing process?

TEST TOOLS

Automated: Will automated tools be used?

Manual: Will manual testing be done?

DOCUMENTATION

Test cases: Are there test cases already prepared or will they need to be
prepared?

Test scripts: Are there test scripts already prepared or will they need to be
prepared?

PROBLEM TRACKING

Tools: What type of tools will be selected?

Processes: Who will be involved in the problem tracking process?

01jwSTOTTLEMYER_37659 6/5/01 11:29 AM Page 8

8 Chapter 1

REPORTING REQUIREMENTS

Testing deliverables: What are the deliverables for the test?

Retests: How will the retesting reporting be documented?

PERSONNEL RESOURCES

Training: Will training be provided?

Implementation: How will training be implemented?

ADDITIONAL DOCUMENTATION

Appendix: Will samples be included?

Reference materials: Will there be a glossary, acronyms, and/or data dictionary?

Once you have written your test plan, you should address some of the fol-
lowing issues and questions:

Verify plan. Make sure the plan is workable, the dates are realistic, and that
the plan is published. How will the test plan be implemented and what are
the deliverables provided to verify the test?

Validate changes. Changes should be recorded by a problem tracking sys-

tem and assigned to a developer to make revisions, retest, and sign off on
changes that have been made.

Acceptance testing. Acceptance testing allows the end users to verify that
the system works according to their expectation and the documentation.
Certification of the Web site should be recorded and signed off by the end
users, testers, and management.

Test reports. Reports should be generated and the data should be checked
and validated by the test team and users.

Web Testing Processes

The purpose of the Web testing process is to provide a clear and concise descrip-
tion of what needs to be done. Specifics of the process are discussed in the fol-
lowing subsections.
01jwSTOTTLEMYER_37659 6/5/01 11:30 AM Page 9

The Web Testing Process 9

Objectives
The objective of testing is to ensure that the Web site is ready for operation. The
test manager and the testing team have this responsibility. A Web test process
will enable a tester or developer to meet critical assignment dates, minimize
errors in testing, and improve the overall site.
It is important to realize when you select the process that it forces your test
team and all parties involved to follow a precise process of testing. The Web test
process builds on a unified process of requirements, analysis of the require-
ments, development, design, and site code.
If the Web test process is followed accurately, measurable results can be doc-
umented and presented to management and eventually the audit team. The V-
process diagram illustrates the involvement of those associated with the testing
process. Figure 1.2 illustrates the V-process; each box represents a different step
in the testing process; they are as follows:

Requirements analysis. A specification that identifies the basic requirement

functionality of the Web site.

Requirements Acceptance
Analysis Test

Architectural System
Design Integration

Detailed Software
Design Integration

Code and
Unit Test

Figure 1.2 The V-process diagram.

www.teleport.com/~qcs/papers/p821.htm
01jwSTOTTLEMYER_37659 6/5/01 11:30 AM Page 10

10 Chapter 1

Architectural design. A design specification that directs the designers in

developing and laying out the Web site.

Detailed design. A detailed layout of the specifications that shows how each
piece of the Web site will fit into place.

Code and unit test. The code is created and a unit test checks that specific
segment of code.

Software integration. The process that allows the designer to set up the soft-
ware and work with the design of the system.

System integration. The process that allows the designer to implement and
begin the implementation of the system.

Acceptance test. The final phase of testing allows the user to put the Web site
into production.

The V-process diagram is a way to look at the software flow and analyze the
development of the Web site. If testing involves the entire Web site, the test cycle
begins at the requirements phase and continues through acceptance testing.
Testing can occur in any or all of the phases in the V-process diagram; it depends
on how thoroughly management wants to test.

The test plan should have quantifiable objectives that relate to testing goals.
The level of testing that is performed will give the user optimal Web site per-
formance. This performance depends on server setup, application setup, and
the general functionality of the Web site. If the testing objectives and goals are
met, the Web site is ready for deployment.

The following are objectives the test manager should consider. To release a
quality Web site, each objective should be discussed and met:

System response. Once testing has started, working through the test cycles
should be a critical aspect. This is needed to make sure that the system is
responding correctly for the particular test. If you are expecting the system to
generate five reports a minute and you are only receiving one, an adjustment
needs to be made, which may involve data, response time, or even resources.

System availability. When working with users, it is important to make sure

that they can log onto the system. When users waste time and energy try-
ing to get onto the system to test the site, it can cause delays in your test
schedule and loss of valuable time and money.
01jwSTOTTLEMYER_37659 6/5/01 11:30 AM Page 11

The Web Testing Process 11

Defect tracking. Before testing, a method for tracking problems should be

put in place. Whether the developer or tester will log problems will be
determined based on the setup of the test team and responsibilities.
Problems need to be tracked as soon as they occur and should be tracked
throughout the development, testing, and retrofit process.

Deliverables defined. Deliverables should be measured as defined in the

test plan. The deliverables are items that can be viewed and measured,
such as the test documentation and the verifiable results on the test cases
and test scripts.

Web site expectation. The end user should be able to enter data into the sys-
tem and receive accurate data in return. This is an important concept
because the Web site is developed and tested for the user. This is a critical
component that affects the objectives; your test cycle should not continue
until there are sign offs for each test cycle.

Adequate documentation. A technical writer or a member of the test team

should be involved throughout the testing and development process to
develop documentation. Each aspect of the Web test process should be sup-
ported by documentation such as the test plan, test cases, or even the test
scripts. Documentation should also cover the Web site as a whole, such as
system, hardware and software, and server specifications.

Goals and strategies. The test team should generate a test plan that will out-
line critical dates and milestones. The test plan will be developed based on
the business requirements outlined by the developers, testers, and end
users. Include a list of goals, strategies, and a test approach as a part of the
test plan.

Web site platforms. Before testing, each platform (such as mainframe,

OS2, or NT) must be configured properly so testing can begin on time.
Once the platform is configured, the environmental test team is ready to
set up a parallel test to mirror the platform. The platforms are managed
by an environmental setup team. Throughout the test cycle process this
team will maintain and correct deficiencies in the testing environment as
they occur.

Preliminary Testing
Before beginning the actual test of the Web site, you should take care of some
preliminary testing steps to determine overall requirements.
01jwSTOTTLEMYER_37659 6/5/01 11:30 AM Page 12

12 Chapter 1

1. Assemble the test team.

2. Prepare documentation. Documentation should include:

• The business requirements that were accepted by the customer and test
team.
• The full functional design from the developer and programmer (if there
are any design problems, the tester will have an idea of the intent and
any corrections made).
• The internal design specifications (if there is a specific internal problem,
testers will know how to address it).
• Any document that may help during the testing (such as previous test
results).

3. Create a project plan.

• The project plan, reporting requirements, and required standards and
processes (such as release processes, change processes, and so on)
should be available.
• Identify high-risk aspects.

4. Design a test approach.

• Set a concrete timetable for each phase of the testing.
• Set priorities for the order of the tests.
• Set the scope and limitations of tests ahead of time. Testers need to be
aware of what should and should not be tested at this point.

5. Set up an environmental team.

• Ensure the hardware is in working order.
• Confirm that the software is ready and the Web site is set for testing.

6. Begin testing.
• All lines of communications are set up and ready to go. Communication
with involved parties is essential; the less time spent looking for
people the more time spent on actually testing the software and the
Web site.
• Decide how many cycles your test team will exercise so the environ-
ment team can make sure space is available before testing begins.
• Any tools, such as record/playback tools, should be set up and ready to go.
01jwSTOTTLEMYER_37659 6/5/01 11:30 AM Page 13

The Web Testing Process 13

7. Create test scripts.

• Test scripts are in place for test tracking. Documentation should be avail-
able at the end of each phase so the tester of the next phase is prepared
and ready to go.

8. Develop a method to do problem tracking.

• Create a form for problem and bug tracking so issues can be docu-
mented and reported to the appropriate person.

9. Decide how many cycles you want to perform.

• Establish estimates for each cycle of testing and discuss them with the
test team. There should be a determination as to how many cycles will
be needed.
• Create a timeline for each cycle (as well as the entire project). The time-
line must be updated and monitored throughout the project.
• Record each milestone as each cycle is completed.
• Log and track Web test processes.

As with any new process, follow a systematic Web testing process to ensure all
steps are covered. The process should begin by identifying the goals for the site and
how the goals will be achieved. A Web site should have an index as its home page
and each page that links from that home page needs to be incorporated into the
design. Because businesses today are relying more and more on the Web for their
success, it is important to understand the development of the site as well as the goals.

Business Requirements
Before beginning the testing project, the tester should have a set of business
requirements that will help in understanding the functionality of the Web site.
Business requirements are a collection of requests and lists from people who
have an interest in the project. There are tools available, such as Requisite Pro
from Rational, that can assist in the layout of business requirements for testing.

A well-written set of business requirements will outline the goals and objec-
tives for the business and serve as the foundation for your test plan. Business
requirements are the high-level objectives of the organization; these objectives
answer the needs of the business as defined by the project’s vision and scope.
01jwSTOTTLEMYER_37659 6/5/01 11:30 AM Page 14

14 Chapter 1

Business requirements are written after a team of developers, users, and pro-
grammers discuss and define the scope of the project. The requirements out-
line the purpose and implementation of the application and describe the
behavior of the system. The team should write the information and develop a
representation of the intended purpose of the application. It is from these ideas
that the functional requirements are established to accomplish the goals of the
proposed Web site.

Following are some ideas for writing effective requirements:

• Keep sentences and paragraphs short and to the point.

• Use the active voice throughout the requirement.

• Check your spelling, grammar, and punctuation.

• Use terminology that is defined in the glossary and data dictionary.

• Do not be vague, and make sure you know exactly what the user means.

• Be detailed enough so the requirement can be carried out.

When a system is designed, users, developers, and programmers work

together to decide how the Web site will function. It is from these sessions that
requirements are developed and set up based on functional requirements. It is
advisable for the developers, programmers, and users to initiate these working
sessions as a team to ensure the accuracy of the development.

Business requirements, which follow, should be written with a results-

oriented focus:

• Complete (developed as a functional entity)

• Consistent (in line with the design)

• Correct (reflect Web site accurately)

• Nonambiguous (only one meaning)

• Noncompound (stand alone)

01jwSTOTTLEMYER_37659 6/5/01 11:30 AM Page 15

The Web Testing Process 15

The format for business requirements follows:

• Overview and summary of product

• Development, operation, and maintenance of the environment

• External interfaces and data flow

• Report format
• User displays
• Data flow diagrams
• Logical data stores
• Data dictionary

• Functional specifications

• Performance requirements

• Condition and handling exceptions

• Priorities

• Modifications and/or enhancements

• Acceptance requirements

• Standards for documentation

• Functional tests

• Performance tests

• Guidelines for design

• Sources of information (for example, white papers, or online documentation)

• Glossary of terms

After the business requirements have been developed, the next step is to
decide how to implement and test them using these requirements.
01jwSTOTTLEMYER_37659 6/5/01 11:30 AM Page 16

16 Chapter 1

Testing Phases
As the business requirements are established and defined, they will become one
of the first phases of your testing process. Understanding their magnitude will
help you determine how you will proceed with the Web test. This complete
understanding will also help you determine the number of test cycles, test the
data used, and set up the test environment.

Communication
Communication is a vital element to make sure that the test team is informed of
specific test tasks and changes to the testing process. The test plan is critical in doc-
umenting the goals and strategies for the product and its test approach and will
help to determine whether the test goals are handled properly. A good way to track
the testing process is to create a checklist to make sure that you are following and
completing the test process. Table 1.1 is an example of a testing checklist. Each item
in the checklist (or checkpoint) should be a part of the test process and depends on
the test life cycle, specification, management, commitment, and communication.

Testing Environment
The test environment should be planned with the number of cycles, type of test
data, and the way to test the Web site in mind. This environment consists of the
hardware, software, network, and logistics. Early in the stages of testing, it is nec-
essary to determine how this will be set up and deployed. You can set up a test lab
or test at the user’s workstation. The test data should be selected and kept separate
from production data. Remember, backups are essential after each test run.

The hardware should be capable of handling the test load. Users and testers
should be given separate passwords for the test environment. The software
needs to be correctly configured and monitored for accuracy. The test environ-
ment should be set up to handle the correct servers and locations as well as
accurate URLs, IP addresses, and any essential server information that is rele-
vant for the test. Testing setup should be readily accessible and the tasks should
be easy to perform. If a lab is designed so that it can be used for other projects,
cost benefits may be realized.

Resources
Resources are critical components of the testing process. It is important to decide
how best to use available resources and how to obtain reliable ones. As a tester, it
01jwSTOTTLEMYER_37659 6/5/01 11:30 AM Page 17

The Web Testing Process 17

Table 1.1 Testing Checklist

QUESTIONS COMMENTS CHECK

Who can test?

Why do you want to test?

Who should test?

What do you test?

How do you document results?

How do you script the test?

Who develops the test bed?

What is the user’s role in testing?

What do you do when you are done testing?

Whom are results reported to?

What happens if the audit team calls you in

to answer questions?

When do you see the developers as opposed to

the programmers?

As a tester, how do you implement fixes

and retest?

How do you track problems?

Who will do the fixes?

How will white papers benefit you when

you test the Web site?

Who will write the test plan?

Who is on the test team?

Who sets up the test environment?

Who maintains the product once it is tested?

Will the test environment be used in the

future?

Who will need training?

How do we rate the bugs?

Have you done enough research to start

the testing?
01jwSTOTTLEMYER_37659 6/5/01 11:30 AM Page 18

18 Chapter 1

is important to focus on the goals. Remember that the purpose of testing is to

uncover errors in code and Web site design. If you work in an organization, you
should focus on several testing items. You need to determine the size of the project
and figure out how many resources are required. The test team will be key in
determining the amount of risks involved in setting up the testing environment
and the testing process. If you have limited testing resources, you will have to
determine what parts of the Web site are critical to test before they are released.

Test goals are achieved by working with objectives that are critical to the suc-
cess of the Web site. Following are questions you should try to answer:

• Does a Web site run as expected?

• Does the Web site meet business requirements?

• Does the Web site run as the programmer designed it?

• Does the Web site run in a timely fashion?

• Is the Web site well documented?

• Are business requirements available?

• Are the design and code available?

• Are definable business requirements set up?

• Are business requirements realistic?

• Has everyone involved agreed on the defined requirements?

• Is the development team aware of the defined requirements?

• Have the users been involved in the development of requirements?

• Is testing performed within a workable schedule so the Web site can be

released in a timely manner?

• Is the timetable reasonable?

• Has everyone been informed of his or her role and task with relation to the
timetable?

• Does testing involve all platforms that will be applicable for the Web site?
01jwSTOTTLEMYER_37659 6/5/01 11:30 AM Page 19

The Web Testing Process 19

• What platforms are involved?

• How many platforms need to be tested?

• Are testers able to uncover errors?

• What are the different types of errors?

• What are expected errors?

• Is there a problem tracking system that will be set up?

• What type of problem tracking system is set up?

• Who will track the problems?

• Does testing involve end users?

• What role does the end user play in the Web test process?

• Who will work with the end user throughout the process?

• Have measurable test specifications been set up?

• How will the standards be measured?

• What will be measured and how will measurements be tracked?

• Will the test include prior measurable results?

Tester Qualifications
A good tester should have a combination of the following skills:

Communication. The ability to convey to the developers, testers, and users

the intent of testing and the roles and responsibilities of all parties.

Technical expertise. The ability to understand the Web site and how it works.

Diplomacy. The ability to work well with others and come up with the best
solution for the team.

Accuracy. The ability to produce error-free results.

Persistence. The ability to test and retest until an adequate result is achieved.
01jwSTOTTLEMYER_37659 6/5/01 11:30 AM Page 20

20 Chapter 1

Of these qualities, persistence is the most important. A tester should have the
ability to continuously test and retest without becoming bored or losing focus.
Being able to endure this process allows the tester to assure accurate results, and
the end result is happy customers.

Deadlines
After the resources and environment have been laid out, it is necessary to set up
deadlines to achieve the testing goals. This can be handled by using a project
management tracking tool such as Microsoft Project. This tool will allow you to
set up a baseline and track your project based on resources, times, constraints,
and milestones. The information can be presented in a manner that is useful for
management and the test team.

Problem Tracking
Tracking the testing process using a problem tracking tool will produce verifi-
able test results. There are many different types of tools available (see Chapter
5, “Web Site Testing Tools,” for a listing of some of these tools). Some of the fea-
tures you should include or look for in a tracking tool are date of test error and
date of resolution, type of problem and its severity, description and resolution,
and problem assigned to/number of days to fix.

Configuration Management
A process that has become important not only to developers but also to testers is
a way to track development using configuration management. Configuration man-
agement is the ability to track and control Web site development and its activities.
By integrating problem management with configuration management, you can
gain control of development activities and improve quality. For example, if multi-
ple developers are working on the same code at the same time, targeting multiple
platforms that support multiple programs, configuration management becomes
crucial. In today’s development and testing field, problem tracking and configu-
ration management have been combined, which allows for problems and changes
to be tracked and assists in the overall process of the Web test.

Web Site Design

The developer/designer of the Web site (who may also be the Webmaster) has
the critical function of designing the Web site and maintaining it to the best inter-
est of the business. It is important for this individual to have the expertise to set
01jwSTOTTLEMYER_37659 6/5/01 11:30 AM Page 21

The Web Testing Process 21

up and design a site that will benefit the business. The Web tester must under-
stand the design and have a basic understanding of the language and technol-
ogy that is used to design the site.

The first important step is to map out the Web site. Mapping out a site involves
laying out the design and making sure that each page links to the assigned page.
After you map out the site, you should identify all URLs (it’s a good idea to set
up a database of these URLs). URLs can be assigned a number so that tracking
the links during the testing process becomes easier. A test script can be written
by going step by step through each URL, and the results can be verified by print-
ing them to an .asp page or a text report (see Chapter 11, “Analyzing the Test
Process and Documentation,” for an example test script). Mapping out your site
will make it easier to develop tests that exercise the Web site design. The
components involved in performing the test are HTML code, broken links,
spelling, download time, and browsers.

A design layout tool such as Visio can be used to set up a schema of the Web
map. Testing tools such as Astra can show the schema of the Web site and its
links, giving the home-page address. As the tester gains an understanding of the
layout of the pages, he or she will have a better understanding of the way the
Web site is organized.

Automated Test Process

A test program that incorporates automated testing will involve a development
effort of strategy, goal planning, test requirements definition, analysis, design,
development, execution, and evaluation. Because organizations are required to
do more with less, automated testing can save time and money.

Automated testing is important to all testing because you can reuse code and
scripts and allow testers to standardize the testing process. In the Web environ-
ment, automated testing is performed across many platforms, multiple layers of
supporting applications, interfaces, databases, and different applications that
can serve as a front or back end to the application. Testing in this environment
can include the following:

• Functional requirement testing

• Server performance testing

• User interface testing

01jwSTOTTLEMYER_37659 6/5/01 11:30 AM Page 22

22 Chapter 1

• Unit testing

• Integration testing

• Program module complexity analysis

• Program code coverage

• System load performance testing

• Boundary testing

• Security testing

• Memory leak testing

• Firewall testing

Automation has made these types of testing more efficient and provided more
accurate results.

Strategy
The test strategy is an overall approach to testing; it identifies the levels of test-
ing applied and the methods, techniques, and tools that will be used. A test
strategy should be adopted by the organization as a whole; it is critical to the suc-
cess of the Web development and should be detailed in the test plan. Figure 1.3
illustrates the approach to test strategies. Web development, test management,
and Web testing are all critical to the overall testing strategy.

A strategy should be developed that encompasses the following:

Software development. A software development strategy should be set up

to illustrate and document the flow through the life cycle development.

Testing. Testing will validate efforts, and measurable results, such as num-
ber of test cases, number of errors detected, severity of errors, time spent,
and which types of testing will be performed, should be documented.

Project management. A workable schedule and timeline should be a major

part of the overall test strategy. This timeline needs to be incorporated into
01jwSTOTTLEMYER_37659 6/5/01 11:30 AM Page 23

The Web Testing Process 23

Software
Development

Project
Testing
Management
Test
Environment

Figure 1.3 Test strategy.

the test plan and published so that all parties involved know their part in
the testing process.

Test tools. Appropriate tools need to be selected that are applicable for the
application that requires testing. Testing tools should be used if repetitive
or a large volume of testing is needed, but it may not be feasible to use
automated tools if you are only testing a small application.

Metrics. The ability to track results is important when working through the test
process. A decision needs to be made about how measurable results will be
tracked and recorded. Several types of metrics that can be tracked are system
response time, reliability, efficiency, usability, portability, and number of
defects. Figure 1.4 illustrates some of the metrics that will need to be tracked.

Employees. Employees and consultants are a critical part of the test process;
how many will be needed should be determined. Employees may also
need appropriate training. Testing success is based on the effectiveness of
testers and their ability to follow the test process.

Setting Up the Test Strategy

When you set up the test strategy, you should determine the approach of your
testing methodology. You should have a strategy for Web site development,
testing, project management, test tools, metrics, and employees. You need to
01jwSTOTTLEMYER_37659 6/5/01 11:30 AM Page 24

24 Chapter 1

100

90

80

70

60
Response
50 Reliability
Efficiency
40

30

20

10

0
Cycle 1 Cycle 2 Cycle 3 Cycle 4

Figure 1.4 Tracking metrics.

decide whether your tests will be manual, automated, or a combination of both.

The test strategy will affect the testing team, developers, and end users. The test
team is instrumental in the decision process and implementation of the testing
process. A test procedure should specify the process and not leave out steps or
make assumptions.

Web Test Analysis

Once the Web test process has been put into place, the following information
should be analyzed to ensure that the Web test process is accurate and will
return required test information:

• The test objectives should be applicable, reasonable, adequate, feasible, and

affordable and should follow the needs of the business requirements.
01jwSTOTTLEMYER_37659 6/5/01 11:30 AM Page 25

The Web Testing Process 25

• The test program should meet the test goals and objectives.

• The correct test program and test plan should be applied to the project.

• The test methodology, which includes the processes, infrastructure, tools,

methods, and planned work products and reviews, should be adequate
and updated to ensure that the test program and test plan are done
correctly.

• The test work products should be adequate to meet the test goals and
objectives.

• Test progress, performance, processes, and process adherence should be

assessed to determine the adequacy of the test program and test plan.

• Adequate testing should be performed to ensure test integrity.

Summary
In this chapter we have focused on the Web test process, objectives, strate-
gies, and essential documentation and goals. Chapter 2 will discuss testing
methodologies.
01jwSTOTTLEMYER_37659 6/5/01 11:30 AM Page 26