Download the Full Version of the Ebook with Added Features ebookname.
com
Effective software testing 50 specific ways to
improve your testing 5th printing Edition Dustin
https://ebookname.com/product/effective-software-
testing-50-specific-ways-to-improve-your-testing-5th-
printing-edition-dustin/
OR CLICK HERE
DOWLOAD NOW
Download more ebook instantly today at https://ebookname.com
Instant digital products (PDF, ePub, MOBI) available
Download now and explore formats that suit you...
Effective C Covers C 4 0 50 Specific Ways to Improve Your
C 2nd Edition Bill Wagner
https://ebookname.com/product/effective-c-covers-c-4-0-50-specific-
ways-to-improve-your-c-2nd-edition-bill-wagner/
ebookname.com
Effective TypeScript 83 Specific Ways to Improve Your
TypeScript 2nd Edition Dan Vanderkam
https://ebookname.com/product/effective-typescript-83-specific-ways-
to-improve-your-typescript-2nd-edition-dan-vanderkam/
ebookname.com
Effective TypeScript 62 Specific Ways to Improve Your
TypeScript 1st Edition Dan Vanderkam
https://ebookname.com/product/effective-typescript-62-specific-ways-
to-improve-your-typescript-1st-edition-dan-vanderkam/
ebookname.com
Science Learning Science Teaching 3rd Edition Jerry
Wellington
https://ebookname.com/product/science-learning-science-teaching-3rd-
edition-jerry-wellington/
ebookname.com
Antisemitism Volume 16 2000 Susan Sarah Cohen (Editor)
https://ebookname.com/product/antisemitism-volume-16-2000-susan-sarah-
cohen-editor/
ebookname.com
Evaluating Clinical and Public Health Interventions A
Practical Guide to Study Design and Statistics 1st Edition
Mitchell H. Katz
https://ebookname.com/product/evaluating-clinical-and-public-health-
interventions-a-practical-guide-to-study-design-and-statistics-1st-
edition-mitchell-h-katz/
ebookname.com
Understanding American Sports In Culture and Society 1st
Edition Gerald R. Gems
https://ebookname.com/product/understanding-american-sports-in-
culture-and-society-1st-edition-gerald-r-gems/
ebookname.com
Vader s Little Princess Jeffrey Brown
https://ebookname.com/product/vader-s-little-princess-jeffrey-brown/
ebookname.com
Services Trade Reform 1st Edition Dee Philippa
https://ebookname.com/product/services-trade-reform-1st-edition-dee-
philippa/
ebookname.com
Fluid mechanics 4ed Edition Kundu P.K.
https://ebookname.com/product/fluid-mechanics-4ed-edition-kundu-p-k/
ebookname.com
EFFECTIVE
S O F T WA R E
TESTING
This page intentionally left blank
EFFECTIVE
S O F T WA R E
TESTING
50 Specific Ways to Improve Your Testing
E L F R I E D E D U S T I N
Boston • San Francisco • New York • Toronto • Montreal
London • Munich • Paris • Madrid
Capetown • Sydney • Tokyo • Singapore • Mexico City
Many of the designations used by manufacturers and sellers to distinguish their products are claimed
as trademarks. Where those designations appear in this book, and Addison-Wesley was aware of a
trademark claim, the designations have been printed with initial capital letters or in all capitals.
The author and publisher have taken care in the preparation of this book, but make no expressed or
implied warranty of any kind and assume no responsibility for errors or omissions. No liability is
assumed for incidental or consequential damages in connection with or arising out of the use of the
information or programs contained herein.
The publisher offers discounts on this book when ordered in quantity for bulk purchases and special
sales. For more information, please contact:
U.S. Corporate and Government Sales
(800) 382-3419
[email protected]For sales outside of the U.S., please contact:
International Sales
(317) 581-3793
[email protected]
Visit Addison-Wesley on the Web: www.awprofessional.com
Library of Congress Cataloging-in-Publication Data
Dustin, Elfriede.
Effective software testing : 50 specific ways to improve your testing / Elfriede Dustin.
p. cm.
Includes bibliographical references and index.
ISBN 0-201-79429-2
1. Computer software--Testing. I. Title.
QA76.76.T48 D873 2002
005.1'4—dc21
2002014338
Copyright © 2003 by Pearson Education, Inc.
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or
transmitted, in any form, or by any means, electronic, mechanical, photocopying, recording, or oth-
erwise, without the prior consent of the publisher. Printed in the United States of America. Published
simultaneously in Canada.
For information on obtaining permission for use of material from this work, please submit a written
request to:
Pearson Education, Inc.
Rights and Contracts Department
75 Arlington Street, Suite 300
Boston, MA 02116
Fax: (617) 848-7047
ISBN: 0-201-79429-2
2 3 4 5 6 7 8 9 10—OPM—070605
2nd printing, March 2005
To Jackie, Erika, and Cedric
This page intentionally left blank
Contents
Preface xi
Acknowledgments xv
1. Requirements Phase 1
Item 1: Involve Testers from the Beginning 3
Item 2: Verify the Requirements 5
Item 3: Design Test Procedures As Soon As Requirements
Are Available 11
Item 4: Ensure That Requirement Changes Are
Communicated 15
Item 5: Beware of Developing and Testing Based on an
Existing System 19
2. Test Planning 23
Item 6: Understand the Task At Hand and the Related
Testing Goal 25
Item 7: Consider the Risks 31
Item 8: Base Testing Efforts on a Prioritized Feature
Schedule 39
Item 9: Keep Software Issues in Mind 41
Item 10: Acquire Effective Test Data 43
Item 11: Plan the Test Environment 47
Item 12: Estimate Test Preparation and Execution Time 51
vii
viii CONTENTS
3. The Testing Team 63
Item 13: Define Roles and Responsibilities 65
Item 14: Require a Mixture of Testing Skills,
Subject-Matter Expertise, and Experience 75
Item 15: Evaluate the Tester’s Effectiveness 79
4. The System Architecture 91
Item 16: Understand the Architecture and
Underlying Components 93
Item 17: Verify That the System Supports Testability 97
Item 18: Use Logging to Increase System Testability 99
Item 19: Verify That the System Supports Debug
and Release Execution Modes 103
5. Test Design and Documentation 107
Item 20: Divide and Conquer 109
Item 21: Mandate the Use of a Test-Procedure
Template and Other Test-Design Standards 115
Item 22: Derive Effective Test Cases from Requirements 121
Item 23: Treat Test Procedures As “Living” Documents 125
Item 24: Utilize System Design and Prototypes 127
Item 25: Use Proven Testing Techniques when
Designing Test-Case Scenarios 129
Item 26: Avoid Including Constraints and Detailed
Data Elements within Test Procedures 135
Item 27: Apply Exploratory Testing 139
6. Unit Testing 143
Item 28: Structure the Development Approach to
Support Effective Unit Testing 145
Item 29: Develop Unit Tests in Parallel or Before
the Implementation 151
Item 30: Make Unit-Test Execution Part of the
Build Process 155
7. Automated Testing Tools 159
Item 31: Know the Different Types of
Testing-Support Tools 161
Item 32: Consider Building a Tool Instead of Buying One 167
CONTENTS ix
Item 33: Know the Impact of Automated Tools
on the Testing Effort 171
Item 34: Focus on the Needs of Your Organization 177
Item 35: Test the Tools on an Application Prototype 183
8. Automated Testing:
Selected Best Practices 185
Item 36: Do Not Rely Solely on Capture/Playback 187
Item 37: Develop a Test Harness When Necessary 191
Item 38: Use Proven Test-Script Development Techniques 197
Item 39: Automate Regression Tests When Feasible 201
Item 40: Implement Automated Builds and Smoke Tests 207
9. Nonfunctional Testing 211
Item 41: Do Not Make Nonfunctional Testing
an Afterthought 213
Item 42: Conduct Performance Testing with
Production-Sized Databases 217
Item 43: Tailor Usability Tests to the Intended Audience 221
Item 44: Consider All Aspects of Security, for
Specific Requirements and System-Wide 225
Item 45: Investigate the System’s Implementation
To Plan for Concurrency Tests 229
Item 46: Set Up an Efficient Environment for
Compatibility Testing 235
10. Managing Test Execution 239
Item 47: Clearly Define the Beginning and End of the
Test-Execution Cycle 241
Item 48: Isolate the Test Environment from the
Development Environment 245
Item 49: Implement a Defect-Tracking Life Cycle 247
Item 50: Track the Execution of the Testing Program 255
Index 259
This page intentionally left blank
Preface
I
n most software-development organizations, the testing program functions as the
final “quality gate” for an application, allowing or preventing the move from the
comfort of the software-engineering environment into the real world. With this
role comes a large responsibility: The success of an application, and possibly of the
organization, can rest on the quality of the software product.
A multitude of small tasks must be performed and managed by the testing
team—so many, in fact, that it is tempting to focus purely on the mechanics of test-
ing a software application and pay little attention to the surrounding tasks required
of a testing program. Issues such as the acquisition of proper test data, testability of
the application’s requirements and architecture, appropriate test-procedure stan-
dards and documentation, and hardware and facilities are often addressed very late,
if at all, in a project’s life cycle. For projects of any significant size, test scripts and
tools alone will not suffice—a fact to which most experienced software testers will
attest.
Knowledge of what constitutes a successful end-to-end testing effort is typically
gained through experience. The realization that a testing program could have been
much more effective had certain tasks been performed earlier in the project life cycle
is a valuable lesson. Of course, at that point, it’s usually too late for the current
project to benefit from the experience.
Effective Software Testing provides experience-based practices and key concepts
that can be used by an organization to implement a successful and efficient testing
program. The goal is to provide a distilled collection of techniques and discussions
xi
xii PREFACE
that can be directly applied by software personnel to improve their products and
avoid costly mistakes and oversights. This book details 50 specific software testing
best practices, contained in ten parts that roughly follow the software life cycle. This
structure itself illustrates a key concept in software testing: To be most effective, the
testing effort must be integrated into the software-development process as a whole.
Isolating the testing effort into one box in the “work flow” (at the end of the software
life cycle) is a common mistake that must be avoided.
The material in the book ranges from process- and management-related topics,
such as managing changing requirements and the makeup of the testing team, to
technical aspects such as ways to improve the testability of the system and the inte-
gration of unit testing into the development process. Although some pseudocode is
given where necessary, the content is not tied to any particular technology or appli-
cation platform.
It is important to note that there are factors outside the scope of the testing pro-
gram that bear heavily on the success or failure of a project. Although a complete
software-development process with its attendant testing program will ensure a suc-
cessful engineering effort, any project must also deal with issues relating to the busi-
ness case, budgets, schedules, and the culture of the organization. In some cases,
these issues will be at odds with the needs of an effective engineering environment.
The recommendations in this book assume that the organization is capable of
adapting, and providing the support to the testing program necessary for its success.
O R G A N I Z AT I O N
This book is organized into 50 separate items covering ten important areas. The
selected best practices are organized in a sequence that parallels the phases of the
system development life cycle.
The reader can approach the material sequentially, item-by-item and part-by-
part, or simply refer to specific items when necessary to gain information about and
understanding of a particular problem. For the most part, each chapter stands on its
own, although there are references to other chapters, and other books, where helpful
to provide the reader with additional information.
Chapter 1 describes requirements-phase considerations for the testing effort.
It is important in the requirements phase for all stakeholders, including a represen-
tative of the testing team, to be involved in and informed of all requirements and
changes. In addition, basing test cases on requirements is an essential concept for
any large project. The importance of having the testing team represented during this
PREFACE xiii
phase cannot be overstated; it is in this phase that a thorough understanding of the
system and its requirements can be obtained.
Chapter 2 covers test-planning activities, including ways to gain understanding
of the goals of the testing effort, approaches to determining the test strategy, and
considerations related to data, environments, and the software itself. Planning must
take place as early as possible in the software life cycle, as lead times must be consid-
ered for implementing the test program successfully. Early planning allows for test-
ing schedules and budgets to be estimated, approved, and incorporated into the
overall software development plan. Estimates must be continually monitored and
compared to actuals, so they can be revised and expectations can be managed as
required.
Chapter 3 focuses on the makeup of the testing team. At the core of any success-
ful testing program are its people. A successful testing team has a mixture of techni-
cal and domain knowledge, as well as a structured and concise division of roles and
responsibilities. Continually evaluating the effectiveness of each test-team member
throughout the testing process is important to ensuring success.
Chapter 4 discusses architectural considerations for the system under test.
Often overlooked, these factors must be taken into account to ensure that the sys-
tem itself is testable, and to enable gray-box testing and effective defect diagnosis.
Chapter 5 details the effective design and development of test procedures,
including considerations for the creation and documentation of tests, and discusses
the most effective testing techniques. As requirements and system design are refined
over time and through system-development iterations, so must the test procedures
be refined to incorporate the new or modified requirements and system functions.
Chapter 6 examines the role of developer unit testing in the overall testing strat-
egy. Unit testing in the implementation phase can result in significant gains in soft-
ware quality. If unit testing is done properly, later testing phases will be more
successful. There is a difference, however, between casual, ad-hoc unit testing based
on knowledge of the problem, and structured, repeatable unit testing based on the
requirements of the system.
Chapter 7 explains automated testing tool issues, including the proper types of
tools to use on a project, the build-versus-buy decision, and factors to consider in
selecting the right tool for the organization. The numerous types of testing tools
available for use throughout the phases in the development life cycle are described
here. In addition, custom tool development is also covered.
Chapter 8 discusses selected best practices for automated testing. The proper use
of capture/playback tools, test harnesses, and regression testing are described.
xiv PREFACE
Chapter 9 provides information on testing nonfunctional aspects of a software
application. Ensuring that nonfunctional requirements are met, including perfor-
mance, security, usability, compatibility, and concurrency testing, adds to the overall
quality of the application.
Chapter 10 provides a strategy for managing the execution of tests, including
appropriate methods of tracking test-procedure execution and the defect life cycle,
and gathering metrics to assess the testing process.
AUDIENCE
The target audience of this book includes Quality Assurance professionals, software
testers, and test leads and managers. Much of the information presented can also be
of value to project managers and software developers looking to improve the qual-
ity of a software project.
Discovering Diverse Content Through
Random Scribd Documents
If the second copy is also defective, you may demand a refund
in writing without further opportunities to fix the problem.
1.F.4. Except for the limited right of replacement or refund set
forth in paragraph 1.F.3, this work is provided to you ‘AS-IS’,
WITH NO OTHER WARRANTIES OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR ANY PURPOSE.
1.F.5. Some states do not allow disclaimers of certain implied
warranties or the exclusion or limitation of certain types of
damages. If any disclaimer or limitation set forth in this
agreement violates the law of the state applicable to this
agreement, the agreement shall be interpreted to make the
maximum disclaimer or limitation permitted by the applicable
state law. The invalidity or unenforceability of any provision of
this agreement shall not void the remaining provisions.
1.F.6. INDEMNITY - You agree to indemnify and hold the
Foundation, the trademark owner, any agent or employee of the
Foundation, anyone providing copies of Project Gutenberg™
electronic works in accordance with this agreement, and any
volunteers associated with the production, promotion and
distribution of Project Gutenberg™ electronic works, harmless
from all liability, costs and expenses, including legal fees, that
arise directly or indirectly from any of the following which you
do or cause to occur: (a) distribution of this or any Project
Gutenberg™ work, (b) alteration, modification, or additions or
deletions to any Project Gutenberg™ work, and (c) any Defect
you cause.
Section 2. Information about the Mission
of Project Gutenberg™
Project Gutenberg™ is synonymous with the free distribution of
electronic works in formats readable by the widest variety of
computers including obsolete, old, middle-aged and new
computers. It exists because of the efforts of hundreds of
volunteers and donations from people in all walks of life.
Volunteers and financial support to provide volunteers with the
assistance they need are critical to reaching Project
Gutenberg™’s goals and ensuring that the Project Gutenberg™
collection will remain freely available for generations to come. In
2001, the Project Gutenberg Literary Archive Foundation was
created to provide a secure and permanent future for Project
Gutenberg™ and future generations. To learn more about the
Project Gutenberg Literary Archive Foundation and how your
efforts and donations can help, see Sections 3 and 4 and the
Foundation information page at www.gutenberg.org.
Section 3. Information about the Project
Gutenberg Literary Archive Foundation
The Project Gutenberg Literary Archive Foundation is a non-
profit 501(c)(3) educational corporation organized under the
laws of the state of Mississippi and granted tax exempt status
by the Internal Revenue Service. The Foundation’s EIN or
federal tax identification number is 64-6221541. Contributions
to the Project Gutenberg Literary Archive Foundation are tax
deductible to the full extent permitted by U.S. federal laws and
your state’s laws.
The Foundation’s business office is located at 809 North 1500
West, Salt Lake City, UT 84116, (801) 596-1887. Email contact
links and up to date contact information can be found at the
Foundation’s website and official page at
www.gutenberg.org/contact
Section 4. Information about Donations to
the Project Gutenberg Literary Archive
Foundation
Project Gutenberg™ depends upon and cannot survive without
widespread public support and donations to carry out its mission
of increasing the number of public domain and licensed works
that can be freely distributed in machine-readable form
accessible by the widest array of equipment including outdated
equipment. Many small donations ($1 to $5,000) are particularly
important to maintaining tax exempt status with the IRS.
The Foundation is committed to complying with the laws
regulating charities and charitable donations in all 50 states of
the United States. Compliance requirements are not uniform
and it takes a considerable effort, much paperwork and many
fees to meet and keep up with these requirements. We do not
solicit donations in locations where we have not received written
confirmation of compliance. To SEND DONATIONS or determine
the status of compliance for any particular state visit
www.gutenberg.org/donate.
While we cannot and do not solicit contributions from states
where we have not met the solicitation requirements, we know
of no prohibition against accepting unsolicited donations from
donors in such states who approach us with offers to donate.
International donations are gratefully accepted, but we cannot
make any statements concerning tax treatment of donations
received from outside the United States. U.S. laws alone swamp
our small staff.
Please check the Project Gutenberg web pages for current
donation methods and addresses. Donations are accepted in a
number of other ways including checks, online payments and
credit card donations. To donate, please visit:
www.gutenberg.org/donate.
Section 5. General Information About
Project Gutenberg™ electronic works
Professor Michael S. Hart was the originator of the Project
Gutenberg™ concept of a library of electronic works that could
be freely shared with anyone. For forty years, he produced and
distributed Project Gutenberg™ eBooks with only a loose
network of volunteer support.
Project Gutenberg™ eBooks are often created from several
printed editions, all of which are confirmed as not protected by
copyright in the U.S. unless a copyright notice is included. Thus,
we do not necessarily keep eBooks in compliance with any
particular paper edition.
Most people start at our website which has the main PG search
facility: www.gutenberg.org.
This website includes information about Project Gutenberg™,
including how to make donations to the Project Gutenberg
Literary Archive Foundation, how to help produce our new
eBooks, and how to subscribe to our email newsletter to hear
about new eBooks.
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade
Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.
Let us accompany you on the journey of exploring knowledge and
personal growth!
ebookname.com
