Overview of Software Testing

Overview of Software Testing

3/9/2010

References
[1], [3], [4] Chapter 1

Overview of Software Testing

3/9/2010

Content
What is Software Testing? Activities of Test Engineering Software Testing Taxonomy Test Case Design Methods Principles of Software Testing Software Testing Myths Software Testing Limits

Overview of Software Testing

3/9/2010

What is Software Testing

Most Common Software problems
Incorrect calculation, Incorrect d edits l l data d Incorrect matching and merging of data Data searches that yields incorrect results y Incorrect processing of data relationship Incorrect coding/implementation of business rules Inadequate software performance I d t ft f Unreliable results or performance Inadequate support of business needs Incorrect or inadequate interfaces with other systems Inadequate performance and security controls
4 Overview of Software Testing 3/9/2010

What is Software Testing

Several definitions:
Testing is the process of demonstrating that errors are not present The purpose of testing is to show that a program performs its intended functions correctly Testing is the process of establishing confidence that a program or system does what it is supposed to. Testing is the process of executing a program or system with the intent of finding errors.

Program testing is more properly viewed as the destructive process of trying to find the errors (whose presence is assumed) in a program
5 Overview of Software Testing 3/9/2010

What is Software Testing

The Major Objectives of Software Testing:
Uncover as many as errors (or bugs) as possible in a given timeline. Demonstrate a given software product matching its requirement specifications. Validate the quality of a software testing using the minimum cost and efforts. Generate high quality test cases, perform effective tests, and issue correct and helpful problem reports.

Overview of Software Testing

3/9/2010

What is Software Testing

Software testing is one element of a broader topic that is often referred to as Verification and Validation (V&V)

Verification:
The Th process of determining whether the products of a given fd i i h h h d f i phase of the software development process fulfill the requirements established during the previous phase (Are we building the product right?) Verification is usually a more technical activity that uses knowledge about the individual software artifacts artifacts, requirements, and specifications. Verification activities include testing and reviews
7 Overview of Software Testing 3/9/2010

What is Software Testing

Validation:
The process of evaluating software at the end of software development to ensure compliance with intended usage (Are we building the right product?) g g p ) Validation activities are used to evaluate whether the features that have been built into the software satisfy the customer requirements and are traceable to customer requirements i d bl i Validation usually depends on domain knowledge

Overview of Software Testing

3/9/2010

What is Software Testing

Overview of Software Testing

3/9/2010

Content
What is Software Testing? Activities of Test Engineer Software Testing Taxonomy Test Case Design Methods Principles of Software Testing Software Testing Myths Software Testing Limits

10

Overview of Software Testing

3/9/2010

Activities of Test Engineer

Test Engineer
An information technology (IT) professional who is in charge of one or more technical test activities
Define test cases, write test specifications, Run tests Analyzing results, and reporting results to developers and managers.

Test Manager
Set test policies and p p processes, interact with other managers on g the project, and otherwise help the engineers do their work

11

Overview of Software Testing

3/9/2010

Activities of Test Engineer

12

Overview of Software Testing

3/9/2010

Activities of Test Engineer

Automation of Test Activities
Software testing is expensive and labor intensive. Software testing S ft t ti i i d l b i t i S ft t ti requires up to 50% of software development costs, and even more for safety-critical applications. => O of th goals of software testing is to automate as much as One f the l f ft t ti i t t t h possible, thereby significantly reducing its cost, minimizing human error, and making regression testing easier. Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions Example: Stress Test, Selenium, TestComplete, IBM Rational Functional Tester
13 Overview of Software Testing 3/9/2010

Content
What is Software Testing? Activities of Test Engineer Software Testing Taxonomy Test Case Design Methods Principles of Software Testing Software Testing Myths Software Testing Limits

14

Overview of Software Testing

3/9/2010

Software Testing Taxonomy

Testing Levels Based on Software Activity Beizers Testing Levels Based on Test Process Maturity
[1] Chapter 1

15

Overview of Software Testing

3/9/2010

Testing Levels Based on Software Activity A ti it

16

Overview of Software Testing

3/9/2010

Testing Levels Based on Software Activity A ti it

Acceptance Testing assess software with respect to requirements. System Testing assess software with respect to architectural design design.
Performance Testing Stress Testing g Security Testing

Integration Testing assess software with respect to subsystem design.

17 Overview of Software Testing 3/9/2010

Testing Levels Based on Software Activity A ti it

Module Testing assess software with respect to detailed design. Unit Testing assess software with respect to implementation. implementation Regression Testing
Is done after changes are made to the software and its purpose software, is to help ensure that the updated software still possesses the functionality it had before the updates.

18

Overview of Software Testing

3/9/2010

Content
What is Software Testing? Activities of Test Engineer Software Testing Taxonomy Test Case Design Methods Principles of Software Testing Software Testing Myths Software Testing Limits

19

Overview of Software Testing

3/9/2010

Test Case
Test Case Values (Input): The input values necessary to complete some execution of the software under test test. Expected Results: The result that will be produced when executing the test if and only if the program satisfies its intended behavior Prefix V l P fi Values: A i t necessary t put th software i t th Any inputs to t the ft into the appropriate state to receive the test case values. Postfix Values: Any inputs that need to be sent to the software after the test t t case values are sent. l t Test Case: composed of the test case values, expected results, prefix values, and postfix values necessary for a complete execution and evaluation of the software under test. l i f h f d Test Suite: a collection of test cases that are intended to be used to test a software program to show that it has some specified set of behaviours
20 Overview of Software Testing 3/9/2010

Test Case Design Methods

Any engineered product (and most other things) can be tested in one of two ways
Black box testing
No knowledge of internal design or code required required. Tests are based on requirements and functionality

White box testing

Knowledge of the internal program design and code required. Tests are based on coverage of code statements, branches, paths, conditions.

21

Overview of Software Testing

3/9/2010

Test Case Design Methods

22

Overview of Software Testing

3/9/2010

Test Case Design Methods

Testing type Unit Testing U it T ti Integration Testing System Testing S T AccceptanceTesting Techniques Used White B Whit Box White Box Black Box Black Box l k Black Box

23

Overview of Software Testing

3/9/2010

Content
What is Software Testing? Activities of Test Engineer Software Testing Taxonomy Test Case Design Methods Principles of Software Testing Software Testing Myths Software Testing Limits

24

Overview of Software Testing

3/9/2010

Software Testing Principles

A necessary part of a test case is a definition of the expected output or result
If the expected result of a test case has not been predefined, chances are that a plausible, but erroneous, result will be plausible erroneous interpreted as a correct result because of the phenomenon of the eye seeing what it wants to see. => a test case must consist of two components:
A description of the input data to the program. A precise description of the correct output of the p g p p p program for that set of input data.

25

Overview of Software Testing

3/9/2010

Software Testing Principles

Testing requires independence.
A programmer should avoid attempting to test his or her own program.
Psychological issues The program may contain errors due to the programmers misunderstanding of the problem statement or specification

A programming organization should not test its own programs programs.

Thoroughly inspect the results of each test

26

Overview of Software Testing

3/9/2010

Software Testing Principles

Test cases must be written for input conditions that are invalid and unexpected, as well as for those that are valid and expected. Examining a program to see if it does not do what it is supposed to do is only half the battle; the other half is seeing whether the program does what it is not supposed to do. p g pp Avoid throwaway test cases unless the program is truly a throwaway program.

27

Overview of Software Testing

3/9/2010

Software Testing Principles

Do not plan a testing effort under the tacit assumption that no errors will be found. The probability of the existence of more errors in a section of a program is proportional to the number of errors already found in that section

28

Overview of Software Testing

3/9/2010

Software Testing Principles

Testing is an extremely creative and intellectually challenging task. Testing should begin in the small and progress toward testing in the large in large. Exhaustive testing is not possible Tests should be planned long before testing begins begins.

29

Overview of Software Testing

3/9/2010

Content
What is Software Testing? Activities of Test Engineering Software Testing Taxonomy Test Case Design Methods Principles of Software Testing Software Testing Myths Software Testing Limits

30

Overview of Software Testing

3/9/2010

Software Testing Myths

We can test a program completely. In other words, we test a program exhaustively exhaustively. We can find all program errors as long as test engineers do a good job. We can test a program by trying all possible inputs and states of a program. A good test suite must include a great number of test cases. Good test cases always are complicated ones. Software test automation can replace test engineers to perform good software testing. d f Software testing is simple and easy. Anyone can do it. No training is needed.
31 Overview of Software Testing 3/9/2010

Software Testing Limits

We can never be sure the specifications are 100% correct. We can never be certain that a testing system (or tool) is correct. No N testing tools can cope with every software program. l h f Tester engineers never be sure that they completely understand a software product product. We never have enough resources to perform software testing. We can never be certain that we achieve 100% adequate software testing

32

Overview of Software Testing

3/9/2010

Q&A

33

Overview of Software Testing

3/9/2010