SOFTWARE TESTING

(ITS 64704)
Fundamentals of Testing
(Part 2)
Terms and Motivation

Topics
Principles of Testing

Fundamental Test Process

Test Cases, Expected Results and Test Oracles

Psychology of Testing

Ethics of Testing
 What is testing?

• Testing is a systematic process with the main aim

of finding defects
• Defects found needs to be fixed by developers
 Testing Objectives

1. Find bugs and provide programmers with the information they

need to fix important bugs
2. Gain confidence about the level of quality of the system
3. Prevent defects (through early involvement in reviews and
advanced test designs)
4. Provide information about the most important aspects of the
quality of the system under test
5. Help management understand system quality
SW Development Model:
Waterfall model (1970)
Verification

SW Development Model:
• examines the consistency of the products being
developed with products developed in previous phases.

Waterfall model (1981)

Validation
• process of evaluating a system or component during or
at the end of the development process to determine
whether it satisfies specified requirements
V-Model (B. Boehm, 1979)
 V-Model (B. Boehm, 1979)

Validate Validate

Verify
Verify
General V-Model
 Test Process

 Closely linked with software development

 It is however, a separate, independent process
 A test plan is necessary for every test level (level test plan)
 Testing cannot be considered as a single activity
 Many individual tasks are performed within testing
 The test process represents these individual testing activities in a
coherent process
 Planning & Control

5 Stages of
Analysis & Design
Fundamental Test Process
Implementation & Execution

Evaluating Exit Criteria and

reporting • Each phase may have to be
executed a number of times in
Test closure activities order to fulfill exit criteria
• Although logically sequential,
the activities in the process may
overlap or take place
concurrently
 Planning & Control

Stage 1: Planning & Control

Analysis & Design

Implementation &
Execution

Planning & Control

Evaluating Exit
1. Specification of test and
Criteria planreporting
• Determine test scope, risks, objectives, strategies
• Specify test approach (techniques, objects, coverage, teams, etc)
Test closure activities
2. Details of Test Plan
• Determine required test resources
• Implement test strategies
• Schedule test analysis & design
• Schedule implementation, execution and evaluation of tests
• Determine the test exit criteria (what level of coverage should be
reached?)

* Test schedule and test strategy are to be recorded in the test plan
 Planning & Control

Stage 1: Planning & Control

Analysis & Design

Implementation &
Execution

Planning & Control

Evaluating Exit
3. Drawing up a rough schedule
Criteria and reporting
• Schedule tasks for test analysis and test specification
• Schedule test implementation, test execution and test evaluation
Test closure activities
4. Test Control
• Measure & analyze result
• Monitor and document progress, coverage and test exit criteria
• Initiate corrective actions
• Make decisions
 Planning & Control

Stage 2: Analysis & Design

Analysis & Design

Implementation &
Execution

Analysis
Evaluating Exit
Design
Criteria
• Review the test and (e.g
basis • Design and prioritize
requirementsreporting
or design specs, combinations of test data,
quality risks, n/w architecture, actions and expected results
Test closure
SRS, FRS etc) activities • Identify the test data needed
• Identify and prioritize test for test conditions and cases
conditions, test requirements, • Design test environment
objectives, required test data • Identify infrastructure, tools
based on analysis of test items
• Evaluate feasibility of the
requirements and system
 Planning & Control

Stage 3: Implementation & Execution

Analysis & Design

Implementation &
Execution

Implementation
Evaluating Exit
Execution
• Develop, implement
Criteria and and • Execute test case (manual or
prioritize test cases, create
reporting automated)
data, write procedures (steps) • Log test results, version of
Test closure
Create test scripts software under test, test tools
• activities
• Organize test suites and and the testware (test log)
sequences of test procedures • Compare actual and expected
• Verify the test environment result
• Report and analyze incidents
• Repeat corrected and/or
updated test
• Run confirmation and/or
regression test

Re-Testing: Repeat of test activities as a result of action taken to

resolve each incident/ defect
 Planning & Control

Stage 4 & 5
Analysis & Design

Implementation &
Execution

Exit &Evaluating
ReportingExit
Closure
Criteria
• Check test logs and the
against • Confirm test deliverables, final
exit criteria inreporting
the test plan resolutions or deferral of bug
• Assess if more tests are
Test closure
reports, and the acceptance
needed or if the exit criteria
activities
of the system
specified should be changed • Finalize, archive the testware,
• Write a test summary report test environment and test
for stakeholders infrastructure
• Deliver testware to the
maintenance organization
• Perform a review to capture
improvements for future
releases, projects, test
process
Test Reporting
Test Reporting
Terms and Motivation

Topics
Principles of Testing

Fundamental Test Process

Test Cases, Expected Results and Test Oracles

Psychology of Testing

Ethics of Testing
Test cases for verification of specified result and of test object delivered results and
reactions

Criteria for Test Cases

• Positive test; expected input (happy path)
- include things like precondition (things
Set of documentation it’s detaining
required/need to have), postcondition (main
things that need to know and do - able
thing to achieve from that particular set of
to carry out the specific testing of a
Test testing)
cases that verify the specified handling of exceptionalparticular
situations and defects
function/module.
- test for cdf - tap by tap guiding - expected result
(testers carry out testing efficiently)
• Negative tests; expected false input
• e.g exception handling cases

Test cases for verification of reactions of the test object for invalid and unexpected inputs
or constraint
Eg: module– registration
no exception handling
system, specified
expect module registered successfully in the system

• Negative test, unexpected erroneous input

- invalid input, expect invalid input

Eg: module registration system, expect module registered
software testing, credit hour 10 instead of 4. Module registration
is not completed successfully

doesn’t matter positive/negative test,

valid/invalid input, outcome is not expected.
(unexpected)
 Test Specification :
High level and specific test cases
Example:
 A company has ordered a program that should calculate the Christmas bonus
of the staff in relation to the time they have been working here. In the
description of the requirement you find the following text:

• Staff who have been with the company for more than 3 years will receive 50% of the
monthly salary as Christmas bonus.
• Staff who have been with the company for more than 5 years will receive 75%.
• Staff who have been with the company for more than 8 years will receive 100% of
their monthly salary.

How do the test cases look like?

 Test Specification :
High level and specific test cases

 You can setup the following relationship between allowance of the bonus and
the time working for the company.
• Staff who have been with the company for more than 3 years will receive 50% of the monthly
salary as Christmas bonus.
• Staff who have been with the company for more than 5 years will receive 75%.
• Staff who have been with the company for more than 8 years will receive 100% of their monthly
salary.

Years with the company Bonus percentage

years with the company <=3 equals bonus = 0%
3 < years with the company <=5 equals bonus = 50%
5 < years with the company <=8 equals bonus = 75%

years with the company >8 equals bonus = 100%

 Test Specification :
High level and specific test cases
(positive tests)
Give the conditions, indicate
the range

High level (logical) Test Case 1 2 3 4

Input value x X <=3 3<x<=5 5<x<=8 X>8
(years with the company)
Expected result 0 50 75 100
(bonus in %)
input/data to test

Specific Test Case 1 2 3 4

Input value x 2 4 7 12
(years with the company)
Expected result 0 50 75 100
(bonus in %)

Remarks:
• No pre- and post-conditions or constraints are considered
• The test cases were not derived systematically
• Only positive tests with expected results
 Test Specification :
High level and specific test cases
(positive tests)
High level (logical) Test Case 1 2 3 4
Input value x X <=3 3<x<=5 5<x<=8 X>8
(years with the company)
Expected result 0 50 75 100
(bonus in %)

Specific Test Case 1 2 3 4

Exact
Input value x i
(bou2n nput an4d ca 7 12
daries
(years with the company) ) will b tegories
Expected result 0EP and B 50 e learne75 d in 100
(bonus in %) VA top
ic

Remarks:
• No pre- and post-conditions or constraints are considered
• The test cases were not derived systematically
• Only positive tests with expected results
 Test Specification :
High level and specific test cases
(negative tests)
High level (logical) Test Case 1 2 3 4
Input value x X <=3 3<x<=5 5<x<=8 X>8
(years with the company)

Expected result Invalid input. Invalid input. Invalid input. Invalid input.
Bonus can’t be Bonus can’t Bonus can’t be Bonus can’t be
(bonus in %) calculated be calculated calculated calculated

Specific Test Case 1 2 3 4

Input value x 4 7 10 2
(years with the company)
Expected result Invalid input. Invalid input. Invalid input. Invalid input.
Bonus can’t be Bonus can’t Bonus can’t be Bonus can’t
(bonus in %) calculated be calculated calculated be calculated

Remarks:
• No pre- and post-conditions or constraints are considered
• The test cases were not derived systematically
• Negative tests with expected results
 Test Specification :
High level and specific test cases
(negative tests)
High level (logical) Test Case 1 2 3 4
Input value x X <=3 3<x<=5 5<x<=8 X>8
(years with the company)

Expected result Invalid input. Invalid input. Invalid input. Invalid input.
Bonus can’t be Bonus can’t Bonus can’t be Bonus can’t be
(bonus in %) calculated be calculated calculated calculated

Specific Test Case 1 2 3 4

Input value x 4 Exact 7 10 2
(years with the company) i
(boun nput and ca
Expected result Invalid input. dInvalid w
tegorinput.
aries)input. Invalid i es Invalid input.
i
Bonus can’t be EPBonus can’tll beBonus
learned be
can’t Bonus can’t
(bonus in %) calculated be acalculated
nd BVA in
topicalculated
c
be calculated

Remarks:
• No pre- and post-conditions or constraints are considered
• The test cases were not derived systematically
• Negative tests with expected results
 Expected Results and Test Oracle

 After each executed test case, it must be decided whether there is a failure or
not
 Compare the actual result vs expected result
 The expected behavior of the test object has to be determined in advance for
each test case
 The tester must obtain this information from appropriate sources when
specifying a test case
Topics

Terms and Motivation

Principles of Testing

Fundamental Test Process

Test Cases, Expected Results and Test Oracles

Psychology of Testing

Ethics of Testing
 Good tester attributes

Professional
Curiosity A critical eye
pessimism

Good
communication
Attention to detail
skills (speaker and
listener)
 Defining Tester Skills

Reading
• Specifications, e-mails, test cases, etc

Writing
• Test cases, bug reports, test documentation, etc

Technology, project, testing skills

• Technology: programming languages, OS, networking,
HTML/web, etc
• Application domain: banking, human factors, office applications,
etc
• Testing: scripting, exploring and attacking the system,
automation, etc
 Differing Mindset

• Testing and reviewing are different than developing, so the

mindsets are different too
• Separation of duties helps focus testing (developers not testing
own code)
• Professional testers are more effective at testing activities
(especially finding failures), as they are more objective and don’t
have the biasness
 Degrees of Independence

• Though developers can test their own code, independent testers

are typically more effective at finding failures
• Levels of test independence:

Developer testing their own code (low level of independence)

Testing done by team member

Testing done by a member from different team / team specialist

Testing done by a person from a different organization or company (high level

independence)
 Clear Objectives

• Should have clear objectives on testing

• For e.g: Should testing find defects or confirm that software
works?
• If it is to find defects, what percentage?
• If it is to build confidence, to what level?
• A test policy can help to clearly state the objectives of testing
 Bad news or Bad Guy?

 Testers are sometimes on receiving end of emotions brought

on by news of project problems
 Communication skills can help:

Collaborate for better quality

Communicate neutrally, about facts , without criticism

Understand your colleagues and how they’ll react to your findings

Conform your colleagues understood what you said

Confirm you understand your colleagues

Topics

Terms and Motivation

Principles of Testing

Fundamental Test Process

Test Cases, Expected Results and Test Oracles

Psychology of Testing

Ethics of Testing
 Tester Ethics

CLIENT &
PUBLIC PRODUCT
EMPLOYER

JUDGEMENT MANAGEMENT PROFESSION

COLLEAGUES SELF
 Tester Ethics

PUBLIC
• Certified software tester shall act consistently with the public interest

CLIENT & EMPLOYER

• Certified software testers shall act in the manner that is in the best
interests of their client and employer, consistent with the public interest

PRODUCT
• Certified software testers shall ensure that the deliverables they provide
(on the products and systems they test) meet the highest professional
standards possible.

JUDGEMENT
• Certified software testers shall maintain integrity and independence in
their professional judgment
 Tester Ethics

MANAGEMENT
• Certified software test managers and leaders shall subscribe to and
promote an ethical approach to the management of software testing.

PROFESSION
• Certified software testers shall advance the integrity and reputation of
the profession consistent with the public interest

COLLEAGUES
• Certified software testers shall be fair to and supportive of their
colleagues, and promote cooperation with software developers.

SELF
• Certified software testers shall participate in lifelong learning regarding
the practise of their profession and shall promote an ethical approach to
the practice of the profession.
Effective vs Efficiency testing

Effective testing Efficient testing

• Use test design • Finding the
techniques to defects with the
write test to find least cost, time
more defects and resources
THANK
YOU