Software Testing

Common Testing Types

Phan Thi Hong

Lecturer
Faculty of Information Technology
Van Lang University

© 2018
Lecture Learning Objectives
 Objectives
▪ Provide basic concepts about some common testing types,
include:
• Regression Testing
• Smoke Testing
• Sanity Testing

 Outcomes
▪ Understand what smoke testing, sanity testing and regression
testing are and when they are applied.

▪ Identify the difference between regression testing and re-testing

as well as between smoke testing and sanity testing that are
the most misunderstood topics in Software Testing.

2
Outline

 Regression Testing

 Smoke Testing

 Sanity Testing

3
04 | Common Testing Types

4.1. Regression Testing

Section Overview
 In this section, we will review the following:
▪ Concepts and techniques of regression testing
▪ Distinguishing regression testing and re-testing

5
Guiding Questions

 What is regression testing?

 When is regression testing performed?

 Why do we need regression testing?

 What are regression testing techniques?

 How to perform regression testing?

 What are differences between regression testing and

re-testing?

6
What is Regression Testing?
 Regression Testing is a form of
software testing carried out to
ensure that:
▪ changes in the fixed defect(s)
▪ or any enhancement changes

are NOT impacting the previously

working functionality.

 Regression test should be a part of

the release cycle and MUST be
considered in the test estimation.

9
Importance of Regression Testing

 It is very important when there are continuous

changes/improvements added in the application.
The new functionality should not negatively affect
existing tested code.
10
What Do We Do in Regression Check?
 Re-run the previously conducted tests

 Compare the current results with previously executed

test results

 The purpose of this test is to determine whether the

change has regressed other aspects of the program.

11
Regression Testing Strategies – 1

 Here are various regression testing techniques:

Source: https://www.softwaretestinghelp.com/regression-testing-tools-and-methods/

12
Regression Testing Strategies – 2
 The simplest strategy is the retest-all approach.
However, it is considered impractical on account of cost,
resource and delivery schedule constraints that projects
are frequently subjected to.

Source: https://www.prototechsolutions.com/regression-testing/

13
Regression Testing Strategies – 3
 Regression test selection
▪ Instead of re-executing the entire test suite, it is better
to select part of test suite to be run.

▪ Test cases selected can be categorized as 1) Reusable

Test Cases 2) Obsolete Test Cases.
• Re-usable Test Cases can be repetitively used in
succeeding regression cycles.

• Obsolete Test Cases cannot be used in succeeding

cycles.

▪ Read more guidelines to select TCs for regression at:

https://www.capgemini.com/2017/01/best-practices-in-
identifying-test-cases-for-regression-suite/

14
 Regression Testing Strategies – 4
 Test Case Prioritization
▪ Priority of test case depends on business impact, critical &
frequently used functionalities.
▪ One of the ideas is to classify the test cases into various
priorities based on importance and customer usage. Here it is
suggested the test cases be classified into three categories;

Priority 0 Priority 1 Priority 2

Sanity test cases which checks This includes the test These are executed
basic functionality (as per the SRS cases which tests as a part of ST cycle
of product) and are run for pre- essential and selected for
system acceptance and when functionalities that regression testing on
product goes through major deliver high project need basis. These
change. These test cases deliver a value to both test cases deliver
very high project value to both engineers and moderate project
engineers and to customers. customers. value.

15
Regression Testing Strategies – 5
 Hybrid
▪ Rather than selecting the entire test suite, select
only the test cases which are re-executed
depending on their priority.

16
How to Perform Regression Test?
Given below are the various steps involved in performing this
testing:

Understand what Identify which might Prepare a test suite

kind of changes be impacted for regression

Automate TCs of the

test suite

Execute regression
Create a test
TCs whenever having
execution report
any change in code

Note that regression test suite should be managed properly by

continuously updating the test cases.
18
Regression Testing vs. Re-testing

Defect (Grayed out)

Dev. Team (Fixed)

Test Team
(Retesting)

19
 Regression Test

Dev. Team (Fixed)

Test Team (Retesting)

… and more for remaining categories

Regression Testing vs. Re-testing

Regression Testing (RT) Re-testing

RT is a type of software testing that Re-testing is done to make sure
intends to ensure that changes like that the tests cases which failed in
defect fixes or enhancements to the last execution are passing after the
module or application have not defects against those failures are
affecting unchanged part. fixed.
RT is not carried out on specific Re-testing is carried out based on
defect fixes. It is planned as specific the defect fixes.
area or full regression testing.
In RT, you can include the test cases In Re-testing, you can include the
which passed earlier. We can say test cases which failed earlier. We
that check the functionality which can say that check the functionality
was working earlier. which was failed in earlier build.

21
Regression Testing vs. Re-testing
Regression Testing (RT) Re-testing
Regression TCs we use are derived Test cases for Re-testing cannot be
from the functional specification, the prepared before starting testing.
user manuals, user tutorials, and
In Re-testing only re-execute the test
defect reports in relation to corrected
cases failed in the prior execution.
problems.
Automation is the key for RT. Manual You cannot automate the test cases
RT tends to get more expensive with for Re-testing.
each new release. RT is right time to
start automating test cases.
Defect verification is not comes under Defect verification is comes under
RT. Re-testing.
Based on the availability of resources Priority of Re-testing over Regression
the RT can be carried out parallel with testing is higher, so it is carried out
Re-testing. before regression testing.

22
 Section Questions

What is regression testing?

When is regression testing performed?

Why do we need regression testing?

What are regression testing techniques?

What are differences between regression testing

and re-testing?
04 | Common Testing Types

4.2. Sanity Testing

Section Overview
 In this section, we will review the following:
▪ Concepts, objectives, and the importance of sanity
testing

25
Guiding Questions

 What is sanity testing?

 When is sanity testing performed?

 Why do we need sanity testing?

 What are the differences between sanity testing and

regression testing?

26
What should you do
– when you have to sign off in a
day or two but the build for
testing is still not released?

– when you were asked to test

and deliver in the same day
and clients sometimes simply
refuse to give extra time?

– Or how could you complete

the whole testing in a few
hours, verify every
functionalities, bugs and is an approach for all such
problems
release it?
What Is Sanity Testing? – 1
 Sanity testing is usually performed when any minor
bug is fixed or when there is a small change in the
functionality. It is a kind of software testing which is
done by the testers to ensure that the functionality is
working as expected.

 Sanity testing should be done only when you are

running short of time, never use this for your regular
releases. Theoretically, this testing is a subset of
regression testing.

 Sanity testing checks the minor bug fixes and the

functionality changes are working at the same time; it
also ensures that the related functionality is intact.
28
What Is Sanity Testing? – 2

 It is a surface level testing which follows narrow and

deep approach concentrating on the detailed testing
of some limited features.

 In sanity testing, the testers verifies the commands

and functions and all the menus in the product.

 Sanity testing is usually not scripted

29
Sanity Testing vs. Regression Testing
Regression Testing Sanity Testing
Regression testing is done to verify Sanity testing is done at random to
that the complete system and bug verify that each functionality is
fixes are working fine. working as expected.
Every tiniest part is regressed in this This is not a planned testing and is
testing. done only when there’s a time
crunch.
It is a well elaborate and planned This is not a planned testing and is
testing. done only when there’s a time
crunch.
An appropriately designed suite of It may not every time be possible to
test cases is created for this testing. create the test cases; a rough set of
test cases is created usually.

30
Sanity Testing vs. Regression Testing
Regression Testing Sanity Testing
This includes in-depth verification of This mainly includes verification of
functionality, UI, performance, business rules, functionality.
browser/OS testing etc. i.e. every
aspect of the system is regressed.

This is a wide and deep testing. This is a wide and shallow testing.
This testing is at times scheduled for This mostly spans over 2-3 days
weeks or even month(s). max.

Source: https://www.softwaretestinghelp.com/smoke-testing-and-sanity-testing-difference/

31
 Section Questions

What is sanity testing?

When is sanity testing performed?

Why do we need sanity testing?

What are the differences between sanity testing

and regression testing?
04 | Common Testing Types

4.3. Smoke Testing

Section Overview
 In this section, we will review the following:
▪ Concepts, objectives and the importance of smoke
testing
▪ Strategy for smoke testing
▪ Distinguishing sanity testing and smoke testing

34
Guiding Questions
 What is smoke testing?

 When is smoke testing performed?

 Why do we need smoke testing?

 What is the strategy for smoke testing?

 What are differences between sanity testing and

smoke testing?

35
Consider the following example:
• Let us assume that there is an application like ‘Student
Network’ which has 15 modules. Among them, there are 4
important components like Login page, Adding student
details, Updating it and Deleting it.

BUILD 1.0
ADD
LOGIN
STUDENT SMOKE PASSED BUILD
NO
TESTING ? REJECTED
UPDATE DELETE
STUDENT STUDENT
YES

FUNCTIONAL
TEST
What is Software Build?
 The term build is the process by which source code is
converted to a stand-alone form that can be run on
any system.

 Builds are created when a certain point in

development has been reached or the code has been
predictable ready for implementation, either for
testing or release.

 A software build is also known as code build.

37
What is Smoke Testing? – 1
 Smoke testing is a type of software testing which
ensures that the major functionalities of the
application are working fine.

 This is a non-exhaustive testing with very limited

number of test cases.

 Smoke testing is like a normal health check up of the

build of an application. It is also known as Build
verification testing where the build is verified by
testing the important features of the application and
then declaring it as good to go for further detailed
testing.

38
What is Smoke Testing? – 2
 Smoke testing can be done by developers before
releasing the build to the testers. It is also tested by
the testing team to ensure that the build is stable
enough to perform the detailed testing.

 It is a type of shallow and wide testing because it

covers all the basic and important functionalities of an
application.

 Usually smoke testing is performed with positive

scenarios and with valid data.

 Usually the smoke testing is documented.

39
Why Smoke Testing?

 The objectives of smoke testing are

▪ to reject a badly broken application, so that the

QA team does not waste time installing and testing
the software application.

▪ NOT to perform exhaustive testing, but to verify

that the critical functionalities of the system are
working fine.

40
Why Should Automate Smoke Tests?
Consider the following example:
• Let us assume that we are testing a financial projection
application, which took inputs about your salary,
savings etc., and projected your taxes, savings, profits
depending on the financial rules. Along with this, we
had customization for countries that depend on the
country and its tax rules used to change (in the code).
• For this project, we had 250 Smoke TCs / 800 TCs.

Manual Test Automation Test with

Selenium
6 – 7 days to get the compared
test result of 250 with 3 – 4 hours to get the
Smoke TCs test result of 250 TCs
41
Smoke Testing Cycle

Source: https://www.guru99.com/smoke-testing.html

42
Smoke Testing vs. Sanity Testing

43
Smoke Testing vs. Sanity Testing

Smoke Testing Sanity Testing

Smoke Testing is performed to Sanity Testing is done to
ascertain that the critical check the new functionalities
functionalities of the program / defects have been fixed.
are working fine.
The objective of this testing is The objective of this testing is
to verify the “stability” of the to verify the “rationality” of
system in order to proceed the system in order to
with more rigorous testing. proceed with more rigorous
testing.
This testing is performed by This testing is usually
the developers or testers. performed by the testers.

44
Smoke Testing vs. Sanity Testing

Smoke Testing Sanity Testing

Smoke Testing is usually Sanity Testing is usually not
documented or scripted. documented and is
unscripted.
Smoke Testing is a subset of Sanity Testing is a subset of
Acceptance testing. Regression Testing.
Smoke Testing exercises the Sanity Testing exercises only
entire system from end to end. the particular component of
the entire system.
Smoke Testing is like general Sanity Testing is like
health check up. specialized health check up.

45
 Section Questions

What is smoke testing?

When is smoke testing performed?

Why do we need smoke testing?

What are differences between sanity testing and

smoke testing?
Take Away

Smoke testing is broad and

shallow, focus more on checking up the
health of application build before
taking it to testing in depth.

Whereas

Sanity testing is narrow and

deep, focus more on verifying whether
requirements are met before taking it to
testing in depth.

47
Take Away
regression testing to make
The main aim of

sure that changed component is NOT

impacting the unchanged parts of

program.

Whereas

re-testing to make sure that

The main aim of

failed test cases in last execution are

passing after the defects against
those failures are fixed.
48
References
1. https://www.softwaretestinghelp.com/regression-
testing-tools-and-methods/

2. https://www.softwaretestinghelp.com/smoke-testing-
and-sanity-testing-difference/

3. http://tryqa.com/what-is-smoke-testing-when-to-use-
it-advantages-and-disadvantages-2/#more-2237

4. http://www.guru99.com/smoke-sanity-testing.html

5. https://www.capgemini.com/2017/01/best-practices-
in-identifying-test-cases-for-regression-suite/#

49
Questions & Answers

50