Regression Testing

Regression testing: Progressives Vs regressive testing, Regression test ability, Objectives of regression testing,
Regression testing types, Regression testing techniques.

• Introduction
• When a new module is added as a (part of integration)testing, the software changes.
• whenever a new bug in the working system appears and it needs some changes.
• The new modifications may affect other parts of the software too.
• It is important to check software with all test cases so that new modifications does not effect other
• parts of software.
.

• changes are due to:

• Some errors
• additional functionality
• due to change in some external policies
• to improve the efficiency and performance
• change in existing technologies
 Progressive Vs Regressive testing:
--All the test case design methods or testing technique, discussed till now are referred to as
progressive testing or development testing.
--The purpose of regression testing is to confirm that a recent program or code change has not
adversely affected existing features.
--Regression testing is nothing but full or partial selection of already executed test cases which are
re-executed to ensure existing functionalities work fine.
--This testing is done to make sure that new code changes should not have side effects on the
existing functionalities.
--It ensures that old code still works once the new code changes are done

• Progressive testing:Test case design methods or testing techniques referred to as progressive testing or
development testing.

Comparison between Regression testing and Progressive or Development testing

Need of regression testing

• Increases confidence in the correctness of the modified program.

• Locates errors in the modified program.
• Preserves the quality and reliability of the software.
• Ensures the software’s continued operation.
• Change in requirements and code is modified according to the requirement
 • New feature is added to the software
• Defect fixing
• Performance issue fix

Definition:
Regression testing is the selective retesting of a system or component to verify that
modifications have not caused unintended effects and that the system or component still complies
with its specified requirements.

REGRESSION TESTING PRODUCES QUALITY SOFTWARE

• Regression testing is performed in case of bug-fixing or whenever there is a need to

incorporate any new requirements in the software.
• changed software requires to be fully tested again so as to ensure:
• (a) bug-fixing has been carried out successfully, (b) the modifications have not affected other
unchanged modules,
• (c) the new requirements have been incorporated correctly.
• the creation, maintenance, and execution of a regression test suite helps to retain the quality of
the software.
• Industry experiences noted by Onoma indicates that regression testing often has a strong
positive influence on software quality.
• We can perform regression testing in the following scenario, these are:
• 1. When new functionality added to the application.
• Example:
• A website has a login functionality which allows users to log in only with Email. Now
providing a new feature to do login using Facebook.
• 2. When there is a Change Requirement.
• Example:
• Remember password removed from the login page which is applicable previously.
• 3. When the defect fixed
• Assume login button is not working in a login page and a tester reports a bug stating that the
login button is broken.
• 4. When there is a performance issue fix
• Example:
• Loading of a home page takes 5 seconds, reducing the load time to 2 seconds.
• 5. When there is an environment change
• Example:
• When we update the database from MySql to Oracle.
• The importance of regression testing is well-understood for the following reasons:
• 􀂄 It validates the parts of the software where changes occur.
• 􀂄 It validates the parts of the software which may be affected by some changes, but otherwise
unrelated.
• 􀂄 It ensures proper functioning of the software, as it was before changes occurred.
• 􀂄 It enhances the quality of software, as it reduces the risk of high-risk bugs.

REGRESSION TESTABILITY

• Regression testability refers to the property of a program, modification, or test suite that lets
it be effectively and efficiently regression-tested.
• Regression testability is a function of both the design of the program and the test suite.
• To consider regression testability, a regression number is computed.
• It is the average number of affected test cases in the test suite that are affected by any
modification to a single instruction.
• Computed using information about the test suite coverage of the program.Provide significant
savings in cost of development and maintenance of software

• Regression number-R(n)
• Test suite-{t1,t2,t3…tn}
• Affected testcases-t5,t8,t9 then
• R(n)=(t5+t8+t9)/3

OBJECTIVES OF REGRESSION TESTING

--It tests to check that the bug has been addressed: The first objective in bug fix testing is to check
whether the bug-fixing has worked or not.
--It finds other related bugs: Regression tests are necessary to validate that the system does not
have any related bugs.
--It tests to check the effect on other parts of the program: It may be possible that bug-fixing has
 unwanted consequences on other parts of a program. Therefore, it is necessary to check the
influence of changes in one part or other parts of the program.

WHEN IS REGRESSION TESTING DONE?

• Software Maintenance
 Corrective maintenance: Changes made to correct a system after a failure has been observed.
 Adaptive maintenance: Changes made to achieve continuing compatibility with the target
environment or other systems.
 Perfective maintenance: Changes designed to improve or add capabilities
 Preventive maintenance: Changes made to increase robustness, maintainability, portability,
and other features

• Rapid Iterative Development: The extreme programming approach requires that a test be
developed for each class and that this test be re-run
every time the class changes
• First Step of Integration: Re-running accumulated test suites, as new components are added
to successive test configurations, builds the regression suite incrementally and reveals
regression bugs.
• Compatibility Assessment and Benchmarking: Some test suites are designed to be run on a
wide range of platforms and applications to establish conformance with a
standard(benchmarking) or to evaluate time and space performance.

TYPES OF REGRESSION TESTING

• Bug-Fix regression: This testing is performed after a bug has been reported and fixed.
• Its goal is to repeat the test cases that expose the problem in the first place.
• Side-Effect regression/Stability regression:It involves retesting a substantial(essential) part of
the product.
• The goal is to prove that the change has no HARMFUL effect on something that was earlier in
order.(before)
• It tests the overall integrity of the program

DEFINING REGRESSION TEST PROBLEM

• Notations used in regression testing

REGRESSION TESTING TECHNIQUES

• Three different techniques for regression testing

• 1. Regression test selection technique
• 2. Test case prioritization technique
• 3. Test suite reduction technique
-->Regression test selection technique: This technique attempts to reduce the time required to
retest a modified program by selecting some subset of the existing test suite.

-->Testcase prioritization technique: Regression test prioritization attempts to reorder a regression

test suite so that those tests with the highest priority according to some established criteria, are
executed earlier in the regression testing process rather than those lower priority. There are two
types of prioritization:

a)General Test Case Prioritization: For a given program P and test suits T, we prioritize the test
 cases in T that will be useful over a succession of subsequent modified versions of P, without any
knowledge of the modified version

b)Version-Specific Test case Prioritization: We prioritize the test cases in T, when P is modified to
P', with the knowledge of the changes made in P.
-->Test Suite Reduction Technique: It reduces testing costs by permanently eliminating
redundant test cases form test suites in terms of codes of functionalities exercised

SELECTIVE RETEST TECHNIQUE

Selective retest technique attempts to reduce the cost of testing by identifying the portions of P'
(modified version of Program) that must be exercised but the regression test suite. Following are the
characteristic features of the selective retest technique:
--> It minimizes the resources required to regression test a new version.
--> It is achieved by minimizing the number of test cases applied to the new version.
--> It analyses the relationship between the test cases and the software elements they cover.
-->It uses the information about changes to select test cases.
 Steps in Selective retest technique:
1. Select T' subset of T, a set of test cases to execute on P'.
2. Test P' with T', establishing correctness of P' with respect to T'.
3. If necessary, create T'‖, a set of new functional test cases for P'.
4. Test P' with T', establishing correctness of P' with respect to T''.
5. Create T'''. a new test suite and test execution profile for P', from T, T' and T''.

Each of these steps involves the following important problems:

• Regression test selection problem:
Step1 involves this problem., The problem is to select a subset T’ of T with which P’ will be
tested.(select T’)
• Coverage identification problem:
 Step3 addresses this problem, The problem is to identify portions of P ‘ or its specifications
that requires additional testing
• Test suite execution problem:
Steps2 and 4 addresses this problem, The problem is to execute test suites efficiently and
checking test results for correctness.
• Test suite maintenance problem: Step5 addresses this problem. The problem is to update and
store test information

• Strategy for Test Case Selection:

• Effective testing requires strategies to trade-off between the two opposing needs of amplifying
testing.
• 1) thoroughness(for which a high number
• of test cases would be desirable)
• 2) reducing time and costs(for which the fewer the test cases, the better).
• A decision procedure for selecting the test cases is provided by a test criterion.

• Selection Criteria Based on Code:

• A) Fault-revealing test cases: (identify the errors)

• B) Modification-revealing test cases(outputs will differ)

• C) Modification-traversing test cases (Executes modified code)

 Fault-revealing test cases:

A test case t detects a fault in P’ if it causes P’ to fail. Hence t is fault-revealing for P’
• Under certain conditions, however, a regression test selection technique can select a
superset(consists of all test cases) of the tests in T that are fault-revealing for P’
• Modification-revealing test cases:
• test case t is modification-revealing for P and P’ if and only if it causes the outputs of P and
P’ to differ.

• Modification-traversing test cases:

• test case t is modification-traversing if and only if it executes new or modified code in P’.

Regression Test Selection Techniques:

Minimization Techniques:
Minimization-based regression test selection techniques attempt to select minimal sets of test cases
from T that yield coverage of modified or affected portions of P.
Ex:- linear equations, integer programming algorithm
• • E.g., every program statement added to or modified for P’ be executed (if possible) by at least
one test in T

• Dataflow Techniques: (coverage based approach)

select test cases that exercise data interactions that have been affected by modifications
E.g., select every test in T, that when executed on P, executed at least one def-use pair that has
been deleted from P’, or at least one def-use pair that has been modified for P’
Ex:- technique of Harrold and Soffa,
• Safe Techniques: Techniques that are not safe can fail to select a test case that would have
revealed a fault in the modified program.
safe regression test selection techniques guarantee that the selected subset T ‘ contains all test
cases in the original test suite T that can reveal faults in P ‘.

• Ad Hoc/Random Techniques:
• developers often select test cases based on ‘intuitions’ or loose associations of test cases with
functionality.
• Another simple approach is to randomly select a predetermined number of test cases from T.

Retest-All Technique: The retest-all technique simply reuses all existing test cases. To test P', the
technique effectively ―selects‖ all test cases in T.

Evaluating Regression Test Selection Technique:

• Metrics, to measure and compare the techniques.
Below given are the certain categories, revealed by the Rotherel[ROTH96a] to compare and assess
the effectiveness of selective re-test techniques

Inclusiveness: Let M be a regression test selection technique. Inclusiveness measures the extent to
which M chooses modification revealing tests from T for inclusion in T'. We define inclusiveness
relative to a particular program, modified program, and test suite, as follows:

DEFINITION
Suppose T contains n tests that are modification revealing for P and P', and suppose M selects m of
these tests. The inclusiveness of M relative to P, P', and T is
1. INCL(M) = (100 * ( m/n)%, if n ≠ 0

2. INCL(M)% = 100%, if n = 0
For example, if T contains 50 tests of which eight are modification-revealing for P and P', and M
selects two of these eight tests, then M is 25% inclusive relative to P, P', and T. If T contains no
modification-revealing tests then every test selection technique is 100% inclusive relative to P, P",
and T.

Precision: Let M be a regression test selection technique. Precision measures the extent to which M
omits tests that are non modification-revealing. We define precision relative to a particular program,
modified program, and test suite, as follows:

DEFINITION
Suppose T contains n tests that are non modification-revealing for P and P' and suppose M omits m
of these tests. The precision of M relative to P, P: and T is
1)Precision = 100 * (m/ n) % if n ≠ 0
2)Precision = 100 % if n = 0
For example, if T contains 50 tests of which 44 are non modification-revealing for P and P', and M
omits 33 of these 44 tests, then M is 75% precise relative to P, P', and T. If T contains no non-
modification-revealing tests, then every test selection technique is 100% precise relative to P, P',
and T.

Efficiency: We measure the efficiency of regression test selection techniques in terms of their space
and time requirements. Where time is concerned, a test selection technique is more economical than
the retest-all technique if the cost of selecting T' is less than the cost of running the tests in T-T'
Space efficiency primarily depends on the test history and program analysis information a technique
must store. Thus, both space and time efficiency depend on the size of the test suite that a technique
selects, and on the computational cost of that technique.
 Regression Test Prioritization:
The regression test prioritization approach is different as compared to selective retest
techniques. Regression test prioritization attempts to reorder a regression test suite so that those
tests with the highest priority, according to some established criterion, are executed earlier in the
regression testing process than those with a lower priority.

The steps for this approach are:

We may have priority codes as follows

• Priority code 1 : Essential test case
• Priority code 2 : Important test case
• Priority code 3 : Execute, if time permits
• Priority code 4 : Not important test case
• Priority code 5 : Redundant test case
• for assigning priorities based on customer requirements or market conditions like
• Priority code 1 : Important for the customer
• Priority code 2 : Required to increase customer satisfaction
• Priority code 3 : Help to increase market share of the product