REGRESSION TESTING

Dr. KOYEL DATTA GUPTA

OUTLINE
• REGRESSION TESTING

• REGRESSION TEST PROCESS

• REGRESSION TEST SELECTION: THE PROBLEM

• SELECTING REGRESSION TESTS

• TEST SELECTION USING EXECUTION TRACE

• TEST SELECTION USING DYNAMIC SLICING

• TEST MINIMIZATION

• TEST PRIORITIZATION

• TOOLS
 REGRESSION TESTING
• Regression testing is a type of software testing that
verifies that software previously developed
and tested still performs correctly after it was
changed or interfaced with other software. Changes
may include software enhancements, patches,
configuration changes, etc.
 REGRESSION TESTING
Version 1
1. Develop P
2. Test P with test set T
3. Release P
Version 2
. Modif P i to P’
. Test P’ fo e fu tio alities
. Pe fo eg essio testi g o P’ to ensure that the code
carried over from P behaves correctly
• Mai p i iple: eusi g tests derived for P!
• Ide tif i g T’, a su set of T
. Release P’
 REGRESSION TEST PROCESS

Test Revalidation
– Checking which tests for P remain valid for P’.
– To ensure that only test that are applicable to P’ are used
during regression testing.
– How can we automatically identify Obsolete test cases?
Test selection
– Should we select and execute all tests in T ?
– Test only modification-traversing tests?
– Classification of tests:
• Re-testable: should be re-run
• Reusable: may not need to be re-run.
Tests in T are either Obsolete (TO), Re-testable (TRT), or Reusable
(TRU).
 REGRESSION TEST PROCESS
Test minimization
– Discards tests seemingly redundant with respect to some
criteria.
– E.g., if two tests t1 and t2 execute function f, one might keep
only t2.
– Purpose: reducing the number of tests to execute for
regression testing.
Test prioritization
– Prioritizing tests based on some criteria
– Context: after revalidation, selection, minimization, one may
still have too many test cases (cannot afford to execute them
all).
 REGRESSION TEST SELECTION: THE
PROBLEM
Let P denote the program that has been tested
– Using test set T
– Against specification S
• Let P’ e ge e ated odif i g P
– P’ ust o fo to “’
• “ a d “’ ould e the sa e e.g., o e ti e ai te a e
• “⊂“’ e.g., “’ o tai s e featu es, edefi ed featu es
• Reg essio Testi g P o le
– Fi d a test set TRT o hi h P’ is to e tested to e su e that
code that implements functionalities carried over from P work
correctly.
• I additio P’ ust e tested to e su e that e l added ode
behaves correctly.
 SELECTING REGRESSION TESTS

• Test All
– T - TO
– Often not a practical solution as T - TO is often too large (too
many test cases to run in the amount of time allocated)
• Random Selection
– Select randomly tests from T - TO
– The tester decides how many tests to select
– May turn out to be better than no regression testing at all
– But may not even select tests that exercise modified parts of the code
• Selecting Modification Traversing Tests
– Selecting a subset of T - TO such that only tests that guarantee the execution of
modified code and code that might be impacted by the modified code in P’
are selected.
– A technique that does not discard any test that will traverse a modified or
impacted statement is known as a safe regression test selection technique.
 TEST SELECTION USING EXECUTION
TRACE
1. P is (has been) executed and execution traces are (have been)
recorded
– Execution trace = sequence of control flow graph (CFG) nodes
hit during one test execution
– trace(t) = execution trace of test case t.
– test(n) = set of tests that hit node n at least once.
We consider execution traces for T - TO.
. P’ is o pa ed ith P
• Build the o t ol flo g aphs G a d G’ of P a d P’, espe ti el
• G a d G’ a ou t fo o t ol flo , as ell as fu tio alls and variable
declarations
• Ide tif odes edges i P a d P’ that a e e ui ale t o ot
• Diffe e t te h i ues e ist e.g., s ta t ees
3. Selection
• Fo ea h ode n∈G that does ot ha e a e ui ale t ode i G’
• TRT = TRT ∪ test(n)
 TEST SELECTION USING DYNAMIC
SLICING
Definition: dynamic slice
– Let P be the program under test
– Let t be a test case against which P has been executed
– Let l be a location in P where variable v is used.
– The dynamic slice of P with respect to t and v is the set of
statements/nodes (CFG) in P that lie in trace(t) and did affect the value of v at l.
1. Computing dynamic slices
– Many different algorithms, with various complexities and
accuracies
. P’ is o pa ed ith P
3. Selection
• Let D“ t de ote a d a i sli e of P ith espe t to t t∈T)
and an output variable of P.
• Let e a ode i P CFG odified to ge e ate P’
• Fo all output a ia les, if ∈ DS(t) then re-test t (TRT = TRT ∪ {t})
 TEST MINIMIZATION
Regression test selection finds a subset TRT of T.
• “uppose P contains n testable entities
– Fu tio s, asi lo ks, o ditio s, …
• “uppose that tests i TRT o e < of the testa le
entities
– There is likely an overlap amongst the entities covered by
two tests in TRT
• Is it possi le a d e efi ial to edu e TRT to TMIN ?
– such that |TMIN| << |TRT| and
– each of the m entities covered by tests in TRT are also
covered by tests in TMIN
 TEST PRIORITIZATION

• Afte regression test selection, TRT might be overly large for

testi g P’
– Not enough budget to execute all those tests.
• Whe e high ualit soft a e is desi ed, it ight ot e ise to
discard test cases as in test minimization.
• I su h ases use Test p io itizatio :
1. Ranking tests
2. Deciding to stop execution of tests after the nth ranked test
• Test p io itizatio e ui es a ite io , o ite ia fo a ki g
• “i gle ite io p io itizatio
– Criteria 1: cost (e.g., execution time)
• Tests ith lo e osts a e a ked fi st hile test with higher costs are ranked last
– Criteria 2: risk (expected risk of not executing a test)
• Tests ith highe isks a e a ked fi st hile test with lower risks are ranked last

• O e goal of p io itizatio is to i ease the likelihood of revealing faults earlier in the

testing process.
 TOOLS

• Reg essio testi g e ui es tools auto atio

– Instrumentation tool to compute traces
– Tools to build control flow graphs, build traces and slices
– Tools implementing selection, minimization, prioritization
algorithms
• U -automated regression testing is equivalent to no
regression
testi g.
• E isti g tools:
– Capture/replay for GUIs
– DejaVOO (Java): research
– Telcordia Software Visualization and Analysis Toolsuite (xSuds)
for C
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner