Scribd
Upload
55 views12 pages

Regression Testing

Regression testing involves re-running tests on a system after changes to check that previous errors have not been reintroduced. It aims to test functionality and architecture through black box and grey box tests. Automating regression tests faces challenges in determining what parts of output should be checked against the oracle. Parameterizing test results by extracting relevant info from output and comparing to an oracle is a solution. Traceability of tests to requirements is important for knowing which tests to update when functionality changes. Tests must be updated for bug fixes, feature additions, platform changes, and system variants. New regression tests are added for reported bugs using either the original input or simplest reproduction.

Uploaded by

Siddharth Agrawal
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
55 views12 pages

Regression Testing

Regression testing involves re-running tests on a system after changes to check that previous errors have not been reintroduced. It aims to test functionality and architecture through black box and grey box tests. Automating regression tests faces challenges in determining what parts of output should be checked against the oracle. Parameterizing test results by extracting relevant info from output and comparing to an oracle is a solution. Traceability of tests to requirements is important for knowing which tests to update when functionality changes. Tests must be updated for bug fixes, feature additions, platform changes, and system variants. New regression tests are added for reported bugs using either the original input or simplest reproduction.

Uploaded by

Siddharth Agrawal
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 12

Regression testing

Tor Stllhane

What is regression testing 1


Regression testing is testing done to check that a system update does not reintroduce errors that have been corrected earlier. All or almost all regression tests aim at checking Functionality black box tests. Architecture grey box tests

What is regression testing 2


Since they are supposed to test all functionality and all previously done changes, regression tests are usually large. Thus, regression testing needs automatic Execution no human intervention Checking. Leaving the checking to developers will not work.

Automating regression tests 1


We face the same challenge when doing automating regression test as we face when doing automatic test checking in general: Which parts of the output should be checked against the oracle? This question gets more important as we need to have more version of the same test due to system variability.

Automating regression tests 2


Simple but annoying and some times expensive problems are e.g. Use of date in the test output Changes in number of blanks or line shifts Other format changes Changes in lead texts

Automating regression tests 3


A simple solution seems to be to use assertions. This is not a good alternative for the following reasons: The assertion will have to be Inside the system all the time extra space and execution time Only in the test version one extra version per variation to maintain.

Automating regression tests 4


The answer to the question on the previous slide is to parameterize the test results. Instead of using a complete output file as the oracle we use: A tool to extract the relevant info from the output file Compare the extracted info with the info stored in the oracle

Automating regression tests 5


Test data

Build results for version x

Build tests for version x

Run tests for version x

Compare

Verdict

Run all regression tests


Since a regression test is large, it is always a need to identify which parts of the regression tests need to be run after a change e.g. an error correction. Thus, traceability which components are used to realize which functionality or requirement is important info for two reasons: Save execution time. Fast and cheap hardware is, however, steadily reducing this need. Know which tests to change when we change functionality.

Improving the regression test


Once we have a regression test, it is important to update it each time we Fix a bug Add, change or remove functionality Change platform If we create variants of the system, we also need to create parallel variants of the regression test suite.

Bug fixing 1
When somebody reports a bug, the first activity is to reproduce it preferably with the simplest input sequence possible. When the bug is reproduced, we need to make a new entry in the regression test. This entry will be one of the following The original reported input sequence The simplest possible input sequence

Bug fixing 2
The original reported input sequence
Pro: is the real thing and will tell us something about how the system is used in real life. Con: can be large, complex and difficult to understand

The simplest possible input sequence


Pro: is (usually) simple to understand Con: is an artificial input sequence and has no info beside being able to reproduce a reported bug

You might also like