Overview

1 Regression testing

2 Automated testing

Wondimagegn D. (AAIT ) Quality Assurance and Software Testing March 22, 2020 1 / 19
 Regression testing

Regression Testing

Sometimes small changes can produce unforeseen effects that lead to

new failures
The new version does not provide correct functionality anymore. This
is regression
Regression testing focuses on regression problems
Regression testing retests the particular bug and recheck the integrity
of the program as a whole
Used during unit testing, integration and system testing
In OO to gain confidence in reused components

Wondimagegn D. (AAIT ) Quality Assurance and Software Testing March 22, 2020 2 / 19
 Regression testing

Regression testing

Now the software product P is launched:

An established set of tests T is available.

Wondimagegn D. (AAIT ) Quality Assurance and Software Testing March 22, 2020 3 / 19
 Regression testing

Regression testing

During operation and maintenance

The software product P can be modified
Regression testing attempts to validate the modified software and
ensure that new errors are not introduced.
This is maintenance regression testing

Wondimagegn D. (AAIT ) Quality Assurance and Software Testing March 22, 2020 4 / 19
 Regression testing

Regression testing

Question : How to design T’ ?

Wondimagegn D. (AAIT ) Quality Assurance and Software Testing March 22, 2020 5 / 19
 Regression testing

Regression testing

Regression testing strategies

Retest-all – reuses all test cases consume excessive time and
resources
Selective testing-attempts to reduce the time required to retest by
selecting only a subset of existing test set.

Wondimagegn D. (AAIT ) Quality Assurance and Software Testing March 22, 2020 6 / 19
 Regression testing

Regression testing

Regression test selection (RTS) techniques

A few are specification based (black box)
Most are code based (white box) based on the differences between
the P and P’
Coverage based - A test will be reused only if the any program entities
it covers have been changed (like make utility)
Safe techniques: select every test in T that can expose one or more
faults in P’.
Minimization techniques: select a minimal set of tests

Wondimagegn D. (AAIT ) Quality Assurance and Software Testing March 22, 2020 7 / 19
 Regression testing

Regression testing

Example: CFG based selection

Based on the difference between the control flow graph of P and P’.
Select the test cases that exercise paths through changed portions of
CFG
Technique is effective only when changes affect a small set of
program paths

Wondimagegn D. (AAIT ) Quality Assurance and Software Testing March 22, 2020 8 / 19
 Regression testing

Regression testing

Test case prioritization

RTS can select a lot of tests.
After that, test case prioritization is sometimes necessary
Simplest: Random sampling
Others, based on:
Execution history: recent/old or #faults found
Structural coverage (#elements covered, old or new elements)
fault-proneness models

Wondimagegn D. (AAIT ) Quality Assurance and Software Testing March 22, 2020 9 / 19
 Automated testing

Automated testing

Repetitive tasks are boring and error prone

Test execution is a mechanical operation and can be automated
Run-time support for testing enables frequent hands-free execution of
test cases
Also test data judgement can be automated, using automated test
oracles
In many organizations a build-and-test cycle occurs during night with
reports each morning

Wondimagegn D. (AAIT ) Quality Assurance and Software Testing March 22, 2020 10 / 19
 Automated testing

Automated testing

If an effective testing process already exists, automated testing can

help
A database with detailed test cases is needed
Mainly used for automating acceptance and regression testing.

Wondimagegn D. (AAIT ) Quality Assurance and Software Testing March 22, 2020 11 / 19
 Automated testing

When to automate?

2 extreme situations; a completely automated test that can run

entirely unattended, and a one-shot manual test that is run once and
thrown away.
A rational decision has to be made for each particular test
Creating an automated test is usually more time consuming
(expensive) than running it once manually.
How to measure the costs of automation?
Usually by measuring the time spent on doing it.
Or evaluating what bugs are not find if I do this?

Wondimagegn D. (AAIT ) Quality Assurance and Software Testing March 22, 2020 12 / 19
 Automated testing

Method

An automatic test does not last forever. At some point the product
will change in a way that that breaks the test.The test has to be
repaired or discarded.
If the test hasn’t repaid the automation effort by that point, you
would have been better off leaving it is a manual test.
Must estimate how many code changes the automatic test will
survive.

Wondimagegn D. (AAIT ) Quality Assurance and Software Testing March 22, 2020 13 / 19
 Automated testing

How to automate?

Automated test script is a program that does the same operations like the
manual tester does.
Languages (C++, Basic, Java script, Perl, Ruby, etc)
The script can be generated by using a keyboard capture/playback
tool
Feature that enables the user to record keystrokes and mouse
movements while interacting with an application.
Keystrokes are recorded in a script and played back during test
execution
Drawback?

Wondimagegn D. (AAIT ) Quality Assurance and Software Testing March 22, 2020 14 / 19
 Automated testing

Problem

The code under test is divided in 2: feature code and support code.
The tester tests the feature code, but support code also can change.
The scripts generated by record/play tools contain hard coded values
which must change if anything changes in the application
If the tester made a error during recording the test has to be
re-recorded
Solution : Data driven automated testing

Wondimagegn D. (AAIT ) Quality Assurance and Software Testing March 22, 2020 15 / 19
 Automated testing

IBM Rational Functional Tester

Automated, functional and regression testing tool

Supports Data-driven testing

Wondimagegn D. (AAIT ) Quality Assurance and Software Testing March 22, 2020 16 / 19
 Automated testing

Watir

Web Application Testing In Ruby

Open source library for automating web browsers
Built on the object-oriented scripting language Ruby
Used for large-scale system testing, functional testing and for
automating user acceptance testing
Automate a repetitive web-based task

Wondimagegn D. (AAIT ) Quality Assurance and Software Testing March 22, 2020 17 / 19
 Automated testing

Ruby scripting language

In Ruby everything is an object

“watir”.reverse (object.message)
A object “watir” string is created and a message “reverse” is sent to
this object.
Ruby Interactive Shell: irb

Wondimagegn D. (AAIT ) Quality Assurance and Software Testing March 22, 2020 18 / 19
 Automated testing

A Watir test script

$LOAD PATH << ’T : ’

$LOAD PATH << ’T: \ w a t i r ’
require ’ watir ’
i n c l u d e Watir
i e=I E . new
i e . g o t o ( ” h t t p : / / g o o g l e . com” )
i e . t e x t f i e l d ( : name , ”q” ) . s e t ( ” p i c k a x e ” )
i e . button ( : value , ” Google zoeken ” ) . c l i c k

How to execute the script

Each line from the script can be tried first using the command
interpreter irb.
The whole script saved as for example testGoogle.rb can be run using
the Ruby interpreter

Wondimagegn D. (AAIT ) Quality Assurance and Software Testing March 22, 2020 19 / 19