Unit 3-Software Testing
Unit 3-Software Testing
Regression testing is a black box testing techniques. It is used to authenticate a code change in
the software does not impact the existing functionality of the product. Regression testing is
making sure that the product works fine with new functionality, bug fixes, or any change in the
existing feature.
Regression testing is a type of software testing. Test cases are re-executed to check the previous
functionality of the application is working fine, and the new changes have not produced any
bugs.
Regression testing can be performed on a new build when there is a significant change in the
original functionality. It ensures that the code still works even when the changes are occurring.
Regression means Re-test those parts of the application, which are unchanged.
Regression tests are also known as the Verification Method. Test cases are often automated. Test
cases are required to execute many times and running the same test case again and again
manually, is time-consuming and tedious too.
Here we are going to take a case to define the regression testing efficiently:
Example:
A website has a login functionality which allows users to log in only with Email. Now providing
a new feature to do login using Facebook.
2. When there is a Change Requirement.
Example:
Remember password removed from the login page which is applicable previously.
Example:
Assume login button is not working in a login page and a tester reports a bug stating that the
login button is broken. Once the bug fixed by developers, tester tests it to make sure Login
Button is working as per the expected result. Simultaneously, tester tests other functionality
which is related to the login button.
Example:
Loading of a home page takes 5 seconds, reducing the load time to 2 seconds.
Example:
The need for regression testing comes when software maintenance includes enhancements, error
corrections, optimization, and deletion of existing features. These modifications may affect
system functionality. Regression Testing becomes necessary in this case.
Re-Test is one of the approaches to do regression testing. In this approach, all the test case suits
should be re-executed. Here we can define re-test as when a test fails, and we determine the
cause of the failure is a software fault. The fault is reported, we can expect a new version of the
software in which defect fixed. In this case, we will need to execute the test again to confirm that
the fault fixed. This is known as re-testing. Some will refer to this as confirmation testing.
o In this technique, a selected test-case suit will execute rather than an entire test-case suit.
o The selected test case suits divided in two cases
1. Reusable Test cases.
2. Obsolete Test cases.
o Reusable test cases can use in succeeding regression cycle.
o Obsolete test cases can't use in succeeding regression cycle.
Prioritize the test case depending on business impact, critical and frequently functionality used.
Selection of test cases will reduce the regression test suite.
WHAT ARE THE REGRESSION TESTING TOOLS?
Regression Testing is a vital part of the QA process; while performing the regression we may
face the below challenges:
o Time Consuming
Regression Testing consumes a lot of time to complete. Regression testing involves
existing tests again, so testers are not excited to re-run the test.
o Complex
Regression Testing is complex as well when there is a need to update any product; lists of
the test are also increasing.
o Communicating business rule
Regression Testing ensures the existing product features are still in working order.
Communication about regression testing with a non-technical leader can be a difficult
task. The executive wants to see the product move forward and making a considerable
time investment in regression testing to ensure existing functionality working can be
hard.
o Identify Impact Area
o Test Cases Increases Release by Release
o Less Resources
o No Accuracy
o Repetitive Task
o Monotonous Job
The regression testing process can be performed across the builds and the releases.
Whenever the bug fixed, we retest the Bug, and if there is any dependent module, we go for a
Regression Testing.
For example, How we perform the regression testing if we have different builds as Build 1, Build
2, and Build 3, which having different scenarios.
Build1
Build2
o Now, the customer asks for 3-4 extra (new) features to be added and also provides the
requirements for the new features.
o The development team starts developing new features.
o After that, the testing team will start writing the test case for the new features, and they
write about 150 new test cases. Therefore, the total number of the test case written is
1050 for both the releases.
o Now the testing team starts testing the new features using 150 new test cases.
o Once it is done, they will begin testing the old features with the help of 900 test cases to
verify that adding the new feature has damaged the old features or not.
o Here, testing the old features is known as Regression Testing.
o Once all the features (New and Old) have been tested, the product is handed over to the
customer, and then the customer will do the acceptance testing.
o Once the acceptance testing is done, the product is moved to the production server.
Build3
o After the second release, the customer wants to remove one of the features like Sales.
o Then he/she will delete all the test cases which are belonging to the sales module (about
120 test cases).
o And then, test the other feature for verifying that if all the other features are working fine
after removing the sales module test cases, and this process is done under the regression
testing.
Note:
o Testing the stable features to ensure that it is broken because of the changes. Here
changes imply that the modification, addition, bug fixing, or the deletion.
o Re-execution of the same test cases in the different builds or releases is to ensure that
changes (modification, addition, bug fixing, or the deletion) are not introducing bugs in
stable features.
The regression testing process starts whenever there is a new Release for same project because
the new feature may affect the old elements in the previous releases.
To understand the regression testing process, we will follow the below steps:
Step1
Step2
The concept of Regression testing starts from Release#2 when the customer gives some new
requirements.
Step3
After getting the new requirements (modifying features) first, they (the developers and test
engineers) will understand the needs before going to the impact analysis.
Step4
After understanding the new requirements, we will perform one round of impact analysis to
avoid the major risk, but here the question arises who will do the Impact analysis?
Step5
The impact analysis is done by the customer based on their business knowledge,
the developer based on their coding knowledge, and most importantly, it is done by the test
engineer because they have the product knowledge.
Note: If a single person does, he/she may not cover all the impact areas, so we include all
persons so that we may cover a maximum impact area, and Impact Analysis should be done at
the early stages of the releases.
Step6
Once we are done with the impact area, then the developer will prepare the impact area
(document), and the customer will also prepare the impact area document so that we can achieve
the maximum coverage of impact analysis.
Step7
After completing the impact analysis, the developer, the customer, and the test engineer will send
the Reports# of the impact area documents to the Test Lead. And in the meantime, the test
engineer and the developer are busy working on the new test case.
Step8
Once the Test lead gets the Reports#, he/she will consolidate the reports and stored in the test
case requirement repository for the release#1.
Note: Test case Repository: Here, we will save all the test case of releases.
Step9
After that, the Test Lead will take the help of RTM and pick the necessary regression test
case from the test case repository, and those files will be placed in the Regression Test Suite.
Note:
o The test lead will store the regression test case in the regression test suite for no further
confusion.
o Regression test suite: Here, we will save all the impact area test documents.
o Regression Test Cases: These are the test cases of the old releases text document which
need to be re-executed as we can see in the below image:
Step10
After that, when the test engineer has done working on the new test cases, the test lead
will assign the regression test case to the test engineer.
Step11
When all the regression test cases and the new features are stable and pass, then check the impact
area using the test case until it is durable for old features plus the new features, and then it will
be handed over to the customer.
TYPES OF REGRESSION TESTING
In this, we are going to test only the changed unit, not the impact area, because it may affect the
components of the same module.
Example1
In the below application, and in the first build, the developer develops the Search button that
accepts 1-15 characters. Then the test engineer tests the Search button with the help of the test
case design technique.
Now, the client does some modification in the requirement and also requests that the Search
button can accept the 1-35 characters. The test engineer will test only the Search button to verify
that it takes 1-35 characters and does not check any further feature of the first build.
Example2
Here, we have Build B001, and a defect is identified, and the report is delivered to the developer.
The developer will fix the bug and sends along with some new features which are developed in
the second Build B002. After that, the test engineer will test only after the defect is fixed.
o The test engineer will identify that clicking on the Submit button goes to the blank page.
o And it is a defect, and it is sent to the developer for fixing it.
o When the new build comes along with the bug fixes, the test engineer will test only the
Submit button.
o And here, we are not going to check other features of the first build and move to test the
new features and sent in the second build.
o We are sure that fixing the Submitbutton is not going to affect the other features, so we
test only the fixed bug.
Therefore, we can say that by testing only the changed feature is called the Unit Regression
Testing.
In this, we are going to test the modification along with the impact area or regions, are called
the Regional Regression testing. Here, we are testing the impact area because if there are
dependable modules, it will affect the other modules also.
For example:
In the below image as we can see that we have four different modules, such as Module A,
Module B, Module C, and Module D, which are provided by the developers for the testing
during the first build. Now, the test engineer will identify the bugs in Module D. The bug report
is sent to the developers, and the development team fixes those defects and sends the second
build.
In the second build, the previous defects are fixed. Now the test engineer understands that the
bug fixing in Module D has impacted some features in Module A and Module C. Hence, the test
engineer first tests the Module D where the bug has been fixed and then checks the impact areas
in Module A and Module C. Therefore, this testing is known as Regional regression testing.
While performing the regional regression testing, we may face the below problem:
Problem:
In the first build, the client sends some modification in requirement and also wants to add new
features in the product. The needs are sent to both the teams, i.e., development and testing.
After getting the requirements, the development team starts doing the modification and also
develops the new features based on the needs.
Now, the test lead sends mail to the clients and asks them that all are the impact areas that will be
affected after the necessary modification have been done. Therefore, the customer will get an
idea, which all features are needed to be tested again. And he/she will also send a mail to the
development team to know which all areas in the application will be affected as a result of the
changes and additions of new features.
And similarly, the customer sends a mail to the testing team for a list of impact areas. Hence, the
test lead will collect the impact list from the client, development team, and the testing team as
well.
This Impact list is sent to all the test engineers who look at the list and check if their features are
modified and if yes, then they do regional regression testing. The impact areas and modified
areas are all tested by the respective engineers. Every test engineer tests only their features that
could have been affected as a result of the modification.
The problem with this above approach is that the test lead may not get the whole idea of the
impact areas because the development team and the client may not have so much time to revert
his/her mails.
Solution
When a new build comes along with the latest features and bug fixes, the testing team will
arrange the meeting where they will talk about if their features are affecting because of the above
modification. Therefore, they will do one round of Impact Analysis and generate the Impact List.
In this particular list, the test engineer tries to enclose the maximum probable impact areas,
which also decreases the chance of getting the defects.
When a new build comes, the testing team will follow the below procedure:
Following are some of the drawbacks of using unit and Regional regression testing:
Note: We can say that the major work we do on the regional regression testing will lead us to get
more number of defects. But, if we will perform the same dedication to work on the full
regressing testing, we will get less number of defects. Therefore, we can determine here that
enhancement in the testing effort will not help us to get more defects.
During the second and the third release of the product, the client asks for adding 3-4 new
features, and also some defects need to be fixed from the previous release. Then the testing team
will do the Impact Analysis and identify that the above modification will lead us to test the entire
product.
Therefore, we can say that testing the modified features and all the remaining (old) features is
called the Full Regression testing.
When we perform Full Regression testing?
o When the modification is happening in the source file of the product. For example, JVM
is the root file of the JAVA application, and if any change is going to happen in JVM,
then the entire JAVA program will be tested.
o When we have to perform n-number of changes.
Note:
The regional regression testing is the ideal approach of regression testing, but the issue is, we
may miss lots of defects while performing the Regional Regression testing.
And here we are going to solve this issue with the help of the following approach:
o When the application is given for the testing, the test engineer will test the first 10-14
cycle, and will do the RRT.
o Then for the 15th cycle, we do FRT. And again, for the next 10-15 cycle, we do Regional
regression testing, and for the 31th cycle, we do the full regression testing, and we will
continue like this.
o But for the last ten cycle of the release, we will perform only complete regression testing.
Hence, we will go for the automation to get over with these issues; when we have n-number of
the regression test cycle, we will go for the automation regression testing process.
Generally we go for the automation whenever there are multiple releases or multiple regression
cycle or there is the repetitive task.
The automation regression testing process can be done in the following steps:
Note1:
The process of testing the application by using some tools is known as automation testing.
Suppose if we take one sample example of a Login module, then how we can perform the
regression testing.
Here, the Login can be done in two ways, which are as follows:
Issues Handle by
Step1
When the new release starts, we don't go for the automation because there is no concept of
regression testing and regression test case as we understood this in the above process.
Step2
When the new release and the enhancement starts, we have two teams, i.e., manual team and the
automation team.
Step3
The manual team will go through the requirements and also identify the impact area and hand
over the requirement test suite to the automation team.
Step4
Now, the manual team starts working on the new features, and the automation team will start
developing the test script and also start automating the test case, which means that the regression
test cases will be converted into the test script.
Step5
Before they (automation team) start automating the test case, they will also analyze which all
cases can be automated or not.
Step6
Based on the analysis, they will start the automation i.e., converting every regression test cases
into the test script.
Step7
During this process, they will take help of the Regression cases because they don't have product
knowledge as well as the tool and the application.
Step8
Once the test script is ready, they will start the execution of these scripts on the new application
[old feature]. Since, the test script is written with the help of the regression feature or the old
feature.
Step9
Step10
If the status is failed, which means it needs to be re-confirmed manually, and if the Bug exists,
then it will report to the concerned developer. When the developer fixes that bug, the Bug needs
to be re-tested along with the Impact area by the manual test engineer, and also the script needs
to be re-executed by the automation test engineer.
Step11
This process goes on until all the new features, and the regression feature will be passed.
Benefits of doing regression testing by the automation testing:
o Accuracy always exists because the task is done by the tools and tools never get bored or
tired.
o The test script can be re-used across multiple releases.
o Batch execution is possible using the automation i.e.; all the written test scripts can be
executed parallel or simultaneously.
o Even though the number of regression test case increase release per release, and we don't
have to increase the automation resource since some regression case are already
automated from the previous release.
o It is a time-saving process because the execution is always faster than the manual
method.
It was found from industry inspection. The several defects reported by the customer were due to
last-minute bug fixes. These creating side effects and hence selecting the Test Case for
regression testing is an art, not an easy task.
If Software undergoes frequent changes, regression testing costs also increase. In those cases,
manual execution of test cases increases test execution time as well as costs. In that case,
automation testing is the best choice. The duration of automation depends on the number of test
cases that remain reusable for successive regression cycles.
Following are the essential tools used for regression testing:
Selenium
Selenium is an open-source tool. This tool used for automated testing of a web application. For
browser-based regression testing, selenium used. Selenium used for UI level regression test for
web-based application.
Ranorex Studio
All in one regression test automation for desktop, web, and mobile apps with built-in Selenium
Web Driver. Ranorex Studio includes full IDE plus tools for codeless automation.
QTP is an automated testing tool used for Regression and Functional Testing. It is a Data-
Driven, keyword-based tool. It used VBScript language for automation. If we open the QTP tool,
we see the three buttons which are Record, Play and Stop. These buttons help to record every
click and action performed on the computer system. It records the actions and play it back.
Rational functional tester is a Java tool used to automate the test cases of software applications.
RTF used for automating regression test cases, and it also integrates with the rational functional
tester.
For more information about regression and automation testing tools refer to the below link:
o Changes are not allowed in the code during the regression testing phase.
o A regression test case must be unaffected developer changes.
o The database used for regression testing must be isolated; changes are not allowed in the
database.
Re-testing Testing means testing the functionality or bug again to ensure the code fixed. If not
set, defects need not be re-opened. If fixed, the defect closed.
Re-testing is a type of testing which performed to check the test-cases that were unsuccessful in
the final execution are successfully pass after the defects repaired.
Regression Testing means testing the software application when it undergoes a code change to
ensure that new code has not affected other parts of the Software.
Regression testing is a type of testing executed to check whether a code has not changed the
existing functionality of the application.
Re-testing is performed to ensure that the Regression Testing is done to confirm whether the
test cases that are failed in the final code change has not affected the existing features.
execution are passing after the defects
fixed.
Re-Testing works on defect fixes. The purpose of regression testing is to ensure that
the code changes adversely not affect the existing
functionality.
Defect verification is the part of the Regression testing does not include defect
Retesting. verification
The priority of Retesting is higher than Based on the project type and availability of
Regression Testing, so it is done before resources, regression testing can be parallel to
the Regression Testing. Retesting.
Re-testing make sure that the original Regression testing checks for unexpected side
fault is corrected. effect.
Retesting executes defects with the same Regression testing is when there is a modification
data and the same environment with or changes become mandatory in an existing
different input with a new build. project.
Re-testing cannot do before start testing. Regression testing can obtain test cases from the
functional specification, user tutorials and
manuals, and defects reports in regards to the
corrected problem.
Differences between the Re-testing and Regression Testing are as follows:
There are several advantages of Regression Testing though there are disadvantages as well.
o Regression Testing should be done for small changes in the code because even a slight
change in the code can create issues in the existing functionality.
o If in case automation is not used in the project for testing, it will time consuming and
tedious task to execute the test again and again.
As the name suggests, test case prioritization refers to prioritizing test cases in test suite on
basis of different factors. Factors could be code coverage, risk/critical modules, functionality,
features, etc.
WHY SHOULD TEST CASES BE PRIORITIZED?
As the size of software increases, test suite also grows bigger and also requires more efforts to
maintain test suite. In order to detect bugs in software as early as possible, it is important to
prioritize test cases so that important test cases can be executed first.
• General Prioritization :
In this type of prioritization, test cases that will be useful for the subsequent modified
versions of product are prioritized. It does not require any information regarding
modifications made in the product.
PRIORITIZATION TECHNIQUES :
This type of prioritization is based on code coverage i.e. test cases are prioritized on basis of
their code coverage.
This technique uses risk analysis to identify potential problem areas which if failed, could lead
to bad consequences. Therefore, test cases are prioritized keeping in mind potential problem
areas. In risk analysis, following steps are performed :
After performing above steps, risk analysis table is formed to present results. The table consists
of columns like Problem ID, Potential problem identified, Severity of Impact, Risk exposure,
etc.
In this type of prioritization, slicing technique is used – when program is modified, all existing
regression test cases are executed in order to make sure that program yields same result as
before, except where it has been modified. For this purpose, we try to find part of program
which has been affected by modification, and then prioritization of test cases is performed for
this affected part. There are 3 parts to slicing technique :
• Execution slice –
The statements executed under test case form execution slice.
• Dynamic slice –
Statements executed under test case that might impact program output.
• Relevant Slice –
Statements that are executed under test case and don’t have any impact on the program
output but may impact output of test case.
Some requirements are more important than others or are more critical in nature, hence test
cases for such requirements should be prioritized first. The following factors can be considered
while prioritizing test cases based on requirements :
• Customer assigned priority –
The customer assigns weight to requirements according to his need or understanding of
requirements of product.
• Developer perceived implementation complexity –
Priority is assigned by developer on basis of efforts or time that would be required to
implement that requirement.
• Requirement volatility –
This factor determines frequency of change of requirement.
• Fault proneness of requirements –
Priority is assigned based on how error-prone requirement has been in previous versions of
software.
where,
TFi = position of first Test case in Test suite T that exposes Fault i
m = total number of Faults exposed under T
n = total number of Test cases in T
AFPD value can range from 0 to 100. The higher APFD value, faster faults detection rate. So
simply put, APFD indicates of how quickly test suite can identify faults or bugs in software. If
test suite can detect faults quickly, then it is considered to be more effective and reliable.
RISK ANALYSIS
The probability of any unwanted incident is defined as Risk. In Software Testing, risk analysis is
the process of identifying the risks in applications or software that you built and prioritizing them
to test. After that, the process of assigning the level of risk is done. The categorization of the
risks takes place; hence, the impact of the risk is calculated.
In any software, using risk analysis at the beginning of a project highlights the potential problem
areas. After knowing about the risk areas, it helps the developers and managers to mitigate the
risks. When a test plan has been created, risks involved in testing the product are to be taken into
consideration along with the possibility of the damage they may cause to your software along
with solutions.
Now, you might think what could be the possible risks that you could encounter? Well here is a
list:
Now, you must know there are certain risks that are unavoidable. I am enumerating them below:
• Instructor-led Sessions
• Real-life Case Studies
• Assessments
• Lifetime Access
Explore Curriculum
In such cases, you have to tackle the situation with care. Following points can be taken care of:
Now, what are these risk magnitude indicators? Well, here is an explanation.
High: means the effect of the risk would be very high and non-tolerable. The company might
face loss.
Medium: it is tolerable but not desirable. The company may suffer financially but there is a
limited risk.
Risk Identification
There are different sets of risks included in the risk identification process. Those are as follows:
1. Business Risks: This risk is the most common risk associated with our topic. It is the risk
that may come from your company or your customer, not from your project.
2. Testing Risks: You should be well acquainted with the platform you are working on,
along with the software testing tools being used.
3. Premature Release Risk: a fair amount of knowledge to analyze the risk associated with
releasing unsatisfactory or untested software is required
4. Software Risks: You should be well versed with the risks associated with the software
development process.
After identifying the risks associated with your software, the next step is to assess the risks; i.e,
Risk Assessment.
RISK ASSESSMENT
In the risk analysis process, these steps prove to be the most important one. It is said that this
step is way too complex and should be tackled with the utmost care. After risk identification,
assessment has to be dealt programmatically. There are a few perspectives on risk assessment.
Read on!
Moving on! Our next topic on Risk Analysis in Software Testing
• Effect
• Cause
• Likelihood
Effect – To assess risk by Effect. In case you identify a condition, event or action and try to
determine its impact.
Cause – To assess risk by Cause is opposite of by Effect. Initialize scanning the problem and
reach to the point that could be the most probable reason behind that.
Likelihood – To assess risk by Likelihood is to say that there is a probability that a requirement
won’t be satisfied.
Now, heading forward, the question that would be hovering over your mind is, how actually shall
we perform risk analysis? Well, here is your solution!