Swhat Is Software Testing? Definition, Basics & Types
Swhat Is Software Testing? Definition, Basics & Types
Definition,
Basics & Types
What is Software Testing?
SOFTWARE TESTING is defined as an activity to check whether the actual
results match the expected results and to ensure that the software system
is Defect free. It involves the execution of a software component or system
component to evaluate one or more properties of interest. Software testing also
helps to identify errors, gaps, or missing requirements in contrary to the actual
requirements. It can be either done manually or using automated tools. Some prefer
saying Software testing as a White Box and Black Box Testing.
Functional Testing
Non-Functional Testing or Performance Testing
Maintenance (Regression and Maintenance)
Integration Testing
Smoke
Localization
Globalization
Interoperability
So on
Endurance
Load
Volume
Scalability
Usability
So on
Maintenance Regression
Maintenance
This is not the complete list as there are more than 150 types of testing types and
still adding. Also, note that not all testing types are applicable to all projects but
depend on the nature & scope of the project.
Integration testing: It focuses on the construction and design of the software. You
need to see that the integrated units are working without errors or not.
System testing: In this method, your software is compiled as a whole and then
tested as a whole. This testing strategy checks the functionality, security,
portability, amongst others.
Program Testing
Program testing is the method of executing any program with the aim of finding
errors. A good testing is one which has a high chance of finding bugs. This testing
process can show whether the errors are present or not. It is possible to write test
cases before writing any program.
Software Validation
Validation is process of examining whether or not the software satisfies the user
requirements. It is carried out at the end of the SDLC. If the software matches
requirements for which it was made, it is validated.
Software Verification
Verification is the process of confirming if the software is meeting the business
requirements, and is developed adhering to the proper specifications and
methodologies.
Testing Approaches
Tests can be conducted based on two approaches –
Functionality testing
Implementation testing
When functionality is being tested without taking the actual implementation in
concern it is known as black-box testing. The other side is known as white-box
testing where not only functionality is tested but the way it is implemented is also
analyzed.
Exhaustive tests are the best-desired method for a perfect testing. Every single
possible value in the range of the input and output values is tested. It is not possible
to test each and every value in real world scenario if the range of values is large.
Black-box testing
It is carried out to test functionality of the program. It is also called ‘Behavioral’
testing. The tester in this case, has a set of input values and respective desired
results. On providing input, if the output matches with the desired results, the
program is tested ‘ok’, and problematic otherwise.
In this testing method, the design and structure of the code are not known to the
tester, and testing engineers and end users conduct this test on the software.
Black-box testing techniques:
Equivalence class - The input is divided into similar classes. If one element
of a class passes the test, it is assumed that all the class is passed.
Boundary values - The input is divided into higher and lower end values. If
these values pass the test, it is assumed that all values in between may pass
too.
Cause-effect graphing - In both previous methods, only one input value at a
time is tested. Cause (input) – Effect (output) is a testing technique where
combinations of input values are tested in a systematic way.
Pair-wise Testing - The behavior of software depends on multiple
parameters. In pairwise testing, the multiple parameters are tested pair-wise
for their different values.
State-based testing - The system changes state on provision of input. These
systems are tested based on their states and input.
White-box testing
In this testing method, the design and structure of the code are known to the tester.
Programmers of the code conduct this test on the code.
The below are some White-box testing techniques:
Control-flow testing - The purpose of the control-flow testing to set up test
cases which covers all statements and branch conditions. The branch
conditions are tested for both being true and false, so that all statements can
be covered.
Data-flow testing - This testing technique emphasis to cover all the data
variables included in the program. It tests where the variables were declared
and defined and where they were used or changed.
Testing Levels
Testing itself may be defined at various levels of SDLC. The testing process runs
parallel to software development. Before jumping on the next stage, a stage is
tested, validated and verified.
Testing separately is done just to make sure that there are no hidden bugs or issues
left in the software. Software is tested on various levels -
Unit Testing
While coding, the programmer performs some tests on that unit of program to know
if it is error free. Testing is performed under white-box testing approach. Unit testing
helps developers decide that individual units of the program are working as per
requirement and are error free.
Integration Testing
Even if the units of software are working fine individually, there is a need to find out
if the units if integrated together would also work without errors. For example,
argument passing and data updation etc.
System Testing
The software is compiled as product and then it is tested as a whole. This can be
accomplished using one or more of the following tests:
Functionality testing - Tests all functionalities of the software against the
requirement.
Performance testing - This test proves how efficient the software is. It tests
the effectiveness and average time taken by the software to do desired task.
Performance testing is done by means of load testing and stress testing
where the software is put under high user and data load under various
environment conditions.
Security & Portability - These tests are done when the software is meant to
work on various platforms and accessed by number of persons.
Acceptance Testing
When the software is ready to hand over to the customer it has to go through last
phase of testing where it is tested for user-interaction and response. This is
important because even if the software matches all user requirements and if user
does not like the way it appears or works, it may be rejected.
Alpha testing - The team of developer themselves perform alpha testing by
using the system as if it is being used in work environment. They try to find
out how user would react to some action in software and how the system
should respond to inputs.
Beta testing - After the software is tested internally, it is handed over to the
users to use it under their production environment only for testing purpose.
This is not as yet the delivered product. Developers expect that users at this
stage will bring minute problems, which were skipped to attend.
Regression Testing
Testing Documentation
Testing documents are prepared at different stages -
Before Testing
Testing starts with test cases generation. Following documents are needed for
reference –
SRS document - Functional Requirements document
Test Policy document - This describes how far testing should take place
before releasing the product.
Test Strategy document - This mentions detail aspects of test team,
responsibility matrix and rights/responsibility of test manager and test
engineer.
Traceability Matrix document - This is SDLC document, which is related to
requirement gathering process. As new requirements come, they are added
to this matrix. These matrices help testers know the source of requirement.
They can be traced forward and backward.
The following documents may be required while testing is started and is being done:
Test Case document - This document contains list of tests required to be
conducted. It includes Unit test plan, Integration test plan, System test plan
and Acceptance test plan.
Test description - This document is a detailed description of all test cases
and procedures to execute them.
Test case report - This document contains test case report as a result of the
test.
Test logs - This document contains test logs for every test case report.
After Testing
Given below are some of the most common myths about software testing.
Serving as a bridge between users and development team of a product, the ultimate goal of
software testing is to troubleshoot all the issues and bugs as well as control the quality of a
resulted product. On the way to the ultimate goal of testing using manual and automated means,
QA specialists are aimed to complete different objectives.
Here are five the most essential software testing objectives:
Bug Prevention. QA engineers prevent defects in a system at the earliest stage of development.
The bug-prevention objective is superior to others and implies not only anticipation but also
prevention of defects from recurring in the future. In the long run, bug prevention helps to shorten
the product time to market, reduce the cost of software quality maintenance and increase the
customer satisfaction and loyalty to your product.
Bug Detection. QA experts detect and root out bugs and malfunctions before customers find
them. It’s a short-term objective that requires a scrutinous approach which can be provided by
manual software testing.
User Satisfaction. QA team make sure that the product satisfies the user requirements and
works as desired. In the process of the software verification & validation, a tester usually writes a
set of test cases which help to determine the software compliance with specific business and
user requirements under positive and negative conditions. For instance, our team establishes the
set of tests to ensure the main functions and features cover different scenarios, work properly in
a range of countries and locations.
Software quality and reliability. Keeping control of software quality and reliability. Keeping
control of software quality means keeping bugs at a low level and making sure software is
compatible. Software compatibility is the capability of a software or an app to work well with other
hardware, software or network, including web, desktop, mobile platform types, all types of
operating systems and web browsers, etc.
#1. High quality of software & user satisfaction. These two things that always go hand-in-hand
define the main goal of software testing. All members of a QA team use different types of testing
and tools to achieve the same goal: to release software without critical bugs and make sure
users are happy with it. Automation testing contributes to it with its special techniques.
#2. Minimizing testing time & human effort. Automation is applied where manual testing would
have taken a lot of time. It helps to shorten test time and simplify testing of complex applications
with many features. QA engineers can set up the system, and it will start running automatically at
night. Every test scenario can be executed multiple times, making automation very useful for
regression. Thus, automation optimizes the time and human resources.
#3. Increasing efficiency and quality while reducing costs. Automation is always more
advantageous over the long run. Timely releases and flawless performance require a lot of effort.
QA engineers can execute tests during off-peak hours, distribute them across multiple servers,
simulate different types of environments, etc. Carrying out all those tests manually will take much
more effort. You will need to pay for more working hours and wait for a long time – not the best
scenario for a dynamic project or tight deadlines.
The objectives are specific actions that help to achieve bigger automation testing goals.
#1. Enhancing software quality after each iteration. QA specialists apply automation to check
software specifications after each iteration. Along with bug detection, they focus on bug
prevention and localization. For this, it is necessary to cover all possible scenarios of software
behavior. Well-written automation tests can work as bug-repellents and prevent mistakes that are
expensive to fix.
#2. Improving stability and reliability. Modeling stress situations and seemingly impossible
scenarios (peak traffic, a huge amount of users requesting the same feature, etc.) helps to
discover how an app behaves in unusual circumstances. It, in turn, allows preventing crashes
and lags if such situations happen. To avoid introducing new risks, any test-specific code is
plugged in by the test and only in a test environment.
#3. Helping to understand the system under test. A system under test, or simply SUT, refers to
the thing that is currently tested. Automated tests help to explore numerous “What if…”
scenarios. Without them, the team would need to pore through code. Instead, a QA engineer
turns on a debugger, runs a test, and sees how code works.
#4. Creating tests that are easy to run. The tests should be fully automated so it is possible to run
them without any effort. They should also be self-checking (detect and report errors without
manual inspection) and repeatable (suitable for multiple uses). Ideally, QA engineers should be
able to run each test independently.
#5. Creating tests that are easy to maintain. When QA professionals write automated tests, they
still need to focus on testing, not on the code for automation. Thus, tests should be easy to write,
read, understand, and modify in the future. They should require minimal maintenance when the
system evolves.
#6. Expanding test coverage. QA engineers aim to cover the widest test area possible. To check
all user scenarios, they need to execute hundreds of test cases, validate boundary and edge
cases, ensure the compatibility with different devices and browsers. Automation can cover both
functional and UI testing.
#7. Boosting QA engineers’ effectiveness and motivation. First off, we aren’t trying to say that our
QA engineers unmotivated or fail to work effectively 🙂 Still, manual testing is an error-prone job.
Repeating the same test cases frequently with only slight changes can get frustrating. Needless
to remind that there are many time-consuming low-level tasks. Automation leaves more time for
QA engineers to focus on challenging and uncommon issues, like exploratory testing and risk
analysis.
What is Software Testing? Terminologies |
Importance | Types
Software Testing is not as easy as it is often presumed to be! One needs to possess a knack
for analytical and logical thinking to formulate different test-case scenarios and present an
appropriate solution accordingly.
It’s very important to understand what Software Testing actually is as there are lots of
misconceptions surrounding this domain. Testing is all about out-of-the-box thinking and
verifying, validating, investigating & exploring the product.
“The bitterness of poor quality remains long after the sweetness of low price is forgotten.”
Testing is an easier job for those who don’t comprehend the significance of good quality.
Stay tuned to our series of blogs on Software Testing where I will be covering each aspect,
FROM SCRATCH! Nothing would be left out!
So, if you wish to venture into this domain, stay tuned as these articles can serve as a holy
grail for you.
In our first blog on Testing, we will be covering the following points,
– John Ruskin
You cannot understate the significance of testing if you wish to deliver a quality product to
your customers.
Creating a good quality product which is bug-free, delivers great user experience and runs
seamlessly across multiple channels requires the product to undergo through a rigorous
testing process. The right kind of testing guarantees that your product is seamless and
delivers on the expectations of the client.
Testing is also essential to enrich User Experience. This is because a badly tested product
will have bugs and other contingencies which will consequently hamper in providing a
seamless UX.
Now, developing this lucrative and seamless product requires the tester to review the product
on several parameters, like,
Accessibility
Compatibility
Efficiency
Functionality
Scalability
Performance
Reliable
Usability
1. Test Plan
A Test Case mentions the detailed inputs, execution conditions, steps to reproduce, test data,
the actual & expected results. Different test cases are designed and noted-down during the
Test Case Development phase of STLC which the testers later refer to, to check the
application performance.
Test cases can be broadly classified into Functional & Non-
functional or Positive & Negative.
People often get confused amongst – Test Scenarios and Test Cases. Here is a simple
example to quickly comprehend the difference between the two,
Test Case 1: Check results on entering valid User Id & valid Password
Test Case 2: Check results on entering Invalid User ID & valid Password
Test Case 3: Check results on entering valid User ID & Invalid Password
Test Case 4: Check results on entering Invalid User ID & Invalid Password
Test Case 5: Check response when fields are Empty & Login Button is pressed.
Test Scenario is to Test the login functionality whereas Test Cases that are included in this
Test Scenario are,
3. Priority vs Severity
Priority specifies the level of urgency under which the bug needs to get resolved. Whereas
severity illustrates how critical the bug is, it basically describes the magnitude of impact if the
bug is not resolved. There are different levels of priority and severity, which are depicted in
the image below,
4. Verification v/s Validation:
Verification is a static routine involving checking documents, structure, code, and program.
It incorporates activities involved in creating fantastic programming, including, assessment,
structure investigation, and detail examination. It is a moderate target process.
Validation is assessing the last item to check whether the product meets the clients’ desires
and prerequisites. It is a dynamic tool of approving and testing the real item.
It’s essential for a tester to possess a clear understanding of the process followed to develop
the product in their organization. Similar to SDLC, a proper process is followed to perform
software testing which is called STLC or Software Testing Life Cycle. Let’s catch some
more details about these processes.
Image Courtesy: plutora.com
SDLC vs STLC
SDLC
Software Development Life Cycle (SDLC) explains the journey of Software
Development. According to Wikipedia, SDLC is a process followed for software
development. It consists of a detailed strategy outlining how to develop, maintain,
replace, alter and enhance a specific software.
STLC
STLC illustrates a systematic and well-planned testing process which includes different
stages to make the testing process quick, effective and accountable. Unlike SDLC, STLC
– software testing life cycle identifies how test cases will be implemented and how the
testing would be conducted successfully.
It’s important that a tester must have knowledge of the process followed to develop any
product in any IT company. This process is named as SDLC – software development life
cycle. Add on to this, a proper process is followed to perform software testing i.e STLC –
software testing life cycle. Let’s catch some more details about these processes.
A tester has to perform numerous types of testing in his career. Let’s take a sneak peek at the
different types of software testing out there,
Functional Testing
Functional testing is a type of software testing where the system is tested against some
functional requirements.
Unit Testing
Unit testing consists of the testing of individual modules or components. Its objective is
to test each unit of the software, validate whether the functionality implemented is stable
or not and most importantly it should work as per the design & requirement.It’s the first
and foremost step to perform Software testing and is generally carried out by software
developers themselves. Unit Testing builds confidence in changing or maintaining the
codes. Reusability of code also increases.
Integration Testing
Integration Testing is the second step of software testing, where individual modules are
combined & integrated modules are tested. It started after completion of Unit Testing.
The objective is to test the defects to discover while integrating any modules. This type of
testing verifies the functionality, performance, and reliability between the integrated
modules.
System Testing
System testing is the testing of complete and integrated software. It ensures that the
developed product meets the specific requirements.System Testing involves the
examination of completely integrated applications incorporating outer peripherals to
check how modules interact with each other and with the framework, all in all.
Sanity Testing
Sanity testing is carried out to quickly evaluate and determine whether functionalities are
working as expected or not. Sanity Testing is also referred to as surface-level-
testing.Sanity testing is a type of Software Testing performed after receiving a software
build, with minor changes in code, or functionalities, to learn that the bugs have been
fixed and no further issues are acquainted due with these progressions.The objective is
“not” to verify the new functionality thoroughly but to determine that the developer has
applied some rationality (sanity) while developing the software.
Smoke Testing
Smoke Testing is performed to discover that the critical functionalities of the program are
working fine. The motto of this testing is to check the “stability” of the framework and
execute “before” any detailed functional or regression tests are executed on the
application.
Interface Testing
An interface is a medium between two different software systems, whereas interface
Testing is a software testing type which verifies whether the correspondence between two
distinctive programming frameworks is done accurately.
Regression Testing
Regression testing is the type of testing in which QA verifies whether changes made in
the old functionality still works with the newly implemented changes.
Beta/Acceptance Testing:
Beta or User Acceptance testing is carried out by “real users” of the developed product in
“real environment.” A Beta version of the application is released for a limited number of
users to gather feedback on product quality.Beta testing decreases item disappointment
hazards and provides an expanded nature of the product through client approval.It is the
final test before delivery product to the clients. Direct criticism from clients is a
noteworthy preferred standpoint of Beta Testing. This testing serves to tests the item in
the client’s environment.
Performance Testing
Performance testing is defined as the type of testing where we ensure proper functionality
of our application under a heavy workload.
Load Testing
Load testing is a sub-classification of Performance Testing. During load testing, we
evaluate the behavior of our application when N number of users simultaneously access
it.
Stress Testing
The objective of stress testing is to analyze the response of the system following a failure.
In this type of testing system should give an error message if it is operating under dire
conditions. This type of testing tries to break the system by testing with overwhelming
data or resources.
Volume Testing
Volume testing is a sort of Software Testing in which the product is subjected to an
enormous volume of information. This testing is carried out to test system performance
by expanding the amount of data in the database.
Security Testing
Security Testing ensures that our applications are free from any vulnerabilities, dangers,
risks that could cause a major misfortune. It additionally helps in identifying all
conceivable security issues in the system and assist users in fixing these issues through
coding.
Compatibility Testing
Compatibility Testing is a sort of Software testing to check whether your product is fit for
running on multiple hardware, different OS, Applications and mobile gadgets. It’s a type
of Non-functional testing.
Recovery Testing
Recovery testing is the process of testing how well an application can recoup from
accidents, hardware failures, and other comparable issues. This testing is the constrained
disappointment of the product in an assortment of approaches to confirm that recovery is
appropriately performed.
Usability Testing
Usability testing ensures – Usefulness, Discoverability, Accessibility, Usability, and
Desirability. The objective is to see how clients interact with your product and to improve
the item dependent on the outcomes. The main focus of a Usability test is to improve a
design.
Compliance Testing :
Compliance testing also referred to as conformance testing, regulation testing, standards
testing, is a sort of testing to decide the consistency of a framework. Most importantly it
is the nonfunctional testing procedure which is done to approve, regardless of whether the
framework created satisfies the association’s recommended guidelines or not.
Determining that the development and maintenance process meets the endorsed
procedure.
Localization Testing
Localization testing is a software testing strategy, where the product is checked to
guarantee that it carries on as indicated by the nearby culture or settings.
Unit testing
Integration testing
System testing
Acceptance testing
Non-functional testing methods incorporate all test types focused on the operational
aspects of a piece of software. These include:
Performance testing
Security testing
Usability testing
Compatibility testing
The key to releasing high quality software that can be easily adopted by your end
users is to build a robust testing framework that implements both functional and non-
functional software testing methodologies.
Unit Testing
Unit testing is the first level of testing and is often performed by the developers
themselves. It is the process of ensuring individual components of a piece of
software at the code level are functional and work as they were designed to.
Developers in a test-driven environment will typically write and run the tests prior to
the software or feature being passed over to the test team. Unit testing can be
conducted manually, but automating the process will speed up delivery cycles and
expand test coverage. Unit testing will also make debugging easier because finding
issues earlier means they take less time to fix than if they were discovered later in
the testing process. TestLeft is a tool that allows advanced testers and developers to
shift left with the fastest test automation tool embedded in any IDE.
Integration Testing
After each unit is thoroughly tested, it is integrated with other units to create modules
or components that are designed to perform specific tasks or activities. These are
then tested as group through integration testing to ensure whole segments of an
application behave as expected (i.e, the interactions between units are seamless).
These tests are often framed by user scenarios, such as logging into an application
or opening files. Integrated tests can be conducted by either developers or
independent testers and are usually comprised of a combination of automated
functional and manual tests.
System Testing
System testing is a black box testing method used to evaluate the completed and
integrated system, as a whole, to ensure it meets specified requirements. The
functionality of the software is tested from end-to-end and is typically conducted by a
separate testing team than the development team before the product is pushed into
production.
Acceptance Testing
Acceptance testing is the last phase of functional testing and is used to assess
whether or not the final piece of software is ready for delivery. It involves ensuring
that the product is in compliance with all of the original business criteria and that it
meets the end user’s needs. This requires the product be tested both internally and
externally, meaning you’ll need to get it into the hands of your end users for beta
testing along with those of your QA team. Beta testing is key to getting real feedback
from potential customers and can address any final usability concerns.
Performance Testing
Performance testing is a non-functional testing technique used to determine how an
application will behave under various conditions. The goal is to test its
responsiveness and stability in real user situations. Performance testing can be
broken down into four types:
Security Testing
With the rise of cloud-based testing platforms and cyber attacks, there is a growing
concern and need for the security of data being used and stored in software.
Security testing is a non-functional software testing technique used to determine if
the information and data in a system is protected. The goal is to purposefully find
loopholes and security risks in the system that could result in unauthorized access to
or the loss of information by probing the application for weaknesses. There are
multiple types of this testing method, each of which aimed at verifying six basic
principles of security:
1. Integrity
2. Confidentiality
3. Authentication
4. Authorization
5. Availability
6. Non-repudiation
Usability Testing
Usability testing is a testing method that measures an application’s ease-of-use from
the end-user perspective and is often performed during the system or acceptance
testing stages. The goal is to determine whether or not the visible design and
aesthetics of an application meet the intended workflow for various processes, such
as logging into an application. Usability testing is a great way for teams to review
separate functions, or the system as a whole, is intuitive to use.
Compatibility Testing
Compatibility testing is used to gauge how an application or piece of software will
work in different environments. It is used to check that your product is compatible
with multiple operating systems, platforms, browsers, or resolution configurations.
The goal is to ensure that your software’s functionality is consistently supported
across any environment you expect your end users to be using.